2024-04-25T08:26:20.482 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T08:26:20.482 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_neesingh@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} --name neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --archive /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993 -- /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/orig.config.yaml 2024-04-25T08:26:20.527 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:26:20.624 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7672993' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v1 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: 5787d42d10024ba410e822a8aba474331cb9af02 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5869 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 5787d42d10024ba410e822a8aba474331cb9af02 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNzYYf0hWcYLWqGvImj9urzBqpKCbclPzRJB7fnHmkQaAcrhtHrqskzVvwmVxrz5G5m5xnFe6dCYRkJlLWWEvWE= smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJULxo3rh4z/CpKYJo4+AyxP4j2+VFk9/h9QDz/570oMboG93QisQsCxYIYLOXm6arpeUuI4D/r+91i3QAQdRc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_08:15:18 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T08:26:20.624 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa; will attempt to use it 2024-04-25T08:26:20.625 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks 2024-04-25T08:26:20.625 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T08:26:20.627 INFO:teuthology.task.internal:Checking packages... 2024-04-25T08:26:20.652 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T08:26:20.652 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T08:26:20.652 INFO:teuthology.packaging:ref: None 2024-04-25T08:26:20.652 INFO:teuthology.packaging:tag: None 2024-04-25T08:26:20.652 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T08:26:20.652 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:26:20.652 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T08:26:20.918 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:26:20.920 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T08:26:20.926 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T08:26:20.926 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T08:26:20.968 INFO:teuthology.task.internal:Saving configuration 2024-04-25T08:26:20.984 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T08:26:20.992 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:26:21.018 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:20:36.594891', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBhhMbya0hQ6jca+YfCDCT2O2GbaIReyvepoNw1fCYgq'} 2024-04-25T08:26:21.041 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:20:36.593335', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEIJcBVaMWm32A/j7kt3JtHLMTouxtXesr/y+uVz4cpH'} 2024-04-25T08:26:21.041 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T08:26:21.049 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T08:26:21.049 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T08:26:21.049 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T08:26:21.136 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8b54aea0d0>, signals=[15]) 2024-04-25T08:26:21.137 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T08:26:21.144 INFO:teuthology.task.internal:Opening connections... 2024-04-25T08:26:21.144 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:26:21.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:26:21.228 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:26:21.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:26:21.309 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T08:26:21.317 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-04-25T08:26:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-04-25T08:26:21.324 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu" 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="22.04" 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=jammy 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T08:26:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T08:26:21.372 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T08:26:21.372 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=jammy 2024-04-25T08:26:21.372 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-04-25T08:26:21.397 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-04-25T08:26:21.401 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-04-25T08:26:21.402 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-04-25T08:26:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T08:26:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Ubuntu" 2024-04-25T08:26:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="22.04" 2024-04-25T08:26:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T08:26:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_CODENAME=jammy 2024-04-25T08:26:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:ID=ubuntu 2024-04-25T08:26:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE=debian 2024-04-25T08:26:21.450 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T08:26:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T08:26:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T08:26:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T08:26:21.451 INFO:teuthology.orchestra.run.smithi082.stdout:UBUNTU_CODENAME=jammy 2024-04-25T08:26:21.451 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-04-25T08:26:21.471 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T08:26:21.482 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T08:26:21.502 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T08:26:21.502 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:26:21.505 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:26:21.509 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T08:26:21.574 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T08:26:21.574 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:26:21.577 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:26:21.583 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T08:26:21.650 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T08:26:21.664 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T08:26:21.665 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T08:26:21.665 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T08:26:21.665 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:26:21.665 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:26:21.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:26:21.670 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-25T08:26:21.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:26:21.674 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-04-25T08:26:21.859 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:26:21.860 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T08:26:21.860 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-04-25T08:26:21.863 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:26:21.863 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T08:26:21.863 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-04-25T08:26:21.865 INFO:teuthology.orchestra.run.smithi046.stdout:5.15.0-56-generic 2024-04-25T08:26:21.865 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:26:21.866 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T08:26:21.866 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-04-25T08:26:21.866 DEBUG:teuthology.orchestra.run.smithi046:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-25T08:26:21.869 INFO:teuthology.orchestra.run.smithi082.stdout:5.15.0-56-generic 2024-04-25T08:26:21.869 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:26:21.869 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T08:26:21.869 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-04-25T08:26:21.869 DEBUG:teuthology.orchestra.run.smithi082:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-04-25T08:26:21.925 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-25T08:26:21.926 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-04-25T08:26:22.054 INFO:teuthology.orchestra.run.smithi082.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-25T08:26:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-25T08:26:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-25T08:26:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:hint: 2024-04-25T08:26:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:hint: git config --global init.defaultBranch 2024-04-25T08:26:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:hint: 2024-04-25T08:26:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-25T08:26:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-25T08:26:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:hint: 2024-04-25T08:26:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:hint: git branch -m 2024-04-25T08:26:22.056 INFO:teuthology.orchestra.run.smithi082.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-25T08:26:22.057 DEBUG:teuthology.orchestra.run.smithi082:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: git config --global init.defaultBranch 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 2024-04-25T08:26:22.081 INFO:teuthology.orchestra.run.smithi046.stderr:hint: git branch -m 2024-04-25T08:26:22.082 INFO:teuthology.orchestra.run.smithi046.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-04-25T08:26:22.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-04-25T08:26:22.121 DEBUG:teuthology.orchestra.run.smithi082:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-25T08:26:22.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-04-25T08:26:22.174 DEBUG:teuthology.orchestra.run.smithi082:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-25T08:26:22.212 DEBUG:teuthology.orchestra.run.smithi046:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-04-25T08:27:26.978 INFO:teuthology.orchestra.run.smithi082.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-25T08:27:26.978 INFO:teuthology.orchestra.run.smithi082.stderr: * [new branch] main -> origin/main 2024-04-25T08:27:27.011 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20190312 -> 20190312 2024-04-25T08:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20190416 -> 20190416 2024-04-25T08:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20190514 -> 20190514 2024-04-25T08:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20190618 -> 20190618 2024-04-25T08:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20190717 -> 20190717 2024-04-25T08:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20190815 -> 20190815 2024-04-25T08:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20190923 -> 20190923 2024-04-25T08:27:27.012 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20191022 -> 20191022 2024-04-25T08:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20191215 -> 20191215 2024-04-25T08:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20200122 -> 20200122 2024-04-25T08:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20200316 -> 20200316 2024-04-25T08:27:27.014 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20200421 -> 20200421 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20200519 -> 20200519 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20200619 -> 20200619 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20200721 -> 20200721 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20200817 -> 20200817 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20200918 -> 20200918 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20201022 -> 20201022 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20201118 -> 20201118 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20201218 -> 20201218 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20210208 -> 20210208 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20210315 -> 20210315 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20210511 -> 20210511 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20210716 -> 20210716 2024-04-25T08:27:27.015 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20210818 -> 20210818 2024-04-25T08:27:27.016 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20210919 -> 20210919 2024-04-25T08:27:27.016 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20211027 -> 20211027 2024-04-25T08:27:27.018 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20211216 -> 20211216 2024-04-25T08:27:27.018 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20220209 -> 20220209 2024-04-25T08:27:27.018 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20220310 -> 20220310 2024-04-25T08:27:27.018 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20220411 -> 20220411 2024-04-25T08:27:27.018 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20220509 -> 20220509 2024-04-25T08:27:27.018 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20220610 -> 20220610 2024-04-25T08:27:27.018 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20220708 -> 20220708 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20220815 -> 20220815 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20220913 -> 20220913 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20221012 -> 20221012 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20221109 -> 20221109 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20221214 -> 20221214 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20230117 -> 20230117 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20230210 -> 20230210 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20230310 -> 20230310 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20230404 -> 20230404 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20230515 -> 20230515 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20230625 -> 20230625 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20230804 -> 20230804 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20230919 -> 20230919 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20231030 -> 20231030 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20231111 -> 20231111 2024-04-25T08:27:27.019 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20231211 -> 20231211 2024-04-25T08:27:27.020 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20240115 -> 20240115 2024-04-25T08:27:27.020 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20240220 -> 20240220 2024-04-25T08:27:27.020 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20240312 -> 20240312 2024-04-25T08:27:27.020 INFO:teuthology.orchestra.run.smithi082.stderr: * [new tag] 20240410 -> 20240410 2024-04-25T08:27:28.812 INFO:teuthology.orchestra.run.smithi046.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-04-25T08:27:28.812 INFO:teuthology.orchestra.run.smithi046.stderr: * [new branch] main -> origin/main 2024-04-25T08:27:28.847 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190312 -> 20190312 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190416 -> 20190416 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190514 -> 20190514 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190618 -> 20190618 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190717 -> 20190717 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190815 -> 20190815 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190923 -> 20190923 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20191022 -> 20191022 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20191215 -> 20191215 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200122 -> 20200122 2024-04-25T08:27:28.848 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200316 -> 20200316 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200421 -> 20200421 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200519 -> 20200519 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200619 -> 20200619 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200721 -> 20200721 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200817 -> 20200817 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200918 -> 20200918 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201022 -> 20201022 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201118 -> 20201118 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201218 -> 20201218 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210208 -> 20210208 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210315 -> 20210315 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210511 -> 20210511 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210716 -> 20210716 2024-04-25T08:27:28.851 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210818 -> 20210818 2024-04-25T08:27:28.852 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210919 -> 20210919 2024-04-25T08:27:28.852 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20211027 -> 20211027 2024-04-25T08:27:28.852 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20211216 -> 20211216 2024-04-25T08:27:28.852 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220209 -> 20220209 2024-04-25T08:27:28.852 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220310 -> 20220310 2024-04-25T08:27:28.852 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220411 -> 20220411 2024-04-25T08:27:28.852 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220509 -> 20220509 2024-04-25T08:27:28.854 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220610 -> 20220610 2024-04-25T08:27:28.854 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220708 -> 20220708 2024-04-25T08:27:28.854 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220815 -> 20220815 2024-04-25T08:27:28.854 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220913 -> 20220913 2024-04-25T08:27:28.854 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221012 -> 20221012 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221109 -> 20221109 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221214 -> 20221214 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230117 -> 20230117 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230210 -> 20230210 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230310 -> 20230310 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230404 -> 20230404 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230515 -> 20230515 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230625 -> 20230625 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230804 -> 20230804 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230919 -> 20230919 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20231030 -> 20231030 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20231111 -> 20231111 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20231211 -> 20231211 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240115 -> 20240115 2024-04-25T08:27:28.855 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240220 -> 20240220 2024-04-25T08:27:28.856 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240312 -> 20240312 2024-04-25T08:27:28.856 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240410 -> 20240410 2024-04-25T08:27:31.309 INFO:teuthology.orchestra.run.smithi082.stderr:Updating files: 44% (1517/3381) Updating files: 45% (1522/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 59% (1995/3381) Updating files: 59% (2005/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 69% (2333/3381) Updating files: 69% (2362/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 83% (2807/3381) Updating files: 83% (2809/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-25T08:27:31.314 INFO:teuthology.orchestra.run.smithi082.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-25T08:27:31.335 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T08:27:31.336 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:27:31.497 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-25T08:27:31.497 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-25T08:27:31.500 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi082.front.sepia.ceph.com to complete... 2024-04-25T08:27:33.367 INFO:teuthology.orchestra.run.smithi046.stderr:Updating files: 44% (1500/3381) Updating files: 45% (1522/3381) Updating files: 46% (1556/3381) Updating files: 47% (1590/3381) Updating files: 48% (1623/3381) Updating files: 49% (1657/3381) Updating files: 50% (1691/3381) Updating files: 51% (1725/3381) Updating files: 52% (1759/3381) Updating files: 53% (1792/3381) Updating files: 54% (1826/3381) Updating files: 55% (1860/3381) Updating files: 56% (1894/3381) Updating files: 57% (1928/3381) Updating files: 58% (1961/3381) Updating files: 58% (1980/3381) Updating files: 59% (1995/3381) Updating files: 60% (2029/3381) Updating files: 61% (2063/3381) Updating files: 62% (2097/3381) Updating files: 63% (2131/3381) Updating files: 64% (2164/3381) Updating files: 65% (2198/3381) Updating files: 66% (2232/3381) Updating files: 67% (2266/3381) Updating files: 68% (2300/3381) Updating files: 68% (2330/3381) Updating files: 69% (2333/3381) Updating files: 70% (2367/3381) Updating files: 71% (2401/3381) Updating files: 72% (2435/3381) Updating files: 73% (2469/3381) Updating files: 74% (2502/3381) Updating files: 75% (2536/3381) Updating files: 76% (2570/3381) Updating files: 77% (2604/3381) Updating files: 78% (2638/3381) Updating files: 79% (2671/3381) Updating files: 80% (2705/3381) Updating files: 81% (2739/3381) Updating files: 82% (2773/3381) Updating files: 82% (2781/3381) Updating files: 83% (2807/3381) Updating files: 84% (2841/3381) Updating files: 85% (2874/3381) Updating files: 86% (2908/3381) Updating files: 87% (2942/3381) Updating files: 88% (2976/3381) Updating files: 89% (3010/3381) Updating files: 90% (3043/3381) Updating files: 91% (3077/3381) Updating files: 92% (3111/3381) Updating files: 93% (3145/3381) Updating files: 94% (3179/3381) Updating files: 95% (3212/3381) Updating files: 96% (3246/3381) Updating files: 97% (3280/3381) Updating files: 98% (3314/3381) Updating files: 99% (3348/3381) Updating files: 100% (3381/3381) Updating files: 100% (3381/3381), done. 2024-04-25T08:27:33.372 INFO:teuthology.orchestra.run.smithi046.stdout:HEAD is now at 63a66b66 Merge branch 'robot/pr-0-1713990671' into 'main' 2024-04-25T08:27:33.395 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T08:27:33.395 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T08:27:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25 08:27:33 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-25T08:27:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:FINISHED --2024-04-25 08:27:33-- 2024-04-25T08:27:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:Total wall clock time: 1.9s 2024-04-25T08:27:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:Downloaded: 1 files, 43M in 1.8s (23.5 MB/s) 2024-04-25T08:27:33.555 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T08:27:33.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo dpkg -i /tmp/linux-image.deb 2024-04-25T08:27:33.561 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ 2024-04-25T08:27:33.562 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /tmp/linux-image.deb && echo linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/ --input-file=- 2024-04-25T08:27:33.564 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-04-25T08:27:33.849 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-25T08:27:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25 08:27:36 URL:https://2.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/ubuntu/jammy/flavors/default/pool/main/l/linux-6.9.0-rc3-g806ec36b2112/linux-image-6.9.0-rc3-g806ec36b2112_6.9.0-rc3-g806ec36b2112-1_amd64.deb [44805284/44805284] -> "/tmp/linux-image.deb" [1] 2024-04-25T08:27:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:FINISHED --2024-04-25 08:27:36-- 2024-04-25T08:27:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:Total wall clock time: 1.7s 2024-04-25T08:27:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:Downloaded: 1 files, 43M in 1.6s (27.2 MB/s) 2024-04-25T08:27:35.307 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T08:27:35.307 DEBUG:teuthology.orchestra.run.smithi046:> sudo dpkg -i /tmp/linux-image.deb 2024-04-25T08:27:35.546 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-image-6.9.0-rc3-g806ec36b2112. 2024-04-25T08:27:35.800 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-25T08:27:35.807 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-25T08:27:35.908 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T08:27:37.538 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-04-25T08:27:37.542 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-04-25T08:27:37.686 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T08:27:39.684 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T08:27:39.822 INFO:teuthology.orchestra.run.smithi082.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T08:27:41.812 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-6.9.0-rc3-g806ec36b2112 (6.9.0-rc3-g806ec36b2112-1) ... 2024-04-25T08:27:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:update-initramfs: Generating /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:02.059 INFO:teuthology.orchestra.run.smithi082.stderr:Sourcing file `/etc/default/grub' 2024-04-25T08:28:02.066 INFO:teuthology.orchestra.run.smithi082.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:28:02.090 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2024-04-25T08:28:02.424 INFO:teuthology.orchestra.run.smithi082.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:02.737 INFO:teuthology.orchestra.run.smithi082.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:28:02.746 INFO:teuthology.orchestra.run.smithi082.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:28:02.807 INFO:teuthology.orchestra.run.smithi082.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:28:02.813 INFO:teuthology.orchestra.run.smithi082.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:28:02.946 INFO:teuthology.orchestra.run.smithi082.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:28:02.946 INFO:teuthology.orchestra.run.smithi082.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:28:02.946 INFO:teuthology.orchestra.run.smithi082.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:28:02.986 INFO:teuthology.orchestra.run.smithi082.stderr:done 2024-04-25T08:28:03.215 DEBUG:teuthology.orchestra.run.smithi082:> dpkg-deb -c /tmp/linux-image.deb 2024-04-25T08:28:03.223 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-25T08:28:03.223 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-25T08:28:03.223 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:03.240 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:03.240 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:03.262 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-25T08:28:03.262 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-25T08:28:03.262 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-25T08:28:03.262 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-25T08:28:03.262 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-25T08:28:03.262 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-25T08:28:03.262 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-25T08:28:03.263 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-25T08:28:03.263 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-25T08:28:03.263 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-25T08:28:03.263 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-25T08:28:03.263 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-25T08:28:03.263 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-25T08:28:03.263 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T08:28:03.266 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-25T08:28:03.267 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-25T08:28:03.268 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T08:28:03.268 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T08:28:03.268 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T08:28:03.289 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-25T08:28:03.289 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-25T08:28:03.289 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-25T08:28:03.289 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-25T08:28:03.289 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T08:28:03.290 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-25T08:28:03.291 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T08:28:03.292 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-25T08:28:03.293 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-25T08:28:03.294 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T08:28:03.295 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T08:28:03.296 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T08:28:03.297 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-25T08:28:03.298 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T08:28:03.299 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T08:28:03.300 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-25T08:28:03.301 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-25T08:28:03.302 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-25T08:28:03.303 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T08:28:03.350 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T08:28:03.351 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-25T08:28:03.352 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-25T08:28:03.353 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-25T08:28:03.354 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T08:28:03.355 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T08:28:03.356 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-25T08:28:03.357 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T08:28:03.358 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T08:28:03.361 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T08:28:03.361 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T08:28:03.361 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T08:28:03.361 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T08:28:03.362 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T08:28:03.363 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-25T08:28:03.364 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T08:28:03.365 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T08:28:03.366 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T08:28:03.367 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T08:28:03.368 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T08:28:03.369 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T08:28:03.370 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T08:28:03.371 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T08:28:03.372 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T08:28:03.373 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T08:28:03.374 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-25T08:28:03.375 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T08:28:03.376 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-25T08:28:03.377 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-25T08:28:03.378 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-25T08:28:03.379 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-25T08:28:03.380 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-25T08:28:03.381 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T08:28:03.382 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-25T08:28:03.383 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T08:28:03.384 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-25T08:28:03.385 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T08:28:03.386 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-25T08:28:03.387 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T08:28:03.388 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-25T08:28:03.389 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-25T08:28:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T08:28:03.391 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-25T08:28:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-25T08:28:03.393 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T08:28:03.394 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T08:28:03.395 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T08:28:03.396 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-25T08:28:03.397 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T08:28:03.398 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-25T08:28:03.398 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T08:28:03.398 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T08:28:03.398 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-25T08:28:03.398 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T08:28:03.398 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T08:28:03.398 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-25T08:28:03.398 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T08:28:03.431 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T08:28:03.431 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-25T08:28:03.431 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T08:28:03.431 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T08:28:03.431 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-25T08:28:03.431 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-25T08:28:03.431 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T08:28:03.431 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-25T08:28:03.431 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-25T08:28:03.432 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T08:28:03.433 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T08:28:03.434 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-25T08:28:03.435 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-25T08:28:03.436 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T08:28:03.437 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-25T08:28:03.438 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T08:28:03.439 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-25T08:28:03.440 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T08:28:03.441 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T08:28:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T08:28:03.443 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T08:28:03.444 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T08:28:03.445 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T08:28:03.446 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-25T08:28:03.447 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T08:28:03.448 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T08:28:03.449 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-25T08:28:03.450 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-25T08:28:03.451 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-25T08:28:03.452 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-25T08:28:03.453 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T08:28:03.454 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-25T08:28:03.455 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-25T08:28:03.456 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-25T08:28:03.457 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T08:28:03.458 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T08:28:03.459 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-25T08:28:03.460 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T08:28:03.461 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-25T08:28:03.462 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T08:28:03.496 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T08:28:03.497 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T08:28:03.498 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T08:28:03.499 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T08:28:03.500 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-25T08:28:03.501 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T08:28:03.502 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-25T08:28:03.503 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-25T08:28:03.504 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-25T08:28:03.505 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-25T08:28:03.506 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-25T08:28:03.507 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-25T08:28:03.508 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T08:28:03.509 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T08:28:03.510 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T08:28:03.511 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T08:28:03.512 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T08:28:03.513 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-25T08:28:03.514 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T08:28:03.515 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T08:28:03.516 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T08:28:03.517 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-25T08:28:03.518 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T08:28:03.519 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T08:28:03.520 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T08:28:03.521 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-25T08:28:03.522 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-25T08:28:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T08:28:03.524 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-25T08:28:03.525 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T08:28:03.526 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T08:28:03.527 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T08:28:03.528 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T08:28:03.529 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T08:28:03.571 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T08:28:03.572 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T08:28:03.573 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T08:28:03.574 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T08:28:03.575 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-25T08:28:03.576 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-25T08:28:03.577 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-25T08:28:03.578 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T08:28:03.579 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-25T08:28:03.580 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-25T08:28:03.581 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T08:28:03.582 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-25T08:28:03.583 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T08:28:03.584 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-25T08:28:03.585 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-25T08:28:03.586 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T08:28:03.587 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T08:28:03.588 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T08:28:03.589 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T08:28:03.590 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T08:28:03.591 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T08:28:03.592 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T08:28:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T08:28:03.594 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T08:28:03.595 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T08:28:03.596 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T08:28:03.597 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T08:28:03.598 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T08:28:03.599 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-25T08:28:03.600 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T08:28:03.601 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T08:28:03.602 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T08:28:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T08:28:03.604 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-25T08:28:03.605 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-25T08:28:03.606 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T08:28:03.607 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T08:28:03.608 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T08:28:03.609 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T08:28:03.610 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T08:28:03.613 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-25T08:28:03.613 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T08:28:03.613 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-25T08:28:03.613 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T08:28:03.613 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T08:28:03.613 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-25T08:28:03.614 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-25T08:28:03.615 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-25T08:28:03.616 INFO:teuthology.orchestra.run.smithi082.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-25T08:28:03.616 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-25T08:28:03.616 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-25T08:28:03.621 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:03.621 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:03.621 DEBUG:teuthology.orchestra.run.smithi082:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-25T08:28:03.661 INFO:teuthology.orchestra.run.smithi082.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T08:28:03.662 INFO:teuthology.orchestra.run.smithi082.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T08:28:03.662 INFO:teuthology.orchestra.run.smithi082.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T08:28:03.662 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-25T08:28:03.662 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:03.663 DEBUG:teuthology.orchestra.run.smithi082:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-25T08:28:03.705 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi082.front.sepia.ceph.com to complete... 2024-04-25T08:28:04.144 INFO:teuthology.orchestra.run.smithi082.stderr:Sourcing file `/etc/default/grub' 2024-04-25T08:28:04.145 INFO:teuthology.orchestra.run.smithi082.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:28:04.174 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2024-04-25T08:28:04.369 INFO:teuthology.orchestra.run.smithi082.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:04.375 INFO:teuthology.orchestra.run.smithi082.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:04.382 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2024-04-25T08:28:04.399 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:28:04.460 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-04-25T08:28:04.610 INFO:teuthology.orchestra.run.smithi082.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:28:04.615 INFO:teuthology.orchestra.run.smithi082.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:28:04.656 INFO:teuthology.orchestra.run.smithi082.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:28:04.662 INFO:teuthology.orchestra.run.smithi082.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:28:04.813 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:04.813 INFO:teuthology.orchestra.run.smithi082.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:28:04.813 INFO:teuthology.orchestra.run.smithi082.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:28:04.813 INFO:teuthology.orchestra.run.smithi082.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:28:04.824 INFO:teuthology.orchestra.run.smithi082.stderr:done 2024-04-25T08:28:04.835 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:05.185 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:28:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:28:05.253 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:28:05.258 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:28:05.418 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:28:05.419 INFO:teuthology.orchestra.run.smithi046.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:28:05.419 INFO:teuthology.orchestra.run.smithi046.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:28:05.463 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-04-25T08:28:05.643 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-deb -c /tmp/linux-image.deb 2024-04-25T08:28:05.650 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./ 2024-04-25T08:28:05.650 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./boot/ 2024-04-25T08:28:05.650 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7002012 2024-04-19 08:55 ./boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:05.667 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 172073 2024-04-19 08:55 ./boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:05.667 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13120512 2024-04-19 08:55 ./boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:05.688 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/ 2024-04-25T08:28:05.688 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/ 2024-04-25T08:28:05.688 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postinst.d/ 2024-04-25T08:28:05.688 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/postrm.d/ 2024-04-25T08:28:05.688 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/preinst.d/ 2024-04-25T08:28:05.688 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./etc/kernel/prerm.d/ 2024-04-25T08:28:05.688 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/ 2024-04-25T08:28:05.688 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/ 2024-04-25T08:28:05.688 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/ 2024-04-25T08:28:05.689 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/ 2024-04-25T08:28:05.689 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/ 2024-04-25T08:28:05.689 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/ 2024-04-25T08:28:05.689 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ 2024-04-25T08:28:05.689 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 507569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/aesni-intel.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 155249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T08:28:05.692 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/ 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/ 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/ 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpuid.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/msr.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/ 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 444425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T08:28:05.693 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1035881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T08:28:05.694 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2849609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T08:28:05.710 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ 2024-04-25T08:28:05.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/af_alg.ko 2024-04-25T08:28:05.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_hash.ko 2024-04-25T08:28:05.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/algif_skcipher.ko 2024-04-25T08:28:05.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T08:28:05.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T08:28:05.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T08:28:05.710 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/ 2024-04-25T08:28:05.710 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/ 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T08:28:05.711 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cryptd.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_simd.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T08:28:05.712 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lrw.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 169065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T08:28:05.713 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xts.ko 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_extlog.ko 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T08:28:05.714 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/ 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/apei/einj.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci.ko 2024-04-25T08:28:05.715 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T08:28:05.716 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_legacy.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T08:28:05.717 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_radisys.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rz1000.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T08:28:05.718 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/ 2024-04-25T08:28:05.719 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 163969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T08:28:05.720 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/ 2024-04-25T08:28:05.720 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/ 2024-04-25T08:28:05.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 140865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T08:28:05.720 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/ 2024-04-25T08:28:05.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1241657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T08:28:05.720 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 203289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T08:28:05.775 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/ 2024-04-25T08:28:05.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T08:28:05.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T08:28:05.775 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/ 2024-04-25T08:28:05.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 126617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T08:28:05.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T08:28:05.775 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 313217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/ 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/ 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/applicom.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/ 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ 2024-04-25T08:28:05.776 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 186921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 179561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/ 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/nvram.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/ 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T08:28:05.777 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/ 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_class.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_core.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/ 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-palmas.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-si5351.ko 2024-04-25T08:28:05.778 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-twl6040.ko 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/clk/clk-wm831x.ko 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/ 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 333657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/ 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/ 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/ 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 751713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T08:28:05.779 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/ 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/ 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/ 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 146769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T08:28:05.780 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/ 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-gpio.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max14577.ko 2024-04-25T08:28:05.781 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max77693.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-max8997.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-palmas.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-rt8973a.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/extcon/extcon-sm5502.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/ 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/dmi-sysfs.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/ 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/efi/efi-pstore.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/ 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-adp5520.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-amd8111.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-arizona.ko 2024-04-25T08:28:05.782 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9052.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-da9055.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-dln2.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-f7188x.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-generic.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-kempld.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-lp3943.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max7300.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max730x.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-max732x.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pca953x.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-pcf857x.ko 2024-04-25T08:28:05.783 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-rdc321x.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-sch311x.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-tps65912.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl4030.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-twl6040.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-vx855.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm831x.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8350.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-wm8994.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/ 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-a4tech.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-apple.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T08:28:05.784 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-belkin.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cherry.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-chicony.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cp2112.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-cypress.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ezkey.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-generic.ko 2024-04-25T08:28:05.785 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kensington.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T08:28:05.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lg-g15.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 121369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-magicmouse.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-microsoft.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-monterey.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ntrig.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T08:28:05.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T08:28:05.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T08:28:05.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T08:28:05.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T08:28:05.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T08:28:05.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T08:28:05.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T08:28:05.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T08:28:05.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 184745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T08:28:05.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T08:28:05.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T08:28:05.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T08:28:05.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 319921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid.ko 2024-04-25T08:28:05.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T08:28:05.791 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/ 2024-04-25T08:28:05.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 153633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbhid.ko 2024-04-25T08:28:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbkbd.ko 2024-04-25T08:28:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/usbhid/usbmouse.ko 2024-04-25T08:28:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 261945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T08:28:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/ 2024-04-25T08:28:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T08:28:05.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 110681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T08:28:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 382033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T08:28:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ 2024-04-25T08:28:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T08:28:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T08:28:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T08:28:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T08:28:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T08:28:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T08:28:05.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T08:28:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9052-hwmon.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/da9055-hwmon.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T08:28:05.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc2103.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hih6130.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T08:28:05.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 156465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/jc42.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T08:28:05.797 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 86497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T08:28:05.798 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mc13783-adc.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 141353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T08:28:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T08:28:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T08:28:05.801 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 116777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T08:28:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/ 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/ 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/ 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T08:28:05.803 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T08:28:05.804 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T08:28:05.805 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/ 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ 2024-04-25T08:28:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 329697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1119913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 408433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 347217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/ 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 590929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/ 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 514641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/ 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 911665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ 2024-04-25T08:28:05.807 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 375545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 197121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/ 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 212553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 334401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 132585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 157937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ 2024-04-25T08:28:05.808 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 156673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/evbug.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/ff-memless.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/ 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/ 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T08:28:05.809 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lkkbd.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8323.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/lm8333.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/newtonkbd.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt1070.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/qt2160.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-04-25T08:28:05.810 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/stowaway.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/sunkbd.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/xtkbd.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/matrix-keymap.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/ 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 160017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 114273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/gpio_mouse.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 428185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/psmouse.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T08:28:05.811 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/ 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 140969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ct82c710.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/parkbd.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/pcips2.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/ps2mult.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serport.ko 2024-04-25T08:28:05.812 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/ 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-88pm860x.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-adp5520.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-bd2802.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da903x.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-da9052.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-gpio.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3533.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm355x.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3642.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp8788.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-max8997.ko 2024-04-25T08:28:05.813 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-mc13783.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-menf21bmc.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca9532.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca955x.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-pca963x.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-tca6507.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm831x-status.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-wm8350.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T08:28:05.814 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/ 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/ 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 790753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 116825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bufio.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 199041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T08:28:05.815 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-era.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-mirror.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-region-hash.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-snapshot.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 234457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T08:28:05.816 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-zero.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/ 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 264329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 157681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 444665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/ 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/ 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 214841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 150761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T08:28:05.817 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/ 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm800.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm805.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/88pm80x.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona-i2c.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 148633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/arizona.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/bcm590xx.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/cros_ec_dev.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/dln2.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/janz-cmodio.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/kempld-core.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-core.ko 2024-04-25T08:28:05.818 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lp3943.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/max8907.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-core.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/menf21bmc.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-adc.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633-gpio.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/pcf50633.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/retu-mfd.ko 2024-04-25T08:28:05.819 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 99425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/si476x-core.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps65010.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/tps6507x.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wm8994.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ad525x_dpot.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/ 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T08:28:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/ 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/c2port/core.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/ 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ds1682.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/dummy-irq.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/ 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/echo/echo.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/ 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T08:28:05.821 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/ 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 191649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hmc6352.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 92345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ics932s401.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/ 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T08:28:05.822 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/ 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 154169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 389033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/phantom.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/ 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/ti-st/st_drv.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/ 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 234905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/ 2024-04-25T08:28:05.823 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 620961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T08:28:05.857 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/ 2024-04-25T08:28:05.857 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/ 2024-04-25T08:28:05.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/ 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 587065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/ 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 158649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ 2024-04-25T08:28:05.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/ 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/ 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/ 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/ 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/ 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T08:28:05.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/ 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/ 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 128409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 110345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/ 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/ 2024-04-25T08:28:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 228985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1733353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 167185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/ 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 161417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 440625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/ 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/ 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 432201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/ 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 257201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-04-25T08:28:05.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 446177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1311433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 206689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/ 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/ 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 269313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/ 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/ 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T08:28:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 169089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/ 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ec_bhf.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/ 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/ 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 356377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T08:28:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/ 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/ 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 366217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/ 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 755793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/ 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 324577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/ 2024-04-25T08:28:05.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1196297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/ 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 362593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/ 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 671801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/ 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 121537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 966705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 193585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/ 2024-04-25T08:28:05.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 126881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 150777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/ 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 905033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 344145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1412417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ 2024-04-25T08:28:05.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/ 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/ 2024-04-25T08:28:05.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 181081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/ 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 171417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/ 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/ 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 300817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 885129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/ 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T08:28:05.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/ 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 264161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/ 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 142449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/ 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1350385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/ 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T08:28:05.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/ 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/ 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/ 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 644537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/ 2024-04-25T08:28:05.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 128497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 207993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/ 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/ 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 110729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/ 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/ 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T08:28:05.871 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/ 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/ 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/defxx.ko 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/ 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 275801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/fddi/skfp/skfp.ko 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/ 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 290441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/ 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T08:28:05.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/ 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-gpio.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/ 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ 2024-04-25T08:28:05.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 121105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T08:28:05.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/ 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/plip/plip.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T08:28:05.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sb1000.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/ 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/ 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 137321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T08:28:05.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/ 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 176497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/ 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 383593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/ 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 175537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/ 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/ 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 335857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 126553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/ 2024-04-25T08:28:05.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 168145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/ 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/ 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pci-stub.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/ 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T08:28:05.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 179961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia.ko 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_core.ko 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 133753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/ 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/ 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/ 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/ 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/ 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/ 2024-04-25T08:28:05.879 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/ 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 164769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T08:28:05.880 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/ibm_rtl.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/ 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T08:28:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/ 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_battery.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/88pm860x_charger.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq2415x_charger.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24190_charger.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/bq24735-charger.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-04-25T08:28:05.882 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9030_battery.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/da9052-battery.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2760_battery.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2780_battery.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2781_battery.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/ds2782_battery.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/gpio-charger.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/isp1704_charger.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/lp8727_charger.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max14577_charger.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max17042_battery.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8903_charger.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/max8925_power.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/pcf50633-charger.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/sbs-battery.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/test_power.ko 2024-04-25T08:28:05.883 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/tps65090-charger.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_backup.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm831x_power.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/power/supply/wm8350_power.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/ 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/ 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/ 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T08:28:05.884 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/ 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/ 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/ 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/ 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm80x.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-88pm860x.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9052.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9055.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-da9063.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T08:28:05.885 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-lp8788.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T08:28:05.886 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t86.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8907.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8925.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8997.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max8998.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-palmas.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf50633.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T08:28:05.887 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rc5t583.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-s35390a.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps6586x.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-tps65910.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm831x.ko 2024-04-25T08:28:05.888 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-wm8350.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 164569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/ 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 340721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/ 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 468081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T08:28:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 380673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/ 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 243705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/ 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 135617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/ 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 304993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/ 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1527113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/ 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 297801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T08:28:05.890 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/ 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 144401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/ 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 390713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/ 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 173017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T08:28:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/ 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 267577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/ 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 194017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fdomain.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/ 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 366529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 251505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/imm.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 327841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T08:28:05.892 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/ 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 397785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/ 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 386849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 178177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/ 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 274009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/ 2024-04-25T08:28:05.893 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2736001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T08:28:05.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/ 2024-04-25T08:28:05.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T08:28:05.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T08:28:05.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 389177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 104769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/ 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 836945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/ 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 158729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/ 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/ 2024-04-25T08:28:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 443745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ppa.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/ 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2324809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/ 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 697337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qlogicfas408.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 322865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 219857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 357153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 121185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T08:28:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 138121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/ 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 205025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/xen-scsifront.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 174913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/ 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/ 2024-04-25T08:28:05.928 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 920169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/ 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1170033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 149721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/ 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/ 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/ 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/ 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T08:28:05.929 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-04-25T08:28:05.930 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 99313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/moxa.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/mxser.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/ 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/ 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T08:28:05.931 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/altera_uart.ko 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/ 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/rp2.ko 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/sc16is7xx.ko 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/ 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 693081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T08:28:05.932 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/ 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_dmem_genirq.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_mf624.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_netx.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/ 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/ 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/c67x00/c67x00.ko 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ 2024-04-25T08:28:05.933 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 189209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/ 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/common/ulpi.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/ 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-04-25T08:28:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/ 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 574593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/dwc3/dwc3.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/ 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/ 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_ether.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/u_serial.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-04-25T08:28:05.935 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 146617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/ 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-04-25T08:28:05.936 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 204049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/libcomposite.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/ 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/ 2024-04-25T08:28:05.937 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 182561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2272.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/net2280.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 195969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/bcma-hcd.ko 2024-04-25T08:28:05.938 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/isp116x-hcd.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 104377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811_cs.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/ssb-hcd.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/ 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-04-25T08:28:05.939 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/cytherm.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ehset.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/lvstest.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/ 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T08:28:05.940 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbtest.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/ 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/mon/usbmon.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/ 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 396209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/musb/musb_hdrc.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/ 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-isp1301.ko 2024-04-25T08:28:05.941 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-tahvo.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/f81232.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 188609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T08:28:05.942 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 142833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T08:28:05.943 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/metro-usb.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mxuport.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 193857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T08:28:05.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70497 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 151137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usbserial.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/wishbone-serial.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T08:28:05.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T08:28:05.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 201193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/ 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/ 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/ 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 206921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/ 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 138721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T08:28:05.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/ 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/ 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/aat2870_bl.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp5520_bl.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8860_bl.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/adp8870_bl.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/as3711_bl.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/bd6107.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da903x_bl.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/da9052_bl.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/gpio_backlight.ko 2024-04-25T08:28:05.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/kb3886_bl.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3533_bl.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lm3639_bl.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lv5207lp.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/max8925_bl.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pandora_bl.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/wm831x_bl.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/ 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T08:28:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/matrox_w1.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/w1-gpio.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/ 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T08:28:05.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/acquirewdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/advantechwdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/cpu5wdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9052_wdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9055_wdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/da9063_wdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/dw_wdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/eurotechwdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T08:28:05.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/kempld_wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/mena21_wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pc87413_wdt.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T08:28:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/retu_wdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sc1200wdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/twl4030_wdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T08:28:05.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wafer5823wdt.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm831x_wdt.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wm8350_wdt.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/ 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/ 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 213913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-scsiback.ko 2024-04-25T08:28:05.954 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/ 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/ 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 192617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/ 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/adfs/adfs.ko 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/ 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 197849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/ 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1261113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/afs/kafs.ko 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/ 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 127377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/autofs/autofs4.ko 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/ 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T08:28:05.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/ 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/bfs/bfs.ko 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/ 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 4489593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/ 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 376857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1586065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/ 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 130073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/ 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/configfs/configfs.ko 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/ 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/ 2024-04-25T08:28:05.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 813225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/ 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/efs/efs.ko 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/ 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1906817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/ 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/ 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/freevxfs/freevxfs.ko 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/ 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/ 2024-04-25T08:28:05.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1486521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T08:28:05.965 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/ 2024-04-25T08:28:05.965 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 183305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/ 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 282097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/ 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 249009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hpfs/hpfs.ko 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/ 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/ 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 533073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/ 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 361369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/ 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 112609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T08:28:05.966 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/ 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 591585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/ 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/ 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/ 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/ 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 147689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1320249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv2.ko 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 151641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2518505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T08:28:05.967 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/ 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/ 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2024049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/ 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 734025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/ 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T08:28:05.968 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ascii.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T08:28:05.969 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T08:28:05.970 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 144209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 167137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T08:28:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 302073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 548169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 3245617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T08:28:05.972 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/ 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/omfs/omfs.ko 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/ 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 462873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/ 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx4/qnx4.ko 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/ 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/qnx6/qnx6.ko 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/ 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_tree.ko 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v1.ko 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/quota/quota_v2.ko 2024-04-25T08:28:05.973 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/ 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 684841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/ 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/ 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/ 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 3319041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/ 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/ 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 164185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/ 2024-04-25T08:28:05.974 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T08:28:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/ 2024-04-25T08:28:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 310017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T08:28:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ 2024-04-25T08:28:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 198233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T08:28:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/ 2024-04-25T08:28:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 5192705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T08:28:05.980 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/ 2024-04-25T08:28:05.980 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/ 2024-04-25T08:28:05.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/locking/locktorture.ko 2024-04-25T08:28:05.980 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/ 2024-04-25T08:28:05.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/time/test_udelay.ko 2024-04-25T08:28:05.980 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T08:28:05.980 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc7.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/ 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/interval_tree_test.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/ 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/ 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7281 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/memory-notifier-error-inject.ko 2024-04-25T08:28:05.981 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/notifier-error-inject.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/percpu_test.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/pm-notifier-error-inject.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/ 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/rbtree_test.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/ 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test-kstrtox.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 4912441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_bpf.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_firmware.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7121 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_module.ko 2024-04-25T08:28:05.982 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/test_user_copy.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/ 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/xz/xz_dec_test.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/ 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/ 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T08:28:05.983 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/ 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13537 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/ 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/ 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 236793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T08:28:05.984 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/ 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 879697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/ 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84761 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1030873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T08:28:05.985 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9569 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70529 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T08:28:05.986 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T08:28:05.987 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T08:28:05.987 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/ 2024-04-25T08:28:05.987 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1243505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T08:28:06.011 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/ 2024-04-25T08:28:06.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 153801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/ 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 244889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71033 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/ 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 507705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/ 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 128873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/hsr/hsr.ko 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ 2024-04-25T08:28:06.012 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 357425 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T08:28:06.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50169 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ip_tables.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T08:28:06.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T08:28:06.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11665 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11737 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T08:28:06.016 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_gre.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 118881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/mip6.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T08:28:06.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T08:28:06.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/ 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/ 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 153217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T08:28:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/ 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/lapb/lapb.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/ 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 332889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc2.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/ 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 388873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/ 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T08:28:06.020 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 147745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124041 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 126681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 126473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 137833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 133681 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 138265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T08:28:06.021 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 136177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 534113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20953 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24577 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12649 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T08:28:06.022 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15977 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12809 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 496817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24513 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 140801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T08:28:06.023 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117201 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 118369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14185 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13345 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T08:28:06.024 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 807985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53049 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11081 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T08:28:06.025 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17921 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16473 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13945 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_osf.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15057 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24993 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_synproxy.ko 2024-04-25T08:28:06.026 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17641 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tproxy.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108393 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/x_tables.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15745 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45217 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15937 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13017 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T08:28:06.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10857 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15361 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23089 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16689 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18985 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20697 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15289 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16633 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T08:28:06.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11961 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14209 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10257 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12657 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T08:28:06.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19193 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9137 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11521 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11561 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T08:28:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13553 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11881 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9273 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36489 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10753 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10849 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18449 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpudp.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12441 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T08:28:06.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/ 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17233 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/ 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14073 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/ 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 488825 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12009 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10873 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/ 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15929 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/ 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 142025 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/phonet.ko 2024-04-25T08:28:06.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/phonet/pn_pep.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/ 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 318617 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 202305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85105 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/ 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1027897 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rxrpc/rxrpc.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/ 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39001 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28721 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27593 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53465 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38705 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T08:28:06.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23969 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43417 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45673 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32129 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41457 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23225 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_cgroup.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37305 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39601 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49321 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79145 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8713 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15385 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48785 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9241 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T08:28:06.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13369 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8377 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25097 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43481 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39177 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq_codel.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62817 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26585 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33545 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36625 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12297 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33065 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T08:28:06.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44793 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32833 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13505 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60249 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33433 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38329 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35777 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34161 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/ 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1187313 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23913 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/ 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/ 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 432409 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T08:28:06.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70113 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T08:28:06.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1940889 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T08:28:06.039 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/ 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 890729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/ 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1122337 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12153 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/ 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18801 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/ 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90265 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 123769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12905 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20401 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/ 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 165841 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/x25/x25.ko 2024-04-25T08:28:06.040 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/ 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26729 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_algo.ko 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23353 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131865 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_user.ko 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 644769 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 628542 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7706 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23609 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9941 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70990 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 146959 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 232835 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 210 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72602 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 879 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T08:28:06.041 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 258450 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T08:28:06.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 313019 2024-04-19 08:55 ./lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T08:28:06.042 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/ 2024-04-25T08:28:06.042 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/ 2024-04-25T08:28:06.042 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/ 2024-04-25T08:28:06.042 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/ 2024-04-25T08:28:06.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 172 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/changelog.Debian.gz 2024-04-25T08:28:06.042 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 691 2024-04-19 08:55 ./usr/share/doc/linux-image-6.9.0-rc3-g806ec36b2112/copyright 2024-04-25T08:28:06.046 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:06.046 INFO:teuthology.task.kernel:searching for kernel 6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:06.046 DEBUG:teuthology.orchestra.run.smithi046:> egrep '(submenu|menuentry.*6.9.0-rc3-g806ec36b2112).*{' /boot/grub/grub.cfg 2024-04-25T08:28:06.087 INFO:teuthology.orchestra.run.smithi046.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T08:28:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T08:28:06.087 INFO:teuthology.orchestra.run.smithi046.stdout: menuentry 'Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.9.0-rc3-g806ec36b2112-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-04-25T08:28:06.088 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-04-25T08:28:06.088 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:06.088 DEBUG:teuthology.orchestra.run.smithi046:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-04-25T08:28:06.130 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-04-25T08:28:06.614 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2024-04-25T08:28:06.615 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:28:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-04-25T08:28:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:06.915 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-6.9.0-rc3-g806ec36b2112 2024-04-25T08:28:07.220 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:28:07.226 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:28:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:28:07.279 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:28:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:28:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:28:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:28:07.532 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-04-25T08:28:35.986 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T08:28:35.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T08:28:35.988 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:28:38.635 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T08:28:38.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-25T08:28:38.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:28:54.430 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-25T08:28:56.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-04-25T08:29:03.434 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T08:29:03.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:29:05.994 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-25T08:29:05.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:29:06.494 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-04-25T08:29:09.058 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-04-25T08:29:18.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T08:29:18.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:29:21.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-25T08:29:21.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:29:50.403 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-04-25T08:29:51.070 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T08:29:51.070 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T08:29:51.071 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-04-25T08:29:51.119 INFO:teuthology.orchestra.run.smithi082.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T08:29:51.119 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T08:29:51.119 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T08:29:51.119 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T08:29:52.120 DEBUG:teuthology.parallel:result is None 2024-04-25T08:30:21.065 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T08:30:36.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-25T08:30:36.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:30:36.452 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-04-25T08:30:37.216 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-25T08:30:37.216 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T08:30:37.216 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-04-25T08:30:37.268 INFO:teuthology.orchestra.run.smithi046.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T08:30:37.268 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T08:30:37.268 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T08:30:37.268 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T08:30:38.269 DEBUG:teuthology.parallel:result is None 2024-04-25T08:30:38.270 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T08:30:38.279 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T08:30:38.280 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:30:38.283 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:30:38.292 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T08:30:38.317 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T08:30:38.349 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T08:30:38.349 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:30:38.352 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:30:38.399 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T08:30:38.410 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T08:30:38.410 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:30:38.419 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:30:38.489 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:30:38.515 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:30:38.538 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:30:38.561 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:30:38.563 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T08:30:38.572 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T08:30:38.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:30:38.576 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:30:38.597 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T08:30:38.610 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T08:30:38.610 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:30:38.643 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:30:38.651 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:30:38.701 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:30:38.753 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:30:38.753 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:30:38.828 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:30:38.837 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:30:38.887 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:30:38.887 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:30:38.960 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-04-25T08:30:38.964 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-04-25T08:30:39.077 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T08:30:39.086 INFO:teuthology.task.internal:Starting timer... 2024-04-25T08:30:39.086 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T08:30:39.135 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T08:30:39.158 DEBUG:teuthology.task.selinux:Excluding smithi046: OS 'ubuntu' does not support SELinux 2024-04-25T08:30:39.158 DEBUG:teuthology.task.selinux:Excluding smithi082: OS 'ubuntu' does not support SELinux 2024-04-25T08:30:39.158 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T08:30:39.158 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T08:30:39.158 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T08:30:39.158 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T08:30:39.184 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T08:30:39.189 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T08:30:39.336 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T08:30:39.351 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T08:30:39.352 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi082.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T08:38:23.354 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi082.front.sepia.ceph.com')] 2024-04-25T08:38:23.356 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-25T08:38:23.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:38:23.444 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-04-25T08:38:23.553 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-04-25T08:38:23.553 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T08:38:23.554 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:38:23.638 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-04-25T08:38:23.761 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T08:38:23.762 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T08:38:23.773 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T08:38:23.773 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:38:23.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:38:23.777 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:38:23.777 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:38:23.813 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T08:38:23.813 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: Command line: ntpd -gq 2024-04-25T08:38:23.813 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: ---------------------------------------------------- 2024-04-25T08:38:23.813 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T08:38:23.813 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T08:38:23.814 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: corporation. Support and training for ntp-4 are 2024-04-25T08:38:23.814 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: available at https://www.nwtime.org/support 2024-04-25T08:38:23.814 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: ---------------------------------------------------- 2024-04-25T08:38:23.814 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: proto: precision = 0.044 usec (-24) 2024-04-25T08:38:23.814 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: basedate set to 2022-02-04 2024-04-25T08:38:23.815 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: gps base set to 2022-02-06 (week 2196) 2024-04-25T08:38:23.815 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:38:23.815 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:38:23.815 INFO:teuthology.orchestra.run.smithi046.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:38:23.815 INFO:teuthology.orchestra.run.smithi046.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:38:23.817 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T08:38:23.817 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T08:38:23.817 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T08:38:23.817 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: Listen normally on 3 ens1f0 172.21.15.46:123 2024-04-25T08:38:23.817 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: Listen normally on 4 lo [::1]:123 2024-04-25T08:38:23.817 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15f8%4]:123 2024-04-25T08:38:23.817 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:23 ntpd[17093]: Listening on routing socket on fd #22 for interface updates 2024-04-25T08:38:23.843 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T08:38:23.843 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: Command line: ntpd -gq 2024-04-25T08:38:23.843 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: ---------------------------------------------------- 2024-04-25T08:38:23.843 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T08:38:23.843 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T08:38:23.843 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: corporation. Support and training for ntp-4 are 2024-04-25T08:38:23.843 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: available at https://www.nwtime.org/support 2024-04-25T08:38:23.843 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: ---------------------------------------------------- 2024-04-25T08:38:23.844 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: proto: precision = 0.042 usec (-24) 2024-04-25T08:38:23.844 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: basedate set to 2022-02-04 2024-04-25T08:38:23.844 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: gps base set to 2022-02-06 (week 2196) 2024-04-25T08:38:23.844 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:38:23.845 INFO:teuthology.orchestra.run.smithi082.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:38:23.845 INFO:teuthology.orchestra.run.smithi082.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:38:23.847 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:38:23.847 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T08:38:23.847 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T08:38:23.847 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T08:38:23.847 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: Listen normally on 3 enp3s0f1 172.21.15.82:123 2024-04-25T08:38:23.847 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: Listen normally on 4 lo [::1]:123 2024-04-25T08:38:23.847 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d3ff%5]:123 2024-04-25T08:38:23.847 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:23 ntpd[17194]: Listening on routing socket on fd #22 for interface updates 2024-04-25T08:38:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:24 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:24 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:24.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:24 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:24.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:24 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:24.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:24 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:24.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:24 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:24.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:24 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:24.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:24 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:26.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:26 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:26.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:26 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:26.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:26 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:26.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:26 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:26 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:26.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:26 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:26.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:26.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:26 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:27.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:27 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:27.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:27 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:27.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:27 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:27.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:27 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:27.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:27 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:27.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:27 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:27 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:27 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:27 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:27 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:27 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:27.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:27 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:28.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:28 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:28.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:28.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:28.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:28.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:28 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:29.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:29 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:29.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:29 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:29.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:29 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:29.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:29 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:29.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:29 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:29.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:29 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:29.843 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:29 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:29.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:29 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:29.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:29 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:29.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:29 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:29.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:29 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:29.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:29 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:30.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:30.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:30 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:30.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:30.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:30.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:30.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:30.846 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:30.846 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:30.846 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:30.846 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:30.846 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:30.847 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:30.847 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:30.847 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:30 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:31 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:31 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:31 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:31 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:31 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:31 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:31.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:31 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:31.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:31 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:31.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:31 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:31.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:31 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:31.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:31 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:31.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:31 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:32.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:32.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:32.815 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:32.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:32.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:32.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:32.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:32.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:32.816 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:32.817 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:32.817 INFO:teuthology.orchestra.run.smithi046.stdout:25 Apr 08:38:32 ntpd[17093]: ntpd: time slew -0.002068 s 2024-04-25T08:38:32.818 INFO:teuthology.orchestra.run.smithi046.stdout:ntpd: time slew -0.002068s 2024-04-25T08:38:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T08:38:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:25 Apr 08:38:32 ntpd[17093]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T08:38:32.843 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:32.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:32.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:32.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:32.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:38:32.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:38:32.844 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:38:32.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:38:32.845 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 08:38:32 ntpd[17194]: ntpd: time slew -0.001238 s 2024-04-25T08:38:32.845 INFO:teuthology.orchestra.run.smithi082.stdout:ntpd: time slew -0.001238s 2024-04-25T08:38:32.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T08:38:32.845 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 08:38:32 ntpd[17194]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T08:38:32.922 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T08:38:32.923 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-04-25T08:38:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:38:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:38:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:38:32.923 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:38:32.945 INFO:teuthology.orchestra.run.smithi082.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T08:38:32.945 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================== 2024-04-25T08:38:32.945 INFO:teuthology.orchestra.run.smithi082.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:38:32.945 INFO:teuthology.orchestra.run.smithi082.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:38:32.945 INFO:teuthology.orchestra.run.smithi082.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:38:32.945 INFO:teuthology.orchestra.run.smithi082.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:38:32.947 INFO:teuthology.run_tasks:Running task install... 2024-04-25T08:38:32.955 DEBUG:teuthology.task.install:project ceph 2024-04-25T08:38:32.956 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T08:38:32.956 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T08:38:32.956 INFO:teuthology.task.install:Using flavor: default 2024-04-25T08:38:32.962 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T08:38:32.962 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T08:38:32.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-key list | grep Ceph 2024-04-25T08:38:32.963 DEBUG:teuthology.orchestra.run.smithi082:> sudo apt-key list | grep Ceph 2024-04-25T08:38:33.070 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T08:38:33.090 INFO:teuthology.orchestra.run.smithi082.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T08:38:33.123 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T08:38:33.123 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T08:38:33.125 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T08:38:33.125 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T08:38:33.125 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:38:33.141 INFO:teuthology.orchestra.run.smithi082.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T08:38:33.142 INFO:teuthology.orchestra.run.smithi082.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T08:38:33.143 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T08:38:33.143 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T08:38:33.143 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:38:33.275 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T08:38:33.276 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:38:33.303 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T08:38:33.304 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:38:33.588 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:38:33.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T08:38:33.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2024-04-25T08:38:33.635 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:38:33.635 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T08:38:33.654 DEBUG:teuthology.orchestra.run.smithi082:> sudo apt-get update 2024-04-25T08:38:33.824 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:38:33.834 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T08:38:33.846 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T08:38:33.847 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:38:33.854 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:38:33.863 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T08:38:33.884 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:38:33.916 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T08:38:34.066 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T08:38:34.095 INFO:teuthology.orchestra.run.smithi082.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T08:38:34.157 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T08:38:34.168 INFO:teuthology.orchestra.run.smithi082.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T08:38:34.230 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T08:38:34.242 INFO:teuthology.orchestra.run.smithi082.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T08:38:34.306 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T08:38:34.316 INFO:teuthology.orchestra.run.smithi082.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T08:38:34.405 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T08:38:34.406 INFO:teuthology.orchestra.run.smithi082.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T08:38:34.520 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 28.9 kB in 1s (42.4 kB/s) 2024-04-25T08:38:34.589 INFO:teuthology.orchestra.run.smithi082.stdout:Fetched 28.9 kB in 1s (40.5 kB/s) 2024-04-25T08:38:35.538 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-04-25T08:38:35.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:38:35.565 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T08:38:35.580 DEBUG:teuthology.orchestra.run.smithi082:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:38:35.630 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-04-25T08:38:35.654 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T08:38:35.808 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-04-25T08:38:35.808 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-04-25T08:38:35.836 INFO:teuthology.orchestra.run.smithi082.stdout:Building dependency tree... 2024-04-25T08:38:35.836 INFO:teuthology.orchestra.run.smithi082.stdout:Reading state information... 2024-04-25T08:38:35.989 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:38:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:38:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:38:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:38:35.989 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:38:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:38:35.991 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:38:35.991 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-04-25T08:38:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T08:38:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T08:38:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:38:35.991 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T08:38:35.992 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T08:38:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T08:38:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T08:38:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T08:38:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T08:38:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T08:38:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T08:38:35.993 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-04-25T08:38:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T08:38:35.993 INFO:teuthology.orchestra.run.smithi046.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi046.stdout: btrfs-tools 2024-04-25T08:38:36.016 INFO:teuthology.orchestra.run.smithi082.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:38:36.016 INFO:teuthology.orchestra.run.smithi082.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:38:36.016 INFO:teuthology.orchestra.run.smithi082.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:38:36.016 INFO:teuthology.orchestra.run.smithi082.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:38:36.017 INFO:teuthology.orchestra.run.smithi082.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:38:36.017 INFO:teuthology.orchestra.run.smithi082.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:38:36.017 INFO:teuthology.orchestra.run.smithi082.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:38:36.018 INFO:teuthology.orchestra.run.smithi082.stdout:The following additional packages will be installed: 2024-04-25T08:38:36.018 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T08:38:36.018 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T08:38:36.018 INFO:teuthology.orchestra.run.smithi082.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:38:36.018 INFO:teuthology.orchestra.run.smithi082.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T08:38:36.019 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T08:38:36.019 INFO:teuthology.orchestra.run.smithi082.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T08:38:36.019 INFO:teuthology.orchestra.run.smithi082.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T08:38:36.020 INFO:teuthology.orchestra.run.smithi082.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout:Suggested packages: 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T08:38:36.021 INFO:teuthology.orchestra.run.smithi082.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T08:38:36.022 INFO:teuthology.orchestra.run.smithi082.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T08:38:36.022 INFO:teuthology.orchestra.run.smithi082.stdout:Recommended packages: 2024-04-25T08:38:36.022 INFO:teuthology.orchestra.run.smithi082.stdout: btrfs-tools 2024-04-25T08:38:36.150 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-04-25T08:38:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T08:38:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T08:38:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T08:38:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T08:38:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:38:36.150 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T08:38:36.151 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T08:38:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T08:38:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T08:38:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T08:38:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T08:38:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T08:38:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T08:38:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T08:38:36.152 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T08:38:36.153 INFO:teuthology.orchestra.run.smithi046.stdout: socat unzip xmlstarlet zip 2024-04-25T08:38:36.170 INFO:teuthology.orchestra.run.smithi082.stdout:The following NEW packages will be installed: 2024-04-25T08:38:36.170 INFO:teuthology.orchestra.run.smithi082.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T08:38:36.170 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T08:38:36.171 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T08:38:36.171 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T08:38:36.171 INFO:teuthology.orchestra.run.smithi082.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:38:36.171 INFO:teuthology.orchestra.run.smithi082.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T08:38:36.171 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T08:38:36.172 INFO:teuthology.orchestra.run.smithi082.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T08:38:36.172 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T08:38:36.172 INFO:teuthology.orchestra.run.smithi082.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T08:38:36.172 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T08:38:36.173 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T08:38:36.174 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T08:38:36.174 INFO:teuthology.orchestra.run.smithi082.stdout: socat unzip xmlstarlet zip 2024-04-25T08:38:36.225 INFO:teuthology.orchestra.run.smithi046.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T08:38:36.225 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 176 MB of archives. 2024-04-25T08:38:36.225 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T08:38:36.225 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T08:38:36.246 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T08:38:36.247 INFO:teuthology.orchestra.run.smithi082.stdout:0 upgraded, 124 newly installed, 0 to remove and 305 not upgraded. 2024-04-25T08:38:36.247 INFO:teuthology.orchestra.run.smithi082.stdout:Need to get 176 MB of archives. 2024-04-25T08:38:36.247 INFO:teuthology.orchestra.run.smithi082.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T08:38:36.247 INFO:teuthology.orchestra.run.smithi082.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T08:38:36.257 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T08:38:36.268 INFO:teuthology.orchestra.run.smithi082.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T08:38:36.280 INFO:teuthology.orchestra.run.smithi082.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T08:38:36.305 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T08:38:36.328 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T08:38:36.329 INFO:teuthology.orchestra.run.smithi082.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T08:38:36.331 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T08:38:36.344 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T08:38:36.355 INFO:teuthology.orchestra.run.smithi082.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T08:38:36.358 INFO:teuthology.orchestra.run.smithi082.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T08:38:36.372 INFO:teuthology.orchestra.run.smithi082.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T08:38:36.391 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T08:38:36.394 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T08:38:36.403 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T08:38:36.406 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T08:38:36.406 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T08:38:36.414 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T08:38:36.416 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T08:38:36.418 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T08:38:36.419 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T08:38:36.422 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T08:38:36.426 INFO:teuthology.orchestra.run.smithi082.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T08:38:36.428 INFO:teuthology.orchestra.run.smithi082.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T08:38:36.435 INFO:teuthology.orchestra.run.smithi082.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T08:38:36.438 INFO:teuthology.orchestra.run.smithi082.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T08:38:36.439 INFO:teuthology.orchestra.run.smithi082.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T08:38:36.440 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T08:38:36.443 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T08:38:36.444 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T08:38:36.445 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T08:38:36.445 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T08:38:36.446 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T08:38:36.447 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T08:38:36.447 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T08:38:36.447 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T08:38:36.448 INFO:teuthology.orchestra.run.smithi082.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T08:38:36.450 INFO:teuthology.orchestra.run.smithi082.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T08:38:36.452 INFO:teuthology.orchestra.run.smithi082.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T08:38:36.453 INFO:teuthology.orchestra.run.smithi082.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T08:38:36.453 INFO:teuthology.orchestra.run.smithi082.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T08:38:36.466 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T08:38:36.466 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T08:38:36.467 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T08:38:36.473 INFO:teuthology.orchestra.run.smithi082.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T08:38:36.474 INFO:teuthology.orchestra.run.smithi082.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T08:38:36.476 INFO:teuthology.orchestra.run.smithi082.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T08:38:36.477 INFO:teuthology.orchestra.run.smithi082.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T08:38:36.477 INFO:teuthology.orchestra.run.smithi082.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T08:38:36.478 INFO:teuthology.orchestra.run.smithi082.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T08:38:36.486 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T08:38:36.486 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T08:38:36.487 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T08:38:36.487 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T08:38:36.488 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T08:38:36.488 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T08:38:36.489 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T08:38:36.493 INFO:teuthology.orchestra.run.smithi082.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T08:38:36.494 INFO:teuthology.orchestra.run.smithi082.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T08:38:36.494 INFO:teuthology.orchestra.run.smithi082.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T08:38:36.494 INFO:teuthology.orchestra.run.smithi082.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T08:38:36.505 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T08:38:36.506 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T08:38:36.506 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T08:38:36.514 INFO:teuthology.orchestra.run.smithi082.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T08:38:36.514 INFO:teuthology.orchestra.run.smithi082.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T08:38:36.517 INFO:teuthology.orchestra.run.smithi082.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T08:38:36.517 INFO:teuthology.orchestra.run.smithi082.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T08:38:36.518 INFO:teuthology.orchestra.run.smithi082.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T08:38:36.519 INFO:teuthology.orchestra.run.smithi082.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T08:38:36.525 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T08:38:36.525 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T08:38:36.526 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T08:38:36.527 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T08:38:36.527 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T08:38:36.529 INFO:teuthology.orchestra.run.smithi046.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T08:38:36.530 INFO:teuthology.orchestra.run.smithi046.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T08:38:36.534 INFO:teuthology.orchestra.run.smithi082.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T08:38:36.535 INFO:teuthology.orchestra.run.smithi082.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T08:38:36.535 INFO:teuthology.orchestra.run.smithi082.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T08:38:36.536 INFO:teuthology.orchestra.run.smithi082.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T08:38:36.544 INFO:teuthology.orchestra.run.smithi046.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T08:38:36.545 INFO:teuthology.orchestra.run.smithi046.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T08:38:36.547 INFO:teuthology.orchestra.run.smithi046.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T08:38:36.555 INFO:teuthology.orchestra.run.smithi082.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T08:38:36.556 INFO:teuthology.orchestra.run.smithi082.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T08:38:36.560 INFO:teuthology.orchestra.run.smithi082.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T08:38:36.560 INFO:teuthology.orchestra.run.smithi082.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T08:38:36.561 INFO:teuthology.orchestra.run.smithi082.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T08:38:36.561 INFO:teuthology.orchestra.run.smithi082.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T08:38:36.565 INFO:teuthology.orchestra.run.smithi046.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T08:38:36.566 INFO:teuthology.orchestra.run.smithi046.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T08:38:36.568 INFO:teuthology.orchestra.run.smithi046.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T08:38:36.588 INFO:teuthology.orchestra.run.smithi082.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T08:38:36.590 INFO:teuthology.orchestra.run.smithi046.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T08:38:36.591 INFO:teuthology.orchestra.run.smithi082.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T08:38:36.591 INFO:teuthology.orchestra.run.smithi082.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T08:38:36.593 INFO:teuthology.orchestra.run.smithi082.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T08:38:36.618 INFO:teuthology.orchestra.run.smithi046.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T08:38:36.619 INFO:teuthology.orchestra.run.smithi046.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T08:38:36.620 INFO:teuthology.orchestra.run.smithi082.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T08:38:36.620 INFO:teuthology.orchestra.run.smithi046.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T08:38:36.621 INFO:teuthology.orchestra.run.smithi046.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T08:38:36.622 INFO:teuthology.orchestra.run.smithi046.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T08:38:36.622 INFO:teuthology.orchestra.run.smithi082.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T08:38:36.622 INFO:teuthology.orchestra.run.smithi082.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T08:38:36.623 INFO:teuthology.orchestra.run.smithi082.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T08:38:36.625 INFO:teuthology.orchestra.run.smithi082.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T08:38:36.673 INFO:teuthology.orchestra.run.smithi046.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T08:38:36.675 INFO:teuthology.orchestra.run.smithi046.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T08:38:36.675 INFO:teuthology.orchestra.run.smithi046.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T08:38:36.682 INFO:teuthology.orchestra.run.smithi082.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T08:38:36.711 INFO:teuthology.orchestra.run.smithi082.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T08:38:36.712 INFO:teuthology.orchestra.run.smithi082.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T08:38:36.713 INFO:teuthology.orchestra.run.smithi082.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T08:38:36.714 INFO:teuthology.orchestra.run.smithi082.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T08:38:36.715 INFO:teuthology.orchestra.run.smithi082.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T08:38:36.742 INFO:teuthology.orchestra.run.smithi046.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T08:38:36.742 INFO:teuthology.orchestra.run.smithi046.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T08:38:36.743 INFO:teuthology.orchestra.run.smithi082.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T08:38:36.743 INFO:teuthology.orchestra.run.smithi046.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T08:38:36.743 INFO:teuthology.orchestra.run.smithi046.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T08:38:36.744 INFO:teuthology.orchestra.run.smithi046.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T08:38:36.744 INFO:teuthology.orchestra.run.smithi082.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T08:38:36.745 INFO:teuthology.orchestra.run.smithi082.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T08:38:36.745 INFO:teuthology.orchestra.run.smithi046.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T08:38:36.763 INFO:teuthology.orchestra.run.smithi046.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T08:38:36.764 INFO:teuthology.orchestra.run.smithi082.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T08:38:36.764 INFO:teuthology.orchestra.run.smithi046.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T08:38:36.766 INFO:teuthology.orchestra.run.smithi046.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T08:38:36.768 INFO:teuthology.orchestra.run.smithi082.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T08:38:36.769 INFO:teuthology.orchestra.run.smithi082.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T08:38:36.769 INFO:teuthology.orchestra.run.smithi082.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T08:38:36.770 INFO:teuthology.orchestra.run.smithi082.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T08:38:36.771 INFO:teuthology.orchestra.run.smithi082.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T08:38:36.782 INFO:teuthology.orchestra.run.smithi046.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T08:38:36.783 INFO:teuthology.orchestra.run.smithi046.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T08:38:36.784 INFO:teuthology.orchestra.run.smithi046.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T08:38:36.787 INFO:teuthology.orchestra.run.smithi046.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T08:38:36.790 INFO:teuthology.orchestra.run.smithi046.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T08:38:36.790 INFO:teuthology.orchestra.run.smithi082.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T08:38:36.791 INFO:teuthology.orchestra.run.smithi046.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T08:38:36.791 INFO:teuthology.orchestra.run.smithi082.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T08:38:36.791 INFO:teuthology.orchestra.run.smithi046.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T08:38:36.794 INFO:teuthology.orchestra.run.smithi082.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T08:38:36.796 INFO:teuthology.orchestra.run.smithi082.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T08:38:36.801 INFO:teuthology.orchestra.run.smithi046.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T08:38:36.806 INFO:teuthology.orchestra.run.smithi046.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T08:38:36.806 INFO:teuthology.orchestra.run.smithi046.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T08:38:36.812 INFO:teuthology.orchestra.run.smithi082.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T08:38:36.812 INFO:teuthology.orchestra.run.smithi082.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T08:38:36.816 INFO:teuthology.orchestra.run.smithi082.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T08:38:36.819 INFO:teuthology.orchestra.run.smithi082.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T08:38:36.819 INFO:teuthology.orchestra.run.smithi082.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T08:38:36.820 INFO:teuthology.orchestra.run.smithi082.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T08:38:36.822 INFO:teuthology.orchestra.run.smithi046.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T08:38:36.823 INFO:teuthology.orchestra.run.smithi046.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T08:38:36.823 INFO:teuthology.orchestra.run.smithi046.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T08:38:36.824 INFO:teuthology.orchestra.run.smithi046.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T08:38:36.825 INFO:teuthology.orchestra.run.smithi046.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T08:38:36.826 INFO:teuthology.orchestra.run.smithi046.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T08:38:36.834 INFO:teuthology.orchestra.run.smithi046.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T08:38:36.835 INFO:teuthology.orchestra.run.smithi082.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T08:38:36.840 INFO:teuthology.orchestra.run.smithi082.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T08:38:36.840 INFO:teuthology.orchestra.run.smithi082.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T08:38:36.843 INFO:teuthology.orchestra.run.smithi082.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T08:38:36.845 INFO:teuthology.orchestra.run.smithi046.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T08:38:36.845 INFO:teuthology.orchestra.run.smithi046.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T08:38:36.847 INFO:teuthology.orchestra.run.smithi046.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T08:38:36.855 INFO:teuthology.orchestra.run.smithi082.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T08:38:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T08:38:36.856 INFO:teuthology.orchestra.run.smithi082.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T08:38:36.858 INFO:teuthology.orchestra.run.smithi082.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T08:38:36.859 INFO:teuthology.orchestra.run.smithi082.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T08:38:36.867 INFO:teuthology.orchestra.run.smithi082.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T08:38:36.869 INFO:teuthology.orchestra.run.smithi046.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T08:38:36.891 INFO:teuthology.orchestra.run.smithi082.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T08:38:36.892 INFO:teuthology.orchestra.run.smithi082.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T08:38:36.894 INFO:teuthology.orchestra.run.smithi082.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T08:38:36.894 INFO:teuthology.orchestra.run.smithi082.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T08:38:37.535 INFO:teuthology.orchestra.run.smithi046.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T08:38:37.593 INFO:teuthology.orchestra.run.smithi082.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T08:38:37.706 INFO:teuthology.orchestra.run.smithi046.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T08:38:37.731 INFO:teuthology.orchestra.run.smithi046.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T08:38:37.742 INFO:teuthology.orchestra.run.smithi046.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T08:38:37.743 INFO:teuthology.orchestra.run.smithi046.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T08:38:37.749 INFO:teuthology.orchestra.run.smithi046.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T08:38:37.752 INFO:teuthology.orchestra.run.smithi046.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T08:38:37.786 INFO:teuthology.orchestra.run.smithi046.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T08:38:37.900 INFO:teuthology.orchestra.run.smithi082.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T08:38:37.950 INFO:teuthology.orchestra.run.smithi082.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T08:38:37.985 INFO:teuthology.orchestra.run.smithi082.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T08:38:37.986 INFO:teuthology.orchestra.run.smithi082.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T08:38:37.996 INFO:teuthology.orchestra.run.smithi082.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T08:38:38.000 INFO:teuthology.orchestra.run.smithi082.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T08:38:38.033 INFO:teuthology.orchestra.run.smithi082.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T08:38:38.138 INFO:teuthology.orchestra.run.smithi046.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T08:38:38.139 INFO:teuthology.orchestra.run.smithi046.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T08:38:38.171 INFO:teuthology.orchestra.run.smithi046.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T08:38:38.570 INFO:teuthology.orchestra.run.smithi082.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T08:38:38.576 INFO:teuthology.orchestra.run.smithi082.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T08:38:38.619 INFO:teuthology.orchestra.run.smithi082.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T08:38:39.491 INFO:teuthology.orchestra.run.smithi046.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T08:38:39.763 INFO:teuthology.orchestra.run.smithi046.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T08:38:39.772 INFO:teuthology.orchestra.run.smithi046.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T08:38:39.778 INFO:teuthology.orchestra.run.smithi046.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T08:38:39.860 INFO:teuthology.orchestra.run.smithi046.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T08:38:40.206 INFO:teuthology.orchestra.run.smithi046.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T08:38:40.599 INFO:teuthology.orchestra.run.smithi082.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T08:38:41.004 INFO:teuthology.orchestra.run.smithi082.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T08:38:41.024 INFO:teuthology.orchestra.run.smithi082.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T08:38:41.033 INFO:teuthology.orchestra.run.smithi082.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T08:38:41.157 INFO:teuthology.orchestra.run.smithi082.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T08:38:41.468 INFO:teuthology.orchestra.run.smithi046.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T08:38:41.469 INFO:teuthology.orchestra.run.smithi046.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T08:38:41.511 INFO:teuthology.orchestra.run.smithi046.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T08:38:41.630 INFO:teuthology.orchestra.run.smithi046.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T08:38:41.659 INFO:teuthology.orchestra.run.smithi082.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T08:38:41.673 INFO:teuthology.orchestra.run.smithi046.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T08:38:41.681 INFO:teuthology.orchestra.run.smithi046.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T08:38:41.780 INFO:teuthology.orchestra.run.smithi046.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi046.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T08:38:42.263 INFO:teuthology.orchestra.run.smithi046.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T08:38:43.650 INFO:teuthology.orchestra.run.smithi082.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T08:38:43.650 INFO:teuthology.orchestra.run.smithi082.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T08:38:43.733 INFO:teuthology.orchestra.run.smithi082.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T08:38:43.948 INFO:teuthology.orchestra.run.smithi082.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T08:38:44.023 INFO:teuthology.orchestra.run.smithi082.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T08:38:44.037 INFO:teuthology.orchestra.run.smithi082.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T08:38:44.220 INFO:teuthology.orchestra.run.smithi082.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T08:38:44.775 INFO:teuthology.orchestra.run.smithi046.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T08:38:44.776 INFO:teuthology.orchestra.run.smithi046.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T08:38:44.779 INFO:teuthology.orchestra.run.smithi046.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T08:38:44.780 INFO:teuthology.orchestra.run.smithi046.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T08:38:44.780 INFO:teuthology.orchestra.run.smithi046.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T08:38:44.783 INFO:teuthology.orchestra.run.smithi046.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T08:38:45.136 INFO:teuthology.orchestra.run.smithi082.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T08:38:45.138 INFO:teuthology.orchestra.run.smithi082.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi046.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T08:38:46.097 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 176 MB in 9s (19.0 MB/s) 2024-04-25T08:38:46.204 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T08:38:46.271 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T08:38:46.274 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T08:38:46.311 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T08:38:46.588 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T08:38:46.602 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T08:38:46.647 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:38:46.983 INFO:teuthology.orchestra.run.smithi082.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T08:38:46.984 INFO:teuthology.orchestra.run.smithi082.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T08:38:46.986 INFO:teuthology.orchestra.run.smithi082.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T08:38:46.986 INFO:teuthology.orchestra.run.smithi082.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T08:38:46.987 INFO:teuthology.orchestra.run.smithi082.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T08:38:46.987 INFO:teuthology.orchestra.run.smithi082.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T08:38:46.999 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T08:38:47.014 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T08:38:47.058 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T08:38:47.394 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T08:38:47.409 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T08:38:47.416 INFO:teuthology.orchestra.run.smithi082.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T08:38:47.494 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:38:47.823 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T08:38:47.838 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T08:38:47.881 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:38:48.048 INFO:teuthology.orchestra.run.smithi082.stdout:Fetched 176 MB in 11s (15.7 MB/s) 2024-04-25T08:38:48.220 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T08:38:48.250 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T08:38:48.266 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T08:38:48.287 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132922 files and directories currently installed.) 2024-04-25T08:38:48.290 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T08:38:48.309 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:38:48.336 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T08:38:48.672 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T08:38:48.685 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T08:38:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:38:48.756 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T08:38:48.771 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:38:48.823 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:38:49.050 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T08:38:49.063 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T08:38:49.099 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T08:38:49.217 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T08:38:49.233 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:38:49.276 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:38:49.394 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T08:38:49.409 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T08:38:49.520 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:38:49.620 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T08:38:49.635 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:38:49.670 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:38:49.898 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T08:38:49.913 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T08:38:49.956 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:38:50.276 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T08:38:50.290 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T08:38:50.359 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:38:50.416 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T08:38:50.431 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T08:38:50.473 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:38:50.754 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T08:38:50.769 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:38:50.818 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librados2. 2024-04-25T08:38:50.834 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:50.846 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:38:50.876 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:51.266 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T08:38:51.279 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librbd1. 2024-04-25T08:38:51.280 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:38:51.294 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:51.324 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:38:51.337 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:51.723 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T08:38:51.735 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T08:38:51.739 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:51.750 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:38:51.781 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:51.811 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:38:52.068 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rados. 2024-04-25T08:38:52.083 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:52.125 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:52.397 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T08:38:52.412 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T08:38:52.455 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:38:52.462 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T08:38:52.477 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:38:52.520 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:52.748 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T08:38:52.763 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:52.797 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:52.808 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package librados2. 2024-04-25T08:38:52.823 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:52.866 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:53.125 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T08:38:53.140 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:38:53.183 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:53.344 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package librbd1. 2024-04-25T08:38:53.358 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:53.402 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:53.495 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T08:38:53.510 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T08:38:53.553 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T08:38:53.814 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T08:38:53.829 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:53.872 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:53.897 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T08:38:53.912 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:53.963 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:54.158 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T08:38:54.174 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T08:38:54.208 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:38:54.291 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-rados. 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:54.349 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:54.444 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T08:38:54.459 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T08:38:54.502 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:38:54.760 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T08:38:54.775 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:38:54.797 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T08:38:54.813 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T08:38:54.827 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:54.855 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:38:55.155 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T08:38:55.170 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:55.175 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua5.1. 2024-04-25T08:38:55.191 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T08:38:55.221 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:55.233 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:38:55.607 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T08:38:55.622 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:38:55.661 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua-any. 2024-04-25T08:38:55.674 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:55.677 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T08:38:55.711 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T08:38:55.965 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package zip. 2024-04-25T08:38:55.980 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T08:38:56.023 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T08:38:56.093 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T08:38:56.108 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T08:38:56.160 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T08:38:56.326 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package unzip. 2024-04-25T08:38:56.341 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T08:38:56.384 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:38:56.488 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T08:38:56.503 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:56.546 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:56.738 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package luarocks. 2024-04-25T08:38:56.753 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T08:38:56.788 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:38:56.899 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T08:38:56.913 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T08:38:56.957 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:38:57.195 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librgw2. 2024-04-25T08:38:57.210 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:57.253 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:57.260 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T08:38:57.274 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T08:38:57.318 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:38:57.596 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T08:38:57.611 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T08:38:57.679 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:38:57.731 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T08:38:57.746 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:57.789 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:57.982 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package lua5.1. 2024-04-25T08:38:57.997 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T08:38:58.032 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:38:58.092 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T08:38:58.108 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T08:38:58.150 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:38:58.344 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package lua-any. 2024-04-25T08:38:58.358 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T08:38:58.402 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T08:38:58.462 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T08:38:58.477 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:58.520 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:58.688 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package zip. 2024-04-25T08:38:58.703 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T08:38:58.746 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T08:38:58.890 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-common. 2024-04-25T08:38:58.905 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:38:58.948 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:38:59.124 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package unzip. 2024-04-25T08:38:59.139 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T08:38:59.191 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:38:59.686 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package luarocks. 2024-04-25T08:38:59.701 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T08:38:59.753 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:39:00.020 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-base. 2024-04-25T08:39:00.036 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:00.088 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:00.152 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package librgw2. 2024-04-25T08:39:00.165 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:00.210 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:00.633 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T08:39:00.647 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T08:39:00.691 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:39:00.721 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T08:39:00.735 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:00.779 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T08:39:00.967 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T08:39:01.002 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:39:01.124 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T08:39:01.137 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T08:39:01.182 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:39:01.272 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T08:39:01.287 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T08:39:01.330 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:39:01.485 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T08:39:01.499 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:01.535 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:01.658 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T08:39:01.674 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T08:39:01.725 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:39:01.963 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-common. 2024-04-25T08:39:01.978 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:02.022 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:02.044 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T08:39:02.060 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T08:39:02.102 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:39:02.414 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T08:39:02.429 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T08:39:02.472 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T08:39:02.741 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-portend. 2024-04-25T08:39:02.757 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T08:39:02.791 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T08:39:03.053 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T08:39:03.068 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T08:39:03.102 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-base. 2024-04-25T08:39:03.102 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T08:39:03.116 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:03.193 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:03.405 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T08:39:03.421 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T08:39:03.472 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:39:03.798 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T08:39:03.811 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T08:39:03.856 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:39:03.959 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T08:39:03.975 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T08:39:04.017 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T08:39:04.134 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T08:39:04.148 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T08:39:04.192 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:39:04.320 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T08:39:04.336 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T08:39:04.370 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T08:39:04.511 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T08:39:04.525 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T08:39:04.561 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:39:04.640 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T08:39:04.655 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T08:39:04.698 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:39:04.889 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T08:39:04.903 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T08:39:04.973 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:39:05.001 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-mako. 2024-04-25T08:39:05.017 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T08:39:05.059 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:39:05.251 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T08:39:05.265 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:39:05.371 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T08:39:05.386 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T08:39:05.446 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:39:05.603 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T08:39:05.618 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T08:39:05.670 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T08:39:05.891 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T08:39:05.906 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T08:39:05.949 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:39:05.998 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-portend. 2024-04-25T08:39:06.012 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T08:39:06.056 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T08:39:06.235 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webob. 2024-04-25T08:39:06.250 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T08:39:06.293 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:39:06.384 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T08:39:06.399 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T08:39:06.442 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T08:39:06.663 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T08:39:06.678 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T08:39:06.724 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:39:06.730 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T08:39:06.745 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T08:39:06.787 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:39:07.041 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T08:39:07.056 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T08:39:07.099 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:39:07.124 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T08:39:07.138 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T08:39:07.207 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T08:39:07.360 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-paste. 2024-04-25T08:39:07.376 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T08:39:07.410 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:39:07.477 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T08:39:07.491 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T08:39:07.526 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T08:39:07.713 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T08:39:07.728 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T08:39:07.771 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:39:07.880 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T08:39:07.894 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T08:39:07.946 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:39:08.049 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T08:39:08.065 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T08:39:08.099 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:39:08.340 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-mako. 2024-04-25T08:39:08.355 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T08:39:08.394 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T08:39:08.407 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:39:08.409 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T08:39:08.452 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T08:39:08.747 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T08:39:08.762 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T08:39:08.805 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T08:39:08.980 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T08:39:09.560 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:39:09.847 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T08:39:09.862 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T08:39:09.885 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T08:39:09.900 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T08:39:09.905 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:39:09.943 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T08:39:10.232 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-webob. 2024-04-25T08:39:10.247 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T08:39:10.271 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-idna. 2024-04-25T08:39:10.286 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T08:39:10.290 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:39:10.329 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T08:39:10.599 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T08:39:10.614 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T08:39:10.649 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:39:10.660 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T08:39:10.675 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T08:39:10.721 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:39:11.039 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T08:39:11.053 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T08:39:11.088 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:39:11.127 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests. 2024-04-25T08:39:11.143 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T08:39:11.185 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:39:11.349 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-paste. 2024-04-25T08:39:11.364 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T08:39:11.399 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:39:11.496 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T08:39:11.512 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T08:39:11.554 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:39:11.702 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T08:39:11.717 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T08:39:11.786 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:39:11.849 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T08:39:11.865 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:11.908 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:12.030 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T08:39:12.045 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T08:39:12.088 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:39:12.287 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T08:39:12.302 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:12.345 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:12.425 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T08:39:12.439 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T08:39:12.483 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T08:39:12.665 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T08:39:12.681 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T08:39:12.723 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:39:12.827 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T08:39:12.842 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T08:39:12.885 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:39:12.984 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T08:39:13.000 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T08:39:13.034 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T08:39:13.266 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T08:39:13.280 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T08:39:13.323 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T08:39:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T08:39:13.395 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:13.437 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:13.626 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-idna. 2024-04-25T08:39:13.640 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T08:39:13.685 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T08:39:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T08:39:13.823 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:13.865 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:13.963 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T08:39:13.978 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T08:39:14.054 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:39:14.349 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-requests. 2024-04-25T08:39:14.358 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T08:39:14.399 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:39:14.444 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T08:39:14.459 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:14.502 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:14.743 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T08:39:14.752 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T08:39:14.793 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:39:15.088 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T08:39:15.096 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:15.138 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:15.232 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph. 2024-04-25T08:39:15.245 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:15.288 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:15.526 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T08:39:15.537 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:15.576 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:15.600 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T08:39:15.615 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:15.666 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T08:39:15.872 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T08:39:15.904 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:39:16.061 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T08:39:16.075 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:16.119 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:16.190 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T08:39:16.199 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T08:39:16.261 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T08:39:16.438 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephadm. 2024-04-25T08:39:16.453 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:16.488 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:16.593 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T08:39:16.602 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:16.642 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:16.783 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T08:39:16.798 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T08:39:16.841 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T08:39:17.038 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T08:39:17.050 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:17.081 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:17.177 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T08:39:17.192 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:39:17.227 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:39:18.066 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T08:39:18.076 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:18.124 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:18.165 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-babel. 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:39:18.215 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:39:18.543 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T08:39:18.558 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T08:39:18.601 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:39:18.921 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T08:39:18.936 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:18.952 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph. 2024-04-25T08:39:18.962 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:18.979 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:19.002 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:19.283 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T08:39:19.298 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T08:39:19.341 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T08:39:19.372 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T08:39:19.382 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:19.430 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:19.677 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-routes. 2024-04-25T08:39:19.693 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T08:39:19.735 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:39:19.833 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T08:39:19.843 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:19.882 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:20.072 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T08:39:20.087 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:20.130 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:20.277 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package cephadm. 2024-04-25T08:39:20.287 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:20.327 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:20.567 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T08:39:20.582 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T08:39:20.616 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:39:20.638 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T08:39:20.649 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T08:39:20.679 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T08:39:21.007 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T08:39:21.017 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:39:21.057 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:39:21.078 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T08:39:21.094 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T08:39:21.128 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:39:21.448 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T08:39:21.463 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T08:39:21.601 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:39:21.887 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T08:39:21.902 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T08:39:21.936 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:39:22.087 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-babel. 2024-04-25T08:39:22.096 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:39:22.137 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:39:22.481 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T08:39:22.491 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T08:39:22.531 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:39:22.587 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T08:39:22.602 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:22.637 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:22.901 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T08:39:22.910 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:22.967 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:23.332 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T08:39:23.342 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T08:39:23.381 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T08:39:23.590 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T08:39:23.606 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T08:39:23.648 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T08:39:23.684 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-routes. 2024-04-25T08:39:23.694 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T08:39:23.725 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:39:23.910 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T08:39:23.925 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T08:39:23.959 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:39:24.045 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T08:39:24.058 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:24.119 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:24.288 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T08:39:24.304 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T08:39:24.346 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:39:24.573 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T08:39:24.581 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T08:39:24.623 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:39:24.682 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T08:39:24.698 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T08:39:24.743 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T08:39:25.022 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T08:39:25.037 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T08:39:25.071 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T08:39:25.126 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T08:39:25.134 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T08:39:25.176 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:39:25.366 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T08:39:25.382 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T08:39:25.424 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:39:25.554 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T08:39:25.562 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T08:39:25.681 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:39:25.719 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T08:39:25.730 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T08:39:25.804 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T08:39:26.026 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T08:39:26.037 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T08:39:26.076 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:39:26.322 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T08:39:26.336 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T08:39:26.380 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:39:26.792 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T08:39:26.801 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:26.842 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:27.085 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T08:39:27.096 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:27.127 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:27.429 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T08:39:27.441 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T08:39:27.479 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:39:27.807 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T08:39:27.819 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:39:27.837 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T08:39:27.849 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T08:39:27.857 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:39:27.887 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T08:39:28.101 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package jq. 2024-04-25T08:39:28.113 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:39:28.151 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T08:39:28.404 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package socat. 2024-04-25T08:39:28.419 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T08:39:28.454 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:39:28.490 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T08:39:28.501 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T08:39:28.531 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:39:28.732 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T08:39:28.745 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T08:39:28.781 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:39:28.843 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T08:39:28.853 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T08:39:28.892 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:39:29.111 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-test. 2024-04-25T08:39:29.123 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:29.152 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:29.162 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T08:39:29.173 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T08:39:29.245 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T08:39:29.558 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T08:39:29.568 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T08:39:29.599 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T08:39:29.902 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T08:39:29.916 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T08:39:29.960 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:39:30.288 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T08:39:30.299 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T08:39:30.338 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T08:39:30.674 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T08:39:30.686 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T08:39:30.724 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:39:31.024 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:31.081 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:31.412 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T08:39:31.423 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:31.460 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T08:39:31.462 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:31.473 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:31.518 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:31.799 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T08:39:31.809 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T08:39:31.830 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T08:39:31.846 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T08:39:31.848 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:39:31.887 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T08:39:32.165 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T08:39:32.168 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T08:39:32.178 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:39:32.179 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:32.215 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:32.218 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:39:32.426 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T08:39:32.440 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:32.475 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:32.520 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package jq. 2024-04-25T08:39:32.531 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:39:32.570 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T08:39:32.745 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T08:39:32.759 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:32.803 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:32.831 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package socat. 2024-04-25T08:39:32.842 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T08:39:32.906 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:39:33.122 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T08:39:33.136 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T08:39:33.180 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:39:33.184 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T08:39:33.195 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T08:39:33.234 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:39:33.604 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-test. 2024-04-25T08:39:33.615 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:33.626 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T08:39:33.640 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T08:39:33.654 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:33.676 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:39:34.038 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T08:39:34.055 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T08:39:34.096 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:39:34.543 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package pkg-config. 2024-04-25T08:39:34.558 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T08:39:34.601 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:39:34.854 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T08:39:34.869 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T08:39:34.912 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:39:35.324 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T08:39:35.338 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T08:39:35.373 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T08:39:35.625 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T08:39:35.637 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:35.643 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T08:39:35.658 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T08:39:35.675 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:35.710 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T08:39:36.037 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T08:39:36.046 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T08:39:36.079 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T08:39:36.088 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:39:36.090 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:36.128 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:36.448 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T08:39:36.460 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T08:39:36.490 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T08:39:36.590 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T08:39:36.602 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T08:39:36.640 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T08:39:36.776 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T08:39:36.787 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:36.825 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:37.001 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-py. 2024-04-25T08:39:37.011 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T08:39:37.051 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T08:39:37.094 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T08:39:37.106 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:39:37.177 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:37.371 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T08:39:37.380 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T08:39:37.421 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:39:37.438 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T08:39:37.450 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:37.488 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:37.808 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T08:39:37.819 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T08:39:37.824 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T08:39:37.833 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T08:39:37.857 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:39:37.866 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:39:38.178 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T08:39:38.190 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T08:39:38.202 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-toml. 2024-04-25T08:39:38.211 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T08:39:38.219 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:39:38.260 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T08:39:38.521 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T08:39:38.531 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T08:39:38.566 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:39:38.607 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T08:39:38.619 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T08:39:38.657 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:39:38.883 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T08:39:38.892 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T08:39:38.924 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:39:39.095 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package pkg-config. 2024-04-25T08:39:39.106 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T08:39:39.144 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:39:39.152 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T08:39:39.161 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T08:39:39.193 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:39:39.440 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T08:39:39.451 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T08:39:39.489 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:39:39.922 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package radosgw. 2024-04-25T08:39:39.932 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:39.934 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T08:39:39.946 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T08:39:39.972 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:39.983 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T08:39:40.261 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T08:39:40.273 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T08:39:40.327 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T08:39:40.558 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T08:39:40.567 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:40.606 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T08:39:40.608 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:40.617 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T08:39:40.656 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:39:40.907 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T08:39:41.000 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T08:39:41.012 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T08:39:41.050 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T08:39:41.180 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:39:41.306 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:39:41.386 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-py. 2024-04-25T08:39:41.398 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T08:39:41.427 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T08:39:41.731 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T08:39:41.743 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T08:39:41.743 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T08:39:41.772 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:39:42.218 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T08:39:42.228 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T08:39:42.245 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T08:39:42.268 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:39:42.595 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-toml. 2024-04-25T08:39:42.607 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T08:39:42.868 INFO:teuthology.orchestra.run.smithi046.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T08:39:42.881 INFO:teuthology.orchestra.run.smithi046.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T08:39:42.933 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:42.939 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T08:39:43.014 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:39:43.359 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T08:39:43.371 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T08:39:43.385 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user cephadm....done 2024-04-25T08:39:43.447 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:39:43.514 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:39:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:39:43.979 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T08:39:43.991 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T08:39:44.028 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:39:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:39:44.225 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:39:44.501 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T08:39:44.757 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package radosgw. 2024-04-25T08:39:44.761 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:39:44.768 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:44.807 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:44.862 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T08:39:45.188 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:39:45.369 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T08:39:45.380 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:39:45.418 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:45.565 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T08:39:45.709 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T08:39:45.866 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T08:39:46.015 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:39:46.134 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T08:39:46.140 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:39:46.401 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:39:46.419 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T08:39:46.628 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:39:46.823 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T08:39:46.929 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:39:47.205 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:39:47.422 INFO:teuthology.orchestra.run.smithi082.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T08:39:47.461 INFO:teuthology.orchestra.run.smithi082.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-04-25T08:39:47.473 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:47.509 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:47.741 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:39:47.842 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:39:47.943 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T08:39:48.094 INFO:teuthology.orchestra.run.smithi082.stdout:Adding system user cephadm....done 2024-04-25T08:39:48.211 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:39:48.239 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:39:48.337 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:39:48.524 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:39:48.605 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:39:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:39:48.918 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:39:49.001 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T08:39:49.035 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T08:39:49.083 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:39:49.177 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T08:39:49.183 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T08:39:49.436 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:39:49.543 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:39:49.562 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T08:39:49.644 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T08:39:49.881 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:39:49.904 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:39:50.180 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:39:50.279 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T08:39:50.497 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T08:39:50.542 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T08:39:50.648 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:39:50.801 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T08:39:51.077 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:39:51.354 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:39:51.359 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:39:51.610 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:39:51.639 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:39:51.915 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:39:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:39:52.037 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:39:52.182 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:52.363 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T08:39:52.458 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:39:52.742 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:39:52.781 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T08:39:52.868 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T08:39:53.074 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:39:53.120 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:39:53.245 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:39:53.467 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:39:53.513 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:39:53.760 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T08:39:53.900 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T08:39:53.977 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T08:39:54.041 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:39:54.094 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:39:54.167 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T08:39:54.454 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:39:54.535 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:39:54.661 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T08:39:54.721 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:39:54.847 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:54.946 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:39:55.157 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T08:39:55.247 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:39:55.500 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:39:55.573 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T08:39:55.732 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:39:56.417 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:39:56.693 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:39:56.819 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:39:56.822 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:39:56.954 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:39:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:39:57.199 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T08:39:57.297 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T08:39:57.467 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:39:57.593 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:39:57.698 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T08:39:57.728 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:39:57.965 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:39:58.054 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:39:58.322 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T08:39:58.392 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:39:58.590 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:39:58.685 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T08:39:58.690 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:39:58.984 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:39:58.985 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:39:59.235 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T08:39:59.345 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:39:59.536 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T08:39:59.622 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:39:59.781 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:39:59.795 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:40:00.063 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T08:40:00.082 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T08:40:00.347 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:40:00.450 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:40:00.465 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T08:40:00.774 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:40:00.900 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:40:01.193 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:40:01.511 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:40:01.697 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:40:01.965 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:40:02.055 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:40:02.099 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T08:40:02.181 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:40:02.262 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T08:40:02.409 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:40:02.516 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:40:02.534 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:40:02.642 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:40:02.661 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:40:02.971 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:40:03.060 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:40:03.238 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T08:40:03.336 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T08:40:03.462 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T08:40:03.523 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:40:03.649 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:40:03.738 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:40:03.864 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:40:03.992 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:40:04.190 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:40:04.251 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T08:40:04.516 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:40:04.518 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T08:40:04.784 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T08:40:04.786 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:40:04.996 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T08:40:05.046 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T08:40:05.272 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T08:40:05.355 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:40:05.481 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T08:40:05.556 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:40:05.799 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:40:05.824 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:40:05.917 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:40:06.201 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T08:40:06.234 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:40:06.485 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:40:06.535 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:40:06.804 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:40:06.838 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:40:06.954 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:40:06.964 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:07.080 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:40:07.090 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:07.169 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T08:40:07.216 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:40:07.415 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:40:07.518 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:07.541 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:40:07.628 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:07.754 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T08:40:07.934 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:40:08.056 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:08.173 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:08.194 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T08:40:08.274 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:08.345 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T08:40:08.474 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:40:08.475 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:40:08.612 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:40:08.780 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:40:09.117 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:09.130 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:40:09.243 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:09.361 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:09.424 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:40:09.470 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:09.596 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:40:09.708 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T08:40:09.920 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T08:40:10.221 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T08:40:10.514 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:40:10.773 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:40:10.941 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:11.076 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:11.158 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T08:40:11.193 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:11.302 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:11.403 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:11.451 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:40:11.529 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:11.829 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:40:11.957 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:12.083 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:12.099 INFO:teuthology.orchestra.run.smithi046.stdout:Adding group ceph....done 2024-04-25T08:40:12.200 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:40:12.418 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user ceph....done 2024-04-25T08:40:12.537 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:12.588 INFO:teuthology.orchestra.run.smithi046.stdout:Setting system user ceph properties....done 2024-04-25T08:40:12.607 INFO:teuthology.orchestra.run.smithi046.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T08:40:12.655 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:12.761 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T08:40:12.814 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T08:40:13.107 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:13.184 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T08:40:13.266 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:13.392 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:13.603 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:40:13.603 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:40:13.865 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:40:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:40:14.277 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:14.396 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:14.741 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:14.842 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:15.130 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:15.356 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:15.419 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:40:15.419 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:40:15.482 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:40:16.069 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:16.420 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T08:40:16.777 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:16.912 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:17.046 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:17.096 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:17.172 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:17.298 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:17.336 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:40:17.336 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:40:17.449 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:17.986 INFO:teuthology.orchestra.run.smithi082.stdout:Adding group ceph....done 2024-04-25T08:40:17.998 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:18.219 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:40:18.219 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:40:18.387 INFO:teuthology.orchestra.run.smithi082.stdout:Adding system user ceph....done 2024-04-25T08:40:18.556 INFO:teuthology.orchestra.run.smithi082.stdout:Setting system user ceph properties....done 2024-04-25T08:40:18.571 INFO:teuthology.orchestra.run.smithi082.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T08:40:18.711 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T08:40:18.866 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:19.082 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T08:40:19.185 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:40:19.186 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:40:19.800 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:19.850 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:19.968 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:20.010 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:40:20.010 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:40:20.111 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:20.329 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:40:20.329 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:40:20.611 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:20.737 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:20.937 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:21.113 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:21.248 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:21.252 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:40:21.252 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:40:21.382 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:21.744 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:40:21.847 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:22.178 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:40:22.242 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T08:40:22.322 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:40:22.874 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:23.113 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:40:23.113 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:40:23.549 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:40:23.692 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:23.930 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:40:23.931 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:40:24.262 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:40:24.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T08:40:24.343 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-04-25T08:40:24.522 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-04-25T08:40:24.522 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-04-25T08:40:24.535 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:24.701 INFO:teuthology.orchestra.run.smithi046.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T08:40:24.701 INFO:teuthology.orchestra.run.smithi046.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T08:40:24.701 INFO:teuthology.orchestra.run.smithi046.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T08:40:24.701 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:40:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:40:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:40:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:40:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:40:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:40:24.703 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:40:24.703 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-04-25T08:40:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T08:40:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T08:40:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T08:40:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T08:40:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:40:24.704 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-04-25T08:40:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T08:40:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T08:40:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T08:40:24.760 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-04-25T08:40:24.761 INFO:teuthology.orchestra.run.smithi046.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T08:40:24.761 INFO:teuthology.orchestra.run.smithi046.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T08:40:24.761 INFO:teuthology.orchestra.run.smithi046.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T08:40:24.761 INFO:teuthology.orchestra.run.smithi046.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T08:40:24.761 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T08:40:24.762 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T08:40:24.762 INFO:teuthology.orchestra.run.smithi046.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:40:24.762 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2024-04-25T08:40:24.763 INFO:teuthology.orchestra.run.smithi046.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T08:40:24.827 INFO:teuthology.orchestra.run.smithi046.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T08:40:24.827 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 55.7 MB of archives. 2024-04-25T08:40:24.827 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T08:40:24.827 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T08:40:24.874 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:40:24.874 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:40:25.008 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T08:40:25.012 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T08:40:25.045 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T08:40:25.046 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T08:40:25.047 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T08:40:25.055 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T08:40:25.056 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T08:40:25.058 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T08:40:25.059 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T08:40:25.059 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T08:40:25.061 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T08:40:25.071 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T08:40:25.071 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T08:40:25.072 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T08:40:25.092 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T08:40:25.344 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T08:40:25.350 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T08:40:25.350 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T08:40:25.356 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T08:40:25.358 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T08:40:25.365 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T08:40:25.366 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T08:40:25.388 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T08:40:25.389 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T08:40:25.401 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T08:40:25.424 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T08:40:25.428 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T08:40:25.430 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T08:40:25.430 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T08:40:25.431 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T08:40:25.444 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T08:40:25.447 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T08:40:25.462 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:25.621 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:25.622 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T08:40:25.622 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T08:40:25.622 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T08:40:25.623 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T08:40:25.623 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T08:40:25.623 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T08:40:25.636 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T08:40:25.637 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T08:40:25.764 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:25.994 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:40:25.994 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:40:26.032 INFO:teuthology.orchestra.run.smithi046.stdout:Preconfiguring packages ... 2024-04-25T08:40:26.181 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 55.7 MB in 1s (64.4 MB/s) 2024-04-25T08:40:26.454 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T08:40:26.458 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:40:26.605 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:40:26.615 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:26.758 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:26.917 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:27.060 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:40:27.198 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T08:40:27.421 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:40:27.521 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:40:27.673 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:40:28.357 INFO:teuthology.orchestra.run.smithi046.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T08:40:28.604 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package flex. 2024-04-25T08:40:28.678 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T08:40:28.681 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T08:40:28.727 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T08:40:28.900 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:40:29.331 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:40:29.500 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T08:40:29.598 INFO:teuthology.orchestra.run.smithi082.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:40:29.601 DEBUG:teuthology.orchestra.run.smithi082:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T08:40:29.675 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T08:40:29.849 INFO:teuthology.orchestra.run.smithi082.stdout:Building dependency tree... 2024-04-25T08:40:29.850 INFO:teuthology.orchestra.run.smithi082.stdout:Reading state information... 2024-04-25T08:40:29.917 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:40:30.024 INFO:teuthology.orchestra.run.smithi082.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T08:40:30.024 INFO:teuthology.orchestra.run.smithi082.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T08:40:30.024 INFO:teuthology.orchestra.run.smithi082.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T08:40:30.024 INFO:teuthology.orchestra.run.smithi082.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:40:30.024 INFO:teuthology.orchestra.run.smithi082.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:40:30.024 INFO:teuthology.orchestra.run.smithi082.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:40:30.025 INFO:teuthology.orchestra.run.smithi082.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:40:30.025 INFO:teuthology.orchestra.run.smithi082.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:40:30.026 INFO:teuthology.orchestra.run.smithi082.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:40:30.026 INFO:teuthology.orchestra.run.smithi082.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:40:30.026 INFO:teuthology.orchestra.run.smithi082.stdout:The following additional packages will be installed: 2024-04-25T08:40:30.026 INFO:teuthology.orchestra.run.smithi082.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T08:40:30.026 INFO:teuthology.orchestra.run.smithi082.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T08:40:30.027 INFO:teuthology.orchestra.run.smithi082.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T08:40:30.027 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T08:40:30.027 INFO:teuthology.orchestra.run.smithi082.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:40:30.027 INFO:teuthology.orchestra.run.smithi082.stdout:Suggested packages: 2024-04-25T08:40:30.027 INFO:teuthology.orchestra.run.smithi082.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T08:40:30.028 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T08:40:30.028 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T08:40:30.081 INFO:teuthology.orchestra.run.smithi082.stdout:The following NEW packages will be installed: 2024-04-25T08:40:30.082 INFO:teuthology.orchestra.run.smithi082.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T08:40:30.082 INFO:teuthology.orchestra.run.smithi082.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T08:40:30.083 INFO:teuthology.orchestra.run.smithi082.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T08:40:30.083 INFO:teuthology.orchestra.run.smithi082.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T08:40:30.083 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T08:40:30.083 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T08:40:30.083 INFO:teuthology.orchestra.run.smithi082.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:40:30.084 INFO:teuthology.orchestra.run.smithi082.stdout:The following packages will be upgraded: 2024-04-25T08:40:30.085 INFO:teuthology.orchestra.run.smithi082.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T08:40:30.166 INFO:teuthology.orchestra.run.smithi082.stdout:4 upgraded, 39 newly installed, 0 to remove and 301 not upgraded. 2024-04-25T08:40:30.166 INFO:teuthology.orchestra.run.smithi082.stdout:Need to get 55.7 MB of archives. 2024-04-25T08:40:30.166 INFO:teuthology.orchestra.run.smithi082.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T08:40:30.166 INFO:teuthology.orchestra.run.smithi082.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T08:40:30.408 INFO:teuthology.orchestra.run.smithi082.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T08:40:30.416 INFO:teuthology.orchestra.run.smithi082.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T08:40:30.463 INFO:teuthology.orchestra.run.smithi082.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T08:40:30.464 INFO:teuthology.orchestra.run.smithi082.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T08:40:30.464 INFO:teuthology.orchestra.run.smithi082.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T08:40:30.480 INFO:teuthology.orchestra.run.smithi082.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T08:40:30.481 INFO:teuthology.orchestra.run.smithi082.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T08:40:30.483 INFO:teuthology.orchestra.run.smithi082.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T08:40:30.484 INFO:teuthology.orchestra.run.smithi082.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T08:40:30.485 INFO:teuthology.orchestra.run.smithi082.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T08:40:30.486 INFO:teuthology.orchestra.run.smithi082.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T08:40:30.492 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T08:40:30.496 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:40:30.496 INFO:teuthology.orchestra.run.smithi082.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T08:40:30.511 INFO:teuthology.orchestra.run.smithi082.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T08:40:30.512 INFO:teuthology.orchestra.run.smithi082.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T08:40:30.513 INFO:teuthology.orchestra.run.smithi082.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T08:40:30.598 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:40:30.888 INFO:teuthology.orchestra.run.smithi082.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T08:40:30.892 INFO:teuthology.orchestra.run.smithi082.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T08:40:30.892 INFO:teuthology.orchestra.run.smithi082.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T08:40:30.895 INFO:teuthology.orchestra.run.smithi082.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T08:40:30.895 INFO:teuthology.orchestra.run.smithi082.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T08:40:30.899 INFO:teuthology.orchestra.run.smithi082.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T08:40:30.901 INFO:teuthology.orchestra.run.smithi082.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T08:40:30.914 INFO:teuthology.orchestra.run.smithi082.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T08:40:30.915 INFO:teuthology.orchestra.run.smithi082.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T08:40:30.953 INFO:teuthology.orchestra.run.smithi082.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T08:40:30.959 INFO:teuthology.orchestra.run.smithi082.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T08:40:30.974 INFO:teuthology.orchestra.run.smithi082.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T08:40:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:40:31.009 INFO:teuthology.orchestra.run.smithi082.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T08:40:31.020 INFO:teuthology.orchestra.run.smithi082.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T08:40:31.020 INFO:teuthology.orchestra.run.smithi082.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T08:40:31.021 INFO:teuthology.orchestra.run.smithi082.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T08:40:31.022 INFO:teuthology.orchestra.run.smithi082.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T08:40:31.033 INFO:teuthology.orchestra.run.smithi082.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T08:40:31.044 INFO:teuthology.orchestra.run.smithi082.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T08:40:31.138 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:40:31.335 INFO:teuthology.orchestra.run.smithi082.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T08:40:31.335 INFO:teuthology.orchestra.run.smithi082.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T08:40:31.335 INFO:teuthology.orchestra.run.smithi082.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T08:40:31.335 INFO:teuthology.orchestra.run.smithi082.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T08:40:31.335 INFO:teuthology.orchestra.run.smithi082.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T08:40:31.335 INFO:teuthology.orchestra.run.smithi082.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T08:40:31.354 INFO:teuthology.orchestra.run.smithi082.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T08:40:31.355 INFO:teuthology.orchestra.run.smithi082.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T08:40:31.488 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:40:31.732 INFO:teuthology.orchestra.run.smithi082.stdout:Preconfiguring packages ... 2024-04-25T08:40:31.855 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package bison. 2024-04-25T08:40:31.916 INFO:teuthology.orchestra.run.smithi082.stdout:Fetched 55.7 MB in 1s (44.2 MB/s) 2024-04-25T08:40:31.937 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T08:40:31.942 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T08:40:31.982 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:40:32.203 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T08:40:32.207 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:40:32.382 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:40:32.391 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T08:40:32.408 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T08:40:32.449 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:40:33.022 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T08:40:33.170 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package dump. 2024-04-25T08:40:33.186 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T08:40:33.228 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T08:40:33.490 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package indent. 2024-04-25T08:40:33.506 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:33.548 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:40:33.818 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T08:40:33.834 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T08:40:33.876 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:40:34.080 INFO:teuthology.orchestra.run.smithi082.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T08:40:34.230 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T08:40:34.246 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T08:40:34.288 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:40:34.337 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package flex. 2024-04-25T08:40:34.407 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140582 files and directories currently installed.) 2024-04-25T08:40:34.411 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T08:40:34.516 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T08:40:34.527 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T08:40:34.533 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T08:40:34.575 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:40:34.911 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T08:40:34.927 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T08:40:34.969 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:40:35.047 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:40:35.216 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T08:40:35.238 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T08:40:35.255 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T08:40:35.296 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:40:35.566 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T08:40:35.582 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T08:40:35.616 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T08:40:35.650 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:40:35.902 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T08:40:35.919 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T08:40:35.960 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:40:36.271 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T08:40:36.275 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:40:36.425 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:40:36.805 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:40:36.923 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:40:37.291 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:40:37.296 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T08:40:37.313 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:40:37.355 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:40:37.616 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T08:40:37.632 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:40:37.674 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:40:37.691 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package bison. 2024-04-25T08:40:37.768 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 140655 files and directories currently installed.) 2024-04-25T08:40:37.771 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T08:40:37.818 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:40:37.943 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T08:40:37.960 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:40:38.002 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:40:38.311 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T08:40:38.326 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T08:40:38.363 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T08:40:38.378 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:40:38.379 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T08:40:38.421 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:40:38.800 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T08:40:38.817 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:40:38.822 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package dump. 2024-04-25T08:40:38.838 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T08:40:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:40:38.889 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T08:40:39.170 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T08:40:39.186 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:39.243 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package indent. 2024-04-25T08:40:39.244 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:39.258 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:39.309 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T08:40:39.622 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:39.646 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T08:40:39.661 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T08:40:39.664 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:40:39.712 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:40:39.959 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T08:40:39.975 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:40.016 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:40:40.141 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T08:40:40.157 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T08:40:40.208 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:40:40.236 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T08:40:40.252 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:40:40.294 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:40:40.502 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T08:40:40.518 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T08:40:40.577 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:40:40.814 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T08:40:40.830 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T08:40:40.872 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:40:41.048 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T08:40:41.064 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T08:40:41.114 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:40:41.151 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package modemmanager. 2024-04-25T08:40:41.167 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:40:41.200 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:40:41.442 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T08:40:41.457 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T08:40:41.509 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:40:41.812 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T08:40:41.824 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager. 2024-04-25T08:40:41.827 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T08:40:41.840 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:40:41.878 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T08:40:41.915 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:40:42.140 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T08:40:42.155 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T08:40:42.231 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:40:42.472 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ppp. 2024-04-25T08:40:42.488 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T08:40:42.858 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:40:43.202 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T08:40:43.216 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T08:40:43.260 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T08:40:43.513 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T08:40:43.527 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T08:40:43.571 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T08:40:43.651 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T08:40:43.667 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:40:43.709 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:40:43.983 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T08:40:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T08:40:44.004 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T08:40:44.020 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:40:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T08:40:44.062 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:40:44.312 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T08:40:44.328 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:44.373 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T08:40:44.389 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:40:44.403 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:44.431 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:40:44.818 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T08:40:44.834 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T08:40:44.876 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:40:44.983 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T08:40:45.000 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T08:40:45.061 INFO:teuthology.orchestra.run.smithi046.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T08:40:45.091 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-common (238) ... 2024-04-25T08:40:45.237 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T08:40:45.253 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:40:45.296 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:40:45.565 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T08:40:45.582 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:45.632 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T08:40:45.643 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:45.648 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:45.682 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:46.444 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T08:40:46.460 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:46.502 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:40:46.805 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T08:40:46.820 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:46.864 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:40:46.867 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql. 2024-04-25T08:40:46.883 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T08:40:46.926 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql (14+238) ... 2024-04-25T08:40:47.108 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T08:40:47.124 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:40:47.153 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T08:40:47.157 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:40:47.168 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T08:40:47.211 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T08:40:47.455 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T08:40:47.471 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T08:40:47.513 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T08:40:47.653 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T08:40:47.669 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T08:40:47.728 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:40:47.807 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T08:40:47.824 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T08:40:47.865 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:40:48.048 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package modemmanager. 2024-04-25T08:40:48.064 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:40:48.097 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:40:48.185 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl. 2024-04-25T08:40:48.201 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T08:40:48.248 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T08:40:48.563 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T08:40:48.579 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T08:40:48.612 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:40:48.713 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package network-manager. 2024-04-25T08:40:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:40:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:40:48.908 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T08:40:48.923 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T08:40:48.975 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T08:40:49.403 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T08:40:49.419 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T08:40:49.420 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ppp. 2024-04-25T08:40:49.436 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T08:40:49.469 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:40:49.747 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T08:40:49.769 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:40:49.934 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T08:40:50.051 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:40:50.078 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T08:40:50.089 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T08:40:50.166 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T08:40:50.177 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:50.303 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:40:50.403 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T08:40:50.417 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T08:40:50.429 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:40:50.461 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T08:40:50.580 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:50.781 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T08:40:50.795 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T08:40:50.839 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T08:40:51.168 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T08:40:51.182 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:51.226 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:51.556 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T08:40:51.652 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:40:51.882 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T08:40:51.896 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T08:40:51.982 INFO:teuthology.orchestra.run.smithi082.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T08:40:52.014 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-common (238) ... 2024-04-25T08:40:52.087 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:40:52.087 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:40:52.431 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T08:40:52.446 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:40:52.484 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:52.797 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:40:52.923 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:40:53.075 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:40:53.201 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T08:40:53.327 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:40:53.453 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:40:53.579 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:40:53.705 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:40:53.751 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql. 2024-04-25T08:40:53.766 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T08:40:53.809 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql (14+238) ... 2024-04-25T08:40:54.078 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T08:40:54.093 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T08:40:54.128 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T08:40:54.355 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T08:40:54.371 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T08:40:54.405 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T08:40:54.724 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T08:40:54.740 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T08:40:54.769 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:40:54.808 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:40:54.895 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T08:40:54.990 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T08:40:55.055 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:40:55.101 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package tcl. 2024-04-25T08:40:55.117 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T08:40:55.156 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:40:55.164 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T08:40:55.282 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T08:40:55.382 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T08:40:55.509 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:40:55.523 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T08:40:55.539 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T08:40:55.581 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:40:55.607 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T08:40:55.677 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:40:55.803 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:40:55.944 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T08:40:55.960 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T08:40:55.979 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:40:56.002 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T08:40:56.105 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:40:56.231 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T08:40:56.390 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T08:40:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T08:40:56.438 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T08:40:56.489 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:40:56.500 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:40:56.789 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T08:40:57.028 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T08:40:57.118 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:40:57.119 INFO:teuthology.orchestra.run.smithi046.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T08:40:57.119 INFO:teuthology.orchestra.run.smithi046.stdout:which means they are currently configured by ifupdown: 2024-04-25T08:40:57.119 INFO:teuthology.orchestra.run.smithi046.stdout:- ens1f0 2024-04-25T08:40:57.119 INFO:teuthology.orchestra.run.smithi046.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T08:40:57.119 INFO:teuthology.orchestra.run.smithi046.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T08:40:57.119 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:40:57.154 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:40:57.258 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T08:40:57.280 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:57.406 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:40:57.549 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:40:57.650 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:40:57.696 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T08:40:58.124 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T08:40:58.595 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T08:40:58.697 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:40:58.819 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:40:58.979 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:40:59.076 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:40:59.076 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:40:59.105 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T08:40:59.256 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-common (238) ... 2024-04-25T08:40:59.742 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:40:59.868 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:40:59.994 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:41:00.153 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T08:41:00.200 INFO:teuthology.orchestra.run.smithi046.stdout:Adding user postgres to group ssl-cert 2024-04-25T08:41:00.288 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:41:00.397 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:41:00.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:00.454 INFO:teuthology.orchestra.run.smithi046.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T08:41:00.506 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:41:00.641 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:41:00.709 INFO:teuthology.orchestra.run.smithi046.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T08:41:00.709 INFO:teuthology.orchestra.run.smithi046.stdout:Removing obsolete dictionary files: 2024-04-25T08:41:01.413 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:41:01.416 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T08:41:01.556 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T08:41:01.642 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T08:41:01.690 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:41:01.808 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:41:01.967 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T08:41:02.006 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:41:02.093 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T08:41:02.219 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:41:02.309 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T08:41:02.346 INFO:teuthology.orchestra.run.smithi046.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T08:41:02.359 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T08:41:02.379 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout:This user must also own the server process. 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout:The default text search configuration will be set to "english". 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout:Data page checksums are disabled. 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:02.415 INFO:teuthology.orchestra.run.smithi046.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T08:41:02.416 INFO:teuthology.orchestra.run.smithi046.stdout:creating subdirectories ... ok 2024-04-25T08:41:02.416 INFO:teuthology.orchestra.run.smithi046.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T08:41:02.440 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default max_connections ... 100 2024-04-25T08:41:02.475 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default shared_buffers ... 128MB 2024-04-25T08:41:02.490 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default time zone ... Etc/UTC 2024-04-25T08:41:02.491 INFO:teuthology.orchestra.run.smithi046.stdout:creating configuration files ... ok 2024-04-25T08:41:02.530 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:41:02.665 INFO:teuthology.orchestra.run.smithi046.stdout:running bootstrap script ... ok 2024-04-25T08:41:02.681 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:41:02.782 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:41:02.933 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T08:41:03.101 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T08:41:03.228 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:41:03.279 INFO:teuthology.orchestra.run.smithi046.stdout:performing post-bootstrap initialization ... ok 2024-04-25T08:41:03.760 INFO:teuthology.orchestra.run.smithi046.stdout:syncing data to disk ... ok 2024-04-25T08:41:04.066 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:04.067 INFO:teuthology.orchestra.run.smithi082.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T08:41:04.067 INFO:teuthology.orchestra.run.smithi082.stdout:which means they are currently configured by ifupdown: 2024-04-25T08:41:04.067 INFO:teuthology.orchestra.run.smithi082.stdout:- enp3s0f1 2024-04-25T08:41:04.067 INFO:teuthology.orchestra.run.smithi082.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T08:41:04.067 INFO:teuthology.orchestra.run.smithi082.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T08:41:04.067 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:04.205 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T08:41:04.290 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T08:41:04.616 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T08:41:05.015 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T08:41:05.713 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:41:05.840 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:41:05.957 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T08:41:06.108 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-common (238) ... 2024-04-25T08:41:07.011 INFO:teuthology.orchestra.run.smithi082.stdout:Adding user postgres to group ssl-cert 2024-04-25T08:41:07.232 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:07.232 INFO:teuthology.orchestra.run.smithi082.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T08:41:07.280 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:41:07.406 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T08:41:07.501 INFO:teuthology.orchestra.run.smithi082.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T08:41:07.501 INFO:teuthology.orchestra.run.smithi082.stdout:Removing obsolete dictionary files: 2024-04-25T08:41:07.515 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql (14+238) ... 2024-04-25T08:41:07.624 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:41:07.793 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:41:08.057 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:41:08.057 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:41:08.224 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T08:41:08.778 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:41:08.826 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:41:08.870 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T08:41:09.175 INFO:teuthology.orchestra.run.smithi082.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T08:41:09.194 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T08:41:09.230 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout:This user must also own the server process. 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout:The default text search configuration will be set to "english". 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout:Data page checksums are disabled. 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:09.260 INFO:teuthology.orchestra.run.smithi082.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T08:41:09.261 INFO:teuthology.orchestra.run.smithi082.stdout:creating subdirectories ... ok 2024-04-25T08:41:09.261 INFO:teuthology.orchestra.run.smithi082.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T08:41:09.285 INFO:teuthology.orchestra.run.smithi082.stdout:selecting default max_connections ... 100 2024-04-25T08:41:09.319 INFO:teuthology.orchestra.run.smithi082.stdout:selecting default shared_buffers ... 128MB 2024-04-25T08:41:09.333 INFO:teuthology.orchestra.run.smithi082.stdout:selecting default time zone ... Etc/UTC 2024-04-25T08:41:09.334 INFO:teuthology.orchestra.run.smithi082.stdout:creating configuration files ... ok 2024-04-25T08:41:09.382 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:41:09.496 INFO:teuthology.orchestra.run.smithi082.stdout:running bootstrap script ... ok 2024-04-25T08:41:10.121 INFO:teuthology.orchestra.run.smithi082.stdout:performing post-bootstrap initialization ... ok 2024-04-25T08:41:10.680 INFO:teuthology.orchestra.run.smithi082.stdout:syncing data to disk ... ok 2024-04-25T08:41:10.827 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T08:41:10.928 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:41:11.171 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T08:41:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:41:11.474 DEBUG:teuthology.parallel:result is None 2024-04-25T08:41:14.276 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:41:14.402 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T08:41:14.530 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql (14+238) ... 2024-04-25T08:41:14.687 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:41:14.805 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:41:15.041 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:41:15.042 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:41:15.765 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:41:15.858 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T08:41:16.176 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:41:16.345 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:41:17.840 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T08:41:17.932 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:41:18.516 INFO:teuthology.orchestra.run.smithi082.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:41:18.523 DEBUG:teuthology.parallel:result is None 2024-04-25T08:41:18.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:41:18.672 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-query -W -f '${Version}' ceph 2024-04-25T08:41:18.696 INFO:teuthology.orchestra.run.smithi046.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:41:18.697 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:41:18.697 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T08:41:18.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:41:18.834 DEBUG:teuthology.orchestra.run.smithi082:> dpkg-query -W -f '${Version}' ceph 2024-04-25T08:41:18.855 INFO:teuthology.orchestra.run.smithi082.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:41:18.855 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:41:18.855 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T08:41:18.857 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T08:41:18.857 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:18.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:41:18.879 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:18.879 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:41:18.916 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T08:41:18.916 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:18.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:41:18.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:41:19.004 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:19.004 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:41:19.022 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:41:19.087 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T08:41:19.087 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:19.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:41:19.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:41:19.172 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:19.172 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:41:19.192 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:41:19.255 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T08:41:19.255 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:19.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:41:19.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:41:19.345 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:19.345 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:41:19.363 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:41:19.423 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T08:41:19.508 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T08:41:19.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2024-04-25T08:41:19.512 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /var/log/ceph 2024-04-25T08:41:19.530 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T08:41:19.531 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:41:19.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:41:19.593 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T08:41:19.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:41:19.639 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:41:19.663 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T08:41:19.663 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T08:41:19.664 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T08:41:19.664 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:19.664 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:41:19.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:41:19.802 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:19.802 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:41:19.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:41:19.904 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T08:41:19.904 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T08:41:19.904 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7672993', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': '5787d42d10024ba410e822a8aba474331cb9af02'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5869, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '5787d42d10024ba410e822a8aba474331cb9af02', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNzYYf0hWcYLWqGvImj9urzBqpKCbclPzRJB7fnHmkQaAcrhtHrqskzVvwmVxrz5G5m5xnFe6dCYRkJlLWWEvWE=', 'smithi082.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDJULxo3rh4z/CpKYJo4+AyxP4j2+VFk9/h9QDz/570oMboG93QisQsCxYIYLOXm6arpeUuI4D/r+91i3QAQdRc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_08:15:18', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T08:41:19.904 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:41:19.908 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:41:19.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:41:19.959 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:41:20.019 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:20.020 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:41:20.029 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:41:20.029 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-04-25T08:41:20.081 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:41:20.081 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:41:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 898 Links: 1 2024-04-25T08:41:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:41:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-25 08:34:50.771412483 +0000 2024-04-25T08:41:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-25 08:34:50.491414981 +0000 2024-04-25T08:41:20.081 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-25 08:34:50.491414981 +0000 2024-04-25T08:41:20.081 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-25 08:34:50.491414981 +0000 2024-04-25T08:41:20.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:41:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-25T08:41:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-25T08:41:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000274497 s, 1.9 MB/s 2024-04-25T08:41:20.145 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:41:20.198 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-04-25T08:41:20.253 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:41:20.253 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:41:20.253 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 913 Links: 1 2024-04-25T08:41:20.253 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:41:20.253 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-25 08:34:51.383407024 +0000 2024-04-25T08:41:20.253 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-25 08:34:51.075409771 +0000 2024-04-25T08:41:20.253 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-25 08:34:51.075409771 +0000 2024-04-25T08:41:20.253 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-25 08:34:51.075409771 +0000 2024-04-25T08:41:20.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:41:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-25T08:41:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-25T08:41:20.319 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000268069 s, 1.9 MB/s 2024-04-25T08:41:20.321 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:41:20.374 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-04-25T08:41:20.424 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:41:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:41:20.425 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 929 Links: 1 2024-04-25T08:41:20.425 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:41:20.425 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-25 08:34:51.643404704 +0000 2024-04-25T08:41:20.425 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-25 08:34:51.639404739 +0000 2024-04-25T08:41:20.425 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-25 08:34:51.639404739 +0000 2024-04-25T08:41:20.425 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-25 08:34:51.639404739 +0000 2024-04-25T08:41:20.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:41:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-25T08:41:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-25T08:41:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000265801 s, 1.9 MB/s 2024-04-25T08:41:20.489 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:41:20.542 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-04-25T08:41:20.592 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:41:20.593 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:41:20.593 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 6h/6d Inode: 947 Links: 1 2024-04-25T08:41:20.593 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:41:20.593 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-04-25 08:34:52.251399280 +0000 2024-04-25T08:41:20.593 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-04-25 08:34:52.247399315 +0000 2024-04-25T08:41:20.593 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-04-25 08:34:52.247399315 +0000 2024-04-25T08:41:20.593 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-04-25 08:34:52.247399315 +0000 2024-04-25T08:41:20.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:41:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-04-25T08:41:20.655 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-04-25T08:41:20.656 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000273102 s, 1.9 MB/s 2024-04-25T08:41:20.657 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:41:20.710 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:20.711 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:20.711 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:41:20.719 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:41:20.719 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-04-25T08:41:20.769 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:41:20.769 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:41:20.769 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 904 Links: 1 2024-04-25T08:41:20.770 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:41:20.770 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-25 08:34:50.749730732 +0000 2024-04-25T08:41:20.770 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-25 08:34:50.493733016 +0000 2024-04-25T08:41:20.770 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-25 08:34:50.493733016 +0000 2024-04-25T08:41:20.770 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-25 08:34:50.493733016 +0000 2024-04-25T08:41:20.770 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:41:20.831 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-25T08:41:20.831 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-25T08:41:20.831 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000394352 s, 1.3 MB/s 2024-04-25T08:41:20.833 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:41:20.887 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-04-25T08:41:20.937 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:41:20.937 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:41:20.937 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 918 Links: 1 2024-04-25T08:41:20.937 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:41:20.937 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-25 08:34:51.337725486 +0000 2024-04-25T08:41:20.937 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-25 08:34:51.049728056 +0000 2024-04-25T08:41:20.938 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-25 08:34:51.049728056 +0000 2024-04-25T08:41:20.938 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-25 08:34:51.049728056 +0000 2024-04-25T08:41:20.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:41:20.999 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-25T08:41:20.999 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-25T08:41:21.000 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000383172 s, 1.3 MB/s 2024-04-25T08:41:21.002 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:41:21.055 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-04-25T08:41:21.109 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:41:21.109 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:41:21.110 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 936 Links: 1 2024-04-25T08:41:21.110 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:41:21.110 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-25 08:34:51.869720740 +0000 2024-04-25T08:41:21.110 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-25 08:34:51.601723131 +0000 2024-04-25T08:41:21.110 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-25 08:34:51.601723131 +0000 2024-04-25T08:41:21.110 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-25 08:34:51.601723131 +0000 2024-04-25T08:41:21.110 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:41:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-25T08:41:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-25T08:41:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000401706 s, 1.3 MB/s 2024-04-25T08:41:21.178 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:41:21.235 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-04-25T08:41:21.285 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:41:21.285 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:41:21.285 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 953 Links: 1 2024-04-25T08:41:21.285 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:41:21.286 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-25 08:34:52.417715852 +0000 2024-04-25T08:41:21.286 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-25 08:34:52.161718136 +0000 2024-04-25T08:41:21.286 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-25 08:34:52.161718136 +0000 2024-04-25T08:41:21.286 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-04-25 08:34:52.161718136 +0000 2024-04-25T08:41:21.286 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:41:21.352 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-25T08:41:21.352 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-25T08:41:21.352 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000397014 s, 1.3 MB/s 2024-04-25T08:41:21.354 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:41:21.407 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:21.407 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:41:21.407 INFO:tasks.ceph:Generating config... 2024-04-25T08:41:21.408 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T08:41:21.409 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T08:41:21.409 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T08:41:21.409 INFO:tasks.ceph:[client] pid file = 2024-04-25T08:41:21.409 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T08:41:21.409 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T08:41:21.409 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T08:41:21.409 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T08:41:21.409 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T08:41:21.409 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T08:41:21.409 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T08:41:21.409 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T08:41:21.409 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T08:41:21.409 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T08:41:21.409 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T08:41:21.409 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T08:41:21.410 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T08:41:21.410 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T08:41:21.410 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T08:41:21.410 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T08:41:21.410 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T08:41:21.410 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T08:41:21.410 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T08:41:21.410 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T08:41:21.410 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T08:41:21.410 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T08:41:21.410 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T08:41:21.410 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T08:41:21.410 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T08:41:21.410 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T08:41:21.410 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T08:41:21.410 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T08:41:21.411 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T08:41:21.411 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T08:41:21.411 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T08:41:21.411 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T08:41:21.411 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T08:41:21.411 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T08:41:21.459 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T08:41:21.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T08:41:21.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:41:21.542 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.46'), ('mon.b', '172.21.15.82'), ('mon.c', '[v2:172.21.15.82:3301,v1:172.21.15.82:6790]')] 2024-04-25T08:41:21.542 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T08:41:21.546 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:21.546 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:41:21.592 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.46 --add b 172.21.15.82 --addv c '[v2:172.21.15.82:3301,v1:172.21.15.82:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: generated fsid 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 0 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:last_changed 2024-04-25T08:41:21.690301+0000 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-04-25T08:41:21.690301+0000 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:min_mon_release 17 (quincy) 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:election_strategy: 1 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-04-25T08:41:21.683 INFO:teuthology.orchestra.run.smithi046.stdout:1: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.b 2024-04-25T08:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout:2: [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] mon.c 2024-04-25T08:41:21.684 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T08:41:21.686 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:41:21.736 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9ca516a0-1b6e-4031-bb83-1b30eee026c8... 2024-04-25T08:41:21.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:41:21.783 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:41:21.823 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-25T08:41:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-25T08:41:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:41:21.823 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-25T08:41:21.824 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:41:21.825 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-25T08:41:21.826 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-25T08:41:21.827 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.828 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-25T08:41:21.829 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:41:21.830 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:41:21.831 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-25T08:41:21.832 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:41:21.833 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:41:21.834 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T08:41:21.835 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:41:21.836 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on skipped message = False 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.837 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T08:41:21.839 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T08:41:21.840 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T08:41:21.841 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T08:41:21.842 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T08:41:21.843 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = True 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = True 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:41:21.844 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:41:21.846 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T08:41:21.846 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T08:41:21.946 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T08:41:21.946 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:21.946 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T08:41:21.955 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:21.955 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T08:41:22.003 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:41:22.004 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:22.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:41:22.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:41:22.075 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:22.075 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:22.123 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:41:22.123 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:22.124 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:41:22.124 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:41:22.158 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:22.158 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:22.213 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T08:41:22.213 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T08:41:22.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:41:22.269 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:41:22.274 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:41:22.333 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:41:22.356 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T08:41:22.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:41:22.412 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:41:22.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T08:41:22.436 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:41:22.535 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:41:22.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T08:41:22.560 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:41:22.619 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:41:22.624 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T08:41:22.644 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:41:22.742 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:41:22.746 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T08:41:22.766 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T08:41:22.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T08:41:22.810 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T08:41:22.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T08:41:22.914 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T08:41:22.931 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T08:41:22.972 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T08:41:22.989 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T08:41:23.073 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T08:41:23.089 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T08:41:23.090 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:41:23.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T08:41:23.109 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:23.110 INFO:tasks.ceph:role: osd.0 2024-04-25T08:41:23.110 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:41:23.110 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:41:23.171 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:41:23.199 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-04-25T08:41:23.201 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-04-25T08:41:23.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T08:41:23.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T08:41:23.335 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:41:23.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:23.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T08:41:23.403 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:23.403 INFO:tasks.ceph:role: osd.1 2024-04-25T08:41:23.403 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:41:23.403 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:41:23.463 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:41:23.489 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-04-25T08:41:23.492 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-04-25T08:41:23.492 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T08:41:23.560 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T08:41:23.619 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:41:23.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:23.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T08:41:23.683 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:23.683 INFO:tasks.ceph:role: osd.2 2024-04-25T08:41:23.683 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:41:23.683 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:41:23.742 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:41:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:41:23.742 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:41:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:41:23.743 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:41:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:41:23.743 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:41:23.743 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:41:23.743 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:41:23.743 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:41:23.769 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-04-25T08:41:23.771 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-04-25T08:41:23.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T08:41:23.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T08:41:23.895 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:41:23.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:23.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T08:41:23.959 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:23.959 INFO:tasks.ceph:role: osd.3 2024-04-25T08:41:23.959 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:41:23.959 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:41:24.019 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:41:24.019 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:41:24.019 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:41:24.019 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:41:24.019 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:41:24.019 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:41:24.020 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:41:24.020 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:41:24.020 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:41:24.020 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:41:24.045 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-04-25T08:41:24.048 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-04-25T08:41:24.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T08:41:24.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T08:41:24.183 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:41:24.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:24.185 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:24.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:24.310+0000 7f2e638a47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T08:41:24.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:24.310+0000 7f2e638a47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T08:41:24.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:24.310+0000 7f2e638a47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:41:24.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:24.310+0000 7f2e638a47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:41:24.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:24.310+0000 7f2e638a47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:41:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:24.314+0000 7f2e638a47c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T08:41:27.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T08:41:27.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:27.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:27.450+0000 7f7eb27e87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T08:41:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:27.450+0000 7f7eb27e87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T08:41:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:27.450+0000 7f7eb27e87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:41:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:27.450+0000 7f7eb27e87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:41:27.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:27.450+0000 7f7eb27e87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:41:27.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:27.450+0000 7f7eb27e87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T08:41:30.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T08:41:30.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:30.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:30.630+0000 7f10da05b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T08:41:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:30.630+0000 7f10da05b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T08:41:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:30.630+0000 7f10da05b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:41:30.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:30.630+0000 7f10da05b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:41:30.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:30.630+0000 7f10da05b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:41:30.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:30.634+0000 7f10da05b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T08:41:33.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T08:41:33.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:33.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:33.766+0000 7f8f41c957c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T08:41:33.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:33.770+0000 7f8f41c957c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T08:41:33.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:33.770+0000 7f8f41c957c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:41:33.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:33.770+0000 7f8f41c957c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:41:33.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:33.770+0000 7f8f41c957c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:41:33.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:33.770+0000 7f8f41c957c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T08:41:36.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T08:41:36.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T08:41:36.845 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:36.845 INFO:tasks.ceph:role: osd.4 2024-04-25T08:41:36.845 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:41:36.845 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:41:36.919 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:41:36.920 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:41:36.920 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:41:36.920 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:41:36.920 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:41:36.920 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:41:36.920 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:41:36.920 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:41:36.920 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:41:36.920 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:41:37.322 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-04-25T08:41:37.326 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-04-25T08:41:37.326 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T08:41:37.421 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T08:41:37.438 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:41:37.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:37.439 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T08:41:37.505 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:37.506 INFO:tasks.ceph:role: osd.5 2024-04-25T08:41:37.506 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:41:37.506 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:41:37.577 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:41:37.578 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:41:37.578 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:41:37.578 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:41:37.578 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:41:37.578 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:41:37.578 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:41:37.578 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:41:37.578 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:41:37.578 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:41:37.974 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-04-25T08:41:37.977 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-04-25T08:41:37.977 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T08:41:38.109 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T08:41:38.126 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:41:38.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:38.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T08:41:38.193 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:38.193 INFO:tasks.ceph:role: osd.6 2024-04-25T08:41:38.193 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:41:38.193 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:41:38.269 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:41:38.682 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-04-25T08:41:38.685 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-04-25T08:41:38.685 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T08:41:38.806 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T08:41:38.823 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:41:38.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:38.825 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T08:41:38.893 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:41:38.893 INFO:tasks.ceph:role: osd.7 2024-04-25T08:41:38.893 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:41:38.893 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:41:38.964 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:41:38.965 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:41:38.965 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:41:38.965 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:41:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:41:38.965 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:41:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:41:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:41:38.965 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:41:38.965 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:41:39.330 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-04-25T08:41:39.334 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-04-25T08:41:39.334 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T08:41:39.419 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T08:41:39.476 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:41:39.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:39.478 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:39.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:39.605+0000 7f2b600e87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T08:41:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:39.605+0000 7f2b600e87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T08:41:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:39.605+0000 7f2b600e87c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:41:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:39.605+0000 7f2b600e87c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:41:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:39.605+0000 7f2b600e87c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:41:39.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:39.609+0000 7f2b600e87c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T08:41:42.615 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T08:41:42.635 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:42.733+0000 7fe46e52d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T08:41:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:42.733+0000 7fe46e52d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T08:41:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:42.737+0000 7fe46e52d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:41:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:42.737+0000 7fe46e52d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:41:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:42.737+0000 7fe46e52d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:41:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:42.737+0000 7fe46e52d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T08:41:45.804 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T08:41:45.825 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:45.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:45.933+0000 7f920afb07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T08:41:45.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:45.933+0000 7f920afb07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T08:41:45.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:45.933+0000 7f920afb07c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:41:45.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:45.933+0000 7f920afb07c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:41:45.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:45.933+0000 7f920afb07c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:41:45.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:45.937+0000 7f920afb07c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T08:41:48.990 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T08:41:49.010 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:49.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:49.109+0000 7f525fcb67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T08:41:49.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:49.113+0000 7f525fcb67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T08:41:49.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:49.113+0000 7f525fcb67c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:41:49.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:49.113+0000 7f525fcb67c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:41:49.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:49.113+0000 7f525fcb67c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:41:49.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:49.113+0000 7f525fcb67c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T08:41:49.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:41:49.910 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:41:52.130 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T08:41:52.151 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T08:41:52.151 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:52.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T08:41:52.174 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:52.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T08:41:52.241 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:52.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T08:41:52.309 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:52.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T08:41:52.377 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:52.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T08:41:52.445 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:52.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T08:41:52.509 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:52.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T08:41:52.577 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:52.577 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T08:41:52.599 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:52.599 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T08:41:52.661 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:52.661 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T08:41:52.725 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:52.725 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T08:41:52.788 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:52.788 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T08:41:52.848 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:52.848 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T08:41:52.908 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:52.908 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T08:41:52.968 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:52.968 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T08:41:52.977 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:52.977 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T08:41:53.028 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:53.028 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T08:41:53.036 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:53.036 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T08:41:53.084 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T08:41:53.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:41:53.087 DEBUG:teuthology.orchestra.run.smithi082:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:41:53.128 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmAY6CEBAA2qHNALNTRKmCJRZatDqAvw== 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmq+gDGRAAtB5ObM/zelMmCZ4cJPghdQ== 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmFahWIBAAORHmzDLXPjuANF+/W2wKpQ== 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQA0FypmaHiqEhAAYzasCl8t+pls5IHe9tCP4g== 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQA3FypmOGTuGhAA37F+qRI/+pj5dkLc3TBIUQ== 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQA6Fypm/YbNJRAAP1QyYDLY/xFvRz63R+H/Pw== 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2024-04-25T08:41:53.129 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQA9FypmiT8ALhAAX/2CJJeJ+2SGNhwah8NTBQ== 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmzhptFBAAlFQZQn1GSseEguy5zfw2CA== 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmyIaAJRAA9zodtgftP7faNth4UPyCTA== 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmRhrPLBAATXOHl1jGJYmIlnYk/JPcOw== 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBDFypmyG88JBAAq1065gzLIIR/6fYaw0bBTg== 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBGFypmNir3KxAAszCXJJ1KFtCusDZhBHAZSQ== 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBJFypm4r7LNxAA42A07ntI6OOgblk+4i1Yiw== 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2024-04-25T08:41:53.130 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQBNFypmG1fOBhAAOKRgw04jIo8H3u2hWIZe2g== 2024-04-25T08:41:53.131 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-25T08:41:53.131 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q== 2024-04-25T08:41:53.131 INFO:teuthology.orchestra.run.smithi046.stdout:[client.2] 2024-04-25T08:41:53.131 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypm2PP0NhAAixV/kkY7eFPXUW/nJDOCOA== 2024-04-25T08:41:53.131 INFO:teuthology.orchestra.run.smithi046.stdout:[client.1] 2024-04-25T08:41:53.131 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmxPyHOhAADBuigZlq3GHURsJkB0nnyA== 2024-04-25T08:41:53.131 INFO:teuthology.orchestra.run.smithi046.stdout:[client.3] 2024-04-25T08:41:53.131 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAzFypmaVDtBBAAKnWk9CxZzg2r6T/IycuFEg== 2024-04-25T08:41:53.142 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.y] 2024-04-25T08:41:53.142 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAyFypmAY6CEBAA2qHNALNTRKmCJRZatDqAvw== 2024-04-25T08:41:53.142 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T08:41:53.142 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAyFypmq+gDGRAAtB5ObM/zelMmCZ4cJPghdQ== 2024-04-25T08:41:53.142 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T08:41:53.142 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAyFypmFahWIBAAORHmzDLXPjuANF+/W2wKpQ== 2024-04-25T08:41:53.142 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.0] 2024-04-25T08:41:53.142 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQA0FypmaHiqEhAAYzasCl8t+pls5IHe9tCP4g== 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.1] 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQA3FypmOGTuGhAA37F+qRI/+pj5dkLc3TBIUQ== 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.2] 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQA6Fypm/YbNJRAAP1QyYDLY/xFvRz63R+H/Pw== 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.3] 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQA9FypmiT8ALhAAX/2CJJeJ+2SGNhwah8NTBQ== 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.x] 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAyFypmzhptFBAAlFQZQn1GSseEguy5zfw2CA== 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAyFypmyIaAJRAA9zodtgftP7faNth4UPyCTA== 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAyFypmRhrPLBAATXOHl1jGJYmIlnYk/JPcOw== 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.4] 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBDFypmyG88JBAAq1065gzLIIR/6fYaw0bBTg== 2024-04-25T08:41:53.143 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.5] 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBGFypmNir3KxAAszCXJJ1KFtCusDZhBHAZSQ== 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.6] 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBJFypm4r7LNxAA42A07ntI6OOgblk+4i1Yiw== 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.7] 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBNFypmG1fOBhAAOKRgw04jIo8H3u2hWIZe2g== 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q== 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout:[client.2] 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAyFypm2PP0NhAAixV/kkY7eFPXUW/nJDOCOA== 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout:[client.1] 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAyFypmxPyHOhAADBuigZlq3GHURsJkB0nnyA== 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout:[client.3] 2024-04-25T08:41:53.144 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAzFypmaVDtBBAAKnWk9CxZzg2r6T/IycuFEg== 2024-04-25T08:41:53.145 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:41:53.179 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:41:53.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:41:53.250 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:41:53.308 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:41:53.312 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:41:53.368 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.371 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.427 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.540 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:41:53.596 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:41:53.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:41:53.652 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:41:53.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:41:53.712 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:41:53.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.771 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.827 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.882 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:41:53.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:41:53.995 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:41:54.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:41:54.055 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:41:54.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:41:54.114 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:41:54.170 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:41:54.173 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:41:54.228 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T08:41:54.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T08:41:54.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:41:54.358 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T08:41:54.378 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T08:41:54.398 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:41:54.506 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T08:41:54.525 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T08:41:54.587 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:41:54.699 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T08:41:54.718 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:54.722 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:41:54.772 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T08:41:54.772 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T08:41:54.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T08:41:54.775 INFO:tasks.ceph.mon.a:Started 2024-04-25T08:41:54.775 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T08:41:54.776 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T08:41:54.819 INFO:tasks.ceph.mon.b:Started 2024-04-25T08:41:54.819 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T08:41:54.819 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T08:41:54.822 INFO:tasks.ceph.mon.c:Started 2024-04-25T08:41:54.822 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T08:41:54.822 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T08:41:54.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T08:41:54.825 INFO:tasks.ceph.mgr.y:Started 2024-04-25T08:41:54.826 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T08:41:54.826 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T08:41:54.829 INFO:tasks.ceph.mgr.x:Started 2024-04-25T08:41:54.829 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T08:41:54.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T08:41:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.974+0000 7f1918800640 1 Processor -- start 2024-04-25T08:41:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.974+0000 7f1918800640 1 -- start start 2024-04-25T08:41:54.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1918800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1910145310 0x7f191006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1918800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 0x7f191015d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1918800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19100cd930 con 0x7f191006dc70 2024-04-25T08:41:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1918800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19100cdbd0 con 0x7f1910145310 2024-04-25T08:41:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1918800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19100cdeb0 con 0x7f191015d090 2024-04-25T08:41:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1915e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1910145310 0x7f191006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1915400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f191006dc70 0x7f19100cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48488/0 (socket says 172.21.15.46:48488) 2024-04-25T08:41:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1915400640 1 -- 172.21.15.46:0/856632170 learned_addr learned my addr 172.21.15.46:0/856632170 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:41:54.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1916800640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 0x7f191015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1916800640 1 -- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 msgr2=0x7f191015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T08:41:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1916800640 1 -- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 msgr2=0x7f191015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:41:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1916800640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 0x7f191015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:41:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1916800640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 0x7f191015d520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:41:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1915e00640 1 -- 172.21.15.46:0/856632170 >> v2:172.21.15.46:3300/0 conn(0x7f1910145310 msgr2=0x7f191006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:41:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1915e00640 1 -- 172.21.15.46:0/856632170 >> v2:172.21.15.46:3300/0 conn(0x7f1910145310 msgr2=0x7f191006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:41:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1915e00640 1 --2- 172.21.15.46:0/856632170 >> v2:172.21.15.46:3300/0 conn(0x7f1910145310 0x7f191006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:41:54.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:54.978+0000 7f1915e00640 1 --2- 172.21.15.46:0/856632170 >> v2:172.21.15.46:3300/0 conn(0x7f1910145310 0x7f191006e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:41:55.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909916922 0 0) 0x7f19100cd930 con 0x7f191006dc70 2024-04-25T08:41:55.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f8003530 con 0x7f191006dc70 2024-04-25T08:41:55.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4125658776 0 0) 0x7f18f8003530 con 0x7f191006dc70 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 msgr2=0x7f191015d520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 0x7f191015d520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 >> v2:172.21.15.46:3300/0 conn(0x7f1910145310 msgr2=0x7f191006e0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 --2- 172.21.15.46:0/856632170 >> v2:172.21.15.46:3300/0 conn(0x7f1910145310 0x7f191006e0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19100ce190 con 0x7f191006dc70 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f1900001680 con 0x7f191006dc70 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 >> v1:172.21.15.46:6789/0 conn(0x7f191006dc70 legacy=0x7f19100cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f18f8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 0x7f18f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8004d50 0x7f18f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100cdbd0 con 0x7f18f8003a30 2024-04-25T08:41:55.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100cdeb0 con 0x7f18f8004d50 2024-04-25T08:41:55.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f800ff70 con 0x7f191015d090 2024-04-25T08:41:55.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1915e00640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f18f8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1915400640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 0x7f18f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1916800640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8004d50 0x7f18f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1915e00640 1 -- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 msgr2=0x7f18f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:55.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1915e00640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f191015d090 0x7f18f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1915e00640 1 -- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8004d50 msgr2=0x7f18f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1915e00640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f8004d50 0x7f18f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.114+0000 7f1915e00640 1 -- 172.21.15.46:0/856632170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f8003530 con 0x7f18f8003a30 2024-04-25T08:41:55.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1915e00640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f18f8003e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f190400c820 tx=0x7f190401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf20fdf7c1bcab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400f040 con 0x7f18f8003a30 2024-04-25T08:41:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19040b0070 con 0x7f18f8003a30 2024-04-25T08:41:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1914a00640 1 -- 172.21.15.46:0/856632170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040a7070 con 0x7f18f8003a30 2024-04-25T08:41:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1918800640 1 -- 172.21.15.46:0/856632170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 msgr2=0x7f18f8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1918800640 1 --2- 172.21.15.46:0/856632170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f18f8003e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f190400c820 tx=0x7f190401c440 comp rx=0 tx=0).stop 2024-04-25T08:41:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1916800640 1 -- 172.21.15.46:0/856632170 reap_dead start 2024-04-25T08:41:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1918800640 1 -- 172.21.15.46:0/856632170 shutdown_connections 2024-04-25T08:41:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1918800640 1 -- 172.21.15.46:0/856632170 >> 172.21.15.46:0/856632170 conn(0x7f191006ac10 msgr2=0x7f191006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1918800640 1 -- 172.21.15.46:0/856632170 shutdown_connections 2024-04-25T08:41:55.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.118+0000 7f1918800640 1 -- 172.21.15.46:0/856632170 wait complete. 2024-04-25T08:41:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1918800640 1 Processor -- start 2024-04-25T08:41:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1918800640 1 -- start start 2024-04-25T08:41:55.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1918800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f19101f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1918800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1910158ad0 0x7f19101435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1918800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1910158f50 0x7f1910143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1918800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191015d8b0 con 0x7f18f8003a30 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1918800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19100cf380 con 0x7f1910158ad0 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1918800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19101644f0 con 0x7f1910158f50 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f19101f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1910158ad0 0x7f19101435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f19101f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53192/0 (socket says 172.21.15.46:53192) 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915e00640 1 -- 172.21.15.46:0/2276064568 learned_addr learned my addr 172.21.15.46:0/2276064568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1916800640 1 --2- 172.21.15.46:0/2276064568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1910158f50 0x7f1910143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915e00640 1 -- 172.21.15.46:0/2276064568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1910158f50 msgr2=0x7f1910143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915e00640 1 --2- 172.21.15.46:0/2276064568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1910158f50 0x7f1910143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915e00640 1 -- 172.21.15.46:0/2276064568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1910158ad0 msgr2=0x7f19101435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915e00640 1 --2- 172.21.15.46:0/2276064568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1910158ad0 0x7f19101435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915e00640 1 -- 172.21.15.46:0/2276064568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19101f43e0 con 0x7f18f8003a30 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915400640 1 --2- 172.21.15.46:0/2276064568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1910158ad0 0x7f19101435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.122+0000 7f1915e00640 1 --2- 172.21.15.46:0/2276064568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f19101f5480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f190401c270 tx=0x7f18f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:55.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.126+0000 7f190a000640 1 -- 172.21.15.46:0/2276064568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040c2070 con 0x7f18f8003a30 2024-04-25T08:41:55.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.126+0000 7f190a000640 1 -- 172.21.15.46:0/2276064568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19040ab7a0 con 0x7f18f8003a30 2024-04-25T08:41:55.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.126+0000 7f190a000640 1 -- 172.21.15.46:0/2276064568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040abb90 con 0x7f18f8003a30 2024-04-25T08:41:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.126+0000 7f1918800640 1 -- 172.21.15.46:0/2276064568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19101f46c0 con 0x7f18f8003a30 2024-04-25T08:41:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.126+0000 7f1918800640 1 -- 172.21.15.46:0/2276064568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19101f4c80 con 0x7f18f8003a30 2024-04-25T08:41:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.126+0000 7f190a000640 1 -- 172.21.15.46:0/2276064568 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f19040bd410 con 0x7f18f8003a30 2024-04-25T08:41:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.126+0000 7f190a000640 1 -- 172.21.15.46:0/2276064568 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f19040c6440 con 0x7f18f8003a30 2024-04-25T08:41:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.126+0000 7f1918800640 1 -- 172.21.15.46:0/2276064568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1910069010 con 0x7f18f8003a30 2024-04-25T08:41:55.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.246+0000 7f190a000640 1 -- 172.21.15.46:0/2276064568 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f19040dddd0 con 0x7f18f8003a30 2024-04-25T08:41:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.310+0000 7f1918800640 1 -- 172.21.15.46:0/2276064568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f191015bee0 con 0x7f18f8003a30 2024-04-25T08:41:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.310+0000 7f190a000640 1 -- 172.21.15.46:0/2276064568 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f19040cc020 con 0x7f18f8003a30 2024-04-25T08:41:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-04-25T08:41:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.314+0000 7f1918800640 1 -- 172.21.15.46:0/2276064568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 msgr2=0x7f19101f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.314+0000 7f1918800640 1 --2- 172.21.15.46:0/2276064568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f19101f5480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f190401c270 tx=0x7f18f8005840 comp rx=0 tx=0).stop 2024-04-25T08:41:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.314+0000 7f1918800640 1 -- 172.21.15.46:0/2276064568 shutdown_connections 2024-04-25T08:41:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.314+0000 7f1918800640 1 --2- 172.21.15.46:0/2276064568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1910158f50 0x7f1910143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.314+0000 7f1918800640 1 --2- 172.21.15.46:0/2276064568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1910158ad0 0x7f19101435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.314+0000 7f1918800640 1 --2- 172.21.15.46:0/2276064568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f8003a30 0x7f19101f5480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.314+0000 7f1918800640 1 -- 172.21.15.46:0/2276064568 >> 172.21.15.46:0/2276064568 conn(0x7f191006ac10 msgr2=0x7f19100b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.314+0000 7f1918800640 1 -- 172.21.15.46:0/2276064568 shutdown_connections 2024-04-25T08:41:55.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:55.314+0000 7f1918800640 1 -- 172.21.15.46:0/2276064568 wait complete. 2024-04-25T08:41:55.327 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T08:41:55.327 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T08:41:55.328 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:55.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T08:41:55.337 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:55.345+0000 7f2f75455180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:41:55.348 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:55.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T08:41:55.412 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:55.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T08:41:55.476 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:41:55.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T08:41:55.523 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:55.526+0000 7f905521d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:41:55.540 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:55.540 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T08:41:55.560 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:55.561 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T08:41:55.623 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:55.623 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T08:41:55.684 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:41:55.684 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T08:41:55.748 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new d4c1a6f5-a079-4d33-8583-52aa28158fd8 0 2024-04-25T08:41:55.877 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:55.885+0000 7f2f75455180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:41:55.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.928+0000 7f769a800640 1 Processor -- start 2024-04-25T08:41:55.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.928+0000 7f769a800640 1 -- start start 2024-04-25T08:41:55.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.928+0000 7f769a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7694160ed0 0x7f76940b3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.928+0000 7f769a800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76940addb0 con 0x7f769406d8a0 2024-04-25T08:41:55.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.928+0000 7f769a800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76940ae070 con 0x7f76940b4730 2024-04-25T08:41:55.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.928+0000 7f769a800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76940ae350 con 0x7f7694160ed0 2024-04-25T08:41:55.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7699400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7694160ed0 0x7f76940b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7699e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f76940b4730 0x7f76940aee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55618/0 (socket says 172.21.15.82:55618) 2024-04-25T08:41:55.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7699e00640 1 -- 172.21.15.82:0/3696652730 learned_addr learned my addr 172.21.15.82:0/3696652730 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T08:41:55.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7693e00640 1 -- 172.21.15.82:0/3696652730 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247614143 0 0) 0x7f76940ae070 con 0x7f76940b4730 2024-04-25T08:41:55.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7699400640 1 -- 172.21.15.82:0/3696652730 >> v1:172.21.15.82:6789/0 conn(0x7f76940b4730 legacy=0x7f76940aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7699400640 1 -- 172.21.15.82:0/3696652730 >> v1:172.21.15.46:6789/0 conn(0x7f769406d8a0 legacy=0x7f76940b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7699400640 1 -- 172.21.15.82:0/3696652730 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76940ae630 con 0x7f7694160ed0 2024-04-25T08:41:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7699400640 1 --2- 172.21.15.82:0/3696652730 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7694160ed0 0x7f76940b3770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76880097f0 tx=0x7f768801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd22c7c6f84c6a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7693e00640 1 -- 172.21.15.82:0/3696652730 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76880b0070 con 0x7f7694160ed0 2024-04-25T08:41:55.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7693e00640 1 -- 172.21.15.82:0/3696652730 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76880ab600 con 0x7f7694160ed0 2024-04-25T08:41:55.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.932+0000 7f7693e00640 1 -- 172.21.15.82:0/3696652730 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76880ab9d0 con 0x7f7694160ed0 2024-04-25T08:41:55.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 -- 172.21.15.82:0/3696652730 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7694160ed0 msgr2=0x7f76940b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 --2- 172.21.15.82:0/3696652730 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7694160ed0 0x7f76940b3770 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76880097f0 tx=0x7f768801c450 comp rx=0 tx=0).stop 2024-04-25T08:41:55.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 -- 172.21.15.82:0/3696652730 shutdown_connections 2024-04-25T08:41:55.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 --2- 172.21.15.82:0/3696652730 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7694160ed0 0x7f76940b3770 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 -- 172.21.15.82:0/3696652730 >> 172.21.15.82:0/3696652730 conn(0x7f7694072760 msgr2=0x7f769406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:55.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 -- 172.21.15.82:0/3696652730 shutdown_connections 2024-04-25T08:41:55.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 -- 172.21.15.82:0/3696652730 wait complete. 2024-04-25T08:41:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 Processor -- start 2024-04-25T08:41:55.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 -- start start 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769406d8a0 0x7f7694147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76940b4730 0x7f769414b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7694160ed0 0x7f769415f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769416be00 con 0x7f7694160ed0 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769416bc80 con 0x7f76940b4730 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f769a800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769416bf80 con 0x7f769406d8a0 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f7698a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76940b4730 0x7f769414b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f7699400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769406d8a0 0x7f7694147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f7698a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76940b4730 0x7f769414b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48766/0 (socket says 172.21.15.82:48766) 2024-04-25T08:41:55.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f7698a00640 1 -- 172.21.15.82:0/59131628 learned_addr learned my addr 172.21.15.82:0/59131628 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f7699400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769406d8a0 0x7f7694147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:40266/0 (socket says 172.21.15.82:40266) 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.936+0000 7f7699e00640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7694160ed0 0x7f769415f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7698a00640 1 -- 172.21.15.82:0/59131628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769406d8a0 msgr2=0x7f7694147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7698a00640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769406d8a0 0x7f7694147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7698a00640 1 -- 172.21.15.82:0/59131628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7694160ed0 msgr2=0x7f769415f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7698a00640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7694160ed0 0x7f769415f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7698a00640 1 -- 172.21.15.82:0/59131628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7694147ba0 con 0x7f76940b4730 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7699400640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769406d8a0 0x7f7694147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7699e00640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7694160ed0 0x7f769415f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7698a00640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76940b4730 0x7f769414b160 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f768000c3c0 tx=0x7f768000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7691600640 1 -- 172.21.15.82:0/59131628 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680017070 con 0x7f76940b4730 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7691600640 1 -- 172.21.15.82:0/59131628 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76800052d0 con 0x7f76940b4730 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f769a800640 1 -- 172.21.15.82:0/59131628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769414ba10 con 0x7f76940b4730 2024-04-25T08:41:55.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7691600640 1 -- 172.21.15.82:0/59131628 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680003840 con 0x7f76940b4730 2024-04-25T08:41:55.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f769a800640 1 -- 172.21.15.82:0/59131628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769414bfd0 con 0x7f76940b4730 2024-04-25T08:41:55.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7691600640 1 -- 172.21.15.82:0/59131628 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f768001a070 con 0x7f76940b4730 2024-04-25T08:41:55.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f7691600640 1 -- 172.21.15.82:0/59131628 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f768001c440 con 0x7f76940b4730 2024-04-25T08:41:55.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.940+0000 7f769a800640 1 -- 172.21.15.82:0/59131628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7694069010 con 0x7f76940b4730 2024-04-25T08:41:55.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:55.960+0000 7f7691600640 1 -- 172.21.15.82:0/59131628 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7680021020 con 0x7f76940b4730 2024-04-25T08:41:56.052 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:56.058+0000 7f905521d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:41:56.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.064+0000 7f769a800640 1 -- 172.21.15.82:0/59131628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d4c1a6f5-a079-4d33-8583-52aa28158fd8", "id": 0} v 0) v1 -- 0x7f76940669a0 con 0x7f76940b4730 2024-04-25T08:41:56.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.068+0000 7f7691600640 1 -- 172.21.15.82:0/59131628 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4c1a6f5-a079-4d33-8583-52aa28158fd8", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7680011590 con 0x7f76940b4730 2024-04-25T08:41:56.064 INFO:teuthology.orchestra.run.smithi082.stdout:0 2024-04-25T08:41:56.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.072+0000 7f769a800640 1 -- 172.21.15.82:0/59131628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76940b4730 msgr2=0x7f769414b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.072+0000 7f769a800640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76940b4730 0x7f769414b160 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f768000c3c0 tx=0x7f768000c880 comp rx=0 tx=0).stop 2024-04-25T08:41:56.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.072+0000 7f769a800640 1 -- 172.21.15.82:0/59131628 shutdown_connections 2024-04-25T08:41:56.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.072+0000 7f769a800640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7694160ed0 0x7f769415f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.072+0000 7f769a800640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76940b4730 0x7f769414b160 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.072+0000 7f769a800640 1 --2- 172.21.15.82:0/59131628 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769406d8a0 0x7f7694147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.072+0000 7f769a800640 1 -- 172.21.15.82:0/59131628 >> 172.21.15.82:0/59131628 conn(0x7f7694072760 msgr2=0x7f769415d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.072+0000 7f769a800640 1 -- 172.21.15.82:0/59131628 shutdown_connections 2024-04-25T08:41:56.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.072+0000 7f769a800640 1 -- 172.21.15.82:0/59131628 wait complete. 2024-04-25T08:41:56.083 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new 6a7c621a-0fc4-40b3-aa8e-86048c60c3ad 1 2024-04-25T08:41:56.116 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:56.120+0000 7f2f75455180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:41:56.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc52800640 1 Processor -- start 2024-04-25T08:41:56.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc52800640 1 -- start start 2024-04-25T08:41:56.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc52800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc4c099c60 0x7efc4c160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc52800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c09a040 0x7efc4c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc52800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c0cc530 con 0x7efc4c0cf210 2024-04-25T08:41:56.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc52800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c0cc300 con 0x7efc4c099c60 2024-04-25T08:41:56.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc52800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0cc810 con 0x7efc4c09a040 2024-04-25T08:41:56.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc4be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c09a040 0x7efc4c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc4be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c09a040 0x7efc4c15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:40276/0 (socket says 172.21.15.82:40276) 2024-04-25T08:41:56.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc4be00640 1 -- 172.21.15.82:0/3817285430 learned_addr learned my addr 172.21.15.82:0/3817285430 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:56.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc4b400640 1 --2- 172.21.15.82:0/3817285430 >> v2:172.21.15.46:3300/0 conn(0x7efc4c099c60 0x7efc4c160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc4be00640 1 -- 172.21.15.82:0/3817285430 >> v2:172.21.15.46:3300/0 conn(0x7efc4c099c60 msgr2=0x7efc4c160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc4be00640 1 --2- 172.21.15.82:0/3817285430 >> v2:172.21.15.46:3300/0 conn(0x7efc4c099c60 0x7efc4c160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc4be00640 1 -- 172.21.15.82:0/3817285430 >> v1:172.21.15.46:6789/0 conn(0x7efc4c0cf210 legacy=0x7efc4c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.224+0000 7efc4be00640 1 -- 172.21.15.82:0/3817285430 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c0ccaf0 con 0x7efc4c09a040 2024-04-25T08:41:56.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc4be00640 1 --2- 172.21.15.82:0/3817285430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c09a040 0x7efc4c15eb60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc44009e50 tx=0x7efc4401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b7b75cf5ef9fccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc4aa00640 1 -- 172.21.15.82:0/3817285430 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440a5810 con 0x7efc4c09a040 2024-04-25T08:41:56.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc4aa00640 1 -- 172.21.15.82:0/3817285430 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc440a59b0 con 0x7efc4c09a040 2024-04-25T08:41:56.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc4aa00640 1 -- 172.21.15.82:0/3817285430 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440a5d80 con 0x7efc4c09a040 2024-04-25T08:41:56.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc52800640 1 -- 172.21.15.82:0/3817285430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c09a040 msgr2=0x7efc4c15eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc52800640 1 --2- 172.21.15.82:0/3817285430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c09a040 0x7efc4c15eb60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efc44009e50 tx=0x7efc4401ca30 comp rx=0 tx=0).stop 2024-04-25T08:41:56.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc52800640 1 -- 172.21.15.82:0/3817285430 shutdown_connections 2024-04-25T08:41:56.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc52800640 1 --2- 172.21.15.82:0/3817285430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c09a040 0x7efc4c15eb60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc52800640 1 --2- 172.21.15.82:0/3817285430 >> v2:172.21.15.46:3300/0 conn(0x7efc4c099c60 0x7efc4c160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc52800640 1 -- 172.21.15.82:0/3817285430 >> 172.21.15.82:0/3817285430 conn(0x7efc4c0b3130 msgr2=0x7efc4c0b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc52800640 1 -- 172.21.15.82:0/3817285430 shutdown_connections 2024-04-25T08:41:56.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.228+0000 7efc52800640 1 -- 172.21.15.82:0/3817285430 wait complete. 2024-04-25T08:41:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.232+0000 7efc52800640 1 Processor -- start 2024-04-25T08:41:56.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.232+0000 7efc52800640 1 -- start start 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.232+0000 7efc52800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c099c60 0x7efc4c15df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.232+0000 7efc52800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c09a040 0x7efc4c155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc52800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4c0cf210 0x7efc4c155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc52800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0b1930 con 0x7efc4c09a040 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc52800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0b17b0 con 0x7efc4c0cf210 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc52800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0b1ab0 con 0x7efc4c099c60 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c099c60 0x7efc4c15df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4c0cf210 0x7efc4c155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c099c60 0x7efc4c15df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:40284/0 (socket says 172.21.15.82:40284) 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4b400640 1 -- 172.21.15.82:0/2005529424 learned_addr learned my addr 172.21.15.82:0/2005529424 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:56.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc43e00640 1 --2- 172.21.15.82:0/2005529424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c09a040 0x7efc4c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4b400640 1 -- 172.21.15.82:0/2005529424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4c0cf210 msgr2=0x7efc4c155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4b400640 1 --2- 172.21.15.82:0/2005529424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4c0cf210 0x7efc4c155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4b400640 1 -- 172.21.15.82:0/2005529424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c09a040 msgr2=0x7efc4c155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4b400640 1 --2- 172.21.15.82:0/2005529424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c09a040 0x7efc4c155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4b400640 1 -- 172.21.15.82:0/2005529424 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c0b3d90 con 0x7efc4c099c60 2024-04-25T08:41:56.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc4b400640 1 --2- 172.21.15.82:0/2005529424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c099c60 0x7efc4c15df00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00e870 tx=0x7efc3c00ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc43400640 1 -- 172.21.15.82:0/2005529424 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c017070 con 0x7efc4c099c60 2024-04-25T08:41:56.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc43400640 1 -- 172.21.15.82:0/2005529424 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c00c940 con 0x7efc4c099c60 2024-04-25T08:41:56.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc52800640 1 -- 172.21.15.82:0/2005529424 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c158720 con 0x7efc4c099c60 2024-04-25T08:41:56.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc43400640 1 -- 172.21.15.82:0/2005529424 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c00ccf0 con 0x7efc4c099c60 2024-04-25T08:41:56.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc52800640 1 -- 172.21.15.82:0/2005529424 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c0afc20 con 0x7efc4c099c60 2024-04-25T08:41:56.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc43400640 1 -- 172.21.15.82:0/2005529424 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7efc3c01a070 con 0x7efc4c099c60 2024-04-25T08:41:56.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc43400640 1 -- 172.21.15.82:0/2005529424 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7efc3c013470 con 0x7efc4c099c60 2024-04-25T08:41:56.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.236+0000 7efc52800640 1 -- 172.21.15.82:0/2005529424 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c069010 con 0x7efc4c099c60 2024-04-25T08:41:56.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.240+0000 7efc43400640 1 -- 172.21.15.82:0/2005529424 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7efc3c006020 con 0x7efc4c099c60 2024-04-25T08:41:56.301 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:56.306+0000 7f905521d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:41:56.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.308+0000 7efc52800640 1 -- 172.21.15.82:0/2005529424 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6a7c621a-0fc4-40b3-aa8e-86048c60c3ad", "id": 1} v 0) v1 -- 0x7efc4c0cb950 con 0x7efc4c099c60 2024-04-25T08:41:56.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc43400640 1 -- 172.21.15.82:0/2005529424 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6a7c621a-0fc4-40b3-aa8e-86048c60c3ad", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7efc3c0059a0 con 0x7efc4c099c60 2024-04-25T08:41:56.307 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T08:41:56.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc52800640 1 -- 172.21.15.82:0/2005529424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c099c60 msgr2=0x7efc4c15df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc52800640 1 --2- 172.21.15.82:0/2005529424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c099c60 0x7efc4c15df00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00e870 tx=0x7efc3c00ec70 comp rx=0 tx=0).stop 2024-04-25T08:41:56.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc52800640 1 -- 172.21.15.82:0/2005529424 shutdown_connections 2024-04-25T08:41:56.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc52800640 1 --2- 172.21.15.82:0/2005529424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc4c0cf210 0x7efc4c155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc52800640 1 --2- 172.21.15.82:0/2005529424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c09a040 0x7efc4c155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc52800640 1 --2- 172.21.15.82:0/2005529424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efc4c099c60 0x7efc4c15df00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc52800640 1 -- 172.21.15.82:0/2005529424 >> 172.21.15.82:0/2005529424 conn(0x7efc4c0b3130 msgr2=0x7efc4c0b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc52800640 1 -- 172.21.15.82:0/2005529424 shutdown_connections 2024-04-25T08:41:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.312+0000 7efc52800640 1 -- 172.21.15.82:0/2005529424 wait complete. 2024-04-25T08:41:56.326 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new f4a9a449-1363-4a3c-8da8-271902210ab4 2 2024-04-25T08:41:56.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.508+0000 7f2a7d000640 1 Processor -- start 2024-04-25T08:41:56.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.508+0000 7f2a7d000640 1 -- start start 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.508+0000 7f2a7d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a780cd500 0x7f2a78070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.508+0000 7f2a7d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a7806f520 0x7f2a7815e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.508+0000 7f2a7d000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a7815ea10 0x7f2a7806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.508+0000 7f2a7d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7815f0a0 con 0x7f2a7806f520 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.508+0000 7f2a7d000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a780cb120 con 0x7f2a7815ea10 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.508+0000 7f2a7d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a780cb3c0 con 0x7f2a780cd500 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a76800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a7815ea10 0x7f2a7806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a75e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a780cd500 0x7f2a78070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a76800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a7815ea10 0x7f2a7806e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48784/0 (socket says 172.21.15.82:48784) 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a76800640 1 -- 172.21.15.82:0/2392016251 learned_addr learned my addr 172.21.15.82:0/2392016251 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:56.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a75e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a780cd500 0x7f2a78070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:40298/0 (socket says 172.21.15.82:40298) 2024-04-25T08:41:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a75400640 1 --2- 172.21.15.82:0/2392016251 >> v2:172.21.15.46:3300/0 conn(0x7f2a7806f520 0x7f2a7815e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a76800640 1 -- 172.21.15.82:0/2392016251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a780cd500 msgr2=0x7f2a78070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a76800640 1 --2- 172.21.15.82:0/2392016251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a780cd500 0x7f2a78070030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a76800640 1 -- 172.21.15.82:0/2392016251 >> v2:172.21.15.46:3300/0 conn(0x7f2a7806f520 msgr2=0x7f2a7815e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a76800640 1 --2- 172.21.15.82:0/2392016251 >> v2:172.21.15.46:3300/0 conn(0x7f2a7806f520 0x7f2a7815e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a76800640 1 -- 172.21.15.82:0/2392016251 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a780b2b20 con 0x7f2a7815ea10 2024-04-25T08:41:56.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a75e00640 1 --2- 172.21.15.82:0/2392016251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a780cd500 0x7f2a78070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:56.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a76800640 1 --2- 172.21.15.82:0/2392016251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a7815ea10 0x7f2a7806e2f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d150 tx=0x7f2a7001e930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d741ac02d40ff7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a74a00640 1 -- 172.21.15.82:0/2392016251 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7000b2d0 con 0x7f2a7815ea10 2024-04-25T08:41:56.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a74a00640 1 -- 172.21.15.82:0/2392016251 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7000b470 con 0x7f2a7815ea10 2024-04-25T08:41:56.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.512+0000 7f2a74a00640 1 -- 172.21.15.82:0/2392016251 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70009820 con 0x7f2a7815ea10 2024-04-25T08:41:56.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 -- 172.21.15.82:0/2392016251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a7815ea10 msgr2=0x7f2a7806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 --2- 172.21.15.82:0/2392016251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a7815ea10 0x7f2a7806e2f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d150 tx=0x7f2a7001e930 comp rx=0 tx=0).stop 2024-04-25T08:41:56.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 -- 172.21.15.82:0/2392016251 shutdown_connections 2024-04-25T08:41:56.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 --2- 172.21.15.82:0/2392016251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a7815ea10 0x7f2a7806e2f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 --2- 172.21.15.82:0/2392016251 >> v2:172.21.15.46:3300/0 conn(0x7f2a7806f520 0x7f2a7815e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 --2- 172.21.15.82:0/2392016251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a780cd500 0x7f2a78070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 -- 172.21.15.82:0/2392016251 >> 172.21.15.82:0/2392016251 conn(0x7f2a78099a00 msgr2=0x7f2a780b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 -- 172.21.15.82:0/2392016251 shutdown_connections 2024-04-25T08:41:56.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 -- 172.21.15.82:0/2392016251 wait complete. 2024-04-25T08:41:56.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 Processor -- start 2024-04-25T08:41:56.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 -- start start 2024-04-25T08:41:56.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7806f520 0x7f2a78158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a780cd500 0x7f2a781f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a7815ea10 0x7f2a780c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7806ad90 con 0x7f2a7806f520 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7806ac10 con 0x7f2a780cd500 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a7d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7806af10 con 0x7f2a7815ea10 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a75400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a780cd500 0x7f2a781f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a76800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a7815ea10 0x7f2a780c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a75400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a780cd500 0x7f2a781f99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48792/0 (socket says 172.21.15.82:48792) 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a75400640 1 -- 172.21.15.82:0/3267181251 learned_addr learned my addr 172.21.15.82:0/3267181251 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a75e00640 1 --2- 172.21.15.82:0/3267181251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7806f520 0x7f2a78158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a76800640 1 -- 172.21.15.82:0/3267181251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a780cd500 msgr2=0x7f2a781f99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a76800640 1 --2- 172.21.15.82:0/3267181251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a780cd500 0x7f2a781f99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a76800640 1 -- 172.21.15.82:0/3267181251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7806f520 msgr2=0x7f2a78158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a76800640 1 --2- 172.21.15.82:0/3267181251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7806f520 0x7f2a78158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.516+0000 7f2a76800640 1 -- 172.21.15.82:0/3267181251 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a78157880 con 0x7f2a7815ea10 2024-04-25T08:41:56.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.520+0000 7f2a76800640 1 --2- 172.21.15.82:0/3267181251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a7815ea10 0x7f2a780c4a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a700ad000 tx=0x7f2a7001b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.520+0000 7f2a6e000640 1 -- 172.21.15.82:0/3267181251 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a700bd020 con 0x7f2a7815ea10 2024-04-25T08:41:56.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.520+0000 7f2a6e000640 1 -- 172.21.15.82:0/3267181251 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a700a75c0 con 0x7f2a7815ea10 2024-04-25T08:41:56.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.520+0000 7f2a7d000640 1 -- 172.21.15.82:0/3267181251 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a78157b60 con 0x7f2a7815ea10 2024-04-25T08:41:56.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.520+0000 7f2a6e000640 1 -- 172.21.15.82:0/3267181251 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7000b720 con 0x7f2a7815ea10 2024-04-25T08:41:56.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.520+0000 7f2a7d000640 1 -- 172.21.15.82:0/3267181251 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a781f9ef0 con 0x7f2a7815ea10 2024-04-25T08:41:56.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.520+0000 7f2a6e000640 1 -- 172.21.15.82:0/3267181251 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2a700c0070 con 0x7f2a7815ea10 2024-04-25T08:41:56.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.520+0000 7f2a6e000640 1 -- 172.21.15.82:0/3267181251 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f2a700af810 con 0x7f2a7815ea10 2024-04-25T08:41:56.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.520+0000 7f2a7d000640 1 -- 172.21.15.82:0/3267181251 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a78069010 con 0x7f2a7815ea10 2024-04-25T08:41:56.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.524+0000 7f2a6e000640 1 -- 172.21.15.82:0/3267181251 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2a700187e0 con 0x7f2a7815ea10 2024-04-25T08:41:56.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.592+0000 7f2a7d000640 1 -- 172.21.15.82:0/3267181251 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f4a9a449-1363-4a3c-8da8-271902210ab4", "id": 2} v 0) v1 -- 0x7f2a780cdc40 con 0x7f2a7815ea10 2024-04-25T08:41:56.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a6e000640 1 -- 172.21.15.82:0/3267181251 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f4a9a449-1363-4a3c-8da8-271902210ab4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2a700093d0 con 0x7f2a7815ea10 2024-04-25T08:41:56.590 INFO:teuthology.orchestra.run.smithi082.stdout:2 2024-04-25T08:41:56.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a7d000640 1 -- 172.21.15.82:0/3267181251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a7815ea10 msgr2=0x7f2a780c4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a7d000640 1 --2- 172.21.15.82:0/3267181251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a7815ea10 0x7f2a780c4a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2a700ad000 tx=0x7f2a7001b6d0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a7d000640 1 -- 172.21.15.82:0/3267181251 shutdown_connections 2024-04-25T08:41:56.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a7d000640 1 --2- 172.21.15.82:0/3267181251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a7815ea10 0x7f2a780c4a90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a7d000640 1 --2- 172.21.15.82:0/3267181251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a780cd500 0x7f2a781f99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a7d000640 1 --2- 172.21.15.82:0/3267181251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a7806f520 0x7f2a78158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a7d000640 1 -- 172.21.15.82:0/3267181251 >> 172.21.15.82:0/3267181251 conn(0x7f2a78099a00 msgr2=0x7f2a780cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a7d000640 1 -- 172.21.15.82:0/3267181251 shutdown_connections 2024-04-25T08:41:56.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.596+0000 7f2a7d000640 1 -- 172.21.15.82:0/3267181251 wait complete. 2024-04-25T08:41:56.609 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new 377bd68c-61e8-4a06-b508-18e213d1f7e7 3 2024-04-25T08:41:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2743e00640 1 Processor -- start 2024-04-25T08:41:56.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2743e00640 1 -- start start 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2743e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f273c0cf890 0x7f273c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2743e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cc180 0x7f273c0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2743e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f273c0cf340 0x7f273c0ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2743e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c0b2d50 con 0x7f273c0cf340 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2743e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c0d01b0 con 0x7f273c0cf890 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2743e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c15f2f0 con 0x7f273c0cc180 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2741400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f273c0cf890 0x7f273c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2741400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f273c0cf890 0x7f273c0b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48798/0 (socket says 172.21.15.82:48798) 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2740a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cc180 0x7f273c0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2741400640 1 -- 172.21.15.82:0/1751936691 learned_addr learned my addr 172.21.15.82:0/1751936691 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2741e00640 1 --2- 172.21.15.82:0/1751936691 >> v2:172.21.15.46:3300/0 conn(0x7f273c0cf340 0x7f273c0ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2741400640 1 -- 172.21.15.82:0/1751936691 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cc180 msgr2=0x7f273c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2741400640 1 --2- 172.21.15.82:0/1751936691 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cc180 0x7f273c0cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2741400640 1 -- 172.21.15.82:0/1751936691 >> v2:172.21.15.46:3300/0 conn(0x7f273c0cf340 msgr2=0x7f273c0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2741400640 1 --2- 172.21.15.82:0/1751936691 >> v2:172.21.15.46:3300/0 conn(0x7f273c0cf340 0x7f273c0ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.752+0000 7f2741400640 1 -- 172.21.15.82:0/1751936691 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c15f5d0 con 0x7f273c0cf890 2024-04-25T08:41:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2741400640 1 --2- 172.21.15.82:0/1751936691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cf890 0x7f273c0b2810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f273c06b8d0 tx=0x7f273001c260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdfe0289e6bc29f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f273be00640 1 -- 172.21.15.82:0/1751936691 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27300b0070 con 0x7f273c0cf890 2024-04-25T08:41:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f273be00640 1 -- 172.21.15.82:0/1751936691 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27300ab600 con 0x7f273c0cf890 2024-04-25T08:41:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f273be00640 1 -- 172.21.15.82:0/1751936691 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27300ab9d0 con 0x7f273c0cf890 2024-04-25T08:41:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 -- 172.21.15.82:0/1751936691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cf890 msgr2=0x7f273c0b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 --2- 172.21.15.82:0/1751936691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cf890 0x7f273c0b2810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f273c06b8d0 tx=0x7f273001c260 comp rx=0 tx=0).stop 2024-04-25T08:41:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 -- 172.21.15.82:0/1751936691 shutdown_connections 2024-04-25T08:41:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 --2- 172.21.15.82:0/1751936691 >> v2:172.21.15.46:3300/0 conn(0x7f273c0cf340 0x7f273c0ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 --2- 172.21.15.82:0/1751936691 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cc180 0x7f273c0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 --2- 172.21.15.82:0/1751936691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cf890 0x7f273c0b2810 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 -- 172.21.15.82:0/1751936691 >> 172.21.15.82:0/1751936691 conn(0x7f273c099a00 msgr2=0x7f273c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 -- 172.21.15.82:0/1751936691 shutdown_connections 2024-04-25T08:41:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 -- 172.21.15.82:0/1751936691 wait complete. 2024-04-25T08:41:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.756+0000 7f2743e00640 1 Processor -- start 2024-04-25T08:41:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 -- start start 2024-04-25T08:41:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cc180 0x7f273c15ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f273c0cf340 0x7f273c0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cf890 0x7f273c0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c15bcf0 con 0x7f273c0cf340 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c15bb70 con 0x7f273c0cc180 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c15be70 con 0x7f273c0cf890 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cc180 0x7f273c15ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cf890 0x7f273c0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cc180 0x7f273c15ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48806/0 (socket says 172.21.15.82:48806) 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cf890 0x7f273c0b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:40324/0 (socket says 172.21.15.82:40324) 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741400640 1 -- 172.21.15.82:0/3840739563 learned_addr learned my addr 172.21.15.82:0/3840739563 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2740a00640 1 --2- 172.21.15.82:0/3840739563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f273c0cf340 0x7f273c0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741400640 1 -- 172.21.15.82:0/3840739563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cf890 msgr2=0x7f273c0b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741400640 1 --2- 172.21.15.82:0/3840739563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cf890 0x7f273c0b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741400640 1 -- 172.21.15.82:0/3840739563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f273c0cf340 msgr2=0x7f273c0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741400640 1 --2- 172.21.15.82:0/3840739563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f273c0cf340 0x7f273c0c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741400640 1 -- 172.21.15.82:0/3840739563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c15ca00 con 0x7f273c0cc180 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741e00640 1 --2- 172.21.15.82:0/3840739563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cf890 0x7f273c0b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2741400640 1 --2- 172.21.15.82:0/3840739563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cc180 0x7f273c15ea50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f273001a910 tx=0x7f2730019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2739600640 1 -- 172.21.15.82:0/3840739563 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27300abe60 con 0x7f273c0cc180 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2739600640 1 -- 172.21.15.82:0/3840739563 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27300b4620 con 0x7f273c0cc180 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2739600640 1 -- 172.21.15.82:0/3840739563 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27300b0040 con 0x7f273c0cc180 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 -- 172.21.15.82:0/3840739563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c0aee10 con 0x7f273c0cc180 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 -- 172.21.15.82:0/3840739563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c0af3d0 con 0x7f273c0cc180 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2739600640 1 -- 172.21.15.82:0/3840739563 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f27300ab4c0 con 0x7f273c0cc180 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2739600640 1 -- 172.21.15.82:0/3840739563 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f27300bd400 con 0x7f273c0cc180 2024-04-25T08:41:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.760+0000 7f2743e00640 1 -- 172.21.15.82:0/3840739563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c069010 con 0x7f273c0cc180 2024-04-25T08:41:56.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.764+0000 7f2739600640 1 -- 172.21.15.82:0/3840739563 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2730016110 con 0x7f273c0cc180 2024-04-25T08:41:56.782 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:56.788+0000 7f2f75455180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:41:56.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.840+0000 7f2743e00640 1 -- 172.21.15.82:0/3840739563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd new", "uuid": "377bd68c-61e8-4a06-b508-18e213d1f7e7", "id": 3} v 0) v1 -- 0x7f273c0b2fd0 con 0x7f273c0cc180 2024-04-25T08:41:56.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.844+0000 7f2739600640 1 -- 172.21.15.82:0/3840739563 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "377bd68c-61e8-4a06-b508-18e213d1f7e7", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f27300b4a10 con 0x7f273c0cc180 2024-04-25T08:41:56.840 INFO:teuthology.orchestra.run.smithi082.stdout:3 2024-04-25T08:41:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.848+0000 7f2743e00640 1 -- 172.21.15.82:0/3840739563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cc180 msgr2=0x7f273c15ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.848+0000 7f2743e00640 1 --2- 172.21.15.82:0/3840739563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cc180 0x7f273c15ea50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f273001a910 tx=0x7f2730019000 comp rx=0 tx=0).stop 2024-04-25T08:41:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.848+0000 7f2743e00640 1 -- 172.21.15.82:0/3840739563 shutdown_connections 2024-04-25T08:41:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.848+0000 7f2743e00640 1 --2- 172.21.15.82:0/3840739563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f273c0cf890 0x7f273c0b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.848+0000 7f2743e00640 1 --2- 172.21.15.82:0/3840739563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f273c0cf340 0x7f273c0c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.848+0000 7f2743e00640 1 --2- 172.21.15.82:0/3840739563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f273c0cc180 0x7f273c15ea50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.848+0000 7f2743e00640 1 -- 172.21.15.82:0/3840739563 >> 172.21.15.82:0/3840739563 conn(0x7f273c099a00 msgr2=0x7f273c0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.848+0000 7f2743e00640 1 -- 172.21.15.82:0/3840739563 shutdown_connections 2024-04-25T08:41:56.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:56.848+0000 7f2743e00640 1 -- 172.21.15.82:0/3840739563 wait complete. 2024-04-25T08:41:56.848 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:56.852+0000 7f2f75455180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:41:56.866 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new 709e6dbb-3635-4856-888e-15eb84401717 4 2024-04-25T08:41:56.918 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:56.924+0000 7f2f75455180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:41:56.950 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:56.954+0000 7f905521d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:41:57.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d48400640 1 Processor -- start 2024-04-25T08:41:57.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d48400640 1 -- start start 2024-04-25T08:41:57.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d48400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d400acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d48400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d400b2c80 0x7f7d400b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d48400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d4015df30 con 0x7f7d4006bc20 2024-04-25T08:41:57.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d48400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4015eb00 con 0x7f7d400b2c80 2024-04-25T08:41:57.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d48400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4015e1d0 con 0x7f7d40072590 2024-04-25T08:41:57.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d45000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d400acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d45000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d400acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:40336/0 (socket says 172.21.15.82:40336) 2024-04-25T08:41:57.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d45000640 1 -- 172.21.15.82:0/1012703031 learned_addr learned my addr 172.21.15.82:0/1012703031 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:57.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d46400640 1 --2- 172.21.15.82:0/1012703031 >> v2:172.21.15.46:3300/0 conn(0x7f7d400b2c80 0x7f7d400b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d45000640 1 -- 172.21.15.82:0/1012703031 >> v2:172.21.15.46:3300/0 conn(0x7f7d400b2c80 msgr2=0x7f7d400b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:57.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d45000640 1 --2- 172.21.15.82:0/1012703031 >> v2:172.21.15.46:3300/0 conn(0x7f7d400b2c80 0x7f7d400b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d45000640 1 -- 172.21.15.82:0/1012703031 >> v1:172.21.15.82:6789/0 conn(0x7f7d4006bc20 legacy=0x7f7d4006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d3fe00640 1 -- 172.21.15.82:0/1012703031 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466006365 0 0) 0x7f7d4015df30 con 0x7f7d4006bc20 2024-04-25T08:41:57.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d45000640 1 -- 172.21.15.82:0/1012703031 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4015e4b0 con 0x7f7d40072590 2024-04-25T08:41:57.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.008+0000 7f7d46400640 1 --2- 172.21.15.82:0/1012703031 >> v2:172.21.15.46:3300/0 conn(0x7f7d400b2c80 0x7f7d400b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:57.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d45000640 1 --2- 172.21.15.82:0/1012703031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d400acd40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006fd0 tx=0x7f7d2c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=350bec336f515b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d3fe00640 1 -- 172.21.15.82:0/1012703031 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0b1070 con 0x7f7d40072590 2024-04-25T08:41:57.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d3fe00640 1 -- 172.21.15.82:0/1012703031 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c0ac620 con 0x7f7d40072590 2024-04-25T08:41:57.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d3fe00640 1 -- 172.21.15.82:0/1012703031 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0ac9f0 con 0x7f7d40072590 2024-04-25T08:41:57.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 -- 172.21.15.82:0/1012703031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 msgr2=0x7f7d400acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 --2- 172.21.15.82:0/1012703031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d400acd40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006fd0 tx=0x7f7d2c01c4b0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 -- 172.21.15.82:0/1012703031 shutdown_connections 2024-04-25T08:41:57.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 --2- 172.21.15.82:0/1012703031 >> v2:172.21.15.46:3300/0 conn(0x7f7d400b2c80 0x7f7d400b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 --2- 172.21.15.82:0/1012703031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d400acd40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 -- 172.21.15.82:0/1012703031 >> 172.21.15.82:0/1012703031 conn(0x7f7d40071a70 msgr2=0x7f7d40071e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 -- 172.21.15.82:0/1012703031 shutdown_connections 2024-04-25T08:41:57.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 -- 172.21.15.82:0/1012703031 wait complete. 2024-04-25T08:41:57.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 Processor -- start 2024-04-25T08:41:57.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.012+0000 7f7d48400640 1 -- start start 2024-04-25T08:41:57.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d48400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d4006bc20 0x7f7d40167550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d48400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d4015f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d48400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d400b2c80 0x7f7d400cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d48400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4016a670 con 0x7f7d400b2c80 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d48400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40158410 con 0x7f7d4006bc20 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d48400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4016a7f0 con 0x7f7d40072590 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d4015f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d4006bc20 0x7f7d40167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d4015f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:40344/0 (socket says 172.21.15.82:40344) 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45000640 1 -- 172.21.15.82:0/3731047693 learned_addr learned my addr 172.21.15.82:0/3731047693 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d46400640 1 --2- 172.21.15.82:0/3731047693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d400b2c80 0x7f7d400cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45000640 1 -- 172.21.15.82:0/3731047693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d4006bc20 msgr2=0x7f7d40167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45000640 1 --2- 172.21.15.82:0/3731047693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d4006bc20 0x7f7d40167550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45000640 1 -- 172.21.15.82:0/3731047693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d400b2c80 msgr2=0x7f7d400cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45000640 1 --2- 172.21.15.82:0/3731047693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d400b2c80 0x7f7d400cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45000640 1 -- 172.21.15.82:0/3731047693 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4015a190 con 0x7f7d40072590 2024-04-25T08:41:57.012 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f2f75455180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:41:57.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d45000640 1 --2- 172.21.15.82:0/3731047693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d4015f2a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c01ca20 tx=0x7f7d2c01acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d3d600640 1 -- 172.21.15.82:0/3731047693 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0b1040 con 0x7f7d40072590 2024-04-25T08:41:57.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d3d600640 1 -- 172.21.15.82:0/3731047693 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c005ba0 con 0x7f7d40072590 2024-04-25T08:41:57.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d48400640 1 -- 172.21.15.82:0/3731047693 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4015a430 con 0x7f7d40072590 2024-04-25T08:41:57.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d3d600640 1 -- 172.21.15.82:0/3731047693 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c0ac7d0 con 0x7f7d40072590 2024-04-25T08:41:57.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d48400640 1 -- 172.21.15.82:0/3731047693 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d40160900 con 0x7f7d40072590 2024-04-25T08:41:57.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d3d600640 1 -- 172.21.15.82:0/3731047693 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7d2c0bc060 con 0x7f7d40072590 2024-04-25T08:41:57.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d3d600640 1 -- 172.21.15.82:0/3731047693 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f7d2c006020 con 0x7f7d40072590 2024-04-25T08:41:57.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.016+0000 7f7d48400640 1 -- 172.21.15.82:0/3731047693 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d40069010 con 0x7f7d40072590 2024-04-25T08:41:57.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.020+0000 7f7d3d600640 1 -- 172.21.15.82:0/3731047693 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7d2c016130 con 0x7f7d40072590 2024-04-25T08:41:57.015 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:57.018+0000 7f905521d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:41:57.072 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:57.076+0000 7f2f75455180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:41:57.080 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:57.086+0000 7f905521d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:41:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.084+0000 7f7d48400640 1 -- 172.21.15.82:0/3731047693 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "709e6dbb-3635-4856-888e-15eb84401717", "id": 4} v 0) v1 -- 0x7f7d400cfab0 con 0x7f7d40072590 2024-04-25T08:41:57.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d3d600640 1 -- 172.21.15.82:0/3731047693 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "709e6dbb-3635-4856-888e-15eb84401717", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7d2c003620 con 0x7f7d40072590 2024-04-25T08:41:57.085 INFO:teuthology.orchestra.run.smithi082.stdout:4 2024-04-25T08:41:57.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d48400640 1 -- 172.21.15.82:0/3731047693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 msgr2=0x7f7d4015f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d48400640 1 --2- 172.21.15.82:0/3731047693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d4015f2a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c01ca20 tx=0x7f7d2c01acf0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d48400640 1 -- 172.21.15.82:0/3731047693 shutdown_connections 2024-04-25T08:41:57.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d48400640 1 --2- 172.21.15.82:0/3731047693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d400b2c80 0x7f7d400cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d48400640 1 --2- 172.21.15.82:0/3731047693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7d40072590 0x7f7d4015f2a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d48400640 1 --2- 172.21.15.82:0/3731047693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7d4006bc20 0x7f7d40167550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d48400640 1 -- 172.21.15.82:0/3731047693 >> 172.21.15.82:0/3731047693 conn(0x7f7d40071a70 msgr2=0x7f7d40071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d48400640 1 -- 172.21.15.82:0/3731047693 shutdown_connections 2024-04-25T08:41:57.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.092+0000 7f7d48400640 1 -- 172.21.15.82:0/3731047693 wait complete. 2024-04-25T08:41:57.105 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new 5954690d-8a26-42e5-9d4d-eddd58943333 5 2024-04-25T08:41:57.167 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:57.172+0000 7f2f75455180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:41:57.174 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:57.178+0000 7f905521d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:41:57.228 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:57.232+0000 7f2f75455180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:41:57.238 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:57.242+0000 7f905521d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:41:57.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.244+0000 7f8e62c00640 1 Processor -- start 2024-04-25T08:41:57.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.244+0000 7f8e62c00640 1 -- start start 2024-04-25T08:41:57.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e62c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8e5c145290 0x7f8e5c0cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e62c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8e5c15c6f0 0x7f8e5c0b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e62c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5c0ce060 con 0x7f8e5c0b3250 2024-04-25T08:41:57.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e62c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b3bf0 con 0x7f8e5c145290 2024-04-25T08:41:57.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e62c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c0cde10 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e60c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8e5c15c6f0 0x7f8e5c0b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e60c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8e5c15c6f0 0x7f8e5c0b36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48822/0 (socket says 172.21.15.82:48822) 2024-04-25T08:41:57.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e60c00640 1 -- 172.21.15.82:0/1280695480 learned_addr learned my addr 172.21.15.82:0/1280695480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:57.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e5be00640 1 --2- 172.21.15.82:0/1280695480 >> v2:172.21.15.46:3300/0 conn(0x7f8e5c145290 0x7f8e5c0cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e60c00640 1 -- 172.21.15.82:0/1280695480 >> v2:172.21.15.46:3300/0 conn(0x7f8e5c145290 msgr2=0x7f8e5c0cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:57.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e60c00640 1 --2- 172.21.15.82:0/1280695480 >> v2:172.21.15.46:3300/0 conn(0x7f8e5c145290 0x7f8e5c0cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e60c00640 1 -- 172.21.15.82:0/1280695480 >> v1:172.21.15.46:6789/0 conn(0x7f8e5c0b3250 legacy=0x7f8e5c15bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.248+0000 7f8e60c00640 1 -- 172.21.15.82:0/1280695480 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0ce340 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e60c00640 1 --2- 172.21.15.82:0/1280695480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 0x7f8e5c0b36b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0cfc50 tx=0x7f8e5401c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38527018f731ef4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e5b400640 1 -- 172.21.15.82:0/1280695480 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e540a56c0 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e5b400640 1 -- 172.21.15.82:0/1280695480 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e540a5860 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e5b400640 1 -- 172.21.15.82:0/1280695480 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e540a5c30 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 -- 172.21.15.82:0/1280695480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 msgr2=0x7f8e5c0b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 --2- 172.21.15.82:0/1280695480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 0x7f8e5c0b36b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0cfc50 tx=0x7f8e5401c9f0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 -- 172.21.15.82:0/1280695480 shutdown_connections 2024-04-25T08:41:57.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 --2- 172.21.15.82:0/1280695480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 0x7f8e5c0b36b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 --2- 172.21.15.82:0/1280695480 >> v2:172.21.15.46:3300/0 conn(0x7f8e5c145290 0x7f8e5c0cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 -- 172.21.15.82:0/1280695480 >> 172.21.15.82:0/1280695480 conn(0x7f8e5c066b70 msgr2=0x7f8e5c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 -- 172.21.15.82:0/1280695480 shutdown_connections 2024-04-25T08:41:57.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 -- 172.21.15.82:0/1280695480 wait complete. 2024-04-25T08:41:57.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 Processor -- start 2024-04-25T08:41:57.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 -- start start 2024-04-25T08:41:57.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e5c0b3250 0x7f8e5c141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c145290 0x7f8e5c0d1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 0x7f8e5c0d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b5930 con 0x7f8e5c145290 2024-04-25T08:41:57.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b57b0 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e62c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b5ab0 con 0x7f8e5c0b3250 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e60c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 0x7f8e5c0d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e5be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e5c0b3250 0x7f8e5c141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.252+0000 7f8e60c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 0x7f8e5c0d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48824/0 (socket says 172.21.15.82:48824) 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e60c00640 1 -- 172.21.15.82:0/2240936577 learned_addr learned my addr 172.21.15.82:0/2240936577 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e53e00640 1 --2- 172.21.15.82:0/2240936577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c145290 0x7f8e5c0d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e60c00640 1 -- 172.21.15.82:0/2240936577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e5c0b3250 msgr2=0x7f8e5c141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e60c00640 1 --2- 172.21.15.82:0/2240936577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e5c0b3250 0x7f8e5c141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e60c00640 1 -- 172.21.15.82:0/2240936577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c145290 msgr2=0x7f8e5c0d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e60c00640 1 --2- 172.21.15.82:0/2240936577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c145290 0x7f8e5c0d1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e60c00640 1 -- 172.21.15.82:0/2240936577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c141670 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e60c00640 1 --2- 172.21.15.82:0/2240936577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 0x7f8e5c0d1b50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8e540af040 tx=0x7f8e54019790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e58c00640 1 -- 172.21.15.82:0/2240936577 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54009510 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e58c00640 1 -- 172.21.15.82:0/2240936577 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e540be070 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e58c00640 1 -- 172.21.15.82:0/2240936577 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e540b8890 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e62c00640 1 -- 172.21.15.82:0/2240936577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c0caae0 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e62c00640 1 -- 172.21.15.82:0/2240936577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c0cb0a0 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e58c00640 1 -- 172.21.15.82:0/2240936577 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8e540b8a30 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e58c00640 1 -- 172.21.15.82:0/2240936577 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f8e540b6440 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.256+0000 7f8e62c00640 1 -- 172.21.15.82:0/2240936577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c069010 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.260+0000 7f8e58c00640 1 -- 172.21.15.82:0/2240936577 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8e540165e0 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.324+0000 7f8e62c00640 1 -- 172.21.15.82:0/2240936577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5954690d-8a26-42e5-9d4d-eddd58943333", "id": 5} v 0) v1 -- 0x7f8e5c0cdbf0 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e58c00640 1 -- 172.21.15.82:0/2240936577 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5954690d-8a26-42e5-9d4d-eddd58943333", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8e540c3460 con 0x7f8e5c15c6f0 2024-04-25T08:41:57.323 INFO:teuthology.orchestra.run.smithi082.stdout:5 2024-04-25T08:41:57.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e62c00640 1 -- 172.21.15.82:0/2240936577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 msgr2=0x7f8e5c0d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e62c00640 1 --2- 172.21.15.82:0/2240936577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 0x7f8e5c0d1b50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8e540af040 tx=0x7f8e54019790 comp rx=0 tx=0).stop 2024-04-25T08:41:57.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e62c00640 1 -- 172.21.15.82:0/2240936577 shutdown_connections 2024-04-25T08:41:57.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e62c00640 1 --2- 172.21.15.82:0/2240936577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e5c15c6f0 0x7f8e5c0d1b50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e62c00640 1 --2- 172.21.15.82:0/2240936577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e5c145290 0x7f8e5c0d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e62c00640 1 --2- 172.21.15.82:0/2240936577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e5c0b3250 0x7f8e5c141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e62c00640 1 -- 172.21.15.82:0/2240936577 >> 172.21.15.82:0/2240936577 conn(0x7f8e5c066b70 msgr2=0x7f8e5c0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e62c00640 1 -- 172.21.15.82:0/2240936577 shutdown_connections 2024-04-25T08:41:57.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.328+0000 7f8e62c00640 1 -- 172.21.15.82:0/2240936577 wait complete. 2024-04-25T08:41:57.333 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:57.338+0000 7f905521d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:41:57.342 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new 77529356-a4a8-4d24-8332-b370296b2b73 6 2024-04-25T08:41:57.400 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:57.402+0000 7f905521d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:41:57.404 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:57.408+0000 7f2f75455180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:41:57.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5dc400640 1 Processor -- start 2024-04-25T08:41:57.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5dc400640 1 -- start start 2024-04-25T08:41:57.478 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:57.484+0000 7f2f75455180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:41:57.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5dc400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5d406b320 0x7fa5d41686c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5dc400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40b5730 con 0x7fa5d40b5270 2024-04-25T08:41:57.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5dc400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40b5a10 con 0x7fa5d4144070 2024-04-25T08:41:57.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5dc400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d41452b0 con 0x7fa5d406b320 2024-04-25T08:41:57.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5d9a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa5d4144070 0x7fa5d40b35e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48784/0 (socket says 172.21.15.82:48784) 2024-04-25T08:41:57.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5d9a00640 1 -- 172.21.15.82:0/4251850361 learned_addr learned my addr 172.21.15.82:0/4251850361 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T08:41:57.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5da400640 1 --2- 172.21.15.82:0/4251850361 >> v2:172.21.15.46:3300/0 conn(0x7fa5d406b320 0x7fa5d41686c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5d3e00640 1 -- 172.21.15.82:0/4251850361 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422252833 0 0) 0x7fa5d40b5a10 con 0x7fa5d4144070 2024-04-25T08:41:57.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5d3e00640 1 -- 172.21.15.82:0/4251850361 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5bc003530 con 0x7fa5d4144070 2024-04-25T08:41:57.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5da400640 1 -- 172.21.15.82:0/4251850361 >> v1:172.21.15.82:6789/0 conn(0x7fa5d4144070 legacy=0x7fa5d40b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5da400640 1 -- 172.21.15.82:0/4251850361 >> v1:172.21.15.46:6789/0 conn(0x7fa5d40b5270 legacy=0x7fa5d406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.484+0000 7fa5da400640 1 -- 172.21.15.82:0/4251850361 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4145590 con 0x7fa5d406b320 2024-04-25T08:41:57.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5da400640 1 --2- 172.21.15.82:0/4251850361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d406b320 0x7fa5d41686c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009dc0 tx=0x7fa5cc01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b5aba7b35d5d1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5d3e00640 1 -- 172.21.15.82:0/4251850361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0b2070 con 0x7fa5d406b320 2024-04-25T08:41:57.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5d3e00640 1 -- 172.21.15.82:0/4251850361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc0a5880 con 0x7fa5d406b320 2024-04-25T08:41:57.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5d3e00640 1 -- 172.21.15.82:0/4251850361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0a5c50 con 0x7fa5d406b320 2024-04-25T08:41:57.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5dc400640 1 -- 172.21.15.82:0/4251850361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d406b320 msgr2=0x7fa5d41686c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5dc400640 1 --2- 172.21.15.82:0/4251850361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d406b320 0x7fa5d41686c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009dc0 tx=0x7fa5cc01c890 comp rx=0 tx=0).stop 2024-04-25T08:41:57.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5dc400640 1 -- 172.21.15.82:0/4251850361 shutdown_connections 2024-04-25T08:41:57.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5dc400640 1 --2- 172.21.15.82:0/4251850361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d406b320 0x7fa5d41686c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5dc400640 1 -- 172.21.15.82:0/4251850361 >> 172.21.15.82:0/4251850361 conn(0x7fa5d40728e0 msgr2=0x7fa5d4099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5dc400640 1 -- 172.21.15.82:0/4251850361 shutdown_connections 2024-04-25T08:41:57.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.488+0000 7fa5dc400640 1 -- 172.21.15.82:0/4251850361 wait complete. 2024-04-25T08:41:57.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 Processor -- start 2024-04-25T08:41:57.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 -- start start 2024-04-25T08:41:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5d406b320 0x7fa5d415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d40b5270 0x7fa5d4160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5d4144070 0x7fa5d4140260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d416b5a0 con 0x7fa5d40b5270 2024-04-25T08:41:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d416b420 con 0x7fa5d406b320 2024-04-25T08:41:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d416b720 con 0x7fa5d4144070 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5d406b320 0x7fa5d415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5da400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5d4144070 0x7fa5d4140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5d406b320 0x7fa5d415d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39066/0 (socket says 172.21.15.82:39066) 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d9a00640 1 -- 172.21.15.82:0/1264514524 learned_addr learned my addr 172.21.15.82:0/1264514524 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d9000640 1 --2- 172.21.15.82:0/1264514524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d40b5270 0x7fa5d4160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5da400640 1 -- 172.21.15.82:0/1264514524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5d406b320 msgr2=0x7fa5d415d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5da400640 1 --2- 172.21.15.82:0/1264514524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5d406b320 0x7fa5d415d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5da400640 1 -- 172.21.15.82:0/1264514524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d40b5270 msgr2=0x7fa5d4160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5da400640 1 --2- 172.21.15.82:0/1264514524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d40b5270 0x7fa5d4160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5da400640 1 -- 172.21.15.82:0/1264514524 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d415f370 con 0x7fa5d4144070 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d9000640 1 --2- 172.21.15.82:0/1264514524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d40b5270 0x7fa5d4160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5da400640 1 --2- 172.21.15.82:0/1264514524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5d4144070 0x7fa5d4140260 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0ac000 tx=0x7fa5cc019630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d1600640 1 -- 172.21.15.82:0/1264514524 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0b2050 con 0x7fa5d4144070 2024-04-25T08:41:57.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d1600640 1 -- 172.21.15.82:0/1264514524 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc0b3040 con 0x7fa5d4144070 2024-04-25T08:41:57.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 -- 172.21.15.82:0/1264514524 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d415f650 con 0x7fa5d4144070 2024-04-25T08:41:57.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d1600640 1 -- 172.21.15.82:0/1264514524 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0ad650 con 0x7fa5d4144070 2024-04-25T08:41:57.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 -- 172.21.15.82:0/1264514524 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d415fc10 con 0x7fa5d4144070 2024-04-25T08:41:57.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d1600640 1 -- 172.21.15.82:0/1264514524 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa5cc0c0020 con 0x7fa5d4144070 2024-04-25T08:41:57.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5d1600640 1 -- 172.21.15.82:0/1264514524 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fa5cc0adde0 con 0x7fa5d4144070 2024-04-25T08:41:57.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.492+0000 7fa5dc400640 1 -- 172.21.15.82:0/1264514524 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d4069010 con 0x7fa5d4144070 2024-04-25T08:41:57.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.496+0000 7fa5d1600640 1 -- 172.21.15.82:0/1264514524 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa5cc016540 con 0x7fa5d4144070 2024-04-25T08:41:57.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.564+0000 7fa5dc400640 1 -- 172.21.15.82:0/1264514524 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd new", "uuid": "77529356-a4a8-4d24-8332-b370296b2b73", "id": 6} v 0) v1 -- 0x7fa5d406aef0 con 0x7fa5d4144070 2024-04-25T08:41:57.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5d1600640 1 -- 172.21.15.82:0/1264514524 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "77529356-a4a8-4d24-8332-b370296b2b73", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa5cc0ad7f0 con 0x7fa5d4144070 2024-04-25T08:41:57.562 INFO:teuthology.orchestra.run.smithi082.stdout:6 2024-04-25T08:41:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5dc400640 1 -- 172.21.15.82:0/1264514524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5d4144070 msgr2=0x7fa5d4140260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5dc400640 1 --2- 172.21.15.82:0/1264514524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5d4144070 0x7fa5d4140260 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0ac000 tx=0x7fa5cc019630 comp rx=0 tx=0).stop 2024-04-25T08:41:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5dc400640 1 -- 172.21.15.82:0/1264514524 shutdown_connections 2024-04-25T08:41:57.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5dc400640 1 --2- 172.21.15.82:0/1264514524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5d4144070 0x7fa5d4140260 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5dc400640 1 --2- 172.21.15.82:0/1264514524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5d40b5270 0x7fa5d4160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5dc400640 1 --2- 172.21.15.82:0/1264514524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5d406b320 0x7fa5d415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5dc400640 1 -- 172.21.15.82:0/1264514524 >> 172.21.15.82:0/1264514524 conn(0x7fa5d40728e0 msgr2=0x7fa5d40b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5dc400640 1 -- 172.21.15.82:0/1264514524 shutdown_connections 2024-04-25T08:41:57.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.568+0000 7fa5dc400640 1 -- 172.21.15.82:0/1264514524 wait complete. 2024-04-25T08:41:57.574 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:57.578+0000 7f905521d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:41:57.581 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd new a4e72542-3f54-4a06-a6a9-044ec460fa4d 7 2024-04-25T08:41:57.649 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:57.654+0000 7f905521d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:41:57.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.720+0000 7f6e6a600640 1 Processor -- start 2024-04-25T08:41:57.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.720+0000 7f6e6a600640 1 -- start start 2024-04-25T08:41:57.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e6a600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6e64145310 0x7f6e6406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e6a600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e640cd930 con 0x7f6e6406dc70 2024-04-25T08:41:57.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e6a600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e640cdbd0 con 0x7f6e6415d090 2024-04-25T08:41:57.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e6a600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e640cdeb0 con 0x7f6e64145310 2024-04-25T08:41:57.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e63400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6e64145310 0x7f6e6406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e63e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6e6415d090 0x7f6e640cc0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48798/0 (socket says 172.21.15.82:48798) 2024-04-25T08:41:57.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e63e00640 1 -- 172.21.15.82:0/1438771832 learned_addr learned my addr 172.21.15.82:0/1438771832 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T08:41:57.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e63400640 1 -- 172.21.15.82:0/1438771832 >> v1:172.21.15.82:6789/0 conn(0x7f6e6415d090 legacy=0x7f6e640cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e63400640 1 -- 172.21.15.82:0/1438771832 >> v1:172.21.15.46:6789/0 conn(0x7f6e6406dc70 legacy=0x7f6e640cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e62a00640 1 -- 172.21.15.82:0/1438771832 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555256057 0 0) 0x7f6e640cdbd0 con 0x7f6e6415d090 2024-04-25T08:41:57.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e63400640 1 -- 172.21.15.82:0/1438771832 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640ce190 con 0x7f6e64145310 2024-04-25T08:41:57.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.724+0000 7f6e63400640 1 --2- 172.21.15.82:0/1438771832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e64145310 0x7f6e6406e0d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e540097c0 tx=0x7f6e5401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f5737b1d0b9b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e62a00640 1 -- 172.21.15.82:0/1438771832 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e540a7070 con 0x7f6e64145310 2024-04-25T08:41:57.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e62a00640 1 -- 172.21.15.82:0/1438771832 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e540b0070 con 0x7f6e64145310 2024-04-25T08:41:57.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e62a00640 1 -- 172.21.15.82:0/1438771832 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e540ab890 con 0x7f6e64145310 2024-04-25T08:41:57.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 -- 172.21.15.82:0/1438771832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e64145310 msgr2=0x7f6e6406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 --2- 172.21.15.82:0/1438771832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e64145310 0x7f6e6406e0d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e540097c0 tx=0x7f6e5401c470 comp rx=0 tx=0).stop 2024-04-25T08:41:57.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 -- 172.21.15.82:0/1438771832 shutdown_connections 2024-04-25T08:41:57.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 --2- 172.21.15.82:0/1438771832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e64145310 0x7f6e6406e0d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 -- 172.21.15.82:0/1438771832 >> 172.21.15.82:0/1438771832 conn(0x7f6e6406ac10 msgr2=0x7f6e6406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 -- 172.21.15.82:0/1438771832 shutdown_connections 2024-04-25T08:41:57.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 -- 172.21.15.82:0/1438771832 wait complete. 2024-04-25T08:41:57.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 Processor -- start 2024-04-25T08:41:57.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 -- start start 2024-04-25T08:41:57.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e64145310 0x7f6e64159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e6415d090 0x7f6e640b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e640b1940 0x7f6e6415be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6415d8b0 con 0x7f6e640b1940 2024-04-25T08:41:57.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640cf380 con 0x7f6e6415d090 2024-04-25T08:41:57.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e6a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64169d30 con 0x7f6e64145310 2024-04-25T08:41:57.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e5be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e6415d090 0x7f6e640b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e63400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e64145310 0x7f6e64159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e63400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e64145310 0x7f6e64159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.82:45098/0 (socket says 172.21.15.82:45098) 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e5be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e6415d090 0x7f6e640b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39076/0 (socket says 172.21.15.82:39076) 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e63400640 1 -- 172.21.15.82:0/2897853917 learned_addr learned my addr 172.21.15.82:0/2897853917 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.728+0000 7f6e63e00640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e640b1940 0x7f6e6415be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5be00640 1 -- 172.21.15.82:0/2897853917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e64145310 msgr2=0x7f6e64159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5be00640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e64145310 0x7f6e64159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5be00640 1 -- 172.21.15.82:0/2897853917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e640b1940 msgr2=0x7f6e6415be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5be00640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e640b1940 0x7f6e6415be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5be00640 1 -- 172.21.15.82:0/2897853917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640b1ec0 con 0x7f6e6415d090 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e63e00640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e640b1940 0x7f6e6415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e63400640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e64145310 0x7f6e64159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5be00640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e6415d090 0x7f6e640b1400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c001ec0 tx=0x7f6e4c00c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5b400640 1 -- 172.21.15.82:0/2897853917 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c017070 con 0x7f6e6415d090 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5b400640 1 -- 172.21.15.82:0/2897853917 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c0052d0 con 0x7f6e6415d090 2024-04-25T08:41:57.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5b400640 1 -- 172.21.15.82:0/2897853917 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c003840 con 0x7f6e6415d090 2024-04-25T08:41:57.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e6a600640 1 -- 172.21.15.82:0/2897853917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e640b21a0 con 0x7f6e6415d090 2024-04-25T08:41:57.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e6a600640 1 -- 172.21.15.82:0/2897853917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e640b2760 con 0x7f6e6415d090 2024-04-25T08:41:57.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5b400640 1 -- 172.21.15.82:0/2897853917 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6e4c01a070 con 0x7f6e6415d090 2024-04-25T08:41:57.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5b400640 1 -- 172.21.15.82:0/2897853917 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f6e4c011590 con 0x7f6e6415d090 2024-04-25T08:41:57.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e6a600640 1 -- 172.21.15.82:0/2897853917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e64069010 con 0x7f6e6415d090 2024-04-25T08:41:57.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.732+0000 7f6e5b400640 1 -- 172.21.15.82:0/2897853917 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6e4c021080 con 0x7f6e6415d090 2024-04-25T08:41:57.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.800+0000 7f6e6a600640 1 -- 172.21.15.82:0/2897853917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a4e72542-3f54-4a06-a6a9-044ec460fa4d", "id": 7} v 0) v1 -- 0x7f6e6415e370 con 0x7f6e6415d090 2024-04-25T08:41:57.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e5b400640 1 -- 172.21.15.82:0/2897853917 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4e72542-3f54-4a06-a6a9-044ec460fa4d", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6e4c005480 con 0x7f6e6415d090 2024-04-25T08:41:57.799 INFO:teuthology.orchestra.run.smithi082.stdout:7 2024-04-25T08:41:57.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e6a600640 1 -- 172.21.15.82:0/2897853917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e6415d090 msgr2=0x7f6e640b1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e6a600640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e6415d090 0x7f6e640b1400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c001ec0 tx=0x7f6e4c00c890 comp rx=0 tx=0).stop 2024-04-25T08:41:57.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e6a600640 1 -- 172.21.15.82:0/2897853917 shutdown_connections 2024-04-25T08:41:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e6a600640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e640b1940 0x7f6e6415be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e6a600640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e6415d090 0x7f6e640b1400 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e6a600640 1 --2- 172.21.15.82:0/2897853917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e64145310 0x7f6e64159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e6a600640 1 -- 172.21.15.82:0/2897853917 >> 172.21.15.82:0/2897853917 conn(0x7f6e6406ac10 msgr2=0x7f6e640af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e6a600640 1 -- 172.21.15.82:0/2897853917 shutdown_connections 2024-04-25T08:41:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T08:41:57.804+0000 7f6e6a600640 1 -- 172.21.15.82:0/2897853917 wait complete. 2024-04-25T08:41:57.834 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T08:41:57.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T08:41:57.838 INFO:tasks.ceph.osd.0:Started 2024-04-25T08:41:57.838 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T08:41:57.838 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T08:41:57.841 INFO:tasks.ceph.osd.1:Started 2024-04-25T08:41:57.841 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T08:41:57.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T08:41:57.844 INFO:tasks.ceph.osd.2:Started 2024-04-25T08:41:57.844 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T08:41:57.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T08:41:57.847 INFO:tasks.ceph.osd.3:Started 2024-04-25T08:41:57.848 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T08:41:57.848 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T08:41:57.852 INFO:tasks.ceph.osd.4:Started 2024-04-25T08:41:57.852 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T08:41:57.852 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T08:41:57.865 INFO:tasks.ceph.osd.5:Started 2024-04-25T08:41:57.865 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T08:41:57.865 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T08:41:57.869 INFO:tasks.ceph.osd.6:Started 2024-04-25T08:41:57.869 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T08:41:57.869 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T08:41:57.881 INFO:tasks.ceph.osd.7:Started 2024-04-25T08:41:57.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:41:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e7200640 1 Processor -- start 2024-04-25T08:41:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e7200640 1 -- start start 2024-04-25T08:41:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e015dfc0 0x7ff4e006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e7200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff4e006b150 0x7ff4e0146a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e7200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e00b4de0 con 0x7ff4e0144030 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e7200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e00b1e80 con 0x7ff4e006b150 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00b5080 con 0x7ff4e015dfc0 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e5e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff4e0144030 0x7ff4e00b1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51030/0 (socket says 172.21.15.46:51030) 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e5e00640 1 -- 172.21.15.46:0/3855614107 learned_addr learned my addr 172.21.15.46:0/3855614107 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e5400640 1 --2- 172.21.15.46:0/3855614107 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e015dfc0 0x7ff4e006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e6800640 1 --2- 172.21.15.46:0/3855614107 >> v2:172.21.15.82:3300/0 conn(0x7ff4e006b150 0x7ff4e0146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e5400640 1 -- 172.21.15.46:0/3855614107 >> v2:172.21.15.82:3300/0 conn(0x7ff4e006b150 msgr2=0x7ff4e0146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e5400640 1 --2- 172.21.15.46:0/3855614107 >> v2:172.21.15.82:3300/0 conn(0x7ff4e006b150 0x7ff4e0146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e5400640 1 -- 172.21.15.46:0/3855614107 >> v1:172.21.15.46:6789/0 conn(0x7ff4e0144030 legacy=0x7ff4e00b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e4a00640 1 -- 172.21.15.46:0/3855614107 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328333757 0 0) 0x7ff4e00b4de0 con 0x7ff4e0144030 2024-04-25T08:41:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.090+0000 7ff4e5400640 1 -- 172.21.15.46:0/3855614107 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e00b5360 con 0x7ff4e015dfc0 2024-04-25T08:41:58.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e5400640 1 --2- 172.21.15.46:0/3855614107 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e015dfc0 0x7ff4e006ac10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00097c0 tx=0x7ff4d001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca67b56993d2cd1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:58.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e4a00640 1 -- 172.21.15.46:0/3855614107 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00b0070 con 0x7ff4e015dfc0 2024-04-25T08:41:58.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e4a00640 1 -- 172.21.15.46:0/3855614107 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d00ab600 con 0x7ff4e015dfc0 2024-04-25T08:41:58.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e4a00640 1 -- 172.21.15.46:0/3855614107 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00ab9d0 con 0x7ff4e015dfc0 2024-04-25T08:41:58.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e7200640 1 -- 172.21.15.46:0/3855614107 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e015dfc0 msgr2=0x7ff4e006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e7200640 1 --2- 172.21.15.46:0/3855614107 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e015dfc0 0x7ff4e006ac10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00097c0 tx=0x7ff4d001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e7200640 1 -- 172.21.15.46:0/3855614107 shutdown_connections 2024-04-25T08:41:58.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e7200640 1 --2- 172.21.15.46:0/3855614107 >> v2:172.21.15.82:3300/0 conn(0x7ff4e006b150 0x7ff4e0146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e7200640 1 --2- 172.21.15.46:0/3855614107 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e015dfc0 0x7ff4e006ac10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e7200640 1 -- 172.21.15.46:0/3855614107 >> 172.21.15.46:0/3855614107 conn(0x7ff4e0099a00 msgr2=0x7ff4e00b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:58.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e7200640 1 -- 172.21.15.46:0/3855614107 shutdown_connections 2024-04-25T08:41:58.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e7200640 1 -- 172.21.15.46:0/3855614107 wait complete. 2024-04-25T08:41:58.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.094+0000 7ff4e7200640 1 Processor -- start 2024-04-25T08:41:58.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 -- start start 2024-04-25T08:41:58.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4e006b150 0x7ff4e014e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e0144030 0x7ff4e009a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e015dfc0 0x7ff4e00b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00b61e0 con 0x7ff4e015dfc0 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e015e420 con 0x7ff4e006b150 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e016a580 con 0x7ff4e0144030 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e015dfc0 0x7ff4e00b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e015dfc0 0x7ff4e00b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54928/0 (socket says 172.21.15.46:54928) 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e6800640 1 -- 172.21.15.46:0/2722562113 learned_addr learned my addr 172.21.15.46:0/2722562113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e5e00640 1 --2- 172.21.15.46:0/2722562113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4e006b150 0x7ff4e014e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e5400640 1 --2- 172.21.15.46:0/2722562113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e0144030 0x7ff4e009a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e6800640 1 -- 172.21.15.46:0/2722562113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e0144030 msgr2=0x7ff4e009a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e6800640 1 --2- 172.21.15.46:0/2722562113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e0144030 0x7ff4e009a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e6800640 1 -- 172.21.15.46:0/2722562113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4e006b150 msgr2=0x7ff4e014e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e6800640 1 --2- 172.21.15.46:0/2722562113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4e006b150 0x7ff4e014e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e6800640 1 -- 172.21.15.46:0/2722562113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e009ae50 con 0x7ff4e015dfc0 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e6800640 1 --2- 172.21.15.46:0/2722562113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e015dfc0 0x7ff4e00b08e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00efc0 tx=0x7ff4dc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4da000640 1 -- 172.21.15.46:0/2722562113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc018020 con 0x7ff4e015dfc0 2024-04-25T08:41:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4da000640 1 -- 172.21.15.46:0/2722562113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc009b10 con 0x7ff4e015dfc0 2024-04-25T08:41:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4da000640 1 -- 172.21.15.46:0/2722562113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc0146c0 con 0x7ff4e015dfc0 2024-04-25T08:41:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 -- 172.21.15.46:0/2722562113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e009b130 con 0x7ff4e015dfc0 2024-04-25T08:41:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 -- 172.21.15.46:0/2722562113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e009b6f0 con 0x7ff4e015dfc0 2024-04-25T08:41:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4da000640 1 -- 172.21.15.46:0/2722562113 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff4dc01b020 con 0x7ff4e015dfc0 2024-04-25T08:41:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4da000640 1 -- 172.21.15.46:0/2722562113 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ff4dc014bf0 con 0x7ff4e015dfc0 2024-04-25T08:41:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.098+0000 7ff4e7200640 1 -- 172.21.15.46:0/2722562113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e0069010 con 0x7ff4e015dfc0 2024-04-25T08:41:58.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.102+0000 7ff4da000640 1 -- 172.21.15.46:0/2722562113 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff4dc022070 con 0x7ff4e015dfc0 2024-04-25T08:41:58.160 INFO:tasks.ceph.mgr.x.smithi082.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T08:41:58.160 INFO:tasks.ceph.mgr.x.smithi082.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T08:41:58.161 INFO:tasks.ceph.mgr.x.smithi082.stderr: from numpy import show_config as show_numpy_config 2024-04-25T08:41:58.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 -- 172.21.15.46:0/2722562113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4e0143840 con 0x7ff4e015dfc0 2024-04-25T08:41:58.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4da000640 1 -- 172.21.15.46:0/2722562113 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7ff4dc010040 con 0x7ff4e015dfc0 2024-04-25T08:41:58.166 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:41:58.166 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:41:57.803830+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:41:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 -- 172.21.15.46:0/2722562113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e015dfc0 msgr2=0x7ff4e00b08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 --2- 172.21.15.46:0/2722562113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e015dfc0 0x7ff4e00b08e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00efc0 tx=0x7ff4dc00c490 comp rx=0 tx=0).stop 2024-04-25T08:41:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 -- 172.21.15.46:0/2722562113 shutdown_connections 2024-04-25T08:41:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 --2- 172.21.15.46:0/2722562113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e015dfc0 0x7ff4e00b08e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 --2- 172.21.15.46:0/2722562113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4e0144030 0x7ff4e009a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 --2- 172.21.15.46:0/2722562113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4e006b150 0x7ff4e014e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 -- 172.21.15.46:0/2722562113 >> 172.21.15.46:0/2722562113 conn(0x7ff4e0099a00 msgr2=0x7ff4e01458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 -- 172.21.15.46:0/2722562113 shutdown_connections 2024-04-25T08:41:58.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:41:58.170+0000 7ff4e7200640 1 -- 172.21.15.46:0/2722562113 wait complete. 2024-04-25T08:41:58.191 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T08:41:58.191 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T08:41:58.227 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:58.232+0000 7f2f75455180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:41:58.320 INFO:tasks.ceph.mgr.y.smithi046.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T08:41:58.320 INFO:tasks.ceph.mgr.y.smithi046.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T08:41:58.320 INFO:tasks.ceph.mgr.y.smithi046.stderr: from numpy import show_config as show_numpy_config 2024-04-25T08:41:58.373 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:58.376+0000 7f2f75455180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:41:58.387 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:58.390+0000 7f905521d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:41:58.510 INFO:tasks.ceph.osd.4.smithi082.stderr:2024-04-25T08:41:58.516+0000 7f3da6f887c0 -1 Falling back to public interface 2024-04-25T08:41:58.513 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-04-25T08:41:58.518+0000 7f65aef867c0 -1 Falling back to public interface 2024-04-25T08:41:58.521 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-04-25T08:41:58.526+0000 7fd7202c17c0 -1 Falling back to public interface 2024-04-25T08:41:58.521 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-04-25T08:41:58.526+0000 7f291cc9e7c0 -1 Falling back to public interface 2024-04-25T08:41:58.534 INFO:tasks.ceph.osd.7.smithi082.stderr:2024-04-25T08:41:58.540+0000 7feebb86b7c0 -1 Falling back to public interface 2024-04-25T08:41:58.540 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-04-25T08:41:58.546+0000 7f8ffd19b7c0 -1 Falling back to public interface 2024-04-25T08:41:58.544 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:58.550+0000 7f905521d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:41:58.548 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:58.552+0000 7f2f75455180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:41:58.558 INFO:tasks.ceph.osd.5.smithi082.stderr:2024-04-25T08:41:58.564+0000 7fd19c2d47c0 -1 Falling back to public interface 2024-04-25T08:41:58.566 INFO:tasks.ceph.osd.6.smithi082.stderr:2024-04-25T08:41:58.572+0000 7fe272f7c7c0 -1 Falling back to public interface 2024-04-25T08:41:58.626 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:58.632+0000 7f2f75455180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:41:58.739 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:58.742+0000 7f905521d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:41:58.773 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:58.780+0000 7f2f75455180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:41:58.816 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:58.822+0000 7f905521d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:41:58.974 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:58.978+0000 7f905521d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:41:58.996 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:41:59.129 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf5be00640 1 Processor -- start 2024-04-25T08:41:59.129 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf5be00640 1 -- start start 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf5be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf5415a1c0 0x7faf54071b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf5be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf540cce70 0x7faf540720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf5be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5406fcf0 con 0x7faf5406e540 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf5be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf540b2660 con 0x7faf5415a1c0 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf5be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf540b2940 con 0x7faf540cce70 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf59400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf5415a1c0 0x7faf54071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf59400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf5415a1c0 0x7faf54071b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54932/0 (socket says 172.21.15.46:54932) 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf59400640 1 -- 172.21.15.46:0/1173716925 learned_addr learned my addr 172.21.15.46:0/1173716925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf58a00640 1 --2- 172.21.15.46:0/1173716925 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf540cce70 0x7faf540720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf59400640 1 -- 172.21.15.46:0/1173716925 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf540cce70 msgr2=0x7faf540720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf59400640 1 --2- 172.21.15.46:0/1173716925 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf540cce70 0x7faf540720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf59400640 1 -- 172.21.15.46:0/1173716925 >> v1:172.21.15.46:6789/0 conn(0x7faf5406e540 legacy=0x7faf5406e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.131 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.134+0000 7faf59400640 1 -- 172.21.15.46:0/1173716925 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf540b2c20 con 0x7faf5415a1c0 2024-04-25T08:41:59.134 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf59400640 1 --2- 172.21.15.46:0/1173716925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5415a1c0 0x7faf54071b70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf48009870 tx=0x7faf4801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75fe548140feeb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:59.134 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf53e00640 1 -- 172.21.15.46:0/1173716925 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480b0070 con 0x7faf5415a1c0 2024-04-25T08:41:59.134 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf53e00640 1 -- 172.21.15.46:0/1173716925 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf480ab600 con 0x7faf5415a1c0 2024-04-25T08:41:59.134 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf53e00640 1 -- 172.21.15.46:0/1173716925 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480ab9d0 con 0x7faf5415a1c0 2024-04-25T08:41:59.135 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf5be00640 1 -- 172.21.15.46:0/1173716925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5415a1c0 msgr2=0x7faf54071b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.135 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf5be00640 1 --2- 172.21.15.46:0/1173716925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5415a1c0 0x7faf54071b70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf48009870 tx=0x7faf4801c4d0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.135 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf5be00640 1 -- 172.21.15.46:0/1173716925 shutdown_connections 2024-04-25T08:41:59.135 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf5be00640 1 --2- 172.21.15.46:0/1173716925 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf540cce70 0x7faf540720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.135 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf5be00640 1 --2- 172.21.15.46:0/1173716925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5415a1c0 0x7faf54071b70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.135 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf5be00640 1 -- 172.21.15.46:0/1173716925 >> 172.21.15.46:0/1173716925 conn(0x7faf540b12c0 msgr2=0x7faf540b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:59.135 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf5be00640 1 -- 172.21.15.46:0/1173716925 shutdown_connections 2024-04-25T08:41:59.135 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.138+0000 7faf5be00640 1 -- 172.21.15.46:0/1173716925 wait complete. 2024-04-25T08:41:59.136 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 Processor -- start 2024-04-25T08:41:59.136 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 -- start start 2024-04-25T08:41:59.139 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5406e540 0x7faf54149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.139 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf540cce70 0x7faf541472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.139 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf5415a1c0 0x7faf5414cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.139 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5415e090 con 0x7faf5406e540 2024-04-25T08:41:59.139 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5415df10 con 0x7faf540cce70 2024-04-25T08:41:59.139 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5415e210 con 0x7faf5415a1c0 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5406e540 0x7faf54149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5406e540 0x7faf54149080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54934/0 (socket says 172.21.15.46:54934) 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59400640 1 -- 172.21.15.46:0/3274447276 learned_addr learned my addr 172.21.15.46:0/3274447276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf58a00640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf540cce70 0x7faf541472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59e00640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf5415a1c0 0x7faf5414cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59400640 1 -- 172.21.15.46:0/3274447276 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf5415a1c0 msgr2=0x7faf5414cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59400640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf5415a1c0 0x7faf5414cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59400640 1 -- 172.21.15.46:0/3274447276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf540cce70 msgr2=0x7faf541472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59400640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf540cce70 0x7faf541472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59400640 1 -- 172.21.15.46:0/3274447276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5413f4d0 con 0x7faf5406e540 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59e00640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf5415a1c0 0x7faf5414cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf59400640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5406e540 0x7faf54149080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf480ae040 tx=0x7faf4801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf51600640 1 -- 172.21.15.46:0/3274447276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480ba020 con 0x7faf5406e540 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf51600640 1 -- 172.21.15.46:0/3274447276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf480b6420 con 0x7faf5406e540 2024-04-25T08:41:59.140 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 -- 172.21.15.46:0/3274447276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf54146510 con 0x7faf5406e540 2024-04-25T08:41:59.141 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf51600640 1 -- 172.21.15.46:0/3274447276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480b0040 con 0x7faf5406e540 2024-04-25T08:41:59.141 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf5be00640 1 -- 172.21.15.46:0/3274447276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf54146ad0 con 0x7faf5406e540 2024-04-25T08:41:59.141 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf51600640 1 -- 172.21.15.46:0/3274447276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7faf480bd020 con 0x7faf5406e540 2024-04-25T08:41:59.141 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.142+0000 7faf51600640 1 -- 172.21.15.46:0/3274447276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7faf480b43f0 con 0x7faf5406e540 2024-04-25T08:41:59.141 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.146+0000 7faf5be00640 1 -- 172.21.15.46:0/3274447276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5404ef30 con 0x7faf5406e540 2024-04-25T08:41:59.142 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.146+0000 7faf51600640 1 -- 172.21.15.46:0/3274447276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7faf480c4030 con 0x7faf5406e540 2024-04-25T08:41:59.201 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 -- 172.21.15.46:0/3274447276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf540b1e40 con 0x7faf5406e540 2024-04-25T08:41:59.201 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf51600640 1 -- 172.21.15.46:0/3274447276 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7faf48016150 con 0x7faf5406e540 2024-04-25T08:41:59.202 INFO:teuthology.misc.health.smithi046.stdout: 2024-04-25T08:41:59.202 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":10,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:41:57.803830+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:41:59.203 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 -- 172.21.15.46:0/3274447276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5406e540 msgr2=0x7faf54149080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.203 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5406e540 0x7faf54149080 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf480ae040 tx=0x7faf4801ad10 comp rx=0 tx=0).stop 2024-04-25T08:41:59.203 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 -- 172.21.15.46:0/3274447276 shutdown_connections 2024-04-25T08:41:59.203 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faf5415a1c0 0x7faf5414cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.203 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf540cce70 0x7faf541472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.203 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 --2- 172.21.15.46:0/3274447276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf5406e540 0x7faf54149080 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.203 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 -- 172.21.15.46:0/3274447276 >> 172.21.15.46:0/3274447276 conn(0x7faf540b12c0 msgr2=0x7faf540b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:59.203 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 -- 172.21.15.46:0/3274447276 shutdown_connections 2024-04-25T08:41:59.203 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:41:59.206+0000 7faf5be00640 1 -- 172.21.15.46:0/3274447276 wait complete. 2024-04-25T08:41:59.219 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:41:59.459 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:59.464+0000 7f2f75455180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:41:59.546 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:59.552+0000 7f2f75455180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:41:59.686 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:59.690+0000 7f905521d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:41:59.746 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:59.752+0000 7f2f75455180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:41:59.778 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:59.782+0000 7f905521d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:41:59.830 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:41:59.836+0000 7f2f75455180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:41:59.989 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:41:59.994+0000 7f905521d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:42:00.080 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:42:00.086+0000 7f905521d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:42:00.157 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:42:00.164+0000 7f2f75455180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:42:00.227 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:42:00.232+0000 7f2f75455180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:42:00.431 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:42:00.434+0000 7f905521d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:42:00.505 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-04-25T08:42:00.510+0000 7f905521d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:42:01.248 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:42:01.252+0000 7f2f1b600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T08:42:01.249 INFO:tasks.ceph.mgr.x.smithi082.stderr:2024-04-25T08:42:01.252+0000 7f2f1b600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T08:42:02.310 INFO:tasks.ceph.osd.4.smithi082.stderr:2024-04-25T08:42:02.316+0000 7f3da6f887c0 -1 osd.4 0 log_to_monitors true 2024-04-25T08:42:02.335 INFO:tasks.ceph.osd.7.smithi082.stderr:2024-04-25T08:42:02.340+0000 7feebb86b7c0 -1 osd.7 0 log_to_monitors true 2024-04-25T08:42:02.343 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-04-25T08:42:02.347+0000 7f65aef867c0 -1 osd.0 0 log_to_monitors true 2024-04-25T08:42:02.415 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-04-25T08:42:02.419+0000 7f291cc9e7c0 -1 osd.2 0 log_to_monitors true 2024-04-25T08:42:02.464 INFO:tasks.ceph.osd.5.smithi082.stderr:2024-04-25T08:42:02.468+0000 7fd19c2d47c0 -1 osd.5 0 log_to_monitors true 2024-04-25T08:42:02.480 INFO:tasks.ceph.osd.6.smithi082.stderr:2024-04-25T08:42:02.484+0000 7fe272f7c7c0 -1 osd.6 0 log_to_monitors true 2024-04-25T08:42:02.495 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-04-25T08:42:02.499+0000 7fd7202c17c0 -1 osd.1 0 log_to_monitors true 2024-04-25T08:42:02.511 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-04-25T08:42:02.515+0000 7f8ffd19b7c0 -1 osd.3 0 log_to_monitors true 2024-04-25T08:42:06.026 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:06.043 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-04-25T08:42:06.047+0000 7fd71bc00640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T08:42:06.059 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-04-25T08:42:06.063+0000 7f65a9e00640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T08:42:06.085 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-04-25T08:42:06.091+0000 7fd711200640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:42:06.104 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-04-25T08:42:06.111+0000 7f8ff8000640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T08:42:06.105 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-04-25T08:42:06.111+0000 7f659fe00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:42:06.133 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-04-25T08:42:06.139+0000 7f8fee000640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:42:06.141 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-04-25T08:42:06.147+0000 7f2917c00640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T08:42:06.177 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82fe200640 1 Processor -- start 2024-04-25T08:42:06.177 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82fe200640 1 -- start start 2024-04-25T08:42:06.178 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82fe200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f815efd0 0x7f82f80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.178 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82fe200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f82f80d1360 0x7f82f80c5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.178 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82fe200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f815f3b0 con 0x7f82f80d06f0 2024-04-25T08:42:06.178 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82fe200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f80c8700 con 0x7f82f80d1360 2024-04-25T08:42:06.178 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82fe200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80ccd70 con 0x7f82f815efd0 2024-04-25T08:42:06.179 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82eea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f82f80d1360 0x7f82f80c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.179 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82f7400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f815efd0 0x7f82f80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.179 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82f7e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f82f80d06f0 0x7f82f80d0ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:56320/0 (socket says 172.21.15.46:56320) 2024-04-25T08:42:06.179 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82f7e00640 1 -- 172.21.15.46:0/330858562 learned_addr learned my addr 172.21.15.46:0/330858562 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:06.179 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82eea00640 1 -- 172.21.15.46:0/330858562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f815efd0 msgr2=0x7f82f80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:06.179 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82eea00640 1 --2- 172.21.15.46:0/330858562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f815efd0 0x7f82f80c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.179 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82eea00640 1 -- 172.21.15.46:0/330858562 >> v1:172.21.15.82:6789/0 conn(0x7f82f80d06f0 legacy=0x7f82f80d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.179 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82f6a00640 1 -- 172.21.15.46:0/330858562 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703195459 0 0) 0x7f82f815f3b0 con 0x7f82f80d06f0 2024-04-25T08:42:06.180 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.183+0000 7f82eea00640 1 -- 172.21.15.46:0/330858562 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80cc7b0 con 0x7f82f80d1360 2024-04-25T08:42:06.180 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f290dc00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:42:06.182 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82eea00640 1 --2- 172.21.15.46:0/330858562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82f80d1360 0x7f82f80c5870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82e00097f0 tx=0x7f82e001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ddbae818bcf650c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:06.182 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82f6a00640 1 -- 172.21.15.46:0/330858562 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e00b0070 con 0x7f82f80d1360 2024-04-25T08:42:06.182 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82f6a00640 1 -- 172.21.15.46:0/330858562 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e00ab600 con 0x7f82f80d1360 2024-04-25T08:42:06.182 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82f6a00640 1 -- 172.21.15.46:0/330858562 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e00ab9d0 con 0x7f82f80d1360 2024-04-25T08:42:06.183 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82fe200640 1 -- 172.21.15.46:0/330858562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82f80d1360 msgr2=0x7f82f80c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.183 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82fe200640 1 --2- 172.21.15.46:0/330858562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82f80d1360 0x7f82f80c5870 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82e00097f0 tx=0x7f82e001c410 comp rx=0 tx=0).stop 2024-04-25T08:42:06.183 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82fe200640 1 -- 172.21.15.46:0/330858562 shutdown_connections 2024-04-25T08:42:06.183 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82fe200640 1 --2- 172.21.15.46:0/330858562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82f80d1360 0x7f82f80c5870 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.183 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82fe200640 1 --2- 172.21.15.46:0/330858562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f815efd0 0x7f82f80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.184 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82fe200640 1 -- 172.21.15.46:0/330858562 >> 172.21.15.46:0/330858562 conn(0x7f82f806f530 msgr2=0x7f82f806f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:06.184 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82fe200640 1 -- 172.21.15.46:0/330858562 shutdown_connections 2024-04-25T08:42:06.184 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.187+0000 7f82fe200640 1 -- 172.21.15.46:0/330858562 wait complete. 2024-04-25T08:42:06.185 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82fe200640 1 Processor -- start 2024-04-25T08:42:06.185 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82fe200640 1 -- start start 2024-04-25T08:42:06.187 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82fe200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82f80d06f0 0x7f82f80b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.187 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82fe200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f80d1360 0x7f82f806c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.187 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82fe200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f815efd0 0x7f82f80b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.187 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82fe200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f8160230 con 0x7f82f815efd0 2024-04-25T08:42:06.187 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82fe200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81600b0 con 0x7f82f80d06f0 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82fe200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f81603b0 con 0x7f82f80d1360 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f815efd0 0x7f82f80b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f815efd0 0x7f82f80b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54968/0 (socket says 172.21.15.46:54968) 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7e00640 1 -- 172.21.15.46:0/1257115579 learned_addr learned my addr 172.21.15.46:0/1257115579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82eea00640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f80d1360 0x7f82f806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7400640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82f80d06f0 0x7f82f80b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7e00640 1 -- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f80d1360 msgr2=0x7f82f806c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7e00640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f80d1360 0x7f82f806c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7e00640 1 -- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82f80d06f0 msgr2=0x7f82f80b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7e00640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82f80d06f0 0x7f82f80b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7e00640 1 -- 172.21.15.46:0/1257115579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80cef90 con 0x7f82f815efd0 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82f7e00640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f815efd0 0x7f82f80b6980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c700 tx=0x7f82f000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82efe00640 1 -- 172.21.15.46:0/1257115579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0018070 con 0x7f82f815efd0 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82efe00640 1 -- 172.21.15.46:0/1257115579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f000ce10 con 0x7f82f815efd0 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f806dca0 con 0x7f82f815efd0 2024-04-25T08:42:06.188 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.191+0000 7f82efe00640 1 -- 172.21.15.46:0/1257115579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000a860 con 0x7f82f815efd0 2024-04-25T08:42:06.189 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.195+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f80c9d50 con 0x7f82f815efd0 2024-04-25T08:42:06.189 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.195+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f81f6c10 con 0x7f82f815efd0 2024-04-25T08:42:06.189 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.195+0000 7f82efe00640 1 -- 172.21.15.46:0/1257115579 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7f82f001b070 con 0x7f82f815efd0 2024-04-25T08:42:06.190 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.195+0000 7f82efe00640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82d4066ed0 0x7f82d4069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.190 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.195+0000 7f82efe00640 1 -- 172.21.15.46:0/1257115579 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f82f0083630 con 0x7f82f815efd0 2024-04-25T08:42:06.191 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.195+0000 7f82f7400640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82d4066ed0 0x7f82d4069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.192 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.199+0000 7f82f7400640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82d4066ed0 0x7f82d4069390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009a50 tx=0x7f82e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:06.195 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.203+0000 7f82efe00640 1 -- 172.21.15.46:0/1257115579 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82f0054e50 con 0x7f82f815efd0 2024-04-25T08:42:06.362 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.367+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82f804ef30 con 0x7f82f815efd0 2024-04-25T08:42:06.362 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.367+0000 7f82efe00640 1 -- 172.21.15.46:0/1257115579 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f82f0058cb0 con 0x7f82f815efd0 2024-04-25T08:42:06.362 INFO:teuthology.misc.health.smithi046.stdout: 2024-04-25T08:42:06.363 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":12,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:04.255703+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82d4066ed0 msgr2=0x7f82d4069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82d4066ed0 0x7f82d4069390 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009a50 tx=0x7f82e8008040 comp rx=0 tx=0).stop 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f815efd0 msgr2=0x7f82f80b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f815efd0 0x7f82f80b6980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f82f000c700 tx=0x7f82f000cbc0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 shutdown_connections 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82d4066ed0 0x7f82d4069390 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f815efd0 0x7f82f80b6980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82f80d1360 0x7f82f806c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 --2- 172.21.15.46:0/1257115579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82f80d06f0 0x7f82f80b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 >> 172.21.15.46:0/1257115579 conn(0x7f82f806f530 msgr2=0x7f82f80c7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 shutdown_connections 2024-04-25T08:42:06.365 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:06.371+0000 7f82fe200640 1 -- 172.21.15.46:0/1257115579 wait complete. 2024-04-25T08:42:06.382 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:42:06.577 INFO:tasks.ceph.osd.6.smithi082.stderr:2024-04-25T08:42:06.584+0000 7fe26de00640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T08:42:06.604 INFO:tasks.ceph.osd.6.smithi082.stderr:2024-04-25T08:42:06.608+0000 7fe263e00640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:42:06.623 INFO:tasks.ceph.osd.5.smithi082.stderr:2024-04-25T08:42:06.628+0000 7fd197c00640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T08:42:06.654 INFO:tasks.ceph.osd.5.smithi082.stderr:2024-04-25T08:42:06.660+0000 7fd18d200640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:42:06.655 INFO:tasks.ceph.osd.4.smithi082.stderr:2024-04-25T08:42:06.660+0000 7f3da1e00640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T08:42:06.682 INFO:tasks.ceph.osd.4.smithi082.stderr:2024-04-25T08:42:06.688+0000 7f3d97e00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:42:06.691 INFO:tasks.ceph.osd.7.smithi082.stderr:2024-04-25T08:42:06.696+0000 7feeb6800640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T08:42:06.726 INFO:tasks.ceph.osd.7.smithi082.stderr:2024-04-25T08:42:06.732+0000 7feeac800640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:42:13.189 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:13.331 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.335+0000 7f185ba00640 1 Processor -- start 2024-04-25T08:42:13.331 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.335+0000 7f185ba00640 1 -- start start 2024-04-25T08:42:13.332 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f185ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f185409a1a0 0x7f18540cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.332 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f185ba00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f18540d12f0 0x7f18540d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.332 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f185ba00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540cf880 con 0x7f18540b1de0 2024-04-25T08:42:13.332 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f185ba00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185406cfd0 con 0x7f18540d12f0 2024-04-25T08:42:13.332 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f185ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185409ab40 con 0x7f185409a1a0 2024-04-25T08:42:13.333 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1853e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f185409a1a0 0x7f18540cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.333 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1859a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f18540d12f0 0x7f18540d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.334 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1859a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f18540d12f0 0x7f18540d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:35786/0 (socket says 172.21.15.46:35786) 2024-04-25T08:42:13.334 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1859000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f18540b1de0 0x7f1854099a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:58734/0 (socket says 172.21.15.46:58734) 2024-04-25T08:42:13.334 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1859a00640 1 -- 172.21.15.46:0/4268313176 learned_addr learned my addr 172.21.15.46:0/4268313176 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:13.334 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1853e00640 1 -- 172.21.15.46:0/4268313176 >> v2:172.21.15.82:3300/0 conn(0x7f18540d12f0 msgr2=0x7f18540d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:13.334 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1853e00640 1 --2- 172.21.15.46:0/4268313176 >> v2:172.21.15.82:3300/0 conn(0x7f18540d12f0 0x7f18540d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.334 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1853e00640 1 -- 172.21.15.46:0/4268313176 >> v1:172.21.15.82:6789/0 conn(0x7f18540b1de0 legacy=0x7f1854099a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.334 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1853e00640 1 -- 172.21.15.46:0/4268313176 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185406ca10 con 0x7f185409a1a0 2024-04-25T08:42:13.334 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1859a00640 1 --2- 172.21.15.46:0/4268313176 >> v2:172.21.15.82:3300/0 conn(0x7f18540d12f0 0x7f18540d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:13.334 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.339+0000 7f1853400640 1 -- 172.21.15.46:0/4268313176 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422828592 0 0) 0x7f1848001120 con 0x7f18540b1de0 2024-04-25T08:42:13.337 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f1853e00640 1 --2- 172.21.15.46:0/4268313176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f185409a1a0 0x7f18540cfd40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18400097c0 tx=0x7f184001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59c5ba89219daaee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:13.337 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f1853400640 1 -- 172.21.15.46:0/4268313176 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18400ab600 con 0x7f185409a1a0 2024-04-25T08:42:13.337 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f1853400640 1 -- 172.21.15.46:0/4268313176 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18400b0070 con 0x7f185409a1a0 2024-04-25T08:42:13.338 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f1853400640 1 -- 172.21.15.46:0/4268313176 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18400ab9c0 con 0x7f185409a1a0 2024-04-25T08:42:13.338 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f185ba00640 1 -- 172.21.15.46:0/4268313176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f185409a1a0 msgr2=0x7f18540cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.338 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f185ba00640 1 --2- 172.21.15.46:0/4268313176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f185409a1a0 0x7f18540cfd40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18400097c0 tx=0x7f184001c3e0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.338 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f185ba00640 1 -- 172.21.15.46:0/4268313176 shutdown_connections 2024-04-25T08:42:13.338 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f185ba00640 1 --2- 172.21.15.46:0/4268313176 >> v2:172.21.15.82:3300/0 conn(0x7f18540d12f0 0x7f18540d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.338 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f185ba00640 1 --2- 172.21.15.46:0/4268313176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f185409a1a0 0x7f18540cfd40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.338 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f185ba00640 1 -- 172.21.15.46:0/4268313176 >> 172.21.15.46:0/4268313176 conn(0x7f18540d17f0 msgr2=0x7f18540d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:13.338 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f185ba00640 1 -- 172.21.15.46:0/4268313176 shutdown_connections 2024-04-25T08:42:13.338 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f185ba00640 1 -- 172.21.15.46:0/4268313176 wait complete. 2024-04-25T08:42:13.339 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.343+0000 7f185ba00640 1 Processor -- start 2024-04-25T08:42:13.339 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 -- start start 2024-04-25T08:42:13.341 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185409a1a0 0x7f185415ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.341 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18540b1de0 0x7f18540aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.341 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18540d12f0 0x7f1854160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.341 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185409a7d0 con 0x7f185409a1a0 2024-04-25T08:42:13.341 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540b21c0 con 0x7f18540d12f0 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18540cbe40 con 0x7f18540b1de0 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185409a1a0 0x7f185415ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185409a1a0 0x7f185415ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51490/0 (socket says 172.21.15.46:51490) 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859000640 1 -- 172.21.15.46:0/4134645044 learned_addr learned my addr 172.21.15.46:0/4134645044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1853e00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18540b1de0 0x7f18540aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859a00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18540d12f0 0x7f1854160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859000640 1 -- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18540b1de0 msgr2=0x7f18540aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859000640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18540b1de0 0x7f18540aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859000640 1 -- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18540d12f0 msgr2=0x7f1854160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859000640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18540d12f0 0x7f1854160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.342 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859000640 1 -- 172.21.15.46:0/4134645044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18541604f0 con 0x7f185409a1a0 2024-04-25T08:42:13.343 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1859000640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185409a1a0 0x7f185415ffb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f184800c780 tx=0x7f184800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:13.343 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1850c00640 1 -- 172.21.15.46:0/4134645044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848018070 con 0x7f185409a1a0 2024-04-25T08:42:13.343 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185415f1f0 con 0x7f185409a1a0 2024-04-25T08:42:13.343 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1850c00640 1 -- 172.21.15.46:0/4134645044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18480072f0 con 0x7f185409a1a0 2024-04-25T08:42:13.343 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f1850c00640 1 -- 172.21.15.46:0/4134645044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18480047f0 con 0x7f185409a1a0 2024-04-25T08:42:13.343 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185415f7b0 con 0x7f185409a1a0 2024-04-25T08:42:13.343 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.347+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185404ef30 con 0x7f185409a1a0 2024-04-25T08:42:13.344 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.351+0000 7f1850c00640 1 -- 172.21.15.46:0/4134645044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f184801b070 con 0x7f185409a1a0 2024-04-25T08:42:13.344 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.351+0000 7f1850c00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1834066f60 0x7f1834069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.344 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.351+0000 7f1850c00640 1 -- 172.21.15.46:0/4134645044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1848084070 con 0x7f185409a1a0 2024-04-25T08:42:13.345 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.351+0000 7f1853e00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1834066f60 0x7f1834069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.345 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.351+0000 7f1853e00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1834066f60 0x7f1834069420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18400af000 tx=0x7f18400a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:13.349 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.355+0000 7f1850c00640 1 -- 172.21.15.46:0/4134645044 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18540cf880 con 0x7f185409a1a0 2024-04-25T08:42:13.512 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18540d0f00 con 0x7f185409a1a0 2024-04-25T08:42:13.512 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f1850c00640 1 -- 172.21.15.46:0/4134645044 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12836 (secure 0 0 0) 0x7f1848054d70 con 0x7f185409a1a0 2024-04-25T08:42:13.513 INFO:teuthology.misc.health.smithi046.stdout: 2024-04-25T08:42:13.513 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":17,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:11.088887+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1834066f60 msgr2=0x7f1834069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1834066f60 0x7f1834069420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f18400af000 tx=0x7f18400a7000 comp rx=0 tx=0).stop 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185409a1a0 msgr2=0x7f185415ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185409a1a0 0x7f185415ffb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f184800c780 tx=0x7f184800cc40 comp rx=0 tx=0).stop 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 shutdown_connections 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1834066f60 0x7f1834069420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18540d12f0 0x7f1854160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18540b1de0 0x7f18540aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 --2- 172.21.15.46:0/4134645044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185409a1a0 0x7f185415ffb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 >> 172.21.15.46:0/4134645044 conn(0x7f18540d17f0 msgr2=0x7f185415b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:13.515 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 shutdown_connections 2024-04-25T08:42:13.516 INFO:teuthology.misc.health.smithi046.stderr:2024-04-25T08:42:13.519+0000 7f185ba00640 1 -- 172.21.15.46:0/4134645044 wait complete. 2024-04-25T08:42:13.532 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T08:42:13.532 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T08:42:13.533 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:42:13.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:42:13.536 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:42:13.536 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:42:13.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:42:13.539 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:42:13.539 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:42:13.539 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:42:13.542 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:42:13.543 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:42:13.543 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:42:13.546 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:42:13.546 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T08:42:13.546 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:13.631 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:42:13.632 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:42:13.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.695+0000 7f1220c00640 1 Processor -- start 2024-04-25T08:42:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.695+0000 7f1220c00640 1 -- start start 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f1220c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f121c0b3140 0x7f121c0716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f1220c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 0x7f121c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f1220c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f121c161530 0x7f121c1619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f1220c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121c072990 con 0x7f121c161530 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f1220c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121c0b3a60 con 0x7f121c0b3140 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f1220c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121c0b3f20 con 0x7f121c15df60 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f121a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f121c161530 0x7f121c1619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f121a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f121c161530 0x7f121c1619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51518/0 (socket says 172.21.15.46:51518) 2024-04-25T08:42:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f121a800640 1 -- 172.21.15.46:0/3614245969 learned_addr learned my addr 172.21.15.46:0/3614245969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f1219e00640 1 --2- 172.21.15.46:0/3614245969 >> v2:172.21.15.82:3300/0 conn(0x7f121c0b3140 0x7f121c0716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f1213e00640 1 --2- 172.21.15.46:0/3614245969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 0x7f121c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f121a800640 1 -- 172.21.15.46:0/3614245969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 msgr2=0x7f121c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f121a800640 1 --2- 172.21.15.46:0/3614245969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 0x7f121c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f121a800640 1 -- 172.21.15.46:0/3614245969 >> v2:172.21.15.82:3300/0 conn(0x7f121c0b3140 msgr2=0x7f121c0716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f121a800640 1 --2- 172.21.15.46:0/3614245969 >> v2:172.21.15.82:3300/0 conn(0x7f121c0b3140 0x7f121c0716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.699+0000 7f121a800640 1 -- 172.21.15.46:0/3614245969 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c071e10 con 0x7f121c161530 2024-04-25T08:42:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f121a800640 1 --2- 172.21.15.46:0/3614245969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 0x7f121c1619c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f121c0b2810 tx=0x7f121401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559a7563c8b97950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:13.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1219400640 1 -- 172.21.15.46:0/3614245969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140a5550 con 0x7f121c161530 2024-04-25T08:42:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1219400640 1 -- 172.21.15.46:0/3614245969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12140a56f0 con 0x7f121c161530 2024-04-25T08:42:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1219400640 1 -- 172.21.15.46:0/3614245969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140a5ac0 con 0x7f121c161530 2024-04-25T08:42:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 -- 172.21.15.46:0/3614245969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 msgr2=0x7f121c1619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 --2- 172.21.15.46:0/3614245969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 0x7f121c1619c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f121c0b2810 tx=0x7f121401c8d0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 -- 172.21.15.46:0/3614245969 shutdown_connections 2024-04-25T08:42:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 --2- 172.21.15.46:0/3614245969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 0x7f121c1619c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 --2- 172.21.15.46:0/3614245969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 0x7f121c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 --2- 172.21.15.46:0/3614245969 >> v2:172.21.15.82:3300/0 conn(0x7f121c0b3140 0x7f121c0716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 -- 172.21.15.46:0/3614245969 >> 172.21.15.46:0/3614245969 conn(0x7f121c0b1750 msgr2=0x7f121c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 -- 172.21.15.46:0/3614245969 shutdown_connections 2024-04-25T08:42:13.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 -- 172.21.15.46:0/3614245969 wait complete. 2024-04-25T08:42:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 Processor -- start 2024-04-25T08:42:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 -- start start 2024-04-25T08:42:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f121c0b3140 0x7f121c142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 0x7f121c140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 0x7f121c167a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c1477c0 con 0x7f121c161530 2024-04-25T08:42:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121c0726b0 con 0x7f121c0b3140 2024-04-25T08:42:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.703+0000 7f1220c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121c147940 con 0x7f121c15df60 2024-04-25T08:42:13.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f121a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 0x7f121c167a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f121a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 0x7f121c167a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51524/0 (socket says 172.21.15.46:51524) 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f121a800640 1 -- 172.21.15.46:0/1832882667 learned_addr learned my addr 172.21.15.46:0/1832882667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1213e00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 0x7f121c140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1219e00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f121c0b3140 0x7f121c142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f121a800640 1 -- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 msgr2=0x7f121c140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f121a800640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 0x7f121c140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f121a800640 1 -- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f121c0b3140 msgr2=0x7f121c142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f121a800640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f121c0b3140 0x7f121c142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f121a800640 1 -- 172.21.15.46:0/1832882667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121c165c70 con 0x7f121c161530 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f121a800640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 0x7f121c167a30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f121401d170 tx=0x7f1214019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1212000640 1 -- 172.21.15.46:0/1832882667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140bc070 con 0x7f121c161530 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1212000640 1 -- 172.21.15.46:0/1832882667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12140b8640 con 0x7f121c161530 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1212000640 1 -- 172.21.15.46:0/1832882667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140a5550 con 0x7f121c161530 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121c165f50 con 0x7f121c161530 2024-04-25T08:42:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121c166510 con 0x7f121c161530 2024-04-25T08:42:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121c069080 con 0x7f121c161530 2024-04-25T08:42:13.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1212000640 1 -- 172.21.15.46:0/1832882667 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12140b8a30 con 0x7f121c161530 2024-04-25T08:42:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.707+0000 7f1212000640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11f8066f90 0x7f11f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.711+0000 7f1212000640 1 -- 172.21.15.46:0/1832882667 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f121411f2c0 con 0x7f121c161530 2024-04-25T08:42:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.711+0000 7f1219e00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11f8066f90 0x7f11f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.711+0000 7f1219e00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11f8066f90 0x7f11f8069450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f120c009a40 tx=0x7f120c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:13.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.715+0000 7f1212000640 1 -- 172.21.15.46:0/1832882667 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12140165e0 con 0x7f121c161530 2024-04-25T08:42:13.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.875+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f121c06c6c0 con 0x7f121c161530 2024-04-25T08:42:13.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1212000640 1 -- 172.21.15.46:0/1832882667 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12836 (secure 0 0 0) 0x7f12140effc0 con 0x7f121c161530 2024-04-25T08:42:13.871 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:13.872 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:11.088887+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:13.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11f8066f90 msgr2=0x7f11f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11f8066f90 0x7f11f8069450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f120c009a40 tx=0x7f120c009190 comp rx=0 tx=0).stop 2024-04-25T08:42:13.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 msgr2=0x7f121c167a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:13.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 0x7f121c167a30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f121401d170 tx=0x7f1214019670 comp rx=0 tx=0).stop 2024-04-25T08:42:13.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 shutdown_connections 2024-04-25T08:42:13.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11f8066f90 0x7f11f8069450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f121c161530 0x7f121c167a30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f121c15df60 0x7f121c140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 --2- 172.21.15.46:0/1832882667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f121c0b3140 0x7f121c142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 >> 172.21.15.46:0/1832882667 conn(0x7f121c0b1750 msgr2=0x7f121c0adb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 shutdown_connections 2024-04-25T08:42:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:13.879+0000 7f1220c00640 1 -- 172.21.15.46:0/1832882667 wait complete. 2024-04-25T08:42:13.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:42:13.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:42:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc193c00640 1 Processor -- start 2024-04-25T08:42:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc193c00640 1 -- start start 2024-04-25T08:42:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc193c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc18c06b230 0x7fc18c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc193c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c0ce220 0x7fc18c161c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc193c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c06b610 con 0x7fc18c0b2ce0 2024-04-25T08:42:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc193c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18c06f980 con 0x7fc18c06b230 2024-04-25T08:42:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc193c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c06fc60 con 0x7fc18c0ce220 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc191c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc18c0b2ce0 0x7fc18c0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42164/0 (socket says 172.21.15.46:42164) 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc191c00640 1 -- 172.21.15.46:0/11776275 learned_addr learned my addr 172.21.15.46:0/11776275 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc183e00640 1 --2- 172.21.15.46:0/11776275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c0ce220 0x7fc18c161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.043+0000 7fc191200640 1 --2- 172.21.15.46:0/11776275 >> v2:172.21.15.82:3300/0 conn(0x7fc18c06b230 0x7fc18c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc18be00640 1 -- 172.21.15.46:0/11776275 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555248208 0 0) 0x7fc18c06b610 con 0x7fc18c0b2ce0 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc18be00640 1 -- 172.21.15.46:0/11776275 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc170003530 con 0x7fc18c0b2ce0 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc183e00640 1 -- 172.21.15.46:0/11776275 >> v2:172.21.15.82:3300/0 conn(0x7fc18c06b230 msgr2=0x7fc18c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc183e00640 1 --2- 172.21.15.46:0/11776275 >> v2:172.21.15.82:3300/0 conn(0x7fc18c06b230 0x7fc18c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc183e00640 1 -- 172.21.15.46:0/11776275 >> v1:172.21.15.46:6789/0 conn(0x7fc18c0b2ce0 legacy=0x7fc18c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc183e00640 1 -- 172.21.15.46:0/11776275 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c06ff40 con 0x7fc18c0ce220 2024-04-25T08:42:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc191200640 1 --2- 172.21.15.46:0/11776275 >> v2:172.21.15.82:3300/0 conn(0x7fc18c06b230 0x7fc18c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc183e00640 1 --2- 172.21.15.46:0/11776275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c0ce220 0x7fc18c161c70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc1740097c0 tx=0x7fc17401c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf48a899796c359 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc18be00640 1 -- 172.21.15.46:0/11776275 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1740b0070 con 0x7fc18c0ce220 2024-04-25T08:42:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc18be00640 1 -- 172.21.15.46:0/11776275 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1740ab600 con 0x7fc18c0ce220 2024-04-25T08:42:14.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc18be00640 1 -- 172.21.15.46:0/11776275 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1740ab9b0 con 0x7fc18c0ce220 2024-04-25T08:42:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc193c00640 1 -- 172.21.15.46:0/11776275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c0ce220 msgr2=0x7fc18c161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.047+0000 7fc193c00640 1 --2- 172.21.15.46:0/11776275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c0ce220 0x7fc18c161c70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc1740097c0 tx=0x7fc17401c350 comp rx=0 tx=0).stop 2024-04-25T08:42:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 -- 172.21.15.46:0/11776275 shutdown_connections 2024-04-25T08:42:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 --2- 172.21.15.46:0/11776275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c0ce220 0x7fc18c161c70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 --2- 172.21.15.46:0/11776275 >> v2:172.21.15.82:3300/0 conn(0x7fc18c06b230 0x7fc18c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 -- 172.21.15.46:0/11776275 >> 172.21.15.46:0/11776275 conn(0x7fc18c099a00 msgr2=0x7fc18c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 -- 172.21.15.46:0/11776275 shutdown_connections 2024-04-25T08:42:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 -- 172.21.15.46:0/11776275 wait complete. 2024-04-25T08:42:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 Processor -- start 2024-04-25T08:42:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 -- start start 2024-04-25T08:42:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc18c06b230 0x7fc18c0b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c0b2ce0 0x7fc18c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c15c480 0x7fc18c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c162330 con 0x7fc18c06b230 2024-04-25T08:42:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c1621b0 con 0x7fc18c0b2ce0 2024-04-25T08:42:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c1624b0 con 0x7fc18c15c480 2024-04-25T08:42:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc18c06b230 0x7fc18c0b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc18c06b230 0x7fc18c0b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51538/0 (socket says 172.21.15.46:51538) 2024-04-25T08:42:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191200640 1 -- 172.21.15.46:0/2796351350 learned_addr learned my addr 172.21.15.46:0/2796351350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc183e00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c0b2ce0 0x7fc18c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191c00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c15c480 0x7fc18c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191200640 1 -- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c15c480 msgr2=0x7fc18c156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191200640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c15c480 0x7fc18c156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191200640 1 -- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c0b2ce0 msgr2=0x7fc18c0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191200640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c0b2ce0 0x7fc18c0c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191200640 1 -- 172.21.15.46:0/2796351350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c0b4690 con 0x7fc18c06b230 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc183e00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c0b2ce0 0x7fc18c0c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191c00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c15c480 0x7fc18c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:14.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc191200640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc18c06b230 0x7fc18c0b4150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00e980 tx=0x7fc17c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc189600640 1 -- 172.21.15.46:0/2796351350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c017070 con 0x7fc18c06b230 2024-04-25T08:42:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc189600640 1 -- 172.21.15.46:0/2796351350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c00ca00 con 0x7fc18c06b230 2024-04-25T08:42:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c0c8d50 con 0x7fc18c06b230 2024-04-25T08:42:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.051+0000 7fc189600640 1 -- 172.21.15.46:0/2796351350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00cdd0 con 0x7fc18c06b230 2024-04-25T08:42:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.055+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c0c92f0 con 0x7fc18c06b230 2024-04-25T08:42:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.055+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18c069010 con 0x7fc18c06b230 2024-04-25T08:42:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.055+0000 7fc189600640 1 -- 172.21.15.46:0/2796351350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc17c01a070 con 0x7fc18c06b230 2024-04-25T08:42:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.055+0000 7fc189600640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc168066f60 0x7fc168069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.055+0000 7fc189600640 1 -- 172.21.15.46:0/2796351350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc17c084c70 con 0x7fc18c06b230 2024-04-25T08:42:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.055+0000 7fc183e00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc168066f60 0x7fc168069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.059+0000 7fc183e00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc168066f60 0x7fc168069420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc1740ad000 tx=0x7fc174008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.059+0000 7fc189600640 1 -- 172.21.15.46:0/2796351350 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc18c0b4690 con 0x7fc18c06b230 2024-04-25T08:42:14.154 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-04-25T08:42:14.154 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-04-25T08:42:14.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.223+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc18c0cd6d0 con 0x7fc18c06b230 2024-04-25T08:42:14.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.223+0000 7fc189600640 1 -- 172.21.15.46:0/2796351350 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc17c055f30 con 0x7fc18c06b230 2024-04-25T08:42:14.218 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:42:14.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc168066f60 msgr2=0x7fc168069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc168066f60 0x7fc168069420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc1740ad000 tx=0x7fc174008040 comp rx=0 tx=0).stop 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc18c06b230 msgr2=0x7fc18c0b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc18c06b230 0x7fc18c0b4150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00e980 tx=0x7fc17c00ee40 comp rx=0 tx=0).stop 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 shutdown_connections 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc168066f60 0x7fc168069420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc18c15c480 0x7fc18c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c0b2ce0 0x7fc18c0c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 --2- 172.21.15.46:0/2796351350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc18c06b230 0x7fc18c0b4150 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 >> 172.21.15.46:0/2796351350 conn(0x7fc18c099a00 msgr2=0x7fc18c0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 shutdown_connections 2024-04-25T08:42:14.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.227+0000 7fc193c00640 1 -- 172.21.15.46:0/2796351350 wait complete. 2024-04-25T08:42:14.239 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:14.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79fae00640 1 Processor -- start 2024-04-25T08:42:14.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79fae00640 1 -- start start 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79fae00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 0x7f79f415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79fae00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f79f40cdc50 0x7f79f415d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79fae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f40cc700 con 0x7f79f40cbeb0 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79fae00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f40cc9e0 con 0x7f79f40cdc50 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79fae00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40cccc0 con 0x7f79f4159230 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f79f40cbeb0 0x7f79f40cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42172/0 (socket says 172.21.15.46:42172) 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f3400640 1 -- 172.21.15.46:0/3968043119 learned_addr learned my addr 172.21.15.46:0/3968043119 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f3e00640 1 --2- 172.21.15.46:0/3968043119 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 0x7f79f415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f8e00640 1 --2- 172.21.15.46:0/3968043119 >> v2:172.21.15.82:3300/0 conn(0x7f79f40cdc50 0x7f79f415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f2a00640 1 -- 172.21.15.46:0/3968043119 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441671602 0 0) 0x7f79f40cc700 con 0x7f79f40cbeb0 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f2a00640 1 -- 172.21.15.46:0/3968043119 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d8003530 con 0x7f79f40cbeb0 2024-04-25T08:42:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f3e00640 1 -- 172.21.15.46:0/3968043119 >> v2:172.21.15.82:3300/0 conn(0x7f79f40cdc50 msgr2=0x7f79f415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f3e00640 1 --2- 172.21.15.46:0/3968043119 >> v2:172.21.15.82:3300/0 conn(0x7f79f40cdc50 0x7f79f415d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f3e00640 1 -- 172.21.15.46:0/3968043119 >> v1:172.21.15.46:6789/0 conn(0x7f79f40cbeb0 legacy=0x7f79f40cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.387+0000 7f79f3e00640 1 -- 172.21.15.46:0/3968043119 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f40ccfa0 con 0x7f79f4159230 2024-04-25T08:42:14.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79f3e00640 1 --2- 172.21.15.46:0/3968043119 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 0x7f79f415bdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f79e40097c0 tx=0x7f79e401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e835235a507d6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:14.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79f2a00640 1 -- 172.21.15.46:0/3968043119 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40ab600 con 0x7f79f4159230 2024-04-25T08:42:14.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79f2a00640 1 -- 172.21.15.46:0/3968043119 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e40b0070 con 0x7f79f4159230 2024-04-25T08:42:14.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79f2a00640 1 -- 172.21.15.46:0/3968043119 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40ab9a0 con 0x7f79f4159230 2024-04-25T08:42:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79fae00640 1 -- 172.21.15.46:0/3968043119 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 msgr2=0x7f79f415bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79fae00640 1 --2- 172.21.15.46:0/3968043119 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 0x7f79f415bdb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f79e40097c0 tx=0x7f79e401c3e0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79fae00640 1 -- 172.21.15.46:0/3968043119 shutdown_connections 2024-04-25T08:42:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79fae00640 1 --2- 172.21.15.46:0/3968043119 >> v2:172.21.15.82:3300/0 conn(0x7f79f40cdc50 0x7f79f415d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79fae00640 1 --2- 172.21.15.46:0/3968043119 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 0x7f79f415bdb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79fae00640 1 -- 172.21.15.46:0/3968043119 >> 172.21.15.46:0/3968043119 conn(0x7f79f40b12a0 msgr2=0x7f79f40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79fae00640 1 -- 172.21.15.46:0/3968043119 shutdown_connections 2024-04-25T08:42:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.391+0000 7f79fae00640 1 -- 172.21.15.46:0/3968043119 wait complete. 2024-04-25T08:42:14.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 Processor -- start 2024-04-25T08:42:14.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 -- start start 2024-04-25T08:42:14.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f40cbeb0 0x7f79f40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40cdc50 0x7f79f415a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 0x7f79f415f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f415df50 con 0x7f79f40cdc50 2024-04-25T08:42:14.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40cc4a0 con 0x7f79f40cbeb0 2024-04-25T08:42:14.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f415e0d0 con 0x7f79f4159230 2024-04-25T08:42:14.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40cdc50 0x7f79f415a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f40cbeb0 0x7f79f40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40cdc50 0x7f79f415a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51544/0 (socket says 172.21.15.46:51544) 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3400640 1 -- 172.21.15.46:0/796581133 learned_addr learned my addr 172.21.15.46:0/796581133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f8e00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 0x7f79f415f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3e00640 1 -- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 msgr2=0x7f79f415f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3e00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 0x7f79f415f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3e00640 1 -- 172.21.15.46:0/796581133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40cdc50 msgr2=0x7f79f415a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3e00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40cdc50 0x7f79f415a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3e00640 1 -- 172.21.15.46:0/796581133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f415abc0 con 0x7f79f40cbeb0 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3400640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40cdc50 0x7f79f415a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79f3e00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f40cbeb0 0x7f79f40ae930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f79e40ac000 tx=0x7f79e401ac20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79ebe00640 1 -- 172.21.15.46:0/796581133 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40bf610 con 0x7f79f40cbeb0 2024-04-25T08:42:14.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79ebe00640 1 -- 172.21.15.46:0/796581133 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e40bf7b0 con 0x7f79f40cbeb0 2024-04-25T08:42:14.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79ebe00640 1 -- 172.21.15.46:0/796581133 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40bfba0 con 0x7f79f40cbeb0 2024-04-25T08:42:14.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f4141b00 con 0x7f79f40cbeb0 2024-04-25T08:42:14.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f415a8d0 con 0x7f79f40cbeb0 2024-04-25T08:42:14.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.395+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f4069080 con 0x7f79f40cbeb0 2024-04-25T08:42:14.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.399+0000 7f79ebe00640 1 -- 172.21.15.46:0/796581133 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79e40bd020 con 0x7f79f40cbeb0 2024-04-25T08:42:14.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.399+0000 7f79ebe00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79d4066f60 0x7f79d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:14.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.399+0000 7f79ebe00640 1 -- 172.21.15.46:0/796581133 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f79e411f9e0 con 0x7f79f40cbeb0 2024-04-25T08:42:14.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.399+0000 7f79f3400640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79d4066f60 0x7f79d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:14.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.399+0000 7f79f3400640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79d4066f60 0x7f79d4069420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f79f41f9980 tx=0x7f79e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:14.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.403+0000 7f79ebe00640 1 -- 172.21.15.46:0/796581133 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79e4016090 con 0x7f79f40cbeb0 2024-04-25T08:42:14.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.603+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79f40723d0 con 0x7f79f40cbeb0 2024-04-25T08:42:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.603+0000 7f79ebe00640 1 -- 172.21.15.46:0/796581133 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f79e40f0760 con 0x7f79f40cbeb0 2024-04-25T08:42:14.599 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:14.599 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T08:42:14.599 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-04-25T08:42:14.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79d4066f60 msgr2=0x7f79d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79d4066f60 0x7f79d4069420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f79f41f9980 tx=0x7f79e0009040 comp rx=0 tx=0).stop 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f40cbeb0 msgr2=0x7f79f40ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f40cbeb0 0x7f79f40ae930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f79e40ac000 tx=0x7f79e401ac20 comp rx=0 tx=0).stop 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 shutdown_connections 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79d4066f60 0x7f79d4069420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79f4159230 0x7f79f415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79f40cdc50 0x7f79f415a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 --2- 172.21.15.46:0/796581133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79f40cbeb0 0x7f79f40ae930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 >> 172.21.15.46:0/796581133 conn(0x7f79f40b12a0 msgr2=0x7f79f415c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 shutdown_connections 2024-04-25T08:42:14.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:14.607+0000 7f79fae00640 1 -- 172.21.15.46:0/796581133 wait complete. 2024-04-25T08:42:16.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.779+0000 7fa16f400640 1 Processor -- start 2024-04-25T08:42:16.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.779+0000 7fa16f400640 1 -- start start 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16f400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa16815b9a0 0x7fa1680af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa1680ad4c0 0x7fa16815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16f400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16815fed0 con 0x7fa16806f570 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16f400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1681601b0 con 0x7fa1680ad4c0 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16f400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16815c0f0 con 0x7fa16815b9a0 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa1680ad4c0 0x7fa16815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa167e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa16806f570 0x7fa1680acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42174/0 (socket says 172.21.15.46:42174) 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa167e00640 1 -- 172.21.15.46:0/2783231429 learned_addr learned my addr 172.21.15.46:0/2783231429 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16ca00640 1 --2- 172.21.15.46:0/2783231429 >> v2:172.21.15.82:3300/0 conn(0x7fa16815b9a0 0x7fa1680af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16d400640 1 -- 172.21.15.46:0/2783231429 >> v2:172.21.15.82:3300/0 conn(0x7fa16815b9a0 msgr2=0x7fa1680af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:16.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16d400640 1 --2- 172.21.15.46:0/2783231429 >> v2:172.21.15.82:3300/0 conn(0x7fa16815b9a0 0x7fa1680af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16d400640 1 -- 172.21.15.46:0/2783231429 >> v1:172.21.15.46:6789/0 conn(0x7fa16806f570 legacy=0x7fa1680acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:16.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16d400640 1 -- 172.21.15.46:0/2783231429 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16815c3d0 con 0x7fa1680ad4c0 2024-04-25T08:42:16.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa167400640 1 -- 172.21.15.46:0/2783231429 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403328172 0 0) 0x7fa16815fed0 con 0x7fa16806f570 2024-04-25T08:42:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa16d400640 1 --2- 172.21.15.46:0/2783231429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 0x7fa16815c8f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa16806d920 tx=0x7fa16001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48bf06dac81d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa167400640 1 -- 172.21.15.46:0/2783231429 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600b2070 con 0x7fa1680ad4c0 2024-04-25T08:42:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa167400640 1 -- 172.21.15.46:0/2783231429 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1600a58c0 con 0x7fa1680ad4c0 2024-04-25T08:42:16.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.783+0000 7fa167400640 1 -- 172.21.15.46:0/2783231429 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600a5c90 con 0x7fa1680ad4c0 2024-04-25T08:42:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 -- 172.21.15.46:0/2783231429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 msgr2=0x7fa16815c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 --2- 172.21.15.46:0/2783231429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 0x7fa16815c8f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa16806d920 tx=0x7fa16001c7e0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 -- 172.21.15.46:0/2783231429 shutdown_connections 2024-04-25T08:42:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 --2- 172.21.15.46:0/2783231429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 0x7fa16815c8f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 --2- 172.21.15.46:0/2783231429 >> v2:172.21.15.82:3300/0 conn(0x7fa16815b9a0 0x7fa1680af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 -- 172.21.15.46:0/2783231429 >> 172.21.15.46:0/2783231429 conn(0x7fa1680cfd90 msgr2=0x7fa1680d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 -- 172.21.15.46:0/2783231429 shutdown_connections 2024-04-25T08:42:16.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 -- 172.21.15.46:0/2783231429 wait complete. 2024-04-25T08:42:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 Processor -- start 2024-04-25T08:42:16.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 -- start start 2024-04-25T08:42:16.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16806f570 0x7fa168150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 0x7fa16815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa16815b9a0 0x7fa168150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16806b7b0 con 0x7fa1680ad4c0 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16806b630 con 0x7fa16806f570 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16f400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa168153b20 con 0x7fa16815b9a0 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa167e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 0x7fa16815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa167e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 0x7fa16815f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51570/0 (socket says 172.21.15.46:51570) 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa167e00640 1 -- 172.21.15.46:0/2251234336 learned_addr learned my addr 172.21.15.46:0/2251234336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16d400640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa16815b9a0 0x7fa168150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa16ca00640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16806f570 0x7fa168150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa167e00640 1 -- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa16815b9a0 msgr2=0x7fa168150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa167e00640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa16815b9a0 0x7fa168150e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa167e00640 1 -- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16806f570 msgr2=0x7fa168150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:16.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa167e00640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16806f570 0x7fa168150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.787+0000 7fa167e00640 1 -- 172.21.15.46:0/2251234336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1681508c0 con 0x7fa1680ad4c0 2024-04-25T08:42:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa16d400640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa16815b9a0 0x7fa168150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa16ca00640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16806f570 0x7fa168150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa167e00640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 0x7fa16815f3d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa15400c520 tx=0x7fa15400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa164c00640 1 -- 172.21.15.46:0/2251234336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154018070 con 0x7fa1680ad4c0 2024-04-25T08:42:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa164c00640 1 -- 172.21.15.46:0/2251234336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1540074e0 con 0x7fa1680ad4c0 2024-04-25T08:42:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa168153280 con 0x7fa1680ad4c0 2024-04-25T08:42:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa164c00640 1 -- 172.21.15.46:0/2251234336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540049f0 con 0x7fa1680ad4c0 2024-04-25T08:42:16.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa168153840 con 0x7fa1680ad4c0 2024-04-25T08:42:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa168069080 con 0x7fa1680ad4c0 2024-04-25T08:42:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa164c00640 1 -- 172.21.15.46:0/2251234336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa15401b070 con 0x7fa1680ad4c0 2024-04-25T08:42:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa164c00640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa148066f60 0x7fa148069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:16.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa164c00640 1 -- 172.21.15.46:0/2251234336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa1540841b0 con 0x7fa1680ad4c0 2024-04-25T08:42:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.791+0000 7fa16ca00640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa148066f60 0x7fa148069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:16.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.795+0000 7fa16ca00640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa148066f60 0x7fa148069420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0097a0 tx=0x7fa15c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:16.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.799+0000 7fa164c00640 1 -- 172.21.15.46:0/2251234336 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa154055500 con 0x7fa1680ad4c0 2024-04-25T08:42:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa168066c40 con 0x7fa1680ad4c0 2024-04-25T08:42:16.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa164c00640 1 -- 172.21.15.46:0/2251234336 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7fa154054e00 con 0x7fa1680ad4c0 2024-04-25T08:42:16.993 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:16.993 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4281,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T08:42:16.993 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa148066f60 msgr2=0x7fa148069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa148066f60 0x7fa148069420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0097a0 tx=0x7fa15c008040 comp rx=0 tx=0).stop 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 msgr2=0x7fa16815f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 0x7fa16815f3d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa15400c520 tx=0x7fa15400c9e0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 shutdown_connections 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa148066f60 0x7fa148069420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa16815b9a0 0x7fa168150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1680ad4c0 0x7fa16815f3d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 --2- 172.21.15.46:0/2251234336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa16806f570 0x7fa168150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 >> 172.21.15.46:0/2251234336 conn(0x7fa1680cfd90 msgr2=0x7fa16815ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 shutdown_connections 2024-04-25T08:42:16.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:16.999+0000 7fa16f400640 1 -- 172.21.15.46:0/2251234336 wait complete. 2024-04-25T08:42:17.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:17.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.159+0000 7f00eaa00640 1 Processor -- start 2024-04-25T08:42:17.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.159+0000 7f00eaa00640 1 -- start start 2024-04-25T08:42:17.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00eaa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00e415a750 0x7f00e406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00eaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 0x7f00e406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00eaa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e406d5e0 con 0x7f00e40aed10 2024-04-25T08:42:17.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00eaa00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e415ab30 con 0x7f00e415a750 2024-04-25T08:42:17.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00eaa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e40ce4f0 con 0x7f00e40cc9e0 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e8a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 0x7f00e406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00e415a750 0x7f00e406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e8a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 0x7f00e406f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:41102/0 (socket says 172.21.15.46:41102) 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00e415a750 0x7f00e406ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:35906/0 (socket says 172.21.15.46:35906) 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e8a00640 1 -- 172.21.15.46:0/3867138376 learned_addr learned my addr 172.21.15.46:0/3867138376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e3e00640 1 -- 172.21.15.46:0/3867138376 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 msgr2=0x7f00e406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e3e00640 1 --2- 172.21.15.46:0/3867138376 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 0x7f00e406f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e3e00640 1 -- 172.21.15.46:0/3867138376 >> v1:172.21.15.82:6789/0 conn(0x7f00e40aed10 legacy=0x7f00e40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e3e00640 1 -- 172.21.15.46:0/3867138376 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e40ce720 con 0x7f00e415a750 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.163+0000 7f00e3400640 1 -- 172.21.15.46:0/3867138376 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978030023 0 0) 0x7f00e406d5e0 con 0x7f00e40aed10 2024-04-25T08:42:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00e3e00640 1 --2- 172.21.15.46:0/3867138376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e415a750 0x7f00e406ec10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00d400ba60 tx=0x7f00d401eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a7f44b8a92fd01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00e3400640 1 -- 172.21.15.46:0/3867138376 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d40b4070 con 0x7f00e415a750 2024-04-25T08:42:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00e3400640 1 -- 172.21.15.46:0/3867138376 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d40a7b30 con 0x7f00e415a750 2024-04-25T08:42:17.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00e3400640 1 -- 172.21.15.46:0/3867138376 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d40af440 con 0x7f00e415a750 2024-04-25T08:42:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 -- 172.21.15.46:0/3867138376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e415a750 msgr2=0x7f00e406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 --2- 172.21.15.46:0/3867138376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e415a750 0x7f00e406ec10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f00d400ba60 tx=0x7f00d401eb70 comp rx=0 tx=0).stop 2024-04-25T08:42:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 -- 172.21.15.46:0/3867138376 shutdown_connections 2024-04-25T08:42:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 --2- 172.21.15.46:0/3867138376 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 0x7f00e406f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 --2- 172.21.15.46:0/3867138376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e415a750 0x7f00e406ec10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 -- 172.21.15.46:0/3867138376 >> 172.21.15.46:0/3867138376 conn(0x7f00e40b12a0 msgr2=0x7f00e40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 -- 172.21.15.46:0/3867138376 shutdown_connections 2024-04-25T08:42:17.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 -- 172.21.15.46:0/3867138376 wait complete. 2024-04-25T08:42:17.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 Processor -- start 2024-04-25T08:42:17.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.167+0000 7f00eaa00640 1 -- start start 2024-04-25T08:42:17.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00eaa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40aed10 0x7f00e4148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00eaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 0x7f00e414b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e415a750 0x7f00e41486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00eaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e406db00 con 0x7f00e415a750 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00eaa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e406d270 con 0x7f00e40aed10 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00eaa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e406dc80 con 0x7f00e40cc9e0 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e415a750 0x7f00e41486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e415a750 0x7f00e41486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51574/0 (socket says 172.21.15.46:51574) 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e8a00640 1 -- 172.21.15.46:0/3312991445 learned_addr learned my addr 172.21.15.46:0/3312991445 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00db400640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 0x7f00e414b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e3e00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40aed10 0x7f00e4148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e8a00640 1 -- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 msgr2=0x7f00e414b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e8a00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 0x7f00e414b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e8a00640 1 -- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40aed10 msgr2=0x7f00e4148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e8a00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40aed10 0x7f00e4148eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e8a00640 1 -- 172.21.15.46:0/3312991445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e414bdf0 con 0x7f00e415a750 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e8a00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e415a750 0x7f00e41486c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e8b0 tx=0x7f00dc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e0c00640 1 -- 172.21.15.46:0/3312991445 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc017070 con 0x7f00e415a750 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e0c00640 1 -- 172.21.15.46:0/3312991445 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc00c940 con 0x7f00e415a750 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e414fef0 con 0x7f00e415a750 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e0c00640 1 -- 172.21.15.46:0/3312991445 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00cd30 con 0x7f00e415a750 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e41504b0 con 0x7f00e415a750 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e4069080 con 0x7f00e415a750 2024-04-25T08:42:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.171+0000 7f00e0c00640 1 -- 172.21.15.46:0/3312991445 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00dc01a070 con 0x7f00e415a750 2024-04-25T08:42:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.175+0000 7f00e0c00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f60 0x7f00c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.175+0000 7f00e0c00640 1 -- 172.21.15.46:0/3312991445 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f00dc084d20 con 0x7f00e415a750 2024-04-25T08:42:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.175+0000 7f00e3e00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f60 0x7f00c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.175+0000 7f00e3e00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f60 0x7f00c0069420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00d40b1000 tx=0x7f00d40a9070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.179+0000 7f00e0c00640 1 -- 172.21.15.46:0/3312991445 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00dc056070 con 0x7f00e415a750 2024-04-25T08:42:17.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.343+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00e40af7d0 con 0x7f00e415a750 2024-04-25T08:42:17.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.343+0000 7f00e0c00640 1 -- 172.21.15.46:0/3312991445 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12836 (secure 0 0 0) 0x7f00dc055970 con 0x7f00e415a750 2024-04-25T08:42:17.337 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:17.337 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:11.088887+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.343+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f60 msgr2=0x7f00c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.343+0000 7f00eaa00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f60 0x7f00c0069420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f00d40b1000 tx=0x7f00d40a9070 comp rx=0 tx=0).stop 2024-04-25T08:42:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.343+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e415a750 msgr2=0x7f00e41486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.343+0000 7f00eaa00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e415a750 0x7f00e41486c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e8b0 tx=0x7f00dc00ed70 comp rx=0 tx=0).stop 2024-04-25T08:42:17.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.347+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 shutdown_connections 2024-04-25T08:42:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.347+0000 7f00eaa00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f60 0x7f00c0069420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.347+0000 7f00eaa00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e415a750 0x7f00e41486c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.347+0000 7f00eaa00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cc9e0 0x7f00e414b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.347+0000 7f00eaa00640 1 --2- 172.21.15.46:0/3312991445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40aed10 0x7f00e4148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.347+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 >> 172.21.15.46:0/3312991445 conn(0x7f00e40b12a0 msgr2=0x7f00e4071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.347+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 shutdown_connections 2024-04-25T08:42:17.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.347+0000 7f00eaa00640 1 -- 172.21.15.46:0/3312991445 wait complete. 2024-04-25T08:42:17.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:42:17.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:42:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.507+0000 7f3575600640 1 Processor -- start 2024-04-25T08:42:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.507+0000 7f3575600640 1 -- start start 2024-04-25T08:42:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3575600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3570157850 0x7f3570071940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3575600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f357015b090 con 0x7f3570071e80 2024-04-25T08:42:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3575600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35700b2640 con 0x7f35700b45b0 2024-04-25T08:42:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3575600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35700b2920 con 0x7f3570157850 2024-04-25T08:42:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3567400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3570071e80 0x7f35700b3ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42180/0 (socket says 172.21.15.46:42180) 2024-04-25T08:42:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3567400640 1 -- 172.21.15.46:0/4252533949 learned_addr learned my addr 172.21.15.46:0/4252533949 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356fe00640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3570157850 0x7f3570071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823296296 0 0) 0x7f357015b090 con 0x7f3570071e80 2024-04-25T08:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3554003450 con 0x7f3570071e80 2024-04-25T08:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619389976 0 0) 0x7f35700b2640 con 0x7f35700b45b0 2024-04-25T08:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f357015b090 con 0x7f35700b45b0 2024-04-25T08:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3155430182 0 0) 0x7f3554003450 con 0x7f3570071e80 2024-04-25T08:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3570157850 msgr2=0x7f3570071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:17.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3570157850 0x7f3570071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 >> v1:172.21.15.82:6789/0 conn(0x7f35700b45b0 legacy=0x7f357006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35700b2c00 con 0x7f3570071e80 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f3558001680 con 0x7f3570071e80 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 >> v1:172.21.15.46:6789/0 conn(0x7f3570071e80 legacy=0x7f35700b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3554003c70 0x7f3554008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3554008da0 0x7f355400b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3570157850 0x7f355400d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35700b2920 con 0x7f3554008da0 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355400dcd0 con 0x7f3554003c70 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355400dea0 con 0x7f3570157850 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3567400640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3554008da0 0x7f355400b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356fe00640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3554003c70 0x7f3554008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3574c00640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3570157850 0x7f355400d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3567400640 1 -- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3570157850 msgr2=0x7f355400d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:17.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3567400640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3570157850 0x7f355400d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3567400640 1 -- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3554003c70 msgr2=0x7f3554008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:17.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3567400640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3554003c70 0x7f3554008860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f3567400640 1 -- 172.21.15.46:0/4252533949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554003450 con 0x7f3554008da0 2024-04-25T08:42:17.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.511+0000 7f356fe00640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3554003c70 0x7f3554008860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f3567400640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3554008da0 0x7f355400b190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f355800c110 tx=0x7f355801cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4560ae0386ff6546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558007c10 con 0x7f3554008da0 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35580a9070 con 0x7f3554008da0 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f356f400640 1 -- 172.21.15.46:0/4252533949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558005d20 con 0x7f3554008da0 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f3575600640 1 -- 172.21.15.46:0/4252533949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3554008da0 msgr2=0x7f355400b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f3575600640 1 --2- 172.21.15.46:0/4252533949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3554008da0 0x7f355400b190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f355800c110 tx=0x7f355801cb40 comp rx=0 tx=0).stop 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f3574c00640 1 -- 172.21.15.46:0/4252533949 reap_dead start 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f3575600640 1 -- 172.21.15.46:0/4252533949 shutdown_connections 2024-04-25T08:42:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f3575600640 1 -- 172.21.15.46:0/4252533949 >> 172.21.15.46:0/4252533949 conn(0x7f3570066fa0 msgr2=0x7f357006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f3575600640 1 -- 172.21.15.46:0/4252533949 shutdown_connections 2024-04-25T08:42:17.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f3575600640 1 -- 172.21.15.46:0/4252533949 wait complete. 2024-04-25T08:42:17.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.515+0000 7f3575600640 1 Processor -- start 2024-04-25T08:42:17.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 -- start start 2024-04-25T08:42:17.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3554008da0 0x7f35701ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3570143ea0 0x7f35701f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f357014f1d0 0x7f3570153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35700cdab0 con 0x7f3570143ea0 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35700cd930 con 0x7f357014f1d0 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357006d7e0 con 0x7f3554008da0 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3567400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3570143ea0 0x7f35701f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3567400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3570143ea0 0x7f35701f0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51592/0 (socket says 172.21.15.46:51592) 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3567400640 1 -- 172.21.15.46:0/3558467833 learned_addr learned my addr 172.21.15.46:0/3558467833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f356fe00640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3554008da0 0x7f35701ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3574c00640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f357014f1d0 0x7f3570153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3567400640 1 -- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3554008da0 msgr2=0x7f35701ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3567400640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3554008da0 0x7f35701ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3567400640 1 -- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f357014f1d0 msgr2=0x7f3570153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3567400640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f357014f1d0 0x7f3570153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3567400640 1 -- 172.21.15.46:0/3558467833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35701f0ab0 con 0x7f3570143ea0 2024-04-25T08:42:17.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3567400640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3570143ea0 0x7f35701f0570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f355801c970 tx=0x7f3554003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:17.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f356cc00640 1 -- 172.21.15.46:0/3558467833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558007c10 con 0x7f3570143ea0 2024-04-25T08:42:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f356cc00640 1 -- 172.21.15.46:0/3558467833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35580047f0 con 0x7f3570143ea0 2024-04-25T08:42:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f356cc00640 1 -- 172.21.15.46:0/3558467833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558004ba0 con 0x7f3570143ea0 2024-04-25T08:42:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35701f0330 con 0x7f3570143ea0 2024-04-25T08:42:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35701efd70 con 0x7f3570143ea0 2024-04-25T08:42:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.519+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3570069010 con 0x7f3570143ea0 2024-04-25T08:42:17.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.523+0000 7f356cc00640 1 -- 172.21.15.46:0/3558467833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3558007070 con 0x7f3570143ea0 2024-04-25T08:42:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.523+0000 7f356cc00640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f354c066f90 0x7f354c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.523+0000 7f356cc00640 1 -- 172.21.15.46:0/3558467833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f355811e6c0 con 0x7f3570143ea0 2024-04-25T08:42:17.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.523+0000 7f356fe00640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f354c066f90 0x7f354c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.523+0000 7f356fe00640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f354c066f90 0x7f354c069450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3560009be0 tx=0x7f3560008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:17.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.527+0000 7f356cc00640 1 -- 172.21.15.46:0/3558467833 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3558016850 con 0x7f3570143ea0 2024-04-25T08:42:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.687+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f357014f930 con 0x7f3570143ea0 2024-04-25T08:42:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f356cc00640 1 -- 172.21.15.46:0/3558467833 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35580ef990 con 0x7f3570143ea0 2024-04-25T08:42:17.683 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:42:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f354c066f90 msgr2=0x7f354c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f354c066f90 0x7f354c069450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3560009be0 tx=0x7f3560008040 comp rx=0 tx=0).stop 2024-04-25T08:42:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3570143ea0 msgr2=0x7f35701f0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3570143ea0 0x7f35701f0570 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f355801c970 tx=0x7f3554003740 comp rx=0 tx=0).stop 2024-04-25T08:42:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 shutdown_connections 2024-04-25T08:42:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f354c066f90 0x7f354c069450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f357014f1d0 0x7f3570153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3570143ea0 0x7f35701f0570 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 --2- 172.21.15.46:0/3558467833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3554008da0 0x7f35701ef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 >> 172.21.15.46:0/3558467833 conn(0x7f3570066fa0 msgr2=0x7f357015bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 shutdown_connections 2024-04-25T08:42:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.691+0000 7f3575600640 1 -- 172.21.15.46:0/3558467833 wait complete. 2024-04-25T08:42:17.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:42:17.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e87e00640 1 Processor -- start 2024-04-25T08:42:17.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e87e00640 1 -- start start 2024-04-25T08:42:17.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e87e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e8015ef20 0x7f9e800cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e800cd6d0 0x7f9e8009a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e87e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e8009ab50 0x7f9e80099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e87e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8015f300 con 0x7f9e8015ef20 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e87e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e800cddb0 con 0x7f9e8009ab50 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e87e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800cc3d0 con 0x7f9e800cd6d0 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e85400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e8015ef20 0x7f9e800cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e85400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e8015ef20 0x7f9e800cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51606/0 (socket says 172.21.15.46:51606) 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e85400640 1 -- 172.21.15.46:0/2054908769 learned_addr learned my addr 172.21.15.46:0/2054908769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e84a00640 1 --2- 172.21.15.46:0/2054908769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e800cd6d0 0x7f9e8009a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e85e00640 1 --2- 172.21.15.46:0/2054908769 >> v2:172.21.15.82:3300/0 conn(0x7f9e8009ab50 0x7f9e80099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e85400640 1 -- 172.21.15.46:0/2054908769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e800cd6d0 msgr2=0x7f9e8009a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e85400640 1 --2- 172.21.15.46:0/2054908769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e800cd6d0 0x7f9e8009a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e85400640 1 -- 172.21.15.46:0/2054908769 >> v2:172.21.15.82:3300/0 conn(0x7f9e8009ab50 msgr2=0x7f9e80099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e85400640 1 --2- 172.21.15.46:0/2054908769 >> v2:172.21.15.82:3300/0 conn(0x7f9e8009ab50 0x7f9e80099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e85400640 1 -- 172.21.15.46:0/2054908769 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e800cc6b0 con 0x7f9e8015ef20 2024-04-25T08:42:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.851+0000 7f9e84a00640 1 --2- 172.21.15.46:0/2054908769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e800cd6d0 0x7f9e8009a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e85400640 1 --2- 172.21.15.46:0/2054908769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8015ef20 0x7f9e800cd190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9e80071e10 tx=0x7f9e7401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61960da9fe99d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e7fe00640 1 -- 172.21.15.46:0/2054908769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e740aa430 con 0x7f9e8015ef20 2024-04-25T08:42:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e7fe00640 1 -- 172.21.15.46:0/2054908769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e740af070 con 0x7f9e8015ef20 2024-04-25T08:42:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e7fe00640 1 -- 172.21.15.46:0/2054908769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e740aa7f0 con 0x7f9e8015ef20 2024-04-25T08:42:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e87e00640 1 -- 172.21.15.46:0/2054908769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8015ef20 msgr2=0x7f9e800cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e87e00640 1 --2- 172.21.15.46:0/2054908769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8015ef20 0x7f9e800cd190 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9e80071e10 tx=0x7f9e7401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e87e00640 1 -- 172.21.15.46:0/2054908769 shutdown_connections 2024-04-25T08:42:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e87e00640 1 --2- 172.21.15.46:0/2054908769 >> v2:172.21.15.82:3300/0 conn(0x7f9e8009ab50 0x7f9e80099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e87e00640 1 --2- 172.21.15.46:0/2054908769 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e800cd6d0 0x7f9e8009a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e87e00640 1 --2- 172.21.15.46:0/2054908769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8015ef20 0x7f9e800cd190 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e87e00640 1 -- 172.21.15.46:0/2054908769 >> 172.21.15.46:0/2054908769 conn(0x7f9e800b3120 msgr2=0x7f9e800b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e87e00640 1 -- 172.21.15.46:0/2054908769 shutdown_connections 2024-04-25T08:42:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.855+0000 7f9e87e00640 1 -- 172.21.15.46:0/2054908769 wait complete. 2024-04-25T08:42:17.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 Processor -- start 2024-04-25T08:42:17.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 -- start start 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8009ab50 0x7f9e8009b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8015ef20 0x7f9e80159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e8015a500 0x7f9e8009cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8015f540 con 0x7f9e8009ab50 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8009a350 con 0x7f9e8015ef20 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8015f6c0 con 0x7f9e8015a500 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8009ab50 0x7f9e8009b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8009ab50 0x7f9e8009b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51622/0 (socket says 172.21.15.46:51622) 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85400640 1 -- 172.21.15.46:0/3297310993 learned_addr learned my addr 172.21.15.46:0/3297310993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e84a00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8015ef20 0x7f9e80159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85e00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e8015a500 0x7f9e8009cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85400640 1 -- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e8015a500 msgr2=0x7f9e8009cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85400640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e8015a500 0x7f9e8009cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85400640 1 -- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8015ef20 msgr2=0x7f9e80159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85400640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8015ef20 0x7f9e80159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85400640 1 -- 172.21.15.46:0/3297310993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8006cfd0 con 0x7f9e8009ab50 2024-04-25T08:42:17.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e85400640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8009ab50 0x7f9e8009b4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e7401aba0 tx=0x7f9e7401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e7d600640 1 -- 172.21.15.46:0/3297310993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e740bd5f0 con 0x7f9e8009ab50 2024-04-25T08:42:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e7d600640 1 -- 172.21.15.46:0/3297310993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e740bd7b0 con 0x7f9e8009ab50 2024-04-25T08:42:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e7d600640 1 -- 172.21.15.46:0/3297310993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e740bdba0 con 0x7f9e8009ab50 2024-04-25T08:42:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8006d2b0 con 0x7f9e8009ab50 2024-04-25T08:42:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8006d870 con 0x7f9e8009ab50 2024-04-25T08:42:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.859+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e80069080 con 0x7f9e8009ab50 2024-04-25T08:42:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.863+0000 7f9e7d600640 1 -- 172.21.15.46:0/3297310993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e740b33f0 con 0x7f9e8009ab50 2024-04-25T08:42:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.863+0000 7f9e7d600640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e60066f60 0x7f9e60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.863+0000 7f9e7d600640 1 -- 172.21.15.46:0/3297310993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9e7411f000 con 0x7f9e8009ab50 2024-04-25T08:42:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.863+0000 7f9e84a00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e60066f60 0x7f9e60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.863+0000 7f9e84a00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e60066f60 0x7f9e60069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9e800b7030 tx=0x7f9e6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:17.867+0000 7f9e7d600640 1 -- 172.21.15.46:0/3297310993 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e740c3060 con 0x7f9e8009ab50 2024-04-25T08:42:18.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9e800d0700 con 0x7f9e8009ab50 2024-04-25T08:42:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e7d600640 1 -- 172.21.15.46:0/3297310993 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9e74016110 con 0x7f9e8009ab50 2024-04-25T08:42:18.024 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:18.024 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:42:18.024 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:42:18.024 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-25T08:42:18.024 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-25T08:42:18.024 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:42:18.024 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e60066f60 msgr2=0x7f9e60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e60066f60 0x7f9e60069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9e800b7030 tx=0x7f9e6c008040 comp rx=0 tx=0).stop 2024-04-25T08:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8009ab50 msgr2=0x7f9e8009b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8009ab50 0x7f9e8009b4c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9e7401aba0 tx=0x7f9e7401ad30 comp rx=0 tx=0).stop 2024-04-25T08:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 shutdown_connections 2024-04-25T08:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e60066f60 0x7f9e60069420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e8015a500 0x7f9e8009cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8015ef20 0x7f9e80159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:18.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 --2- 172.21.15.46:0/3297310993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e8009ab50 0x7f9e8009b4c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:18.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 >> 172.21.15.46:0/3297310993 conn(0x7f9e800b3120 msgr2=0x7f9e8015d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:18.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 shutdown_connections 2024-04-25T08:42:18.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.031+0000 7f9e87e00640 1 -- 172.21.15.46:0/3297310993 wait complete. 2024-04-25T08:42:18.045 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:42:18.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:42:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.195+0000 7fdc20c00640 1 Processor -- start 2024-04-25T08:42:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.195+0000 7fdc20c00640 1 -- start start 2024-04-25T08:42:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc20c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc1c09a110 0x7fdc1c0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc20c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc1c15deb0 0x7fdc1c1622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc20c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c0cf750 con 0x7fdc1c15dad0 2024-04-25T08:42:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc20c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0d0320 con 0x7fdc1c15deb0 2024-04-25T08:42:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc20c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cf9f0 con 0x7fdc1c09a110 2024-04-25T08:42:18.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc1be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc1c15deb0 0x7fdc1c1622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc1be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc1c15deb0 0x7fdc1c1622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44104/0 (socket says 172.21.15.46:44104) 2024-04-25T08:42:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc1be00640 1 -- 172.21.15.46:0/3127790578 learned_addr learned my addr 172.21.15.46:0/3127790578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc13e00640 1 --2- 172.21.15.46:0/3127790578 >> v2:172.21.15.82:3300/0 conn(0x7fdc1c09a110 0x7fdc1c0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc1be00640 1 -- 172.21.15.46:0/3127790578 >> v2:172.21.15.82:3300/0 conn(0x7fdc1c09a110 msgr2=0x7fdc1c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc1be00640 1 --2- 172.21.15.46:0/3127790578 >> v2:172.21.15.82:3300/0 conn(0x7fdc1c09a110 0x7fdc1c0d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc1be00640 1 -- 172.21.15.46:0/3127790578 >> v1:172.21.15.82:6789/0 conn(0x7fdc1c15dad0 legacy=0x7fdc1c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:18.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc1be00640 1 -- 172.21.15.46:0/3127790578 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0cfcd0 con 0x7fdc1c15deb0 2024-04-25T08:42:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.199+0000 7fdc1be00640 1 --2- 172.21.15.46:0/3127790578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c15deb0 0x7fdc1c1622a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c06daf0 tx=0x7fdc1401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03f80bc713e6df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc1aa00640 1 -- 172.21.15.46:0/3127790578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140a5860 con 0x7fdc1c15deb0 2024-04-25T08:42:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc1aa00640 1 -- 172.21.15.46:0/3127790578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc140a5a00 con 0x7fdc1c15deb0 2024-04-25T08:42:18.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc1aa00640 1 -- 172.21.15.46:0/3127790578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140a5dd0 con 0x7fdc1c15deb0 2024-04-25T08:42:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 -- 172.21.15.46:0/3127790578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c15deb0 msgr2=0x7fdc1c1622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 --2- 172.21.15.46:0/3127790578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c15deb0 0x7fdc1c1622a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c06daf0 tx=0x7fdc1401c980 comp rx=0 tx=0).stop 2024-04-25T08:42:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 -- 172.21.15.46:0/3127790578 shutdown_connections 2024-04-25T08:42:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 --2- 172.21.15.46:0/3127790578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c15deb0 0x7fdc1c1622a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:18.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 --2- 172.21.15.46:0/3127790578 >> v2:172.21.15.82:3300/0 conn(0x7fdc1c09a110 0x7fdc1c0d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 -- 172.21.15.46:0/3127790578 >> 172.21.15.46:0/3127790578 conn(0x7fdc1c0b2f70 msgr2=0x7fdc1c0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 -- 172.21.15.46:0/3127790578 shutdown_connections 2024-04-25T08:42:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 -- 172.21.15.46:0/3127790578 wait complete. 2024-04-25T08:42:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 Processor -- start 2024-04-25T08:42:18.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 -- start start 2024-04-25T08:42:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c09a110 0x7fdc1c15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc1c15dad0 0x7fdc1c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c15deb0 0x7fdc1c0b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cc670 con 0x7fdc1c09a110 2024-04-25T08:42:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cc4f0 con 0x7fdc1c15deb0 2024-04-25T08:42:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc20c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cc7f0 con 0x7fdc1c15dad0 2024-04-25T08:42:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc1b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c09a110 0x7fdc1c15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:18.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc1b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c09a110 0x7fdc1c15aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44118/0 (socket says 172.21.15.46:44118) 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc1b400640 1 -- 172.21.15.46:0/4015375011 learned_addr learned my addr 172.21.15.46:0/4015375011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc1be00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c15deb0 0x7fdc1c0b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.203+0000 7fdc13e00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc1c15dad0 0x7fdc1c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc1b400640 1 -- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc1c15dad0 msgr2=0x7fdc1c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc1b400640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc1c15dad0 0x7fdc1c0b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc1b400640 1 -- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c15deb0 msgr2=0x7fdc1c0b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc1b400640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c15deb0 0x7fdc1c0b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc1b400640 1 -- 172.21.15.46:0/4015375011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c15f690 con 0x7fdc1c09a110 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc1b400640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c09a110 0x7fdc1c15aa10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c450 tx=0x7fdc0c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc13400640 1 -- 172.21.15.46:0/4015375011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c011790 con 0x7fdc1c09a110 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc13400640 1 -- 172.21.15.46:0/4015375011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c017070 con 0x7fdc1c09a110 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc13400640 1 -- 172.21.15.46:0/4015375011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c011b90 con 0x7fdc1c09a110 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c15f970 con 0x7fdc1c09a110 2024-04-25T08:42:18.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c15ff30 con 0x7fdc1c09a110 2024-04-25T08:42:18.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c069010 con 0x7fdc1c09a110 2024-04-25T08:42:18.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc13400640 1 -- 172.21.15.46:0/4015375011 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc0c011d30 con 0x7fdc1c09a110 2024-04-25T08:42:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc13400640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbf8066f60 0x7fdbf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.207+0000 7fdc13400640 1 -- 172.21.15.46:0/4015375011 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdc0c0849b0 con 0x7fdc1c09a110 2024-04-25T08:42:18.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.211+0000 7fdc13e00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbf8066f60 0x7fdbf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:18.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.211+0000 7fdc13e00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbf8066f60 0x7fdbf8069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0c7a40 tx=0x7fdc04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.211+0000 7fdc13400640 1 -- 172.21.15.46:0/4015375011 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc0c0556b0 con 0x7fdc1c09a110 2024-04-25T08:42:18.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:18.379+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdc1c0cb8b0 con 0x7fdc1c09a110 2024-04-25T08:42:19.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.315+0000 7fdc13400640 1 -- 172.21.15.46:0/4015375011 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdc0c059750 con 0x7fdc1c09a110 2024-04-25T08:42:19.309 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-25T08:42:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbf8066f60 msgr2=0x7fdbf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbf8066f60 0x7fdbf8069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0c7a40 tx=0x7fdc04008040 comp rx=0 tx=0).stop 2024-04-25T08:42:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c09a110 msgr2=0x7fdc1c15aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c09a110 0x7fdc1c15aa10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00c450 tx=0x7fdc0c00c910 comp rx=0 tx=0).stop 2024-04-25T08:42:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 shutdown_connections 2024-04-25T08:42:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbf8066f60 0x7fdbf8069420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc1c15deb0 0x7fdc1c0b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc1c15dad0 0x7fdc1c0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 --2- 172.21.15.46:0/4015375011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc1c09a110 0x7fdc1c15aa10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 >> 172.21.15.46:0/4015375011 conn(0x7fdc1c0b2f70 msgr2=0x7fdc1c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 shutdown_connections 2024-04-25T08:42:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.319+0000 7fdc20c00640 1 -- 172.21.15.46:0/4015375011 wait complete. 2024-04-25T08:42:19.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T08:42:19.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.499+0000 7f8002600640 1 Processor -- start 2024-04-25T08:42:19.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.499+0000 7f8002600640 1 -- start start 2024-04-25T08:42:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8002600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06bf30 0x7f7ffc0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8002600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ffc09acb0 con 0x7f7ffc06c730 2024-04-25T08:42:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8002600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ffc0d0aa0 con 0x7f7ffc09a410 2024-04-25T08:42:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8002600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc0b3dc0 con 0x7f7ffc06bf30 2024-04-25T08:42:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8001200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7ffc06c730 0x7f7ffc06cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48774/0 (socket says 172.21.15.46:48774) 2024-04-25T08:42:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8001200640 1 -- 172.21.15.46:0/2309245528 learned_addr learned my addr 172.21.15.46:0/2309245528 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:19.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8001c00640 1 --2- 172.21.15.46:0/2309245528 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06bf30 0x7f7ffc0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f7ffbe00640 1 -- 172.21.15.46:0/2309245528 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467527756 0 0) 0x7f7ffc09acb0 con 0x7f7ffc06c730 2024-04-25T08:42:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f7ffbe00640 1 -- 172.21.15.46:0/2309245528 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe0003530 con 0x7f7ffc06c730 2024-04-25T08:42:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f7ffbe00640 1 -- 172.21.15.46:0/2309245528 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051285832 0 0) 0x7f7ffc0d0aa0 con 0x7f7ffc09a410 2024-04-25T08:42:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f7ffbe00640 1 -- 172.21.15.46:0/2309245528 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ffc09acb0 con 0x7f7ffc09a410 2024-04-25T08:42:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8001c00640 1 -- 172.21.15.46:0/2309245528 >> v1:172.21.15.82:6789/0 conn(0x7f7ffc09a410 legacy=0x7f7ffc0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8001c00640 1 -- 172.21.15.46:0/2309245528 >> v1:172.21.15.46:6789/0 conn(0x7f7ffc06c730 legacy=0x7f7ffc06cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:19.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.503+0000 7f8001c00640 1 -- 172.21.15.46:0/2309245528 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc09a9d0 con 0x7f7ffc06bf30 2024-04-25T08:42:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8001c00640 1 --2- 172.21.15.46:0/2309245528 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06bf30 0x7f7ffc0cbab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009df0 tx=0x7f7ff401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c00d999c7a22d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f7ffbe00640 1 -- 172.21.15.46:0/2309245528 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40b2070 con 0x7f7ffc06bf30 2024-04-25T08:42:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f7ffbe00640 1 -- 172.21.15.46:0/2309245528 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff40a5850 con 0x7f7ffc06bf30 2024-04-25T08:42:19.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f7ffbe00640 1 -- 172.21.15.46:0/2309245528 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40a5c20 con 0x7f7ffc06bf30 2024-04-25T08:42:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 -- 172.21.15.46:0/2309245528 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06bf30 msgr2=0x7f7ffc0cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 --2- 172.21.15.46:0/2309245528 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06bf30 0x7f7ffc0cbab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009df0 tx=0x7f7ff401c990 comp rx=0 tx=0).stop 2024-04-25T08:42:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 -- 172.21.15.46:0/2309245528 shutdown_connections 2024-04-25T08:42:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 --2- 172.21.15.46:0/2309245528 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06bf30 0x7f7ffc0cbab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:19.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 -- 172.21.15.46:0/2309245528 >> 172.21.15.46:0/2309245528 conn(0x7f7ffc0b2f50 msgr2=0x7f7ffc0b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 -- 172.21.15.46:0/2309245528 shutdown_connections 2024-04-25T08:42:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 -- 172.21.15.46:0/2309245528 wait complete. 2024-04-25T08:42:19.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 Processor -- start 2024-04-25T08:42:19.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 -- start start 2024-04-25T08:42:19.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc06bf30 0x7f7ffc070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:19.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06c730 0x7f7ffc150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:19.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ffc09a410 0x7f7ffc150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:19.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc0cc400 con 0x7f7ffc06bf30 2024-04-25T08:42:19.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc15dd60 con 0x7f7ffc09a410 2024-04-25T08:42:19.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.507+0000 7f8002600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ffc16c460 con 0x7f7ffc06c730 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc06bf30 0x7f7ffc070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc06bf30 0x7f7ffc070250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44124/0 (socket says 172.21.15.46:44124) 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001200640 1 -- 172.21.15.46:0/1456525826 learned_addr learned my addr 172.21.15.46:0/1456525826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f7ff3e00640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06c730 0x7f7ffc150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001c00640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ffc09a410 0x7f7ffc150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001200640 1 -- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06c730 msgr2=0x7f7ffc150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001200640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06c730 0x7f7ffc150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001200640 1 -- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ffc09a410 msgr2=0x7f7ffc150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001200640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ffc09a410 0x7f7ffc150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001200640 1 -- 172.21.15.46:0/1456525826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc1553b0 con 0x7f7ffc06bf30 2024-04-25T08:42:19.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8001200640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc06bf30 0x7f7ffc070250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00c420 tx=0x7f7fec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f7ff9600640 1 -- 172.21.15.46:0/1456525826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec017070 con 0x7f7ffc06bf30 2024-04-25T08:42:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f7ff9600640 1 -- 172.21.15.46:0/1456525826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec005cf0 con 0x7f7ffc06bf30 2024-04-25T08:42:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f7ff9600640 1 -- 172.21.15.46:0/1456525826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec0072d0 con 0x7f7ffc06bf30 2024-04-25T08:42:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ffc155690 con 0x7f7ffc06bf30 2024-04-25T08:42:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ffc155c50 con 0x7f7ffc06bf30 2024-04-25T08:42:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ffc069010 con 0x7f7ffc06bf30 2024-04-25T08:42:19.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.511+0000 7f7ff9600640 1 -- 172.21.15.46:0/1456525826 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fec01a070 con 0x7f7ffc06bf30 2024-04-25T08:42:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.515+0000 7f7ff9600640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd8066f60 0x7f7fd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.515+0000 7f7ff9600640 1 -- 172.21.15.46:0/1456525826 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f7fec0840b0 con 0x7f7ffc06bf30 2024-04-25T08:42:19.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.515+0000 7f7ff3e00640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd8066f60 0x7f7fd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:19.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.515+0000 7f7ff3e00640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd8066f60 0x7f7fd8069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc156410 tx=0x7f7fe4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:19.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.519+0000 7f7ff9600640 1 -- 172.21.15.46:0/1456525826 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7fec054ce0 con 0x7f7ffc06bf30 2024-04-25T08:42:19.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:19.683+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f7ffc06c510 con 0x7f7ffc06bf30 2024-04-25T08:42:19.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:42:19.987 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:42:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.323+0000 7f7ff9600640 1 -- 172.21.15.46:0/1456525826 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f7fec058d80 con 0x7f7ffc06bf30 2024-04-25T08:42:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-25T08:42:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd8066f60 msgr2=0x7f7fd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd8066f60 0x7f7fd8069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc156410 tx=0x7f7fe4009040 comp rx=0 tx=0).stop 2024-04-25T08:42:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc06bf30 msgr2=0x7f7ffc070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc06bf30 0x7f7ffc070250 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00c420 tx=0x7f7fec00c8e0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 shutdown_connections 2024-04-25T08:42:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd8066f60 0x7f7fd8069420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ffc09a410 0x7f7ffc150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ffc06c730 0x7f7ffc150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 --2- 172.21.15.46:0/1456525826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc06bf30 0x7f7ffc070250 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 >> 172.21.15.46:0/1456525826 conn(0x7f7ffc0b2f50 msgr2=0x7f7ffc06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 shutdown_connections 2024-04-25T08:42:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.327+0000 7f8002600640 1 -- 172.21.15.46:0/1456525826 wait complete. 2024-04-25T08:42:20.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:42:20.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.507+0000 7f63a4e00640 1 Processor -- start 2024-04-25T08:42:20.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.507+0000 7f63a4e00640 1 -- start start 2024-04-25T08:42:20.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f63a4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63a015dd50 0x7f63a00cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:20.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f63a4e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a00d0450 0x7f63a00cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:20.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f63a4e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a009a780 con 0x7f63a006f4d0 2024-04-25T08:42:20.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f63a4e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a00708c0 con 0x7f63a015dd50 2024-04-25T08:42:20.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f63a4e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a009aa60 con 0x7f63a00d0450 2024-04-25T08:42:20.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f6395400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63a015dd50 0x7f63a00cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:20.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f6395400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63a015dd50 0x7f63a00cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44126/0 (socket says 172.21.15.46:44126) 2024-04-25T08:42:20.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f6395400640 1 -- 172.21.15.46:0/2690317551 learned_addr learned my addr 172.21.15.46:0/2690317551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:20.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f639e800640 1 --2- 172.21.15.46:0/2690317551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a00d0450 0x7f63a00cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:20.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f6395400640 1 -- 172.21.15.46:0/2690317551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a00d0450 msgr2=0x7f63a00cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:20.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f6395400640 1 --2- 172.21.15.46:0/2690317551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a00d0450 0x7f63a00cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f6395400640 1 -- 172.21.15.46:0/2690317551 >> v1:172.21.15.82:6789/0 conn(0x7f63a006f4d0 legacy=0x7f63a00cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:20.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f6395400640 1 -- 172.21.15.46:0/2690317551 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a009ad40 con 0x7f63a015dd50 2024-04-25T08:42:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.511+0000 7f6395400640 1 --2- 172.21.15.46:0/2690317551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a015dd50 0x7f63a00cff10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6388009870 tx=0x7f638801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9747d7537e22fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f639d400640 1 -- 172.21.15.46:0/2690317551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880a95f0 con 0x7f63a015dd50 2024-04-25T08:42:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f639d400640 1 -- 172.21.15.46:0/2690317551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63880a9790 con 0x7f63a015dd50 2024-04-25T08:42:20.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f639d400640 1 -- 172.21.15.46:0/2690317551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880a9b60 con 0x7f63a015dd50 2024-04-25T08:42:20.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 -- 172.21.15.46:0/2690317551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a015dd50 msgr2=0x7f63a00cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:20.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 --2- 172.21.15.46:0/2690317551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a015dd50 0x7f63a00cff10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6388009870 tx=0x7f638801c530 comp rx=0 tx=0).stop 2024-04-25T08:42:20.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 -- 172.21.15.46:0/2690317551 shutdown_connections 2024-04-25T08:42:20.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 --2- 172.21.15.46:0/2690317551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a00d0450 0x7f63a00cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 --2- 172.21.15.46:0/2690317551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a015dd50 0x7f63a00cff10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 -- 172.21.15.46:0/2690317551 >> 172.21.15.46:0/2690317551 conn(0x7f63a00b3160 msgr2=0x7f63a00b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:20.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 -- 172.21.15.46:0/2690317551 shutdown_connections 2024-04-25T08:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 -- 172.21.15.46:0/2690317551 wait complete. 2024-04-25T08:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 Processor -- start 2024-04-25T08:42:20.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 -- start start 2024-04-25T08:42:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a006f4d0 0x7f63a014c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a00d0450 0x7f63a014b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a015dd50 0x7f63a014b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00cd390 con 0x7f63a006f4d0 2024-04-25T08:42:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00cd210 con 0x7f63a00d0450 2024-04-25T08:42:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f63a4e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00cd510 con 0x7f63a015dd50 2024-04-25T08:42:20.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f639de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a006f4d0 0x7f63a014c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f639de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a006f4d0 0x7f63a014c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44142/0 (socket says 172.21.15.46:44142) 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.515+0000 7f639de00640 1 -- 172.21.15.46:0/3524172024 learned_addr learned my addr 172.21.15.46:0/3524172024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f6395400640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a00d0450 0x7f63a014b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f639e800640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a015dd50 0x7f63a014b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f639de00640 1 -- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a015dd50 msgr2=0x7f63a014b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f639de00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a015dd50 0x7f63a014b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f639de00640 1 -- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a00d0450 msgr2=0x7f63a014b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f639de00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a00d0450 0x7f63a014b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f639de00640 1 -- 172.21.15.46:0/3524172024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00c8740 con 0x7f63a006f4d0 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f6395400640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a00d0450 0x7f63a014b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f639de00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a006f4d0 0x7f63a014c770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f639000c790 tx=0x7f639000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:20.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f6396a00640 1 -- 172.21.15.46:0/3524172024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390016020 con 0x7f63a006f4d0 2024-04-25T08:42:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f6396a00640 1 -- 172.21.15.46:0/3524172024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390006d60 con 0x7f63a006f4d0 2024-04-25T08:42:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a006d710 con 0x7f63a006f4d0 2024-04-25T08:42:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f6396a00640 1 -- 172.21.15.46:0/3524172024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390003bc0 con 0x7f63a006f4d0 2024-04-25T08:42:20.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a014d5a0 con 0x7f63a006f4d0 2024-04-25T08:42:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a0069010 con 0x7f63a006f4d0 2024-04-25T08:42:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f6396a00640 1 -- 172.21.15.46:0/3524172024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f639000a850 con 0x7f63a006f4d0 2024-04-25T08:42:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.519+0000 7f6396a00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f637c066f60 0x7f637c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.523+0000 7f6396a00640 1 -- 172.21.15.46:0/3524172024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6390084ca0 con 0x7f63a006f4d0 2024-04-25T08:42:20.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.523+0000 7f6395400640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f637c066f60 0x7f637c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:20.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.523+0000 7f6395400640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f637c066f60 0x7f637c069420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f63880ae000 tx=0x7f63880a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:20.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.527+0000 7f6396a00640 1 -- 172.21.15.46:0/3524172024 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6390020020 con 0x7f63a006f4d0 2024-04-25T08:42:20.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:20.719+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f63a0159180 con 0x7f63a006f4d0 2024-04-25T08:42:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.327+0000 7f6396a00640 1 -- 172.21.15.46:0/3524172024 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6390019310 con 0x7f63a006f4d0 2024-04-25T08:42:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T08:42:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f637c066f60 msgr2=0x7f637c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f637c066f60 0x7f637c069420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f63880ae000 tx=0x7f63880a7000 comp rx=0 tx=0).stop 2024-04-25T08:42:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a006f4d0 msgr2=0x7f63a014c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a006f4d0 0x7f63a014c770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f639000c790 tx=0x7f639000cc50 comp rx=0 tx=0).stop 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 shutdown_connections 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f637c066f60 0x7f637c069420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f63a015dd50 0x7f63a014b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63a00d0450 0x7f63a014b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 --2- 172.21.15.46:0/3524172024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63a006f4d0 0x7f63a014c770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 >> 172.21.15.46:0/3524172024 conn(0x7f63a00b3160 msgr2=0x7f63a00b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 shutdown_connections 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.331+0000 7f63a4e00640 1 -- 172.21.15.46:0/3524172024 wait complete. 2024-04-25T08:42:21.377 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T08:42:21.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T08:42:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbeb200640 1 Processor -- start 2024-04-25T08:42:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbeb200640 1 -- start start 2024-04-25T08:42:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbeb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40cff70 0x7fdbe40b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbeb200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbe40b2d50 0x7fdbe40ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbeb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdbe40cd250 0x7fdbe4161d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbeb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe406f830 con 0x7fdbe40cd250 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbeb200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe4070320 con 0x7fdbe40b2d50 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbeb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40d0350 con 0x7fdbe40cff70 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbe9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdbe40cd250 0x7fdbe4161d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbe9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdbe40cd250 0x7fdbe4161d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44152/0 (socket says 172.21.15.46:44152) 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbe9200640 1 -- 172.21.15.46:0/1406591456 learned_addr learned my addr 172.21.15.46:0/1406591456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbe3e00640 1 --2- 172.21.15.46:0/1406591456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40cff70 0x7fdbe40b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbdbe00640 1 --2- 172.21.15.46:0/1406591456 >> v2:172.21.15.82:3300/0 conn(0x7fdbe40b2d50 0x7fdbe40ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbe9200640 1 -- 172.21.15.46:0/1406591456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40cff70 msgr2=0x7fdbe40b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbe9200640 1 --2- 172.21.15.46:0/1406591456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40cff70 0x7fdbe40b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbe9200640 1 -- 172.21.15.46:0/1406591456 >> v2:172.21.15.82:3300/0 conn(0x7fdbe40b2d50 msgr2=0x7fdbe40ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbe9200640 1 --2- 172.21.15.46:0/1406591456 >> v2:172.21.15.82:3300/0 conn(0x7fdbe40b2d50 0x7fdbe40ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbe9200640 1 -- 172.21.15.46:0/1406591456 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe406fa80 con 0x7fdbe40cd250 2024-04-25T08:42:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.531+0000 7fdbdbe00640 1 --2- 172.21.15.46:0/1406591456 >> v2:172.21.15.82:3300/0 conn(0x7fdbe40b2d50 0x7fdbe40ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbe9200640 1 --2- 172.21.15.46:0/1406591456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cd250 0x7fdbe4161d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40732f0 tx=0x7fdbdc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ef90d23c5e5708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbe3400640 1 -- 172.21.15.46:0/1406591456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0a5670 con 0x7fdbe40cd250 2024-04-25T08:42:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbe3400640 1 -- 172.21.15.46:0/1406591456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc0a5810 con 0x7fdbe40cd250 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbe3400640 1 -- 172.21.15.46:0/1406591456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0a5be0 con 0x7fdbe40cd250 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbeb200640 1 -- 172.21.15.46:0/1406591456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cd250 msgr2=0x7fdbe4161d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbeb200640 1 --2- 172.21.15.46:0/1406591456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cd250 0x7fdbe4161d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40732f0 tx=0x7fdbdc01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbeb200640 1 -- 172.21.15.46:0/1406591456 shutdown_connections 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbeb200640 1 --2- 172.21.15.46:0/1406591456 >> v2:172.21.15.82:3300/0 conn(0x7fdbe40b2d50 0x7fdbe40ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbeb200640 1 --2- 172.21.15.46:0/1406591456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cd250 0x7fdbe4161d00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbeb200640 1 --2- 172.21.15.46:0/1406591456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40cff70 0x7fdbe40b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbeb200640 1 -- 172.21.15.46:0/1406591456 >> 172.21.15.46:0/1406591456 conn(0x7fdbe4099a00 msgr2=0x7fdbe40b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbeb200640 1 -- 172.21.15.46:0/1406591456 shutdown_connections 2024-04-25T08:42:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.535+0000 7fdbeb200640 1 -- 172.21.15.46:0/1406591456 wait complete. 2024-04-25T08:42:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 Processor -- start 2024-04-25T08:42:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 -- start start 2024-04-25T08:42:21.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40b2d50 0x7fdbe414e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe40cd250 0x7fdbe41f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cff70 0x7fdbe40b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe415d350 con 0x7fdbe40cff70 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe415d1d0 con 0x7fdbe40cd250 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe415d4d0 con 0x7fdbe40b2d50 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cff70 0x7fdbe40b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cff70 0x7fdbe40b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44168/0 (socket says 172.21.15.46:44168) 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe9200640 1 -- 172.21.15.46:0/2187048139 learned_addr learned my addr 172.21.15.46:0/2187048139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbdbe00640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe40cd250 0x7fdbe41f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe3e00640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40b2d50 0x7fdbe414e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe9200640 1 -- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40b2d50 msgr2=0x7fdbe414e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe9200640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40b2d50 0x7fdbe414e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe9200640 1 -- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe40cd250 msgr2=0x7fdbe41f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe9200640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe40cd250 0x7fdbe41f7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe9200640 1 -- 172.21.15.46:0/2187048139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe4154a70 con 0x7fdbe40cff70 2024-04-25T08:42:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbdbe00640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe40cd250 0x7fdbe41f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe3e00640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40b2d50 0x7fdbe414e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe9200640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cff70 0x7fdbe40b1490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc01d290 tx=0x7fdbdc019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe0c00640 1 -- 172.21.15.46:0/2187048139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc009510 con 0x7fdbe40cff70 2024-04-25T08:42:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe0c00640 1 -- 172.21.15.46:0/2187048139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc0be070 con 0x7fdbe40cff70 2024-04-25T08:42:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbe0c00640 1 -- 172.21.15.46:0/2187048139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0b88b0 con 0x7fdbe40cff70 2024-04-25T08:42:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe415cbe0 con 0x7fdbe40cff70 2024-04-25T08:42:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.539+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe4155010 con 0x7fdbe40cff70 2024-04-25T08:42:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.543+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe4069080 con 0x7fdbe40cff70 2024-04-25T08:42:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.543+0000 7fdbe0c00640 1 -- 172.21.15.46:0/2187048139 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdbdc0b8a50 con 0x7fdbe40cff70 2024-04-25T08:42:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.543+0000 7fdbe0c00640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbc0066f60 0x7fdbc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:21.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.543+0000 7fdbe0c00640 1 -- 172.21.15.46:0/2187048139 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbdc120210 con 0x7fdbe40cff70 2024-04-25T08:42:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.543+0000 7fdbe3e00640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbc0066f60 0x7fdbc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:21.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.543+0000 7fdbe3e00640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbc0066f60 0x7fdbc0069420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400b970 tx=0x7fdbd4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:21.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.547+0000 7fdbe0c00640 1 -- 172.21.15.46:0/2187048139 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdbdc0165e0 con 0x7fdbe40cff70 2024-04-25T08:42:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:21.715+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fdbe40aee90 con 0x7fdbe40cff70 2024-04-25T08:42:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.327+0000 7fdbe0c00640 1 -- 172.21.15.46:0/2187048139 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7fdbdc0f1340 con 0x7fdbe40cff70 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbc0066f60 msgr2=0x7fdbc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbc0066f60 0x7fdbc0069420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdbd400b970 tx=0x7fdbd4008040 comp rx=0 tx=0).stop 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cff70 msgr2=0x7fdbe40b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cff70 0x7fdbe40b1490 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc01d290 tx=0x7fdbdc019790 comp rx=0 tx=0).stop 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 shutdown_connections 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdbc0066f60 0x7fdbc0069420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbe40cff70 0x7fdbe40b1490 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe40cd250 0x7fdbe41f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 --2- 172.21.15.46:0/2187048139 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdbe40b2d50 0x7fdbe414e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 >> 172.21.15.46:0/2187048139 conn(0x7fdbe4099a00 msgr2=0x7fdbe40cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 shutdown_connections 2024-04-25T08:42:22.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.331+0000 7fdbeb200640 1 -- 172.21.15.46:0/2187048139 wait complete. 2024-04-25T08:42:22.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:42:22.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3e600640 1 Processor -- start 2024-04-25T08:42:22.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3e600640 1 -- start start 2024-04-25T08:42:22.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3e600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcb380cf0f0 0x7fcb380cf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:22.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb380cbf90 0x7fcb380cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:22.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3815f210 con 0x7fcb3815ee30 2024-04-25T08:42:22.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3e600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb380d0430 con 0x7fcb380cf0f0 2024-04-25T08:42:22.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3809aa10 con 0x7fcb380cbf90 2024-04-25T08:42:22.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3d200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcb3815ee30 0x7fcb380b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48778/0 (socket says 172.21.15.46:48778) 2024-04-25T08:42:22.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3d200640 1 -- 172.21.15.46:0/1303395561 learned_addr learned my addr 172.21.15.46:0/1303395561 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:22.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37e00640 1 --2- 172.21.15.46:0/1303395561 >> v2:172.21.15.82:3300/0 conn(0x7fcb380cf0f0 0x7fcb380cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:22.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3dc00640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb380cbf90 0x7fcb380cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:22.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083076949 0 0) 0x7fcb3815f210 con 0x7fcb3815ee30 2024-04-25T08:42:22.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb20003530 con 0x7fcb3815ee30 2024-04-25T08:42:22.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1188298311 0 0) 0x7fcb20003530 con 0x7fcb3815ee30 2024-04-25T08:42:22.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb380cbf90 msgr2=0x7fcb380cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:22.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb380cbf90 0x7fcb380cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 >> v2:172.21.15.82:3300/0 conn(0x7fcb380cf0f0 msgr2=0x7fcb380cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 --2- 172.21.15.46:0/1303395561 >> v2:172.21.15.82:3300/0 conn(0x7fcb380cf0f0 0x7fcb380cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380cfe70 con 0x7fcb3815ee30 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fcb2c001680 con 0x7fcb3815ee30 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 >> v1:172.21.15.46:6789/0 conn(0x7fcb3815ee30 legacy=0x7fcb380b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb20004810 0x7fcb20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb380cbf90 0x7fcb200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380d0430 con 0x7fcb200039c0 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3809aa10 con 0x7fcb20004810 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2000b9d0 con 0x7fcb380cbf90 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3d200640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3dc00640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb380cbf90 0x7fcb200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb37e00640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb20004810 0x7fcb20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3d200640 1 -- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb380cbf90 msgr2=0x7fcb200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3d200640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb380cbf90 0x7fcb200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3d200640 1 -- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb20004810 msgr2=0x7fcb20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3d200640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb20004810 0x7fcb20004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.511+0000 7fcb3d200640 1 -- 172.21.15.46:0/1303395561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb20003530 con 0x7fcb200039c0 2024-04-25T08:42:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb3d200640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb20003de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c001060 tx=0x7fcb2c01cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8505bf2e64802149 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c007cb0 con 0x7fcb200039c0 2024-04-25T08:42:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c0a9020 con 0x7fcb200039c0 2024-04-25T08:42:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb37400640 1 -- 172.21.15.46:0/1303395561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c005b20 con 0x7fcb200039c0 2024-04-25T08:42:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb3e600640 1 -- 172.21.15.46:0/1303395561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 msgr2=0x7fcb20003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb3e600640 1 --2- 172.21.15.46:0/1303395561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb20003de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c001060 tx=0x7fcb2c01cb50 comp rx=0 tx=0).stop 2024-04-25T08:42:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb3dc00640 1 -- 172.21.15.46:0/1303395561 reap_dead start 2024-04-25T08:42:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb3e600640 1 -- 172.21.15.46:0/1303395561 shutdown_connections 2024-04-25T08:42:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb3e600640 1 -- 172.21.15.46:0/1303395561 >> 172.21.15.46:0/1303395561 conn(0x7fcb380b3010 msgr2=0x7fcb380b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb3e600640 1 -- 172.21.15.46:0/1303395561 shutdown_connections 2024-04-25T08:42:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.515+0000 7fcb3e600640 1 -- 172.21.15.46:0/1303395561 wait complete. 2024-04-25T08:42:22.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 Processor -- start 2024-04-25T08:42:22.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 -- start start 2024-04-25T08:42:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb380c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb3806c120 0x7fcb3814b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb3806c5a0 0x7fcb3814ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380cfc20 con 0x7fcb200039c0 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380cfaa0 con 0x7fcb3806c5a0 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb380ccbb0 con 0x7fcb3806c120 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb380c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb380c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44186/0 (socket says 172.21.15.46:44186) 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3d200640 1 -- 172.21.15.46:0/2755971322 learned_addr learned my addr 172.21.15.46:0/2755971322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb37e00640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb3806c120 0x7fcb3814b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3dc00640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb3806c5a0 0x7fcb3814ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3d200640 1 -- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb3806c120 msgr2=0x7fcb3814b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3d200640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb3806c120 0x7fcb3814b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3d200640 1 -- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb3806c5a0 msgr2=0x7fcb3814ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3d200640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb3806c5a0 0x7fcb3814ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3d200640 1 -- 172.21.15.46:0/2755971322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380b4140 con 0x7fcb200039c0 2024-04-25T08:42:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb37e00640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb3806c120 0x7fcb3814b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3dc00640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb3806c5a0 0x7fcb3814ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3d200640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb380c7a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0ab040 tx=0x7fcb20005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb34c00640 1 -- 172.21.15.46:0/2755971322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c005d00 con 0x7fcb200039c0 2024-04-25T08:42:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb34c00640 1 -- 172.21.15.46:0/2755971322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c0047f0 con 0x7fcb200039c0 2024-04-25T08:42:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb34c00640 1 -- 172.21.15.46:0/2755971322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c004ba0 con 0x7fcb200039c0 2024-04-25T08:42:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb380aeb10 con 0x7fcb200039c0 2024-04-25T08:42:22.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.519+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb380af090 con 0x7fcb200039c0 2024-04-25T08:42:22.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.523+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb38069010 con 0x7fcb200039c0 2024-04-25T08:42:22.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.523+0000 7fcb34c00640 1 -- 172.21.15.46:0/2755971322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb2c0a57a0 con 0x7fcb200039c0 2024-04-25T08:42:22.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.523+0000 7fcb34c00640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb18066f90 0x7fcb18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:22.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.523+0000 7fcb34c00640 1 -- 172.21.15.46:0/2755971322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6007+0+0 (secure 0 0 0) 0x7fcb2c11e530 con 0x7fcb200039c0 2024-04-25T08:42:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.523+0000 7fcb37e00640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb18066f90 0x7fcb18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.523+0000 7fcb37e00640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb18066f90 0x7fcb18069450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb3806ce30 tx=0x7fcb24008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.527+0000 7fcb34c00640 1 -- 172.21.15.46:0/2755971322 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb380b4140 con 0x7fcb200039c0 2024-04-25T08:42:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:22.687+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcb380cd920 con 0x7fcb200039c0 2024-04-25T08:42:24.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.371+0000 7fcb34c00640 1 -- 172.21.15.46:0/2755971322 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7fcb2c0168f0 con 0x7fcb200039c0 2024-04-25T08:42:24.363 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' created 2024-04-25T08:42:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb18066f90 msgr2=0x7fcb18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb18066f90 0x7fcb18069450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb3806ce30 tx=0x7fcb24008040 comp rx=0 tx=0).stop 2024-04-25T08:42:24.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 msgr2=0x7fcb380c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb380c7a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0ab040 tx=0x7fcb20005840 comp rx=0 tx=0).stop 2024-04-25T08:42:24.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 shutdown_connections 2024-04-25T08:42:24.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb18066f90 0x7fcb18069450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb3806c5a0 0x7fcb3814ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcb3806c120 0x7fcb3814b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 --2- 172.21.15.46:0/2755971322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb200039c0 0x7fcb380c7a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 >> 172.21.15.46:0/2755971322 conn(0x7fcb380b3010 msgr2=0x7fcb3806f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:24.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 shutdown_connections 2024-04-25T08:42:24.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.375+0000 7fcb3e600640 1 -- 172.21.15.46:0/2755971322 wait complete. 2024-04-25T08:42:24.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T08:42:24.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.555+0000 7fdf3d000640 1 Processor -- start 2024-04-25T08:42:24.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.555+0000 7fdf3d000640 1 -- start start 2024-04-25T08:42:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf3d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf3806bc20 0x7fdf3806c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf3d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf380ae1a0 0x7fdf3806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf3d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3806c540 con 0x7fdf3815dce0 2024-04-25T08:42:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf3d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3806ee30 con 0x7fdf3806bc20 2024-04-25T08:42:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf3d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3806f110 con 0x7fdf380ae1a0 2024-04-25T08:42:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf35e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf3806bc20 0x7fdf3806c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf35400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdf3815dce0 0x7fdf3806db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48790/0 (socket says 172.21.15.46:48790) 2024-04-25T08:42:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf35400640 1 -- 172.21.15.46:0/3557971093 learned_addr learned my addr 172.21.15.46:0/3557971093 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf36800640 1 --2- 172.21.15.46:0/3557971093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf380ae1a0 0x7fdf3806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf35e00640 1 -- 172.21.15.46:0/3557971093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf380ae1a0 msgr2=0x7fdf3806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf35e00640 1 --2- 172.21.15.46:0/3557971093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf380ae1a0 0x7fdf3806e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf35e00640 1 -- 172.21.15.46:0/3557971093 >> v1:172.21.15.46:6789/0 conn(0x7fdf3815dce0 legacy=0x7fdf3806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf35e00640 1 -- 172.21.15.46:0/3557971093 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3806f3f0 con 0x7fdf3806bc20 2024-04-25T08:42:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf36800640 1 --2- 172.21.15.46:0/3557971093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf380ae1a0 0x7fdf3806e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:24.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf34a00640 1 -- 172.21.15.46:0/3557971093 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527807140 0 0) 0x7fdf3806c540 con 0x7fdf3815dce0 2024-04-25T08:42:24.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.559+0000 7fdf35e00640 1 --2- 172.21.15.46:0/3557971093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf3806bc20 0x7fdf3806c000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf280097c0 tx=0x7fdf2801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4039e0062e9ba14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:24.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf34a00640 1 -- 172.21.15.46:0/3557971093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf280ab430 con 0x7fdf3806bc20 2024-04-25T08:42:24.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf34a00640 1 -- 172.21.15.46:0/3557971093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf280b0070 con 0x7fdf3806bc20 2024-04-25T08:42:24.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf34a00640 1 -- 172.21.15.46:0/3557971093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf280ab7f0 con 0x7fdf3806bc20 2024-04-25T08:42:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 -- 172.21.15.46:0/3557971093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf3806bc20 msgr2=0x7fdf3806c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 --2- 172.21.15.46:0/3557971093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf3806bc20 0x7fdf3806c000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf280097c0 tx=0x7fdf2801c470 comp rx=0 tx=0).stop 2024-04-25T08:42:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 -- 172.21.15.46:0/3557971093 shutdown_connections 2024-04-25T08:42:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 --2- 172.21.15.46:0/3557971093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf380ae1a0 0x7fdf3806e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 --2- 172.21.15.46:0/3557971093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf3806bc20 0x7fdf3806c000 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 -- 172.21.15.46:0/3557971093 >> 172.21.15.46:0/3557971093 conn(0x7fdf38071ae0 msgr2=0x7fdf38071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 -- 172.21.15.46:0/3557971093 shutdown_connections 2024-04-25T08:42:24.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 -- 172.21.15.46:0/3557971093 wait complete. 2024-04-25T08:42:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 Processor -- start 2024-04-25T08:42:24.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 -- start start 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf3806bc20 0x7fdf38147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf380ae1a0 0x7fdf38160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf3815dce0 0x7fdf38160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3806eba0 con 0x7fdf380ae1a0 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380b3d20 con 0x7fdf3806bc20 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf3d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3816ae40 con 0x7fdf3815dce0 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf380ae1a0 0x7fdf38160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf380ae1a0 0x7fdf38160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44192/0 (socket says 172.21.15.46:44192) 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf35400640 1 -- 172.21.15.46:0/1616625049 learned_addr learned my addr 172.21.15.46:0/1616625049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf36800640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf3815dce0 0x7fdf38160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.563+0000 7fdf35e00640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf3806bc20 0x7fdf38147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf35400640 1 -- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf3815dce0 msgr2=0x7fdf38160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf35400640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf3815dce0 0x7fdf38160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf35400640 1 -- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf3806bc20 msgr2=0x7fdf38147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf35400640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf3806bc20 0x7fdf38147310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf35400640 1 -- 172.21.15.46:0/1616625049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38155750 con 0x7fdf380ae1a0 2024-04-25T08:42:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf36800640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf3815dce0 0x7fdf38160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf35e00640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf3806bc20 0x7fdf38147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:24.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf35400640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf380ae1a0 0x7fdf38160570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c480 tx=0x7fdf2400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf2e000640 1 -- 172.21.15.46:0/1616625049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24018070 con 0x7fdf380ae1a0 2024-04-25T08:42:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf2e000640 1 -- 172.21.15.46:0/1616625049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24007500 con 0x7fdf380ae1a0 2024-04-25T08:42:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf2e000640 1 -- 172.21.15.46:0/1616625049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24004a30 con 0x7fdf380ae1a0 2024-04-25T08:42:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf38155470 con 0x7fdf380ae1a0 2024-04-25T08:42:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3814c560 con 0x7fdf380ae1a0 2024-04-25T08:42:24.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf38067820 con 0x7fdf380ae1a0 2024-04-25T08:42:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf2e000640 1 -- 172.21.15.46:0/1616625049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf24004bd0 con 0x7fdf380ae1a0 2024-04-25T08:42:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf2e000640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf18066f60 0x7fdf18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.567+0000 7fdf2e000640 1 -- 172.21.15.46:0/1616625049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fdf24084640 con 0x7fdf380ae1a0 2024-04-25T08:42:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.571+0000 7fdf35e00640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf18066f60 0x7fdf18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.571+0000 7fdf35e00640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf18066f60 0x7fdf18069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf280af000 tx=0x7fdf280a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.575+0000 7fdf2e000640 1 -- 172.21.15.46:0/1616625049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf24055320 con 0x7fdf380ae1a0 2024-04-25T08:42:24.649 INFO:tasks.ceph.mds.b.smithi082.stderr:2024-04-25T08:42:24.656+0000 7f34ad400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:42:24.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:24.739+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fdf38069080 con 0x7fdf380ae1a0 2024-04-25T08:42:25.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.375+0000 7fdf2e000640 1 -- 172.21.15.46:0/1616625049 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdf24054c20 con 0x7fdf380ae1a0 2024-04-25T08:42:25.369 INFO:teuthology.orchestra.run.smithi046.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T08:42:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.379+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf18066f60 msgr2=0x7fdf18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.379+0000 7fdf3d000640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf18066f60 0x7fdf18069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdf280af000 tx=0x7fdf280a7000 comp rx=0 tx=0).stop 2024-04-25T08:42:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.379+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf380ae1a0 msgr2=0x7fdf38160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.379+0000 7fdf3d000640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf380ae1a0 0x7fdf38160570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c480 tx=0x7fdf2400c940 comp rx=0 tx=0).stop 2024-04-25T08:42:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.383+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 shutdown_connections 2024-04-25T08:42:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.383+0000 7fdf3d000640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf18066f60 0x7fdf18069420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.383+0000 7fdf3d000640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf3815dce0 0x7fdf38160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.383+0000 7fdf3d000640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf380ae1a0 0x7fdf38160570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.383+0000 7fdf3d000640 1 --2- 172.21.15.46:0/1616625049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf3806bc20 0x7fdf38147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.383+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 >> 172.21.15.46:0/1616625049 conn(0x7fdf38071ae0 msgr2=0x7fdf3815ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:25.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.383+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 shutdown_connections 2024-04-25T08:42:25.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.383+0000 7fdf3d000640 1 -- 172.21.15.46:0/1616625049 wait complete. 2024-04-25T08:42:25.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T08:42:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7ce4a00640 1 Processor -- start 2024-04-25T08:42:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7ce4a00640 1 -- start start 2024-04-25T08:42:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7ce4a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ce009a7c0 0x7f7ce006bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7ce4a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce00cf8a0 con 0x7f7ce009ac20 2024-04-25T08:42:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7ce4a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce00cfb80 con 0x7f7ce0099a00 2024-04-25T08:42:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7ce4a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce015efa0 con 0x7f7ce009a7c0 2024-04-25T08:42:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cde800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7ce009ac20 0x7f7ce006c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48806/0 (socket says 172.21.15.46:48806) 2024-04-25T08:42:25.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cde800640 1 -- 172.21.15.46:0/4134767580 learned_addr learned my addr 172.21.15.46:0/4134767580 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:25.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cdd400640 1 --2- 172.21.15.46:0/4134767580 >> v2:172.21.15.82:3300/0 conn(0x7f7ce009a7c0 0x7f7ce006bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cdca00640 1 -- 172.21.15.46:0/4134767580 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130125624 0 0) 0x7f7ce00cf8a0 con 0x7f7ce009ac20 2024-04-25T08:42:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cdca00640 1 -- 172.21.15.46:0/4134767580 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cc4003530 con 0x7f7ce009ac20 2024-04-25T08:42:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cdca00640 1 -- 172.21.15.46:0/4134767580 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720527061 0 0) 0x7f7ce00cfb80 con 0x7f7ce0099a00 2024-04-25T08:42:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cdca00640 1 -- 172.21.15.46:0/4134767580 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce00cf8a0 con 0x7f7ce0099a00 2024-04-25T08:42:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cdd400640 1 -- 172.21.15.46:0/4134767580 >> v1:172.21.15.82:6789/0 conn(0x7f7ce0099a00 legacy=0x7f7ce0099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cdd400640 1 -- 172.21.15.46:0/4134767580 >> v1:172.21.15.46:6789/0 conn(0x7f7ce009ac20 legacy=0x7f7ce006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cdca00640 1 -- 172.21.15.46:0/4134767580 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1955179962 0 0) 0x7f7cc4003530 con 0x7f7ce009ac20 2024-04-25T08:42:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.579+0000 7f7cdd400640 1 -- 172.21.15.46:0/4134767580 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce00cfe60 con 0x7f7ce009a7c0 2024-04-25T08:42:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7cdd400640 1 --2- 172.21.15.46:0/4134767580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009a7c0 0x7f7ce006bd20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0097c0 tx=0x7f7ccc01c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98d1ec4d989c5f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7cdca00640 1 -- 172.21.15.46:0/4134767580 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc0b0070 con 0x7f7ce009a7c0 2024-04-25T08:42:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7cdca00640 1 -- 172.21.15.46:0/4134767580 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc0ab400 con 0x7f7ce009a7c0 2024-04-25T08:42:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7cdca00640 1 -- 172.21.15.46:0/4134767580 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8001a00 con 0x7f7ce009a7c0 2024-04-25T08:42:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7ce4a00640 1 -- 172.21.15.46:0/4134767580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009a7c0 msgr2=0x7f7ce006bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7ce4a00640 1 --2- 172.21.15.46:0/4134767580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009a7c0 0x7f7ce006bd20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0097c0 tx=0x7f7ccc01c390 comp rx=0 tx=0).stop 2024-04-25T08:42:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7ce4a00640 1 -- 172.21.15.46:0/4134767580 shutdown_connections 2024-04-25T08:42:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7ce4a00640 1 --2- 172.21.15.46:0/4134767580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009a7c0 0x7f7ce006bd20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7ce4a00640 1 -- 172.21.15.46:0/4134767580 >> 172.21.15.46:0/4134767580 conn(0x7f7ce00b3130 msgr2=0x7f7ce00b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7ce4a00640 1 -- 172.21.15.46:0/4134767580 shutdown_connections 2024-04-25T08:42:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7ce4a00640 1 -- 172.21.15.46:0/4134767580 wait complete. 2024-04-25T08:42:25.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7ce4a00640 1 Processor -- start 2024-04-25T08:42:25.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.583+0000 7f7ce4a00640 1 -- start start 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7ce4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0099a00 0x7f7ce0070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7ce4a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce009a7c0 0x7f7ce00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7ce4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009ac20 0x7f7ce00b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7ce4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b0870 con 0x7f7ce0099a00 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7ce4a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce00b06f0 con 0x7f7ce009ac20 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7ce4a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce016c5d0 con 0x7f7ce009a7c0 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0099a00 0x7f7ce0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0099a00 0x7f7ce0070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44206/0 (socket says 172.21.15.46:44206) 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdde00640 1 -- 172.21.15.46:0/854253263 learned_addr learned my addr 172.21.15.46:0/854253263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cde800640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009ac20 0x7f7ce00b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdd400640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce009a7c0 0x7f7ce00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdde00640 1 -- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce009a7c0 msgr2=0x7f7ce00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdde00640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce009a7c0 0x7f7ce00709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdde00640 1 -- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009ac20 msgr2=0x7f7ce00b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdde00640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009ac20 0x7f7ce00b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdde00640 1 -- 172.21.15.46:0/854253263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce00b24a0 con 0x7f7ce0099a00 2024-04-25T08:42:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cde800640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009ac20 0x7f7ce00b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdd400640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce009a7c0 0x7f7ce00709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cdde00640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0099a00 0x7f7ce0070470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000c460 tx=0x7f7cd000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cd6000640 1 -- 172.21.15.46:0/854253263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0017070 con 0x7f7ce0099a00 2024-04-25T08:42:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cd6000640 1 -- 172.21.15.46:0/854253263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd00052f0 con 0x7f7ce0099a00 2024-04-25T08:42:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce0158d20 con 0x7f7ce0099a00 2024-04-25T08:42:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7cd6000640 1 -- 172.21.15.46:0/854253263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0003840 con 0x7f7ce0099a00 2024-04-25T08:42:25.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce0158960 con 0x7f7ce0099a00 2024-04-25T08:42:25.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.587+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce0069010 con 0x7f7ce0099a00 2024-04-25T08:42:25.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.591+0000 7f7cd6000640 1 -- 172.21.15.46:0/854253263 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cd001a070 con 0x7f7ce0099a00 2024-04-25T08:42:25.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.591+0000 7f7cd6000640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cc0066f60 0x7f7cc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.591+0000 7f7cd6000640 1 -- 172.21.15.46:0/854253263 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7cd0084790 con 0x7f7ce0099a00 2024-04-25T08:42:25.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.591+0000 7f7cdd400640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cc0066f60 0x7f7cc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.591+0000 7f7cdd400640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cc0066f60 0x7f7cc0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0ad000 tx=0x7f7ccc0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:25.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.595+0000 7f7cd6000640 1 -- 172.21.15.46:0/854253263 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7cd00553e0 con 0x7f7ce0099a00 2024-04-25T08:42:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:25.791+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f7ce015ca80 con 0x7f7ce0099a00 2024-04-25T08:42:26.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7cd6000640 1 -- 172.21.15.46:0/854253263 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f7cd0054ce0 con 0x7f7ce0099a00 2024-04-25T08:42:26.380 INFO:teuthology.orchestra.run.smithi046.stderr:added data pool 4 to fsmap 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cc0066f60 msgr2=0x7f7cc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cc0066f60 0x7f7cc0069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0ad000 tx=0x7f7ccc0a6000 comp rx=0 tx=0).stop 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0099a00 msgr2=0x7f7ce0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0099a00 0x7f7ce0070470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000c460 tx=0x7f7cd000c920 comp rx=0 tx=0).stop 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 shutdown_connections 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cc0066f60 0x7f7cc0069420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce009ac20 0x7f7ce00b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce009a7c0 0x7f7ce00709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 --2- 172.21.15.46:0/854253263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce0099a00 0x7f7ce0070470 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.387+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 >> 172.21.15.46:0/854253263 conn(0x7f7ce00b3130 msgr2=0x7f7ce00b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.391+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 shutdown_connections 2024-04-25T08:42:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.391+0000 7f7ce4a00640 1 -- 172.21.15.46:0/854253263 wait complete. 2024-04-25T08:42:26.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f7597400640 1 Processor -- start 2024-04-25T08:42:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f7597400640 1 -- start start 2024-04-25T08:42:26.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f7597400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f75900ae360 0x7f759015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f7597400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7590161330 con 0x7f759006ea00 2024-04-25T08:42:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f7597400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75900ae810 con 0x7f7590160f50 2024-04-25T08:42:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f7597400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759006d680 con 0x7f75900ae360 2024-04-25T08:42:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f7595400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f759006ea00 0x7f759006db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48816/0 (socket says 172.21.15.46:48816) 2024-04-25T08:42:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f7595400640 1 -- 172.21.15.46:0/1272026615 learned_addr learned my addr 172.21.15.46:0/1272026615 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758fe00640 1 --2- 172.21.15.46:0/1272026615 >> v2:172.21.15.82:3300/0 conn(0x7f75900ae360 0x7f759015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342582894 0 0) 0x7f7590161330 con 0x7f759006ea00 2024-04-25T08:42:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7578003530 con 0x7f759006ea00 2024-04-25T08:42:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478915419 0 0) 0x7f75900ae810 con 0x7f7590160f50 2024-04-25T08:42:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7590161330 con 0x7f7590160f50 2024-04-25T08:42:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2171046498 0 0) 0x7f7578003530 con 0x7f759006ea00 2024-04-25T08:42:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 >> v2:172.21.15.82:3300/0 conn(0x7f75900ae360 msgr2=0x7f759015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 --2- 172.21.15.46:0/1272026615 >> v2:172.21.15.82:3300/0 conn(0x7f75900ae360 0x7f759015f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 >> v1:172.21.15.82:6789/0 conn(0x7f7590160f50 legacy=0x7f75900adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75900aeaf0 con 0x7f759006ea00 2024-04-25T08:42:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f7588001a00 con 0x7f759006ea00 2024-04-25T08:42:26.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758fe00640 1 --2- 172.21.15.46:0/1272026615 >> v2:172.21.15.82:3300/0 conn(0x7f75900ae360 0x7f759015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 >> v1:172.21.15.46:6789/0 conn(0x7f759006ea00 legacy=0x7f759006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7578003ba0 0x7f7578003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7578004810 0x7f7578004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75780051d0 0x7f757800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759006d680 con 0x7f7578003ba0 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757800c0a0 con 0x7f7578004810 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757800c2a0 con 0x7f75780051d0 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.559+0000 7f7594a00640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7578003ba0 0x7f7578003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f758fe00640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7578004810 0x7f7578004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7595400640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75780051d0 0x7f757800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7594a00640 1 -- 172.21.15.46:0/1272026615 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75780051d0 msgr2=0x7f757800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7594a00640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75780051d0 0x7f757800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7594a00640 1 -- 172.21.15.46:0/1272026615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7578004810 msgr2=0x7f7578004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7594a00640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7578004810 0x7f7578004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7594a00640 1 -- 172.21.15.46:0/1272026615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7578003530 con 0x7f7578003ba0 2024-04-25T08:42:26.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7595400640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f75780051d0 0x7f757800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:26.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7595400640 1 -- 172.21.15.46:0/1272026615 reap_dead start 2024-04-25T08:42:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7594a00640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7578003ba0 0x7f7578003fa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f75840018d0 tx=0x7f758401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f66d34331f2fec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584001520 con 0x7f7578003ba0 2024-04-25T08:42:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75840052f0 con 0x7f7578003ba0 2024-04-25T08:42:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f758f400640 1 -- 172.21.15.46:0/1272026615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584003840 con 0x7f7578003ba0 2024-04-25T08:42:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7597400640 1 -- 172.21.15.46:0/1272026615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7578003ba0 msgr2=0x7f7578003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.563+0000 7f7597400640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7578003ba0 0x7f7578003fa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f75840018d0 tx=0x7f758401c530 comp rx=0 tx=0).stop 2024-04-25T08:42:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 -- 172.21.15.46:0/1272026615 shutdown_connections 2024-04-25T08:42:26.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 --2- 172.21.15.46:0/1272026615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7578003ba0 0x7f7578003fa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 -- 172.21.15.46:0/1272026615 >> 172.21.15.46:0/1272026615 conn(0x7f75900b12a0 msgr2=0x7f75900b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 -- 172.21.15.46:0/1272026615 shutdown_connections 2024-04-25T08:42:26.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 -- 172.21.15.46:0/1272026615 wait complete. 2024-04-25T08:42:26.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 Processor -- start 2024-04-25T08:42:26.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 -- start start 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7578003ba0 0x7f759015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7590159da0 0x7f75901e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75901e8830 0x7f7590135f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75900aedd0 con 0x7f75901e8830 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759015f5b0 con 0x7f7590159da0 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7597400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759006e300 con 0x7f7578003ba0 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7595400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75901e8830 0x7f7590135f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7594a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7578003ba0 0x7f759015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7595400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75901e8830 0x7f7590135f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44230/0 (socket says 172.21.15.46:44230) 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7595400640 1 -- 172.21.15.46:0/1480170235 learned_addr learned my addr 172.21.15.46:0/1480170235 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f758fe00640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7590159da0 0x7f75901e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7595400640 1 -- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7578003ba0 msgr2=0x7f759015a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7595400640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7578003ba0 0x7f759015a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7595400640 1 -- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7590159da0 msgr2=0x7f75901e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7595400640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7590159da0 0x7f75901e82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.567+0000 7f7595400640 1 -- 172.21.15.46:0/1480170235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7590134090 con 0x7f75901e8830 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f7595400640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75901e8830 0x7f7590135f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f75900cba60 tx=0x7f7588007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f758cc00640 1 -- 172.21.15.46:0/1480170235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758800ea20 con 0x7f75901e8830 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f758cc00640 1 -- 172.21.15.46:0/1480170235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758800ec00 con 0x7f75901e8830 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f758cc00640 1 -- 172.21.15.46:0/1480170235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588014430 con 0x7f75901e8830 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7590134370 con 0x7f75901e8830 2024-04-25T08:42:26.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7590134930 con 0x7f75901e8830 2024-04-25T08:42:26.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7590069080 con 0x7f75901e8830 2024-04-25T08:42:26.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f758cc00640 1 -- 172.21.15.46:0/1480170235 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75880145d0 con 0x7f75901e8830 2024-04-25T08:42:26.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f758cc00640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7570066f60 0x7f7570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f758cc00640 1 -- 172.21.15.46:0/1480170235 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7588085070 con 0x7f75901e8830 2024-04-25T08:42:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.571+0000 7f7594a00640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7570066f60 0x7f7570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.575+0000 7f7594a00640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7570066f60 0x7f7570069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f75840ac000 tx=0x7f75840a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.575+0000 7f758cc00640 1 -- 172.21.15.46:0/1480170235 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75880556e0 con 0x7f75901e8830 2024-04-25T08:42:26.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.771+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7590073ca0 con 0x7f75901e8830 2024-04-25T08:42:26.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.771+0000 7f758cc00640 1 -- 172.21.15.46:0/1480170235 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f7588059780 con 0x7f75901e8830 2024-04-25T08:42:26.766 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:26.766 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:25.795379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:42:26.766 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 7 2024-04-25T08:42:26.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7570066f60 msgr2=0x7f7570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7570066f60 0x7f7570069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f75840ac000 tx=0x7f75840a6000 comp rx=0 tx=0).stop 2024-04-25T08:42:26.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75901e8830 msgr2=0x7f7590135f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75901e8830 0x7f7590135f80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f75900cba60 tx=0x7f7588007bc0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 shutdown_connections 2024-04-25T08:42:26.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7570066f60 0x7f7570069420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75901e8830 0x7f7590135f80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7590159da0 0x7f75901e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 --2- 172.21.15.46:0/1480170235 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7578003ba0 0x7f759015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 >> 172.21.15.46:0/1480170235 conn(0x7f75900b12a0 msgr2=0x7f75900b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 shutdown_connections 2024-04-25T08:42:26.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.775+0000 7f7597400640 1 -- 172.21.15.46:0/1480170235 wait complete. 2024-04-25T08:42:26.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9368000640 1 Processor -- start 2024-04-25T08:42:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9368000640 1 -- start start 2024-04-25T08:42:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9368000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f936015ef20 0x7f93600cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9368000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93600cd6d0 0x7f936009a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9368000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f936009ab50 0x7f9360099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9368000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936015f300 con 0x7f936009ab50 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9368000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93600cddb0 con 0x7f936015ef20 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9368000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600cc3d0 con 0x7f93600cd6d0 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9366000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f936009ab50 0x7f9360099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9366000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f936009ab50 0x7f9360099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44238/0 (socket says 172.21.15.46:44238) 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9366000640 1 -- 172.21.15.46:0/1810715422 learned_addr learned my addr 172.21.15.46:0/1810715422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9364c00640 1 --2- 172.21.15.46:0/1810715422 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93600cd6d0 0x7f936009a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9365600640 1 --2- 172.21.15.46:0/1810715422 >> v2:172.21.15.82:3300/0 conn(0x7f936015ef20 0x7f93600cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9366000640 1 -- 172.21.15.46:0/1810715422 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93600cd6d0 msgr2=0x7f936009a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9366000640 1 --2- 172.21.15.46:0/1810715422 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93600cd6d0 0x7f936009a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9366000640 1 -- 172.21.15.46:0/1810715422 >> v2:172.21.15.82:3300/0 conn(0x7f936015ef20 msgr2=0x7f93600cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9366000640 1 --2- 172.21.15.46:0/1810715422 >> v2:172.21.15.82:3300/0 conn(0x7f936015ef20 0x7f93600cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9366000640 1 -- 172.21.15.46:0/1810715422 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600cc6b0 con 0x7f936009ab50 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.935+0000 7f9365600640 1 --2- 172.21.15.46:0/1810715422 >> v2:172.21.15.82:3300/0 conn(0x7f936015ef20 0x7f93600cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9366000640 1 --2- 172.21.15.46:0/1810715422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 0x7f9360099a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f936009a5e0 tx=0x7f935801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3e4a60681249b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f935fe00640 1 -- 172.21.15.46:0/1810715422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580a5750 con 0x7f936009ab50 2024-04-25T08:42:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f935fe00640 1 -- 172.21.15.46:0/1810715422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93580a58f0 con 0x7f936009ab50 2024-04-25T08:42:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f935fe00640 1 -- 172.21.15.46:0/1810715422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580a5cc0 con 0x7f936009ab50 2024-04-25T08:42:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9368000640 1 -- 172.21.15.46:0/1810715422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 msgr2=0x7f9360099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9368000640 1 --2- 172.21.15.46:0/1810715422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 0x7f9360099a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f936009a5e0 tx=0x7f935801c9c0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9368000640 1 -- 172.21.15.46:0/1810715422 shutdown_connections 2024-04-25T08:42:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9368000640 1 --2- 172.21.15.46:0/1810715422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 0x7f9360099a00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9368000640 1 --2- 172.21.15.46:0/1810715422 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93600cd6d0 0x7f936009a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9368000640 1 --2- 172.21.15.46:0/1810715422 >> v2:172.21.15.82:3300/0 conn(0x7f936015ef20 0x7f93600cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9368000640 1 -- 172.21.15.46:0/1810715422 >> 172.21.15.46:0/1810715422 conn(0x7f93600b3120 msgr2=0x7f93600b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9368000640 1 -- 172.21.15.46:0/1810715422 shutdown_connections 2024-04-25T08:42:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.939+0000 7f9368000640 1 -- 172.21.15.46:0/1810715422 wait complete. 2024-04-25T08:42:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 Processor -- start 2024-04-25T08:42:26.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 -- start start 2024-04-25T08:42:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 0x7f936015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f936015ef20 0x7f93600caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93600cafa0 0x7f936015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936015f6c0 con 0x7f936009ab50 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936015f540 con 0x7f93600cafa0 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936015f840 con 0x7f936015ef20 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9365600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 0x7f936015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9365600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 0x7f936015a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44248/0 (socket says 172.21.15.46:44248) 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9365600640 1 -- 172.21.15.46:0/3338452970 learned_addr learned my addr 172.21.15.46:0/3338452970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9364c00640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f936015ef20 0x7f93600caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9366000640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93600cafa0 0x7f936015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9365600640 1 -- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f936015ef20 msgr2=0x7f93600caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9365600640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f936015ef20 0x7f93600caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9365600640 1 -- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93600cafa0 msgr2=0x7f936015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9365600640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93600cafa0 0x7f936015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9365600640 1 -- 172.21.15.46:0/3338452970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936015a560 con 0x7f936009ab50 2024-04-25T08:42:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9365600640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 0x7f936015a020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f935400d800 tx=0x7f935400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f935d600640 1 -- 172.21.15.46:0/3338452970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354018020 con 0x7f936009ab50 2024-04-25T08:42:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f935d600640 1 -- 172.21.15.46:0/3338452970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93540096c0 con 0x7f936009ab50 2024-04-25T08:42:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f935d600640 1 -- 172.21.15.46:0/3338452970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354009ab0 con 0x7f936009ab50 2024-04-25T08:42:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93601f73f0 con 0x7f936009ab50 2024-04-25T08:42:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93601f79b0 con 0x7f936009ab50 2024-04-25T08:42:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.943+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93600cb3b0 con 0x7f936009ab50 2024-04-25T08:42:26.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.947+0000 7f935d600640 1 -- 172.21.15.46:0/3338452970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f935401b020 con 0x7f936009ab50 2024-04-25T08:42:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.947+0000 7f935d600640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9340066f60 0x7f9340069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.947+0000 7f935d600640 1 -- 172.21.15.46:0/3338452970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f93540857c0 con 0x7f936009ab50 2024-04-25T08:42:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.947+0000 7f9364c00640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9340066f60 0x7f9340069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.947+0000 7f9364c00640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9340066f60 0x7f9340069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f93600cb6e0 tx=0x7f934c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:26.951+0000 7f935d600640 1 -- 172.21.15.46:0/3338452970 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93600cb3b0 con 0x7f936009ab50 2024-04-25T08:42:27.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.115+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93600d0700 con 0x7f936009ab50 2024-04-25T08:42:27.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f935d600640 1 -- 172.21.15.46:0/3338452970 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18479 (secure 0 0 0) 0x7f9354022020 con 0x7f936009ab50 2024-04-25T08:42:27.111 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:27.112 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:25.795177+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9340066f60 msgr2=0x7f9340069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9340066f60 0x7f9340069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f93600cb6e0 tx=0x7f934c009190 comp rx=0 tx=0).stop 2024-04-25T08:42:27.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 msgr2=0x7f936015a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 0x7f936015a020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f935400d800 tx=0x7f935400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 shutdown_connections 2024-04-25T08:42:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9340066f60 0x7f9340069420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93600cafa0 0x7f936015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f936015ef20 0x7f93600caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 --2- 172.21.15.46:0/3338452970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f936009ab50 0x7f936015a020 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 >> 172.21.15.46:0/3338452970 conn(0x7f93600b3120 msgr2=0x7f936015a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 shutdown_connections 2024-04-25T08:42:27.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.119+0000 7f9368000640 1 -- 172.21.15.46:0/3338452970 wait complete. 2024-04-25T08:42:27.132 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.279+0000 7f5aeea00640 1 Processor -- start 2024-04-25T08:42:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.279+0000 7f5aeea00640 1 -- start start 2024-04-25T08:42:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aeea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ae80b29c0 0x7f5ae815f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aeea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae806c2b0 0x7f5ae8161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aeea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae80b2da0 con 0x7f5ae8099a00 2024-04-25T08:42:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aeea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae81623a0 con 0x7f5ae80b29c0 2024-04-25T08:42:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aeea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8161de0 con 0x7f5ae806c2b0 2024-04-25T08:42:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aed600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ae80b29c0 0x7f5ae815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aed600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ae80b29c0 0x7f5ae815f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44256/0 (socket says 172.21.15.46:44256) 2024-04-25T08:42:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aed600640 1 -- 172.21.15.46:0/301165900 learned_addr learned my addr 172.21.15.46:0/301165900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aecc00640 1 --2- 172.21.15.46:0/301165900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae806c2b0 0x7f5ae8161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aed600640 1 -- 172.21.15.46:0/301165900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae806c2b0 msgr2=0x7f5ae8161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aed600640 1 --2- 172.21.15.46:0/301165900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae806c2b0 0x7f5ae8161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aed600640 1 -- 172.21.15.46:0/301165900 >> v1:172.21.15.46:6789/0 conn(0x7f5ae8099a00 legacy=0x7f5ae80d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5aed600640 1 -- 172.21.15.46:0/301165900 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae81620c0 con 0x7f5ae80b29c0 2024-04-25T08:42:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.283+0000 7f5ae7e00640 1 -- 172.21.15.46:0/301165900 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2207989081 0 0) 0x7f5ae80b2da0 con 0x7f5ae8099a00 2024-04-25T08:42:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aed600640 1 --2- 172.21.15.46:0/301165900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80b29c0 0x7f5ae815f250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009870 tx=0x7f5adc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50053327caeb18b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5ae7e00640 1 -- 172.21.15.46:0/301165900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0a95f0 con 0x7f5ae80b29c0 2024-04-25T08:42:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5ae7e00640 1 -- 172.21.15.46:0/301165900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc0a9790 con 0x7f5ae80b29c0 2024-04-25T08:42:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5ae7e00640 1 -- 172.21.15.46:0/301165900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0a9b60 con 0x7f5ae80b29c0 2024-04-25T08:42:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 -- 172.21.15.46:0/301165900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80b29c0 msgr2=0x7f5ae815f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 --2- 172.21.15.46:0/301165900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80b29c0 0x7f5ae815f250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009870 tx=0x7f5adc01c530 comp rx=0 tx=0).stop 2024-04-25T08:42:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 -- 172.21.15.46:0/301165900 shutdown_connections 2024-04-25T08:42:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 --2- 172.21.15.46:0/301165900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae806c2b0 0x7f5ae8161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 --2- 172.21.15.46:0/301165900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae80b29c0 0x7f5ae815f250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 -- 172.21.15.46:0/301165900 >> 172.21.15.46:0/301165900 conn(0x7f5ae80b2f90 msgr2=0x7f5ae80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 -- 172.21.15.46:0/301165900 shutdown_connections 2024-04-25T08:42:27.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 -- 172.21.15.46:0/301165900 wait complete. 2024-04-25T08:42:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 Processor -- start 2024-04-25T08:42:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 -- start start 2024-04-25T08:42:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae806c2b0 0x7f5ae80b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae8099a00 0x7f5ae81f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae80b29c0 0x7f5ae80b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80d0e50 con 0x7f5ae806c2b0 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80d0cd0 con 0x7f5ae8099a00 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aeea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81619b0 con 0x7f5ae80b29c0 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae806c2b0 0x7f5ae80b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.287+0000 7f5aed600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae806c2b0 0x7f5ae80b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44258/0 (socket says 172.21.15.46:44258) 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aed600640 1 -- 172.21.15.46:0/2465422549 learned_addr learned my addr 172.21.15.46:0/2465422549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aecc00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae8099a00 0x7f5ae81f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aee000640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae80b29c0 0x7f5ae80b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aed600640 1 -- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae80b29c0 msgr2=0x7f5ae80b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aed600640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae80b29c0 0x7f5ae80b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aed600640 1 -- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae8099a00 msgr2=0x7f5ae81f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aed600640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae8099a00 0x7f5ae81f9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aed600640 1 -- 172.21.15.46:0/2465422549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae80b7030 con 0x7f5ae806c2b0 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aecc00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae8099a00 0x7f5ae81f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aee000640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae80b29c0 0x7f5ae80b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aed600640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae806c2b0 0x7f5ae80b4ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0ab040 tx=0x7f5adc01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5ae5600640 1 -- 172.21.15.46:0/2465422549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0a95f0 con 0x7f5ae806c2b0 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5ae5600640 1 -- 172.21.15.46:0/2465422549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc0b1820 con 0x7f5ae806c2b0 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5ae5600640 1 -- 172.21.15.46:0/2465422549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0b1c10 con 0x7f5ae806c2b0 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae8151960 con 0x7f5ae806c2b0 2024-04-25T08:42:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae80b5440 con 0x7f5ae806c2b0 2024-04-25T08:42:27.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae8069010 con 0x7f5ae806c2b0 2024-04-25T08:42:27.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.291+0000 7f5ae5600640 1 -- 172.21.15.46:0/2465422549 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5adc0a8350 con 0x7f5ae806c2b0 2024-04-25T08:42:27.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.295+0000 7f5ae5600640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ac8066f60 0x7f5ac8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.295+0000 7f5ae5600640 1 -- 172.21.15.46:0/2465422549 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5adc11f7d0 con 0x7f5ae806c2b0 2024-04-25T08:42:27.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.295+0000 7f5aecc00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ac8066f60 0x7f5ac8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.295+0000 7f5aecc00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ac8066f60 0x7f5ac8069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80b5fd0 tx=0x7f5ad4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.299+0000 7f5ae5600640 1 -- 172.21.15.46:0/2465422549 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5adc0c3070 con 0x7f5ae806c2b0 2024-04-25T08:42:27.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.479+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ae809a530 con 0x7f5ae806c2b0 2024-04-25T08:42:27.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.479+0000 7f5ae5600640 1 -- 172.21.15.46:0/2465422549 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f5ae8151960 con 0x7f5ae806c2b0 2024-04-25T08:42:27.473 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:27.473 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ac8066f60 msgr2=0x7f5ac8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ac8066f60 0x7f5ac8069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80b5fd0 tx=0x7f5ad4008040 comp rx=0 tx=0).stop 2024-04-25T08:42:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae806c2b0 msgr2=0x7f5ae80b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae806c2b0 0x7f5ae80b4ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0ab040 tx=0x7f5adc01ad70 comp rx=0 tx=0).stop 2024-04-25T08:42:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 shutdown_connections 2024-04-25T08:42:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ac8066f60 0x7f5ac8069420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ae80b29c0 0x7f5ae80b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae8099a00 0x7f5ae81f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 --2- 172.21.15.46:0/2465422549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ae806c2b0 0x7f5ae80b4ad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 >> 172.21.15.46:0/2465422549 conn(0x7f5ae80b2f90 msgr2=0x7f5ae80c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 shutdown_connections 2024-04-25T08:42:27.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.483+0000 7f5aeea00640 1 -- 172.21.15.46:0/2465422549 wait complete. 2024-04-25T08:42:27.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.655+0000 7f8ced600640 1 Processor -- start 2024-04-25T08:42:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.655+0000 7f8ced600640 1 -- start start 2024-04-25T08:42:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ced600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ce815a750 0x7f8ce806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ced600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80aed10 0x7f8ce806f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ced600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce806d5e0 con 0x7f8ce80cc9e0 2024-04-25T08:42:27.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ced600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ce815ab30 con 0x7f8ce815a750 2024-04-25T08:42:27.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ced600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce80ce4f0 con 0x7f8ce80aed10 2024-04-25T08:42:27.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ce6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ce815a750 0x7f8ce806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ce7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8ce80cc9e0 0x7f8ce80b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48838/0 (socket says 172.21.15.46:48838) 2024-04-25T08:42:27.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ce7400640 1 -- 172.21.15.46:0/1841174877 learned_addr learned my addr 172.21.15.46:0/1841174877 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8cde000640 1 --2- 172.21.15.46:0/1841174877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80aed10 0x7f8ce806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1841174877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80aed10 msgr2=0x7f8ce806f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/1841174877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80aed10 0x7f8ce806f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1841174877 >> v1:172.21.15.46:6789/0 conn(0x7f8ce80cc9e0 legacy=0x7f8ce80b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ce6a00640 1 -- 172.21.15.46:0/1841174877 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce80ce720 con 0x7f8ce815a750 2024-04-25T08:42:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8cde000640 1 --2- 172.21.15.46:0/1841174877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80aed10 0x7f8ce806f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.659+0000 7f8ce6000640 1 -- 172.21.15.46:0/1841174877 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831365478 0 0) 0x7f8ce806d5e0 con 0x7f8ce80cc9e0 2024-04-25T08:42:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/1841174877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 0x7f8ce806ec10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800ba60 tx=0x7f8cd801eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6059caafcce98ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ce6000640 1 -- 172.21.15.46:0/1841174877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80b4070 con 0x7f8ce815a750 2024-04-25T08:42:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ce6000640 1 -- 172.21.15.46:0/1841174877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd80a7b30 con 0x7f8ce815a750 2024-04-25T08:42:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ce6000640 1 -- 172.21.15.46:0/1841174877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80af440 con 0x7f8ce815a750 2024-04-25T08:42:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 -- 172.21.15.46:0/1841174877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 msgr2=0x7f8ce806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 --2- 172.21.15.46:0/1841174877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 0x7f8ce806ec10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800ba60 tx=0x7f8cd801eb70 comp rx=0 tx=0).stop 2024-04-25T08:42:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 -- 172.21.15.46:0/1841174877 shutdown_connections 2024-04-25T08:42:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 --2- 172.21.15.46:0/1841174877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80aed10 0x7f8ce806f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 --2- 172.21.15.46:0/1841174877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 0x7f8ce806ec10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 -- 172.21.15.46:0/1841174877 >> 172.21.15.46:0/1841174877 conn(0x7f8ce80b12a0 msgr2=0x7f8ce80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 -- 172.21.15.46:0/1841174877 shutdown_connections 2024-04-25T08:42:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 -- 172.21.15.46:0/1841174877 wait complete. 2024-04-25T08:42:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 Processor -- start 2024-04-25T08:42:27.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 -- start start 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ce80aed10 0x7f8ce813d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80cc9e0 0x7f8ce8153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 0x7f8ce8151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce806db00 con 0x7f8ce815a750 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce806d270 con 0x7f8ce80aed10 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.663+0000 7f8ced600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ce806dc80 con 0x7f8ce80cc9e0 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 0x7f8ce8151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 0x7f8ce8151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44276/0 (socket says 172.21.15.46:44276) 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce7400640 1 -- 172.21.15.46:0/2959510506 learned_addr learned my addr 172.21.15.46:0/2959510506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8cde000640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80cc9e0 0x7f8ce8153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ce80aed10 0x7f8ce813d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce7400640 1 -- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80cc9e0 msgr2=0x7f8ce8153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce7400640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80cc9e0 0x7f8ce8153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce7400640 1 -- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ce80aed10 msgr2=0x7f8ce813d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce7400640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ce80aed10 0x7f8ce813d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce7400640 1 -- 172.21.15.46:0/2959510506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce81550a0 con 0x7f8ce815a750 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ce80aed10 0x7f8ce813d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ce7400640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 0x7f8ce8151fe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c7a0 tx=0x7f8ce000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8cdf400640 1 -- 172.21.15.46:0/2959510506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0018070 con 0x7f8ce815a750 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8cdf400640 1 -- 172.21.15.46:0/2959510506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce000a640 con 0x7f8ce815a750 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce8155380 con 0x7f8ce815a750 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8cdf400640 1 -- 172.21.15.46:0/2959510506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000aab0 con 0x7f8ce815a750 2024-04-25T08:42:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ce81539a0 con 0x7f8ce815a750 2024-04-25T08:42:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ce8155550 con 0x7f8ce815a750 2024-04-25T08:42:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.667+0000 7f8cdf400640 1 -- 172.21.15.46:0/2959510506 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ce001b070 con 0x7f8ce815a750 2024-04-25T08:42:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.671+0000 7f8cdf400640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cc4066f60 0x7f8cc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.671+0000 7f8cdf400640 1 -- 172.21.15.46:0/2959510506 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8ce00849b0 con 0x7f8ce815a750 2024-04-25T08:42:27.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.671+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cc4066f60 0x7f8cc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.671+0000 7f8ce6a00640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cc4066f60 0x7f8cc4069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80b1000 tx=0x7f8cd80a9070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.675+0000 7f8cdf400640 1 -- 172.21.15.46:0/2959510506 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ce0054f90 con 0x7f8ce815a750 2024-04-25T08:42:27.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.867+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ce8069080 con 0x7f8ce815a750 2024-04-25T08:42:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8cdf400640 1 -- 172.21.15.46:0/2959510506 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f8ce0059030 con 0x7f8ce815a750 2024-04-25T08:42:27.863 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:27.863 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:25.795379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:42:27.863 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 7 2024-04-25T08:42:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cc4066f60 msgr2=0x7f8cc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cc4066f60 0x7f8cc4069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80b1000 tx=0x7f8cd80a9070 comp rx=0 tx=0).stop 2024-04-25T08:42:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 msgr2=0x7f8ce8151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 0x7f8ce8151fe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000c7a0 tx=0x7f8ce000cc60 comp rx=0 tx=0).stop 2024-04-25T08:42:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 shutdown_connections 2024-04-25T08:42:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cc4066f60 0x7f8cc4069420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ce815a750 0x7f8ce8151fe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ce80cc9e0 0x7f8ce8153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 --2- 172.21.15.46:0/2959510506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ce80aed10 0x7f8ce813d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 >> 172.21.15.46:0/2959510506 conn(0x7f8ce80b12a0 msgr2=0x7f8ce80ad5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 shutdown_connections 2024-04-25T08:42:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:27.871+0000 7f8ced600640 1 -- 172.21.15.46:0/2959510506 wait complete. 2024-04-25T08:42:27.883 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.031+0000 7f6eca400640 1 Processor -- start 2024-04-25T08:42:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.031+0000 7f6eca400640 1 -- start start 2024-04-25T08:42:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.031+0000 7f6eca400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ec406c190 0x7f6ec4070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.031+0000 7f6eca400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec40cf5b0 con 0x7f6ec4099a00 2024-04-25T08:42:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.031+0000 7f6eca400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec406fa90 con 0x7f6ec406bd20 2024-04-25T08:42:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.031+0000 7f6eca400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec406d060 con 0x7f6ec406c190 2024-04-25T08:42:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6ec4099a00 0x7f6ec4099de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48846/0 (socket says 172.21.15.46:48846) 2024-04-25T08:42:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec3400640 1 -- 172.21.15.46:0/2877944227 learned_addr learned my addr 172.21.15.46:0/2877944227 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:28.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec3e00640 1 --2- 172.21.15.46:0/2877944227 >> v2:172.21.15.46:3300/0 conn(0x7f6ec406c190 0x7f6ec4070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec2a00640 1 -- 172.21.15.46:0/2877944227 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704952766 0 0) 0x7f6ec40cf5b0 con 0x7f6ec4099a00 2024-04-25T08:42:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec2a00640 1 -- 172.21.15.46:0/2877944227 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea8003530 con 0x7f6ec4099a00 2024-04-25T08:42:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec3e00640 1 -- 172.21.15.46:0/2877944227 >> v1:172.21.15.82:6789/0 conn(0x7f6ec406bd20 legacy=0x7f6ec4070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec3e00640 1 -- 172.21.15.46:0/2877944227 >> v1:172.21.15.46:6789/0 conn(0x7f6ec4099a00 legacy=0x7f6ec4099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec3e00640 1 -- 172.21.15.46:0/2877944227 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec406c7c0 con 0x7f6ec406c190 2024-04-25T08:42:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec3e00640 1 --2- 172.21.15.46:0/2877944227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 0x7f6ec4070980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009d90 tx=0x7f6ebc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b04d8dfea96d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec2a00640 1 -- 172.21.15.46:0/2877944227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0b2070 con 0x7f6ec406c190 2024-04-25T08:42:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec2a00640 1 -- 172.21.15.46:0/2877944227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc0a5a00 con 0x7f6ec406c190 2024-04-25T08:42:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.035+0000 7f6ec2a00640 1 -- 172.21.15.46:0/2877944227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0a5dd0 con 0x7f6ec406c190 2024-04-25T08:42:28.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 -- 172.21.15.46:0/2877944227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 msgr2=0x7f6ec4070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 --2- 172.21.15.46:0/2877944227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 0x7f6ec4070980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009d90 tx=0x7f6ebc01c940 comp rx=0 tx=0).stop 2024-04-25T08:42:28.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 -- 172.21.15.46:0/2877944227 shutdown_connections 2024-04-25T08:42:28.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 --2- 172.21.15.46:0/2877944227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 0x7f6ec4070980 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 -- 172.21.15.46:0/2877944227 >> 172.21.15.46:0/2877944227 conn(0x7f6ec40d16b0 msgr2=0x7f6ec40d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 -- 172.21.15.46:0/2877944227 shutdown_connections 2024-04-25T08:42:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 -- 172.21.15.46:0/2877944227 wait complete. 2024-04-25T08:42:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 Processor -- start 2024-04-25T08:42:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 -- start start 2024-04-25T08:42:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ec406bd20 0x7f6ec4161870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 0x7f6ec40c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4099a00 0x7f6ec40b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec409a750 con 0x7f6ec406c190 2024-04-25T08:42:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40712d0 con 0x7f6ec4099a00 2024-04-25T08:42:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6eca400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec415ef30 con 0x7f6ec406bd20 2024-04-25T08:42:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ebaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 0x7f6ec40c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ebaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 0x7f6ec40c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44288/0 (socket says 172.21.15.46:44288) 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ebaa00640 1 -- 172.21.15.46:0/1267041777 learned_addr learned my addr 172.21.15.46:0/1267041777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ec3400640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ec406bd20 0x7f6ec4161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ec3e00640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4099a00 0x7f6ec40b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ebaa00640 1 -- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ec406bd20 msgr2=0x7f6ec4161870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ebaa00640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ec406bd20 0x7f6ec4161870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ebaa00640 1 -- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4099a00 msgr2=0x7f6ec40b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ebaa00640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4099a00 0x7f6ec40b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ebaa00640 1 -- 172.21.15.46:0/1267041777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec4161f10 con 0x7f6ec406c190 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.039+0000 7f6ec3e00640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4099a00 0x7f6ec40b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.043+0000 7f6ebaa00640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 0x7f6ec40c66b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00c420 tx=0x7f6eac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.043+0000 7f6ebbe00640 1 -- 172.21.15.46:0/1267041777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0052d0 con 0x7f6ec406c190 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.043+0000 7f6ebbe00640 1 -- 172.21.15.46:0/1267041777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac005470 con 0x7f6ec406c190 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.043+0000 7f6ebbe00640 1 -- 172.21.15.46:0/1267041777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac007260 con 0x7f6ec406c190 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.043+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec40b5010 con 0x7f6ec406c190 2024-04-25T08:42:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.043+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec40b55d0 con 0x7f6ec406c190 2024-04-25T08:42:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.043+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec4069080 con 0x7f6ec406c190 2024-04-25T08:42:28.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.043+0000 7f6ebbe00640 1 -- 172.21.15.46:0/1267041777 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6eac007400 con 0x7f6ec406c190 2024-04-25T08:42:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.043+0000 7f6ebbe00640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ea0066f60 0x7f6ea0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.047+0000 7f6ebbe00640 1 -- 172.21.15.46:0/1267041777 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6eac084800 con 0x7f6ec406c190 2024-04-25T08:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.047+0000 7f6ec3400640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ea0066f60 0x7f6ea0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.047+0000 7f6ec3400640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ea0066f60 0x7f6ea0069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40011f0 tx=0x7f6eb4007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.051+0000 7f6ebbe00640 1 -- 172.21.15.46:0/1267041777 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6eac054e70 con 0x7f6ec406c190 2024-04-25T08:42:28.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.211+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ec40b7630 con 0x7f6ec406c190 2024-04-25T08:42:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.211+0000 7f6ebbe00640 1 -- 172.21.15.46:0/1267041777 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f6eac058f10 con 0x7f6ec406c190 2024-04-25T08:42:28.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:28.207 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:28.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ea0066f60 msgr2=0x7f6ea0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ea0066f60 0x7f6ea0069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40011f0 tx=0x7f6eb4007d40 comp rx=0 tx=0).stop 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 msgr2=0x7f6ec40c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 0x7f6ec40c66b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00c420 tx=0x7f6eac00c8e0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 shutdown_connections 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ea0066f60 0x7f6ea0069420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec4099a00 0x7f6ec40b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec406c190 0x7f6ec40c66b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 --2- 172.21.15.46:0/1267041777 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ec406bd20 0x7f6ec4161870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 >> 172.21.15.46:0/1267041777 conn(0x7f6ec40d16b0 msgr2=0x7f6ec4099de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 shutdown_connections 2024-04-25T08:42:28.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.215+0000 7f6eca400640 1 -- 172.21.15.46:0/1267041777 wait complete. 2024-04-25T08:42:28.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:42:20.723239+0000', 'modified': '2024-04-25T08:42:25.795379+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/3580064760', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 3580064760}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 3580064760}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T08:42:28.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:42:28.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T08:42:28.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa10ac00640 1 Processor -- start 2024-04-25T08:42:28.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa10ac00640 1 -- start start 2024-04-25T08:42:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa10ac00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa104099a00 0x7fa104099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa10ac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa10406bd20 0x7fa10406c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa10ac00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1040cf5b0 con 0x7fa1041616b0 2024-04-25T08:42:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa10ac00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10406d060 con 0x7fa104099a00 2024-04-25T08:42:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa10ac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10409a320 con 0x7fa10406bd20 2024-04-25T08:42:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa103e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa104099a00 0x7fa104099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa103e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa104099a00 0x7fa104099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34730/0 (socket says 172.21.15.46:34730) 2024-04-25T08:42:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.783+0000 7fa103e00640 1 -- 172.21.15.46:0/768401827 learned_addr learned my addr 172.21.15.46:0/768401827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa0fb400640 1 --2- 172.21.15.46:0/768401827 >> v2:172.21.15.82:3300/0 conn(0x7fa10406bd20 0x7fa10406c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa103e00640 1 -- 172.21.15.46:0/768401827 >> v2:172.21.15.82:3300/0 conn(0x7fa10406bd20 msgr2=0x7fa10406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa103e00640 1 --2- 172.21.15.46:0/768401827 >> v2:172.21.15.82:3300/0 conn(0x7fa10406bd20 0x7fa10406c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa103e00640 1 -- 172.21.15.46:0/768401827 >> v1:172.21.15.82:6789/0 conn(0x7fa1041616b0 legacy=0x7fa104161b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa103e00640 1 -- 172.21.15.46:0/768401827 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10406fa90 con 0x7fa104099a00 2024-04-25T08:42:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa103e00640 1 --2- 172.21.15.46:0/768401827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa104099a00 0x7fa104099de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa10406f880 tx=0x7fa0f401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9c203715a020ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa103400640 1 -- 172.21.15.46:0/768401827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40aa650 con 0x7fa104099a00 2024-04-25T08:42:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa103400640 1 -- 172.21.15.46:0/768401827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f40af070 con 0x7fa104099a00 2024-04-25T08:42:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa103400640 1 -- 172.21.15.46:0/768401827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40aaa10 con 0x7fa104099a00 2024-04-25T08:42:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa10ac00640 1 -- 172.21.15.46:0/768401827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa104099a00 msgr2=0x7fa104099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.787+0000 7fa10ac00640 1 --2- 172.21.15.46:0/768401827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa104099a00 0x7fa104099de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa10406f880 tx=0x7fa0f401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 -- 172.21.15.46:0/768401827 shutdown_connections 2024-04-25T08:42:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 --2- 172.21.15.46:0/768401827 >> v2:172.21.15.82:3300/0 conn(0x7fa10406bd20 0x7fa10406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 --2- 172.21.15.46:0/768401827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa104099a00 0x7fa104099de0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 -- 172.21.15.46:0/768401827 >> 172.21.15.46:0/768401827 conn(0x7fa1040d16b0 msgr2=0x7fa1040d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 -- 172.21.15.46:0/768401827 shutdown_connections 2024-04-25T08:42:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 -- 172.21.15.46:0/768401827 wait complete. 2024-04-25T08:42:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 Processor -- start 2024-04-25T08:42:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 -- start start 2024-04-25T08:42:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10406bd20 0x7fa1041512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa104099a00 0x7fa104151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1041616b0 0x7fa104151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10409a750 con 0x7fa1041616b0 2024-04-25T08:42:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10406fd50 con 0x7fa10406bd20 2024-04-25T08:42:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa10ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa104071280 con 0x7fa104099a00 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa108c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1041616b0 0x7fa104151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa108c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1041616b0 0x7fa104151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34738/0 (socket says 172.21.15.46:34738) 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa108c00640 1 -- 172.21.15.46:0/2813164633 learned_addr learned my addr 172.21.15.46:0/2813164633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa103e00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10406bd20 0x7fa1041512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa0fb400640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa104099a00 0x7fa104151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa108c00640 1 -- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa104099a00 msgr2=0x7fa104151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa108c00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa104099a00 0x7fa104151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa108c00640 1 -- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10406bd20 msgr2=0x7fa1041512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa108c00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10406bd20 0x7fa1041512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa108c00640 1 -- 172.21.15.46:0/2813164633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa104151820 con 0x7fa1041616b0 2024-04-25T08:42:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.791+0000 7fa108c00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1041616b0 0x7fa104151f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00c770 tx=0x7fa0fc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.795+0000 7fa100c00640 1 -- 172.21.15.46:0/2813164633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc003b80 con 0x7fa1041616b0 2024-04-25T08:42:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.795+0000 7fa100c00640 1 -- 172.21.15.46:0/2813164633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc003d20 con 0x7fa1041616b0 2024-04-25T08:42:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.795+0000 7fa100c00640 1 -- 172.21.15.46:0/2813164633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00aa90 con 0x7fa1041616b0 2024-04-25T08:42:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.795+0000 7fa10ac00640 1 -- 172.21.15.46:0/2813164633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa104152590 con 0x7fa1041616b0 2024-04-25T08:42:28.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.795+0000 7fa10ac00640 1 -- 172.21.15.46:0/2813164633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa104152b50 con 0x7fa1041616b0 2024-04-25T08:42:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.795+0000 7fa100c00640 1 -- 172.21.15.46:0/2813164633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0fc00ac30 con 0x7fa1041616b0 2024-04-25T08:42:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.795+0000 7fa100c00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa0e0066f60 0x7fa0e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.795+0000 7fa100c00640 1 -- 172.21.15.46:0/2813164633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa0fc085200 con 0x7fa1041616b0 2024-04-25T08:42:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 Processor -- start 2024-04-25T08:42:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa103e00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa0e0066f60 0x7fa0e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- start start 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc148cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148d240 0x55cfc14429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1443310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc132a5f0 con 0x55cfc1442f30 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa103e00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa0e0066f60 0x7fa0e0069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40ae000 tx=0x7fa0f40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc1291da0 con 0x55cfc148d240 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfc121a150 con 0x55cfc148c920 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa108c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1443310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa108c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1443310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34748/0 (socket says 172.21.15.46:34748) 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa108c00640 1 -- 172.21.15.46:0/3444370308 learned_addr learned my addr 172.21.15.46:0/3444370308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa103e00640 1 --2- 172.21.15.46:0/3444370308 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc148cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa0fb400640 1 --2- 172.21.15.46:0/3444370308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148d240 0x55cfc14429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa108c00640 1 -- 172.21.15.46:0/3444370308 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 msgr2=0x55cfc148cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa108c00640 1 --2- 172.21.15.46:0/3444370308 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc148cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa108c00640 1 -- 172.21.15.46:0/3444370308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148d240 msgr2=0x55cfc14429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa108c00640 1 --2- 172.21.15.46:0/3444370308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148d240 0x55cfc14429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa108c00640 1 -- 172.21.15.46:0/3444370308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfc1233ca0 con 0x55cfc1442f30 2024-04-25T08:42:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa108c00640 1 --2- 172.21.15.46:0/3444370308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1443310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc007fd0 tx=0x7fa0fc055450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159722f11491c476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa0f8c00640 1 -- 172.21.15.46:0/3444370308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc05de40 con 0x55cfc1442f30 2024-04-25T08:42:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa0f8c00640 1 -- 172.21.15.46:0/3444370308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc053ae0 con 0x55cfc1442f30 2024-04-25T08:42:28.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa0f8c00640 1 -- 172.21.15.46:0/3444370308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc051e50 con 0x55cfc1442f30 2024-04-25T08:42:28.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- 172.21.15.46:0/3444370308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 msgr2=0x55cfc1443310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 --2- 172.21.15.46:0/3444370308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1443310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc007fd0 tx=0x7fa0fc055450 comp rx=0 tx=0).stop 2024-04-25T08:42:28.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- 172.21.15.46:0/3444370308 shutdown_connections 2024-04-25T08:42:28.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 --2- 172.21.15.46:0/3444370308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1443310 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 --2- 172.21.15.46:0/3444370308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148d240 0x55cfc14429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 --2- 172.21.15.46:0/3444370308 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc148cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- 172.21.15.46:0/3444370308 >> 172.21.15.46:0/3444370308 conn(0x55cfc14492a0 msgr2=0x55cfc14412d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- 172.21.15.46:0/3444370308 shutdown_connections 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- 172.21.15.46:0/3444370308 wait complete. 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 -- start start 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.799+0000 7fa10e031000 1 Processor -- start 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1493ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148c920 0x55cfc1494410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148d240 0x55cfc1446aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc124c990 con 0x55cfc1442f30 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc128de40 con 0x55cfc148c920 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfc121d900 con 0x55cfc148d240 2024-04-25T08:42:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 10 client.0 _ll_register_callbacks cb 0x55cfc13451c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa103e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1493ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa103e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1493ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34758/0 (socket says 172.21.15.46:34758) 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa103e00640 1 -- 172.21.15.46:0/2771334651 learned_addr learned my addr 172.21.15.46:0/2771334651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0fb400640 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148c920 0x55cfc1494410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa108c00640 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148d240 0x55cfc1446aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa103e00640 1 -- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148d240 msgr2=0x55cfc1446aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa103e00640 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148d240 0x55cfc1446aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa103e00640 1 -- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148c920 msgr2=0x55cfc1494410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa103e00640 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148c920 0x55cfc1494410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa103e00640 1 -- 172.21.15.46:0/2771334651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfc124a630 con 0x55cfc1442f30 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa103e00640 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1493ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4019100 tx=0x7fa0f401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b333a82f3c1469a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0f1600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0f1600640 1 -- 172.21.15.46:0/2771334651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40cd060 con 0x55cfc1442f30 2024-04-25T08:42:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0f1600640 1 -- 172.21.15.46:0/2771334651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa0f40c07a0 con 0x55cfc1442f30 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 -- 172.21.15.46:0/2771334651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0fc053ae0 con 0x55cfc1442f30 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0f1600640 1 -- 172.21.15.46:0/2771334651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f40c09a0 con 0x55cfc1442f30 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 10 client.4407 fetch_fsmap learned FSMap version 7 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 -- 172.21.15.46:0/2771334651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55cfc121a3a0 con 0x55cfc1442f30 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0f1600640 1 -- 172.21.15.46:0/2771334651 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc053ae0 con 0x55cfc1442f30 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0f1600640 1 -- 172.21.15.46:0/2771334651 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fa0f40c7030 con 0x55cfc1442f30 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 10 client.4407 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 20 client.4407 populate_metadata read hostname 'smithi046' 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x55cfc1448160 0x55cfc149df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 4 client.4407 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:42:28.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa10e031000 1 -- 172.21.15.46:0/2771334651 --> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfc122da50 con 0x55cfc1448160 2024-04-25T08:42:28.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0fb400640 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x55cfc1448160 0x55cfc149df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0fb400640 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x55cfc1448160 0x55cfc149df30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea39f5b15d94aa0 server_cookie=5129d5995e66a4b9 in_seq=0 out_seq=0 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.803+0000 7fa0f1600640 10 client.4407 ms_handle_connect on v2:172.21.15.82:6835/3580064760 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa0f1600640 1 -- 172.21.15.46:0/2771334651 <== mds.0 v2:172.21.15.82:6835/3580064760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cfc122da50 con 0x55cfc1448160 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa0f1600640 10 client.4407 handle_command_reply: tid=0 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 client.4407 shutdown 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 -- 172.21.15.46:0/2771334651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 msgr2=0x55cfc1493ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1493ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4019100 tx=0x7fa0f401ad30 comp rx=0 tx=0).stop 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 -- 172.21.15.46:0/2771334651 shutdown_connections 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x55cfc1448160 0x55cfc149df30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148d240 0x55cfc1446aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc148c920 0x55cfc1494410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 --2- 172.21.15.46:0/2771334651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc1442f30 0x55cfc1493ed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa0f1600640 0 client.4407 ms_handle_reset on v2:172.21.15.82:6835/3580064760 2024-04-25T08:42:28.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 -- 172.21.15.46:0/2771334651 >> 172.21.15.46:0/2771334651 conn(0x55cfc14492a0 msgr2=0x55cfc14419e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 -- 172.21.15.46:0/2771334651 shutdown_connections 2024-04-25T08:42:28.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 1 -- 172.21.15.46:0/2771334651 wait complete. 2024-04-25T08:42:28.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.807+0000 7fa10e031000 20 client.4407 trim_cache size 0 max 16384 2024-04-25T08:42:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 Processor -- start 2024-04-25T08:42:28.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 -- start start 2024-04-25T08:42:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 0x55cfc1446330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc1446870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc1496b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104203620 con 0x55cfc148d240 2024-04-25T08:42:28.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104202b30 con 0x55cfc1442f30 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa104203020 con 0x55cfc148c920 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa108c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc1496b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa108c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc1496b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34760/0 (socket says 172.21.15.46:34760) 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa108c00640 1 -- 172.21.15.46:0/2793104104 learned_addr learned my addr 172.21.15.46:0/2793104104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa103e00640 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 0x55cfc1446330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa0fb400640 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc1446870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa108c00640 1 -- 172.21.15.46:0/2793104104 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 msgr2=0x55cfc1446870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa108c00640 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc1446870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa108c00640 1 -- 172.21.15.46:0/2793104104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 msgr2=0x55cfc1446330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa108c00640 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 0x55cfc1446330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa108c00640 1 -- 172.21.15.46:0/2793104104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfc148d070 con 0x55cfc148d240 2024-04-25T08:42:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa103e00640 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 0x55cfc1446330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa108c00640 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc1496b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc051bf0 tx=0x7fa0fc005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0df8c0da54cbabb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa0f3e00640 1 -- 172.21.15.46:0/2793104104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc072070 con 0x55cfc148d240 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa0f3e00640 1 -- 172.21.15.46:0/2793104104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc053ae0 con 0x55cfc148d240 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa0f3e00640 1 -- 172.21.15.46:0/2793104104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc051e50 con 0x55cfc148d240 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 -- 172.21.15.46:0/2793104104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 msgr2=0x55cfc1496b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.823+0000 7fa10e031000 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc1496b00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc051bf0 tx=0x7fa0fc005050 comp rx=0 tx=0).stop 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- 172.21.15.46:0/2793104104 shutdown_connections 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc1496b00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc1446870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 --2- 172.21.15.46:0/2793104104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 0x55cfc1446330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- 172.21.15.46:0/2793104104 >> 172.21.15.46:0/2793104104 conn(0x55cfc14492a0 msgr2=0x55cfc1479010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- 172.21.15.46:0/2793104104 shutdown_connections 2024-04-25T08:42:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- 172.21.15.46:0/2793104104 wait complete. 2024-04-25T08:42:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- start start 2024-04-25T08:42:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 Processor -- start 2024-04-25T08:42:28.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 0x55cfc14d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc14d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc14d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc12963d0 con 0x55cfc148d240 2024-04-25T08:42:28.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc134dd50 con 0x55cfc1442f30 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfc122c980 con 0x55cfc148c920 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 10 client.0 _ll_register_callbacks cb 0x7fa0fc00ce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa108c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc14d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa108c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc14d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34766/0 (socket says 172.21.15.46:34766) 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa108c00640 1 -- 172.21.15.46:0/3638632903 learned_addr learned my addr 172.21.15.46:0/3638632903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa103e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 0x55cfc14d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa0fb400640 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc14d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa108c00640 1 -- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 msgr2=0x55cfc14d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa108c00640 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc14d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa108c00640 1 -- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 msgr2=0x55cfc14d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa108c00640 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 0x55cfc14d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa108c00640 1 -- 172.21.15.46:0/3638632903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfc1313600 con 0x55cfc148d240 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa108c00640 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc14d17b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc055770 tx=0x7fa0fc01d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84976a38cf21f161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa0f1600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa0f1600640 1 -- 172.21.15.46:0/3638632903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc072050 con 0x55cfc148d240 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa0f1600640 1 -- 172.21.15.46:0/3638632903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa0fc073b70 con 0x55cfc148d240 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- 172.21.15.46:0/3638632903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0fc053ae0 con 0x55cfc148d240 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa0f1600640 1 -- 172.21.15.46:0/3638632903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0fc073d70 con 0x55cfc148d240 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 10 client.4413 fetch_fsmap learned FSMap version 7 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- 172.21.15.46:0/3638632903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55cfc12269b0 con 0x55cfc148d240 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa0f1600640 1 -- 172.21.15.46:0/3638632903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc053ae0 con 0x55cfc148d240 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa0f1600640 1 -- 172.21.15.46:0/3638632903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fa0fc054c30 con 0x55cfc148d240 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 10 client.4413 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 10 client.4413 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 20 client.4413 populate_metadata read hostname 'smithi046' 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x55cfc14d1cf0 0x55cfc14997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 4 client.4413 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.827+0000 7fa10e031000 1 -- 172.21.15.46:0/3638632903 --> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cfc129f2e0 con 0x55cfc14d1cf0 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa103e00640 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x55cfc14d1cf0 0x55cfc14997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa103e00640 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x55cfc14d1cf0 0x55cfc14997d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e8b7c7f7532d22 server_cookie=faf1e14a50381294 in_seq=0 out_seq=0 2024-04-25T08:42:28.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa0f1600640 10 client.4413 ms_handle_connect on v2:172.21.15.82:6835/3580064760 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa0f1600640 1 -- 172.21.15.46:0/3638632903 <== mds.0 v2:172.21.15.82:6835/3580064760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55cfc129f2e0 con 0x55cfc14d1cf0 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa0f1600640 10 client.4413 handle_command_reply: tid=0 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 client.4413 shutdown 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 -- 172.21.15.46:0/3638632903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 msgr2=0x55cfc14d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc14d17b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc055770 tx=0x7fa0fc01d040 comp rx=0 tx=0).stop 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 -- 172.21.15.46:0/3638632903 shutdown_connections 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x55cfc14d1cf0 0x55cfc14997d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfc148d240 0x55cfc14d17b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cfc148c920 0x55cfc14d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 --2- 172.21.15.46:0/3638632903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cfc1442f30 0x55cfc14d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 -- 172.21.15.46:0/3638632903 >> 172.21.15.46:0/3638632903 conn(0x55cfc14492a0 msgr2=0x55cfc1445320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa0f1600640 0 client.4413 ms_handle_reset on v2:172.21.15.82:6835/3580064760 2024-04-25T08:42:28.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 -- 172.21.15.46:0/3638632903 shutdown_connections 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 1 -- 172.21.15.46:0/3638632903 wait complete. 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10e031000 20 client.4413 trim_cache size 0 max 16384 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4281, 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 7.5038002060000002, 2024-04-25T08:42:28.828 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 7, 2024-04-25T08:42:28.829 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 26, 2024-04-25T08:42:28.829 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T08:42:28.829 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 14.665847201 2024-04-25T08:42:28.829 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:42:28.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 -- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa0e0066f60 msgr2=0x7fa0e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa0e0066f60 0x7fa0e0069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40ae000 tx=0x7fa0f40a7000 comp rx=0 tx=0).stop 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 -- 172.21.15.46:0/2813164633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1041616b0 msgr2=0x7fa104151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1041616b0 0x7fa104151f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00c770 tx=0x7fa0fc00cc30 comp rx=0 tx=0).stop 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 -- 172.21.15.46:0/2813164633 shutdown_connections 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa0e0066f60 0x7fa0e0069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1041616b0 0x7fa104151f60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa104099a00 0x7fa104151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 --2- 172.21.15.46:0/2813164633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa10406bd20 0x7fa1041512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 -- 172.21.15.46:0/2813164633 >> 172.21.15.46:0/2813164633 conn(0x7fa1040d16b0 msgr2=0x7fa104160bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 -- 172.21.15.46:0/2813164633 shutdown_connections 2024-04-25T08:42:28.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.831+0000 7fa10ac00640 1 -- 172.21.15.46:0/2813164633 wait complete. 2024-04-25T08:42:28.843 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:42:28.844 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:42:28.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:42:28.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d8ac00640 1 Processor -- start 2024-04-25T08:42:28.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d8ac00640 1 -- start start 2024-04-25T08:42:28.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d8ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 0x7f3d840cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d8ac00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d840cc700 con 0x7f3d840cda80 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d8ac00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d840cc9e0 con 0x7f3d84159230 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d8ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840cccc0 con 0x7f3d8415bdb0 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d88c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3d840cda80 0x7f3d8415d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33256/0 (socket says 172.21.15.46:33256) 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d88c00640 1 -- 172.21.15.46:0/3202914222 learned_addr learned my addr 172.21.15.46:0/3202914222 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d7be00640 1 --2- 172.21.15.46:0/3202914222 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 0x7f3d840cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d83400640 1 -- 172.21.15.46:0/3202914222 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816079401 0 0) 0x7f3d840cc700 con 0x7f3d840cda80 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.991+0000 7f3d83400640 1 -- 172.21.15.46:0/3202914222 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d68003530 con 0x7f3d840cda80 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d7be00640 1 -- 172.21.15.46:0/3202914222 >> v1:172.21.15.82:6789/0 conn(0x7f3d84159230 legacy=0x7f3d840cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d7be00640 1 -- 172.21.15.46:0/3202914222 >> v1:172.21.15.46:6789/0 conn(0x7f3d840cda80 legacy=0x7f3d8415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d7be00640 1 -- 172.21.15.46:0/3202914222 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d840ccfa0 con 0x7f3d8415bdb0 2024-04-25T08:42:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d7be00640 1 --2- 172.21.15.46:0/3202914222 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 0x7f3d840cd540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0097f0 tx=0x7f3d6c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9562298ee9a6951c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d83400640 1 -- 172.21.15.46:0/3202914222 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0b0070 con 0x7f3d8415bdb0 2024-04-25T08:42:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d83400640 1 -- 172.21.15.46:0/3202914222 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c0ab600 con 0x7f3d8415bdb0 2024-04-25T08:42:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d83400640 1 -- 172.21.15.46:0/3202914222 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0ab9d0 con 0x7f3d8415bdb0 2024-04-25T08:42:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3202914222 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 msgr2=0x7f3d840cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d8ac00640 1 --2- 172.21.15.46:0/3202914222 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 0x7f3d840cd540 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0097f0 tx=0x7f3d6c01c450 comp rx=0 tx=0).stop 2024-04-25T08:42:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3202914222 shutdown_connections 2024-04-25T08:42:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d8ac00640 1 --2- 172.21.15.46:0/3202914222 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 0x7f3d840cd540 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.995+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3202914222 >> 172.21.15.46:0/3202914222 conn(0x7f3d840b12a0 msgr2=0x7f3d840b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3202914222 shutdown_connections 2024-04-25T08:42:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3202914222 wait complete. 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 Processor -- start 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 -- start start 2024-04-25T08:42:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d840cda80 0x7f3d8413b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d84159230 0x7f3d840b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 0x7f3d84157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8415e4b0 con 0x7f3d84159230 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8415e330 con 0x7f3d840cda80 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d84169790 con 0x7f3d8415bdb0 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d84159230 0x7f3d840b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d84159230 0x7f3d840b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34778/0 (socket says 172.21.15.46:34778) 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d7be00640 1 -- 172.21.15.46:0/3409514756 learned_addr learned my addr 172.21.15.46:0/3409514756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d88c00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 0x7f3d84157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d83e00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d840cda80 0x7f3d8413b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d7be00640 1 -- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 msgr2=0x7f3d84157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d7be00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 0x7f3d84157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d7be00640 1 -- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d840cda80 msgr2=0x7f3d8413b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d7be00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d840cda80 0x7f3d8413b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d7be00640 1 -- 172.21.15.46:0/3409514756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d840ae6f0 con 0x7f3d84159230 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d7be00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d84159230 0x7f3d840b2390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0097f0 tx=0x7f3d6c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d80c00640 1 -- 172.21.15.46:0/3409514756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0b8020 con 0x7f3d84159230 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d80c00640 1 -- 172.21.15.46:0/3409514756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c0b4410 con 0x7f3d84159230 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d840ae9d0 con 0x7f3d84159230 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:28.999+0000 7f3d80c00640 1 -- 172.21.15.46:0/3409514756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0b0040 con 0x7f3d84159230 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.003+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8413a080 con 0x7f3d84159230 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.003+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d84069080 con 0x7f3d84159230 2024-04-25T08:42:28.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.003+0000 7f3d80c00640 1 -- 172.21.15.46:0/3409514756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d6c0bb020 con 0x7f3d84159230 2024-04-25T08:42:28.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.003+0000 7f3d80c00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d60066f60 0x7f3d60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.003+0000 7f3d80c00640 1 -- 172.21.15.46:0/3409514756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3d6c11ed70 con 0x7f3d84159230 2024-04-25T08:42:28.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.003+0000 7f3d83e00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d60066f60 0x7f3d60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.003+0000 7f3d83e00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d60066f60 0x7f3d60069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d74000f70 tx=0x7f3d74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.007+0000 7f3d80c00640 1 -- 172.21.15.46:0/3409514756 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d6c0c2030 con 0x7f3d84159230 2024-04-25T08:42:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.207+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3d840b1ba0 con 0x7f3d84159230 2024-04-25T08:42:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d80c00640 1 -- 172.21.15.46:0/3409514756 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3d6c0160d0 con 0x7f3d84159230 2024-04-25T08:42:29.203 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:42:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d60066f60 msgr2=0x7f3d60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d60066f60 0x7f3d60069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d74000f70 tx=0x7f3d74009040 comp rx=0 tx=0).stop 2024-04-25T08:42:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d84159230 msgr2=0x7f3d840b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d84159230 0x7f3d840b2390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0097f0 tx=0x7f3d6c0191f0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 shutdown_connections 2024-04-25T08:42:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d60066f60 0x7f3d60069420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d8415bdb0 0x7f3d84157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d84159230 0x7f3d840b2390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 --2- 172.21.15.46:0/3409514756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d840cda80 0x7f3d8413b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 >> 172.21.15.46:0/3409514756 conn(0x7f3d840b12a0 msgr2=0x7f3d84140770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 shutdown_connections 2024-04-25T08:42:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:29.211+0000 7f3d8ac00640 1 -- 172.21.15.46:0/3409514756 wait complete. 2024-04-25T08:42:29.224 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:42:29.224 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:42:29.224 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:42:29.224 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T08:42:29.224 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:42:29.224 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:42:29.224 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:42:29.224 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:42:29.224 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:42:29.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.225 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:42:29.234 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:29.234 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:29.234 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:29.234 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:29.234 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:29.234 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:29.234 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:29.234 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:29.234 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42453sec preferred_lft 42453sec 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:42:29.235 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:29.236 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:42:29.236 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:29.236 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:42:29.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.236 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:29.236 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:29.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:42:29.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-04-25T08:42:29.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-04-25T08:42:29.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:42:29.237 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:29.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.348 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:42:29.368 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:42:29.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.371 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:42:29.438 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:42:29.438 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:42:29.438 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:42:29.438 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:42:29.438 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:42:29.438 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:42:29.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.439 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:29.439 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:29.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:42:29.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:42:29.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:42:29.439 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:29.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.528 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:42:29.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.538 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-04-25T08:42:29.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.590 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:29.590 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:29.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:42:29.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T08:42:29.591 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:29.674 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T08:42:29.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.674 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:29.674 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:29.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T08:42:29.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:42:29.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T08:42:29.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T08:42:29.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T08:42:29.675 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:29.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:29.843 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:29.844 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:29.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:42:29.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:42:29.844 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:29.883 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T08:42:29.883 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:29.883 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:42:29.938 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:42:29.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:29.939 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:42:29.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-04-25T08:42:30.051 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:42:30.107 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/run/lock 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-04-25T08:42:30.108 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/fs/nfsd 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/run/rpc_pipefs 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:42:30.109 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:42:30.110 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-04-25T08:42:30.110 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:42:30.110 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:42:30.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:30.111 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-04-25T08:42:30.162 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:42:30.162 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T08:42:30.205 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-04-25T08:42:30.226 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:42:30.275 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-04-25T08:42:30.275 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-04-25T08:42:30.275 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/run/lock 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-04-25T08:42:30.276 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/fs/nfsd 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/run/rpc_pipefs 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:30.278 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-04-25T08:42:30.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:2024-04-25T08:42:30.311+0000 7ff7ab78c580 -1 init, newargv = 0x55e20b020580 newargc=15 2024-04-25T08:42:30.303 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[32522]: starting ceph client 2024-04-25T08:42:30.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[32522]: starting fuse 2024-04-25T08:42:31.414 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-04-25T08:42:31.437 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:42:31.488 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-04-25T08:42:31.488 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/run/lock 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-04-25T08:42:31.489 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/fs/nfsd 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/run/rpc_pipefs 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:42:31.490 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-04-25T08:42:31.491 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:42:31.491 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:42:31.491 INFO:teuthology.orchestra.run.smithi046.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T08:42:31.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:31.492 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-04-25T08:42:31.546 INFO:teuthology.orchestra.run.smithi046.stdout:44 2024-04-25T08:42:31.547 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-04-25T08:42:31.547 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.admin")) 2024-04-25T08:42:31.548 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:42:31.683 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.admin.32522.asok 2024-04-25T08:42:31.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:42:31.695 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.32522.asok 2024-04-25T08:42:31.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:31.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.32522.asok status 2024-04-25T08:42:31.874 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "admin", 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "32522", 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4425, 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-04-25T08:42:31.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 4425 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4114594630 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4114594630 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.4425 192.168.0.1:0/4114594630", 2024-04-25T08:42:31.876 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/4114594630", 2024-04-25T08:42:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-04-25T08:42:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 7, 2024-04-25T08:42:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 26, 2024-04-25T08:42:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:42:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-04-25T08:42:31.877 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-04-25T08:42:31.877 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:42:31.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:31.890 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-04-25T08:42:31.890 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:42:31.899 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-04-25T08:42:31.899 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:42:31.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:31.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:42:31.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:31.963 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T08:42:32.053 INFO:teuthology.orchestra.run.smithi046.stdout:# file: . 2024-04-25T08:42:32.053 INFO:teuthology.orchestra.run.smithi046.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T08:42:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:32.054 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:42:32.054 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:32.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.255+0000 7f1e12e00640 1 Processor -- start 2024-04-25T08:42:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.255+0000 7f1e12e00640 1 -- start start 2024-04-25T08:42:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e12e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf140 0x7f1e0c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e12e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c06ed70 con 0x7f1e0c0afd40 2024-04-25T08:42:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e12e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e0c0cde00 con 0x7f1e0c0cf520 2024-04-25T08:42:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e12e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c0afb40 con 0x7f1e0c0cf140 2024-04-25T08:42:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e0b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1e0c0afd40 0x7f1e0c0ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33270/0 (socket says 172.21.15.46:33270) 2024-04-25T08:42:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e0b400640 1 -- 172.21.15.46:0/2898708464 learned_addr learned my addr 172.21.15.46:0/2898708464 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e0be00640 1 --2- 172.21.15.46:0/2898708464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf140 0x7f1e0c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e0aa00640 1 -- 172.21.15.46:0/2898708464 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82724454 0 0) 0x7f1df8001120 con 0x7f1e0c0afd40 2024-04-25T08:42:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e0aa00640 1 -- 172.21.15.46:0/2898708464 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df0003530 con 0x7f1e0c0afd40 2024-04-25T08:42:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e0be00640 1 -- 172.21.15.46:0/2898708464 >> v1:172.21.15.82:6789/0 conn(0x7f1e0c0cf520 legacy=0x7f1e0c0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e0be00640 1 -- 172.21.15.46:0/2898708464 >> v1:172.21.15.46:6789/0 conn(0x7f1e0c0afd40 legacy=0x7f1e0c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e0be00640 1 -- 172.21.15.46:0/2898708464 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c0d13f0 con 0x7f1e0c0cf140 2024-04-25T08:42:32.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.259+0000 7f1e0aa00640 1 -- 172.21.15.46:0/2898708464 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796201323 0 0) 0x7f1e0c0cde00 con 0x7f1e0c0cf520 2024-04-25T08:42:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e0be00640 1 --2- 172.21.15.46:0/2898708464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf140 0x7f1e0c15d590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0097f0 tx=0x7f1dfc01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b15cae15b32daa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e0aa00640 1 -- 172.21.15.46:0/2898708464 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0b1070 con 0x7f1e0c0cf140 2024-04-25T08:42:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e0aa00640 1 -- 172.21.15.46:0/2898708464 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc0ac620 con 0x7f1e0c0cf140 2024-04-25T08:42:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e0aa00640 1 -- 172.21.15.46:0/2898708464 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0ac9f0 con 0x7f1e0c0cf140 2024-04-25T08:42:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 -- 172.21.15.46:0/2898708464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf140 msgr2=0x7f1e0c15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 --2- 172.21.15.46:0/2898708464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf140 0x7f1e0c15d590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0097f0 tx=0x7f1dfc01c3c0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 -- 172.21.15.46:0/2898708464 shutdown_connections 2024-04-25T08:42:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 --2- 172.21.15.46:0/2898708464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf140 0x7f1e0c15d590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 -- 172.21.15.46:0/2898708464 >> 172.21.15.46:0/2898708464 conn(0x7f1e0c0728c0 msgr2=0x7f1e0c099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 -- 172.21.15.46:0/2898708464 shutdown_connections 2024-04-25T08:42:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 -- 172.21.15.46:0/2898708464 wait complete. 2024-04-25T08:42:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 Processor -- start 2024-04-25T08:42:32.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 -- start start 2024-04-25T08:42:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e0c0afd40 0x7f1e0c14a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c0cf140 0x7f1e0c154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.263+0000 7f1e12e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf520 0x7f1e0c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e12e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c162010 con 0x7f1e0c0afd40 2024-04-25T08:42:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e12e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c161d30 con 0x7f1e0c0cf140 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e12e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c15dee0 con 0x7f1e0c0cf520 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e0be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e0c0afd40 0x7f1e0c14a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e0be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e0c0afd40 0x7f1e0c14a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34784/0 (socket says 172.21.15.46:34784) 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e0be00640 1 -- 172.21.15.46:0/1815369507 learned_addr learned my addr 172.21.15.46:0/1815369507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e0be00640 1 -- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf520 msgr2=0x7f1e0c154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e0be00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf520 0x7f1e0c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e0be00640 1 -- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c0cf140 msgr2=0x7f1e0c154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e0be00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c0cf140 0x7f1e0c154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e0be00640 1 -- 172.21.15.46:0/1815369507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c1523e0 con 0x7f1e0c0afd40 2024-04-25T08:42:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e0be00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e0c0afd40 0x7f1e0c14a530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0097f0 tx=0x7f1dfc01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e03e00640 1 -- 172.21.15.46:0/1815369507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc0b1050 con 0x7f1e0c0afd40 2024-04-25T08:42:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e03e00640 1 -- 172.21.15.46:0/1815369507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc0b2040 con 0x7f1e0c0afd40 2024-04-25T08:42:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e0c1526c0 con 0x7f1e0c0afd40 2024-04-25T08:42:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e03e00640 1 -- 172.21.15.46:0/1815369507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc006e20 con 0x7f1e0c0afd40 2024-04-25T08:42:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e0c152c80 con 0x7f1e0c0afd40 2024-04-25T08:42:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e0c06ccb0 con 0x7f1e0c0afd40 2024-04-25T08:42:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.267+0000 7f1e03e00640 1 -- 172.21.15.46:0/1815369507 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1dfc0bf030 con 0x7f1e0c0afd40 2024-04-25T08:42:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.271+0000 7f1e03e00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dec066f60 0x7f1dec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.271+0000 7f1e03e00640 1 -- 172.21.15.46:0/1815369507 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1dfc11ee40 con 0x7f1e0c0afd40 2024-04-25T08:42:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.271+0000 7f1e0b400640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dec066f60 0x7f1dec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.271+0000 7f1e0b400640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dec066f60 0x7f1dec069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000f70 tx=0x7f1df8001120 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.275+0000 7f1e03e00640 1 -- 172.21.15.46:0/1815369507 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1dfc0c6060 con 0x7f1e0c0afd40 2024-04-25T08:42:32.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.439+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e0c072f90 con 0x7f1e0c0afd40 2024-04-25T08:42:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.439+0000 7f1e03e00640 1 -- 172.21.15.46:0/1815369507 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f1dfc0160d0 con 0x7f1e0c0afd40 2024-04-25T08:42:32.433 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:32.433 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dec066f60 msgr2=0x7f1dec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dec066f60 0x7f1dec069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1df8000f70 tx=0x7f1df8001120 comp rx=0 tx=0).stop 2024-04-25T08:42:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e0c0afd40 msgr2=0x7f1e0c14a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e0c0afd40 0x7f1e0c14a530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0097f0 tx=0x7f1dfc01ac00 comp rx=0 tx=0).stop 2024-04-25T08:42:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 shutdown_connections 2024-04-25T08:42:32.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dec066f60 0x7f1dec069420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e0c0cf520 0x7f1e0c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e0c0cf140 0x7f1e0c154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 --2- 172.21.15.46:0/1815369507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e0c0afd40 0x7f1e0c14a530 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 >> 172.21.15.46:0/1815369507 conn(0x7f1e0c0728c0 msgr2=0x7f1e0c151110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 shutdown_connections 2024-04-25T08:42:32.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.443+0000 7f1e12e00640 1 -- 172.21.15.46:0/1815369507 wait complete. 2024-04-25T08:42:32.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:42:32.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:42:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cfc200640 1 Processor -- start 2024-04-25T08:42:32.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cfc200640 1 -- start start 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cfc200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf4099c60 0x7f5cf4160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cfc200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf40cc530 con 0x7f5cf40cf210 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cfc200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf40cc810 con 0x7f5cf409a040 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cfc200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40cc300 con 0x7f5cf4099c60 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf8e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5cf40cf210 0x7f5cf40cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33280/0 (socket says 172.21.15.46:33280) 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf8e00640 1 -- 172.21.15.46:0/3581818365 learned_addr learned my addr 172.21.15.46:0/3581818365 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf9800640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf4099c60 0x7f5cf4160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507187152 0 0) 0x7f5cf40cc530 con 0x7f5cf40cf210 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cdc003530 con 0x7f5cf40cf210 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004314831 0 0) 0x7f5cf40cc810 con 0x7f5cf409a040 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf40cc530 con 0x7f5cf409a040 2024-04-25T08:42:32.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2954975703 0 0) 0x7f5cdc003530 con 0x7f5cf40cf210 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf4099c60 msgr2=0x7f5cf4160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf4099c60 0x7f5cf4160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 >> v1:172.21.15.82:6789/0 conn(0x7f5cf409a040 legacy=0x7f5cf415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40ccaf0 con 0x7f5cf40cf210 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f5ce0001680 con 0x7f5cf40cf210 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 >> v1:172.21.15.46:6789/0 conn(0x7f5cf40cf210 legacy=0x7f5cf40cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cdc003bf0 0x7f5cdc003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cdc004810 0x7f5cdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0051d0 0x7f5cdc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc00bff0 con 0x7f5cdc0051d0 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc00c1a0 con 0x7f5cdc004810 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc00c3a0 con 0x7f5cdc003bf0 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf9800640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf4099c60 0x7f5cf4160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cfa200640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0051d0 0x7f5cdc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf8e00640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cdc004810 0x7f5cdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.607+0000 7f5cf9800640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cdc003bf0 0x7f5cdc003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfa200640 1 -- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cdc003bf0 msgr2=0x7f5cdc003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfa200640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cdc003bf0 0x7f5cdc003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfa200640 1 -- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cdc004810 msgr2=0x7f5cdc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfa200640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cdc004810 0x7f5cdc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfa200640 1 -- 172.21.15.46:0/3581818365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc003530 con 0x7f5cdc0051d0 2024-04-25T08:42:32.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cf9800640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cdc003bf0 0x7f5cdc003ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:32.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfa200640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0051d0 0x7f5cdc00b9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cec001fd0 tx=0x7f5cec01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f66cb39adef819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec001c20 con 0x7f5cdc0051d0 2024-04-25T08:42:32.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec003ba0 con 0x7f5cdc0051d0 2024-04-25T08:42:32.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cf3e00640 1 -- 172.21.15.46:0/3581818365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec0050d0 con 0x7f5cdc0051d0 2024-04-25T08:42:32.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfc200640 1 -- 172.21.15.46:0/3581818365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0051d0 msgr2=0x7f5cdc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfc200640 1 --2- 172.21.15.46:0/3581818365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cdc0051d0 0x7f5cdc00b9d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cec001fd0 tx=0x7f5cec01c900 comp rx=0 tx=0).stop 2024-04-25T08:42:32.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfa200640 1 -- 172.21.15.46:0/3581818365 reap_dead start 2024-04-25T08:42:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfc200640 1 -- 172.21.15.46:0/3581818365 shutdown_connections 2024-04-25T08:42:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfc200640 1 -- 172.21.15.46:0/3581818365 >> 172.21.15.46:0/3581818365 conn(0x7f5cf40b3130 msgr2=0x7f5cf40b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfc200640 1 -- 172.21.15.46:0/3581818365 shutdown_connections 2024-04-25T08:42:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.611+0000 7f5cfc200640 1 -- 172.21.15.46:0/3581818365 wait complete. 2024-04-25T08:42:32.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 Processor -- start 2024-04-25T08:42:32.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 -- start start 2024-04-25T08:42:32.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf40cb120 0x7f5cf40b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf40cb500 0x7f5cf40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cf40b4180 0x7f5cf40b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf415f270 con 0x7f5cf40b4180 2024-04-25T08:42:32.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf40cd0b0 con 0x7f5cf40cb500 2024-04-25T08:42:32.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf415f3f0 con 0x7f5cf40cb120 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cf40b4180 0x7f5cf40b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cf40b4180 0x7f5cf40b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34794/0 (socket says 172.21.15.46:34794) 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfa200640 1 -- 172.21.15.46:0/3141975085 learned_addr learned my addr 172.21.15.46:0/3141975085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cf9800640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf40cb120 0x7f5cf40b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cf8e00640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf40cb500 0x7f5cf40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfa200640 1 -- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf40cb120 msgr2=0x7f5cf40b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfa200640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf40cb120 0x7f5cf40b5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfa200640 1 -- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf40cb500 msgr2=0x7f5cf40b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfa200640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf40cb500 0x7f5cf40b6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfa200640 1 -- 172.21.15.46:0/3141975085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf40b4d90 con 0x7f5cf40b4180 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cf8e00640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf40cb500 0x7f5cf40b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfa200640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cf40b4180 0x7f5cf40b4600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cec005040 tx=0x7f5cdc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cf1600640 1 -- 172.21.15.46:0/3141975085 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec007e00 con 0x7f5cf40b4180 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cf1600640 1 -- 172.21.15.46:0/3141975085 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec001c00 con 0x7f5cf40b4180 2024-04-25T08:42:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cf1600640 1 -- 172.21.15.46:0/3141975085 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec0055b0 con 0x7f5cf40b4180 2024-04-25T08:42:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf40b5070 con 0x7f5cf40b4180 2024-04-25T08:42:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.615+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf40b5630 con 0x7f5cf40b4180 2024-04-25T08:42:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.619+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf4069010 con 0x7f5cf40b4180 2024-04-25T08:42:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.619+0000 7f5cf1600640 1 -- 172.21.15.46:0/3141975085 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cec006b60 con 0x7f5cf40b4180 2024-04-25T08:42:32.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.619+0000 7f5cf1600640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5cd4066f90 0x7f5cd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.619+0000 7f5cf1600640 1 -- 172.21.15.46:0/3141975085 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5cec11dff0 con 0x7f5cf40b4180 2024-04-25T08:42:32.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.619+0000 7f5cf9800640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5cd4066f90 0x7f5cd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.619+0000 7f5cf9800640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5cd4066f90 0x7f5cd4069450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4072a30 tx=0x7f5ce8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.623+0000 7f5cf1600640 1 -- 172.21.15.46:0/3141975085 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cec016610 con 0x7f5cf40b4180 2024-04-25T08:42:32.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cf40b3f60 con 0x7f5cf40b4180 2024-04-25T08:42:32.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cf1600640 1 -- 172.21.15.46:0/3141975085 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5cec0ab340 con 0x7f5cf40b4180 2024-04-25T08:42:32.780 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5cd4066f90 msgr2=0x7f5cd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5cd4066f90 0x7f5cd4069450 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4072a30 tx=0x7f5ce8008040 comp rx=0 tx=0).stop 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cf40b4180 msgr2=0x7f5cf40b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cf40b4180 0x7f5cf40b4600 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cec005040 tx=0x7f5cdc005840 comp rx=0 tx=0).stop 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 shutdown_connections 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5cd4066f90 0x7f5cd4069450 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5cf40b4180 0x7f5cf40b4600 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf40cb500 0x7f5cf40b6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 --2- 172.21.15.46:0/3141975085 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5cf40cb120 0x7f5cf40b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.787+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 >> 172.21.15.46:0/3141975085 conn(0x7f5cf40b3130 msgr2=0x7f5cf40c7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.791+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 shutdown_connections 2024-04-25T08:42:32.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.791+0000 7f5cfc200640 1 -- 172.21.15.46:0/3141975085 wait complete. 2024-04-25T08:42:32.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:42:32.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa69aa00640 1 Processor -- start 2024-04-25T08:42:32.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa69aa00640 1 -- start start 2024-04-25T08:42:32.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa69aa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa69415ee00 0x7fa69409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa69aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69409a110 0x7fa694070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa69aa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6940cfd00 con 0x7fa69415f1e0 2024-04-25T08:42:32.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa69aa00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa694071560 con 0x7fa69415ee00 2024-04-25T08:42:32.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa69aa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940ccb00 con 0x7fa69409a110 2024-04-25T08:42:32.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa698a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69409a110 0x7fa694070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa693e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa69415ee00 0x7fa69409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa698a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69409a110 0x7fa694070820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:57480/0 (socket says 172.21.15.46:57480) 2024-04-25T08:42:32.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.951+0000 7fa698a00640 1 -- 172.21.15.46:0/1602046544 learned_addr learned my addr 172.21.15.46:0/1602046544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:32.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa698a00640 1 -- 172.21.15.46:0/1602046544 >> v2:172.21.15.82:3300/0 conn(0x7fa69415ee00 msgr2=0x7fa69409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:32.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa698a00640 1 --2- 172.21.15.46:0/1602046544 >> v2:172.21.15.82:3300/0 conn(0x7fa69415ee00 0x7fa69409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa698a00640 1 -- 172.21.15.46:0/1602046544 >> v1:172.21.15.82:6789/0 conn(0x7fa69415f1e0 legacy=0x7fa694099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa698a00640 1 -- 172.21.15.46:0/1602046544 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6940cffa0 con 0x7fa69409a110 2024-04-25T08:42:32.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa698a00640 1 --2- 172.21.15.46:0/1602046544 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69409a110 0x7fa694070820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009e50 tx=0x7fa68c01ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9b9774517a794a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa692a00640 1 -- 172.21.15.46:0/1602046544 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0a58f0 con 0x7fa69409a110 2024-04-25T08:42:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa692a00640 1 -- 172.21.15.46:0/1602046544 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c0a5a90 con 0x7fa69409a110 2024-04-25T08:42:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa692a00640 1 -- 172.21.15.46:0/1602046544 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0a5e60 con 0x7fa69409a110 2024-04-25T08:42:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa69aa00640 1 -- 172.21.15.46:0/1602046544 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69409a110 msgr2=0x7fa694070820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa69aa00640 1 --2- 172.21.15.46:0/1602046544 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69409a110 0x7fa694070820 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa68c009e50 tx=0x7fa68c01ca00 comp rx=0 tx=0).stop 2024-04-25T08:42:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa69aa00640 1 -- 172.21.15.46:0/1602046544 shutdown_connections 2024-04-25T08:42:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa69aa00640 1 --2- 172.21.15.46:0/1602046544 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69409a110 0x7fa694070820 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa69aa00640 1 --2- 172.21.15.46:0/1602046544 >> v2:172.21.15.82:3300/0 conn(0x7fa69415ee00 0x7fa69409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.955+0000 7fa69aa00640 1 -- 172.21.15.46:0/1602046544 >> 172.21.15.46:0/1602046544 conn(0x7fa6940b3170 msgr2=0x7fa6940b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 -- 172.21.15.46:0/1602046544 shutdown_connections 2024-04-25T08:42:32.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 -- 172.21.15.46:0/1602046544 wait complete. 2024-04-25T08:42:32.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 Processor -- start 2024-04-25T08:42:32.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 -- start start 2024-04-25T08:42:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa69409a110 0x7fa69415cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69415ee00 0x7fa694148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69415f1e0 0x7fa6940b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940cd920 con 0x7fa69415ee00 2024-04-25T08:42:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940cd7a0 con 0x7fa69409a110 2024-04-25T08:42:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa69aa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940cdaa0 con 0x7fa69415f1e0 2024-04-25T08:42:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69415ee00 0x7fa694148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69415ee00 0x7fa694148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34800/0 (socket says 172.21.15.46:34800) 2024-04-25T08:42:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa698a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69415f1e0 0x7fa6940b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693400640 1 -- 172.21.15.46:0/1683076375 learned_addr learned my addr 172.21.15.46:0/1683076375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693e00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa69409a110 0x7fa69415cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693400640 1 -- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69415f1e0 msgr2=0x7fa6940b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693400640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69415f1e0 0x7fa6940b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693400640 1 -- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa69409a110 msgr2=0x7fa69415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693400640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa69409a110 0x7fa69415cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693400640 1 -- 172.21.15.46:0/1683076375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6941491e0 con 0x7fa69415ee00 2024-04-25T08:42:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa698a00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69415f1e0 0x7fa6940b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa693400640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69415ee00 0x7fa694148ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c3c0 tx=0x7fa68000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.959+0000 7fa68be00640 1 -- 172.21.15.46:0/1683076375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680017070 con 0x7fa69415ee00 2024-04-25T08:42:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.963+0000 7fa68be00640 1 -- 172.21.15.46:0/1683076375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6800052f0 con 0x7fa69415ee00 2024-04-25T08:42:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.963+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6940b56a0 con 0x7fa69415ee00 2024-04-25T08:42:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.963+0000 7fa68be00640 1 -- 172.21.15.46:0/1683076375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680003860 con 0x7fa69415ee00 2024-04-25T08:42:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.963+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6940b5c60 con 0x7fa69415ee00 2024-04-25T08:42:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.963+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6941494e0 con 0x7fa69415ee00 2024-04-25T08:42:32.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.963+0000 7fa68be00640 1 -- 172.21.15.46:0/1683076375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa68001a070 con 0x7fa69415ee00 2024-04-25T08:42:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.963+0000 7fa68be00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.963+0000 7fa68be00640 1 -- 172.21.15.46:0/1683076375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa680084590 con 0x7fa69415ee00 2024-04-25T08:42:32.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.963+0000 7fa693e00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.967+0000 7fa693e00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa684009b60 tx=0x7fa684009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:32.967+0000 7fa68be00640 1 -- 172.21.15.46:0/1683076375 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa680054b70 con 0x7fa69415ee00 2024-04-25T08:42:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa694069010 con 0x7fa69415ee00 2024-04-25T08:42:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa68be00640 1 -- 172.21.15.46:0/1683076375 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa680058c10 con 0x7fa69415ee00 2024-04-25T08:42:33.124 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:42:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 msgr2=0x7fa674069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa684009b60 tx=0x7fa684009190 comp rx=0 tx=0).stop 2024-04-25T08:42:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69415ee00 msgr2=0x7fa694148ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69415ee00 0x7fa694148ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa68000c3c0 tx=0x7fa68000c880 comp rx=0 tx=0).stop 2024-04-25T08:42:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 shutdown_connections 2024-04-25T08:42:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa69415f1e0 0x7fa6940b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa69415ee00 0x7fa694148ca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 --2- 172.21.15.46:0/1683076375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa69409a110 0x7fa69415cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 >> 172.21.15.46:0/1683076375 conn(0x7fa6940b3170 msgr2=0x7fa694165c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 shutdown_connections 2024-04-25T08:42:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.131+0000 7fa69aa00640 1 -- 172.21.15.46:0/1683076375 wait complete. 2024-04-25T08:42:33.145 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:42:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fccab600640 1 Processor -- start 2024-04-25T08:42:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fccab600640 1 -- start start 2024-04-25T08:42:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fccab600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcca4161250 0x7fcca409a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fccab600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcca4099a00 0x7fcca4099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fccab600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca406bd20 0x7fcca406c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fccab600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca406ddb0 con 0x7fcca4161250 2024-04-25T08:42:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fccab600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca40cf7a0 con 0x7fcca4099a00 2024-04-25T08:42:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fccab600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca406d230 con 0x7fcca406bd20 2024-04-25T08:42:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fcca8c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcca4161250 0x7fcca409a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fcca8c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcca4161250 0x7fcca409a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34806/0 (socket says 172.21.15.46:34806) 2024-04-25T08:42:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.295+0000 7fcca8c00640 1 -- 172.21.15.46:0/2220582741 learned_addr learned my addr 172.21.15.46:0/2220582741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca8c00640 1 -- 172.21.15.46:0/2220582741 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca406bd20 msgr2=0x7fcca406c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca9600640 1 --2- 172.21.15.46:0/2220582741 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca406bd20 0x7fcca406c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcc9be00640 1 --2- 172.21.15.46:0/2220582741 >> v2:172.21.15.82:3300/0 conn(0x7fcca4099a00 0x7fcca4099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca8c00640 1 --2- 172.21.15.46:0/2220582741 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca406bd20 0x7fcca406c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca8c00640 1 -- 172.21.15.46:0/2220582741 >> v2:172.21.15.82:3300/0 conn(0x7fcca4099a00 msgr2=0x7fcca4099e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:33.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca8c00640 1 --2- 172.21.15.46:0/2220582741 >> v2:172.21.15.82:3300/0 conn(0x7fcca4099a00 0x7fcca4099e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca8c00640 1 -- 172.21.15.46:0/2220582741 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca406d510 con 0x7fcca4161250 2024-04-25T08:42:33.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcc9be00640 1 --2- 172.21.15.46:0/2220582741 >> v2:172.21.15.82:3300/0 conn(0x7fcca4099a00 0x7fcca4099e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:33.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca9600640 1 --2- 172.21.15.46:0/2220582741 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca406bd20 0x7fcca406c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:33.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca8c00640 1 --2- 172.21.15.46:0/2220582741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4161250 0x7fcca409a7e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcca4072990 tx=0x7fcc9401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a11c58de157b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca3e00640 1 -- 172.21.15.46:0/2220582741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940b1070 con 0x7fcca4161250 2024-04-25T08:42:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca3e00640 1 -- 172.21.15.46:0/2220582741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc940ac630 con 0x7fcca4161250 2024-04-25T08:42:33.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fcca3e00640 1 -- 172.21.15.46:0/2220582741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940a8050 con 0x7fcca4161250 2024-04-25T08:42:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fccab600640 1 -- 172.21.15.46:0/2220582741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4161250 msgr2=0x7fcca409a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.299+0000 7fccab600640 1 --2- 172.21.15.46:0/2220582741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4161250 0x7fcca409a7e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcca4072990 tx=0x7fcc9401c520 comp rx=0 tx=0).stop 2024-04-25T08:42:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 -- 172.21.15.46:0/2220582741 shutdown_connections 2024-04-25T08:42:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 --2- 172.21.15.46:0/2220582741 >> v2:172.21.15.82:3300/0 conn(0x7fcca4099a00 0x7fcca4099e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 --2- 172.21.15.46:0/2220582741 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca406bd20 0x7fcca406c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 --2- 172.21.15.46:0/2220582741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4161250 0x7fcca409a7e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 -- 172.21.15.46:0/2220582741 >> 172.21.15.46:0/2220582741 conn(0x7fcca40b3170 msgr2=0x7fcca40b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 -- 172.21.15.46:0/2220582741 shutdown_connections 2024-04-25T08:42:33.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 -- 172.21.15.46:0/2220582741 wait complete. 2024-04-25T08:42:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 Processor -- start 2024-04-25T08:42:33.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 -- start start 2024-04-25T08:42:33.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca406bd20 0x7fcca41546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4099a00 0x7fcca40c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca4161250 0x7fcca40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40cfdc0 con 0x7fcca4099a00 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40cfc40 con 0x7fcca406bd20 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca40cff40 con 0x7fcca4161250 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4099a00 0x7fcca40c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcc9be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4099a00 0x7fcca40c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34808/0 (socket says 172.21.15.46:34808) 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcc9be00640 1 -- 172.21.15.46:0/4070685199 learned_addr learned my addr 172.21.15.46:0/4070685199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcca8c00640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca406bd20 0x7fcca41546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcca9600640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca4161250 0x7fcca40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcc9be00640 1 -- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca4161250 msgr2=0x7fcca40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcc9be00640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca4161250 0x7fcca40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcc9be00640 1 -- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca406bd20 msgr2=0x7fcca41546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcc9be00640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca406bd20 0x7fcca41546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcc9be00640 1 -- 172.21.15.46:0/4070685199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca4154c30 con 0x7fcca4099a00 2024-04-25T08:42:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcca9600640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca4161250 0x7fcca40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcc9be00640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4099a00 0x7fcca40c7220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00c7c0 tx=0x7fcc8c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcca1600640 1 -- 172.21.15.46:0/4070685199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c007a30 con 0x7fcca4099a00 2024-04-25T08:42:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcca1600640 1 -- 172.21.15.46:0/4070685199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c007bd0 con 0x7fcca4099a00 2024-04-25T08:42:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fcca1600640 1 -- 172.21.15.46:0/4070685199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c0113f0 con 0x7fcca4099a00 2024-04-25T08:42:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.303+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca40b5680 con 0x7fcca4099a00 2024-04-25T08:42:33.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.307+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca40b5c40 con 0x7fcca4099a00 2024-04-25T08:42:33.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.307+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca4069010 con 0x7fcca4099a00 2024-04-25T08:42:33.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.307+0000 7fcca1600640 1 -- 172.21.15.46:0/4070685199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc8c011590 con 0x7fcca4099a00 2024-04-25T08:42:33.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.307+0000 7fcca1600640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcc80066f90 0x7fcc80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.307+0000 7fcca1600640 1 -- 172.21.15.46:0/4070685199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcc8c085610 con 0x7fcca4099a00 2024-04-25T08:42:33.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.307+0000 7fcca8c00640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcc80066f90 0x7fcc80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.307+0000 7fcca8c00640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcc80066f90 0x7fcc80069450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc940ae000 tx=0x7fcc94008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.311+0000 7fcca1600640 1 -- 172.21.15.46:0/4070685199 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc8c015020 con 0x7fcca4099a00 2024-04-25T08:42:33.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.475+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcca40b2460 con 0x7fcca4099a00 2024-04-25T08:42:33.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.475+0000 7fcca1600640 1 -- 172.21.15.46:0/4070685199 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc8c056240 con 0x7fcca4099a00 2024-04-25T08:42:33.470 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcc80066f90 msgr2=0x7fcc80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcc80066f90 0x7fcc80069450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc940ae000 tx=0x7fcc94008040 comp rx=0 tx=0).stop 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4099a00 msgr2=0x7fcca40c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4099a00 0x7fcca40c7220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00c7c0 tx=0x7fcc8c00cc80 comp rx=0 tx=0).stop 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 shutdown_connections 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcc80066f90 0x7fcc80069450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcca4161250 0x7fcca40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcca4099a00 0x7fcca40c7220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 --2- 172.21.15.46:0/4070685199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca406bd20 0x7fcca41546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 >> 172.21.15.46:0/4070685199 conn(0x7fcca40b3170 msgr2=0x7fcca415e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 shutdown_connections 2024-04-25T08:42:33.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.479+0000 7fccab600640 1 -- 172.21.15.46:0/4070685199 wait complete. 2024-04-25T08:42:33.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:42:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff55ec00640 1 Processor -- start 2024-04-25T08:42:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff55ec00640 1 -- start start 2024-04-25T08:42:33.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff55ec00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff558099a00 0x7ff558099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff55ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809ac20 0x7ff55815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff55ec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55809a320 con 0x7ff55809a7c0 2024-04-25T08:42:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff55ec00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55806cd30 con 0x7ff558099a00 2024-04-25T08:42:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff55ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580cf8a0 con 0x7ff55809ac20 2024-04-25T08:42:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff557400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff55809a7c0 0x7ff55815df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33296/0 (socket says 172.21.15.46:33296) 2024-04-25T08:42:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff557400640 1 -- 172.21.15.46:0/3054701131 learned_addr learned my addr 172.21.15.46:0/3054701131 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.643+0000 7ff55cc00640 1 --2- 172.21.15.46:0/3054701131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809ac20 0x7ff55815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff557e00640 1 --2- 172.21.15.46:0/3054701131 >> v2:172.21.15.82:3300/0 conn(0x7ff558099a00 0x7ff558099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff556a00640 1 -- 172.21.15.46:0/3054701131 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930485089 0 0) 0x7ff55809a320 con 0x7ff55809a7c0 2024-04-25T08:42:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff556a00640 1 -- 172.21.15.46:0/3054701131 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff53c003530 con 0x7ff55809a7c0 2024-04-25T08:42:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff55cc00640 1 -- 172.21.15.46:0/3054701131 >> v2:172.21.15.82:3300/0 conn(0x7ff558099a00 msgr2=0x7ff558099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff55cc00640 1 --2- 172.21.15.46:0/3054701131 >> v2:172.21.15.82:3300/0 conn(0x7ff558099a00 0x7ff558099de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff55cc00640 1 -- 172.21.15.46:0/3054701131 >> v1:172.21.15.46:6789/0 conn(0x7ff55809a7c0 legacy=0x7ff55815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff55cc00640 1 -- 172.21.15.46:0/3054701131 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580cfb80 con 0x7ff55809ac20 2024-04-25T08:42:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff55cc00640 1 --2- 172.21.15.46:0/3054701131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809ac20 0x7ff55815e6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff550009d90 tx=0x7ff55001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f1ea0f84b3b6b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff556a00640 1 -- 172.21.15.46:0/3054701131 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5500b2070 con 0x7ff55809ac20 2024-04-25T08:42:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff556a00640 1 -- 172.21.15.46:0/3054701131 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5500a5910 con 0x7ff55809ac20 2024-04-25T08:42:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.647+0000 7ff556a00640 1 -- 172.21.15.46:0/3054701131 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5500a5ce0 con 0x7ff55809ac20 2024-04-25T08:42:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 -- 172.21.15.46:0/3054701131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809ac20 msgr2=0x7ff55815e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 --2- 172.21.15.46:0/3054701131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809ac20 0x7ff55815e6a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff550009d90 tx=0x7ff55001c940 comp rx=0 tx=0).stop 2024-04-25T08:42:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 -- 172.21.15.46:0/3054701131 shutdown_connections 2024-04-25T08:42:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 --2- 172.21.15.46:0/3054701131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809ac20 0x7ff55815e6a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 --2- 172.21.15.46:0/3054701131 >> v2:172.21.15.82:3300/0 conn(0x7ff558099a00 0x7ff558099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 -- 172.21.15.46:0/3054701131 >> 172.21.15.46:0/3054701131 conn(0x7ff5580b3130 msgr2=0x7ff5580b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 -- 172.21.15.46:0/3054701131 shutdown_connections 2024-04-25T08:42:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 -- 172.21.15.46:0/3054701131 wait complete. 2024-04-25T08:42:33.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 Processor -- start 2024-04-25T08:42:33.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 -- start start 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff558099a00 0x7ff55814cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809a7c0 0x7ff5580705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55809ac20 0x7ff558070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580d0940 con 0x7ff558099a00 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580b06f0 con 0x7ff55809ac20 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff558152fd0 con 0x7ff55809a7c0 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff558099a00 0x7ff55814cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff558099a00 0x7ff55814cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34812/0 (socket says 172.21.15.46:34812) 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557e00640 1 -- 172.21.15.46:0/1643158858 learned_addr learned my addr 172.21.15.46:0/1643158858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff55cc00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55809ac20 0x7ff558070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557400640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809a7c0 0x7ff5580705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557e00640 1 -- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809a7c0 msgr2=0x7ff5580705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557e00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809a7c0 0x7ff5580705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557e00640 1 -- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55809ac20 msgr2=0x7ff558070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557e00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55809ac20 0x7ff558070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557e00640 1 -- 172.21.15.46:0/1643158858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580b2b90 con 0x7ff558099a00 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.651+0000 7ff557e00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff558099a00 0x7ff55814cb10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e870 tx=0x7ff54800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff54fe00640 1 -- 172.21.15.46:0/1643158858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548017070 con 0x7ff558099a00 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff54fe00640 1 -- 172.21.15.46:0/1643158858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54800c940 con 0x7ff558099a00 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff54fe00640 1 -- 172.21.15.46:0/1643158858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54800ccf0 con 0x7ff558099a00 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5580cbd10 con 0x7ff558099a00 2024-04-25T08:42:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5580cc2d0 con 0x7ff558099a00 2024-04-25T08:42:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff558069010 con 0x7ff558099a00 2024-04-25T08:42:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff54fe00640 1 -- 172.21.15.46:0/1643158858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff54801a070 con 0x7ff558099a00 2024-04-25T08:42:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff54fe00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff538066f60 0x7ff538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff54fe00640 1 -- 172.21.15.46:0/1643158858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff5480855f0 con 0x7ff558099a00 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.655+0000 7ff557400640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff538066f60 0x7ff538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.659+0000 7ff557400640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff538066f60 0x7ff538069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff558154010 tx=0x7ff544009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.659+0000 7ff54fe00640 1 -- 172.21.15.46:0/1643158858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff548055bd0 con 0x7ff558099a00 2024-04-25T08:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff55809a530 con 0x7ff558099a00 2024-04-25T08:42:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff54fe00640 1 -- 172.21.15.46:0/1643158858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff548059c70 con 0x7ff558099a00 2024-04-25T08:42:33.816 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff538066f60 msgr2=0x7ff538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff538066f60 0x7ff538069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff558154010 tx=0x7ff544009040 comp rx=0 tx=0).stop 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff558099a00 msgr2=0x7ff55814cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff558099a00 0x7ff55814cb10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e870 tx=0x7ff54800ec70 comp rx=0 tx=0).stop 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 shutdown_connections 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff538066f60 0x7ff538069420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff55809ac20 0x7ff558070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff55809a7c0 0x7ff5580705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 --2- 172.21.15.46:0/1643158858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff558099a00 0x7ff55814cb10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 >> 172.21.15.46:0/1643158858 conn(0x7ff5580b3130 msgr2=0x7ff55815d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 shutdown_connections 2024-04-25T08:42:33.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.823+0000 7ff55ec00640 1 -- 172.21.15.46:0/1643158858 wait complete. 2024-04-25T08:42:33.838 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:42:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6c200640 1 Processor -- start 2024-04-25T08:42:33.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6c200640 1 -- start start 2024-04-25T08:42:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6c200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa640b1f20 0x7ffa6406d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6c200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa640b43c0 0x7ffa6406db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6c200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6415aea0 con 0x7ffa6406f110 2024-04-25T08:42:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6c200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6415b4d0 con 0x7ffa640b43c0 2024-04-25T08:42:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6c200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6406fd60 con 0x7ffa640b1f20 2024-04-25T08:42:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa640b43c0 0x7ffa6406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa69800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa6406f110 0x7ffa640b3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33298/0 (socket says 172.21.15.46:33298) 2024-04-25T08:42:33.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa69800640 1 -- 172.21.15.46:0/1529516097 learned_addr learned my addr 172.21.15.46:0/1529516097 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:33.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa68e00640 1 --2- 172.21.15.46:0/1529516097 >> v2:172.21.15.82:3300/0 conn(0x7ffa640b1f20 0x7ffa6406d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6a200640 1 -- 172.21.15.46:0/1529516097 >> v2:172.21.15.82:3300/0 conn(0x7ffa640b1f20 msgr2=0x7ffa6406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:33.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6a200640 1 --2- 172.21.15.46:0/1529516097 >> v2:172.21.15.82:3300/0 conn(0x7ffa640b1f20 0x7ffa6406d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6a200640 1 -- 172.21.15.46:0/1529516097 >> v1:172.21.15.46:6789/0 conn(0x7ffa6406f110 legacy=0x7ffa640b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa6a200640 1 -- 172.21.15.46:0/1529516097 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6415b160 con 0x7ffa640b43c0 2024-04-25T08:42:33.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.987+0000 7ffa63e00640 1 -- 172.21.15.46:0/1529516097 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199954314 0 0) 0x7ffa6415aea0 con 0x7ffa6406f110 2024-04-25T08:42:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa6a200640 1 --2- 172.21.15.46:0/1529516097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa640b43c0 0x7ffa6406db70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffa6415b650 tx=0x7ffa5c01d390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4da3449cfd876a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa63e00640 1 -- 172.21.15.46:0/1529516097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0b2070 con 0x7ffa640b43c0 2024-04-25T08:42:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa63e00640 1 -- 172.21.15.46:0/1529516097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c0a5900 con 0x7ffa640b43c0 2024-04-25T08:42:33.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa63e00640 1 -- 172.21.15.46:0/1529516097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0a5cd0 con 0x7ffa640b43c0 2024-04-25T08:42:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa6c200640 1 -- 172.21.15.46:0/1529516097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa640b43c0 msgr2=0x7ffa6406db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa6c200640 1 --2- 172.21.15.46:0/1529516097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa640b43c0 0x7ffa6406db70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffa6415b650 tx=0x7ffa5c01d390 comp rx=0 tx=0).stop 2024-04-25T08:42:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa6c200640 1 -- 172.21.15.46:0/1529516097 shutdown_connections 2024-04-25T08:42:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa6c200640 1 --2- 172.21.15.46:0/1529516097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa640b43c0 0x7ffa6406db70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa6c200640 1 --2- 172.21.15.46:0/1529516097 >> v2:172.21.15.82:3300/0 conn(0x7ffa640b1f20 0x7ffa6406d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa6c200640 1 -- 172.21.15.46:0/1529516097 >> 172.21.15.46:0/1529516097 conn(0x7ffa6406b270 msgr2=0x7ffa640b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa6c200640 1 -- 172.21.15.46:0/1529516097 shutdown_connections 2024-04-25T08:42:33.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.991+0000 7ffa6c200640 1 -- 172.21.15.46:0/1529516097 wait complete. 2024-04-25T08:42:33.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 Processor -- start 2024-04-25T08:42:33.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 -- start start 2024-04-25T08:42:33.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa6406f110 0x7ffa64163cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa640b43c0 0x7ffa640b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa640b3070 0x7ffa64151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6406e0b0 con 0x7ffa6406f110 2024-04-25T08:42:33.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640cd0a0 con 0x7ffa640b43c0 2024-04-25T08:42:33.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6406e230 con 0x7ffa640b3070 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa69800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa6406f110 0x7ffa64163cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa69800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa6406f110 0x7ffa64163cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34834/0 (socket says 172.21.15.46:34834) 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa69800640 1 -- 172.21.15.46:0/2071282667 learned_addr learned my addr 172.21.15.46:0/2071282667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa68e00640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa640b43c0 0x7ffa640b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6a200640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa640b3070 0x7ffa64151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa69800640 1 -- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa640b3070 msgr2=0x7ffa64151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa69800640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa640b3070 0x7ffa64151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa69800640 1 -- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa640b43c0 msgr2=0x7ffa640b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa69800640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa640b43c0 0x7ffa640b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa69800640 1 -- 172.21.15.46:0/2071282667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa64164210 con 0x7ffa6406f110 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6a200640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa640b3070 0x7ffa64151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa69800640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa6406f110 0x7ffa64163cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa580011f0 tx=0x7ffa5800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa61600640 1 -- 172.21.15.46:0/2071282667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58018070 con 0x7ffa6406f110 2024-04-25T08:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa61600640 1 -- 172.21.15.46:0/2071282667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa580074e0 con 0x7ffa6406f110 2024-04-25T08:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa64152a00 con 0x7ffa6406f110 2024-04-25T08:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa61600640 1 -- 172.21.15.46:0/2071282667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58004a10 con 0x7ffa6406f110 2024-04-25T08:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.995+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa64152fc0 con 0x7ffa6406f110 2024-04-25T08:42:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.999+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa640b3480 con 0x7ffa6406f110 2024-04-25T08:42:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.999+0000 7ffa61600640 1 -- 172.21.15.46:0/2071282667 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa5801b070 con 0x7ffa6406f110 2024-04-25T08:42:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.999+0000 7ffa61600640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa44066f90 0x7ffa44069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.999+0000 7ffa61600640 1 -- 172.21.15.46:0/2071282667 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffa58084650 con 0x7ffa6406f110 2024-04-25T08:42:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.999+0000 7ffa68e00640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa44066f90 0x7ffa44069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:33.999+0000 7ffa68e00640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa44066f90 0x7ffa44069450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa640ae7c0 tx=0x7ffa50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.003+0000 7ffa61600640 1 -- 172.21.15.46:0/2071282667 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa640b3480 con 0x7ffa6406f110 2024-04-25T08:42:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.175+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffa64069080 con 0x7ffa6406f110 2024-04-25T08:42:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa61600640 1 -- 172.21.15.46:0/2071282667 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffa58054c30 con 0x7ffa6406f110 2024-04-25T08:42:34.171 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:42:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa44066f90 msgr2=0x7ffa44069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa44066f90 0x7ffa44069450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa640ae7c0 tx=0x7ffa50008040 comp rx=0 tx=0).stop 2024-04-25T08:42:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa6406f110 msgr2=0x7ffa64163cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa6406f110 0x7ffa64163cd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffa580011f0 tx=0x7ffa5800c9b0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 shutdown_connections 2024-04-25T08:42:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa44066f90 0x7ffa44069450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa640b3070 0x7ffa64151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa640b43c0 0x7ffa640b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 --2- 172.21.15.46:0/2071282667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa6406f110 0x7ffa64163cd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 >> 172.21.15.46:0/2071282667 conn(0x7ffa6406b270 msgr2=0x7ffa640cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 shutdown_connections 2024-04-25T08:42:34.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.179+0000 7ffa6c200640 1 -- 172.21.15.46:0/2071282667 wait complete. 2024-04-25T08:42:34.192 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:42:34.192 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:34.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:34.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.387+0000 7f9a72400640 1 Processor -- start 2024-04-25T08:42:34.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.387+0000 7f9a72400640 1 -- start start 2024-04-25T08:42:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a72400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a6c099a00 0x7f9a6c15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a72400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c06c7c0 0x7f9a6c06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a72400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a6c0d0f70 con 0x7f9a6c0d0980 2024-04-25T08:42:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a72400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6c15f360 con 0x7f9a6c099a00 2024-04-25T08:42:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a72400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c09a7c0 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a6be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c06c7c0 0x7f9a6c06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a6b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a6c099a00 0x7f9a6c15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a6b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a6c099a00 0x7f9a6c15ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:39054/0 (socket says 172.21.15.46:39054) 2024-04-25T08:42:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a6b400640 1 -- 172.21.15.46:0/3457495433 learned_addr learned my addr 172.21.15.46:0/3457495433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a6b400640 1 -- 172.21.15.46:0/3457495433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c06c7c0 msgr2=0x7f9a6c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a6b400640 1 --2- 172.21.15.46:0/3457495433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c06c7c0 0x7f9a6c06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a6b400640 1 -- 172.21.15.46:0/3457495433 >> v1:172.21.15.82:6789/0 conn(0x7f9a6c0d0980 legacy=0x7f9a6c099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.391+0000 7f9a6b400640 1 -- 172.21.15.46:0/3457495433 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0d1250 con 0x7f9a6c099a00 2024-04-25T08:42:34.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a6b400640 1 --2- 172.21.15.46:0/3457495433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c099a00 0x7f9a6c15ee20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009870 tx=0x7f9a5c01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=687662da46cfd469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a6aa00640 1 -- 172.21.15.46:0/3457495433 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0b0070 con 0x7f9a6c099a00 2024-04-25T08:42:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a6aa00640 1 -- 172.21.15.46:0/3457495433 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c004230 con 0x7f9a6c099a00 2024-04-25T08:42:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a6aa00640 1 -- 172.21.15.46:0/3457495433 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c003620 con 0x7f9a6c099a00 2024-04-25T08:42:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 -- 172.21.15.46:0/3457495433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c099a00 msgr2=0x7f9a6c15ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 --2- 172.21.15.46:0/3457495433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c099a00 0x7f9a6c15ee20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009870 tx=0x7f9a5c01c440 comp rx=0 tx=0).stop 2024-04-25T08:42:34.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 -- 172.21.15.46:0/3457495433 shutdown_connections 2024-04-25T08:42:34.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 --2- 172.21.15.46:0/3457495433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c06c7c0 0x7f9a6c06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 --2- 172.21.15.46:0/3457495433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c099a00 0x7f9a6c15ee20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 -- 172.21.15.46:0/3457495433 >> 172.21.15.46:0/3457495433 conn(0x7f9a6c0d16b0 msgr2=0x7f9a6c0d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:34.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 -- 172.21.15.46:0/3457495433 shutdown_connections 2024-04-25T08:42:34.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 -- 172.21.15.46:0/3457495433 wait complete. 2024-04-25T08:42:34.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 Processor -- start 2024-04-25T08:42:34.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.395+0000 7f9a72400640 1 -- start start 2024-04-25T08:42:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a72400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c06c7c0 0x7f9a6c0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a72400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c099a00 0x7f9a6c0b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a72400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c0d0980 0x7f9a6c1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a72400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c16afe0 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a72400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c16ae60 con 0x7f9a6c0d0980 2024-04-25T08:42:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a72400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c16b160 con 0x7f9a6c099a00 2024-04-25T08:42:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c06c7c0 0x7f9a6c0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c06c7c0 0x7f9a6c0b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34842/0 (socket says 172.21.15.46:34842) 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6b400640 1 -- 172.21.15.46:0/3608472653 learned_addr learned my addr 172.21.15.46:0/3608472653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6be00640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c0d0980 0x7f9a6c1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a63e00640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c099a00 0x7f9a6c0b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6b400640 1 -- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c099a00 msgr2=0x7f9a6c0b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6b400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c099a00 0x7f9a6c0b42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6b400640 1 -- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c0d0980 msgr2=0x7f9a6c1594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6b400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c0d0980 0x7f9a6c1594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6b400640 1 -- 172.21.15.46:0/3608472653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0b4980 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6be00640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c0d0980 0x7f9a6c1594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:34.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a6b400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c06c7c0 0x7f9a6c0b5a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0ae040 tx=0x7f9a5c0191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a63400640 1 -- 172.21.15.46:0/3608472653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0b9020 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a63400640 1 -- 172.21.15.46:0/3608472653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c0b1040 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6c0b4c60 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a63400640 1 -- 172.21.15.46:0/3608472653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c0b0050 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c0b5220 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6c069080 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.399+0000 7f9a63400640 1 -- 172.21.15.46:0/3608472653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a5c0bc020 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.403+0000 7f9a63400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a48066f90 0x7f9a48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.403+0000 7f9a63400640 1 -- 172.21.15.46:0/3608472653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9a5c11eef0 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.403+0000 7f9a63e00640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a48066f90 0x7f9a48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.403+0000 7f9a63e00640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a48066f90 0x7f9a48069450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c157290 tx=0x7f9a54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:34.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.407+0000 7f9a63400640 1 -- 172.21.15.46:0/3608472653 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a5c0c3030 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a6c0d23e0 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a63400640 1 -- 172.21.15.46:0/3608472653 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f9a5c016150 con 0x7f9a6c06c7c0 2024-04-25T08:42:34.564 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:34.565 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:34.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a48066f90 msgr2=0x7f9a48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a48066f90 0x7f9a48069450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c157290 tx=0x7f9a54009040 comp rx=0 tx=0).stop 2024-04-25T08:42:34.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c06c7c0 msgr2=0x7f9a6c0b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c06c7c0 0x7f9a6c0b5a20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0ae040 tx=0x7f9a5c0191e0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 shutdown_connections 2024-04-25T08:42:34.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a48066f90 0x7f9a48069450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a6c0d0980 0x7f9a6c1594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a6c099a00 0x7f9a6c0b42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 --2- 172.21.15.46:0/3608472653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a6c06c7c0 0x7f9a6c0b5a20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 >> 172.21.15.46:0/3608472653 conn(0x7f9a6c0d16b0 msgr2=0x7f9a6c06f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:34.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 shutdown_connections 2024-04-25T08:42:34.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.571+0000 7f9a72400640 1 -- 172.21.15.46:0/3608472653 wait complete. 2024-04-25T08:42:34.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:42:34.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:42:34.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.735+0000 7f7a10200640 1 Processor -- start 2024-04-25T08:42:34.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.735+0000 7f7a10200640 1 -- start start 2024-04-25T08:42:34.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.735+0000 7f7a10200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a0815dae0 0x7f7a08072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.735+0000 7f7a10200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806ea60 0x7f7a080acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.735+0000 7f7a10200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a080b3850 con 0x7f7a0806e620 2024-04-25T08:42:34.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.735+0000 7f7a10200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0806e190 con 0x7f7a0815dae0 2024-04-25T08:42:34.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.735+0000 7f7a10200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080b3af0 con 0x7f7a0806ea60 2024-04-25T08:42:34.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a0d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a0815dae0 0x7f7a08072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a0d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a0815dae0 0x7f7a08072600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34852/0 (socket says 172.21.15.46:34852) 2024-04-25T08:42:34.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a0d800640 1 -- 172.21.15.46:0/1205224420 learned_addr learned my addr 172.21.15.46:0/1205224420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:34.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a0e200640 1 --2- 172.21.15.46:0/1205224420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806ea60 0x7f7a080acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a0d800640 1 -- 172.21.15.46:0/1205224420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806ea60 msgr2=0x7f7a080acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:34.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a0d800640 1 --2- 172.21.15.46:0/1205224420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806ea60 0x7f7a080acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a0d800640 1 -- 172.21.15.46:0/1205224420 >> v1:172.21.15.82:6789/0 conn(0x7f7a0806e620 legacy=0x7f7a0806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a0d800640 1 -- 172.21.15.46:0/1205224420 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a080b3dd0 con 0x7f7a0815dae0 2024-04-25T08:42:34.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a0d800640 1 --2- 172.21.15.46:0/1205224420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0815dae0 0x7f7a08072600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a0806e310 tx=0x7f79fc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7611d2293fed2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:34.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a07e00640 1 -- 172.21.15.46:0/1205224420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0aa670 con 0x7f7a0815dae0 2024-04-25T08:42:34.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a07e00640 1 -- 172.21.15.46:0/1205224420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc0af070 con 0x7f7a0815dae0 2024-04-25T08:42:34.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.739+0000 7f7a07e00640 1 -- 172.21.15.46:0/1205224420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0aaa30 con 0x7f7a0815dae0 2024-04-25T08:42:34.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 -- 172.21.15.46:0/1205224420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0815dae0 msgr2=0x7f7a08072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 --2- 172.21.15.46:0/1205224420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0815dae0 0x7f7a08072600 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7a0806e310 tx=0x7f79fc01c510 comp rx=0 tx=0).stop 2024-04-25T08:42:34.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 -- 172.21.15.46:0/1205224420 shutdown_connections 2024-04-25T08:42:34.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 --2- 172.21.15.46:0/1205224420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806ea60 0x7f7a080acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 --2- 172.21.15.46:0/1205224420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0815dae0 0x7f7a08072600 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 -- 172.21.15.46:0/1205224420 >> 172.21.15.46:0/1205224420 conn(0x7f7a08071ae0 msgr2=0x7f7a08071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:34.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 -- 172.21.15.46:0/1205224420 shutdown_connections 2024-04-25T08:42:34.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 -- 172.21.15.46:0/1205224420 wait complete. 2024-04-25T08:42:34.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 Processor -- start 2024-04-25T08:42:34.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 -- start start 2024-04-25T08:42:34.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806e620 0x7f7a08143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0806ea60 0x7f7a08150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a0815dae0 0x7f7a0814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0806f6d0 con 0x7f7a0806ea60 2024-04-25T08:42:34.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0815dec0 con 0x7f7a0815dae0 2024-04-25T08:42:34.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a10200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0806f850 con 0x7f7a0806e620 2024-04-25T08:42:34.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0806ea60 0x7f7a08150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0806ea60 0x7f7a08150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34860/0 (socket says 172.21.15.46:34860) 2024-04-25T08:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0ce00640 1 -- 172.21.15.46:0/4036761393 learned_addr learned my addr 172.21.15.46:0/4036761393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0d800640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806e620 0x7f7a08143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0e200640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a0815dae0 0x7f7a0814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0ce00640 1 -- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806e620 msgr2=0x7f7a08143db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0ce00640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806e620 0x7f7a08143db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0ce00640 1 -- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a0815dae0 msgr2=0x7f7a0814af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0ce00640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a0815dae0 0x7f7a0814af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.743+0000 7f7a0ce00640 1 -- 172.21.15.46:0/4036761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a080b12c0 con 0x7f7a0806ea60 2024-04-25T08:42:34.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a0ce00640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0806ea60 0x7f7a08150da0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79f400c400 tx=0x7f79f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:34.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a05600640 1 -- 172.21.15.46:0/4036761393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4017070 con 0x7f7a0806ea60 2024-04-25T08:42:34.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a05600640 1 -- 172.21.15.46:0/4036761393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f4005310 con 0x7f7a0806ea60 2024-04-25T08:42:34.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a080b15a0 con 0x7f7a0806ea60 2024-04-25T08:42:34.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a05600640 1 -- 172.21.15.46:0/4036761393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f40054c0 con 0x7f7a0806ea60 2024-04-25T08:42:34.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a08147a70 con 0x7f7a0806ea60 2024-04-25T08:42:34.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a08069010 con 0x7f7a0806ea60 2024-04-25T08:42:34.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a05600640 1 -- 172.21.15.46:0/4036761393 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79f401a070 con 0x7f7a0806ea60 2024-04-25T08:42:34.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a05600640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79e8066f60 0x7f79e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.747+0000 7f7a05600640 1 -- 172.21.15.46:0/4036761393 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f79f40845e0 con 0x7f7a0806ea60 2024-04-25T08:42:34.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.751+0000 7f7a0d800640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79e8066f60 0x7f79e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.751+0000 7f7a0d800640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79e8066f60 0x7f79e8069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0ae000 tx=0x7f79fc0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:34.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.751+0000 7f7a05600640 1 -- 172.21.15.46:0/4036761393 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79f4054c50 con 0x7f7a0806ea60 2024-04-25T08:42:34.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a08152d90 con 0x7f7a0806ea60 2024-04-25T08:42:34.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a05600640 1 -- 172.21.15.46:0/4036761393 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79f4058cf0 con 0x7f7a0806ea60 2024-04-25T08:42:34.908 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:42:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79e8066f60 msgr2=0x7f79e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79e8066f60 0x7f79e8069420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0ae000 tx=0x7f79fc0a7000 comp rx=0 tx=0).stop 2024-04-25T08:42:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0806ea60 msgr2=0x7f7a08150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0806ea60 0x7f7a08150da0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79f400c400 tx=0x7f79f400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 shutdown_connections 2024-04-25T08:42:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f79e8066f60 0x7f79e8069420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a0815dae0 0x7f7a0814af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0806ea60 0x7f7a08150da0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 --2- 172.21.15.46:0/4036761393 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a0806e620 0x7f7a08143db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 >> 172.21.15.46:0/4036761393 conn(0x7f7a08071ae0 msgr2=0x7f7a0806baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 shutdown_connections 2024-04-25T08:42:34.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:34.915+0000 7f7a10200640 1 -- 172.21.15.46:0/4036761393 wait complete. 2024-04-25T08:42:34.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:42:35.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1c5600640 1 Processor -- start 2024-04-25T08:42:35.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1c5600640 1 -- start start 2024-04-25T08:42:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1c5600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 0x7fd1c006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1c5600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c00b3850 con 0x7fd1c015dae0 2024-04-25T08:42:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1c5600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c00b3af0 con 0x7fd1c006ea60 2024-04-25T08:42:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1c5600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c006e190 con 0x7fd1c006e620 2024-04-25T08:42:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1bea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd1c015dae0 0x7fd1c0072600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33308/0 (socket says 172.21.15.46:33308) 2024-04-25T08:42:35.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1bea00640 1 -- 172.21.15.46:0/679527126 learned_addr learned my addr 172.21.15.46:0/679527126 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1b7e00640 1 --2- 172.21.15.46:0/679527126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 0x7fd1c006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1be000640 1 -- 172.21.15.46:0/679527126 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167738034 0 0) 0x7fd1c00b3850 con 0x7fd1c015dae0 2024-04-25T08:42:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1be000640 1 -- 172.21.15.46:0/679527126 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1a4003530 con 0x7fd1c015dae0 2024-04-25T08:42:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1be000640 1 -- 172.21.15.46:0/679527126 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951455759 0 0) 0x7fd1c00b3af0 con 0x7fd1c006ea60 2024-04-25T08:42:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1be000640 1 -- 172.21.15.46:0/679527126 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1c00b3850 con 0x7fd1c006ea60 2024-04-25T08:42:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1be000640 1 -- 172.21.15.46:0/679527126 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3579615432 0 0) 0x7fd1a4003530 con 0x7fd1c015dae0 2024-04-25T08:42:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1b7e00640 1 -- 172.21.15.46:0/679527126 >> v1:172.21.15.82:6789/0 conn(0x7fd1c006ea60 legacy=0x7fd1c00acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1b7e00640 1 -- 172.21.15.46:0/679527126 >> v1:172.21.15.46:6789/0 conn(0x7fd1c015dae0 legacy=0x7fd1c0072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.079+0000 7fd1b7e00640 1 -- 172.21.15.46:0/679527126 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c00b3dd0 con 0x7fd1c006e620 2024-04-25T08:42:35.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1b7e00640 1 --2- 172.21.15.46:0/679527126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 0x7fd1c006bc20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80097f0 tx=0x7fd1a801c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d63dca9f9034cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1be000640 1 -- 172.21.15.46:0/679527126 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80b0070 con 0x7fd1c006e620 2024-04-25T08:42:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1be000640 1 -- 172.21.15.46:0/679527126 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a80ab400 con 0x7fd1c006e620 2024-04-25T08:42:35.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1be000640 1 -- 172.21.15.46:0/679527126 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0001680 con 0x7fd1c006e620 2024-04-25T08:42:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1c5600640 1 -- 172.21.15.46:0/679527126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 msgr2=0x7fd1c006bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1c5600640 1 --2- 172.21.15.46:0/679527126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 0x7fd1c006bc20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80097f0 tx=0x7fd1a801c3d0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1c5600640 1 -- 172.21.15.46:0/679527126 shutdown_connections 2024-04-25T08:42:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1c5600640 1 --2- 172.21.15.46:0/679527126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 0x7fd1c006bc20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1c5600640 1 -- 172.21.15.46:0/679527126 >> 172.21.15.46:0/679527126 conn(0x7fd1c0071ae0 msgr2=0x7fd1c0071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1c5600640 1 -- 172.21.15.46:0/679527126 shutdown_connections 2024-04-25T08:42:35.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.083+0000 7fd1c5600640 1 -- 172.21.15.46:0/679527126 wait complete. 2024-04-25T08:42:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 Processor -- start 2024-04-25T08:42:35.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 -- start start 2024-04-25T08:42:35.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 0x7fd1c014f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1c006ea60 0x7fd1c01513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c015dae0 0x7fd1c0147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c006f6d0 con 0x7fd1c015dae0 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c006c570 con 0x7fd1c006ea60 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c006f850 con 0x7fd1c006e620 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c015dae0 0x7fd1c0147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c015dae0 0x7fd1c0147180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34870/0 (socket says 172.21.15.46:34870) 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bf400640 1 -- 172.21.15.46:0/961778486 learned_addr learned my addr 172.21.15.46:0/961778486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1b7e00640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1c006ea60 0x7fd1c01513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bea00640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 0x7fd1c014f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bf400640 1 -- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 msgr2=0x7fd1c014f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bf400640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 0x7fd1c014f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bf400640 1 -- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1c006ea60 msgr2=0x7fd1c01513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bf400640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1c006ea60 0x7fd1c01513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bf400640 1 -- 172.21.15.46:0/961778486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c0149990 con 0x7fd1c015dae0 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1bf400640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c015dae0 0x7fd1c0147180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800c790 tx=0x7fd1b800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1b6a00640 1 -- 172.21.15.46:0/961778486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8003b80 con 0x7fd1c015dae0 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1b6a00640 1 -- 172.21.15.46:0/961778486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b8003d20 con 0x7fd1c015dae0 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c0149c70 con 0x7fd1c015dae0 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1b6a00640 1 -- 172.21.15.46:0/961778486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b800aa50 con 0x7fd1c015dae0 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.087+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c014a230 con 0x7fd1c015dae0 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.091+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c0069010 con 0x7fd1c015dae0 2024-04-25T08:42:35.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.091+0000 7fd1b6a00640 1 -- 172.21.15.46:0/961778486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd1b800abf0 con 0x7fd1c015dae0 2024-04-25T08:42:35.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.091+0000 7fd1b6a00640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd19c066f60 0x7fd19c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.091+0000 7fd1b6a00640 1 -- 172.21.15.46:0/961778486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd1b80852c0 con 0x7fd1c015dae0 2024-04-25T08:42:35.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.091+0000 7fd1bea00640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd19c066f60 0x7fd19c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.091+0000 7fd1bea00640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd19c066f60 0x7fd19c069420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00011f0 tx=0x7fd1b00016a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:35.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.095+0000 7fd1b6a00640 1 -- 172.21.15.46:0/961778486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd1b80558a0 con 0x7fd1c015dae0 2024-04-25T08:42:35.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.255+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1c00acf50 con 0x7fd1c015dae0 2024-04-25T08:42:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.255+0000 7fd1b6a00640 1 -- 172.21.15.46:0/961778486 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd1b8059940 con 0x7fd1c015dae0 2024-04-25T08:42:35.250 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd19c066f60 msgr2=0x7fd19c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd19c066f60 0x7fd19c069420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1b00011f0 tx=0x7fd1b00016a0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c015dae0 msgr2=0x7fd1c0147180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c015dae0 0x7fd1c0147180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd1b800c790 tx=0x7fd1b800cc50 comp rx=0 tx=0).stop 2024-04-25T08:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 shutdown_connections 2024-04-25T08:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd19c066f60 0x7fd19c069420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1c015dae0 0x7fd1c0147180 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1c006ea60 0x7fd1c01513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 --2- 172.21.15.46:0/961778486 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd1c006e620 0x7fd1c014f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 >> 172.21.15.46:0/961778486 conn(0x7fd1c0071ae0 msgr2=0x7fd1c00afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 shutdown_connections 2024-04-25T08:42:35.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.259+0000 7fd1c5600640 1 -- 172.21.15.46:0/961778486 wait complete. 2024-04-25T08:42:35.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:42:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4039c00640 1 Processor -- start 2024-04-25T08:42:35.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4039c00640 1 -- start start 2024-04-25T08:42:35.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4039c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40341548d0 0x7f40340b5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4039c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f40340b5860 0x7f403406c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4039c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403415e3b0 con 0x7f403415df10 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4039c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403415e690 con 0x7f40341548d0 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4039c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403415e970 con 0x7f40340b5860 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4032a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40341548d0 0x7f40340b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4032000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f403415df10 0x7f40340b3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33318/0 (socket says 172.21.15.46:33318) 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4032000640 1 -- 172.21.15.46:0/2496204421 learned_addr learned my addr 172.21.15.46:0/2496204421 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4033400640 1 --2- 172.21.15.46:0/2496204421 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f40340b5860 0x7f403406c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4032a00640 1 -- 172.21.15.46:0/2496204421 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f40340b5860 msgr2=0x7f403406c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4032a00640 1 --2- 172.21.15.46:0/2496204421 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f40340b5860 0x7f403406c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4032a00640 1 -- 172.21.15.46:0/2496204421 >> v1:172.21.15.46:6789/0 conn(0x7f403415df10 legacy=0x7f40340b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4032a00640 1 -- 172.21.15.46:0/2496204421 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403415ec50 con 0x7f40341548d0 2024-04-25T08:42:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4033400640 1 --2- 172.21.15.46:0/2496204421 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f40340b5860 0x7f403406c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:35.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.423+0000 7f4031600640 1 -- 172.21.15.46:0/2496204421 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182634323 0 0) 0x7f403415e3b0 con 0x7f403415df10 2024-04-25T08:42:35.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4032a00640 1 --2- 172.21.15.46:0/2496204421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40341548d0 0x7f40340b5320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f40240097c0 tx=0x7f402401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541c9122eeb6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4031600640 1 -- 172.21.15.46:0/2496204421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240b0070 con 0x7f40341548d0 2024-04-25T08:42:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4031600640 1 -- 172.21.15.46:0/2496204421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40240ab600 con 0x7f40341548d0 2024-04-25T08:42:35.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4031600640 1 -- 172.21.15.46:0/2496204421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240ab9d0 con 0x7f40341548d0 2024-04-25T08:42:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 -- 172.21.15.46:0/2496204421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40341548d0 msgr2=0x7f40340b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 --2- 172.21.15.46:0/2496204421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40341548d0 0x7f40340b5320 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f40240097c0 tx=0x7f402401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 -- 172.21.15.46:0/2496204421 shutdown_connections 2024-04-25T08:42:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 --2- 172.21.15.46:0/2496204421 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f40340b5860 0x7f403406c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 --2- 172.21.15.46:0/2496204421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40341548d0 0x7f40340b5320 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 -- 172.21.15.46:0/2496204421 >> 172.21.15.46:0/2496204421 conn(0x7f4034066b70 msgr2=0x7f403406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 -- 172.21.15.46:0/2496204421 shutdown_connections 2024-04-25T08:42:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 -- 172.21.15.46:0/2496204421 wait complete. 2024-04-25T08:42:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 Processor -- start 2024-04-25T08:42:35.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.427+0000 7f4039c00640 1 -- start start 2024-04-25T08:42:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4039c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40340b5860 0x7f4034146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4039c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40341548d0 0x7f40341472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4039c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403415df10 0x7f4034147820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4039c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403406d5a0 con 0x7f40340b5860 2024-04-25T08:42:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4039c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40340b3b70 con 0x7f40341548d0 2024-04-25T08:42:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4039c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403406d720 con 0x7f403415df10 2024-04-25T08:42:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40340b5860 0x7f4034146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40340b5860 0x7f4034146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34880/0 (socket says 172.21.15.46:34880) 2024-04-25T08:42:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032a00640 1 -- 172.21.15.46:0/3184101545 learned_addr learned my addr 172.21.15.46:0/3184101545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032000640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40341548d0 0x7f40341472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4033400640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403415df10 0x7f4034147820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032a00640 1 -- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403415df10 msgr2=0x7f4034147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032a00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403415df10 0x7f4034147820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032a00640 1 -- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40341548d0 msgr2=0x7f40341472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032a00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40341548d0 0x7f40341472e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032a00640 1 -- 172.21.15.46:0/3184101545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40341575d0 con 0x7f40340b5860 2024-04-25T08:42:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4032a00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40340b5860 0x7f4034146da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f402401a0d0 tx=0x7f402401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:35.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f402aa00640 1 -- 172.21.15.46:0/3184101545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240b0050 con 0x7f40340b5860 2024-04-25T08:42:35.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f402aa00640 1 -- 172.21.15.46:0/3184101545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40240bc070 con 0x7f40340b5860 2024-04-25T08:42:35.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f402aa00640 1 -- 172.21.15.46:0/3184101545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240aac00 con 0x7f40340b5860 2024-04-25T08:42:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40341578b0 con 0x7f40340b5860 2024-04-25T08:42:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40340d0070 con 0x7f40340b5860 2024-04-25T08:42:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.431+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4034069010 con 0x7f40340b5860 2024-04-25T08:42:35.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.435+0000 7f402aa00640 1 -- 172.21.15.46:0/3184101545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40240c1460 con 0x7f40340b5860 2024-04-25T08:42:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.435+0000 7f402aa00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4014066f90 0x7f4014069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.435+0000 7f402aa00640 1 -- 172.21.15.46:0/3184101545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f402411fd20 con 0x7f40340b5860 2024-04-25T08:42:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.435+0000 7f4032000640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4014066f90 0x7f4014069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.435+0000 7f4032000640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4014066f90 0x7f4014069450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f403414dfd0 tx=0x7f4020007df0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:35.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.439+0000 7f402aa00640 1 -- 172.21.15.46:0/3184101545 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4024016090 con 0x7f40340b5860 2024-04-25T08:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f403414a800 con 0x7f40340b5860 2024-04-25T08:42:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f402aa00640 1 -- 172.21.15.46:0/3184101545 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f40240f0300 con 0x7f40340b5860 2024-04-25T08:42:35.595 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:42:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4014066f90 msgr2=0x7f4014069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4014066f90 0x7f4014069450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f403414dfd0 tx=0x7f4020007df0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40340b5860 msgr2=0x7f4034146da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40340b5860 0x7f4034146da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f402401a0d0 tx=0x7f402401ace0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 shutdown_connections 2024-04-25T08:42:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4014066f90 0x7f4014069450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403415df10 0x7f4034147820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40341548d0 0x7f40341472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 --2- 172.21.15.46:0/3184101545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40340b5860 0x7f4034146da0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 >> 172.21.15.46:0/3184101545 conn(0x7f4034066b70 msgr2=0x7f40340cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 shutdown_connections 2024-04-25T08:42:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.603+0000 7f4039c00640 1 -- 172.21.15.46:0/3184101545 wait complete. 2024-04-25T08:42:35.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:42:35.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.763+0000 7fcac7800640 1 Processor -- start 2024-04-25T08:42:35.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.763+0000 7fcac7800640 1 -- start start 2024-04-25T08:42:35.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac7800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcac006f4d0 0x7fcac00cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac7800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 0x7fcac00cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac7800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac00d06f0 con 0x7fcac00d0280 2024-04-25T08:42:35.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac7800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac009a780 con 0x7fcac006f4d0 2024-04-25T08:42:35.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac7800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac009aa60 con 0x7fcac015dd50 2024-04-25T08:42:35.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac4e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcac006f4d0 0x7fcac00cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcabfe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 0x7fcac00cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac5800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcac00d0280 0x7fcac006f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:42738/0 (socket says 172.21.15.46:42738) 2024-04-25T08:42:35.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac5800640 1 -- 172.21.15.46:0/856930913 learned_addr learned my addr 172.21.15.46:0/856930913 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac4e00640 1 -- 172.21.15.46:0/856930913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 msgr2=0x7fcac00cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac4e00640 1 --2- 172.21.15.46:0/856930913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 0x7fcac00cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac4e00640 1 -- 172.21.15.46:0/856930913 >> v1:172.21.15.82:6789/0 conn(0x7fcac00d0280 legacy=0x7fcac006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.767+0000 7fcac4e00640 1 -- 172.21.15.46:0/856930913 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac009ad40 con 0x7fcac006f4d0 2024-04-25T08:42:35.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac4e00640 1 --2- 172.21.15.46:0/856930913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac006f4d0 0x7fcac00cf800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcab40097c0 tx=0x7fcab401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6afc95a8ad9a3231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:35.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcabf400640 1 -- 172.21.15.46:0/856930913 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab40a7070 con 0x7fcac006f4d0 2024-04-25T08:42:35.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcabf400640 1 -- 172.21.15.46:0/856930913 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab40b0070 con 0x7fcac006f4d0 2024-04-25T08:42:35.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcabf400640 1 -- 172.21.15.46:0/856930913 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab40ab890 con 0x7fcac006f4d0 2024-04-25T08:42:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 -- 172.21.15.46:0/856930913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac006f4d0 msgr2=0x7fcac00cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 --2- 172.21.15.46:0/856930913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac006f4d0 0x7fcac00cf800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcab40097c0 tx=0x7fcab401c470 comp rx=0 tx=0).stop 2024-04-25T08:42:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 -- 172.21.15.46:0/856930913 shutdown_connections 2024-04-25T08:42:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 --2- 172.21.15.46:0/856930913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 0x7fcac00cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 --2- 172.21.15.46:0/856930913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac006f4d0 0x7fcac00cf800 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 -- 172.21.15.46:0/856930913 >> 172.21.15.46:0/856930913 conn(0x7fcac00b3160 msgr2=0x7fcac00b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 -- 172.21.15.46:0/856930913 shutdown_connections 2024-04-25T08:42:35.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 -- 172.21.15.46:0/856930913 wait complete. 2024-04-25T08:42:35.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 Processor -- start 2024-04-25T08:42:35.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.771+0000 7fcac7800640 1 -- start start 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac006f4d0 0x7fcac015b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac7800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac00d0280 0x7fcac015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac7800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 0x7fcac015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac016a820 con 0x7fcac006f4d0 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac7800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac016a6a0 con 0x7fcac00d0280 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac7800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac016a9a0 con 0x7fcac015dd50 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac006f4d0 0x7fcac015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac006f4d0 0x7fcac015b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34890/0 (socket says 172.21.15.46:34890) 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac4e00640 1 -- 172.21.15.46:0/225568138 learned_addr learned my addr 172.21.15.46:0/225568138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac5800640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 0x7fcac015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcabfe00640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac00d0280 0x7fcac015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac4e00640 1 -- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 msgr2=0x7fcac015bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac4e00640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 0x7fcac015bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac4e00640 1 -- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac00d0280 msgr2=0x7fcac015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac4e00640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac00d0280 0x7fcac015ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac4e00640 1 -- 172.21.15.46:0/225568138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac00b1d30 con 0x7fcac006f4d0 2024-04-25T08:42:35.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcabfe00640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac00d0280 0x7fcac015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:35.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac4e00640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac006f4d0 0x7fcac015b500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcab40ad040 tx=0x7fcab401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcabcc00640 1 -- 172.21.15.46:0/225568138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab40bf610 con 0x7fcac006f4d0 2024-04-25T08:42:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcabcc00640 1 -- 172.21.15.46:0/225568138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab40bf7d0 con 0x7fcac006f4d0 2024-04-25T08:42:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac00b2010 con 0x7fcac006f4d0 2024-04-25T08:42:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcabcc00640 1 -- 172.21.15.46:0/225568138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab40bfbc0 con 0x7fcac006f4d0 2024-04-25T08:42:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac0153750 con 0x7fcac006f4d0 2024-04-25T08:42:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0069010 con 0x7fcac006f4d0 2024-04-25T08:42:35.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.775+0000 7fcabcc00640 1 -- 172.21.15.46:0/225568138 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcab40bd020 con 0x7fcac006f4d0 2024-04-25T08:42:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.779+0000 7fcabcc00640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcaa0066f60 0x7fcaa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.779+0000 7fcabcc00640 1 -- 172.21.15.46:0/225568138 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcab4120040 con 0x7fcac006f4d0 2024-04-25T08:42:35.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.779+0000 7fcabfe00640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcaa0066f60 0x7fcaa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:35.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.779+0000 7fcabfe00640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcaa0066f60 0x7fcaa0069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcac00b7700 tx=0x7fcaac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:35.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.783+0000 7fcabcc00640 1 -- 172.21.15.46:0/225568138 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcab4016090 con 0x7fcac006f4d0 2024-04-25T08:42:35.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.939+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcac01539e0 con 0x7fcac006f4d0 2024-04-25T08:42:35.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcabcc00640 1 -- 172.21.15.46:0/225568138 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcab40f0d90 con 0x7fcac006f4d0 2024-04-25T08:42:35.935 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:42:35.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcaa0066f60 msgr2=0x7fcaa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcaa0066f60 0x7fcaa0069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcac00b7700 tx=0x7fcaac008040 comp rx=0 tx=0).stop 2024-04-25T08:42:35.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac006f4d0 msgr2=0x7fcac015b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac006f4d0 0x7fcac015b500 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcab40ad040 tx=0x7fcab401acb0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 shutdown_connections 2024-04-25T08:42:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcaa0066f60 0x7fcaa0069420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcac015dd50 0x7fcac015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcac00d0280 0x7fcac015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 --2- 172.21.15.46:0/225568138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac006f4d0 0x7fcac015b500 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 >> 172.21.15.46:0/225568138 conn(0x7fcac00b3160 msgr2=0x7fcac0157020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 shutdown_connections 2024-04-25T08:42:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:35.943+0000 7fcac7800640 1 -- 172.21.15.46:0/225568138 wait complete. 2024-04-25T08:42:35.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:42:36.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.103+0000 7fa46c800640 1 Processor -- start 2024-04-25T08:42:36.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46c800640 1 -- start start 2024-04-25T08:42:36.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46c800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa4641611d0 0x7fa46406c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46c800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 0x7fa4640cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46c800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4640cc290 0x7fa4640d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46c800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4640cfba0 con 0x7fa4640cc290 2024-04-25T08:42:36.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46c800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4640cfe80 con 0x7fa4641611d0 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46c800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4640d0160 con 0x7fa46406cc70 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4640cc290 0x7fa4640d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4640cc290 0x7fa4640d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34898/0 (socket says 172.21.15.46:34898) 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46a800640 1 -- 172.21.15.46:0/2221465001 learned_addr learned my addr 172.21.15.46:0/2221465001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa469e00640 1 --2- 172.21.15.46:0/2221465001 >> v2:172.21.15.82:3300/0 conn(0x7fa4641611d0 0x7fa46406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa469400640 1 --2- 172.21.15.46:0/2221465001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 0x7fa4640cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46a800640 1 -- 172.21.15.46:0/2221465001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 msgr2=0x7fa4640cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46a800640 1 --2- 172.21.15.46:0/2221465001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 0x7fa4640cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46a800640 1 -- 172.21.15.46:0/2221465001 >> v2:172.21.15.82:3300/0 conn(0x7fa4641611d0 msgr2=0x7fa46406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46a800640 1 --2- 172.21.15.46:0/2221465001 >> v2:172.21.15.82:3300/0 conn(0x7fa4641611d0 0x7fa46406c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa46a800640 1 -- 172.21.15.46:0/2221465001 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4640d0440 con 0x7fa4640cc290 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa469e00640 1 --2- 172.21.15.46:0/2221465001 >> v2:172.21.15.82:3300/0 conn(0x7fa4641611d0 0x7fa46406c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:36.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.107+0000 7fa469400640 1 --2- 172.21.15.46:0/2221465001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 0x7fa4640cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:36.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46a800640 1 --2- 172.21.15.46:0/2221465001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 0x7fa4640d0960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa464072760 tx=0x7fa46001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cf375c3f3e289e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa468a00640 1 -- 172.21.15.46:0/2221465001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4600a56b0 con 0x7fa4640cc290 2024-04-25T08:42:36.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa468a00640 1 -- 172.21.15.46:0/2221465001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4600a5850 con 0x7fa4640cc290 2024-04-25T08:42:36.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa468a00640 1 -- 172.21.15.46:0/2221465001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4600a5c20 con 0x7fa4640cc290 2024-04-25T08:42:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 -- 172.21.15.46:0/2221465001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 msgr2=0x7fa4640d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 --2- 172.21.15.46:0/2221465001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 0x7fa4640d0960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa464072760 tx=0x7fa46001c960 comp rx=0 tx=0).stop 2024-04-25T08:42:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 -- 172.21.15.46:0/2221465001 shutdown_connections 2024-04-25T08:42:36.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 --2- 172.21.15.46:0/2221465001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 0x7fa4640d0960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 --2- 172.21.15.46:0/2221465001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 0x7fa4640cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 --2- 172.21.15.46:0/2221465001 >> v2:172.21.15.82:3300/0 conn(0x7fa4641611d0 0x7fa46406c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 -- 172.21.15.46:0/2221465001 >> 172.21.15.46:0/2221465001 conn(0x7fa4640d1700 msgr2=0x7fa4640d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 -- 172.21.15.46:0/2221465001 shutdown_connections 2024-04-25T08:42:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 -- 172.21.15.46:0/2221465001 wait complete. 2024-04-25T08:42:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 Processor -- start 2024-04-25T08:42:36.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.111+0000 7fa46c800640 1 -- start start 2024-04-25T08:42:36.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46c800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 0x7fa4640b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 0x7fa464155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46c800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4641611d0 0x7fa464153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464161880 con 0x7fa4640cc290 2024-04-25T08:42:36.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46c800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46409a860 con 0x7fa4641611d0 2024-04-25T08:42:36.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46c800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa464161a00 con 0x7fa46406cc70 2024-04-25T08:42:36.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 0x7fa464155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 0x7fa464155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34908/0 (socket says 172.21.15.46:34908) 2024-04-25T08:42:36.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 0x7fa4640b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46a800640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4641611d0 0x7fa464153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469400640 1 -- 172.21.15.46:0/530686808 learned_addr learned my addr 172.21.15.46:0/530686808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469400640 1 -- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 msgr2=0x7fa4640b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469400640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 0x7fa4640b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469400640 1 -- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4641611d0 msgr2=0x7fa464153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469400640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4641611d0 0x7fa464153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469400640 1 -- 172.21.15.46:0/530686808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4641565b0 con 0x7fa4640cc290 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa469400640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 0x7fa464155140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa45400c7c0 tx=0x7fa45400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa45e000640 1 -- 172.21.15.46:0/530686808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454015070 con 0x7fa4640cc290 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa45e000640 1 -- 172.21.15.46:0/530686808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa454007a30 con 0x7fa4640cc290 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa45e000640 1 -- 172.21.15.46:0/530686808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa454007e00 con 0x7fa4640cc290 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4640b53d0 con 0x7fa4640cc290 2024-04-25T08:42:36.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4640b5c70 con 0x7fa4640cc290 2024-04-25T08:42:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4640b6730 con 0x7fa4640cc290 2024-04-25T08:42:36.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.115+0000 7fa45e000640 1 -- 172.21.15.46:0/530686808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa454018070 con 0x7fa4640cc290 2024-04-25T08:42:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.119+0000 7fa45e000640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa448066f90 0x7fa448069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.119+0000 7fa45e000640 1 -- 172.21.15.46:0/530686808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa4540851c0 con 0x7fa4640cc290 2024-04-25T08:42:36.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.119+0000 7fa469e00640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa448066f90 0x7fa448069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.119+0000 7fa469e00640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa448066f90 0x7fa448069450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa458006fd0 tx=0x7fa458008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.123+0000 7fa45e000640 1 -- 172.21.15.46:0/530686808 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa454055840 con 0x7fa4640cc290 2024-04-25T08:42:36.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa4640b5e40 con 0x7fa4640cc290 2024-04-25T08:42:36.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa45e000640 1 -- 172.21.15.46:0/530686808 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa4540598e0 con 0x7fa4640cc290 2024-04-25T08:42:36.288 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:42:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa448066f90 msgr2=0x7fa448069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa448066f90 0x7fa448069450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa458006fd0 tx=0x7fa458008040 comp rx=0 tx=0).stop 2024-04-25T08:42:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 msgr2=0x7fa464155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 0x7fa464155140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa45400c7c0 tx=0x7fa45400cc80 comp rx=0 tx=0).stop 2024-04-25T08:42:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 shutdown_connections 2024-04-25T08:42:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa448066f90 0x7fa448069450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa4641611d0 0x7fa464153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4640cc290 0x7fa464155140 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 --2- 172.21.15.46:0/530686808 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa46406cc70 0x7fa4640b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 >> 172.21.15.46:0/530686808 conn(0x7fa4640d1700 msgr2=0x7fa4640cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:36.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 shutdown_connections 2024-04-25T08:42:36.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.295+0000 7fa46c800640 1 -- 172.21.15.46:0/530686808 wait complete. 2024-04-25T08:42:36.309 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-04-25T08:42:36.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:36.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:42:36.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi046.stderr:ceph-fuse[32522]: fuse finished with error 0 and tester_r 0 2024-04-25T08:42:36.443 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:42:36.489 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:42:36.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:36.490 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T08:42:36.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea2800640 1 Processor -- start 2024-04-25T08:42:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea2800640 1 -- start start 2024-04-25T08:42:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e9c15ec80 0x7f0e9c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0e9c0cf4d0 0x7f0e9c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea2800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9c0d0530 con 0x7f0e9c099f40 2024-04-25T08:42:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9c09a3b0 con 0x7f0e9c0cf4d0 2024-04-25T08:42:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c15f060 con 0x7f0e9c15ec80 2024-04-25T08:42:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0e9c099f40 0x7f0e9c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33320/0 (socket says 172.21.15.46:33320) 2024-04-25T08:42:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea1e00640 1 -- 172.21.15.46:0/3913895420 learned_addr learned my addr 172.21.15.46:0/3913895420 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302905007 0 0) 0x7f0e940014a0 con 0x7f0e9c099f40 2024-04-25T08:42:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e84003530 con 0x7f0e9c099f40 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea0a00640 1 --2- 172.21.15.46:0/3913895420 >> v2:172.21.15.82:3300/0 conn(0x7f0e9c0cf4d0 0x7f0e9c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea1400640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e9c15ec80 0x7f0e9c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2046971237 0 0) 0x7f0e84003530 con 0x7f0e9c099f40 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e9c15ec80 msgr2=0x7f0e9c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e9c15ec80 0x7f0e9c15fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 >> v2:172.21.15.82:3300/0 conn(0x7f0e9c0cf4d0 msgr2=0x7f0e9c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 --2- 172.21.15.46:0/3913895420 >> v2:172.21.15.82:3300/0 conn(0x7f0e9c0cf4d0 0x7f0e9c099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c0cc020 con 0x7f0e9c099f40 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f0e94001bd0 con 0x7f0e9c099f40 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea0a00640 1 --2- 172.21.15.46:0/3913895420 >> v2:172.21.15.82:3300/0 conn(0x7f0e9c0cf4d0 0x7f0e9c099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea1400640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e9c15ec80 0x7f0e9c15fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 >> v1:172.21.15.46:6789/0 conn(0x7f0e9c099f40 legacy=0x7f0e9c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e9c15ec80 0x7f0e840039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e84004000 0x7f0e84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e84004d50 0x7f0e840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c09a3b0 con 0x7f0e84004000 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c15f060 con 0x7f0e84004d50 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8400bb90 con 0x7f0e9c15ec80 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea0a00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e84004000 0x7f0e84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea1400640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e9c15ec80 0x7f0e840039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea1e00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e84004d50 0x7f0e840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea0a00640 1 -- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e9c15ec80 msgr2=0x7f0e840039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea0a00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e9c15ec80 0x7f0e840039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea0a00640 1 -- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e84004d50 msgr2=0x7f0e840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea0a00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e84004d50 0x7f0e840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea0a00640 1 -- 172.21.15.46:0/3913895420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e84003530 con 0x7f0e84004000 2024-04-25T08:42:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.783+0000 7f0ea1e00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e84004d50 0x7f0e840051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0ea0a00640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e84004000 0x7f0e84004810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0e880098d0 tx=0x7f0e8801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2ac3dbd0989e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8800f040 con 0x7f0e84004000 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e880a9650 con 0x7f0e84004000 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0e9be00640 1 -- 172.21.15.46:0/3913895420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e880a9a00 con 0x7f0e84004000 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0ea2800640 1 -- 172.21.15.46:0/3913895420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e84004000 msgr2=0x7f0e84004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0ea2800640 1 --2- 172.21.15.46:0/3913895420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e84004000 0x7f0e84004810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0e880098d0 tx=0x7f0e8801c560 comp rx=0 tx=0).stop 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0ea1e00640 1 -- 172.21.15.46:0/3913895420 reap_dead start 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0ea2800640 1 -- 172.21.15.46:0/3913895420 shutdown_connections 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0ea2800640 1 -- 172.21.15.46:0/3913895420 >> 172.21.15.46:0/3913895420 conn(0x7f0e9c0b2f90 msgr2=0x7f0e9c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0ea2800640 1 -- 172.21.15.46:0/3913895420 shutdown_connections 2024-04-25T08:42:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.787+0000 7f0ea2800640 1 -- 172.21.15.46:0/3913895420 wait complete. 2024-04-25T08:42:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 Processor -- start 2024-04-25T08:42:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 -- start start 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e84004000 0x7f0e9c1f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e9c154750 0x7f0e9c147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e9c1ea5a0 0x7f0e9c0c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c0d0250 con 0x7f0e9c154750 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c0cff70 con 0x7f0e9c1ea5a0 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c1600a0 con 0x7f0e84004000 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e9c154750 0x7f0e9c147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e9c154750 0x7f0e9c147760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34930/0 (socket says 172.21.15.46:34930) 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea0a00640 1 -- 172.21.15.46:0/1143932469 learned_addr learned my addr 172.21.15.46:0/1143932469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea1400640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e84004000 0x7f0e9c1f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea1e00640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e9c1ea5a0 0x7f0e9c0c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea0a00640 1 -- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e84004000 msgr2=0x7f0e9c1f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea0a00640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e84004000 0x7f0e9c1f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea0a00640 1 -- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e9c1ea5a0 msgr2=0x7f0e9c0c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea0a00640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e9c1ea5a0 0x7f0e9c0c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea0a00640 1 -- 172.21.15.46:0/1143932469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c1e9500 con 0x7f0e9c154750 2024-04-25T08:42:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea0a00640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e9c154750 0x7f0e9c147760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e8801c390 tx=0x7f0e88019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0e99600640 1 -- 172.21.15.46:0/1143932469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e880c1070 con 0x7f0e9c154750 2024-04-25T08:42:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0e99600640 1 -- 172.21.15.46:0/1143932469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e880bc3f0 con 0x7f0e9c154750 2024-04-25T08:42:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9c1e97e0 con 0x7f0e9c154750 2024-04-25T08:42:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0e99600640 1 -- 172.21.15.46:0/1143932469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e880bc7c0 con 0x7f0e9c154750 2024-04-25T08:42:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.791+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9c1e9da0 con 0x7f0e9c154750 2024-04-25T08:42:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.795+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9c069010 con 0x7f0e9c154750 2024-04-25T08:42:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.795+0000 7f0e99600640 1 -- 172.21.15.46:0/1143932469 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e880c4070 con 0x7f0e9c154750 2024-04-25T08:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.795+0000 7f0e99600640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e7c066f90 0x7f0e7c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.795+0000 7f0e99600640 1 -- 172.21.15.46:0/1143932469 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0e8811fb30 con 0x7f0e9c154750 2024-04-25T08:42:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.795+0000 7f0ea1400640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e7c066f90 0x7f0e7c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.795+0000 7f0ea1400640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e7c066f90 0x7f0e7c069450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e90006f90 tx=0x7f0e90008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.799+0000 7f0e99600640 1 -- 172.21.15.46:0/1143932469 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e880ad020 con 0x7f0e9c154750 2024-04-25T08:42:36.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.967+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e9c0cb560 con 0x7f0e9c154750 2024-04-25T08:42:36.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.967+0000 7f0e99600640 1 -- 172.21.15.46:0/1143932469 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f0e880162a0 con 0x7f0e9c154750 2024-04-25T08:42:36.962 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:36.962 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e7c066f90 msgr2=0x7f0e7c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e7c066f90 0x7f0e7c069450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0e90006f90 tx=0x7f0e90008040 comp rx=0 tx=0).stop 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e9c154750 msgr2=0x7f0e9c147760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e9c154750 0x7f0e9c147760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e8801c390 tx=0x7f0e88019300 comp rx=0 tx=0).stop 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 shutdown_connections 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e7c066f90 0x7f0e7c069450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e9c1ea5a0 0x7f0e9c0c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e9c154750 0x7f0e9c147760 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 --2- 172.21.15.46:0/1143932469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e84004000 0x7f0e9c1f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 >> 172.21.15.46:0/1143932469 conn(0x7f0e9c0b2f90 msgr2=0x7f0e9c15f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 shutdown_connections 2024-04-25T08:42:36.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:36.971+0000 7f0ea2800640 1 -- 172.21.15.46:0/1143932469 wait complete. 2024-04-25T08:42:36.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:37.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.131+0000 7fc77f800640 1 Processor -- start 2024-04-25T08:42:37.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.131+0000 7fc77f800640 1 -- start start 2024-04-25T08:42:37.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc77f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 0x7fc7780cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc77f800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7780cc700 con 0x7fc7780cda80 2024-04-25T08:42:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc77f800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7780cc9e0 con 0x7fc778159230 2024-04-25T08:42:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc77f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780cccc0 con 0x7fc77815bdb0 2024-04-25T08:42:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc77d800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc7780cda80 0x7fc77815d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33332/0 (socket says 172.21.15.46:33332) 2024-04-25T08:42:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc77d800640 1 -- 172.21.15.46:0/2122362583 learned_addr learned my addr 172.21.15.46:0/2122362583 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc76fe00640 1 --2- 172.21.15.46:0/2122362583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 0x7fc7780cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc777e00640 1 -- 172.21.15.46:0/2122362583 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378017995 0 0) 0x7fc7780cc700 con 0x7fc7780cda80 2024-04-25T08:42:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc777e00640 1 -- 172.21.15.46:0/2122362583 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc75c003530 con 0x7fc7780cda80 2024-04-25T08:42:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc777e00640 1 -- 172.21.15.46:0/2122362583 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 361626225 0 0) 0x7fc75c003530 con 0x7fc7780cda80 2024-04-25T08:42:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc76fe00640 1 -- 172.21.15.46:0/2122362583 >> v1:172.21.15.82:6789/0 conn(0x7fc778159230 legacy=0x7fc7780cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc76fe00640 1 -- 172.21.15.46:0/2122362583 >> v1:172.21.15.46:6789/0 conn(0x7fc7780cda80 legacy=0x7fc77815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc76fe00640 1 -- 172.21.15.46:0/2122362583 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7780ccfa0 con 0x7fc77815bdb0 2024-04-25T08:42:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.135+0000 7fc76fe00640 1 --2- 172.21.15.46:0/2122362583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 0x7fc7780cd540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097f0 tx=0x7fc76001c220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db0030a866f7bb56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc777e00640 1 -- 172.21.15.46:0/2122362583 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7600b0070 con 0x7fc77815bdb0 2024-04-25T08:42:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc777e00640 1 -- 172.21.15.46:0/2122362583 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7600ab400 con 0x7fc77815bdb0 2024-04-25T08:42:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc777e00640 1 -- 172.21.15.46:0/2122362583 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770001a00 con 0x7fc77815bdb0 2024-04-25T08:42:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 -- 172.21.15.46:0/2122362583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 msgr2=0x7fc7780cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 --2- 172.21.15.46:0/2122362583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 0x7fc7780cd540 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097f0 tx=0x7fc76001c220 comp rx=0 tx=0).stop 2024-04-25T08:42:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 -- 172.21.15.46:0/2122362583 shutdown_connections 2024-04-25T08:42:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 --2- 172.21.15.46:0/2122362583 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 0x7fc7780cd540 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 -- 172.21.15.46:0/2122362583 >> 172.21.15.46:0/2122362583 conn(0x7fc7780b12a0 msgr2=0x7fc7780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 -- 172.21.15.46:0/2122362583 shutdown_connections 2024-04-25T08:42:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 -- 172.21.15.46:0/2122362583 wait complete. 2024-04-25T08:42:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 Processor -- start 2024-04-25T08:42:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 -- start start 2024-04-25T08:42:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7780cda80 0x7fc7781fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc778159230 0x7fc7780ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 0x7fc77813da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77815e4b0 con 0x7fc778159230 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77815e330 con 0x7fc7780cda80 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc77f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc778169790 con 0x7fc77815bdb0 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc76fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc778159230 0x7fc7780ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc76fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc778159230 0x7fc7780ae1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34938/0 (socket says 172.21.15.46:34938) 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.139+0000 7fc76fe00640 1 -- 172.21.15.46:0/2289928876 learned_addr learned my addr 172.21.15.46:0/2289928876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc77ce00640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7780cda80 0x7fc7781fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc77d800640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 0x7fc77813da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc76fe00640 1 -- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 msgr2=0x7fc77813da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc76fe00640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 0x7fc77813da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc76fe00640 1 -- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7780cda80 msgr2=0x7fc7781fa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc76fe00640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7780cda80 0x7fc7781fa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc76fe00640 1 -- 172.21.15.46:0/2289928876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778141350 con 0x7fc778159230 2024-04-25T08:42:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc76fe00640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc778159230 0x7fc7780ae1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097f0 tx=0x7fc76001aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc775600640 1 -- 172.21.15.46:0/2289928876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7600bb020 con 0x7fc778159230 2024-04-25T08:42:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc775600640 1 -- 172.21.15.46:0/2289928876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7600b1040 con 0x7fc778159230 2024-04-25T08:42:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc775600640 1 -- 172.21.15.46:0/2289928876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7600b0050 con 0x7fc778159230 2024-04-25T08:42:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc778141630 con 0x7fc778159230 2024-04-25T08:42:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc778141bf0 con 0x7fc778159230 2024-04-25T08:42:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc778069080 con 0x7fc778159230 2024-04-25T08:42:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc775600640 1 -- 172.21.15.46:0/2289928876 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7600be070 con 0x7fc778159230 2024-04-25T08:42:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc775600640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc754066f60 0x7fc754069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.143+0000 7fc775600640 1 -- 172.21.15.46:0/2289928876 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc76011ea50 con 0x7fc778159230 2024-04-25T08:42:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.147+0000 7fc77ce00640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc754066f60 0x7fc754069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.147+0000 7fc77ce00640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc754066f60 0x7fc754069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc768007fd0 tx=0x7fc768009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:37.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.147+0000 7fc775600640 1 -- 172.21.15.46:0/2289928876 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc760015ff0 con 0x7fc778159230 2024-04-25T08:42:37.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.311+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc778167560 con 0x7fc778159230 2024-04-25T08:42:37.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.311+0000 7fc775600640 1 -- 172.21.15.46:0/2289928876 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7fc7600ef030 con 0x7fc778159230 2024-04-25T08:42:37.306 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:37.306 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:37.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc754066f60 msgr2=0x7fc754069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc754066f60 0x7fc754069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc768007fd0 tx=0x7fc768009040 comp rx=0 tx=0).stop 2024-04-25T08:42:37.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc778159230 msgr2=0x7fc7780ae1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc778159230 0x7fc7780ae1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc7600097f0 tx=0x7fc76001aa60 comp rx=0 tx=0).stop 2024-04-25T08:42:37.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 shutdown_connections 2024-04-25T08:42:37.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc754066f60 0x7fc754069420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77815bdb0 0x7fc77813da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc778159230 0x7fc7780ae1c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 --2- 172.21.15.46:0/2289928876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7780cda80 0x7fc7781fa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 >> 172.21.15.46:0/2289928876 conn(0x7fc7780b12a0 msgr2=0x7fc77815d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:37.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 shutdown_connections 2024-04-25T08:42:37.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.315+0000 7fc77f800640 1 -- 172.21.15.46:0/2289928876 wait complete. 2024-04-25T08:42:37.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:42:37.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.479+0000 7f9d90200640 1 Processor -- start 2024-04-25T08:42:37.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.479+0000 7f9d90200640 1 -- start start 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.479+0000 7f9d90200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9d88099a00 0x7f9d88099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.479+0000 7f9d90200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d8815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.479+0000 7f9d90200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8809a320 con 0x7f9d8809a7c0 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.479+0000 7f9d90200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8806cd30 con 0x7f9d88099a00 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.479+0000 7f9d90200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880cf8a0 con 0x7f9d8809ac20 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.479+0000 7f9d8ce00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9d8809a7c0 0x7f9d8815df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33334/0 (socket says 172.21.15.46:33334) 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.479+0000 7f9d8ce00640 1 -- 172.21.15.46:0/294186536 learned_addr learned my addr 172.21.15.46:0/294186536 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d8d800640 1 --2- 172.21.15.46:0/294186536 >> v2:172.21.15.82:3300/0 conn(0x7f9d88099a00 0x7f9d88099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d8e200640 1 --2- 172.21.15.46:0/294186536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d8815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d87e00640 1 -- 172.21.15.46:0/294186536 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469344658 0 0) 0x7f9d8809a320 con 0x7f9d8809a7c0 2024-04-25T08:42:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d87e00640 1 -- 172.21.15.46:0/294186536 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d70003530 con 0x7f9d8809a7c0 2024-04-25T08:42:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d87e00640 1 -- 172.21.15.46:0/294186536 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1146267093 0 0) 0x7f9d70003530 con 0x7f9d8809a7c0 2024-04-25T08:42:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d8d800640 1 -- 172.21.15.46:0/294186536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 msgr2=0x7f9d8815e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d8d800640 1 --2- 172.21.15.46:0/294186536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d8815e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d8d800640 1 -- 172.21.15.46:0/294186536 >> v1:172.21.15.46:6789/0 conn(0x7f9d8809a7c0 legacy=0x7f9d8815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d8d800640 1 -- 172.21.15.46:0/294186536 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880cfb80 con 0x7f9d88099a00 2024-04-25T08:42:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d8e200640 1 --2- 172.21.15.46:0/294186536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d8815e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:37.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.483+0000 7f9d8d800640 1 --2- 172.21.15.46:0/294186536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d88099a00 0x7f9d88099de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0097c0 tx=0x7f9d7c01c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14936546df6ef248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:37.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d87e00640 1 -- 172.21.15.46:0/294186536 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0b0070 con 0x7f9d88099a00 2024-04-25T08:42:37.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d87e00640 1 -- 172.21.15.46:0/294186536 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c0ab400 con 0x7f9d88099a00 2024-04-25T08:42:37.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d87e00640 1 -- 172.21.15.46:0/294186536 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74001680 con 0x7f9d88099a00 2024-04-25T08:42:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 -- 172.21.15.46:0/294186536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d88099a00 msgr2=0x7f9d88099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 --2- 172.21.15.46:0/294186536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d88099a00 0x7f9d88099de0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0097c0 tx=0x7f9d7c01c390 comp rx=0 tx=0).stop 2024-04-25T08:42:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 -- 172.21.15.46:0/294186536 shutdown_connections 2024-04-25T08:42:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 --2- 172.21.15.46:0/294186536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d8815e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 --2- 172.21.15.46:0/294186536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d88099a00 0x7f9d88099de0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 -- 172.21.15.46:0/294186536 >> 172.21.15.46:0/294186536 conn(0x7f9d880b3130 msgr2=0x7f9d880b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:37.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 -- 172.21.15.46:0/294186536 shutdown_connections 2024-04-25T08:42:37.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 -- 172.21.15.46:0/294186536 wait complete. 2024-04-25T08:42:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 Processor -- start 2024-04-25T08:42:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 -- start start 2024-04-25T08:42:37.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d88099a00 0x7f9d880c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d8809a7c0 0x7f9d881541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d880af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880d0940 con 0x7f9d8809a7c0 2024-04-25T08:42:37.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880b06f0 con 0x7f9d88099a00 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d90200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880c79a0 con 0x7f9d8809ac20 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d8ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d8809a7c0 0x7f9d881541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d8ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d8809a7c0 0x7f9d881541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34940/0 (socket says 172.21.15.46:34940) 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.487+0000 7f9d8ce00640 1 -- 172.21.15.46:0/2261095797 learned_addr learned my addr 172.21.15.46:0/2261095797 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d8d800640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d88099a00 0x7f9d880c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d8e200640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d880af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d8ce00640 1 -- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 msgr2=0x7f9d880af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d8ce00640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d880af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d8ce00640 1 -- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d88099a00 msgr2=0x7f9d880c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d8ce00640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d88099a00 0x7f9d880c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d8ce00640 1 -- 172.21.15.46:0/2261095797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880c5670 con 0x7f9d8809a7c0 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d8e200640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d880af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d8ce00640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d8809a7c0 0x7f9d881541d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400c780 tx=0x7f9d7400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:37.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d85600640 1 -- 172.21.15.46:0/2261095797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74005ae0 con 0x7f9d8809a7c0 2024-04-25T08:42:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d85600640 1 -- 172.21.15.46:0/2261095797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d74005ca0 con 0x7f9d8809a7c0 2024-04-25T08:42:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d880c7070 con 0x7f9d8809a7c0 2024-04-25T08:42:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d85600640 1 -- 172.21.15.46:0/2261095797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d740072b0 con 0x7f9d8809a7c0 2024-04-25T08:42:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d880c7630 con 0x7f9d8809a7c0 2024-04-25T08:42:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d88069010 con 0x7f9d8809a7c0 2024-04-25T08:42:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d85600640 1 -- 172.21.15.46:0/2261095797 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d74007450 con 0x7f9d8809a7c0 2024-04-25T08:42:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.491+0000 7f9d85600640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d68066f60 0x7f9d68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.495+0000 7f9d85600640 1 -- 172.21.15.46:0/2261095797 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9d74085240 con 0x7f9d8809a7c0 2024-04-25T08:42:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.495+0000 7f9d8d800640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d68066f60 0x7f9d68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.495+0000 7f9d8d800640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d68066f60 0x7f9d68069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0ad000 tx=0x7f9d7c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.495+0000 7f9d85600640 1 -- 172.21.15.46:0/2261095797 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d74055e70 con 0x7f9d8809a7c0 2024-04-25T08:42:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:37.695+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9d8809a530 con 0x7f9d8809a7c0 2024-04-25T08:42:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.427+0000 7f9d85600640 1 -- 172.21.15.46:0/2261095797 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9d74055770 con 0x7f9d8809a7c0 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d68066f60 msgr2=0x7f9d68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d68066f60 0x7f9d68069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0ad000 tx=0x7f9d7c0a6000 comp rx=0 tx=0).stop 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d8809a7c0 msgr2=0x7f9d881541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d8809a7c0 0x7f9d881541d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400c780 tx=0x7f9d7400cc40 comp rx=0 tx=0).stop 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 shutdown_connections 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d68066f60 0x7f9d68069420 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d8809ac20 0x7f9d880af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d8809a7c0 0x7f9d881541d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 --2- 172.21.15.46:0/2261095797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d88099a00 0x7f9d880c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 >> 172.21.15.46:0/2261095797 conn(0x7f9d880b3130 msgr2=0x7f9d88156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 shutdown_connections 2024-04-25T08:42:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.431+0000 7f9d90200640 1 -- 172.21.15.46:0/2261095797 wait complete. 2024-04-25T08:42:38.449 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T08:42:38.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:42:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bbac00640 1 Processor -- start 2024-04-25T08:42:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bbac00640 1 -- start start 2024-04-25T08:42:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bbac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb4143990 0x7f1bb40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bbac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1bb41440d0 0x7f1bb406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bbac00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb415e440 con 0x7f1bb406ef80 2024-04-25T08:42:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bbac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb415ed30 con 0x7f1bb41440d0 2024-04-25T08:42:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bbac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb40b3730 con 0x7f1bb4143990 2024-04-25T08:42:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bb8c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1bb406ef80 0x7f1bb40cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58982/0 (socket says 172.21.15.46:58982) 2024-04-25T08:42:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bb8c00640 1 -- 172.21.15.46:0/2928040396 learned_addr learned my addr 172.21.15.46:0/2928040396 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bb3e00640 1 --2- 172.21.15.46:0/2928040396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb4143990 0x7f1bb40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1babe00640 1 --2- 172.21.15.46:0/2928040396 >> v2:172.21.15.82:3300/0 conn(0x7f1bb41440d0 0x7f1bb406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bb3400640 1 -- 172.21.15.46:0/2928040396 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140550441 0 0) 0x7f1bb415e440 con 0x7f1bb406ef80 2024-04-25T08:42:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1bb3400640 1 -- 172.21.15.46:0/2928040396 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b98003530 con 0x7f1bb406ef80 2024-04-25T08:42:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1babe00640 1 -- 172.21.15.46:0/2928040396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb4143990 msgr2=0x7f1bb40cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1babe00640 1 --2- 172.21.15.46:0/2928040396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb4143990 0x7f1bb40cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1babe00640 1 -- 172.21.15.46:0/2928040396 >> v1:172.21.15.46:6789/0 conn(0x7f1bb406ef80 legacy=0x7f1bb40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.607+0000 7f1babe00640 1 -- 172.21.15.46:0/2928040396 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb415e6e0 con 0x7f1bb41440d0 2024-04-25T08:42:38.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1babe00640 1 --2- 172.21.15.46:0/2928040396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb41440d0 0x7f1bb406ea40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0097c0 tx=0x7f1b9c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12cfad9cb5dfdb55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bb3400640 1 -- 172.21.15.46:0/2928040396 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0b0070 con 0x7f1bb41440d0 2024-04-25T08:42:38.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bb3400640 1 -- 172.21.15.46:0/2928040396 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c0ab600 con 0x7f1bb41440d0 2024-04-25T08:42:38.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bb3400640 1 -- 172.21.15.46:0/2928040396 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0ab9d0 con 0x7f1bb41440d0 2024-04-25T08:42:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bbac00640 1 -- 172.21.15.46:0/2928040396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb41440d0 msgr2=0x7f1bb406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bbac00640 1 --2- 172.21.15.46:0/2928040396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb41440d0 0x7f1bb406ea40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0097c0 tx=0x7f1b9c01c3c0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bbac00640 1 -- 172.21.15.46:0/2928040396 shutdown_connections 2024-04-25T08:42:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bbac00640 1 --2- 172.21.15.46:0/2928040396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb41440d0 0x7f1bb406ea40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bbac00640 1 --2- 172.21.15.46:0/2928040396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb4143990 0x7f1bb40cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bbac00640 1 -- 172.21.15.46:0/2928040396 >> 172.21.15.46:0/2928040396 conn(0x7f1bb406ac10 msgr2=0x7f1bb406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bbac00640 1 -- 172.21.15.46:0/2928040396 shutdown_connections 2024-04-25T08:42:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.611+0000 7f1bbac00640 1 -- 172.21.15.46:0/2928040396 wait complete. 2024-04-25T08:42:38.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 Processor -- start 2024-04-25T08:42:38.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 -- start start 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb406ef80 0x7f1bb40b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb4143990 0x7f1bb40b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb41440d0 0x7f1bb4159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb4143d70 con 0x7f1bb4143990 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb415f1a0 con 0x7f1bb406ef80 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb4143ef0 con 0x7f1bb41440d0 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1babe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb4143990 0x7f1bb40b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1babe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb4143990 0x7f1bb40b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59876/0 (socket says 172.21.15.46:59876) 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1babe00640 1 -- 172.21.15.46:0/662374228 learned_addr learned my addr 172.21.15.46:0/662374228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bb8c00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb41440d0 0x7f1bb4159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bb3e00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb406ef80 0x7f1bb40b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1babe00640 1 -- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb41440d0 msgr2=0x7f1bb4159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1babe00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb41440d0 0x7f1bb4159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1babe00640 1 -- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb406ef80 msgr2=0x7f1bb40b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1babe00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb406ef80 0x7f1bb40b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1babe00640 1 -- 172.21.15.46:0/662374228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb4141280 con 0x7f1bb4143990 2024-04-25T08:42:38.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1babe00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb4143990 0x7f1bb40b1630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0097c0 tx=0x7f1b9c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bb0c00640 1 -- 172.21.15.46:0/662374228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0b0040 con 0x7f1bb4143990 2024-04-25T08:42:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bb0c00640 1 -- 172.21.15.46:0/662374228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c0b9070 con 0x7f1bb4143990 2024-04-25T08:42:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bb0c00640 1 -- 172.21.15.46:0/662374228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0b4650 con 0x7f1bb4143990 2024-04-25T08:42:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb4138580 con 0x7f1bb4143990 2024-04-25T08:42:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.615+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb4140df0 con 0x7f1bb4143990 2024-04-25T08:42:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.619+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb4069010 con 0x7f1bb4143990 2024-04-25T08:42:38.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.619+0000 7f1bb0c00640 1 -- 172.21.15.46:0/662374228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b9c0b47f0 con 0x7f1bb4143990 2024-04-25T08:42:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.619+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b90066f90 0x7f1b90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.619+0000 7f1bb0c00640 1 -- 172.21.15.46:0/662374228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1b9c11ef30 con 0x7f1bb4143990 2024-04-25T08:42:38.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.619+0000 7f1bb3e00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b90066f90 0x7f1b90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.619+0000 7f1bb3e00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b90066f90 0x7f1b90069450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009b60 tx=0x7f1ba4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.623+0000 7f1bb0c00640 1 -- 172.21.15.46:0/662374228 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b9c015fb0 con 0x7f1bb4143990 2024-04-25T08:42:38.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:38.823+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1bb40cd710 con 0x7f1bb4143990 2024-04-25T08:42:39.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.431+0000 7f1bb0c00640 1 -- 172.21.15.46:0/662374228 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1b9c0ef5b0 con 0x7f1bb4143990 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b90066f90 msgr2=0x7f1b90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b90066f90 0x7f1b90069450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009b60 tx=0x7f1ba4009190 comp rx=0 tx=0).stop 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb4143990 msgr2=0x7f1bb40b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb4143990 0x7f1bb40b1630 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0097c0 tx=0x7f1b9c019160 comp rx=0 tx=0).stop 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 shutdown_connections 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b90066f90 0x7f1b90069450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb41440d0 0x7f1bb4159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb4143990 0x7f1bb40b1630 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 --2- 172.21.15.46:0/662374228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb406ef80 0x7f1bb40b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 >> 172.21.15.46:0/662374228 conn(0x7f1bb406ac10 msgr2=0x7f1bb406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 shutdown_connections 2024-04-25T08:42:39.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.435+0000 7f1bbac00640 1 -- 172.21.15.46:0/662374228 wait complete. 2024-04-25T08:42:39.453 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:42:39.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.603+0000 7f06eb200640 1 Processor -- start 2024-04-25T08:42:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.603+0000 7f06eb200640 1 -- start start 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06eb200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f06e406c2b0 0x7f06e415f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06eb200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e40b2da0 con 0x7f06e4099a00 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06eb200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e41623a0 con 0x7f06e40b29c0 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06eb200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e4161b80 con 0x7f06e406c2b0 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06ea800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f06e4099a00 0x7f06e4099e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58988/0 (socket says 172.21.15.46:58988) 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06ea800640 1 -- 172.21.15.46:0/839697220 learned_addr learned my addr 172.21.15.46:0/839697220 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e9400640 1 --2- 172.21.15.46:0/839697220 >> v2:172.21.15.82:3300/0 conn(0x7f06e406c2b0 0x7f06e415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778139484 0 0) 0x7f06e40b2da0 con 0x7f06e4099a00 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06cc003530 con 0x7f06e4099a00 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361387672 0 0) 0x7f06e41623a0 con 0x7f06e40b29c0 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06e40b2da0 con 0x7f06e40b29c0 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4160662749 0 0) 0x7f06cc003530 con 0x7f06e4099a00 2024-04-25T08:42:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 >> v2:172.21.15.82:3300/0 conn(0x7f06e406c2b0 msgr2=0x7f06e415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 --2- 172.21.15.46:0/839697220 >> v2:172.21.15.82:3300/0 conn(0x7f06e406c2b0 0x7f06e415f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 >> v1:172.21.15.82:6789/0 conn(0x7f06e40b29c0 legacy=0x7f06e4161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e4161de0 con 0x7f06e4099a00 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e9400640 1 --2- 172.21.15.46:0/839697220 >> v2:172.21.15.82:3300/0 conn(0x7f06e406c2b0 0x7f06e415f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f06e0001a00 con 0x7f06e4099a00 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 >> v1:172.21.15.46:6789/0 conn(0x7f06e4099a00 legacy=0x7f06e4099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06cc003b50 0x7f06cc003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06cc004810 0x7f06cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0051d0 0x7f06cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e4161b80 con 0x7f06cc0051d0 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc00c050 con 0x7f06cc004810 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06cc00c250 con 0x7f06cc003b50 2024-04-25T08:42:39.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06ea800640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0051d0 0x7f06cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e9400640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06cc004810 0x7f06cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e9e00640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06cc003b50 0x7f06cc003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06ea800640 1 -- 172.21.15.46:0/839697220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06cc003b50 msgr2=0x7f06cc003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06ea800640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06cc003b50 0x7f06cc003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06ea800640 1 -- 172.21.15.46:0/839697220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06cc004810 msgr2=0x7f06cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06ea800640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06cc004810 0x7f06cc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06ea800640 1 -- 172.21.15.46:0/839697220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06cc003530 con 0x7f06cc0051d0 2024-04-25T08:42:39.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.607+0000 7f06e9e00640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06cc003b50 0x7f06cc003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06ea800640 1 -- 172.21.15.46:0/839697220 reap_dead start 2024-04-25T08:42:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06ea800640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0051d0 0x7f06cc00b9d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f06e000a820 tx=0x7f06e001fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50058f2a79df7a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e000a450 con 0x7f06cc0051d0 2024-04-25T08:42:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e00aa6d0 con 0x7f06cc0051d0 2024-04-25T08:42:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06e8a00640 1 -- 172.21.15.46:0/839697220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e00aaac0 con 0x7f06cc0051d0 2024-04-25T08:42:39.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06eb200640 1 -- 172.21.15.46:0/839697220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0051d0 msgr2=0x7f06cc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06eb200640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0051d0 0x7f06cc00b9d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f06e000a820 tx=0x7f06e001fa60 comp rx=0 tx=0).stop 2024-04-25T08:42:39.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06eb200640 1 -- 172.21.15.46:0/839697220 shutdown_connections 2024-04-25T08:42:39.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06eb200640 1 --2- 172.21.15.46:0/839697220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06cc0051d0 0x7f06cc00b9d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06eb200640 1 -- 172.21.15.46:0/839697220 >> 172.21.15.46:0/839697220 conn(0x7f06e40b2f90 msgr2=0x7f06e40b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:39.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06eb200640 1 -- 172.21.15.46:0/839697220 shutdown_connections 2024-04-25T08:42:39.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06eb200640 1 -- 172.21.15.46:0/839697220 wait complete. 2024-04-25T08:42:39.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06eb200640 1 Processor -- start 2024-04-25T08:42:39.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.611+0000 7f06eb200640 1 -- start start 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06eb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06cc0051d0 0x7f06e415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06eb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06e415c6a0 0x7f06e40c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06eb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06e40af870 0x7f06e40afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06eb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40cfc90 con 0x7f06e40af870 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06eb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e40cf9b0 con 0x7f06cc0051d0 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06eb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e415f250 con 0x7f06e415c6a0 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06e40af870 0x7f06e40afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06e40af870 0x7f06e40afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59894/0 (socket says 172.21.15.46:59894) 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06ea800640 1 -- 172.21.15.46:0/4049937676 learned_addr learned my addr 172.21.15.46:0/4049937676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06e9e00640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06cc0051d0 0x7f06e415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06e9400640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06e415c6a0 0x7f06e40c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06ea800640 1 -- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06e415c6a0 msgr2=0x7f06e40c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06ea800640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06e415c6a0 0x7f06e40c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06ea800640 1 -- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06cc0051d0 msgr2=0x7f06e415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06ea800640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06cc0051d0 0x7f06e415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06ea800640 1 -- 172.21.15.46:0/4049937676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e40c5510 con 0x7f06e40af870 2024-04-25T08:42:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06ea800640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06e40af870 0x7f06e40afcf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06e001f890 tx=0x7f06cc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:39.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06de000640 1 -- 172.21.15.46:0/4049937676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e00af070 con 0x7f06e40af870 2024-04-25T08:42:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06de000640 1 -- 172.21.15.46:0/4049937676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e00b0040 con 0x7f06e40af870 2024-04-25T08:42:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06de000640 1 -- 172.21.15.46:0/4049937676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0001fa0 con 0x7f06e40af870 2024-04-25T08:42:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e40c57c0 con 0x7f06e40af870 2024-04-25T08:42:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e40c5d80 con 0x7f06e40af870 2024-04-25T08:42:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.615+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e4069010 con 0x7f06e40af870 2024-04-25T08:42:39.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.619+0000 7f06de000640 1 -- 172.21.15.46:0/4049937676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06e00be020 con 0x7f06e40af870 2024-04-25T08:42:39.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.619+0000 7f06de000640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06c8066f90 0x7f06c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.619+0000 7f06de000640 1 -- 172.21.15.46:0/4049937676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f06e0122760 con 0x7f06e40af870 2024-04-25T08:42:39.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.619+0000 7f06e9e00640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06c8066f90 0x7f06c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.619+0000 7f06e9e00640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06c8066f90 0x7f06c8069450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06d8001720 tx=0x7f06d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:39.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.623+0000 7f06de000640 1 -- 172.21.15.46:0/4049937676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06e0019770 con 0x7f06e40af870 2024-04-25T08:42:39.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.819+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06e409a530 con 0x7f06e40af870 2024-04-25T08:42:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.819+0000 7f06de000640 1 -- 172.21.15.46:0/4049937676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f06e00f2d50 con 0x7f06e40af870 2024-04-25T08:42:39.813 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:39.813 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:38.824358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:42:39.814 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-25T08:42:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06c8066f90 msgr2=0x7f06c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06c8066f90 0x7f06c8069450 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06d8001720 tx=0x7f06d8009040 comp rx=0 tx=0).stop 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06e40af870 msgr2=0x7f06e40afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06e40af870 0x7f06e40afcf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f06e001f890 tx=0x7f06cc005840 comp rx=0 tx=0).stop 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 shutdown_connections 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06c8066f90 0x7f06c8069450 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06e40af870 0x7f06e40afcf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06e415c6a0 0x7f06e40c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 --2- 172.21.15.46:0/4049937676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06cc0051d0 0x7f06e415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 >> 172.21.15.46:0/4049937676 conn(0x7f06e40b2f90 msgr2=0x7f06e409a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 shutdown_connections 2024-04-25T08:42:39.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.823+0000 7f06eb200640 1 -- 172.21.15.46:0/4049937676 wait complete. 2024-04-25T08:42:39.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45d0c00640 1 Processor -- start 2024-04-25T08:42:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45d0c00640 1 -- start start 2024-04-25T08:42:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45d0c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f45cc15ef20 0x7f45cc0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45d0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45cc0cd6d0 0x7f45cc09a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45d0c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc09ab50 0x7f45cc099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45d0c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45cc15f300 con 0x7f45cc0cd6d0 2024-04-25T08:42:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45d0c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45cc0cddb0 con 0x7f45cc15ef20 2024-04-25T08:42:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45d0c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc0cc3d0 con 0x7f45cc09ab50 2024-04-25T08:42:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45c3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45cc0cd6d0 0x7f45cc09a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45c3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45cc0cd6d0 0x7f45cc09a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59898/0 (socket says 172.21.15.46:59898) 2024-04-25T08:42:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45c3e00640 1 -- 172.21.15.46:0/3047156168 learned_addr learned my addr 172.21.15.46:0/3047156168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45c9e00640 1 --2- 172.21.15.46:0/3047156168 >> v2:172.21.15.82:3300/0 conn(0x7f45cc15ef20 0x7f45cc0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45ca800640 1 --2- 172.21.15.46:0/3047156168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc09ab50 0x7f45cc099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45c3e00640 1 -- 172.21.15.46:0/3047156168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc09ab50 msgr2=0x7f45cc099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45c3e00640 1 --2- 172.21.15.46:0/3047156168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc09ab50 0x7f45cc099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45c3e00640 1 -- 172.21.15.46:0/3047156168 >> v2:172.21.15.82:3300/0 conn(0x7f45cc15ef20 msgr2=0x7f45cc0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45c3e00640 1 --2- 172.21.15.46:0/3047156168 >> v2:172.21.15.82:3300/0 conn(0x7f45cc15ef20 0x7f45cc0cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.983+0000 7f45c3e00640 1 -- 172.21.15.46:0/3047156168 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc0cc6b0 con 0x7f45cc0cd6d0 2024-04-25T08:42:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45c3e00640 1 --2- 172.21.15.46:0/3047156168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc0cd6d0 0x7f45cc09a610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f45cc09a5e0 tx=0x7f45b401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790c36143d35e3d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45c9400640 1 -- 172.21.15.46:0/3047156168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b40aa430 con 0x7f45cc0cd6d0 2024-04-25T08:42:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45c9400640 1 -- 172.21.15.46:0/3047156168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b40af070 con 0x7f45cc0cd6d0 2024-04-25T08:42:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45c9400640 1 -- 172.21.15.46:0/3047156168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b40aa7f0 con 0x7f45cc0cd6d0 2024-04-25T08:42:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45d0c00640 1 -- 172.21.15.46:0/3047156168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc0cd6d0 msgr2=0x7f45cc09a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45d0c00640 1 --2- 172.21.15.46:0/3047156168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc0cd6d0 0x7f45cc09a610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f45cc09a5e0 tx=0x7f45b401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45d0c00640 1 -- 172.21.15.46:0/3047156168 shutdown_connections 2024-04-25T08:42:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45d0c00640 1 --2- 172.21.15.46:0/3047156168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc09ab50 0x7f45cc099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45d0c00640 1 --2- 172.21.15.46:0/3047156168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc0cd6d0 0x7f45cc09a610 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45d0c00640 1 --2- 172.21.15.46:0/3047156168 >> v2:172.21.15.82:3300/0 conn(0x7f45cc15ef20 0x7f45cc0cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45d0c00640 1 -- 172.21.15.46:0/3047156168 >> 172.21.15.46:0/3047156168 conn(0x7f45cc0b3120 msgr2=0x7f45cc0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45d0c00640 1 -- 172.21.15.46:0/3047156168 shutdown_connections 2024-04-25T08:42:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.987+0000 7f45d0c00640 1 -- 172.21.15.46:0/3047156168 wait complete. 2024-04-25T08:42:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 Processor -- start 2024-04-25T08:42:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 -- start start 2024-04-25T08:42:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45cc09ab50 0x7f45cc06bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc15ef20 0x7f45cc09c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc09cbf0 0x7f45cc0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc15f540 con 0x7f45cc09cbf0 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc09a350 con 0x7f45cc09ab50 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc15f6c0 con 0x7f45cc15ef20 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc09cbf0 0x7f45cc0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc09cbf0 0x7f45cc0ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59904/0 (socket says 172.21.15.46:59904) 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45ca800640 1 -- 172.21.15.46:0/1789727547 learned_addr learned my addr 172.21.15.46:0/1789727547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45c9e00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45cc09ab50 0x7f45cc06bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45c3e00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc15ef20 0x7f45cc09c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45ca800640 1 -- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc15ef20 msgr2=0x7f45cc09c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45ca800640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc15ef20 0x7f45cc09c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45ca800640 1 -- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45cc09ab50 msgr2=0x7f45cc06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45ca800640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45cc09ab50 0x7f45cc06bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45ca800640 1 -- 172.21.15.46:0/1789727547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc0c9640 con 0x7f45cc09cbf0 2024-04-25T08:42:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45ca800640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc09cbf0 0x7f45cc0ca8d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f45c4007e60 tx=0x7f45c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45c2000640 1 -- 172.21.15.46:0/1789727547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4014620 con 0x7f45cc09cbf0 2024-04-25T08:42:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45c2000640 1 -- 172.21.15.46:0/1789727547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c4019070 con 0x7f45cc09cbf0 2024-04-25T08:42:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc0c9920 con 0x7f45cc09cbf0 2024-04-25T08:42:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45c2000640 1 -- 172.21.15.46:0/1789727547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4014a20 con 0x7f45cc09cbf0 2024-04-25T08:42:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc06ca10 con 0x7f45cc09cbf0 2024-04-25T08:42:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.991+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45cc0c9ec0 con 0x7f45cc09cbf0 2024-04-25T08:42:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.995+0000 7f45c2000640 1 -- 172.21.15.46:0/1789727547 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45c4014bc0 con 0x7f45cc09cbf0 2024-04-25T08:42:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.995+0000 7f45c2000640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45a8066f60 0x7f45a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.995+0000 7f45c2000640 1 -- 172.21.15.46:0/1789727547 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f45c4085b40 con 0x7f45cc09cbf0 2024-04-25T08:42:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.995+0000 7f45c9e00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45a8066f60 0x7f45a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:39.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.995+0000 7f45c9e00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45a8066f60 0x7f45a8069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0079c0 tx=0x7f45bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:39.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:39.999+0000 7f45c2000640 1 -- 172.21.15.46:0/1789727547 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45c4056120 con 0x7f45cc09cbf0 2024-04-25T08:42:40.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.163+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45cc069080 con 0x7f45cc09cbf0 2024-04-25T08:42:40.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.163+0000 7f45c2000640 1 -- 172.21.15.46:0/1789727547 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f45c405a1c0 con 0x7f45cc09cbf0 2024-04-25T08:42:40.158 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:40.158 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45a8066f60 msgr2=0x7f45a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45a8066f60 0x7f45a8069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0079c0 tx=0x7f45bc008040 comp rx=0 tx=0).stop 2024-04-25T08:42:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc09cbf0 msgr2=0x7f45cc0ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc09cbf0 0x7f45cc0ca8d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f45c4007e60 tx=0x7f45c400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 shutdown_connections 2024-04-25T08:42:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45a8066f60 0x7f45a8069420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45cc09cbf0 0x7f45cc0ca8d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45cc15ef20 0x7f45cc09c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 --2- 172.21.15.46:0/1789727547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45cc09ab50 0x7f45cc06bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 >> 172.21.15.46:0/1789727547 conn(0x7f45cc0b3120 msgr2=0x7f45cc0c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 shutdown_connections 2024-04-25T08:42:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.167+0000 7f45d0c00640 1 -- 172.21.15.46:0/1789727547 wait complete. 2024-04-25T08:42:40.180 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:42:40.180 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:40.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.327+0000 7fca26e00640 1 Processor -- start 2024-04-25T08:42:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca26e00640 1 -- start start 2024-04-25T08:42:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca26e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca2015ef20 0x7fca200cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca26e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200cd6d0 0x7fca2009a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca26e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca2009ab50 0x7fca20099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca26e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2015f300 con 0x7fca2015ef20 2024-04-25T08:42:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca26e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca200cddb0 con 0x7fca2009ab50 2024-04-25T08:42:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca26e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca200cc3d0 con 0x7fca200cd6d0 2024-04-25T08:42:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca1fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca2015ef20 0x7fca200cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca1fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca2015ef20 0x7fca200cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59918/0 (socket says 172.21.15.46:59918) 2024-04-25T08:42:40.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca1fe00640 1 -- 172.21.15.46:0/2201821790 learned_addr learned my addr 172.21.15.46:0/2201821790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca1f400640 1 --2- 172.21.15.46:0/2201821790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200cd6d0 0x7fca2009a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca24e00640 1 --2- 172.21.15.46:0/2201821790 >> v2:172.21.15.82:3300/0 conn(0x7fca2009ab50 0x7fca20099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca1fe00640 1 -- 172.21.15.46:0/2201821790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200cd6d0 msgr2=0x7fca2009a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca1fe00640 1 --2- 172.21.15.46:0/2201821790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200cd6d0 0x7fca2009a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca1fe00640 1 -- 172.21.15.46:0/2201821790 >> v2:172.21.15.82:3300/0 conn(0x7fca2009ab50 msgr2=0x7fca20099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca1fe00640 1 --2- 172.21.15.46:0/2201821790 >> v2:172.21.15.82:3300/0 conn(0x7fca2009ab50 0x7fca20099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.331+0000 7fca1fe00640 1 -- 172.21.15.46:0/2201821790 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca200cc6b0 con 0x7fca2015ef20 2024-04-25T08:42:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca1fe00640 1 --2- 172.21.15.46:0/2201821790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2015ef20 0x7fca200cd190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fca20071e10 tx=0x7fca1001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b478f62e12327d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca1ea00640 1 -- 172.21.15.46:0/2201821790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca100aa430 con 0x7fca2015ef20 2024-04-25T08:42:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca1ea00640 1 -- 172.21.15.46:0/2201821790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca100af070 con 0x7fca2015ef20 2024-04-25T08:42:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca1ea00640 1 -- 172.21.15.46:0/2201821790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca100aa7f0 con 0x7fca2015ef20 2024-04-25T08:42:40.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 -- 172.21.15.46:0/2201821790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2015ef20 msgr2=0x7fca200cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 --2- 172.21.15.46:0/2201821790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2015ef20 0x7fca200cd190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fca20071e10 tx=0x7fca1001c4f0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 -- 172.21.15.46:0/2201821790 shutdown_connections 2024-04-25T08:42:40.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 --2- 172.21.15.46:0/2201821790 >> v2:172.21.15.82:3300/0 conn(0x7fca2009ab50 0x7fca20099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 --2- 172.21.15.46:0/2201821790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200cd6d0 0x7fca2009a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 --2- 172.21.15.46:0/2201821790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2015ef20 0x7fca200cd190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 -- 172.21.15.46:0/2201821790 >> 172.21.15.46:0/2201821790 conn(0x7fca200b3120 msgr2=0x7fca200b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 -- 172.21.15.46:0/2201821790 shutdown_connections 2024-04-25T08:42:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 -- 172.21.15.46:0/2201821790 wait complete. 2024-04-25T08:42:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 Processor -- start 2024-04-25T08:42:40.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.335+0000 7fca26e00640 1 -- start start 2024-04-25T08:42:40.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca26e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2009ab50 0x7fca200cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca26e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca2015ef20 0x7fca201f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca26e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200c9110 0x7fca201f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca26e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2015f540 con 0x7fca2009ab50 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca26e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2009a350 con 0x7fca2015ef20 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca26e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2015f6c0 con 0x7fca200c9110 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2009ab50 0x7fca200cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2009ab50 0x7fca200cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59926/0 (socket says 172.21.15.46:59926) 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1fe00640 1 -- 172.21.15.46:0/4023259251 learned_addr learned my addr 172.21.15.46:0/4023259251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1f400640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca2015ef20 0x7fca201f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca24e00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200c9110 0x7fca201f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1fe00640 1 -- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200c9110 msgr2=0x7fca201f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1fe00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200c9110 0x7fca201f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1fe00640 1 -- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca2015ef20 msgr2=0x7fca201f7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1fe00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca2015ef20 0x7fca201f7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1fe00640 1 -- 172.21.15.46:0/4023259251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca20156110 con 0x7fca2009ab50 2024-04-25T08:42:40.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca1fe00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2009ab50 0x7fca200cac30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fca1001aba0 tx=0x7fca1001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca17e00640 1 -- 172.21.15.46:0/4023259251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca100bd5f0 con 0x7fca2009ab50 2024-04-25T08:42:40.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca17e00640 1 -- 172.21.15.46:0/4023259251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca100bd7b0 con 0x7fca2009ab50 2024-04-25T08:42:40.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca201563f0 con 0x7fca2009ab50 2024-04-25T08:42:40.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca17e00640 1 -- 172.21.15.46:0/4023259251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca100bdba0 con 0x7fca2009ab50 2024-04-25T08:42:40.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca201569b0 con 0x7fca2009ab50 2024-04-25T08:42:40.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca20069080 con 0x7fca2009ab50 2024-04-25T08:42:40.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.339+0000 7fca17e00640 1 -- 172.21.15.46:0/4023259251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca100b33f0 con 0x7fca2009ab50 2024-04-25T08:42:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.343+0000 7fca17e00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca00066f60 0x7fca00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.343+0000 7fca17e00640 1 -- 172.21.15.46:0/4023259251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fca1011fd20 con 0x7fca2009ab50 2024-04-25T08:42:40.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.343+0000 7fca1f400640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca00066f60 0x7fca00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.343+0000 7fca1f400640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca00066f60 0x7fca00069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca200cb820 tx=0x7fca0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.347+0000 7fca17e00640 1 -- 172.21.15.46:0/4023259251 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca100c3060 con 0x7fca2009ab50 2024-04-25T08:42:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.507+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca20073db0 con 0x7fca00066f60 2024-04-25T08:42:40.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.519+0000 7fca17e00640 1 -- 172.21.15.46:0/4023259251 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332562 (secure 0 0 0) 0x7fca20073db0 con 0x7fca00066f60 2024-04-25T08:42:40.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:40.519 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca00066f60 msgr2=0x7fca00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca00066f60 0x7fca00069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fca200cb820 tx=0x7fca0c008040 comp rx=0 tx=0).stop 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2009ab50 msgr2=0x7fca200cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2009ab50 0x7fca200cac30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fca1001aba0 tx=0x7fca1001ad30 comp rx=0 tx=0).stop 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 shutdown_connections 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca00066f60 0x7fca00069420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca200c9110 0x7fca201f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca2015ef20 0x7fca201f7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 --2- 172.21.15.46:0/4023259251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2009ab50 0x7fca200cac30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 >> 172.21.15.46:0/4023259251 conn(0x7fca200b3120 msgr2=0x7fca20070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 shutdown_connections 2024-04-25T08:42:40.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.523+0000 7fca26e00640 1 -- 172.21.15.46:0/4023259251 wait complete. 2024-04-25T08:42:40.537 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-25T08:42:40.256911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223716,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751004,"statfs":{"total":773094113280,"available":772865028096,"internally_reserved":0,"allocated":3481600,"data_stored":2086387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003140"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683286+0000","last_change":"2024-04-25T08:42:20.357496+0000","last_active":"2024-04-25T08:42:28.683286+0000","last_peered":"2024-04-25T08:42:28.683286+0000","last_clean":"2024-04-25T08:42:28.683286+0000","last_became_active":"2024-04-25T08:42:20.357232+0000","last_became_peered":"2024-04-25T08:42:20.357232+0000","last_unstale":"2024-04-25T08:42:28.683286+0000","last_undegraded":"2024-04-25T08:42:28.683286+0000","last_fullsized":"2024-04-25T08:42:28.683286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:43.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385143+0000","last_change":"2024-04-25T08:42:21.359572+0000","last_active":"2024-04-25T08:42:27.385143+0000","last_peered":"2024-04-25T08:42:27.385143+0000","last_clean":"2024-04-25T08:42:27.385143+0000","last_became_active":"2024-04-25T08:42:21.357850+0000","last_became_peered":"2024-04-25T08:42:21.357850+0000","last_unstale":"2024-04-25T08:42:27.385143+0000","last_undegraded":"2024-04-25T08:42:27.385143+0000","last_fullsized":"2024-04-25T08:42:27.385143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:25.717765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384099+0000","last_change":"2024-04-25T08:42:25.834318+0000","last_active":"2024-04-25T08:42:27.384099+0000","last_peered":"2024-04-25T08:42:27.384099+0000","last_clean":"2024-04-25T08:42:27.384099+0000","last_became_active":"2024-04-25T08:42:25.833964+0000","last_became_peered":"2024-04-25T08:42:25.833964+0000","last_unstale":"2024-04-25T08:42:27.384099+0000","last_undegraded":"2024-04-25T08:42:27.384099+0000","last_fullsized":"2024-04-25T08:42:27.384099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:37.125230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385326+0000","last_change":"2024-04-25T08:42:20.351984+0000","last_active":"2024-04-25T08:42:27.385326+0000","last_peered":"2024-04-25T08:42:27.385326+0000","last_clean":"2024-04-25T08:42:27.385326+0000","last_became_active":"2024-04-25T08:42:20.351590+0000","last_became_peered":"2024-04-25T08:42:20.351590+0000","last_unstale":"2024-04-25T08:42:27.385326+0000","last_undegraded":"2024-04-25T08:42:27.385326+0000","last_fullsized":"2024-04-25T08:42:27.385326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683138+0000","last_change":"2024-04-25T08:42:21.364800+0000","last_active":"2024-04-25T08:42:28.683138+0000","last_peered":"2024-04-25T08:42:28.683138+0000","last_clean":"2024-04-25T08:42:28.683138+0000","last_became_active":"2024-04-25T08:42:21.363796+0000","last_became_peered":"2024-04-25T08:42:21.363796+0000","last_unstale":"2024-04-25T08:42:28.683138+0000","last_undegraded":"2024-04-25T08:42:28.683138+0000","last_fullsized":"2024-04-25T08:42:28.683138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389657+0000","last_change":"2024-04-25T08:42:25.410345+0000","last_active":"2024-04-25T08:42:27.389657+0000","last_peered":"2024-04-25T08:42:27.389657+0000","last_clean":"2024-04-25T08:42:27.389657+0000","last_became_active":"2024-04-25T08:42:25.409807+0000","last_became_peered":"2024-04-25T08:42:25.409807+0000","last_unstale":"2024-04-25T08:42:27.389657+0000","last_undegraded":"2024-04-25T08:42:27.389657+0000","last_fullsized":"2024-04-25T08:42:27.389657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.971525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683514+0000","last_change":"2024-04-25T08:42:20.363027+0000","last_active":"2024-04-25T08:42:28.683514+0000","last_peered":"2024-04-25T08:42:28.683514+0000","last_clean":"2024-04-25T08:42:28.683514+0000","last_became_active":"2024-04-25T08:42:20.362598+0000","last_became_peered":"2024-04-25T08:42:20.362598+0000","last_unstale":"2024-04-25T08:42:28.683514+0000","last_undegraded":"2024-04-25T08:42:28.683514+0000","last_fullsized":"2024-04-25T08:42:28.683514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:26.564850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582927+0000","last_change":"2024-04-25T08:42:21.367716+0000","last_active":"2024-04-25T08:42:27.582927+0000","last_peered":"2024-04-25T08:42:27.582927+0000","last_clean":"2024-04-25T08:42:27.582927+0000","last_became_active":"2024-04-25T08:42:21.366362+0000","last_became_peered":"2024-04-25T08:42:21.366362+0000","last_unstale":"2024-04-25T08:42:27.582927+0000","last_undegraded":"2024-04-25T08:42:27.582927+0000","last_fullsized":"2024-04-25T08:42:27.582927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:30.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390443+0000","last_change":"2024-04-25T08:42:25.407011+0000","last_active":"2024-04-25T08:42:27.390443+0000","last_peered":"2024-04-25T08:42:27.390443+0000","last_clean":"2024-04-25T08:42:27.390443+0000","last_became_active":"2024-04-25T08:42:25.405963+0000","last_became_peered":"2024-04-25T08:42:25.405963+0000","last_unstale":"2024-04-25T08:42:27.390443+0000","last_undegraded":"2024-04-25T08:42:27.390443+0000","last_fullsized":"2024-04-25T08:42:27.390443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683493+0000","last_change":"2024-04-25T08:42:20.365950+0000","last_active":"2024-04-25T08:42:28.683493+0000","last_peered":"2024-04-25T08:42:28.683493+0000","last_clean":"2024-04-25T08:42:28.683493+0000","last_became_active":"2024-04-25T08:42:20.365648+0000","last_became_peered":"2024-04-25T08:42:20.365648+0000","last_unstale":"2024-04-25T08:42:28.683493+0000","last_undegraded":"2024-04-25T08:42:28.683493+0000","last_fullsized":"2024-04-25T08:42:28.683493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:31.011923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583016+0000","last_change":"2024-04-25T08:42:21.366598+0000","last_active":"2024-04-25T08:42:27.583016+0000","last_peered":"2024-04-25T08:42:27.583016+0000","last_clean":"2024-04-25T08:42:27.583016+0000","last_became_active":"2024-04-25T08:42:21.365936+0000","last_became_peered":"2024-04-25T08:42:21.365936+0000","last_unstale":"2024-04-25T08:42:27.583016+0000","last_undegraded":"2024-04-25T08:42:27.583016+0000","last_fullsized":"2024-04-25T08:42:27.583016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389911+0000","last_change":"2024-04-25T08:42:25.400233+0000","last_active":"2024-04-25T08:42:27.389911+0000","last_peered":"2024-04-25T08:42:27.389911+0000","last_clean":"2024-04-25T08:42:27.389911+0000","last_became_active":"2024-04-25T08:42:25.399809+0000","last_became_peered":"2024-04-25T08:42:25.399809+0000","last_unstale":"2024-04-25T08:42:27.389911+0000","last_undegraded":"2024-04-25T08:42:27.389911+0000","last_fullsized":"2024-04-25T08:42:27.389911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:19.971594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388746+0000","last_change":"2024-04-25T08:42:21.354062+0000","last_active":"2024-04-25T08:42:27.388746+0000","last_peered":"2024-04-25T08:42:27.388746+0000","last_clean":"2024-04-25T08:42:27.388746+0000","last_became_active":"2024-04-25T08:42:21.353804+0000","last_became_peered":"2024-04-25T08:42:21.353804+0000","last_unstale":"2024-04-25T08:42:27.388746+0000","last_undegraded":"2024-04-25T08:42:27.388746+0000","last_fullsized":"2024-04-25T08:42:27.388746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683480+0000","last_change":"2024-04-25T08:42:20.361718+0000","last_active":"2024-04-25T08:42:28.683480+0000","last_peered":"2024-04-25T08:42:28.683480+0000","last_clean":"2024-04-25T08:42:28.683480+0000","last_became_active":"2024-04-25T08:42:20.361531+0000","last_became_peered":"2024-04-25T08:42:20.361531+0000","last_unstale":"2024-04-25T08:42:28.683480+0000","last_undegraded":"2024-04-25T08:42:28.683480+0000","last_fullsized":"2024-04-25T08:42:28.683480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:47.546463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390866+0000","last_change":"2024-04-25T08:42:25.410313+0000","last_active":"2024-04-25T08:42:27.390866+0000","last_peered":"2024-04-25T08:42:27.390866+0000","last_clean":"2024-04-25T08:42:27.390866+0000","last_became_active":"2024-04-25T08:42:25.409794+0000","last_became_peered":"2024-04-25T08:42:25.409794+0000","last_unstale":"2024-04-25T08:42:27.390866+0000","last_undegraded":"2024-04-25T08:42:27.390866+0000","last_fullsized":"2024-04-25T08:42:27.390866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384856+0000","last_change":"2024-04-25T08:42:21.358366+0000","last_active":"2024-04-25T08:42:27.384856+0000","last_peered":"2024-04-25T08:42:27.384856+0000","last_clean":"2024-04-25T08:42:27.384856+0000","last_became_active":"2024-04-25T08:42:21.357447+0000","last_became_peered":"2024-04-25T08:42:21.357447+0000","last_unstale":"2024-04-25T08:42:27.384856+0000","last_undegraded":"2024-04-25T08:42:27.384856+0000","last_fullsized":"2024-04-25T08:42:27.384856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677905+0000","last_change":"2024-04-25T08:42:20.352044+0000","last_active":"2024-04-25T08:42:28.677905+0000","last_peered":"2024-04-25T08:42:28.677905+0000","last_clean":"2024-04-25T08:42:28.677905+0000","last_became_active":"2024-04-25T08:42:20.351667+0000","last_became_peered":"2024-04-25T08:42:20.351667+0000","last_unstale":"2024-04-25T08:42:28.677905+0000","last_undegraded":"2024-04-25T08:42:28.677905+0000","last_fullsized":"2024-04-25T08:42:28.677905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390597+0000","last_change":"2024-04-25T08:42:25.406565+0000","last_active":"2024-04-25T08:42:27.390597+0000","last_peered":"2024-04-25T08:42:27.390597+0000","last_clean":"2024-04-25T08:42:27.390597+0000","last_became_active":"2024-04-25T08:42:25.405840+0000","last_became_peered":"2024-04-25T08:42:25.405840+0000","last_unstale":"2024-04-25T08:42:27.390597+0000","last_undegraded":"2024-04-25T08:42:27.390597+0000","last_fullsized":"2024-04-25T08:42:27.390597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384326+0000","last_change":"2024-04-25T08:42:21.354614+0000","last_active":"2024-04-25T08:42:27.384326+0000","last_peered":"2024-04-25T08:42:27.384326+0000","last_clean":"2024-04-25T08:42:27.384326+0000","last_became_active":"2024-04-25T08:42:21.353873+0000","last_became_peered":"2024-04-25T08:42:21.353873+0000","last_unstale":"2024-04-25T08:42:27.384326+0000","last_undegraded":"2024-04-25T08:42:27.384326+0000","last_fullsized":"2024-04-25T08:42:27.384326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678307+0000","last_change":"2024-04-25T08:42:20.349084+0000","last_active":"2024-04-25T08:42:28.678307+0000","last_peered":"2024-04-25T08:42:28.678307+0000","last_clean":"2024-04-25T08:42:28.678307+0000","last_became_active":"2024-04-25T08:42:20.348850+0000","last_became_peered":"2024-04-25T08:42:20.348850+0000","last_unstale":"2024-04-25T08:42:28.678307+0000","last_undegraded":"2024-04-25T08:42:28.678307+0000","last_fullsized":"2024-04-25T08:42:28.678307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678770+0000","last_change":"2024-04-25T08:42:25.832820+0000","last_active":"2024-04-25T08:42:28.678770+0000","last_peered":"2024-04-25T08:42:28.678770+0000","last_clean":"2024-04-25T08:42:28.678770+0000","last_became_active":"2024-04-25T08:42:25.832090+0000","last_became_peered":"2024-04-25T08:42:25.832090+0000","last_unstale":"2024-04-25T08:42:28.678770+0000","last_undegraded":"2024-04-25T08:42:28.678770+0000","last_fullsized":"2024-04-25T08:42:28.678770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390081+0000","last_change":"2024-04-25T08:42:21.338500+0000","last_active":"2024-04-25T08:42:27.390081+0000","last_peered":"2024-04-25T08:42:27.390081+0000","last_clean":"2024-04-25T08:42:27.390081+0000","last_became_active":"2024-04-25T08:42:21.338153+0000","last_became_peered":"2024-04-25T08:42:21.338153+0000","last_unstale":"2024-04-25T08:42:27.390081+0000","last_undegraded":"2024-04-25T08:42:27.390081+0000","last_fullsized":"2024-04-25T08:42:27.390081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:00.647619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683714+0000","last_change":"2024-04-25T08:42:20.361253+0000","last_active":"2024-04-25T08:42:28.683714+0000","last_peered":"2024-04-25T08:42:28.683714+0000","last_clean":"2024-04-25T08:42:28.683714+0000","last_became_active":"2024-04-25T08:42:20.360866+0000","last_became_peered":"2024-04-25T08:42:20.360866+0000","last_unstale":"2024-04-25T08:42:28.683714+0000","last_undegraded":"2024-04-25T08:42:28.683714+0000","last_fullsized":"2024-04-25T08:42:28.683714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.823720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390784+0000","last_change":"2024-04-25T08:42:25.406555+0000","last_active":"2024-04-25T08:42:27.390784+0000","last_peered":"2024-04-25T08:42:27.390784+0000","last_clean":"2024-04-25T08:42:27.390784+0000","last_became_active":"2024-04-25T08:42:25.405977+0000","last_became_peered":"2024-04-25T08:42:25.405977+0000","last_unstale":"2024-04-25T08:42:27.390784+0000","last_undegraded":"2024-04-25T08:42:27.390784+0000","last_fullsized":"2024-04-25T08:42:27.390784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683559+0000","last_change":"2024-04-25T08:42:21.361932+0000","last_active":"2024-04-25T08:42:28.683559+0000","last_peered":"2024-04-25T08:42:28.683559+0000","last_clean":"2024-04-25T08:42:28.683559+0000","last_became_active":"2024-04-25T08:42:21.361599+0000","last_became_peered":"2024-04-25T08:42:21.361599+0000","last_unstale":"2024-04-25T08:42:28.683559+0000","last_undegraded":"2024-04-25T08:42:28.683559+0000","last_fullsized":"2024-04-25T08:42:28.683559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389472+0000","last_change":"2024-04-25T08:42:20.359118+0000","last_active":"2024-04-25T08:42:27.389472+0000","last_peered":"2024-04-25T08:42:27.389472+0000","last_clean":"2024-04-25T08:42:27.389472+0000","last_became_active":"2024-04-25T08:42:20.358850+0000","last_became_peered":"2024-04-25T08:42:20.358850+0000","last_unstale":"2024-04-25T08:42:27.389472+0000","last_undegraded":"2024-04-25T08:42:27.389472+0000","last_fullsized":"2024-04-25T08:42:27.389472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389832+0000","last_change":"2024-04-25T08:42:25.836412+0000","last_active":"2024-04-25T08:42:27.389832+0000","last_peered":"2024-04-25T08:42:27.389832+0000","last_clean":"2024-04-25T08:42:27.389832+0000","last_became_active":"2024-04-25T08:42:25.835926+0000","last_became_peered":"2024-04-25T08:42:25.835926+0000","last_unstale":"2024-04-25T08:42:27.389832+0000","last_undegraded":"2024-04-25T08:42:27.389832+0000","last_fullsized":"2024-04-25T08:42:27.389832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.610141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385472+0000","last_change":"2024-04-25T08:42:21.366095+0000","last_active":"2024-04-25T08:42:27.385472+0000","last_peered":"2024-04-25T08:42:27.385472+0000","last_clean":"2024-04-25T08:42:27.385472+0000","last_became_active":"2024-04-25T08:42:21.365608+0000","last_became_peered":"2024-04-25T08:42:21.365608+0000","last_unstale":"2024-04-25T08:42:27.385472+0000","last_undegraded":"2024-04-25T08:42:27.385472+0000","last_fullsized":"2024-04-25T08:42:27.385472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677918+0000","last_change":"2024-04-25T08:42:20.347127+0000","last_active":"2024-04-25T08:42:28.677918+0000","last_peered":"2024-04-25T08:42:28.677918+0000","last_clean":"2024-04-25T08:42:28.677918+0000","last_became_active":"2024-04-25T08:42:20.346840+0000","last_became_peered":"2024-04-25T08:42:20.346840+0000","last_unstale":"2024-04-25T08:42:28.677918+0000","last_undegraded":"2024-04-25T08:42:28.677918+0000","last_fullsized":"2024-04-25T08:42:28.677918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384051+0000","last_change":"2024-04-25T08:42:25.832430+0000","last_active":"2024-04-25T08:42:27.384051+0000","last_peered":"2024-04-25T08:42:27.384051+0000","last_clean":"2024-04-25T08:42:27.384051+0000","last_became_active":"2024-04-25T08:42:25.832026+0000","last_became_peered":"2024-04-25T08:42:25.832026+0000","last_unstale":"2024-04-25T08:42:27.384051+0000","last_undegraded":"2024-04-25T08:42:27.384051+0000","last_fullsized":"2024-04-25T08:42:27.384051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:39.589658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.679001+0000","last_change":"2024-04-25T08:42:21.340393+0000","last_active":"2024-04-25T08:42:28.679001+0000","last_peered":"2024-04-25T08:42:28.679001+0000","last_clean":"2024-04-25T08:42:28.679001+0000","last_became_active":"2024-04-25T08:42:21.340043+0000","last_became_peered":"2024-04-25T08:42:21.340043+0000","last_unstale":"2024-04-25T08:42:28.679001+0000","last_undegraded":"2024-04-25T08:42:28.679001+0000","last_fullsized":"2024-04-25T08:42:28.679001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684534+0000","last_change":"2024-04-25T08:42:20.360592+0000","last_active":"2024-04-25T08:42:28.684534+0000","last_peered":"2024-04-25T08:42:28.684534+0000","last_clean":"2024-04-25T08:42:28.684534+0000","last_became_active":"2024-04-25T08:42:20.360361+0000","last_became_peered":"2024-04-25T08:42:20.360361+0000","last_unstale":"2024-04-25T08:42:28.684534+0000","last_undegraded":"2024-04-25T08:42:28.684534+0000","last_fullsized":"2024-04-25T08:42:28.684534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:53.741991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682832+0000","last_change":"2024-04-25T08:42:25.834135+0000","last_active":"2024-04-25T08:42:28.682832+0000","last_peered":"2024-04-25T08:42:28.682832+0000","last_clean":"2024-04-25T08:42:28.682832+0000","last_became_active":"2024-04-25T08:42:25.833757+0000","last_became_peered":"2024-04-25T08:42:25.833757+0000","last_unstale":"2024-04-25T08:42:28.682832+0000","last_undegraded":"2024-04-25T08:42:28.682832+0000","last_fullsized":"2024-04-25T08:42:28.682832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583189+0000","last_change":"2024-04-25T08:42:21.366345+0000","last_active":"2024-04-25T08:42:27.583189+0000","last_peered":"2024-04-25T08:42:27.583189+0000","last_clean":"2024-04-25T08:42:27.583189+0000","last_became_active":"2024-04-25T08:42:21.365644+0000","last_became_peered":"2024-04-25T08:42:21.365644+0000","last_unstale":"2024-04-25T08:42:27.583189+0000","last_undegraded":"2024-04-25T08:42:27.583189+0000","last_fullsized":"2024-04-25T08:42:27.583189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.679043+0000","last_change":"2024-04-25T08:42:20.344087+0000","last_active":"2024-04-25T08:42:28.679043+0000","last_peered":"2024-04-25T08:42:28.679043+0000","last_clean":"2024-04-25T08:42:28.679043+0000","last_became_active":"2024-04-25T08:42:20.343799+0000","last_became_peered":"2024-04-25T08:42:20.343799+0000","last_unstale":"2024-04-25T08:42:28.679043+0000","last_undegraded":"2024-04-25T08:42:28.679043+0000","last_fullsized":"2024-04-25T08:42:28.679043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.823720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678152+0000","last_change":"2024-04-25T08:42:25.412243+0000","last_active":"2024-04-25T08:42:28.678152+0000","last_peered":"2024-04-25T08:42:28.678152+0000","last_clean":"2024-04-25T08:42:28.678152+0000","last_became_active":"2024-04-25T08:42:25.406351+0000","last_became_peered":"2024-04-25T08:42:25.406351+0000","last_unstale":"2024-04-25T08:42:28.678152+0000","last_undegraded":"2024-04-25T08:42:28.678152+0000","last_fullsized":"2024-04-25T08:42:28.678152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:42.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390626+0000","last_change":"2024-04-25T08:42:21.366092+0000","last_active":"2024-04-25T08:42:27.390626+0000","last_peered":"2024-04-25T08:42:27.390626+0000","last_clean":"2024-04-25T08:42:27.390626+0000","last_became_active":"2024-04-25T08:42:21.365152+0000","last_became_peered":"2024-04-25T08:42:21.365152+0000","last_unstale":"2024-04-25T08:42:27.390626+0000","last_undegraded":"2024-04-25T08:42:27.390626+0000","last_fullsized":"2024-04-25T08:42:27.390626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:29.830442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582739+0000","last_change":"2024-04-25T08:42:20.352850+0000","last_active":"2024-04-25T08:42:27.582739+0000","last_peered":"2024-04-25T08:42:27.582739+0000","last_clean":"2024-04-25T08:42:27.582739+0000","last_became_active":"2024-04-25T08:42:20.352047+0000","last_became_peered":"2024-04-25T08:42:20.352047+0000","last_unstale":"2024-04-25T08:42:27.582739+0000","last_undegraded":"2024-04-25T08:42:27.582739+0000","last_fullsized":"2024-04-25T08:42:27.582739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678979+0000","last_change":"2024-04-25T08:42:25.405939+0000","last_active":"2024-04-25T08:42:28.678979+0000","last_peered":"2024-04-25T08:42:28.678979+0000","last_clean":"2024-04-25T08:42:28.678979+0000","last_became_active":"2024-04-25T08:42:25.405333+0000","last_became_peered":"2024-04-25T08:42:25.405333+0000","last_unstale":"2024-04-25T08:42:28.678979+0000","last_undegraded":"2024-04-25T08:42:28.678979+0000","last_fullsized":"2024-04-25T08:42:28.678979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678442+0000","last_change":"2024-04-25T08:42:21.338660+0000","last_active":"2024-04-25T08:42:28.678442+0000","last_peered":"2024-04-25T08:42:28.678442+0000","last_clean":"2024-04-25T08:42:28.678442+0000","last_became_active":"2024-04-25T08:42:21.338357+0000","last_became_peered":"2024-04-25T08:42:21.338357+0000","last_unstale":"2024-04-25T08:42:28.678442+0000","last_undegraded":"2024-04-25T08:42:28.678442+0000","last_fullsized":"2024-04-25T08:42:28.678442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583032+0000","last_change":"2024-04-25T08:42:20.352888+0000","last_active":"2024-04-25T08:42:28.583032+0000","last_peered":"2024-04-25T08:42:28.583032+0000","last_clean":"2024-04-25T08:42:28.583032+0000","last_became_active":"2024-04-25T08:42:20.351108+0000","last_became_peered":"2024-04-25T08:42:20.351108+0000","last_unstale":"2024-04-25T08:42:28.583032+0000","last_undegraded":"2024-04-25T08:42:28.583032+0000","last_fullsized":"2024-04-25T08:42:28.583032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678770+0000","last_change":"2024-04-25T08:42:25.409801+0000","last_active":"2024-04-25T08:42:28.678770+0000","last_peered":"2024-04-25T08:42:28.678770+0000","last_clean":"2024-04-25T08:42:28.678770+0000","last_became_active":"2024-04-25T08:42:25.409331+0000","last_became_peered":"2024-04-25T08:42:25.409331+0000","last_unstale":"2024-04-25T08:42:28.678770+0000","last_undegraded":"2024-04-25T08:42:28.678770+0000","last_fullsized":"2024-04-25T08:42:28.678770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682464+0000","last_change":"2024-04-25T08:42:21.363349+0000","last_active":"2024-04-25T08:42:28.682464+0000","last_peered":"2024-04-25T08:42:28.682464+0000","last_clean":"2024-04-25T08:42:28.682464+0000","last_became_active":"2024-04-25T08:42:21.362903+0000","last_became_peered":"2024-04-25T08:42:21.362903+0000","last_unstale":"2024-04-25T08:42:28.682464+0000","last_undegraded":"2024-04-25T08:42:28.682464+0000","last_fullsized":"2024-04-25T08:42:28.682464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:48.553185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683717+0000","last_change":"2024-04-25T08:42:20.356423+0000","last_active":"2024-04-25T08:42:28.683717+0000","last_peered":"2024-04-25T08:42:28.683717+0000","last_clean":"2024-04-25T08:42:28.683717+0000","last_became_active":"2024-04-25T08:42:20.356110+0000","last_became_peered":"2024-04-25T08:42:20.356110+0000","last_unstale":"2024-04-25T08:42:28.683717+0000","last_undegraded":"2024-04-25T08:42:28.683717+0000","last_fullsized":"2024-04-25T08:42:28.683717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:44.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683389+0000","last_change":"2024-04-25T08:42:25.834561+0000","last_active":"2024-04-25T08:42:28.683389+0000","last_peered":"2024-04-25T08:42:28.683389+0000","last_clean":"2024-04-25T08:42:28.683389+0000","last_became_active":"2024-04-25T08:42:25.834038+0000","last_became_peered":"2024-04-25T08:42:25.834038+0000","last_unstale":"2024-04-25T08:42:28.683389+0000","last_undegraded":"2024-04-25T08:42:28.683389+0000","last_fullsized":"2024-04-25T08:42:28.683389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:09.194186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384703+0000","last_change":"2024-04-25T08:42:21.369030+0000","last_active":"2024-04-25T08:42:27.384703+0000","last_peered":"2024-04-25T08:42:27.384703+0000","last_clean":"2024-04-25T08:42:27.384703+0000","last_became_active":"2024-04-25T08:42:21.368760+0000","last_became_peered":"2024-04-25T08:42:21.368760+0000","last_unstale":"2024-04-25T08:42:27.384703+0000","last_undegraded":"2024-04-25T08:42:27.384703+0000","last_fullsized":"2024-04-25T08:42:27.384703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582740+0000","last_change":"2024-04-25T08:42:20.359912+0000","last_active":"2024-04-25T08:42:27.582740+0000","last_peered":"2024-04-25T08:42:27.582740+0000","last_clean":"2024-04-25T08:42:27.582740+0000","last_became_active":"2024-04-25T08:42:20.359616+0000","last_became_peered":"2024-04-25T08:42:20.359616+0000","last_unstale":"2024-04-25T08:42:27.582740+0000","last_undegraded":"2024-04-25T08:42:27.582740+0000","last_fullsized":"2024-04-25T08:42:27.582740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683854+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:42:28.683854+0000","last_peered":"2024-04-25T08:42:28.683854+0000","last_clean":"2024-04-25T08:42:28.683854+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:42:28.683854+0000","last_undegraded":"2024-04-25T08:42:28.683854+0000","last_fullsized":"2024-04-25T08:42:28.683854+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389063+0000","last_change":"2024-04-25T08:42:25.836852+0000","last_active":"2024-04-25T08:42:27.389063+0000","last_peered":"2024-04-25T08:42:27.389063+0000","last_clean":"2024-04-25T08:42:27.389063+0000","last_became_active":"2024-04-25T08:42:25.836428+0000","last_became_peered":"2024-04-25T08:42:25.836428+0000","last_unstale":"2024-04-25T08:42:27.389063+0000","last_undegraded":"2024-04-25T08:42:27.389063+0000","last_fullsized":"2024-04-25T08:42:27.389063+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.544013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684361+0000","last_change":"2024-04-25T08:42:21.376058+0000","last_active":"2024-04-25T08:42:28.684361+0000","last_peered":"2024-04-25T08:42:28.684361+0000","last_clean":"2024-04-25T08:42:28.684361+0000","last_became_active":"2024-04-25T08:42:21.375825+0000","last_became_peered":"2024-04-25T08:42:21.375825+0000","last_unstale":"2024-04-25T08:42:28.684361+0000","last_undegraded":"2024-04-25T08:42:28.684361+0000","last_fullsized":"2024-04-25T08:42:28.684361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:24.495752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583044+0000","last_change":"2024-04-25T08:42:20.353588+0000","last_active":"2024-04-25T08:42:28.583044+0000","last_peered":"2024-04-25T08:42:28.583044+0000","last_clean":"2024-04-25T08:42:28.583044+0000","last_became_active":"2024-04-25T08:42:20.353335+0000","last_became_peered":"2024-04-25T08:42:20.353335+0000","last_unstale":"2024-04-25T08:42:28.583044+0000","last_undegraded":"2024-04-25T08:42:28.583044+0000","last_fullsized":"2024-04-25T08:42:28.583044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388628+0000","last_change":"2024-04-25T08:42:25.833110+0000","last_active":"2024-04-25T08:42:27.388628+0000","last_peered":"2024-04-25T08:42:27.388628+0000","last_clean":"2024-04-25T08:42:27.388628+0000","last_became_active":"2024-04-25T08:42:25.832656+0000","last_became_peered":"2024-04-25T08:42:25.832656+0000","last_unstale":"2024-04-25T08:42:27.388628+0000","last_undegraded":"2024-04-25T08:42:27.388628+0000","last_fullsized":"2024-04-25T08:42:27.388628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.971525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582934+0000","last_change":"2024-04-25T08:42:21.368021+0000","last_active":"2024-04-25T08:42:27.582934+0000","last_peered":"2024-04-25T08:42:27.582934+0000","last_clean":"2024-04-25T08:42:27.582934+0000","last_became_active":"2024-04-25T08:42:21.367516+0000","last_became_peered":"2024-04-25T08:42:21.367516+0000","last_unstale":"2024-04-25T08:42:27.582934+0000","last_undegraded":"2024-04-25T08:42:27.582934+0000","last_fullsized":"2024-04-25T08:42:27.582934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:24.495752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677925+0000","last_change":"2024-04-25T08:42:20.340089+0000","last_active":"2024-04-25T08:42:28.677925+0000","last_peered":"2024-04-25T08:42:28.677925+0000","last_clean":"2024-04-25T08:42:28.677925+0000","last_became_active":"2024-04-25T08:42:20.338267+0000","last_became_peered":"2024-04-25T08:42:20.338267+0000","last_unstale":"2024-04-25T08:42:28.677925+0000","last_undegraded":"2024-04-25T08:42:28.677925+0000","last_fullsized":"2024-04-25T08:42:28.677925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:44.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682886+0000","last_change":"2024-04-25T08:42:25.418168+0000","last_active":"2024-04-25T08:42:28.682886+0000","last_peered":"2024-04-25T08:42:28.682886+0000","last_clean":"2024-04-25T08:42:28.682886+0000","last_became_active":"2024-04-25T08:42:25.417852+0000","last_became_peered":"2024-04-25T08:42:25.417852+0000","last_unstale":"2024-04-25T08:42:28.682886+0000","last_undegraded":"2024-04-25T08:42:28.682886+0000","last_fullsized":"2024-04-25T08:42:28.682886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684774+0000","last_change":"2024-04-25T08:42:21.370059+0000","last_active":"2024-04-25T08:42:28.684774+0000","last_peered":"2024-04-25T08:42:28.684774+0000","last_clean":"2024-04-25T08:42:28.684774+0000","last_became_active":"2024-04-25T08:42:21.359776+0000","last_became_peered":"2024-04-25T08:42:21.359776+0000","last_unstale":"2024-04-25T08:42:28.684774+0000","last_undegraded":"2024-04-25T08:42:28.684774+0000","last_fullsized":"2024-04-25T08:42:28.684774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:21.412723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583070+0000","last_change":"2024-04-25T08:42:20.351406+0000","last_active":"2024-04-25T08:42:28.583070+0000","last_peered":"2024-04-25T08:42:28.583070+0000","last_clean":"2024-04-25T08:42:28.583070+0000","last_became_active":"2024-04-25T08:42:20.351114+0000","last_became_peered":"2024-04-25T08:42:20.351114+0000","last_unstale":"2024-04-25T08:42:28.583070+0000","last_undegraded":"2024-04-25T08:42:28.583070+0000","last_fullsized":"2024-04-25T08:42:28.583070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582385+0000","last_change":"2024-04-25T08:42:25.835941+0000","last_active":"2024-04-25T08:42:27.582385+0000","last_peered":"2024-04-25T08:42:27.582385+0000","last_clean":"2024-04-25T08:42:27.582385+0000","last_became_active":"2024-04-25T08:42:25.834320+0000","last_became_peered":"2024-04-25T08:42:25.834320+0000","last_unstale":"2024-04-25T08:42:27.582385+0000","last_undegraded":"2024-04-25T08:42:27.582385+0000","last_fullsized":"2024-04-25T08:42:27.582385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:32.335929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684548+0000","last_change":"2024-04-25T08:42:21.363962+0000","last_active":"2024-04-25T08:42:28.684548+0000","last_peered":"2024-04-25T08:42:28.684548+0000","last_clean":"2024-04-25T08:42:28.684548+0000","last_became_active":"2024-04-25T08:42:21.363695+0000","last_became_peered":"2024-04-25T08:42:21.363695+0000","last_unstale":"2024-04-25T08:42:28.684548+0000","last_undegraded":"2024-04-25T08:42:28.684548+0000","last_fullsized":"2024-04-25T08:42:28.684548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:15.923334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.391021+0000","last_change":"2024-04-25T08:42:20.345476+0000","last_active":"2024-04-25T08:42:27.391021+0000","last_peered":"2024-04-25T08:42:27.391021+0000","last_clean":"2024-04-25T08:42:27.391021+0000","last_became_active":"2024-04-25T08:42:20.345085+0000","last_became_peered":"2024-04-25T08:42:20.345085+0000","last_unstale":"2024-04-25T08:42:27.391021+0000","last_undegraded":"2024-04-25T08:42:27.391021+0000","last_fullsized":"2024-04-25T08:42:27.391021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:19.301171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389733+0000","last_change":"2024-04-25T08:42:25.410444+0000","last_active":"2024-04-25T08:42:27.389733+0000","last_peered":"2024-04-25T08:42:27.389733+0000","last_clean":"2024-04-25T08:42:27.389733+0000","last_became_active":"2024-04-25T08:42:25.409574+0000","last_became_peered":"2024-04-25T08:42:25.409574+0000","last_unstale":"2024-04-25T08:42:27.389733+0000","last_undegraded":"2024-04-25T08:42:27.389733+0000","last_fullsized":"2024-04-25T08:42:27.389733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.544013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683914+0000","last_change":"2024-04-25T08:42:21.357665+0000","last_active":"2024-04-25T08:42:28.683914+0000","last_peered":"2024-04-25T08:42:28.683914+0000","last_clean":"2024-04-25T08:42:28.683914+0000","last_became_active":"2024-04-25T08:42:21.357081+0000","last_became_peered":"2024-04-25T08:42:21.357081+0000","last_unstale":"2024-04-25T08:42:28.683914+0000","last_undegraded":"2024-04-25T08:42:28.683914+0000","last_fullsized":"2024-04-25T08:42:28.683914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:24.505703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583188+0000","last_change":"2024-04-25T08:42:20.345496+0000","last_active":"2024-04-25T08:42:28.583188+0000","last_peered":"2024-04-25T08:42:28.583188+0000","last_clean":"2024-04-25T08:42:28.583188+0000","last_became_active":"2024-04-25T08:42:20.345230+0000","last_became_peered":"2024-04-25T08:42:20.345230+0000","last_unstale":"2024-04-25T08:42:28.583188+0000","last_undegraded":"2024-04-25T08:42:28.583188+0000","last_fullsized":"2024-04-25T08:42:28.583188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:33.260664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683216+0000","last_change":"2024-04-25T08:42:25.412932+0000","last_active":"2024-04-25T08:42:28.683216+0000","last_peered":"2024-04-25T08:42:28.683216+0000","last_clean":"2024-04-25T08:42:28.683216+0000","last_became_active":"2024-04-25T08:42:25.412212+0000","last_became_peered":"2024-04-25T08:42:25.412212+0000","last_unstale":"2024-04-25T08:42:28.683216+0000","last_undegraded":"2024-04-25T08:42:28.683216+0000","last_fullsized":"2024-04-25T08:42:28.683216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388745+0000","last_change":"2024-04-25T08:42:21.351855+0000","last_active":"2024-04-25T08:42:27.388745+0000","last_peered":"2024-04-25T08:42:27.388745+0000","last_clean":"2024-04-25T08:42:27.388745+0000","last_became_active":"2024-04-25T08:42:21.351412+0000","last_became_peered":"2024-04-25T08:42:21.351412+0000","last_unstale":"2024-04-25T08:42:27.388745+0000","last_undegraded":"2024-04-25T08:42:27.388745+0000","last_fullsized":"2024-04-25T08:42:27.388745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388741+0000","last_change":"2024-04-25T08:42:20.350412+0000","last_active":"2024-04-25T08:42:27.388741+0000","last_peered":"2024-04-25T08:42:27.388741+0000","last_clean":"2024-04-25T08:42:27.388741+0000","last_became_active":"2024-04-25T08:42:20.350113+0000","last_became_peered":"2024-04-25T08:42:20.350113+0000","last_unstale":"2024-04-25T08:42:27.388741+0000","last_undegraded":"2024-04-25T08:42:27.388741+0000","last_fullsized":"2024-04-25T08:42:27.388741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385384+0000","last_change":"2024-04-25T08:42:25.838652+0000","last_active":"2024-04-25T08:42:27.385384+0000","last_peered":"2024-04-25T08:42:27.385384+0000","last_clean":"2024-04-25T08:42:27.385384+0000","last_became_active":"2024-04-25T08:42:25.838168+0000","last_became_peered":"2024-04-25T08:42:25.838168+0000","last_unstale":"2024-04-25T08:42:27.385384+0000","last_undegraded":"2024-04-25T08:42:27.385384+0000","last_fullsized":"2024-04-25T08:42:27.385384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:46.882274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684243+0000","last_change":"2024-04-25T08:42:21.373789+0000","last_active":"2024-04-25T08:42:28.684243+0000","last_peered":"2024-04-25T08:42:28.684243+0000","last_clean":"2024-04-25T08:42:28.684243+0000","last_became_active":"2024-04-25T08:42:21.372582+0000","last_became_peered":"2024-04-25T08:42:21.372582+0000","last_unstale":"2024-04-25T08:42:28.684243+0000","last_undegraded":"2024-04-25T08:42:28.684243+0000","last_fullsized":"2024-04-25T08:42:28.684243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.525554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582667+0000","last_change":"2024-04-25T08:42:20.357655+0000","last_active":"2024-04-25T08:42:27.582667+0000","last_peered":"2024-04-25T08:42:27.582667+0000","last_clean":"2024-04-25T08:42:27.582667+0000","last_became_active":"2024-04-25T08:42:20.357354+0000","last_became_peered":"2024-04-25T08:42:20.357354+0000","last_unstale":"2024-04-25T08:42:27.582667+0000","last_undegraded":"2024-04-25T08:42:27.582667+0000","last_fullsized":"2024-04-25T08:42:27.582667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385040+0000","last_change":"2024-04-25T08:42:25.835505+0000","last_active":"2024-04-25T08:42:27.385040+0000","last_peered":"2024-04-25T08:42:27.385040+0000","last_clean":"2024-04-25T08:42:27.385040+0000","last_became_active":"2024-04-25T08:42:25.835079+0000","last_became_peered":"2024-04-25T08:42:25.835079+0000","last_unstale":"2024-04-25T08:42:27.385040+0000","last_undegraded":"2024-04-25T08:42:27.385040+0000","last_fullsized":"2024-04-25T08:42:27.385040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:00.799395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682748+0000","last_change":"2024-04-25T08:42:21.366223+0000","last_active":"2024-04-25T08:42:28.682748+0000","last_peered":"2024-04-25T08:42:28.682748+0000","last_clean":"2024-04-25T08:42:28.682748+0000","last_became_active":"2024-04-25T08:42:21.365913+0000","last_became_peered":"2024-04-25T08:42:21.365913+0000","last_unstale":"2024-04-25T08:42:28.682748+0000","last_undegraded":"2024-04-25T08:42:28.682748+0000","last_fullsized":"2024-04-25T08:42:28.682748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682183+0000","last_change":"2024-04-25T08:42:20.354779+0000","last_active":"2024-04-25T08:42:28.682183+0000","last_peered":"2024-04-25T08:42:28.682183+0000","last_clean":"2024-04-25T08:42:28.682183+0000","last_became_active":"2024-04-25T08:42:20.354504+0000","last_became_peered":"2024-04-25T08:42:20.354504+0000","last_unstale":"2024-04-25T08:42:28.682183+0000","last_undegraded":"2024-04-25T08:42:28.682183+0000","last_fullsized":"2024-04-25T08:42:28.682183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.311211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583098+0000","last_change":"2024-04-25T08:42:25.832677+0000","last_active":"2024-04-25T08:42:27.583098+0000","last_peered":"2024-04-25T08:42:27.583098+0000","last_clean":"2024-04-25T08:42:27.583098+0000","last_became_active":"2024-04-25T08:42:25.832204+0000","last_became_peered":"2024-04-25T08:42:25.832204+0000","last_unstale":"2024-04-25T08:42:27.583098+0000","last_undegraded":"2024-04-25T08:42:27.583098+0000","last_fullsized":"2024-04-25T08:42:27.583098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:02.694380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684296+0000","last_change":"2024-04-25T08:42:21.365924+0000","last_active":"2024-04-25T08:42:28.684296+0000","last_peered":"2024-04-25T08:42:28.684296+0000","last_clean":"2024-04-25T08:42:28.684296+0000","last_became_active":"2024-04-25T08:42:21.365581+0000","last_became_peered":"2024-04-25T08:42:21.365581+0000","last_unstale":"2024-04-25T08:42:28.684296+0000","last_undegraded":"2024-04-25T08:42:28.684296+0000","last_fullsized":"2024-04-25T08:42:28.684296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.923264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:36.452272+0000","last_change":"2024-04-25T08:42:20.354736+0000","last_active":"2024-04-25T08:42:36.452272+0000","last_peered":"2024-04-25T08:42:36.452272+0000","last_clean":"2024-04-25T08:42:36.452272+0000","last_became_active":"2024-04-25T08:42:20.354322+0000","last_became_peered":"2024-04-25T08:42:20.354322+0000","last_unstale":"2024-04-25T08:42:36.452272+0000","last_undegraded":"2024-04-25T08:42:36.452272+0000","last_fullsized":"2024-04-25T08:42:36.452272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678428+0000","last_change":"2024-04-25T08:42:25.414706+0000","last_active":"2024-04-25T08:42:28.678428+0000","last_peered":"2024-04-25T08:42:28.678428+0000","last_clean":"2024-04-25T08:42:28.678428+0000","last_became_active":"2024-04-25T08:42:25.414302+0000","last_became_peered":"2024-04-25T08:42:25.414302+0000","last_unstale":"2024-04-25T08:42:28.678428+0000","last_undegraded":"2024-04-25T08:42:28.678428+0000","last_fullsized":"2024-04-25T08:42:28.678428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390755+0000","last_change":"2024-04-25T08:42:21.361175+0000","last_active":"2024-04-25T08:42:27.390755+0000","last_peered":"2024-04-25T08:42:27.390755+0000","last_clean":"2024-04-25T08:42:27.390755+0000","last_became_active":"2024-04-25T08:42:21.358285+0000","last_became_peered":"2024-04-25T08:42:21.358285+0000","last_unstale":"2024-04-25T08:42:27.390755+0000","last_undegraded":"2024-04-25T08:42:27.390755+0000","last_fullsized":"2024-04-25T08:42:27.390755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:45.573590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677940+0000","last_change":"2024-04-25T08:42:20.357678+0000","last_active":"2024-04-25T08:42:28.677940+0000","last_peered":"2024-04-25T08:42:28.677940+0000","last_clean":"2024-04-25T08:42:28.677940+0000","last_became_active":"2024-04-25T08:42:20.357378+0000","last_became_peered":"2024-04-25T08:42:20.357378+0000","last_unstale":"2024-04-25T08:42:28.677940+0000","last_undegraded":"2024-04-25T08:42:28.677940+0000","last_fullsized":"2024-04-25T08:42:28.677940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390203+0000","last_change":"2024-04-25T08:42:25.406294+0000","last_active":"2024-04-25T08:42:27.390203+0000","last_peered":"2024-04-25T08:42:27.390203+0000","last_clean":"2024-04-25T08:42:27.390203+0000","last_became_active":"2024-04-25T08:42:25.405833+0000","last_became_peered":"2024-04-25T08:42:25.405833+0000","last_unstale":"2024-04-25T08:42:27.390203+0000","last_undegraded":"2024-04-25T08:42:27.390203+0000","last_fullsized":"2024-04-25T08:42:27.390203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:42.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683217+0000","last_change":"2024-04-25T08:42:21.363445+0000","last_active":"2024-04-25T08:42:28.683217+0000","last_peered":"2024-04-25T08:42:28.683217+0000","last_clean":"2024-04-25T08:42:28.683217+0000","last_became_active":"2024-04-25T08:42:21.363129+0000","last_became_peered":"2024-04-25T08:42:21.363129+0000","last_unstale":"2024-04-25T08:42:28.683217+0000","last_undegraded":"2024-04-25T08:42:28.683217+0000","last_fullsized":"2024-04-25T08:42:28.683217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583076+0000","last_change":"2024-04-25T08:42:20.352637+0000","last_active":"2024-04-25T08:42:27.583076+0000","last_peered":"2024-04-25T08:42:27.583076+0000","last_clean":"2024-04-25T08:42:27.583076+0000","last_became_active":"2024-04-25T08:42:20.351776+0000","last_became_peered":"2024-04-25T08:42:20.351776+0000","last_unstale":"2024-04-25T08:42:27.583076+0000","last_undegraded":"2024-04-25T08:42:27.583076+0000","last_fullsized":"2024-04-25T08:42:27.583076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:53.741991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384221+0000","last_change":"2024-04-25T08:42:25.406488+0000","last_active":"2024-04-25T08:42:27.384221+0000","last_peered":"2024-04-25T08:42:27.384221+0000","last_clean":"2024-04-25T08:42:27.384221+0000","last_became_active":"2024-04-25T08:42:25.405903+0000","last_became_peered":"2024-04-25T08:42:25.405903+0000","last_unstale":"2024-04-25T08:42:27.384221+0000","last_undegraded":"2024-04-25T08:42:27.384221+0000","last_fullsized":"2024-04-25T08:42:27.384221+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:52.897838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583032+0000","last_change":"2024-04-25T08:42:21.376939+0000","last_active":"2024-04-25T08:42:27.583032+0000","last_peered":"2024-04-25T08:42:27.583032+0000","last_clean":"2024-04-25T08:42:27.583032+0000","last_became_active":"2024-04-25T08:42:21.376693+0000","last_became_peered":"2024-04-25T08:42:21.376693+0000","last_unstale":"2024-04-25T08:42:27.583032+0000","last_undegraded":"2024-04-25T08:42:27.583032+0000","last_fullsized":"2024-04-25T08:42:27.583032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:16.561880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389379+0000","last_change":"2024-04-25T08:42:20.355744+0000","last_active":"2024-04-25T08:42:27.389379+0000","last_peered":"2024-04-25T08:42:27.389379+0000","last_clean":"2024-04-25T08:42:27.389379+0000","last_became_active":"2024-04-25T08:42:20.355365+0000","last_became_peered":"2024-04-25T08:42:20.355365+0000","last_unstale":"2024-04-25T08:42:27.389379+0000","last_undegraded":"2024-04-25T08:42:27.389379+0000","last_fullsized":"2024-04-25T08:42:27.389379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384187+0000","last_change":"2024-04-25T08:42:25.833881+0000","last_active":"2024-04-25T08:42:27.384187+0000","last_peered":"2024-04-25T08:42:27.384187+0000","last_clean":"2024-04-25T08:42:27.384187+0000","last_became_active":"2024-04-25T08:42:25.833475+0000","last_became_peered":"2024-04-25T08:42:25.833475+0000","last_unstale":"2024-04-25T08:42:27.384187+0000","last_undegraded":"2024-04-25T08:42:27.384187+0000","last_fullsized":"2024-04-25T08:42:27.384187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:49.130328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683043+0000","last_change":"2024-04-25T08:42:25.835305+0000","last_active":"2024-04-25T08:42:28.683043+0000","last_peered":"2024-04-25T08:42:28.683043+0000","last_clean":"2024-04-25T08:42:28.683043+0000","last_became_active":"2024-04-25T08:42:25.834632+0000","last_became_peered":"2024-04-25T08:42:25.834632+0000","last_unstale":"2024-04-25T08:42:28.683043+0000","last_undegraded":"2024-04-25T08:42:28.683043+0000","last_fullsized":"2024-04-25T08:42:28.683043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678200+0000","last_change":"2024-04-25T08:42:21.350716+0000","last_active":"2024-04-25T08:42:28.678200+0000","last_peered":"2024-04-25T08:42:28.678200+0000","last_clean":"2024-04-25T08:42:28.678200+0000","last_became_active":"2024-04-25T08:42:21.349383+0000","last_became_peered":"2024-04-25T08:42:21.349383+0000","last_unstale":"2024-04-25T08:42:28.678200+0000","last_undegraded":"2024-04-25T08:42:28.678200+0000","last_fullsized":"2024-04-25T08:42:28.678200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582596+0000","last_change":"2024-04-25T08:42:20.345077+0000","last_active":"2024-04-25T08:42:27.582596+0000","last_peered":"2024-04-25T08:42:27.582596+0000","last_clean":"2024-04-25T08:42:27.582596+0000","last_became_active":"2024-04-25T08:42:20.344526+0000","last_became_peered":"2024-04-25T08:42:20.344526+0000","last_unstale":"2024-04-25T08:42:27.582596+0000","last_undegraded":"2024-04-25T08:42:27.582596+0000","last_fullsized":"2024-04-25T08:42:27.582596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389292+0000","last_change":"2024-04-25T08:42:25.407527+0000","last_active":"2024-04-25T08:42:27.389292+0000","last_peered":"2024-04-25T08:42:27.389292+0000","last_clean":"2024-04-25T08:42:27.389292+0000","last_became_active":"2024-04-25T08:42:25.407221+0000","last_became_peered":"2024-04-25T08:42:25.407221+0000","last_unstale":"2024-04-25T08:42:27.389292+0000","last_undegraded":"2024-04-25T08:42:27.389292+0000","last_fullsized":"2024-04-25T08:42:27.389292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.610141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384221+0000","last_change":"2024-04-25T08:42:21.351795+0000","last_active":"2024-04-25T08:42:27.384221+0000","last_peered":"2024-04-25T08:42:27.384221+0000","last_clean":"2024-04-25T08:42:27.384221+0000","last_became_active":"2024-04-25T08:42:21.351544+0000","last_became_peered":"2024-04-25T08:42:21.351544+0000","last_unstale":"2024-04-25T08:42:27.384221+0000","last_undegraded":"2024-04-25T08:42:27.384221+0000","last_fullsized":"2024-04-25T08:42:27.384221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384681+0000","last_change":"2024-04-25T08:42:20.354629+0000","last_active":"2024-04-25T08:42:27.384681+0000","last_peered":"2024-04-25T08:42:27.384681+0000","last_clean":"2024-04-25T08:42:27.384681+0000","last_became_active":"2024-04-25T08:42:20.354388+0000","last_became_peered":"2024-04-25T08:42:20.354388+0000","last_unstale":"2024-04-25T08:42:27.384681+0000","last_undegraded":"2024-04-25T08:42:27.384681+0000","last_fullsized":"2024-04-25T08:42:27.384681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.311211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388979+0000","last_change":"2024-04-25T08:42:20.350141+0000","last_active":"2024-04-25T08:42:27.388979+0000","last_peered":"2024-04-25T08:42:27.388979+0000","last_clean":"2024-04-25T08:42:27.388979+0000","last_became_active":"2024-04-25T08:42:20.349168+0000","last_became_peered":"2024-04-25T08:42:20.349168+0000","last_unstale":"2024-04-25T08:42:27.388979+0000","last_undegraded":"2024-04-25T08:42:27.388979+0000","last_fullsized":"2024-04-25T08:42:27.388979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683007+0000","last_change":"2024-04-25T08:42:21.363608+0000","last_active":"2024-04-25T08:42:28.683007+0000","last_peered":"2024-04-25T08:42:28.683007+0000","last_clean":"2024-04-25T08:42:28.683007+0000","last_became_active":"2024-04-25T08:42:21.363152+0000","last_became_peered":"2024-04-25T08:42:21.363152+0000","last_unstale":"2024-04-25T08:42:28.683007+0000","last_undegraded":"2024-04-25T08:42:28.683007+0000","last_fullsized":"2024-04-25T08:42:28.683007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583434+0000","last_change":"2024-04-25T08:42:25.413462+0000","last_active":"2024-04-25T08:42:28.583434+0000","last_peered":"2024-04-25T08:42:28.583434+0000","last_clean":"2024-04-25T08:42:28.583434+0000","last_became_active":"2024-04-25T08:42:25.412613+0000","last_became_peered":"2024-04-25T08:42:25.412613+0000","last_unstale":"2024-04-25T08:42:28.583434+0000","last_undegraded":"2024-04-25T08:42:28.583434+0000","last_fullsized":"2024-04-25T08:42:28.583434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:52.601446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384923+0000","last_change":"2024-04-25T08:42:20.355705+0000","last_active":"2024-04-25T08:42:27.384923+0000","last_peered":"2024-04-25T08:42:27.384923+0000","last_clean":"2024-04-25T08:42:27.384923+0000","last_became_active":"2024-04-25T08:42:20.355433+0000","last_became_peered":"2024-04-25T08:42:20.355433+0000","last_unstale":"2024-04-25T08:42:27.384923+0000","last_undegraded":"2024-04-25T08:42:27.384923+0000","last_fullsized":"2024-04-25T08:42:27.384923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:33.260664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390028+0000","last_change":"2024-04-25T08:42:21.344759+0000","last_active":"2024-04-25T08:42:27.390028+0000","last_peered":"2024-04-25T08:42:27.390028+0000","last_clean":"2024-04-25T08:42:27.390028+0000","last_became_active":"2024-04-25T08:42:21.344560+0000","last_became_peered":"2024-04-25T08:42:21.344560+0000","last_unstale":"2024-04-25T08:42:27.390028+0000","last_undegraded":"2024-04-25T08:42:27.390028+0000","last_fullsized":"2024-04-25T08:42:27.390028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:32.295514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385225+0000","last_change":"2024-04-25T08:42:25.413565+0000","last_active":"2024-04-25T08:42:27.385225+0000","last_peered":"2024-04-25T08:42:27.385225+0000","last_clean":"2024-04-25T08:42:27.385225+0000","last_became_active":"2024-04-25T08:42:25.413198+0000","last_became_peered":"2024-04-25T08:42:25.413198+0000","last_unstale":"2024-04-25T08:42:27.385225+0000","last_undegraded":"2024-04-25T08:42:27.385225+0000","last_fullsized":"2024-04-25T08:42:27.385225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:13.096012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388406+0000","last_change":"2024-04-25T08:42:21.363343+0000","last_active":"2024-04-25T08:42:27.388406+0000","last_peered":"2024-04-25T08:42:27.388406+0000","last_clean":"2024-04-25T08:42:27.388406+0000","last_became_active":"2024-04-25T08:42:21.363101+0000","last_became_peered":"2024-04-25T08:42:21.363101+0000","last_unstale":"2024-04-25T08:42:27.388406+0000","last_undegraded":"2024-04-25T08:42:27.388406+0000","last_fullsized":"2024-04-25T08:42:27.388406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583299+0000","last_change":"2024-04-25T08:42:21.361026+0000","last_active":"2024-04-25T08:42:28.583299+0000","last_peered":"2024-04-25T08:42:28.583299+0000","last_clean":"2024-04-25T08:42:28.583299+0000","last_became_active":"2024-04-25T08:42:21.360012+0000","last_became_peered":"2024-04-25T08:42:21.360012+0000","last_unstale":"2024-04-25T08:42:28.583299+0000","last_undegraded":"2024-04-25T08:42:28.583299+0000","last_fullsized":"2024-04-25T08:42:28.583299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:29.830442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582614+0000","last_change":"2024-04-25T08:42:21.367540+0000","last_active":"2024-04-25T08:42:27.582614+0000","last_peered":"2024-04-25T08:42:27.582614+0000","last_clean":"2024-04-25T08:42:27.582614+0000","last_became_active":"2024-04-25T08:42:21.367291+0000","last_became_peered":"2024-04-25T08:42:21.367291+0000","last_unstale":"2024-04-25T08:42:27.582614+0000","last_undegraded":"2024-04-25T08:42:27.582614+0000","last_fullsized":"2024-04-25T08:42:27.582614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.923264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683000+0000","last_change":"2024-04-25T08:42:21.366788+0000","last_active":"2024-04-25T08:42:28.683000+0000","last_peered":"2024-04-25T08:42:28.683000+0000","last_clean":"2024-04-25T08:42:28.683000+0000","last_became_active":"2024-04-25T08:42:21.365945+0000","last_became_peered":"2024-04-25T08:42:21.365945+0000","last_unstale":"2024-04-25T08:42:28.683000+0000","last_undegraded":"2024-04-25T08:42:28.683000+0000","last_fullsized":"2024-04-25T08:42:28.683000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385215+0000","last_change":"2024-04-25T08:42:21.351963+0000","last_active":"2024-04-25T08:42:27.385215+0000","last_peered":"2024-04-25T08:42:27.385215+0000","last_clean":"2024-04-25T08:42:27.385215+0000","last_became_active":"2024-04-25T08:42:21.351587+0000","last_became_peered":"2024-04-25T08:42:21.351587+0000","last_unstale":"2024-04-25T08:42:27.385215+0000","last_undegraded":"2024-04-25T08:42:27.385215+0000","last_fullsized":"2024-04-25T08:42:27.385215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:30.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678682+0000","last_change":"2024-04-25T08:42:21.363969+0000","last_active":"2024-04-25T08:42:28.678682+0000","last_peered":"2024-04-25T08:42:28.678682+0000","last_clean":"2024-04-25T08:42:28.678682+0000","last_became_active":"2024-04-25T08:42:21.354733+0000","last_became_peered":"2024-04-25T08:42:21.354733+0000","last_unstale":"2024-04-25T08:42:28.678682+0000","last_undegraded":"2024-04-25T08:42:28.678682+0000","last_fullsized":"2024-04-25T08:42:28.678682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389184+0000","last_change":"2024-04-25T08:42:21.375124+0000","last_active":"2024-04-25T08:42:27.389184+0000","last_peered":"2024-04-25T08:42:27.389184+0000","last_clean":"2024-04-25T08:42:27.389184+0000","last_became_active":"2024-04-25T08:42:21.374653+0000","last_became_peered":"2024-04-25T08:42:21.374653+0000","last_unstale":"2024-04-25T08:42:27.389184+0000","last_undegraded":"2024-04-25T08:42:27.389184+0000","last_fullsized":"2024-04-25T08:42:27.389184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385387+0000","last_change":"2024-04-25T08:42:21.362798+0000","last_active":"2024-04-25T08:42:27.385387+0000","last_peered":"2024-04-25T08:42:27.385387+0000","last_clean":"2024-04-25T08:42:27.385387+0000","last_became_active":"2024-04-25T08:42:21.362412+0000","last_became_peered":"2024-04-25T08:42:21.362412+0000","last_unstale":"2024-04-25T08:42:27.385387+0000","last_undegraded":"2024-04-25T08:42:27.385387+0000","last_fullsized":"2024-04-25T08:42:27.385387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582584+0000","last_change":"2024-04-25T08:42:21.368298+0000","last_active":"2024-04-25T08:42:27.582584+0000","last_peered":"2024-04-25T08:42:27.582584+0000","last_clean":"2024-04-25T08:42:27.582584+0000","last_became_active":"2024-04-25T08:42:21.368073+0000","last_became_peered":"2024-04-25T08:42:21.368073+0000","last_unstale":"2024-04-25T08:42:27.582584+0000","last_undegraded":"2024-04-25T08:42:27.582584+0000","last_fullsized":"2024-04-25T08:42:27.582584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384593+0000","last_change":"2024-04-25T08:42:21.356172+0000","last_active":"2024-04-25T08:42:27.384593+0000","last_peered":"2024-04-25T08:42:27.384593+0000","last_clean":"2024-04-25T08:42:27.384593+0000","last_became_active":"2024-04-25T08:42:21.355920+0000","last_became_peered":"2024-04-25T08:42:21.355920+0000","last_unstale":"2024-04-25T08:42:27.384593+0000","last_undegraded":"2024-04-25T08:42:27.384593+0000","last_fullsized":"2024-04-25T08:42:27.384593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582542+0000","last_change":"2024-04-25T08:42:21.376313+0000","last_active":"2024-04-25T08:42:27.582542+0000","last_peered":"2024-04-25T08:42:27.582542+0000","last_clean":"2024-04-25T08:42:27.582542+0000","last_became_active":"2024-04-25T08:42:21.376047+0000","last_became_peered":"2024-04-25T08:42:21.376047+0000","last_unstale":"2024-04-25T08:42:27.582542+0000","last_undegraded":"2024-04-25T08:42:27.582542+0000","last_fullsized":"2024-04-25T08:42:27.582542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:24.505703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682755+0000","last_change":"2024-04-25T08:42:21.367017+0000","last_active":"2024-04-25T08:42:28.682755+0000","last_peered":"2024-04-25T08:42:28.682755+0000","last_clean":"2024-04-25T08:42:28.682755+0000","last_became_active":"2024-04-25T08:42:21.366480+0000","last_became_peered":"2024-04-25T08:42:21.366480+0000","last_unstale":"2024-04-25T08:42:28.682755+0000","last_undegraded":"2024-04-25T08:42:28.682755+0000","last_fullsized":"2024-04-25T08:42:28.682755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388881+0000","last_change":"2024-04-25T08:42:21.354458+0000","last_active":"2024-04-25T08:42:27.388881+0000","last_peered":"2024-04-25T08:42:27.388881+0000","last_clean":"2024-04-25T08:42:27.388881+0000","last_became_active":"2024-04-25T08:42:21.351083+0000","last_became_peered":"2024-04-25T08:42:21.351083+0000","last_unstale":"2024-04-25T08:42:27.388881+0000","last_undegraded":"2024-04-25T08:42:27.388881+0000","last_fullsized":"2024-04-25T08:42:27.388881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684511+0000","last_change":"2024-04-25T08:42:21.369898+0000","last_active":"2024-04-25T08:42:28.684511+0000","last_peered":"2024-04-25T08:42:28.684511+0000","last_clean":"2024-04-25T08:42:28.684511+0000","last_became_active":"2024-04-25T08:42:21.369443+0000","last_became_peered":"2024-04-25T08:42:21.369443+0000","last_unstale":"2024-04-25T08:42:28.684511+0000","last_undegraded":"2024-04-25T08:42:28.684511+0000","last_fullsized":"2024-04-25T08:42:28.684511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:16.561880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583661+0000","last_change":"2024-04-25T08:42:21.353156+0000","last_active":"2024-04-25T08:42:28.583661+0000","last_peered":"2024-04-25T08:42:28.583661+0000","last_clean":"2024-04-25T08:42:28.583661+0000","last_became_active":"2024-04-25T08:42:21.352670+0000","last_became_peered":"2024-04-25T08:42:21.352670+0000","last_unstale":"2024-04-25T08:42:28.583661+0000","last_undegraded":"2024-04-25T08:42:28.583661+0000","last_fullsized":"2024-04-25T08:42:28.583661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:45.573590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583361+0000","last_change":"2024-04-25T08:42:21.376357+0000","last_active":"2024-04-25T08:42:27.583361+0000","last_peered":"2024-04-25T08:42:27.583361+0000","last_clean":"2024-04-25T08:42:27.583361+0000","last_became_active":"2024-04-25T08:42:21.376092+0000","last_became_peered":"2024-04-25T08:42:21.376092+0000","last_unstale":"2024-04-25T08:42:27.583361+0000","last_undegraded":"2024-04-25T08:42:27.583361+0000","last_fullsized":"2024-04-25T08:42:27.583361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:15.923334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582441+0000","last_change":"2024-04-25T08:42:21.371951+0000","last_active":"2024-04-25T08:42:27.582441+0000","last_peered":"2024-04-25T08:42:27.582441+0000","last_clean":"2024-04-25T08:42:27.582441+0000","last_became_active":"2024-04-25T08:42:21.371306+0000","last_became_peered":"2024-04-25T08:42:21.371306+0000","last_unstale":"2024-04-25T08:42:27.582441+0000","last_undegraded":"2024-04-25T08:42:27.582441+0000","last_fullsized":"2024-04-25T08:42:27.582441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678232+0000","last_change":"2024-04-25T08:42:21.341496+0000","last_active":"2024-04-25T08:42:28.678232+0000","last_peered":"2024-04-25T08:42:28.678232+0000","last_clean":"2024-04-25T08:42:28.678232+0000","last_became_active":"2024-04-25T08:42:21.341224+0000","last_became_peered":"2024-04-25T08:42:21.341224+0000","last_unstale":"2024-04-25T08:42:28.678232+0000","last_undegraded":"2024-04-25T08:42:28.678232+0000","last_fullsized":"2024-04-25T08:42:28.678232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682722+0000","last_change":"2024-04-25T08:42:21.360668+0000","last_active":"2024-04-25T08:42:28.682722+0000","last_peered":"2024-04-25T08:42:28.682722+0000","last_clean":"2024-04-25T08:42:28.682722+0000","last_became_active":"2024-04-25T08:42:21.360398+0000","last_became_peered":"2024-04-25T08:42:21.360398+0000","last_unstale":"2024-04-25T08:42:28.682722+0000","last_undegraded":"2024-04-25T08:42:28.682722+0000","last_fullsized":"2024-04-25T08:42:28.682722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:33.626044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384826+0000","last_change":"2024-04-25T08:42:21.353592+0000","last_active":"2024-04-25T08:42:27.384826+0000","last_peered":"2024-04-25T08:42:27.384826+0000","last_clean":"2024-04-25T08:42:27.384826+0000","last_became_active":"2024-04-25T08:42:21.353329+0000","last_became_peered":"2024-04-25T08:42:21.353329+0000","last_unstale":"2024-04-25T08:42:27.384826+0000","last_undegraded":"2024-04-25T08:42:27.384826+0000","last_fullsized":"2024-04-25T08:42:27.384826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390624+0000","last_change":"2024-04-25T08:42:21.356315+0000","last_active":"2024-04-25T08:42:27.390624+0000","last_peered":"2024-04-25T08:42:27.390624+0000","last_clean":"2024-04-25T08:42:27.390624+0000","last_became_active":"2024-04-25T08:42:21.355905+0000","last_became_peered":"2024-04-25T08:42:21.355905+0000","last_unstale":"2024-04-25T08:42:27.390624+0000","last_undegraded":"2024-04-25T08:42:27.390624+0000","last_fullsized":"2024-04-25T08:42:27.390624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:48.723555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389154+0000","last_change":"2024-04-25T08:42:21.371337+0000","last_active":"2024-04-25T08:42:27.389154+0000","last_peered":"2024-04-25T08:42:27.389154+0000","last_clean":"2024-04-25T08:42:27.389154+0000","last_became_active":"2024-04-25T08:42:21.370950+0000","last_became_peered":"2024-04-25T08:42:21.370950+0000","last_unstale":"2024-04-25T08:42:27.389154+0000","last_undegraded":"2024-04-25T08:42:27.389154+0000","last_fullsized":"2024-04-25T08:42:27.389154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389045+0000","last_change":"2024-04-25T08:42:21.365819+0000","last_active":"2024-04-25T08:42:27.389045+0000","last_peered":"2024-04-25T08:42:27.389045+0000","last_clean":"2024-04-25T08:42:27.389045+0000","last_became_active":"2024-04-25T08:42:21.365562+0000","last_became_peered":"2024-04-25T08:42:21.365562+0000","last_unstale":"2024-04-25T08:42:27.389045+0000","last_undegraded":"2024-04-25T08:42:27.389045+0000","last_fullsized":"2024-04-25T08:42:27.389045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684772+0000","last_change":"2024-04-25T08:42:21.366180+0000","last_active":"2024-04-25T08:42:28.684772+0000","last_peered":"2024-04-25T08:42:28.684772+0000","last_clean":"2024-04-25T08:42:28.684772+0000","last_became_active":"2024-04-25T08:42:21.365934+0000","last_became_peered":"2024-04-25T08:42:21.365934+0000","last_unstale":"2024-04-25T08:42:28.684772+0000","last_undegraded":"2024-04-25T08:42:28.684772+0000","last_fullsized":"2024-04-25T08:42:28.684772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:36.740234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384130+0000","last_change":"2024-04-25T08:42:21.368124+0000","last_active":"2024-04-25T08:42:27.384130+0000","last_peered":"2024-04-25T08:42:27.384130+0000","last_clean":"2024-04-25T08:42:27.384130+0000","last_became_active":"2024-04-25T08:42:21.367798+0000","last_became_peered":"2024-04-25T08:42:21.367798+0000","last_unstale":"2024-04-25T08:42:27.384130+0000","last_undegraded":"2024-04-25T08:42:27.384130+0000","last_fullsized":"2024-04-25T08:42:27.384130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683866+0000","last_change":"2024-04-25T08:42:21.357410+0000","last_active":"2024-04-25T08:42:28.683866+0000","last_peered":"2024-04-25T08:42:28.683866+0000","last_clean":"2024-04-25T08:42:28.683866+0000","last_became_active":"2024-04-25T08:42:21.356770+0000","last_became_peered":"2024-04-25T08:42:21.356770+0000","last_unstale":"2024-04-25T08:42:28.683866+0000","last_undegraded":"2024-04-25T08:42:28.683866+0000","last_fullsized":"2024-04-25T08:42:28.683866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582479+0000","last_change":"2024-04-25T08:42:21.372610+0000","last_active":"2024-04-25T08:42:27.582479+0000","last_peered":"2024-04-25T08:42:27.582479+0000","last_clean":"2024-04-25T08:42:27.582479+0000","last_became_active":"2024-04-25T08:42:21.372336+0000","last_became_peered":"2024-04-25T08:42:21.372336+0000","last_unstale":"2024-04-25T08:42:27.582479+0000","last_undegraded":"2024-04-25T08:42:27.582479+0000","last_fullsized":"2024-04-25T08:42:27.582479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:25.717765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684118+0000","last_change":"2024-04-25T08:42:21.358088+0000","last_active":"2024-04-25T08:42:28.684118+0000","last_peered":"2024-04-25T08:42:28.684118+0000","last_clean":"2024-04-25T08:42:28.684118+0000","last_became_active":"2024-04-25T08:42:21.357310+0000","last_became_peered":"2024-04-25T08:42:21.357310+0000","last_unstale":"2024-04-25T08:42:28.684118+0000","last_undegraded":"2024-04-25T08:42:28.684118+0000","last_fullsized":"2024-04-25T08:42:28.684118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582808+0000","last_change":"2024-04-25T08:42:21.367883+0000","last_active":"2024-04-25T08:42:27.582808+0000","last_peered":"2024-04-25T08:42:27.582808+0000","last_clean":"2024-04-25T08:42:27.582808+0000","last_became_active":"2024-04-25T08:42:21.367176+0000","last_became_peered":"2024-04-25T08:42:21.367176+0000","last_unstale":"2024-04-25T08:42:27.582808+0000","last_undegraded":"2024-04-25T08:42:27.582808+0000","last_fullsized":"2024-04-25T08:42:27.582808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.525554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682913+0000","last_change":"2024-04-25T08:42:21.362174+0000","last_active":"2024-04-25T08:42:28.682913+0000","last_peered":"2024-04-25T08:42:28.682913+0000","last_clean":"2024-04-25T08:42:28.682913+0000","last_became_active":"2024-04-25T08:42:21.361897+0000","last_became_peered":"2024-04-25T08:42:21.361897+0000","last_unstale":"2024-04-25T08:42:28.682913+0000","last_undegraded":"2024-04-25T08:42:28.682913+0000","last_fullsized":"2024-04-25T08:42:28.682913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678838+0000","last_change":"2024-04-25T08:42:21.370457+0000","last_active":"2024-04-25T08:42:28.678838+0000","last_peered":"2024-04-25T08:42:28.678838+0000","last_clean":"2024-04-25T08:42:28.678838+0000","last_became_active":"2024-04-25T08:42:21.370091+0000","last_became_peered":"2024-04-25T08:42:21.370091+0000","last_unstale":"2024-04-25T08:42:28.678838+0000","last_undegraded":"2024-04-25T08:42:28.678838+0000","last_fullsized":"2024-04-25T08:42:28.678838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684699+0000","last_change":"2024-04-25T08:42:21.370936+0000","last_active":"2024-04-25T08:42:28.684699+0000","last_peered":"2024-04-25T08:42:28.684699+0000","last_clean":"2024-04-25T08:42:28.684699+0000","last_became_active":"2024-04-25T08:42:21.369447+0000","last_became_peered":"2024-04-25T08:42:21.369447+0000","last_unstale":"2024-04-25T08:42:28.684699+0000","last_undegraded":"2024-04-25T08:42:28.684699+0000","last_fullsized":"2024-04-25T08:42:28.684699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:38.445314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:40.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:40.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.723+0000 7fa9a3600640 1 Processor -- start 2024-04-25T08:42:40.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.723+0000 7fa9a3600640 1 -- start start 2024-04-25T08:42:40.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa9a3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c15a750 0x7fa99c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa9a3600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c06d5e0 con 0x7fa99c0aed10 2024-04-25T08:42:40.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa9a3600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c0ce720 con 0x7fa99c0cc9e0 2024-04-25T08:42:40.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa9a3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c15ab30 con 0x7fa99c15a750 2024-04-25T08:42:40.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa993e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa99c0aed10 0x7fa99c0b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59002/0 (socket says 172.21.15.46:59002) 2024-04-25T08:42:40.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa993e00640 1 -- 172.21.15.46:0/1489771229 learned_addr learned my addr 172.21.15.46:0/1489771229 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa9a0c00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c15a750 0x7fa99c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995667940 0 0) 0x7fa99c06d5e0 con 0x7fa99c0aed10 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa980003530 con 0x7fa99c0aed10 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2498258303 0 0) 0x7fa980003530 con 0x7fa99c0aed10 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c15a750 msgr2=0x7fa99c06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c15a750 0x7fa99c06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 >> v1:172.21.15.82:6789/0 conn(0x7fa99c0cc9e0 legacy=0x7fa99c0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c0cea00 con 0x7fa99c0aed10 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fa984001680 con 0x7fa99c0aed10 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 >> v1:172.21.15.46:6789/0 conn(0x7fa99c0aed10 legacy=0x7fa99c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9800039f0 0x7fa980003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa980004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c15a750 0x7fa9800050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98000b9d0 con 0x7fa9800043a0 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98000bb50 con 0x7fa9800039f0 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98000bd50 con 0x7fa99c15a750 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa993e00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa980004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa9a1600640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c15a750 0x7fa9800050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa9a0c00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9800039f0 0x7fa980003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa993e00640 1 -- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c15a750 msgr2=0x7fa9800050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa993e00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c15a750 0x7fa9800050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa993e00640 1 -- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9800039f0 msgr2=0x7fa980003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa993e00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9800039f0 0x7fa980003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.727+0000 7fa993e00640 1 -- 172.21.15.46:0/1489771229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa980003530 con 0x7fa9800043a0 2024-04-25T08:42:40.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa993e00640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa980004870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c240 tx=0x7fa98401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10047006e69c491b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984007d40 con 0x7fa9800043a0 2024-04-25T08:42:40.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9840a9070 con 0x7fa9800043a0 2024-04-25T08:42:40.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa99be00640 1 -- 172.21.15.46:0/1489771229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984005d40 con 0x7fa9800043a0 2024-04-25T08:42:40.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa9a3600640 1 -- 172.21.15.46:0/1489771229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 msgr2=0x7fa980004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa9a3600640 1 --2- 172.21.15.46:0/1489771229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa980004870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c240 tx=0x7fa98401cc80 comp rx=0 tx=0).stop 2024-04-25T08:42:40.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa9a1600640 1 -- 172.21.15.46:0/1489771229 reap_dead start 2024-04-25T08:42:40.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa9a3600640 1 -- 172.21.15.46:0/1489771229 shutdown_connections 2024-04-25T08:42:40.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa9a3600640 1 -- 172.21.15.46:0/1489771229 >> 172.21.15.46:0/1489771229 conn(0x7fa99c0b12a0 msgr2=0x7fa99c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa9a3600640 1 -- 172.21.15.46:0/1489771229 shutdown_connections 2024-04-25T08:42:40.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa9a3600640 1 -- 172.21.15.46:0/1489771229 wait complete. 2024-04-25T08:42:40.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.731+0000 7fa9a3600640 1 Processor -- start 2024-04-25T08:42:40.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 -- start start 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa99c145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99c0b25a0 0x7fa99c0b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c1fa2e0 0x7fa99c1fa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c1663b0 con 0x7fa9800043a0 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c166230 con 0x7fa99c0b25a0 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c166530 con 0x7fa99c1fa2e0 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa99c145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa99c145da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59934/0 (socket says 172.21.15.46:59934) 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a0c00640 1 -- 172.21.15.46:0/1519149303 learned_addr learned my addr 172.21.15.46:0/1519149303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a1600640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c1fa2e0 0x7fa99c1fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa993e00640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99c0b25a0 0x7fa99c0b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a0c00640 1 -- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c1fa2e0 msgr2=0x7fa99c1fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a0c00640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c1fa2e0 0x7fa99c1fa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a0c00640 1 -- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99c0b25a0 msgr2=0x7fa99c0b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a0c00640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99c0b25a0 0x7fa99c0b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a0c00640 1 -- 172.21.15.46:0/1519149303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c1f6c90 con 0x7fa9800043a0 2024-04-25T08:42:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a1600640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c1fa2e0 0x7fa99c1fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a0c00640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa99c145da0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa98c011960 tx=0x7fa98c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa999600640 1 -- 172.21.15.46:0/1519149303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00faf0 con 0x7fa9800043a0 2024-04-25T08:42:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa999600640 1 -- 172.21.15.46:0/1519149303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c00fc90 con 0x7fa9800043a0 2024-04-25T08:42:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa999600640 1 -- 172.21.15.46:0/1519149303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c014440 con 0x7fa9800043a0 2024-04-25T08:42:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c1fb010 con 0x7fa9800043a0 2024-04-25T08:42:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c156840 con 0x7fa9800043a0 2024-04-25T08:42:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.735+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c0b2f60 con 0x7fa9800043a0 2024-04-25T08:42:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.739+0000 7fa999600640 1 -- 172.21.15.46:0/1519149303 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa98c012040 con 0x7fa9800043a0 2024-04-25T08:42:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.739+0000 7fa999600640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa978066f60 0x7fa978069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.739+0000 7fa999600640 1 -- 172.21.15.46:0/1519149303 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa98c085a20 con 0x7fa9800043a0 2024-04-25T08:42:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.739+0000 7fa993e00640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa978066f60 0x7fa978069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.739+0000 7fa993e00640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa978066f60 0x7fa978069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c240 tx=0x7fa9840b1040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.743+0000 7fa999600640 1 -- 172.21.15.46:0/1519149303 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa98c019030 con 0x7fa9800043a0 2024-04-25T08:42:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.899+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa99c069080 con 0x7fa978066f60 2024-04-25T08:42:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.907+0000 7fa999600640 1 -- 172.21.15.46:0/1519149303 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332562 (secure 0 0 0) 0x7fa99c069080 con 0x7fa978066f60 2024-04-25T08:42:40.905 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:40.907 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa978066f60 msgr2=0x7fa978069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa978066f60 0x7fa978069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa98400c240 tx=0x7fa9840b1040 comp rx=0 tx=0).stop 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 msgr2=0x7fa99c145da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa99c145da0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa98c011960 tx=0x7fa98c011e20 comp rx=0 tx=0).stop 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 shutdown_connections 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa978066f60 0x7fa978069420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa99c1fa2e0 0x7fa99c1fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa99c0b25a0 0x7fa99c0b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 --2- 172.21.15.46:0/1519149303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9800043a0 0x7fa99c145da0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 >> 172.21.15.46:0/1519149303 conn(0x7fa99c0b12a0 msgr2=0x7fa99c0b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 shutdown_connections 2024-04-25T08:42:40.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:40.911+0000 7fa9a3600640 1 -- 172.21.15.46:0/1519149303 wait complete. 2024-04-25T08:42:40.925 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-25T08:42:40.256911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223716,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751004,"statfs":{"total":773094113280,"available":772865028096,"internally_reserved":0,"allocated":3481600,"data_stored":2086387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003140"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683286+0000","last_change":"2024-04-25T08:42:20.357496+0000","last_active":"2024-04-25T08:42:28.683286+0000","last_peered":"2024-04-25T08:42:28.683286+0000","last_clean":"2024-04-25T08:42:28.683286+0000","last_became_active":"2024-04-25T08:42:20.357232+0000","last_became_peered":"2024-04-25T08:42:20.357232+0000","last_unstale":"2024-04-25T08:42:28.683286+0000","last_undegraded":"2024-04-25T08:42:28.683286+0000","last_fullsized":"2024-04-25T08:42:28.683286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:43.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385143+0000","last_change":"2024-04-25T08:42:21.359572+0000","last_active":"2024-04-25T08:42:27.385143+0000","last_peered":"2024-04-25T08:42:27.385143+0000","last_clean":"2024-04-25T08:42:27.385143+0000","last_became_active":"2024-04-25T08:42:21.357850+0000","last_became_peered":"2024-04-25T08:42:21.357850+0000","last_unstale":"2024-04-25T08:42:27.385143+0000","last_undegraded":"2024-04-25T08:42:27.385143+0000","last_fullsized":"2024-04-25T08:42:27.385143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:25.717765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384099+0000","last_change":"2024-04-25T08:42:25.834318+0000","last_active":"2024-04-25T08:42:27.384099+0000","last_peered":"2024-04-25T08:42:27.384099+0000","last_clean":"2024-04-25T08:42:27.384099+0000","last_became_active":"2024-04-25T08:42:25.833964+0000","last_became_peered":"2024-04-25T08:42:25.833964+0000","last_unstale":"2024-04-25T08:42:27.384099+0000","last_undegraded":"2024-04-25T08:42:27.384099+0000","last_fullsized":"2024-04-25T08:42:27.384099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:37.125230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385326+0000","last_change":"2024-04-25T08:42:20.351984+0000","last_active":"2024-04-25T08:42:27.385326+0000","last_peered":"2024-04-25T08:42:27.385326+0000","last_clean":"2024-04-25T08:42:27.385326+0000","last_became_active":"2024-04-25T08:42:20.351590+0000","last_became_peered":"2024-04-25T08:42:20.351590+0000","last_unstale":"2024-04-25T08:42:27.385326+0000","last_undegraded":"2024-04-25T08:42:27.385326+0000","last_fullsized":"2024-04-25T08:42:27.385326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683138+0000","last_change":"2024-04-25T08:42:21.364800+0000","last_active":"2024-04-25T08:42:28.683138+0000","last_peered":"2024-04-25T08:42:28.683138+0000","last_clean":"2024-04-25T08:42:28.683138+0000","last_became_active":"2024-04-25T08:42:21.363796+0000","last_became_peered":"2024-04-25T08:42:21.363796+0000","last_unstale":"2024-04-25T08:42:28.683138+0000","last_undegraded":"2024-04-25T08:42:28.683138+0000","last_fullsized":"2024-04-25T08:42:28.683138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389657+0000","last_change":"2024-04-25T08:42:25.410345+0000","last_active":"2024-04-25T08:42:27.389657+0000","last_peered":"2024-04-25T08:42:27.389657+0000","last_clean":"2024-04-25T08:42:27.389657+0000","last_became_active":"2024-04-25T08:42:25.409807+0000","last_became_peered":"2024-04-25T08:42:25.409807+0000","last_unstale":"2024-04-25T08:42:27.389657+0000","last_undegraded":"2024-04-25T08:42:27.389657+0000","last_fullsized":"2024-04-25T08:42:27.389657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.971525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683514+0000","last_change":"2024-04-25T08:42:20.363027+0000","last_active":"2024-04-25T08:42:28.683514+0000","last_peered":"2024-04-25T08:42:28.683514+0000","last_clean":"2024-04-25T08:42:28.683514+0000","last_became_active":"2024-04-25T08:42:20.362598+0000","last_became_peered":"2024-04-25T08:42:20.362598+0000","last_unstale":"2024-04-25T08:42:28.683514+0000","last_undegraded":"2024-04-25T08:42:28.683514+0000","last_fullsized":"2024-04-25T08:42:28.683514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:26.564850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582927+0000","last_change":"2024-04-25T08:42:21.367716+0000","last_active":"2024-04-25T08:42:27.582927+0000","last_peered":"2024-04-25T08:42:27.582927+0000","last_clean":"2024-04-25T08:42:27.582927+0000","last_became_active":"2024-04-25T08:42:21.366362+0000","last_became_peered":"2024-04-25T08:42:21.366362+0000","last_unstale":"2024-04-25T08:42:27.582927+0000","last_undegraded":"2024-04-25T08:42:27.582927+0000","last_fullsized":"2024-04-25T08:42:27.582927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:30.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390443+0000","last_change":"2024-04-25T08:42:25.407011+0000","last_active":"2024-04-25T08:42:27.390443+0000","last_peered":"2024-04-25T08:42:27.390443+0000","last_clean":"2024-04-25T08:42:27.390443+0000","last_became_active":"2024-04-25T08:42:25.405963+0000","last_became_peered":"2024-04-25T08:42:25.405963+0000","last_unstale":"2024-04-25T08:42:27.390443+0000","last_undegraded":"2024-04-25T08:42:27.390443+0000","last_fullsized":"2024-04-25T08:42:27.390443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683493+0000","last_change":"2024-04-25T08:42:20.365950+0000","last_active":"2024-04-25T08:42:28.683493+0000","last_peered":"2024-04-25T08:42:28.683493+0000","last_clean":"2024-04-25T08:42:28.683493+0000","last_became_active":"2024-04-25T08:42:20.365648+0000","last_became_peered":"2024-04-25T08:42:20.365648+0000","last_unstale":"2024-04-25T08:42:28.683493+0000","last_undegraded":"2024-04-25T08:42:28.683493+0000","last_fullsized":"2024-04-25T08:42:28.683493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:31.011923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583016+0000","last_change":"2024-04-25T08:42:21.366598+0000","last_active":"2024-04-25T08:42:27.583016+0000","last_peered":"2024-04-25T08:42:27.583016+0000","last_clean":"2024-04-25T08:42:27.583016+0000","last_became_active":"2024-04-25T08:42:21.365936+0000","last_became_peered":"2024-04-25T08:42:21.365936+0000","last_unstale":"2024-04-25T08:42:27.583016+0000","last_undegraded":"2024-04-25T08:42:27.583016+0000","last_fullsized":"2024-04-25T08:42:27.583016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389911+0000","last_change":"2024-04-25T08:42:25.400233+0000","last_active":"2024-04-25T08:42:27.389911+0000","last_peered":"2024-04-25T08:42:27.389911+0000","last_clean":"2024-04-25T08:42:27.389911+0000","last_became_active":"2024-04-25T08:42:25.399809+0000","last_became_peered":"2024-04-25T08:42:25.399809+0000","last_unstale":"2024-04-25T08:42:27.389911+0000","last_undegraded":"2024-04-25T08:42:27.389911+0000","last_fullsized":"2024-04-25T08:42:27.389911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:19.971594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388746+0000","last_change":"2024-04-25T08:42:21.354062+0000","last_active":"2024-04-25T08:42:27.388746+0000","last_peered":"2024-04-25T08:42:27.388746+0000","last_clean":"2024-04-25T08:42:27.388746+0000","last_became_active":"2024-04-25T08:42:21.353804+0000","last_became_peered":"2024-04-25T08:42:21.353804+0000","last_unstale":"2024-04-25T08:42:27.388746+0000","last_undegraded":"2024-04-25T08:42:27.388746+0000","last_fullsized":"2024-04-25T08:42:27.388746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683480+0000","last_change":"2024-04-25T08:42:20.361718+0000","last_active":"2024-04-25T08:42:28.683480+0000","last_peered":"2024-04-25T08:42:28.683480+0000","last_clean":"2024-04-25T08:42:28.683480+0000","last_became_active":"2024-04-25T08:42:20.361531+0000","last_became_peered":"2024-04-25T08:42:20.361531+0000","last_unstale":"2024-04-25T08:42:28.683480+0000","last_undegraded":"2024-04-25T08:42:28.683480+0000","last_fullsized":"2024-04-25T08:42:28.683480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:47.546463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390866+0000","last_change":"2024-04-25T08:42:25.410313+0000","last_active":"2024-04-25T08:42:27.390866+0000","last_peered":"2024-04-25T08:42:27.390866+0000","last_clean":"2024-04-25T08:42:27.390866+0000","last_became_active":"2024-04-25T08:42:25.409794+0000","last_became_peered":"2024-04-25T08:42:25.409794+0000","last_unstale":"2024-04-25T08:42:27.390866+0000","last_undegraded":"2024-04-25T08:42:27.390866+0000","last_fullsized":"2024-04-25T08:42:27.390866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384856+0000","last_change":"2024-04-25T08:42:21.358366+0000","last_active":"2024-04-25T08:42:27.384856+0000","last_peered":"2024-04-25T08:42:27.384856+0000","last_clean":"2024-04-25T08:42:27.384856+0000","last_became_active":"2024-04-25T08:42:21.357447+0000","last_became_peered":"2024-04-25T08:42:21.357447+0000","last_unstale":"2024-04-25T08:42:27.384856+0000","last_undegraded":"2024-04-25T08:42:27.384856+0000","last_fullsized":"2024-04-25T08:42:27.384856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677905+0000","last_change":"2024-04-25T08:42:20.352044+0000","last_active":"2024-04-25T08:42:28.677905+0000","last_peered":"2024-04-25T08:42:28.677905+0000","last_clean":"2024-04-25T08:42:28.677905+0000","last_became_active":"2024-04-25T08:42:20.351667+0000","last_became_peered":"2024-04-25T08:42:20.351667+0000","last_unstale":"2024-04-25T08:42:28.677905+0000","last_undegraded":"2024-04-25T08:42:28.677905+0000","last_fullsized":"2024-04-25T08:42:28.677905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390597+0000","last_change":"2024-04-25T08:42:25.406565+0000","last_active":"2024-04-25T08:42:27.390597+0000","last_peered":"2024-04-25T08:42:27.390597+0000","last_clean":"2024-04-25T08:42:27.390597+0000","last_became_active":"2024-04-25T08:42:25.405840+0000","last_became_peered":"2024-04-25T08:42:25.405840+0000","last_unstale":"2024-04-25T08:42:27.390597+0000","last_undegraded":"2024-04-25T08:42:27.390597+0000","last_fullsized":"2024-04-25T08:42:27.390597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384326+0000","last_change":"2024-04-25T08:42:21.354614+0000","last_active":"2024-04-25T08:42:27.384326+0000","last_peered":"2024-04-25T08:42:27.384326+0000","last_clean":"2024-04-25T08:42:27.384326+0000","last_became_active":"2024-04-25T08:42:21.353873+0000","last_became_peered":"2024-04-25T08:42:21.353873+0000","last_unstale":"2024-04-25T08:42:27.384326+0000","last_undegraded":"2024-04-25T08:42:27.384326+0000","last_fullsized":"2024-04-25T08:42:27.384326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678307+0000","last_change":"2024-04-25T08:42:20.349084+0000","last_active":"2024-04-25T08:42:28.678307+0000","last_peered":"2024-04-25T08:42:28.678307+0000","last_clean":"2024-04-25T08:42:28.678307+0000","last_became_active":"2024-04-25T08:42:20.348850+0000","last_became_peered":"2024-04-25T08:42:20.348850+0000","last_unstale":"2024-04-25T08:42:28.678307+0000","last_undegraded":"2024-04-25T08:42:28.678307+0000","last_fullsized":"2024-04-25T08:42:28.678307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678770+0000","last_change":"2024-04-25T08:42:25.832820+0000","last_active":"2024-04-25T08:42:28.678770+0000","last_peered":"2024-04-25T08:42:28.678770+0000","last_clean":"2024-04-25T08:42:28.678770+0000","last_became_active":"2024-04-25T08:42:25.832090+0000","last_became_peered":"2024-04-25T08:42:25.832090+0000","last_unstale":"2024-04-25T08:42:28.678770+0000","last_undegraded":"2024-04-25T08:42:28.678770+0000","last_fullsized":"2024-04-25T08:42:28.678770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390081+0000","last_change":"2024-04-25T08:42:21.338500+0000","last_active":"2024-04-25T08:42:27.390081+0000","last_peered":"2024-04-25T08:42:27.390081+0000","last_clean":"2024-04-25T08:42:27.390081+0000","last_became_active":"2024-04-25T08:42:21.338153+0000","last_became_peered":"2024-04-25T08:42:21.338153+0000","last_unstale":"2024-04-25T08:42:27.390081+0000","last_undegraded":"2024-04-25T08:42:27.390081+0000","last_fullsized":"2024-04-25T08:42:27.390081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:00.647619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683714+0000","last_change":"2024-04-25T08:42:20.361253+0000","last_active":"2024-04-25T08:42:28.683714+0000","last_peered":"2024-04-25T08:42:28.683714+0000","last_clean":"2024-04-25T08:42:28.683714+0000","last_became_active":"2024-04-25T08:42:20.360866+0000","last_became_peered":"2024-04-25T08:42:20.360866+0000","last_unstale":"2024-04-25T08:42:28.683714+0000","last_undegraded":"2024-04-25T08:42:28.683714+0000","last_fullsized":"2024-04-25T08:42:28.683714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.823720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390784+0000","last_change":"2024-04-25T08:42:25.406555+0000","last_active":"2024-04-25T08:42:27.390784+0000","last_peered":"2024-04-25T08:42:27.390784+0000","last_clean":"2024-04-25T08:42:27.390784+0000","last_became_active":"2024-04-25T08:42:25.405977+0000","last_became_peered":"2024-04-25T08:42:25.405977+0000","last_unstale":"2024-04-25T08:42:27.390784+0000","last_undegraded":"2024-04-25T08:42:27.390784+0000","last_fullsized":"2024-04-25T08:42:27.390784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683559+0000","last_change":"2024-04-25T08:42:21.361932+0000","last_active":"2024-04-25T08:42:28.683559+0000","last_peered":"2024-04-25T08:42:28.683559+0000","last_clean":"2024-04-25T08:42:28.683559+0000","last_became_active":"2024-04-25T08:42:21.361599+0000","last_became_peered":"2024-04-25T08:42:21.361599+0000","last_unstale":"2024-04-25T08:42:28.683559+0000","last_undegraded":"2024-04-25T08:42:28.683559+0000","last_fullsized":"2024-04-25T08:42:28.683559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389472+0000","last_change":"2024-04-25T08:42:20.359118+0000","last_active":"2024-04-25T08:42:27.389472+0000","last_peered":"2024-04-25T08:42:27.389472+0000","last_clean":"2024-04-25T08:42:27.389472+0000","last_became_active":"2024-04-25T08:42:20.358850+0000","last_became_peered":"2024-04-25T08:42:20.358850+0000","last_unstale":"2024-04-25T08:42:27.389472+0000","last_undegraded":"2024-04-25T08:42:27.389472+0000","last_fullsized":"2024-04-25T08:42:27.389472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389832+0000","last_change":"2024-04-25T08:42:25.836412+0000","last_active":"2024-04-25T08:42:27.389832+0000","last_peered":"2024-04-25T08:42:27.389832+0000","last_clean":"2024-04-25T08:42:27.389832+0000","last_became_active":"2024-04-25T08:42:25.835926+0000","last_became_peered":"2024-04-25T08:42:25.835926+0000","last_unstale":"2024-04-25T08:42:27.389832+0000","last_undegraded":"2024-04-25T08:42:27.389832+0000","last_fullsized":"2024-04-25T08:42:27.389832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.610141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385472+0000","last_change":"2024-04-25T08:42:21.366095+0000","last_active":"2024-04-25T08:42:27.385472+0000","last_peered":"2024-04-25T08:42:27.385472+0000","last_clean":"2024-04-25T08:42:27.385472+0000","last_became_active":"2024-04-25T08:42:21.365608+0000","last_became_peered":"2024-04-25T08:42:21.365608+0000","last_unstale":"2024-04-25T08:42:27.385472+0000","last_undegraded":"2024-04-25T08:42:27.385472+0000","last_fullsized":"2024-04-25T08:42:27.385472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677918+0000","last_change":"2024-04-25T08:42:20.347127+0000","last_active":"2024-04-25T08:42:28.677918+0000","last_peered":"2024-04-25T08:42:28.677918+0000","last_clean":"2024-04-25T08:42:28.677918+0000","last_became_active":"2024-04-25T08:42:20.346840+0000","last_became_peered":"2024-04-25T08:42:20.346840+0000","last_unstale":"2024-04-25T08:42:28.677918+0000","last_undegraded":"2024-04-25T08:42:28.677918+0000","last_fullsized":"2024-04-25T08:42:28.677918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384051+0000","last_change":"2024-04-25T08:42:25.832430+0000","last_active":"2024-04-25T08:42:27.384051+0000","last_peered":"2024-04-25T08:42:27.384051+0000","last_clean":"2024-04-25T08:42:27.384051+0000","last_became_active":"2024-04-25T08:42:25.832026+0000","last_became_peered":"2024-04-25T08:42:25.832026+0000","last_unstale":"2024-04-25T08:42:27.384051+0000","last_undegraded":"2024-04-25T08:42:27.384051+0000","last_fullsized":"2024-04-25T08:42:27.384051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:39.589658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.679001+0000","last_change":"2024-04-25T08:42:21.340393+0000","last_active":"2024-04-25T08:42:28.679001+0000","last_peered":"2024-04-25T08:42:28.679001+0000","last_clean":"2024-04-25T08:42:28.679001+0000","last_became_active":"2024-04-25T08:42:21.340043+0000","last_became_peered":"2024-04-25T08:42:21.340043+0000","last_unstale":"2024-04-25T08:42:28.679001+0000","last_undegraded":"2024-04-25T08:42:28.679001+0000","last_fullsized":"2024-04-25T08:42:28.679001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684534+0000","last_change":"2024-04-25T08:42:20.360592+0000","last_active":"2024-04-25T08:42:28.684534+0000","last_peered":"2024-04-25T08:42:28.684534+0000","last_clean":"2024-04-25T08:42:28.684534+0000","last_became_active":"2024-04-25T08:42:20.360361+0000","last_became_peered":"2024-04-25T08:42:20.360361+0000","last_unstale":"2024-04-25T08:42:28.684534+0000","last_undegraded":"2024-04-25T08:42:28.684534+0000","last_fullsized":"2024-04-25T08:42:28.684534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:53.741991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682832+0000","last_change":"2024-04-25T08:42:25.834135+0000","last_active":"2024-04-25T08:42:28.682832+0000","last_peered":"2024-04-25T08:42:28.682832+0000","last_clean":"2024-04-25T08:42:28.682832+0000","last_became_active":"2024-04-25T08:42:25.833757+0000","last_became_peered":"2024-04-25T08:42:25.833757+0000","last_unstale":"2024-04-25T08:42:28.682832+0000","last_undegraded":"2024-04-25T08:42:28.682832+0000","last_fullsized":"2024-04-25T08:42:28.682832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583189+0000","last_change":"2024-04-25T08:42:21.366345+0000","last_active":"2024-04-25T08:42:27.583189+0000","last_peered":"2024-04-25T08:42:27.583189+0000","last_clean":"2024-04-25T08:42:27.583189+0000","last_became_active":"2024-04-25T08:42:21.365644+0000","last_became_peered":"2024-04-25T08:42:21.365644+0000","last_unstale":"2024-04-25T08:42:27.583189+0000","last_undegraded":"2024-04-25T08:42:27.583189+0000","last_fullsized":"2024-04-25T08:42:27.583189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.679043+0000","last_change":"2024-04-25T08:42:20.344087+0000","last_active":"2024-04-25T08:42:28.679043+0000","last_peered":"2024-04-25T08:42:28.679043+0000","last_clean":"2024-04-25T08:42:28.679043+0000","last_became_active":"2024-04-25T08:42:20.343799+0000","last_became_peered":"2024-04-25T08:42:20.343799+0000","last_unstale":"2024-04-25T08:42:28.679043+0000","last_undegraded":"2024-04-25T08:42:28.679043+0000","last_fullsized":"2024-04-25T08:42:28.679043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.823720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678152+0000","last_change":"2024-04-25T08:42:25.412243+0000","last_active":"2024-04-25T08:42:28.678152+0000","last_peered":"2024-04-25T08:42:28.678152+0000","last_clean":"2024-04-25T08:42:28.678152+0000","last_became_active":"2024-04-25T08:42:25.406351+0000","last_became_peered":"2024-04-25T08:42:25.406351+0000","last_unstale":"2024-04-25T08:42:28.678152+0000","last_undegraded":"2024-04-25T08:42:28.678152+0000","last_fullsized":"2024-04-25T08:42:28.678152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:42.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390626+0000","last_change":"2024-04-25T08:42:21.366092+0000","last_active":"2024-04-25T08:42:27.390626+0000","last_peered":"2024-04-25T08:42:27.390626+0000","last_clean":"2024-04-25T08:42:27.390626+0000","last_became_active":"2024-04-25T08:42:21.365152+0000","last_became_peered":"2024-04-25T08:42:21.365152+0000","last_unstale":"2024-04-25T08:42:27.390626+0000","last_undegraded":"2024-04-25T08:42:27.390626+0000","last_fullsized":"2024-04-25T08:42:27.390626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:29.830442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582739+0000","last_change":"2024-04-25T08:42:20.352850+0000","last_active":"2024-04-25T08:42:27.582739+0000","last_peered":"2024-04-25T08:42:27.582739+0000","last_clean":"2024-04-25T08:42:27.582739+0000","last_became_active":"2024-04-25T08:42:20.352047+0000","last_became_peered":"2024-04-25T08:42:20.352047+0000","last_unstale":"2024-04-25T08:42:27.582739+0000","last_undegraded":"2024-04-25T08:42:27.582739+0000","last_fullsized":"2024-04-25T08:42:27.582739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678979+0000","last_change":"2024-04-25T08:42:25.405939+0000","last_active":"2024-04-25T08:42:28.678979+0000","last_peered":"2024-04-25T08:42:28.678979+0000","last_clean":"2024-04-25T08:42:28.678979+0000","last_became_active":"2024-04-25T08:42:25.405333+0000","last_became_peered":"2024-04-25T08:42:25.405333+0000","last_unstale":"2024-04-25T08:42:28.678979+0000","last_undegraded":"2024-04-25T08:42:28.678979+0000","last_fullsized":"2024-04-25T08:42:28.678979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678442+0000","last_change":"2024-04-25T08:42:21.338660+0000","last_active":"2024-04-25T08:42:28.678442+0000","last_peered":"2024-04-25T08:42:28.678442+0000","last_clean":"2024-04-25T08:42:28.678442+0000","last_became_active":"2024-04-25T08:42:21.338357+0000","last_became_peered":"2024-04-25T08:42:21.338357+0000","last_unstale":"2024-04-25T08:42:28.678442+0000","last_undegraded":"2024-04-25T08:42:28.678442+0000","last_fullsized":"2024-04-25T08:42:28.678442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583032+0000","last_change":"2024-04-25T08:42:20.352888+0000","last_active":"2024-04-25T08:42:28.583032+0000","last_peered":"2024-04-25T08:42:28.583032+0000","last_clean":"2024-04-25T08:42:28.583032+0000","last_became_active":"2024-04-25T08:42:20.351108+0000","last_became_peered":"2024-04-25T08:42:20.351108+0000","last_unstale":"2024-04-25T08:42:28.583032+0000","last_undegraded":"2024-04-25T08:42:28.583032+0000","last_fullsized":"2024-04-25T08:42:28.583032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678770+0000","last_change":"2024-04-25T08:42:25.409801+0000","last_active":"2024-04-25T08:42:28.678770+0000","last_peered":"2024-04-25T08:42:28.678770+0000","last_clean":"2024-04-25T08:42:28.678770+0000","last_became_active":"2024-04-25T08:42:25.409331+0000","last_became_peered":"2024-04-25T08:42:25.409331+0000","last_unstale":"2024-04-25T08:42:28.678770+0000","last_undegraded":"2024-04-25T08:42:28.678770+0000","last_fullsized":"2024-04-25T08:42:28.678770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682464+0000","last_change":"2024-04-25T08:42:21.363349+0000","last_active":"2024-04-25T08:42:28.682464+0000","last_peered":"2024-04-25T08:42:28.682464+0000","last_clean":"2024-04-25T08:42:28.682464+0000","last_became_active":"2024-04-25T08:42:21.362903+0000","last_became_peered":"2024-04-25T08:42:21.362903+0000","last_unstale":"2024-04-25T08:42:28.682464+0000","last_undegraded":"2024-04-25T08:42:28.682464+0000","last_fullsized":"2024-04-25T08:42:28.682464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:48.553185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683717+0000","last_change":"2024-04-25T08:42:20.356423+0000","last_active":"2024-04-25T08:42:28.683717+0000","last_peered":"2024-04-25T08:42:28.683717+0000","last_clean":"2024-04-25T08:42:28.683717+0000","last_became_active":"2024-04-25T08:42:20.356110+0000","last_became_peered":"2024-04-25T08:42:20.356110+0000","last_unstale":"2024-04-25T08:42:28.683717+0000","last_undegraded":"2024-04-25T08:42:28.683717+0000","last_fullsized":"2024-04-25T08:42:28.683717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:44.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683389+0000","last_change":"2024-04-25T08:42:25.834561+0000","last_active":"2024-04-25T08:42:28.683389+0000","last_peered":"2024-04-25T08:42:28.683389+0000","last_clean":"2024-04-25T08:42:28.683389+0000","last_became_active":"2024-04-25T08:42:25.834038+0000","last_became_peered":"2024-04-25T08:42:25.834038+0000","last_unstale":"2024-04-25T08:42:28.683389+0000","last_undegraded":"2024-04-25T08:42:28.683389+0000","last_fullsized":"2024-04-25T08:42:28.683389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:09.194186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384703+0000","last_change":"2024-04-25T08:42:21.369030+0000","last_active":"2024-04-25T08:42:27.384703+0000","last_peered":"2024-04-25T08:42:27.384703+0000","last_clean":"2024-04-25T08:42:27.384703+0000","last_became_active":"2024-04-25T08:42:21.368760+0000","last_became_peered":"2024-04-25T08:42:21.368760+0000","last_unstale":"2024-04-25T08:42:27.384703+0000","last_undegraded":"2024-04-25T08:42:27.384703+0000","last_fullsized":"2024-04-25T08:42:27.384703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582740+0000","last_change":"2024-04-25T08:42:20.359912+0000","last_active":"2024-04-25T08:42:27.582740+0000","last_peered":"2024-04-25T08:42:27.582740+0000","last_clean":"2024-04-25T08:42:27.582740+0000","last_became_active":"2024-04-25T08:42:20.359616+0000","last_became_peered":"2024-04-25T08:42:20.359616+0000","last_unstale":"2024-04-25T08:42:27.582740+0000","last_undegraded":"2024-04-25T08:42:27.582740+0000","last_fullsized":"2024-04-25T08:42:27.582740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683854+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:42:28.683854+0000","last_peered":"2024-04-25T08:42:28.683854+0000","last_clean":"2024-04-25T08:42:28.683854+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:42:28.683854+0000","last_undegraded":"2024-04-25T08:42:28.683854+0000","last_fullsized":"2024-04-25T08:42:28.683854+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389063+0000","last_change":"2024-04-25T08:42:25.836852+0000","last_active":"2024-04-25T08:42:27.389063+0000","last_peered":"2024-04-25T08:42:27.389063+0000","last_clean":"2024-04-25T08:42:27.389063+0000","last_became_active":"2024-04-25T08:42:25.836428+0000","last_became_peered":"2024-04-25T08:42:25.836428+0000","last_unstale":"2024-04-25T08:42:27.389063+0000","last_undegraded":"2024-04-25T08:42:27.389063+0000","last_fullsized":"2024-04-25T08:42:27.389063+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.544013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684361+0000","last_change":"2024-04-25T08:42:21.376058+0000","last_active":"2024-04-25T08:42:28.684361+0000","last_peered":"2024-04-25T08:42:28.684361+0000","last_clean":"2024-04-25T08:42:28.684361+0000","last_became_active":"2024-04-25T08:42:21.375825+0000","last_became_peered":"2024-04-25T08:42:21.375825+0000","last_unstale":"2024-04-25T08:42:28.684361+0000","last_undegraded":"2024-04-25T08:42:28.684361+0000","last_fullsized":"2024-04-25T08:42:28.684361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:24.495752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583044+0000","last_change":"2024-04-25T08:42:20.353588+0000","last_active":"2024-04-25T08:42:28.583044+0000","last_peered":"2024-04-25T08:42:28.583044+0000","last_clean":"2024-04-25T08:42:28.583044+0000","last_became_active":"2024-04-25T08:42:20.353335+0000","last_became_peered":"2024-04-25T08:42:20.353335+0000","last_unstale":"2024-04-25T08:42:28.583044+0000","last_undegraded":"2024-04-25T08:42:28.583044+0000","last_fullsized":"2024-04-25T08:42:28.583044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388628+0000","last_change":"2024-04-25T08:42:25.833110+0000","last_active":"2024-04-25T08:42:27.388628+0000","last_peered":"2024-04-25T08:42:27.388628+0000","last_clean":"2024-04-25T08:42:27.388628+0000","last_became_active":"2024-04-25T08:42:25.832656+0000","last_became_peered":"2024-04-25T08:42:25.832656+0000","last_unstale":"2024-04-25T08:42:27.388628+0000","last_undegraded":"2024-04-25T08:42:27.388628+0000","last_fullsized":"2024-04-25T08:42:27.388628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.971525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582934+0000","last_change":"2024-04-25T08:42:21.368021+0000","last_active":"2024-04-25T08:42:27.582934+0000","last_peered":"2024-04-25T08:42:27.582934+0000","last_clean":"2024-04-25T08:42:27.582934+0000","last_became_active":"2024-04-25T08:42:21.367516+0000","last_became_peered":"2024-04-25T08:42:21.367516+0000","last_unstale":"2024-04-25T08:42:27.582934+0000","last_undegraded":"2024-04-25T08:42:27.582934+0000","last_fullsized":"2024-04-25T08:42:27.582934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:24.495752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677925+0000","last_change":"2024-04-25T08:42:20.340089+0000","last_active":"2024-04-25T08:42:28.677925+0000","last_peered":"2024-04-25T08:42:28.677925+0000","last_clean":"2024-04-25T08:42:28.677925+0000","last_became_active":"2024-04-25T08:42:20.338267+0000","last_became_peered":"2024-04-25T08:42:20.338267+0000","last_unstale":"2024-04-25T08:42:28.677925+0000","last_undegraded":"2024-04-25T08:42:28.677925+0000","last_fullsized":"2024-04-25T08:42:28.677925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:44.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682886+0000","last_change":"2024-04-25T08:42:25.418168+0000","last_active":"2024-04-25T08:42:28.682886+0000","last_peered":"2024-04-25T08:42:28.682886+0000","last_clean":"2024-04-25T08:42:28.682886+0000","last_became_active":"2024-04-25T08:42:25.417852+0000","last_became_peered":"2024-04-25T08:42:25.417852+0000","last_unstale":"2024-04-25T08:42:28.682886+0000","last_undegraded":"2024-04-25T08:42:28.682886+0000","last_fullsized":"2024-04-25T08:42:28.682886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684774+0000","last_change":"2024-04-25T08:42:21.370059+0000","last_active":"2024-04-25T08:42:28.684774+0000","last_peered":"2024-04-25T08:42:28.684774+0000","last_clean":"2024-04-25T08:42:28.684774+0000","last_became_active":"2024-04-25T08:42:21.359776+0000","last_became_peered":"2024-04-25T08:42:21.359776+0000","last_unstale":"2024-04-25T08:42:28.684774+0000","last_undegraded":"2024-04-25T08:42:28.684774+0000","last_fullsized":"2024-04-25T08:42:28.684774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:21.412723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583070+0000","last_change":"2024-04-25T08:42:20.351406+0000","last_active":"2024-04-25T08:42:28.583070+0000","last_peered":"2024-04-25T08:42:28.583070+0000","last_clean":"2024-04-25T08:42:28.583070+0000","last_became_active":"2024-04-25T08:42:20.351114+0000","last_became_peered":"2024-04-25T08:42:20.351114+0000","last_unstale":"2024-04-25T08:42:28.583070+0000","last_undegraded":"2024-04-25T08:42:28.583070+0000","last_fullsized":"2024-04-25T08:42:28.583070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582385+0000","last_change":"2024-04-25T08:42:25.835941+0000","last_active":"2024-04-25T08:42:27.582385+0000","last_peered":"2024-04-25T08:42:27.582385+0000","last_clean":"2024-04-25T08:42:27.582385+0000","last_became_active":"2024-04-25T08:42:25.834320+0000","last_became_peered":"2024-04-25T08:42:25.834320+0000","last_unstale":"2024-04-25T08:42:27.582385+0000","last_undegraded":"2024-04-25T08:42:27.582385+0000","last_fullsized":"2024-04-25T08:42:27.582385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:32.335929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684548+0000","last_change":"2024-04-25T08:42:21.363962+0000","last_active":"2024-04-25T08:42:28.684548+0000","last_peered":"2024-04-25T08:42:28.684548+0000","last_clean":"2024-04-25T08:42:28.684548+0000","last_became_active":"2024-04-25T08:42:21.363695+0000","last_became_peered":"2024-04-25T08:42:21.363695+0000","last_unstale":"2024-04-25T08:42:28.684548+0000","last_undegraded":"2024-04-25T08:42:28.684548+0000","last_fullsized":"2024-04-25T08:42:28.684548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:15.923334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.391021+0000","last_change":"2024-04-25T08:42:20.345476+0000","last_active":"2024-04-25T08:42:27.391021+0000","last_peered":"2024-04-25T08:42:27.391021+0000","last_clean":"2024-04-25T08:42:27.391021+0000","last_became_active":"2024-04-25T08:42:20.345085+0000","last_became_peered":"2024-04-25T08:42:20.345085+0000","last_unstale":"2024-04-25T08:42:27.391021+0000","last_undegraded":"2024-04-25T08:42:27.391021+0000","last_fullsized":"2024-04-25T08:42:27.391021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:19.301171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389733+0000","last_change":"2024-04-25T08:42:25.410444+0000","last_active":"2024-04-25T08:42:27.389733+0000","last_peered":"2024-04-25T08:42:27.389733+0000","last_clean":"2024-04-25T08:42:27.389733+0000","last_became_active":"2024-04-25T08:42:25.409574+0000","last_became_peered":"2024-04-25T08:42:25.409574+0000","last_unstale":"2024-04-25T08:42:27.389733+0000","last_undegraded":"2024-04-25T08:42:27.389733+0000","last_fullsized":"2024-04-25T08:42:27.389733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.544013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683914+0000","last_change":"2024-04-25T08:42:21.357665+0000","last_active":"2024-04-25T08:42:28.683914+0000","last_peered":"2024-04-25T08:42:28.683914+0000","last_clean":"2024-04-25T08:42:28.683914+0000","last_became_active":"2024-04-25T08:42:21.357081+0000","last_became_peered":"2024-04-25T08:42:21.357081+0000","last_unstale":"2024-04-25T08:42:28.683914+0000","last_undegraded":"2024-04-25T08:42:28.683914+0000","last_fullsized":"2024-04-25T08:42:28.683914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:24.505703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583188+0000","last_change":"2024-04-25T08:42:20.345496+0000","last_active":"2024-04-25T08:42:28.583188+0000","last_peered":"2024-04-25T08:42:28.583188+0000","last_clean":"2024-04-25T08:42:28.583188+0000","last_became_active":"2024-04-25T08:42:20.345230+0000","last_became_peered":"2024-04-25T08:42:20.345230+0000","last_unstale":"2024-04-25T08:42:28.583188+0000","last_undegraded":"2024-04-25T08:42:28.583188+0000","last_fullsized":"2024-04-25T08:42:28.583188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:33.260664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683216+0000","last_change":"2024-04-25T08:42:25.412932+0000","last_active":"2024-04-25T08:42:28.683216+0000","last_peered":"2024-04-25T08:42:28.683216+0000","last_clean":"2024-04-25T08:42:28.683216+0000","last_became_active":"2024-04-25T08:42:25.412212+0000","last_became_peered":"2024-04-25T08:42:25.412212+0000","last_unstale":"2024-04-25T08:42:28.683216+0000","last_undegraded":"2024-04-25T08:42:28.683216+0000","last_fullsized":"2024-04-25T08:42:28.683216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388745+0000","last_change":"2024-04-25T08:42:21.351855+0000","last_active":"2024-04-25T08:42:27.388745+0000","last_peered":"2024-04-25T08:42:27.388745+0000","last_clean":"2024-04-25T08:42:27.388745+0000","last_became_active":"2024-04-25T08:42:21.351412+0000","last_became_peered":"2024-04-25T08:42:21.351412+0000","last_unstale":"2024-04-25T08:42:27.388745+0000","last_undegraded":"2024-04-25T08:42:27.388745+0000","last_fullsized":"2024-04-25T08:42:27.388745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388741+0000","last_change":"2024-04-25T08:42:20.350412+0000","last_active":"2024-04-25T08:42:27.388741+0000","last_peered":"2024-04-25T08:42:27.388741+0000","last_clean":"2024-04-25T08:42:27.388741+0000","last_became_active":"2024-04-25T08:42:20.350113+0000","last_became_peered":"2024-04-25T08:42:20.350113+0000","last_unstale":"2024-04-25T08:42:27.388741+0000","last_undegraded":"2024-04-25T08:42:27.388741+0000","last_fullsized":"2024-04-25T08:42:27.388741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385384+0000","last_change":"2024-04-25T08:42:25.838652+0000","last_active":"2024-04-25T08:42:27.385384+0000","last_peered":"2024-04-25T08:42:27.385384+0000","last_clean":"2024-04-25T08:42:27.385384+0000","last_became_active":"2024-04-25T08:42:25.838168+0000","last_became_peered":"2024-04-25T08:42:25.838168+0000","last_unstale":"2024-04-25T08:42:27.385384+0000","last_undegraded":"2024-04-25T08:42:27.385384+0000","last_fullsized":"2024-04-25T08:42:27.385384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:46.882274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684243+0000","last_change":"2024-04-25T08:42:21.373789+0000","last_active":"2024-04-25T08:42:28.684243+0000","last_peered":"2024-04-25T08:42:28.684243+0000","last_clean":"2024-04-25T08:42:28.684243+0000","last_became_active":"2024-04-25T08:42:21.372582+0000","last_became_peered":"2024-04-25T08:42:21.372582+0000","last_unstale":"2024-04-25T08:42:28.684243+0000","last_undegraded":"2024-04-25T08:42:28.684243+0000","last_fullsized":"2024-04-25T08:42:28.684243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.525554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582667+0000","last_change":"2024-04-25T08:42:20.357655+0000","last_active":"2024-04-25T08:42:27.582667+0000","last_peered":"2024-04-25T08:42:27.582667+0000","last_clean":"2024-04-25T08:42:27.582667+0000","last_became_active":"2024-04-25T08:42:20.357354+0000","last_became_peered":"2024-04-25T08:42:20.357354+0000","last_unstale":"2024-04-25T08:42:27.582667+0000","last_undegraded":"2024-04-25T08:42:27.582667+0000","last_fullsized":"2024-04-25T08:42:27.582667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385040+0000","last_change":"2024-04-25T08:42:25.835505+0000","last_active":"2024-04-25T08:42:27.385040+0000","last_peered":"2024-04-25T08:42:27.385040+0000","last_clean":"2024-04-25T08:42:27.385040+0000","last_became_active":"2024-04-25T08:42:25.835079+0000","last_became_peered":"2024-04-25T08:42:25.835079+0000","last_unstale":"2024-04-25T08:42:27.385040+0000","last_undegraded":"2024-04-25T08:42:27.385040+0000","last_fullsized":"2024-04-25T08:42:27.385040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:00.799395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682748+0000","last_change":"2024-04-25T08:42:21.366223+0000","last_active":"2024-04-25T08:42:28.682748+0000","last_peered":"2024-04-25T08:42:28.682748+0000","last_clean":"2024-04-25T08:42:28.682748+0000","last_became_active":"2024-04-25T08:42:21.365913+0000","last_became_peered":"2024-04-25T08:42:21.365913+0000","last_unstale":"2024-04-25T08:42:28.682748+0000","last_undegraded":"2024-04-25T08:42:28.682748+0000","last_fullsized":"2024-04-25T08:42:28.682748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682183+0000","last_change":"2024-04-25T08:42:20.354779+0000","last_active":"2024-04-25T08:42:28.682183+0000","last_peered":"2024-04-25T08:42:28.682183+0000","last_clean":"2024-04-25T08:42:28.682183+0000","last_became_active":"2024-04-25T08:42:20.354504+0000","last_became_peered":"2024-04-25T08:42:20.354504+0000","last_unstale":"2024-04-25T08:42:28.682183+0000","last_undegraded":"2024-04-25T08:42:28.682183+0000","last_fullsized":"2024-04-25T08:42:28.682183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.311211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583098+0000","last_change":"2024-04-25T08:42:25.832677+0000","last_active":"2024-04-25T08:42:27.583098+0000","last_peered":"2024-04-25T08:42:27.583098+0000","last_clean":"2024-04-25T08:42:27.583098+0000","last_became_active":"2024-04-25T08:42:25.832204+0000","last_became_peered":"2024-04-25T08:42:25.832204+0000","last_unstale":"2024-04-25T08:42:27.583098+0000","last_undegraded":"2024-04-25T08:42:27.583098+0000","last_fullsized":"2024-04-25T08:42:27.583098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:02.694380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684296+0000","last_change":"2024-04-25T08:42:21.365924+0000","last_active":"2024-04-25T08:42:28.684296+0000","last_peered":"2024-04-25T08:42:28.684296+0000","last_clean":"2024-04-25T08:42:28.684296+0000","last_became_active":"2024-04-25T08:42:21.365581+0000","last_became_peered":"2024-04-25T08:42:21.365581+0000","last_unstale":"2024-04-25T08:42:28.684296+0000","last_undegraded":"2024-04-25T08:42:28.684296+0000","last_fullsized":"2024-04-25T08:42:28.684296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.923264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:36.452272+0000","last_change":"2024-04-25T08:42:20.354736+0000","last_active":"2024-04-25T08:42:36.452272+0000","last_peered":"2024-04-25T08:42:36.452272+0000","last_clean":"2024-04-25T08:42:36.452272+0000","last_became_active":"2024-04-25T08:42:20.354322+0000","last_became_peered":"2024-04-25T08:42:20.354322+0000","last_unstale":"2024-04-25T08:42:36.452272+0000","last_undegraded":"2024-04-25T08:42:36.452272+0000","last_fullsized":"2024-04-25T08:42:36.452272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678428+0000","last_change":"2024-04-25T08:42:25.414706+0000","last_active":"2024-04-25T08:42:28.678428+0000","last_peered":"2024-04-25T08:42:28.678428+0000","last_clean":"2024-04-25T08:42:28.678428+0000","last_became_active":"2024-04-25T08:42:25.414302+0000","last_became_peered":"2024-04-25T08:42:25.414302+0000","last_unstale":"2024-04-25T08:42:28.678428+0000","last_undegraded":"2024-04-25T08:42:28.678428+0000","last_fullsized":"2024-04-25T08:42:28.678428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390755+0000","last_change":"2024-04-25T08:42:21.361175+0000","last_active":"2024-04-25T08:42:27.390755+0000","last_peered":"2024-04-25T08:42:27.390755+0000","last_clean":"2024-04-25T08:42:27.390755+0000","last_became_active":"2024-04-25T08:42:21.358285+0000","last_became_peered":"2024-04-25T08:42:21.358285+0000","last_unstale":"2024-04-25T08:42:27.390755+0000","last_undegraded":"2024-04-25T08:42:27.390755+0000","last_fullsized":"2024-04-25T08:42:27.390755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:45.573590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677940+0000","last_change":"2024-04-25T08:42:20.357678+0000","last_active":"2024-04-25T08:42:28.677940+0000","last_peered":"2024-04-25T08:42:28.677940+0000","last_clean":"2024-04-25T08:42:28.677940+0000","last_became_active":"2024-04-25T08:42:20.357378+0000","last_became_peered":"2024-04-25T08:42:20.357378+0000","last_unstale":"2024-04-25T08:42:28.677940+0000","last_undegraded":"2024-04-25T08:42:28.677940+0000","last_fullsized":"2024-04-25T08:42:28.677940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390203+0000","last_change":"2024-04-25T08:42:25.406294+0000","last_active":"2024-04-25T08:42:27.390203+0000","last_peered":"2024-04-25T08:42:27.390203+0000","last_clean":"2024-04-25T08:42:27.390203+0000","last_became_active":"2024-04-25T08:42:25.405833+0000","last_became_peered":"2024-04-25T08:42:25.405833+0000","last_unstale":"2024-04-25T08:42:27.390203+0000","last_undegraded":"2024-04-25T08:42:27.390203+0000","last_fullsized":"2024-04-25T08:42:27.390203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:42.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683217+0000","last_change":"2024-04-25T08:42:21.363445+0000","last_active":"2024-04-25T08:42:28.683217+0000","last_peered":"2024-04-25T08:42:28.683217+0000","last_clean":"2024-04-25T08:42:28.683217+0000","last_became_active":"2024-04-25T08:42:21.363129+0000","last_became_peered":"2024-04-25T08:42:21.363129+0000","last_unstale":"2024-04-25T08:42:28.683217+0000","last_undegraded":"2024-04-25T08:42:28.683217+0000","last_fullsized":"2024-04-25T08:42:28.683217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583076+0000","last_change":"2024-04-25T08:42:20.352637+0000","last_active":"2024-04-25T08:42:27.583076+0000","last_peered":"2024-04-25T08:42:27.583076+0000","last_clean":"2024-04-25T08:42:27.583076+0000","last_became_active":"2024-04-25T08:42:20.351776+0000","last_became_peered":"2024-04-25T08:42:20.351776+0000","last_unstale":"2024-04-25T08:42:27.583076+0000","last_undegraded":"2024-04-25T08:42:27.583076+0000","last_fullsized":"2024-04-25T08:42:27.583076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:53.741991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384221+0000","last_change":"2024-04-25T08:42:25.406488+0000","last_active":"2024-04-25T08:42:27.384221+0000","last_peered":"2024-04-25T08:42:27.384221+0000","last_clean":"2024-04-25T08:42:27.384221+0000","last_became_active":"2024-04-25T08:42:25.405903+0000","last_became_peered":"2024-04-25T08:42:25.405903+0000","last_unstale":"2024-04-25T08:42:27.384221+0000","last_undegraded":"2024-04-25T08:42:27.384221+0000","last_fullsized":"2024-04-25T08:42:27.384221+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:52.897838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583032+0000","last_change":"2024-04-25T08:42:21.376939+0000","last_active":"2024-04-25T08:42:27.583032+0000","last_peered":"2024-04-25T08:42:27.583032+0000","last_clean":"2024-04-25T08:42:27.583032+0000","last_became_active":"2024-04-25T08:42:21.376693+0000","last_became_peered":"2024-04-25T08:42:21.376693+0000","last_unstale":"2024-04-25T08:42:27.583032+0000","last_undegraded":"2024-04-25T08:42:27.583032+0000","last_fullsized":"2024-04-25T08:42:27.583032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:16.561880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389379+0000","last_change":"2024-04-25T08:42:20.355744+0000","last_active":"2024-04-25T08:42:27.389379+0000","last_peered":"2024-04-25T08:42:27.389379+0000","last_clean":"2024-04-25T08:42:27.389379+0000","last_became_active":"2024-04-25T08:42:20.355365+0000","last_became_peered":"2024-04-25T08:42:20.355365+0000","last_unstale":"2024-04-25T08:42:27.389379+0000","last_undegraded":"2024-04-25T08:42:27.389379+0000","last_fullsized":"2024-04-25T08:42:27.389379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384187+0000","last_change":"2024-04-25T08:42:25.833881+0000","last_active":"2024-04-25T08:42:27.384187+0000","last_peered":"2024-04-25T08:42:27.384187+0000","last_clean":"2024-04-25T08:42:27.384187+0000","last_became_active":"2024-04-25T08:42:25.833475+0000","last_became_peered":"2024-04-25T08:42:25.833475+0000","last_unstale":"2024-04-25T08:42:27.384187+0000","last_undegraded":"2024-04-25T08:42:27.384187+0000","last_fullsized":"2024-04-25T08:42:27.384187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:49.130328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683043+0000","last_change":"2024-04-25T08:42:25.835305+0000","last_active":"2024-04-25T08:42:28.683043+0000","last_peered":"2024-04-25T08:42:28.683043+0000","last_clean":"2024-04-25T08:42:28.683043+0000","last_became_active":"2024-04-25T08:42:25.834632+0000","last_became_peered":"2024-04-25T08:42:25.834632+0000","last_unstale":"2024-04-25T08:42:28.683043+0000","last_undegraded":"2024-04-25T08:42:28.683043+0000","last_fullsized":"2024-04-25T08:42:28.683043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678200+0000","last_change":"2024-04-25T08:42:21.350716+0000","last_active":"2024-04-25T08:42:28.678200+0000","last_peered":"2024-04-25T08:42:28.678200+0000","last_clean":"2024-04-25T08:42:28.678200+0000","last_became_active":"2024-04-25T08:42:21.349383+0000","last_became_peered":"2024-04-25T08:42:21.349383+0000","last_unstale":"2024-04-25T08:42:28.678200+0000","last_undegraded":"2024-04-25T08:42:28.678200+0000","last_fullsized":"2024-04-25T08:42:28.678200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582596+0000","last_change":"2024-04-25T08:42:20.345077+0000","last_active":"2024-04-25T08:42:27.582596+0000","last_peered":"2024-04-25T08:42:27.582596+0000","last_clean":"2024-04-25T08:42:27.582596+0000","last_became_active":"2024-04-25T08:42:20.344526+0000","last_became_peered":"2024-04-25T08:42:20.344526+0000","last_unstale":"2024-04-25T08:42:27.582596+0000","last_undegraded":"2024-04-25T08:42:27.582596+0000","last_fullsized":"2024-04-25T08:42:27.582596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389292+0000","last_change":"2024-04-25T08:42:25.407527+0000","last_active":"2024-04-25T08:42:27.389292+0000","last_peered":"2024-04-25T08:42:27.389292+0000","last_clean":"2024-04-25T08:42:27.389292+0000","last_became_active":"2024-04-25T08:42:25.407221+0000","last_became_peered":"2024-04-25T08:42:25.407221+0000","last_unstale":"2024-04-25T08:42:27.389292+0000","last_undegraded":"2024-04-25T08:42:27.389292+0000","last_fullsized":"2024-04-25T08:42:27.389292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.610141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384221+0000","last_change":"2024-04-25T08:42:21.351795+0000","last_active":"2024-04-25T08:42:27.384221+0000","last_peered":"2024-04-25T08:42:27.384221+0000","last_clean":"2024-04-25T08:42:27.384221+0000","last_became_active":"2024-04-25T08:42:21.351544+0000","last_became_peered":"2024-04-25T08:42:21.351544+0000","last_unstale":"2024-04-25T08:42:27.384221+0000","last_undegraded":"2024-04-25T08:42:27.384221+0000","last_fullsized":"2024-04-25T08:42:27.384221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384681+0000","last_change":"2024-04-25T08:42:20.354629+0000","last_active":"2024-04-25T08:42:27.384681+0000","last_peered":"2024-04-25T08:42:27.384681+0000","last_clean":"2024-04-25T08:42:27.384681+0000","last_became_active":"2024-04-25T08:42:20.354388+0000","last_became_peered":"2024-04-25T08:42:20.354388+0000","last_unstale":"2024-04-25T08:42:27.384681+0000","last_undegraded":"2024-04-25T08:42:27.384681+0000","last_fullsized":"2024-04-25T08:42:27.384681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.311211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388979+0000","last_change":"2024-04-25T08:42:20.350141+0000","last_active":"2024-04-25T08:42:27.388979+0000","last_peered":"2024-04-25T08:42:27.388979+0000","last_clean":"2024-04-25T08:42:27.388979+0000","last_became_active":"2024-04-25T08:42:20.349168+0000","last_became_peered":"2024-04-25T08:42:20.349168+0000","last_unstale":"2024-04-25T08:42:27.388979+0000","last_undegraded":"2024-04-25T08:42:27.388979+0000","last_fullsized":"2024-04-25T08:42:27.388979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683007+0000","last_change":"2024-04-25T08:42:21.363608+0000","last_active":"2024-04-25T08:42:28.683007+0000","last_peered":"2024-04-25T08:42:28.683007+0000","last_clean":"2024-04-25T08:42:28.683007+0000","last_became_active":"2024-04-25T08:42:21.363152+0000","last_became_peered":"2024-04-25T08:42:21.363152+0000","last_unstale":"2024-04-25T08:42:28.683007+0000","last_undegraded":"2024-04-25T08:42:28.683007+0000","last_fullsized":"2024-04-25T08:42:28.683007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583434+0000","last_change":"2024-04-25T08:42:25.413462+0000","last_active":"2024-04-25T08:42:28.583434+0000","last_peered":"2024-04-25T08:42:28.583434+0000","last_clean":"2024-04-25T08:42:28.583434+0000","last_became_active":"2024-04-25T08:42:25.412613+0000","last_became_peered":"2024-04-25T08:42:25.412613+0000","last_unstale":"2024-04-25T08:42:28.583434+0000","last_undegraded":"2024-04-25T08:42:28.583434+0000","last_fullsized":"2024-04-25T08:42:28.583434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:52.601446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384923+0000","last_change":"2024-04-25T08:42:20.355705+0000","last_active":"2024-04-25T08:42:27.384923+0000","last_peered":"2024-04-25T08:42:27.384923+0000","last_clean":"2024-04-25T08:42:27.384923+0000","last_became_active":"2024-04-25T08:42:20.355433+0000","last_became_peered":"2024-04-25T08:42:20.355433+0000","last_unstale":"2024-04-25T08:42:27.384923+0000","last_undegraded":"2024-04-25T08:42:27.384923+0000","last_fullsized":"2024-04-25T08:42:27.384923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:33.260664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390028+0000","last_change":"2024-04-25T08:42:21.344759+0000","last_active":"2024-04-25T08:42:27.390028+0000","last_peered":"2024-04-25T08:42:27.390028+0000","last_clean":"2024-04-25T08:42:27.390028+0000","last_became_active":"2024-04-25T08:42:21.344560+0000","last_became_peered":"2024-04-25T08:42:21.344560+0000","last_unstale":"2024-04-25T08:42:27.390028+0000","last_undegraded":"2024-04-25T08:42:27.390028+0000","last_fullsized":"2024-04-25T08:42:27.390028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:32.295514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385225+0000","last_change":"2024-04-25T08:42:25.413565+0000","last_active":"2024-04-25T08:42:27.385225+0000","last_peered":"2024-04-25T08:42:27.385225+0000","last_clean":"2024-04-25T08:42:27.385225+0000","last_became_active":"2024-04-25T08:42:25.413198+0000","last_became_peered":"2024-04-25T08:42:25.413198+0000","last_unstale":"2024-04-25T08:42:27.385225+0000","last_undegraded":"2024-04-25T08:42:27.385225+0000","last_fullsized":"2024-04-25T08:42:27.385225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:13.096012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388406+0000","last_change":"2024-04-25T08:42:21.363343+0000","last_active":"2024-04-25T08:42:27.388406+0000","last_peered":"2024-04-25T08:42:27.388406+0000","last_clean":"2024-04-25T08:42:27.388406+0000","last_became_active":"2024-04-25T08:42:21.363101+0000","last_became_peered":"2024-04-25T08:42:21.363101+0000","last_unstale":"2024-04-25T08:42:27.388406+0000","last_undegraded":"2024-04-25T08:42:27.388406+0000","last_fullsized":"2024-04-25T08:42:27.388406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583299+0000","last_change":"2024-04-25T08:42:21.361026+0000","last_active":"2024-04-25T08:42:28.583299+0000","last_peered":"2024-04-25T08:42:28.583299+0000","last_clean":"2024-04-25T08:42:28.583299+0000","last_became_active":"2024-04-25T08:42:21.360012+0000","last_became_peered":"2024-04-25T08:42:21.360012+0000","last_unstale":"2024-04-25T08:42:28.583299+0000","last_undegraded":"2024-04-25T08:42:28.583299+0000","last_fullsized":"2024-04-25T08:42:28.583299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:29.830442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582614+0000","last_change":"2024-04-25T08:42:21.367540+0000","last_active":"2024-04-25T08:42:27.582614+0000","last_peered":"2024-04-25T08:42:27.582614+0000","last_clean":"2024-04-25T08:42:27.582614+0000","last_became_active":"2024-04-25T08:42:21.367291+0000","last_became_peered":"2024-04-25T08:42:21.367291+0000","last_unstale":"2024-04-25T08:42:27.582614+0000","last_undegraded":"2024-04-25T08:42:27.582614+0000","last_fullsized":"2024-04-25T08:42:27.582614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.923264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683000+0000","last_change":"2024-04-25T08:42:21.366788+0000","last_active":"2024-04-25T08:42:28.683000+0000","last_peered":"2024-04-25T08:42:28.683000+0000","last_clean":"2024-04-25T08:42:28.683000+0000","last_became_active":"2024-04-25T08:42:21.365945+0000","last_became_peered":"2024-04-25T08:42:21.365945+0000","last_unstale":"2024-04-25T08:42:28.683000+0000","last_undegraded":"2024-04-25T08:42:28.683000+0000","last_fullsized":"2024-04-25T08:42:28.683000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385215+0000","last_change":"2024-04-25T08:42:21.351963+0000","last_active":"2024-04-25T08:42:27.385215+0000","last_peered":"2024-04-25T08:42:27.385215+0000","last_clean":"2024-04-25T08:42:27.385215+0000","last_became_active":"2024-04-25T08:42:21.351587+0000","last_became_peered":"2024-04-25T08:42:21.351587+0000","last_unstale":"2024-04-25T08:42:27.385215+0000","last_undegraded":"2024-04-25T08:42:27.385215+0000","last_fullsized":"2024-04-25T08:42:27.385215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:30.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678682+0000","last_change":"2024-04-25T08:42:21.363969+0000","last_active":"2024-04-25T08:42:28.678682+0000","last_peered":"2024-04-25T08:42:28.678682+0000","last_clean":"2024-04-25T08:42:28.678682+0000","last_became_active":"2024-04-25T08:42:21.354733+0000","last_became_peered":"2024-04-25T08:42:21.354733+0000","last_unstale":"2024-04-25T08:42:28.678682+0000","last_undegraded":"2024-04-25T08:42:28.678682+0000","last_fullsized":"2024-04-25T08:42:28.678682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389184+0000","last_change":"2024-04-25T08:42:21.375124+0000","last_active":"2024-04-25T08:42:27.389184+0000","last_peered":"2024-04-25T08:42:27.389184+0000","last_clean":"2024-04-25T08:42:27.389184+0000","last_became_active":"2024-04-25T08:42:21.374653+0000","last_became_peered":"2024-04-25T08:42:21.374653+0000","last_unstale":"2024-04-25T08:42:27.389184+0000","last_undegraded":"2024-04-25T08:42:27.389184+0000","last_fullsized":"2024-04-25T08:42:27.389184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385387+0000","last_change":"2024-04-25T08:42:21.362798+0000","last_active":"2024-04-25T08:42:27.385387+0000","last_peered":"2024-04-25T08:42:27.385387+0000","last_clean":"2024-04-25T08:42:27.385387+0000","last_became_active":"2024-04-25T08:42:21.362412+0000","last_became_peered":"2024-04-25T08:42:21.362412+0000","last_unstale":"2024-04-25T08:42:27.385387+0000","last_undegraded":"2024-04-25T08:42:27.385387+0000","last_fullsized":"2024-04-25T08:42:27.385387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582584+0000","last_change":"2024-04-25T08:42:21.368298+0000","last_active":"2024-04-25T08:42:27.582584+0000","last_peered":"2024-04-25T08:42:27.582584+0000","last_clean":"2024-04-25T08:42:27.582584+0000","last_became_active":"2024-04-25T08:42:21.368073+0000","last_became_peered":"2024-04-25T08:42:21.368073+0000","last_unstale":"2024-04-25T08:42:27.582584+0000","last_undegraded":"2024-04-25T08:42:27.582584+0000","last_fullsized":"2024-04-25T08:42:27.582584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384593+0000","last_change":"2024-04-25T08:42:21.356172+0000","last_active":"2024-04-25T08:42:27.384593+0000","last_peered":"2024-04-25T08:42:27.384593+0000","last_clean":"2024-04-25T08:42:27.384593+0000","last_became_active":"2024-04-25T08:42:21.355920+0000","last_became_peered":"2024-04-25T08:42:21.355920+0000","last_unstale":"2024-04-25T08:42:27.384593+0000","last_undegraded":"2024-04-25T08:42:27.384593+0000","last_fullsized":"2024-04-25T08:42:27.384593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582542+0000","last_change":"2024-04-25T08:42:21.376313+0000","last_active":"2024-04-25T08:42:27.582542+0000","last_peered":"2024-04-25T08:42:27.582542+0000","last_clean":"2024-04-25T08:42:27.582542+0000","last_became_active":"2024-04-25T08:42:21.376047+0000","last_became_peered":"2024-04-25T08:42:21.376047+0000","last_unstale":"2024-04-25T08:42:27.582542+0000","last_undegraded":"2024-04-25T08:42:27.582542+0000","last_fullsized":"2024-04-25T08:42:27.582542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:24.505703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682755+0000","last_change":"2024-04-25T08:42:21.367017+0000","last_active":"2024-04-25T08:42:28.682755+0000","last_peered":"2024-04-25T08:42:28.682755+0000","last_clean":"2024-04-25T08:42:28.682755+0000","last_became_active":"2024-04-25T08:42:21.366480+0000","last_became_peered":"2024-04-25T08:42:21.366480+0000","last_unstale":"2024-04-25T08:42:28.682755+0000","last_undegraded":"2024-04-25T08:42:28.682755+0000","last_fullsized":"2024-04-25T08:42:28.682755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388881+0000","last_change":"2024-04-25T08:42:21.354458+0000","last_active":"2024-04-25T08:42:27.388881+0000","last_peered":"2024-04-25T08:42:27.388881+0000","last_clean":"2024-04-25T08:42:27.388881+0000","last_became_active":"2024-04-25T08:42:21.351083+0000","last_became_peered":"2024-04-25T08:42:21.351083+0000","last_unstale":"2024-04-25T08:42:27.388881+0000","last_undegraded":"2024-04-25T08:42:27.388881+0000","last_fullsized":"2024-04-25T08:42:27.388881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684511+0000","last_change":"2024-04-25T08:42:21.369898+0000","last_active":"2024-04-25T08:42:28.684511+0000","last_peered":"2024-04-25T08:42:28.684511+0000","last_clean":"2024-04-25T08:42:28.684511+0000","last_became_active":"2024-04-25T08:42:21.369443+0000","last_became_peered":"2024-04-25T08:42:21.369443+0000","last_unstale":"2024-04-25T08:42:28.684511+0000","last_undegraded":"2024-04-25T08:42:28.684511+0000","last_fullsized":"2024-04-25T08:42:28.684511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:16.561880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583661+0000","last_change":"2024-04-25T08:42:21.353156+0000","last_active":"2024-04-25T08:42:28.583661+0000","last_peered":"2024-04-25T08:42:28.583661+0000","last_clean":"2024-04-25T08:42:28.583661+0000","last_became_active":"2024-04-25T08:42:21.352670+0000","last_became_peered":"2024-04-25T08:42:21.352670+0000","last_unstale":"2024-04-25T08:42:28.583661+0000","last_undegraded":"2024-04-25T08:42:28.583661+0000","last_fullsized":"2024-04-25T08:42:28.583661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:45.573590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583361+0000","last_change":"2024-04-25T08:42:21.376357+0000","last_active":"2024-04-25T08:42:27.583361+0000","last_peered":"2024-04-25T08:42:27.583361+0000","last_clean":"2024-04-25T08:42:27.583361+0000","last_became_active":"2024-04-25T08:42:21.376092+0000","last_became_peered":"2024-04-25T08:42:21.376092+0000","last_unstale":"2024-04-25T08:42:27.583361+0000","last_undegraded":"2024-04-25T08:42:27.583361+0000","last_fullsized":"2024-04-25T08:42:27.583361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:15.923334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582441+0000","last_change":"2024-04-25T08:42:21.371951+0000","last_active":"2024-04-25T08:42:27.582441+0000","last_peered":"2024-04-25T08:42:27.582441+0000","last_clean":"2024-04-25T08:42:27.582441+0000","last_became_active":"2024-04-25T08:42:21.371306+0000","last_became_peered":"2024-04-25T08:42:21.371306+0000","last_unstale":"2024-04-25T08:42:27.582441+0000","last_undegraded":"2024-04-25T08:42:27.582441+0000","last_fullsized":"2024-04-25T08:42:27.582441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678232+0000","last_change":"2024-04-25T08:42:21.341496+0000","last_active":"2024-04-25T08:42:28.678232+0000","last_peered":"2024-04-25T08:42:28.678232+0000","last_clean":"2024-04-25T08:42:28.678232+0000","last_became_active":"2024-04-25T08:42:21.341224+0000","last_became_peered":"2024-04-25T08:42:21.341224+0000","last_unstale":"2024-04-25T08:42:28.678232+0000","last_undegraded":"2024-04-25T08:42:28.678232+0000","last_fullsized":"2024-04-25T08:42:28.678232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682722+0000","last_change":"2024-04-25T08:42:21.360668+0000","last_active":"2024-04-25T08:42:28.682722+0000","last_peered":"2024-04-25T08:42:28.682722+0000","last_clean":"2024-04-25T08:42:28.682722+0000","last_became_active":"2024-04-25T08:42:21.360398+0000","last_became_peered":"2024-04-25T08:42:21.360398+0000","last_unstale":"2024-04-25T08:42:28.682722+0000","last_undegraded":"2024-04-25T08:42:28.682722+0000","last_fullsized":"2024-04-25T08:42:28.682722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:33.626044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384826+0000","last_change":"2024-04-25T08:42:21.353592+0000","last_active":"2024-04-25T08:42:27.384826+0000","last_peered":"2024-04-25T08:42:27.384826+0000","last_clean":"2024-04-25T08:42:27.384826+0000","last_became_active":"2024-04-25T08:42:21.353329+0000","last_became_peered":"2024-04-25T08:42:21.353329+0000","last_unstale":"2024-04-25T08:42:27.384826+0000","last_undegraded":"2024-04-25T08:42:27.384826+0000","last_fullsized":"2024-04-25T08:42:27.384826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390624+0000","last_change":"2024-04-25T08:42:21.356315+0000","last_active":"2024-04-25T08:42:27.390624+0000","last_peered":"2024-04-25T08:42:27.390624+0000","last_clean":"2024-04-25T08:42:27.390624+0000","last_became_active":"2024-04-25T08:42:21.355905+0000","last_became_peered":"2024-04-25T08:42:21.355905+0000","last_unstale":"2024-04-25T08:42:27.390624+0000","last_undegraded":"2024-04-25T08:42:27.390624+0000","last_fullsized":"2024-04-25T08:42:27.390624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:48.723555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389154+0000","last_change":"2024-04-25T08:42:21.371337+0000","last_active":"2024-04-25T08:42:27.389154+0000","last_peered":"2024-04-25T08:42:27.389154+0000","last_clean":"2024-04-25T08:42:27.389154+0000","last_became_active":"2024-04-25T08:42:21.370950+0000","last_became_peered":"2024-04-25T08:42:21.370950+0000","last_unstale":"2024-04-25T08:42:27.389154+0000","last_undegraded":"2024-04-25T08:42:27.389154+0000","last_fullsized":"2024-04-25T08:42:27.389154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389045+0000","last_change":"2024-04-25T08:42:21.365819+0000","last_active":"2024-04-25T08:42:27.389045+0000","last_peered":"2024-04-25T08:42:27.389045+0000","last_clean":"2024-04-25T08:42:27.389045+0000","last_became_active":"2024-04-25T08:42:21.365562+0000","last_became_peered":"2024-04-25T08:42:21.365562+0000","last_unstale":"2024-04-25T08:42:27.389045+0000","last_undegraded":"2024-04-25T08:42:27.389045+0000","last_fullsized":"2024-04-25T08:42:27.389045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684772+0000","last_change":"2024-04-25T08:42:21.366180+0000","last_active":"2024-04-25T08:42:28.684772+0000","last_peered":"2024-04-25T08:42:28.684772+0000","last_clean":"2024-04-25T08:42:28.684772+0000","last_became_active":"2024-04-25T08:42:21.365934+0000","last_became_peered":"2024-04-25T08:42:21.365934+0000","last_unstale":"2024-04-25T08:42:28.684772+0000","last_undegraded":"2024-04-25T08:42:28.684772+0000","last_fullsized":"2024-04-25T08:42:28.684772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:36.740234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384130+0000","last_change":"2024-04-25T08:42:21.368124+0000","last_active":"2024-04-25T08:42:27.384130+0000","last_peered":"2024-04-25T08:42:27.384130+0000","last_clean":"2024-04-25T08:42:27.384130+0000","last_became_active":"2024-04-25T08:42:21.367798+0000","last_became_peered":"2024-04-25T08:42:21.367798+0000","last_unstale":"2024-04-25T08:42:27.384130+0000","last_undegraded":"2024-04-25T08:42:27.384130+0000","last_fullsized":"2024-04-25T08:42:27.384130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683866+0000","last_change":"2024-04-25T08:42:21.357410+0000","last_active":"2024-04-25T08:42:28.683866+0000","last_peered":"2024-04-25T08:42:28.683866+0000","last_clean":"2024-04-25T08:42:28.683866+0000","last_became_active":"2024-04-25T08:42:21.356770+0000","last_became_peered":"2024-04-25T08:42:21.356770+0000","last_unstale":"2024-04-25T08:42:28.683866+0000","last_undegraded":"2024-04-25T08:42:28.683866+0000","last_fullsized":"2024-04-25T08:42:28.683866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582479+0000","last_change":"2024-04-25T08:42:21.372610+0000","last_active":"2024-04-25T08:42:27.582479+0000","last_peered":"2024-04-25T08:42:27.582479+0000","last_clean":"2024-04-25T08:42:27.582479+0000","last_became_active":"2024-04-25T08:42:21.372336+0000","last_became_peered":"2024-04-25T08:42:21.372336+0000","last_unstale":"2024-04-25T08:42:27.582479+0000","last_undegraded":"2024-04-25T08:42:27.582479+0000","last_fullsized":"2024-04-25T08:42:27.582479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:25.717765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684118+0000","last_change":"2024-04-25T08:42:21.358088+0000","last_active":"2024-04-25T08:42:28.684118+0000","last_peered":"2024-04-25T08:42:28.684118+0000","last_clean":"2024-04-25T08:42:28.684118+0000","last_became_active":"2024-04-25T08:42:21.357310+0000","last_became_peered":"2024-04-25T08:42:21.357310+0000","last_unstale":"2024-04-25T08:42:28.684118+0000","last_undegraded":"2024-04-25T08:42:28.684118+0000","last_fullsized":"2024-04-25T08:42:28.684118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582808+0000","last_change":"2024-04-25T08:42:21.367883+0000","last_active":"2024-04-25T08:42:27.582808+0000","last_peered":"2024-04-25T08:42:27.582808+0000","last_clean":"2024-04-25T08:42:27.582808+0000","last_became_active":"2024-04-25T08:42:21.367176+0000","last_became_peered":"2024-04-25T08:42:21.367176+0000","last_unstale":"2024-04-25T08:42:27.582808+0000","last_undegraded":"2024-04-25T08:42:27.582808+0000","last_fullsized":"2024-04-25T08:42:27.582808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.525554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682913+0000","last_change":"2024-04-25T08:42:21.362174+0000","last_active":"2024-04-25T08:42:28.682913+0000","last_peered":"2024-04-25T08:42:28.682913+0000","last_clean":"2024-04-25T08:42:28.682913+0000","last_became_active":"2024-04-25T08:42:21.361897+0000","last_became_peered":"2024-04-25T08:42:21.361897+0000","last_unstale":"2024-04-25T08:42:28.682913+0000","last_undegraded":"2024-04-25T08:42:28.682913+0000","last_fullsized":"2024-04-25T08:42:28.682913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678838+0000","last_change":"2024-04-25T08:42:21.370457+0000","last_active":"2024-04-25T08:42:28.678838+0000","last_peered":"2024-04-25T08:42:28.678838+0000","last_clean":"2024-04-25T08:42:28.678838+0000","last_became_active":"2024-04-25T08:42:21.370091+0000","last_became_peered":"2024-04-25T08:42:21.370091+0000","last_unstale":"2024-04-25T08:42:28.678838+0000","last_undegraded":"2024-04-25T08:42:28.678838+0000","last_fullsized":"2024-04-25T08:42:28.678838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684699+0000","last_change":"2024-04-25T08:42:21.370936+0000","last_active":"2024-04-25T08:42:28.684699+0000","last_peered":"2024-04-25T08:42:28.684699+0000","last_clean":"2024-04-25T08:42:28.684699+0000","last_became_active":"2024-04-25T08:42:21.369447+0000","last_became_peered":"2024-04-25T08:42:21.369447+0000","last_unstale":"2024-04-25T08:42:28.684699+0000","last_undegraded":"2024-04-25T08:42:28.684699+0000","last_fullsized":"2024-04-25T08:42:28.684699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:38.445314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:40.929 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:42:40.929 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T08:42:40.930 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T08:42:40.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T08:42:40.930 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T08:42:41.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.111+0000 7f0c00600640 1 Processor -- start 2024-04-25T08:42:41.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.111+0000 7f0c00600640 1 -- start start 2024-04-25T08:42:41.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0c00600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf8159230 0x7f0bf815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0c00600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0bf80cbeb0 0x7f0bf80cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0c00600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf80cc700 con 0x7f0bf80cda80 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0c00600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf80cc9e0 con 0x7f0bf80cbeb0 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0c00600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf80cccc0 con 0x7f0bf8159230 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfd200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0bf80cbeb0 0x7f0bf80cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfe600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0bf80cda80 0x7f0bf815f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59016/0 (socket says 172.21.15.46:59016) 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfe600640 1 -- 172.21.15.46:0/3787363860 learned_addr learned my addr 172.21.15.46:0/3787363860 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfd200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0bf80cbeb0 0x7f0bf80cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59940/0 (socket says 172.21.15.46:59940) 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfdc00640 1 --2- 172.21.15.46:0/3787363860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf8159230 0x7f0bf815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfd200640 1 -- 172.21.15.46:0/3787363860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf8159230 msgr2=0x7f0bf815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfd200640 1 --2- 172.21.15.46:0/3787363860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf8159230 0x7f0bf815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfd200640 1 -- 172.21.15.46:0/3787363860 >> v1:172.21.15.46:6789/0 conn(0x7f0bf80cda80 legacy=0x7f0bf815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfd200640 1 -- 172.21.15.46:0/3787363860 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf80ccfa0 con 0x7f0bf80cbeb0 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bfdc00640 1 --2- 172.21.15.46:0/3787363860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf8159230 0x7f0bf815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:41.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.115+0000 7f0bf7e00640 1 -- 172.21.15.46:0/3787363860 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372145118 0 0) 0x7f0bf80cc700 con 0x7f0bf80cda80 2024-04-25T08:42:41.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0bfd200640 1 --2- 172.21.15.46:0/3787363860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cbeb0 0x7f0bf80cd540 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0be40097c0 tx=0x7f0be401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2193457b1120ac4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0bf7e00640 1 -- 172.21.15.46:0/3787363860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40b0070 con 0x7f0bf80cbeb0 2024-04-25T08:42:41.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0bf7e00640 1 -- 172.21.15.46:0/3787363860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be40ab600 con 0x7f0bf80cbeb0 2024-04-25T08:42:41.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0bf7e00640 1 -- 172.21.15.46:0/3787363860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40ab9d0 con 0x7f0bf80cbeb0 2024-04-25T08:42:41.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 -- 172.21.15.46:0/3787363860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cbeb0 msgr2=0x7f0bf80cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 --2- 172.21.15.46:0/3787363860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cbeb0 0x7f0bf80cd540 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0be40097c0 tx=0x7f0be401c410 comp rx=0 tx=0).stop 2024-04-25T08:42:41.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 -- 172.21.15.46:0/3787363860 shutdown_connections 2024-04-25T08:42:41.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 --2- 172.21.15.46:0/3787363860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cbeb0 0x7f0bf80cd540 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 --2- 172.21.15.46:0/3787363860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf8159230 0x7f0bf815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 -- 172.21.15.46:0/3787363860 >> 172.21.15.46:0/3787363860 conn(0x7f0bf80b12a0 msgr2=0x7f0bf80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 -- 172.21.15.46:0/3787363860 shutdown_connections 2024-04-25T08:42:41.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 -- 172.21.15.46:0/3787363860 wait complete. 2024-04-25T08:42:41.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 Processor -- start 2024-04-25T08:42:41.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 -- start start 2024-04-25T08:42:41.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf80cbeb0 0x7f0bf81411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cda80 0x7f0bf813d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf8159230 0x7f0bf81436c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf8168900 con 0x7f0bf80cda80 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf8168780 con 0x7f0bf8159230 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0c00600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf8168a80 con 0x7f0bf80cbeb0 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.119+0000 7f0bfd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cda80 0x7f0bf813d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cda80 0x7f0bf813d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59952/0 (socket says 172.21.15.46:59952) 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfd200640 1 -- 172.21.15.46:0/2541178263 learned_addr learned my addr 172.21.15.46:0/2541178263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfe600640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf8159230 0x7f0bf81436c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfdc00640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf80cbeb0 0x7f0bf81411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfd200640 1 -- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf80cbeb0 msgr2=0x7f0bf81411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfd200640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf80cbeb0 0x7f0bf81411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfd200640 1 -- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf8159230 msgr2=0x7f0bf81436c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfd200640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf8159230 0x7f0bf81436c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfd200640 1 -- 172.21.15.46:0/2541178263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf8144650 con 0x7f0bf80cda80 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfe600640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf8159230 0x7f0bf81436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:41.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bfd200640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cda80 0x7f0bf813d550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0be40ae040 tx=0x7f0be401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bf5600640 1 -- 172.21.15.46:0/2541178263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40b0050 con 0x7f0bf80cda80 2024-04-25T08:42:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bf5600640 1 -- 172.21.15.46:0/2541178263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be40b1040 con 0x7f0bf80cda80 2024-04-25T08:42:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bf5600640 1 -- 172.21.15.46:0/2541178263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be40b5870 con 0x7f0bf80cda80 2024-04-25T08:42:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf8144930 con 0x7f0bf80cda80 2024-04-25T08:42:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf81431a0 con 0x7f0bf80cda80 2024-04-25T08:42:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf8142cd0 con 0x7f0bf80cda80 2024-04-25T08:42:41.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.123+0000 7f0bf5600640 1 -- 172.21.15.46:0/2541178263 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0be40be030 con 0x7f0bf80cda80 2024-04-25T08:42:41.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.127+0000 7f0bf5600640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0bd8066f60 0x7f0bd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.127+0000 7f0bf5600640 1 -- 172.21.15.46:0/2541178263 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0be411f130 con 0x7f0bf80cda80 2024-04-25T08:42:41.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.127+0000 7f0bfdc00640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0bd8066f60 0x7f0bd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.127+0000 7f0bfdc00640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0bd8066f60 0x7f0bd8069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00a9a0 tx=0x7f0bec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.131+0000 7f0bf5600640 1 -- 172.21.15.46:0/2541178263 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0be40c5060 con 0x7f0bf80cda80 2024-04-25T08:42:41.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.331+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0bf8069080 con 0x7f0bf80cda80 2024-04-25T08:42:41.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0bf5600640 1 -- 172.21.15.46:0/2541178263 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165732 (secure 0 0 0) 0x7f0be4016090 con 0x7f0bf80cda80 2024-04-25T08:42:41.328 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0bd8066f60 msgr2=0x7f0bd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0bd8066f60 0x7f0bd8069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0bec00a9a0 tx=0x7f0bec008040 comp rx=0 tx=0).stop 2024-04-25T08:42:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cda80 msgr2=0x7f0bf813d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cda80 0x7f0bf813d550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0be40ae040 tx=0x7f0be401ac50 comp rx=0 tx=0).stop 2024-04-25T08:42:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 shutdown_connections 2024-04-25T08:42:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0bd8066f60 0x7f0bd8069420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bf8159230 0x7f0bf81436c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bf80cda80 0x7f0bf813d550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 --2- 172.21.15.46:0/2541178263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bf80cbeb0 0x7f0bf81411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 >> 172.21.15.46:0/2541178263 conn(0x7f0bf80b12a0 msgr2=0x7f0bf80b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 shutdown_connections 2024-04-25T08:42:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.335+0000 7f0c00600640 1 -- 172.21.15.46:0/2541178263 wait complete. 2024-04-25T08:42:41.349 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2153338604},{"type":"v1","addr":"172.21.15.82:6833","nonce":2153338604}]},"active_addr":"172.21.15.82:6833/2153338604","active_change":"2024-04-25T08:42:00.237367+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4100,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":3979036056}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2603082844}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1804410947}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":929990739}]}]} 2024-04-25T08:42:41.352 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T08:42:41.352 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T08:42:41.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:41.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb637000640 1 Processor -- start 2024-04-25T08:42:41.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb637000640 1 -- start start 2024-04-25T08:42:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb637000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb63015ef20 0x7fb6300cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb637000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6300cd6d0 0x7fb63009a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb637000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63015f300 con 0x7fb63009ab50 2024-04-25T08:42:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb637000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6300cddb0 con 0x7fb63015ef20 2024-04-25T08:42:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb637000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300cc3d0 con 0x7fb6300cd6d0 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb635000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb63009ab50 0x7fb630099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59026/0 (socket says 172.21.15.46:59026) 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb635000640 1 -- 172.21.15.46:0/3143593100 learned_addr learned my addr 172.21.15.46:0/3143593100 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb627400640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6300cd6d0 0x7fb63009a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62fe00640 1 --2- 172.21.15.46:0/3143593100 >> v2:172.21.15.82:3300/0 conn(0x7fb63015ef20 0x7fb6300cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935429937 0 0) 0x7fb63015f300 con 0x7fb63009ab50 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb614003530 con 0x7fb63009ab50 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 626305681 0 0) 0x7fb614003530 con 0x7fb63009ab50 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6300cd6d0 msgr2=0x7fb63009a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6300cd6d0 0x7fb63009a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 >> v2:172.21.15.82:3300/0 conn(0x7fb63015ef20 msgr2=0x7fb6300cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 --2- 172.21.15.46:0/3143593100 >> v2:172.21.15.82:3300/0 conn(0x7fb63015ef20 0x7fb6300cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6300cc6b0 con 0x7fb63009ab50 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fb628001680 con 0x7fb63009ab50 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62fe00640 1 --2- 172.21.15.46:0/3143593100 >> v2:172.21.15.82:3300/0 conn(0x7fb63015ef20 0x7fb6300cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 >> v1:172.21.15.46:6789/0 conn(0x7fb63009ab50 legacy=0x7fb630099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6140039c0 0x7fb614003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb614004810 0x7fb614004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb614005190 0x7fb61400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300cddb0 con 0x7fb6140039c0 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300cc3d0 con 0x7fb614005190 2024-04-25T08:42:41.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.503+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb614004610 con 0x7fb614004810 2024-04-25T08:42:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62fe00640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6140039c0 0x7fb614003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb627400640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb614004810 0x7fb614004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb635000640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb614005190 0x7fb61400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62fe00640 1 -- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb614004810 msgr2=0x7fb614004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62fe00640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb614004810 0x7fb614004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62fe00640 1 -- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb614005190 msgr2=0x7fb61400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62fe00640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb614005190 0x7fb61400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62fe00640 1 -- 172.21.15.46:0/3143593100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614003530 con 0x7fb6140039c0 2024-04-25T08:42:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb627400640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb614004810 0x7fb614004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62fe00640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6140039c0 0x7fb614003de0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb620009cb0 tx=0x7fb62001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2830e44122efea50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200b2070 con 0x7fb6140039c0 2024-04-25T08:42:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6200a5900 con 0x7fb6140039c0 2024-04-25T08:42:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb62f400640 1 -- 172.21.15.46:0/3143593100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620009880 con 0x7fb6140039c0 2024-04-25T08:42:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb637000640 1 -- 172.21.15.46:0/3143593100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6140039c0 msgr2=0x7fb614003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb637000640 1 --2- 172.21.15.46:0/3143593100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6140039c0 0x7fb614003de0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb620009cb0 tx=0x7fb62001cc60 comp rx=0 tx=0).stop 2024-04-25T08:42:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb635000640 1 -- 172.21.15.46:0/3143593100 reap_dead start 2024-04-25T08:42:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb637000640 1 -- 172.21.15.46:0/3143593100 shutdown_connections 2024-04-25T08:42:41.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.507+0000 7fb637000640 1 -- 172.21.15.46:0/3143593100 >> 172.21.15.46:0/3143593100 conn(0x7fb6300b3120 msgr2=0x7fb6300b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 -- 172.21.15.46:0/3143593100 shutdown_connections 2024-04-25T08:42:41.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 -- 172.21.15.46:0/3143593100 wait complete. 2024-04-25T08:42:41.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 Processor -- start 2024-04-25T08:42:41.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 -- start start 2024-04-25T08:42:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6140039c0 0x7fb63015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb630160390 0x7fb630160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb630161b70 0x7fb630161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300c5b10 con 0x7fb630160390 2024-04-25T08:42:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300c5990 con 0x7fb6140039c0 2024-04-25T08:42:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63009a110 con 0x7fb630161b70 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb627400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb630160390 0x7fb630160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb627400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb630160390 0x7fb630160810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59968/0 (socket says 172.21.15.46:59968) 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb627400640 1 -- 172.21.15.46:0/1282281386 learned_addr learned my addr 172.21.15.46:0/1282281386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb62fe00640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6140039c0 0x7fb63015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb635000640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb630161b70 0x7fb630161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb627400640 1 -- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb630161b70 msgr2=0x7fb630161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb627400640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb630161b70 0x7fb630161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb627400640 1 -- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6140039c0 msgr2=0x7fb63015a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb627400640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6140039c0 0x7fb63015a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb627400640 1 -- 172.21.15.46:0/1282281386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb630160ec0 con 0x7fb630160390 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb627400640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb630160390 0x7fb630160810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb6180101c0 tx=0x7fb618010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb62cc00640 1 -- 172.21.15.46:0/1282281386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61801a030 con 0x7fb630160390 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb62cc00640 1 -- 172.21.15.46:0/1282281386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6180039a0 con 0x7fb630160390 2024-04-25T08:42:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.511+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6301611a0 con 0x7fb630160390 2024-04-25T08:42:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.515+0000 7fb62cc00640 1 -- 172.21.15.46:0/1282281386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618003d70 con 0x7fb630160390 2024-04-25T08:42:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.515+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6300b41b0 con 0x7fb630160390 2024-04-25T08:42:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.515+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb630069080 con 0x7fb630160390 2024-04-25T08:42:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.515+0000 7fb62cc00640 1 -- 172.21.15.46:0/1282281386 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb61801d070 con 0x7fb630160390 2024-04-25T08:42:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.515+0000 7fb62cc00640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb60c066f90 0x7fb60c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.515+0000 7fb62cc00640 1 -- 172.21.15.46:0/1282281386 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb6180861a0 con 0x7fb630160390 2024-04-25T08:42:41.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.515+0000 7fb62fe00640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb60c066f90 0x7fb60c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.515+0000 7fb62fe00640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb60c066f90 0x7fb60c069450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6200af000 tx=0x7fb6200046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.519+0000 7fb62cc00640 1 -- 172.21.15.46:0/1282281386 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb618056e60 con 0x7fb630160390 2024-04-25T08:42:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.683+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6300d0700 con 0x7fb630160390 2024-04-25T08:42:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.683+0000 7fb62cc00640 1 -- 172.21.15.46:0/1282281386 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7fb618056760 con 0x7fb630160390 2024-04-25T08:42:41.677 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:41.677 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb60c066f90 msgr2=0x7fb60c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb60c066f90 0x7fb60c069450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6200af000 tx=0x7fb6200046d0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb630160390 msgr2=0x7fb630160810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb630160390 0x7fb630160810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb6180101c0 tx=0x7fb618010680 comp rx=0 tx=0).stop 2024-04-25T08:42:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 shutdown_connections 2024-04-25T08:42:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb60c066f90 0x7fb60c069450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb630161b70 0x7fb630161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb630160390 0x7fb630160810 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 --2- 172.21.15.46:0/1282281386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6140039c0 0x7fb63015a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 >> 172.21.15.46:0/1282281386 conn(0x7fb6300b3120 msgr2=0x7fb630099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 shutdown_connections 2024-04-25T08:42:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.687+0000 7fb637000640 1 -- 172.21.15.46:0/1282281386 wait complete. 2024-04-25T08:42:41.699 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T08:42:41.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:41.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.847+0000 7f1c81600640 1 Processor -- start 2024-04-25T08:42:41.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.847+0000 7f1c81600640 1 -- start start 2024-04-25T08:42:41.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.847+0000 7f1c81600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c7c15dbd0 0x7f1c7c0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.847+0000 7f1c81600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c0713a0 0x7f1c7c06c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.847+0000 7f1c81600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c7c0b5250 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.847+0000 7f1c81600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c0b5af0 con 0x7f1c7c15dbd0 2024-04-25T08:42:41.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.847+0000 7f1c81600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0b5530 con 0x7f1c7c0713a0 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c0713a0 0x7f1c7c06c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7aa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c7c15dbd0 0x7f1c7c0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c72000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1c7c0b0c30 0x7f1c7c070c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:35474/0 (socket says 172.21.15.46:35474) 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7aa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c7c15dbd0 0x7f1c7c0b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:49954/0 (socket says 172.21.15.46:49954) 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c72000640 1 -- 172.21.15.46:0/3619374980 learned_addr learned my addr 172.21.15.46:0/3619374980 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c0713a0 0x7f1c7c06c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35804/0 (socket says 172.21.15.46:35804) 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7b400640 1 -- 172.21.15.46:0/3619374980 >> v2:172.21.15.82:3300/0 conn(0x7f1c7c15dbd0 msgr2=0x7f1c7c0b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7b400640 1 --2- 172.21.15.46:0/3619374980 >> v2:172.21.15.82:3300/0 conn(0x7f1c7c15dbd0 0x7f1c7c0b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7b400640 1 -- 172.21.15.46:0/3619374980 >> v1:172.21.15.82:6789/0 conn(0x7f1c7c0b0c30 legacy=0x7f1c7c070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7b400640 1 -- 172.21.15.46:0/3619374980 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0b5810 con 0x7f1c7c0713a0 2024-04-25T08:42:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7a000640 1 -- 172.21.15.46:0/3619374980 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585491405 0 0) 0x7f1c7c0b5250 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7b400640 1 --2- 172.21.15.46:0/3619374980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c0713a0 0x7f1c7c06c380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009d90 tx=0x7f1c7401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6b4a57700e7261f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7a000640 1 -- 172.21.15.46:0/3619374980 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740b2070 con 0x7f1c7c0713a0 2024-04-25T08:42:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7a000640 1 -- 172.21.15.46:0/3619374980 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c740a5940 con 0x7f1c7c0713a0 2024-04-25T08:42:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.851+0000 7f1c7a000640 1 -- 172.21.15.46:0/3619374980 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740a5d10 con 0x7f1c7c0713a0 2024-04-25T08:42:41.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 -- 172.21.15.46:0/3619374980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c0713a0 msgr2=0x7f1c7c06c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 --2- 172.21.15.46:0/3619374980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c0713a0 0x7f1c7c06c380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009d90 tx=0x7f1c7401c970 comp rx=0 tx=0).stop 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 -- 172.21.15.46:0/3619374980 shutdown_connections 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 --2- 172.21.15.46:0/3619374980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c0713a0 0x7f1c7c06c380 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 --2- 172.21.15.46:0/3619374980 >> v2:172.21.15.82:3300/0 conn(0x7f1c7c15dbd0 0x7f1c7c0b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 -- 172.21.15.46:0/3619374980 >> 172.21.15.46:0/3619374980 conn(0x7f1c7c06f4f0 msgr2=0x7f1c7c06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 -- 172.21.15.46:0/3619374980 shutdown_connections 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 -- 172.21.15.46:0/3619374980 wait complete. 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 Processor -- start 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 -- start start 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0713a0 0x7f1c7c1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0b0c30 0x7f1c7c154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c15dbd0 0x7f1c7c0d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c06d4e0 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c06d360 con 0x7f1c7c0713a0 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c81600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c06d660 con 0x7f1c7c15dbd0 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c72000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0b0c30 0x7f1c7c154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c7aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0713a0 0x7f1c7c1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c72000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0b0c30 0x7f1c7c154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59984/0 (socket says 172.21.15.46:59984) 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c72000640 1 -- 172.21.15.46:0/1231500505 learned_addr learned my addr 172.21.15.46:0/1231500505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c7b400640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c15dbd0 0x7f1c7c0d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c72000640 1 -- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c15dbd0 msgr2=0x7f1c7c0d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c72000640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c15dbd0 0x7f1c7c0d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c72000640 1 -- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0713a0 msgr2=0x7f1c7c1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c72000640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0713a0 0x7f1c7c1539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.855+0000 7f1c72000640 1 -- 172.21.15.46:0/1231500505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c155640 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c72000640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0b0c30 0x7f1c7c154cb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c520 tx=0x7f1c6400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c73400640 1 -- 172.21.15.46:0/1231500505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64018070 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c73400640 1 -- 172.21.15.46:0/1231500505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c640074e0 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c155920 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c73400640 1 -- 172.21.15.46:0/1231500505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640049f0 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c15e300 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c069080 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c73400640 1 -- 172.21.15.46:0/1231500505 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c6401b070 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c73400640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c58066f60 0x7f1c58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c73400640 1 -- 172.21.15.46:0/1231500505 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1c640847f0 con 0x7f1c7c0b0c30 2024-04-25T08:42:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.859+0000 7f1c7aa00640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c58066f60 0x7f1c58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.863+0000 7f1c7aa00640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c58066f60 0x7f1c58069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009ac0 tx=0x7f1c6c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:41.863+0000 7f1c73400640 1 -- 172.21.15.46:0/1231500505 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c64054de0 con 0x7f1c7c0b0c30 2024-04-25T08:42:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c7c0d25a0 con 0x7f1c7c0b0c30 2024-04-25T08:42:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c73400640 1 -- 172.21.15.46:0/1231500505 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f1c64058e80 con 0x7f1c7c0b0c30 2024-04-25T08:42:42.020 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:42.020 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c58066f60 msgr2=0x7f1c58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c58066f60 0x7f1c58069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009ac0 tx=0x7f1c6c009190 comp rx=0 tx=0).stop 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0b0c30 msgr2=0x7f1c7c154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0b0c30 0x7f1c7c154cb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400c520 tx=0x7f1c6400c9e0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 shutdown_connections 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c58066f60 0x7f1c58069420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c7c15dbd0 0x7f1c7c0d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c7c0b0c30 0x7f1c7c154cb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 --2- 172.21.15.46:0/1231500505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0713a0 0x7f1c7c1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 >> 172.21.15.46:0/1231500505 conn(0x7f1c7c06f4f0 msgr2=0x7f1c7c0b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 shutdown_connections 2024-04-25T08:42:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.027+0000 7f1c81600640 1 -- 172.21.15.46:0/1231500505 wait complete. 2024-04-25T08:42:42.041 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T08:42:42.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T08:42:42.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T08:42:42.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T08:42:42.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T08:42:42.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T08:42:42.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T08:42:42.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T08:42:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7f4166600640 1 Processor -- start 2024-04-25T08:42:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7f4166600640 1 -- start start 2024-04-25T08:42:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7fd449400640 1 Processor -- start 2024-04-25T08:42:42.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7fd449400640 1 -- start start 2024-04-25T08:42:42.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7fd449400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd43c033e40 0x7fd43c0eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7fd449400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0eb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7f4166600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41580eaa80 0x7f4158058630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7f4166600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f415803eb40 0x7f415803c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7f4166600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f4158053550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7f4166600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41580581a0 con 0x7f41580eaa80 2024-04-25T08:42:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7f4166600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41580572b0 con 0x7f415803eb40 2024-04-25T08:42:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7f4166600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4158057590 con 0x7f415803c5e0 2024-04-25T08:42:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7fd449400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43c034220 con 0x7fd43c0ebc00 2024-04-25T08:42:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7fd449400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c050210 con 0x7fd43c033e40 2024-04-25T08:42:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.295+0000 7fd449400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c037bc0 con 0x7fd43c047c20 2024-04-25T08:42:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469e800640 1 Processor -- start 2024-04-25T08:42:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469e800640 1 -- start start 2024-04-25T08:42:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd442800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd43c033e40 0x7fd43c0eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd443200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd43c0ebc00 0x7fd43c052f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59042/0 (socket says 172.21.15.46:59042) 2024-04-25T08:42:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd443200640 1 -- 172.21.15.46:0/3045606062 learned_addr learned my addr 172.21.15.46:0/3045606062 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41580eaa80 0x7f4158058630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41580eaa80 0x7f4158058630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59994/0 (socket says 172.21.15.46:59994) 2024-04-25T08:42:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415f400640 1 -- 172.21.15.46:0/2037824505 learned_addr learned my addr 172.21.15.46:0/2037824505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415fe00640 1 --2- 172.21.15.46:0/2037824505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f4158053550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415ea00640 1 --2- 172.21.15.46:0/2037824505 >> v2:172.21.15.82:3300/0 conn(0x7f415803eb40 0x7f415803c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd441e00640 1 --2- 172.21.15.46:0/3045606062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0eb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd441400640 1 -- 172.21.15.46:0/3045606062 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165387686 0 0) 0x7fd43c034220 con 0x7fd43c0ebc00 2024-04-25T08:42:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd441400640 1 -- 172.21.15.46:0/3045606062 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd428003530 con 0x7fd43c0ebc00 2024-04-25T08:42:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469e800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4698161c70 0x7f4698162e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469e800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4698071750 0x7f4698165880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469e800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46980cd210 con 0x7f46980cdc20 2024-04-25T08:42:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469e800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46980cd4f0 con 0x7f4698071750 2024-04-25T08:42:42.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469e800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46980cd7d0 con 0x7f4698161c70 2024-04-25T08:42:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469de00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4698071750 0x7f4698165880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469de00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4698071750 0x7f4698165880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:49986/0 (socket says 172.21.15.46:49986) 2024-04-25T08:42:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469de00640 1 -- 172.21.15.46:0/4062129983 learned_addr learned my addr 172.21.15.46:0/4062129983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd441e00640 1 -- 172.21.15.46:0/3045606062 >> v2:172.21.15.82:3300/0 conn(0x7fd43c033e40 msgr2=0x7fd43c0eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd441e00640 1 --2- 172.21.15.46:0/3045606062 >> v2:172.21.15.82:3300/0 conn(0x7fd43c033e40 0x7fd43c0eb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd441e00640 1 -- 172.21.15.46:0/3045606062 >> v1:172.21.15.46:6789/0 conn(0x7fd43c0ebc00 legacy=0x7fd43c052f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7fd441e00640 1 -- 172.21.15.46:0/3045606062 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c04fa30 con 0x7fd43c047c20 2024-04-25T08:42:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415f400640 1 -- 172.21.15.46:0/2037824505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 msgr2=0x7f4158053550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415f400640 1 --2- 172.21.15.46:0/2037824505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f4158053550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415f400640 1 -- 172.21.15.46:0/2037824505 >> v2:172.21.15.82:3300/0 conn(0x7f415803eb40 msgr2=0x7f415803c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415f400640 1 --2- 172.21.15.46:0/2037824505 >> v2:172.21.15.82:3300/0 conn(0x7f415803eb40 0x7f415803c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415f400640 1 -- 172.21.15.46:0/2037824505 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4158057870 con 0x7f41580eaa80 2024-04-25T08:42:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f415fe00640 1 --2- 172.21.15.46:0/2037824505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f4158053550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469de00640 1 -- 172.21.15.46:0/4062129983 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4698161c70 msgr2=0x7f4698162e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:42:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469de00640 1 --2- 172.21.15.46:0/4062129983 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4698161c70 0x7f4698162e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.299+0000 7f469de00640 1 -- 172.21.15.46:0/4062129983 >> v1:172.21.15.46:6789/0 conn(0x7f46980cdc20 legacy=0x7f4698071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7f469de00640 1 -- 172.21.15.46:0/4062129983 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469806e7d0 con 0x7f4698071750 2024-04-25T08:42:42.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd441e00640 1 --2- 172.21.15.46:0/3045606062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0eb6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd44402d380 tx=0x7fd444099d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8203227b2fbc19f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd441400640 1 -- 172.21.15.46:0/3045606062 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444123940 con 0x7fd43c047c20 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd441400640 1 -- 172.21.15.46:0/3045606062 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd444123ae0 con 0x7fd43c047c20 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd441400640 1 -- 172.21.15.46:0/3045606062 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44412b440 con 0x7fd43c047c20 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd449400640 1 -- 172.21.15.46:0/3045606062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 msgr2=0x7fd43c0eb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd449400640 1 --2- 172.21.15.46:0/3045606062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0eb6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd44402d380 tx=0x7fd444099d00 comp rx=0 tx=0).stop 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd449400640 1 -- 172.21.15.46:0/3045606062 shutdown_connections 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd449400640 1 --2- 172.21.15.46:0/3045606062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0eb6c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd449400640 1 --2- 172.21.15.46:0/3045606062 >> v2:172.21.15.82:3300/0 conn(0x7fd43c033e40 0x7fd43c0eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd449400640 1 -- 172.21.15.46:0/3045606062 >> 172.21.15.46:0/3045606062 conn(0x7fd43c009fc0 msgr2=0x7fd43c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd449400640 1 -- 172.21.15.46:0/3045606062 shutdown_connections 2024-04-25T08:42:42.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.303+0000 7fd449400640 1 -- 172.21.15.46:0/3045606062 wait complete. 2024-04-25T08:42:42.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f5c79200640 1 Processor -- start 2024-04-25T08:42:42.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f5c79200640 1 -- start start 2024-04-25T08:42:42.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7fd449400640 1 Processor -- start 2024-04-25T08:42:42.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f415f400640 1 --2- 172.21.15.46:0/2037824505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41580eaa80 0x7f4158058630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f415804fd40 tx=0x7f415401c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa782c9365903031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7fd449400640 1 -- start start 2024-04-25T08:42:42.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f469de00640 1 --2- 172.21.15.46:0/4062129983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4698071750 0x7f4698165880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4690009e50 tx=0x7f469001d440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef7b38a5479ed820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f415e000640 1 -- 172.21.15.46:0/2037824505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540b0070 con 0x7f41580eaa80 2024-04-25T08:42:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f4697e00640 1 -- 172.21.15.46:0/4062129983 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900a57a0 con 0x7f4698071750 2024-04-25T08:42:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f5c79200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c74161510 0x7f5c74070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f5c79200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c740711f0 0x7f5c7406c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f5c79200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7406bd20 con 0x7f5c740b0cf0 2024-04-25T08:42:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f5c79200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c740b10d0 con 0x7f5c740711f0 2024-04-25T08:42:42.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f5c79200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c741619f0 con 0x7f5c74161510 2024-04-25T08:42:42.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f4697e00640 1 -- 172.21.15.46:0/4062129983 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46900a5940 con 0x7f4698071750 2024-04-25T08:42:42.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f4697e00640 1 -- 172.21.15.46:0/4062129983 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46900a5d10 con 0x7f4698071750 2024-04-25T08:42:42.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f415e000640 1 -- 172.21.15.46:0/2037824505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41540ab600 con 0x7f41580eaa80 2024-04-25T08:42:42.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f5c71e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5c740b0cf0 0x7f5c740b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59052/0 (socket says 172.21.15.46:59052) 2024-04-25T08:42:42.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.307+0000 7f5c71e00640 1 -- 172.21.15.46:0/813389213 learned_addr learned my addr 172.21.15.46:0/813389213 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd449400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c033e40 0x7fd43c04e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f469e800640 1 -- 172.21.15.46:0/4062129983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4698071750 msgr2=0x7f4698165880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f469e800640 1 --2- 172.21.15.46:0/4062129983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4698071750 0x7f4698165880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4690009e50 tx=0x7f469001d440 comp rx=0 tx=0).stop 2024-04-25T08:42:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f5c72800640 1 --2- 172.21.15.46:0/813389213 >> v2:172.21.15.82:3300/0 conn(0x7f5c740711f0 0x7f5c7406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f5c71400640 1 --2- 172.21.15.46:0/813389213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c74161510 0x7f5c74070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd449400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0e9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd449400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd43c0ebc00 0x7fd43c04bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f469e800640 1 -- 172.21.15.46:0/4062129983 shutdown_connections 2024-04-25T08:42:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f469e800640 1 --2- 172.21.15.46:0/4062129983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4698071750 0x7f4698165880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd449400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c04fd80 con 0x7fd43c0ebc00 2024-04-25T08:42:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd449400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c04f090 con 0x7fd43c033e40 2024-04-25T08:42:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd449400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c04f5c0 con 0x7fd43c047c20 2024-04-25T08:42:42.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f469e800640 1 --2- 172.21.15.46:0/4062129983 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4698161c70 0x7f4698162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f415e000640 1 -- 172.21.15.46:0/2037824505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540ab9d0 con 0x7f41580eaa80 2024-04-25T08:42:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f469e800640 1 -- 172.21.15.46:0/4062129983 >> 172.21.15.46:0/4062129983 conn(0x7f4698066fa0 msgr2=0x7f469806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f7860200640 1 Processor -- start 2024-04-25T08:42:42.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd442800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c033e40 0x7fd43c04e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd441e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd442800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c033e40 0x7fd43c04e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50008/0 (socket says 172.21.15.46:50008) 2024-04-25T08:42:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd442800640 1 -- 172.21.15.46:0/1528608427 learned_addr learned my addr 172.21.15.46:0/1528608427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd441e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0e9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35840/0 (socket says 172.21.15.46:35840) 2024-04-25T08:42:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f5c70a00640 1 -- 172.21.15.46:0/813389213 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923829389 0 0) 0x7f5c7406bd20 con 0x7f5c740b0cf0 2024-04-25T08:42:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f5c70a00640 1 -- 172.21.15.46:0/813389213 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c58003530 con 0x7f5c740b0cf0 2024-04-25T08:42:42.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f5c71400640 1 -- 172.21.15.46:0/813389213 >> v2:172.21.15.82:3300/0 conn(0x7f5c740711f0 msgr2=0x7f5c7406c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f5c71400640 1 --2- 172.21.15.46:0/813389213 >> v2:172.21.15.82:3300/0 conn(0x7f5c740711f0 0x7f5c7406c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f5c71400640 1 -- 172.21.15.46:0/813389213 >> v1:172.21.15.46:6789/0 conn(0x7f5c740b0cf0 legacy=0x7f5c740b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f5c71400640 1 -- 172.21.15.46:0/813389213 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74161cd0 con 0x7f5c74161510 2024-04-25T08:42:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f7860200640 1 -- start start 2024-04-25T08:42:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f469e800640 1 -- 172.21.15.46:0/4062129983 shutdown_connections 2024-04-25T08:42:42.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f469e800640 1 -- 172.21.15.46:0/4062129983 wait complete. 2024-04-25T08:42:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd441e00640 1 -- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c033e40 msgr2=0x7fd43c04e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd441e00640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43c033e40 0x7fd43c04e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd441e00640 1 -- 172.21.15.46:0/1528608427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd43c0ebc00 msgr2=0x7fd43c04bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd441e00640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd43c0ebc00 0x7fd43c04bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f4166600640 1 -- 172.21.15.46:0/2037824505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41580eaa80 msgr2=0x7f4158058630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd441e00640 1 -- 172.21.15.46:0/1528608427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c04d750 con 0x7fd43c047c20 2024-04-25T08:42:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f4166600640 1 --2- 172.21.15.46:0/2037824505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41580eaa80 0x7f4158058630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f415804fd40 tx=0x7f415401c250 comp rx=0 tx=0).stop 2024-04-25T08:42:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd441e00640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0e9310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd44406a580 tx=0x7fd444098540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd43aa00640 1 -- 172.21.15.46:0/1528608427 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444139070 con 0x7fd43c047c20 2024-04-25T08:42:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd43aa00640 1 -- 172.21.15.46:0/1528608427 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd444123740 con 0x7fd43c047c20 2024-04-25T08:42:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd449400640 1 -- 172.21.15.46:0/1528608427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43c04d9f0 con 0x7fd43c047c20 2024-04-25T08:42:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd43aa00640 1 -- 172.21.15.46:0/1528608427 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4441345f0 con 0x7fd43c047c20 2024-04-25T08:42:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd449400640 1 -- 172.21.15.46:0/1528608427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c04dfb0 con 0x7fd43c047c20 2024-04-25T08:42:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f7860200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7858161ea0 0x7f78580b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f7860200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78580b1350 0x7f785806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f7860200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78581625f0 con 0x7f785806ac10 2024-04-25T08:42:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f7860200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785806b5b0 con 0x7f7858161ea0 2024-04-25T08:42:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f469e800640 1 Processor -- start 2024-04-25T08:42:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f7860200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7858163200 con 0x7f78580b1350 2024-04-25T08:42:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f785d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7858161ea0 0x7f78580b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f785d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7858161ea0 0x7f78580b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60026/0 (socket says 172.21.15.46:60026) 2024-04-25T08:42:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7f785d800640 1 -- 172.21.15.46:0/1482070896 learned_addr learned my addr 172.21.15.46:0/1482070896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.311+0000 7fd42fe00640 1 -- 172.21.15.46:0/1528608427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd418001040 con 0x7fd43c047c20 2024-04-25T08:42:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f785e200640 1 --2- 172.21.15.46:0/1482070896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78580b1350 0x7f785806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f4166600640 1 -- 172.21.15.46:0/2037824505 shutdown_connections 2024-04-25T08:42:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f4166600640 1 --2- 172.21.15.46:0/2037824505 >> v2:172.21.15.82:3300/0 conn(0x7f415803eb40 0x7f415803c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f4166600640 1 --2- 172.21.15.46:0/2037824505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f4158053550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f4166600640 1 --2- 172.21.15.46:0/2037824505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41580eaa80 0x7f4158058630 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f4166600640 1 -- 172.21.15.46:0/2037824505 >> 172.21.15.46:0/2037824505 conn(0x7f4158058f00 msgr2=0x7f4158059310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f469e800640 1 -- start start 2024-04-25T08:42:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fce4b800640 1 Processor -- start 2024-04-25T08:42:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fce4b800640 1 -- start start 2024-04-25T08:42:42.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f785d800640 1 -- 172.21.15.46:0/1482070896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78580b1350 msgr2=0x7f785806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f785d800640 1 --2- 172.21.15.46:0/1482070896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78580b1350 0x7f785806b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f785d800640 1 -- 172.21.15.46:0/1482070896 >> v1:172.21.15.46:6789/0 conn(0x7f785806ac10 legacy=0x7f78581659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f785d800640 1 -- 172.21.15.46:0/1482070896 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78581628d0 con 0x7f7858161ea0 2024-04-25T08:42:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f785e200640 1 --2- 172.21.15.46:0/1482070896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78580b1350 0x7f785806b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fd43aa00640 1 -- 172.21.15.46:0/1528608427 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd44413c070 con 0x7fd43c047c20 2024-04-25T08:42:42.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f4166600640 1 -- 172.21.15.46:0/2037824505 shutdown_connections 2024-04-25T08:42:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c71400640 1 --2- 172.21.15.46:0/813389213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c74161510 0x7f5c74070cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5c640097c0 tx=0x7f5c6401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c968225081c977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c70a00640 1 -- 172.21.15.46:0/813389213 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640b0070 con 0x7f5c74161510 2024-04-25T08:42:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f469e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4698071750 0x7f4698157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c70a00640 1 -- 172.21.15.46:0/813389213 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c640ab600 con 0x7f5c74161510 2024-04-25T08:42:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c70a00640 1 -- 172.21.15.46:0/813389213 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640ab9d0 con 0x7f5c74161510 2024-04-25T08:42:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f0e69e00640 1 Processor -- start 2024-04-25T08:42:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fce4b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fce440ced80 0x7fce441689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fce4b800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce44161e60 con 0x7fce4415db10 2024-04-25T08:42:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fce4b800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce44162140 con 0x7fce440b9460 2024-04-25T08:42:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fce4b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce440b98c0 con 0x7fce440ced80 2024-04-25T08:42:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f0e69e00640 1 -- start start 2024-04-25T08:42:42.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f469e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46980cdc20 0x7f46980b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fce49800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fce440ced80 0x7fce441689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fce43e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fce440b9460 0x7fce440ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:35480/0 (socket says 172.21.15.46:35480) 2024-04-25T08:42:42.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fce43e00640 1 -- 172.21.15.46:0/2064399224 learned_addr learned my addr 172.21.15.46:0/2064399224 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4698071750 0x7f4698157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469e800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4698161c70 0x7f469806d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980b2ac0 con 0x7f46980cdc20 2024-04-25T08:42:42.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469e800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46980b2940 con 0x7f4698071750 2024-04-25T08:42:42.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469e800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469816ec20 con 0x7f4698161c70 2024-04-25T08:42:42.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f4166600640 1 -- 172.21.15.46:0/2037824505 wait complete. 2024-04-25T08:42:42.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46980cdc20 0x7f46980b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46980cdc20 0x7f46980b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60036/0 (socket says 172.21.15.46:60036) 2024-04-25T08:42:42.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469ca00640 1 -- 172.21.15.46:0/1246396416 learned_addr learned my addr 172.21.15.46:0/1246396416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e69e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e6415db10 0x7f0e640b9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e69e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e64161e60 con 0x7f0e64161680 2024-04-25T08:42:42.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e69e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e64162140 con 0x7f0e640ced80 2024-04-25T08:42:42.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e69e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640d1ca0 con 0x7f0e6415db10 2024-04-25T08:42:42.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fce43400640 1 -- 172.21.15.46:0/2064399224 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219308977 0 0) 0x7fce44162140 con 0x7fce440b9460 2024-04-25T08:42:42.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fce43400640 1 -- 172.21.15.46:0/2064399224 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce28003530 con 0x7fce440b9460 2024-04-25T08:42:42.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fce49800640 1 -- 172.21.15.46:0/2064399224 >> v1:172.21.15.82:6789/0 conn(0x7fce440b9460 legacy=0x7fce440ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fce49800640 1 -- 172.21.15.46:0/2064399224 >> v1:172.21.15.46:6789/0 conn(0x7fce4415db10 legacy=0x7fce44161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fce49800640 1 -- 172.21.15.46:0/2064399224 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44162420 con 0x7fce440ced80 2024-04-25T08:42:42.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fd43aa00640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd424067030 0x7fd4240694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fd43aa00640 1 -- 172.21.15.46:0/1528608427 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd44419cde0 con 0x7fd43c047c20 2024-04-25T08:42:42.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fd442800640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd424067030 0x7fd4240694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7fd43aa00640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7fd4240707b0 0x7fd424072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fd443200640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7fd4240707b0 0x7fd424072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fd442800640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd424067030 0x7fd4240694f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd43000ba90 tx=0x7fd430008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f4166600640 1 Processor -- start 2024-04-25T08:42:42.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c79200640 1 -- 172.21.15.46:0/813389213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c74161510 msgr2=0x7f5c74070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c79200640 1 --2- 172.21.15.46:0/813389213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c74161510 0x7f5c74070cb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5c640097c0 tx=0x7f5c6401c4c0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c79200640 1 -- 172.21.15.46:0/813389213 shutdown_connections 2024-04-25T08:42:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c79200640 1 --2- 172.21.15.46:0/813389213 >> v2:172.21.15.82:3300/0 conn(0x7f5c740711f0 0x7f5c7406c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c79200640 1 --2- 172.21.15.46:0/813389213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c74161510 0x7f5c74070cb0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5c640097c0 tx=0x7f5c6401c4c0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fd43aa00640 1 -- 172.21.15.46:0/1528608427 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd4440626b0 con 0x7fd4240707b0 2024-04-25T08:42:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.315+0000 7f5c79200640 1 -- 172.21.15.46:0/813389213 >> 172.21.15.46:0/813389213 conn(0x7f5c7406f510 msgr2=0x7f5c7406f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fd443200640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7fd4240707b0 0x7fd424072bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fd43aa00640 1 -- 172.21.15.46:0/1528608427 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd44419f440 con 0x7fd43c047c20 2024-04-25T08:42:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f5c79200640 1 -- 172.21.15.46:0/813389213 shutdown_connections 2024-04-25T08:42:42.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469d400640 1 -- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4698161c70 msgr2=0x7f469806d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f5c79200640 1 -- 172.21.15.46:0/813389213 wait complete. 2024-04-25T08:42:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e62000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0e64161680 0x7f0e640ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59078/0 (socket says 172.21.15.46:59078) 2024-04-25T08:42:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e62000640 1 -- 172.21.15.46:0/1970632323 learned_addr learned my addr 172.21.15.46:0/1970632323 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469de00640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4698161c70 0x7f469806d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469d400640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4698161c70 0x7f469806d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469d400640 1 -- 172.21.15.46:0/1246396416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46980cdc20 msgr2=0x7f46980b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469d400640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46980cdc20 0x7f46980b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469d400640 1 -- 172.21.15.46:0/1246396416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469814f9a0 con 0x7f4698071750 2024-04-25T08:42:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469ca00640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46980cdc20 0x7f46980b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469de00640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4698161c70 0x7f469806d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f785d800640 1 --2- 172.21.15.46:0/1482070896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858161ea0 0x7f78580b3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f784c009870 tx=0x7f784c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36330f8acbb75a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654035769 0 0) 0x7f0e64162140 con 0x7f0e640ced80 2024-04-25T08:42:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e48003530 con 0x7f0e640ced80 2024-04-25T08:42:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f469d400640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4698071750 0x7f4698157380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f468400b3c0 tx=0x7f468400b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377364284 0 0) 0x7f0e64161e60 con 0x7f0e64161680 2024-04-25T08:42:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e62a00640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e6415db10 0x7f0e640b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e64162140 con 0x7f0e64161680 2024-04-25T08:42:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f4166600640 1 -- start start 2024-04-25T08:42:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f4166600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f415804e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f4695600640 1 -- 172.21.15.46:0/1246396416 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684011830 con 0x7f4698071750 2024-04-25T08:42:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 <== mon.1 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3059003676 0 0) 0x7f0e48003530 con 0x7f0e640ced80 2024-04-25T08:42:42.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e6415db10 msgr2=0x7f0e640b9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.319+0000 7fd43aa00640 1 -- 172.21.15.46:0/1528608427 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd4440626b0 con 0x7fd4240707b0 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e6415db10 0x7f0e640b9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 >> v1:172.21.15.46:6789/0 conn(0x7f0e64161680 legacy=0x7f0e640ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e64162420 con 0x7f0e640ced80 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 <== mon.1 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f0e5c001a00 con 0x7f0e640ced80 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 >> v1:172.21.15.82:6789/0 conn(0x7f0e640ced80 legacy=0x7f0e640d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 0x7f0e48003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e6415db10 0x7f0e48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e48004d50 0x7f0e480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640d1ca0 con 0x7f0e48004d50 2024-04-25T08:42:42.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4800ba80 con 0x7f0e48003b50 2024-04-25T08:42:42.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4800bc80 con 0x7f0e6415db10 2024-04-25T08:42:42.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4166600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f415803eb40 0x7f41580d78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f415804e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4166600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41580eaa80 0x7f4158050740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7857e00640 1 -- 172.21.15.46:0/1482070896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0b1070 con 0x7f7858161ea0 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7857e00640 1 -- 172.21.15.46:0/1482070896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c0ac630 con 0x7f7858161ea0 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e62a00640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 0x7f0e48003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f415804e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35870/0 (socket says 172.21.15.46:35870) 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41580eaa80 0x7f4158050740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41580eaa80 0x7f4158050740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50056/0 (socket says 172.21.15.46:50056) 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4166600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4158058b70 con 0x7f415803eb40 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4166600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415803ca30 con 0x7f41580eaa80 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4166600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415804f090 con 0x7f415803c5e0 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415f400640 1 -- 172.21.15.46:0/1705395690 learned_addr learned my addr 172.21.15.46:0/1705395690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469814fc80 con 0x7f4698071750 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4698150240 con 0x7f4698071750 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4695600640 1 -- 172.21.15.46:0/1246396416 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4684016070 con 0x7f4698071750 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4695600640 1 -- 172.21.15.46:0/1246396416 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684011830 con 0x7f4698071750 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4698069010 con 0x7f4698071750 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f5c79200640 1 Processor -- start 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7857e00640 1 -- 172.21.15.46:0/1482070896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0b1070 con 0x7f7858161ea0 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e62000640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e6415db10 0x7f0e48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e63400640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e48004d50 0x7f0e480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e62a00640 1 -- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e6415db10 msgr2=0x7f0e48004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e62a00640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e6415db10 0x7f0e48004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e62a00640 1 -- 172.21.15.46:0/1970632323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e48004d50 msgr2=0x7f0e480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e62a00640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e48004d50 0x7f0e480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e62a00640 1 -- 172.21.15.46:0/1970632323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e64162420 con 0x7f0e48003b50 2024-04-25T08:42:42.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e62000640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e6415db10 0x7f0e48004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:42.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f0e63400640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e48004d50 0x7f0e480051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:42.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f3027400640 1 Processor -- start 2024-04-25T08:42:42.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f3027400640 1 -- start start 2024-04-25T08:42:42.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7fce49800640 1 --2- 172.21.15.46:0/2064399224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce440ced80 0x7fce441689b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009dc0 tx=0x7fce3c01c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=375dd00e82964bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7860200640 1 -- 172.21.15.46:0/1482070896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858161ea0 msgr2=0x7f78580b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7860200640 1 --2- 172.21.15.46:0/1482070896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858161ea0 0x7f78580b3650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f784c009870 tx=0x7f784c01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415ea00640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f415803eb40 0x7f41580d78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7860200640 1 -- 172.21.15.46:0/1482070896 shutdown_connections 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7860200640 1 --2- 172.21.15.46:0/1482070896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78580b1350 0x7f785806b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7860200640 1 --2- 172.21.15.46:0/1482070896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7858161ea0 0x7f78580b3650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415f400640 1 -- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41580eaa80 msgr2=0x7f4158050740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415f400640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41580eaa80 0x7f4158050740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415f400640 1 -- 172.21.15.46:0/1705395690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f415803eb40 msgr2=0x7f41580d78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415f400640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f415803eb40 0x7f41580d78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415f400640 1 -- 172.21.15.46:0/1705395690 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415804f370 con 0x7f415803c5e0 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f415f400640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f415804e9b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c10 tx=0x7f4154018ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f5c79200640 1 -- start start 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7fce43400640 1 -- 172.21.15.46:0/2064399224 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0b2070 con 0x7fce440ced80 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7fce43400640 1 -- 172.21.15.46:0/2064399224 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c0a5790 con 0x7fce440ced80 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7860200640 1 -- 172.21.15.46:0/1482070896 >> 172.21.15.46:0/1482070896 conn(0x7f7858072920 msgr2=0x7f7858099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7860200640 1 -- 172.21.15.46:0/1482070896 shutdown_connections 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4153400640 1 -- 172.21.15.46:0/1705395690 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540ba020 con 0x7f415803c5e0 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4153400640 1 -- 172.21.15.46:0/1705395690 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4154004240 con 0x7f415803c5e0 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4153400640 1 -- 172.21.15.46:0/1705395690 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540b0040 con 0x7f415803c5e0 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415804f650 con 0x7f415803c5e0 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f3027400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f302006f2e0 0x7f3020157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f3027400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200b4640 0x7f30200aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f3027400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30200b2a00 con 0x7f30200ae6d0 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f3027400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30200b37c0 con 0x7f302006f2e0 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f3027400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30200af080 con 0x7f30200b4640 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7fce4b800640 1 -- 172.21.15.46:0/2064399224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce440ced80 msgr2=0x7fce441689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7fce43400640 1 -- 172.21.15.46:0/2064399224 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0b2070 con 0x7fce440ced80 2024-04-25T08:42:42.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41580de400 con 0x7f415803c5e0 2024-04-25T08:42:42.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7fce4b800640 1 --2- 172.21.15.46:0/2064399224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce440ced80 0x7fce441689b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009dc0 tx=0x7fce3c01c8d0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f3025400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200b4640 0x7f30200aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f3024a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f302006f2e0 0x7f3020157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f3024a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f302006f2e0 0x7f3020157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50058/0 (socket says 172.21.15.46:50058) 2024-04-25T08:42:42.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f3024a00640 1 -- 172.21.15.46:0/839469137 learned_addr learned my addr 172.21.15.46:0/839469137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41580e58f0 con 0x7f415803c5e0 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7fce4b800640 1 -- 172.21.15.46:0/2064399224 shutdown_connections 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7fce4b800640 1 --2- 172.21.15.46:0/2064399224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce440ced80 0x7fce441689b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7fce4b800640 1 -- 172.21.15.46:0/2064399224 >> 172.21.15.46:0/2064399224 conn(0x7fce440d0720 msgr2=0x7fce440d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c79200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c740711f0 0x7f5c74148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f3024a00640 1 -- 172.21.15.46:0/839469137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200b4640 msgr2=0x7f30200aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f3024a00640 1 --2- 172.21.15.46:0/839469137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200b4640 0x7f30200aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f3024a00640 1 -- 172.21.15.46:0/839469137 >> v1:172.21.15.46:6789/0 conn(0x7f30200ae6d0 legacy=0x7f30200b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f3024a00640 1 -- 172.21.15.46:0/839469137 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30200b2ce0 con 0x7f302006f2e0 2024-04-25T08:42:42.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c79200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c740b0cf0 0x7f5c741624d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7fce4b800640 1 -- 172.21.15.46:0/2064399224 shutdown_connections 2024-04-25T08:42:42.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f4695600640 1 -- 172.21.15.46:0/1246396416 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46840119f0 con 0x7f4698071750 2024-04-25T08:42:42.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f4695600640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4678067160 0x7f4678069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f4695600640 1 -- 172.21.15.46:0/1246396416 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4684085600 con 0x7f4698071750 2024-04-25T08:42:42.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f4695600640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f46780708d0 0x7f4678072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f4695600640 1 -- 172.21.15.46:0/1246396416 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f46840067e0 con 0x7f46780708d0 2024-04-25T08:42:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c79200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c741491e0 0x7f5c74149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c79200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7406d2e0 con 0x7f5c740711f0 2024-04-25T08:42:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c79200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7406d160 con 0x7f5c740b0cf0 2024-04-25T08:42:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c79200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7406d460 con 0x7f5c741491e0 2024-04-25T08:42:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c740711f0 0x7f5c74148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c740b0cf0 0x7f5c741624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c740b0cf0 0x7f5c741624d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50066/0 (socket says 172.21.15.46:50066) 2024-04-25T08:42:42.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71400640 1 -- 172.21.15.46:0/3967731305 learned_addr learned my addr 172.21.15.46:0/3967731305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7fce4b800640 1 -- 172.21.15.46:0/2064399224 wait complete. 2024-04-25T08:42:42.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.323+0000 7f7860200640 1 -- 172.21.15.46:0/1482070896 wait complete. 2024-04-25T08:42:42.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f4695600640 1 -- 172.21.15.46:0/1246396416 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46840859f0 con 0x7f4698071750 2024-04-25T08:42:42.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71400640 1 -- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c741491e0 msgr2=0x7f5c74149690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:42:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f0e62a00640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 0x7f0e48003f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000ba30 tx=0x7f0e5001c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de2d2f73586025a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71400640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c741491e0 0x7f5c74149690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71400640 1 -- 172.21.15.46:0/3967731305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c740711f0 msgr2=0x7f5c74148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71400640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c740711f0 0x7f5c74148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71400640 1 -- 172.21.15.46:0/3967731305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c740d1d50 con 0x7f5c740b0cf0 2024-04-25T08:42:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f4153400640 1 -- 172.21.15.46:0/1705395690 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f41540bd020 con 0x7f415803c5e0 2024-04-25T08:42:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f469ca00640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4678067160 0x7f4678069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5000b600 con 0x7f0e48003b50 2024-04-25T08:42:42.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e500b0070 con 0x7f0e48003b50 2024-04-25T08:42:42.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f0e61600640 1 -- 172.21.15.46:0/1970632323 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500ab8b0 con 0x7f0e48003b50 2024-04-25T08:42:42.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.327+0000 7f5c71400640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c740b0cf0 0x7f5c741624d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5c640097c0 tx=0x7f5c64019260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f469de00640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f46780708d0 0x7f4678072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f7860200640 1 Processor -- start 2024-04-25T08:42:42.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f5c6a000640 1 -- 172.21.15.46:0/3967731305 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640b0040 con 0x7f5c740b0cf0 2024-04-25T08:42:42.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f5c6a000640 1 -- 172.21.15.46:0/3967731305 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c640b9070 con 0x7f5c740b0cf0 2024-04-25T08:42:42.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f5c6a000640 1 -- 172.21.15.46:0/3967731305 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64006e20 con 0x7f5c740b0cf0 2024-04-25T08:42:42.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c740cf6a0 con 0x7f5c740b0cf0 2024-04-25T08:42:42.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c740d19b0 con 0x7f5c740b0cf0 2024-04-25T08:42:42.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f7860200640 1 -- start start 2024-04-25T08:42:42.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f0e69e00640 1 -- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 msgr2=0x7f0e48003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f0e69e00640 1 --2- 172.21.15.46:0/1970632323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 0x7f0e48003f50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000ba30 tx=0x7f0e5001c660 comp rx=0 tx=0).stop 2024-04-25T08:42:42.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f0e63400640 1 -- 172.21.15.46:0/1970632323 reap_dead start 2024-04-25T08:42:42.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f4153400640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4140066f90 0x7f4140069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f4153400640 1 -- 172.21.15.46:0/1705395690 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f415411f740 con 0x7f415803c5e0 2024-04-25T08:42:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f0e69e00640 1 -- 172.21.15.46:0/1970632323 shutdown_connections 2024-04-25T08:42:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f0e69e00640 1 -- 172.21.15.46:0/1970632323 >> 172.21.15.46:0/1970632323 conn(0x7f0e640d0720 msgr2=0x7f0e640d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f4153400640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f4140070680 0x7f4140072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f4153400640 1 -- 172.21.15.46:0/1705395690 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41540b7e80 con 0x7f4140070680 2024-04-25T08:42:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f4153400640 1 -- 172.21.15.46:0/1705395690 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4154126050 con 0x7f415803c5e0 2024-04-25T08:42:42.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7fce4b800640 1 Processor -- start 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f415fe00640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f4140070680 0x7f4140072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7fce4b800640 1 -- start start 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f3024a00640 1 --2- 172.21.15.46:0/839469137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 0x7f3020157280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3018009e50 tx=0x7f301801c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=759d83b299ee29fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f7860200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f785806ac10 0x7f7858163770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f7860200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78580b1350 0x7f78580ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f415fe00640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f4140070680 0x7f4140072ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f469de00640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f46780708d0 0x7f4678072d10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f7860200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7858161ea0 0x7f78581fa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f7860200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580b1eb0 con 0x7f78580b1350 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f7860200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580b1d30 con 0x7f7858161ea0 2024-04-25T08:42:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f469ca00640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4678067160 0x7f4678069620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46981fe450 tx=0x7f468c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f7860200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78580b3b90 con 0x7f785806ac10 2024-04-25T08:42:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f785ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78580b1350 0x7f78580ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.331+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5c74069080 con 0x7f5c740b0cf0 2024-04-25T08:42:42.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f5c6a000640 1 -- 172.21.15.46:0/3967731305 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c640b4670 con 0x7f5c740b0cf0 2024-04-25T08:42:42.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce4b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce440b9460 0x7fce44157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce4b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce440ced80 0x7fce440c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78580b1350 0x7f78580ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60070/0 (socket says 172.21.15.46:60070) 2024-04-25T08:42:42.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785d800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f785806ac10 0x7f7858163770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785ce00640 1 -- 172.21.15.46:0/664618434 learned_addr learned my addr 172.21.15.46:0/664618434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785e200640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7858161ea0 0x7f78581fa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f415ea00640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4140066f90 0x7f4140069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785ce00640 1 -- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f785806ac10 msgr2=0x7f7858163770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785ce00640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f785806ac10 0x7f7858163770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785ce00640 1 -- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7858161ea0 msgr2=0x7f78581fa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785ce00640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7858161ea0 0x7f78581fa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785ce00640 1 -- 172.21.15.46:0/664618434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580cd060 con 0x7f78580b1350 2024-04-25T08:42:42.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f4695600640 1 -- 172.21.15.46:0/1246396416 <== osd.2 v2:172.21.15.46:6808/3102391403 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f46840067e0 con 0x7f46780708d0 2024-04-25T08:42:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f0e69e00640 1 -- 172.21.15.46:0/1970632323 shutdown_connections 2024-04-25T08:42:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce48e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce440b9460 0x7fce44157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f301f400640 1 -- 172.21.15.46:0/839469137 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30180a5750 con 0x7f302006f2e0 2024-04-25T08:42:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f301f400640 1 -- 172.21.15.46:0/839469137 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30180a58f0 con 0x7f302006f2e0 2024-04-25T08:42:42.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f301f400640 1 -- 172.21.15.46:0/839469137 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30180a5cc0 con 0x7f302006f2e0 2024-04-25T08:42:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f415ea00640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4140066f90 0x7f4140069450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41580db6b0 tx=0x7f414c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f5c6a000640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c54066f90 0x7f5c54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f5c6a000640 1 -- 172.21.15.46:0/3967731305 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5c6411ede0 con 0x7f5c740b0cf0 2024-04-25T08:42:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f5c6a000640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f5c54070690 0x7f5c54072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f5c6a000640 1 -- 172.21.15.46:0/3967731305 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c640b6e70 con 0x7f5c54070690 2024-04-25T08:42:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f5c6a000640 1 -- 172.21.15.46:0/3967731305 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c641214c0 con 0x7f5c740b0cf0 2024-04-25T08:42:42.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce48e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce440b9460 0x7fce44157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60078/0 (socket says 172.21.15.46:60078) 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce48e00640 1 -- 172.21.15.46:0/2697378279 learned_addr learned my addr 172.21.15.46:0/2697378279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce43e00640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce440ced80 0x7fce440c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce4b800640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4415db10 0x7fce440b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4416b920 con 0x7fce440b9460 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440d1f80 con 0x7fce440ced80 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4416baa0 con 0x7fce4415db10 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce49800640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4415db10 0x7fce440b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f785ce00640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78580b1350 0x7f78580ce100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f784400c490 tx=0x7f7844001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f7855600640 1 -- 172.21.15.46:0/664618434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844017070 con 0x7f78580b1350 2024-04-25T08:42:42.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f7855600640 1 -- 172.21.15.46:0/664618434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7844003600 con 0x7f78580b1350 2024-04-25T08:42:42.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f7855600640 1 -- 172.21.15.46:0/664618434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844017070 con 0x7f78580b1350 2024-04-25T08:42:42.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f0e69e00640 1 -- 172.21.15.46:0/1970632323 wait complete. 2024-04-25T08:42:42.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78580cd340 con 0x7f78580b1350 2024-04-25T08:42:42.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce48e00640 1 -- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4415db10 msgr2=0x7fce440b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce48e00640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4415db10 0x7fce440b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce48e00640 1 -- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce440ced80 msgr2=0x7fce440c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce48e00640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce440ced80 0x7fce440c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78580cd900 con 0x7f78580b1350 2024-04-25T08:42:42.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce48e00640 1 -- 172.21.15.46:0/2697378279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce441f6ec0 con 0x7fce440b9460 2024-04-25T08:42:42.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f3027400640 1 -- 172.21.15.46:0/839469137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 msgr2=0x7f3020157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f3027400640 1 --2- 172.21.15.46:0/839469137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 0x7f3020157280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3018009e50 tx=0x7f301801c9c0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce48e00640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce440b9460 0x7fce44157f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c450 tx=0x7fce3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f5c72800640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f5c54070690 0x7f5c54072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f5c71e00640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c54066f90 0x7f5c54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f5c71e00640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c54066f90 0x7f5c54069450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c60007fd0 tx=0x7f5c60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f5c72800640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f5c54070690 0x7f5c54072ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce40c00640 1 -- 172.21.15.46:0/2697378279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34012830 con 0x7fce440b9460 2024-04-25T08:42:42.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce441f71a0 con 0x7fce440b9460 2024-04-25T08:42:42.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce441f7760 con 0x7fce440b9460 2024-04-25T08:42:42.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7fce40c00640 1 -- 172.21.15.46:0/2697378279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce34017070 con 0x7fce440b9460 2024-04-25T08:42:42.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.335+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7858069080 con 0x7f78580b1350 2024-04-25T08:42:42.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e69e00640 1 Processor -- start 2024-04-25T08:42:42.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e69e00640 1 -- start start 2024-04-25T08:42:42.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce1c001040 con 0x7fce440b9460 2024-04-25T08:42:42.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7fce40c00640 1 -- 172.21.15.46:0/2697378279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34012830 con 0x7fce440b9460 2024-04-25T08:42:42.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f4153400640 1 -- 172.21.15.46:0/1705395690 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f41540b7e80 con 0x7f4140070680 2024-04-25T08:42:42.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e69e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 0x7f0e64152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e69e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e641e7f00 0x7f0e641e8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e69e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e64159f70 0x7f0e6415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e69e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e64161ae0 con 0x7f0e64159f70 2024-04-25T08:42:42.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e69e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640d1f80 con 0x7f0e48003b50 2024-04-25T08:42:42.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e69e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640b9460 con 0x7f0e641e7f00 2024-04-25T08:42:42.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f3027400640 1 -- 172.21.15.46:0/839469137 shutdown_connections 2024-04-25T08:42:42.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f3027400640 1 --2- 172.21.15.46:0/839469137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200b4640 0x7f30200aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f3027400640 1 --2- 172.21.15.46:0/839469137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 0x7f3020157280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f3027400640 1 -- 172.21.15.46:0/839469137 >> 172.21.15.46:0/839469137 conn(0x7f30200b12a0 msgr2=0x7f30200b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f3027400640 1 -- 172.21.15.46:0/839469137 shutdown_connections 2024-04-25T08:42:42.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e62a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 0x7f0e64152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e62000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e641e7f00 0x7f0e641e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e62a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 0x7f0e64152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50102/0 (socket says 172.21.15.46:50102) 2024-04-25T08:42:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e62a00640 1 -- 172.21.15.46:0/3800803999 learned_addr learned my addr 172.21.15.46:0/3800803999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e62a00640 1 -- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e641e7f00 msgr2=0x7f0e641e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e62a00640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e641e7f00 0x7f0e641e8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e62a00640 1 -- 172.21.15.46:0/3800803999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e64159f70 msgr2=0x7f0e6415a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e62a00640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e64159f70 0x7f0e6415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f0e62a00640 1 -- 172.21.15.46:0/3800803999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640cc0f0 con 0x7f0e48003b50 2024-04-25T08:42:42.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f5c6a000640 1 -- 172.21.15.46:0/3967731305 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5c640b6e70 con 0x7f5c54070690 2024-04-25T08:42:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f3027400640 1 -- 172.21.15.46:0/839469137 wait complete. 2024-04-25T08:42:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7fce40c00640 1 -- 172.21.15.46:0/2697378279 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce340129d0 con 0x7fce440b9460 2024-04-25T08:42:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.339+0000 7f7855600640 1 -- 172.21.15.46:0/664618434 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7844003810 con 0x7f78580b1350 2024-04-25T08:42:42.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f0e62a00640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 0x7f0e64152c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e500ad040 tx=0x7f0e48005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f0e5b400640 1 -- 172.21.15.46:0/3800803999 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500ba070 con 0x7f0e48003b50 2024-04-25T08:42:42.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f0e5b400640 1 -- 172.21.15.46:0/3800803999 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e50004220 con 0x7f0e48003b50 2024-04-25T08:42:42.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f0e5b400640 1 -- 172.21.15.46:0/3800803999 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500ab7c0 con 0x7f0e48003b50 2024-04-25T08:42:42.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e641e8f60 con 0x7f0e48003b50 2024-04-25T08:42:42.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e641e8c80 con 0x7f0e48003b50 2024-04-25T08:42:42.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7fce40c00640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce24067160 0x7fce24069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7fce40c00640 1 -- 172.21.15.46:0/2697378279 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fce340845f0 con 0x7fce440b9460 2024-04-25T08:42:42.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7fce43e00640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce24067160 0x7fce24069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7fce40c00640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7fce240707d0 0x7fce24072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7fce40c00640 1 -- 172.21.15.46:0/2697378279 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce34019e50 con 0x7fce240707d0 2024-04-25T08:42:42.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7fce40c00640 1 -- 172.21.15.46:0/2697378279 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce3408b050 con 0x7fce440b9460 2024-04-25T08:42:42.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0e640cc490 con 0x7f0e48003b50 2024-04-25T08:42:42.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f7855600640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7838067190 0x7f7838069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7fce43e00640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce24067160 0x7fce24069620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce441588b0 tx=0x7fce30007dd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f7855600640 1 -- 172.21.15.46:0/664618434 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7844084480 con 0x7f78580b1350 2024-04-25T08:42:42.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7fce49800640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7fce240707d0 0x7fce24072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f3027400640 1 Processor -- start 2024-04-25T08:42:42.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f7855600640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f7838070910 0x7f7838072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f7855600640 1 -- 172.21.15.46:0/664618434 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7844014ec0 con 0x7f7838070910 2024-04-25T08:42:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f7855600640 1 -- 172.21.15.46:0/664618434 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7844084870 con 0x7f78580b1350 2024-04-25T08:42:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f3027400640 1 -- start start 2024-04-25T08:42:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f785d800640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7838067190 0x7f7838069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.343+0000 7f785e200640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f7838070910 0x7f7838072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7fce49800640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7fce240707d0 0x7fce24072c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e5b400640 1 -- 172.21.15.46:0/3800803999 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e500ab960 con 0x7f0e48003b50 2024-04-25T08:42:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f785d800640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7838067190 0x7f7838069650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ae000 tx=0x7f784c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f785e200640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f7838070910 0x7f7838072d50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e5b400640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e44066fe0 0x7f0e440694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7fce40c00640 1 -- 172.21.15.46:0/2697378279 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fce34019e50 con 0x7fce240707d0 2024-04-25T08:42:42.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3027400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 0x7f302013b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7fd42fe00640 1 -- 172.21.15.46:0/1528608427 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd418002de0 con 0x7fd4240707b0 2024-04-25T08:42:42.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3027400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200ae6d0 0x7f302013ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e5b400640 1 -- 172.21.15.46:0/3800803999 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0e5011fa90 con 0x7f0e48003b50 2024-04-25T08:42:42.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f7855600640 1 -- 172.21.15.46:0/664618434 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7844014ec0 con 0x7f7838070910 2024-04-25T08:42:42.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e5b400640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7f0e44070760 0x7f0e44072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e5b400640 1 -- 172.21.15.46:0/3800803999 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0e500ace90 con 0x7f0e44070760 2024-04-25T08:42:42.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e5b400640 1 -- 172.21.15.46:0/3800803999 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e501242a0 con 0x7f0e48003b50 2024-04-25T08:42:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3027400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30200b4640 0x7f302013d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e62000640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e44066fe0 0x7f0e440694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f301fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200ae6d0 0x7f302013ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3024a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 0x7f302013b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f301fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200ae6d0 0x7f302013ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35926/0 (socket says 172.21.15.46:35926) 2024-04-25T08:42:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e63400640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7f0e44070760 0x7f0e44072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3024a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 0x7f302013b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50108/0 (socket says 172.21.15.46:50108) 2024-04-25T08:42:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3025400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30200b4640 0x7f302013d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30200ce270 con 0x7f30200b4640 2024-04-25T08:42:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f301fe00640 1 -- 172.21.15.46:0/2056526726 learned_addr learned my addr 172.21.15.46:0/2056526726 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30200ce0f0 con 0x7f302006f2e0 2024-04-25T08:42:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30200ce3f0 con 0x7f30200ae6d0 2024-04-25T08:42:42.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e62000640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e44066fe0 0x7f0e440694a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0e540078d0 tx=0x7f0e54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f0e63400640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7f0e44070760 0x7f0e44072ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f301fe00640 1 -- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 msgr2=0x7f302013b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f301fe00640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 0x7f302013b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f301fe00640 1 -- 172.21.15.46:0/2056526726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30200b4640 msgr2=0x7f302013d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f301fe00640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30200b4640 0x7f302013d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f301fe00640 1 -- 172.21.15.46:0/2056526726 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020145f30 con 0x7f30200ae6d0 2024-04-25T08:42:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.347+0000 7f3024a00640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f302006f2e0 0x7f302013b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.351+0000 7f0e5b400640 1 -- 172.21.15.46:0/3800803999 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0e500ace90 con 0x7f0e44070760 2024-04-25T08:42:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.351+0000 7f301fe00640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200ae6d0 0x7f302013ce70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f300800c490 tx=0x7f3008001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.351+0000 7f301cc00640 1 -- 172.21.15.46:0/2056526726 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008017070 con 0x7f30200ae6d0 2024-04-25T08:42:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.351+0000 7f301cc00640 1 -- 172.21.15.46:0/2056526726 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3008003600 con 0x7f30200ae6d0 2024-04-25T08:42:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.351+0000 7f301cc00640 1 -- 172.21.15.46:0/2056526726 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3008003a30 con 0x7f30200ae6d0 2024-04-25T08:42:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.351+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3020146210 con 0x7f30200ae6d0 2024-04-25T08:42:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.351+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30201467d0 con 0x7f30200ae6d0 2024-04-25T08:42:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.351+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3020069080 con 0x7f30200ae6d0 2024-04-25T08:42:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.351+0000 7fd43aa00640 1 -- 172.21.15.46:0/1528608427 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd418002de0 con 0x7fd4240707b0 2024-04-25T08:42:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f301cc00640 1 -- 172.21.15.46:0/2056526726 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f300801a070 con 0x7f30200ae6d0 2024-04-25T08:42:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 -- 172.21.15.46:0/1528608427 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7fd4240707b0 msgr2=0x7fd424072bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7fd4240707b0 0x7fd424072bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 -- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd424067030 msgr2=0x7fd4240694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd424067030 0x7fd4240694f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd43000ba90 tx=0x7fd430008040 comp rx=0 tx=0).stop 2024-04-25T08:42:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 -- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 msgr2=0x7fd43c0e9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 --2- 172.21.15.46:0/1528608427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd43c047c20 0x7fd43c0e9310 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd44406a580 tx=0x7fd444098540 comp rx=0 tx=0).stop 2024-04-25T08:42:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd443200640 1 -- 172.21.15.46:0/1528608427 reap_dead start 2024-04-25T08:42:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 -- 172.21.15.46:0/1528608427 shutdown_connections 2024-04-25T08:42:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 -- 172.21.15.46:0/1528608427 >> 172.21.15.46:0/1528608427 conn(0x7fd43c009fc0 msgr2=0x7fd43c034680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 -- 172.21.15.46:0/1528608427 shutdown_connections 2024-04-25T08:42:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7fd42fe00640 1 -- 172.21.15.46:0/1528608427 wait complete. 2024-04-25T08:42:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f301cc00640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ffc067060 0x7f2ffc069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f301cc00640 1 -- 172.21.15.46:0/2056526726 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3008084620 con 0x7f30200ae6d0 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f301cc00640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f2ffc0707e0 0x7f2ffc072c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f301cc00640 1 -- 172.21.15.46:0/2056526726 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ffc0732f0 con 0x7f2ffc0707e0 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f301cc00640 1 -- 172.21.15.46:0/2056526726 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f300808b050 con 0x7f30200ae6d0 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f3025400640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f2ffc0707e0 0x7f2ffc072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f3024a00640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ffc067060 0x7f2ffc069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f3025400640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f2ffc0707e0 0x7f2ffc072c20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.355+0000 7f3024a00640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ffc067060 0x7f2ffc069520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f30180ae000 tx=0x7f30180a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.359+0000 7f301cc00640 1 -- 172.21.15.46:0/2056526726 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2ffc0732f0 con 0x7f2ffc0707e0 2024-04-25T08:42:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.367+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4158005720 con 0x7f4140070680 2024-04-25T08:42:42.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.367+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f785806b970 con 0x7f7838070910 2024-04-25T08:42:42.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4153400640 1 -- 172.21.15.46:0/1705395690 <== osd.1 v2:172.21.15.46:6809/2552657765 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4158005720 con 0x7f4140070680 2024-04-25T08:42:42.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7855600640 1 -- 172.21.15.46:0/664618434 <== osd.5 v2:172.21.15.82:6816/541310136 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f785806b970 con 0x7f7838070910 2024-04-25T08:42:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f7838070910 msgr2=0x7f7838072d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f7838070910 0x7f7838072d50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7838067190 msgr2=0x7f7838069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7838067190 0x7f7838069650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f784c0ae000 tx=0x7f784c009040 comp rx=0 tx=0).stop 2024-04-25T08:42:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78580b1350 msgr2=0x7f78580ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 --2- 172.21.15.46:0/664618434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78580b1350 0x7f78580ce100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f784400c490 tx=0x7f7844001e80 comp rx=0 tx=0).stop 2024-04-25T08:42:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f785e200640 1 -- 172.21.15.46:0/664618434 reap_dead start 2024-04-25T08:42:42.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f4140070680 msgr2=0x7f4140072ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4166600640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f4140070680 0x7f4140072ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4698071e30 con 0x7f46780708d0 2024-04-25T08:42:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4140066f90 msgr2=0x7f4140069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4166600640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4140066f90 0x7f4140069450 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f41580db6b0 tx=0x7f414c008040 comp rx=0 tx=0).stop 2024-04-25T08:42:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 msgr2=0x7f415804e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4166600640 1 --2- 172.21.15.46:0/1705395690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f415803c5e0 0x7f415804e9b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4154000c10 tx=0x7f4154018ff0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f415fe00640 1 -- 172.21.15.46:0/1705395690 reap_dead start 2024-04-25T08:42:42.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4695600640 1 -- 172.21.15.46:0/1246396416 <== osd.2 v2:172.21.15.46:6808/3102391403 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4698071e30 con 0x7f46780708d0 2024-04-25T08:42:42.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5c740701e0 con 0x7f5c54070690 2024-04-25T08:42:42.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 shutdown_connections 2024-04-25T08:42:42.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 >> 172.21.15.46:0/664618434 conn(0x7f7858072920 msgr2=0x7f785806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 shutdown_connections 2024-04-25T08:42:42.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 >> 172.21.15.46:0/1705395690 conn(0x7f4158058f00 msgr2=0x7f4158052690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 shutdown_connections 2024-04-25T08:42:42.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f7860200640 1 -- 172.21.15.46:0/664618434 wait complete. 2024-04-25T08:42:42.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.371+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 shutdown_connections 2024-04-25T08:42:42.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f4166600640 1 -- 172.21.15.46:0/1705395690 wait complete. 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f46780708d0 msgr2=0x7f4678072d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f46780708d0 0x7f4678072d10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4678067160 msgr2=0x7f4678069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4678067160 0x7f4678069620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46981fe450 tx=0x7f468c009040 comp rx=0 tx=0).stop 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4698071750 msgr2=0x7f4698157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 --2- 172.21.15.46:0/1246396416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4698071750 0x7f4698157380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f468400b3c0 tx=0x7f468400b880 comp rx=0 tx=0).stop 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469de00640 1 -- 172.21.15.46:0/1246396416 reap_dead start 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 shutdown_connections 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 >> 172.21.15.46:0/1246396416 conn(0x7f4698066fa0 msgr2=0x7f4698071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 shutdown_connections 2024-04-25T08:42:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f469e800640 1 -- 172.21.15.46:0/1246396416 wait complete. 2024-04-25T08:42:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f5c6a000640 1 -- 172.21.15.46:0/3967731305 <== osd.3 v2:172.21.15.46:6824/1647449001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5c740701e0 con 0x7f5c54070690 2024-04-25T08:42:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f5c54070690 msgr2=0x7f5c54072ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7f5c79200640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f5c54070690 0x7f5c54072ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.375+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce1c002de0 con 0x7fce240707d0 2024-04-25T08:42:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c54066f90 msgr2=0x7f5c54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7f5c79200640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c54066f90 0x7f5c54069450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5c60007fd0 tx=0x7f5c60009040 comp rx=0 tx=0).stop 2024-04-25T08:42:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c740b0cf0 msgr2=0x7f5c741624d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7f5c79200640 1 --2- 172.21.15.46:0/3967731305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c740b0cf0 0x7f5c741624d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5c640097c0 tx=0x7f5c64019260 comp rx=0 tx=0).stop 2024-04-25T08:42:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7f5c72800640 1 -- 172.21.15.46:0/3967731305 reap_dead start 2024-04-25T08:42:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 shutdown_connections 2024-04-25T08:42:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 >> 172.21.15.46:0/3967731305 conn(0x7f5c7406f510 msgr2=0x7f5c740b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 shutdown_connections 2024-04-25T08:42:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7f5c79200640 1 -- 172.21.15.46:0/3967731305 wait complete. 2024-04-25T08:42:42.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce40c00640 1 -- 172.21.15.46:0/2697378279 <== osd.4 v2:172.21.15.82:6800/2293006647 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fce1c002de0 con 0x7fce240707d0 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7fce240707d0 msgr2=0x7fce24072c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce4b800640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7fce240707d0 0x7fce24072c10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce24067160 msgr2=0x7fce24069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce4b800640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce24067160 0x7fce24069620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fce441588b0 tx=0x7fce30007dd0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce440b9460 msgr2=0x7fce44157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce4b800640 1 --2- 172.21.15.46:0/2697378279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce440b9460 0x7fce44157f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fce3400c450 tx=0x7fce3400c910 comp rx=0 tx=0).stop 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce49800640 1 -- 172.21.15.46:0/2697378279 reap_dead start 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 shutdown_connections 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 >> 172.21.15.46:0/2697378279 conn(0x7fce440d0720 msgr2=0x7fce4406c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.379+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 shutdown_connections 2024-04-25T08:42:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.383+0000 7fce4b800640 1 -- 172.21.15.46:0/2697378279 wait complete. 2024-04-25T08:42:42.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.387+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0e64069080 con 0x7f0e44070760 2024-04-25T08:42:42.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.387+0000 7f0e5b400640 1 -- 172.21.15.46:0/3800803999 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0e64069080 con 0x7f0e44070760 2024-04-25T08:42:42.399 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:42.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T08:42:42.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.387+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7f0e44070760 msgr2=0x7f0e44072ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.387+0000 7f0e69e00640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7f0e44070760 0x7f0e44072ba0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e44066fe0 msgr2=0x7f0e440694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f0e69e00640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e44066fe0 0x7f0e440694a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0e540078d0 tx=0x7f0e54009040 comp rx=0 tx=0).stop 2024-04-25T08:42:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 msgr2=0x7f0e64152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f0e69e00640 1 --2- 172.21.15.46:0/3800803999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e48003b50 0x7f0e64152c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e500ad040 tx=0x7f0e48005840 comp rx=0 tx=0).stop 2024-04-25T08:42:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.387+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f302013edd0 con 0x7f2ffc0707e0 2024-04-25T08:42:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f0e63400640 1 -- 172.21.15.46:0/3800803999 reap_dead start 2024-04-25T08:42:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 shutdown_connections 2024-04-25T08:42:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 >> 172.21.15.46:0/3800803999 conn(0x7f0e640d0720 msgr2=0x7f0e640b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f301cc00640 1 -- 172.21.15.46:0/2056526726 <== osd.7 v2:172.21.15.82:6808/1420997799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f302013edd0 con 0x7f2ffc0707e0 2024-04-25T08:42:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f2ffc0707e0 msgr2=0x7f2ffc072c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f2ffc0707e0 0x7f2ffc072c20 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 shutdown_connections 2024-04-25T08:42:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f0e69e00640 1 -- 172.21.15.46:0/3800803999 wait complete. 2024-04-25T08:42:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ffc067060 msgr2=0x7f2ffc069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ffc067060 0x7f2ffc069520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f30180ae000 tx=0x7f30180a7070 comp rx=0 tx=0).stop 2024-04-25T08:42:42.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200ae6d0 msgr2=0x7f302013ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 --2- 172.21.15.46:0/2056526726 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30200ae6d0 0x7f302013ce70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f300800c490 tx=0x7f3008001e80 comp rx=0 tx=0).stop 2024-04-25T08:42:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3025400640 1 -- 172.21.15.46:0/2056526726 reap_dead start 2024-04-25T08:42:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 shutdown_connections 2024-04-25T08:42:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 >> 172.21.15.46:0/2056526726 conn(0x7f30200b12a0 msgr2=0x7f302015c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 shutdown_connections 2024-04-25T08:42:42.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.391+0000 7f3027400640 1 -- 172.21.15.46:0/2056526726 wait complete. 2024-04-25T08:42:42.404 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-25T08:42:42.405 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T08:42:42.408 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-25T08:42:42.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T08:42:42.409 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:42.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T08:42:42.412 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:42.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T08:42:42.421 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:42.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T08:42:42.422 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-25T08:42:42.422 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T08:42:42.423 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-25T08:42:42.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T08:42:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.631+0000 7f94c9000640 1 Processor -- start 2024-04-25T08:42:42.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.631+0000 7f94c9000640 1 -- start start 2024-04-25T08:42:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c4140820 0x7f94c4161de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c9000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94c4071090 0x7f94c4071510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c4162320 con 0x7f94c4142fa0 2024-04-25T08:42:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c9000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c40cee70 con 0x7f94c4071090 2024-04-25T08:42:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4140c00 con 0x7f94c4140820 2024-04-25T08:42:42.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f94c4142fa0 0x7f94c41433e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59100/0 (socket says 172.21.15.46:59100) 2024-04-25T08:42:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c2800640 1 -- 172.21.15.46:0/3093915580 learned_addr learned my addr 172.21.15.46:0/3093915580 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759845240 0 0) 0x7f94bc0014a0 con 0x7f94c4142fa0 2024-04-25T08:42:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a8003530 con 0x7f94c4142fa0 2024-04-25T08:42:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2587090248 0 0) 0x7f94a8003530 con 0x7f94c4142fa0 2024-04-25T08:42:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c4140820 msgr2=0x7f94c4161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c0a00640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c4140820 0x7f94c4161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 >> v2:172.21.15.82:3300/0 conn(0x7f94c4071090 msgr2=0x7f94c4071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c0a00640 1 --2- 172.21.15.46:0/3093915580 >> v2:172.21.15.82:3300/0 conn(0x7f94c4071090 0x7f94c4071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.635+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c415e180 con 0x7f94c4142fa0 2024-04-25T08:42:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f94bc001bb0 con 0x7f94c4142fa0 2024-04-25T08:42:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 >> v1:172.21.15.46:6789/0 conn(0x7f94c4142fa0 legacy=0x7f94c41433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c0a00640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c4140820 0x7f94a8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c0a00640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a8004110 0x7f94a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c0a00640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a8004da0 0x7f94a8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c40cee70 con 0x7f94a8004110 2024-04-25T08:42:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4140c00 con 0x7f94a8004da0 2024-04-25T08:42:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a800bd30 con 0x7f94c4140820 2024-04-25T08:42:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c1400640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a8004110 0x7f94a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c1e00640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c4140820 0x7f94a8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c2800640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a8004da0 0x7f94a8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c1e00640 1 -- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a8004da0 msgr2=0x7f94a8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c1e00640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a8004da0 0x7f94a8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c1e00640 1 -- 172.21.15.46:0/3093915580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a8004110 msgr2=0x7f94a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c1e00640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a8004110 0x7f94a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c1e00640 1 -- 172.21.15.46:0/3093915580 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c415e180 con 0x7f94c4140820 2024-04-25T08:42:42.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.639+0000 7f94c2800640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94a8004da0 0x7f94a8005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.643+0000 7f94c1400640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a8004110 0x7f94a8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.643+0000 7f94c1e00640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c4140820 0x7f94a8003a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009860 tx=0x7f94b001c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110ab67568773199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.643+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b00b0070 con 0x7f94c4140820 2024-04-25T08:42:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.643+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b00ab620 con 0x7f94c4140820 2024-04-25T08:42:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.647+0000 7f94c0a00640 1 -- 172.21.15.46:0/3093915580 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0009460 con 0x7f94c4140820 2024-04-25T08:42:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.647+0000 7f94c9000640 1 -- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c4140820 msgr2=0x7f94a8003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.647+0000 7f94c9000640 1 --2- 172.21.15.46:0/3093915580 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c4140820 0x7f94a8003a00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009860 tx=0x7f94b001c380 comp rx=0 tx=0).stop 2024-04-25T08:42:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.647+0000 7f94c2800640 1 -- 172.21.15.46:0/3093915580 reap_dead start 2024-04-25T08:42:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.647+0000 7f94c9000640 1 -- 172.21.15.46:0/3093915580 shutdown_connections 2024-04-25T08:42:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.647+0000 7f94c9000640 1 -- 172.21.15.46:0/3093915580 >> 172.21.15.46:0/3093915580 conn(0x7f94c40b1130 msgr2=0x7f94c40b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.651+0000 7fc77d600640 1 Processor -- start 2024-04-25T08:42:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.651+0000 7fc77d600640 1 -- start start 2024-04-25T08:42:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.651+0000 7f94c9000640 1 -- 172.21.15.46:0/3093915580 shutdown_connections 2024-04-25T08:42:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.651+0000 7f94c9000640 1 -- 172.21.15.46:0/3093915580 wait complete. 2024-04-25T08:42:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.651+0000 7f64bf600640 1 Processor -- start 2024-04-25T08:42:42.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bf600640 1 -- start start 2024-04-25T08:42:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fc77d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc7781579b0 0x7fc77815a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fc77d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77806d0a0 0x7fc7780cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fc77d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc778157d90 con 0x7fc778071040 2024-04-25T08:42:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fc77d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7780b32b0 con 0x7fc7781579b0 2024-04-25T08:42:42.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fc77d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780ce040 con 0x7fc77806d0a0 2024-04-25T08:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bf600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64b81654c0 0x7f64b8163400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bf600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64b8162330 0x7f64b806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bf600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b806d270 0x7f64b806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bf600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b806e9d0 con 0x7f64b81654c0 2024-04-25T08:42:42.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bf600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b806ec70 con 0x7f64b8162330 2024-04-25T08:42:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bf600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b806ef50 con 0x7f64b806d270 2024-04-25T08:42:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bcc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64b81654c0 0x7f64b8163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bcc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64b81654c0 0x7f64b8163400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60114/0 (socket says 172.21.15.46:60114) 2024-04-25T08:42:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bcc00640 1 -- 172.21.15.46:0/1636284065 learned_addr learned my addr 172.21.15.46:0/1636284065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f94c9000640 1 Processor -- start 2024-04-25T08:42:42.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f94c9000640 1 -- start start 2024-04-25T08:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64b7e00640 1 --2- 172.21.15.46:0/1636284065 >> v2:172.21.15.82:3300/0 conn(0x7f64b8162330 0x7f64b806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7f64bd600640 1 --2- 172.21.15.46:0/1636284065 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b806d270 0x7f64b806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fc77cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc778071040 0x7fc7780b3a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59104/0 (socket says 172.21.15.46:59104) 2024-04-25T08:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fc77cc00640 1 -- 172.21.15.46:0/1882763344 learned_addr learned my addr 172.21.15.46:0/1882763344 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fc777e00640 1 --2- 172.21.15.46:0/1882763344 >> v2:172.21.15.46:3300/0 conn(0x7fc7781579b0 0x7fc77815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fb47b200640 1 Processor -- start 2024-04-25T08:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.655+0000 7fb47b200640 1 -- start start 2024-04-25T08:42:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7fc777400640 1 --2- 172.21.15.46:0/1882763344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77806d0a0 0x7fc7780cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f64bcc00640 1 -- 172.21.15.46:0/1636284065 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b806d270 msgr2=0x7f64b806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f64bcc00640 1 --2- 172.21.15.46:0/1636284065 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b806d270 0x7f64b806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f64bcc00640 1 -- 172.21.15.46:0/1636284065 >> v2:172.21.15.82:3300/0 conn(0x7f64b8162330 msgr2=0x7f64b806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f64bcc00640 1 --2- 172.21.15.46:0/1636284065 >> v2:172.21.15.82:3300/0 conn(0x7f64b8162330 0x7f64b806e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c4202e20 0x7f94c4203200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c41550a0 0x7f94c41610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c41598a0 0x7f94c4159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f64bcc00640 1 -- 172.21.15.46:0/1636284065 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b806f230 con 0x7f64b81654c0 2024-04-25T08:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7fc777400640 1 -- 172.21.15.46:0/1882763344 >> v2:172.21.15.46:3300/0 conn(0x7fc7781579b0 msgr2=0x7fc77815a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7fc777400640 1 --2- 172.21.15.46:0/1882763344 >> v2:172.21.15.46:3300/0 conn(0x7fc7781579b0 0x7fc77815a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7fc777400640 1 -- 172.21.15.46:0/1882763344 >> v1:172.21.15.46:6789/0 conn(0x7fc778071040 legacy=0x7fc7780b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7fc777400640 1 -- 172.21.15.46:0/1882763344 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7780ce2e0 con 0x7fc77806d0a0 2024-04-25T08:42:42.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c415dde0 con 0x7f94c4202e20 2024-04-25T08:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f64bd600640 1 --2- 172.21.15.46:0/1636284065 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b806d270 0x7f64b806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f64b7e00640 1 --2- 172.21.15.46:0/1636284065 >> v2:172.21.15.82:3300/0 conn(0x7f64b8162330 0x7f64b806e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7fb47b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4740e3450 0x7fb474053760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c41550a0 0x7f94c41610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c9000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c415ed00 con 0x7f94c41550a0 2024-04-25T08:42:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c415df60 con 0x7f94c41598a0 2024-04-25T08:42:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c41550a0 0x7f94c41610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50128/0 (socket says 172.21.15.46:50128) 2024-04-25T08:42:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c1400640 1 -- 172.21.15.46:0/2869152550 learned_addr learned my addr 172.21.15.46:0/2869152550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7fb479e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb4740e3070 0x7fb474057290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:35500/0 (socket says 172.21.15.46:35500) 2024-04-25T08:42:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c2800640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c41598a0 0x7f94c4159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c1e00640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c4202e20 0x7f94c4203200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7fb479e00640 1 -- 172.21.15.46:0/2740968045 learned_addr learned my addr 172.21.15.46:0/2740968045 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7fb479400640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4740e3450 0x7fb474053760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c2800640 1 -- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c41550a0 msgr2=0x7f94c41610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c2800640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c41550a0 0x7f94c41610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c2800640 1 -- 172.21.15.46:0/2869152550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c4202e20 msgr2=0x7f94c4203200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c2800640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c4202e20 0x7f94c4203200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.659+0000 7f94c2800640 1 -- 172.21.15.46:0/2869152550 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c41615f0 con 0x7f94c41598a0 2024-04-25T08:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f94c2800640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c41598a0 0x7f94c4159d20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94c40b36e0 tx=0x7f94bc00a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb47b200640 1 -- 172.21.15.46:0/2740968045 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb474058d10 con 0x7fb474053ca0 2024-04-25T08:42:42.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb47b200640 1 -- 172.21.15.46:0/2740968045 --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4740579a0 con 0x7fb4740e3070 2024-04-25T08:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f94ba000640 1 -- 172.21.15.46:0/2869152550 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4009460 con 0x7f94c41598a0 2024-04-25T08:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f94ba000640 1 -- 172.21.15.46:0/2869152550 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc018070 con 0x7f94c41598a0 2024-04-25T08:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb47b200640 1 -- 172.21.15.46:0/2740968045 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740540d0 con 0x7fb4740e3450 2024-04-25T08:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f94ba000640 1 -- 172.21.15.46:0/2869152550 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc012790 con 0x7f94c41598a0 2024-04-25T08:42:42.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f94c1400640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c41550a0 0x7f94c41610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241878729 0 0) 0x7fb4740579a0 con 0x7fb4740e3070 2024-04-25T08:42:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb460003530 con 0x7fb4740e3070 2024-04-25T08:42:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 <== mon.1 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1286624940 0 0) 0x7fb460003530 con 0x7fb4740e3070 2024-04-25T08:42:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4740e3450 msgr2=0x7fb474053760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4740e3450 0x7fb474053760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 >> v1:172.21.15.46:6789/0 conn(0x7fb474053ca0 legacy=0x7fb47403c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c4155f90 con 0x7f94c41598a0 2024-04-25T08:42:42.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f64bcc00640 1 --2- 172.21.15.46:0/1636284065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b81654c0 0x7f64b8163400 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f64b80cf440 tx=0x7f64a001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a97a5bba9b8214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c41559d0 con 0x7f94c41598a0 2024-04-25T08:42:42.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f64b7400640 1 -- 172.21.15.46:0/1636284065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a00b0070 con 0x7f64b81654c0 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f64b7400640 1 -- 172.21.15.46:0/1636284065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a00ab600 con 0x7f64b81654c0 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7f64b7400640 1 -- 172.21.15.46:0/1636284065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a00ab9d0 con 0x7f64b81654c0 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb474057c70 con 0x7fb4740e3070 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 <== mon.1 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fb46c001680 con 0x7fb4740e3070 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 >> v1:172.21.15.82:6789/0 conn(0x7fb4740e3070 legacy=0x7fb474057290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4600039c0 0x7fb460003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4740e3450 0x7fb460004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460004d50 0x7fb4600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740540d0 con 0x7fb460004d50 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600142a0 con 0x7fb4600039c0 2024-04-25T08:42:42.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600144a0 con 0x7fb4740e3450 2024-04-25T08:42:42.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb479e00640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4600039c0 0x7fb460003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb479400640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4740e3450 0x7fb460004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb47a800640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460004d50 0x7fb4600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fc777400640 1 --2- 172.21.15.46:0/1882763344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77806d0a0 0x7fc7780cdb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc76400a960 tx=0x7fc76401d7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2712b99260600594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb47a800640 1 -- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4740e3450 msgr2=0x7fb460004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb47a800640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb4740e3450 0x7fb460004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb47a800640 1 -- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4600039c0 msgr2=0x7fb460003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb47a800640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4600039c0 0x7fb460003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb47a800640 1 -- 172.21.15.46:0/2740968045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb474057c70 con 0x7fb460004d50 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.663+0000 7fb479e00640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4600039c0 0x7fb460003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc776a00640 1 -- 172.21.15.46:0/1882763344 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640b3070 con 0x7fc77806d0a0 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc776a00640 1 -- 172.21.15.46:0/1882763344 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7640a6780 con 0x7fc77806d0a0 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc776a00640 1 -- 172.21.15.46:0/1882763344 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76400b040 con 0x7fc77806d0a0 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f64bf600640 1 -- 172.21.15.46:0/1636284065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b81654c0 msgr2=0x7f64b8163400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f64bf600640 1 --2- 172.21.15.46:0/1636284065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b81654c0 0x7f64b8163400 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f64b80cf440 tx=0x7f64a001c560 comp rx=0 tx=0).stop 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fcdf3e00640 1 Processor -- start 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f64bf600640 1 -- 172.21.15.46:0/1636284065 shutdown_connections 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f64bf600640 1 --2- 172.21.15.46:0/1636284065 >> v2:172.21.15.82:3300/0 conn(0x7f64b8162330 0x7f64b806e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f64bf600640 1 --2- 172.21.15.46:0/1636284065 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b806d270 0x7f64b806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f64bf600640 1 --2- 172.21.15.46:0/1636284065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b81654c0 0x7f64b8163400 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f64bf600640 1 -- 172.21.15.46:0/1636284065 >> 172.21.15.46:0/1636284065 conn(0x7f64b80b12a0 msgr2=0x7f64b80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f64bf600640 1 -- 172.21.15.46:0/1636284065 shutdown_connections 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f64bf600640 1 -- 172.21.15.46:0/1636284065 wait complete. 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc77d600640 1 -- 172.21.15.46:0/1882763344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77806d0a0 msgr2=0x7fc7780cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc77d600640 1 --2- 172.21.15.46:0/1882763344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77806d0a0 0x7fc7780cdb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc76400a960 tx=0x7fc76401d7e0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc77d600640 1 -- 172.21.15.46:0/1882763344 shutdown_connections 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc77d600640 1 --2- 172.21.15.46:0/1882763344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc77806d0a0 0x7fc7780cdb00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc77d600640 1 --2- 172.21.15.46:0/1882763344 >> v2:172.21.15.46:3300/0 conn(0x7fc7781579b0 0x7fc77815a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc77d600640 1 -- 172.21.15.46:0/1882763344 >> 172.21.15.46:0/1882763344 conn(0x7fc778066fa0 msgr2=0x7fc77806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fcdf3e00640 1 -- start start 2024-04-25T08:42:42.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c4069010 con 0x7f94c41598a0 2024-04-25T08:42:42.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc77d600640 1 -- 172.21.15.46:0/1882763344 shutdown_connections 2024-04-25T08:42:42.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fc77d600640 1 -- 172.21.15.46:0/1882763344 wait complete. 2024-04-25T08:42:42.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fcdf3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcdf4066f80 0x7fcdf4159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fcdf3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4150820 0x7fcdf40ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fcdf3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf415a560 con 0x7fcdf406d630 2024-04-25T08:42:42.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fcdf3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf415d6a0 con 0x7fcdf4066f80 2024-04-25T08:42:42.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7fcdf3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf415a840 con 0x7fcdf4150820 2024-04-25T08:42:42.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.667+0000 7f94ba000640 1 -- 172.21.15.46:0/2869152550 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94bc012930 con 0x7f94c41598a0 2024-04-25T08:42:42.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fcdf2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4150820 0x7fcdf40ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fcdf2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4150820 0x7fcdf40ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35980/0 (socket says 172.21.15.46:35980) 2024-04-25T08:42:42.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fcdf2000640 1 -- 172.21.15.46:0/2326002283 learned_addr learned my addr 172.21.15.46:0/2326002283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7f94ba000640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a4066f60 0x7f94a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fcdf2a00640 1 --2- 172.21.15.46:0/2326002283 >> v2:172.21.15.46:3300/0 conn(0x7fcdf4066f80 0x7fcdf4159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7f64bf600640 1 Processor -- start 2024-04-25T08:42:42.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fcdf2000640 1 -- 172.21.15.46:0/2326002283 >> v2:172.21.15.46:3300/0 conn(0x7fcdf4066f80 msgr2=0x7fcdf4159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fcdf2000640 1 --2- 172.21.15.46:0/2326002283 >> v2:172.21.15.46:3300/0 conn(0x7fcdf4066f80 0x7fcdf4159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc77d600640 1 Processor -- start 2024-04-25T08:42:42.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fcdf2000640 1 -- 172.21.15.46:0/2326002283 >> v1:172.21.15.46:6789/0 conn(0x7fcdf406d630 legacy=0x7fcdf415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fcdf2000640 1 -- 172.21.15.46:0/2326002283 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf415ab20 con 0x7fcdf4150820 2024-04-25T08:42:42.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7f94ba000640 1 -- 172.21.15.46:0/2869152550 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f94bc085c00 con 0x7f94c41598a0 2024-04-25T08:42:42.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7f94c1e00640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a4066f60 0x7f94a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc77d600640 1 -- start start 2024-04-25T08:42:42.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7f64bf600640 1 -- start start 2024-04-25T08:42:42.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7f94c1e00640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a4066f60 0x7f94a4069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f94b00ad000 tx=0x7f94b0004850 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fb47a800640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460004d50 0x7fb4600051d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb47c073560 tx=0x7fb47c097240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30ae4bb94560c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc77d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc77806d0a0 0x7fc778142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c04fec0 con 0x7fb460004d50 2024-04-25T08:42:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc77d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc778071040 0x7fc778150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc77d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7781579b0 0x7fc77814e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc77d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780718a0 con 0x7fc77806d0a0 2024-04-25T08:42:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc77d600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77806d500 con 0x7fc7781579b0 2024-04-25T08:42:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc77d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780b4590 con 0x7fc778071040 2024-04-25T08:42:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c126660 con 0x7fb460004d50 2024-04-25T08:42:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7f64bf600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b806d270 0x7f64b815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7f64bf600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b8162330 0x7f64b81599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc777e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc77806d0a0 0x7fc778142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fb478a00640 1 -- 172.21.15.46:0/2740968045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c126b10 con 0x7fb460004d50 2024-04-25T08:42:42.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7f64bcc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b806d270 0x7f64b815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bcc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b806d270 0x7f64b815b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50158/0 (socket says 172.21.15.46:50158) 2024-04-25T08:42:42.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bcc00640 1 -- 172.21.15.46:0/1199260494 learned_addr learned my addr 172.21.15.46:0/1199260494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc777e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc77806d0a0 0x7fc778142c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60148/0 (socket says 172.21.15.46:60148) 2024-04-25T08:42:42.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc777e00640 1 -- 172.21.15.46:0/1773903847 learned_addr learned my addr 172.21.15.46:0/1773903847 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc777400640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc778071040 0x7fc778150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.671+0000 7fc77cc00640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7781579b0 0x7fc77814e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bf600640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b8168450 0x7f64b8168900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80ce610 con 0x7f64b8162330 2024-04-25T08:42:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80ce490 con 0x7f64b806d270 2024-04-25T08:42:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80ce790 con 0x7f64b8168450 2024-04-25T08:42:42.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64b7e00640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b8162330 0x7f64b81599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc777e00640 1 -- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc778071040 msgr2=0x7fc778150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc777e00640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc778071040 0x7fc778150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc777e00640 1 -- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7781579b0 msgr2=0x7fc77814e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc777e00640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7781579b0 0x7fc77814e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc777e00640 1 -- 172.21.15.46:0/1773903847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778143160 con 0x7fc77806d0a0 2024-04-25T08:42:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bd600640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b8168450 0x7f64b8168900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f94ba000640 1 -- 172.21.15.46:0/2869152550 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94c41615f0 con 0x7f94c41598a0 2024-04-25T08:42:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bcc00640 1 -- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b8168450 msgr2=0x7f64b8168900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bcc00640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b8168450 0x7f64b8168900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bcc00640 1 -- 172.21.15.46:0/1199260494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b8162330 msgr2=0x7f64b81599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bcc00640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b8162330 0x7f64b81599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bcc00640 1 -- 172.21.15.46:0/1199260494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8159f30 con 0x7f64b806d270 2024-04-25T08:42:42.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc777e00640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc77806d0a0 0x7fc778142c20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc76800e870 tx=0x7fc76800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fb47b200640 1 -- 172.21.15.46:0/2740968045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460004d50 msgr2=0x7fb4600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fb47b200640 1 --2- 172.21.15.46:0/2740968045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb460004d50 0x7fb4600051d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb47c073560 tx=0x7fb47c097240 comp rx=0 tx=0).stop 2024-04-25T08:42:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc76fe00640 1 -- 172.21.15.46:0/1773903847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768017070 con 0x7fc77806d0a0 2024-04-25T08:42:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc76fe00640 1 -- 172.21.15.46:0/1773903847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76800c940 con 0x7fc77806d0a0 2024-04-25T08:42:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc76fe00640 1 -- 172.21.15.46:0/1773903847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76800ccf0 con 0x7fc77806d0a0 2024-04-25T08:42:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bcc00640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b806d270 0x7f64b815b660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f64a001ac10 tx=0x7f64a0019300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fb47a800640 1 -- 172.21.15.46:0/2740968045 reap_dead start 2024-04-25T08:42:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fb47b200640 1 -- 172.21.15.46:0/2740968045 shutdown_connections 2024-04-25T08:42:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fb47b200640 1 -- 172.21.15.46:0/2740968045 >> 172.21.15.46:0/2740968045 conn(0x7fb474058f00 msgr2=0x7fb474059310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fb47b200640 1 -- 172.21.15.46:0/2740968045 shutdown_connections 2024-04-25T08:42:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fcdf2000640 1 --2- 172.21.15.46:0/2326002283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4150820 0x7fcdf40ae820 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009870 tx=0x7fcde401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aede59905420680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fcdf1600640 1 -- 172.21.15.46:0/2326002283 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde40b5070 con 0x7fcdf4150820 2024-04-25T08:42:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64b4c00640 1 -- 172.21.15.46:0/1199260494 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a00b0050 con 0x7f64b806d270 2024-04-25T08:42:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64b4c00640 1 -- 172.21.15.46:0/1199260494 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a00ba070 con 0x7f64b806d270 2024-04-25T08:42:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64b4c00640 1 -- 172.21.15.46:0/1199260494 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a00b47d0 con 0x7f64b806d270 2024-04-25T08:42:42.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fb47b200640 1 -- 172.21.15.46:0/2740968045 wait complete. 2024-04-25T08:42:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fcdf1600640 1 -- 172.21.15.46:0/2326002283 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde40a8a50 con 0x7fcdf4150820 2024-04-25T08:42:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fcdf3e00640 1 -- 172.21.15.46:0/2326002283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4150820 msgr2=0x7fcdf40ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fcdf3e00640 1 --2- 172.21.15.46:0/2326002283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4150820 0x7fcdf40ae820 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009870 tx=0x7fcde401ed50 comp rx=0 tx=0).stop 2024-04-25T08:42:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b80aef90 con 0x7f64b806d270 2024-04-25T08:42:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b816b430 con 0x7f64b806d270 2024-04-25T08:42:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fcdf3e00640 1 -- 172.21.15.46:0/2326002283 shutdown_connections 2024-04-25T08:42:42.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fcdf3e00640 1 --2- 172.21.15.46:0/2326002283 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4150820 0x7fcdf40ae820 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fcdf3e00640 1 --2- 172.21.15.46:0/2326002283 >> v2:172.21.15.46:3300/0 conn(0x7fcdf4066f80 0x7fcdf4159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fcdf3e00640 1 -- 172.21.15.46:0/2326002283 >> 172.21.15.46:0/2326002283 conn(0x7fcdf4066b70 msgr2=0x7fcdf406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7781556b0 con 0x7fc77806d0a0 2024-04-25T08:42:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.675+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc778155c70 con 0x7fc77806d0a0 2024-04-25T08:42:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77815b400 con 0x7fc77806d0a0 2024-04-25T08:42:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fcdf3e00640 1 -- 172.21.15.46:0/2326002283 shutdown_connections 2024-04-25T08:42:42.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fb47b200640 1 Processor -- start 2024-04-25T08:42:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b8069080 con 0x7f64b806d270 2024-04-25T08:42:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7fcdf3e00640 1 -- 172.21.15.46:0/2326002283 wait complete. 2024-04-25T08:42:42.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.679+0000 7f64b4c00640 1 -- 172.21.15.46:0/1199260494 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64a00b4970 con 0x7f64b806d270 2024-04-25T08:42:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3e00640 1 Processor -- start 2024-04-25T08:42:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3e00640 1 -- start start 2024-04-25T08:42:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fc76fe00640 1 -- 172.21.15.46:0/1773903847 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc76801a070 con 0x7fc77806d0a0 2024-04-25T08:42:42.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7f64b4c00640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6494067060 0x7f6494069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7f64b4c00640 1 -- 172.21.15.46:0/1199260494 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f64a011f190 con 0x7f64b806d270 2024-04-25T08:42:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4066f80 0x7fcdf4140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf406d630 0x7fcdf4143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdf4150820 0x7fcdf40b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4150c80 con 0x7fcdf406d630 2024-04-25T08:42:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40aed60 con 0x7fcdf4150820 2024-04-25T08:42:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4150e00 con 0x7fcdf4066f80 2024-04-25T08:42:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fc76fe00640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc758066f60 0x7fc758069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fc76fe00640 1 -- 172.21.15.46:0/1773903847 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc7680858d0 con 0x7fc77806d0a0 2024-04-25T08:42:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdf4150820 0x7fcdf40b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fb47b200640 1 -- start start 2024-04-25T08:42:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fb47b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb47403e490 0x7fb47404ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fb47b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb47403e870 0x7fb4740e1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fb47b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4740decc0 0x7fb4740d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fb47b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb474058510 con 0x7fb47403e870 2024-04-25T08:42:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fb47b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb474058230 con 0x7fb4740decc0 2024-04-25T08:42:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fb47b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4740587f0 con 0x7fb47403e490 2024-04-25T08:42:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf406d630 0x7fcdf4143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdf4150820 0x7fcdf40b0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50170/0 (socket says 172.21.15.46:50170) 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3400640 1 -- 172.21.15.46:0/1968264627 learned_addr learned my addr 172.21.15.46:0/1968264627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf2a00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4066f80 0x7fcdf4140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3400640 1 -- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4066f80 msgr2=0x7fcdf4140670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3400640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4066f80 0x7fcdf4140670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3400640 1 -- 172.21.15.46:0/1968264627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf406d630 msgr2=0x7fcdf4143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3400640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf406d630 0x7fcdf4143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.683+0000 7fcdf3400640 1 -- 172.21.15.46:0/1968264627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4203920 con 0x7fcdf4150820 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fcdf2a00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4066f80 0x7fcdf4140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fc777400640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc758066f60 0x7fc758069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fcdf3400640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdf4150820 0x7fcdf40b0190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00c710 tx=0x7fcdec00cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb479400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb47403e870 0x7fb4740e1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb479400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb47403e870 0x7fb4740e1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60184/0 (socket says 172.21.15.46:60184) 2024-04-25T08:42:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb479400640 1 -- 172.21.15.46:0/2700041270 learned_addr learned my addr 172.21.15.46:0/2700041270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb47a800640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4740decc0 0x7fb4740d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb47a800640 1 -- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb47403e490 msgr2=0x7fb47404ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb47a800640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb47403e490 0x7fb47404ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb47a800640 1 -- 172.21.15.46:0/2700041270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb47403e870 msgr2=0x7fb4740e1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb47a800640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb47403e870 0x7fb4740e1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb47a800640 1 -- 172.21.15.46:0/2700041270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb474050f00 con 0x7fb4740decc0 2024-04-25T08:42:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb479400640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb47403e870 0x7fb4740e1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fc777400640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc758066f60 0x7fc758069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc7640b0000 tx=0x7fc76400c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fcdf2000640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf406d630 0x7fcdf4143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fcdeaa00640 1 -- 172.21.15.46:0/1968264627 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec016020 con 0x7fcdf4150820 2024-04-25T08:42:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fcdeaa00640 1 -- 172.21.15.46:0/1968264627 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec006d60 con 0x7fcdf4150820 2024-04-25T08:42:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fcdeaa00640 1 -- 172.21.15.46:0/1968264627 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec003ba0 con 0x7fcdf4150820 2024-04-25T08:42:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb47a800640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4740decc0 0x7fb4740d8f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb47c04fd80 tx=0x7fb460005840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb472000640 1 -- 172.21.15.46:0/2700041270 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c136070 con 0x7fb4740decc0 2024-04-25T08:42:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb472000640 1 -- 172.21.15.46:0/2700041270 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c131660 con 0x7fb4740decc0 2024-04-25T08:42:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf4203c00 con 0x7fcdf4150820 2024-04-25T08:42:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7f64b4c00640 1 -- 172.21.15.46:0/1199260494 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64a0016150 con 0x7f64b806d270 2024-04-25T08:42:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf42041c0 con 0x7fcdf4150820 2024-04-25T08:42:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.691+0000 7f64b7e00640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6494067060 0x7f6494069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.691+0000 7f64b7e00640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6494067060 0x7f6494069520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64b816c080 tx=0x7f64b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.687+0000 7fb472000640 1 -- 172.21.15.46:0/2700041270 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c007d80 con 0x7fb4740decc0 2024-04-25T08:42:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.691+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb474050660 con 0x7fb4740decc0 2024-04-25T08:42:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.691+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb474050c20 con 0x7fb4740decc0 2024-04-25T08:42:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.691+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf4069010 con 0x7fcdf4150820 2024-04-25T08:42:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.691+0000 7fb472000640 1 -- 172.21.15.46:0/2700041270 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb47c131a10 con 0x7fb4740decc0 2024-04-25T08:42:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.691+0000 7fc76fe00640 1 -- 172.21.15.46:0/1773903847 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc768056030 con 0x7fc77806d0a0 2024-04-25T08:42:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.695+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb474005720 con 0x7fb4740decc0 2024-04-25T08:42:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.695+0000 7fb472000640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb45c066f60 0x7fb45c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.695+0000 7fb472000640 1 -- 172.21.15.46:0/2700041270 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb47c1996f0 con 0x7fb4740decc0 2024-04-25T08:42:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.695+0000 7fb479e00640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb45c066f60 0x7fb45c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.695+0000 7fcdeaa00640 1 -- 172.21.15.46:0/1968264627 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcdec00a850 con 0x7fcdf4150820 2024-04-25T08:42:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.695+0000 7fb479e00640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb45c066f60 0x7fb45c069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb46c001d30 tx=0x7fb46c001c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.695+0000 7fcdeaa00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdd4067030 0x7fcdd40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.699+0000 7fcdeaa00640 1 -- 172.21.15.46:0/1968264627 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcdec084e80 con 0x7fcdf4150820 2024-04-25T08:42:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.699+0000 7fcdf2a00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdd4067030 0x7fcdd40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.699+0000 7fcdeaa00640 1 -- 172.21.15.46:0/1968264627 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcdec020020 con 0x7fcdf4150820 2024-04-25T08:42:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.703+0000 7fcdf2a00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdd4067030 0x7fcdd40694f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009ed0 tx=0x7fcde0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.703+0000 7fb472000640 1 -- 172.21.15.46:0/2700041270 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb47c0910b0 con 0x7fb4740decc0 2024-04-25T08:42:42.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.731+0000 7fb35e800640 1 Processor -- start 2024-04-25T08:42:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.731+0000 7fb35e800640 1 -- start start 2024-04-25T08:42:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.735+0000 7fb35e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3580cd650 0x7fb3581653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.735+0000 7fb35e800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3580b2c90 con 0x7fb35815a600 2024-04-25T08:42:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.735+0000 7fb35e800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3580b2f70 con 0x7fb35815a9e0 2024-04-25T08:42:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.735+0000 7fb35e800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3580afd10 con 0x7fb3580cd650 2024-04-25T08:42:42.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.735+0000 7fb356a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb35815a9e0 0x7fb3580ccf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:35502/0 (socket says 172.21.15.46:35502) 2024-04-25T08:42:42.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.735+0000 7fb356a00640 1 -- 172.21.15.46:0/351250266 learned_addr learned my addr 172.21.15.46:0/351250266 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.735+0000 7fb357e00640 1 --2- 172.21.15.46:0/351250266 >> v2:172.21.15.46:3300/0 conn(0x7fb3580cd650 0x7fb3581653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7fb356000640 1 -- 172.21.15.46:0/351250266 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385548573 0 0) 0x7fb3580b2f70 con 0x7fb35815a9e0 2024-04-25T08:42:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7fb356000640 1 -- 172.21.15.46:0/351250266 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb33c003530 con 0x7fb35815a9e0 2024-04-25T08:42:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7fb356000640 1 -- 172.21.15.46:0/351250266 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909063013 0 0) 0x7fb3580b2c90 con 0x7fb35815a600 2024-04-25T08:42:42.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7fb356000640 1 -- 172.21.15.46:0/351250266 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3580b2f70 con 0x7fb35815a600 2024-04-25T08:42:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7fb356000640 1 -- 172.21.15.46:0/351250266 <== mon.1 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1380737368 0 0) 0x7fb33c003530 con 0x7fb35815a9e0 2024-04-25T08:42:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7fb357e00640 1 -- 172.21.15.46:0/351250266 >> v1:172.21.15.82:6789/0 conn(0x7fb35815a9e0 legacy=0x7fb3580ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7fb357e00640 1 -- 172.21.15.46:0/351250266 >> v1:172.21.15.46:6789/0 conn(0x7fb35815a600 legacy=0x7fb35806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7fb357e00640 1 -- 172.21.15.46:0/351250266 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3580ced40 con 0x7fb3580cd650 2024-04-25T08:42:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7ff357200640 1 Processor -- start 2024-04-25T08:42:42.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.739+0000 7ff357200640 1 -- start start 2024-04-25T08:42:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7f487fe00640 1 Processor -- start 2024-04-25T08:42:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff357200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff3500cd920 0x7ff350140e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7f487fe00640 1 -- start start 2024-04-25T08:42:42.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff355e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff35013e540 0x7ff3500cd210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59160/0 (socket says 172.21.15.46:59160) 2024-04-25T08:42:42.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff355e00640 1 -- 172.21.15.46:0/2421068594 learned_addr learned my addr 172.21.15.46:0/2421068594 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff357200640 1 -- 172.21.15.46:0/2421068594 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3500710e0 con 0x7ff35013e540 2024-04-25T08:42:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243755911 0 0) 0x7ff3500710e0 con 0x7ff35013e540 2024-04-25T08:42:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff357200640 1 -- 172.21.15.46:0/2421068594 --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3500713c0 con 0x7ff350071ea0 2024-04-25T08:42:42.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7fb357e00640 1 --2- 172.21.15.46:0/351250266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3580cd650 0x7fb3581653f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb350009d90 tx=0x7fb35001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbaf8433b1a2363 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7fb356000640 1 -- 172.21.15.46:0/351250266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348001680 con 0x7fb3580cd650 2024-04-25T08:42:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7fb356000640 1 -- 172.21.15.46:0/351250266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3500a6470 con 0x7fb3580cd650 2024-04-25T08:42:42.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7f487fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f488015d4f0 0x7f488015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7fb356000640 1 -- 172.21.15.46:0/351250266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500a6840 con 0x7fb3580cd650 2024-04-25T08:42:42.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7f487fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48800b2910 0x7f488006e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff357200640 1 -- 172.21.15.46:0/2421068594 --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3500716a0 con 0x7ff3500cd920 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff338007940 con 0x7ff35013e540 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4157664238 0 0) 0x7ff338007940 con 0x7ff35013e540 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 >> v2:172.21.15.82:3300/0 conn(0x7ff3500cd920 msgr2=0x7ff350140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 --2- 172.21.15.46:0/2421068594 >> v2:172.21.15.82:3300/0 conn(0x7ff3500cd920 0x7ff350140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 >> v1:172.21.15.82:6789/0 conn(0x7ff350071ea0 legacy=0x7ff350141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff350071980 con 0x7ff35013e540 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7ff34c001a00 con 0x7ff35013e540 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 >> v1:172.21.15.46:6789/0 conn(0x7ff35013e540 legacy=0x7ff3500cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 --2- 172.21.15.46:0/2421068594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff338007ea0 0x7ff338008280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 --2- 172.21.15.46:0/2421068594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff338008c20 0x7ff3380090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 --2- 172.21.15.46:0/2421068594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3380097e0 0x7ff33800fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500716a0 con 0x7ff3380097e0 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380106a0 con 0x7ff338007ea0 2024-04-25T08:42:42.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.743+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380108a0 con 0x7ff338008c20 2024-04-25T08:42:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7ff356800640 1 --2- 172.21.15.46:0/2421068594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3380097e0 0x7ff33800fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487fe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48800b2dc0 con 0x7f48800cbda0 2024-04-25T08:42:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f488015d4f0 0x7f488015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4880162340 con 0x7f48800b2910 2024-04-25T08:42:42.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487fe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48801617c0 con 0x7f488015d4f0 2024-04-25T08:42:42.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7fb35e800640 1 -- 172.21.15.46:0/351250266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3580cd650 msgr2=0x7fb3581653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7fb35e800640 1 --2- 172.21.15.46:0/351250266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3580cd650 0x7fb3581653f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb350009d90 tx=0x7fb35001c820 comp rx=0 tx=0).stop 2024-04-25T08:42:42.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7ff356800640 1 -- 172.21.15.46:0/2421068594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff338008c20 msgr2=0x7ff3380090a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:42:42.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7ff356800640 1 --2- 172.21.15.46:0/2421068594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff338008c20 0x7ff3380090a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7ff356800640 1 -- 172.21.15.46:0/2421068594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff338007ea0 msgr2=0x7ff338008280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:42:42.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7ff356800640 1 --2- 172.21.15.46:0/2421068594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff338007ea0 0x7ff338008280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7ff356800640 1 -- 172.21.15.46:0/2421068594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338007940 con 0x7ff3380097e0 2024-04-25T08:42:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487f400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f48800cbda0 0x7f488006d5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:35514/0 (socket says 172.21.15.46:35514) 2024-04-25T08:42:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487f400640 1 -- 172.21.15.46:0/1864769058 learned_addr learned my addr 172.21.15.46:0/1864769058 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f488015d4f0 0x7f488015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50206/0 (socket says 172.21.15.46:50206) 2024-04-25T08:42:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487e000640 1 --2- 172.21.15.46:0/1864769058 >> v2:172.21.15.46:3300/0 conn(0x7f48800b2910 0x7f488006e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487ea00640 1 -- 172.21.15.46:0/1864769058 >> v2:172.21.15.46:3300/0 conn(0x7f48800b2910 msgr2=0x7f488006e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487ea00640 1 --2- 172.21.15.46:0/1864769058 >> v2:172.21.15.46:3300/0 conn(0x7f48800b2910 0x7f488006e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7fb35e800640 1 -- 172.21.15.46:0/351250266 shutdown_connections 2024-04-25T08:42:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487ea00640 1 -- 172.21.15.46:0/1864769058 >> v1:172.21.15.82:6789/0 conn(0x7f48800cbda0 legacy=0x7f488006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7fb35e800640 1 --2- 172.21.15.46:0/351250266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3580cd650 0x7fb3581653f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7fb35e800640 1 -- 172.21.15.46:0/351250266 >> 172.21.15.46:0/351250266 conn(0x7fb358099a00 msgr2=0x7fb3580b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7f487ea00640 1 -- 172.21.15.46:0/1864769058 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4880161aa0 con 0x7f488015d4f0 2024-04-25T08:42:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7fb35e800640 1 -- 172.21.15.46:0/351250266 shutdown_connections 2024-04-25T08:42:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.747+0000 7fb35e800640 1 -- 172.21.15.46:0/351250266 wait complete. 2024-04-25T08:42:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.751+0000 7fb35e800640 1 Processor -- start 2024-04-25T08:42:42.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.751+0000 7fb35e800640 1 -- start start 2024-04-25T08:42:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.751+0000 7ff356800640 1 -- 172.21.15.46:0/2421068594 reap_dead start 2024-04-25T08:42:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.751+0000 7ff356800640 1 --2- 172.21.15.46:0/2421068594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3380097e0 0x7ff33800fde0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff3400018d0 tx=0x7ff34001c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56deb221918801e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb35e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3580cd650 0x7fb35815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb35e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb35815a600 0x7fb3580b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7f487ea00640 1 --2- 172.21.15.46:0/1864769058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488015d4f0 0x7f488015d8d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4880072450 tx=0x7f487001c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32fde967abd62605 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb35e800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb35815a9e0 0x7fb35815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3580cd650 0x7fb35815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3580cd650 0x7fb35815d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50216/0 (socket says 172.21.15.46:50216) 2024-04-25T08:42:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357400640 1 -- 172.21.15.46:0/1886326427 learned_addr learned my addr 172.21.15.46:0/1886326427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3581688f0 con 0x7fb35815a600 2024-04-25T08:42:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb358168770 con 0x7fb3580cd650 2024-04-25T08:42:42.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb358168a70 con 0x7fb35815a9e0 2024-04-25T08:42:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb356a00640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb35815a600 0x7fb3580b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357e00640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb35815a9e0 0x7fb35815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357e00640 1 -- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3580cd650 msgr2=0x7fb35815d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357e00640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3580cd650 0x7fb35815d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357e00640 1 -- 172.21.15.46:0/1886326427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb35815a600 msgr2=0x7fb3580b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357e00640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb35815a600 0x7fb3580b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357e00640 1 -- 172.21.15.46:0/1886326427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3580b22d0 con 0x7fb35815a9e0 2024-04-25T08:42:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb357e00640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb35815a9e0 0x7fb35815b200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb358072120 tx=0x7fb3500195c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340001270 con 0x7ff3380097e0 2024-04-25T08:42:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb34f400640 1 -- 172.21.15.46:0/1886326427 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500a7650 con 0x7fb35815a9e0 2024-04-25T08:42:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff340005270 con 0x7ff3380097e0 2024-04-25T08:42:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7ff354a00640 1 -- 172.21.15.46:0/2421068594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340003600 con 0x7ff3380097e0 2024-04-25T08:42:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35814f8a0 con 0x7fb35815a9e0 2024-04-25T08:42:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35814fe60 con 0x7fb35815a9e0 2024-04-25T08:42:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7f487d600640 1 -- 172.21.15.46:0/1864769058 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700b0070 con 0x7f488015d4f0 2024-04-25T08:42:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7f487d600640 1 -- 172.21.15.46:0/1864769058 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48700ab600 con 0x7f488015d4f0 2024-04-25T08:42:42.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7f487d600640 1 -- 172.21.15.46:0/1864769058 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700ab9d0 con 0x7f488015d4f0 2024-04-25T08:42:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7f487fe00640 1 -- 172.21.15.46:0/1864769058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488015d4f0 msgr2=0x7f488015d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.755+0000 7f487fe00640 1 --2- 172.21.15.46:0/1864769058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488015d4f0 0x7f488015d8d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4880072450 tx=0x7f487001c280 comp rx=0 tx=0).stop 2024-04-25T08:42:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7ff357200640 1 -- 172.21.15.46:0/2421068594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3380097e0 msgr2=0x7ff33800fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7ff357200640 1 --2- 172.21.15.46:0/2421068594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3380097e0 0x7ff33800fde0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff3400018d0 tx=0x7ff34001c580 comp rx=0 tx=0).stop 2024-04-25T08:42:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7ff357200640 1 -- 172.21.15.46:0/2421068594 shutdown_connections 2024-04-25T08:42:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7ff357200640 1 --2- 172.21.15.46:0/2421068594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3380097e0 0x7ff33800fde0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7ff357200640 1 -- 172.21.15.46:0/2421068594 >> 172.21.15.46:0/2421068594 conn(0x7ff350066fa0 msgr2=0x7ff35006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7f487fe00640 1 -- 172.21.15.46:0/1864769058 shutdown_connections 2024-04-25T08:42:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7f487fe00640 1 --2- 172.21.15.46:0/1864769058 >> v2:172.21.15.46:3300/0 conn(0x7f48800b2910 0x7f488006e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7f487fe00640 1 --2- 172.21.15.46:0/1864769058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f488015d4f0 0x7f488015d8d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7f487fe00640 1 -- 172.21.15.46:0/1864769058 >> 172.21.15.46:0/1864769058 conn(0x7f4880066b70 msgr2=0x7f488006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7f487fe00640 1 -- 172.21.15.46:0/1864769058 shutdown_connections 2024-04-25T08:42:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7f487fe00640 1 -- 172.21.15.46:0/1864769058 wait complete. 2024-04-25T08:42:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7fb34f400640 1 -- 172.21.15.46:0/1886326427 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3500be070 con 0x7fb35815a9e0 2024-04-25T08:42:42.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7fb34f400640 1 -- 172.21.15.46:0/1886326427 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500a79f0 con 0x7fb35815a9e0 2024-04-25T08:42:42.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7ff357200640 1 -- 172.21.15.46:0/2421068594 shutdown_connections 2024-04-25T08:42:42.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7ff357200640 1 -- 172.21.15.46:0/2421068594 wait complete. 2024-04-25T08:42:42.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7ff357200640 1 Processor -- start 2024-04-25T08:42:42.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7f487fe00640 1 Processor -- start 2024-04-25T08:42:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3300052a0 con 0x7fb35815a9e0 2024-04-25T08:42:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.759+0000 7fb34f400640 1 -- 172.21.15.46:0/1886326427 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3500b8070 con 0x7fb35815a9e0 2024-04-25T08:42:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7fb34f400640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb338066f60 0x7fb338069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7f487fe00640 1 -- start start 2024-04-25T08:42:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7fb34f400640 1 -- 172.21.15.46:0/1886326427 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb35011fd40 con 0x7fb35815a9e0 2024-04-25T08:42:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7fb357400640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb338066f60 0x7fb338069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7fb357400640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb338066f60 0x7fb338069420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb344001020 tx=0x7fb344009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7ff357200640 1 -- start start 2024-04-25T08:42:42.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7f487fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48800b2910 0x7f4880151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7ff357200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3380097e0 0x7ff35013bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7ff357200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff35013dae0 0x7ff3501ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7ff357200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff350139b60 0x7ff35006e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7f487fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48800cbda0 0x7f48800ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7f487ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48800b2910 0x7f4880151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7ff355400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff35013dae0 0x7ff3501ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7ff355400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff35013dae0 0x7ff3501ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50232/0 (socket says 172.21.15.46:50232) 2024-04-25T08:42:42.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7f487ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48800b2910 0x7f4880151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36048/0 (socket says 172.21.15.46:36048) 2024-04-25T08:42:42.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7ff355e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3380097e0 0x7ff35013bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7ff355e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3380097e0 0x7ff35013bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36056/0 (socket says 172.21.15.46:36056) 2024-04-25T08:42:42.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7f487e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48800cbda0 0x7f48800ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.763+0000 7f487e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48800cbda0 0x7f48800ad8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50248/0 (socket says 172.21.15.46:50248) 2024-04-25T08:42:42.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff355400640 1 -- 172.21.15.46:0/1533276584 learned_addr learned my addr 172.21.15.46:0/1533276584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500cd390 con 0x7ff350139b60 2024-04-25T08:42:42.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff356800640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff350139b60 0x7ff35006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500cd210 con 0x7ff35013dae0 2024-04-25T08:42:42.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487ea00640 1 -- 172.21.15.46:0/2911529707 learned_addr learned my addr 172.21.15.46:0/2911529707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:42.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3500cd510 con 0x7ff3380097e0 2024-04-25T08:42:42.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487fe00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f488015d4f0 0x7f48800ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48801613e0 con 0x7f488015d4f0 2024-04-25T08:42:42.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4880162060 con 0x7f48800cbda0 2024-04-25T08:42:42.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4880161560 con 0x7f48800b2910 2024-04-25T08:42:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff355400640 1 -- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3380097e0 msgr2=0x7ff35013bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff355400640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3380097e0 0x7ff35013bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff355400640 1 -- 172.21.15.46:0/1533276584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff350139b60 msgr2=0x7ff35006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff355400640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff350139b60 0x7ff35006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff355400640 1 -- 172.21.15.46:0/1533276584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff350154670 con 0x7ff35013dae0 2024-04-25T08:42:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff355400640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff35013dae0 0x7ff3501ffc80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b490 tx=0x7ff34400b950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487f400640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f488015d4f0 0x7f48800ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487e000640 1 -- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48800b2910 msgr2=0x7f4880151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487e000640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48800b2910 0x7f4880151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487e000640 1 -- 172.21.15.46:0/2911529707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f488015d4f0 msgr2=0x7f48800ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff34a000640 1 -- 172.21.15.46:0/1533276584 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344015020 con 0x7ff35013dae0 2024-04-25T08:42:42.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487e000640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f488015d4f0 0x7f48800ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487e000640 1 -- 172.21.15.46:0/2911529707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4880160780 con 0x7f48800cbda0 2024-04-25T08:42:42.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487ea00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48800b2910 0x7f4880151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff350139220 con 0x7ff35013dae0 2024-04-25T08:42:42.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff350139fe0 con 0x7ff35013dae0 2024-04-25T08:42:42.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff34a000640 1 -- 172.21.15.46:0/1533276584 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff344011830 con 0x7ff35013dae0 2024-04-25T08:42:42.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7f487e000640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48800cbda0 0x7f48800ad8c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c7c0 tx=0x7f486c00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.767+0000 7ff34a000640 1 -- 172.21.15.46:0/1533276584 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344015020 con 0x7ff35013dae0 2024-04-25T08:42:42.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7f4876a00640 1 -- 172.21.15.46:0/2911529707 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c015070 con 0x7f48800cbda0 2024-04-25T08:42:42.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7f4876a00640 1 -- 172.21.15.46:0/2911529707 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c007820 con 0x7f48800cbda0 2024-04-25T08:42:42.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7f4876a00640 1 -- 172.21.15.46:0/2911529707 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c007bf0 con 0x7f48800cbda0 2024-04-25T08:42:42.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7fb34f400640 1 -- 172.21.15.46:0/1886326427 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3500b6040 con 0x7fb35815a9e0 2024-04-25T08:42:42.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488015fb70 con 0x7f48800cbda0 2024-04-25T08:42:42.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4880160130 con 0x7f48800cbda0 2024-04-25T08:42:42.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c0052a0 con 0x7ff35013dae0 2024-04-25T08:42:42.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4880069010 con 0x7f48800cbda0 2024-04-25T08:42:42.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7ff34a000640 1 -- 172.21.15.46:0/1533276584 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff344018020 con 0x7ff35013dae0 2024-04-25T08:42:42.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.771+0000 7f4876a00640 1 -- 172.21.15.46:0/2911529707 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f486c018070 con 0x7f48800cbda0 2024-04-25T08:42:42.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.775+0000 7ff34a000640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff334067160 0x7ff334069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.775+0000 7ff34a000640 1 -- 172.21.15.46:0/1533276584 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff344084a00 con 0x7ff35013dae0 2024-04-25T08:42:42.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.775+0000 7f4876a00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4860066fe0 0x7f48600694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.775+0000 7f4876a00640 1 -- 172.21.15.46:0/2911529707 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f486c085070 con 0x7f48800cbda0 2024-04-25T08:42:42.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.775+0000 7ff355e00640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff334067160 0x7ff334069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.775+0000 7f487ea00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4860066fe0 0x7f48600694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.775+0000 7ff355e00640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff334067160 0x7ff334069620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0013e0 tx=0x7ff34c001ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.775+0000 7f487ea00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4860066fe0 0x7f48600694a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4870000c10 tx=0x7f48700a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.779+0000 7ff34a000640 1 -- 172.21.15.46:0/1533276584 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff34401f030 con 0x7ff35013dae0 2024-04-25T08:42:42.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:42.783+0000 7f4876a00640 1 -- 172.21.15.46:0/2911529707 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f486c055760 con 0x7f48800cbda0 2024-04-25T08:42:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.019+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f94c406ba80 con 0x7f94c41598a0 2024-04-25T08:42:43.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.019+0000 7f94ba000640 1 -- 172.21.15.46:0/2869152550 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f94c4162320 con 0x7f94c41598a0 2024-04-25T08:42:43.014 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2024-04-25T08:42:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.023+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a4066f60 msgr2=0x7f94a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.023+0000 7f94c9000640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a4066f60 0x7f94a4069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f94b00ad000 tx=0x7f94b0004850 comp rx=0 tx=0).stop 2024-04-25T08:42:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c41598a0 msgr2=0x7f94c4159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c41598a0 0x7f94c4159d20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f94c40b36e0 tx=0x7f94bc00a690 comp rx=0 tx=0).stop 2024-04-25T08:42:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 shutdown_connections 2024-04-25T08:42:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a4066f60 0x7f94a4069420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c41598a0 0x7f94c4159d20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c41550a0 0x7f94c41610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 --2- 172.21.15.46:0/2869152550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c4202e20 0x7f94c4203200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 >> 172.21.15.46:0/2869152550 conn(0x7f94c40b1130 msgr2=0x7f94c406e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 shutdown_connections 2024-04-25T08:42:43.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7f94c9000640 1 -- 172.21.15.46:0/2869152550 wait complete. 2024-04-25T08:42:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc778068720 con 0x7fc77806d0a0 2024-04-25T08:42:43.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.027+0000 7fc76fe00640 1 -- 172.21.15.46:0/1773903847 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc76805a0d0 con 0x7fc77806d0a0 2024-04-25T08:42:43.022 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2024-04-25T08:42:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f64b80b0030 con 0x7f64b806d270 2024-04-25T08:42:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.031+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc758066f60 msgr2=0x7fc758069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.031+0000 7fc77d600640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc758066f60 0x7fc758069420 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc7640b0000 tx=0x7fc76400c040 comp rx=0 tx=0).stop 2024-04-25T08:42:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.031+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc77806d0a0 msgr2=0x7fc778142c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.031+0000 7fc77d600640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc77806d0a0 0x7fc778142c20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc76800e870 tx=0x7fc76800ec70 comp rx=0 tx=0).stop 2024-04-25T08:42:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 shutdown_connections 2024-04-25T08:42:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7fc77d600640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc758066f60 0x7fc758069420 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7fc77d600640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7781579b0 0x7fc77814e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7fc77d600640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc778071040 0x7fc778150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7fc77d600640 1 --2- 172.21.15.46:0/1773903847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc77806d0a0 0x7fc778142c20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 >> 172.21.15.46:0/1773903847 conn(0x7fc778066fa0 msgr2=0x7fc77815b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 shutdown_connections 2024-04-25T08:42:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7fc77d600640 1 -- 172.21.15.46:0/1773903847 wait complete. 2024-04-25T08:42:43.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.035+0000 7f64b4c00640 1 -- 172.21.15.46:0/1199260494 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64a00efe40 con 0x7f64b806d270 2024-04-25T08:42:43.030 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2024-04-25T08:42:43.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6494067060 msgr2=0x7f6494069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6494067060 0x7f6494069520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64b816c080 tx=0x7f64b0008040 comp rx=0 tx=0).stop 2024-04-25T08:42:43.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b806d270 msgr2=0x7f64b815b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b806d270 0x7f64b815b660 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f64a001ac10 tx=0x7f64a0019300 comp rx=0 tx=0).stop 2024-04-25T08:42:43.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb474053a80 con 0x7fb4740decc0 2024-04-25T08:42:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 shutdown_connections 2024-04-25T08:42:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6494067060 0x7f6494069520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64b8168450 0x7f64b8168900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64b8162330 0x7f64b81599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 --2- 172.21.15.46:0/1199260494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64b806d270 0x7f64b815b660 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 >> 172.21.15.46:0/1199260494 conn(0x7f64b80b12a0 msgr2=0x7f64b80af210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.039+0000 7fb472000640 1 -- 172.21.15.46:0/2700041270 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb47c170020 con 0x7fb4740decc0 2024-04-25T08:42:43.035 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.043+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 shutdown_connections 2024-04-25T08:42:43.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.043+0000 7f64bf600640 1 -- 172.21.15.46:0/1199260494 wait complete. 2024-04-25T08:42:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb45c066f60 msgr2=0x7fb45c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb45c066f60 0x7fb45c069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb46c001d30 tx=0x7fb46c001c80 comp rx=0 tx=0).stop 2024-04-25T08:42:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4740decc0 msgr2=0x7fb4740d8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4740decc0 0x7fb4740d8f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb47c04fd80 tx=0x7fb460005840 comp rx=0 tx=0).stop 2024-04-25T08:42:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 shutdown_connections 2024-04-25T08:42:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb45c066f60 0x7fb45c069420 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4740decc0 0x7fb4740d8f00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb47403e870 0x7fb4740e1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 --2- 172.21.15.46:0/2700041270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb47403e490 0x7fb47404ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 >> 172.21.15.46:0/2700041270 conn(0x7fb474058f00 msgr2=0x7fb4740069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 shutdown_connections 2024-04-25T08:42:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.047+0000 7fb47b200640 1 -- 172.21.15.46:0/2700041270 wait complete. 2024-04-25T08:42:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.051+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcdf4200070 con 0x7fcdf4150820 2024-04-25T08:42:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.051+0000 7fcdeaa00640 1 -- 172.21.15.46:0/1968264627 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcdec00ce20 con 0x7fcdf4150820 2024-04-25T08:42:43.046 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2024-04-25T08:42:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdd4067030 msgr2=0x7fcdd40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdd4067030 0x7fcdd40694f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009ed0 tx=0x7fcde0009190 comp rx=0 tx=0).stop 2024-04-25T08:42:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdf4150820 msgr2=0x7fcdf40b0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdf4150820 0x7fcdf40b0190 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00c710 tx=0x7fcdec00cbd0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 shutdown_connections 2024-04-25T08:42:43.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdd4067030 0x7fcdd40694f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdf4150820 0x7fcdf40b0190 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf406d630 0x7fcdf4143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 --2- 172.21.15.46:0/1968264627 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcdf4066f80 0x7fcdf4140670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.055+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 >> 172.21.15.46:0/1968264627 conn(0x7fcdf4066b70 msgr2=0x7fcdf406e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.063+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 shutdown_connections 2024-04-25T08:42:43.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.063+0000 7fcdf3e00640 1 -- 172.21.15.46:0/1968264627 wait complete. 2024-04-25T08:42:43.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.0 2024-04-25T08:42:43.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.5 2024-04-25T08:42:43.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.2 2024-04-25T08:42:43.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.1 2024-04-25T08:42:43.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.3 2024-04-25T08:42:43.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4880152350 con 0x7f48800cbda0 2024-04-25T08:42:43.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f4876a00640 1 -- 172.21.15.46:0/2911529707 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f486c059800 con 0x7f48800cbda0 2024-04-25T08:42:43.091 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff32c005590 con 0x7ff35013dae0 2024-04-25T08:42:43.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7ff34a000640 1 -- 172.21.15.46:0/1533276584 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff344054fe0 con 0x7ff35013dae0 2024-04-25T08:42:43.092 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2024-04-25T08:42:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4860066fe0 msgr2=0x7f48600694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4860066fe0 0x7f48600694a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4870000c10 tx=0x7f48700a6000 comp rx=0 tx=0).stop 2024-04-25T08:42:43.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48800cbda0 msgr2=0x7f48800ad8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48800cbda0 0x7f48800ad8c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c7c0 tx=0x7f486c00cc80 comp rx=0 tx=0).stop 2024-04-25T08:42:43.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 shutdown_connections 2024-04-25T08:42:43.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4860066fe0 0x7f48600694a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f488015d4f0 0x7f48800ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48800cbda0 0x7f48800ad8c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 --2- 172.21.15.46:0/2911529707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f48800b2910 0x7f4880151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 >> 172.21.15.46:0/2911529707 conn(0x7f4880066b70 msgr2=0x7f48800b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 shutdown_connections 2024-04-25T08:42:43.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.099+0000 7f487fe00640 1 -- 172.21.15.46:0/2911529707 wait complete. 2024-04-25T08:42:43.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff334067160 msgr2=0x7ff334069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff334067160 0x7ff334069620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0013e0 tx=0x7ff34c001ba0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff35013dae0 msgr2=0x7ff3501ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff35013dae0 0x7ff3501ffc80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff34400b490 tx=0x7ff34400b950 comp rx=0 tx=0).stop 2024-04-25T08:42:43.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 shutdown_connections 2024-04-25T08:42:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff334067160 0x7ff334069620 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff350139b60 0x7ff35006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff35013dae0 0x7ff3501ffc80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 --2- 172.21.15.46:0/1533276584 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3380097e0 0x7ff35013bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 >> 172.21.15.46:0/1533276584 conn(0x7ff350066fa0 msgr2=0x7ff3500b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 shutdown_connections 2024-04-25T08:42:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7ff357200640 1 -- 172.21.15.46:0/1533276584 wait complete. 2024-04-25T08:42:43.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb330005590 con 0x7fb35815a9e0 2024-04-25T08:42:43.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.103+0000 7fb34f400640 1 -- 172.21.15.46:0/1886326427 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3500ac030 con 0x7fb35815a9e0 2024-04-25T08:42:43.097 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb338066f60 msgr2=0x7fb338069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb338066f60 0x7fb338069420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb344001020 tx=0x7fb344009040 comp rx=0 tx=0).stop 2024-04-25T08:42:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb35815a9e0 msgr2=0x7fb35815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb35815a9e0 0x7fb35815b200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb358072120 tx=0x7fb3500195c0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 shutdown_connections 2024-04-25T08:42:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb338066f60 0x7fb338069420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb35815a9e0 0x7fb35815b200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb35815a600 0x7fb3580b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 --2- 172.21.15.46:0/1886326427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3580cd650 0x7fb35815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 >> 172.21.15.46:0/1886326427 conn(0x7fb358099a00 msgr2=0x7fb3580cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 shutdown_connections 2024-04-25T08:42:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:43.107+0000 7fb35e800640 1 -- 172.21.15.46:0/1886326427 wait complete. 2024-04-25T08:42:43.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.7 2024-04-25T08:42:43.120 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.4 2024-04-25T08:42:43.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.6 2024-04-25T08:42:44.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T08:42:44.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T08:42:44.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T08:42:44.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T08:42:44.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T08:42:44.118 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T08:42:44.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T08:42:44.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T08:42:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.263+0000 7f5dd6200640 1 Processor -- start 2024-04-25T08:42:44.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.263+0000 7f5dd6200640 1 -- start start 2024-04-25T08:42:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.263+0000 7f5dd6200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5dd015d8d0 0x7f5dd00b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.263+0000 7f5dd6200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5dd006e7d0 0x7f5dd00cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.263+0000 7f5dd6200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd01623a0 con 0x7f5dd006ec30 2024-04-25T08:42:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.263+0000 7f5dd6200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd015dcb0 con 0x7f5dd015d8d0 2024-04-25T08:42:44.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.263+0000 7f5dd6200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd0161820 con 0x7f5dd006e7d0 2024-04-25T08:42:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.267+0000 7f5dd4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5dd015d8d0 0x7f5dd00b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.267+0000 7f5dd4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5dd015d8d0 0x7f5dd00b2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60250/0 (socket says 172.21.15.46:60250) 2024-04-25T08:42:44.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.267+0000 7f5dd4e00640 1 -- 172.21.15.46:0/3815857563 learned_addr learned my addr 172.21.15.46:0/3815857563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.267+0000 7f5dd4e00640 1 -- 172.21.15.46:0/3815857563 >> v2:172.21.15.82:3300/0 conn(0x7f5dd006e7d0 msgr2=0x7f5dd00cd900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:42:44.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.267+0000 7f5dd4e00640 1 --2- 172.21.15.46:0/3815857563 >> v2:172.21.15.82:3300/0 conn(0x7f5dd006e7d0 0x7f5dd00cd900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.267+0000 7f5dd4e00640 1 -- 172.21.15.46:0/3815857563 >> v1:172.21.15.82:6789/0 conn(0x7f5dd006ec30 legacy=0x7f5dd006d5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:42:44.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.267+0000 7f5dd4e00640 1 -- 172.21.15.46:0/3815857563 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0161b00 con 0x7f5dd015d8d0 2024-04-25T08:42:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.271+0000 7f5dd4e00640 1 --2- 172.21.15.46:0/3815857563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd015d8d0 0x7f5dd00b2980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00724e0 tx=0x7f5dc001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee1637ba311524d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.271+0000 7f5dcf400640 1 -- 172.21.15.46:0/3815857563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00b0070 con 0x7f5dd015d8d0 2024-04-25T08:42:44.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dcf400640 1 -- 172.21.15.46:0/3815857563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc00ab600 con 0x7f5dd015d8d0 2024-04-25T08:42:44.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dcf400640 1 -- 172.21.15.46:0/3815857563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00ab9d0 con 0x7f5dd015d8d0 2024-04-25T08:42:44.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dd6200640 1 -- 172.21.15.46:0/3815857563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd015d8d0 msgr2=0x7f5dd00b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dd6200640 1 --2- 172.21.15.46:0/3815857563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd015d8d0 0x7f5dd00b2980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00724e0 tx=0x7f5dc001c470 comp rx=0 tx=0).stop 2024-04-25T08:42:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dd6200640 1 -- 172.21.15.46:0/3815857563 shutdown_connections 2024-04-25T08:42:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dd6200640 1 --2- 172.21.15.46:0/3815857563 >> v2:172.21.15.82:3300/0 conn(0x7f5dd006e7d0 0x7f5dd00cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dd6200640 1 --2- 172.21.15.46:0/3815857563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd015d8d0 0x7f5dd00b2980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dd6200640 1 -- 172.21.15.46:0/3815857563 >> 172.21.15.46:0/3815857563 conn(0x7f5dd0066b70 msgr2=0x7f5dd006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dd6200640 1 -- 172.21.15.46:0/3815857563 shutdown_connections 2024-04-25T08:42:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.275+0000 7f5dd6200640 1 -- 172.21.15.46:0/3815857563 wait complete. 2024-04-25T08:42:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.279+0000 7f5dd6200640 1 Processor -- start 2024-04-25T08:42:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.279+0000 7f5dd6200640 1 -- start start 2024-04-25T08:42:44.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.279+0000 7f5dd6200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd006e7d0 0x7f5dd014bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.279+0000 7f5dd6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd006ec30 0x7f5dd014ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd6200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5dd015d8d0 0x7f5dd014df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd01615e0 con 0x7f5dd006ec30 2024-04-25T08:42:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd6200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd0161460 con 0x7f5dd015d8d0 2024-04-25T08:42:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd6200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd016a800 con 0x7f5dd006e7d0 2024-04-25T08:42:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd006ec30 0x7f5dd014ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd006ec30 0x7f5dd014ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60264/0 (socket says 172.21.15.46:60264) 2024-04-25T08:42:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dcfe00640 1 -- 172.21.15.46:0/724567121 learned_addr learned my addr 172.21.15.46:0/724567121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd5800640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5dd015d8d0 0x7f5dd014df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd4e00640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd006e7d0 0x7f5dd014bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd5800640 1 -- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd006e7d0 msgr2=0x7f5dd014bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd5800640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd006e7d0 0x7f5dd014bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd5800640 1 -- 172.21.15.46:0/724567121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd006ec30 msgr2=0x7f5dd014ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd5800640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd006ec30 0x7f5dd014ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd5800640 1 -- 172.21.15.46:0/724567121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd014b360 con 0x7f5dd015d8d0 2024-04-25T08:42:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dcfe00640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd006ec30 0x7f5dd014ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd4e00640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd006e7d0 0x7f5dd014bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd5800640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5dd015d8d0 0x7f5dd014df80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00b410 tx=0x7f5dbc00b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dccc00640 1 -- 172.21.15.46:0/724567121 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc015020 con 0x7f5dd015d8d0 2024-04-25T08:42:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dccc00640 1 -- 172.21.15.46:0/724567121 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc011830 con 0x7f5dd015d8d0 2024-04-25T08:42:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dccc00640 1 -- 172.21.15.46:0/724567121 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc011c20 con 0x7f5dd015d8d0 2024-04-25T08:42:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.283+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd0153610 con 0x7f5dd015d8d0 2024-04-25T08:42:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.287+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd0153bd0 con 0x7f5dd015d8d0 2024-04-25T08:42:44.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.287+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd014ce50 con 0x7f5dd015d8d0 2024-04-25T08:42:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.291+0000 7f5dccc00640 1 -- 172.21.15.46:0/724567121 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dbc018020 con 0x7f5dd015d8d0 2024-04-25T08:42:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.291+0000 7f5dccc00640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066fe0 0x7f5db00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.291+0000 7f5dccc00640 1 -- 172.21.15.46:0/724567121 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5dbc0537f0 con 0x7f5dd015d8d0 2024-04-25T08:42:44.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.291+0000 7f5dd4e00640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066fe0 0x7f5db00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.291+0000 7f5dd4e00640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066fe0 0x7f5db00694a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00ad000 tx=0x7f5dc00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.295+0000 7f5dccc00640 1 -- 172.21.15.46:0/724567121 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5dbc01f020 con 0x7f5dd015d8d0 2024-04-25T08:42:44.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.339+0000 7eff43200640 1 Processor -- start 2024-04-25T08:42:44.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.339+0000 7eff43200640 1 -- start start 2024-04-25T08:42:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.339+0000 7eff43200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7eff340e30d0 0x7eff34005ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.339+0000 7eff43200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff3404f300 0x7eff340e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.339+0000 7eff43200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff340e7850 con 0x7eff34054500 2024-04-25T08:42:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.339+0000 7eff43200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff34037bc0 con 0x7eff340e30d0 2024-04-25T08:42:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.339+0000 7eff43200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340542c0 con 0x7eff3404f300 2024-04-25T08:42:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7eff3b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7eff34054500 0x7eff3404faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59166/0 (socket says 172.21.15.46:59166) 2024-04-25T08:42:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7eff3b400640 1 -- 172.21.15.46:0/4110424352 learned_addr learned my addr 172.21.15.46:0/4110424352 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7eff41200640 1 --2- 172.21.15.46:0/4110424352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff3404f300 0x7eff340e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7eff3be00640 1 --2- 172.21.15.46:0/4110424352 >> v2:172.21.15.46:3300/0 conn(0x7eff340e30d0 0x7eff34005ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7eff41200640 1 -- 172.21.15.46:0/4110424352 >> v2:172.21.15.46:3300/0 conn(0x7eff340e30d0 msgr2=0x7eff34005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7eff41200640 1 --2- 172.21.15.46:0/4110424352 >> v2:172.21.15.46:3300/0 conn(0x7eff340e30d0 0x7eff34005ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7eff41200640 1 -- 172.21.15.46:0/4110424352 >> v1:172.21.15.46:6789/0 conn(0x7eff34054500 legacy=0x7eff3404faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7eff41200640 1 -- 172.21.15.46:0/4110424352 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34053c90 con 0x7eff3404f300 2024-04-25T08:42:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7fec65e00640 1 Processor -- start 2024-04-25T08:42:44.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.343+0000 7fec65e00640 1 -- start start 2024-04-25T08:42:44.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7fec65e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec60159470 0x7fec6015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7fec5ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec60159470 0x7fec6015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7eff3be00640 1 --2- 172.21.15.46:0/4110424352 >> v2:172.21.15.46:3300/0 conn(0x7eff340e30d0 0x7eff34005ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7fec65e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc080 0x7fec60099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7fec5ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec60159470 0x7fec6015bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60286/0 (socket says 172.21.15.46:60286) 2024-04-25T08:42:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7fec5ea00640 1 -- 172.21.15.46:0/140845851 learned_addr learned my addr 172.21.15.46:0/140845851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7fec65e00640 1 -- 172.21.15.46:0/140845851 --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec600cc9d0 con 0x7fec600cc510 2024-04-25T08:42:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7fec65e00640 1 -- 172.21.15.46:0/140845851 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec600d05a0 con 0x7fec60159470 2024-04-25T08:42:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7fec65e00640 1 -- 172.21.15.46:0/140845851 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec600cccb0 con 0x7fec600cc080 2024-04-25T08:42:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7fec5e000640 1 --2- 172.21.15.46:0/140845851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc080 0x7fec60099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.347+0000 7eff41200640 1 --2- 172.21.15.46:0/4110424352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff3404f300 0x7eff340e34b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7eff280097c0 tx=0x7eff2801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea97cae37cedb3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.351+0000 7fec5ea00640 1 -- 172.21.15.46:0/140845851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc080 msgr2=0x7fec60099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.351+0000 7fec5ea00640 1 --2- 172.21.15.46:0/140845851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc080 0x7fec60099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.351+0000 7fec5ea00640 1 -- 172.21.15.46:0/140845851 >> v1:172.21.15.82:6789/0 conn(0x7fec600cc510 legacy=0x7fec600cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.351+0000 7fec5ea00640 1 -- 172.21.15.46:0/140845851 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600ccf90 con 0x7fec60159470 2024-04-25T08:42:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.351+0000 7fec5e000640 1 --2- 172.21.15.46:0/140845851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc080 0x7fec60099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.351+0000 7eff3aa00640 1 -- 172.21.15.46:0/4110424352 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280b0070 con 0x7eff3404f300 2024-04-25T08:42:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.351+0000 7eff3aa00640 1 -- 172.21.15.46:0/4110424352 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff280ab600 con 0x7eff3404f300 2024-04-25T08:42:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.351+0000 7eff3aa00640 1 -- 172.21.15.46:0/4110424352 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff280ab9d0 con 0x7eff3404f300 2024-04-25T08:42:44.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.351+0000 7eff43200640 1 -- 172.21.15.46:0/4110424352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff3404f300 msgr2=0x7eff340e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7eff43200640 1 --2- 172.21.15.46:0/4110424352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff3404f300 0x7eff340e34b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7eff280097c0 tx=0x7eff2801c3e0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7eff43200640 1 -- 172.21.15.46:0/4110424352 shutdown_connections 2024-04-25T08:42:44.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7eff43200640 1 --2- 172.21.15.46:0/4110424352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff3404f300 0x7eff340e34b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7eff43200640 1 --2- 172.21.15.46:0/4110424352 >> v2:172.21.15.46:3300/0 conn(0x7eff340e30d0 0x7eff34005ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7eff43200640 1 -- 172.21.15.46:0/4110424352 >> 172.21.15.46:0/4110424352 conn(0x7eff34009fc0 msgr2=0x7eff3400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7eff43200640 1 -- 172.21.15.46:0/4110424352 shutdown_connections 2024-04-25T08:42:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7eff43200640 1 -- 172.21.15.46:0/4110424352 wait complete. 2024-04-25T08:42:44.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec5ea00640 1 --2- 172.21.15.46:0/140845851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec60159470 0x7fec6015bcc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00dc90 tx=0x7fec4c0206e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9fe9ffeefb7b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec5d600640 1 -- 172.21.15.46:0/140845851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0b0020 con 0x7fec60159470 2024-04-25T08:42:44.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec5d600640 1 -- 172.21.15.46:0/140845851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c00d110 con 0x7fec60159470 2024-04-25T08:42:44.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec5d600640 1 -- 172.21.15.46:0/140845851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c0b0020 con 0x7fec60159470 2024-04-25T08:42:44.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec65e00640 1 -- 172.21.15.46:0/140845851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec60159470 msgr2=0x7fec6015bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec65e00640 1 --2- 172.21.15.46:0/140845851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec60159470 0x7fec6015bcc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00dc90 tx=0x7fec4c0206e0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec65e00640 1 -- 172.21.15.46:0/140845851 shutdown_connections 2024-04-25T08:42:44.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec65e00640 1 --2- 172.21.15.46:0/140845851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc080 0x7fec60099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec65e00640 1 --2- 172.21.15.46:0/140845851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec60159470 0x7fec6015bcc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7fec65e00640 1 -- 172.21.15.46:0/140845851 >> 172.21.15.46:0/140845851 conn(0x7fec600d15e0 msgr2=0x7fec600d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7fec65e00640 1 -- 172.21.15.46:0/140845851 shutdown_connections 2024-04-25T08:42:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7fec65e00640 1 -- 172.21.15.46:0/140845851 wait complete. 2024-04-25T08:42:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7fec65e00640 1 Processor -- start 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.355+0000 7eff43200640 1 Processor -- start 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff43200640 1 -- start start 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff43200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3404f300 0x7eff340e1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff43200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff34054500 0x7eff3404cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff43200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff340e30d0 0x7eff3404a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff3b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff34054500 0x7eff3404cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff3b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff34054500 0x7eff3404cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36100/0 (socket says 172.21.15.46:36100) 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff3be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3404f300 0x7eff340e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff3be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3404f300 0x7eff340e1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50274/0 (socket says 172.21.15.46:50274) 2024-04-25T08:42:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff3be00640 1 -- 172.21.15.46:0/458571463 learned_addr learned my addr 172.21.15.46:0/458571463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7eff41200640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff340e30d0 0x7eff3404a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff43200640 1 -- 172.21.15.46:0/458571463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340342d0 con 0x7eff340e30d0 2024-04-25T08:42:44.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff43200640 1 -- 172.21.15.46:0/458571463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340501b0 con 0x7eff3404f300 2024-04-25T08:42:44.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.359+0000 7fec65e00640 1 -- start start 2024-04-25T08:42:44.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff43200640 1 -- 172.21.15.46:0/458571463 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340e7af0 con 0x7eff34054500 2024-04-25T08:42:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff3be00640 1 -- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff34054500 msgr2=0x7eff3404cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff3be00640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff34054500 0x7eff3404cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff3be00640 1 -- 172.21.15.46:0/458571463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff340e30d0 msgr2=0x7eff3404a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff3be00640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff340e30d0 0x7eff3404a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff3be00640 1 -- 172.21.15.46:0/458571463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3404bd40 con 0x7eff3404f300 2024-04-25T08:42:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec65e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec600cc080 0x7fec600c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff3be00640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3404f300 0x7eff340e1790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff3c050870 tx=0x7eff3c050050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec65e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc510 0x7fec60203510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7f433d000640 1 Processor -- start 2024-04-25T08:42:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff2fe00640 1 -- 172.21.15.46:0/458571463 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c097070 con 0x7eff3404f300 2024-04-25T08:42:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec6016ec10 0x7fec60203a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff2fe00640 1 -- 172.21.15.46:0/458571463 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c073b10 con 0x7eff3404f300 2024-04-25T08:42:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff2fe00640 1 -- 172.21.15.46:0/458571463 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0923f0 con 0x7eff3404f300 2024-04-25T08:42:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff43200640 1 -- 172.21.15.46:0/458571463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3404bfe0 con 0x7eff3404f300 2024-04-25T08:42:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec65e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec60166680 con 0x7fec6016ec10 2024-04-25T08:42:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec65e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec60166500 con 0x7fec600cc080 2024-04-25T08:42:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec5e000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc510 0x7fec60203510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7eff43200640 1 -- 172.21.15.46:0/458571463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3404c5a0 con 0x7eff3404f300 2024-04-25T08:42:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec5e000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc510 0x7fec60203510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36110/0 (socket says 172.21.15.46:36110) 2024-04-25T08:42:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec5e000640 1 -- 172.21.15.46:0/1866342225 learned_addr learned my addr 172.21.15.46:0/1866342225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec60166800 con 0x7fec600cc510 2024-04-25T08:42:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.363+0000 7fec5ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec600cc080 0x7fec600c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f433d000640 1 -- start start 2024-04-25T08:42:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec5e000640 1 -- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec600cc080 msgr2=0x7fec600c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec5e000640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec600cc080 0x7fec600c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec5e000640 1 -- 172.21.15.46:0/1866342225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec6016ec10 msgr2=0x7fec60203a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec5e000640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec6016ec10 0x7fec60203a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec5e000640 1 -- 172.21.15.46:0/1866342225 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600cb090 con 0x7fec600cc510 2024-04-25T08:42:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec5e000640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc510 0x7fec60203510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec5800cf90 tx=0x7fec5800a7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7eff2d600640 1 -- 172.21.15.46:0/458571463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff340052d0 con 0x7eff3404f300 2024-04-25T08:42:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f433d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 0x7f43380ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f433d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4338159a40 0x7f43380ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f433d000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43380cbda0 0x7f43380cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f433d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380aec10 con 0x7f4338159a40 2024-04-25T08:42:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f433d000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4338071a60 con 0x7f43380cbda0 2024-04-25T08:42:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f433d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43380aee40 con 0x7f4338159660 2024-04-25T08:42:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f4337e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43380cbda0 0x7f43380cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f4337e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43380cbda0 0x7f43380cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50292/0 (socket says 172.21.15.46:50292) 2024-04-25T08:42:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f4337e00640 1 -- 172.21.15.46:0/3107730153 learned_addr learned my addr 172.21.15.46:0/3107730153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7f4337400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 0x7f43380ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7eff2fe00640 1 -- 172.21.15.46:0/458571463 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff3c092590 con 0x7eff3404f300 2024-04-25T08:42:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7f4337e00640 1 -- 172.21.15.46:0/3107730153 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 msgr2=0x7f43380ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7f432fe00640 1 --2- 172.21.15.46:0/3107730153 >> v2:172.21.15.46:3300/0 conn(0x7f4338159a40 0x7f43380ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7f4337e00640 1 --2- 172.21.15.46:0/3107730153 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 0x7f43380ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7f4337e00640 1 -- 172.21.15.46:0/3107730153 >> v2:172.21.15.46:3300/0 conn(0x7f4338159a40 msgr2=0x7f43380ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec57400640 1 -- 172.21.15.46:0/1866342225 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5801a070 con 0x7fec600cc510 2024-04-25T08:42:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec600cb370 con 0x7fec600cc510 2024-04-25T08:42:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec600b5090 con 0x7fec600cc510 2024-04-25T08:42:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.367+0000 7fec57400640 1 -- 172.21.15.46:0/1866342225 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5800d840 con 0x7fec600cc510 2024-04-25T08:42:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7fec57400640 1 -- 172.21.15.46:0/1866342225 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5800dc10 con 0x7fec600cc510 2024-04-25T08:42:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7f4337e00640 1 --2- 172.21.15.46:0/3107730153 >> v2:172.21.15.46:3300/0 conn(0x7f4338159a40 0x7f43380ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7f4337e00640 1 -- 172.21.15.46:0/3107730153 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4338071170 con 0x7f43380cbda0 2024-04-25T08:42:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7f432fe00640 1 --2- 172.21.15.46:0/3107730153 >> v2:172.21.15.46:3300/0 conn(0x7f4338159a40 0x7f43380ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec380052a0 con 0x7fec600cc510 2024-04-25T08:42:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7eff2fe00640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff1c066f60 0x7eff1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7eff2fe00640 1 -- 172.21.15.46:0/458571463 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7eff3c0ff480 con 0x7eff3404f300 2024-04-25T08:42:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7ffa11e00640 1 Processor -- start 2024-04-25T08:42:44.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7eff3b400640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff1c066f60 0x7eff1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7ffa11e00640 1 -- start start 2024-04-25T08:42:44.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7eff3b400640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff1c066f60 0x7eff1c069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff34036260 tx=0x7eff3000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.371+0000 7fec57400640 1 -- 172.21.15.46:0/1866342225 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec5801d070 con 0x7fec600cc510 2024-04-25T08:42:44.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa0c0cdb40 0x7ffa0c0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0c06e950 con 0x7ffa0c071750 2024-04-25T08:42:44.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0c0ce020 con 0x7ffa0c161b50 2024-04-25T08:42:44.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0c06eef0 con 0x7ffa0c0cdb40 2024-04-25T08:42:44.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa0c0cdb40 0x7ffa0c0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7fec57400640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec40066f60 0x7fec40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7fec57400640 1 -- 172.21.15.46:0/1866342225 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fec58085bb0 con 0x7fec600cc510 2024-04-25T08:42:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa0c0cdb40 0x7ffa0c0cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60328/0 (socket says 172.21.15.46:60328) 2024-04-25T08:42:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11400640 1 -- 172.21.15.46:0/1284232630 learned_addr learned my addr 172.21.15.46:0/1284232630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa10a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffa0c161b50 0x7ffa0c071040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:35532/0 (socket says 172.21.15.46:35532) 2024-04-25T08:42:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7fec5ea00640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec40066f60 0x7fec40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11400640 1 -- 172.21.15.46:0/1284232630 >> v1:172.21.15.82:6789/0 conn(0x7ffa0c161b50 legacy=0x7ffa0c071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11400640 1 -- 172.21.15.46:0/1284232630 >> v1:172.21.15.46:6789/0 conn(0x7ffa0c071750 legacy=0x7ffa0c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa11400640 1 -- 172.21.15.46:0/1284232630 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c06ec10 con 0x7ffa0c0cdb40 2024-04-25T08:42:44.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7ffa0b400640 1 -- 172.21.15.46:0/1284232630 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283035034 0 0) 0x7ffa0c06e950 con 0x7ffa0c071750 2024-04-25T08:42:44.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7f4337e00640 1 --2- 172.21.15.46:0/3107730153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 0x7f43380cc230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f43380ccf70 tx=0x7f433001c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1571382d22b768f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7f4336a00640 1 -- 172.21.15.46:0/3107730153 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43300a56d0 con 0x7f43380cbda0 2024-04-25T08:42:44.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7f4336a00640 1 -- 172.21.15.46:0/3107730153 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43300a5870 con 0x7f43380cbda0 2024-04-25T08:42:44.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7f4336a00640 1 -- 172.21.15.46:0/3107730153 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43300a56d0 con 0x7f43380cbda0 2024-04-25T08:42:44.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.375+0000 7eff2fe00640 1 -- 172.21.15.46:0/458571463 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff3c08f070 con 0x7eff3404f300 2024-04-25T08:42:44.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7f433d000640 1 -- 172.21.15.46:0/3107730153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 msgr2=0x7f43380cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7f433d000640 1 --2- 172.21.15.46:0/3107730153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 0x7f43380cc230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f43380ccf70 tx=0x7f433001c960 comp rx=0 tx=0).stop 2024-04-25T08:42:44.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7f433d000640 1 -- 172.21.15.46:0/3107730153 shutdown_connections 2024-04-25T08:42:44.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7f433d000640 1 --2- 172.21.15.46:0/3107730153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 0x7f43380cc230 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7f433d000640 1 --2- 172.21.15.46:0/3107730153 >> v2:172.21.15.46:3300/0 conn(0x7f4338159a40 0x7f43380ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7f433d000640 1 --2- 172.21.15.46:0/3107730153 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 0x7f43380ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7f433d000640 1 -- 172.21.15.46:0/3107730153 >> 172.21.15.46:0/3107730153 conn(0x7f4338066b70 msgr2=0x7f433806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7f433d000640 1 -- 172.21.15.46:0/3107730153 shutdown_connections 2024-04-25T08:42:44.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7fec57400640 1 -- 172.21.15.46:0/1866342225 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec58056190 con 0x7fec600cc510 2024-04-25T08:42:44.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7f433d000640 1 -- 172.21.15.46:0/3107730153 wait complete. 2024-04-25T08:42:44.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.379+0000 7ffa11400640 1 --2- 172.21.15.46:0/1284232630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 0x7ffa0c0cd210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c06f070 tx=0x7ffa0401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b417f766550574 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7fec5ea00640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec40066f60 0x7fec40069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0b1000 tx=0x7fec4c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f433d000640 1 Processor -- start 2024-04-25T08:42:44.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f433d000640 1 -- start start 2024-04-25T08:42:44.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa0b400640 1 -- 172.21.15.46:0/1284232630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040a55b0 con 0x7ffa0c0cdb40 2024-04-25T08:42:44.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa0b400640 1 -- 172.21.15.46:0/1284232630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa040a5750 con 0x7ffa0c0cdb40 2024-04-25T08:42:44.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa0b400640 1 -- 172.21.15.46:0/1284232630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040a5b20 con 0x7ffa0c0cdb40 2024-04-25T08:42:44.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa11e00640 1 -- 172.21.15.46:0/1284232630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 msgr2=0x7ffa0c0cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa11e00640 1 --2- 172.21.15.46:0/1284232630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 0x7ffa0c0cd210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c06f070 tx=0x7ffa0401c8f0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa11e00640 1 -- 172.21.15.46:0/1284232630 shutdown_connections 2024-04-25T08:42:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa11e00640 1 --2- 172.21.15.46:0/1284232630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 0x7ffa0c0cd210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa11e00640 1 -- 172.21.15.46:0/1284232630 >> 172.21.15.46:0/1284232630 conn(0x7ffa0c066fa0 msgr2=0x7ffa0c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa11e00640 1 -- 172.21.15.46:0/1284232630 shutdown_connections 2024-04-25T08:42:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f433d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 0x7f4338155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f433d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 0x7f433815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f433d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4338159a40 0x7f4338154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f433d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433815ddb0 con 0x7f4338159a40 2024-04-25T08:42:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f433d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433815dc30 con 0x7f43380cbda0 2024-04-25T08:42:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f433d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433815df30 con 0x7f4338159660 2024-04-25T08:42:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7ffa11e00640 1 -- 172.21.15.46:0/1284232630 wait complete. 2024-04-25T08:42:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f4337e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4338159a40 0x7f4338154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f4337400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 0x7f4338155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f4337400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 0x7f4338155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50306/0 (socket says 172.21.15.46:50306) 2024-04-25T08:42:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.383+0000 7f4337400640 1 -- 172.21.15.46:0/525702753 learned_addr learned my addr 172.21.15.46:0/525702753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f432fe00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 0x7f433815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f4337400640 1 -- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 msgr2=0x7f433815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f4337400640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 0x7f433815e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f4337400640 1 -- 172.21.15.46:0/525702753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4338159a40 msgr2=0x7f4338154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f4337400640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4338159a40 0x7f4338154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f4337400640 1 -- 172.21.15.46:0/525702753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4338161260 con 0x7f43380cbda0 2024-04-25T08:42:44.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f432fe00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 0x7f433815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:44.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f4337400640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 0x7f4338155b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4328007ab0 tx=0x7f4328007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f432f400640 1 -- 172.21.15.46:0/525702753 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328019070 con 0x7f43380cbda0 2024-04-25T08:42:44.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f432f400640 1 -- 172.21.15.46:0/525702753 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432800aaa0 con 0x7f43380cbda0 2024-04-25T08:42:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f432f400640 1 -- 172.21.15.46:0/525702753 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328014410 con 0x7f43380cbda0 2024-04-25T08:42:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f433d000640 1 -- 172.21.15.46:0/525702753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4338161540 con 0x7f43380cbda0 2024-04-25T08:42:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7f433d000640 1 -- 172.21.15.46:0/525702753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4338161b00 con 0x7f43380cbda0 2024-04-25T08:42:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7ffa11e00640 1 Processor -- start 2024-04-25T08:42:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.387+0000 7ffa11e00640 1 -- start start 2024-04-25T08:42:44.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa0c071750 0x7ffa0c1553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 0x7ffa0c15cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0c161b50 0x7ffa0c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c16fba0 con 0x7ffa0c0cdb40 2024-04-25T08:42:44.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c16fa20 con 0x7ffa0c161b50 2024-04-25T08:42:44.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c16fd20 con 0x7ffa0c071750 2024-04-25T08:42:44.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0c161b50 0x7ffa0c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0c161b50 0x7ffa0c15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50310/0 (socket says 172.21.15.46:50310) 2024-04-25T08:42:44.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11400640 1 -- 172.21.15.46:0/4075298900 learned_addr learned my addr 172.21.15.46:0/4075298900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7f432f400640 1 -- 172.21.15.46:0/525702753 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f432801c070 con 0x7f43380cbda0 2024-04-25T08:42:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa0be00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 0x7ffa0c15cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa10a00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa0c071750 0x7ffa0c1553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11400640 1 -- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa0c071750 msgr2=0x7ffa0c1553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11400640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa0c071750 0x7ffa0c1553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7f432f400640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4314066f60 0x7f4314069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11400640 1 -- 172.21.15.46:0/4075298900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 msgr2=0x7ffa0c15cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11400640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 0x7ffa0c15cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa11400640 1 -- 172.21.15.46:0/4075298900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c1f57d0 con 0x7ffa0c161b50 2024-04-25T08:42:44.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7ffa10a00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa0c071750 0x7ffa0c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:44.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.395+0000 7ffa0be00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 0x7ffa0c15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:44.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.391+0000 7f432f400640 1 -- 172.21.15.46:0/525702753 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4328084bd0 con 0x7f43380cbda0 2024-04-25T08:42:44.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.395+0000 7f432fe00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4314066f60 0x7f4314069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.395+0000 7f432cc00640 1 -- 172.21.15.46:0/525702753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43080052a0 con 0x7f43380cbda0 2024-04-25T08:42:44.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.395+0000 7ffa11400640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0c161b50 0x7ffa0c15d120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa040af040 tx=0x7ffa04019690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.395+0000 7f432fe00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4314066f60 0x7f4314069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f433814bdc0 tx=0x7f4320008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.399+0000 7ffa08c00640 1 -- 172.21.15.46:0/4075298900 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040bc020 con 0x7ffa0c161b50 2024-04-25T08:42:44.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.399+0000 7ffa08c00640 1 -- 172.21.15.46:0/4075298900 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa040b88a0 con 0x7ffa0c161b50 2024-04-25T08:42:44.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.399+0000 7ffa08c00640 1 -- 172.21.15.46:0/4075298900 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040b8c90 con 0x7ffa0c161b50 2024-04-25T08:42:44.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.399+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c15b4f0 con 0x7ffa0c161b50 2024-04-25T08:42:44.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.399+0000 7f432f400640 1 -- 172.21.15.46:0/525702753 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4328084e20 con 0x7f43380cbda0 2024-04-25T08:42:44.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.399+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c15bab0 con 0x7ffa0c161b50 2024-04-25T08:42:44.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.403+0000 7ffa08c00640 1 -- 172.21.15.46:0/4075298900 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa040bf020 con 0x7ffa0c161b50 2024-04-25T08:42:44.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.403+0000 7ffa08c00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9ec066f60 0x7ff9ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.403+0000 7ffa08c00640 1 -- 172.21.15.46:0/4075298900 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffa041208c0 con 0x7ffa0c161b50 2024-04-25T08:42:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.407+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c069010 con 0x7ffa0c161b50 2024-04-25T08:42:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.407+0000 7ffa10a00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9ec066f60 0x7ff9ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.407+0000 7ffa10a00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9ec066f60 0x7ff9ec069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000f70 tx=0x7ff9fc007e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.411+0000 7ffa08c00640 1 -- 172.21.15.46:0/4075298900 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa04016600 con 0x7ffa0c161b50 2024-04-25T08:42:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.435+0000 7f54d1600640 1 Processor -- start 2024-04-25T08:42:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.435+0000 7f44b9000640 1 Processor -- start 2024-04-25T08:42:44.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.435+0000 7f44b9000640 1 -- start start 2024-04-25T08:42:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.435+0000 7f44b9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b415e220 0x7f44b4150b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.435+0000 7f44b9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b406f260 con 0x7f44b4072260 2024-04-25T08:42:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b9000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b406f810 con 0x7f44b415d5d0 2024-04-25T08:42:44.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b4160950 con 0x7f44b415e220 2024-04-25T08:42:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.435+0000 7f54d1600640 1 -- start start 2024-04-25T08:42:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54d1600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54cc1654c0 0x7f54cc163400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54d1600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f54cc162330 0x7f54cc06e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54d1600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc06e9d0 con 0x7f54cc06d270 2024-04-25T08:42:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54d1600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc06ec70 con 0x7f54cc162330 2024-04-25T08:42:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54d1600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc06ef50 con 0x7f54cc1654c0 2024-04-25T08:42:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54caa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54cc1654c0 0x7f54cc163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54caa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54cc1654c0 0x7f54cc163400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50324/0 (socket says 172.21.15.46:50324) 2024-04-25T08:42:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54caa00640 1 -- 172.21.15.46:0/2504583261 learned_addr learned my addr 172.21.15.46:0/2504583261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b2800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f44b415d5d0 0x7f44b40ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:35534/0 (socket says 172.21.15.46:35534) 2024-04-25T08:42:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b2800640 1 -- 172.21.15.46:0/3578155299 learned_addr learned my addr 172.21.15.46:0/3578155299 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b1e00640 1 --2- 172.21.15.46:0/3578155299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b415e220 0x7f44b4150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b0a00640 1 -- 172.21.15.46:0/3578155299 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846144931 0 0) 0x7f44b406f810 con 0x7f44b415d5d0 2024-04-25T08:42:44.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b0a00640 1 -- 172.21.15.46:0/3578155299 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4498003530 con 0x7f44b415d5d0 2024-04-25T08:42:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54ca000640 1 --2- 172.21.15.46:0/2504583261 >> v2:172.21.15.46:3300/0 conn(0x7f54cc162330 0x7f54cc06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54caa00640 1 -- 172.21.15.46:0/2504583261 >> v2:172.21.15.46:3300/0 conn(0x7f54cc162330 msgr2=0x7f54cc06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54caa00640 1 --2- 172.21.15.46:0/2504583261 >> v2:172.21.15.46:3300/0 conn(0x7f54cc162330 0x7f54cc06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54caa00640 1 -- 172.21.15.46:0/2504583261 >> v1:172.21.15.46:6789/0 conn(0x7f54cc06d270 legacy=0x7f54cc0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54caa00640 1 -- 172.21.15.46:0/2504583261 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc06f230 con 0x7f54cc1654c0 2024-04-25T08:42:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f54ca000640 1 --2- 172.21.15.46:0/2504583261 >> v2:172.21.15.46:3300/0 conn(0x7f54cc162330 0x7f54cc06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b1e00640 1 -- 172.21.15.46:0/3578155299 >> v1:172.21.15.82:6789/0 conn(0x7f44b415d5d0 legacy=0x7f44b40ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b1e00640 1 -- 172.21.15.46:0/3578155299 >> v1:172.21.15.46:6789/0 conn(0x7f44b4072260 legacy=0x7f44b406eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.439+0000 7f44b1e00640 1 -- 172.21.15.46:0/3578155299 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b406faf0 con 0x7f44b415e220 2024-04-25T08:42:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54caa00640 1 --2- 172.21.15.46:0/2504583261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc1654c0 0x7f54cc163400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54cc070b00 tx=0x7f54b801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8a8dfea4b698a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b1e00640 1 --2- 172.21.15.46:0/3578155299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b415e220 0x7f44b4150b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44a40097f0 tx=0x7f44a401ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9332fd6ba4aaae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54c9600640 1 -- 172.21.15.46:0/2504583261 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80b1070 con 0x7f54cc1654c0 2024-04-25T08:42:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b0a00640 1 -- 172.21.15.46:0/3578155299 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a40b5070 con 0x7f44b415e220 2024-04-25T08:42:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b0a00640 1 -- 172.21.15.46:0/3578155299 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a40a8a10 con 0x7f44b415e220 2024-04-25T08:42:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b0a00640 1 -- 172.21.15.46:0/3578155299 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a40a8de0 con 0x7f44b415e220 2024-04-25T08:42:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54c9600640 1 -- 172.21.15.46:0/2504583261 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b80ac640 con 0x7f54cc1654c0 2024-04-25T08:42:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54c9600640 1 -- 172.21.15.46:0/2504583261 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80aca50 con 0x7f54cc1654c0 2024-04-25T08:42:44.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b9000640 1 -- 172.21.15.46:0/3578155299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b415e220 msgr2=0x7f44b4150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b9000640 1 --2- 172.21.15.46:0/3578155299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b415e220 0x7f44b4150b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44a40097f0 tx=0x7f44a401ed10 comp rx=0 tx=0).stop 2024-04-25T08:42:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b9000640 1 -- 172.21.15.46:0/3578155299 shutdown_connections 2024-04-25T08:42:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b9000640 1 --2- 172.21.15.46:0/3578155299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b415e220 0x7f44b4150b30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b9000640 1 -- 172.21.15.46:0/3578155299 >> 172.21.15.46:0/3578155299 conn(0x7f44b40b12a0 msgr2=0x7f44b40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54d1600640 1 -- 172.21.15.46:0/2504583261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc1654c0 msgr2=0x7f54cc163400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54d1600640 1 --2- 172.21.15.46:0/2504583261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc1654c0 0x7f54cc163400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f54cc070b00 tx=0x7f54b801c540 comp rx=0 tx=0).stop 2024-04-25T08:42:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b9000640 1 -- 172.21.15.46:0/3578155299 shutdown_connections 2024-04-25T08:42:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f44b9000640 1 -- 172.21.15.46:0/3578155299 wait complete. 2024-04-25T08:42:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54d1600640 1 -- 172.21.15.46:0/2504583261 shutdown_connections 2024-04-25T08:42:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54d1600640 1 --2- 172.21.15.46:0/2504583261 >> v2:172.21.15.46:3300/0 conn(0x7f54cc162330 0x7f54cc06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54d1600640 1 --2- 172.21.15.46:0/2504583261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc1654c0 0x7f54cc163400 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54d1600640 1 -- 172.21.15.46:0/2504583261 >> 172.21.15.46:0/2504583261 conn(0x7f54cc0b12a0 msgr2=0x7f54cc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54d1600640 1 -- 172.21.15.46:0/2504583261 shutdown_connections 2024-04-25T08:42:44.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.451+0000 7f44b9000640 1 Processor -- start 2024-04-25T08:42:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.447+0000 7f54d1600640 1 -- 172.21.15.46:0/2504583261 wait complete. 2024-04-25T08:42:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.451+0000 7f54d1600640 1 Processor -- start 2024-04-25T08:42:44.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.451+0000 7f54d1600640 1 -- start start 2024-04-25T08:42:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.451+0000 7f54d1600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54cc06d270 0x7f54cc16b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54d1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54cc162330 0x7f54cc166a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54d1600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc0b34e0 0x7f54cc166f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54ca000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54cc162330 0x7f54cc166a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54caa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54cc06d270 0x7f54cc16b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54ca000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54cc162330 0x7f54cc166a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60376/0 (socket says 172.21.15.46:60376) 2024-04-25T08:42:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54cb400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc0b34e0 0x7f54cc166f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54cb400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc0b34e0 0x7f54cc166f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50334/0 (socket says 172.21.15.46:50334) 2024-04-25T08:42:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54caa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54cc06d270 0x7f54cc16b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36158/0 (socket says 172.21.15.46:36158) 2024-04-25T08:42:44.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f44b9000640 1 -- start start 2024-04-25T08:42:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc16ce90 con 0x7f54cc162330 2024-04-25T08:42:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54ca000640 1 -- 172.21.15.46:0/818151238 learned_addr learned my addr 172.21.15.46:0/818151238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc16cd10 con 0x7f54cc0b34e0 2024-04-25T08:42:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc16d010 con 0x7f54cc06d270 2024-04-25T08:42:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f44b9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b4072260 0x7f44b4152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f44b9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b415d5d0 0x7f44b414fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f44b9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b41f7d10 0x7f44b41f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f44b9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40b25a0 con 0x7f44b4072260 2024-04-25T08:42:44.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f44b1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b4072260 0x7f44b4152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f44b1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b4072260 0x7f44b4152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60388/0 (socket says 172.21.15.46:60388) 2024-04-25T08:42:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.455+0000 7f44b1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b415d5d0 0x7f44b414fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b9000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40cdd40 con 0x7f44b415d5d0 2024-04-25T08:42:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b1e00640 1 -- 172.21.15.46:0/2787314563 learned_addr learned my addr 172.21.15.46:0/2787314563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b2800640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b41f7d10 0x7f44b41f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b415da60 con 0x7f44b41f7d10 2024-04-25T08:42:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f54ca000640 1 -- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54cc06d270 msgr2=0x7f54cc16b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f54ca000640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54cc06d270 0x7f54cc16b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f54ca000640 1 -- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc0b34e0 msgr2=0x7f54cc166f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f54ca000640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc0b34e0 0x7f54cc166f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f54ca000640 1 -- 172.21.15.46:0/818151238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc16c8b0 con 0x7f54cc162330 2024-04-25T08:42:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b2800640 1 -- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b415d5d0 msgr2=0x7f44b414fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b2800640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b415d5d0 0x7f44b414fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b2800640 1 -- 172.21.15.46:0/2787314563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b4072260 msgr2=0x7f44b4152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b2800640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b4072260 0x7f44b4152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b2800640 1 -- 172.21.15.46:0/2787314563 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b41543c0 con 0x7f44b41f7d10 2024-04-25T08:42:44.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b2800640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b41f7d10 0x7f44b41f81c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00cab0 tx=0x7f44ac00cf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f54ca000640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54cc162330 0x7f54cc166a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f54c400cf90 tx=0x7f54c400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b1e00640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b4072260 0x7f44b4152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:44.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f54c2a00640 1 -- 172.21.15.46:0/818151238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c401a070 con 0x7f54cc162330 2024-04-25T08:42:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44aa000640 1 -- 172.21.15.46:0/2787314563 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac017070 con 0x7f44b41f7d10 2024-04-25T08:42:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b4141360 con 0x7f44b41f7d10 2024-04-25T08:42:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b4152e00 con 0x7f44b41f7d10 2024-04-25T08:42:44.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f44aa000640 1 -- 172.21.15.46:0/2787314563 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac00ac30 con 0x7f44b41f7d10 2024-04-25T08:42:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f54c2a00640 1 -- 172.21.15.46:0/818151238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c400d840 con 0x7f54cc162330 2024-04-25T08:42:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.459+0000 7f54c2a00640 1 -- 172.21.15.46:0/818151238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c400dc10 con 0x7f54cc162330 2024-04-25T08:42:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.463+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc0b3a20 con 0x7f54cc162330 2024-04-25T08:42:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.463+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc0b3fe0 con 0x7f54cc162330 2024-04-25T08:42:44.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.463+0000 7f44aa000640 1 -- 172.21.15.46:0/2787314563 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac017070 con 0x7f44b41f7d10 2024-04-25T08:42:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.463+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448c0052a0 con 0x7f44b41f7d10 2024-04-25T08:42:44.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.463+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc069080 con 0x7f54cc162330 2024-04-25T08:42:44.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.467+0000 7f44aa000640 1 -- 172.21.15.46:0/2787314563 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44ac007070 con 0x7f44b41f7d10 2024-04-25T08:42:44.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.467+0000 7f44aa000640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4494067390 0x7f4494069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.467+0000 7f44aa000640 1 -- 172.21.15.46:0/2787314563 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f44ac00aa30 con 0x7f44b41f7d10 2024-04-25T08:42:44.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.467+0000 7f54c2a00640 1 -- 172.21.15.46:0/818151238 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54c401d070 con 0x7f54cc162330 2024-04-25T08:42:44.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.467+0000 7f54c2a00640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f54ac066fb0 0x7f54ac069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.467+0000 7f54caa00640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f54ac066fb0 0x7f54ac069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.467+0000 7f54caa00640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f54ac066fb0 0x7f54ac069470 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54b80ae000 tx=0x7f54b80a6070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.471+0000 7f54c2a00640 1 -- 172.21.15.46:0/818151238 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f54c4054210 con 0x7f54cc162330 2024-04-25T08:42:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.467+0000 7f44b1e00640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4494067390 0x7f4494069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.471+0000 7f44b1e00640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4494067390 0x7f4494069850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009740 tx=0x7f44a40aa070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.475+0000 7f44aa000640 1 -- 172.21.15.46:0/2787314563 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44ac054f90 con 0x7f44b41f7d10 2024-04-25T08:42:44.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f20e00640 1 Processor -- start 2024-04-25T08:42:44.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f20e00640 1 -- start start 2024-04-25T08:42:44.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f54c2a00640 1 -- 172.21.15.46:0/818151238 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54c4056170 con 0x7f54cc162330 2024-04-25T08:42:44.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f20e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f14039df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f20e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f14050d50 0x7f5f1403a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f20e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1404f620 con 0x7f5f14038640 2024-04-25T08:42:44.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f20e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1403a870 con 0x7f5f14050d50 2024-04-25T08:42:44.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f20e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1404f900 con 0x7f5f140e0ef0 2024-04-25T08:42:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f1b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f14039df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f1b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f14039df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36178/0 (socket says 172.21.15.46:36178) 2024-04-25T08:42:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.483+0000 7f5f1b400640 1 -- 172.21.15.46:0/3550356315 learned_addr learned my addr 172.21.15.46:0/3550356315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.487+0000 7f5f1aa00640 1 --2- 172.21.15.46:0/3550356315 >> v2:172.21.15.82:3300/0 conn(0x7f5f14050d50 0x7f5f1403a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.487+0000 7f5f1a000640 1 -- 172.21.15.46:0/3550356315 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581484350 0 0) 0x7f5f1404f620 con 0x7f5f14038640 2024-04-25T08:42:44.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.487+0000 7f5f1a000640 1 -- 172.21.15.46:0/3550356315 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f04003530 con 0x7f5f14038640 2024-04-25T08:42:44.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.487+0000 7f5f1b400640 1 -- 172.21.15.46:0/3550356315 >> v2:172.21.15.82:3300/0 conn(0x7f5f14050d50 msgr2=0x7f5f1403a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:44.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.487+0000 7f5f1b400640 1 --2- 172.21.15.46:0/3550356315 >> v2:172.21.15.82:3300/0 conn(0x7f5f14050d50 0x7f5f1403a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.487+0000 7f5f1b400640 1 -- 172.21.15.46:0/3550356315 >> v1:172.21.15.82:6789/0 conn(0x7f5f14038640 legacy=0x7f5f14005ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.487+0000 7f5f1b400640 1 -- 172.21.15.46:0/3550356315 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1404fbe0 con 0x7f5f140e0ef0 2024-04-25T08:42:44.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.491+0000 7f5f1aa00640 1 --2- 172.21.15.46:0/3550356315 >> v2:172.21.15.82:3300/0 conn(0x7f5f14050d50 0x7f5f1403a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:44.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.491+0000 7f5f1b400640 1 --2- 172.21.15.46:0/3550356315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f14039df0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009870 tx=0x7f5f0801c570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1557c993978716ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.491+0000 7f5f1a000640 1 -- 172.21.15.46:0/3550356315 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080a9630 con 0x7f5f140e0ef0 2024-04-25T08:42:44.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.491+0000 7f5f1a000640 1 -- 172.21.15.46:0/3550356315 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f080a97d0 con 0x7f5f140e0ef0 2024-04-25T08:42:44.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.491+0000 7f5f1a000640 1 -- 172.21.15.46:0/3550356315 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f080a9be0 con 0x7f5f140e0ef0 2024-04-25T08:42:44.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.495+0000 7f5f20e00640 1 -- 172.21.15.46:0/3550356315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 msgr2=0x7f5f14039df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.495+0000 7f5f20e00640 1 --2- 172.21.15.46:0/3550356315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f14039df0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5f08009870 tx=0x7f5f0801c570 comp rx=0 tx=0).stop 2024-04-25T08:42:44.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.495+0000 7f5f20e00640 1 -- 172.21.15.46:0/3550356315 shutdown_connections 2024-04-25T08:42:44.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.495+0000 7f5f20e00640 1 --2- 172.21.15.46:0/3550356315 >> v2:172.21.15.82:3300/0 conn(0x7f5f14050d50 0x7f5f1403a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.495+0000 7f5f20e00640 1 --2- 172.21.15.46:0/3550356315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f14039df0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.495+0000 7f5f20e00640 1 -- 172.21.15.46:0/3550356315 >> 172.21.15.46:0/3550356315 conn(0x7f5f14009fc0 msgr2=0x7f5f1400a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.495+0000 7f5f20e00640 1 -- 172.21.15.46:0/3550356315 shutdown_connections 2024-04-25T08:42:44.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.495+0000 7f5f20e00640 1 -- 172.21.15.46:0/3550356315 wait complete. 2024-04-25T08:42:44.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.499+0000 7f5f20e00640 1 Processor -- start 2024-04-25T08:42:44.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.499+0000 7f5f20e00640 1 -- start start 2024-04-25T08:42:44.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.499+0000 7f5f20e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f14038640 0x7f5f140e3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.499+0000 7f5f20e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f14050d50 0x7f5f1404a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f20e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f141881d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f141881d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f141881d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36194/0 (socket says 172.21.15.46:36194) 2024-04-25T08:42:44.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f14050d50 0x7f5f1404a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f14050d50 0x7f5f1404a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50374/0 (socket says 172.21.15.46:50374) 2024-04-25T08:42:44.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1be00640 1 -- 172.21.15.46:0/4275827509 learned_addr learned my addr 172.21.15.46:0/4275827509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:44.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f20e00640 1 -- 172.21.15.46:0/4275827509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14006e10 con 0x7f5f14038640 2024-04-25T08:42:44.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f20e00640 1 -- 172.21.15.46:0/4275827509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1404f2a0 con 0x7f5f14050d50 2024-04-25T08:42:44.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1b400640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f14038640 0x7f5f140e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f20e00640 1 -- 172.21.15.46:0/4275827509 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f14006f90 con 0x7f5f140e0ef0 2024-04-25T08:42:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1aa00640 1 -- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 msgr2=0x7f5f141881d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1aa00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f141881d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1aa00640 1 -- 172.21.15.46:0/4275827509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f14038640 msgr2=0x7f5f140e3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1aa00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f14038640 0x7f5f140e3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1aa00640 1 -- 172.21.15.46:0/4275827509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f140e2a90 con 0x7f5f14050d50 2024-04-25T08:42:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1be00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f141881d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:44.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1b400640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f14038640 0x7f5f140e3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:44.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.503+0000 7f5f1aa00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f14050d50 0x7f5f1404a860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000e9b0 tx=0x7f5f1000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.507+0000 7f5f0f400640 1 -- 172.21.15.46:0/4275827509 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f10017070 con 0x7f5f14050d50 2024-04-25T08:42:44.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.507+0000 7f5f0f400640 1 -- 172.21.15.46:0/4275827509 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1000c9a0 con 0x7f5f14050d50 2024-04-25T08:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.507+0000 7f5f0f400640 1 -- 172.21.15.46:0/4275827509 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1000cd90 con 0x7f5f14050d50 2024-04-25T08:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.507+0000 7f5f20e00640 1 -- 172.21.15.46:0/4275827509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1404b120 con 0x7f5f14050d50 2024-04-25T08:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.507+0000 7f5f20e00640 1 -- 172.21.15.46:0/4275827509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f140e2d30 con 0x7f5f14050d50 2024-04-25T08:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.511+0000 7f5f0f400640 1 -- 172.21.15.46:0/4275827509 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f1001a070 con 0x7f5f14050d50 2024-04-25T08:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.511+0000 7f5f0cc00640 1 -- 172.21.15.46:0/4275827509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef00052a0 con 0x7f5f14050d50 2024-04-25T08:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.511+0000 7f5f0f400640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5efc066f60 0x7f5efc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:44.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.511+0000 7f5f0f400640 1 -- 172.21.15.46:0/4275827509 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5f10053630 con 0x7f5f14050d50 2024-04-25T08:42:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.511+0000 7f5f1b400640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5efc066f60 0x7f5efc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.511+0000 7f5f1b400640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5efc066f60 0x7f5efc069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5f080ae000 tx=0x7f5f080b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:44.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.515+0000 7f5f0f400640 1 -- 172.21.15.46:0/4275827509 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f10055590 con 0x7f5f14050d50 2024-04-25T08:42:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.651+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5dd00cce40 con 0x7f5dd015d8d0 2024-04-25T08:42:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dccc00640 1 -- 172.21.15.46:0/724567121 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5dbc055e50 con 0x7f5dd015d8d0 2024-04-25T08:42:44.647 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066fe0 msgr2=0x7f5db00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066fe0 0x7f5db00694a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00ad000 tx=0x7f5dc00a6000 comp rx=0 tx=0).stop 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5dd015d8d0 msgr2=0x7f5dd014df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5dd015d8d0 0x7f5dd014df80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00b410 tx=0x7f5dbc00b8d0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 shutdown_connections 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066fe0 0x7f5db00694a0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5dc00ad000 tx=0x7f5dc00a6000 comp rx=0 tx=0).stop 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5dd015d8d0 0x7f5dd014df80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd006ec30 0x7f5dd014ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 --2- 172.21.15.46:0/724567121 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd006e7d0 0x7f5dd014bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 >> 172.21.15.46:0/724567121 conn(0x7f5dd0066b70 msgr2=0x7f5dd0152080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 shutdown_connections 2024-04-25T08:42:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.655+0000 7f5dd6200640 1 -- 172.21.15.46:0/724567121 wait complete. 2024-04-25T08:42:44.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-04-25T08:42:44.690 DEBUG:teuthology.parallel:result is None 2024-04-25T08:42:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.711+0000 7f432cc00640 1 -- 172.21.15.46:0/525702753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4308005590 con 0x7f43380cbda0 2024-04-25T08:42:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.711+0000 7f432f400640 1 -- 172.21.15.46:0/525702753 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4328084e20 con 0x7f43380cbda0 2024-04-25T08:42:44.706 INFO:teuthology.orchestra.run.smithi046.stdout:55834574858 2024-04-25T08:42:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 -- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4314066f60 msgr2=0x7f4314069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4314066f60 0x7f4314069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f433814bdc0 tx=0x7f4320008040 comp rx=0 tx=0).stop 2024-04-25T08:42:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 -- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 msgr2=0x7f4338155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 0x7f4338155b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4328007ab0 tx=0x7f4328007f70 comp rx=0 tx=0).stop 2024-04-25T08:42:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 -- 172.21.15.46:0/525702753 shutdown_connections 2024-04-25T08:42:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4314066f60 0x7f4314069420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4338159a40 0x7f4338154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4338159660 0x7f433815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 --2- 172.21.15.46:0/525702753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43380cbda0 0x7f4338155b60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 -- 172.21.15.46:0/525702753 >> 172.21.15.46:0/525702753 conn(0x7f4338066b70 msgr2=0x7f43380af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 -- 172.21.15.46:0/525702753 shutdown_connections 2024-04-25T08:42:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.715+0000 7f432cc00640 1 -- 172.21.15.46:0/525702753 wait complete. 2024-04-25T08:42:44.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.723+0000 7eff2d600640 1 -- 172.21.15.46:0/458571463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7eff34052830 con 0x7eff3404f300 2024-04-25T08:42:44.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.723+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fec38005590 con 0x7fec600cc510 2024-04-25T08:42:44.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.723+0000 7fec57400640 1 -- 172.21.15.46:0/1866342225 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fec5805a230 con 0x7fec600cc510 2024-04-25T08:42:44.717 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-25T08:42:44.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.723+0000 7eff2fe00640 1 -- 172.21.15.46:0/458571463 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff3c0cfb90 con 0x7eff3404f300 2024-04-25T08:42:44.718 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:44.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 -- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff1c066f60 msgr2=0x7eff1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff1c066f60 0x7eff1c069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7eff34036260 tx=0x7eff3000b040 comp rx=0 tx=0).stop 2024-04-25T08:42:44.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec40066f60 msgr2=0x7fec40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec40066f60 0x7fec40069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0b1000 tx=0x7fec4c009040 comp rx=0 tx=0).stop 2024-04-25T08:42:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc510 msgr2=0x7fec60203510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc510 0x7fec60203510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec5800cf90 tx=0x7fec5800a7d0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 -- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3404f300 msgr2=0x7eff340e1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3404f300 0x7eff340e1790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7eff3c050870 tx=0x7eff3c050050 comp rx=0 tx=0).stop 2024-04-25T08:42:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 shutdown_connections 2024-04-25T08:42:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec40066f60 0x7fec40069420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec6016ec10 0x7fec60203a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fec600cc510 0x7fec60203510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 --2- 172.21.15.46:0/1866342225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec600cc080 0x7fec600c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 >> 172.21.15.46:0/1866342225 conn(0x7fec600d15e0 msgr2=0x7fec600b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 -- 172.21.15.46:0/458571463 shutdown_connections 2024-04-25T08:42:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff1c066f60 0x7eff1c069420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff340e30d0 0x7eff3404a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff34054500 0x7eff3404cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 --2- 172.21.15.46:0/458571463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3404f300 0x7eff340e1790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 -- 172.21.15.46:0/458571463 >> 172.21.15.46:0/458571463 conn(0x7eff34009fc0 msgr2=0x7eff340e34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 shutdown_connections 2024-04-25T08:42:44.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7fec65e00640 1 -- 172.21.15.46:0/1866342225 wait complete. 2024-04-25T08:42:44.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 -- 172.21.15.46:0/458571463 shutdown_connections 2024-04-25T08:42:44.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.727+0000 7eff2d600640 1 -- 172.21.15.46:0/458571463 wait complete. 2024-04-25T08:42:44.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-25T08:42:44.737 DEBUG:teuthology.parallel:result is None 2024-04-25T08:42:44.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-25T08:42:44.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.759+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ffa0c068720 con 0x7ffa0c161b50 2024-04-25T08:42:44.751 DEBUG:teuthology.parallel:result is None 2024-04-25T08:42:44.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.759+0000 7ffa08c00640 1 -- 172.21.15.46:0/4075298900 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffa040f1610 con 0x7ffa0c161b50 2024-04-25T08:42:44.751 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:44.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2024-04-25T08:42:44.755 DEBUG:teuthology.parallel:result is None 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9ec066f60 msgr2=0x7ff9ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9ec066f60 0x7ff9ec069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000f70 tx=0x7ff9fc007e90 comp rx=0 tx=0).stop 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0c161b50 msgr2=0x7ffa0c15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0c161b50 0x7ffa0c15d120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffa040af040 tx=0x7ffa04019690 comp rx=0 tx=0).stop 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 shutdown_connections 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9ec066f60 0x7ff9ec069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0c161b50 0x7ffa0c15d120 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0c0cdb40 0x7ffa0c15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 --2- 172.21.15.46:0/4075298900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa0c071750 0x7ffa0c1553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 >> 172.21.15.46:0/4075298900 conn(0x7ffa0c066fa0 msgr2=0x7ffa0c071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 shutdown_connections 2024-04-25T08:42:44.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.763+0000 7ffa11e00640 1 -- 172.21.15.46:0/4075298900 wait complete. 2024-04-25T08:42:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.779+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f448c005590 con 0x7f44b41f7d10 2024-04-25T08:42:44.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.779+0000 7f44aa000640 1 -- 172.21.15.46:0/2787314563 <== mon.2 v2:172.21.15.82:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44ac059030 con 0x7f44b41f7d10 2024-04-25T08:42:44.774 INFO:teuthology.orchestra.run.smithi046.stdout:55834574857 2024-04-25T08:42:44.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-04-25T08:42:44.775 DEBUG:teuthology.parallel:result is None 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4494067390 msgr2=0x7f4494069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4494067390 0x7f4494069850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009740 tx=0x7f44a40aa070 comp rx=0 tx=0).stop 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b41f7d10 msgr2=0x7f44b41f81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b41f7d10 0x7f44b41f81c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00cab0 tx=0x7f44ac00cf70 comp rx=0 tx=0).stop 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 shutdown_connections 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4494067390 0x7f4494069850 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44b41f7d10 0x7f44b41f81c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44b415d5d0 0x7f44b414fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 --2- 172.21.15.46:0/2787314563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44b4072260 0x7f44b4152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 >> 172.21.15.46:0/2787314563 conn(0x7f44b40b12a0 msgr2=0x7f44b415e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 shutdown_connections 2024-04-25T08:42:44.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.783+0000 7f44b9000640 1 -- 172.21.15.46:0/2787314563 wait complete. 2024-04-25T08:42:44.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-04-25T08:42:44.796 DEBUG:teuthology.parallel:result is None 2024-04-25T08:42:44.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.835+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f54cc161e70 con 0x7f54cc162330 2024-04-25T08:42:44.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.835+0000 7f54c2a00640 1 -- 172.21.15.46:0/818151238 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54c405a210 con 0x7f54cc162330 2024-04-25T08:42:44.829 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-25T08:42:44.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f54ac066fb0 msgr2=0x7f54ac069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f54ac066fb0 0x7f54ac069470 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54b80ae000 tx=0x7f54b80a6070 comp rx=0 tx=0).stop 2024-04-25T08:42:44.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54cc162330 msgr2=0x7f54cc166a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54cc162330 0x7f54cc166a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f54c400cf90 tx=0x7f54c400a7d0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 shutdown_connections 2024-04-25T08:42:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f54ac066fb0 0x7f54ac069470 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54cc0b34e0 0x7f54cc166f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54cc162330 0x7f54cc166a30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 --2- 172.21.15.46:0/818151238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54cc06d270 0x7f54cc16b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 >> 172.21.15.46:0/818151238 conn(0x7f54cc0b12a0 msgr2=0x7f54cc16d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 shutdown_connections 2024-04-25T08:42:44.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.839+0000 7f54d1600640 1 -- 172.21.15.46:0/818151238 wait complete. 2024-04-25T08:42:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.851+0000 7f5f0cc00640 1 -- 172.21.15.46:0/4275827509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5ef0005860 con 0x7f5f14050d50 2024-04-25T08:42:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.851+0000 7f5f0f400640 1 -- 172.21.15.46:0/4275827509 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f10059630 con 0x7f5f14050d50 2024-04-25T08:42:44.845 INFO:teuthology.orchestra.run.smithi046.stdout:55834574859 2024-04-25T08:42:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 -- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5efc066f60 msgr2=0x7f5efc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5efc066f60 0x7f5efc069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5f080ae000 tx=0x7f5f080b3040 comp rx=0 tx=0).stop 2024-04-25T08:42:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 -- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f14050d50 msgr2=0x7f5f1404a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f14050d50 0x7f5f1404a860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000e9b0 tx=0x7f5f1000ee70 comp rx=0 tx=0).stop 2024-04-25T08:42:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 -- 172.21.15.46:0/4275827509 shutdown_connections 2024-04-25T08:42:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5efc066f60 0x7f5efc069420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f140e0ef0 0x7f5f141881d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f14050d50 0x7f5f1404a860 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 --2- 172.21.15.46:0/4275827509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f14038640 0x7f5f140e3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 -- 172.21.15.46:0/4275827509 >> 172.21.15.46:0/4275827509 conn(0x7f5f14009fc0 msgr2=0x7f5f140dff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 -- 172.21.15.46:0/4275827509 shutdown_connections 2024-04-25T08:42:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:44.855+0000 7f5f0cc00640 1 -- 172.21.15.46:0/4275827509 wait complete. 2024-04-25T08:42:44.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-04-25T08:42:44.853 DEBUG:teuthology.parallel:result is None 2024-04-25T08:42:44.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-25T08:42:44.866 DEBUG:teuthology.parallel:result is None 2024-04-25T08:42:44.866 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T08:42:44.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.055+0000 7f64a9000640 1 Processor -- start 2024-04-25T08:42:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.055+0000 7f64a9000640 1 -- start start 2024-04-25T08:42:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f64a9000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64a406f320 0x7f64a4162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f64a9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 0x7f64a4164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f64a9000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a40b35b0 con 0x7f64a415a750 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f64a9000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a415ab30 con 0x7f64a406f320 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f64a9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a40ce4f0 con 0x7f64a40aed10 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f649be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64a406f320 0x7f64a4162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f649be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64a406f320 0x7f64a4162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60406/0 (socket says 172.21.15.46:60406) 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f649be00640 1 -- 172.21.15.46:0/3772976582 learned_addr learned my addr 172.21.15.46:0/3772976582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f64a2800640 1 --2- 172.21.15.46:0/3772976582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 0x7f64a4164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f649be00640 1 -- 172.21.15.46:0/3772976582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 msgr2=0x7f64a4164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f649be00640 1 --2- 172.21.15.46:0/3772976582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 0x7f64a4164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f649be00640 1 -- 172.21.15.46:0/3772976582 >> v1:172.21.15.82:6789/0 conn(0x7f64a415a750 legacy=0x7f64a406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.059+0000 7f649be00640 1 -- 172.21.15.46:0/3772976582 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a406d5e0 con 0x7f64a406f320 2024-04-25T08:42:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f649be00640 1 --2- 172.21.15.46:0/3772976582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a406f320 0x7f64a4162200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f648c009870 tx=0x7f648c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37465b497b0dbbbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a1400640 1 -- 172.21.15.46:0/3772976582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c0ab650 con 0x7f64a406f320 2024-04-25T08:42:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a1400640 1 -- 172.21.15.46:0/3772976582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c0b0070 con 0x7f64a406f320 2024-04-25T08:42:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a1400640 1 -- 172.21.15.46:0/3772976582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c0aba10 con 0x7f64a406f320 2024-04-25T08:42:45.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 -- 172.21.15.46:0/3772976582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a406f320 msgr2=0x7f64a4162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 --2- 172.21.15.46:0/3772976582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a406f320 0x7f64a4162200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f648c009870 tx=0x7f648c01c560 comp rx=0 tx=0).stop 2024-04-25T08:42:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 -- 172.21.15.46:0/3772976582 shutdown_connections 2024-04-25T08:42:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 --2- 172.21.15.46:0/3772976582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 0x7f64a4164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 --2- 172.21.15.46:0/3772976582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a406f320 0x7f64a4162200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 -- 172.21.15.46:0/3772976582 >> 172.21.15.46:0/3772976582 conn(0x7f64a40b12a0 msgr2=0x7f64a40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 -- 172.21.15.46:0/3772976582 shutdown_connections 2024-04-25T08:42:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 -- 172.21.15.46:0/3772976582 wait complete. 2024-04-25T08:42:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 Processor -- start 2024-04-25T08:42:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 -- start start 2024-04-25T08:42:45.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64a406f320 0x7f64a414deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 0x7f64a415acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a415a750 0x7f64a414f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a406d270 con 0x7f64a415a750 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a40cf200 con 0x7f64a406f320 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.063+0000 7f64a9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a41674b0 con 0x7f64a40aed10 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a415a750 0x7f64a414f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a415a750 0x7f64a414f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60408/0 (socket says 172.21.15.46:60408) 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a2800640 1 -- 172.21.15.46:0/1749441197 learned_addr learned my addr 172.21.15.46:0/1749441197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a1e00640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64a406f320 0x7f64a414deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f649be00640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 0x7f64a415acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a2800640 1 -- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 msgr2=0x7f64a415acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a2800640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 0x7f64a415acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a2800640 1 -- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64a406f320 msgr2=0x7f64a414deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a2800640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64a406f320 0x7f64a414deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a2800640 1 -- 172.21.15.46:0/1749441197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a415b780 con 0x7f64a415a750 2024-04-25T08:42:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a1e00640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64a406f320 0x7f64a414deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a2800640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a415a750 0x7f64a414f730 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f649c007e40 tx=0x7f649c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f649a000640 1 -- 172.21.15.46:0/1749441197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c014620 con 0x7f64a415a750 2024-04-25T08:42:45.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f649a000640 1 -- 172.21.15.46:0/1749441197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c019070 con 0x7f64a415a750 2024-04-25T08:42:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f649a000640 1 -- 172.21.15.46:0/1749441197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c014a20 con 0x7f64a415a750 2024-04-25T08:42:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a415ba60 con 0x7f64a415a750 2024-04-25T08:42:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a415c020 con 0x7f64a415a750 2024-04-25T08:42:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a4069080 con 0x7f64a415a750 2024-04-25T08:42:45.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f649a000640 1 -- 172.21.15.46:0/1749441197 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f649c014bc0 con 0x7f64a415a750 2024-04-25T08:42:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.067+0000 7f649a000640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6480066f60 0x7f6480069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.071+0000 7f649a000640 1 -- 172.21.15.46:0/1749441197 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f649c085c30 con 0x7f64a415a750 2024-04-25T08:42:45.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.071+0000 7f64a1e00640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6480066f60 0x7f6480069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.071+0000 7f64a1e00640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6480066f60 0x7f6480069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f64a4157740 tx=0x7f6494009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.075+0000 7f649a000640 1 -- 172.21.15.46:0/1749441197 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f649c056860 con 0x7f64a415a750 2024-04-25T08:42:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.231+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f64a40b21c0 con 0x7f6480066f60 2024-04-25T08:42:45.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.239+0000 7f649a000640 1 -- 172.21.15.46:0/1749441197 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332538 (secure 0 0 0) 0x7f64a40b21c0 con 0x7f6480066f60 2024-04-25T08:42:45.237 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:45.239 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:42:45.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6480066f60 msgr2=0x7f6480069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6480066f60 0x7f6480069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f64a4157740 tx=0x7f6494009040 comp rx=0 tx=0).stop 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a415a750 msgr2=0x7f64a414f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a415a750 0x7f64a414f730 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f649c007e40 tx=0x7f649c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 shutdown_connections 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6480066f60 0x7f6480069420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a415a750 0x7f64a414f730 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64a40aed10 0x7f64a415acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 --2- 172.21.15.46:0/1749441197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64a406f320 0x7f64a414deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 >> 172.21.15.46:0/1749441197 conn(0x7f64a40b12a0 msgr2=0x7f64a40af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 shutdown_connections 2024-04-25T08:42:45.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.243+0000 7f64a9000640 1 -- 172.21.15.46:0/1749441197 wait complete. 2024-04-25T08:42:45.257 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T08:42:44.257962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223716,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751004,"statfs":{"total":773094113280,"available":772865028096,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003315"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683286+0000","last_change":"2024-04-25T08:42:20.357496+0000","last_active":"2024-04-25T08:42:28.683286+0000","last_peered":"2024-04-25T08:42:28.683286+0000","last_clean":"2024-04-25T08:42:28.683286+0000","last_became_active":"2024-04-25T08:42:20.357232+0000","last_became_peered":"2024-04-25T08:42:20.357232+0000","last_unstale":"2024-04-25T08:42:28.683286+0000","last_undegraded":"2024-04-25T08:42:28.683286+0000","last_fullsized":"2024-04-25T08:42:28.683286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:43.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385143+0000","last_change":"2024-04-25T08:42:21.359572+0000","last_active":"2024-04-25T08:42:27.385143+0000","last_peered":"2024-04-25T08:42:27.385143+0000","last_clean":"2024-04-25T08:42:27.385143+0000","last_became_active":"2024-04-25T08:42:21.357850+0000","last_became_peered":"2024-04-25T08:42:21.357850+0000","last_unstale":"2024-04-25T08:42:27.385143+0000","last_undegraded":"2024-04-25T08:42:27.385143+0000","last_fullsized":"2024-04-25T08:42:27.385143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:25.717765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384099+0000","last_change":"2024-04-25T08:42:25.834318+0000","last_active":"2024-04-25T08:42:27.384099+0000","last_peered":"2024-04-25T08:42:27.384099+0000","last_clean":"2024-04-25T08:42:27.384099+0000","last_became_active":"2024-04-25T08:42:25.833964+0000","last_became_peered":"2024-04-25T08:42:25.833964+0000","last_unstale":"2024-04-25T08:42:27.384099+0000","last_undegraded":"2024-04-25T08:42:27.384099+0000","last_fullsized":"2024-04-25T08:42:27.384099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:37.125230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385326+0000","last_change":"2024-04-25T08:42:20.351984+0000","last_active":"2024-04-25T08:42:27.385326+0000","last_peered":"2024-04-25T08:42:27.385326+0000","last_clean":"2024-04-25T08:42:27.385326+0000","last_became_active":"2024-04-25T08:42:20.351590+0000","last_became_peered":"2024-04-25T08:42:20.351590+0000","last_unstale":"2024-04-25T08:42:27.385326+0000","last_undegraded":"2024-04-25T08:42:27.385326+0000","last_fullsized":"2024-04-25T08:42:27.385326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683138+0000","last_change":"2024-04-25T08:42:21.364800+0000","last_active":"2024-04-25T08:42:28.683138+0000","last_peered":"2024-04-25T08:42:28.683138+0000","last_clean":"2024-04-25T08:42:28.683138+0000","last_became_active":"2024-04-25T08:42:21.363796+0000","last_became_peered":"2024-04-25T08:42:21.363796+0000","last_unstale":"2024-04-25T08:42:28.683138+0000","last_undegraded":"2024-04-25T08:42:28.683138+0000","last_fullsized":"2024-04-25T08:42:28.683138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389657+0000","last_change":"2024-04-25T08:42:25.410345+0000","last_active":"2024-04-25T08:42:27.389657+0000","last_peered":"2024-04-25T08:42:27.389657+0000","last_clean":"2024-04-25T08:42:27.389657+0000","last_became_active":"2024-04-25T08:42:25.409807+0000","last_became_peered":"2024-04-25T08:42:25.409807+0000","last_unstale":"2024-04-25T08:42:27.389657+0000","last_undegraded":"2024-04-25T08:42:27.389657+0000","last_fullsized":"2024-04-25T08:42:27.389657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.971525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683514+0000","last_change":"2024-04-25T08:42:20.363027+0000","last_active":"2024-04-25T08:42:28.683514+0000","last_peered":"2024-04-25T08:42:28.683514+0000","last_clean":"2024-04-25T08:42:28.683514+0000","last_became_active":"2024-04-25T08:42:20.362598+0000","last_became_peered":"2024-04-25T08:42:20.362598+0000","last_unstale":"2024-04-25T08:42:28.683514+0000","last_undegraded":"2024-04-25T08:42:28.683514+0000","last_fullsized":"2024-04-25T08:42:28.683514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:26.564850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582927+0000","last_change":"2024-04-25T08:42:21.367716+0000","last_active":"2024-04-25T08:42:27.582927+0000","last_peered":"2024-04-25T08:42:27.582927+0000","last_clean":"2024-04-25T08:42:27.582927+0000","last_became_active":"2024-04-25T08:42:21.366362+0000","last_became_peered":"2024-04-25T08:42:21.366362+0000","last_unstale":"2024-04-25T08:42:27.582927+0000","last_undegraded":"2024-04-25T08:42:27.582927+0000","last_fullsized":"2024-04-25T08:42:27.582927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:30.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390443+0000","last_change":"2024-04-25T08:42:25.407011+0000","last_active":"2024-04-25T08:42:27.390443+0000","last_peered":"2024-04-25T08:42:27.390443+0000","last_clean":"2024-04-25T08:42:27.390443+0000","last_became_active":"2024-04-25T08:42:25.405963+0000","last_became_peered":"2024-04-25T08:42:25.405963+0000","last_unstale":"2024-04-25T08:42:27.390443+0000","last_undegraded":"2024-04-25T08:42:27.390443+0000","last_fullsized":"2024-04-25T08:42:27.390443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683493+0000","last_change":"2024-04-25T08:42:20.365950+0000","last_active":"2024-04-25T08:42:28.683493+0000","last_peered":"2024-04-25T08:42:28.683493+0000","last_clean":"2024-04-25T08:42:28.683493+0000","last_became_active":"2024-04-25T08:42:20.365648+0000","last_became_peered":"2024-04-25T08:42:20.365648+0000","last_unstale":"2024-04-25T08:42:28.683493+0000","last_undegraded":"2024-04-25T08:42:28.683493+0000","last_fullsized":"2024-04-25T08:42:28.683493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:31.011923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583016+0000","last_change":"2024-04-25T08:42:21.366598+0000","last_active":"2024-04-25T08:42:27.583016+0000","last_peered":"2024-04-25T08:42:27.583016+0000","last_clean":"2024-04-25T08:42:27.583016+0000","last_became_active":"2024-04-25T08:42:21.365936+0000","last_became_peered":"2024-04-25T08:42:21.365936+0000","last_unstale":"2024-04-25T08:42:27.583016+0000","last_undegraded":"2024-04-25T08:42:27.583016+0000","last_fullsized":"2024-04-25T08:42:27.583016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389911+0000","last_change":"2024-04-25T08:42:25.400233+0000","last_active":"2024-04-25T08:42:27.389911+0000","last_peered":"2024-04-25T08:42:27.389911+0000","last_clean":"2024-04-25T08:42:27.389911+0000","last_became_active":"2024-04-25T08:42:25.399809+0000","last_became_peered":"2024-04-25T08:42:25.399809+0000","last_unstale":"2024-04-25T08:42:27.389911+0000","last_undegraded":"2024-04-25T08:42:27.389911+0000","last_fullsized":"2024-04-25T08:42:27.389911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:19.971594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388746+0000","last_change":"2024-04-25T08:42:21.354062+0000","last_active":"2024-04-25T08:42:27.388746+0000","last_peered":"2024-04-25T08:42:27.388746+0000","last_clean":"2024-04-25T08:42:27.388746+0000","last_became_active":"2024-04-25T08:42:21.353804+0000","last_became_peered":"2024-04-25T08:42:21.353804+0000","last_unstale":"2024-04-25T08:42:27.388746+0000","last_undegraded":"2024-04-25T08:42:27.388746+0000","last_fullsized":"2024-04-25T08:42:27.388746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683480+0000","last_change":"2024-04-25T08:42:20.361718+0000","last_active":"2024-04-25T08:42:28.683480+0000","last_peered":"2024-04-25T08:42:28.683480+0000","last_clean":"2024-04-25T08:42:28.683480+0000","last_became_active":"2024-04-25T08:42:20.361531+0000","last_became_peered":"2024-04-25T08:42:20.361531+0000","last_unstale":"2024-04-25T08:42:28.683480+0000","last_undegraded":"2024-04-25T08:42:28.683480+0000","last_fullsized":"2024-04-25T08:42:28.683480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:47.546463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390866+0000","last_change":"2024-04-25T08:42:25.410313+0000","last_active":"2024-04-25T08:42:27.390866+0000","last_peered":"2024-04-25T08:42:27.390866+0000","last_clean":"2024-04-25T08:42:27.390866+0000","last_became_active":"2024-04-25T08:42:25.409794+0000","last_became_peered":"2024-04-25T08:42:25.409794+0000","last_unstale":"2024-04-25T08:42:27.390866+0000","last_undegraded":"2024-04-25T08:42:27.390866+0000","last_fullsized":"2024-04-25T08:42:27.390866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384856+0000","last_change":"2024-04-25T08:42:21.358366+0000","last_active":"2024-04-25T08:42:27.384856+0000","last_peered":"2024-04-25T08:42:27.384856+0000","last_clean":"2024-04-25T08:42:27.384856+0000","last_became_active":"2024-04-25T08:42:21.357447+0000","last_became_peered":"2024-04-25T08:42:21.357447+0000","last_unstale":"2024-04-25T08:42:27.384856+0000","last_undegraded":"2024-04-25T08:42:27.384856+0000","last_fullsized":"2024-04-25T08:42:27.384856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677905+0000","last_change":"2024-04-25T08:42:20.352044+0000","last_active":"2024-04-25T08:42:28.677905+0000","last_peered":"2024-04-25T08:42:28.677905+0000","last_clean":"2024-04-25T08:42:28.677905+0000","last_became_active":"2024-04-25T08:42:20.351667+0000","last_became_peered":"2024-04-25T08:42:20.351667+0000","last_unstale":"2024-04-25T08:42:28.677905+0000","last_undegraded":"2024-04-25T08:42:28.677905+0000","last_fullsized":"2024-04-25T08:42:28.677905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390597+0000","last_change":"2024-04-25T08:42:25.406565+0000","last_active":"2024-04-25T08:42:27.390597+0000","last_peered":"2024-04-25T08:42:27.390597+0000","last_clean":"2024-04-25T08:42:27.390597+0000","last_became_active":"2024-04-25T08:42:25.405840+0000","last_became_peered":"2024-04-25T08:42:25.405840+0000","last_unstale":"2024-04-25T08:42:27.390597+0000","last_undegraded":"2024-04-25T08:42:27.390597+0000","last_fullsized":"2024-04-25T08:42:27.390597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384326+0000","last_change":"2024-04-25T08:42:21.354614+0000","last_active":"2024-04-25T08:42:27.384326+0000","last_peered":"2024-04-25T08:42:27.384326+0000","last_clean":"2024-04-25T08:42:27.384326+0000","last_became_active":"2024-04-25T08:42:21.353873+0000","last_became_peered":"2024-04-25T08:42:21.353873+0000","last_unstale":"2024-04-25T08:42:27.384326+0000","last_undegraded":"2024-04-25T08:42:27.384326+0000","last_fullsized":"2024-04-25T08:42:27.384326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678307+0000","last_change":"2024-04-25T08:42:20.349084+0000","last_active":"2024-04-25T08:42:28.678307+0000","last_peered":"2024-04-25T08:42:28.678307+0000","last_clean":"2024-04-25T08:42:28.678307+0000","last_became_active":"2024-04-25T08:42:20.348850+0000","last_became_peered":"2024-04-25T08:42:20.348850+0000","last_unstale":"2024-04-25T08:42:28.678307+0000","last_undegraded":"2024-04-25T08:42:28.678307+0000","last_fullsized":"2024-04-25T08:42:28.678307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678770+0000","last_change":"2024-04-25T08:42:25.832820+0000","last_active":"2024-04-25T08:42:28.678770+0000","last_peered":"2024-04-25T08:42:28.678770+0000","last_clean":"2024-04-25T08:42:28.678770+0000","last_became_active":"2024-04-25T08:42:25.832090+0000","last_became_peered":"2024-04-25T08:42:25.832090+0000","last_unstale":"2024-04-25T08:42:28.678770+0000","last_undegraded":"2024-04-25T08:42:28.678770+0000","last_fullsized":"2024-04-25T08:42:28.678770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390081+0000","last_change":"2024-04-25T08:42:21.338500+0000","last_active":"2024-04-25T08:42:27.390081+0000","last_peered":"2024-04-25T08:42:27.390081+0000","last_clean":"2024-04-25T08:42:27.390081+0000","last_became_active":"2024-04-25T08:42:21.338153+0000","last_became_peered":"2024-04-25T08:42:21.338153+0000","last_unstale":"2024-04-25T08:42:27.390081+0000","last_undegraded":"2024-04-25T08:42:27.390081+0000","last_fullsized":"2024-04-25T08:42:27.390081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:00.647619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683714+0000","last_change":"2024-04-25T08:42:20.361253+0000","last_active":"2024-04-25T08:42:28.683714+0000","last_peered":"2024-04-25T08:42:28.683714+0000","last_clean":"2024-04-25T08:42:28.683714+0000","last_became_active":"2024-04-25T08:42:20.360866+0000","last_became_peered":"2024-04-25T08:42:20.360866+0000","last_unstale":"2024-04-25T08:42:28.683714+0000","last_undegraded":"2024-04-25T08:42:28.683714+0000","last_fullsized":"2024-04-25T08:42:28.683714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.823720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390784+0000","last_change":"2024-04-25T08:42:25.406555+0000","last_active":"2024-04-25T08:42:27.390784+0000","last_peered":"2024-04-25T08:42:27.390784+0000","last_clean":"2024-04-25T08:42:27.390784+0000","last_became_active":"2024-04-25T08:42:25.405977+0000","last_became_peered":"2024-04-25T08:42:25.405977+0000","last_unstale":"2024-04-25T08:42:27.390784+0000","last_undegraded":"2024-04-25T08:42:27.390784+0000","last_fullsized":"2024-04-25T08:42:27.390784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683559+0000","last_change":"2024-04-25T08:42:21.361932+0000","last_active":"2024-04-25T08:42:28.683559+0000","last_peered":"2024-04-25T08:42:28.683559+0000","last_clean":"2024-04-25T08:42:28.683559+0000","last_became_active":"2024-04-25T08:42:21.361599+0000","last_became_peered":"2024-04-25T08:42:21.361599+0000","last_unstale":"2024-04-25T08:42:28.683559+0000","last_undegraded":"2024-04-25T08:42:28.683559+0000","last_fullsized":"2024-04-25T08:42:28.683559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389472+0000","last_change":"2024-04-25T08:42:20.359118+0000","last_active":"2024-04-25T08:42:27.389472+0000","last_peered":"2024-04-25T08:42:27.389472+0000","last_clean":"2024-04-25T08:42:27.389472+0000","last_became_active":"2024-04-25T08:42:20.358850+0000","last_became_peered":"2024-04-25T08:42:20.358850+0000","last_unstale":"2024-04-25T08:42:27.389472+0000","last_undegraded":"2024-04-25T08:42:27.389472+0000","last_fullsized":"2024-04-25T08:42:27.389472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389832+0000","last_change":"2024-04-25T08:42:25.836412+0000","last_active":"2024-04-25T08:42:27.389832+0000","last_peered":"2024-04-25T08:42:27.389832+0000","last_clean":"2024-04-25T08:42:27.389832+0000","last_became_active":"2024-04-25T08:42:25.835926+0000","last_became_peered":"2024-04-25T08:42:25.835926+0000","last_unstale":"2024-04-25T08:42:27.389832+0000","last_undegraded":"2024-04-25T08:42:27.389832+0000","last_fullsized":"2024-04-25T08:42:27.389832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.610141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385472+0000","last_change":"2024-04-25T08:42:21.366095+0000","last_active":"2024-04-25T08:42:27.385472+0000","last_peered":"2024-04-25T08:42:27.385472+0000","last_clean":"2024-04-25T08:42:27.385472+0000","last_became_active":"2024-04-25T08:42:21.365608+0000","last_became_peered":"2024-04-25T08:42:21.365608+0000","last_unstale":"2024-04-25T08:42:27.385472+0000","last_undegraded":"2024-04-25T08:42:27.385472+0000","last_fullsized":"2024-04-25T08:42:27.385472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677918+0000","last_change":"2024-04-25T08:42:20.347127+0000","last_active":"2024-04-25T08:42:28.677918+0000","last_peered":"2024-04-25T08:42:28.677918+0000","last_clean":"2024-04-25T08:42:28.677918+0000","last_became_active":"2024-04-25T08:42:20.346840+0000","last_became_peered":"2024-04-25T08:42:20.346840+0000","last_unstale":"2024-04-25T08:42:28.677918+0000","last_undegraded":"2024-04-25T08:42:28.677918+0000","last_fullsized":"2024-04-25T08:42:28.677918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384051+0000","last_change":"2024-04-25T08:42:25.832430+0000","last_active":"2024-04-25T08:42:27.384051+0000","last_peered":"2024-04-25T08:42:27.384051+0000","last_clean":"2024-04-25T08:42:27.384051+0000","last_became_active":"2024-04-25T08:42:25.832026+0000","last_became_peered":"2024-04-25T08:42:25.832026+0000","last_unstale":"2024-04-25T08:42:27.384051+0000","last_undegraded":"2024-04-25T08:42:27.384051+0000","last_fullsized":"2024-04-25T08:42:27.384051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:39.589658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.679001+0000","last_change":"2024-04-25T08:42:21.340393+0000","last_active":"2024-04-25T08:42:28.679001+0000","last_peered":"2024-04-25T08:42:28.679001+0000","last_clean":"2024-04-25T08:42:28.679001+0000","last_became_active":"2024-04-25T08:42:21.340043+0000","last_became_peered":"2024-04-25T08:42:21.340043+0000","last_unstale":"2024-04-25T08:42:28.679001+0000","last_undegraded":"2024-04-25T08:42:28.679001+0000","last_fullsized":"2024-04-25T08:42:28.679001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684534+0000","last_change":"2024-04-25T08:42:20.360592+0000","last_active":"2024-04-25T08:42:28.684534+0000","last_peered":"2024-04-25T08:42:28.684534+0000","last_clean":"2024-04-25T08:42:28.684534+0000","last_became_active":"2024-04-25T08:42:20.360361+0000","last_became_peered":"2024-04-25T08:42:20.360361+0000","last_unstale":"2024-04-25T08:42:28.684534+0000","last_undegraded":"2024-04-25T08:42:28.684534+0000","last_fullsized":"2024-04-25T08:42:28.684534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:53.741991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682832+0000","last_change":"2024-04-25T08:42:25.834135+0000","last_active":"2024-04-25T08:42:28.682832+0000","last_peered":"2024-04-25T08:42:28.682832+0000","last_clean":"2024-04-25T08:42:28.682832+0000","last_became_active":"2024-04-25T08:42:25.833757+0000","last_became_peered":"2024-04-25T08:42:25.833757+0000","last_unstale":"2024-04-25T08:42:28.682832+0000","last_undegraded":"2024-04-25T08:42:28.682832+0000","last_fullsized":"2024-04-25T08:42:28.682832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583189+0000","last_change":"2024-04-25T08:42:21.366345+0000","last_active":"2024-04-25T08:42:27.583189+0000","last_peered":"2024-04-25T08:42:27.583189+0000","last_clean":"2024-04-25T08:42:27.583189+0000","last_became_active":"2024-04-25T08:42:21.365644+0000","last_became_peered":"2024-04-25T08:42:21.365644+0000","last_unstale":"2024-04-25T08:42:27.583189+0000","last_undegraded":"2024-04-25T08:42:27.583189+0000","last_fullsized":"2024-04-25T08:42:27.583189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.679043+0000","last_change":"2024-04-25T08:42:20.344087+0000","last_active":"2024-04-25T08:42:28.679043+0000","last_peered":"2024-04-25T08:42:28.679043+0000","last_clean":"2024-04-25T08:42:28.679043+0000","last_became_active":"2024-04-25T08:42:20.343799+0000","last_became_peered":"2024-04-25T08:42:20.343799+0000","last_unstale":"2024-04-25T08:42:28.679043+0000","last_undegraded":"2024-04-25T08:42:28.679043+0000","last_fullsized":"2024-04-25T08:42:28.679043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.823720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678152+0000","last_change":"2024-04-25T08:42:25.412243+0000","last_active":"2024-04-25T08:42:28.678152+0000","last_peered":"2024-04-25T08:42:28.678152+0000","last_clean":"2024-04-25T08:42:28.678152+0000","last_became_active":"2024-04-25T08:42:25.406351+0000","last_became_peered":"2024-04-25T08:42:25.406351+0000","last_unstale":"2024-04-25T08:42:28.678152+0000","last_undegraded":"2024-04-25T08:42:28.678152+0000","last_fullsized":"2024-04-25T08:42:28.678152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:42.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390626+0000","last_change":"2024-04-25T08:42:21.366092+0000","last_active":"2024-04-25T08:42:27.390626+0000","last_peered":"2024-04-25T08:42:27.390626+0000","last_clean":"2024-04-25T08:42:27.390626+0000","last_became_active":"2024-04-25T08:42:21.365152+0000","last_became_peered":"2024-04-25T08:42:21.365152+0000","last_unstale":"2024-04-25T08:42:27.390626+0000","last_undegraded":"2024-04-25T08:42:27.390626+0000","last_fullsized":"2024-04-25T08:42:27.390626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:29.830442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582739+0000","last_change":"2024-04-25T08:42:20.352850+0000","last_active":"2024-04-25T08:42:27.582739+0000","last_peered":"2024-04-25T08:42:27.582739+0000","last_clean":"2024-04-25T08:42:27.582739+0000","last_became_active":"2024-04-25T08:42:20.352047+0000","last_became_peered":"2024-04-25T08:42:20.352047+0000","last_unstale":"2024-04-25T08:42:27.582739+0000","last_undegraded":"2024-04-25T08:42:27.582739+0000","last_fullsized":"2024-04-25T08:42:27.582739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678979+0000","last_change":"2024-04-25T08:42:25.405939+0000","last_active":"2024-04-25T08:42:28.678979+0000","last_peered":"2024-04-25T08:42:28.678979+0000","last_clean":"2024-04-25T08:42:28.678979+0000","last_became_active":"2024-04-25T08:42:25.405333+0000","last_became_peered":"2024-04-25T08:42:25.405333+0000","last_unstale":"2024-04-25T08:42:28.678979+0000","last_undegraded":"2024-04-25T08:42:28.678979+0000","last_fullsized":"2024-04-25T08:42:28.678979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678442+0000","last_change":"2024-04-25T08:42:21.338660+0000","last_active":"2024-04-25T08:42:28.678442+0000","last_peered":"2024-04-25T08:42:28.678442+0000","last_clean":"2024-04-25T08:42:28.678442+0000","last_became_active":"2024-04-25T08:42:21.338357+0000","last_became_peered":"2024-04-25T08:42:21.338357+0000","last_unstale":"2024-04-25T08:42:28.678442+0000","last_undegraded":"2024-04-25T08:42:28.678442+0000","last_fullsized":"2024-04-25T08:42:28.678442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583032+0000","last_change":"2024-04-25T08:42:20.352888+0000","last_active":"2024-04-25T08:42:28.583032+0000","last_peered":"2024-04-25T08:42:28.583032+0000","last_clean":"2024-04-25T08:42:28.583032+0000","last_became_active":"2024-04-25T08:42:20.351108+0000","last_became_peered":"2024-04-25T08:42:20.351108+0000","last_unstale":"2024-04-25T08:42:28.583032+0000","last_undegraded":"2024-04-25T08:42:28.583032+0000","last_fullsized":"2024-04-25T08:42:28.583032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678770+0000","last_change":"2024-04-25T08:42:25.409801+0000","last_active":"2024-04-25T08:42:28.678770+0000","last_peered":"2024-04-25T08:42:28.678770+0000","last_clean":"2024-04-25T08:42:28.678770+0000","last_became_active":"2024-04-25T08:42:25.409331+0000","last_became_peered":"2024-04-25T08:42:25.409331+0000","last_unstale":"2024-04-25T08:42:28.678770+0000","last_undegraded":"2024-04-25T08:42:28.678770+0000","last_fullsized":"2024-04-25T08:42:28.678770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682464+0000","last_change":"2024-04-25T08:42:21.363349+0000","last_active":"2024-04-25T08:42:28.682464+0000","last_peered":"2024-04-25T08:42:28.682464+0000","last_clean":"2024-04-25T08:42:28.682464+0000","last_became_active":"2024-04-25T08:42:21.362903+0000","last_became_peered":"2024-04-25T08:42:21.362903+0000","last_unstale":"2024-04-25T08:42:28.682464+0000","last_undegraded":"2024-04-25T08:42:28.682464+0000","last_fullsized":"2024-04-25T08:42:28.682464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:48.553185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683717+0000","last_change":"2024-04-25T08:42:20.356423+0000","last_active":"2024-04-25T08:42:28.683717+0000","last_peered":"2024-04-25T08:42:28.683717+0000","last_clean":"2024-04-25T08:42:28.683717+0000","last_became_active":"2024-04-25T08:42:20.356110+0000","last_became_peered":"2024-04-25T08:42:20.356110+0000","last_unstale":"2024-04-25T08:42:28.683717+0000","last_undegraded":"2024-04-25T08:42:28.683717+0000","last_fullsized":"2024-04-25T08:42:28.683717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:44.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683389+0000","last_change":"2024-04-25T08:42:25.834561+0000","last_active":"2024-04-25T08:42:28.683389+0000","last_peered":"2024-04-25T08:42:28.683389+0000","last_clean":"2024-04-25T08:42:28.683389+0000","last_became_active":"2024-04-25T08:42:25.834038+0000","last_became_peered":"2024-04-25T08:42:25.834038+0000","last_unstale":"2024-04-25T08:42:28.683389+0000","last_undegraded":"2024-04-25T08:42:28.683389+0000","last_fullsized":"2024-04-25T08:42:28.683389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:09.194186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384703+0000","last_change":"2024-04-25T08:42:21.369030+0000","last_active":"2024-04-25T08:42:27.384703+0000","last_peered":"2024-04-25T08:42:27.384703+0000","last_clean":"2024-04-25T08:42:27.384703+0000","last_became_active":"2024-04-25T08:42:21.368760+0000","last_became_peered":"2024-04-25T08:42:21.368760+0000","last_unstale":"2024-04-25T08:42:27.384703+0000","last_undegraded":"2024-04-25T08:42:27.384703+0000","last_fullsized":"2024-04-25T08:42:27.384703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582740+0000","last_change":"2024-04-25T08:42:20.359912+0000","last_active":"2024-04-25T08:42:27.582740+0000","last_peered":"2024-04-25T08:42:27.582740+0000","last_clean":"2024-04-25T08:42:27.582740+0000","last_became_active":"2024-04-25T08:42:20.359616+0000","last_became_peered":"2024-04-25T08:42:20.359616+0000","last_unstale":"2024-04-25T08:42:27.582740+0000","last_undegraded":"2024-04-25T08:42:27.582740+0000","last_fullsized":"2024-04-25T08:42:27.582740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683854+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:42:28.683854+0000","last_peered":"2024-04-25T08:42:28.683854+0000","last_clean":"2024-04-25T08:42:28.683854+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:42:28.683854+0000","last_undegraded":"2024-04-25T08:42:28.683854+0000","last_fullsized":"2024-04-25T08:42:28.683854+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389063+0000","last_change":"2024-04-25T08:42:25.836852+0000","last_active":"2024-04-25T08:42:27.389063+0000","last_peered":"2024-04-25T08:42:27.389063+0000","last_clean":"2024-04-25T08:42:27.389063+0000","last_became_active":"2024-04-25T08:42:25.836428+0000","last_became_peered":"2024-04-25T08:42:25.836428+0000","last_unstale":"2024-04-25T08:42:27.389063+0000","last_undegraded":"2024-04-25T08:42:27.389063+0000","last_fullsized":"2024-04-25T08:42:27.389063+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.544013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684361+0000","last_change":"2024-04-25T08:42:21.376058+0000","last_active":"2024-04-25T08:42:28.684361+0000","last_peered":"2024-04-25T08:42:28.684361+0000","last_clean":"2024-04-25T08:42:28.684361+0000","last_became_active":"2024-04-25T08:42:21.375825+0000","last_became_peered":"2024-04-25T08:42:21.375825+0000","last_unstale":"2024-04-25T08:42:28.684361+0000","last_undegraded":"2024-04-25T08:42:28.684361+0000","last_fullsized":"2024-04-25T08:42:28.684361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:24.495752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583044+0000","last_change":"2024-04-25T08:42:20.353588+0000","last_active":"2024-04-25T08:42:28.583044+0000","last_peered":"2024-04-25T08:42:28.583044+0000","last_clean":"2024-04-25T08:42:28.583044+0000","last_became_active":"2024-04-25T08:42:20.353335+0000","last_became_peered":"2024-04-25T08:42:20.353335+0000","last_unstale":"2024-04-25T08:42:28.583044+0000","last_undegraded":"2024-04-25T08:42:28.583044+0000","last_fullsized":"2024-04-25T08:42:28.583044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388628+0000","last_change":"2024-04-25T08:42:25.833110+0000","last_active":"2024-04-25T08:42:27.388628+0000","last_peered":"2024-04-25T08:42:27.388628+0000","last_clean":"2024-04-25T08:42:27.388628+0000","last_became_active":"2024-04-25T08:42:25.832656+0000","last_became_peered":"2024-04-25T08:42:25.832656+0000","last_unstale":"2024-04-25T08:42:27.388628+0000","last_undegraded":"2024-04-25T08:42:27.388628+0000","last_fullsized":"2024-04-25T08:42:27.388628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.971525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582934+0000","last_change":"2024-04-25T08:42:21.368021+0000","last_active":"2024-04-25T08:42:27.582934+0000","last_peered":"2024-04-25T08:42:27.582934+0000","last_clean":"2024-04-25T08:42:27.582934+0000","last_became_active":"2024-04-25T08:42:21.367516+0000","last_became_peered":"2024-04-25T08:42:21.367516+0000","last_unstale":"2024-04-25T08:42:27.582934+0000","last_undegraded":"2024-04-25T08:42:27.582934+0000","last_fullsized":"2024-04-25T08:42:27.582934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:24.495752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677925+0000","last_change":"2024-04-25T08:42:20.340089+0000","last_active":"2024-04-25T08:42:28.677925+0000","last_peered":"2024-04-25T08:42:28.677925+0000","last_clean":"2024-04-25T08:42:28.677925+0000","last_became_active":"2024-04-25T08:42:20.338267+0000","last_became_peered":"2024-04-25T08:42:20.338267+0000","last_unstale":"2024-04-25T08:42:28.677925+0000","last_undegraded":"2024-04-25T08:42:28.677925+0000","last_fullsized":"2024-04-25T08:42:28.677925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:44.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682886+0000","last_change":"2024-04-25T08:42:25.418168+0000","last_active":"2024-04-25T08:42:28.682886+0000","last_peered":"2024-04-25T08:42:28.682886+0000","last_clean":"2024-04-25T08:42:28.682886+0000","last_became_active":"2024-04-25T08:42:25.417852+0000","last_became_peered":"2024-04-25T08:42:25.417852+0000","last_unstale":"2024-04-25T08:42:28.682886+0000","last_undegraded":"2024-04-25T08:42:28.682886+0000","last_fullsized":"2024-04-25T08:42:28.682886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684774+0000","last_change":"2024-04-25T08:42:21.370059+0000","last_active":"2024-04-25T08:42:28.684774+0000","last_peered":"2024-04-25T08:42:28.684774+0000","last_clean":"2024-04-25T08:42:28.684774+0000","last_became_active":"2024-04-25T08:42:21.359776+0000","last_became_peered":"2024-04-25T08:42:21.359776+0000","last_unstale":"2024-04-25T08:42:28.684774+0000","last_undegraded":"2024-04-25T08:42:28.684774+0000","last_fullsized":"2024-04-25T08:42:28.684774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:21.412723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583070+0000","last_change":"2024-04-25T08:42:20.351406+0000","last_active":"2024-04-25T08:42:28.583070+0000","last_peered":"2024-04-25T08:42:28.583070+0000","last_clean":"2024-04-25T08:42:28.583070+0000","last_became_active":"2024-04-25T08:42:20.351114+0000","last_became_peered":"2024-04-25T08:42:20.351114+0000","last_unstale":"2024-04-25T08:42:28.583070+0000","last_undegraded":"2024-04-25T08:42:28.583070+0000","last_fullsized":"2024-04-25T08:42:28.583070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582385+0000","last_change":"2024-04-25T08:42:25.835941+0000","last_active":"2024-04-25T08:42:27.582385+0000","last_peered":"2024-04-25T08:42:27.582385+0000","last_clean":"2024-04-25T08:42:27.582385+0000","last_became_active":"2024-04-25T08:42:25.834320+0000","last_became_peered":"2024-04-25T08:42:25.834320+0000","last_unstale":"2024-04-25T08:42:27.582385+0000","last_undegraded":"2024-04-25T08:42:27.582385+0000","last_fullsized":"2024-04-25T08:42:27.582385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:32.335929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684548+0000","last_change":"2024-04-25T08:42:21.363962+0000","last_active":"2024-04-25T08:42:28.684548+0000","last_peered":"2024-04-25T08:42:28.684548+0000","last_clean":"2024-04-25T08:42:28.684548+0000","last_became_active":"2024-04-25T08:42:21.363695+0000","last_became_peered":"2024-04-25T08:42:21.363695+0000","last_unstale":"2024-04-25T08:42:28.684548+0000","last_undegraded":"2024-04-25T08:42:28.684548+0000","last_fullsized":"2024-04-25T08:42:28.684548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:15.923334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.391021+0000","last_change":"2024-04-25T08:42:20.345476+0000","last_active":"2024-04-25T08:42:27.391021+0000","last_peered":"2024-04-25T08:42:27.391021+0000","last_clean":"2024-04-25T08:42:27.391021+0000","last_became_active":"2024-04-25T08:42:20.345085+0000","last_became_peered":"2024-04-25T08:42:20.345085+0000","last_unstale":"2024-04-25T08:42:27.391021+0000","last_undegraded":"2024-04-25T08:42:27.391021+0000","last_fullsized":"2024-04-25T08:42:27.391021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:19.301171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389733+0000","last_change":"2024-04-25T08:42:25.410444+0000","last_active":"2024-04-25T08:42:27.389733+0000","last_peered":"2024-04-25T08:42:27.389733+0000","last_clean":"2024-04-25T08:42:27.389733+0000","last_became_active":"2024-04-25T08:42:25.409574+0000","last_became_peered":"2024-04-25T08:42:25.409574+0000","last_unstale":"2024-04-25T08:42:27.389733+0000","last_undegraded":"2024-04-25T08:42:27.389733+0000","last_fullsized":"2024-04-25T08:42:27.389733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.544013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683914+0000","last_change":"2024-04-25T08:42:21.357665+0000","last_active":"2024-04-25T08:42:28.683914+0000","last_peered":"2024-04-25T08:42:28.683914+0000","last_clean":"2024-04-25T08:42:28.683914+0000","last_became_active":"2024-04-25T08:42:21.357081+0000","last_became_peered":"2024-04-25T08:42:21.357081+0000","last_unstale":"2024-04-25T08:42:28.683914+0000","last_undegraded":"2024-04-25T08:42:28.683914+0000","last_fullsized":"2024-04-25T08:42:28.683914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:24.505703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583188+0000","last_change":"2024-04-25T08:42:20.345496+0000","last_active":"2024-04-25T08:42:28.583188+0000","last_peered":"2024-04-25T08:42:28.583188+0000","last_clean":"2024-04-25T08:42:28.583188+0000","last_became_active":"2024-04-25T08:42:20.345230+0000","last_became_peered":"2024-04-25T08:42:20.345230+0000","last_unstale":"2024-04-25T08:42:28.583188+0000","last_undegraded":"2024-04-25T08:42:28.583188+0000","last_fullsized":"2024-04-25T08:42:28.583188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:33.260664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683216+0000","last_change":"2024-04-25T08:42:25.412932+0000","last_active":"2024-04-25T08:42:28.683216+0000","last_peered":"2024-04-25T08:42:28.683216+0000","last_clean":"2024-04-25T08:42:28.683216+0000","last_became_active":"2024-04-25T08:42:25.412212+0000","last_became_peered":"2024-04-25T08:42:25.412212+0000","last_unstale":"2024-04-25T08:42:28.683216+0000","last_undegraded":"2024-04-25T08:42:28.683216+0000","last_fullsized":"2024-04-25T08:42:28.683216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388745+0000","last_change":"2024-04-25T08:42:21.351855+0000","last_active":"2024-04-25T08:42:27.388745+0000","last_peered":"2024-04-25T08:42:27.388745+0000","last_clean":"2024-04-25T08:42:27.388745+0000","last_became_active":"2024-04-25T08:42:21.351412+0000","last_became_peered":"2024-04-25T08:42:21.351412+0000","last_unstale":"2024-04-25T08:42:27.388745+0000","last_undegraded":"2024-04-25T08:42:27.388745+0000","last_fullsized":"2024-04-25T08:42:27.388745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388741+0000","last_change":"2024-04-25T08:42:20.350412+0000","last_active":"2024-04-25T08:42:27.388741+0000","last_peered":"2024-04-25T08:42:27.388741+0000","last_clean":"2024-04-25T08:42:27.388741+0000","last_became_active":"2024-04-25T08:42:20.350113+0000","last_became_peered":"2024-04-25T08:42:20.350113+0000","last_unstale":"2024-04-25T08:42:27.388741+0000","last_undegraded":"2024-04-25T08:42:27.388741+0000","last_fullsized":"2024-04-25T08:42:27.388741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385384+0000","last_change":"2024-04-25T08:42:25.838652+0000","last_active":"2024-04-25T08:42:27.385384+0000","last_peered":"2024-04-25T08:42:27.385384+0000","last_clean":"2024-04-25T08:42:27.385384+0000","last_became_active":"2024-04-25T08:42:25.838168+0000","last_became_peered":"2024-04-25T08:42:25.838168+0000","last_unstale":"2024-04-25T08:42:27.385384+0000","last_undegraded":"2024-04-25T08:42:27.385384+0000","last_fullsized":"2024-04-25T08:42:27.385384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:46.882274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684243+0000","last_change":"2024-04-25T08:42:21.373789+0000","last_active":"2024-04-25T08:42:28.684243+0000","last_peered":"2024-04-25T08:42:28.684243+0000","last_clean":"2024-04-25T08:42:28.684243+0000","last_became_active":"2024-04-25T08:42:21.372582+0000","last_became_peered":"2024-04-25T08:42:21.372582+0000","last_unstale":"2024-04-25T08:42:28.684243+0000","last_undegraded":"2024-04-25T08:42:28.684243+0000","last_fullsized":"2024-04-25T08:42:28.684243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.525554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582667+0000","last_change":"2024-04-25T08:42:20.357655+0000","last_active":"2024-04-25T08:42:27.582667+0000","last_peered":"2024-04-25T08:42:27.582667+0000","last_clean":"2024-04-25T08:42:27.582667+0000","last_became_active":"2024-04-25T08:42:20.357354+0000","last_became_peered":"2024-04-25T08:42:20.357354+0000","last_unstale":"2024-04-25T08:42:27.582667+0000","last_undegraded":"2024-04-25T08:42:27.582667+0000","last_fullsized":"2024-04-25T08:42:27.582667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385040+0000","last_change":"2024-04-25T08:42:25.835505+0000","last_active":"2024-04-25T08:42:27.385040+0000","last_peered":"2024-04-25T08:42:27.385040+0000","last_clean":"2024-04-25T08:42:27.385040+0000","last_became_active":"2024-04-25T08:42:25.835079+0000","last_became_peered":"2024-04-25T08:42:25.835079+0000","last_unstale":"2024-04-25T08:42:27.385040+0000","last_undegraded":"2024-04-25T08:42:27.385040+0000","last_fullsized":"2024-04-25T08:42:27.385040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:00.799395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682748+0000","last_change":"2024-04-25T08:42:21.366223+0000","last_active":"2024-04-25T08:42:28.682748+0000","last_peered":"2024-04-25T08:42:28.682748+0000","last_clean":"2024-04-25T08:42:28.682748+0000","last_became_active":"2024-04-25T08:42:21.365913+0000","last_became_peered":"2024-04-25T08:42:21.365913+0000","last_unstale":"2024-04-25T08:42:28.682748+0000","last_undegraded":"2024-04-25T08:42:28.682748+0000","last_fullsized":"2024-04-25T08:42:28.682748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682183+0000","last_change":"2024-04-25T08:42:20.354779+0000","last_active":"2024-04-25T08:42:28.682183+0000","last_peered":"2024-04-25T08:42:28.682183+0000","last_clean":"2024-04-25T08:42:28.682183+0000","last_became_active":"2024-04-25T08:42:20.354504+0000","last_became_peered":"2024-04-25T08:42:20.354504+0000","last_unstale":"2024-04-25T08:42:28.682183+0000","last_undegraded":"2024-04-25T08:42:28.682183+0000","last_fullsized":"2024-04-25T08:42:28.682183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.311211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583098+0000","last_change":"2024-04-25T08:42:25.832677+0000","last_active":"2024-04-25T08:42:27.583098+0000","last_peered":"2024-04-25T08:42:27.583098+0000","last_clean":"2024-04-25T08:42:27.583098+0000","last_became_active":"2024-04-25T08:42:25.832204+0000","last_became_peered":"2024-04-25T08:42:25.832204+0000","last_unstale":"2024-04-25T08:42:27.583098+0000","last_undegraded":"2024-04-25T08:42:27.583098+0000","last_fullsized":"2024-04-25T08:42:27.583098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:02.694380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684296+0000","last_change":"2024-04-25T08:42:21.365924+0000","last_active":"2024-04-25T08:42:28.684296+0000","last_peered":"2024-04-25T08:42:28.684296+0000","last_clean":"2024-04-25T08:42:28.684296+0000","last_became_active":"2024-04-25T08:42:21.365581+0000","last_became_peered":"2024-04-25T08:42:21.365581+0000","last_unstale":"2024-04-25T08:42:28.684296+0000","last_undegraded":"2024-04-25T08:42:28.684296+0000","last_fullsized":"2024-04-25T08:42:28.684296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.923264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:36.452272+0000","last_change":"2024-04-25T08:42:20.354736+0000","last_active":"2024-04-25T08:42:36.452272+0000","last_peered":"2024-04-25T08:42:36.452272+0000","last_clean":"2024-04-25T08:42:36.452272+0000","last_became_active":"2024-04-25T08:42:20.354322+0000","last_became_peered":"2024-04-25T08:42:20.354322+0000","last_unstale":"2024-04-25T08:42:36.452272+0000","last_undegraded":"2024-04-25T08:42:36.452272+0000","last_fullsized":"2024-04-25T08:42:36.452272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678428+0000","last_change":"2024-04-25T08:42:25.414706+0000","last_active":"2024-04-25T08:42:28.678428+0000","last_peered":"2024-04-25T08:42:28.678428+0000","last_clean":"2024-04-25T08:42:28.678428+0000","last_became_active":"2024-04-25T08:42:25.414302+0000","last_became_peered":"2024-04-25T08:42:25.414302+0000","last_unstale":"2024-04-25T08:42:28.678428+0000","last_undegraded":"2024-04-25T08:42:28.678428+0000","last_fullsized":"2024-04-25T08:42:28.678428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390755+0000","last_change":"2024-04-25T08:42:21.361175+0000","last_active":"2024-04-25T08:42:27.390755+0000","last_peered":"2024-04-25T08:42:27.390755+0000","last_clean":"2024-04-25T08:42:27.390755+0000","last_became_active":"2024-04-25T08:42:21.358285+0000","last_became_peered":"2024-04-25T08:42:21.358285+0000","last_unstale":"2024-04-25T08:42:27.390755+0000","last_undegraded":"2024-04-25T08:42:27.390755+0000","last_fullsized":"2024-04-25T08:42:27.390755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:45.573590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677940+0000","last_change":"2024-04-25T08:42:20.357678+0000","last_active":"2024-04-25T08:42:28.677940+0000","last_peered":"2024-04-25T08:42:28.677940+0000","last_clean":"2024-04-25T08:42:28.677940+0000","last_became_active":"2024-04-25T08:42:20.357378+0000","last_became_peered":"2024-04-25T08:42:20.357378+0000","last_unstale":"2024-04-25T08:42:28.677940+0000","last_undegraded":"2024-04-25T08:42:28.677940+0000","last_fullsized":"2024-04-25T08:42:28.677940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390203+0000","last_change":"2024-04-25T08:42:25.406294+0000","last_active":"2024-04-25T08:42:27.390203+0000","last_peered":"2024-04-25T08:42:27.390203+0000","last_clean":"2024-04-25T08:42:27.390203+0000","last_became_active":"2024-04-25T08:42:25.405833+0000","last_became_peered":"2024-04-25T08:42:25.405833+0000","last_unstale":"2024-04-25T08:42:27.390203+0000","last_undegraded":"2024-04-25T08:42:27.390203+0000","last_fullsized":"2024-04-25T08:42:27.390203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:42.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683217+0000","last_change":"2024-04-25T08:42:21.363445+0000","last_active":"2024-04-25T08:42:28.683217+0000","last_peered":"2024-04-25T08:42:28.683217+0000","last_clean":"2024-04-25T08:42:28.683217+0000","last_became_active":"2024-04-25T08:42:21.363129+0000","last_became_peered":"2024-04-25T08:42:21.363129+0000","last_unstale":"2024-04-25T08:42:28.683217+0000","last_undegraded":"2024-04-25T08:42:28.683217+0000","last_fullsized":"2024-04-25T08:42:28.683217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583076+0000","last_change":"2024-04-25T08:42:20.352637+0000","last_active":"2024-04-25T08:42:27.583076+0000","last_peered":"2024-04-25T08:42:27.583076+0000","last_clean":"2024-04-25T08:42:27.583076+0000","last_became_active":"2024-04-25T08:42:20.351776+0000","last_became_peered":"2024-04-25T08:42:20.351776+0000","last_unstale":"2024-04-25T08:42:27.583076+0000","last_undegraded":"2024-04-25T08:42:27.583076+0000","last_fullsized":"2024-04-25T08:42:27.583076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:53.741991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384221+0000","last_change":"2024-04-25T08:42:25.406488+0000","last_active":"2024-04-25T08:42:27.384221+0000","last_peered":"2024-04-25T08:42:27.384221+0000","last_clean":"2024-04-25T08:42:27.384221+0000","last_became_active":"2024-04-25T08:42:25.405903+0000","last_became_peered":"2024-04-25T08:42:25.405903+0000","last_unstale":"2024-04-25T08:42:27.384221+0000","last_undegraded":"2024-04-25T08:42:27.384221+0000","last_fullsized":"2024-04-25T08:42:27.384221+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:52.897838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583032+0000","last_change":"2024-04-25T08:42:21.376939+0000","last_active":"2024-04-25T08:42:27.583032+0000","last_peered":"2024-04-25T08:42:27.583032+0000","last_clean":"2024-04-25T08:42:27.583032+0000","last_became_active":"2024-04-25T08:42:21.376693+0000","last_became_peered":"2024-04-25T08:42:21.376693+0000","last_unstale":"2024-04-25T08:42:27.583032+0000","last_undegraded":"2024-04-25T08:42:27.583032+0000","last_fullsized":"2024-04-25T08:42:27.583032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:16.561880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389379+0000","last_change":"2024-04-25T08:42:20.355744+0000","last_active":"2024-04-25T08:42:27.389379+0000","last_peered":"2024-04-25T08:42:27.389379+0000","last_clean":"2024-04-25T08:42:27.389379+0000","last_became_active":"2024-04-25T08:42:20.355365+0000","last_became_peered":"2024-04-25T08:42:20.355365+0000","last_unstale":"2024-04-25T08:42:27.389379+0000","last_undegraded":"2024-04-25T08:42:27.389379+0000","last_fullsized":"2024-04-25T08:42:27.389379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384187+0000","last_change":"2024-04-25T08:42:25.833881+0000","last_active":"2024-04-25T08:42:27.384187+0000","last_peered":"2024-04-25T08:42:27.384187+0000","last_clean":"2024-04-25T08:42:27.384187+0000","last_became_active":"2024-04-25T08:42:25.833475+0000","last_became_peered":"2024-04-25T08:42:25.833475+0000","last_unstale":"2024-04-25T08:42:27.384187+0000","last_undegraded":"2024-04-25T08:42:27.384187+0000","last_fullsized":"2024-04-25T08:42:27.384187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:49.130328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683043+0000","last_change":"2024-04-25T08:42:25.835305+0000","last_active":"2024-04-25T08:42:28.683043+0000","last_peered":"2024-04-25T08:42:28.683043+0000","last_clean":"2024-04-25T08:42:28.683043+0000","last_became_active":"2024-04-25T08:42:25.834632+0000","last_became_peered":"2024-04-25T08:42:25.834632+0000","last_unstale":"2024-04-25T08:42:28.683043+0000","last_undegraded":"2024-04-25T08:42:28.683043+0000","last_fullsized":"2024-04-25T08:42:28.683043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678200+0000","last_change":"2024-04-25T08:42:21.350716+0000","last_active":"2024-04-25T08:42:28.678200+0000","last_peered":"2024-04-25T08:42:28.678200+0000","last_clean":"2024-04-25T08:42:28.678200+0000","last_became_active":"2024-04-25T08:42:21.349383+0000","last_became_peered":"2024-04-25T08:42:21.349383+0000","last_unstale":"2024-04-25T08:42:28.678200+0000","last_undegraded":"2024-04-25T08:42:28.678200+0000","last_fullsized":"2024-04-25T08:42:28.678200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582596+0000","last_change":"2024-04-25T08:42:20.345077+0000","last_active":"2024-04-25T08:42:27.582596+0000","last_peered":"2024-04-25T08:42:27.582596+0000","last_clean":"2024-04-25T08:42:27.582596+0000","last_became_active":"2024-04-25T08:42:20.344526+0000","last_became_peered":"2024-04-25T08:42:20.344526+0000","last_unstale":"2024-04-25T08:42:27.582596+0000","last_undegraded":"2024-04-25T08:42:27.582596+0000","last_fullsized":"2024-04-25T08:42:27.582596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389292+0000","last_change":"2024-04-25T08:42:25.407527+0000","last_active":"2024-04-25T08:42:27.389292+0000","last_peered":"2024-04-25T08:42:27.389292+0000","last_clean":"2024-04-25T08:42:27.389292+0000","last_became_active":"2024-04-25T08:42:25.407221+0000","last_became_peered":"2024-04-25T08:42:25.407221+0000","last_unstale":"2024-04-25T08:42:27.389292+0000","last_undegraded":"2024-04-25T08:42:27.389292+0000","last_fullsized":"2024-04-25T08:42:27.389292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.610141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384221+0000","last_change":"2024-04-25T08:42:21.351795+0000","last_active":"2024-04-25T08:42:27.384221+0000","last_peered":"2024-04-25T08:42:27.384221+0000","last_clean":"2024-04-25T08:42:27.384221+0000","last_became_active":"2024-04-25T08:42:21.351544+0000","last_became_peered":"2024-04-25T08:42:21.351544+0000","last_unstale":"2024-04-25T08:42:27.384221+0000","last_undegraded":"2024-04-25T08:42:27.384221+0000","last_fullsized":"2024-04-25T08:42:27.384221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384681+0000","last_change":"2024-04-25T08:42:20.354629+0000","last_active":"2024-04-25T08:42:27.384681+0000","last_peered":"2024-04-25T08:42:27.384681+0000","last_clean":"2024-04-25T08:42:27.384681+0000","last_became_active":"2024-04-25T08:42:20.354388+0000","last_became_peered":"2024-04-25T08:42:20.354388+0000","last_unstale":"2024-04-25T08:42:27.384681+0000","last_undegraded":"2024-04-25T08:42:27.384681+0000","last_fullsized":"2024-04-25T08:42:27.384681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.311211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388979+0000","last_change":"2024-04-25T08:42:20.350141+0000","last_active":"2024-04-25T08:42:27.388979+0000","last_peered":"2024-04-25T08:42:27.388979+0000","last_clean":"2024-04-25T08:42:27.388979+0000","last_became_active":"2024-04-25T08:42:20.349168+0000","last_became_peered":"2024-04-25T08:42:20.349168+0000","last_unstale":"2024-04-25T08:42:27.388979+0000","last_undegraded":"2024-04-25T08:42:27.388979+0000","last_fullsized":"2024-04-25T08:42:27.388979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683007+0000","last_change":"2024-04-25T08:42:21.363608+0000","last_active":"2024-04-25T08:42:28.683007+0000","last_peered":"2024-04-25T08:42:28.683007+0000","last_clean":"2024-04-25T08:42:28.683007+0000","last_became_active":"2024-04-25T08:42:21.363152+0000","last_became_peered":"2024-04-25T08:42:21.363152+0000","last_unstale":"2024-04-25T08:42:28.683007+0000","last_undegraded":"2024-04-25T08:42:28.683007+0000","last_fullsized":"2024-04-25T08:42:28.683007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583434+0000","last_change":"2024-04-25T08:42:25.413462+0000","last_active":"2024-04-25T08:42:28.583434+0000","last_peered":"2024-04-25T08:42:28.583434+0000","last_clean":"2024-04-25T08:42:28.583434+0000","last_became_active":"2024-04-25T08:42:25.412613+0000","last_became_peered":"2024-04-25T08:42:25.412613+0000","last_unstale":"2024-04-25T08:42:28.583434+0000","last_undegraded":"2024-04-25T08:42:28.583434+0000","last_fullsized":"2024-04-25T08:42:28.583434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:52.601446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384923+0000","last_change":"2024-04-25T08:42:20.355705+0000","last_active":"2024-04-25T08:42:27.384923+0000","last_peered":"2024-04-25T08:42:27.384923+0000","last_clean":"2024-04-25T08:42:27.384923+0000","last_became_active":"2024-04-25T08:42:20.355433+0000","last_became_peered":"2024-04-25T08:42:20.355433+0000","last_unstale":"2024-04-25T08:42:27.384923+0000","last_undegraded":"2024-04-25T08:42:27.384923+0000","last_fullsized":"2024-04-25T08:42:27.384923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:33.260664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390028+0000","last_change":"2024-04-25T08:42:21.344759+0000","last_active":"2024-04-25T08:42:27.390028+0000","last_peered":"2024-04-25T08:42:27.390028+0000","last_clean":"2024-04-25T08:42:27.390028+0000","last_became_active":"2024-04-25T08:42:21.344560+0000","last_became_peered":"2024-04-25T08:42:21.344560+0000","last_unstale":"2024-04-25T08:42:27.390028+0000","last_undegraded":"2024-04-25T08:42:27.390028+0000","last_fullsized":"2024-04-25T08:42:27.390028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:32.295514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385225+0000","last_change":"2024-04-25T08:42:25.413565+0000","last_active":"2024-04-25T08:42:27.385225+0000","last_peered":"2024-04-25T08:42:27.385225+0000","last_clean":"2024-04-25T08:42:27.385225+0000","last_became_active":"2024-04-25T08:42:25.413198+0000","last_became_peered":"2024-04-25T08:42:25.413198+0000","last_unstale":"2024-04-25T08:42:27.385225+0000","last_undegraded":"2024-04-25T08:42:27.385225+0000","last_fullsized":"2024-04-25T08:42:27.385225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:13.096012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388406+0000","last_change":"2024-04-25T08:42:21.363343+0000","last_active":"2024-04-25T08:42:27.388406+0000","last_peered":"2024-04-25T08:42:27.388406+0000","last_clean":"2024-04-25T08:42:27.388406+0000","last_became_active":"2024-04-25T08:42:21.363101+0000","last_became_peered":"2024-04-25T08:42:21.363101+0000","last_unstale":"2024-04-25T08:42:27.388406+0000","last_undegraded":"2024-04-25T08:42:27.388406+0000","last_fullsized":"2024-04-25T08:42:27.388406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583299+0000","last_change":"2024-04-25T08:42:21.361026+0000","last_active":"2024-04-25T08:42:28.583299+0000","last_peered":"2024-04-25T08:42:28.583299+0000","last_clean":"2024-04-25T08:42:28.583299+0000","last_became_active":"2024-04-25T08:42:21.360012+0000","last_became_peered":"2024-04-25T08:42:21.360012+0000","last_unstale":"2024-04-25T08:42:28.583299+0000","last_undegraded":"2024-04-25T08:42:28.583299+0000","last_fullsized":"2024-04-25T08:42:28.583299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:29.830442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582614+0000","last_change":"2024-04-25T08:42:21.367540+0000","last_active":"2024-04-25T08:42:27.582614+0000","last_peered":"2024-04-25T08:42:27.582614+0000","last_clean":"2024-04-25T08:42:27.582614+0000","last_became_active":"2024-04-25T08:42:21.367291+0000","last_became_peered":"2024-04-25T08:42:21.367291+0000","last_unstale":"2024-04-25T08:42:27.582614+0000","last_undegraded":"2024-04-25T08:42:27.582614+0000","last_fullsized":"2024-04-25T08:42:27.582614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.923264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683000+0000","last_change":"2024-04-25T08:42:21.366788+0000","last_active":"2024-04-25T08:42:28.683000+0000","last_peered":"2024-04-25T08:42:28.683000+0000","last_clean":"2024-04-25T08:42:28.683000+0000","last_became_active":"2024-04-25T08:42:21.365945+0000","last_became_peered":"2024-04-25T08:42:21.365945+0000","last_unstale":"2024-04-25T08:42:28.683000+0000","last_undegraded":"2024-04-25T08:42:28.683000+0000","last_fullsized":"2024-04-25T08:42:28.683000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385215+0000","last_change":"2024-04-25T08:42:21.351963+0000","last_active":"2024-04-25T08:42:27.385215+0000","last_peered":"2024-04-25T08:42:27.385215+0000","last_clean":"2024-04-25T08:42:27.385215+0000","last_became_active":"2024-04-25T08:42:21.351587+0000","last_became_peered":"2024-04-25T08:42:21.351587+0000","last_unstale":"2024-04-25T08:42:27.385215+0000","last_undegraded":"2024-04-25T08:42:27.385215+0000","last_fullsized":"2024-04-25T08:42:27.385215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:30.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678682+0000","last_change":"2024-04-25T08:42:21.363969+0000","last_active":"2024-04-25T08:42:28.678682+0000","last_peered":"2024-04-25T08:42:28.678682+0000","last_clean":"2024-04-25T08:42:28.678682+0000","last_became_active":"2024-04-25T08:42:21.354733+0000","last_became_peered":"2024-04-25T08:42:21.354733+0000","last_unstale":"2024-04-25T08:42:28.678682+0000","last_undegraded":"2024-04-25T08:42:28.678682+0000","last_fullsized":"2024-04-25T08:42:28.678682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389184+0000","last_change":"2024-04-25T08:42:21.375124+0000","last_active":"2024-04-25T08:42:27.389184+0000","last_peered":"2024-04-25T08:42:27.389184+0000","last_clean":"2024-04-25T08:42:27.389184+0000","last_became_active":"2024-04-25T08:42:21.374653+0000","last_became_peered":"2024-04-25T08:42:21.374653+0000","last_unstale":"2024-04-25T08:42:27.389184+0000","last_undegraded":"2024-04-25T08:42:27.389184+0000","last_fullsized":"2024-04-25T08:42:27.389184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385387+0000","last_change":"2024-04-25T08:42:21.362798+0000","last_active":"2024-04-25T08:42:27.385387+0000","last_peered":"2024-04-25T08:42:27.385387+0000","last_clean":"2024-04-25T08:42:27.385387+0000","last_became_active":"2024-04-25T08:42:21.362412+0000","last_became_peered":"2024-04-25T08:42:21.362412+0000","last_unstale":"2024-04-25T08:42:27.385387+0000","last_undegraded":"2024-04-25T08:42:27.385387+0000","last_fullsized":"2024-04-25T08:42:27.385387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582584+0000","last_change":"2024-04-25T08:42:21.368298+0000","last_active":"2024-04-25T08:42:27.582584+0000","last_peered":"2024-04-25T08:42:27.582584+0000","last_clean":"2024-04-25T08:42:27.582584+0000","last_became_active":"2024-04-25T08:42:21.368073+0000","last_became_peered":"2024-04-25T08:42:21.368073+0000","last_unstale":"2024-04-25T08:42:27.582584+0000","last_undegraded":"2024-04-25T08:42:27.582584+0000","last_fullsized":"2024-04-25T08:42:27.582584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384593+0000","last_change":"2024-04-25T08:42:21.356172+0000","last_active":"2024-04-25T08:42:27.384593+0000","last_peered":"2024-04-25T08:42:27.384593+0000","last_clean":"2024-04-25T08:42:27.384593+0000","last_became_active":"2024-04-25T08:42:21.355920+0000","last_became_peered":"2024-04-25T08:42:21.355920+0000","last_unstale":"2024-04-25T08:42:27.384593+0000","last_undegraded":"2024-04-25T08:42:27.384593+0000","last_fullsized":"2024-04-25T08:42:27.384593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582542+0000","last_change":"2024-04-25T08:42:21.376313+0000","last_active":"2024-04-25T08:42:27.582542+0000","last_peered":"2024-04-25T08:42:27.582542+0000","last_clean":"2024-04-25T08:42:27.582542+0000","last_became_active":"2024-04-25T08:42:21.376047+0000","last_became_peered":"2024-04-25T08:42:21.376047+0000","last_unstale":"2024-04-25T08:42:27.582542+0000","last_undegraded":"2024-04-25T08:42:27.582542+0000","last_fullsized":"2024-04-25T08:42:27.582542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:24.505703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682755+0000","last_change":"2024-04-25T08:42:21.367017+0000","last_active":"2024-04-25T08:42:28.682755+0000","last_peered":"2024-04-25T08:42:28.682755+0000","last_clean":"2024-04-25T08:42:28.682755+0000","last_became_active":"2024-04-25T08:42:21.366480+0000","last_became_peered":"2024-04-25T08:42:21.366480+0000","last_unstale":"2024-04-25T08:42:28.682755+0000","last_undegraded":"2024-04-25T08:42:28.682755+0000","last_fullsized":"2024-04-25T08:42:28.682755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388881+0000","last_change":"2024-04-25T08:42:21.354458+0000","last_active":"2024-04-25T08:42:27.388881+0000","last_peered":"2024-04-25T08:42:27.388881+0000","last_clean":"2024-04-25T08:42:27.388881+0000","last_became_active":"2024-04-25T08:42:21.351083+0000","last_became_peered":"2024-04-25T08:42:21.351083+0000","last_unstale":"2024-04-25T08:42:27.388881+0000","last_undegraded":"2024-04-25T08:42:27.388881+0000","last_fullsized":"2024-04-25T08:42:27.388881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684511+0000","last_change":"2024-04-25T08:42:21.369898+0000","last_active":"2024-04-25T08:42:28.684511+0000","last_peered":"2024-04-25T08:42:28.684511+0000","last_clean":"2024-04-25T08:42:28.684511+0000","last_became_active":"2024-04-25T08:42:21.369443+0000","last_became_peered":"2024-04-25T08:42:21.369443+0000","last_unstale":"2024-04-25T08:42:28.684511+0000","last_undegraded":"2024-04-25T08:42:28.684511+0000","last_fullsized":"2024-04-25T08:42:28.684511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:16.561880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583661+0000","last_change":"2024-04-25T08:42:21.353156+0000","last_active":"2024-04-25T08:42:28.583661+0000","last_peered":"2024-04-25T08:42:28.583661+0000","last_clean":"2024-04-25T08:42:28.583661+0000","last_became_active":"2024-04-25T08:42:21.352670+0000","last_became_peered":"2024-04-25T08:42:21.352670+0000","last_unstale":"2024-04-25T08:42:28.583661+0000","last_undegraded":"2024-04-25T08:42:28.583661+0000","last_fullsized":"2024-04-25T08:42:28.583661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:45.573590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583361+0000","last_change":"2024-04-25T08:42:21.376357+0000","last_active":"2024-04-25T08:42:27.583361+0000","last_peered":"2024-04-25T08:42:27.583361+0000","last_clean":"2024-04-25T08:42:27.583361+0000","last_became_active":"2024-04-25T08:42:21.376092+0000","last_became_peered":"2024-04-25T08:42:21.376092+0000","last_unstale":"2024-04-25T08:42:27.583361+0000","last_undegraded":"2024-04-25T08:42:27.583361+0000","last_fullsized":"2024-04-25T08:42:27.583361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:15.923334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582441+0000","last_change":"2024-04-25T08:42:21.371951+0000","last_active":"2024-04-25T08:42:27.582441+0000","last_peered":"2024-04-25T08:42:27.582441+0000","last_clean":"2024-04-25T08:42:27.582441+0000","last_became_active":"2024-04-25T08:42:21.371306+0000","last_became_peered":"2024-04-25T08:42:21.371306+0000","last_unstale":"2024-04-25T08:42:27.582441+0000","last_undegraded":"2024-04-25T08:42:27.582441+0000","last_fullsized":"2024-04-25T08:42:27.582441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678232+0000","last_change":"2024-04-25T08:42:21.341496+0000","last_active":"2024-04-25T08:42:28.678232+0000","last_peered":"2024-04-25T08:42:28.678232+0000","last_clean":"2024-04-25T08:42:28.678232+0000","last_became_active":"2024-04-25T08:42:21.341224+0000","last_became_peered":"2024-04-25T08:42:21.341224+0000","last_unstale":"2024-04-25T08:42:28.678232+0000","last_undegraded":"2024-04-25T08:42:28.678232+0000","last_fullsized":"2024-04-25T08:42:28.678232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682722+0000","last_change":"2024-04-25T08:42:21.360668+0000","last_active":"2024-04-25T08:42:28.682722+0000","last_peered":"2024-04-25T08:42:28.682722+0000","last_clean":"2024-04-25T08:42:28.682722+0000","last_became_active":"2024-04-25T08:42:21.360398+0000","last_became_peered":"2024-04-25T08:42:21.360398+0000","last_unstale":"2024-04-25T08:42:28.682722+0000","last_undegraded":"2024-04-25T08:42:28.682722+0000","last_fullsized":"2024-04-25T08:42:28.682722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:33.626044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384826+0000","last_change":"2024-04-25T08:42:21.353592+0000","last_active":"2024-04-25T08:42:27.384826+0000","last_peered":"2024-04-25T08:42:27.384826+0000","last_clean":"2024-04-25T08:42:27.384826+0000","last_became_active":"2024-04-25T08:42:21.353329+0000","last_became_peered":"2024-04-25T08:42:21.353329+0000","last_unstale":"2024-04-25T08:42:27.384826+0000","last_undegraded":"2024-04-25T08:42:27.384826+0000","last_fullsized":"2024-04-25T08:42:27.384826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390624+0000","last_change":"2024-04-25T08:42:21.356315+0000","last_active":"2024-04-25T08:42:27.390624+0000","last_peered":"2024-04-25T08:42:27.390624+0000","last_clean":"2024-04-25T08:42:27.390624+0000","last_became_active":"2024-04-25T08:42:21.355905+0000","last_became_peered":"2024-04-25T08:42:21.355905+0000","last_unstale":"2024-04-25T08:42:27.390624+0000","last_undegraded":"2024-04-25T08:42:27.390624+0000","last_fullsized":"2024-04-25T08:42:27.390624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:48.723555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389154+0000","last_change":"2024-04-25T08:42:21.371337+0000","last_active":"2024-04-25T08:42:27.389154+0000","last_peered":"2024-04-25T08:42:27.389154+0000","last_clean":"2024-04-25T08:42:27.389154+0000","last_became_active":"2024-04-25T08:42:21.370950+0000","last_became_peered":"2024-04-25T08:42:21.370950+0000","last_unstale":"2024-04-25T08:42:27.389154+0000","last_undegraded":"2024-04-25T08:42:27.389154+0000","last_fullsized":"2024-04-25T08:42:27.389154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389045+0000","last_change":"2024-04-25T08:42:21.365819+0000","last_active":"2024-04-25T08:42:27.389045+0000","last_peered":"2024-04-25T08:42:27.389045+0000","last_clean":"2024-04-25T08:42:27.389045+0000","last_became_active":"2024-04-25T08:42:21.365562+0000","last_became_peered":"2024-04-25T08:42:21.365562+0000","last_unstale":"2024-04-25T08:42:27.389045+0000","last_undegraded":"2024-04-25T08:42:27.389045+0000","last_fullsized":"2024-04-25T08:42:27.389045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684772+0000","last_change":"2024-04-25T08:42:21.366180+0000","last_active":"2024-04-25T08:42:28.684772+0000","last_peered":"2024-04-25T08:42:28.684772+0000","last_clean":"2024-04-25T08:42:28.684772+0000","last_became_active":"2024-04-25T08:42:21.365934+0000","last_became_peered":"2024-04-25T08:42:21.365934+0000","last_unstale":"2024-04-25T08:42:28.684772+0000","last_undegraded":"2024-04-25T08:42:28.684772+0000","last_fullsized":"2024-04-25T08:42:28.684772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:36.740234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384130+0000","last_change":"2024-04-25T08:42:21.368124+0000","last_active":"2024-04-25T08:42:27.384130+0000","last_peered":"2024-04-25T08:42:27.384130+0000","last_clean":"2024-04-25T08:42:27.384130+0000","last_became_active":"2024-04-25T08:42:21.367798+0000","last_became_peered":"2024-04-25T08:42:21.367798+0000","last_unstale":"2024-04-25T08:42:27.384130+0000","last_undegraded":"2024-04-25T08:42:27.384130+0000","last_fullsized":"2024-04-25T08:42:27.384130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683866+0000","last_change":"2024-04-25T08:42:21.357410+0000","last_active":"2024-04-25T08:42:28.683866+0000","last_peered":"2024-04-25T08:42:28.683866+0000","last_clean":"2024-04-25T08:42:28.683866+0000","last_became_active":"2024-04-25T08:42:21.356770+0000","last_became_peered":"2024-04-25T08:42:21.356770+0000","last_unstale":"2024-04-25T08:42:28.683866+0000","last_undegraded":"2024-04-25T08:42:28.683866+0000","last_fullsized":"2024-04-25T08:42:28.683866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582479+0000","last_change":"2024-04-25T08:42:21.372610+0000","last_active":"2024-04-25T08:42:27.582479+0000","last_peered":"2024-04-25T08:42:27.582479+0000","last_clean":"2024-04-25T08:42:27.582479+0000","last_became_active":"2024-04-25T08:42:21.372336+0000","last_became_peered":"2024-04-25T08:42:21.372336+0000","last_unstale":"2024-04-25T08:42:27.582479+0000","last_undegraded":"2024-04-25T08:42:27.582479+0000","last_fullsized":"2024-04-25T08:42:27.582479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:25.717765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684118+0000","last_change":"2024-04-25T08:42:21.358088+0000","last_active":"2024-04-25T08:42:28.684118+0000","last_peered":"2024-04-25T08:42:28.684118+0000","last_clean":"2024-04-25T08:42:28.684118+0000","last_became_active":"2024-04-25T08:42:21.357310+0000","last_became_peered":"2024-04-25T08:42:21.357310+0000","last_unstale":"2024-04-25T08:42:28.684118+0000","last_undegraded":"2024-04-25T08:42:28.684118+0000","last_fullsized":"2024-04-25T08:42:28.684118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582808+0000","last_change":"2024-04-25T08:42:21.367883+0000","last_active":"2024-04-25T08:42:27.582808+0000","last_peered":"2024-04-25T08:42:27.582808+0000","last_clean":"2024-04-25T08:42:27.582808+0000","last_became_active":"2024-04-25T08:42:21.367176+0000","last_became_peered":"2024-04-25T08:42:21.367176+0000","last_unstale":"2024-04-25T08:42:27.582808+0000","last_undegraded":"2024-04-25T08:42:27.582808+0000","last_fullsized":"2024-04-25T08:42:27.582808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.525554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682913+0000","last_change":"2024-04-25T08:42:21.362174+0000","last_active":"2024-04-25T08:42:28.682913+0000","last_peered":"2024-04-25T08:42:28.682913+0000","last_clean":"2024-04-25T08:42:28.682913+0000","last_became_active":"2024-04-25T08:42:21.361897+0000","last_became_peered":"2024-04-25T08:42:21.361897+0000","last_unstale":"2024-04-25T08:42:28.682913+0000","last_undegraded":"2024-04-25T08:42:28.682913+0000","last_fullsized":"2024-04-25T08:42:28.682913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678838+0000","last_change":"2024-04-25T08:42:21.370457+0000","last_active":"2024-04-25T08:42:28.678838+0000","last_peered":"2024-04-25T08:42:28.678838+0000","last_clean":"2024-04-25T08:42:28.678838+0000","last_became_active":"2024-04-25T08:42:21.370091+0000","last_became_peered":"2024-04-25T08:42:21.370091+0000","last_unstale":"2024-04-25T08:42:28.678838+0000","last_undegraded":"2024-04-25T08:42:28.678838+0000","last_fullsized":"2024-04-25T08:42:28.678838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684699+0000","last_change":"2024-04-25T08:42:21.370936+0000","last_active":"2024-04-25T08:42:28.684699+0000","last_peered":"2024-04-25T08:42:28.684699+0000","last_clean":"2024-04-25T08:42:28.684699+0000","last_became_active":"2024-04-25T08:42:21.369447+0000","last_became_peered":"2024-04-25T08:42:21.369447+0000","last_unstale":"2024-04-25T08:42:28.684699+0000","last_undegraded":"2024-04-25T08:42:28.684699+0000","last_fullsized":"2024-04-25T08:42:28.684699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:38.445314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:45.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:45.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.451+0000 7f26bec00640 1 Processor -- start 2024-04-25T08:42:45.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.451+0000 7f26bec00640 1 -- start start 2024-04-25T08:42:45.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.451+0000 7f26bec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26b815a750 0x7f26b806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.451+0000 7f26bec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b806d5e0 con 0x7f26b80aed10 2024-04-25T08:42:45.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.451+0000 7f26bec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b80ce720 con 0x7f26b80cc9e0 2024-04-25T08:42:45.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.451+0000 7f26bec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b815ab30 con 0x7f26b815a750 2024-04-25T08:42:45.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.451+0000 7f26b7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26b815a750 0x7f26b806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.451+0000 7f26afe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26b80aed10 0x7f26b80b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59192/0 (socket says 172.21.15.46:59192) 2024-04-25T08:42:45.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.451+0000 7f26afe00640 1 -- 172.21.15.46:0/634771399 learned_addr learned my addr 172.21.15.46:0/634771399 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:45.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.455+0000 7f26b7400640 1 -- 172.21.15.46:0/634771399 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107343484 0 0) 0x7f26b806d5e0 con 0x7f26b80aed10 2024-04-25T08:42:45.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.455+0000 7f26b7400640 1 -- 172.21.15.46:0/634771399 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f269c003530 con 0x7f26b80aed10 2024-04-25T08:42:45.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.455+0000 7f26b7e00640 1 -- 172.21.15.46:0/634771399 >> v1:172.21.15.82:6789/0 conn(0x7f26b80cc9e0 legacy=0x7f26b80b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.455+0000 7f26b7e00640 1 -- 172.21.15.46:0/634771399 >> v1:172.21.15.46:6789/0 conn(0x7f26b80aed10 legacy=0x7f26b80b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.455+0000 7f26b7e00640 1 -- 172.21.15.46:0/634771399 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b80cea00 con 0x7f26b815a750 2024-04-25T08:42:45.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.455+0000 7f26b7e00640 1 --2- 172.21.15.46:0/634771399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 0x7f26b806ec10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26a80097c0 tx=0x7f26a801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acf2907a1e20600 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:45.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.455+0000 7f26b7400640 1 -- 172.21.15.46:0/634771399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a80b0070 con 0x7f26b815a750 2024-04-25T08:42:45.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.455+0000 7f26b7400640 1 -- 172.21.15.46:0/634771399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a80ab600 con 0x7f26b815a750 2024-04-25T08:42:45.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.455+0000 7f26b7400640 1 -- 172.21.15.46:0/634771399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a80ab9d0 con 0x7f26b815a750 2024-04-25T08:42:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.459+0000 7f26bec00640 1 -- 172.21.15.46:0/634771399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 msgr2=0x7f26b806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.459+0000 7f26bec00640 1 --2- 172.21.15.46:0/634771399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 0x7f26b806ec10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f26a80097c0 tx=0x7f26a801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.459+0000 7f26bec00640 1 -- 172.21.15.46:0/634771399 shutdown_connections 2024-04-25T08:42:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.459+0000 7f26bec00640 1 --2- 172.21.15.46:0/634771399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 0x7f26b806ec10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.459+0000 7f26bec00640 1 -- 172.21.15.46:0/634771399 >> 172.21.15.46:0/634771399 conn(0x7f26b80b12a0 msgr2=0x7f26b80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.459+0000 7f26bec00640 1 -- 172.21.15.46:0/634771399 shutdown_connections 2024-04-25T08:42:45.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.459+0000 7f26bec00640 1 -- 172.21.15.46:0/634771399 wait complete. 2024-04-25T08:42:45.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.459+0000 7f26bec00640 1 Processor -- start 2024-04-25T08:42:45.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.459+0000 7f26bec00640 1 -- start start 2024-04-25T08:42:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b80aed10 0x7f26b8158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26b80cc9e0 0x7f26b8141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 0x7f26b815b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8168a50 con 0x7f26b815a750 2024-04-25T08:42:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b806f560 con 0x7f26b80aed10 2024-04-25T08:42:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8168bd0 con 0x7f26b80cc9e0 2024-04-25T08:42:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 0x7f26b815b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 0x7f26b815b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60430/0 (socket says 172.21.15.46:60430) 2024-04-25T08:42:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bcc00640 1 -- 172.21.15.46:0/2421303189 learned_addr learned my addr 172.21.15.46:0/2421303189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26b7e00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b80aed10 0x7f26b8158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26afe00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26b80cc9e0 0x7f26b8141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bcc00640 1 -- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26b80cc9e0 msgr2=0x7f26b8141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bcc00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26b80cc9e0 0x7f26b8141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bcc00640 1 -- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b80aed10 msgr2=0x7f26b8158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bcc00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b80aed10 0x7f26b8158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bcc00640 1 -- 172.21.15.46:0/2421303189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b81590d0 con 0x7f26b815a750 2024-04-25T08:42:45.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bcc00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 0x7f26b815b430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26b000c7a0 tx=0x7f26b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26b4c00640 1 -- 172.21.15.46:0/2421303189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0003b80 con 0x7f26b815a750 2024-04-25T08:42:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26b4c00640 1 -- 172.21.15.46:0/2421303189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b0003d20 con 0x7f26b815a750 2024-04-25T08:42:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26b4c00640 1 -- 172.21.15.46:0/2421303189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b000aa90 con 0x7f26b815a750 2024-04-25T08:42:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b81593b0 con 0x7f26b815a750 2024-04-25T08:42:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b815af10 con 0x7f26b815a750 2024-04-25T08:42:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b8069080 con 0x7f26b815a750 2024-04-25T08:42:45.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.463+0000 7f26b4c00640 1 -- 172.21.15.46:0/2421303189 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26b000ac30 con 0x7f26b815a750 2024-04-25T08:42:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.467+0000 7f26b4c00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2694066f90 0x7f2694069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.467+0000 7f26b4c00640 1 -- 172.21.15.46:0/2421303189 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f26b00851e0 con 0x7f26b815a750 2024-04-25T08:42:45.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.467+0000 7f26b7e00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2694066f90 0x7f2694069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.467+0000 7f26b7e00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2694066f90 0x7f2694069450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f26a80ad000 tx=0x7f26a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:45.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.471+0000 7f26b4c00640 1 -- 172.21.15.46:0/2421303189 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26b0055860 con 0x7f26b815a750 2024-04-25T08:42:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.627+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26b815b0e0 con 0x7f2694066f90 2024-04-25T08:42:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.639+0000 7f26b4c00640 1 -- 172.21.15.46:0/2421303189 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332538 (secure 0 0 0) 0x7f26b815b0e0 con 0x7f2694066f90 2024-04-25T08:42:45.634 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:45.635 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:42:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.639+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2694066f90 msgr2=0x7f2694069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.639+0000 7f26bec00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2694066f90 0x7f2694069450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f26a80ad000 tx=0x7f26a8008040 comp rx=0 tx=0).stop 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.639+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 msgr2=0x7f26b815b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.639+0000 7f26bec00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 0x7f26b815b430 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26b000c7a0 tx=0x7f26b000cc60 comp rx=0 tx=0).stop 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.643+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 shutdown_connections 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.643+0000 7f26bec00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2694066f90 0x7f2694069450 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.643+0000 7f26bec00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26b815a750 0x7f26b815b430 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.643+0000 7f26bec00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26b80cc9e0 0x7f26b8141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.643+0000 7f26bec00640 1 --2- 172.21.15.46:0/2421303189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26b80aed10 0x7f26b8158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.643+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 >> 172.21.15.46:0/2421303189 conn(0x7f26b80b12a0 msgr2=0x7f26b8151d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.643+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 shutdown_connections 2024-04-25T08:42:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.643+0000 7f26bec00640 1 -- 172.21.15.46:0/2421303189 wait complete. 2024-04-25T08:42:45.653 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T08:42:44.257962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223716,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751004,"statfs":{"total":773094113280,"available":772865028096,"internally_reserved":0,"allocated":3481600,"data_stored":2086528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":225496626},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003315"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":28,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683286+0000","last_change":"2024-04-25T08:42:20.357496+0000","last_active":"2024-04-25T08:42:28.683286+0000","last_peered":"2024-04-25T08:42:28.683286+0000","last_clean":"2024-04-25T08:42:28.683286+0000","last_became_active":"2024-04-25T08:42:20.357232+0000","last_became_peered":"2024-04-25T08:42:20.357232+0000","last_unstale":"2024-04-25T08:42:28.683286+0000","last_undegraded":"2024-04-25T08:42:28.683286+0000","last_fullsized":"2024-04-25T08:42:28.683286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:43.347447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385143+0000","last_change":"2024-04-25T08:42:21.359572+0000","last_active":"2024-04-25T08:42:27.385143+0000","last_peered":"2024-04-25T08:42:27.385143+0000","last_clean":"2024-04-25T08:42:27.385143+0000","last_became_active":"2024-04-25T08:42:21.357850+0000","last_became_peered":"2024-04-25T08:42:21.357850+0000","last_unstale":"2024-04-25T08:42:27.385143+0000","last_undegraded":"2024-04-25T08:42:27.385143+0000","last_fullsized":"2024-04-25T08:42:27.385143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:25.717765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384099+0000","last_change":"2024-04-25T08:42:25.834318+0000","last_active":"2024-04-25T08:42:27.384099+0000","last_peered":"2024-04-25T08:42:27.384099+0000","last_clean":"2024-04-25T08:42:27.384099+0000","last_became_active":"2024-04-25T08:42:25.833964+0000","last_became_peered":"2024-04-25T08:42:25.833964+0000","last_unstale":"2024-04-25T08:42:27.384099+0000","last_undegraded":"2024-04-25T08:42:27.384099+0000","last_fullsized":"2024-04-25T08:42:27.384099+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:37.125230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385326+0000","last_change":"2024-04-25T08:42:20.351984+0000","last_active":"2024-04-25T08:42:27.385326+0000","last_peered":"2024-04-25T08:42:27.385326+0000","last_clean":"2024-04-25T08:42:27.385326+0000","last_became_active":"2024-04-25T08:42:20.351590+0000","last_became_peered":"2024-04-25T08:42:20.351590+0000","last_unstale":"2024-04-25T08:42:27.385326+0000","last_undegraded":"2024-04-25T08:42:27.385326+0000","last_fullsized":"2024-04-25T08:42:27.385326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683138+0000","last_change":"2024-04-25T08:42:21.364800+0000","last_active":"2024-04-25T08:42:28.683138+0000","last_peered":"2024-04-25T08:42:28.683138+0000","last_clean":"2024-04-25T08:42:28.683138+0000","last_became_active":"2024-04-25T08:42:21.363796+0000","last_became_peered":"2024-04-25T08:42:21.363796+0000","last_unstale":"2024-04-25T08:42:28.683138+0000","last_undegraded":"2024-04-25T08:42:28.683138+0000","last_fullsized":"2024-04-25T08:42:28.683138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389657+0000","last_change":"2024-04-25T08:42:25.410345+0000","last_active":"2024-04-25T08:42:27.389657+0000","last_peered":"2024-04-25T08:42:27.389657+0000","last_clean":"2024-04-25T08:42:27.389657+0000","last_became_active":"2024-04-25T08:42:25.409807+0000","last_became_peered":"2024-04-25T08:42:25.409807+0000","last_unstale":"2024-04-25T08:42:27.389657+0000","last_undegraded":"2024-04-25T08:42:27.389657+0000","last_fullsized":"2024-04-25T08:42:27.389657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.971525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683514+0000","last_change":"2024-04-25T08:42:20.363027+0000","last_active":"2024-04-25T08:42:28.683514+0000","last_peered":"2024-04-25T08:42:28.683514+0000","last_clean":"2024-04-25T08:42:28.683514+0000","last_became_active":"2024-04-25T08:42:20.362598+0000","last_became_peered":"2024-04-25T08:42:20.362598+0000","last_unstale":"2024-04-25T08:42:28.683514+0000","last_undegraded":"2024-04-25T08:42:28.683514+0000","last_fullsized":"2024-04-25T08:42:28.683514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:26.564850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582927+0000","last_change":"2024-04-25T08:42:21.367716+0000","last_active":"2024-04-25T08:42:27.582927+0000","last_peered":"2024-04-25T08:42:27.582927+0000","last_clean":"2024-04-25T08:42:27.582927+0000","last_became_active":"2024-04-25T08:42:21.366362+0000","last_became_peered":"2024-04-25T08:42:21.366362+0000","last_unstale":"2024-04-25T08:42:27.582927+0000","last_undegraded":"2024-04-25T08:42:27.582927+0000","last_fullsized":"2024-04-25T08:42:27.582927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:30.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390443+0000","last_change":"2024-04-25T08:42:25.407011+0000","last_active":"2024-04-25T08:42:27.390443+0000","last_peered":"2024-04-25T08:42:27.390443+0000","last_clean":"2024-04-25T08:42:27.390443+0000","last_became_active":"2024-04-25T08:42:25.405963+0000","last_became_peered":"2024-04-25T08:42:25.405963+0000","last_unstale":"2024-04-25T08:42:27.390443+0000","last_undegraded":"2024-04-25T08:42:27.390443+0000","last_fullsized":"2024-04-25T08:42:27.390443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683493+0000","last_change":"2024-04-25T08:42:20.365950+0000","last_active":"2024-04-25T08:42:28.683493+0000","last_peered":"2024-04-25T08:42:28.683493+0000","last_clean":"2024-04-25T08:42:28.683493+0000","last_became_active":"2024-04-25T08:42:20.365648+0000","last_became_peered":"2024-04-25T08:42:20.365648+0000","last_unstale":"2024-04-25T08:42:28.683493+0000","last_undegraded":"2024-04-25T08:42:28.683493+0000","last_fullsized":"2024-04-25T08:42:28.683493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:31.011923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583016+0000","last_change":"2024-04-25T08:42:21.366598+0000","last_active":"2024-04-25T08:42:27.583016+0000","last_peered":"2024-04-25T08:42:27.583016+0000","last_clean":"2024-04-25T08:42:27.583016+0000","last_became_active":"2024-04-25T08:42:21.365936+0000","last_became_peered":"2024-04-25T08:42:21.365936+0000","last_unstale":"2024-04-25T08:42:27.583016+0000","last_undegraded":"2024-04-25T08:42:27.583016+0000","last_fullsized":"2024-04-25T08:42:27.583016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389911+0000","last_change":"2024-04-25T08:42:25.400233+0000","last_active":"2024-04-25T08:42:27.389911+0000","last_peered":"2024-04-25T08:42:27.389911+0000","last_clean":"2024-04-25T08:42:27.389911+0000","last_became_active":"2024-04-25T08:42:25.399809+0000","last_became_peered":"2024-04-25T08:42:25.399809+0000","last_unstale":"2024-04-25T08:42:27.389911+0000","last_undegraded":"2024-04-25T08:42:27.389911+0000","last_fullsized":"2024-04-25T08:42:27.389911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:19.971594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388746+0000","last_change":"2024-04-25T08:42:21.354062+0000","last_active":"2024-04-25T08:42:27.388746+0000","last_peered":"2024-04-25T08:42:27.388746+0000","last_clean":"2024-04-25T08:42:27.388746+0000","last_became_active":"2024-04-25T08:42:21.353804+0000","last_became_peered":"2024-04-25T08:42:21.353804+0000","last_unstale":"2024-04-25T08:42:27.388746+0000","last_undegraded":"2024-04-25T08:42:27.388746+0000","last_fullsized":"2024-04-25T08:42:27.388746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683480+0000","last_change":"2024-04-25T08:42:20.361718+0000","last_active":"2024-04-25T08:42:28.683480+0000","last_peered":"2024-04-25T08:42:28.683480+0000","last_clean":"2024-04-25T08:42:28.683480+0000","last_became_active":"2024-04-25T08:42:20.361531+0000","last_became_peered":"2024-04-25T08:42:20.361531+0000","last_unstale":"2024-04-25T08:42:28.683480+0000","last_undegraded":"2024-04-25T08:42:28.683480+0000","last_fullsized":"2024-04-25T08:42:28.683480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:47.546463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390866+0000","last_change":"2024-04-25T08:42:25.410313+0000","last_active":"2024-04-25T08:42:27.390866+0000","last_peered":"2024-04-25T08:42:27.390866+0000","last_clean":"2024-04-25T08:42:27.390866+0000","last_became_active":"2024-04-25T08:42:25.409794+0000","last_became_peered":"2024-04-25T08:42:25.409794+0000","last_unstale":"2024-04-25T08:42:27.390866+0000","last_undegraded":"2024-04-25T08:42:27.390866+0000","last_fullsized":"2024-04-25T08:42:27.390866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384856+0000","last_change":"2024-04-25T08:42:21.358366+0000","last_active":"2024-04-25T08:42:27.384856+0000","last_peered":"2024-04-25T08:42:27.384856+0000","last_clean":"2024-04-25T08:42:27.384856+0000","last_became_active":"2024-04-25T08:42:21.357447+0000","last_became_peered":"2024-04-25T08:42:21.357447+0000","last_unstale":"2024-04-25T08:42:27.384856+0000","last_undegraded":"2024-04-25T08:42:27.384856+0000","last_fullsized":"2024-04-25T08:42:27.384856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677905+0000","last_change":"2024-04-25T08:42:20.352044+0000","last_active":"2024-04-25T08:42:28.677905+0000","last_peered":"2024-04-25T08:42:28.677905+0000","last_clean":"2024-04-25T08:42:28.677905+0000","last_became_active":"2024-04-25T08:42:20.351667+0000","last_became_peered":"2024-04-25T08:42:20.351667+0000","last_unstale":"2024-04-25T08:42:28.677905+0000","last_undegraded":"2024-04-25T08:42:28.677905+0000","last_fullsized":"2024-04-25T08:42:28.677905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390597+0000","last_change":"2024-04-25T08:42:25.406565+0000","last_active":"2024-04-25T08:42:27.390597+0000","last_peered":"2024-04-25T08:42:27.390597+0000","last_clean":"2024-04-25T08:42:27.390597+0000","last_became_active":"2024-04-25T08:42:25.405840+0000","last_became_peered":"2024-04-25T08:42:25.405840+0000","last_unstale":"2024-04-25T08:42:27.390597+0000","last_undegraded":"2024-04-25T08:42:27.390597+0000","last_fullsized":"2024-04-25T08:42:27.390597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384326+0000","last_change":"2024-04-25T08:42:21.354614+0000","last_active":"2024-04-25T08:42:27.384326+0000","last_peered":"2024-04-25T08:42:27.384326+0000","last_clean":"2024-04-25T08:42:27.384326+0000","last_became_active":"2024-04-25T08:42:21.353873+0000","last_became_peered":"2024-04-25T08:42:21.353873+0000","last_unstale":"2024-04-25T08:42:27.384326+0000","last_undegraded":"2024-04-25T08:42:27.384326+0000","last_fullsized":"2024-04-25T08:42:27.384326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678307+0000","last_change":"2024-04-25T08:42:20.349084+0000","last_active":"2024-04-25T08:42:28.678307+0000","last_peered":"2024-04-25T08:42:28.678307+0000","last_clean":"2024-04-25T08:42:28.678307+0000","last_became_active":"2024-04-25T08:42:20.348850+0000","last_became_peered":"2024-04-25T08:42:20.348850+0000","last_unstale":"2024-04-25T08:42:28.678307+0000","last_undegraded":"2024-04-25T08:42:28.678307+0000","last_fullsized":"2024-04-25T08:42:28.678307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678770+0000","last_change":"2024-04-25T08:42:25.832820+0000","last_active":"2024-04-25T08:42:28.678770+0000","last_peered":"2024-04-25T08:42:28.678770+0000","last_clean":"2024-04-25T08:42:28.678770+0000","last_became_active":"2024-04-25T08:42:25.832090+0000","last_became_peered":"2024-04-25T08:42:25.832090+0000","last_unstale":"2024-04-25T08:42:28.678770+0000","last_undegraded":"2024-04-25T08:42:28.678770+0000","last_fullsized":"2024-04-25T08:42:28.678770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390081+0000","last_change":"2024-04-25T08:42:21.338500+0000","last_active":"2024-04-25T08:42:27.390081+0000","last_peered":"2024-04-25T08:42:27.390081+0000","last_clean":"2024-04-25T08:42:27.390081+0000","last_became_active":"2024-04-25T08:42:21.338153+0000","last_became_peered":"2024-04-25T08:42:21.338153+0000","last_unstale":"2024-04-25T08:42:27.390081+0000","last_undegraded":"2024-04-25T08:42:27.390081+0000","last_fullsized":"2024-04-25T08:42:27.390081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:00.647619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683714+0000","last_change":"2024-04-25T08:42:20.361253+0000","last_active":"2024-04-25T08:42:28.683714+0000","last_peered":"2024-04-25T08:42:28.683714+0000","last_clean":"2024-04-25T08:42:28.683714+0000","last_became_active":"2024-04-25T08:42:20.360866+0000","last_became_peered":"2024-04-25T08:42:20.360866+0000","last_unstale":"2024-04-25T08:42:28.683714+0000","last_undegraded":"2024-04-25T08:42:28.683714+0000","last_fullsized":"2024-04-25T08:42:28.683714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.823720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390784+0000","last_change":"2024-04-25T08:42:25.406555+0000","last_active":"2024-04-25T08:42:27.390784+0000","last_peered":"2024-04-25T08:42:27.390784+0000","last_clean":"2024-04-25T08:42:27.390784+0000","last_became_active":"2024-04-25T08:42:25.405977+0000","last_became_peered":"2024-04-25T08:42:25.405977+0000","last_unstale":"2024-04-25T08:42:27.390784+0000","last_undegraded":"2024-04-25T08:42:27.390784+0000","last_fullsized":"2024-04-25T08:42:27.390784+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683559+0000","last_change":"2024-04-25T08:42:21.361932+0000","last_active":"2024-04-25T08:42:28.683559+0000","last_peered":"2024-04-25T08:42:28.683559+0000","last_clean":"2024-04-25T08:42:28.683559+0000","last_became_active":"2024-04-25T08:42:21.361599+0000","last_became_peered":"2024-04-25T08:42:21.361599+0000","last_unstale":"2024-04-25T08:42:28.683559+0000","last_undegraded":"2024-04-25T08:42:28.683559+0000","last_fullsized":"2024-04-25T08:42:28.683559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389472+0000","last_change":"2024-04-25T08:42:20.359118+0000","last_active":"2024-04-25T08:42:27.389472+0000","last_peered":"2024-04-25T08:42:27.389472+0000","last_clean":"2024-04-25T08:42:27.389472+0000","last_became_active":"2024-04-25T08:42:20.358850+0000","last_became_peered":"2024-04-25T08:42:20.358850+0000","last_unstale":"2024-04-25T08:42:27.389472+0000","last_undegraded":"2024-04-25T08:42:27.389472+0000","last_fullsized":"2024-04-25T08:42:27.389472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389832+0000","last_change":"2024-04-25T08:42:25.836412+0000","last_active":"2024-04-25T08:42:27.389832+0000","last_peered":"2024-04-25T08:42:27.389832+0000","last_clean":"2024-04-25T08:42:27.389832+0000","last_became_active":"2024-04-25T08:42:25.835926+0000","last_became_peered":"2024-04-25T08:42:25.835926+0000","last_unstale":"2024-04-25T08:42:27.389832+0000","last_undegraded":"2024-04-25T08:42:27.389832+0000","last_fullsized":"2024-04-25T08:42:27.389832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.610141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385472+0000","last_change":"2024-04-25T08:42:21.366095+0000","last_active":"2024-04-25T08:42:27.385472+0000","last_peered":"2024-04-25T08:42:27.385472+0000","last_clean":"2024-04-25T08:42:27.385472+0000","last_became_active":"2024-04-25T08:42:21.365608+0000","last_became_peered":"2024-04-25T08:42:21.365608+0000","last_unstale":"2024-04-25T08:42:27.385472+0000","last_undegraded":"2024-04-25T08:42:27.385472+0000","last_fullsized":"2024-04-25T08:42:27.385472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677918+0000","last_change":"2024-04-25T08:42:20.347127+0000","last_active":"2024-04-25T08:42:28.677918+0000","last_peered":"2024-04-25T08:42:28.677918+0000","last_clean":"2024-04-25T08:42:28.677918+0000","last_became_active":"2024-04-25T08:42:20.346840+0000","last_became_peered":"2024-04-25T08:42:20.346840+0000","last_unstale":"2024-04-25T08:42:28.677918+0000","last_undegraded":"2024-04-25T08:42:28.677918+0000","last_fullsized":"2024-04-25T08:42:28.677918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384051+0000","last_change":"2024-04-25T08:42:25.832430+0000","last_active":"2024-04-25T08:42:27.384051+0000","last_peered":"2024-04-25T08:42:27.384051+0000","last_clean":"2024-04-25T08:42:27.384051+0000","last_became_active":"2024-04-25T08:42:25.832026+0000","last_became_peered":"2024-04-25T08:42:25.832026+0000","last_unstale":"2024-04-25T08:42:27.384051+0000","last_undegraded":"2024-04-25T08:42:27.384051+0000","last_fullsized":"2024-04-25T08:42:27.384051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:39.589658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.679001+0000","last_change":"2024-04-25T08:42:21.340393+0000","last_active":"2024-04-25T08:42:28.679001+0000","last_peered":"2024-04-25T08:42:28.679001+0000","last_clean":"2024-04-25T08:42:28.679001+0000","last_became_active":"2024-04-25T08:42:21.340043+0000","last_became_peered":"2024-04-25T08:42:21.340043+0000","last_unstale":"2024-04-25T08:42:28.679001+0000","last_undegraded":"2024-04-25T08:42:28.679001+0000","last_fullsized":"2024-04-25T08:42:28.679001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684534+0000","last_change":"2024-04-25T08:42:20.360592+0000","last_active":"2024-04-25T08:42:28.684534+0000","last_peered":"2024-04-25T08:42:28.684534+0000","last_clean":"2024-04-25T08:42:28.684534+0000","last_became_active":"2024-04-25T08:42:20.360361+0000","last_became_peered":"2024-04-25T08:42:20.360361+0000","last_unstale":"2024-04-25T08:42:28.684534+0000","last_undegraded":"2024-04-25T08:42:28.684534+0000","last_fullsized":"2024-04-25T08:42:28.684534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:53.741991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682832+0000","last_change":"2024-04-25T08:42:25.834135+0000","last_active":"2024-04-25T08:42:28.682832+0000","last_peered":"2024-04-25T08:42:28.682832+0000","last_clean":"2024-04-25T08:42:28.682832+0000","last_became_active":"2024-04-25T08:42:25.833757+0000","last_became_peered":"2024-04-25T08:42:25.833757+0000","last_unstale":"2024-04-25T08:42:28.682832+0000","last_undegraded":"2024-04-25T08:42:28.682832+0000","last_fullsized":"2024-04-25T08:42:28.682832+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583189+0000","last_change":"2024-04-25T08:42:21.366345+0000","last_active":"2024-04-25T08:42:27.583189+0000","last_peered":"2024-04-25T08:42:27.583189+0000","last_clean":"2024-04-25T08:42:27.583189+0000","last_became_active":"2024-04-25T08:42:21.365644+0000","last_became_peered":"2024-04-25T08:42:21.365644+0000","last_unstale":"2024-04-25T08:42:27.583189+0000","last_undegraded":"2024-04-25T08:42:27.583189+0000","last_fullsized":"2024-04-25T08:42:27.583189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.679043+0000","last_change":"2024-04-25T08:42:20.344087+0000","last_active":"2024-04-25T08:42:28.679043+0000","last_peered":"2024-04-25T08:42:28.679043+0000","last_clean":"2024-04-25T08:42:28.679043+0000","last_became_active":"2024-04-25T08:42:20.343799+0000","last_became_peered":"2024-04-25T08:42:20.343799+0000","last_unstale":"2024-04-25T08:42:28.679043+0000","last_undegraded":"2024-04-25T08:42:28.679043+0000","last_fullsized":"2024-04-25T08:42:28.679043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.823720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678152+0000","last_change":"2024-04-25T08:42:25.412243+0000","last_active":"2024-04-25T08:42:28.678152+0000","last_peered":"2024-04-25T08:42:28.678152+0000","last_clean":"2024-04-25T08:42:28.678152+0000","last_became_active":"2024-04-25T08:42:25.406351+0000","last_became_peered":"2024-04-25T08:42:25.406351+0000","last_unstale":"2024-04-25T08:42:28.678152+0000","last_undegraded":"2024-04-25T08:42:28.678152+0000","last_fullsized":"2024-04-25T08:42:28.678152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:42.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390626+0000","last_change":"2024-04-25T08:42:21.366092+0000","last_active":"2024-04-25T08:42:27.390626+0000","last_peered":"2024-04-25T08:42:27.390626+0000","last_clean":"2024-04-25T08:42:27.390626+0000","last_became_active":"2024-04-25T08:42:21.365152+0000","last_became_peered":"2024-04-25T08:42:21.365152+0000","last_unstale":"2024-04-25T08:42:27.390626+0000","last_undegraded":"2024-04-25T08:42:27.390626+0000","last_fullsized":"2024-04-25T08:42:27.390626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:29.830442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582739+0000","last_change":"2024-04-25T08:42:20.352850+0000","last_active":"2024-04-25T08:42:27.582739+0000","last_peered":"2024-04-25T08:42:27.582739+0000","last_clean":"2024-04-25T08:42:27.582739+0000","last_became_active":"2024-04-25T08:42:20.352047+0000","last_became_peered":"2024-04-25T08:42:20.352047+0000","last_unstale":"2024-04-25T08:42:27.582739+0000","last_undegraded":"2024-04-25T08:42:27.582739+0000","last_fullsized":"2024-04-25T08:42:27.582739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678979+0000","last_change":"2024-04-25T08:42:25.405939+0000","last_active":"2024-04-25T08:42:28.678979+0000","last_peered":"2024-04-25T08:42:28.678979+0000","last_clean":"2024-04-25T08:42:28.678979+0000","last_became_active":"2024-04-25T08:42:25.405333+0000","last_became_peered":"2024-04-25T08:42:25.405333+0000","last_unstale":"2024-04-25T08:42:28.678979+0000","last_undegraded":"2024-04-25T08:42:28.678979+0000","last_fullsized":"2024-04-25T08:42:28.678979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678442+0000","last_change":"2024-04-25T08:42:21.338660+0000","last_active":"2024-04-25T08:42:28.678442+0000","last_peered":"2024-04-25T08:42:28.678442+0000","last_clean":"2024-04-25T08:42:28.678442+0000","last_became_active":"2024-04-25T08:42:21.338357+0000","last_became_peered":"2024-04-25T08:42:21.338357+0000","last_unstale":"2024-04-25T08:42:28.678442+0000","last_undegraded":"2024-04-25T08:42:28.678442+0000","last_fullsized":"2024-04-25T08:42:28.678442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583032+0000","last_change":"2024-04-25T08:42:20.352888+0000","last_active":"2024-04-25T08:42:28.583032+0000","last_peered":"2024-04-25T08:42:28.583032+0000","last_clean":"2024-04-25T08:42:28.583032+0000","last_became_active":"2024-04-25T08:42:20.351108+0000","last_became_peered":"2024-04-25T08:42:20.351108+0000","last_unstale":"2024-04-25T08:42:28.583032+0000","last_undegraded":"2024-04-25T08:42:28.583032+0000","last_fullsized":"2024-04-25T08:42:28.583032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678770+0000","last_change":"2024-04-25T08:42:25.409801+0000","last_active":"2024-04-25T08:42:28.678770+0000","last_peered":"2024-04-25T08:42:28.678770+0000","last_clean":"2024-04-25T08:42:28.678770+0000","last_became_active":"2024-04-25T08:42:25.409331+0000","last_became_peered":"2024-04-25T08:42:25.409331+0000","last_unstale":"2024-04-25T08:42:28.678770+0000","last_undegraded":"2024-04-25T08:42:28.678770+0000","last_fullsized":"2024-04-25T08:42:28.678770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682464+0000","last_change":"2024-04-25T08:42:21.363349+0000","last_active":"2024-04-25T08:42:28.682464+0000","last_peered":"2024-04-25T08:42:28.682464+0000","last_clean":"2024-04-25T08:42:28.682464+0000","last_became_active":"2024-04-25T08:42:21.362903+0000","last_became_peered":"2024-04-25T08:42:21.362903+0000","last_unstale":"2024-04-25T08:42:28.682464+0000","last_undegraded":"2024-04-25T08:42:28.682464+0000","last_fullsized":"2024-04-25T08:42:28.682464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:48.553185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683717+0000","last_change":"2024-04-25T08:42:20.356423+0000","last_active":"2024-04-25T08:42:28.683717+0000","last_peered":"2024-04-25T08:42:28.683717+0000","last_clean":"2024-04-25T08:42:28.683717+0000","last_became_active":"2024-04-25T08:42:20.356110+0000","last_became_peered":"2024-04-25T08:42:20.356110+0000","last_unstale":"2024-04-25T08:42:28.683717+0000","last_undegraded":"2024-04-25T08:42:28.683717+0000","last_fullsized":"2024-04-25T08:42:28.683717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:44.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683389+0000","last_change":"2024-04-25T08:42:25.834561+0000","last_active":"2024-04-25T08:42:28.683389+0000","last_peered":"2024-04-25T08:42:28.683389+0000","last_clean":"2024-04-25T08:42:28.683389+0000","last_became_active":"2024-04-25T08:42:25.834038+0000","last_became_peered":"2024-04-25T08:42:25.834038+0000","last_unstale":"2024-04-25T08:42:28.683389+0000","last_undegraded":"2024-04-25T08:42:28.683389+0000","last_fullsized":"2024-04-25T08:42:28.683389+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:09.194186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384703+0000","last_change":"2024-04-25T08:42:21.369030+0000","last_active":"2024-04-25T08:42:27.384703+0000","last_peered":"2024-04-25T08:42:27.384703+0000","last_clean":"2024-04-25T08:42:27.384703+0000","last_became_active":"2024-04-25T08:42:21.368760+0000","last_became_peered":"2024-04-25T08:42:21.368760+0000","last_unstale":"2024-04-25T08:42:27.384703+0000","last_undegraded":"2024-04-25T08:42:27.384703+0000","last_fullsized":"2024-04-25T08:42:27.384703+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582740+0000","last_change":"2024-04-25T08:42:20.359912+0000","last_active":"2024-04-25T08:42:27.582740+0000","last_peered":"2024-04-25T08:42:27.582740+0000","last_clean":"2024-04-25T08:42:27.582740+0000","last_became_active":"2024-04-25T08:42:20.359616+0000","last_became_peered":"2024-04-25T08:42:20.359616+0000","last_unstale":"2024-04-25T08:42:27.582740+0000","last_undegraded":"2024-04-25T08:42:27.582740+0000","last_fullsized":"2024-04-25T08:42:27.582740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":180,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683854+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:42:28.683854+0000","last_peered":"2024-04-25T08:42:28.683854+0000","last_clean":"2024-04-25T08:42:28.683854+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:42:28.683854+0000","last_undegraded":"2024-04-25T08:42:28.683854+0000","last_fullsized":"2024-04-25T08:42:28.683854+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389063+0000","last_change":"2024-04-25T08:42:25.836852+0000","last_active":"2024-04-25T08:42:27.389063+0000","last_peered":"2024-04-25T08:42:27.389063+0000","last_clean":"2024-04-25T08:42:27.389063+0000","last_became_active":"2024-04-25T08:42:25.836428+0000","last_became_peered":"2024-04-25T08:42:25.836428+0000","last_unstale":"2024-04-25T08:42:27.389063+0000","last_undegraded":"2024-04-25T08:42:27.389063+0000","last_fullsized":"2024-04-25T08:42:27.389063+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.544013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684361+0000","last_change":"2024-04-25T08:42:21.376058+0000","last_active":"2024-04-25T08:42:28.684361+0000","last_peered":"2024-04-25T08:42:28.684361+0000","last_clean":"2024-04-25T08:42:28.684361+0000","last_became_active":"2024-04-25T08:42:21.375825+0000","last_became_peered":"2024-04-25T08:42:21.375825+0000","last_unstale":"2024-04-25T08:42:28.684361+0000","last_undegraded":"2024-04-25T08:42:28.684361+0000","last_fullsized":"2024-04-25T08:42:28.684361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:24.495752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583044+0000","last_change":"2024-04-25T08:42:20.353588+0000","last_active":"2024-04-25T08:42:28.583044+0000","last_peered":"2024-04-25T08:42:28.583044+0000","last_clean":"2024-04-25T08:42:28.583044+0000","last_became_active":"2024-04-25T08:42:20.353335+0000","last_became_peered":"2024-04-25T08:42:20.353335+0000","last_unstale":"2024-04-25T08:42:28.583044+0000","last_undegraded":"2024-04-25T08:42:28.583044+0000","last_fullsized":"2024-04-25T08:42:28.583044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388628+0000","last_change":"2024-04-25T08:42:25.833110+0000","last_active":"2024-04-25T08:42:27.388628+0000","last_peered":"2024-04-25T08:42:27.388628+0000","last_clean":"2024-04-25T08:42:27.388628+0000","last_became_active":"2024-04-25T08:42:25.832656+0000","last_became_peered":"2024-04-25T08:42:25.832656+0000","last_unstale":"2024-04-25T08:42:27.388628+0000","last_undegraded":"2024-04-25T08:42:27.388628+0000","last_fullsized":"2024-04-25T08:42:27.388628+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.971525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582934+0000","last_change":"2024-04-25T08:42:21.368021+0000","last_active":"2024-04-25T08:42:27.582934+0000","last_peered":"2024-04-25T08:42:27.582934+0000","last_clean":"2024-04-25T08:42:27.582934+0000","last_became_active":"2024-04-25T08:42:21.367516+0000","last_became_peered":"2024-04-25T08:42:21.367516+0000","last_unstale":"2024-04-25T08:42:27.582934+0000","last_undegraded":"2024-04-25T08:42:27.582934+0000","last_fullsized":"2024-04-25T08:42:27.582934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:24.495752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677925+0000","last_change":"2024-04-25T08:42:20.340089+0000","last_active":"2024-04-25T08:42:28.677925+0000","last_peered":"2024-04-25T08:42:28.677925+0000","last_clean":"2024-04-25T08:42:28.677925+0000","last_became_active":"2024-04-25T08:42:20.338267+0000","last_became_peered":"2024-04-25T08:42:20.338267+0000","last_unstale":"2024-04-25T08:42:28.677925+0000","last_undegraded":"2024-04-25T08:42:28.677925+0000","last_fullsized":"2024-04-25T08:42:28.677925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:44.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682886+0000","last_change":"2024-04-25T08:42:25.418168+0000","last_active":"2024-04-25T08:42:28.682886+0000","last_peered":"2024-04-25T08:42:28.682886+0000","last_clean":"2024-04-25T08:42:28.682886+0000","last_became_active":"2024-04-25T08:42:25.417852+0000","last_became_peered":"2024-04-25T08:42:25.417852+0000","last_unstale":"2024-04-25T08:42:28.682886+0000","last_undegraded":"2024-04-25T08:42:28.682886+0000","last_fullsized":"2024-04-25T08:42:28.682886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:40.788495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684774+0000","last_change":"2024-04-25T08:42:21.370059+0000","last_active":"2024-04-25T08:42:28.684774+0000","last_peered":"2024-04-25T08:42:28.684774+0000","last_clean":"2024-04-25T08:42:28.684774+0000","last_became_active":"2024-04-25T08:42:21.359776+0000","last_became_peered":"2024-04-25T08:42:21.359776+0000","last_unstale":"2024-04-25T08:42:28.684774+0000","last_undegraded":"2024-04-25T08:42:28.684774+0000","last_fullsized":"2024-04-25T08:42:28.684774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:21.412723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583070+0000","last_change":"2024-04-25T08:42:20.351406+0000","last_active":"2024-04-25T08:42:28.583070+0000","last_peered":"2024-04-25T08:42:28.583070+0000","last_clean":"2024-04-25T08:42:28.583070+0000","last_became_active":"2024-04-25T08:42:20.351114+0000","last_became_peered":"2024-04-25T08:42:20.351114+0000","last_unstale":"2024-04-25T08:42:28.583070+0000","last_undegraded":"2024-04-25T08:42:28.583070+0000","last_fullsized":"2024-04-25T08:42:28.583070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582385+0000","last_change":"2024-04-25T08:42:25.835941+0000","last_active":"2024-04-25T08:42:27.582385+0000","last_peered":"2024-04-25T08:42:27.582385+0000","last_clean":"2024-04-25T08:42:27.582385+0000","last_became_active":"2024-04-25T08:42:25.834320+0000","last_became_peered":"2024-04-25T08:42:25.834320+0000","last_unstale":"2024-04-25T08:42:27.582385+0000","last_undegraded":"2024-04-25T08:42:27.582385+0000","last_fullsized":"2024-04-25T08:42:27.582385+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:32.335929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684548+0000","last_change":"2024-04-25T08:42:21.363962+0000","last_active":"2024-04-25T08:42:28.684548+0000","last_peered":"2024-04-25T08:42:28.684548+0000","last_clean":"2024-04-25T08:42:28.684548+0000","last_became_active":"2024-04-25T08:42:21.363695+0000","last_became_peered":"2024-04-25T08:42:21.363695+0000","last_unstale":"2024-04-25T08:42:28.684548+0000","last_undegraded":"2024-04-25T08:42:28.684548+0000","last_fullsized":"2024-04-25T08:42:28.684548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:15.923334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.391021+0000","last_change":"2024-04-25T08:42:20.345476+0000","last_active":"2024-04-25T08:42:27.391021+0000","last_peered":"2024-04-25T08:42:27.391021+0000","last_clean":"2024-04-25T08:42:27.391021+0000","last_became_active":"2024-04-25T08:42:20.345085+0000","last_became_peered":"2024-04-25T08:42:20.345085+0000","last_unstale":"2024-04-25T08:42:27.391021+0000","last_undegraded":"2024-04-25T08:42:27.391021+0000","last_fullsized":"2024-04-25T08:42:27.391021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:19.301171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389733+0000","last_change":"2024-04-25T08:42:25.410444+0000","last_active":"2024-04-25T08:42:27.389733+0000","last_peered":"2024-04-25T08:42:27.389733+0000","last_clean":"2024-04-25T08:42:27.389733+0000","last_became_active":"2024-04-25T08:42:25.409574+0000","last_became_peered":"2024-04-25T08:42:25.409574+0000","last_unstale":"2024-04-25T08:42:27.389733+0000","last_undegraded":"2024-04-25T08:42:27.389733+0000","last_fullsized":"2024-04-25T08:42:27.389733+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:28.544013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683914+0000","last_change":"2024-04-25T08:42:21.357665+0000","last_active":"2024-04-25T08:42:28.683914+0000","last_peered":"2024-04-25T08:42:28.683914+0000","last_clean":"2024-04-25T08:42:28.683914+0000","last_became_active":"2024-04-25T08:42:21.357081+0000","last_became_peered":"2024-04-25T08:42:21.357081+0000","last_unstale":"2024-04-25T08:42:28.683914+0000","last_undegraded":"2024-04-25T08:42:28.683914+0000","last_fullsized":"2024-04-25T08:42:28.683914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:24.505703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":32,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583188+0000","last_change":"2024-04-25T08:42:20.345496+0000","last_active":"2024-04-25T08:42:28.583188+0000","last_peered":"2024-04-25T08:42:28.583188+0000","last_clean":"2024-04-25T08:42:28.583188+0000","last_became_active":"2024-04-25T08:42:20.345230+0000","last_became_peered":"2024-04-25T08:42:20.345230+0000","last_unstale":"2024-04-25T08:42:28.583188+0000","last_undegraded":"2024-04-25T08:42:28.583188+0000","last_fullsized":"2024-04-25T08:42:28.583188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:33.260664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683216+0000","last_change":"2024-04-25T08:42:25.412932+0000","last_active":"2024-04-25T08:42:28.683216+0000","last_peered":"2024-04-25T08:42:28.683216+0000","last_clean":"2024-04-25T08:42:28.683216+0000","last_became_active":"2024-04-25T08:42:25.412212+0000","last_became_peered":"2024-04-25T08:42:25.412212+0000","last_unstale":"2024-04-25T08:42:28.683216+0000","last_undegraded":"2024-04-25T08:42:28.683216+0000","last_fullsized":"2024-04-25T08:42:28.683216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:37.438634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388745+0000","last_change":"2024-04-25T08:42:21.351855+0000","last_active":"2024-04-25T08:42:27.388745+0000","last_peered":"2024-04-25T08:42:27.388745+0000","last_clean":"2024-04-25T08:42:27.388745+0000","last_became_active":"2024-04-25T08:42:21.351412+0000","last_became_peered":"2024-04-25T08:42:21.351412+0000","last_unstale":"2024-04-25T08:42:27.388745+0000","last_undegraded":"2024-04-25T08:42:27.388745+0000","last_fullsized":"2024-04-25T08:42:27.388745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388741+0000","last_change":"2024-04-25T08:42:20.350412+0000","last_active":"2024-04-25T08:42:27.388741+0000","last_peered":"2024-04-25T08:42:27.388741+0000","last_clean":"2024-04-25T08:42:27.388741+0000","last_became_active":"2024-04-25T08:42:20.350113+0000","last_became_peered":"2024-04-25T08:42:20.350113+0000","last_unstale":"2024-04-25T08:42:27.388741+0000","last_undegraded":"2024-04-25T08:42:27.388741+0000","last_fullsized":"2024-04-25T08:42:27.388741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385384+0000","last_change":"2024-04-25T08:42:25.838652+0000","last_active":"2024-04-25T08:42:27.385384+0000","last_peered":"2024-04-25T08:42:27.385384+0000","last_clean":"2024-04-25T08:42:27.385384+0000","last_became_active":"2024-04-25T08:42:25.838168+0000","last_became_peered":"2024-04-25T08:42:25.838168+0000","last_unstale":"2024-04-25T08:42:27.385384+0000","last_undegraded":"2024-04-25T08:42:27.385384+0000","last_fullsized":"2024-04-25T08:42:27.385384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:46.882274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684243+0000","last_change":"2024-04-25T08:42:21.373789+0000","last_active":"2024-04-25T08:42:28.684243+0000","last_peered":"2024-04-25T08:42:28.684243+0000","last_clean":"2024-04-25T08:42:28.684243+0000","last_became_active":"2024-04-25T08:42:21.372582+0000","last_became_peered":"2024-04-25T08:42:21.372582+0000","last_unstale":"2024-04-25T08:42:28.684243+0000","last_undegraded":"2024-04-25T08:42:28.684243+0000","last_fullsized":"2024-04-25T08:42:28.684243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.525554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582667+0000","last_change":"2024-04-25T08:42:20.357655+0000","last_active":"2024-04-25T08:42:27.582667+0000","last_peered":"2024-04-25T08:42:27.582667+0000","last_clean":"2024-04-25T08:42:27.582667+0000","last_became_active":"2024-04-25T08:42:20.357354+0000","last_became_peered":"2024-04-25T08:42:20.357354+0000","last_unstale":"2024-04-25T08:42:27.582667+0000","last_undegraded":"2024-04-25T08:42:27.582667+0000","last_fullsized":"2024-04-25T08:42:27.582667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:59.640897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385040+0000","last_change":"2024-04-25T08:42:25.835505+0000","last_active":"2024-04-25T08:42:27.385040+0000","last_peered":"2024-04-25T08:42:27.385040+0000","last_clean":"2024-04-25T08:42:27.385040+0000","last_became_active":"2024-04-25T08:42:25.835079+0000","last_became_peered":"2024-04-25T08:42:25.835079+0000","last_unstale":"2024-04-25T08:42:27.385040+0000","last_undegraded":"2024-04-25T08:42:27.385040+0000","last_fullsized":"2024-04-25T08:42:27.385040+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:00.799395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682748+0000","last_change":"2024-04-25T08:42:21.366223+0000","last_active":"2024-04-25T08:42:28.682748+0000","last_peered":"2024-04-25T08:42:28.682748+0000","last_clean":"2024-04-25T08:42:28.682748+0000","last_became_active":"2024-04-25T08:42:21.365913+0000","last_became_peered":"2024-04-25T08:42:21.365913+0000","last_unstale":"2024-04-25T08:42:28.682748+0000","last_undegraded":"2024-04-25T08:42:28.682748+0000","last_fullsized":"2024-04-25T08:42:28.682748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682183+0000","last_change":"2024-04-25T08:42:20.354779+0000","last_active":"2024-04-25T08:42:28.682183+0000","last_peered":"2024-04-25T08:42:28.682183+0000","last_clean":"2024-04-25T08:42:28.682183+0000","last_became_active":"2024-04-25T08:42:20.354504+0000","last_became_peered":"2024-04-25T08:42:20.354504+0000","last_unstale":"2024-04-25T08:42:28.682183+0000","last_undegraded":"2024-04-25T08:42:28.682183+0000","last_fullsized":"2024-04-25T08:42:28.682183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.311211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583098+0000","last_change":"2024-04-25T08:42:25.832677+0000","last_active":"2024-04-25T08:42:27.583098+0000","last_peered":"2024-04-25T08:42:27.583098+0000","last_clean":"2024-04-25T08:42:27.583098+0000","last_became_active":"2024-04-25T08:42:25.832204+0000","last_became_peered":"2024-04-25T08:42:25.832204+0000","last_unstale":"2024-04-25T08:42:27.583098+0000","last_undegraded":"2024-04-25T08:42:27.583098+0000","last_fullsized":"2024-04-25T08:42:27.583098+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:02.694380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684296+0000","last_change":"2024-04-25T08:42:21.365924+0000","last_active":"2024-04-25T08:42:28.684296+0000","last_peered":"2024-04-25T08:42:28.684296+0000","last_clean":"2024-04-25T08:42:28.684296+0000","last_became_active":"2024-04-25T08:42:21.365581+0000","last_became_peered":"2024-04-25T08:42:21.365581+0000","last_unstale":"2024-04-25T08:42:28.684296+0000","last_undegraded":"2024-04-25T08:42:28.684296+0000","last_fullsized":"2024-04-25T08:42:28.684296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.923264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"26'10","reported_seq":34,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:36.452272+0000","last_change":"2024-04-25T08:42:20.354736+0000","last_active":"2024-04-25T08:42:36.452272+0000","last_peered":"2024-04-25T08:42:36.452272+0000","last_clean":"2024-04-25T08:42:36.452272+0000","last_became_active":"2024-04-25T08:42:20.354322+0000","last_became_peered":"2024-04-25T08:42:20.354322+0000","last_unstale":"2024-04-25T08:42:36.452272+0000","last_undegraded":"2024-04-25T08:42:36.452272+0000","last_fullsized":"2024-04-25T08:42:36.452272+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.716832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678428+0000","last_change":"2024-04-25T08:42:25.414706+0000","last_active":"2024-04-25T08:42:28.678428+0000","last_peered":"2024-04-25T08:42:28.678428+0000","last_clean":"2024-04-25T08:42:28.678428+0000","last_became_active":"2024-04-25T08:42:25.414302+0000","last_became_peered":"2024-04-25T08:42:25.414302+0000","last_unstale":"2024-04-25T08:42:28.678428+0000","last_undegraded":"2024-04-25T08:42:28.678428+0000","last_fullsized":"2024-04-25T08:42:28.678428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:25.460984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390755+0000","last_change":"2024-04-25T08:42:21.361175+0000","last_active":"2024-04-25T08:42:27.390755+0000","last_peered":"2024-04-25T08:42:27.390755+0000","last_clean":"2024-04-25T08:42:27.390755+0000","last_became_active":"2024-04-25T08:42:21.358285+0000","last_became_peered":"2024-04-25T08:42:21.358285+0000","last_unstale":"2024-04-25T08:42:27.390755+0000","last_undegraded":"2024-04-25T08:42:27.390755+0000","last_fullsized":"2024-04-25T08:42:27.390755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:45.573590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.677940+0000","last_change":"2024-04-25T08:42:20.357678+0000","last_active":"2024-04-25T08:42:28.677940+0000","last_peered":"2024-04-25T08:42:28.677940+0000","last_clean":"2024-04-25T08:42:28.677940+0000","last_became_active":"2024-04-25T08:42:20.357378+0000","last_became_peered":"2024-04-25T08:42:20.357378+0000","last_unstale":"2024-04-25T08:42:28.677940+0000","last_undegraded":"2024-04-25T08:42:28.677940+0000","last_fullsized":"2024-04-25T08:42:28.677940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.288791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390203+0000","last_change":"2024-04-25T08:42:25.406294+0000","last_active":"2024-04-25T08:42:27.390203+0000","last_peered":"2024-04-25T08:42:27.390203+0000","last_clean":"2024-04-25T08:42:27.390203+0000","last_became_active":"2024-04-25T08:42:25.405833+0000","last_became_peered":"2024-04-25T08:42:25.405833+0000","last_unstale":"2024-04-25T08:42:27.390203+0000","last_undegraded":"2024-04-25T08:42:27.390203+0000","last_fullsized":"2024-04-25T08:42:27.390203+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:42.493575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683217+0000","last_change":"2024-04-25T08:42:21.363445+0000","last_active":"2024-04-25T08:42:28.683217+0000","last_peered":"2024-04-25T08:42:28.683217+0000","last_clean":"2024-04-25T08:42:28.683217+0000","last_became_active":"2024-04-25T08:42:21.363129+0000","last_became_peered":"2024-04-25T08:42:21.363129+0000","last_unstale":"2024-04-25T08:42:28.683217+0000","last_undegraded":"2024-04-25T08:42:28.683217+0000","last_fullsized":"2024-04-25T08:42:28.683217+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583076+0000","last_change":"2024-04-25T08:42:20.352637+0000","last_active":"2024-04-25T08:42:27.583076+0000","last_peered":"2024-04-25T08:42:27.583076+0000","last_clean":"2024-04-25T08:42:27.583076+0000","last_became_active":"2024-04-25T08:42:20.351776+0000","last_became_peered":"2024-04-25T08:42:20.351776+0000","last_unstale":"2024-04-25T08:42:27.583076+0000","last_undegraded":"2024-04-25T08:42:27.583076+0000","last_fullsized":"2024-04-25T08:42:27.583076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:53.741991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384221+0000","last_change":"2024-04-25T08:42:25.406488+0000","last_active":"2024-04-25T08:42:27.384221+0000","last_peered":"2024-04-25T08:42:27.384221+0000","last_clean":"2024-04-25T08:42:27.384221+0000","last_became_active":"2024-04-25T08:42:25.405903+0000","last_became_peered":"2024-04-25T08:42:25.405903+0000","last_unstale":"2024-04-25T08:42:27.384221+0000","last_undegraded":"2024-04-25T08:42:27.384221+0000","last_fullsized":"2024-04-25T08:42:27.384221+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:52.897838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583032+0000","last_change":"2024-04-25T08:42:21.376939+0000","last_active":"2024-04-25T08:42:27.583032+0000","last_peered":"2024-04-25T08:42:27.583032+0000","last_clean":"2024-04-25T08:42:27.583032+0000","last_became_active":"2024-04-25T08:42:21.376693+0000","last_became_peered":"2024-04-25T08:42:21.376693+0000","last_unstale":"2024-04-25T08:42:27.583032+0000","last_undegraded":"2024-04-25T08:42:27.583032+0000","last_fullsized":"2024-04-25T08:42:27.583032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:16.561880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389379+0000","last_change":"2024-04-25T08:42:20.355744+0000","last_active":"2024-04-25T08:42:27.389379+0000","last_peered":"2024-04-25T08:42:27.389379+0000","last_clean":"2024-04-25T08:42:27.389379+0000","last_became_active":"2024-04-25T08:42:20.355365+0000","last_became_peered":"2024-04-25T08:42:20.355365+0000","last_unstale":"2024-04-25T08:42:27.389379+0000","last_undegraded":"2024-04-25T08:42:27.389379+0000","last_fullsized":"2024-04-25T08:42:27.389379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384187+0000","last_change":"2024-04-25T08:42:25.833881+0000","last_active":"2024-04-25T08:42:27.384187+0000","last_peered":"2024-04-25T08:42:27.384187+0000","last_clean":"2024-04-25T08:42:27.384187+0000","last_became_active":"2024-04-25T08:42:25.833475+0000","last_became_peered":"2024-04-25T08:42:25.833475+0000","last_unstale":"2024-04-25T08:42:27.384187+0000","last_undegraded":"2024-04-25T08:42:27.384187+0000","last_fullsized":"2024-04-25T08:42:27.384187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:49.130328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683043+0000","last_change":"2024-04-25T08:42:25.835305+0000","last_active":"2024-04-25T08:42:28.683043+0000","last_peered":"2024-04-25T08:42:28.683043+0000","last_clean":"2024-04-25T08:42:28.683043+0000","last_became_active":"2024-04-25T08:42:25.834632+0000","last_became_peered":"2024-04-25T08:42:25.834632+0000","last_unstale":"2024-04-25T08:42:28.683043+0000","last_undegraded":"2024-04-25T08:42:28.683043+0000","last_fullsized":"2024-04-25T08:42:28.683043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:03:22.823331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678200+0000","last_change":"2024-04-25T08:42:21.350716+0000","last_active":"2024-04-25T08:42:28.678200+0000","last_peered":"2024-04-25T08:42:28.678200+0000","last_clean":"2024-04-25T08:42:28.678200+0000","last_became_active":"2024-04-25T08:42:21.349383+0000","last_became_peered":"2024-04-25T08:42:21.349383+0000","last_unstale":"2024-04-25T08:42:28.678200+0000","last_undegraded":"2024-04-25T08:42:28.678200+0000","last_fullsized":"2024-04-25T08:42:28.678200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582596+0000","last_change":"2024-04-25T08:42:20.345077+0000","last_active":"2024-04-25T08:42:27.582596+0000","last_peered":"2024-04-25T08:42:27.582596+0000","last_clean":"2024-04-25T08:42:27.582596+0000","last_became_active":"2024-04-25T08:42:20.344526+0000","last_became_peered":"2024-04-25T08:42:20.344526+0000","last_unstale":"2024-04-25T08:42:27.582596+0000","last_undegraded":"2024-04-25T08:42:27.582596+0000","last_fullsized":"2024-04-25T08:42:27.582596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:40.128002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389292+0000","last_change":"2024-04-25T08:42:25.407527+0000","last_active":"2024-04-25T08:42:27.389292+0000","last_peered":"2024-04-25T08:42:27.389292+0000","last_clean":"2024-04-25T08:42:27.389292+0000","last_became_active":"2024-04-25T08:42:25.407221+0000","last_became_peered":"2024-04-25T08:42:25.407221+0000","last_unstale":"2024-04-25T08:42:27.389292+0000","last_undegraded":"2024-04-25T08:42:27.389292+0000","last_fullsized":"2024-04-25T08:42:27.389292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:20.610141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384221+0000","last_change":"2024-04-25T08:42:21.351795+0000","last_active":"2024-04-25T08:42:27.384221+0000","last_peered":"2024-04-25T08:42:27.384221+0000","last_clean":"2024-04-25T08:42:27.384221+0000","last_became_active":"2024-04-25T08:42:21.351544+0000","last_became_peered":"2024-04-25T08:42:21.351544+0000","last_unstale":"2024-04-25T08:42:27.384221+0000","last_undegraded":"2024-04-25T08:42:27.384221+0000","last_fullsized":"2024-04-25T08:42:27.384221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384681+0000","last_change":"2024-04-25T08:42:20.354629+0000","last_active":"2024-04-25T08:42:27.384681+0000","last_peered":"2024-04-25T08:42:27.384681+0000","last_clean":"2024-04-25T08:42:27.384681+0000","last_became_active":"2024-04-25T08:42:20.354388+0000","last_became_peered":"2024-04-25T08:42:20.354388+0000","last_unstale":"2024-04-25T08:42:27.384681+0000","last_undegraded":"2024-04-25T08:42:27.384681+0000","last_fullsized":"2024-04-25T08:42:27.384681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.311211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388979+0000","last_change":"2024-04-25T08:42:20.350141+0000","last_active":"2024-04-25T08:42:27.388979+0000","last_peered":"2024-04-25T08:42:27.388979+0000","last_clean":"2024-04-25T08:42:27.388979+0000","last_became_active":"2024-04-25T08:42:20.349168+0000","last_became_peered":"2024-04-25T08:42:20.349168+0000","last_unstale":"2024-04-25T08:42:27.388979+0000","last_undegraded":"2024-04-25T08:42:27.388979+0000","last_fullsized":"2024-04-25T08:42:27.388979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:24.161945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683007+0000","last_change":"2024-04-25T08:42:21.363608+0000","last_active":"2024-04-25T08:42:28.683007+0000","last_peered":"2024-04-25T08:42:28.683007+0000","last_clean":"2024-04-25T08:42:28.683007+0000","last_became_active":"2024-04-25T08:42:21.363152+0000","last_became_peered":"2024-04-25T08:42:21.363152+0000","last_unstale":"2024-04-25T08:42:28.683007+0000","last_undegraded":"2024-04-25T08:42:28.683007+0000","last_fullsized":"2024-04-25T08:42:28.683007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583434+0000","last_change":"2024-04-25T08:42:25.413462+0000","last_active":"2024-04-25T08:42:28.583434+0000","last_peered":"2024-04-25T08:42:28.583434+0000","last_clean":"2024-04-25T08:42:28.583434+0000","last_became_active":"2024-04-25T08:42:25.412613+0000","last_became_peered":"2024-04-25T08:42:25.412613+0000","last_unstale":"2024-04-25T08:42:28.583434+0000","last_undegraded":"2024-04-25T08:42:28.583434+0000","last_fullsized":"2024-04-25T08:42:28.583434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:52.601446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384923+0000","last_change":"2024-04-25T08:42:20.355705+0000","last_active":"2024-04-25T08:42:27.384923+0000","last_peered":"2024-04-25T08:42:27.384923+0000","last_clean":"2024-04-25T08:42:27.384923+0000","last_became_active":"2024-04-25T08:42:20.355433+0000","last_became_peered":"2024-04-25T08:42:20.355433+0000","last_unstale":"2024-04-25T08:42:27.384923+0000","last_undegraded":"2024-04-25T08:42:27.384923+0000","last_fullsized":"2024-04-25T08:42:27.384923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:19.311121+0000","last_clean_scrub_stamp":"2024-04-25T08:42:19.311121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:33.260664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390028+0000","last_change":"2024-04-25T08:42:21.344759+0000","last_active":"2024-04-25T08:42:27.390028+0000","last_peered":"2024-04-25T08:42:27.390028+0000","last_clean":"2024-04-25T08:42:27.390028+0000","last_became_active":"2024-04-25T08:42:21.344560+0000","last_became_peered":"2024-04-25T08:42:21.344560+0000","last_unstale":"2024-04-25T08:42:27.390028+0000","last_undegraded":"2024-04-25T08:42:27.390028+0000","last_fullsized":"2024-04-25T08:42:27.390028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:32.295514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385225+0000","last_change":"2024-04-25T08:42:25.413565+0000","last_active":"2024-04-25T08:42:27.385225+0000","last_peered":"2024-04-25T08:42:27.385225+0000","last_clean":"2024-04-25T08:42:27.385225+0000","last_became_active":"2024-04-25T08:42:25.413198+0000","last_became_peered":"2024-04-25T08:42:25.413198+0000","last_unstale":"2024-04-25T08:42:27.385225+0000","last_undegraded":"2024-04-25T08:42:27.385225+0000","last_fullsized":"2024-04-25T08:42:27.385225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:24.366105+0000","last_clean_scrub_stamp":"2024-04-25T08:42:24.366105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:13.096012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388406+0000","last_change":"2024-04-25T08:42:21.363343+0000","last_active":"2024-04-25T08:42:27.388406+0000","last_peered":"2024-04-25T08:42:27.388406+0000","last_clean":"2024-04-25T08:42:27.388406+0000","last_became_active":"2024-04-25T08:42:21.363101+0000","last_became_peered":"2024-04-25T08:42:21.363101+0000","last_unstale":"2024-04-25T08:42:27.388406+0000","last_undegraded":"2024-04-25T08:42:27.388406+0000","last_fullsized":"2024-04-25T08:42:27.388406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583299+0000","last_change":"2024-04-25T08:42:21.361026+0000","last_active":"2024-04-25T08:42:28.583299+0000","last_peered":"2024-04-25T08:42:28.583299+0000","last_clean":"2024-04-25T08:42:28.583299+0000","last_became_active":"2024-04-25T08:42:21.360012+0000","last_became_peered":"2024-04-25T08:42:21.360012+0000","last_unstale":"2024-04-25T08:42:28.583299+0000","last_undegraded":"2024-04-25T08:42:28.583299+0000","last_fullsized":"2024-04-25T08:42:28.583299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:29.830442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582614+0000","last_change":"2024-04-25T08:42:21.367540+0000","last_active":"2024-04-25T08:42:27.582614+0000","last_peered":"2024-04-25T08:42:27.582614+0000","last_clean":"2024-04-25T08:42:27.582614+0000","last_became_active":"2024-04-25T08:42:21.367291+0000","last_became_peered":"2024-04-25T08:42:21.367291+0000","last_unstale":"2024-04-25T08:42:27.582614+0000","last_undegraded":"2024-04-25T08:42:27.582614+0000","last_fullsized":"2024-04-25T08:42:27.582614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:24.923264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683000+0000","last_change":"2024-04-25T08:42:21.366788+0000","last_active":"2024-04-25T08:42:28.683000+0000","last_peered":"2024-04-25T08:42:28.683000+0000","last_clean":"2024-04-25T08:42:28.683000+0000","last_became_active":"2024-04-25T08:42:21.365945+0000","last_became_peered":"2024-04-25T08:42:21.365945+0000","last_unstale":"2024-04-25T08:42:28.683000+0000","last_undegraded":"2024-04-25T08:42:28.683000+0000","last_fullsized":"2024-04-25T08:42:28.683000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385215+0000","last_change":"2024-04-25T08:42:21.351963+0000","last_active":"2024-04-25T08:42:27.385215+0000","last_peered":"2024-04-25T08:42:27.385215+0000","last_clean":"2024-04-25T08:42:27.385215+0000","last_became_active":"2024-04-25T08:42:21.351587+0000","last_became_peered":"2024-04-25T08:42:21.351587+0000","last_unstale":"2024-04-25T08:42:27.385215+0000","last_undegraded":"2024-04-25T08:42:27.385215+0000","last_fullsized":"2024-04-25T08:42:27.385215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:30.980133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678682+0000","last_change":"2024-04-25T08:42:21.363969+0000","last_active":"2024-04-25T08:42:28.678682+0000","last_peered":"2024-04-25T08:42:28.678682+0000","last_clean":"2024-04-25T08:42:28.678682+0000","last_became_active":"2024-04-25T08:42:21.354733+0000","last_became_peered":"2024-04-25T08:42:21.354733+0000","last_unstale":"2024-04-25T08:42:28.678682+0000","last_undegraded":"2024-04-25T08:42:28.678682+0000","last_fullsized":"2024-04-25T08:42:28.678682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389184+0000","last_change":"2024-04-25T08:42:21.375124+0000","last_active":"2024-04-25T08:42:27.389184+0000","last_peered":"2024-04-25T08:42:27.389184+0000","last_clean":"2024-04-25T08:42:27.389184+0000","last_became_active":"2024-04-25T08:42:21.374653+0000","last_became_peered":"2024-04-25T08:42:21.374653+0000","last_unstale":"2024-04-25T08:42:27.389184+0000","last_undegraded":"2024-04-25T08:42:27.389184+0000","last_fullsized":"2024-04-25T08:42:27.389184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.385387+0000","last_change":"2024-04-25T08:42:21.362798+0000","last_active":"2024-04-25T08:42:27.385387+0000","last_peered":"2024-04-25T08:42:27.385387+0000","last_clean":"2024-04-25T08:42:27.385387+0000","last_became_active":"2024-04-25T08:42:21.362412+0000","last_became_peered":"2024-04-25T08:42:21.362412+0000","last_unstale":"2024-04-25T08:42:27.385387+0000","last_undegraded":"2024-04-25T08:42:27.385387+0000","last_fullsized":"2024-04-25T08:42:27.385387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582584+0000","last_change":"2024-04-25T08:42:21.368298+0000","last_active":"2024-04-25T08:42:27.582584+0000","last_peered":"2024-04-25T08:42:27.582584+0000","last_clean":"2024-04-25T08:42:27.582584+0000","last_became_active":"2024-04-25T08:42:21.368073+0000","last_became_peered":"2024-04-25T08:42:21.368073+0000","last_unstale":"2024-04-25T08:42:27.582584+0000","last_undegraded":"2024-04-25T08:42:27.582584+0000","last_fullsized":"2024-04-25T08:42:27.582584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:10.074415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384593+0000","last_change":"2024-04-25T08:42:21.356172+0000","last_active":"2024-04-25T08:42:27.384593+0000","last_peered":"2024-04-25T08:42:27.384593+0000","last_clean":"2024-04-25T08:42:27.384593+0000","last_became_active":"2024-04-25T08:42:21.355920+0000","last_became_peered":"2024-04-25T08:42:21.355920+0000","last_unstale":"2024-04-25T08:42:27.384593+0000","last_undegraded":"2024-04-25T08:42:27.384593+0000","last_fullsized":"2024-04-25T08:42:27.384593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:44.283547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582542+0000","last_change":"2024-04-25T08:42:21.376313+0000","last_active":"2024-04-25T08:42:27.582542+0000","last_peered":"2024-04-25T08:42:27.582542+0000","last_clean":"2024-04-25T08:42:27.582542+0000","last_became_active":"2024-04-25T08:42:21.376047+0000","last_became_peered":"2024-04-25T08:42:21.376047+0000","last_unstale":"2024-04-25T08:42:27.582542+0000","last_undegraded":"2024-04-25T08:42:27.582542+0000","last_fullsized":"2024-04-25T08:42:27.582542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:24.505703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682755+0000","last_change":"2024-04-25T08:42:21.367017+0000","last_active":"2024-04-25T08:42:28.682755+0000","last_peered":"2024-04-25T08:42:28.682755+0000","last_clean":"2024-04-25T08:42:28.682755+0000","last_became_active":"2024-04-25T08:42:21.366480+0000","last_became_peered":"2024-04-25T08:42:21.366480+0000","last_unstale":"2024-04-25T08:42:28.682755+0000","last_undegraded":"2024-04-25T08:42:28.682755+0000","last_fullsized":"2024-04-25T08:42:28.682755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.388881+0000","last_change":"2024-04-25T08:42:21.354458+0000","last_active":"2024-04-25T08:42:27.388881+0000","last_peered":"2024-04-25T08:42:27.388881+0000","last_clean":"2024-04-25T08:42:27.388881+0000","last_became_active":"2024-04-25T08:42:21.351083+0000","last_became_peered":"2024-04-25T08:42:21.351083+0000","last_unstale":"2024-04-25T08:42:27.388881+0000","last_undegraded":"2024-04-25T08:42:27.388881+0000","last_fullsized":"2024-04-25T08:42:27.388881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684511+0000","last_change":"2024-04-25T08:42:21.369898+0000","last_active":"2024-04-25T08:42:28.684511+0000","last_peered":"2024-04-25T08:42:28.684511+0000","last_clean":"2024-04-25T08:42:28.684511+0000","last_became_active":"2024-04-25T08:42:21.369443+0000","last_became_peered":"2024-04-25T08:42:21.369443+0000","last_unstale":"2024-04-25T08:42:28.684511+0000","last_undegraded":"2024-04-25T08:42:28.684511+0000","last_fullsized":"2024-04-25T08:42:28.684511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:16.561880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.583661+0000","last_change":"2024-04-25T08:42:21.353156+0000","last_active":"2024-04-25T08:42:28.583661+0000","last_peered":"2024-04-25T08:42:28.583661+0000","last_clean":"2024-04-25T08:42:28.583661+0000","last_became_active":"2024-04-25T08:42:21.352670+0000","last_became_peered":"2024-04-25T08:42:21.352670+0000","last_unstale":"2024-04-25T08:42:28.583661+0000","last_undegraded":"2024-04-25T08:42:28.583661+0000","last_fullsized":"2024-04-25T08:42:28.583661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:45.573590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.583361+0000","last_change":"2024-04-25T08:42:21.376357+0000","last_active":"2024-04-25T08:42:27.583361+0000","last_peered":"2024-04-25T08:42:27.583361+0000","last_clean":"2024-04-25T08:42:27.583361+0000","last_became_active":"2024-04-25T08:42:21.376092+0000","last_became_peered":"2024-04-25T08:42:21.376092+0000","last_unstale":"2024-04-25T08:42:27.583361+0000","last_undegraded":"2024-04-25T08:42:27.583361+0000","last_fullsized":"2024-04-25T08:42:27.583361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:15.923334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582441+0000","last_change":"2024-04-25T08:42:21.371951+0000","last_active":"2024-04-25T08:42:27.582441+0000","last_peered":"2024-04-25T08:42:27.582441+0000","last_clean":"2024-04-25T08:42:27.582441+0000","last_became_active":"2024-04-25T08:42:21.371306+0000","last_became_peered":"2024-04-25T08:42:21.371306+0000","last_unstale":"2024-04-25T08:42:27.582441+0000","last_undegraded":"2024-04-25T08:42:27.582441+0000","last_fullsized":"2024-04-25T08:42:27.582441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678232+0000","last_change":"2024-04-25T08:42:21.341496+0000","last_active":"2024-04-25T08:42:28.678232+0000","last_peered":"2024-04-25T08:42:28.678232+0000","last_clean":"2024-04-25T08:42:28.678232+0000","last_became_active":"2024-04-25T08:42:21.341224+0000","last_became_peered":"2024-04-25T08:42:21.341224+0000","last_unstale":"2024-04-25T08:42:28.678232+0000","last_undegraded":"2024-04-25T08:42:28.678232+0000","last_fullsized":"2024-04-25T08:42:28.678232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:14.302300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682722+0000","last_change":"2024-04-25T08:42:21.360668+0000","last_active":"2024-04-25T08:42:28.682722+0000","last_peered":"2024-04-25T08:42:28.682722+0000","last_clean":"2024-04-25T08:42:28.682722+0000","last_became_active":"2024-04-25T08:42:21.360398+0000","last_became_peered":"2024-04-25T08:42:21.360398+0000","last_unstale":"2024-04-25T08:42:28.682722+0000","last_undegraded":"2024-04-25T08:42:28.682722+0000","last_fullsized":"2024-04-25T08:42:28.682722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:33.626044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384826+0000","last_change":"2024-04-25T08:42:21.353592+0000","last_active":"2024-04-25T08:42:27.384826+0000","last_peered":"2024-04-25T08:42:27.384826+0000","last_clean":"2024-04-25T08:42:27.384826+0000","last_became_active":"2024-04-25T08:42:21.353329+0000","last_became_peered":"2024-04-25T08:42:21.353329+0000","last_unstale":"2024-04-25T08:42:27.384826+0000","last_undegraded":"2024-04-25T08:42:27.384826+0000","last_fullsized":"2024-04-25T08:42:27.384826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.390624+0000","last_change":"2024-04-25T08:42:21.356315+0000","last_active":"2024-04-25T08:42:27.390624+0000","last_peered":"2024-04-25T08:42:27.390624+0000","last_clean":"2024-04-25T08:42:27.390624+0000","last_became_active":"2024-04-25T08:42:21.355905+0000","last_became_peered":"2024-04-25T08:42:21.355905+0000","last_unstale":"2024-04-25T08:42:27.390624+0000","last_undegraded":"2024-04-25T08:42:27.390624+0000","last_fullsized":"2024-04-25T08:42:27.390624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:48.723555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389154+0000","last_change":"2024-04-25T08:42:21.371337+0000","last_active":"2024-04-25T08:42:27.389154+0000","last_peered":"2024-04-25T08:42:27.389154+0000","last_clean":"2024-04-25T08:42:27.389154+0000","last_became_active":"2024-04-25T08:42:21.370950+0000","last_became_peered":"2024-04-25T08:42:21.370950+0000","last_unstale":"2024-04-25T08:42:27.389154+0000","last_undegraded":"2024-04-25T08:42:27.389154+0000","last_fullsized":"2024-04-25T08:42:27.389154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:00.881802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.389045+0000","last_change":"2024-04-25T08:42:21.365819+0000","last_active":"2024-04-25T08:42:27.389045+0000","last_peered":"2024-04-25T08:42:27.389045+0000","last_clean":"2024-04-25T08:42:27.389045+0000","last_became_active":"2024-04-25T08:42:21.365562+0000","last_became_peered":"2024-04-25T08:42:21.365562+0000","last_unstale":"2024-04-25T08:42:27.389045+0000","last_undegraded":"2024-04-25T08:42:27.389045+0000","last_fullsized":"2024-04-25T08:42:27.389045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:19.937626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684772+0000","last_change":"2024-04-25T08:42:21.366180+0000","last_active":"2024-04-25T08:42:28.684772+0000","last_peered":"2024-04-25T08:42:28.684772+0000","last_clean":"2024-04-25T08:42:28.684772+0000","last_became_active":"2024-04-25T08:42:21.365934+0000","last_became_peered":"2024-04-25T08:42:21.365934+0000","last_unstale":"2024-04-25T08:42:28.684772+0000","last_undegraded":"2024-04-25T08:42:28.684772+0000","last_fullsized":"2024-04-25T08:42:28.684772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:36.740234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.384130+0000","last_change":"2024-04-25T08:42:21.368124+0000","last_active":"2024-04-25T08:42:27.384130+0000","last_peered":"2024-04-25T08:42:27.384130+0000","last_clean":"2024-04-25T08:42:27.384130+0000","last_became_active":"2024-04-25T08:42:21.367798+0000","last_became_peered":"2024-04-25T08:42:21.367798+0000","last_unstale":"2024-04-25T08:42:27.384130+0000","last_undegraded":"2024-04-25T08:42:27.384130+0000","last_fullsized":"2024-04-25T08:42:27.384130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.992009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.683866+0000","last_change":"2024-04-25T08:42:21.357410+0000","last_active":"2024-04-25T08:42:28.683866+0000","last_peered":"2024-04-25T08:42:28.683866+0000","last_clean":"2024-04-25T08:42:28.683866+0000","last_became_active":"2024-04-25T08:42:21.356770+0000","last_became_peered":"2024-04-25T08:42:21.356770+0000","last_unstale":"2024-04-25T08:42:28.683866+0000","last_undegraded":"2024-04-25T08:42:28.683866+0000","last_fullsized":"2024-04-25T08:42:28.683866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:34.569458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582479+0000","last_change":"2024-04-25T08:42:21.372610+0000","last_active":"2024-04-25T08:42:27.582479+0000","last_peered":"2024-04-25T08:42:27.582479+0000","last_clean":"2024-04-25T08:42:27.582479+0000","last_became_active":"2024-04-25T08:42:21.372336+0000","last_became_peered":"2024-04-25T08:42:21.372336+0000","last_unstale":"2024-04-25T08:42:27.582479+0000","last_undegraded":"2024-04-25T08:42:27.582479+0000","last_fullsized":"2024-04-25T08:42:27.582479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:25.717765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684118+0000","last_change":"2024-04-25T08:42:21.358088+0000","last_active":"2024-04-25T08:42:28.684118+0000","last_peered":"2024-04-25T08:42:28.684118+0000","last_clean":"2024-04-25T08:42:28.684118+0000","last_became_active":"2024-04-25T08:42:21.357310+0000","last_became_peered":"2024-04-25T08:42:21.357310+0000","last_unstale":"2024-04-25T08:42:28.684118+0000","last_undegraded":"2024-04-25T08:42:28.684118+0000","last_fullsized":"2024-04-25T08:42:28.684118+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:27.495278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:27.582808+0000","last_change":"2024-04-25T08:42:21.367883+0000","last_active":"2024-04-25T08:42:27.582808+0000","last_peered":"2024-04-25T08:42:27.582808+0000","last_clean":"2024-04-25T08:42:27.582808+0000","last_became_active":"2024-04-25T08:42:21.367176+0000","last_became_peered":"2024-04-25T08:42:21.367176+0000","last_unstale":"2024-04-25T08:42:27.582808+0000","last_undegraded":"2024-04-25T08:42:27.582808+0000","last_fullsized":"2024-04-25T08:42:27.582808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:52.525554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.682913+0000","last_change":"2024-04-25T08:42:21.362174+0000","last_active":"2024-04-25T08:42:28.682913+0000","last_peered":"2024-04-25T08:42:28.682913+0000","last_clean":"2024-04-25T08:42:28.682913+0000","last_became_active":"2024-04-25T08:42:21.361897+0000","last_became_peered":"2024-04-25T08:42:21.361897+0000","last_unstale":"2024-04-25T08:42:28.682913+0000","last_undegraded":"2024-04-25T08:42:28.682913+0000","last_fullsized":"2024-04-25T08:42:28.682913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:38.397190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.678838+0000","last_change":"2024-04-25T08:42:21.370457+0000","last_active":"2024-04-25T08:42:28.678838+0000","last_peered":"2024-04-25T08:42:28.678838+0000","last_clean":"2024-04-25T08:42:28.678838+0000","last_became_active":"2024-04-25T08:42:21.370091+0000","last_became_peered":"2024-04-25T08:42:21.370091+0000","last_unstale":"2024-04-25T08:42:28.678838+0000","last_undegraded":"2024-04-25T08:42:28.678838+0000","last_fullsized":"2024-04-25T08:42:28.678838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:07.771858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T08:42:28.684699+0000","last_change":"2024-04-25T08:42:21.370936+0000","last_active":"2024-04-25T08:42:28.684699+0000","last_peered":"2024-04-25T08:42:28.684699+0000","last_clean":"2024-04-25T08:42:28.684699+0000","last_became_active":"2024-04-25T08:42:21.369447+0000","last_became_peered":"2024-04-25T08:42:21.369447+0000","last_unstale":"2024-04-25T08:42:28.684699+0000","last_undegraded":"2024-04-25T08:42:28.684699+0000","last_fullsized":"2024-04-25T08:42:28.684699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:20.317844+0000","last_clean_scrub_stamp":"2024-04-25T08:42:20.317844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:38.445314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":876544,"data_stored":703014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":290816,"data_stored":112190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":290816,"data_stored":115906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":112064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":112224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":703070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:45.658 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T08:42:45.658 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T08:42:45.659 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T08:42:45.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:42:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.843+0000 7f992a600640 1 Processor -- start 2024-04-25T08:42:45.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.843+0000 7f992a600640 1 -- start start 2024-04-25T08:42:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f992a600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f992406da60 0x7f992406f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f992a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f992406f7c0 0x7f9924099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f992a600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992406fcd0 con 0x7f9924099f40 2024-04-25T08:42:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f992a600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99240d0df0 con 0x7f992406f7c0 2024-04-25T08:42:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f992a600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992406ffb0 con 0x7f992406da60 2024-04-25T08:42:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f991be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f992406f7c0 0x7f9924099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f991be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f992406f7c0 0x7f9924099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60446/0 (socket says 172.21.15.46:60446) 2024-04-25T08:42:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f991be00640 1 -- 172.21.15.46:0/3852504417 learned_addr learned my addr 172.21.15.46:0/3852504417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f9929200640 1 --2- 172.21.15.46:0/3852504417 >> v2:172.21.15.82:3300/0 conn(0x7f992406da60 0x7f992406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f991be00640 1 -- 172.21.15.46:0/3852504417 >> v2:172.21.15.82:3300/0 conn(0x7f992406da60 msgr2=0x7f992406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f991be00640 1 --2- 172.21.15.46:0/3852504417 >> v2:172.21.15.82:3300/0 conn(0x7f992406da60 0x7f992406f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f991be00640 1 -- 172.21.15.46:0/3852504417 >> v1:172.21.15.82:6789/0 conn(0x7f9924099f40 legacy=0x7f992415d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.847+0000 7f991be00640 1 -- 172.21.15.46:0/3852504417 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924070290 con 0x7f992406f7c0 2024-04-25T08:42:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f991be00640 1 --2- 172.21.15.46:0/3852504417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f992406f7c0 0x7f9924099a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99240b4100 tx=0x7f990c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30a5c654e4a0ece server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f9923e00640 1 -- 172.21.15.46:0/3852504417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0b0070 con 0x7f992406f7c0 2024-04-25T08:42:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f9923e00640 1 -- 172.21.15.46:0/3852504417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c0ab600 con 0x7f992406f7c0 2024-04-25T08:42:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f9923e00640 1 -- 172.21.15.46:0/3852504417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0ab9d0 con 0x7f992406f7c0 2024-04-25T08:42:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 -- 172.21.15.46:0/3852504417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f992406f7c0 msgr2=0x7f9924099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 --2- 172.21.15.46:0/3852504417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f992406f7c0 0x7f9924099a00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99240b4100 tx=0x7f990c01c490 comp rx=0 tx=0).stop 2024-04-25T08:42:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 -- 172.21.15.46:0/3852504417 shutdown_connections 2024-04-25T08:42:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 --2- 172.21.15.46:0/3852504417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f992406f7c0 0x7f9924099a00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 --2- 172.21.15.46:0/3852504417 >> v2:172.21.15.82:3300/0 conn(0x7f992406da60 0x7f992406f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 -- 172.21.15.46:0/3852504417 >> 172.21.15.46:0/3852504417 conn(0x7f99240b3000 msgr2=0x7f99240b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 -- 172.21.15.46:0/3852504417 shutdown_connections 2024-04-25T08:42:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 -- 172.21.15.46:0/3852504417 wait complete. 2024-04-25T08:42:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 Processor -- start 2024-04-25T08:42:45.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 -- start start 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f992406da60 0x7f9924159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f992406f7c0 0x7f99240ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9924099f40 0x7f99240ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992415e250 con 0x7f9924099f40 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992415e0d0 con 0x7f992406f7c0 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f992a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99240cf7e0 con 0x7f992406da60 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.851+0000 7f9929c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9924099f40 0x7f99240ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9929c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9924099f40 0x7f99240ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60450/0 (socket says 172.21.15.46:60450) 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9929c00640 1 -- 172.21.15.46:0/2095592699 learned_addr learned my addr 172.21.15.46:0/2095592699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f991be00640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f992406f7c0 0x7f99240ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9929200640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f992406da60 0x7f9924159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9929c00640 1 -- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f992406da60 msgr2=0x7f9924159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9929c00640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f992406da60 0x7f9924159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9929c00640 1 -- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f992406f7c0 msgr2=0x7f99240ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:45.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9929c00640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f992406f7c0 0x7f99240ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9929c00640 1 -- 172.21.15.46:0/2095592699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240b4410 con 0x7f9924099f40 2024-04-25T08:42:45.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9929c00640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9924099f40 0x7f99240ce870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c770 tx=0x7f991c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9921600640 1 -- 172.21.15.46:0/2095592699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c003b80 con 0x7f9924099f40 2024-04-25T08:42:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9921600640 1 -- 172.21.15.46:0/2095592699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c003d20 con 0x7f9924099f40 2024-04-25T08:42:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99241f7420 con 0x7f9924099f40 2024-04-25T08:42:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9921600640 1 -- 172.21.15.46:0/2095592699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c00aa90 con 0x7f9924099f40 2024-04-25T08:42:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99241f79e0 con 0x7f9924099f40 2024-04-25T08:42:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9924069010 con 0x7f9924099f40 2024-04-25T08:42:45.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9921600640 1 -- 172.21.15.46:0/2095592699 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f991c00ac30 con 0x7f9924099f40 2024-04-25T08:42:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9921600640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9900066f90 0x7f9900069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.855+0000 7f9921600640 1 -- 172.21.15.46:0/2095592699 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f991c0850c0 con 0x7f9924099f40 2024-04-25T08:42:45.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.859+0000 7f9929200640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9900066f90 0x7f9900069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:45.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.859+0000 7f9929200640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9900066f90 0x7f9900069450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f99240c8220 tx=0x7f9914008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:45.859+0000 7f9921600640 1 -- 172.21.15.46:0/2095592699 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f991c0557b0 con 0x7f9924099f40 2024-04-25T08:42:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.055+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f99240b7490 con 0x7f9924099f40 2024-04-25T08:42:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.055+0000 7f9921600640 1 -- 172.21.15.46:0/2095592699 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f991c059850 con 0x7f9924099f40 2024-04-25T08:42:46.050 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:46.050 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9900066f90 msgr2=0x7f9900069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9900066f90 0x7f9900069450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f99240c8220 tx=0x7f9914008040 comp rx=0 tx=0).stop 2024-04-25T08:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9924099f40 msgr2=0x7f99240ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9924099f40 0x7f99240ce870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c770 tx=0x7f991c00cc30 comp rx=0 tx=0).stop 2024-04-25T08:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 shutdown_connections 2024-04-25T08:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9900066f90 0x7f9900069450 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9924099f40 0x7f99240ce870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f992406f7c0 0x7f99240ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 --2- 172.21.15.46:0/2095592699 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f992406da60 0x7f9924159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 >> 172.21.15.46:0/2095592699 conn(0x7f99240b3000 msgr2=0x7f992415f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 shutdown_connections 2024-04-25T08:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.059+0000 7f992a600640 1 -- 172.21.15.46:0/2095592699 wait complete. 2024-04-25T08:42:46.071 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T08:42:46.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f5b000640 1 Processor -- start 2024-04-25T08:42:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f5b000640 1 -- start start 2024-04-25T08:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f5b000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f540c3580 0x7f3f54166450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f5b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f540b2810 0x7f3f5406e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f5b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f54166990 con 0x7f3f54162130 2024-04-25T08:42:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f5b000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f540ced00 con 0x7f3f540c3580 2024-04-25T08:42:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f5b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cef30 con 0x7f3f540b2810 2024-04-25T08:42:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f4b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3f54162130 0x7f3f540c3960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59200/0 (socket says 172.21.15.46:59200) 2024-04-25T08:42:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f4b400640 1 -- 172.21.15.46:0/252814299 learned_addr learned my addr 172.21.15.46:0/252814299 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f59000640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f540b2810 0x7f3f5406e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f53e00640 1 --2- 172.21.15.46:0/252814299 >> v2:172.21.15.82:3300/0 conn(0x7f3f540c3580 0x7f3f54166450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597821821 0 0) 0x7f3f3c001120 con 0x7f3f54162130 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.219+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f38003530 con 0x7f3f54162130 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855633441 0 0) 0x7f3f38003530 con 0x7f3f54162130 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f540b2810 msgr2=0x7f3f5406e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f540b2810 0x7f3f5406e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 >> v2:172.21.15.82:3300/0 conn(0x7f3f540c3580 msgr2=0x7f3f54166450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 --2- 172.21.15.46:0/252814299 >> v2:172.21.15.82:3300/0 conn(0x7f3f540c3580 0x7f3f54166450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540cf210 con 0x7f3f54162130 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f3f3c001850 con 0x7f3f54162130 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 >> v1:172.21.15.46:6789/0 conn(0x7f3f54162130 legacy=0x7f3f540c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f540b2810 0x7f3f380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38003f90 0x7f3f38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f38004d50 0x7f3f380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cef30 con 0x7f3f38003f90 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3800b9d0 con 0x7f3f38004d50 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3800bb50 con 0x7f3f540b2810 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f4b400640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38003f90 0x7f3f38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f53e00640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f540b2810 0x7f3f380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f59000640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f38004d50 0x7f3f380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f4b400640 1 -- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f540b2810 msgr2=0x7f3f380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f4b400640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f540b2810 0x7f3f380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f4b400640 1 -- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f38004d50 msgr2=0x7f3f380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f4b400640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f38004d50 0x7f3f380051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f4b400640 1 -- 172.21.15.46:0/252814299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f38003530 con 0x7f3f38003f90 2024-04-25T08:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.223+0000 7f3f4b400640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38003f90 0x7f3f38004810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c350 tx=0x7f3f3c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96189d2e7b8ea072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007e60 con 0x7f3f38003f90 2024-04-25T08:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c005b00 con 0x7f3f38003f90 2024-04-25T08:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f53400640 1 -- 172.21.15.46:0/252814299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007070 con 0x7f3f38003f90 2024-04-25T08:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 -- 172.21.15.46:0/252814299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38003f90 msgr2=0x7f3f38004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 --2- 172.21.15.46:0/252814299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38003f90 0x7f3f38004810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c350 tx=0x7f3f3c01f330 comp rx=0 tx=0).stop 2024-04-25T08:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f59000640 1 -- 172.21.15.46:0/252814299 reap_dead start 2024-04-25T08:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 -- 172.21.15.46:0/252814299 shutdown_connections 2024-04-25T08:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 -- 172.21.15.46:0/252814299 >> 172.21.15.46:0/252814299 conn(0x7f3f54099a00 msgr2=0x7f3f540b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 -- 172.21.15.46:0/252814299 shutdown_connections 2024-04-25T08:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 -- 172.21.15.46:0/252814299 wait complete. 2024-04-25T08:42:46.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 Processor -- start 2024-04-25T08:42:46.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 -- start start 2024-04-25T08:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f38003f90 0x7f3f540cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f54152a60 0x7f3f541f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f541f83d0 0x7f3f540ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cfab0 con 0x7f3f541f83d0 2024-04-25T08:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cf7d0 con 0x7f3f54152a60 2024-04-25T08:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f5b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540cffd0 con 0x7f3f38003f90 2024-04-25T08:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f59000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f541f83d0 0x7f3f540ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f59000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f541f83d0 0x7f3f540ae560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60456/0 (socket says 172.21.15.46:60456) 2024-04-25T08:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f59000640 1 -- 172.21.15.46:0/109432691 learned_addr learned my addr 172.21.15.46:0/109432691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:46.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f59000640 1 -- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f38003f90 msgr2=0x7f3f540cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f4b400640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f54152a60 0x7f3f541f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.227+0000 7f3f59000640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f38003f90 0x7f3f540cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f59000640 1 -- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f54152a60 msgr2=0x7f3f541f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f59000640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f54152a60 0x7f3f541f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f59000640 1 -- 172.21.15.46:0/109432691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f541f8a50 con 0x7f3f541f83d0 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f4b400640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f54152a60 0x7f3f541f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f59000640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f541f83d0 0x7f3f540ae560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3f540caa00 tx=0x7f3f4c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f50c00640 1 -- 172.21.15.46:0/109432691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c019020 con 0x7f3f541f83d0 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f50c00640 1 -- 172.21.15.46:0/109432691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c003620 con 0x7f3f541f83d0 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f540caad0 con 0x7f3f541f83d0 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f50c00640 1 -- 172.21.15.46:0/109432691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c003a10 con 0x7f3f541f83d0 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f540cb090 con 0x7f3f541f83d0 2024-04-25T08:42:46.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f540cb290 con 0x7f3f541f83d0 2024-04-25T08:42:46.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f50c00640 1 -- 172.21.15.46:0/109432691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f4c01c020 con 0x7f3f541f83d0 2024-04-25T08:42:46.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f50c00640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f30066f60 0x7f3f30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f50c00640 1 -- 172.21.15.46:0/109432691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3f4c085430 con 0x7f3f541f83d0 2024-04-25T08:42:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.231+0000 7f3f53e00640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f30066f60 0x7f3f30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.235+0000 7f3f53e00640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f30066f60 0x7f3f30069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009fc0 tx=0x7f3f44009750 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.235+0000 7f3f50c00640 1 -- 172.21.15.46:0/109432691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f540cb290 con 0x7f3f541f83d0 2024-04-25T08:42:46.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.399+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f54069080 con 0x7f3f541f83d0 2024-04-25T08:42:46.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.399+0000 7f3f50c00640 1 -- 172.21.15.46:0/109432691 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f3f4c023030 con 0x7f3f541f83d0 2024-04-25T08:42:46.395 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:46.395 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f30066f60 msgr2=0x7f3f30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f30066f60 0x7f3f30069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009fc0 tx=0x7f3f44009750 comp rx=0 tx=0).stop 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f541f83d0 msgr2=0x7f3f540ae560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f541f83d0 0x7f3f540ae560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3f540caa00 tx=0x7f3f4c011920 comp rx=0 tx=0).stop 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 shutdown_connections 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f30066f60 0x7f3f30069420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f541f83d0 0x7f3f540ae560 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f54152a60 0x7f3f541f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 --2- 172.21.15.46:0/109432691 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f38003f90 0x7f3f540cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 >> 172.21.15.46:0/109432691 conn(0x7f3f54099a00 msgr2=0x7f3f5415fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:46.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 shutdown_connections 2024-04-25T08:42:46.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.403+0000 7f3f5b000640 1 -- 172.21.15.46:0/109432691 wait complete. 2024-04-25T08:42:46.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:42:46.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:42:46.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a3800640 1 Processor -- start 2024-04-25T08:42:46.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a3800640 1 -- start start 2024-04-25T08:42:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a3800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f839c145310 0x7f839c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a3800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c06dc70 0x7f839c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a3800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c0cd930 con 0x7f839c0cf1b0 2024-04-25T08:42:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a3800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839c0cdbd0 con 0x7f839c145310 2024-04-25T08:42:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a3800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0cdeb0 con 0x7f839c06dc70 2024-04-25T08:42:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a0e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f839c145310 0x7f839c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a0e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f839c145310 0x7f839c06e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60470/0 (socket says 172.21.15.46:60470) 2024-04-25T08:42:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.567+0000 7f83a0e00640 1 -- 172.21.15.46:0/3274422079 learned_addr learned my addr 172.21.15.46:0/3274422079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f8393e00640 1 --2- 172.21.15.46:0/3274422079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c06dc70 0x7f839c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a0e00640 1 -- 172.21.15.46:0/3274422079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c06dc70 msgr2=0x7f839c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a0e00640 1 --2- 172.21.15.46:0/3274422079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c06dc70 0x7f839c0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a0e00640 1 -- 172.21.15.46:0/3274422079 >> v1:172.21.15.82:6789/0 conn(0x7f839c0cf1b0 legacy=0x7f839c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a0e00640 1 -- 172.21.15.46:0/3274422079 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0ce190 con 0x7f839c145310 2024-04-25T08:42:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f8393e00640 1 --2- 172.21.15.46:0/3274422079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c06dc70 0x7f839c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a0e00640 1 --2- 172.21.15.46:0/3274422079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c145310 0x7f839c06e0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f838c009870 tx=0x7f838c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfdf097c98821f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f839be00640 1 -- 172.21.15.46:0/3274422079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c0aa610 con 0x7f839c145310 2024-04-25T08:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f839be00640 1 -- 172.21.15.46:0/3274422079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c0af070 con 0x7f839c145310 2024-04-25T08:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f839be00640 1 -- 172.21.15.46:0/3274422079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c0aa9d0 con 0x7f839c145310 2024-04-25T08:42:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a3800640 1 -- 172.21.15.46:0/3274422079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c145310 msgr2=0x7f839c06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a3800640 1 --2- 172.21.15.46:0/3274422079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c145310 0x7f839c06e0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f838c009870 tx=0x7f838c01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a3800640 1 -- 172.21.15.46:0/3274422079 shutdown_connections 2024-04-25T08:42:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a3800640 1 --2- 172.21.15.46:0/3274422079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c06dc70 0x7f839c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a3800640 1 --2- 172.21.15.46:0/3274422079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c145310 0x7f839c06e0d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a3800640 1 -- 172.21.15.46:0/3274422079 >> 172.21.15.46:0/3274422079 conn(0x7f839c06ac10 msgr2=0x7f839c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a3800640 1 -- 172.21.15.46:0/3274422079 shutdown_connections 2024-04-25T08:42:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.571+0000 7f83a3800640 1 -- 172.21.15.46:0/3274422079 wait complete. 2024-04-25T08:42:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 Processor -- start 2024-04-25T08:42:46.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 -- start start 2024-04-25T08:42:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c0cf1b0 0x7f839c144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f839c145310 0x7f839c158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c0a9550 0x7f839c13b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c15d920 con 0x7f839c0cf1b0 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c15d7a0 con 0x7f839c145310 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c168510 con 0x7f839c0a9550 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c0cf1b0 0x7f839c144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c0cf1b0 0x7f839c144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60486/0 (socket says 172.21.15.46:60486) 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a0e00640 1 -- 172.21.15.46:0/1907188444 learned_addr learned my addr 172.21.15.46:0/1907188444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a1800640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c0a9550 0x7f839c13b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f8393e00640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f839c145310 0x7f839c158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a0e00640 1 -- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c0a9550 msgr2=0x7f839c13b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a0e00640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c0a9550 0x7f839c13b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a0e00640 1 -- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f839c145310 msgr2=0x7f839c158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a0e00640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f839c145310 0x7f839c158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a0e00640 1 -- 172.21.15.46:0/1907188444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c144870 con 0x7f839c0cf1b0 2024-04-25T08:42:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f8393e00640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f839c145310 0x7f839c158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a0e00640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c0cf1b0 0x7f839c144000 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f838c0ac040 tx=0x7f838c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f8399600640 1 -- 172.21.15.46:0/1907188444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c0bd5f0 con 0x7f839c0cf1b0 2024-04-25T08:42:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f8399600640 1 -- 172.21.15.46:0/1907188444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c0bd7b0 con 0x7f839c0cf1b0 2024-04-25T08:42:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f8399600640 1 -- 172.21.15.46:0/1907188444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c0bdba0 con 0x7f839c0cf1b0 2024-04-25T08:42:46.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c0b15c0 con 0x7f839c0cf1b0 2024-04-25T08:42:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.575+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c144540 con 0x7f839c0cf1b0 2024-04-25T08:42:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.579+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839c069010 con 0x7f839c0cf1b0 2024-04-25T08:42:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.579+0000 7f8399600640 1 -- 172.21.15.46:0/1907188444 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f838c0b33f0 con 0x7f839c0cf1b0 2024-04-25T08:42:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.579+0000 7f8399600640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8378066f60 0x7f8378069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.579+0000 7f8399600640 1 -- 172.21.15.46:0/1907188444 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f838c11fda0 con 0x7f839c0cf1b0 2024-04-25T08:42:46.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.579+0000 7f8393e00640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8378066f60 0x7f8378069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.579+0000 7f8393e00640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8378066f60 0x7f8378069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f839c1f9890 tx=0x7f8384008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.583+0000 7f8399600640 1 -- 172.21.15.46:0/1907188444 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f838c0a7070 con 0x7f839c0cf1b0 2024-04-25T08:42:46.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f839c069010 con 0x7f839c0cf1b0 2024-04-25T08:42:46.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f8399600640 1 -- 172.21.15.46:0/1907188444 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f838c016110 con 0x7f839c0cf1b0 2024-04-25T08:42:46.739 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:42:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8378066f60 msgr2=0x7f8378069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8378066f60 0x7f8378069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f839c1f9890 tx=0x7f8384008040 comp rx=0 tx=0).stop 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c0cf1b0 msgr2=0x7f839c144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c0cf1b0 0x7f839c144000 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f838c0ac040 tx=0x7f838c01ad30 comp rx=0 tx=0).stop 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 shutdown_connections 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8378066f60 0x7f8378069420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839c0a9550 0x7f839c13b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f839c145310 0x7f839c158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 --2- 172.21.15.46:0/1907188444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839c0cf1b0 0x7f839c144000 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 >> 172.21.15.46:0/1907188444 conn(0x7f839c06ac10 msgr2=0x7f839c0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 shutdown_connections 2024-04-25T08:42:46.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.747+0000 7f83a3800640 1 -- 172.21.15.46:0/1907188444 wait complete. 2024-04-25T08:42:46.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:42:46.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba87e00640 1 Processor -- start 2024-04-25T08:42:46.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba87e00640 1 -- start start 2024-04-25T08:42:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba87e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba8015ef30 0x7fba800b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba87e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba800b2d40 0x7fba80099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba87e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fba8015ff80 0x7fba80099f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba87e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba801604d0 con 0x7fba8015ef30 2024-04-25T08:42:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba87e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba800b17b0 con 0x7fba8015ff80 2024-04-25T08:42:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba87e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8015f310 con 0x7fba800b2d40 2024-04-25T08:42:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba8015ef30 0x7fba800b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba8015ef30 0x7fba800b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60500/0 (socket says 172.21.15.46:60500) 2024-04-25T08:42:46.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85400640 1 -- 172.21.15.46:0/4162218067 learned_addr learned my addr 172.21.15.46:0/4162218067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba84a00640 1 --2- 172.21.15.46:0/4162218067 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba800b2d40 0x7fba80099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85e00640 1 --2- 172.21.15.46:0/4162218067 >> v2:172.21.15.82:3300/0 conn(0x7fba8015ff80 0x7fba80099f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85400640 1 -- 172.21.15.46:0/4162218067 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba800b2d40 msgr2=0x7fba80099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85400640 1 --2- 172.21.15.46:0/4162218067 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba800b2d40 0x7fba80099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85400640 1 -- 172.21.15.46:0/4162218067 >> v2:172.21.15.82:3300/0 conn(0x7fba8015ff80 msgr2=0x7fba80099f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85400640 1 --2- 172.21.15.46:0/4162218067 >> v2:172.21.15.82:3300/0 conn(0x7fba8015ff80 0x7fba80099f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85400640 1 -- 172.21.15.46:0/4162218067 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba800b0e80 con 0x7fba8015ef30 2024-04-25T08:42:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba84a00640 1 --2- 172.21.15.46:0/4162218067 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba800b2d40 0x7fba80099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:46.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.911+0000 7fba85e00640 1 --2- 172.21.15.46:0/4162218067 >> v2:172.21.15.82:3300/0 conn(0x7fba8015ff80 0x7fba80099f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:46.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba85400640 1 --2- 172.21.15.46:0/4162218067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 0x7fba800b1c40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fba800d07f0 tx=0x7fba7401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2b427f738a3cea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba7fe00640 1 -- 172.21.15.46:0/4162218067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba740b0070 con 0x7fba8015ef30 2024-04-25T08:42:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba7fe00640 1 -- 172.21.15.46:0/4162218067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba740ab600 con 0x7fba8015ef30 2024-04-25T08:42:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba7fe00640 1 -- 172.21.15.46:0/4162218067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba740ab9d0 con 0x7fba8015ef30 2024-04-25T08:42:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba87e00640 1 -- 172.21.15.46:0/4162218067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 msgr2=0x7fba800b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba87e00640 1 --2- 172.21.15.46:0/4162218067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 0x7fba800b1c40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fba800d07f0 tx=0x7fba7401c490 comp rx=0 tx=0).stop 2024-04-25T08:42:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba87e00640 1 -- 172.21.15.46:0/4162218067 shutdown_connections 2024-04-25T08:42:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba87e00640 1 --2- 172.21.15.46:0/4162218067 >> v2:172.21.15.82:3300/0 conn(0x7fba8015ff80 0x7fba80099f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba87e00640 1 --2- 172.21.15.46:0/4162218067 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba800b2d40 0x7fba80099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba87e00640 1 --2- 172.21.15.46:0/4162218067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 0x7fba800b1c40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba87e00640 1 -- 172.21.15.46:0/4162218067 >> 172.21.15.46:0/4162218067 conn(0x7fba800b3130 msgr2=0x7fba800b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba87e00640 1 -- 172.21.15.46:0/4162218067 shutdown_connections 2024-04-25T08:42:46.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.915+0000 7fba87e00640 1 -- 172.21.15.46:0/4162218067 wait complete. 2024-04-25T08:42:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 Processor -- start 2024-04-25T08:42:46.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 -- start start 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 0x7fba80152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8015ff80 0x7fba8006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba80070210 0x7fba80157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800ccbc0 con 0x7fba8015ef30 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800cca40 con 0x7fba80070210 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba800ccd40 con 0x7fba8015ff80 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 0x7fba80152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 0x7fba80152be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60510/0 (socket says 172.21.15.46:60510) 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85400640 1 -- 172.21.15.46:0/3305196610 learned_addr learned my addr 172.21.15.46:0/3305196610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85e00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba80070210 0x7fba80157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85400640 1 -- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8015ff80 msgr2=0x7fba8006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba84a00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8015ff80 0x7fba8006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85400640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8015ff80 0x7fba8006fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85400640 1 -- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba80070210 msgr2=0x7fba80157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85400640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba80070210 0x7fba80157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85400640 1 -- 172.21.15.46:0/3305196610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba801584b0 con 0x7fba8015ef30 2024-04-25T08:42:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85e00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba80070210 0x7fba80157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba84a00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8015ff80 0x7fba8006fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:46.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba85400640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 0x7fba80152be0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fba740ae040 tx=0x7fba74019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba7d600640 1 -- 172.21.15.46:0/3305196610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba740b0040 con 0x7fba8015ef30 2024-04-25T08:42:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba7d600640 1 -- 172.21.15.46:0/3305196610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba740b9070 con 0x7fba8015ef30 2024-04-25T08:42:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba80158790 con 0x7fba8015ef30 2024-04-25T08:42:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba7d600640 1 -- 172.21.15.46:0/3305196610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba740b46a0 con 0x7fba8015ef30 2024-04-25T08:42:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.919+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba80165000 con 0x7fba8015ef30 2024-04-25T08:42:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.923+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba80070620 con 0x7fba8015ef30 2024-04-25T08:42:46.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.923+0000 7fba7d600640 1 -- 172.21.15.46:0/3305196610 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba740b4840 con 0x7fba8015ef30 2024-04-25T08:42:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.923+0000 7fba7d600640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba60066f90 0x7fba60069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.923+0000 7fba7d600640 1 -- 172.21.15.46:0/3305196610 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fba7411f110 con 0x7fba8015ef30 2024-04-25T08:42:46.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.923+0000 7fba84a00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba60066f90 0x7fba60069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:46.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.923+0000 7fba84a00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba60066f90 0x7fba60069450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fba6c006fd0 tx=0x7fba6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:46.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:46.927+0000 7fba7d600640 1 -- 172.21.15.46:0/3305196610 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba80070620 con 0x7fba8015ef30 2024-04-25T08:42:47.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.087+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba80069010 con 0x7fba8015ef30 2024-04-25T08:42:47.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba7d600640 1 -- 172.21.15.46:0/3305196610 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba74016110 con 0x7fba8015ef30 2024-04-25T08:42:47.083 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:42:47.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba60066f90 msgr2=0x7fba60069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba60066f90 0x7fba60069450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fba6c006fd0 tx=0x7fba6c008040 comp rx=0 tx=0).stop 2024-04-25T08:42:47.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 msgr2=0x7fba80152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 0x7fba80152be0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fba740ae040 tx=0x7fba74019230 comp rx=0 tx=0).stop 2024-04-25T08:42:47.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 shutdown_connections 2024-04-25T08:42:47.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba60066f90 0x7fba60069450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba80070210 0x7fba80157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba8015ff80 0x7fba8006fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 --2- 172.21.15.46:0/3305196610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba8015ef30 0x7fba80152be0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 >> 172.21.15.46:0/3305196610 conn(0x7fba800b3130 msgr2=0x7fba800aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:47.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 shutdown_connections 2024-04-25T08:42:47.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.091+0000 7fba87e00640 1 -- 172.21.15.46:0/3305196610 wait complete. 2024-04-25T08:42:47.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:42:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba34200640 1 Processor -- start 2024-04-25T08:42:47.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba34200640 1 -- start start 2024-04-25T08:42:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba34200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c06f150 0x7fba2c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba34200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c15e440 con 0x7fba2c143990 2024-04-25T08:42:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba34200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2c15e6e0 con 0x7fba2c1440d0 2024-04-25T08:42:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba34200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c15ed30 con 0x7fba2c06f150 2024-04-25T08:42:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba31800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fba2c143990 0x7fba2c06ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59206/0 (socket says 172.21.15.46:59206) 2024-04-25T08:42:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba31800640 1 -- 172.21.15.46:0/269964277 learned_addr learned my addr 172.21.15.46:0/269964277 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:47.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba32200640 1 --2- 172.21.15.46:0/269964277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c06f150 0x7fba2c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba2be00640 1 -- 172.21.15.46:0/269964277 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765027806 0 0) 0x7fba2c15e440 con 0x7fba2c143990 2024-04-25T08:42:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba2be00640 1 -- 172.21.15.46:0/269964277 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba14003530 con 0x7fba2c143990 2024-04-25T08:42:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba2be00640 1 -- 172.21.15.46:0/269964277 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184006931 0 0) 0x7fba2c15e6e0 con 0x7fba2c1440d0 2024-04-25T08:42:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba32200640 1 -- 172.21.15.46:0/269964277 >> v1:172.21.15.82:6789/0 conn(0x7fba2c1440d0 legacy=0x7fba2c0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba32200640 1 -- 172.21.15.46:0/269964277 >> v1:172.21.15.46:6789/0 conn(0x7fba2c143990 legacy=0x7fba2c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.255+0000 7fba32200640 1 -- 172.21.15.46:0/269964277 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c15e9c0 con 0x7fba2c06f150 2024-04-25T08:42:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba32200640 1 --2- 172.21.15.46:0/269964277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c06f150 0x7fba2c0cecc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba2c15eeb0 tx=0x7fba2401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=613f87d39ee029a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba2be00640 1 -- 172.21.15.46:0/269964277 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240b2070 con 0x7fba2c06f150 2024-04-25T08:42:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba2be00640 1 -- 172.21.15.46:0/269964277 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba240a5960 con 0x7fba2c06f150 2024-04-25T08:42:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba2be00640 1 -- 172.21.15.46:0/269964277 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240a5d30 con 0x7fba2c06f150 2024-04-25T08:42:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba34200640 1 -- 172.21.15.46:0/269964277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c06f150 msgr2=0x7fba2c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba34200640 1 --2- 172.21.15.46:0/269964277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c06f150 0x7fba2c0cecc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba2c15eeb0 tx=0x7fba2401c990 comp rx=0 tx=0).stop 2024-04-25T08:42:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba34200640 1 -- 172.21.15.46:0/269964277 shutdown_connections 2024-04-25T08:42:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba34200640 1 --2- 172.21.15.46:0/269964277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c06f150 0x7fba2c0cecc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba34200640 1 -- 172.21.15.46:0/269964277 >> 172.21.15.46:0/269964277 conn(0x7fba2c06ac10 msgr2=0x7fba2c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba34200640 1 -- 172.21.15.46:0/269964277 shutdown_connections 2024-04-25T08:42:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.259+0000 7fba34200640 1 -- 172.21.15.46:0/269964277 wait complete. 2024-04-25T08:42:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 Processor -- start 2024-04-25T08:42:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 -- start start 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2c06f150 0x7fba2c159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c143990 0x7fba2c0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba2c1440d0 0x7fba2c0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c1686a0 con 0x7fba2c06f150 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c168520 con 0x7fba2c1440d0 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2c168820 con 0x7fba2c143990 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba31800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2c06f150 0x7fba2c159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba31800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2c06f150 0x7fba2c159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60522/0 (socket says 172.21.15.46:60522) 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba31800640 1 -- 172.21.15.46:0/141207911 learned_addr learned my addr 172.21.15.46:0/141207911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba30e00640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c143990 0x7fba2c0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba32200640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba2c1440d0 0x7fba2c0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba31800640 1 -- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c143990 msgr2=0x7fba2c0b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba31800640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c143990 0x7fba2c0b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba31800640 1 -- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba2c1440d0 msgr2=0x7fba2c0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba31800640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba2c1440d0 0x7fba2c0b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba31800640 1 -- 172.21.15.46:0/141207911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c1605f0 con 0x7fba2c06f150 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba31800640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2c06f150 0x7fba2c159b40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c450 tx=0x7fba2000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba29600640 1 -- 172.21.15.46:0/141207911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20017070 con 0x7fba2c06f150 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba29600640 1 -- 172.21.15.46:0/141207911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba20005cf0 con 0x7fba2c06f150 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba29600640 1 -- 172.21.15.46:0/141207911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20007290 con 0x7fba2c06f150 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2c15fd50 con 0x7fba2c06f150 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c160310 con 0x7fba2c06f150 2024-04-25T08:42:47.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.263+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2c069010 con 0x7fba2c06f150 2024-04-25T08:42:47.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.267+0000 7fba29600640 1 -- 172.21.15.46:0/141207911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba2001a070 con 0x7fba2c06f150 2024-04-25T08:42:47.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.267+0000 7fba29600640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba0c066f60 0x7fba0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.267+0000 7fba29600640 1 -- 172.21.15.46:0/141207911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fba20084770 con 0x7fba2c06f150 2024-04-25T08:42:47.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.267+0000 7fba30e00640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba0c066f60 0x7fba0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.267+0000 7fba30e00640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba0c066f60 0x7fba0c069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fba2c1664e0 tx=0x7fba18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.271+0000 7fba29600640 1 -- 172.21.15.46:0/141207911 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba20054d50 con 0x7fba2c06f150 2024-04-25T08:42:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.431+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba2c0b10f0 con 0x7fba2c06f150 2024-04-25T08:42:47.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.431+0000 7fba29600640 1 -- 172.21.15.46:0/141207911 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba20058df0 con 0x7fba2c06f150 2024-04-25T08:42:47.426 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:42:47.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba0c066f60 msgr2=0x7fba0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba0c066f60 0x7fba0c069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fba2c1664e0 tx=0x7fba18009040 comp rx=0 tx=0).stop 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2c06f150 msgr2=0x7fba2c159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2c06f150 0x7fba2c159b40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c450 tx=0x7fba2000c910 comp rx=0 tx=0).stop 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 shutdown_connections 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba0c066f60 0x7fba0c069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba2c1440d0 0x7fba2c0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba2c143990 0x7fba2c0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 --2- 172.21.15.46:0/141207911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba2c06f150 0x7fba2c159b40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 >> 172.21.15.46:0/141207911 conn(0x7fba2c06ac10 msgr2=0x7fba2c15ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 shutdown_connections 2024-04-25T08:42:47.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.435+0000 7fba34200640 1 -- 172.21.15.46:0/141207911 wait complete. 2024-04-25T08:42:47.447 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:42:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c7600640 1 Processor -- start 2024-04-25T08:42:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c7600640 1 -- start start 2024-04-25T08:42:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c7600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99c015ebe0 0x7f99c0070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c7600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99c006f520 0x7f99c00cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c7600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c015f0a0 con 0x7f99c00cd500 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c7600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00cb120 con 0x7f99c006f520 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c7600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00cb3c0 con 0x7f99c015ebe0 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c5600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99c006f520 0x7f99c00cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c5600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99c006f520 0x7f99c00cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60526/0 (socket says 172.21.15.46:60526) 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c5600640 1 -- 172.21.15.46:0/2224784921 learned_addr learned my addr 172.21.15.46:0/2224784921 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c5600640 1 -- 172.21.15.46:0/2224784921 >> v2:172.21.15.82:3300/0 conn(0x7f99c015ebe0 msgr2=0x7f99c0070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99bfe00640 1 --2- 172.21.15.46:0/2224784921 >> v2:172.21.15.82:3300/0 conn(0x7f99c015ebe0 0x7f99c0070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c5600640 1 --2- 172.21.15.46:0/2224784921 >> v2:172.21.15.82:3300/0 conn(0x7f99c015ebe0 0x7f99c0070030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c5600640 1 -- 172.21.15.46:0/2224784921 >> v1:172.21.15.82:6789/0 conn(0x7f99c00cd500 legacy=0x7f99c015e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99c5600640 1 -- 172.21.15.46:0/2224784921 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00b2b20 con 0x7f99c006f520 2024-04-25T08:42:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.599+0000 7f99bfe00640 1 --2- 172.21.15.46:0/2224784921 >> v2:172.21.15.82:3300/0 conn(0x7f99c015ebe0 0x7f99c0070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c5600640 1 --2- 172.21.15.46:0/2224784921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 0x7f99c00cb8e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f99c00b2d50 tx=0x7f99b801d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0089464a66e0a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99bf400640 1 -- 172.21.15.46:0/2224784921 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80a5830 con 0x7f99c006f520 2024-04-25T08:42:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99bf400640 1 -- 172.21.15.46:0/2224784921 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b80a59d0 con 0x7f99c006f520 2024-04-25T08:42:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99bf400640 1 -- 172.21.15.46:0/2224784921 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80a5da0 con 0x7f99c006f520 2024-04-25T08:42:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c7600640 1 -- 172.21.15.46:0/2224784921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 msgr2=0x7f99c00cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c7600640 1 --2- 172.21.15.46:0/2224784921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 0x7f99c00cb8e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f99c00b2d50 tx=0x7f99b801d3b0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c7600640 1 -- 172.21.15.46:0/2224784921 shutdown_connections 2024-04-25T08:42:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c7600640 1 --2- 172.21.15.46:0/2224784921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 0x7f99c00cb8e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c7600640 1 --2- 172.21.15.46:0/2224784921 >> v2:172.21.15.82:3300/0 conn(0x7f99c015ebe0 0x7f99c0070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c7600640 1 -- 172.21.15.46:0/2224784921 >> 172.21.15.46:0/2224784921 conn(0x7f99c0099a00 msgr2=0x7f99c00b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c7600640 1 -- 172.21.15.46:0/2224784921 shutdown_connections 2024-04-25T08:42:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c7600640 1 -- 172.21.15.46:0/2224784921 wait complete. 2024-04-25T08:42:47.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.603+0000 7f99c7600640 1 Processor -- start 2024-04-25T08:42:47.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 -- start start 2024-04-25T08:42:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 0x7f99c006e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99c00cd500 0x7f99c00b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c015ebe0 0x7f99c00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00cc7b0 con 0x7f99c006f520 2024-04-25T08:42:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00cc630 con 0x7f99c015ebe0 2024-04-25T08:42:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00cc930 con 0x7f99c00cd500 2024-04-25T08:42:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 0x7f99c006e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 0x7f99c006e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60536/0 (socket says 172.21.15.46:60536) 2024-04-25T08:42:47.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c4c00640 1 -- 172.21.15.46:0/3774458693 learned_addr learned my addr 172.21.15.46:0/3774458693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99bfe00640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99c00cd500 0x7f99c00b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c5600640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c015ebe0 0x7f99c00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c4c00640 1 -- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99c00cd500 msgr2=0x7f99c00b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c4c00640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99c00cd500 0x7f99c00b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c4c00640 1 -- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c015ebe0 msgr2=0x7f99c00b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c4c00640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c015ebe0 0x7f99c00b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c4c00640 1 -- 172.21.15.46:0/3774458693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c009d5a0 con 0x7f99c006f520 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c5600640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c015ebe0 0x7f99c00b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99bfe00640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99c00cd500 0x7f99c00b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c4c00640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 0x7f99c006e550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f99b400c450 tx=0x7f99b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99bcc00640 1 -- 172.21.15.46:0/3774458693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4012830 con 0x7f99c006f520 2024-04-25T08:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99bcc00640 1 -- 172.21.15.46:0/3774458693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b4017070 con 0x7f99c006f520 2024-04-25T08:42:47.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99bcc00640 1 -- 172.21.15.46:0/3774458693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4012c30 con 0x7f99c006f520 2024-04-25T08:42:47.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c009d880 con 0x7f99c006f520 2024-04-25T08:42:47.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c009de40 con 0x7f99c006f520 2024-04-25T08:42:47.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.607+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c0069010 con 0x7f99c006f520 2024-04-25T08:42:47.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.611+0000 7f99bcc00640 1 -- 172.21.15.46:0/3774458693 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99b401c460 con 0x7f99c006f520 2024-04-25T08:42:47.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.611+0000 7f99bcc00640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99a0066f60 0x7f99a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.611+0000 7f99bcc00640 1 -- 172.21.15.46:0/3774458693 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f99b40851f0 con 0x7f99c006f520 2024-04-25T08:42:47.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.611+0000 7f99bfe00640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99a0066f60 0x7f99a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.611+0000 7f99bfe00640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99a0066f60 0x7f99a0069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99c00cf660 tx=0x7f99ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.615+0000 7f99bcc00640 1 -- 172.21.15.46:0/3774458693 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99b4055e20 con 0x7f99c006f520 2024-04-25T08:42:47.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f99c00cdc40 con 0x7f99c006f520 2024-04-25T08:42:47.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99bcc00640 1 -- 172.21.15.46:0/3774458693 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f99b4055720 con 0x7f99c006f520 2024-04-25T08:42:47.768 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:42:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99a0066f60 msgr2=0x7f99a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99a0066f60 0x7f99a0069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99c00cf660 tx=0x7f99ac008040 comp rx=0 tx=0).stop 2024-04-25T08:42:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 msgr2=0x7f99c006e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 0x7f99c006e550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f99b400c450 tx=0x7f99b400c910 comp rx=0 tx=0).stop 2024-04-25T08:42:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 shutdown_connections 2024-04-25T08:42:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99a0066f60 0x7f99a0069420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c015ebe0 0x7f99c00b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99c00cd500 0x7f99c00b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 --2- 172.21.15.46:0/3774458693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c006f520 0x7f99c006e550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 >> 172.21.15.46:0/3774458693 conn(0x7f99c0099a00 msgr2=0x7f99c015dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 shutdown_connections 2024-04-25T08:42:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.775+0000 7f99c7600640 1 -- 172.21.15.46:0/3774458693 wait complete. 2024-04-25T08:42:47.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.935+0000 7fd383a00640 1 Processor -- start 2024-04-25T08:42:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.935+0000 7fd383a00640 1 -- start start 2024-04-25T08:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd383a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd37c159230 0x7fd37c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd383a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 0x7fd37c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd383a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c0cc700 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd383a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37c0cc9e0 con 0x7fd37c159230 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd383a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0cccc0 con 0x7fd37c0cdc50 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd37c159230 0x7fd37c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd37c159230 0x7fd37c15bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60546/0 (socket says 172.21.15.46:60546) 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381000640 1 -- 172.21.15.46:0/511652885 learned_addr learned my addr 172.21.15.46:0/511652885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd37be00640 1 --1- 172.21.15.46:0/511652885 >> v1:172.21.15.46:6789/0 conn(0x7fd37c0cbeb0 0x7fd37c0cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59210/0 (socket says 172.21.15.46:59210) 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381a00640 1 --2- 172.21.15.46:0/511652885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 0x7fd37c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381000640 1 -- 172.21.15.46:0/511652885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 msgr2=0x7fd37c15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381000640 1 --2- 172.21.15.46:0/511652885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 0x7fd37c15d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381000640 1 -- 172.21.15.46:0/511652885 >> v1:172.21.15.46:6789/0 conn(0x7fd37c0cbeb0 legacy=0x7fd37c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381000640 1 -- 172.21.15.46:0/511652885 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0ccfa0 con 0x7fd37c159230 2024-04-25T08:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381a00640 1 --2- 172.21.15.46:0/511652885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 0x7fd37c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.939+0000 7fd381000640 1 --2- 172.21.15.46:0/511652885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c159230 0x7fd37c15bdb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd370009870 tx=0x7fd37001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359989758fb996be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd37b400640 1 -- 172.21.15.46:0/511652885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700aa620 con 0x7fd37c159230 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd37b400640 1 -- 172.21.15.46:0/511652885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3700af070 con 0x7fd37c159230 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd37b400640 1 -- 172.21.15.46:0/511652885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700aa9e0 con 0x7fd37c159230 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 -- 172.21.15.46:0/511652885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c159230 msgr2=0x7fd37c15bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 --2- 172.21.15.46:0/511652885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c159230 0x7fd37c15bdb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd370009870 tx=0x7fd37001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 -- 172.21.15.46:0/511652885 shutdown_connections 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 --2- 172.21.15.46:0/511652885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 0x7fd37c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 --2- 172.21.15.46:0/511652885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c159230 0x7fd37c15bdb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 -- 172.21.15.46:0/511652885 >> 172.21.15.46:0/511652885 conn(0x7fd37c0b12a0 msgr2=0x7fd37c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 -- 172.21.15.46:0/511652885 shutdown_connections 2024-04-25T08:42:47.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 -- 172.21.15.46:0/511652885 wait complete. 2024-04-25T08:42:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 Processor -- start 2024-04-25T08:42:47.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 -- start start 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c0cbeb0 0x7fd37c157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 0x7fd37c13b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd37c159230 0x7fd37c138a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c15df50 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0cc4e0 con 0x7fd37c159230 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd383a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c15e0d0 con 0x7fd37c0cdc50 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd381000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c0cbeb0 0x7fd37c157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd381000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c0cbeb0 0x7fd37c157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60548/0 (socket says 172.21.15.46:60548) 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd381000640 1 -- 172.21.15.46:0/3710301865 learned_addr learned my addr 172.21.15.46:0/3710301865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd381000640 1 -- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 msgr2=0x7fd37c13b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd381000640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 0x7fd37c13b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd381000640 1 -- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd37c159230 msgr2=0x7fd37c138a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:47.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd381000640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd37c159230 0x7fd37c138a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.943+0000 7fd381000640 1 -- 172.21.15.46:0/3710301865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c13ba30 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd381000640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c0cbeb0 0x7fd37c157b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3700ac040 tx=0x7fd37001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd378c00640 1 -- 172.21.15.46:0/3710301865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700bd3f0 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd378c00640 1 -- 172.21.15.46:0/3710301865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3700bd590 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c13bd10 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd378c00640 1 -- 172.21.15.46:0/3710301865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700bd980 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c159d00 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c069080 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd378c00640 1 -- 172.21.15.46:0/3710301865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3700b33f0 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd378c00640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd35c066f60 0x7fd35c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:47.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd378c00640 1 -- 172.21.15.46:0/3710301865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd37011f850 con 0x7fd37c0cbeb0 2024-04-25T08:42:47.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.947+0000 7fd37be00640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd35c066f60 0x7fd35c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:47.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.951+0000 7fd37be00640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd35c066f60 0x7fd35c069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd368000f70 tx=0x7fd368007a10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:47.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:47.951+0000 7fd378c00640 1 -- 172.21.15.46:0/3710301865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3700c3060 con 0x7fd37c0cbeb0 2024-04-25T08:42:48.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd37c0723d0 con 0x7fd37c0cbeb0 2024-04-25T08:42:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd378c00640 1 -- 172.21.15.46:0/3710301865 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fd370016150 con 0x7fd37c0cbeb0 2024-04-25T08:42:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:48.144 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:38.824358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:42:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd35c066f60 msgr2=0x7fd35c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd35c066f60 0x7fd35c069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd368000f70 tx=0x7fd368007a10 comp rx=0 tx=0).stop 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c0cbeb0 msgr2=0x7fd37c157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c0cbeb0 0x7fd37c157b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd3700ac040 tx=0x7fd37001ace0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 shutdown_connections 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd35c066f60 0x7fd35c069420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd37c159230 0x7fd37c138a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd37c0cdc50 0x7fd37c13b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 --2- 172.21.15.46:0/3710301865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c0cbeb0 0x7fd37c157b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 >> 172.21.15.46:0/3710301865 conn(0x7fd37c0b12a0 msgr2=0x7fd37c06d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 shutdown_connections 2024-04-25T08:42:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.151+0000 7fd383a00640 1 -- 172.21.15.46:0/3710301865 wait complete. 2024-04-25T08:42:48.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.311+0000 7f38bac00640 1 Processor -- start 2024-04-25T08:42:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.311+0000 7f38bac00640 1 -- start start 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.311+0000 7f38bac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 0x7f38b406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.311+0000 7f38bac00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b40b5250 con 0x7f38b4070c90 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.311+0000 7f38bac00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b40b5530 con 0x7f38b415dbd0 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.311+0000 7f38bac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b40b5af0 con 0x7f38b4071110 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.311+0000 7f38ab400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f38b4070c90 0x7f38b406c380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33336/0 (socket says 172.21.15.46:33336) 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.311+0000 7f38ab400640 1 -- 172.21.15.46:0/1763485351 learned_addr learned my addr 172.21.15.46:0/1763485351 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b8c00640 1 --2- 172.21.15.46:0/1763485351 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 0x7f38b406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b3400640 1 -- 172.21.15.46:0/1763485351 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930069051 0 0) 0x7f38b40b5250 con 0x7f38b4070c90 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b3400640 1 -- 172.21.15.46:0/1763485351 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3898003530 con 0x7f38b4070c90 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b3400640 1 -- 172.21.15.46:0/1763485351 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644726647 0 0) 0x7f38b40b5530 con 0x7f38b415dbd0 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b8c00640 1 -- 172.21.15.46:0/1763485351 >> v1:172.21.15.82:6789/0 conn(0x7f38b415dbd0 legacy=0x7f38b40b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b8c00640 1 -- 172.21.15.46:0/1763485351 >> v1:172.21.15.46:6789/0 conn(0x7f38b4070c90 legacy=0x7f38b406c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b8c00640 1 -- 172.21.15.46:0/1763485351 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b40b5810 con 0x7f38b4071110 2024-04-25T08:42:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b8c00640 1 --2- 172.21.15.46:0/1763485351 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 0x7f38b406ca90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38b40b5c70 tx=0x7f38ac01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6cc40305be72ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b3400640 1 -- 172.21.15.46:0/1763485351 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0b2070 con 0x7f38b4071110 2024-04-25T08:42:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b3400640 1 -- 172.21.15.46:0/1763485351 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac0a5850 con 0x7f38b4071110 2024-04-25T08:42:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.315+0000 7f38b3400640 1 -- 172.21.15.46:0/1763485351 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac0a5c20 con 0x7f38b4071110 2024-04-25T08:42:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 -- 172.21.15.46:0/1763485351 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 msgr2=0x7f38b406ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 --2- 172.21.15.46:0/1763485351 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 0x7f38b406ca90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38b40b5c70 tx=0x7f38ac01c990 comp rx=0 tx=0).stop 2024-04-25T08:42:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 -- 172.21.15.46:0/1763485351 shutdown_connections 2024-04-25T08:42:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 --2- 172.21.15.46:0/1763485351 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 0x7f38b406ca90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 -- 172.21.15.46:0/1763485351 >> 172.21.15.46:0/1763485351 conn(0x7f38b406f4f0 msgr2=0x7f38b406f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 -- 172.21.15.46:0/1763485351 shutdown_connections 2024-04-25T08:42:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 -- 172.21.15.46:0/1763485351 wait complete. 2024-04-25T08:42:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 Processor -- start 2024-04-25T08:42:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 -- start start 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38b4070c90 0x7f38b41496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 0x7f38b40cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b415dbd0 0x7f38b4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b416b7d0 con 0x7f38b4070c90 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b416b650 con 0x7f38b415dbd0 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38bac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b416b950 con 0x7f38b4071110 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38b4070c90 0x7f38b41496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38b4070c90 0x7f38b41496c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39086/0 (socket says 172.21.15.46:39086) 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b3e00640 1 -- 172.21.15.46:0/2684573072 learned_addr learned my addr 172.21.15.46:0/2684573072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38ab400640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 0x7f38b40cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b8c00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b415dbd0 0x7f38b4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b3e00640 1 -- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 msgr2=0x7f38b40cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b3e00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 0x7f38b40cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b3e00640 1 -- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b415dbd0 msgr2=0x7f38b4160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b3e00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b415dbd0 0x7f38b4160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b3e00640 1 -- 172.21.15.46:0/2684573072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b40cd070 con 0x7f38b4070c90 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.319+0000 7f38b8c00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b415dbd0 0x7f38b4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38b3e00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38b4070c90 0x7f38b41496c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c450 tx=0x7f38a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38b0c00640 1 -- 172.21.15.46:0/2684573072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4017070 con 0x7f38b4070c90 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38b0c00640 1 -- 172.21.15.46:0/2684573072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a4005480 con 0x7f38b4070c90 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38b0c00640 1 -- 172.21.15.46:0/2684573072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4003840 con 0x7f38b4070c90 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b415e1c0 con 0x7f38b4070c90 2024-04-25T08:42:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b4160a80 con 0x7f38b4070c90 2024-04-25T08:42:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b415dfb0 con 0x7f38b4070c90 2024-04-25T08:42:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38b0c00640 1 -- 172.21.15.46:0/2684573072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38a401a070 con 0x7f38b4070c90 2024-04-25T08:42:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38b0c00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3890066f60 0x7f3890069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38b0c00640 1 -- 172.21.15.46:0/2684573072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f38a4084670 con 0x7f38b4070c90 2024-04-25T08:42:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.323+0000 7f38ab400640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3890066f60 0x7f3890069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.327+0000 7f38ab400640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3890066f60 0x7f3890069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f389c007fd0 tx=0x7f389c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.327+0000 7f38b0c00640 1 -- 172.21.15.46:0/2684573072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38a4054c50 con 0x7f38b4070c90 2024-04-25T08:42:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38b4069080 con 0x7f38b4070c90 2024-04-25T08:42:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38b0c00640 1 -- 172.21.15.46:0/2684573072 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f38a4058cf0 con 0x7f38b4070c90 2024-04-25T08:42:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:48.484 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:48.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3890066f60 msgr2=0x7f3890069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3890066f60 0x7f3890069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f389c007fd0 tx=0x7f389c009040 comp rx=0 tx=0).stop 2024-04-25T08:42:48.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38b4070c90 msgr2=0x7f38b41496c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38b4070c90 0x7f38b41496c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38a400c450 tx=0x7f38a400c910 comp rx=0 tx=0).stop 2024-04-25T08:42:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 shutdown_connections 2024-04-25T08:42:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3890066f60 0x7f3890069420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38b415dbd0 0x7f38b4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38b4071110 0x7f38b40cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 --2- 172.21.15.46:0/2684573072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38b4070c90 0x7f38b41496c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 >> 172.21.15.46:0/2684573072 conn(0x7f38b406f4f0 msgr2=0x7f38b415c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.491+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 shutdown_connections 2024-04-25T08:42:48.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.495+0000 7f38bac00640 1 -- 172.21.15.46:0/2684573072 wait complete. 2024-04-25T08:42:48.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:42:48.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:42:48.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.655+0000 7fa69a600640 1 Processor -- start 2024-04-25T08:42:48.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.655+0000 7fa69a600640 1 -- start start 2024-04-25T08:42:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa69a600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6940cf280 0x7fa69409a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa69a600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6940ae210 con 0x7fa69406ff40 2024-04-25T08:42:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa69a600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69409ad20 con 0x7fa694099a00 2024-04-25T08:42:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa69a600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69406f210 con 0x7fa6940cf280 2024-04-25T08:42:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa69406ff40 0x7fa6940703f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33342/0 (socket says 172.21.15.46:33342) 2024-04-25T08:42:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699c00640 1 -- 172.21.15.46:0/689896536 learned_addr learned my addr 172.21.15.46:0/689896536 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693e00640 1 --2- 172.21.15.46:0/689896536 >> v2:172.21.15.82:3300/0 conn(0x7fa6940cf280 0x7fa69409a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144115917 0 0) 0x7fa68c0014a0 con 0x7fa69406ff40 2024-04-25T08:42:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa67c003530 con 0x7fa69406ff40 2024-04-25T08:42:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3519979245 0 0) 0x7fa67c003530 con 0x7fa69406ff40 2024-04-25T08:42:48.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 >> v2:172.21.15.82:3300/0 conn(0x7fa6940cf280 msgr2=0x7fa69409a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 --2- 172.21.15.46:0/689896536 >> v2:172.21.15.82:3300/0 conn(0x7fa6940cf280 0x7fa69409a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 >> v1:172.21.15.82:6789/0 conn(0x7fa694099a00 legacy=0x7fa694099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69406f460 con 0x7fa69406ff40 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fa68c001bd0 con 0x7fa69406ff40 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 >> v1:172.21.15.46:6789/0 conn(0x7fa69406ff40 legacy=0x7fa6940703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa67c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c004810 0x7fa67c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa67c0051d0 0x7fa67c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69406f210 con 0x7fa67c0039c0 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c00c070 con 0x7fa67c004810 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c00c270 con 0x7fa67c0051d0 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699200640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa67c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa693e00640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c004810 0x7fa67c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699c00640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa67c0051d0 0x7fa67c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699200640 1 -- 172.21.15.46:0/689896536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa67c0051d0 msgr2=0x7fa67c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699200640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa67c0051d0 0x7fa67c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699200640 1 -- 172.21.15.46:0/689896536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c004810 msgr2=0x7fa67c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699200640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67c004810 0x7fa67c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699200640 1 -- 172.21.15.46:0/689896536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c003530 con 0x7fa67c0039c0 2024-04-25T08:42:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.659+0000 7fa699c00640 1 -- 172.21.15.46:0/689896536 reap_dead start 2024-04-25T08:42:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa699200640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa67c003de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa688001780 tx=0x7fa68801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdb37a53ada31c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6880013a0 con 0x7fa67c0039c0 2024-04-25T08:42:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6880052f0 con 0x7fa67c0039c0 2024-04-25T08:42:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa693400640 1 -- 172.21.15.46:0/689896536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688003840 con 0x7fa67c0039c0 2024-04-25T08:42:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa69a600640 1 -- 172.21.15.46:0/689896536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 msgr2=0x7fa67c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa69a600640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa67c003de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa688001780 tx=0x7fa68801c3d0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa69a600640 1 -- 172.21.15.46:0/689896536 shutdown_connections 2024-04-25T08:42:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa69a600640 1 --2- 172.21.15.46:0/689896536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa67c003de0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa69a600640 1 -- 172.21.15.46:0/689896536 >> 172.21.15.46:0/689896536 conn(0x7fa6940b2f90 msgr2=0x7fa6940b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:48.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa69a600640 1 -- 172.21.15.46:0/689896536 shutdown_connections 2024-04-25T08:42:48.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa69a600640 1 -- 172.21.15.46:0/689896536 wait complete. 2024-04-25T08:42:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa69a600640 1 Processor -- start 2024-04-25T08:42:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.663+0000 7fa69a600640 1 -- start start 2024-04-25T08:42:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa69a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa6941fa810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa69a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6940c3ce0 0x7fa69406cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa69a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6940cd930 0x7fa6941526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa69a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69415bc80 con 0x7fa67c0039c0 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa69a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940cd3c0 con 0x7fa6940c3ce0 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa69a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69415be00 con 0x7fa6940cd930 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa6941fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa6941fa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39098/0 (socket says 172.21.15.46:39098) 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699200640 1 -- 172.21.15.46:0/3457259734 learned_addr learned my addr 172.21.15.46:0/3457259734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa693e00640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6940c3ce0 0x7fa69406cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699c00640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6940cd930 0x7fa6941526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699200640 1 -- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6940cd930 msgr2=0x7fa6941526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699200640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6940cd930 0x7fa6941526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699200640 1 -- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6940c3ce0 msgr2=0x7fa69406cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699200640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6940c3ce0 0x7fa69406cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699200640 1 -- 172.21.15.46:0/3457259734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69406d710 con 0x7fa67c0039c0 2024-04-25T08:42:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa699200640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa6941fa810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa68801c200 tx=0x7fa67c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa690c00640 1 -- 172.21.15.46:0/3457259734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688007a40 con 0x7fa67c0039c0 2024-04-25T08:42:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa690c00640 1 -- 172.21.15.46:0/3457259734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6880ba070 con 0x7fa67c0039c0 2024-04-25T08:42:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa690c00640 1 -- 172.21.15.46:0/3457259734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6880013a0 con 0x7fa67c0039c0 2024-04-25T08:42:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69406d9f0 con 0x7fa67c0039c0 2024-04-25T08:42:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6941fa2f0 con 0x7fa67c0039c0 2024-04-25T08:42:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa694069010 con 0x7fa67c0039c0 2024-04-25T08:42:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.667+0000 7fa690c00640 1 -- 172.21.15.46:0/3457259734 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6880052f0 con 0x7fa67c0039c0 2024-04-25T08:42:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.671+0000 7fa690c00640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.671+0000 7fa690c00640 1 -- 172.21.15.46:0/3457259734 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa68811f710 con 0x7fa67c0039c0 2024-04-25T08:42:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.671+0000 7fa693e00640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.671+0000 7fa693e00640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa6940b2b60 tx=0x7fa680009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.675+0000 7fa690c00640 1 -- 172.21.15.46:0/3457259734 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa688016110 con 0x7fa67c0039c0 2024-04-25T08:42:48.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.835+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6940cf860 con 0x7fa67c0039c0 2024-04-25T08:42:48.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.835+0000 7fa690c00640 1 -- 172.21.15.46:0/3457259734 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6880f03c0 con 0x7fa67c0039c0 2024-04-25T08:42:48.828 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 msgr2=0x7fa674069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa6940b2b60 tx=0x7fa680009190 comp rx=0 tx=0).stop 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 msgr2=0x7fa6941fa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa6941fa810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa68801c200 tx=0x7fa67c005840 comp rx=0 tx=0).stop 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 shutdown_connections 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa674066f60 0x7fa674069420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6940cd930 0x7fa6941526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6940c3ce0 0x7fa69406cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 --2- 172.21.15.46:0/3457259734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa67c0039c0 0x7fa6941fa810 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 >> 172.21.15.46:0/3457259734 conn(0x7fa6940b2f90 msgr2=0x7fa6940cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 shutdown_connections 2024-04-25T08:42:48.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:48.839+0000 7fa69a600640 1 -- 172.21.15.46:0/3457259734 wait complete. 2024-04-25T08:42:48.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:42:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68aec00640 1 Processor -- start 2024-04-25T08:42:48.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68aec00640 1 -- start start 2024-04-25T08:42:48.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68aec00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f68a80cf890 0x7f68a80b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68aec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68a80cf340 0x7f68a815bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:48.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68aec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a80b2d50 con 0x7f68a80cc180 2024-04-25T08:42:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68aec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a815f2f0 con 0x7f68a80cf340 2024-04-25T08:42:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68aec00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a815f5d0 con 0x7f68a80cf890 2024-04-25T08:42:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68acc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68a80cf340 0x7f68a815bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f689f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f68a80cc180 0x7f68a80cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33354/0 (socket says 172.21.15.46:33354) 2024-04-25T08:42:48.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f689f400640 1 -- 172.21.15.46:0/170402929 learned_addr learned my addr 172.21.15.46:0/170402929 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:48.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68a7e00640 1 --2- 172.21.15.46:0/170402929 >> v2:172.21.15.82:3300/0 conn(0x7f68a80cf890 0x7f68a80b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:48.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68acc00640 1 -- 172.21.15.46:0/170402929 >> v2:172.21.15.82:3300/0 conn(0x7f68a80cf890 msgr2=0x7f68a80b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:48.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68acc00640 1 --2- 172.21.15.46:0/170402929 >> v2:172.21.15.82:3300/0 conn(0x7f68a80cf890 0x7f68a80b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:48.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68acc00640 1 -- 172.21.15.46:0/170402929 >> v1:172.21.15.46:6789/0 conn(0x7f68a80cc180 legacy=0x7f68a80cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:48.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.003+0000 7f68acc00640 1 -- 172.21.15.46:0/170402929 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a815f8b0 con 0x7f68a80cf340 2024-04-25T08:42:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68acc00640 1 --2- 172.21.15.46:0/170402929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 0x7f68a815bb70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f68a806b8d0 tx=0x7f68a001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83748a88a2963757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68a7400640 1 -- 172.21.15.46:0/170402929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a00b2070 con 0x7f68a80cf340 2024-04-25T08:42:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68a7400640 1 -- 172.21.15.46:0/170402929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a00a58e0 con 0x7f68a80cf340 2024-04-25T08:42:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68a7400640 1 -- 172.21.15.46:0/170402929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a00a5cb0 con 0x7f68a80cf340 2024-04-25T08:42:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68aec00640 1 -- 172.21.15.46:0/170402929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 msgr2=0x7f68a815bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68aec00640 1 --2- 172.21.15.46:0/170402929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 0x7f68a815bb70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f68a806b8d0 tx=0x7f68a001c930 comp rx=0 tx=0).stop 2024-04-25T08:42:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68aec00640 1 -- 172.21.15.46:0/170402929 shutdown_connections 2024-04-25T08:42:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68aec00640 1 --2- 172.21.15.46:0/170402929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 0x7f68a815bb70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68aec00640 1 --2- 172.21.15.46:0/170402929 >> v2:172.21.15.82:3300/0 conn(0x7f68a80cf890 0x7f68a80b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68aec00640 1 -- 172.21.15.46:0/170402929 >> 172.21.15.46:0/170402929 conn(0x7f68a8099a00 msgr2=0x7f68a80b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68aec00640 1 -- 172.21.15.46:0/170402929 shutdown_connections 2024-04-25T08:42:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.007+0000 7f68aec00640 1 -- 172.21.15.46:0/170402929 wait complete. 2024-04-25T08:42:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 Processor -- start 2024-04-25T08:42:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 -- start start 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68a80cc180 0x7f68a8157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 0x7f68a80b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68a80cf890 0x7f68a806f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a815cc50 con 0x7f68a80cf340 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a815cad0 con 0x7f68a80cf890 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a8168690 con 0x7f68a80cc180 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f689f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 0x7f68a80b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f689f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 0x7f68a80b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39118/0 (socket says 172.21.15.46:39118) 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f689f400640 1 -- 172.21.15.46:0/1103740345 learned_addr learned my addr 172.21.15.46:0/1103740345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68a7e00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68a80cc180 0x7f68a8157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68acc00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68a80cf890 0x7f68a806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f689f400640 1 -- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68a80cc180 msgr2=0x7f68a8157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f689f400640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68a80cc180 0x7f68a8157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f689f400640 1 -- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68a80cf890 msgr2=0x7f68a806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f689f400640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68a80cf890 0x7f68a806f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f689f400640 1 -- 172.21.15.46:0/1103740345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a80afe00 con 0x7f68a80cf340 2024-04-25T08:42:49.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f689f400640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 0x7f68a80b15a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f689000c400 tx=0x7f689000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68a4c00640 1 -- 172.21.15.46:0/1103740345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890017070 con 0x7f68a80cf340 2024-04-25T08:42:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68a4c00640 1 -- 172.21.15.46:0/1103740345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6890005cf0 con 0x7f68a80cf340 2024-04-25T08:42:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a80b00e0 con 0x7f68a80cf340 2024-04-25T08:42:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68a4c00640 1 -- 172.21.15.46:0/1103740345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890007310 con 0x7f68a80cf340 2024-04-25T08:42:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a80b06a0 con 0x7f68a80cf340 2024-04-25T08:42:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.011+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a8157af0 con 0x7f68a80cf340 2024-04-25T08:42:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.015+0000 7f68a4c00640 1 -- 172.21.15.46:0/1103740345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f689001a070 con 0x7f68a80cf340 2024-04-25T08:42:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.015+0000 7f68a4c00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6884066f60 0x7f6884069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.015+0000 7f68a4c00640 1 -- 172.21.15.46:0/1103740345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6890084740 con 0x7f68a80cf340 2024-04-25T08:42:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.015+0000 7f68a7e00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6884066f60 0x7f6884069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.015+0000 7f68a7e00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6884066f60 0x7f6884069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f68a81f9ee0 tx=0x7f6898008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:49.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.019+0000 7f68a4c00640 1 -- 172.21.15.46:0/1103740345 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68a80b2d50 con 0x7f68a80cf340 2024-04-25T08:42:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.179+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68a80b2fd0 con 0x7f68a80cf340 2024-04-25T08:42:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.179+0000 7f68a4c00640 1 -- 172.21.15.46:0/1103740345 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6890054d20 con 0x7f68a80cf340 2024-04-25T08:42:49.174 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:42:49.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6884066f60 msgr2=0x7f6884069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6884066f60 0x7f6884069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f68a81f9ee0 tx=0x7f6898008040 comp rx=0 tx=0).stop 2024-04-25T08:42:49.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 msgr2=0x7f68a80b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 0x7f68a80b15a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f689000c400 tx=0x7f689000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 shutdown_connections 2024-04-25T08:42:49.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6884066f60 0x7f6884069420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68a80cf890 0x7f68a806f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68a80cf340 0x7f68a80b15a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 --2- 172.21.15.46:0/1103740345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68a80cc180 0x7f68a8157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 >> 172.21.15.46:0/1103740345 conn(0x7f68a8099a00 msgr2=0x7f68a80cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:49.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 shutdown_connections 2024-04-25T08:42:49.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.183+0000 7f68aec00640 1 -- 172.21.15.46:0/1103740345 wait complete. 2024-04-25T08:42:49.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:42:49.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f830a000640 1 Processor -- start 2024-04-25T08:42:49.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f830a000640 1 -- start start 2024-04-25T08:42:49.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f830a000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f830406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f830a000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f830406dc70 0x7f83040cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f830a000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83040cd930 con 0x7f83040cf1b0 2024-04-25T08:42:49.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f830a000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83040cdbd0 con 0x7f830406dc70 2024-04-25T08:42:49.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f830a000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83040cdeb0 con 0x7f8304145310 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f82faa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f830406dc70 0x7f83040cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f8303400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f830406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f8303400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f830406e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36924/0 (socket says 172.21.15.46:36924) 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f82faa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f830406dc70 0x7f83040cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:42864/0 (socket says 172.21.15.46:42864) 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f8303400640 1 -- 172.21.15.46:0/996742225 learned_addr learned my addr 172.21.15.46:0/996742225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f8303400640 1 -- 172.21.15.46:0/996742225 >> v2:172.21.15.82:3300/0 conn(0x7f830406dc70 msgr2=0x7f83040cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f8303400640 1 --2- 172.21.15.46:0/996742225 >> v2:172.21.15.82:3300/0 conn(0x7f830406dc70 0x7f83040cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f8303400640 1 -- 172.21.15.46:0/996742225 >> v1:172.21.15.82:6789/0 conn(0x7f83040cf1b0 legacy=0x7f830415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f8303400640 1 -- 172.21.15.46:0/996742225 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83040ce190 con 0x7f8304145310 2024-04-25T08:42:49.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.347+0000 7f82faa00640 1 --2- 172.21.15.46:0/996742225 >> v2:172.21.15.82:3300/0 conn(0x7f830406dc70 0x7f83040cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f8303400640 1 --2- 172.21.15.46:0/996742225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f830406e0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009870 tx=0x7f82f401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d44eb2bf243ad279 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f8302a00640 1 -- 172.21.15.46:0/996742225 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40a93f0 con 0x7f8304145310 2024-04-25T08:42:49.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f8302a00640 1 -- 172.21.15.46:0/996742225 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40a9590 con 0x7f8304145310 2024-04-25T08:42:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f8302a00640 1 -- 172.21.15.46:0/996742225 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40a9960 con 0x7f8304145310 2024-04-25T08:42:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f830a000640 1 -- 172.21.15.46:0/996742225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 msgr2=0x7f830406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f830a000640 1 --2- 172.21.15.46:0/996742225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f830406e0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009870 tx=0x7f82f401c530 comp rx=0 tx=0).stop 2024-04-25T08:42:49.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f830a000640 1 -- 172.21.15.46:0/996742225 shutdown_connections 2024-04-25T08:42:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f830a000640 1 --2- 172.21.15.46:0/996742225 >> v2:172.21.15.82:3300/0 conn(0x7f830406dc70 0x7f83040cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f830a000640 1 --2- 172.21.15.46:0/996742225 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f830406e0d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f830a000640 1 -- 172.21.15.46:0/996742225 >> 172.21.15.46:0/996742225 conn(0x7f830406ac10 msgr2=0x7f830406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f830a000640 1 -- 172.21.15.46:0/996742225 shutdown_connections 2024-04-25T08:42:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.351+0000 7f830a000640 1 -- 172.21.15.46:0/996742225 wait complete. 2024-04-25T08:42:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 Processor -- start 2024-04-25T08:42:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 -- start start 2024-04-25T08:42:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83040cf1b0 0x7f83041444d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f8304159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830415a2c0 0x7f8304158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830415d920 con 0x7f83040cf1b0 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830415d7a0 con 0x7f830415a2c0 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8304166500 con 0x7f8304145310 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83040cf1b0 0x7f83041444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83040cf1b0 0x7f83041444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39132/0 (socket says 172.21.15.46:39132) 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303400640 1 -- 172.21.15.46:0/1572901794 learned_addr learned my addr 172.21.15.46:0/1572901794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f82faa00640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f8304159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303e00640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830415a2c0 0x7f8304158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303400640 1 -- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 msgr2=0x7f8304159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303400640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f8304159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303400640 1 -- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830415a2c0 msgr2=0x7f8304158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303400640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830415a2c0 0x7f8304158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303400640 1 -- 172.21.15.46:0/1572901794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83040a9fa0 con 0x7f83040cf1b0 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f8303400640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83040cf1b0 0x7f83041444d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f82f401abe0 tx=0x7f82f40192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f82fbe00640 1 -- 172.21.15.46:0/1572901794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40a93f0 con 0x7f83040cf1b0 2024-04-25T08:42:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f82fbe00640 1 -- 172.21.15.46:0/1572901794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40b1820 con 0x7f83040cf1b0 2024-04-25T08:42:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83040aa280 con 0x7f83040cf1b0 2024-04-25T08:42:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f82fbe00640 1 -- 172.21.15.46:0/1572901794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40b1c10 con 0x7f83040cf1b0 2024-04-25T08:42:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.355+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83040ad290 con 0x7f83040cf1b0 2024-04-25T08:42:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.359+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830415a6d0 con 0x7f83040cf1b0 2024-04-25T08:42:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.359+0000 7f82fbe00640 1 -- 172.21.15.46:0/1572901794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82f40c43f0 con 0x7f83040cf1b0 2024-04-25T08:42:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.359+0000 7f82fbe00640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82e0066f90 0x7f82e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.359+0000 7f82fbe00640 1 -- 172.21.15.46:0/1572901794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f82f411fa10 con 0x7f83040cf1b0 2024-04-25T08:42:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.359+0000 7f82faa00640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82e0066f90 0x7f82e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.359+0000 7f82faa00640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82e0066f90 0x7f82e0069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f83041591c0 tx=0x7f82ec00db60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.363+0000 7f82fbe00640 1 -- 172.21.15.46:0/1572901794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f830415a6d0 con 0x7f83040cf1b0 2024-04-25T08:42:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.527+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8304069010 con 0x7f83040cf1b0 2024-04-25T08:42:49.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.527+0000 7f82fbe00640 1 -- 172.21.15.46:0/1572901794 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f82f40ad020 con 0x7f83040cf1b0 2024-04-25T08:42:49.522 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82e0066f90 msgr2=0x7f82e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82e0066f90 0x7f82e0069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f83041591c0 tx=0x7f82ec00db60 comp rx=0 tx=0).stop 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83040cf1b0 msgr2=0x7f83041444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83040cf1b0 0x7f83041444d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f82f401abe0 tx=0x7f82f40192d0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 shutdown_connections 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82e0066f90 0x7f82e0069450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830415a2c0 0x7f8304158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8304145310 0x7f8304159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 --2- 172.21.15.46:0/1572901794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83040cf1b0 0x7f83041444d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 >> 172.21.15.46:0/1572901794 conn(0x7f830406ac10 msgr2=0x7f83040b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 shutdown_connections 2024-04-25T08:42:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.531+0000 7f830a000640 1 -- 172.21.15.46:0/1572901794 wait complete. 2024-04-25T08:42:49.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:42:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.691+0000 7fcfd2000640 1 Processor -- start 2024-04-25T08:42:49.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.691+0000 7fcfd2000640 1 -- start start 2024-04-25T08:42:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfd2000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcfcc145310 0x7fcfcc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfd2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfcc15d090 0x7fcfcc15d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfd2000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc0cd930 con 0x7fcfcc06dc70 2024-04-25T08:42:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfd2000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc0cdbd0 con 0x7fcfcc145310 2024-04-25T08:42:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfd2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0cdeb0 con 0x7fcfcc15d090 2024-04-25T08:42:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcaa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcfcc06dc70 0x7fcfcc0cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33360/0 (socket says 172.21.15.46:33360) 2024-04-25T08:42:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcaa00640 1 -- 172.21.15.46:0/2919884134 learned_addr learned my addr 172.21.15.46:0/2919884134 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:49.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422608552 0 0) 0x7fcfcc0cd930 con 0x7fcfcc06dc70 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcbe00640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfcc15d090 0x7fcfcc15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcb400640 1 --2- 172.21.15.46:0/2919884134 >> v2:172.21.15.82:3300/0 conn(0x7fcfcc145310 0x7fcfcc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb0003530 con 0x7fcfcc06dc70 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2685392028 0 0) 0x7fcfb0003530 con 0x7fcfcc06dc70 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfcc15d090 msgr2=0x7fcfcc15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfcc15d090 0x7fcfcc15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 >> v2:172.21.15.82:3300/0 conn(0x7fcfcc145310 msgr2=0x7fcfcc06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 --2- 172.21.15.46:0/2919884134 >> v2:172.21.15.82:3300/0 conn(0x7fcfcc145310 0x7fcfcc06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0ce190 con 0x7fcfcc06dc70 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fcfb8001680 con 0x7fcfcc06dc70 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 >> v1:172.21.15.46:6789/0 conn(0x7fcfcc06dc70 legacy=0x7fcfcc0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfb0003a30 0x7fcfb0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfcc15d090 0x7fcfb0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfb0004d50 0x7fcfb00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0cdbd0 con 0x7fcfb0003a30 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0cdeb0 con 0x7fcfb0004d50 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb000b9d0 con 0x7fcfcc15d090 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcb400640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfb0003a30 0x7fcfb0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcaa00640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfcc15d090 0x7fcfb0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcbe00640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfb0004d50 0x7fcfb00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcb400640 1 -- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfcc15d090 msgr2=0x7fcfb0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcb400640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfcc15d090 0x7fcfb0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcb400640 1 -- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfb0004d50 msgr2=0x7fcfb00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcb400640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfb0004d50 0x7fcfb00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.695+0000 7fcfcb400640 1 -- 172.21.15.46:0/2919884134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb0003530 con 0x7fcfb0003a30 2024-04-25T08:42:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfcb400640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfb0003a30 0x7fcfb0003e50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009ae0 tx=0x7fcfbc01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26f446fe4318a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0b2070 con 0x7fcfb0003a30 2024-04-25T08:42:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfbc0a5650 con 0x7fcfb0003a30 2024-04-25T08:42:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfca000640 1 -- 172.21.15.46:0/2919884134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0096b0 con 0x7fcfb0003a30 2024-04-25T08:42:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfd2000640 1 -- 172.21.15.46:0/2919884134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfb0003a30 msgr2=0x7fcfb0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfd2000640 1 --2- 172.21.15.46:0/2919884134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfb0003a30 0x7fcfb0003e50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009ae0 tx=0x7fcfbc01ca80 comp rx=0 tx=0).stop 2024-04-25T08:42:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfcbe00640 1 -- 172.21.15.46:0/2919884134 reap_dead start 2024-04-25T08:42:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfd2000640 1 -- 172.21.15.46:0/2919884134 shutdown_connections 2024-04-25T08:42:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfd2000640 1 -- 172.21.15.46:0/2919884134 >> 172.21.15.46:0/2919884134 conn(0x7fcfcc06ac10 msgr2=0x7fcfcc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfd2000640 1 -- 172.21.15.46:0/2919884134 shutdown_connections 2024-04-25T08:42:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.699+0000 7fcfd2000640 1 -- 172.21.15.46:0/2919884134 wait complete. 2024-04-25T08:42:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 Processor -- start 2024-04-25T08:42:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 -- start start 2024-04-25T08:42:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfb0003a30 0x7fcfcc153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfcc138bb0 0x7fcfcc153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfcc1406f0 0x7fcfcc140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc15d8b0 con 0x7fcfcc1406f0 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0cf380 con 0x7fcfcc138bb0 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc166500 con 0x7fcfb0003a30 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfcc1406f0 0x7fcfcc140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfcc1406f0 0x7fcfcc140b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39154/0 (socket says 172.21.15.46:39154) 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcbe00640 1 -- 172.21.15.46:0/659560419 learned_addr learned my addr 172.21.15.46:0/659560419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcb400640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfb0003a30 0x7fcfcc153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcaa00640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfcc138bb0 0x7fcfcc153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcbe00640 1 -- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfb0003a30 msgr2=0x7fcfcc153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcbe00640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfb0003a30 0x7fcfcc153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcbe00640 1 -- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfcc138bb0 msgr2=0x7fcfcc153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcbe00640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfcc138bb0 0x7fcfcc153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcbe00640 1 -- 172.21.15.46:0/659560419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc141420 con 0x7fcfcc1406f0 2024-04-25T08:42:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcaa00640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfcc138bb0 0x7fcfcc153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfcbe00640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfcc1406f0 0x7fcfcc140b70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0685b0 tx=0x7fcfc400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfc3400640 1 -- 172.21.15.46:0/659560419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400fdb0 con 0x7fcfcc1406f0 2024-04-25T08:42:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfc3400640 1 -- 172.21.15.46:0/659560419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc401d070 con 0x7fcfcc1406f0 2024-04-25T08:42:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfc3400640 1 -- 172.21.15.46:0/659560419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4015670 con 0x7fcfcc1406f0 2024-04-25T08:42:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc141700 con 0x7fcfcc1406f0 2024-04-25T08:42:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.703+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc141cc0 con 0x7fcfcc1406f0 2024-04-25T08:42:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.707+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc069010 con 0x7fcfcc1406f0 2024-04-25T08:42:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.707+0000 7fcfc3400640 1 -- 172.21.15.46:0/659560419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfc4015810 con 0x7fcfcc1406f0 2024-04-25T08:42:49.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.707+0000 7fcfc3400640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcfac066f90 0x7fcfac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:49.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.707+0000 7fcfc3400640 1 -- 172.21.15.46:0/659560419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcfc4085c40 con 0x7fcfcc1406f0 2024-04-25T08:42:49.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.707+0000 7fcfcb400640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcfac066f90 0x7fcfac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:49.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.707+0000 7fcfcb400640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcfac066f90 0x7fcfac069450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0af000 tx=0x7fcfbc0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:49.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.711+0000 7fcfc3400640 1 -- 172.21.15.46:0/659560419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfc4010040 con 0x7fcfcc1406f0 2024-04-25T08:42:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.871+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcfcc073bd0 con 0x7fcfcc1406f0 2024-04-25T08:42:49.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfc3400640 1 -- 172.21.15.46:0/659560419 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcfc40850a0 con 0x7fcfcc1406f0 2024-04-25T08:42:49.867 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:42:49.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcfac066f90 msgr2=0x7fcfac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcfac066f90 0x7fcfac069450 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0af000 tx=0x7fcfbc0046d0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfcc1406f0 msgr2=0x7fcfcc140b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfcc1406f0 0x7fcfcc140b70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0685b0 tx=0x7fcfc400f580 comp rx=0 tx=0).stop 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 shutdown_connections 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcfac066f90 0x7fcfac069450 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfcc1406f0 0x7fcfcc140b70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfcc138bb0 0x7fcfcc153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 --2- 172.21.15.46:0/659560419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcfb0003a30 0x7fcfcc153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 >> 172.21.15.46:0/659560419 conn(0x7fcfcc06ac10 msgr2=0x7fcfcc158b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 shutdown_connections 2024-04-25T08:42:49.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:49.875+0000 7fcfd2000640 1 -- 172.21.15.46:0/659560419 wait complete. 2024-04-25T08:42:49.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e29000640 1 Processor -- start 2024-04-25T08:42:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e29000640 1 -- start start 2024-04-25T08:42:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e29000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2e24099a00 0x7f2e24099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e29000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240ccde0 0x7f2e240cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e29000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2409a320 con 0x7f2e240b2240 2024-04-25T08:42:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e29000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2406db90 con 0x7f2e24099a00 2024-04-25T08:42:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e29000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2406bdc0 con 0x7f2e240ccde0 2024-04-25T08:42:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e22800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2e240b2240 0x7f2e240b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33372/0 (socket says 172.21.15.46:33372) 2024-04-25T08:42:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e22800640 1 -- 172.21.15.46:0/3041513914 learned_addr learned my addr 172.21.15.46:0/3041513914 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e19400640 1 --2- 172.21.15.46:0/3041513914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240ccde0 0x7f2e240cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e21e00640 1 --2- 172.21.15.46:0/3041513914 >> v2:172.21.15.82:3300/0 conn(0x7f2e24099a00 0x7f2e24099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e21400640 1 -- 172.21.15.46:0/3041513914 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789239550 0 0) 0x7f2e2409a320 con 0x7f2e240b2240 2024-04-25T08:42:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e19400640 1 -- 172.21.15.46:0/3041513914 >> v2:172.21.15.82:3300/0 conn(0x7f2e24099a00 msgr2=0x7f2e24099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e19400640 1 --2- 172.21.15.46:0/3041513914 >> v2:172.21.15.82:3300/0 conn(0x7f2e24099a00 0x7f2e24099de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e19400640 1 -- 172.21.15.46:0/3041513914 >> v1:172.21.15.46:6789/0 conn(0x7f2e240b2240 legacy=0x7f2e240b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e19400640 1 -- 172.21.15.46:0/3041513914 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2406c0a0 con 0x7f2e240ccde0 2024-04-25T08:42:50.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.035+0000 7f2e21e00640 1 --2- 172.21.15.46:0/3041513914 >> v2:172.21.15.82:3300/0 conn(0x7f2e24099a00 0x7f2e24099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e19400640 1 --2- 172.21.15.46:0/3041513914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240ccde0 0x7f2e240cd260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009d90 tx=0x7f2e0c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5f7606a8fdb15b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e21400640 1 -- 172.21.15.46:0/3041513914 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0b2070 con 0x7f2e240ccde0 2024-04-25T08:42:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e21400640 1 -- 172.21.15.46:0/3041513914 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c0a5970 con 0x7f2e240ccde0 2024-04-25T08:42:50.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e21400640 1 -- 172.21.15.46:0/3041513914 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0a5d40 con 0x7f2e240ccde0 2024-04-25T08:42:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e29000640 1 -- 172.21.15.46:0/3041513914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240ccde0 msgr2=0x7f2e240cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e29000640 1 --2- 172.21.15.46:0/3041513914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240ccde0 0x7f2e240cd260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009d90 tx=0x7f2e0c01c8b0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e29000640 1 -- 172.21.15.46:0/3041513914 shutdown_connections 2024-04-25T08:42:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e29000640 1 --2- 172.21.15.46:0/3041513914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240ccde0 0x7f2e240cd260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e29000640 1 --2- 172.21.15.46:0/3041513914 >> v2:172.21.15.82:3300/0 conn(0x7f2e24099a00 0x7f2e24099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e29000640 1 -- 172.21.15.46:0/3041513914 >> 172.21.15.46:0/3041513914 conn(0x7f2e240b3120 msgr2=0x7f2e240b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e29000640 1 -- 172.21.15.46:0/3041513914 shutdown_connections 2024-04-25T08:42:50.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.039+0000 7f2e29000640 1 -- 172.21.15.46:0/3041513914 wait complete. 2024-04-25T08:42:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 Processor -- start 2024-04-25T08:42:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 -- start start 2024-04-25T08:42:50.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:42:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e24099a00 0x7f2e240b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240b2240 0x7f2e24158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccde0 0x7f2e24156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240cd7a0 con 0x7f2e24099a00 2024-04-25T08:42:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240cdb80 con 0x7f2e240ccde0 2024-04-25T08:42:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240cd920 con 0x7f2e240b2240 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e21e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e24099a00 0x7f2e240b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e21e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e24099a00 0x7f2e240b6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39170/0 (socket says 172.21.15.46:39170) 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e21e00640 1 -- 172.21.15.46:0/2544299254 learned_addr learned my addr 172.21.15.46:0/2544299254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e19400640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240b2240 0x7f2e24158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e21e00640 1 -- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240b2240 msgr2=0x7f2e24158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e22800640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccde0 0x7f2e24156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e21e00640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240b2240 0x7f2e24158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e21e00640 1 -- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccde0 msgr2=0x7f2e24156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e21e00640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccde0 0x7f2e24156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e21e00640 1 -- 172.21.15.46:0/2544299254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240b7000 con 0x7f2e24099a00 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e19400640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240b2240 0x7f2e24158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e22800640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccde0 0x7f2e24156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e21e00640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e24099a00 0x7f2e240b6460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400e980 tx=0x7f2e1400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e1aa00640 1 -- 172.21.15.46:0/2544299254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14017070 con 0x7f2e24099a00 2024-04-25T08:42:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e1aa00640 1 -- 172.21.15.46:0/2544299254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1400ca00 con 0x7f2e24099a00 2024-04-25T08:42:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e240b6a40 con 0x7f2e24099a00 2024-04-25T08:42:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e1aa00640 1 -- 172.21.15.46:0/2544299254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400cdd0 con 0x7f2e24099a00 2024-04-25T08:42:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.043+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e240c6570 con 0x7f2e24099a00 2024-04-25T08:42:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.047+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e24069080 con 0x7f2e24099a00 2024-04-25T08:42:50.042 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:42:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.047+0000 7f2e1aa00640 1 -- 172.21.15.46:0/2544299254 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e1401a070 con 0x7f2e24099a00 2024-04-25T08:42:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.047+0000 7f2e1aa00640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e00066f90 0x7f2e00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.047+0000 7f2e1aa00640 1 -- 172.21.15.46:0/2544299254 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2e140852a0 con 0x7f2e24099a00 2024-04-25T08:42:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.047+0000 7f2e19400640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e00066f90 0x7f2e00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.047+0000 7f2e19400640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e00066f90 0x7f2e00069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0af000 tx=0x7f2e0c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.051+0000 7f2e1aa00640 1 -- 172.21.15.46:0/2544299254 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e14055ee0 con 0x7f2e24099a00 2024-04-25T08:42:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e240af960 con 0x7f2e24099a00 2024-04-25T08:42:50.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e1aa00640 1 -- 172.21.15.46:0/2544299254 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f2e140557e0 con 0x7f2e24099a00 2024-04-25T08:42:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:50.243 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:38.824358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:42:50.243 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-25T08:42:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e00066f90 msgr2=0x7f2e00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e00066f90 0x7f2e00069450 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0af000 tx=0x7f2e0c0a7070 comp rx=0 tx=0).stop 2024-04-25T08:42:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e24099a00 msgr2=0x7f2e240b6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e24099a00 0x7f2e240b6460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400e980 tx=0x7f2e1400ee40 comp rx=0 tx=0).stop 2024-04-25T08:42:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 shutdown_connections 2024-04-25T08:42:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e00066f90 0x7f2e00069450 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccde0 0x7f2e24156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e240b2240 0x7f2e24158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 --2- 172.21.15.46:0/2544299254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e24099a00 0x7f2e240b6460 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 >> 172.21.15.46:0/2544299254 conn(0x7f2e240b3120 msgr2=0x7f2e24073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 shutdown_connections 2024-04-25T08:42:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.251+0000 7f2e29000640 1 -- 172.21.15.46:0/2544299254 wait complete. 2024-04-25T08:42:50.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a5b400640 1 Processor -- start 2024-04-25T08:42:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a5b400640 1 -- start start 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a5b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a54099a00 0x7f0a54099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a5b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 0x7f0a540cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a5b400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a5409a320 con 0x7f0a540b2240 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a5b400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5406db90 con 0x7f0a54099a00 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a5b400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5406bdc0 con 0x7f0a540ccde0 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a58a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a54099a00 0x7f0a54099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a58a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a54099a00 0x7f0a54099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39186/0 (socket says 172.21.15.46:39186) 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a58a00640 1 -- 172.21.15.46:0/3966515712 learned_addr learned my addr 172.21.15.46:0/3966515712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a58a00640 1 -- 172.21.15.46:0/3966515712 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 msgr2=0x7f0a540cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a58a00640 1 --2- 172.21.15.46:0/3966515712 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 0x7f0a540cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a58a00640 1 -- 172.21.15.46:0/3966515712 >> v1:172.21.15.82:6789/0 conn(0x7f0a540b2240 legacy=0x7f0a540b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.415+0000 7f0a58a00640 1 -- 172.21.15.46:0/3966515712 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5406c0a0 con 0x7f0a54099a00 2024-04-25T08:42:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a58a00640 1 --2- 172.21.15.46:0/3966515712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a54099a00 0x7f0a54099de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009870 tx=0x7f0a4401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb65096e7f0ba690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a53e00640 1 -- 172.21.15.46:0/3966515712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a440ab620 con 0x7f0a54099a00 2024-04-25T08:42:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a53e00640 1 -- 172.21.15.46:0/3966515712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a440b0070 con 0x7f0a54099a00 2024-04-25T08:42:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a53e00640 1 -- 172.21.15.46:0/3966515712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a440ab9e0 con 0x7f0a54099a00 2024-04-25T08:42:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a5b400640 1 -- 172.21.15.46:0/3966515712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a54099a00 msgr2=0x7f0a54099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a5b400640 1 --2- 172.21.15.46:0/3966515712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a54099a00 0x7f0a54099de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009870 tx=0x7f0a4401c460 comp rx=0 tx=0).stop 2024-04-25T08:42:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a5b400640 1 -- 172.21.15.46:0/3966515712 shutdown_connections 2024-04-25T08:42:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a5b400640 1 --2- 172.21.15.46:0/3966515712 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 0x7f0a540cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a5b400640 1 --2- 172.21.15.46:0/3966515712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a54099a00 0x7f0a54099de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a5b400640 1 -- 172.21.15.46:0/3966515712 >> 172.21.15.46:0/3966515712 conn(0x7f0a540b3120 msgr2=0x7f0a540b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a5b400640 1 -- 172.21.15.46:0/3966515712 shutdown_connections 2024-04-25T08:42:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.419+0000 7f0a5b400640 1 -- 172.21.15.46:0/3966515712 wait complete. 2024-04-25T08:42:50.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 Processor -- start 2024-04-25T08:42:50.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 -- start start 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a54099a00 0x7f0a540cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a540b2240 0x7f0a540ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 0x7f0a541f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540cd7a0 con 0x7f0a540b2240 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540cdb80 con 0x7f0a54099a00 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540cd920 con 0x7f0a540ccde0 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a4be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a540b2240 0x7f0a540ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a4be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a540b2240 0x7f0a540ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39200/0 (socket says 172.21.15.46:39200) 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a4be00640 1 -- 172.21.15.46:0/1100755135 learned_addr learned my addr 172.21.15.46:0/1100755135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a59400640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 0x7f0a541f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a58a00640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a54099a00 0x7f0a540cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a4be00640 1 -- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 msgr2=0x7f0a541f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a4be00640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 0x7f0a541f63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a4be00640 1 -- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a54099a00 msgr2=0x7f0a540cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a4be00640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a54099a00 0x7f0a540cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a4be00640 1 -- 172.21.15.46:0/1100755135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a54158380 con 0x7f0a540b2240 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a59400640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 0x7f0a541f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a58a00640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a54099a00 0x7f0a540cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a4be00640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a540b2240 0x7f0a540ce470 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00b550 tx=0x7f0a3c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a51600640 1 -- 172.21.15.46:0/1100755135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c014070 con 0x7f0a540b2240 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a51600640 1 -- 172.21.15.46:0/1100755135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c010890 con 0x7f0a540b2240 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a51600640 1 -- 172.21.15.46:0/1100755135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c010c80 con 0x7f0a540b2240 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a541578e0 con 0x7f0a540b2240 2024-04-25T08:42:50.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a54157b80 con 0x7f0a540b2240 2024-04-25T08:42:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.423+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a54069080 con 0x7f0a540b2240 2024-04-25T08:42:50.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.427+0000 7f0a51600640 1 -- 172.21.15.46:0/1100755135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a3c017070 con 0x7f0a540b2240 2024-04-25T08:42:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.427+0000 7f0a51600640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a30066f60 0x7f0a30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.427+0000 7f0a51600640 1 -- 172.21.15.46:0/1100755135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0a3c0851a0 con 0x7f0a540b2240 2024-04-25T08:42:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.427+0000 7f0a58a00640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a30066f60 0x7f0a30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.427+0000 7f0a58a00640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a30066f60 0x7f0a30069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a440af000 tx=0x7f0a440a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:50.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.431+0000 7f0a51600640 1 -- 172.21.15.46:0/1100755135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a3c055e60 con 0x7f0a540b2240 2024-04-25T08:42:50.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a540af160 con 0x7f0a540b2240 2024-04-25T08:42:50.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a51600640 1 -- 172.21.15.46:0/1100755135 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f0a3c010e20 con 0x7f0a540b2240 2024-04-25T08:42:50.588 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:50.588 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a30066f60 msgr2=0x7f0a30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a30066f60 0x7f0a30069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a440af000 tx=0x7f0a440a7000 comp rx=0 tx=0).stop 2024-04-25T08:42:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a540b2240 msgr2=0x7f0a540ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a540b2240 0x7f0a540ce470 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00b550 tx=0x7f0a3c00ba10 comp rx=0 tx=0).stop 2024-04-25T08:42:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 shutdown_connections 2024-04-25T08:42:50.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a30066f60 0x7f0a30069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a540ccde0 0x7f0a541f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a540b2240 0x7f0a540ce470 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 --2- 172.21.15.46:0/1100755135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a54099a00 0x7f0a540cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 >> 172.21.15.46:0/1100755135 conn(0x7f0a540b3120 msgr2=0x7f0a540ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:50.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 shutdown_connections 2024-04-25T08:42:50.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.595+0000 7f0a5b400640 1 -- 172.21.15.46:0/1100755135 wait complete. 2024-04-25T08:42:50.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:50.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c3200640 1 Processor -- start 2024-04-25T08:42:50.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c3200640 1 -- start start 2024-04-25T08:42:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c3200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc9c413e540 0x7fc9c40cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c40cd750 0x7fc9c4141560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c40710e0 con 0x7fc9c4071ea0 2024-04-25T08:42:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c3200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c40713c0 con 0x7fc9c413e540 2024-04-25T08:42:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c40716a0 con 0x7fc9c40cd750 2024-04-25T08:42:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc9c4071ea0 0x7fc9c4140e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33374/0 (socket says 172.21.15.46:33374) 2024-04-25T08:42:50.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c1400640 1 -- 172.21.15.46:0/2092237603 learned_addr learned my addr 172.21.15.46:0/2092237603 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c2800640 1 --2- 172.21.15.46:0/2092237603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c40cd750 0x7fc9c4141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c1e00640 1 --2- 172.21.15.46:0/2092237603 >> v2:172.21.15.82:3300/0 conn(0x7fc9c413e540 0x7fc9c40cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c0a00640 1 -- 172.21.15.46:0/2092237603 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582976239 0 0) 0x7fc9c40710e0 con 0x7fc9c4071ea0 2024-04-25T08:42:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c0a00640 1 -- 172.21.15.46:0/2092237603 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a8003530 con 0x7fc9c4071ea0 2024-04-25T08:42:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c1e00640 1 -- 172.21.15.46:0/2092237603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c40cd750 msgr2=0x7fc9c4141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c1e00640 1 --2- 172.21.15.46:0/2092237603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c40cd750 0x7fc9c4141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c1e00640 1 -- 172.21.15.46:0/2092237603 >> v1:172.21.15.46:6789/0 conn(0x7fc9c4071ea0 legacy=0x7fc9c4140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c1e00640 1 -- 172.21.15.46:0/2092237603 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c4071980 con 0x7fc9c413e540 2024-04-25T08:42:50.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.759+0000 7fc9c2800640 1 --2- 172.21.15.46:0/2092237603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c40cd750 0x7fc9c4141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:50.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c1e00640 1 --2- 172.21.15.46:0/2092237603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9c413e540 0x7fc9c40cd210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40097c0 tx=0x7fc9b401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3f30bb80367483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:50.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c0a00640 1 -- 172.21.15.46:0/2092237603 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40b0070 con 0x7fc9c413e540 2024-04-25T08:42:50.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c0a00640 1 -- 172.21.15.46:0/2092237603 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b40ab600 con 0x7fc9c413e540 2024-04-25T08:42:50.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c0a00640 1 -- 172.21.15.46:0/2092237603 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40ab9d0 con 0x7fc9c413e540 2024-04-25T08:42:50.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c3200640 1 -- 172.21.15.46:0/2092237603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9c413e540 msgr2=0x7fc9c40cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c3200640 1 --2- 172.21.15.46:0/2092237603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9c413e540 0x7fc9c40cd210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40097c0 tx=0x7fc9b401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c3200640 1 -- 172.21.15.46:0/2092237603 shutdown_connections 2024-04-25T08:42:50.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c3200640 1 --2- 172.21.15.46:0/2092237603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c40cd750 0x7fc9c4141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c3200640 1 --2- 172.21.15.46:0/2092237603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9c413e540 0x7fc9c40cd210 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c3200640 1 -- 172.21.15.46:0/2092237603 >> 172.21.15.46:0/2092237603 conn(0x7fc9c4066fa0 msgr2=0x7fc9c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:50.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c3200640 1 -- 172.21.15.46:0/2092237603 shutdown_connections 2024-04-25T08:42:50.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c3200640 1 -- 172.21.15.46:0/2092237603 wait complete. 2024-04-25T08:42:50.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.763+0000 7fc9c3200640 1 Processor -- start 2024-04-25T08:42:50.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 -- start start 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9c4071ea0 0x7fc9c41512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9c40cd750 0x7fc9c40b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c413e540 0x7fc9c40b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c413f8b0 con 0x7fc9c40cd750 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c413f730 con 0x7fc9c4071ea0 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c413fa30 con 0x7fc9c413e540 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9c40cd750 0x7fc9c40b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9c40cd750 0x7fc9c40b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39212/0 (socket says 172.21.15.46:39212) 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1400640 1 -- 172.21.15.46:0/3798169066 learned_addr learned my addr 172.21.15.46:0/3798169066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1e00640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9c4071ea0 0x7fc9c41512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c2800640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c413e540 0x7fc9c40b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1400640 1 -- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c413e540 msgr2=0x7fc9c40b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1400640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c413e540 0x7fc9c40b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1400640 1 -- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9c4071ea0 msgr2=0x7fc9c41512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1400640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9c4071ea0 0x7fc9c41512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1400640 1 -- 172.21.15.46:0/3798169066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c4151830 con 0x7fc9c40cd750 2024-04-25T08:42:50.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c1400640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9c40cd750 0x7fc9c40b2880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000c3f0 tx=0x7fc9b000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:50.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9ba000640 1 -- 172.21.15.46:0/3798169066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0017070 con 0x7fc9c40cd750 2024-04-25T08:42:50.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9ba000640 1 -- 172.21.15.46:0/3798169066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b0005cf0 con 0x7fc9c40cd750 2024-04-25T08:42:50.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9ba000640 1 -- 172.21.15.46:0/3798169066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00072f0 con 0x7fc9c40cd750 2024-04-25T08:42:50.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c413a920 con 0x7fc9c40cd750 2024-04-25T08:42:50.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c413aee0 con 0x7fc9c40cd750 2024-04-25T08:42:50.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.767+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c4069010 con 0x7fc9c40cd750 2024-04-25T08:42:50.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.771+0000 7fc9ba000640 1 -- 172.21.15.46:0/3798169066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9b001a070 con 0x7fc9c40cd750 2024-04-25T08:42:50.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.771+0000 7fc9ba000640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc9a4066f90 0x7fc9a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:50.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.771+0000 7fc9ba000640 1 -- 172.21.15.46:0/3798169066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc9b0084680 con 0x7fc9c40cd750 2024-04-25T08:42:50.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.771+0000 7fc9c1e00640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc9a4066f90 0x7fc9a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:50.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.771+0000 7fc9c1e00640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc9a4066f90 0x7fc9a4069450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40ad000 tx=0x7fc9b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:50.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.775+0000 7fc9ba000640 1 -- 172.21.15.46:0/3798169066 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc9b0054c60 con 0x7fc9c40cd750 2024-04-25T08:42:50.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9c4141640 con 0x7fc9c40cd750 2024-04-25T08:42:50.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9ba000640 1 -- 172.21.15.46:0/3798169066 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fc9b0058d00 con 0x7fc9c40cd750 2024-04-25T08:42:50.963 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:42:50.964 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:38.824358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:42:50.964 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-25T08:42:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc9a4066f90 msgr2=0x7fc9a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc9a4066f90 0x7fc9a4069450 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40ad000 tx=0x7fc9b4008040 comp rx=0 tx=0).stop 2024-04-25T08:42:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9c40cd750 msgr2=0x7fc9c40b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9c40cd750 0x7fc9c40b2880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9b000c3f0 tx=0x7fc9b000c7f0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 shutdown_connections 2024-04-25T08:42:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc9a4066f90 0x7fc9a4069450 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9c413e540 0x7fc9c40b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9c40cd750 0x7fc9c40b2880 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 --2- 172.21.15.46:0/3798169066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9c4071ea0 0x7fc9c41512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 >> 172.21.15.46:0/3798169066 conn(0x7fc9c4066fa0 msgr2=0x7fc9c415efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 shutdown_connections 2024-04-25T08:42:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:50.971+0000 7fc9c3200640 1 -- 172.21.15.46:0/3798169066 wait complete. 2024-04-25T08:42:51.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:42:20.723239+0000', 'modified': '2024-04-25T08:42:38.824358+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/3580064760', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 3580064760}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 3580064760}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:42:51.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:42:51.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T08:42:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.543+0000 7f9479000640 1 Processor -- start 2024-04-25T08:42:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.543+0000 7f9479000640 1 -- start start 2024-04-25T08:42:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9479000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f947409a610 0x7f947409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9479000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947415f300 con 0x7f9474099a00 2024-04-25T08:42:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9479000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94740cd8a0 con 0x7f947415ef20 2024-04-25T08:42:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9479000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94740cddb0 con 0x7f947409a610 2024-04-25T08:42:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9472800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9474099a00 0x7f94740c5180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33378/0 (socket says 172.21.15.46:33378) 2024-04-25T08:42:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9472800640 1 -- 172.21.15.46:0/2880944479 learned_addr learned my addr 172.21.15.46:0/2880944479 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f946be00640 1 --2- 172.21.15.46:0/2880944479 >> v2:172.21.15.82:3300/0 conn(0x7f947409a610 0x7f947409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407074596 0 0) 0x7f947415f300 con 0x7f9474099a00 2024-04-25T08:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9458003530 con 0x7f9474099a00 2024-04-25T08:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642808018 0 0) 0x7f9458003530 con 0x7f9474099a00 2024-04-25T08:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 >> v2:172.21.15.82:3300/0 conn(0x7f947409a610 msgr2=0x7f947409aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 --2- 172.21.15.46:0/2880944479 >> v2:172.21.15.82:3300/0 conn(0x7f947409a610 0x7f947409aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 >> v1:172.21.15.82:6789/0 conn(0x7f947415ef20 legacy=0x7f94740cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94740cc3d0 con 0x7f9474099a00 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f946c001a00 con 0x7f9474099a00 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 >> v1:172.21.15.46:6789/0 conn(0x7f9474099a00 legacy=0x7f94740c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9458003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9458004810 0x7f9458004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94580051d0 0x7f945800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740cddb0 con 0x7f94580039c0 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945800c070 con 0x7f94580051d0 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945800c270 con 0x7f9458004810 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471e00640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9458003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f946be00640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9458004810 0x7f9458004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9472800640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94580051d0 0x7f945800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471e00640 1 -- 172.21.15.46:0/2880944479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9458004810 msgr2=0x7f9458004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471e00640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9458004810 0x7f9458004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471e00640 1 -- 172.21.15.46:0/2880944479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94580051d0 msgr2=0x7f945800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471e00640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94580051d0 0x7f945800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9471e00640 1 -- 172.21.15.46:0/2880944479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9458003530 con 0x7f94580039c0 2024-04-25T08:42:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.547+0000 7f9472800640 1 -- 172.21.15.46:0/2880944479 reap_dead start 2024-04-25T08:42:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9471e00640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9458003de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9464001780 tx=0x7f946401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c81cfe5eb162ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94640013a0 con 0x7f94580039c0 2024-04-25T08:42:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94640052f0 con 0x7f94580039c0 2024-04-25T08:42:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9471400640 1 -- 172.21.15.46:0/2880944479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464003840 con 0x7f94580039c0 2024-04-25T08:42:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9479000640 1 -- 172.21.15.46:0/2880944479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 msgr2=0x7f9458003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9479000640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9458003de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9464001780 tx=0x7f946401c3d0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9479000640 1 -- 172.21.15.46:0/2880944479 shutdown_connections 2024-04-25T08:42:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9479000640 1 --2- 172.21.15.46:0/2880944479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9458003de0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9479000640 1 -- 172.21.15.46:0/2880944479 >> 172.21.15.46:0/2880944479 conn(0x7f94740b3120 msgr2=0x7f94740b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9479000640 1 -- 172.21.15.46:0/2880944479 shutdown_connections 2024-04-25T08:42:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.551+0000 7f9479000640 1 -- 172.21.15.46:0/2880944479 wait complete. 2024-04-25T08:42:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 Processor -- start 2024-04-25T08:42:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 -- start start 2024-04-25T08:42:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9474151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f947406d590 0x7f94740b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f947415ab70 0x7f94740b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740cd5a0 con 0x7f94580039c0 2024-04-25T08:42:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947409ae00 con 0x7f947406d590 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94740cd720 con 0x7f947415ab70 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9471e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9474151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9471e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9474151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39236/0 (socket says 172.21.15.46:39236) 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9471e00640 1 -- 172.21.15.46:0/30024344 learned_addr learned my addr 172.21.15.46:0/30024344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9472800640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f947415ab70 0x7f94740b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f946be00640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f947406d590 0x7f94740b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9471e00640 1 -- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f947415ab70 msgr2=0x7f94740b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9471e00640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f947415ab70 0x7f94740b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9471e00640 1 -- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f947406d590 msgr2=0x7f94740b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9471e00640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f947406d590 0x7f94740b6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9471e00640 1 -- 172.21.15.46:0/30024344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947415dcc0 con 0x7f94580039c0 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9471e00640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9474151ac0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f946401c200 tx=0x7f9458005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:51.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f946a000640 1 -- 172.21.15.46:0/30024344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94640013a0 con 0x7f94580039c0 2024-04-25T08:42:51.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f946a000640 1 -- 172.21.15.46:0/30024344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94640ba070 con 0x7f94580039c0 2024-04-25T08:42:51.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f946a000640 1 -- 172.21.15.46:0/30024344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464007a40 con 0x7f94580039c0 2024-04-25T08:42:51.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 -- 172.21.15.46:0/30024344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947406c7d0 con 0x7f94580039c0 2024-04-25T08:42:51.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.555+0000 7f9479000640 1 -- 172.21.15.46:0/30024344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947406cd90 con 0x7f94580039c0 2024-04-25T08:42:51.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f946a000640 1 -- 172.21.15.46:0/30024344 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9464005290 con 0x7f94580039c0 2024-04-25T08:42:51.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f946a000640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9450066f60 0x7f9450069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f946a000640 1 -- 172.21.15.46:0/30024344 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f946411f130 con 0x7f94580039c0 2024-04-25T08:42:51.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f946be00640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9450066f60 0x7f9450069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f947c4a3000 1 Processor -- start 2024-04-25T08:42:51.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f947c4a3000 1 -- start start 2024-04-25T08:42:51.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f946be00640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9450066f60 0x7f9450069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f94740cf690 tx=0x7f945c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:51.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbaf98d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbaf4e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf4f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f947c4a3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbae365f0 con 0x563bbaf98920 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f947c4a3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbad9dda0 con 0x563bbaf4ef30 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f947c4a3000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bbad26150 con 0x563bbaf99240 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9471e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbaf98d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9471e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbaf98d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39250/0 (socket says 172.21.15.46:39250) 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9471e00640 1 -- 172.21.15.46:0/3973601343 learned_addr learned my addr 172.21.15.46:0/3973601343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f946be00640 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbaf4e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9472800640 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf4f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9471e00640 1 -- 172.21.15.46:0/3973601343 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 msgr2=0x563bbaf4e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9471e00640 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbaf4e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9471e00640 1 -- 172.21.15.46:0/3973601343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 msgr2=0x563bbaf4f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9471e00640 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf4f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9471e00640 1 -- 172.21.15.46:0/3973601343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bbad3fca0 con 0x563bbaf98920 2024-04-25T08:42:51.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.559+0000 7f9472800640 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf4f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:51.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9471e00640 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbaf98d00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9464005630 tx=0x7f94640efd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53290e7a0c816a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:51.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9462a00640 1 -- 172.21.15.46:0/3973601343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94640f37b0 con 0x563bbaf98920 2024-04-25T08:42:51.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9462a00640 1 -- 172.21.15.46:0/3973601343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94640f7d70 con 0x563bbaf98920 2024-04-25T08:42:51.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9462a00640 1 -- 172.21.15.46:0/3973601343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94640ebcd0 con 0x563bbaf98920 2024-04-25T08:42:51.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 -- 172.21.15.46:0/3973601343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 msgr2=0x563bbaf98d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbaf98d00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9464005630 tx=0x7f94640efd60 comp rx=0 tx=0).stop 2024-04-25T08:42:51.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 -- 172.21.15.46:0/3973601343 shutdown_connections 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf4f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbaf4e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3973601343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbaf98d00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 -- 172.21.15.46:0/3973601343 >> 172.21.15.46:0/3973601343 conn(0x563bbaf552a0 msgr2=0x563bbaf4d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 -- 172.21.15.46:0/3973601343 shutdown_connections 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 -- 172.21.15.46:0/3973601343 wait complete. 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 -- start start 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 Processor -- start 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf52e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf98920 0x563bbaf533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf99240 0x563bbaf53910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbad29900 con 0x563bbaf99240 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbae365f0 con 0x563bbaf4ef30 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bbad9dda0 con 0x563bbaf98920 2024-04-25T08:42:51.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f947c4a3000 10 client.0 _ll_register_callbacks cb 0x563bbae511c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9472800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf99240 0x563bbaf53910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9472800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf99240 0x563bbaf53910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39258/0 (socket says 172.21.15.46:39258) 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9472800640 1 -- 172.21.15.46:0/3833936430 learned_addr learned my addr 172.21.15.46:0/3833936430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9471e00640 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf52e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f946be00640 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf98920 0x563bbaf533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9472800640 1 -- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf98920 msgr2=0x563bbaf533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9472800640 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf98920 0x563bbaf533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9472800640 1 -- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 msgr2=0x563bbaf52e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9472800640 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf52e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9472800640 1 -- 172.21.15.46:0/3833936430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bbad56630 con 0x563bbaf99240 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9472800640 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf99240 0x563bbaf53910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f94740d1110 tx=0x7f946c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76de6411297fa3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.563+0000 7f9457400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9457400640 1 -- 172.21.15.46:0/3833936430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c00aa40 con 0x563bbaf99240 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9457400640 1 -- 172.21.15.46:0/3833936430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f946c00abe0 con 0x563bbaf99240 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 -- 172.21.15.46:0/3833936430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94640f7d70 con 0x563bbaf99240 2024-04-25T08:42:51.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9457400640 1 -- 172.21.15.46:0/3833936430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f946c00ade0 con 0x563bbaf99240 2024-04-25T08:42:51.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 10 client.4806 fetch_fsmap learned FSMap version 9 2024-04-25T08:42:51.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 -- 172.21.15.46:0/3833936430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x563bbad263a0 con 0x563bbaf99240 2024-04-25T08:42:51.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9457400640 1 -- 172.21.15.46:0/3833936430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94640f7d70 con 0x563bbaf99240 2024-04-25T08:42:51.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9457400640 1 -- 172.21.15.46:0/3833936430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f946c028020 con 0x563bbaf99240 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 10 client.4806 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 10 client.4806 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 20 client.4806 populate_metadata read hostname 'smithi046' 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x563bbafa7bc0 0x563bbafa9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 4 client.4806 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 -- 172.21.15.46:0/3833936430 --> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bbad39a50 con 0x563bbafa7bc0 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9471e00640 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x563bbafa7bc0 0x563bbafa9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9471e00640 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x563bbafa7bc0 0x563bbafa9fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cdd37dd6b8ada90 server_cookie=c836d851f08fdb90 in_seq=0 out_seq=0 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9457400640 10 client.4806 ms_handle_connect on v2:172.21.15.82:6835/3580064760 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9457400640 1 -- 172.21.15.46:0/3833936430 <== mds.0 v2:172.21.15.82:6835/3580064760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563bbad39a50 con 0x563bbafa7bc0 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9457400640 10 client.4806 handle_command_reply: tid=0 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 client.4806 shutdown 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 -- 172.21.15.46:0/3833936430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf99240 msgr2=0x563bbaf53910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf99240 0x563bbaf53910 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f94740d1110 tx=0x7f946c0078c0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 -- 172.21.15.46:0/3833936430 shutdown_connections 2024-04-25T08:42:51.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x563bbafa7bc0 0x563bbafa9fb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf99240 0x563bbaf53910 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf98920 0x563bbaf533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3833936430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf52e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f9457400640 0 client.4806 ms_handle_reset on v2:172.21.15.82:6835/3580064760 2024-04-25T08:42:51.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 -- 172.21.15.46:0/3833936430 >> 172.21.15.46:0/3833936430 conn(0x563bbaf552a0 msgr2=0x563bbaf4d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:51.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 -- 172.21.15.46:0/3833936430 shutdown_connections 2024-04-25T08:42:51.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 1 -- 172.21.15.46:0/3833936430 wait complete. 2024-04-25T08:42:51.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.567+0000 7f947c4a3000 20 client.4806 trim_cache size 0 max 16384 2024-04-25T08:42:51.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.583+0000 7f947c4a3000 1 Processor -- start 2024-04-25T08:42:51.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.583+0000 7f947c4a3000 1 -- start start 2024-04-25T08:42:51.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.583+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf4ef30 0x563bbaff4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.583+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf98920 0x563bbaff4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.583+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbaff4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.583+0000 7f947c4a3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9474204010 con 0x563bbaf4ef30 2024-04-25T08:42:51.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.583+0000 7f947c4a3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9474203440 con 0x563bbaf98920 2024-04-25T08:42:51.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.583+0000 7f947c4a3000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9474203a10 con 0x563bbaf99240 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf4ef30 0x563bbaff4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf4ef30 0x563bbaff4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39274/0 (socket says 172.21.15.46:39274) 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 -- 172.21.15.46:0/2573994286 learned_addr learned my addr 172.21.15.46:0/2573994286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f946be00640 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf98920 0x563bbaff4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9472800640 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbaff4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 -- 172.21.15.46:0/2573994286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 msgr2=0x563bbaff4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbaff4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 -- 172.21.15.46:0/2573994286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf98920 msgr2=0x563bbaff4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf98920 0x563bbaff4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 -- 172.21.15.46:0/2573994286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bbae1f600 con 0x563bbaf4ef30 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f946be00640 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf98920 0x563bbaff4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf4ef30 0x563bbaff4040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9464097dc0 tx=0x7f946411d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f887311d853a1c17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9462000640 1 -- 172.21.15.46:0/2573994286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464107120 con 0x563bbaf4ef30 2024-04-25T08:42:51.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9462000640 1 -- 172.21.15.46:0/2573994286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94640ed960 con 0x563bbaf4ef30 2024-04-25T08:42:51.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9462000640 1 -- 172.21.15.46:0/2573994286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464116be0 con 0x563bbaf4ef30 2024-04-25T08:42:51.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 -- 172.21.15.46:0/2573994286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf4ef30 msgr2=0x563bbaff4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf4ef30 0x563bbaff4040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9464097dc0 tx=0x7f946411d040 comp rx=0 tx=0).stop 2024-04-25T08:42:51.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 -- 172.21.15.46:0/2573994286 shutdown_connections 2024-04-25T08:42:51.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbaff4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf98920 0x563bbaff4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 --2- 172.21.15.46:0/2573994286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf4ef30 0x563bbaff4040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 -- 172.21.15.46:0/2573994286 >> 172.21.15.46:0/2573994286 conn(0x563bbaf552a0 msgr2=0x563bbaf96d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:51.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 -- 172.21.15.46:0/2573994286 shutdown_connections 2024-04-25T08:42:51.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 -- 172.21.15.46:0/2573994286 wait complete. 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 -- start start 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 Processor -- start 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf533a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbafdc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbafdca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbada23d0 con 0x563bbaf98920 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bbae59d50 con 0x563bbaf4ef30 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bbad38980 con 0x563bbaf99240 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f947c4a3000 10 client.0 _ll_register_callbacks cb 0x7f9464003ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f946be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbafdc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9471e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f946be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbafdc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39282/0 (socket says 172.21.15.46:39282) 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f946be00640 1 -- 172.21.15.46:0/3198775595 learned_addr learned my addr 172.21.15.46:0/3198775595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:51.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.587+0000 7f9472800640 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbafdca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f946be00640 1 -- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 msgr2=0x563bbafdca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f946be00640 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbafdca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f946be00640 1 -- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 msgr2=0x563bbaf533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f946be00640 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f946be00640 1 -- 172.21.15.46:0/3198775595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bbad329b0 con 0x563bbaf98920 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9471e00640 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f946be00640 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbafdc510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f945c0038a0 tx=0x7f945c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fc5a037b0590a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 1 -- 172.21.15.46:0/3198775595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c032060 con 0x563bbaf98920 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 1 -- 172.21.15.46:0/3198775595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f945c022bc0 con 0x563bbaf98920 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 -- 172.21.15.46:0/3198775595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94640ed960 con 0x563bbaf98920 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 1 -- 172.21.15.46:0/3198775595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f945c038050 con 0x563bbaf98920 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 10 client.4812 fetch_fsmap learned FSMap version 9 2024-04-25T08:42:51.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 -- 172.21.15.46:0/3198775595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x563bbad450b0 con 0x563bbaf98920 2024-04-25T08:42:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 1 -- 172.21.15.46:0/3198775595 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94640ed960 con 0x563bbaf98920 2024-04-25T08:42:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 1 -- 172.21.15.46:0/3198775595 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f945c03b020 con 0x563bbaf98920 2024-04-25T08:42:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 10 client.4812 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T08:42:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 10 client.4812 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:42:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 20 client.4812 populate_metadata read hostname 'smithi046' 2024-04-25T08:42:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x563bbaf538e0 0x563bbafdd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 4 client.4812 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:42:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 -- 172.21.15.46:0/3198775595 --> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563bbadab2e0 con 0x563bbaf538e0 2024-04-25T08:42:51.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9471e00640 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x563bbaf538e0 0x563bbafdd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:51.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9471e00640 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x563bbaf538e0 0x563bbafdd2f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dbde86bc6d5d9f9 server_cookie=2d5144c6afb026a9 in_seq=0 out_seq=0 2024-04-25T08:42:51.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 10 client.4812 ms_handle_connect on v2:172.21.15.82:6835/3580064760 2024-04-25T08:42:51.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 1 -- 172.21.15.46:0/3198775595 <== mds.0 v2:172.21.15.82:6835/3580064760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x563bbadab2e0 con 0x563bbaf538e0 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 10 client.4812 handle_command_reply: tid=0 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 client.4812 shutdown 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 -- 172.21.15.46:0/3198775595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 msgr2=0x563bbafdc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbafdc510 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f945c0038a0 tx=0x7f945c0046d0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 -- 172.21.15.46:0/3198775595 shutdown_connections 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:6835/3580064760,v1:172.21.15.82:6837/3580064760] conn(0x563bbaf538e0 0x563bbafdd2f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563bbaf99240 0x563bbafdca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563bbaf98920 0x563bbafdc510 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9457400640 0 client.4812 ms_handle_reset on v2:172.21.15.82:6835/3580064760 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 --2- 172.21.15.46:0/3198775595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563bbaf4ef30 0x563bbaf533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 -- 172.21.15.46:0/3198775595 >> 172.21.15.46:0/3198775595 conn(0x563bbaf552a0 msgr2=0x563bbaf97430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 -- 172.21.15.46:0/3198775595 shutdown_connections 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 1 -- 172.21.15.46:0/3198775595 wait complete. 2024-04-25T08:42:51.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f947c4a3000 20 client.4812 trim_cache size 0 max 16384 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4281, 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 30.264984988999998, 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 9, 2024-04-25T08:42:51.590 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 26, 2024-04-25T08:42:51.591 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 26, 2024-04-25T08:42:51.591 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 37.427032062999999 2024-04-25T08:42:51.591 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:42:51.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 -- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9450066f60 msgr2=0x7f9450069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9450066f60 0x7f9450069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f94740cf690 tx=0x7f945c009190 comp rx=0 tx=0).stop 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 -- 172.21.15.46:0/30024344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 msgr2=0x7f9474151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9474151ac0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f946401c200 tx=0x7f9458005840 comp rx=0 tx=0).stop 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 -- 172.21.15.46:0/30024344 shutdown_connections 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9450066f60 0x7f9450069420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f947415ab70 0x7f94740b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f947406d590 0x7f94740b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 --2- 172.21.15.46:0/30024344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94580039c0 0x7f9474151ac0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.591+0000 7f9479000640 1 -- 172.21.15.46:0/30024344 >> 172.21.15.46:0/30024344 conn(0x7f94740b3120 msgr2=0x7f947415f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.595+0000 7f9479000640 1 -- 172.21.15.46:0/30024344 shutdown_connections 2024-04-25T08:42:51.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:51.595+0000 7f9479000640 1 -- 172.21.15.46:0/30024344 wait complete. 2024-04-25T08:42:51.605 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T08:42:51.619 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T08:42:51.619 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T08:42:51.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:51.620 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:42:51.629 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T08:42:51.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:51.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:42:51.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:51.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-25T08:42:51.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:51.881 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:42:51.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:51.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-25T08:42:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-04-25T08:42:51.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:42:51.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:51.959 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T08:42:51.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:51.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-25T08:42:52.038 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-04-25T08:42:52.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:42:52.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.040 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T08:42:52.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.094 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-25T08:42:52.161 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-04-25T08:42:52.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:42:52.164 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T08:42:52.164 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:42:52.164 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:52.164 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:42:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.311+0000 7f0ab5a00640 1 Processor -- start 2024-04-25T08:42:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.311+0000 7f0ab5a00640 1 -- start start 2024-04-25T08:42:52.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00b0be0 0x7f0ab00b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:52.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab0162240 con 0x7f0ab006ac10 2024-04-25T08:42:52.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab00b4f80 con 0x7f0ab0146210 2024-04-25T08:42:52.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00b5590 con 0x7f0ab00b0be0 2024-04-25T08:42:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aa7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0ab006ac10 0x7f0ab0161420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33382/0 (socket says 172.21.15.46:33382) 2024-04-25T08:42:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aa7e00640 1 -- 172.21.15.46:0/3678151825 learned_addr learned my addr 172.21.15.46:0/3678151825 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aafe00640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00b0be0 0x7f0ab00b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494232255 0 0) 0x7f0ab0162240 con 0x7f0ab006ac10 2024-04-25T08:42:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a94003530 con 0x7f0ab006ac10 2024-04-25T08:42:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4132004612 0 0) 0x7f0a94003530 con 0x7f0ab006ac10 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00b0be0 msgr2=0x7f0ab00b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00b0be0 0x7f0ab00b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 >> v1:172.21.15.82:6789/0 conn(0x7f0ab0146210 legacy=0x7f0ab0161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab00b1b40 con 0x7f0ab006ac10 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f0a98001680 con 0x7f0ab006ac10 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 >> v1:172.21.15.46:6789/0 conn(0x7f0ab006ac10 legacy=0x7f0ab0161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a940039f0 0x7f0a94003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00b0be0 0x7f0a94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a94004d50 0x7f0a940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9400bb60 con 0x7f0a94004d50 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9400bce0 con 0x7f0a940039f0 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9400bee0 con 0x7f0ab00b0be0 2024-04-25T08:42:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5000640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a94004d50 0x7f0a940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aafe00640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a940039f0 0x7f0a94003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0aa7e00640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00b0be0 0x7f0a94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5000640 1 -- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00b0be0 msgr2=0x7f0a94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5000640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00b0be0 0x7f0a94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5000640 1 -- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a940039f0 msgr2=0x7f0a94003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5000640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a940039f0 0x7f0a94003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.315+0000 7f0ab5000640 1 -- 172.21.15.46:0/3678151825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a94003530 con 0x7f0a94004d50 2024-04-25T08:42:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0ab5000640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a94004d50 0x7f0a940051d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8001e40 tx=0x7f0aa801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aac6c3c313afe79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8001a90 con 0x7f0a94004d50 2024-04-25T08:42:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8003b80 con 0x7f0a94004d50 2024-04-25T08:42:52.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0aaf400640 1 -- 172.21.15.46:0/3678151825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98007e60 con 0x7f0a94004d50 2024-04-25T08:42:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0ab5a00640 1 -- 172.21.15.46:0/3678151825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a94004d50 msgr2=0x7f0a940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0ab5a00640 1 --2- 172.21.15.46:0/3678151825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a94004d50 0x7f0a940051d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8001e40 tx=0x7f0aa801c760 comp rx=0 tx=0).stop 2024-04-25T08:42:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0ab5000640 1 -- 172.21.15.46:0/3678151825 reap_dead start 2024-04-25T08:42:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0ab5a00640 1 -- 172.21.15.46:0/3678151825 shutdown_connections 2024-04-25T08:42:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0ab5a00640 1 -- 172.21.15.46:0/3678151825 >> 172.21.15.46:0/3678151825 conn(0x7f0ab0099a00 msgr2=0x7f0ab00b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0ab5a00640 1 -- 172.21.15.46:0/3678151825 shutdown_connections 2024-04-25T08:42:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.319+0000 7f0ab5a00640 1 -- 172.21.15.46:0/3678151825 wait complete. 2024-04-25T08:42:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 Processor -- start 2024-04-25T08:42:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 -- start start 2024-04-25T08:42:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a94004d50 0x7f0ab015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ab0144520 0x7f0ab01449a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00afc80 0x7f0ab00b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00b1e20 con 0x7f0ab0144520 2024-04-25T08:42:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab0147370 con 0x7f0a94004d50 2024-04-25T08:42:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00b0fc0 con 0x7f0ab00afc80 2024-04-25T08:42:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aa7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ab0144520 0x7f0ab01449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aa7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ab0144520 0x7f0ab01449a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39296/0 (socket says 172.21.15.46:39296) 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aa7e00640 1 -- 172.21.15.46:0/2647632513 learned_addr learned my addr 172.21.15.46:0/2647632513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5000640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00afc80 0x7f0ab00b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aafe00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a94004d50 0x7f0ab015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aa7e00640 1 -- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00afc80 msgr2=0x7f0ab00b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aa7e00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00afc80 0x7f0ab00b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aa7e00640 1 -- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a94004d50 msgr2=0x7f0ab015cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aa7e00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a94004d50 0x7f0ab015cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aa7e00640 1 -- 172.21.15.46:0/2647632513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab00ae7d0 con 0x7f0ab0144520 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aafe00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a94004d50 0x7f0ab015cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5000640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00afc80 0x7f0ab00b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aa7e00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ab0144520 0x7f0ab01449a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0a98001050 tx=0x7f0a980126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aacc00640 1 -- 172.21.15.46:0/2647632513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9801c070 con 0x7f0ab0144520 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aacc00640 1 -- 172.21.15.46:0/2647632513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a98014770 con 0x7f0ab0144520 2024-04-25T08:42:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0aacc00640 1 -- 172.21.15.46:0/2647632513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98014b60 con 0x7f0ab0144520 2024-04-25T08:42:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab00ad730 con 0x7f0ab0144520 2024-04-25T08:42:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab00adcf0 con 0x7f0ab0144520 2024-04-25T08:42:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.323+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab0069010 con 0x7f0ab0144520 2024-04-25T08:42:52.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.327+0000 7f0aacc00640 1 -- 172.21.15.46:0/2647632513 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a98014d00 con 0x7f0ab0144520 2024-04-25T08:42:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.327+0000 7f0aacc00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a8c066f90 0x7f0a8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.327+0000 7f0aacc00640 1 -- 172.21.15.46:0/2647632513 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0a98087bc0 con 0x7f0ab0144520 2024-04-25T08:42:52.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.327+0000 7f0aafe00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a8c066f90 0x7f0a8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.327+0000 7f0aafe00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a8c066f90 0x7f0a8c069450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009ad0 tx=0x7f0aa0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:52.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.331+0000 7f0aacc00640 1 -- 172.21.15.46:0/2647632513 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a98058880 con 0x7f0ab0144520 2024-04-25T08:42:52.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.531+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0ab006d9b0 con 0x7f0ab0144520 2024-04-25T08:42:52.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.531+0000 7f0aacc00640 1 -- 172.21.15.46:0/2647632513 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0a98058180 con 0x7f0ab0144520 2024-04-25T08:42:52.524 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:42:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.531+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a8c066f90 msgr2=0x7f0a8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.531+0000 7f0ab5a00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a8c066f90 0x7f0a8c069450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009ad0 tx=0x7f0aa0008040 comp rx=0 tx=0).stop 2024-04-25T08:42:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.531+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ab0144520 msgr2=0x7f0ab01449a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.531+0000 7f0ab5a00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ab0144520 0x7f0ab01449a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0a98001050 tx=0x7f0a980126f0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.531+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 shutdown_connections 2024-04-25T08:42:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.535+0000 7f0ab5a00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a8c066f90 0x7f0a8c069450 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.535+0000 7f0ab5a00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ab00afc80 0x7f0ab00b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.535+0000 7f0ab5a00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ab0144520 0x7f0ab01449a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.535+0000 7f0ab5a00640 1 --2- 172.21.15.46:0/2647632513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a94004d50 0x7f0ab015cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.535+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 >> 172.21.15.46:0/2647632513 conn(0x7f0ab0099a00 msgr2=0x7f0ab01611f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.535+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 shutdown_connections 2024-04-25T08:42:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:52.535+0000 7f0ab5a00640 1 -- 172.21.15.46:0/2647632513 wait complete. 2024-04-25T08:42:52.546 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:42:52.546 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:42:52.546 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:42:52.546 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:42:52.546 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:52.546 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:42:52.546 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:42:52.546 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:42:52.546 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:42:52.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.547 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:42:52.555 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:52.555 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:52.555 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:52.555 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42430sec preferred_lft 42430sec 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:42:52.556 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:52.557 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:52.557 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:52.557 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:42:52.557 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:42:52.557 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:52.557 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:42:52.557 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:52.558 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:42:52.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.558 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:52.558 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:52.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:42:52.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-04-25T08:42:52.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-04-25T08:42:52.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:42:52.558 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:52.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.669 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:42:52.688 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:42:52.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.690 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:42:52.743 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:42:52.744 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:42:52.744 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:42:52.744 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:42:52.744 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:42:52.744 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:42:52.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.744 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:52.744 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:52.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:42:52.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:42:52.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:42:52.745 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:52.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.840 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:42:52.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.849 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-04-25T08:42:52.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.903 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:52.903 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:52.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:42:52.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T08:42:52.903 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:52.984 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T08:42:52.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:52.985 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:52.985 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:52.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T08:42:52.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:42:52.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T08:42:52.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T08:42:52.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T08:42:52.985 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:53.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:53.123 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:53.123 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:53.123 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:42:53.123 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:42:53.123 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:53.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:53.163 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:53.218 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:42:53.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:53.219 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:53.273 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:42:53.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:53.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:42:53.365 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:42:53.365 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:42:53.365 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:42:53.973 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:42:53.973 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:42:53.973 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:42:53.974 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:42:53.974 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:42:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:42:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:42:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:42:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:42:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:42:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:42:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:42:53.986 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:42:53.990 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:42:53.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:53.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:54.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.011 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:54.069 INFO:teuthology.orchestra.run.smithi046.stdout:client4658 2024-04-25T08:42:54.070 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:42:54.070 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:54.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.118 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:54.177 INFO:teuthology.orchestra.run.smithi046.stdout:client4658 2024-04-25T08:42:54.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.177 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:54.229 INFO:teuthology.orchestra.run.smithi046.stdout:client4658 2024-04-25T08:42:54.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.229 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ca516a0-1b6e-4031-bb83-1b30eee026c8.client4658/status') 2024-04-25T08:42:54.293 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.4658 (3)172.21.15.46:0/762062937 2024-04-25T08:42:54.293 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-04-25T08:42:54.294 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-04-25T08:42:54.294 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-04-25T08:42:54.294 INFO:teuthology.orchestra.run.smithi046.stderr:69 bytes copied, 5.7381e-05 s, 1.2 MB/s 2024-04-25T08:42:54.296 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T08:42:54.296 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:42:54.296 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:42:54.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:42:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e9600640 1 Processor -- start 2024-04-25T08:42:54.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e9600640 1 -- start start 2024-04-25T08:42:54.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e9600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f26e409a110 0x7f26e409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:54.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e9600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e415f300 con 0x7f26e415ef20 2024-04-25T08:42:54.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e9600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e40cd8a0 con 0x7f26e409a610 2024-04-25T08:42:54.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e9600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e40cddb0 con 0x7f26e409a110 2024-04-25T08:42:54.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26e415ef20 0x7f26e40cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33392/0 (socket says 172.21.15.46:33392) 2024-04-25T08:42:54.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e2a00640 1 -- 172.21.15.46:0/21315365 learned_addr learned my addr 172.21.15.46:0/21315365 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:54.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e3400640 1 --2- 172.21.15.46:0/21315365 >> v2:172.21.15.82:3300/0 conn(0x7f26e409a110 0x7f26e409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:54.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705052195 0 0) 0x7f26e415f300 con 0x7f26e415ef20 2024-04-25T08:42:54.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c8003530 con 0x7f26e415ef20 2024-04-25T08:42:54.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1066188856 0 0) 0x7f26c8003530 con 0x7f26e415ef20 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 >> v2:172.21.15.82:3300/0 conn(0x7f26e409a110 msgr2=0x7f26e409aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 --2- 172.21.15.46:0/21315365 >> v2:172.21.15.82:3300/0 conn(0x7f26e409a110 0x7f26e409aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 >> v1:172.21.15.82:6789/0 conn(0x7f26e409a610 legacy=0x7f26e4099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40cc3d0 con 0x7f26e415ef20 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f26d4001680 con 0x7f26e415ef20 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 >> v1:172.21.15.46:6789/0 conn(0x7f26e415ef20 legacy=0x7f26e40cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26c8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26c8004810 0x7f26c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26c80051d0 0x7f26c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40cddb0 con 0x7f26c8003a10 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c800c0d0 con 0x7f26c80051d0 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c800c2d0 con 0x7f26c8004810 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e3400640 1 --2- 172.21.15.46:0/21315365 >> v2:172.21.15.82:3300/0 conn(0x7f26e409a110 0x7f26e409aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:54.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e2a00640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26c8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e2000640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26c8004810 0x7f26c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e3400640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26c80051d0 0x7f26c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e2a00640 1 -- 172.21.15.46:0/21315365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26c8004810 msgr2=0x7f26c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e2a00640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26c8004810 0x7f26c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e2a00640 1 -- 172.21.15.46:0/21315365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26c80051d0 msgr2=0x7f26c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e2a00640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26c80051d0 0x7f26c800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e2a00640 1 -- 172.21.15.46:0/21315365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8003530 con 0x7f26c8003a10 2024-04-25T08:42:54.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.487+0000 7f26e3400640 1 -- 172.21.15.46:0/21315365 reap_dead start 2024-04-25T08:42:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e2a00640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26c8003e30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26d400c190 tx=0x7f26d401cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb88b5bbaa8689e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4007c70 con 0x7f26c8003a10 2024-04-25T08:42:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d40a9070 con 0x7f26c8003a10 2024-04-25T08:42:54.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e1600640 1 -- 172.21.15.46:0/21315365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4005da0 con 0x7f26c8003a10 2024-04-25T08:42:54.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e9600640 1 -- 172.21.15.46:0/21315365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 msgr2=0x7f26c8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:54.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e9600640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26c8003e30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26d400c190 tx=0x7f26d401cb30 comp rx=0 tx=0).stop 2024-04-25T08:42:54.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e9600640 1 -- 172.21.15.46:0/21315365 shutdown_connections 2024-04-25T08:42:54.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e9600640 1 --2- 172.21.15.46:0/21315365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26c8003e30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e9600640 1 -- 172.21.15.46:0/21315365 >> 172.21.15.46:0/21315365 conn(0x7f26e40b3120 msgr2=0x7f26e40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:54.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e9600640 1 -- 172.21.15.46:0/21315365 shutdown_connections 2024-04-25T08:42:54.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.491+0000 7f26e9600640 1 -- 172.21.15.46:0/21315365 wait complete. 2024-04-25T08:42:54.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 Processor -- start 2024-04-25T08:42:54.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 -- start start 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26e4149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e41497a0 0x7f26e4158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e4159260 0x7f26e406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4099e10 con 0x7f26c8003a10 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e409ae00 con 0x7f26e41497a0 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4099f90 con 0x7f26e4159260 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26e4149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26e4149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39314/0 (socket says 172.21.15.46:39314) 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2a00640 1 -- 172.21.15.46:0/2842078299 learned_addr learned my addr 172.21.15.46:0/2842078299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:54.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2000640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e41497a0 0x7f26e4158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e3400640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e4159260 0x7f26e406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2a00640 1 -- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e4159260 msgr2=0x7f26e406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2a00640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e4159260 0x7f26e406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2a00640 1 -- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e41497a0 msgr2=0x7f26e4158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2a00640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e41497a0 0x7f26e4158d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2a00640 1 -- 172.21.15.46:0/2842078299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e4159bf0 con 0x7f26c8003a10 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e3400640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e4159260 0x7f26e406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e2a00640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26e4149260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f26d401c960 tx=0x7f26c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26daa00640 1 -- 172.21.15.46:0/2842078299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4007140 con 0x7f26c8003a10 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26daa00640 1 -- 172.21.15.46:0/2842078299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d40072e0 con 0x7f26c8003a10 2024-04-25T08:42:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e4159ed0 con 0x7f26c8003a10 2024-04-25T08:42:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26daa00640 1 -- 172.21.15.46:0/2842078299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4007690 con 0x7f26c8003a10 2024-04-25T08:42:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.495+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e415a490 con 0x7f26c8003a10 2024-04-25T08:42:54.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.499+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e4069080 con 0x7f26c8003a10 2024-04-25T08:42:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.499+0000 7f26daa00640 1 -- 172.21.15.46:0/2842078299 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26d40047f0 con 0x7f26c8003a10 2024-04-25T08:42:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.499+0000 7f26daa00640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c4066f90 0x7f26c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.499+0000 7f26daa00640 1 -- 172.21.15.46:0/2842078299 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f26d411eea0 con 0x7f26c8003a10 2024-04-25T08:42:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.499+0000 7f26e2000640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c4066f90 0x7f26c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:54.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.499+0000 7f26e2000640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c4066f90 0x7f26c4069450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26e41fa840 tx=0x7f26d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:54.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.503+0000 7f26daa00640 1 -- 172.21.15.46:0/2842078299 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26d4016870 con 0x7f26c8003a10 2024-04-25T08:42:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.703+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f26e40b3e60 con 0x7f26c8003a10 2024-04-25T08:42:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.703+0000 7f26daa00640 1 -- 172.21.15.46:0/2842078299 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f26d40efc00 con 0x7f26c8003a10 2024-04-25T08:42:54.697 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c4066f90 msgr2=0x7f26c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c4066f90 0x7f26c4069450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26e41fa840 tx=0x7f26d0009040 comp rx=0 tx=0).stop 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 msgr2=0x7f26e4149260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26e4149260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f26d401c960 tx=0x7f26c8005840 comp rx=0 tx=0).stop 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 shutdown_connections 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c4066f90 0x7f26c4069450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e4159260 0x7f26e406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e41497a0 0x7f26e4158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 --2- 172.21.15.46:0/2842078299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8003a10 0x7f26e4149260 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 >> 172.21.15.46:0/2842078299 conn(0x7f26e40b3120 msgr2=0x7f26e406f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 shutdown_connections 2024-04-25T08:42:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:54.707+0000 7f26e9600640 1 -- 172.21.15.46:0/2842078299 wait complete. 2024-04-25T08:42:54.718 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:42:54.718 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:42:54.718 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:42:54.718 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T08:42:54.718 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:42:54.718 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:42:54.718 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:42:54.718 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:42:54.718 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:42:54.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.719 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:54.728 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42428sec preferred_lft 42428sec 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:42:54.729 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:42:54.730 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:54.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.732 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:42:54.792 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:42:54.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.794 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:42:54.847 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:42:54.847 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:42:54.847 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:42:54.847 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:42:54.847 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:42:54.847 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:42:54.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.847 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:54.847 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:54.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:42:54.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:42:54.848 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:42:54.848 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:54.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.939 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:42:54.948 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:42:54.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:54.949 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-04-25T08:42:55.001 INFO:teuthology.orchestra.run.smithi046.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T08:42:55.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.003 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:55.003 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:55.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:42:55.003 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T08:42:55.003 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:55.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:42:55.146 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:55.146 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:42:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 8a:4d:78:a8:c8:df brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:42:55.146 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:42:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::884d:78ff:fea8:c8df/64 scope link 2024-04-25T08:42:55.147 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:55.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 8a:4d:78:a8:c8:df brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::884d:78ff:fea8:c8df/64 scope link 2024-04-25T08:42:55.214 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:55.216 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T08:42:55.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.216 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:55.216 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:55.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T08:42:55.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:42:55.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T08:42:55.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T08:42:55.216 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T08:42:55.216 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:55.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.351 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:42:55.352 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:42:55.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.1 up 2024-04-25T08:42:55.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:42:55.352 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:42:55.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:55.391 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T08:42:55.446 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:42:55.446 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:55.446 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:42:55.501 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:42:55.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.501 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T08:42:55.595 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T08:42:55.595 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T08:42:55.595 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:42:55.610 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T08:42:55.610 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T08:42:55.611 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:42:55.611 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:42:55.611 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:42:55.619 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T08:42:55.620 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T08:42:55.620 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:42:55.620 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:42:55.620 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:42:55.620 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:42:55.620 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:42:55.620 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:42:55.623 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:42:55.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:42:55.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.643 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:55.697 INFO:teuthology.orchestra.run.smithi046.stdout:client4807 2024-04-25T08:42:55.697 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:42:55.698 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:55.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.746 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:55.805 INFO:teuthology.orchestra.run.smithi046.stdout:client4807 2024-04-25T08:42:55.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.805 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:55.857 INFO:teuthology.orchestra.run.smithi046.stdout:client4807 2024-04-25T08:42:55.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:55.857 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ca516a0-1b6e-4031-bb83-1b30eee026c8.client4807/status') 2024-04-25T08:42:55.917 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.4807 (3)172.21.15.46:0/89977370 2024-04-25T08:42:55.917 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-04-25T08:42:55.917 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-04-25T08:42:55.917 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-04-25T08:42:55.917 INFO:teuthology.orchestra.run.smithi046.stderr:68 bytes copied, 6.7606e-05 s, 1.0 MB/s 2024-04-25T08:42:55.920 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T08:42:55.920 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:42:55.920 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:42:55.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:42:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1aef200640 1 Processor -- start 2024-04-25T08:42:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1aef200640 1 -- start start 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1aef200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ae815daf0 0x7f1ae80d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1aef200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ae80d1e20 0x7f1ae80ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1aef200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae80b7020 con 0x7f1ae815d710 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1aef200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae80b72d0 con 0x7f1ae815daf0 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1aef200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae80b7590 con 0x7f1ae80d1e20 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1ae7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1ae815d710 0x7f1ae80b6910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33396/0 (socket says 172.21.15.46:33396) 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1ae7e00640 1 -- 172.21.15.46:0/682380493 learned_addr learned my addr 172.21.15.46:0/682380493 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1adf400640 1 --2- 172.21.15.46:0/682380493 >> v2:172.21.15.46:3300/0 conn(0x7f1ae815daf0 0x7f1ae80d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1aed200640 1 --2- 172.21.15.46:0/682380493 >> v2:172.21.15.82:3300/0 conn(0x7f1ae80d1e20 0x7f1ae80ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1ae7400640 1 -- 172.21.15.46:0/682380493 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270075813 0 0) 0x7f1ae80b7020 con 0x7f1ae815d710 2024-04-25T08:42:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1ae7400640 1 -- 172.21.15.46:0/682380493 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1acc003530 con 0x7f1ae815d710 2024-04-25T08:42:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1adf400640 1 -- 172.21.15.46:0/682380493 >> v2:172.21.15.82:3300/0 conn(0x7f1ae80d1e20 msgr2=0x7f1ae80ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1adf400640 1 --2- 172.21.15.46:0/682380493 >> v2:172.21.15.82:3300/0 conn(0x7f1ae80d1e20 0x7f1ae80ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1adf400640 1 -- 172.21.15.46:0/682380493 >> v1:172.21.15.46:6789/0 conn(0x7f1ae815d710 legacy=0x7f1ae80b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1adf400640 1 -- 172.21.15.46:0/682380493 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80b7870 con 0x7f1ae815daf0 2024-04-25T08:42:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.111+0000 7f1aed200640 1 --2- 172.21.15.46:0/682380493 >> v2:172.21.15.82:3300/0 conn(0x7f1ae80d1e20 0x7f1ae80ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1adf400640 1 --2- 172.21.15.46:0/682380493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 0x7f1ae80d18e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00097c0 tx=0x7f1ad001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb3b10a5f8d5178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1ae7400640 1 -- 172.21.15.46:0/682380493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00ab430 con 0x7f1ae815daf0 2024-04-25T08:42:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1ae7400640 1 -- 172.21.15.46:0/682380493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad00b0070 con 0x7f1ae815daf0 2024-04-25T08:42:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1ae7400640 1 -- 172.21.15.46:0/682380493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00ab7f0 con 0x7f1ae815daf0 2024-04-25T08:42:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1aef200640 1 -- 172.21.15.46:0/682380493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 msgr2=0x7f1ae80d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1aef200640 1 --2- 172.21.15.46:0/682380493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 0x7f1ae80d18e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00097c0 tx=0x7f1ad001c430 comp rx=0 tx=0).stop 2024-04-25T08:42:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1aef200640 1 -- 172.21.15.46:0/682380493 shutdown_connections 2024-04-25T08:42:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1aef200640 1 --2- 172.21.15.46:0/682380493 >> v2:172.21.15.82:3300/0 conn(0x7f1ae80d1e20 0x7f1ae80ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1aef200640 1 --2- 172.21.15.46:0/682380493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 0x7f1ae80d18e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1aef200640 1 -- 172.21.15.46:0/682380493 >> 172.21.15.46:0/682380493 conn(0x7f1ae8072770 msgr2=0x7f1ae8072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1aef200640 1 -- 172.21.15.46:0/682380493 shutdown_connections 2024-04-25T08:42:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1aef200640 1 -- 172.21.15.46:0/682380493 wait complete. 2024-04-25T08:42:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.115+0000 7f1aef200640 1 Processor -- start 2024-04-25T08:42:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 -- start start 2024-04-25T08:42:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae815d710 0x7f1ae80c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 0x7f1ae80d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ae8159bf0 0x7f1ae80d3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80ceea0 con 0x7f1ae815daf0 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae80b8d70 con 0x7f1ae815d710 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae816a620 con 0x7f1ae8159bf0 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1adf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 0x7f1ae80d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1adf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 0x7f1ae80d2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39324/0 (socket says 172.21.15.46:39324) 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1adf400640 1 -- 172.21.15.46:0/1539353633 learned_addr learned my addr 172.21.15.46:0/1539353633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1ae7e00640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae815d710 0x7f1ae80c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aed200640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ae8159bf0 0x7f1ae80d3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1adf400640 1 -- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ae8159bf0 msgr2=0x7f1ae80d3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1adf400640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ae8159bf0 0x7f1ae80d3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1adf400640 1 -- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae815d710 msgr2=0x7f1ae80c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1adf400640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae815d710 0x7f1ae80c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1adf400640 1 -- 172.21.15.46:0/1539353633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80cc160 con 0x7f1ae815daf0 2024-04-25T08:42:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aed200640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ae8159bf0 0x7f1ae80d3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1adf400640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 0x7f1ae80d2c20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0019040 tx=0x7f1ad001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1ae4c00640 1 -- 172.21.15.46:0/1539353633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00bf610 con 0x7f1ae815daf0 2024-04-25T08:42:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1ae4c00640 1 -- 172.21.15.46:0/1539353633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad00bf7d0 con 0x7f1ae815daf0 2024-04-25T08:42:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae80cc440 con 0x7f1ae815daf0 2024-04-25T08:42:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1ae4c00640 1 -- 172.21.15.46:0/1539353633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00bfbc0 con 0x7f1ae815daf0 2024-04-25T08:42:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae80cca00 con 0x7f1ae815daf0 2024-04-25T08:42:56.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.119+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae8069080 con 0x7f1ae815daf0 2024-04-25T08:42:56.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.123+0000 7f1ae4c00640 1 -- 172.21.15.46:0/1539353633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ad00bd020 con 0x7f1ae815daf0 2024-04-25T08:42:56.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.123+0000 7f1ae4c00640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ac4066f60 0x7f1ac4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:56.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.123+0000 7f1ae4c00640 1 -- 172.21.15.46:0/1539353633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1ad00a8030 con 0x7f1ae815daf0 2024-04-25T08:42:56.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.123+0000 7f1ae7e00640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ac4066f60 0x7f1ac4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:56.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.123+0000 7f1ae7e00640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ac4066f60 0x7f1ac4069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8007fd0 tx=0x7f1ad8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.127+0000 7f1ae4c00640 1 -- 172.21.15.46:0/1539353633 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ad00c4060 con 0x7f1ae815daf0 2024-04-25T08:42:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.323+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ae80680e0 con 0x7f1ae815daf0 2024-04-25T08:42:56.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.323+0000 7f1ae4c00640 1 -- 172.21.15.46:0/1539353633 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1ad0016050 con 0x7f1ae815daf0 2024-04-25T08:42:56.318 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ac4066f60 msgr2=0x7f1ac4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ac4066f60 0x7f1ac4069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8007fd0 tx=0x7f1ad8009040 comp rx=0 tx=0).stop 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 msgr2=0x7f1ae80d2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 0x7f1ae80d2c20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0019040 tx=0x7f1ad001ac70 comp rx=0 tx=0).stop 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 shutdown_connections 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ac4066f60 0x7f1ac4069420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ae8159bf0 0x7f1ae80d3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ae815daf0 0x7f1ae80d2c20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 --2- 172.21.15.46:0/1539353633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae815d710 0x7f1ae80c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:56.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 >> 172.21.15.46:0/1539353633 conn(0x7f1ae8072770 msgr2=0x7f1ae8157790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 shutdown_connections 2024-04-25T08:42:56.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:56.327+0000 7f1aef200640 1 -- 172.21.15.46:0/1539353633 wait complete. 2024-04-25T08:42:56.339 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:42:56.339 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:42:56.339 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T08:42:56.339 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T08:42:56.339 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:42:56.340 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:42:56.340 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:42:56.340 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:42:56.340 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:42:56.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:56.340 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:56.350 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42407sec preferred_lft 42407sec 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:4b:8a:05:f0 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:42:56.351 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:56.352 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:42:56.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:56.352 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T08:42:56.352 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T08:42:56.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:42:56.353 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-04-25T08:42:56.353 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-04-25T08:42:56.353 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:42:56.353 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T08:42:56.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:56.466 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:42:56.485 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T08:42:56.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:56.487 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T08:42:56.788 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T08:42:56.788 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:42:56.788 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T08:42:56.788 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:42:56.788 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T08:42:56.788 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:42:56.788 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:42:56.788 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:42:56.789 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:42:56.789 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:42:56.789 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:42:56.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:56.789 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T08:42:56.789 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T08:42:56.789 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:42:56.789 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:42:56.789 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:42:56.789 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T08:42:56.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:56.841 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T08:42:56.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:56.851 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-04-25T08:42:56.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:56.905 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T08:42:56.905 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T08:42:56.905 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:42:56.905 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T08:42:56.905 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T08:42:56.991 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T08:42:56.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:56.991 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T08:42:56.992 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T08:42:56.992 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T08:42:56.992 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:42:56.992 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T08:42:56.992 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T08:42:56.992 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T08:42:56.992 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T08:42:57.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:57.157 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T08:42:57.157 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T08:42:57.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T08:42:57.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:42:57.157 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T08:42:57.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:57.196 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T08:42:57.245 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:42:57.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:57.245 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:42:57.300 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:42:57.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:57.300 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T08:42:57.388 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T08:42:57.388 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T08:42:57.388 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-25T08:42:57.976 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T08:42:57.976 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T08:42:57.976 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-25T08:42:57.977 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:42:57.977 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:42:57.988 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T08:42:57.988 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T08:42:57.988 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-25T08:42:57.988 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:42:57.989 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:42:57.989 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:42:57.989 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:42:57.989 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:42:57.993 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:42:57.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:57.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:42:58.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.012 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:58.080 INFO:teuthology.orchestra.run.smithi082.stdout:client4667 2024-04-25T08:42:58.081 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:42:58.081 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:58.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.132 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:58.183 INFO:teuthology.orchestra.run.smithi082.stdout:client4667 2024-04-25T08:42:58.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.183 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:58.235 INFO:teuthology.orchestra.run.smithi082.stdout:client4667 2024-04-25T08:42:58.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.235 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ca516a0-1b6e-4031-bb83-1b30eee026c8.client4667/status') 2024-04-25T08:42:58.299 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.4667 (3)192.168.0.1:0/1346783609 2024-04-25T08:42:58.299 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-04-25T08:42:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-04-25T08:42:58.300 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-04-25T08:42:58.300 INFO:teuthology.orchestra.run.smithi082.stderr:69 bytes copied, 5.3496e-05 s, 1.3 MB/s 2024-04-25T08:42:58.301 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T08:42:58.301 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:42:58.301 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:42:58.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:42:58.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.459+0000 7f847f600640 1 Processor -- start 2024-04-25T08:42:58.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.459+0000 7f847f600640 1 -- start start 2024-04-25T08:42:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f847f600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 0x7f847806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f847f600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f84780aed10 0x7f847806f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f847f600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847806d5e0 con 0x7f84780cc9e0 2024-04-25T08:42:58.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f847f600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847815ab30 con 0x7f84780aed10 2024-04-25T08:42:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f847f600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84780ce4f0 con 0x7f847815a750 2024-04-25T08:42:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f8477e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f84780aed10 0x7f847806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f8477e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f84780aed10 0x7f847806f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42732/0 (socket says 172.21.15.46:42732) 2024-04-25T08:42:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f8477e00640 1 -- 172.21.15.46:0/303005801 learned_addr learned my addr 172.21.15.46:0/303005801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f847cc00640 1 --2- 172.21.15.46:0/303005801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 0x7f847806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f8477e00640 1 -- 172.21.15.46:0/303005801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 msgr2=0x7f847806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f8477e00640 1 --2- 172.21.15.46:0/303005801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 0x7f847806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f8477e00640 1 -- 172.21.15.46:0/303005801 >> v1:172.21.15.82:6789/0 conn(0x7f84780cc9e0 legacy=0x7f84780b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:58.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.463+0000 7f8477e00640 1 -- 172.21.15.46:0/303005801 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780ce720 con 0x7f84780aed10 2024-04-25T08:42:58.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f8477e00640 1 --2- 172.21.15.46:0/303005801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 0x7f847806f150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f846400bb10 tx=0x7f846401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85eda99ad47f6b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f8477400640 1 -- 172.21.15.46:0/303005801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640b4070 con 0x7f84780aed10 2024-04-25T08:42:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f8477400640 1 -- 172.21.15.46:0/303005801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84640a7bf0 con 0x7f84780aed10 2024-04-25T08:42:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f8477400640 1 -- 172.21.15.46:0/303005801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640af440 con 0x7f84780aed10 2024-04-25T08:42:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 -- 172.21.15.46:0/303005801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 msgr2=0x7f847806f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 --2- 172.21.15.46:0/303005801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 0x7f847806f150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f846400bb10 tx=0x7f846401ec30 comp rx=0 tx=0).stop 2024-04-25T08:42:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 -- 172.21.15.46:0/303005801 shutdown_connections 2024-04-25T08:42:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 --2- 172.21.15.46:0/303005801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 0x7f847806f150 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 --2- 172.21.15.46:0/303005801 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 0x7f847806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 -- 172.21.15.46:0/303005801 >> 172.21.15.46:0/303005801 conn(0x7f84780b12a0 msgr2=0x7f84780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 -- 172.21.15.46:0/303005801 shutdown_connections 2024-04-25T08:42:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 -- 172.21.15.46:0/303005801 wait complete. 2024-04-25T08:42:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 Processor -- start 2024-04-25T08:42:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.467+0000 7f847f600640 1 -- start start 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 0x7f8478151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847f600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84780cc9e0 0x7f8478152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847f600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 0x7f847814b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847806db00 con 0x7f84780aed10 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847f600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847806d270 con 0x7f84780cc9e0 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847f600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847806dc80 con 0x7f847815a750 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 0x7f8478151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 0x7f8478151080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42742/0 (socket says 172.21.15.46:42742) 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847cc00640 1 -- 172.21.15.46:0/2439463696 learned_addr learned my addr 172.21.15.46:0/2439463696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:42:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f8477e00640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84780cc9e0 0x7f8478152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847d600640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 0x7f847814b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847cc00640 1 -- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 msgr2=0x7f847814b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847cc00640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 0x7f847814b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847cc00640 1 -- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84780cc9e0 msgr2=0x7f8478152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847cc00640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84780cc9e0 0x7f8478152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847cc00640 1 -- 172.21.15.46:0/2439463696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847815c240 con 0x7f84780aed10 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847cc00640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 0x7f8478151080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f846c00e870 tx=0x7f846c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f8474c00640 1 -- 172.21.15.46:0/2439463696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c017070 con 0x7f84780aed10 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f8474c00640 1 -- 172.21.15.46:0/2439463696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c00c940 con 0x7f84780aed10 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847815c520 con 0x7f84780aed10 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f8474c00640 1 -- 172.21.15.46:0/2439463696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c00ccf0 con 0x7f84780aed10 2024-04-25T08:42:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8478158ba0 con 0x7f84780aed10 2024-04-25T08:42:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8478069080 con 0x7f84780aed10 2024-04-25T08:42:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.471+0000 7f8474c00640 1 -- 172.21.15.46:0/2439463696 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f846c01a070 con 0x7f84780aed10 2024-04-25T08:42:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.475+0000 7f8474c00640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8458066f60 0x7f8458069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.475+0000 7f8474c00640 1 -- 172.21.15.46:0/2439463696 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f846c0855d0 con 0x7f84780aed10 2024-04-25T08:42:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.475+0000 7f8477e00640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8458066f60 0x7f8458069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.475+0000 7f8477e00640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8458066f60 0x7f8458069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f84640b1000 tx=0x7f8464008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:58.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.479+0000 7f8474c00640 1 -- 172.21.15.46:0/2439463696 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f846c055bb0 con 0x7f84780aed10 2024-04-25T08:42:58.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8478164f70 con 0x7f84780aed10 2024-04-25T08:42:58.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f8474c00640 1 -- 172.21.15.46:0/2439463696 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f846c059c50 con 0x7f84780aed10 2024-04-25T08:42:58.672 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8458066f60 msgr2=0x7f8458069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8458066f60 0x7f8458069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f84640b1000 tx=0x7f8464008040 comp rx=0 tx=0).stop 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 msgr2=0x7f8478151080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 0x7f8478151080 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f846c00e870 tx=0x7f846c00ec70 comp rx=0 tx=0).stop 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 shutdown_connections 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8458066f60 0x7f8458069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f847815a750 0x7f847814b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84780cc9e0 0x7f8478152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 --2- 172.21.15.46:0/2439463696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84780aed10 0x7f8478151080 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:58.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 >> 172.21.15.46:0/2439463696 conn(0x7f84780b12a0 msgr2=0x7f84781506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:58.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 shutdown_connections 2024-04-25T08:42:58.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:42:58.679+0000 7f847f600640 1 -- 172.21.15.46:0/2439463696 wait complete. 2024-04-25T08:42:58.693 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:42:58.693 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:42:58.693 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T08:42:58.693 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T08:42:58.693 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:42:58.693 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:42:58.693 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:42:58.693 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:42:58.693 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:42:58.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.694 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T08:42:58.703 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42405sec preferred_lft 42405sec 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:4b:8a:05:f0 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:58.704 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-04-25T08:42:58.705 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:58.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.706 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:42:58.766 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T08:42:58.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.768 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T08:42:58.820 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T08:42:58.820 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:42:58.820 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T08:42:58.820 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:42:58.820 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T08:42:58.820 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:42:58.820 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:42:58.820 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:42:58.820 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:42:58.821 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:42:58.821 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:42:58.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.821 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T08:42:58.821 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T08:42:58.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:42:58.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:42:58.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:42:58.821 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T08:42:58.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.910 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T08:42:58.919 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T08:42:58.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.919 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-04-25T08:42:58.969 INFO:teuthology.orchestra.run.smithi082.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T08:42:58.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:58.969 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T08:42:58.969 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T08:42:58.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:42:58.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T08:42:58.969 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T08:42:59.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.054 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 3a:04:5a:b5:f7:29 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::3804:5aff:feb5:f729/64 scope link 2024-04-25T08:42:59.120 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:59.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.122 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:42:59.184 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:59.184 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:59.184 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:59.184 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:59.184 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:59.184 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:59.185 INFO:teuthology.orchestra.run.smithi082.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:42:59.185 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 3a:04:5a:b5:f7:29 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:42:59.185 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:42:59.185 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:59.185 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::3804:5aff:feb5:f729/64 scope link 2024-04-25T08:42:59.185 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:59.186 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T08:42:59.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.187 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T08:42:59.187 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T08:42:59.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T08:42:59.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:42:59.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T08:42:59.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T08:42:59.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T08:42:59.187 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T08:42:59.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.324 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T08:42:59.324 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T08:42:59.324 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.1 up 2024-04-25T08:42:59.324 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:42:59.324 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T08:42:59.362 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:59.363 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T08:42:59.417 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:42:59.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:59.417 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:42:59.472 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:42:59.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.472 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T08:42:59.560 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T08:42:59.560 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T08:42:59.560 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-25T08:42:59.575 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T08:42:59.575 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T08:42:59.575 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-25T08:42:59.575 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:42:59.575 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:42:59.585 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T08:42:59.585 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T08:42:59.585 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-04-25T08:42:59.585 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:42:59.585 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:42:59.585 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:42:59.585 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:42:59.585 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:42:59.589 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:42:59.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.589 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:42:59.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.655 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:59.707 INFO:teuthology.orchestra.run.smithi082.stdout:client4842 2024-04-25T08:42:59.707 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:42:59.708 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:59.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.756 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:59.807 INFO:teuthology.orchestra.run.smithi082.stdout:client4842 2024-04-25T08:42:59.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.807 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:42:59.859 INFO:teuthology.orchestra.run.smithi082.stdout:client4842 2024-04-25T08:42:59.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:59.859 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9ca516a0-1b6e-4031-bb83-1b30eee026c8.client4842/status') 2024-04-25T08:42:59.923 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.4842 (3)172.21.15.82:0/3277758470 2024-04-25T08:42:59.924 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-04-25T08:42:59.924 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-04-25T08:42:59.924 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-04-25T08:42:59.924 INFO:teuthology.orchestra.run.smithi082.stderr:70 bytes copied, 4.8925e-05 s, 1.4 MB/s 2024-04-25T08:42:59.926 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T08:42:59.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.099+0000 7f0ba5800640 1 Processor -- start 2024-04-25T08:43:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.099+0000 7f0ba5800640 1 -- start start 2024-04-25T08:43:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0ba5800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0ba015a750 0x7f0ba006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0ba5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ba00aed10 0x7f0ba006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0ba5800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba006d5e0 con 0x7f0ba00cc9e0 2024-04-25T08:43:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0ba5800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba015ab30 con 0x7f0ba00aed10 2024-04-25T08:43:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0ba5800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba00ce4f0 con 0x7f0ba015a750 2024-04-25T08:43:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0b9f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0ba00cc9e0 0x7f0ba00b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48338/0 (socket says 172.21.15.46:48338) 2024-04-25T08:43:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0b96000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ba00aed10 0x7f0ba006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0b9f400640 1 -- 172.21.15.46:0/2443015331 learned_addr learned my addr 172.21.15.46:0/2443015331 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0b9ea00640 1 --2- 172.21.15.46:0/2443015331 >> v2:172.21.15.82:3300/0 conn(0x7f0ba015a750 0x7f0ba006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0b9e000640 1 -- 172.21.15.46:0/2443015331 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743465533 0 0) 0x7f0ba006d5e0 con 0x7f0ba00cc9e0 2024-04-25T08:43:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0b96000640 1 -- 172.21.15.46:0/2443015331 >> v2:172.21.15.82:3300/0 conn(0x7f0ba015a750 msgr2=0x7f0ba006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0b96000640 1 --2- 172.21.15.46:0/2443015331 >> v2:172.21.15.82:3300/0 conn(0x7f0ba015a750 0x7f0ba006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0b96000640 1 -- 172.21.15.46:0/2443015331 >> v1:172.21.15.46:6789/0 conn(0x7f0ba00cc9e0 legacy=0x7f0ba00b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.103+0000 7f0b96000640 1 -- 172.21.15.46:0/2443015331 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba00ce720 con 0x7f0ba00aed10 2024-04-25T08:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0b96000640 1 --2- 172.21.15.46:0/2443015331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 0x7f0ba006f150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ba004ffd0 tx=0x7f0b8801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f261e7f42d730f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0b9e000640 1 -- 172.21.15.46:0/2443015331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b880b0070 con 0x7f0ba00aed10 2024-04-25T08:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0b9e000640 1 -- 172.21.15.46:0/2443015331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b880ab600 con 0x7f0ba00aed10 2024-04-25T08:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0b9e000640 1 -- 172.21.15.46:0/2443015331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b880ab9d0 con 0x7f0ba00aed10 2024-04-25T08:43:00.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 -- 172.21.15.46:0/2443015331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 msgr2=0x7f0ba006f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 --2- 172.21.15.46:0/2443015331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 0x7f0ba006f150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ba004ffd0 tx=0x7f0b8801c480 comp rx=0 tx=0).stop 2024-04-25T08:43:00.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 -- 172.21.15.46:0/2443015331 shutdown_connections 2024-04-25T08:43:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 --2- 172.21.15.46:0/2443015331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 0x7f0ba006f150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 --2- 172.21.15.46:0/2443015331 >> v2:172.21.15.82:3300/0 conn(0x7f0ba015a750 0x7f0ba006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 -- 172.21.15.46:0/2443015331 >> 172.21.15.46:0/2443015331 conn(0x7f0ba00b12a0 msgr2=0x7f0ba00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 -- 172.21.15.46:0/2443015331 shutdown_connections 2024-04-25T08:43:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 -- 172.21.15.46:0/2443015331 wait complete. 2024-04-25T08:43:00.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 Processor -- start 2024-04-25T08:43:00.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.107+0000 7f0ba5800640 1 -- start start 2024-04-25T08:43:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0ba5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 0x7f0ba0156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0ba5800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba00cc9e0 0x7f0ba0157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0ba5800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ba015a750 0x7f0ba0148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0ba5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba0167230 con 0x7f0ba00aed10 2024-04-25T08:43:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0ba5800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba006d270 con 0x7f0ba00cc9e0 2024-04-25T08:43:00.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0ba5800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba01673b0 con 0x7f0ba015a750 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 0x7f0ba0156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 0x7f0ba0156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42748/0 (socket says 172.21.15.46:42748) 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9ea00640 1 -- 172.21.15.46:0/1710363582 learned_addr learned my addr 172.21.15.46:0/1710363582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b96000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba00cc9e0 0x7f0ba0157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9f400640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ba015a750 0x7f0ba0148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9ea00640 1 -- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ba015a750 msgr2=0x7f0ba0148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9ea00640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ba015a750 0x7f0ba0148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9ea00640 1 -- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba00cc9e0 msgr2=0x7f0ba0157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9ea00640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba00cc9e0 0x7f0ba0157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9ea00640 1 -- 172.21.15.46:0/1710363582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba0158290 con 0x7f0ba00aed10 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9f400640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ba015a750 0x7f0ba0148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b9ea00640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 0x7f0ba0156570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000e8a0 tx=0x7f0b9000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b97400640 1 -- 172.21.15.46:0/1710363582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b90017070 con 0x7f0ba00aed10 2024-04-25T08:43:00.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b97400640 1 -- 172.21.15.46:0/1710363582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9000c940 con 0x7f0ba00aed10 2024-04-25T08:43:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0b97400640 1 -- 172.21.15.46:0/1710363582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000ccf0 con 0x7f0ba00aed10 2024-04-25T08:43:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba01595d0 con 0x7f0ba00aed10 2024-04-25T08:43:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba0158810 con 0x7f0ba00aed10 2024-04-25T08:43:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.111+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba0069080 con 0x7f0ba00aed10 2024-04-25T08:43:00.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.115+0000 7f0b97400640 1 -- 172.21.15.46:0/1710363582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b9001a070 con 0x7f0ba00aed10 2024-04-25T08:43:00.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.115+0000 7f0b97400640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b7c066f60 0x7f0b7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.115+0000 7f0b97400640 1 -- 172.21.15.46:0/1710363582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b90085200 con 0x7f0ba00aed10 2024-04-25T08:43:00.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.115+0000 7f0b96000640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b7c066f60 0x7f0b7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.115+0000 7f0b96000640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b7c066f60 0x7f0b7c069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b880ad000 tx=0x7f0b880a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:00.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.119+0000 7f0b97400640 1 -- 172.21.15.46:0/1710363582 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b90055870 con 0x7f0ba00aed10 2024-04-25T08:43:00.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.279+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ba0157440 con 0x7f0ba00aed10 2024-04-25T08:43:00.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0b97400640 1 -- 172.21.15.46:0/1710363582 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f0b90059910 con 0x7f0ba00aed10 2024-04-25T08:43:00.275 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:00.275 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:42:27.380179+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b7c066f60 msgr2=0x7f0b7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b7c066f60 0x7f0b7c069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b880ad000 tx=0x7f0b880a6000 comp rx=0 tx=0).stop 2024-04-25T08:43:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 msgr2=0x7f0ba0156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 0x7f0ba0156570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0b9000e8a0 tx=0x7f0b9000eca0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 shutdown_connections 2024-04-25T08:43:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b7c066f60 0x7f0b7c069420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ba015a750 0x7f0ba0148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba00cc9e0 0x7f0ba0157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 --2- 172.21.15.46:0/1710363582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ba00aed10 0x7f0ba0156570 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 >> 172.21.15.46:0/1710363582 conn(0x7f0ba00b12a0 msgr2=0x7f0ba00af690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 shutdown_connections 2024-04-25T08:43:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.283+0000 7f0ba5800640 1 -- 172.21.15.46:0/1710363582 wait complete. 2024-04-25T08:43:00.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:00.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.447+0000 7f3da9200640 1 Processor -- start 2024-04-25T08:43:00.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.447+0000 7f3da9200640 1 -- start start 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da9200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da406f4d0 0x7f3da40cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da9200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da409a780 con 0x7f3da415dd50 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da9200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da409aa60 con 0x7f3da40d0450 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da9200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40708c0 con 0x7f3da406f4d0 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3d9be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3da415dd50 0x7f3da40cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48348/0 (socket says 172.21.15.46:48348) 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3d9be00640 1 -- 172.21.15.46:0/3950722285 learned_addr learned my addr 172.21.15.46:0/3950722285 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1e00640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da406f4d0 0x7f3da40cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175389384 0 0) 0x7f3da409a780 con 0x7f3da415dd50 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d88003530 con 0x7f3da415dd50 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690417106 0 0) 0x7f3da409aa60 con 0x7f3da40d0450 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da409a780 con 0x7f3da40d0450 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1560164415 0 0) 0x7f3d88003530 con 0x7f3da415dd50 2024-04-25T08:43:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da406f4d0 msgr2=0x7f3da40cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da406f4d0 0x7f3da40cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 >> v1:172.21.15.82:6789/0 conn(0x7f3da40d0450 legacy=0x7f3da40cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da409ad40 con 0x7f3da415dd50 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f3d8c001680 con 0x7f3da415dd50 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1e00640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da406f4d0 0x7f3da40cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 >> v1:172.21.15.46:6789/0 conn(0x7f3da415dd50 legacy=0x7f3da40cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3d88003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d88004810 0x7f3d88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da406f4d0 0x7f3d880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40708c0 con 0x7f3d88003bf0 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8800ba80 con 0x7f3d88004810 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8800bc80 con 0x7f3da406f4d0 2024-04-25T08:43:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1e00640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3d88003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3d9be00640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d88004810 0x7f3d88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da2800640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da406f4d0 0x7f3d880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1e00640 1 -- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da406f4d0 msgr2=0x7f3d880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1e00640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da406f4d0 0x7f3d880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1e00640 1 -- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d88004810 msgr2=0x7f3d88004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1e00640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d88004810 0x7f3d88004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3da1e00640 1 -- 172.21.15.46:0/3950722285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d88003530 con 0x7f3d88003bf0 2024-04-25T08:43:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.451+0000 7f3d9be00640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d88004810 0x7f3d88004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da1e00640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3d88003ff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009c80 tx=0x7f3d9401cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f589786fd261ee5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940b2070 con 0x7f3d88003bf0 2024-04-25T08:43:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d940a5730 con 0x7f3d88003bf0 2024-04-25T08:43:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da1400640 1 -- 172.21.15.46:0/3950722285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d94009880 con 0x7f3d88003bf0 2024-04-25T08:43:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da9200640 1 -- 172.21.15.46:0/3950722285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 msgr2=0x7f3d88003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da9200640 1 --2- 172.21.15.46:0/3950722285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3d88003ff0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009c80 tx=0x7f3d9401cb80 comp rx=0 tx=0).stop 2024-04-25T08:43:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da2800640 1 -- 172.21.15.46:0/3950722285 reap_dead start 2024-04-25T08:43:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da9200640 1 -- 172.21.15.46:0/3950722285 shutdown_connections 2024-04-25T08:43:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da9200640 1 -- 172.21.15.46:0/3950722285 >> 172.21.15.46:0/3950722285 conn(0x7f3da40b3160 msgr2=0x7f3da40b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da9200640 1 -- 172.21.15.46:0/3950722285 shutdown_connections 2024-04-25T08:43:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da9200640 1 -- 172.21.15.46:0/3950722285 wait complete. 2024-04-25T08:43:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da9200640 1 Processor -- start 2024-04-25T08:43:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.455+0000 7f3da9200640 1 -- start start 2024-04-25T08:43:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3da4158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da9200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3da40c7f10 0x7f3da415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da9200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da40c51a0 0x7f3da40c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40cd560 con 0x7f3d88003bf0 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da9200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40cd3e0 con 0x7f3da40c7f10 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da9200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40cd6e0 con 0x7f3da40c51a0 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3da4158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3da4158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42768/0 (socket says 172.21.15.46:42768) 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da1e00640 1 -- 172.21.15.46:0/1656323290 learned_addr learned my addr 172.21.15.46:0/1656323290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3d9be00640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3da40c7f10 0x7f3da415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da2800640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da40c51a0 0x7f3da40c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da1e00640 1 -- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da40c51a0 msgr2=0x7f3da40c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da1e00640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da40c51a0 0x7f3da40c5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da1e00640 1 -- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3da40c7f10 msgr2=0x7f3da415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da1e00640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3da40c7f10 0x7f3da415e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da1e00640 1 -- 172.21.15.46:0/1656323290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da41594b0 con 0x7f3d88003bf0 2024-04-25T08:43:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da2800640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da40c51a0 0x7f3da40c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da1e00640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3da4158940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d9401c9b0 tx=0x7f3d88005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3d9a000640 1 -- 172.21.15.46:0/1656323290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940b2040 con 0x7f3d88003bf0 2024-04-25T08:43:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3d9a000640 1 -- 172.21.15.46:0/1656323290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d94009400 con 0x7f3d88003bf0 2024-04-25T08:43:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da4159790 con 0x7f3d88003bf0 2024-04-25T08:43:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3d9a000640 1 -- 172.21.15.46:0/1656323290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d94009880 con 0x7f3d88003bf0 2024-04-25T08:43:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da4159d50 con 0x7f3d88003bf0 2024-04-25T08:43:00.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.459+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da4069010 con 0x7f3d88003bf0 2024-04-25T08:43:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.463+0000 7f3d9a000640 1 -- 172.21.15.46:0/1656323290 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d940c0020 con 0x7f3d88003bf0 2024-04-25T08:43:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.463+0000 7f3d9a000640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d80066f60 0x7f3d80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.463+0000 7f3d9a000640 1 -- 172.21.15.46:0/1656323290 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3d9411ec50 con 0x7f3d88003bf0 2024-04-25T08:43:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.463+0000 7f3d9be00640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d80066f60 0x7f3d80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.463+0000 7f3d9be00640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d80066f60 0x7f3d80069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0018b0 tx=0x7f3d8c007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.467+0000 7f3d9a000640 1 -- 172.21.15.46:0/1656323290 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d940168c0 con 0x7f3d88003bf0 2024-04-25T08:43:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3da415d580 con 0x7f3d88003bf0 2024-04-25T08:43:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3d9a000640 1 -- 172.21.15.46:0/1656323290 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d940ef350 con 0x7f3d88003bf0 2024-04-25T08:43:00.631 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d80066f60 msgr2=0x7f3d80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d80066f60 0x7f3d80069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0018b0 tx=0x7f3d8c007040 comp rx=0 tx=0).stop 2024-04-25T08:43:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 msgr2=0x7f3da4158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3da4158940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d9401c9b0 tx=0x7f3d88005840 comp rx=0 tx=0).stop 2024-04-25T08:43:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 shutdown_connections 2024-04-25T08:43:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d80066f60 0x7f3d80069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3da40c51a0 0x7f3da40c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3da40c7f10 0x7f3da415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 --2- 172.21.15.46:0/1656323290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d88003bf0 0x7f3da4158940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 >> 172.21.15.46:0/1656323290 conn(0x7f3da40b3160 msgr2=0x7f3da406f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 shutdown_connections 2024-04-25T08:43:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.639+0000 7f3da9200640 1 -- 172.21.15.46:0/1656323290 wait complete. 2024-04-25T08:43:00.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:00.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f7625600640 1 Processor -- start 2024-04-25T08:43:00.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f7625600640 1 -- start start 2024-04-25T08:43:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f7625600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f762006c730 0x7f762006bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f7625600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76200d0390 0x7f76200cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f7625600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762009acb0 con 0x7f762009a410 2024-04-25T08:43:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f7625600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200b3dc0 con 0x7f762006c730 2024-04-25T08:43:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f7625600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200cf680 con 0x7f76200d0390 2024-04-25T08:43:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f761fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f762006c730 0x7f762006bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f761fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f762006c730 0x7f762006bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42780/0 (socket says 172.21.15.46:42780) 2024-04-25T08:43:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f761fe00640 1 -- 172.21.15.46:0/1607363948 learned_addr learned my addr 172.21.15.46:0/1607363948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:00.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f7624c00640 1 --2- 172.21.15.46:0/1607363948 >> v2:172.21.15.82:3300/0 conn(0x7f76200d0390 0x7f76200cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f761fe00640 1 -- 172.21.15.46:0/1607363948 >> v2:172.21.15.82:3300/0 conn(0x7f76200d0390 msgr2=0x7f76200cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f761fe00640 1 --2- 172.21.15.46:0/1607363948 >> v2:172.21.15.82:3300/0 conn(0x7f76200d0390 0x7f76200cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f761fe00640 1 -- 172.21.15.46:0/1607363948 >> v1:172.21.15.82:6789/0 conn(0x7f762009a410 legacy=0x7f762006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f761fe00640 1 -- 172.21.15.46:0/1607363948 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762009a9d0 con 0x7f762006c730 2024-04-25T08:43:00.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.803+0000 7f7624c00640 1 --2- 172.21.15.46:0/1607363948 >> v2:172.21.15.82:3300/0 conn(0x7f76200d0390 0x7f76200cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f761fe00640 1 --2- 172.21.15.46:0/1607363948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f762006c730 0x7f762006bf30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7610009870 tx=0x7f761001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502f5791e7805f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f761f400640 1 -- 172.21.15.46:0/1607363948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76100b1070 con 0x7f762006c730 2024-04-25T08:43:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f761f400640 1 -- 172.21.15.46:0/1607363948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76100ac620 con 0x7f762006c730 2024-04-25T08:43:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f761f400640 1 -- 172.21.15.46:0/1607363948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76100ac9f0 con 0x7f762006c730 2024-04-25T08:43:00.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f7625600640 1 -- 172.21.15.46:0/1607363948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f762006c730 msgr2=0x7f762006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f7625600640 1 --2- 172.21.15.46:0/1607363948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f762006c730 0x7f762006bf30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7610009870 tx=0x7f761001c4b0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f7625600640 1 -- 172.21.15.46:0/1607363948 shutdown_connections 2024-04-25T08:43:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f7625600640 1 --2- 172.21.15.46:0/1607363948 >> v2:172.21.15.82:3300/0 conn(0x7f76200d0390 0x7f76200cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f7625600640 1 --2- 172.21.15.46:0/1607363948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f762006c730 0x7f762006bf30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f7625600640 1 -- 172.21.15.46:0/1607363948 >> 172.21.15.46:0/1607363948 conn(0x7f76200b2f50 msgr2=0x7f76200b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f7625600640 1 -- 172.21.15.46:0/1607363948 shutdown_connections 2024-04-25T08:43:00.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.807+0000 7f7625600640 1 -- 172.21.15.46:0/1607363948 wait complete. 2024-04-25T08:43:00.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 Processor -- start 2024-04-25T08:43:00.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 -- start start 2024-04-25T08:43:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f762006c730 0x7f76201514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f762009a410 0x7f76200af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76200d0390 0x7f76200ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762015dd60 con 0x7f76200d0390 2024-04-25T08:43:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200cc400 con 0x7f762009a410 2024-04-25T08:43:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762016b040 con 0x7f762006c730 2024-04-25T08:43:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7624c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76200d0390 0x7f76200ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7624c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76200d0390 0x7f76200ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42788/0 (socket says 172.21.15.46:42788) 2024-04-25T08:43:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7624c00640 1 -- 172.21.15.46:0/600412166 learned_addr learned my addr 172.21.15.46:0/600412166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f761fe00640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f762006c730 0x7f76201514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7617400640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f762009a410 0x7f76200af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7624c00640 1 -- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f762006c730 msgr2=0x7f76201514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7624c00640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f762006c730 0x7f76201514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7624c00640 1 -- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f762009a410 msgr2=0x7f76200af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7624c00640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f762009a410 0x7f76200af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7624c00640 1 -- 172.21.15.46:0/600412166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76200af890 con 0x7f76200d0390 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7617400640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f762009a410 0x7f76200af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7624c00640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76200d0390 0x7f76200ae110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f761800cd80 tx=0x7f761800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f761cc00640 1 -- 172.21.15.46:0/600412166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800f680 con 0x7f76200d0390 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f761cc00640 1 -- 172.21.15.46:0/600412166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761800f840 con 0x7f76200d0390 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f761cc00640 1 -- 172.21.15.46:0/600412166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800fc30 con 0x7f76200d0390 2024-04-25T08:43:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76200b4b30 con 0x7f76200d0390 2024-04-25T08:43:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76200c5410 con 0x7f76200d0390 2024-04-25T08:43:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.811+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76200b4d00 con 0x7f76200d0390 2024-04-25T08:43:00.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.815+0000 7f761cc00640 1 -- 172.21.15.46:0/600412166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f761800e3e0 con 0x7f76200d0390 2024-04-25T08:43:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.815+0000 7f761cc00640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f75fc066f60 0x7f75fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.815+0000 7f761cc00640 1 -- 172.21.15.46:0/600412166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f76180860f0 con 0x7f76200d0390 2024-04-25T08:43:00.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.815+0000 7f761fe00640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f75fc066f60 0x7f75fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:00.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.815+0000 7f761fe00640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f75fc066f60 0x7f75fc069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f76100ae000 tx=0x7f7610008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:00.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.819+0000 7f761cc00640 1 -- 172.21.15.46:0/600412166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7618014030 con 0x7f76200d0390 2024-04-25T08:43:00.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7620069010 con 0x7f76200d0390 2024-04-25T08:43:00.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f761cc00640 1 -- 172.21.15.46:0/600412166 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7618056cf0 con 0x7f76200d0390 2024-04-25T08:43:00.975 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f75fc066f60 msgr2=0x7f75fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f75fc066f60 0x7f75fc069420 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f76100ae000 tx=0x7f7610008040 comp rx=0 tx=0).stop 2024-04-25T08:43:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76200d0390 msgr2=0x7f76200ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76200d0390 0x7f76200ae110 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f761800cd80 tx=0x7f761800a650 comp rx=0 tx=0).stop 2024-04-25T08:43:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 shutdown_connections 2024-04-25T08:43:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f75fc066f60 0x7f75fc069420 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76200d0390 0x7f76200ae110 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f762009a410 0x7f76200af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 --2- 172.21.15.46:0/600412166 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f762006c730 0x7f76201514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 >> 172.21.15.46:0/600412166 conn(0x7f76200b2f50 msgr2=0x7f76200b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 shutdown_connections 2024-04-25T08:43:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:00.983+0000 7f7625600640 1 -- 172.21.15.46:0/600412166 wait complete. 2024-04-25T08:43:00.997 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:01.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.147+0000 7f172fe00640 1 Processor -- start 2024-04-25T08:43:01.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.147+0000 7f172fe00640 1 -- start start 2024-04-25T08:43:01.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1728071980 0x7f172815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172815eab0 con 0x7f172815db50 2024-04-25T08:43:01.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172fe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172815e7d0 con 0x7f172813d0b0 2024-04-25T08:43:01.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172fe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172813d490 con 0x7f1728071980 2024-04-25T08:43:01.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f172815db50 0x7f172815df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48358/0 (socket says 172.21.15.46:48358) 2024-04-25T08:43:01.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172de00640 1 -- 172.21.15.46:0/2797230990 learned_addr learned my addr 172.21.15.46:0/2797230990 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:01.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172ca00640 1 --2- 172.21.15.46:0/2797230990 >> v2:172.21.15.82:3300/0 conn(0x7f1728071980 0x7f172815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f1727e00640 1 -- 172.21.15.46:0/2797230990 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258059879 0 0) 0x7f172815eab0 con 0x7f172815db50 2024-04-25T08:43:01.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f1727e00640 1 -- 172.21.15.46:0/2797230990 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1710005740 con 0x7f172815db50 2024-04-25T08:43:01.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f1727e00640 1 -- 172.21.15.46:0/2797230990 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143552814 0 0) 0x7f172815e7d0 con 0x7f172813d0b0 2024-04-25T08:43:01.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f1727e00640 1 -- 172.21.15.46:0/2797230990 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f172815eab0 con 0x7f172813d0b0 2024-04-25T08:43:01.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f1727e00640 1 -- 172.21.15.46:0/2797230990 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2978850906 0 0) 0x7f1710005740 con 0x7f172815db50 2024-04-25T08:43:01.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172ca00640 1 -- 172.21.15.46:0/2797230990 >> v1:172.21.15.82:6789/0 conn(0x7f172813d0b0 legacy=0x7f1728071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172ca00640 1 -- 172.21.15.46:0/2797230990 >> v1:172.21.15.46:6789/0 conn(0x7f172815db50 legacy=0x7f172815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.151+0000 7f172ca00640 1 -- 172.21.15.46:0/2797230990 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172815a6e0 con 0x7f1728071980 2024-04-25T08:43:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172ca00640 1 --2- 172.21.15.46:0/2797230990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1728071980 0x7f172815b1c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17140097c0 tx=0x7f171401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6317ec15e2baec9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:01.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f1727e00640 1 -- 172.21.15.46:0/2797230990 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140b0070 con 0x7f1728071980 2024-04-25T08:43:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f1727e00640 1 -- 172.21.15.46:0/2797230990 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17140ab400 con 0x7f1728071980 2024-04-25T08:43:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f1727e00640 1 -- 172.21.15.46:0/2797230990 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720001a00 con 0x7f1728071980 2024-04-25T08:43:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 -- 172.21.15.46:0/2797230990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1728071980 msgr2=0x7f172815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 --2- 172.21.15.46:0/2797230990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1728071980 0x7f172815b1c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17140097c0 tx=0x7f171401c3c0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 -- 172.21.15.46:0/2797230990 shutdown_connections 2024-04-25T08:43:01.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 --2- 172.21.15.46:0/2797230990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1728071980 0x7f172815b1c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 -- 172.21.15.46:0/2797230990 >> 172.21.15.46:0/2797230990 conn(0x7f172806ac10 msgr2=0x7f172806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:01.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 -- 172.21.15.46:0/2797230990 shutdown_connections 2024-04-25T08:43:01.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 -- 172.21.15.46:0/2797230990 wait complete. 2024-04-25T08:43:01.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 Processor -- start 2024-04-25T08:43:01.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 -- start start 2024-04-25T08:43:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1728071980 0x7f172813a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172813d0b0 0x7f172813c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f172815db50 0x7f172813ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280b2ab0 con 0x7f172813d0b0 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280b2930 con 0x7f1728071980 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.155+0000 7f172fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17280b2c30 con 0x7f172815db50 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172813d0b0 0x7f172813c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172813d0b0 0x7f172813c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42796/0 (socket says 172.21.15.46:42796) 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172ca00640 1 -- 172.21.15.46:0/2498907563 learned_addr learned my addr 172.21.15.46:0/2498907563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172d400640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1728071980 0x7f172813a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172de00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f172815db50 0x7f172813ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172ca00640 1 -- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f172815db50 msgr2=0x7f172813ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172ca00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f172815db50 0x7f172813ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172ca00640 1 -- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1728071980 msgr2=0x7f172813a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172ca00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1728071980 0x7f172813a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172ca00640 1 -- 172.21.15.46:0/2498907563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728152f30 con 0x7f172813d0b0 2024-04-25T08:43:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172ca00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172813d0b0 0x7f172813c0b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f17140ae040 tx=0x7f1714019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f1725600640 1 -- 172.21.15.46:0/2498907563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140ba020 con 0x7f172813d0b0 2024-04-25T08:43:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f1725600640 1 -- 172.21.15.46:0/2498907563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714004020 con 0x7f172813d0b0 2024-04-25T08:43:01.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f1725600640 1 -- 172.21.15.46:0/2498907563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140b0040 con 0x7f172813d0b0 2024-04-25T08:43:01.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17281534b0 con 0x7f172813d0b0 2024-04-25T08:43:01.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1728137fb0 con 0x7f172813d0b0 2024-04-25T08:43:01.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1728069010 con 0x7f172813d0b0 2024-04-25T08:43:01.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.159+0000 7f1725600640 1 -- 172.21.15.46:0/2498907563 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17140bd070 con 0x7f172813d0b0 2024-04-25T08:43:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.163+0000 7f1725600640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1708066f60 0x7f1708069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.163+0000 7f1725600640 1 -- 172.21.15.46:0/2498907563 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f171411e160 con 0x7f172813d0b0 2024-04-25T08:43:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.163+0000 7f172d400640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1708066f60 0x7f1708069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.163+0000 7f172d400640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1708066f60 0x7f1708069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f171c0017c0 tx=0x7f171c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:01.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.163+0000 7f1725600640 1 -- 172.21.15.46:0/2498907563 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17140c3080 con 0x7f172813d0b0 2024-04-25T08:43:01.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f172806d950 con 0x7f172813d0b0 2024-04-25T08:43:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f1725600640 1 -- 172.21.15.46:0/2498907563 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1714015fb0 con 0x7f172813d0b0 2024-04-25T08:43:01.320 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1708066f60 msgr2=0x7f1708069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1708066f60 0x7f1708069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f171c0017c0 tx=0x7f171c009040 comp rx=0 tx=0).stop 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172813d0b0 msgr2=0x7f172813c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172813d0b0 0x7f172813c0b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f17140ae040 tx=0x7f1714019160 comp rx=0 tx=0).stop 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 shutdown_connections 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1708066f60 0x7f1708069420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f172815db50 0x7f172813ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f172813d0b0 0x7f172813c0b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 --2- 172.21.15.46:0/2498907563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1728071980 0x7f172813a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 >> 172.21.15.46:0/2498907563 conn(0x7f172806ac10 msgr2=0x7f172806efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 shutdown_connections 2024-04-25T08:43:01.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.327+0000 7f172fe00640 1 -- 172.21.15.46:0/2498907563 wait complete. 2024-04-25T08:43:01.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:01.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.491+0000 7f846cc00640 1 Processor -- start 2024-04-25T08:43:01.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f846cc00640 1 -- start start 2024-04-25T08:43:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f846cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8468143990 0x7f84680cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f846cc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f84681440d0 0x7f846806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f846cc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846815e440 con 0x7f846806ef80 2024-04-25T08:43:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f846cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846815ed30 con 0x7f8468143990 2024-04-25T08:43:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f846cc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84680b3730 con 0x7f84681440d0 2024-04-25T08:43:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f8465e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8468143990 0x7f84680cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f8465e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8468143990 0x7f84680cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42798/0 (socket says 172.21.15.46:42798) 2024-04-25T08:43:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f8465e00640 1 -- 172.21.15.46:0/2933116038 learned_addr learned my addr 172.21.15.46:0/2933116038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:01.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f845fe00640 1 --2- 172.21.15.46:0/2933116038 >> v2:172.21.15.82:3300/0 conn(0x7f84681440d0 0x7f846806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f8465e00640 1 -- 172.21.15.46:0/2933116038 >> v2:172.21.15.82:3300/0 conn(0x7f84681440d0 msgr2=0x7f846806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:01.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f8465e00640 1 --2- 172.21.15.46:0/2933116038 >> v2:172.21.15.82:3300/0 conn(0x7f84681440d0 0x7f846806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f8465e00640 1 -- 172.21.15.46:0/2933116038 >> v1:172.21.15.82:6789/0 conn(0x7f846806ef80 legacy=0x7f84680cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.495+0000 7f8465e00640 1 -- 172.21.15.46:0/2933116038 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846815e6e0 con 0x7f8468143990 2024-04-25T08:43:01.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f8465e00640 1 --2- 172.21.15.46:0/2933116038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8468143990 0x7f84680cecc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8468070c80 tx=0x7f845801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0967be6acfdd92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:01.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f8465400640 1 -- 172.21.15.46:0/2933116038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84580b0070 con 0x7f8468143990 2024-04-25T08:43:01.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f8465400640 1 -- 172.21.15.46:0/2933116038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84580ab600 con 0x7f8468143990 2024-04-25T08:43:01.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f8465400640 1 -- 172.21.15.46:0/2933116038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84580ab9d0 con 0x7f8468143990 2024-04-25T08:43:01.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 -- 172.21.15.46:0/2933116038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8468143990 msgr2=0x7f84680cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 --2- 172.21.15.46:0/2933116038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8468143990 0x7f84680cecc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8468070c80 tx=0x7f845801c520 comp rx=0 tx=0).stop 2024-04-25T08:43:01.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 -- 172.21.15.46:0/2933116038 shutdown_connections 2024-04-25T08:43:01.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 --2- 172.21.15.46:0/2933116038 >> v2:172.21.15.82:3300/0 conn(0x7f84681440d0 0x7f846806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 --2- 172.21.15.46:0/2933116038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8468143990 0x7f84680cecc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 -- 172.21.15.46:0/2933116038 >> 172.21.15.46:0/2933116038 conn(0x7f846806ac10 msgr2=0x7f846806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:01.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 -- 172.21.15.46:0/2933116038 shutdown_connections 2024-04-25T08:43:01.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 -- 172.21.15.46:0/2933116038 wait complete. 2024-04-25T08:43:01.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 Processor -- start 2024-04-25T08:43:01.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.499+0000 7f846cc00640 1 -- start start 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f846cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f846806ef80 0x7f8468154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f846cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8468143990 0x7f8468155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f846cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84681440d0 0x7f8468138f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f846cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8468143d70 con 0x7f84681440d0 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f846cc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846806f390 con 0x7f846806ef80 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f846cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8468143ef0 con 0x7f8468143990 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8465e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f846806ef80 0x7f8468154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8465e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f846806ef80 0x7f8468154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:58286/0 (socket says 172.21.15.46:58286) 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8465e00640 1 -- 172.21.15.46:0/1849295787 learned_addr learned my addr 172.21.15.46:0/1849295787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8466800640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84681440d0 0x7f8468138f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f845fe00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8468143990 0x7f8468155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8466800640 1 -- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8468143990 msgr2=0x7f8468155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8466800640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8468143990 0x7f8468155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8466800640 1 -- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f846806ef80 msgr2=0x7f8468154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8466800640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f846806ef80 0x7f8468154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8466800640 1 -- 172.21.15.46:0/1849295787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468155b20 con 0x7f84681440d0 2024-04-25T08:43:01.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8465e00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f846806ef80 0x7f8468154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:01.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.503+0000 7f8466800640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84681440d0 0x7f8468138f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f846000c700 tx=0x7f846000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:01.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.507+0000 7f845e000640 1 -- 172.21.15.46:0/1849295787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8460003b80 con 0x7f84681440d0 2024-04-25T08:43:01.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.507+0000 7f845e000640 1 -- 172.21.15.46:0/1849295787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8460003d20 con 0x7f84681440d0 2024-04-25T08:43:01.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.507+0000 7f845e000640 1 -- 172.21.15.46:0/1849295787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846000aa30 con 0x7f84681440d0 2024-04-25T08:43:01.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.507+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8468155e00 con 0x7f84681440d0 2024-04-25T08:43:01.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.507+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8468157b60 con 0x7f84681440d0 2024-04-25T08:43:01.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.507+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8468157d60 con 0x7f84681440d0 2024-04-25T08:43:01.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.507+0000 7f845e000640 1 -- 172.21.15.46:0/1849295787 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f846000abd0 con 0x7f84681440d0 2024-04-25T08:43:01.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.511+0000 7f845e000640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8444066f60 0x7f8444069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.511+0000 7f845e000640 1 -- 172.21.15.46:0/1849295787 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8460085180 con 0x7f84681440d0 2024-04-25T08:43:01.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.511+0000 7f8465e00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8444066f60 0x7f8444069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.515+0000 7f8465e00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8444066f60 0x7f8444069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f845801c560 tx=0x7f8458008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:01.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.519+0000 7f845e000640 1 -- 172.21.15.46:0/1849295787 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8468157d60 con 0x7f84681440d0 2024-04-25T08:43:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.679+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8468069010 con 0x7f84681440d0 2024-04-25T08:43:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.679+0000 7f845e000640 1 -- 172.21.15.46:0/1849295787 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8460055730 con 0x7f84681440d0 2024-04-25T08:43:01.674 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8444066f60 msgr2=0x7f8444069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8444066f60 0x7f8444069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f845801c560 tx=0x7f8458008040 comp rx=0 tx=0).stop 2024-04-25T08:43:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84681440d0 msgr2=0x7f8468138f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84681440d0 0x7f8468138f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f846000c700 tx=0x7f846000cbc0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 shutdown_connections 2024-04-25T08:43:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8444066f60 0x7f8444069420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84681440d0 0x7f8468138f70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8468143990 0x7f8468155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 --2- 172.21.15.46:0/1849295787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f846806ef80 0x7f8468154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 >> 172.21.15.46:0/1849295787 conn(0x7f846806ac10 msgr2=0x7f846813ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 shutdown_connections 2024-04-25T08:43:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.683+0000 7f846cc00640 1 -- 172.21.15.46:0/1849295787 wait complete. 2024-04-25T08:43:01.697 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.843+0000 7f6853a00640 1 Processor -- start 2024-04-25T08:43:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.843+0000 7f6853a00640 1 -- start start 2024-04-25T08:43:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6853a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f684c159230 0x7f684c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6853a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f684c0cbeb0 0x7f684c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6853a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f684c0cc700 con 0x7f684c0cda80 2024-04-25T08:43:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6853a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684c0cc9e0 con 0x7f684c159230 2024-04-25T08:43:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6853a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0cccc0 con 0x7f684c0cbeb0 2024-04-25T08:43:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6851a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f684c0cda80 0x7f684c15f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48372/0 (socket says 172.21.15.46:48372) 2024-04-25T08:43:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6851a00640 1 -- 172.21.15.46:0/1684854567 learned_addr learned my addr 172.21.15.46:0/1684854567 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6851000640 1 --2- 172.21.15.46:0/1684854567 >> v2:172.21.15.46:3300/0 conn(0x7f684c159230 0x7f684c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f684be00640 1 --2- 172.21.15.46:0/1684854567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f684c0cbeb0 0x7f684c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6851000640 1 -- 172.21.15.46:0/1684854567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f684c0cbeb0 msgr2=0x7f684c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6851000640 1 --2- 172.21.15.46:0/1684854567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f684c0cbeb0 0x7f684c0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6851000640 1 -- 172.21.15.46:0/1684854567 >> v1:172.21.15.46:6789/0 conn(0x7f684c0cda80 legacy=0x7f684c15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f6851000640 1 -- 172.21.15.46:0/1684854567 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c0ccfa0 con 0x7f684c159230 2024-04-25T08:43:01.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.847+0000 7f684b400640 1 -- 172.21.15.46:0/1684854567 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279399899 0 0) 0x7f684c0cc700 con 0x7f684c0cda80 2024-04-25T08:43:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6851000640 1 --2- 172.21.15.46:0/1684854567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c159230 0x7f684c15bdb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f68400097c0 tx=0x7f684001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35124fa1b640783 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f684b400640 1 -- 172.21.15.46:0/1684854567 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400b0070 con 0x7f684c159230 2024-04-25T08:43:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f684b400640 1 -- 172.21.15.46:0/1684854567 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68400ab600 con 0x7f684c159230 2024-04-25T08:43:01.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f684b400640 1 -- 172.21.15.46:0/1684854567 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400ab9d0 con 0x7f684c159230 2024-04-25T08:43:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 -- 172.21.15.46:0/1684854567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c159230 msgr2=0x7f684c15bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 --2- 172.21.15.46:0/1684854567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c159230 0x7f684c15bdb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f68400097c0 tx=0x7f684001c410 comp rx=0 tx=0).stop 2024-04-25T08:43:01.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 -- 172.21.15.46:0/1684854567 shutdown_connections 2024-04-25T08:43:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 --2- 172.21.15.46:0/1684854567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f684c0cbeb0 0x7f684c0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 --2- 172.21.15.46:0/1684854567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c159230 0x7f684c15bdb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 -- 172.21.15.46:0/1684854567 >> 172.21.15.46:0/1684854567 conn(0x7f684c0b12a0 msgr2=0x7f684c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 -- 172.21.15.46:0/1684854567 shutdown_connections 2024-04-25T08:43:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 -- 172.21.15.46:0/1684854567 wait complete. 2024-04-25T08:43:01.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 Processor -- start 2024-04-25T08:43:01.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 -- start start 2024-04-25T08:43:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f684c0cbeb0 0x7f684c140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0cda80 0x7f684c0af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f684c159230 0x7f684c0af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c168780 con 0x7f684c0cda80 2024-04-25T08:43:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c0cc4a0 con 0x7f684c159230 2024-04-25T08:43:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f6853a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684c168900 con 0x7f684c0cbeb0 2024-04-25T08:43:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.851+0000 7f684be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0cda80 0x7f684c0af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f684be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0cda80 0x7f684c0af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42822/0 (socket says 172.21.15.46:42822) 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f684be00640 1 -- 172.21.15.46:0/3614198305 learned_addr learned my addr 172.21.15.46:0/3614198305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f684be00640 1 -- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f684c0cbeb0 msgr2=0x7f684c140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f684be00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f684c0cbeb0 0x7f684c140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f684be00640 1 -- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f684c159230 msgr2=0x7f684c0af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f684be00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f684c159230 0x7f684c0af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f684be00640 1 -- 172.21.15.46:0/3614198305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c15ae70 con 0x7f684c0cda80 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f684be00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0cda80 0x7f684c0af1b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f683800e870 tx=0x7f683800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f6848c00640 1 -- 172.21.15.46:0/3614198305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838017070 con 0x7f684c0cda80 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f6848c00640 1 -- 172.21.15.46:0/3614198305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683800c940 con 0x7f684c0cda80 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f6848c00640 1 -- 172.21.15.46:0/3614198305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683800ccf0 con 0x7f684c0cda80 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c1407d0 con 0x7f684c0cda80 2024-04-25T08:43:01.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c06d5d0 con 0x7f684c0cda80 2024-04-25T08:43:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c069080 con 0x7f684c0cda80 2024-04-25T08:43:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f6848c00640 1 -- 172.21.15.46:0/3614198305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f683801a070 con 0x7f684c0cda80 2024-04-25T08:43:01.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.855+0000 7f6848c00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f682c066f60 0x7f682c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:01.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.859+0000 7f6848c00640 1 -- 172.21.15.46:0/3614198305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f68380853b0 con 0x7f684c0cda80 2024-04-25T08:43:01.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.859+0000 7f6851000640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f682c066f60 0x7f682c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.859+0000 7f6851000640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f682c066f60 0x7f682c069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68400ad000 tx=0x7f6840008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:01.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:01.859+0000 7f6848c00640 1 -- 172.21.15.46:0/3614198305 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6838055f20 con 0x7f684c0cda80 2024-04-25T08:43:02.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.027+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f684c0723d0 con 0x7f684c0cda80 2024-04-25T08:43:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.027+0000 7f6848c00640 1 -- 172.21.15.46:0/3614198305 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7f6838055820 con 0x7f684c0cda80 2024-04-25T08:43:02.021 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:02.021 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f682c066f60 msgr2=0x7f682c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f682c066f60 0x7f682c069420 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68400ad000 tx=0x7f6840008040 comp rx=0 tx=0).stop 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0cda80 msgr2=0x7f684c0af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0cda80 0x7f684c0af1b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f683800e870 tx=0x7f683800ec70 comp rx=0 tx=0).stop 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 shutdown_connections 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f682c066f60 0x7f682c069420 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f684c159230 0x7f684c0af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f684c0cda80 0x7f684c0af1b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 --2- 172.21.15.46:0/3614198305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f684c0cbeb0 0x7f684c140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 >> 172.21.15.46:0/3614198305 conn(0x7f684c0b12a0 msgr2=0x7f684c0b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 shutdown_connections 2024-04-25T08:43:02.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.031+0000 7f6853a00640 1 -- 172.21.15.46:0/3614198305 wait complete. 2024-04-25T08:43:02.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:02.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.191+0000 7f0e64a00640 1 Processor -- start 2024-04-25T08:43:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.191+0000 7f0e64a00640 1 -- start start 2024-04-25T08:43:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e64a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 0x7f0e600b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e64a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e6006e1a0 0x7f0e6006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e64a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6006e9f0 con 0x7f0e6006b150 2024-04-25T08:43:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e64a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6006ecd0 con 0x7f0e6006e1a0 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e64a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6006efb0 con 0x7f0e601473f0 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e5fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0e6006b150 0x7f0e60071490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48386/0 (socket says 172.21.15.46:48386) 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e5fe00640 1 -- 172.21.15.46:0/732932190 learned_addr learned my addr 172.21.15.46:0/732932190 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e56a00640 1 --2- 172.21.15.46:0/732932190 >> v2:172.21.15.46:3300/0 conn(0x7f0e6006e1a0 0x7f0e6006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e5f400640 1 --2- 172.21.15.46:0/732932190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 0x7f0e600b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e56a00640 1 -- 172.21.15.46:0/732932190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 msgr2=0x7f0e600b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e56a00640 1 --2- 172.21.15.46:0/732932190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 0x7f0e600b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e56a00640 1 -- 172.21.15.46:0/732932190 >> v1:172.21.15.46:6789/0 conn(0x7f0e6006b150 legacy=0x7f0e60071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e56a00640 1 -- 172.21.15.46:0/732932190 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6006f290 con 0x7f0e6006e1a0 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e5ea00640 1 -- 172.21.15.46:0/732932190 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129294181 0 0) 0x7f0e6006e9f0 con 0x7f0e6006b150 2024-04-25T08:43:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.195+0000 7f0e5f400640 1 --2- 172.21.15.46:0/732932190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 0x7f0e600b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:02.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e56a00640 1 --2- 172.21.15.46:0/732932190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 0x7f0e6006ac10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0e480097c0 tx=0x7f0e4801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366f0af80eb6ced2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e5ea00640 1 -- 172.21.15.46:0/732932190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480b0070 con 0x7f0e6006e1a0 2024-04-25T08:43:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e5ea00640 1 -- 172.21.15.46:0/732932190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e480ab600 con 0x7f0e6006e1a0 2024-04-25T08:43:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e5ea00640 1 -- 172.21.15.46:0/732932190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480ab9d0 con 0x7f0e6006e1a0 2024-04-25T08:43:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 -- 172.21.15.46:0/732932190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 msgr2=0x7f0e6006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 --2- 172.21.15.46:0/732932190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 0x7f0e6006ac10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0e480097c0 tx=0x7f0e4801c410 comp rx=0 tx=0).stop 2024-04-25T08:43:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 -- 172.21.15.46:0/732932190 shutdown_connections 2024-04-25T08:43:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 --2- 172.21.15.46:0/732932190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 0x7f0e6006ac10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 --2- 172.21.15.46:0/732932190 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 0x7f0e600b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 -- 172.21.15.46:0/732932190 >> 172.21.15.46:0/732932190 conn(0x7f0e60067000 msgr2=0x7f0e600b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 -- 172.21.15.46:0/732932190 shutdown_connections 2024-04-25T08:43:02.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 -- 172.21.15.46:0/732932190 wait complete. 2024-04-25T08:43:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 Processor -- start 2024-04-25T08:43:02.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 -- start start 2024-04-25T08:43:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e6006b150 0x7f0e6015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 0x7f0e60160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 0x7f0e60160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6016a880 con 0x7f0e6006e1a0 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6016a700 con 0x7f0e6006b150 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.199+0000 7f0e64a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6016aa00 con 0x7f0e601473f0 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e56a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 0x7f0e60160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e56a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 0x7f0e60160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42842/0 (socket says 172.21.15.46:42842) 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e56a00640 1 -- 172.21.15.46:0/3804510835 learned_addr learned my addr 172.21.15.46:0/3804510835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e5fe00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 0x7f0e60160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e5f400640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e6006b150 0x7f0e6015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e56a00640 1 -- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 msgr2=0x7f0e60160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e56a00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 0x7f0e60160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e56a00640 1 -- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e6006b150 msgr2=0x7f0e6015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e56a00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e6006b150 0x7f0e6015d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e56a00640 1 -- 172.21.15.46:0/3804510835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6013fa40 con 0x7f0e6006e1a0 2024-04-25T08:43:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e56a00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 0x7f0e60160690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e480ae040 tx=0x7f0e480191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e57e00640 1 -- 172.21.15.46:0/3804510835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480b8030 con 0x7f0e6006e1a0 2024-04-25T08:43:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e57e00640 1 -- 172.21.15.46:0/3804510835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e480b4410 con 0x7f0e6006e1a0 2024-04-25T08:43:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e57e00640 1 -- 172.21.15.46:0/3804510835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480b0040 con 0x7f0e6006e1a0 2024-04-25T08:43:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e60144d30 con 0x7f0e6006e1a0 2024-04-25T08:43:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e601452f0 con 0x7f0e6006e1a0 2024-04-25T08:43:02.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e60069010 con 0x7f0e6006e1a0 2024-04-25T08:43:02.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.203+0000 7f0e57e00640 1 -- 172.21.15.46:0/3804510835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e480bb070 con 0x7f0e6006e1a0 2024-04-25T08:43:02.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.207+0000 7f0e57e00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e3c066f60 0x7f0e3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.207+0000 7f0e57e00640 1 -- 172.21.15.46:0/3804510835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0e4811e9b0 con 0x7f0e6006e1a0 2024-04-25T08:43:02.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.207+0000 7f0e5f400640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e3c066f60 0x7f0e3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.207+0000 7f0e5f400640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e3c066f60 0x7f0e3c069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000a9a0 tx=0x7f0e50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:02.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.211+0000 7f0e57e00640 1 -- 172.21.15.46:0/3804510835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e480c3020 con 0x7f0e6006e1a0 2024-04-25T08:43:02.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.371+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e600b31e0 con 0x7f0e6006e1a0 2024-04-25T08:43:02.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.371+0000 7f0e57e00640 1 -- 172.21.15.46:0/3804510835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e48016090 con 0x7f0e6006e1a0 2024-04-25T08:43:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e3c066f60 msgr2=0x7f0e3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e3c066f60 0x7f0e3c069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000a9a0 tx=0x7f0e50008040 comp rx=0 tx=0).stop 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 msgr2=0x7f0e60160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 0x7f0e60160690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e480ae040 tx=0x7f0e480191b0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 shutdown_connections 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e3c066f60 0x7f0e3c069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0e601473f0 0x7f0e60160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6006e1a0 0x7f0e60160690 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 --2- 172.21.15.46:0/3804510835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e6006b150 0x7f0e6015d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 >> 172.21.15.46:0/3804510835 conn(0x7f0e60067000 msgr2=0x7f0e600afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 shutdown_connections 2024-04-25T08:43:02.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.375+0000 7f0e64a00640 1 -- 172.21.15.46:0/3804510835 wait complete. 2024-04-25T08:43:02.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.535+0000 7ff1acc00640 1 Processor -- start 2024-04-25T08:43:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.535+0000 7ff1acc00640 1 -- start start 2024-04-25T08:43:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1acc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff1a815df90 0x7ff1a80d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1acc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a809a580 0x7ff1a809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1acc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a80d08e0 con 0x7ff1a815dbb0 2024-04-25T08:43:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1acc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a80b2c00 con 0x7ff1a815df90 2024-04-25T08:43:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1acc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a809af50 con 0x7ff1a809a580 2024-04-25T08:43:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff1a815dbb0 0x7ff1a80cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48388/0 (socket says 172.21.15.46:48388) 2024-04-25T08:43:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5e00640 1 -- 172.21.15.46:0/2714971789 learned_addr learned my addr 172.21.15.46:0/2714971789 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a6800640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a809a580 0x7ff1a809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff19fe00640 1 --2- 172.21.15.46:0/2714971789 >> v2:172.21.15.82:3300/0 conn(0x7ff1a815df90 0x7ff1a80d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3502648383 0 0) 0x7ff1a80d08e0 con 0x7ff1a815dbb0 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff18c003530 con 0x7ff1a815dbb0 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 821911076 0 0) 0x7ff18c003530 con 0x7ff1a815dbb0 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a809a580 msgr2=0x7ff1a809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a809a580 0x7ff1a809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 >> v2:172.21.15.82:3300/0 conn(0x7ff1a815df90 msgr2=0x7ff1a80d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 --2- 172.21.15.46:0/2714971789 >> v2:172.21.15.82:3300/0 conn(0x7ff1a815df90 0x7ff1a80d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a80d0bc0 con 0x7ff1a815dbb0 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7ff198001680 con 0x7ff1a815dbb0 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 >> v1:172.21.15.46:6789/0 conn(0x7ff1a815dbb0 legacy=0x7ff1a80cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c003a30 0x7ff18c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a809a580 0x7ff18c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff18c004d50 0x7ff18c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c00b9d0 con 0x7ff18c003a30 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c00bb50 con 0x7ff18c004d50 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c00bcd0 con 0x7ff1a809a580 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5e00640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c003a30 0x7ff18c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a6800640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff18c004d50 0x7ff18c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff19fe00640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a809a580 0x7ff18c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5e00640 1 -- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a809a580 msgr2=0x7ff18c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5e00640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a809a580 0x7ff18c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5e00640 1 -- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff18c004d50 msgr2=0x7ff18c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5e00640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff18c004d50 0x7ff18c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.539+0000 7ff1a5e00640 1 -- 172.21.15.46:0/2714971789 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c003530 con 0x7ff18c003a30 2024-04-25T08:43:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1a5e00640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c003a30 0x7ff18c003e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c240 tx=0x7ff19801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20520f5ed5967144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198007d60 con 0x7ff18c003a30 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1980a9020 con 0x7ff18c003a30 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1a5400640 1 -- 172.21.15.46:0/2714971789 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198005ae0 con 0x7ff18c003a30 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1acc00640 1 -- 172.21.15.46:0/2714971789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c003a30 msgr2=0x7ff18c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1acc00640 1 --2- 172.21.15.46:0/2714971789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c003a30 0x7ff18c003e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff19800c240 tx=0x7ff19801ccc0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1a6800640 1 -- 172.21.15.46:0/2714971789 reap_dead start 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1acc00640 1 -- 172.21.15.46:0/2714971789 shutdown_connections 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1acc00640 1 -- 172.21.15.46:0/2714971789 >> 172.21.15.46:0/2714971789 conn(0x7ff1a80b3150 msgr2=0x7ff1a80b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1acc00640 1 -- 172.21.15.46:0/2714971789 shutdown_connections 2024-04-25T08:43:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.543+0000 7ff1acc00640 1 -- 172.21.15.46:0/2714971789 wait complete. 2024-04-25T08:43:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 Processor -- start 2024-04-25T08:43:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 -- start start 2024-04-25T08:43:02.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a81f6010 0x7ff1a80b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a80b6780 0x7ff1a80cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a80ccb10 0x7ff1a815f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81621d0 con 0x7ff1a81f6010 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a809ada0 con 0x7ff1a80ccb10 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a8162350 con 0x7ff1a80b6780 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a81f6010 0x7ff1a80b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a81f6010 0x7ff1a80b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42862/0 (socket says 172.21.15.46:42862) 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a5e00640 1 -- 172.21.15.46:0/4183586217 learned_addr learned my addr 172.21.15.46:0/4183586217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff19fe00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a80b6780 0x7ff1a80cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a6800640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a80ccb10 0x7ff1a815f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a5e00640 1 -- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a80b6780 msgr2=0x7ff1a80cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a5e00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a80b6780 0x7ff1a80cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a5e00640 1 -- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a80ccb10 msgr2=0x7ff1a815f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a5e00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a80ccb10 0x7ff1a815f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a5e00640 1 -- 172.21.15.46:0/4183586217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a80c8980 con 0x7ff1a81f6010 2024-04-25T08:43:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1a5e00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a81f6010 0x7ff1a80b57b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff1980ab040 tx=0x7ff18c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff19e000640 1 -- 172.21.15.46:0/4183586217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198001ca0 con 0x7ff1a81f6010 2024-04-25T08:43:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff19e000640 1 -- 172.21.15.46:0/4183586217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198004ab0 con 0x7ff1a81f6010 2024-04-25T08:43:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff19e000640 1 -- 172.21.15.46:0/4183586217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980a58b0 con 0x7ff1a81f6010 2024-04-25T08:43:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a80c8c60 con 0x7ff1a81f6010 2024-04-25T08:43:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a80c9220 con 0x7ff1a81f6010 2024-04-25T08:43:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.547+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a80c8710 con 0x7ff1a81f6010 2024-04-25T08:43:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.551+0000 7ff19e000640 1 -- 172.21.15.46:0/4183586217 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1980a5a50 con 0x7ff1a81f6010 2024-04-25T08:43:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.551+0000 7ff19e000640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff184066f90 0x7ff184069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.551+0000 7ff19e000640 1 -- 172.21.15.46:0/4183586217 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff19811e900 con 0x7ff1a81f6010 2024-04-25T08:43:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.551+0000 7ff19fe00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff184066f90 0x7ff184069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.551+0000 7ff19fe00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff184066f90 0x7ff184069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff1a815cbd0 tx=0x7ff190009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:02.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.555+0000 7ff19e000640 1 -- 172.21.15.46:0/4183586217 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1980169d0 con 0x7ff1a81f6010 2024-04-25T08:43:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1a8069010 con 0x7ff1a81f6010 2024-04-25T08:43:02.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff19e000640 1 -- 172.21.15.46:0/4183586217 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1980ef530 con 0x7ff1a81f6010 2024-04-25T08:43:02.711 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff184066f90 msgr2=0x7ff184069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff184066f90 0x7ff184069450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff1a815cbd0 tx=0x7ff190009190 comp rx=0 tx=0).stop 2024-04-25T08:43:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a81f6010 msgr2=0x7ff1a80b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a81f6010 0x7ff1a80b57b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff1980ab040 tx=0x7ff18c005840 comp rx=0 tx=0).stop 2024-04-25T08:43:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 shutdown_connections 2024-04-25T08:43:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff184066f90 0x7ff184069450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a80ccb10 0x7ff1a815f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a80b6780 0x7ff1a80cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 --2- 172.21.15.46:0/4183586217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a81f6010 0x7ff1a80b57b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 >> 172.21.15.46:0/4183586217 conn(0x7ff1a80b3150 msgr2=0x7ff1a80cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 shutdown_connections 2024-04-25T08:43:02.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.719+0000 7ff1acc00640 1 -- 172.21.15.46:0/4183586217 wait complete. 2024-04-25T08:43:02.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.883+0000 7f16c8600640 1 Processor -- start 2024-04-25T08:43:02.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.883+0000 7f16c8600640 1 -- start start 2024-04-25T08:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.883+0000 7f16c8600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f16c01440d0 0x7f16c00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.883+0000 7f16c8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 0x7f16c00b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.883+0000 7f16c8600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c015e440 con 0x7f16c0143990 2024-04-25T08:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.883+0000 7f16c8600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c015ed30 con 0x7f16c01440d0 2024-04-25T08:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.883+0000 7f16c8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c015e6e0 con 0x7f16c006f150 2024-04-25T08:43:02.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c5200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f16c01440d0 0x7f16c00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c5c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f16c0143990 0x7f16c006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48394/0 (socket says 172.21.15.46:48394) 2024-04-25T08:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c5c00640 1 -- 172.21.15.46:0/1549864915 learned_addr learned my addr 172.21.15.46:0/1549864915 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c6600640 1 --2- 172.21.15.46:0/1549864915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 0x7f16c00b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c5200640 1 -- 172.21.15.46:0/1549864915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 msgr2=0x7f16c00b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:02.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c5200640 1 --2- 172.21.15.46:0/1549864915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 0x7f16c00b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c5200640 1 -- 172.21.15.46:0/1549864915 >> v1:172.21.15.46:6789/0 conn(0x7f16c0143990 legacy=0x7f16c006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16bfe00640 1 -- 172.21.15.46:0/1549864915 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154118851 0 0) 0x7f16c015e440 con 0x7f16c0143990 2024-04-25T08:43:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c5200640 1 -- 172.21.15.46:0/1549864915 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c015e9c0 con 0x7f16c01440d0 2024-04-25T08:43:02.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c6600640 1 --2- 172.21.15.46:0/1549864915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 0x7f16c00b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16c5200640 1 --2- 172.21.15.46:0/1549864915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 0x7f16c00cecc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f16c015eeb0 tx=0x7f16ac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2defdd7d086b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16bfe00640 1 -- 172.21.15.46:0/1549864915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0b0070 con 0x7f16c01440d0 2024-04-25T08:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16bfe00640 1 -- 172.21.15.46:0/1549864915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac0ab600 con 0x7f16c01440d0 2024-04-25T08:43:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.887+0000 7f16bfe00640 1 -- 172.21.15.46:0/1549864915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac0ab9d0 con 0x7f16c01440d0 2024-04-25T08:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 -- 172.21.15.46:0/1549864915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 msgr2=0x7f16c00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 --2- 172.21.15.46:0/1549864915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 0x7f16c00cecc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f16c015eeb0 tx=0x7f16ac01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 -- 172.21.15.46:0/1549864915 shutdown_connections 2024-04-25T08:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 --2- 172.21.15.46:0/1549864915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 0x7f16c00b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 --2- 172.21.15.46:0/1549864915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 0x7f16c00cecc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 -- 172.21.15.46:0/1549864915 >> 172.21.15.46:0/1549864915 conn(0x7f16c006ac10 msgr2=0x7f16c006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 -- 172.21.15.46:0/1549864915 shutdown_connections 2024-04-25T08:43:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 -- 172.21.15.46:0/1549864915 wait complete. 2024-04-25T08:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 Processor -- start 2024-04-25T08:43:02.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 -- start start 2024-04-25T08:43:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 0x7f16c01f9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c0143990 0x7f16c00cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 0x7f16c015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00b35f0 con 0x7f16c01440d0 2024-04-25T08:43:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00b3470 con 0x7f16c0143990 2024-04-25T08:43:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c00b3770 con 0x7f16c006f150 2024-04-25T08:43:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 0x7f16c015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 0x7f16c015d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42868/0 (socket says 172.21.15.46:42868) 2024-04-25T08:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c6600640 1 -- 172.21.15.46:0/1887376726 learned_addr learned my addr 172.21.15.46:0/1887376726 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c5c00640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 0x7f16c01f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.891+0000 7f16c5200640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c0143990 0x7f16c00cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c6600640 1 -- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 msgr2=0x7f16c01f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c6600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 0x7f16c01f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c6600640 1 -- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c0143990 msgr2=0x7f16c00cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c6600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c0143990 0x7f16c00cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c6600640 1 -- 172.21.15.46:0/1887376726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c01fa510 con 0x7f16c01440d0 2024-04-25T08:43:02.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c5200640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c0143990 0x7f16c00cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c6600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 0x7f16c015d5a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16b800cef0 tx=0x7f16b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16bd600640 1 -- 172.21.15.46:0/1887376726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b801a070 con 0x7f16c01440d0 2024-04-25T08:43:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16bd600640 1 -- 172.21.15.46:0/1887376726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800d860 con 0x7f16c01440d0 2024-04-25T08:43:02.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16bd600640 1 -- 172.21.15.46:0/1887376726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800dc50 con 0x7f16c01440d0 2024-04-25T08:43:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c01f9f50 con 0x7f16c01440d0 2024-04-25T08:43:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c014ad40 con 0x7f16c01440d0 2024-04-25T08:43:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c0069010 con 0x7f16c01440d0 2024-04-25T08:43:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16bd600640 1 -- 172.21.15.46:0/1887376726 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16b801d070 con 0x7f16c01440d0 2024-04-25T08:43:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16bd600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f16a0066f60 0x7f16a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:02.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16bd600640 1 -- 172.21.15.46:0/1887376726 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f16b8085cd0 con 0x7f16c01440d0 2024-04-25T08:43:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.895+0000 7f16c5c00640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f16a0066f60 0x7f16a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:02.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.899+0000 7f16c5c00640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f16a0066f60 0x7f16a0069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007fd0 tx=0x7f16b4007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:02.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:02.899+0000 7f16bd600640 1 -- 172.21.15.46:0/1887376726 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16b80562b0 con 0x7f16c01440d0 2024-04-25T08:43:03.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16c014b0e0 con 0x7f16c01440d0 2024-04-25T08:43:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16bd600640 1 -- 172.21.15.46:0/1887376726 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16b805a350 con 0x7f16c01440d0 2024-04-25T08:43:03.055 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f16a0066f60 msgr2=0x7f16a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f16a0066f60 0x7f16a0069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007fd0 tx=0x7f16b4007d60 comp rx=0 tx=0).stop 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 msgr2=0x7f16c015d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 0x7f16c015d5a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f16b800cef0 tx=0x7f16b800a710 comp rx=0 tx=0).stop 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 shutdown_connections 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f16a0066f60 0x7f16a0069420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16c01440d0 0x7f16c015d5a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c0143990 0x7f16c00cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 --2- 172.21.15.46:0/1887376726 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f16c006f150 0x7f16c01f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 >> 172.21.15.46:0/1887376726 conn(0x7f16c006ac10 msgr2=0x7f16c0164130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 shutdown_connections 2024-04-25T08:43:03.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.063+0000 7f16c8600640 1 -- 172.21.15.46:0/1887376726 wait complete. 2024-04-25T08:43:03.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:03.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.227+0000 7fca73e00640 1 Processor -- start 2024-04-25T08:43:03.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.227+0000 7fca73e00640 1 -- start start 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca7406c730 0x7fca7406bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca740d0390 0x7fca740cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7409acb0 con 0x7fca7409a410 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca740b3dc0 con 0x7fca740d0390 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca740cf680 con 0x7fca7406c730 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca740d0390 0x7fca740cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca740d0390 0x7fca740cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42882/0 (socket says 172.21.15.46:42882) 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73400640 1 -- 172.21.15.46:0/2035156670 learned_addr learned my addr 172.21.15.46:0/2035156670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca72a00640 1 --2- 172.21.15.46:0/2035156670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca7406c730 0x7fca7406bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73400640 1 -- 172.21.15.46:0/2035156670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca7406c730 msgr2=0x7fca7406bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:03.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73400640 1 --2- 172.21.15.46:0/2035156670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca7406c730 0x7fca7406bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73400640 1 -- 172.21.15.46:0/2035156670 >> v1:172.21.15.82:6789/0 conn(0x7fca7409a410 legacy=0x7fca7406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:03.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73400640 1 -- 172.21.15.46:0/2035156670 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7409a9d0 con 0x7fca740d0390 2024-04-25T08:43:03.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca72a00640 1 --2- 172.21.15.46:0/2035156670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca7406c730 0x7fca7406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:03.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca73400640 1 --2- 172.21.15.46:0/2035156670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca740d0390 0x7fca740cbab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009e50 tx=0x7fca6c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d287c2ea565c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.231+0000 7fca72000640 1 -- 172.21.15.46:0/2035156670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0a5840 con 0x7fca740d0390 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca72000640 1 -- 172.21.15.46:0/2035156670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c0a59e0 con 0x7fca740d0390 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca72000640 1 -- 172.21.15.46:0/2035156670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0a5db0 con 0x7fca740d0390 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 -- 172.21.15.46:0/2035156670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca740d0390 msgr2=0x7fca740cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 --2- 172.21.15.46:0/2035156670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca740d0390 0x7fca740cbab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009e50 tx=0x7fca6c01c9c0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 -- 172.21.15.46:0/2035156670 shutdown_connections 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 --2- 172.21.15.46:0/2035156670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca740d0390 0x7fca740cbab0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 --2- 172.21.15.46:0/2035156670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca7406c730 0x7fca7406bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 -- 172.21.15.46:0/2035156670 >> 172.21.15.46:0/2035156670 conn(0x7fca740b2f50 msgr2=0x7fca740b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 -- 172.21.15.46:0/2035156670 shutdown_connections 2024-04-25T08:43:03.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 -- 172.21.15.46:0/2035156670 wait complete. 2024-04-25T08:43:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 Processor -- start 2024-04-25T08:43:03.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 -- start start 2024-04-25T08:43:03.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7406c730 0x7fca7414fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7409a410 0x7fca740ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca740d0390 0x7fca741553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7415dd60 con 0x7fca7409a410 2024-04-25T08:43:03.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca740cc400 con 0x7fca7406c730 2024-04-25T08:43:03.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7416b040 con 0x7fca740d0390 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca6a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7409a410 0x7fca740ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca72a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7406c730 0x7fca7414fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca6a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7409a410 0x7fca740ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42886/0 (socket says 172.21.15.46:42886) 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca6a000640 1 -- 172.21.15.46:0/2283813771 learned_addr learned my addr 172.21.15.46:0/2283813771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca73400640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca740d0390 0x7fca741553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca6a000640 1 -- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca740d0390 msgr2=0x7fca741553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca6a000640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca740d0390 0x7fca741553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca6a000640 1 -- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7406c730 msgr2=0x7fca7414fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca6a000640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7406c730 0x7fca7414fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.235+0000 7fca6a000640 1 -- 172.21.15.46:0/2283813771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca740c9940 con 0x7fca7409a410 2024-04-25T08:43:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca6a000640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7409a410 0x7fca740ca9e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b410 tx=0x7fca5c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca6b400640 1 -- 172.21.15.46:0/2283813771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c015030 con 0x7fca7409a410 2024-04-25T08:43:03.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca6b400640 1 -- 172.21.15.46:0/2283813771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c004020 con 0x7fca7409a410 2024-04-25T08:43:03.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca6b400640 1 -- 172.21.15.46:0/2283813771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c004450 con 0x7fca7409a410 2024-04-25T08:43:03.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca740c9c20 con 0x7fca7409a410 2024-04-25T08:43:03.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca740ca1e0 con 0x7fca7409a410 2024-04-25T08:43:03.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca74069010 con 0x7fca7409a410 2024-04-25T08:43:03.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca6b400640 1 -- 172.21.15.46:0/2283813771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca5c018070 con 0x7fca7409a410 2024-04-25T08:43:03.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca6b400640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca50066f90 0x7fca50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca6b400640 1 -- 172.21.15.46:0/2283813771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fca5c084e90 con 0x7fca7409a410 2024-04-25T08:43:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.239+0000 7fca72a00640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca50066f90 0x7fca50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.243+0000 7fca72a00640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca50066f90 0x7fca50069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fca740d1090 tx=0x7fca64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.243+0000 7fca6b400640 1 -- 172.21.15.46:0/2283813771 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca5c055450 con 0x7fca7409a410 2024-04-25T08:43:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fca740719f0 con 0x7fca7409a410 2024-04-25T08:43:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca6b400640 1 -- 172.21.15.46:0/2283813771 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fca5c0594f0 con 0x7fca7409a410 2024-04-25T08:43:03.399 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:03.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca50066f90 msgr2=0x7fca50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca50066f90 0x7fca50069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fca740d1090 tx=0x7fca64008040 comp rx=0 tx=0).stop 2024-04-25T08:43:03.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7409a410 msgr2=0x7fca740ca9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7409a410 0x7fca740ca9e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00b410 tx=0x7fca5c00b8d0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 shutdown_connections 2024-04-25T08:43:03.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca50066f90 0x7fca50069450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca740d0390 0x7fca741553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca7409a410 0x7fca740ca9e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 --2- 172.21.15.46:0/2283813771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca7406c730 0x7fca7414fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 >> 172.21.15.46:0/2283813771 conn(0x7fca740b2f50 msgr2=0x7fca740aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 shutdown_connections 2024-04-25T08:43:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.407+0000 7fca73e00640 1 -- 172.21.15.46:0/2283813771 wait complete. 2024-04-25T08:43:03.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.567+0000 7ffa61600640 1 Processor -- start 2024-04-25T08:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.567+0000 7ffa61600640 1 -- start start 2024-04-25T08:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa61600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa5c15ee20 0x7ffa5c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa61600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c0d0f70 con 0x7ffa5c0d0980 2024-04-25T08:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa61600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c0d1250 con 0x7ffa5c099a00 2024-04-25T08:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa61600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5c09a7c0 con 0x7ffa5c15ee20 2024-04-25T08:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa53e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa5c0d0980 0x7ffa5c06c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48396/0 (socket says 172.21.15.46:48396) 2024-04-25T08:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa53e00640 1 -- 172.21.15.46:0/637131960 learned_addr learned my addr 172.21.15.46:0/637131960 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa5b400640 1 --2- 172.21.15.46:0/637131960 >> v2:172.21.15.82:3300/0 conn(0x7ffa5c15ee20 0x7ffa5c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa5a000640 1 -- 172.21.15.46:0/637131960 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750332651 0 0) 0x7ffa5c0d0f70 con 0x7ffa5c0d0980 2024-04-25T08:43:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa5a000640 1 -- 172.21.15.46:0/637131960 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa40003530 con 0x7ffa5c0d0980 2024-04-25T08:43:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa5a000640 1 -- 172.21.15.46:0/637131960 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289717049 0 0) 0x7ffa5c0d1250 con 0x7ffa5c099a00 2024-04-25T08:43:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa5a000640 1 -- 172.21.15.46:0/637131960 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa5c0d0f70 con 0x7ffa5c099a00 2024-04-25T08:43:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa5b400640 1 -- 172.21.15.46:0/637131960 >> v1:172.21.15.82:6789/0 conn(0x7ffa5c099a00 legacy=0x7ffa5c099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa5b400640 1 -- 172.21.15.46:0/637131960 >> v1:172.21.15.46:6789/0 conn(0x7ffa5c0d0980 legacy=0x7ffa5c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.571+0000 7ffa5b400640 1 -- 172.21.15.46:0/637131960 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c06bea0 con 0x7ffa5c15ee20 2024-04-25T08:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa5b400640 1 --2- 172.21.15.46:0/637131960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c15ee20 0x7ffa5c0cf5b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009d90 tx=0x7ffa5401c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76f2cd30a130e3c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa5a000640 1 -- 172.21.15.46:0/637131960 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540b2070 con 0x7ffa5c15ee20 2024-04-25T08:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa5a000640 1 -- 172.21.15.46:0/637131960 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa540a5820 con 0x7ffa5c15ee20 2024-04-25T08:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa5a000640 1 -- 172.21.15.46:0/637131960 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540a5bf0 con 0x7ffa5c15ee20 2024-04-25T08:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 -- 172.21.15.46:0/637131960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c15ee20 msgr2=0x7ffa5c0cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 --2- 172.21.15.46:0/637131960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c15ee20 0x7ffa5c0cf5b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009d90 tx=0x7ffa5401c850 comp rx=0 tx=0).stop 2024-04-25T08:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 -- 172.21.15.46:0/637131960 shutdown_connections 2024-04-25T08:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 --2- 172.21.15.46:0/637131960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c15ee20 0x7ffa5c0cf5b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 -- 172.21.15.46:0/637131960 >> 172.21.15.46:0/637131960 conn(0x7ffa5c0d16b0 msgr2=0x7ffa5c0d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 -- 172.21.15.46:0/637131960 shutdown_connections 2024-04-25T08:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 -- 172.21.15.46:0/637131960 wait complete. 2024-04-25T08:43:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 Processor -- start 2024-04-25T08:43:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 -- start start 2024-04-25T08:43:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.575+0000 7ffa61600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c099a00 0x7ffa5c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa61600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa5c0d0980 0x7ffa5c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa61600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c15ee20 0x7ffa5c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa61600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c06d050 con 0x7ffa5c15ee20 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa61600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c06ced0 con 0x7ffa5c099a00 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa61600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c16c470 con 0x7ffa5c0d0980 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c15ee20 0x7ffa5c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c15ee20 0x7ffa5c159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42890/0 (socket says 172.21.15.46:42890) 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5b400640 1 -- 172.21.15.46:0/297483204 learned_addr learned my addr 172.21.15.46:0/297483204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5aa00640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c099a00 0x7ffa5c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa53e00640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa5c0d0980 0x7ffa5c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5b400640 1 -- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa5c0d0980 msgr2=0x7ffa5c15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5b400640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa5c0d0980 0x7ffa5c15cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5b400640 1 -- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c099a00 msgr2=0x7ffa5c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5b400640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c099a00 0x7ffa5c15c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5b400640 1 -- 172.21.15.46:0/297483204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c0c83e0 con 0x7ffa5c15ee20 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa5b400640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c15ee20 0x7ffa5c159170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa5401a480 tx=0x7ffa540195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa52a00640 1 -- 172.21.15.46:0/297483204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540bd020 con 0x7ffa5c15ee20 2024-04-25T08:43:03.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa52a00640 1 -- 172.21.15.46:0/297483204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa540b3040 con 0x7ffa5c15ee20 2024-04-25T08:43:03.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c15a540 con 0x7ffa5c15ee20 2024-04-25T08:43:03.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa52a00640 1 -- 172.21.15.46:0/297483204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa540b2070 con 0x7ffa5c15ee20 2024-04-25T08:43:03.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c15ab00 con 0x7ffa5c15ee20 2024-04-25T08:43:03.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c069080 con 0x7ffa5c15ee20 2024-04-25T08:43:03.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.579+0000 7ffa52a00640 1 -- 172.21.15.46:0/297483204 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa540c0020 con 0x7ffa5c15ee20 2024-04-25T08:43:03.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.583+0000 7ffa52a00640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa38066f60 0x7ffa38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.583+0000 7ffa52a00640 1 -- 172.21.15.46:0/297483204 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ffa5411fee0 con 0x7ffa5c15ee20 2024-04-25T08:43:03.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.583+0000 7ffa5aa00640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa38066f60 0x7ffa38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.583+0000 7ffa5aa00640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa38066f60 0x7ffa38069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0b4e80 tx=0x7ffa4c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.587+0000 7ffa52a00640 1 -- 172.21.15.46:0/297483204 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa540c7030 con 0x7ffa5c15ee20 2024-04-25T08:43:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.747+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa5c0d23e0 con 0x7ffa5c15ee20 2024-04-25T08:43:03.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.747+0000 7ffa52a00640 1 -- 172.21.15.46:0/297483204 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7ffa54016560 con 0x7ffa5c15ee20 2024-04-25T08:43:03.741 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:03.741 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa38066f60 msgr2=0x7ffa38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa38066f60 0x7ffa38069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0b4e80 tx=0x7ffa4c009040 comp rx=0 tx=0).stop 2024-04-25T08:43:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c15ee20 msgr2=0x7ffa5c159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c15ee20 0x7ffa5c159170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffa5401a480 tx=0x7ffa540195f0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 shutdown_connections 2024-04-25T08:43:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffa38066f60 0x7ffa38069420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa5c15ee20 0x7ffa5c159170 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa5c0d0980 0x7ffa5c15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 --2- 172.21.15.46:0/297483204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa5c099a00 0x7ffa5c15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 >> 172.21.15.46:0/297483204 conn(0x7ffa5c0d16b0 msgr2=0x7ffa5c073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 shutdown_connections 2024-04-25T08:43:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.751+0000 7ffa61600640 1 -- 172.21.15.46:0/297483204 wait complete. 2024-04-25T08:43:03.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:03.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:03.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef8400640 1 Processor -- start 2024-04-25T08:43:03.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef8400640 1 -- start start 2024-04-25T08:43:03.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef8400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ef006f4d0 0x7f1ef00cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef8400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ef015dd50 0x7f1ef00cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef8400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef009a780 con 0x7f1ef00d0280 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef8400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef009aa60 con 0x7f1ef006f4d0 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef8400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef009ad40 con 0x7f1ef015dd50 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ef006f4d0 0x7f1ef00cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ef006f4d0 0x7f1ef00cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42896/0 (socket says 172.21.15.46:42896) 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef5a00640 1 -- 172.21.15.46:0/2466365718 learned_addr learned my addr 172.21.15.46:0/2466365718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef5000640 1 --2- 172.21.15.46:0/2466365718 >> v2:172.21.15.82:3300/0 conn(0x7f1ef015dd50 0x7f1ef00cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef5a00640 1 -- 172.21.15.46:0/2466365718 >> v2:172.21.15.82:3300/0 conn(0x7f1ef015dd50 msgr2=0x7f1ef00cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef5a00640 1 --2- 172.21.15.46:0/2466365718 >> v2:172.21.15.82:3300/0 conn(0x7f1ef015dd50 0x7f1ef00cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef5a00640 1 -- 172.21.15.46:0/2466365718 >> v1:172.21.15.82:6789/0 conn(0x7f1ef00d0280 legacy=0x7f1ef006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef5a00640 1 -- 172.21.15.46:0/2466365718 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef00cbcc0 con 0x7f1ef006f4d0 2024-04-25T08:43:03.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.915+0000 7f1ef5000640 1 --2- 172.21.15.46:0/2466365718 >> v2:172.21.15.82:3300/0 conn(0x7f1ef015dd50 0x7f1ef00cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:03.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef5a00640 1 --2- 172.21.15.46:0/2466365718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 0x7f1ef00cf800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0071720 tx=0x7f1ee401cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca123795bbd254b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1eefe00640 1 -- 172.21.15.46:0/2466365718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40aa670 con 0x7f1ef006f4d0 2024-04-25T08:43:03.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1eefe00640 1 -- 172.21.15.46:0/2466365718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee40af070 con 0x7f1ef006f4d0 2024-04-25T08:43:03.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1eefe00640 1 -- 172.21.15.46:0/2466365718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40aaa30 con 0x7f1ef006f4d0 2024-04-25T08:43:03.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 -- 172.21.15.46:0/2466365718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 msgr2=0x7f1ef00cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 --2- 172.21.15.46:0/2466365718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 0x7f1ef00cf800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0071720 tx=0x7f1ee401cf50 comp rx=0 tx=0).stop 2024-04-25T08:43:03.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 -- 172.21.15.46:0/2466365718 shutdown_connections 2024-04-25T08:43:03.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 --2- 172.21.15.46:0/2466365718 >> v2:172.21.15.82:3300/0 conn(0x7f1ef015dd50 0x7f1ef00cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 --2- 172.21.15.46:0/2466365718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 0x7f1ef00cf800 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 -- 172.21.15.46:0/2466365718 >> 172.21.15.46:0/2466365718 conn(0x7f1ef00b3160 msgr2=0x7f1ef00b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:03.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 -- 172.21.15.46:0/2466365718 shutdown_connections 2024-04-25T08:43:03.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 -- 172.21.15.46:0/2466365718 wait complete. 2024-04-25T08:43:03.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 Processor -- start 2024-04-25T08:43:03.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.919+0000 7f1ef8400640 1 -- start start 2024-04-25T08:43:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef8400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 0x7f1ef0151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef8400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ef00d0280 0x7f1ef015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef8400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ef015dd50 0x7f1ef00b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef8400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef016a820 con 0x7f1ef006f4d0 2024-04-25T08:43:03.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef8400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef016a6a0 con 0x7f1ef015dd50 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef8400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef016a9a0 con 0x7f1ef00d0280 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 0x7f1ef0151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 0x7f1ef0151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42910/0 (socket says 172.21.15.46:42910) 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5a00640 1 -- 172.21.15.46:0/1360934174 learned_addr learned my addr 172.21.15.46:0/1360934174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef6400640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ef015dd50 0x7f1ef00b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5000640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ef00d0280 0x7f1ef015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5a00640 1 -- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ef00d0280 msgr2=0x7f1ef015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5a00640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ef00d0280 0x7f1ef015c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5a00640 1 -- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ef015dd50 msgr2=0x7f1ef00b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5a00640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ef015dd50 0x7f1ef00b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5a00640 1 -- 172.21.15.46:0/1360934174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0153440 con 0x7f1ef006f4d0 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef6400640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ef015dd50 0x7f1ef00b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5000640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ef00d0280 0x7f1ef015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:03.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef5a00640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 0x7f1ef0151fc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4019100 tx=0x7f1ee401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1eed600640 1 -- 172.21.15.46:0/1360934174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40bd5f0 con 0x7f1ef006f4d0 2024-04-25T08:43:03.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1eed600640 1 -- 172.21.15.46:0/1360934174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee40bd790 con 0x7f1ef006f4d0 2024-04-25T08:43:03.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1eed600640 1 -- 172.21.15.46:0/1360934174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40bdb60 con 0x7f1ef006f4d0 2024-04-25T08:43:03.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef0153720 con 0x7f1ef006f4d0 2024-04-25T08:43:03.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef01530d0 con 0x7f1ef006f4d0 2024-04-25T08:43:03.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.923+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef0069010 con 0x7f1ef006f4d0 2024-04-25T08:43:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.927+0000 7f1eed600640 1 -- 172.21.15.46:0/1360934174 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ee40b33f0 con 0x7f1ef006f4d0 2024-04-25T08:43:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.927+0000 7f1eed600640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ed0066f60 0x7f1ed0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.927+0000 7f1eed600640 1 -- 172.21.15.46:0/1360934174 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1ee411fce0 con 0x7f1ef006f4d0 2024-04-25T08:43:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.927+0000 7f1ef5000640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ed0066f60 0x7f1ed0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.927+0000 7f1ef5000640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ed0066f60 0x7f1ed0069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1ef01596f0 tx=0x7f1edc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:03.931+0000 7f1eed600640 1 -- 172.21.15.46:0/1360934174 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ee40c3060 con 0x7f1ef006f4d0 2024-04-25T08:43:04.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.091+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ef00cfe90 con 0x7f1ef006f4d0 2024-04-25T08:43:04.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1eed600640 1 -- 172.21.15.46:0/1360934174 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ee4016110 con 0x7f1ef006f4d0 2024-04-25T08:43:04.086 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ed0066f60 msgr2=0x7f1ed0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ed0066f60 0x7f1ed0069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1ef01596f0 tx=0x7f1edc008040 comp rx=0 tx=0).stop 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 msgr2=0x7f1ef0151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 0x7f1ef0151fc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4019100 tx=0x7f1ee401ad30 comp rx=0 tx=0).stop 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 shutdown_connections 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ed0066f60 0x7f1ed0069420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ef015dd50 0x7f1ef00b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ef00d0280 0x7f1ef015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 --2- 172.21.15.46:0/1360934174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ef006f4d0 0x7f1ef0151fc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 >> 172.21.15.46:0/1360934174 conn(0x7f1ef00b3160 msgr2=0x7f1ef0073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 shutdown_connections 2024-04-25T08:43:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.095+0000 7f1ef8400640 1 -- 172.21.15.46:0/1360934174 wait complete. 2024-04-25T08:43:04.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf32600640 1 Processor -- start 2024-04-25T08:43:04.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf32600640 1 -- start start 2024-04-25T08:43:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf32600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf2c06f150 0x7fcf2c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf32600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2c15e440 con 0x7fcf2c1440d0 2024-04-25T08:43:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf32600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2c15e6e0 con 0x7fcf2c143990 2024-04-25T08:43:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf32600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c15ed30 con 0x7fcf2c06f150 2024-04-25T08:43:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcf2c1440d0 0x7fcf2c0b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48400/0 (socket says 172.21.15.46:48400) 2024-04-25T08:43:04.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2aa00640 1 -- 172.21.15.46:0/1466470982 learned_addr learned my addr 172.21.15.46:0/1466470982 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2be00640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf2c06f150 0x7fcf2c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183996394 0 0) 0x7fcf2c15e440 con 0x7fcf2c1440d0 2024-04-25T08:43:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf10003530 con 0x7fcf2c1440d0 2024-04-25T08:43:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2158623485 0 0) 0x7fcf10003530 con 0x7fcf2c1440d0 2024-04-25T08:43:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf2c06f150 msgr2=0x7fcf2c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf2c06f150 0x7fcf2c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 >> v1:172.21.15.82:6789/0 conn(0x7fcf2c143990 legacy=0x7fcf2c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c15e9c0 con 0x7fcf2c1440d0 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fcf18001680 con 0x7fcf2c1440d0 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 >> v1:172.21.15.46:6789/0 conn(0x7fcf2c1440d0 legacy=0x7fcf2c0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10003a60 0x7fcf10003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf10004810 0x7fcf10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf2c06f150 0x7fcf100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c15ed30 con 0x7fcf10004810 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000dcc0 con 0x7fcf10003a60 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.259+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000ba80 con 0x7fcf2c06f150 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2aa00640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf10004810 0x7fcf10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2b400640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10003a60 0x7fcf10003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2be00640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf2c06f150 0x7fcf100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2aa00640 1 -- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf2c06f150 msgr2=0x7fcf100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2aa00640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf2c06f150 0x7fcf100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2aa00640 1 -- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10003a60 msgr2=0x7fcf10003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2aa00640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf10003a60 0x7fcf10003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2aa00640 1 -- 172.21.15.46:0/1466470982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10003530 con 0x7fcf10004810 2024-04-25T08:43:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2be00640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf2c06f150 0x7fcf100051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2aa00640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf10004810 0x7fcf10004c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c160 tx=0x7fcf1801cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab6101982e0702b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18007c70 con 0x7fcf10004810 2024-04-25T08:43:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf180a9070 con 0x7fcf10004810 2024-04-25T08:43:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.263+0000 7fcf2a000640 1 -- 172.21.15.46:0/1466470982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18005d20 con 0x7fcf10004810 2024-04-25T08:43:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 -- 172.21.15.46:0/1466470982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf10004810 msgr2=0x7fcf10004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 --2- 172.21.15.46:0/1466470982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf10004810 0x7fcf10004c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf1800c160 tx=0x7fcf1801cb60 comp rx=0 tx=0).stop 2024-04-25T08:43:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf2be00640 1 -- 172.21.15.46:0/1466470982 reap_dead start 2024-04-25T08:43:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 -- 172.21.15.46:0/1466470982 shutdown_connections 2024-04-25T08:43:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 -- 172.21.15.46:0/1466470982 >> 172.21.15.46:0/1466470982 conn(0x7fcf2c06ac10 msgr2=0x7fcf2c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 -- 172.21.15.46:0/1466470982 shutdown_connections 2024-04-25T08:43:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 -- 172.21.15.46:0/1466470982 wait complete. 2024-04-25T08:43:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 Processor -- start 2024-04-25T08:43:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 -- start start 2024-04-25T08:43:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf10004810 0x7fcf2c14ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf2c159f50 0x7fcf2c142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf2c13f6e0 0x7fcf2c13fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c143d70 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c15f1c0 con 0x7fcf2c159f50 2024-04-25T08:43:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf32600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c143ef0 con 0x7fcf10004810 2024-04-25T08:43:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf2c13f6e0 0x7fcf2c13fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf2b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf10004810 0x7fcf2c14ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf2c13f6e0 0x7fcf2c13fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42940/0 (socket says 172.21.15.46:42940) 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf2be00640 1 -- 172.21.15.46:0/4035649993 learned_addr learned my addr 172.21.15.46:0/4035649993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.267+0000 7fcf2aa00640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf2c159f50 0x7fcf2c142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf2be00640 1 -- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf10004810 msgr2=0x7fcf2c14ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf2be00640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf10004810 0x7fcf2c14ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf2be00640 1 -- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf2c159f50 msgr2=0x7fcf2c142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf2be00640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf2c159f50 0x7fcf2c142260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf2be00640 1 -- 172.21.15.46:0/4035649993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c142b60 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf2be00640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf2c13f6e0 0x7fcf2c13fb60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c06d060 tx=0x7fcf24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf23400640 1 -- 172.21.15.46:0/4035649993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2401a030 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf23400640 1 -- 172.21.15.46:0/4035649993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf24010e50 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c142e40 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf23400640 1 -- 172.21.15.46:0/4035649993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24003be0 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c143120 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c069010 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf23400640 1 -- 172.21.15.46:0/4035649993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf2401d070 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf23400640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf0c066f60 0x7fcf0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf23400640 1 -- 172.21.15.46:0/4035649993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcf24085a80 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.271+0000 7fcf2b400640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf0c066f60 0x7fcf0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.275+0000 7fcf2b400640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf0c066f60 0x7fcf0c069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c007fd0 tx=0x7fcf1c0013f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.275+0000 7fcf23400640 1 -- 172.21.15.46:0/4035649993 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf24087e20 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.439+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf2c0cc960 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.439+0000 7fcf23400640 1 -- 172.21.15.46:0/4035649993 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf24056030 con 0x7fcf2c13f6e0 2024-04-25T08:43:04.434 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:04.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf0c066f60 msgr2=0x7fcf0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf0c066f60 0x7fcf0c069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c007fd0 tx=0x7fcf1c0013f0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf2c13f6e0 msgr2=0x7fcf2c13fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf2c13f6e0 0x7fcf2c13fb60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c06d060 tx=0x7fcf24009f90 comp rx=0 tx=0).stop 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 shutdown_connections 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf0c066f60 0x7fcf0c069420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf2c13f6e0 0x7fcf2c13fb60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf2c159f50 0x7fcf2c142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 --2- 172.21.15.46:0/4035649993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf10004810 0x7fcf2c14ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 >> 172.21.15.46:0/4035649993 conn(0x7fcf2c06ac10 msgr2=0x7fcf2c13e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 shutdown_connections 2024-04-25T08:43:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.443+0000 7fcf32600640 1 -- 172.21.15.46:0/4035649993 wait complete. 2024-04-25T08:43:04.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:04.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b15c00640 1 Processor -- start 2024-04-25T08:43:04.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b15c00640 1 -- start start 2024-04-25T08:43:04.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b15c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b10145310 0x7f8b1006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b15c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b1006dc70 0x7f8b100cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b15c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b100cd930 con 0x7f8b100cf1b0 2024-04-25T08:43:04.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b15c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b100cdbd0 con 0x7f8b1006dc70 2024-04-25T08:43:04.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b15c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b100cdeb0 con 0x7f8b10145310 2024-04-25T08:43:04.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b0f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8b100cf1b0 0x7f8b1015d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48410/0 (socket says 172.21.15.46:48410) 2024-04-25T08:43:04.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b0f400640 1 -- 172.21.15.46:0/2293495726 learned_addr learned my addr 172.21.15.46:0/2293495726 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:04.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b07e00640 1 --2- 172.21.15.46:0/2293495726 >> v2:172.21.15.46:3300/0 conn(0x7f8b1006dc70 0x7f8b100cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b0ea00640 1 --2- 172.21.15.46:0/2293495726 >> v2:172.21.15.82:3300/0 conn(0x7f8b10145310 0x7f8b1006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b0e000640 1 -- 172.21.15.46:0/2293495726 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825428023 0 0) 0x7f8b100cd930 con 0x7f8b100cf1b0 2024-04-25T08:43:04.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b0e000640 1 -- 172.21.15.46:0/2293495726 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8af4003530 con 0x7f8b100cf1b0 2024-04-25T08:43:04.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b07e00640 1 -- 172.21.15.46:0/2293495726 >> v2:172.21.15.82:3300/0 conn(0x7f8b10145310 msgr2=0x7f8b1006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:04.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b07e00640 1 --2- 172.21.15.46:0/2293495726 >> v2:172.21.15.82:3300/0 conn(0x7f8b10145310 0x7f8b1006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b07e00640 1 -- 172.21.15.46:0/2293495726 >> v1:172.21.15.46:6789/0 conn(0x7f8b100cf1b0 legacy=0x7f8b1015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.607+0000 7f8b07e00640 1 -- 172.21.15.46:0/2293495726 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b100ce190 con 0x7f8b1006dc70 2024-04-25T08:43:04.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b07e00640 1 --2- 172.21.15.46:0/2293495726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b1006dc70 0x7f8b100cec70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b10072530 tx=0x7f8af801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb17d975e9395d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b0e000640 1 -- 172.21.15.46:0/2293495726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af80ab650 con 0x7f8b1006dc70 2024-04-25T08:43:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b0e000640 1 -- 172.21.15.46:0/2293495726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af80b0070 con 0x7f8b1006dc70 2024-04-25T08:43:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b0e000640 1 -- 172.21.15.46:0/2293495726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af80aba10 con 0x7f8b1006dc70 2024-04-25T08:43:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 -- 172.21.15.46:0/2293495726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b1006dc70 msgr2=0x7f8b100cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 --2- 172.21.15.46:0/2293495726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b1006dc70 0x7f8b100cec70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8b10072530 tx=0x7f8af801c450 comp rx=0 tx=0).stop 2024-04-25T08:43:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 -- 172.21.15.46:0/2293495726 shutdown_connections 2024-04-25T08:43:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 --2- 172.21.15.46:0/2293495726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b1006dc70 0x7f8b100cec70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 --2- 172.21.15.46:0/2293495726 >> v2:172.21.15.82:3300/0 conn(0x7f8b10145310 0x7f8b1006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 -- 172.21.15.46:0/2293495726 >> 172.21.15.46:0/2293495726 conn(0x7f8b1006ac10 msgr2=0x7f8b1006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 -- 172.21.15.46:0/2293495726 shutdown_connections 2024-04-25T08:43:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 -- 172.21.15.46:0/2293495726 wait complete. 2024-04-25T08:43:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 Processor -- start 2024-04-25T08:43:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.611+0000 7f8b15c00640 1 -- start start 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b15c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b100cf1b0 0x7f8b10139100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b15c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10145310 0x7f8b1013c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b15c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b101415f0 0x7f8b10141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b15c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1015d920 con 0x7f8b10145310 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b15c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1015d7a0 con 0x7f8b101415f0 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b15c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b10168510 con 0x7f8b100cf1b0 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10145310 0x7f8b1013c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10145310 0x7f8b1013c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42960/0 (socket says 172.21.15.46:42960) 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b07e00640 1 -- 172.21.15.46:0/3094756778 learned_addr learned my addr 172.21.15.46:0/3094756778 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b0f400640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b101415f0 0x7f8b10141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b0ea00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b100cf1b0 0x7f8b10139100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b07e00640 1 -- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b100cf1b0 msgr2=0x7f8b10139100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b07e00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b100cf1b0 0x7f8b10139100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b07e00640 1 -- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b101415f0 msgr2=0x7f8b10141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b07e00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b101415f0 0x7f8b10141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b07e00640 1 -- 172.21.15.46:0/3094756778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b10139640 con 0x7f8b10145310 2024-04-25T08:43:04.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b07e00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10145310 0x7f8b1013c910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8af8019060 tx=0x7f8af801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b06a00640 1 -- 172.21.15.46:0/3094756778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af80bf610 con 0x7f8b10145310 2024-04-25T08:43:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b06a00640 1 -- 172.21.15.46:0/3094756778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af80bf7d0 con 0x7f8b10145310 2024-04-25T08:43:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b10139a20 con 0x7f8b10145310 2024-04-25T08:43:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b06a00640 1 -- 172.21.15.46:0/3094756778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af80bfbc0 con 0x7f8b10145310 2024-04-25T08:43:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b10139fe0 con 0x7f8b10145310 2024-04-25T08:43:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.615+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b10069010 con 0x7f8b10145310 2024-04-25T08:43:04.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.619+0000 7f8b06a00640 1 -- 172.21.15.46:0/3094756778 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8af80bd020 con 0x7f8b10145310 2024-04-25T08:43:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.619+0000 7f8b06a00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8aec066f90 0x7f8aec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.619+0000 7f8b06a00640 1 -- 172.21.15.46:0/3094756778 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8af8120220 con 0x7f8b10145310 2024-04-25T08:43:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.619+0000 7f8b0ea00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8aec066f90 0x7f8aec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.619+0000 7f8b0ea00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8aec066f90 0x7f8aec069450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b1013f1d0 tx=0x7f8b00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.623+0000 7f8b06a00640 1 -- 172.21.15.46:0/3094756778 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8af8016070 con 0x7f8b10145310 2024-04-25T08:43:04.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.783+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b10072050 con 0x7f8b10145310 2024-04-25T08:43:04.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.783+0000 7f8b06a00640 1 -- 172.21.15.46:0/3094756778 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8af80f0eb0 con 0x7f8b10145310 2024-04-25T08:43:04.776 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.783+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8aec066f90 msgr2=0x7f8aec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.783+0000 7f8b15c00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8aec066f90 0x7f8aec069450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b1013f1d0 tx=0x7f8b00008040 comp rx=0 tx=0).stop 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10145310 msgr2=0x7f8b1013c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10145310 0x7f8b1013c910 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8af8019060 tx=0x7f8af801ac90 comp rx=0 tx=0).stop 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 shutdown_connections 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8aec066f90 0x7f8aec069450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b101415f0 0x7f8b10141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b10145310 0x7f8b1013c910 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 --2- 172.21.15.46:0/3094756778 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8b100cf1b0 0x7f8b10139100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 >> 172.21.15.46:0/3094756778 conn(0x7f8b1006ac10 msgr2=0x7f8b100afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 shutdown_connections 2024-04-25T08:43:04.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.787+0000 7f8b15c00640 1 -- 172.21.15.46:0/3094756778 wait complete. 2024-04-25T08:43:04.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:04.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.947+0000 7fce52000640 1 Processor -- start 2024-04-25T08:43:04.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.947+0000 7fce52000640 1 -- start start 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce52000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 0x7fce4c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce52000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fce4c06f150 0x7fce4c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce52000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce4c15e440 con 0x7fce4c143990 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce52000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce4c15ed30 con 0x7fce4c06f150 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce52000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c15e6e0 con 0x7fce4c1440d0 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce4b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce4c143990 0x7fce4c06ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48412/0 (socket says 172.21.15.46:48412) 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce4b400640 1 -- 172.21.15.46:0/1027080752 learned_addr learned my addr 172.21.15.46:0/1027080752 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce4be00640 1 --2- 172.21.15.46:0/1027080752 >> v2:172.21.15.82:3300/0 conn(0x7fce4c06f150 0x7fce4c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce43e00640 1 --2- 172.21.15.46:0/1027080752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 0x7fce4c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce4aa00640 1 -- 172.21.15.46:0/1027080752 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833647031 0 0) 0x7fce4c15e440 con 0x7fce4c143990 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce4aa00640 1 -- 172.21.15.46:0/1027080752 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce30003530 con 0x7fce4c143990 2024-04-25T08:43:04.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce4be00640 1 -- 172.21.15.46:0/1027080752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 msgr2=0x7fce4c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce4be00640 1 --2- 172.21.15.46:0/1027080752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 0x7fce4c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce4be00640 1 -- 172.21.15.46:0/1027080752 >> v1:172.21.15.46:6789/0 conn(0x7fce4c143990 legacy=0x7fce4c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.951+0000 7fce4be00640 1 -- 172.21.15.46:0/1027080752 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c15e9c0 con 0x7fce4c06f150 2024-04-25T08:43:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce4be00640 1 --2- 172.21.15.46:0/1027080752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c06f150 0x7fce4c0b2720 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fce44009d90 tx=0x7fce4401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1e606d001fe21fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce4aa00640 1 -- 172.21.15.46:0/1027080752 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440b2070 con 0x7fce4c06f150 2024-04-25T08:43:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce4aa00640 1 -- 172.21.15.46:0/1027080752 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce440a57a0 con 0x7fce4c06f150 2024-04-25T08:43:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce4aa00640 1 -- 172.21.15.46:0/1027080752 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440a5b50 con 0x7fce4c06f150 2024-04-25T08:43:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 -- 172.21.15.46:0/1027080752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c06f150 msgr2=0x7fce4c0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 --2- 172.21.15.46:0/1027080752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c06f150 0x7fce4c0b2720 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fce44009d90 tx=0x7fce4401c8e0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 -- 172.21.15.46:0/1027080752 shutdown_connections 2024-04-25T08:43:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 --2- 172.21.15.46:0/1027080752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c06f150 0x7fce4c0b2720 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 --2- 172.21.15.46:0/1027080752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 0x7fce4c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 -- 172.21.15.46:0/1027080752 >> 172.21.15.46:0/1027080752 conn(0x7fce4c06ac10 msgr2=0x7fce4c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 -- 172.21.15.46:0/1027080752 shutdown_connections 2024-04-25T08:43:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 -- 172.21.15.46:0/1027080752 wait complete. 2024-04-25T08:43:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 Processor -- start 2024-04-25T08:43:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.955+0000 7fce52000640 1 -- start start 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce52000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce4c06f150 0x7fce4c13b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce52000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c143990 0x7fce4c164260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce52000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 0x7fce4c1647a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce52000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0b35f0 con 0x7fce4c06f150 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce52000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0b3470 con 0x7fce4c143990 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce52000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c0b3770 con 0x7fce4c1440d0 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce4c06f150 0x7fce4c13b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce4c06f150 0x7fce4c13b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42966/0 (socket says 172.21.15.46:42966) 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4b400640 1 -- 172.21.15.46:0/3532805 learned_addr learned my addr 172.21.15.46:0/3532805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce43e00640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c143990 0x7fce4c164260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4be00640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 0x7fce4c1647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4b400640 1 -- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 msgr2=0x7fce4c1647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4b400640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 0x7fce4c1647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4b400640 1 -- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c143990 msgr2=0x7fce4c164260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4b400640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c143990 0x7fce4c164260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4b400640 1 -- 172.21.15.46:0/3532805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c15a4d0 con 0x7fce4c06f150 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce43e00640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c143990 0x7fce4c164260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce4b400640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce4c06f150 0x7fce4c13b2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c450 tx=0x7fce3c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce43400640 1 -- 172.21.15.46:0/3532805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c017070 con 0x7fce4c06f150 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce43400640 1 -- 172.21.15.46:0/3532805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c005cf0 con 0x7fce4c06f150 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce43400640 1 -- 172.21.15.46:0/3532805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c007290 con 0x7fce4c06f150 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c164e70 con 0x7fce4c06f150 2024-04-25T08:43:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c147c00 con 0x7fce4c06f150 2024-04-25T08:43:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4c069010 con 0x7fce4c06f150 2024-04-25T08:43:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.959+0000 7fce43400640 1 -- 172.21.15.46:0/3532805 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce3c01a070 con 0x7fce4c06f150 2024-04-25T08:43:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.963+0000 7fce43400640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce28066f60 0x7fce28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.963+0000 7fce43400640 1 -- 172.21.15.46:0/3532805 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fce3c084ba0 con 0x7fce4c06f150 2024-04-25T08:43:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.963+0000 7fce43e00640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce28066f60 0x7fce28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.963+0000 7fce43e00640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce28066f60 0x7fce28069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce4c147a30 tx=0x7fce34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:04.963+0000 7fce43400640 1 -- 172.21.15.46:0/3532805 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce3c084e20 con 0x7fce4c06f150 2024-04-25T08:43:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fce4c0af780 con 0x7fce4c06f150 2024-04-25T08:43:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce43400640 1 -- 172.21.15.46:0/3532805 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fce3c021080 con 0x7fce4c06f150 2024-04-25T08:43:05.120 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce28066f60 msgr2=0x7fce28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce28066f60 0x7fce28069420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce4c147a30 tx=0x7fce34008040 comp rx=0 tx=0).stop 2024-04-25T08:43:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce4c06f150 msgr2=0x7fce4c13b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce4c06f150 0x7fce4c13b2f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00c450 tx=0x7fce3c00c910 comp rx=0 tx=0).stop 2024-04-25T08:43:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 shutdown_connections 2024-04-25T08:43:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce28066f60 0x7fce28069420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce4c1440d0 0x7fce4c1647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce4c143990 0x7fce4c164260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 --2- 172.21.15.46:0/3532805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce4c06f150 0x7fce4c13b2f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 >> 172.21.15.46:0/3532805 conn(0x7fce4c06ac10 msgr2=0x7fce4c0afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 shutdown_connections 2024-04-25T08:43:05.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.127+0000 7fce52000640 1 -- 172.21.15.46:0/3532805 wait complete. 2024-04-25T08:43:05.141 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.287+0000 7ff9fe000640 1 Processor -- start 2024-04-25T08:43:05.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.287+0000 7ff9fe000640 1 -- start start 2024-04-25T08:43:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9fe000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8159230 0x7ff9f815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9fe000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff9f80cdc50 0x7ff9f815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9fe000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f80cc700 con 0x7ff9f80cbeb0 2024-04-25T08:43:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9fe000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f80cc9e0 con 0x7ff9f80cdc50 2024-04-25T08:43:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9fe000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80cccc0 con 0x7ff9f8159230 2024-04-25T08:43:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9eea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9f80cbeb0 0x7ff9f80cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48426/0 (socket says 172.21.15.46:48426) 2024-04-25T08:43:05.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9eea00640 1 -- 172.21.15.46:0/1982752315 learned_addr learned my addr 172.21.15.46:0/1982752315 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f7400640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8159230 0x7ff9f815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f7e00640 1 --2- 172.21.15.46:0/1982752315 >> v2:172.21.15.82:3300/0 conn(0x7ff9f80cdc50 0x7ff9f815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985746006 0 0) 0x7ff9f80cc700 con 0x7ff9f80cbeb0 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9dc003530 con 0x7ff9f80cbeb0 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1067125585 0 0) 0x7ff9dc003530 con 0x7ff9f80cbeb0 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8159230 msgr2=0x7ff9f815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8159230 0x7ff9f815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 >> v2:172.21.15.82:3300/0 conn(0x7ff9f80cdc50 msgr2=0x7ff9f815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 --2- 172.21.15.46:0/1982752315 >> v2:172.21.15.82:3300/0 conn(0x7ff9f80cdc50 0x7ff9f815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80ccfa0 con 0x7ff9f80cbeb0 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7ff9e0001680 con 0x7ff9f80cbeb0 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 >> v1:172.21.15.46:6789/0 conn(0x7ff9f80cbeb0 legacy=0x7ff9f80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8159230 0x7ff9dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9dc004d50 0x7ff9dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80cc9e0 con 0x7ff9dc0039c0 2024-04-25T08:43:05.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80cccc0 con 0x7ff9dc004d50 2024-04-25T08:43:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9dc00ba00 con 0x7ff9f8159230 2024-04-25T08:43:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f7400640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f7e00640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9dc004d50 0x7ff9dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9eea00640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8159230 0x7ff9dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f7400640 1 -- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8159230 msgr2=0x7ff9dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f7400640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8159230 0x7ff9dc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f7400640 1 -- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9dc004d50 msgr2=0x7ff9dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f7400640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9dc004d50 0x7ff9dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.291+0000 7ff9f7400640 1 -- 172.21.15.46:0/1982752315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc003530 con 0x7ff9dc0039c0 2024-04-25T08:43:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9f7400640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9dc003de0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009ae0 tx=0x7ff9e801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d166eb4e35aa49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80b2070 con 0x7ff9dc0039c0 2024-04-25T08:43:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e80a5520 con 0x7ff9dc0039c0 2024-04-25T08:43:05.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9f6a00640 1 -- 172.21.15.46:0/1982752315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80096b0 con 0x7ff9dc0039c0 2024-04-25T08:43:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9fe000640 1 -- 172.21.15.46:0/1982752315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 msgr2=0x7ff9dc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9fe000640 1 --2- 172.21.15.46:0/1982752315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9dc003de0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009ae0 tx=0x7ff9e801ca40 comp rx=0 tx=0).stop 2024-04-25T08:43:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9f7e00640 1 -- 172.21.15.46:0/1982752315 reap_dead start 2024-04-25T08:43:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9fe000640 1 -- 172.21.15.46:0/1982752315 shutdown_connections 2024-04-25T08:43:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9fe000640 1 -- 172.21.15.46:0/1982752315 >> 172.21.15.46:0/1982752315 conn(0x7ff9f80b12a0 msgr2=0x7ff9f80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9fe000640 1 -- 172.21.15.46:0/1982752315 shutdown_connections 2024-04-25T08:43:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.295+0000 7ff9fe000640 1 -- 172.21.15.46:0/1982752315 wait complete. 2024-04-25T08:43:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 Processor -- start 2024-04-25T08:43:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 -- start start 2024-04-25T08:43:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9f815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f8141960 0x7ff9f80b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8157d10 0x7ff9f81581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f815df50 con 0x7ff9dc0039c0 2024-04-25T08:43:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f80cc4a0 con 0x7ff9f8141960 2024-04-25T08:43:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f815e0d0 con 0x7ff9f8157d10 2024-04-25T08:43:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9f815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9f815b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42990/0 (socket says 172.21.15.46:42990) 2024-04-25T08:43:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7400640 1 -- 172.21.15.46:0/638473450 learned_addr learned my addr 172.21.15.46:0/638473450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9eea00640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f8141960 0x7ff9f80b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7e00640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8157d10 0x7ff9f81581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7400640 1 -- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8157d10 msgr2=0x7ff9f81581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7400640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8157d10 0x7ff9f81581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7400640 1 -- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f8141960 msgr2=0x7ff9f80b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7400640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f8141960 0x7ff9f80b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7400640 1 -- 172.21.15.46:0/638473450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f8141df0 con 0x7ff9dc0039c0 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9eea00640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f8141960 0x7ff9f80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9f7400640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9f815b740 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80b0040 tx=0x7ff9dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9efe00640 1 -- 172.21.15.46:0/638473450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80b2040 con 0x7ff9dc0039c0 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9efe00640 1 -- 172.21.15.46:0/638473450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e800f000 con 0x7ff9dc0039c0 2024-04-25T08:43:05.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9efe00640 1 -- 172.21.15.46:0/638473450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8004020 con 0x7ff9dc0039c0 2024-04-25T08:43:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f81420d0 con 0x7ff9dc0039c0 2024-04-25T08:43:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f8142690 con 0x7ff9dc0039c0 2024-04-25T08:43:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.299+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f8069080 con 0x7ff9dc0039c0 2024-04-25T08:43:05.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.303+0000 7ff9efe00640 1 -- 172.21.15.46:0/638473450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9e80c0030 con 0x7ff9dc0039c0 2024-04-25T08:43:05.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.303+0000 7ff9efe00640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9d4066f60 0x7ff9d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.303+0000 7ff9efe00640 1 -- 172.21.15.46:0/638473450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff9e811f040 con 0x7ff9dc0039c0 2024-04-25T08:43:05.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.303+0000 7ff9eea00640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9d4066f60 0x7ff9d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.303+0000 7ff9eea00640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9d4066f60 0x7ff9d4069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8141910 tx=0x7ff9e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:05.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.307+0000 7ff9efe00640 1 -- 172.21.15.46:0/638473450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9e80c7060 con 0x7ff9dc0039c0 2024-04-25T08:43:05.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.471+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9f8144420 con 0x7ff9dc0039c0 2024-04-25T08:43:05.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.471+0000 7ff9efe00640 1 -- 172.21.15.46:0/638473450 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7ff9e8016780 con 0x7ff9dc0039c0 2024-04-25T08:43:05.494 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:05.494 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9d4066f60 msgr2=0x7ff9d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9d4066f60 0x7ff9d4069420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8141910 tx=0x7ff9e0009040 comp rx=0 tx=0).stop 2024-04-25T08:43:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 msgr2=0x7ff9f815b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9f815b740 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80b0040 tx=0x7ff9dc005840 comp rx=0 tx=0).stop 2024-04-25T08:43:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 shutdown_connections 2024-04-25T08:43:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9d4066f60 0x7ff9d4069420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9f8157d10 0x7ff9f81581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f8141960 0x7ff9f80b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 --2- 172.21.15.46:0/638473450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc0039c0 0x7ff9f815b740 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 >> 172.21.15.46:0/638473450 conn(0x7ff9f80b12a0 msgr2=0x7ff9f8073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 shutdown_connections 2024-04-25T08:43:05.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.475+0000 7ff9fe000640 1 -- 172.21.15.46:0/638473450 wait complete. 2024-04-25T08:43:05.497 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:05.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.647+0000 7fca37000640 1 Processor -- start 2024-04-25T08:43:05.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.647+0000 7fca37000640 1 -- start start 2024-04-25T08:43:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca37000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca3006dc70 0x7fca300cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca37000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca300cd930 con 0x7fca30145310 2024-04-25T08:43:05.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca37000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca300cdbd0 con 0x7fca300cf1b0 2024-04-25T08:43:05.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca37000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca300cdeb0 con 0x7fca3006dc70 2024-04-25T08:43:05.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fca30145310 0x7fca3006e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48428/0 (socket says 172.21.15.46:48428) 2024-04-25T08:43:05.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2fe00640 1 -- 172.21.15.46:0/156550040 learned_addr learned my addr 172.21.15.46:0/156550040 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:05.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2f400640 1 --2- 172.21.15.46:0/156550040 >> v2:172.21.15.82:3300/0 conn(0x7fca3006dc70 0x7fca300cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2ea00640 1 -- 172.21.15.46:0/156550040 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685015566 0 0) 0x7fca300cd930 con 0x7fca30145310 2024-04-25T08:43:05.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2ea00640 1 -- 172.21.15.46:0/156550040 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca14003530 con 0x7fca30145310 2024-04-25T08:43:05.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2ea00640 1 -- 172.21.15.46:0/156550040 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180704227 0 0) 0x7fca300cdbd0 con 0x7fca300cf1b0 2024-04-25T08:43:05.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2ea00640 1 -- 172.21.15.46:0/156550040 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca300cd930 con 0x7fca300cf1b0 2024-04-25T08:43:05.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2f400640 1 -- 172.21.15.46:0/156550040 >> v1:172.21.15.82:6789/0 conn(0x7fca300cf1b0 legacy=0x7fca3015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2f400640 1 -- 172.21.15.46:0/156550040 >> v1:172.21.15.46:6789/0 conn(0x7fca30145310 legacy=0x7fca3006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.651+0000 7fca2f400640 1 -- 172.21.15.46:0/156550040 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300ce190 con 0x7fca3006dc70 2024-04-25T08:43:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca2f400640 1 --2- 172.21.15.46:0/156550040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca3006dc70 0x7fca300cec70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b1325091f96d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca2ea00640 1 -- 172.21.15.46:0/156550040 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0a7070 con 0x7fca3006dc70 2024-04-25T08:43:05.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca2ea00640 1 -- 172.21.15.46:0/156550040 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0b0070 con 0x7fca3006dc70 2024-04-25T08:43:05.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca2ea00640 1 -- 172.21.15.46:0/156550040 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0ab890 con 0x7fca3006dc70 2024-04-25T08:43:05.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 -- 172.21.15.46:0/156550040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca3006dc70 msgr2=0x7fca300cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 --2- 172.21.15.46:0/156550040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca3006dc70 0x7fca300cec70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c470 comp rx=0 tx=0).stop 2024-04-25T08:43:05.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 -- 172.21.15.46:0/156550040 shutdown_connections 2024-04-25T08:43:05.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 --2- 172.21.15.46:0/156550040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca3006dc70 0x7fca300cec70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 -- 172.21.15.46:0/156550040 >> 172.21.15.46:0/156550040 conn(0x7fca3006ac10 msgr2=0x7fca3006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:05.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 -- 172.21.15.46:0/156550040 shutdown_connections 2024-04-25T08:43:05.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 -- 172.21.15.46:0/156550040 wait complete. 2024-04-25T08:43:05.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 Processor -- start 2024-04-25T08:43:05.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 -- start start 2024-04-25T08:43:05.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca300cf1b0 0x7fca30138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30145310 0x7fca3013c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30157610 0x7fca300b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca301696a0 con 0x7fca30145310 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca30169520 con 0x7fca30157610 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca37000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca30169820 con 0x7fca300cf1b0 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.655+0000 7fca2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30145310 0x7fca3013c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30145310 0x7fca3013c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42992/0 (socket says 172.21.15.46:42992) 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2f400640 1 -- 172.21.15.46:0/1143877463 learned_addr learned my addr 172.21.15.46:0/1143877463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2fe00640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca300cf1b0 0x7fca30138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca35000640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30157610 0x7fca300b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2f400640 1 -- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca300cf1b0 msgr2=0x7fca30138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2f400640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca300cf1b0 0x7fca30138860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2f400640 1 -- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30157610 msgr2=0x7fca300b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2f400640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30157610 0x7fca300b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2f400640 1 -- 172.21.15.46:0/1143877463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca30159140 con 0x7fca30145310 2024-04-25T08:43:05.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2fe00640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca300cf1b0 0x7fca30138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:05.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca35000640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30157610 0x7fca300b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:05.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca2f400640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30145310 0x7fca3013c5b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0ad040 tx=0x7fca1c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca27e00640 1 -- 172.21.15.46:0/1143877463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0bf610 con 0x7fca30145310 2024-04-25T08:43:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca27e00640 1 -- 172.21.15.46:0/1143877463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0bf7f0 con 0x7fca30145310 2024-04-25T08:43:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca30159420 con 0x7fca30145310 2024-04-25T08:43:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca27e00640 1 -- 172.21.15.46:0/1143877463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0bfbe0 con 0x7fca30145310 2024-04-25T08:43:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca300b29c0 con 0x7fca30145310 2024-04-25T08:43:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca30069010 con 0x7fca30145310 2024-04-25T08:43:05.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca27e00640 1 -- 172.21.15.46:0/1143877463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca1c0bd020 con 0x7fca30145310 2024-04-25T08:43:05.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca27e00640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.659+0000 7fca27e00640 1 -- 172.21.15.46:0/1143877463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fca1c11fed0 con 0x7fca30145310 2024-04-25T08:43:05.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.663+0000 7fca2fe00640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.663+0000 7fca2fe00640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f60 0x7fca10069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fca20000f70 tx=0x7fca20009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:05.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.663+0000 7fca27e00640 1 -- 172.21.15.46:0/1143877463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca1c016090 con 0x7fca30145310 2024-04-25T08:43:05.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca301423c0 con 0x7fca30145310 2024-04-25T08:43:05.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca27e00640 1 -- 172.21.15.46:0/1143877463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca1c0f0b60 con 0x7fca30145310 2024-04-25T08:43:05.820 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f60 msgr2=0x7fca10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f60 0x7fca10069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fca20000f70 tx=0x7fca20009040 comp rx=0 tx=0).stop 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30145310 msgr2=0x7fca3013c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30145310 0x7fca3013c5b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0ad040 tx=0x7fca1c01acb0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 shutdown_connections 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f60 0x7fca10069420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30157610 0x7fca300b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca30145310 0x7fca3013c5b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 --2- 172.21.15.46:0/1143877463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca300cf1b0 0x7fca30138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 >> 172.21.15.46:0/1143877463 conn(0x7fca3006ac10 msgr2=0x7fca30071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:05.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 shutdown_connections 2024-04-25T08:43:05.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.827+0000 7fca37000640 1 -- 172.21.15.46:0/1143877463 wait complete. 2024-04-25T08:43:05.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5437800640 1 Processor -- start 2024-04-25T08:43:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5437800640 1 -- start start 2024-04-25T08:43:05.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5437800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015d900 0x7f54300af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5437800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f543006b470 0x7f54300d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5437800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54300b0040 con 0x7f54300afc20 2024-04-25T08:43:05.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5437800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543006fca0 con 0x7f543006b470 2024-04-25T08:43:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5437800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543006ff80 con 0x7f543015d900 2024-04-25T08:43:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5427e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f54300afc20 0x7f543006ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48436/0 (socket says 172.21.15.46:48436) 2024-04-25T08:43:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5427e00640 1 -- 172.21.15.46:0/2433018238 learned_addr learned my addr 172.21.15.46:0/2433018238 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5435800640 1 --2- 172.21.15.46:0/2433018238 >> v2:172.21.15.82:3300/0 conn(0x7f543006b470 0x7f54300d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5434e00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015d900 0x7f54300af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517969282 0 0) 0x7f54300b0040 con 0x7f54300afc20 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5414003530 con 0x7f54300afc20 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 401205487 0 0) 0x7f5414003530 con 0x7f54300afc20 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015d900 msgr2=0x7f54300af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015d900 0x7f54300af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 >> v2:172.21.15.82:3300/0 conn(0x7f543006b470 msgr2=0x7f54300d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 --2- 172.21.15.46:0/2433018238 >> v2:172.21.15.82:3300/0 conn(0x7f543006b470 0x7f54300d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5430070260 con 0x7f54300afc20 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f5418001680 con 0x7f54300afc20 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5435800640 1 --2- 172.21.15.46:0/2433018238 >> v2:172.21.15.82:3300/0 conn(0x7f543006b470 0x7f54300d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 >> v1:172.21.15.46:6789/0 conn(0x7f54300afc20 legacy=0x7f543006ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5414003a30 0x7f5414003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015d900 0x7f5414004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5414004d50 0x7f54140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543006fca0 con 0x7f5414003a30 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543006ff80 con 0x7f5414004d50 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541400ba20 con 0x7f543015d900 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5434e00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5414003a30 0x7f5414003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5427e00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015d900 0x7f5414004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.995+0000 7f5435800640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5414004d50 0x7f54140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5434e00640 1 -- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015d900 msgr2=0x7f5414004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5434e00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015d900 0x7f5414004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5434e00640 1 -- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5414004d50 msgr2=0x7f54140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5434e00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5414004d50 0x7f54140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5434e00640 1 -- 172.21.15.46:0/2433018238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414003530 con 0x7f5414003a30 2024-04-25T08:43:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5427e00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015d900 0x7f5414004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:05.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5434e00640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5414003a30 0x7f5414003e50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5420009ae0 tx=0x7f542001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6f9f1d16b2e582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:05.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54200096b0 con 0x7f5414003a30 2024-04-25T08:43:05.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54200a6450 con 0x7f5414003a30 2024-04-25T08:43:05.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f542fe00640 1 -- 172.21.15.46:0/2433018238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54200a6840 con 0x7f5414003a30 2024-04-25T08:43:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5437800640 1 -- 172.21.15.46:0/2433018238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5414003a30 msgr2=0x7f5414003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5437800640 1 --2- 172.21.15.46:0/2433018238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5414003a30 0x7f5414003e50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5420009ae0 tx=0x7f542001ca70 comp rx=0 tx=0).stop 2024-04-25T08:43:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5435800640 1 -- 172.21.15.46:0/2433018238 reap_dead start 2024-04-25T08:43:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5437800640 1 -- 172.21.15.46:0/2433018238 shutdown_connections 2024-04-25T08:43:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5437800640 1 -- 172.21.15.46:0/2433018238 >> 172.21.15.46:0/2433018238 conn(0x7f543006e3e0 msgr2=0x7f543006e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5437800640 1 -- 172.21.15.46:0/2433018238 shutdown_connections 2024-04-25T08:43:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:05.999+0000 7f5437800640 1 -- 172.21.15.46:0/2433018238 wait complete. 2024-04-25T08:43:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 Processor -- start 2024-04-25T08:43:05.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 -- start start 2024-04-25T08:43:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5414003a30 0x7f54301f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015c720 0x7f543015cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54300cba00 0x7f54300cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54300d10e0 con 0x7f54300cba00 2024-04-25T08:43:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54300d0f60 con 0x7f5414003a30 2024-04-25T08:43:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54300d1260 con 0x7f543015c720 2024-04-25T08:43:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5435800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54300cba00 0x7f54300cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5435800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54300cba00 0x7f54300cbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43006/0 (socket says 172.21.15.46:43006) 2024-04-25T08:43:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5435800640 1 -- 172.21.15.46:0/159619915 learned_addr learned my addr 172.21.15.46:0/159619915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5427e00640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015c720 0x7f543015cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5434e00640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5414003a30 0x7f54301f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5435800640 1 -- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015c720 msgr2=0x7f543015cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5435800640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015c720 0x7f543015cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5435800640 1 -- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5414003a30 msgr2=0x7f54301f7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5435800640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5414003a30 0x7f54301f7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5435800640 1 -- 172.21.15.46:0/159619915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54300cac40 con 0x7f54300cba00 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5435800640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54300cba00 0x7f54300cbe80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f54300cfc00 tx=0x7f542800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f542d600640 1 -- 172.21.15.46:0/159619915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800fd70 con 0x7f54300cba00 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f542d600640 1 -- 172.21.15.46:0/159619915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54280039a0 con 0x7f54300cba00 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f542d600640 1 -- 172.21.15.46:0/159619915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428003d90 con 0x7f54300cba00 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54300caf20 con 0x7f54300cba00 2024-04-25T08:43:05.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.003+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54300cc3c0 con 0x7f54300cba00 2024-04-25T08:43:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.007+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5430069010 con 0x7f54300cba00 2024-04-25T08:43:06.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.007+0000 7f542d600640 1 -- 172.21.15.46:0/159619915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f542801f020 con 0x7f54300cba00 2024-04-25T08:43:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.007+0000 7f542d600640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f540c066f90 0x7f540c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.007+0000 7f542d600640 1 -- 172.21.15.46:0/159619915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5428085650 con 0x7f54300cba00 2024-04-25T08:43:06.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.007+0000 7f5434e00640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f540c066f90 0x7f540c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.011+0000 7f5434e00640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f540c066f90 0x7f540c069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f54200ad000 tx=0x7f54200b2040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:06.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.011+0000 7f542d600640 1 -- 172.21.15.46:0/159619915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5428010040 con 0x7f54300cba00 2024-04-25T08:43:06.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5430072890 con 0x7f54300cba00 2024-04-25T08:43:06.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f542d600640 1 -- 172.21.15.46:0/159619915 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5428055b70 con 0x7f54300cba00 2024-04-25T08:43:06.168 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f540c066f90 msgr2=0x7f540c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f540c066f90 0x7f540c069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f54200ad000 tx=0x7f54200b2040 comp rx=0 tx=0).stop 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54300cba00 msgr2=0x7f54300cbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54300cba00 0x7f54300cbe80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f54300cfc00 tx=0x7f542800f580 comp rx=0 tx=0).stop 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 shutdown_connections 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f540c066f90 0x7f540c069450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54300cba00 0x7f54300cbe80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f543015c720 0x7f543015cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 --2- 172.21.15.46:0/159619915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5414003a30 0x7f54301f7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 >> 172.21.15.46:0/159619915 conn(0x7f543006e3e0 msgr2=0x7f5430161380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:06.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 shutdown_connections 2024-04-25T08:43:06.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.175+0000 7f5437800640 1 -- 172.21.15.46:0/159619915 wait complete. 2024-04-25T08:43:06.189 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.339+0000 7fe0b0800640 1 Processor -- start 2024-04-25T08:43:06.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.339+0000 7fe0b0800640 1 -- start start 2024-04-25T08:43:06.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.339+0000 7fe0b0800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a815d900 0x7fe0a80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.339+0000 7fe0b0800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0a80afc20 0x7fe0a806ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.339+0000 7fe0b0800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a80b0040 con 0x7fe0a806b2a0 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.339+0000 7fe0b0800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a806b6c0 con 0x7fe0a80afc20 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.339+0000 7fe0b0800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a806fca0 con 0x7fe0a815d900 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ad400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0a80afc20 0x7fe0a806ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ad400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0a80afc20 0x7fe0a806ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43012/0 (socket says 172.21.15.46:43012) 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ad400640 1 -- 172.21.15.46:0/4279854705 learned_addr learned my addr 172.21.15.46:0/4279854705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ade00640 1 --2- 172.21.15.46:0/4279854705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a815d900 0x7fe0a80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ad400640 1 -- 172.21.15.46:0/4279854705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a815d900 msgr2=0x7fe0a80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ad400640 1 --2- 172.21.15.46:0/4279854705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a815d900 0x7fe0a80af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ad400640 1 -- 172.21.15.46:0/4279854705 >> v1:172.21.15.82:6789/0 conn(0x7fe0a806b2a0 legacy=0x7fe0a815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ad400640 1 -- 172.21.15.46:0/4279854705 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a806ff80 con 0x7fe0a80afc20 2024-04-25T08:43:06.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ade00640 1 --2- 172.21.15.46:0/4279854705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a815d900 0x7fe0a80af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:06.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.343+0000 7fe0ad400640 1 --2- 172.21.15.46:0/4279854705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a80afc20 0x7fe0a806ad60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe098009870 tx=0x7fe09801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4344c105a0a6e7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:06.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0aca00640 1 -- 172.21.15.46:0/4279854705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980b1070 con 0x7fe0a80afc20 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0aca00640 1 -- 172.21.15.46:0/4279854705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0980ac620 con 0x7fe0a80afc20 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0aca00640 1 -- 172.21.15.46:0/4279854705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0980ac9f0 con 0x7fe0a80afc20 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 -- 172.21.15.46:0/4279854705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a80afc20 msgr2=0x7fe0a806ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 --2- 172.21.15.46:0/4279854705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a80afc20 0x7fe0a806ad60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe098009870 tx=0x7fe09801c4d0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 -- 172.21.15.46:0/4279854705 shutdown_connections 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 --2- 172.21.15.46:0/4279854705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a80afc20 0x7fe0a806ad60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 --2- 172.21.15.46:0/4279854705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a815d900 0x7fe0a80af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 -- 172.21.15.46:0/4279854705 >> 172.21.15.46:0/4279854705 conn(0x7fe0a806e3e0 msgr2=0x7fe0a806e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 -- 172.21.15.46:0/4279854705 shutdown_connections 2024-04-25T08:43:06.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 -- 172.21.15.46:0/4279854705 wait complete. 2024-04-25T08:43:06.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 Processor -- start 2024-04-25T08:43:06.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 -- start start 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0a806b2a0 0x7fe0a814e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a80afc20 0x7fe0a8146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a815d900 0x7fe0a8149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80d13a0 con 0x7fe0a815d900 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a80d1220 con 0x7fe0a806b2a0 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0b0800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a816a900 con 0x7fe0a80afc20 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0ae800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a815d900 0x7fe0a8149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0ae800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a815d900 0x7fe0a8149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43024/0 (socket says 172.21.15.46:43024) 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0a806b2a0 0x7fe0a814e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0ae800640 1 -- 172.21.15.46:0/4263558326 learned_addr learned my addr 172.21.15.46:0/4263558326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.347+0000 7fe0ad400640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a80afc20 0x7fe0a8146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0ae800640 1 -- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a80afc20 msgr2=0x7fe0a8146d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0ae800640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a80afc20 0x7fe0a8146d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0ae800640 1 -- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0a806b2a0 msgr2=0x7fe0a814e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0ae800640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0a806b2a0 0x7fe0a814e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0ae800640 1 -- 172.21.15.46:0/4263558326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a814ec80 con 0x7fe0a815d900 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0ae800640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a815d900 0x7fe0a8149640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400c770 tx=0x7fe0a400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0a2000640 1 -- 172.21.15.46:0/4263558326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4003b80 con 0x7fe0a815d900 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0a2000640 1 -- 172.21.15.46:0/4263558326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a4003d20 con 0x7fe0a815d900 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0a2000640 1 -- 172.21.15.46:0/4263558326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a400aa90 con 0x7fe0a815d900 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a80ce8c0 con 0x7fe0a815d900 2024-04-25T08:43:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a80cee80 con 0x7fe0a815d900 2024-04-25T08:43:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a8069010 con 0x7fe0a815d900 2024-04-25T08:43:06.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0a2000640 1 -- 172.21.15.46:0/4263558326 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0a400ac30 con 0x7fe0a815d900 2024-04-25T08:43:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0a2000640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe08c066f60 0x7fe08c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0a2000640 1 -- 172.21.15.46:0/4263558326 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe0a4085320 con 0x7fe0a815d900 2024-04-25T08:43:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.351+0000 7fe0ade00640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe08c066f60 0x7fe08c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.355+0000 7fe0ade00640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe08c066f60 0x7fe08c069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0045f0 tx=0x7fe09c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:06.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.355+0000 7fe0a2000640 1 -- 172.21.15.46:0/4263558326 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe0a40557b0 con 0x7fe0a815d900 2024-04-25T08:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0a8072890 con 0x7fe0a815d900 2024-04-25T08:43:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0a2000640 1 -- 172.21.15.46:0/4263558326 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe0a4059850 con 0x7fe0a815d900 2024-04-25T08:43:06.511 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe08c066f60 msgr2=0x7fe08c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe08c066f60 0x7fe08c069420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0045f0 tx=0x7fe09c008040 comp rx=0 tx=0).stop 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a815d900 msgr2=0x7fe0a8149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a815d900 0x7fe0a8149640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400c770 tx=0x7fe0a400cc30 comp rx=0 tx=0).stop 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 shutdown_connections 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe08c066f60 0x7fe08c069420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0a815d900 0x7fe0a8149640 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0a80afc20 0x7fe0a8146d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 --2- 172.21.15.46:0/4263558326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0a806b2a0 0x7fe0a814e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 >> 172.21.15.46:0/4263558326 conn(0x7fe0a806e3e0 msgr2=0x7fe0a80b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 shutdown_connections 2024-04-25T08:43:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.519+0000 7fe0b0800640 1 -- 172.21.15.46:0/4263558326 wait complete. 2024-04-25T08:43:06.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.683+0000 7fe1c1000640 1 Processor -- start 2024-04-25T08:43:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.683+0000 7fe1c1000640 1 -- start start 2024-04-25T08:43:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.683+0000 7fe1c1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1bc15dae0 0x7fe1bc072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.683+0000 7fe1c1000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc0b3850 con 0x7fe1bc06ea60 2024-04-25T08:43:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.683+0000 7fe1c1000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc0b3af0 con 0x7fe1bc06e620 2024-04-25T08:43:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.683+0000 7fe1c1000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1bc06e190 con 0x7fe1bc15dae0 2024-04-25T08:43:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.683+0000 7fe1b9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1bc15dae0 0x7fe1bc072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.683+0000 7fe1ba800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe1bc06ea60 0x7fe1bc0acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48440/0 (socket says 172.21.15.46:48440) 2024-04-25T08:43:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.683+0000 7fe1ba800640 1 -- 172.21.15.46:0/470872952 learned_addr learned my addr 172.21.15.46:0/470872952 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1b9e00640 1 -- 172.21.15.46:0/470872952 >> v1:172.21.15.82:6789/0 conn(0x7fe1bc06e620 legacy=0x7fe1bc06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1b9e00640 1 -- 172.21.15.46:0/470872952 >> v1:172.21.15.46:6789/0 conn(0x7fe1bc06ea60 legacy=0x7fe1bc0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1b9e00640 1 -- 172.21.15.46:0/470872952 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0b3dd0 con 0x7fe1bc15dae0 2024-04-25T08:43:06.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1b9400640 1 -- 172.21.15.46:0/470872952 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682127159 0 0) 0x7fe1bc0b3850 con 0x7fe1bc06ea60 2024-04-25T08:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/470872952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc15dae0 0x7fe1bc072600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009d90 tx=0x7fe1ac01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f8c005ba2b9f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1b9400640 1 -- 172.21.15.46:0/470872952 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0b2070 con 0x7fe1bc15dae0 2024-04-25T08:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1b9400640 1 -- 172.21.15.46:0/470872952 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac0a5a00 con 0x7fe1bc15dae0 2024-04-25T08:43:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1b9400640 1 -- 172.21.15.46:0/470872952 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0a5dd0 con 0x7fe1bc15dae0 2024-04-25T08:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1c1000640 1 -- 172.21.15.46:0/470872952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc15dae0 msgr2=0x7fe1bc072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1c1000640 1 --2- 172.21.15.46:0/470872952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc15dae0 0x7fe1bc072600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac009d90 tx=0x7fe1ac01c940 comp rx=0 tx=0).stop 2024-04-25T08:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1c1000640 1 -- 172.21.15.46:0/470872952 shutdown_connections 2024-04-25T08:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1c1000640 1 --2- 172.21.15.46:0/470872952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc15dae0 0x7fe1bc072600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1c1000640 1 -- 172.21.15.46:0/470872952 >> 172.21.15.46:0/470872952 conn(0x7fe1bc071ae0 msgr2=0x7fe1bc071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1c1000640 1 -- 172.21.15.46:0/470872952 shutdown_connections 2024-04-25T08:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.687+0000 7fe1c1000640 1 -- 172.21.15.46:0/470872952 wait complete. 2024-04-25T08:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1c1000640 1 Processor -- start 2024-04-25T08:43:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1c1000640 1 -- start start 2024-04-25T08:43:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc06e620 0x7fe1bc15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1c1000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1bc06ea60 0x7fe1bc153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1c1000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc15dae0 0x7fe1bc1490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1c1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc06f6d0 con 0x7fe1bc06e620 2024-04-25T08:43:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1c1000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc15dec0 con 0x7fe1bc15dae0 2024-04-25T08:43:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1c1000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc06f850 con 0x7fe1bc06ea60 2024-04-25T08:43:06.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc06e620 0x7fe1bc15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc06e620 0x7fe1bc15e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43028/0 (socket says 172.21.15.46:43028) 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b9e00640 1 -- 172.21.15.46:0/638291999 learned_addr learned my addr 172.21.15.46:0/638291999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b1400640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1bc06ea60 0x7fe1bc153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1ba800640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc15dae0 0x7fe1bc1490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b9e00640 1 -- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1bc06ea60 msgr2=0x7fe1bc153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1bc06ea60 0x7fe1bc153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b9e00640 1 -- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc15dae0 msgr2=0x7fe1bc1490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc15dae0 0x7fe1bc1490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b9e00640 1 -- 172.21.15.46:0/638291999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc14c700 con 0x7fe1bc06e620 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1ba800640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc15dae0 0x7fe1bc1490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b9e00640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc06e620 0x7fe1bc15e0d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac019550 tx=0x7fe1ac0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b2a00640 1 -- 172.21.15.46:0/638291999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0b2040 con 0x7fe1bc06e620 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b2a00640 1 -- 172.21.15.46:0/638291999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac0bf070 con 0x7fe1bc06e620 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1b2a00640 1 -- 172.21.15.46:0/638291999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0ada80 con 0x7fe1bc06e620 2024-04-25T08:43:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.691+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc14c9e0 con 0x7fe1bc06e620 2024-04-25T08:43:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.695+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc14cfa0 con 0x7fe1bc06e620 2024-04-25T08:43:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.695+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc069010 con 0x7fe1bc06e620 2024-04-25T08:43:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.695+0000 7fe1b2a00640 1 -- 172.21.15.46:0/638291999 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1ac0adc20 con 0x7fe1bc06e620 2024-04-25T08:43:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.695+0000 7fe1b2a00640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe198066f60 0x7fe198069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.695+0000 7fe1b2a00640 1 -- 172.21.15.46:0/638291999 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe1ac1208b0 con 0x7fe1bc06e620 2024-04-25T08:43:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.695+0000 7fe1b1400640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe198066f60 0x7fe198069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.695+0000 7fe1b1400640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe198066f60 0x7fe198069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc1f95f0 tx=0x7fe1a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:06.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.699+0000 7fe1b2a00640 1 -- 172.21.15.46:0/638291999 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe1ac016560 con 0x7fe1bc06e620 2024-04-25T08:43:06.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.859+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe1bc06cae0 con 0x7fe1bc06e620 2024-04-25T08:43:06.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1b2a00640 1 -- 172.21.15.46:0/638291999 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe1ac0ab340 con 0x7fe1bc06e620 2024-04-25T08:43:06.855 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe198066f60 msgr2=0x7fe198069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe198066f60 0x7fe198069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc1f95f0 tx=0x7fe1a4009040 comp rx=0 tx=0).stop 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc06e620 msgr2=0x7fe1bc15e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc06e620 0x7fe1bc15e0d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac019550 tx=0x7fe1ac0196e0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 shutdown_connections 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe198066f60 0x7fe198069420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc15dae0 0x7fe1bc1490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1bc06ea60 0x7fe1bc153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 --2- 172.21.15.46:0/638291999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1bc06e620 0x7fe1bc15e0d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 >> 172.21.15.46:0/638291999 conn(0x7fe1bc071ae0 msgr2=0x7fe1bc072380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 shutdown_connections 2024-04-25T08:43:06.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:06.863+0000 7fe1c1000640 1 -- 172.21.15.46:0/638291999 wait complete. 2024-04-25T08:43:06.883 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:43:06.883 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.031+0000 7fb2a5000640 1 Processor -- start 2024-04-25T08:43:07.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.031+0000 7fb2a5000640 1 -- start start 2024-04-25T08:43:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.031+0000 7fb2a5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a015bfe0 0x7fb2a00b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.031+0000 7fb2a5000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb2a00b4510 0x7fb2a009ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.031+0000 7fb2a5000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a015d770 con 0x7fb2a0144280 2024-04-25T08:43:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.031+0000 7fb2a5000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a0144660 con 0x7fb2a00b4510 2024-04-25T08:43:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.031+0000 7fb2a5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a015da50 con 0x7fb2a015bfe0 2024-04-25T08:43:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29de00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb2a0144280 0x7fb2a015ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:54404/0 (socket says 172.21.15.46:54404) 2024-04-25T08:43:07.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29de00640 1 -- 172.21.15.46:0/3523178167 learned_addr learned my addr 172.21.15.46:0/3523178167 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29e800640 1 --2- 172.21.15.46:0/3523178167 >> v2:172.21.15.82:3300/0 conn(0x7fb2a00b4510 0x7fb2a009ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb297e00640 1 --2- 172.21.15.46:0/3523178167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a015bfe0 0x7fb2a00b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29d400640 1 -- 172.21.15.46:0/3523178167 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314509311 0 0) 0x7fb2a015d770 con 0x7fb2a0144280 2024-04-25T08:43:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29d400640 1 -- 172.21.15.46:0/3523178167 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb284003530 con 0x7fb2a0144280 2024-04-25T08:43:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29e800640 1 -- 172.21.15.46:0/3523178167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a015bfe0 msgr2=0x7fb2a00b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29e800640 1 --2- 172.21.15.46:0/3523178167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a015bfe0 0x7fb2a00b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29e800640 1 -- 172.21.15.46:0/3523178167 >> v1:172.21.15.82:6789/0 conn(0x7fb2a0144280 legacy=0x7fb2a015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29e800640 1 -- 172.21.15.46:0/3523178167 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a015dd30 con 0x7fb2a00b4510 2024-04-25T08:43:07.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29e800640 1 --2- 172.21.15.46:0/3523178167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a00b4510 0x7fb2a009ab00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb298009d90 tx=0x7fb29801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=468cc6c96e6be1b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.035+0000 7fb29d400640 1 -- 172.21.15.46:0/3523178167 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2980b2070 con 0x7fb2a00b4510 2024-04-25T08:43:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb29d400640 1 -- 172.21.15.46:0/3523178167 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2980a57a0 con 0x7fb2a00b4510 2024-04-25T08:43:07.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb29d400640 1 -- 172.21.15.46:0/3523178167 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2980a5b70 con 0x7fb2a00b4510 2024-04-25T08:43:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 -- 172.21.15.46:0/3523178167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a00b4510 msgr2=0x7fb2a009ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 --2- 172.21.15.46:0/3523178167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a00b4510 0x7fb2a009ab00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb298009d90 tx=0x7fb29801c8e0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 -- 172.21.15.46:0/3523178167 shutdown_connections 2024-04-25T08:43:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 --2- 172.21.15.46:0/3523178167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a00b4510 0x7fb2a009ab00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 --2- 172.21.15.46:0/3523178167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a015bfe0 0x7fb2a00b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 -- 172.21.15.46:0/3523178167 >> 172.21.15.46:0/3523178167 conn(0x7fb2a0072920 msgr2=0x7fb2a0099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 -- 172.21.15.46:0/3523178167 shutdown_connections 2024-04-25T08:43:07.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 -- 172.21.15.46:0/3523178167 wait complete. 2024-04-25T08:43:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 Processor -- start 2024-04-25T08:43:07.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 -- start start 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a00b4510 0x7fb2a0139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a0144280 0x7fb2a015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a015bfe0 0x7fb2a009c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a009bbe0 con 0x7fb2a00b4510 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a009ba60 con 0x7fb2a015bfe0 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb2a5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a009bd60 con 0x7fb2a0144280 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb29de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a00b4510 0x7fb2a0139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb29de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a00b4510 0x7fb2a0139b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43042/0 (socket says 172.21.15.46:43042) 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb29de00640 1 -- 172.21.15.46:0/2156331915 learned_addr learned my addr 172.21.15.46:0/2156331915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb297e00640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a0144280 0x7fb2a015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.039+0000 7fb29e800640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a015bfe0 0x7fb2a009c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb29de00640 1 -- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a0144280 msgr2=0x7fb2a015a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb29de00640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a0144280 0x7fb2a015a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb29de00640 1 -- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a015bfe0 msgr2=0x7fb2a009c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb29de00640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a015bfe0 0x7fb2a009c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb29de00640 1 -- 172.21.15.46:0/2156331915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a009ced0 con 0x7fb2a00b4510 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb29de00640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a00b4510 0x7fb2a0139b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb29000c3f0 tx=0x7fb29000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb296000640 1 -- 172.21.15.46:0/2156331915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290017070 con 0x7fb2a00b4510 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb296000640 1 -- 172.21.15.46:0/2156331915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb290005cf0 con 0x7fb2a00b4510 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb296000640 1 -- 172.21.15.46:0/2156331915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900072f0 con 0x7fb2a00b4510 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a009d150 con 0x7fb2a00b4510 2024-04-25T08:43:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a009d710 con 0x7fb2a00b4510 2024-04-25T08:43:07.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a006ccb0 con 0x7fb2a00b4510 2024-04-25T08:43:07.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb296000640 1 -- 172.21.15.46:0/2156331915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb29001a070 con 0x7fb2a00b4510 2024-04-25T08:43:07.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb296000640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb27c066f60 0x7fb27c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.043+0000 7fb296000640 1 -- 172.21.15.46:0/2156331915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb290084860 con 0x7fb2a00b4510 2024-04-25T08:43:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.047+0000 7fb297e00640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb27c066f60 0x7fb27c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.047+0000 7fb297e00640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb27c066f60 0x7fb27c069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0137c70 tx=0x7fb288009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:07.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.047+0000 7fb296000640 1 -- 172.21.15.46:0/2156331915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2a009ced0 con 0x7fb2a00b4510 2024-04-25T08:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.215+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2a009d8e0 con 0x7fb2a00b4510 2024-04-25T08:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.215+0000 7fb296000640 1 -- 172.21.15.46:0/2156331915 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7fb29005b0d0 con 0x7fb2a00b4510 2024-04-25T08:43:07.209 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:07.209 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb27c066f60 msgr2=0x7fb27c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb27c066f60 0x7fb27c069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0137c70 tx=0x7fb288009190 comp rx=0 tx=0).stop 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a00b4510 msgr2=0x7fb2a0139b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a00b4510 0x7fb2a0139b40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb29000c3f0 tx=0x7fb29000c8b0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 shutdown_connections 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb27c066f60 0x7fb27c069420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a015bfe0 0x7fb2a009c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2a0144280 0x7fb2a015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 --2- 172.21.15.46:0/2156331915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2a00b4510 0x7fb2a0139b40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 >> 172.21.15.46:0/2156331915 conn(0x7fb2a0072920 msgr2=0x7fb2a0143260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 shutdown_connections 2024-04-25T08:43:07.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.219+0000 7fb2a5000640 1 -- 172.21.15.46:0/2156331915 wait complete. 2024-04-25T08:43:07.230 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:07.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.379+0000 7f651da00640 1 Processor -- start 2024-04-25T08:43:07.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.379+0000 7f651da00640 1 -- start start 2024-04-25T08:43:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f651da00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6518143990 0x7f65180cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f651da00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651815e440 con 0x7f65181440d0 2024-04-25T08:43:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f651da00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651815e6e0 con 0x7f651806f150 2024-04-25T08:43:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f651da00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651815ed30 con 0x7f6518143990 2024-04-25T08:43:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f650fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f65181440d0 0x7f651806ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48454/0 (socket says 172.21.15.46:48454) 2024-04-25T08:43:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f650fe00640 1 -- 172.21.15.46:0/2947683546 learned_addr learned my addr 172.21.15.46:0/2947683546 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:07.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516a00640 1 --2- 172.21.15.46:0/2947683546 >> v2:172.21.15.82:3300/0 conn(0x7f6518143990 0x7f65180cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341142908 0 0) 0x7f651815e440 con 0x7f65181440d0 2024-04-25T08:43:07.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64fc005740 con 0x7f65181440d0 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497773547 0 0) 0x7f64fc005740 con 0x7f65181440d0 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 >> v2:172.21.15.82:3300/0 conn(0x7f6518143990 msgr2=0x7f65180cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 --2- 172.21.15.46:0/2947683546 >> v2:172.21.15.82:3300/0 conn(0x7f6518143990 0x7f65180cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 >> v1:172.21.15.82:6789/0 conn(0x7f651806f150 legacy=0x7f65180b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651815e9c0 con 0x7f65181440d0 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f6500001680 con 0x7f65181440d0 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516a00640 1 --2- 172.21.15.46:0/2947683546 >> v2:172.21.15.82:3300/0 conn(0x7f6518143990 0x7f65180cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 >> v1:172.21.15.46:6789/0 conn(0x7f65181440d0 legacy=0x7f651806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f64fc006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64fc006a20 0x7f64fc006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64fc0073e0 0x7f64fc00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651815ed30 con 0x7f64fc005c40 2024-04-25T08:43:07.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc00e280 con 0x7f64fc0073e0 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc00e480 con 0x7f64fc006a20 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516a00640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f64fc006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f650fe00640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64fc006a20 0x7f64fc006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6517400640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64fc0073e0 0x7f64fc00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516a00640 1 -- 172.21.15.46:0/2947683546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64fc006a20 msgr2=0x7f64fc006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516a00640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64fc006a20 0x7f64fc006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516a00640 1 -- 172.21.15.46:0/2947683546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64fc0073e0 msgr2=0x7f64fc00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516a00640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64fc0073e0 0x7f64fc00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6516a00640 1 -- 172.21.15.46:0/2947683546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc005740 con 0x7f64fc005c40 2024-04-25T08:43:07.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.383+0000 7f6517400640 1 -- 172.21.15.46:0/2947683546 reap_dead start 2024-04-25T08:43:07.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f6516a00640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f64fc006040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6508009cb0 tx=0x7f650801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f57e6e8aa937329 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:07.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080b2070 con 0x7f64fc005c40 2024-04-25T08:43:07.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65080a5850 con 0x7f64fc005c40 2024-04-25T08:43:07.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f6516000640 1 -- 172.21.15.46:0/2947683546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508009880 con 0x7f64fc005c40 2024-04-25T08:43:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f651da00640 1 -- 172.21.15.46:0/2947683546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 msgr2=0x7f64fc006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f651da00640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f64fc006040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6508009cb0 tx=0x7f650801cc10 comp rx=0 tx=0).stop 2024-04-25T08:43:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f651da00640 1 -- 172.21.15.46:0/2947683546 shutdown_connections 2024-04-25T08:43:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f651da00640 1 --2- 172.21.15.46:0/2947683546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f64fc006040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f651da00640 1 -- 172.21.15.46:0/2947683546 >> 172.21.15.46:0/2947683546 conn(0x7f651806ac10 msgr2=0x7f651806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f651da00640 1 -- 172.21.15.46:0/2947683546 shutdown_connections 2024-04-25T08:43:07.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.387+0000 7f651da00640 1 -- 172.21.15.46:0/2947683546 wait complete. 2024-04-25T08:43:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 Processor -- start 2024-04-25T08:43:07.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 -- start start 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f65181557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651813fa60 0x7f65181f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f651813fee0 0x7f65181f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6518143d70 con 0x7f64fc005c40 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651815f1c0 con 0x7f651813fa60 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6518143ef0 con 0x7f651813fee0 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f6516a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f65181557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f6516a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f65181557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43066/0 (socket says 172.21.15.46:43066) 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f6516a00640 1 -- 172.21.15.46:0/510053317 learned_addr learned my addr 172.21.15.46:0/510053317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f6516a00640 1 -- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f651813fee0 msgr2=0x7f65181f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f6516a00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f651813fee0 0x7f65181f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f6516a00640 1 -- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651813fa60 msgr2=0x7f65181f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f650fe00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651813fa60 0x7f65181f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f6516a00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651813fa60 0x7f65181f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f6516a00640 1 -- 172.21.15.46:0/510053317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65181f0f10 con 0x7f64fc005c40 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f650fe00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651813fa60 0x7f65181f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f6516a00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f65181557e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f650801ca40 tx=0x7f65080199b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f650ea00640 1 -- 172.21.15.46:0/510053317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080bd070 con 0x7f64fc005c40 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f650ea00640 1 -- 172.21.15.46:0/510053317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65080ad660 con 0x7f64fc005c40 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65181f11f0 con 0x7f64fc005c40 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f650ea00640 1 -- 172.21.15.46:0/510053317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080b2040 con 0x7f64fc005c40 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65181f17b0 con 0x7f64fc005c40 2024-04-25T08:43:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.391+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65180ccdf0 con 0x7f64fc005c40 2024-04-25T08:43:07.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.395+0000 7f650ea00640 1 -- 172.21.15.46:0/510053317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65080c0070 con 0x7f64fc005c40 2024-04-25T08:43:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.395+0000 7f650ea00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64f4066f60 0x7f64f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.395+0000 7f650ea00640 1 -- 172.21.15.46:0/510053317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f650811f190 con 0x7f64fc005c40 2024-04-25T08:43:07.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.395+0000 7f650fe00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64f4066f60 0x7f64f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.395+0000 7f650fe00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64f4066f60 0x7f64f4069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6518139e50 tx=0x7f6500009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:07.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.399+0000 7f650ea00640 1 -- 172.21.15.46:0/510053317 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6508016920 con 0x7f64fc005c40 2024-04-25T08:43:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.559+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6518069010 con 0x7f64fc005c40 2024-04-25T08:43:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.559+0000 7f650ea00640 1 -- 172.21.15.46:0/510053317 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65080ef730 con 0x7f64fc005c40 2024-04-25T08:43:07.553 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:07.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64f4066f60 msgr2=0x7f64f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64f4066f60 0x7f64f4069420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6518139e50 tx=0x7f6500009040 comp rx=0 tx=0).stop 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 msgr2=0x7f65181557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f65181557e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f650801ca40 tx=0x7f65080199b0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 shutdown_connections 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64f4066f60 0x7f64f4069420 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f651813fee0 0x7f65181f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651813fa60 0x7f65181f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 --2- 172.21.15.46:0/510053317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64fc005c40 0x7f65181557e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 >> 172.21.15.46:0/510053317 conn(0x7f651806ac10 msgr2=0x7f651813ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 shutdown_connections 2024-04-25T08:43:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.563+0000 7f651da00640 1 -- 172.21.15.46:0/510053317 wait complete. 2024-04-25T08:43:07.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.723+0000 7ff832c00640 1 Processor -- start 2024-04-25T08:43:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff832c00640 1 -- start start 2024-04-25T08:43:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff832c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff82c06dc70 0x7ff82c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff832c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff82c0cf1b0 0x7ff82c15d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff832c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82c0cd930 con 0x7ff82c145310 2024-04-25T08:43:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff832c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0cdbd0 con 0x7ff82c0cf1b0 2024-04-25T08:43:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff832c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0cdeb0 con 0x7ff82c06dc70 2024-04-25T08:43:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff830c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff82c0cf1b0 0x7ff82c15d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff82be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff82c145310 0x7ff82c06e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48462/0 (socket says 172.21.15.46:48462) 2024-04-25T08:43:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff82be00640 1 -- 172.21.15.46:0/4026282015 learned_addr learned my addr 172.21.15.46:0/4026282015 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff823e00640 1 --2- 172.21.15.46:0/4026282015 >> v2:172.21.15.82:3300/0 conn(0x7ff82c06dc70 0x7ff82c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff830c00640 1 -- 172.21.15.46:0/4026282015 >> v2:172.21.15.82:3300/0 conn(0x7ff82c06dc70 msgr2=0x7ff82c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff830c00640 1 --2- 172.21.15.46:0/4026282015 >> v2:172.21.15.82:3300/0 conn(0x7ff82c06dc70 0x7ff82c0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff830c00640 1 -- 172.21.15.46:0/4026282015 >> v1:172.21.15.46:6789/0 conn(0x7ff82c145310 legacy=0x7ff82c06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff830c00640 1 -- 172.21.15.46:0/4026282015 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0ce190 con 0x7ff82c0cf1b0 2024-04-25T08:43:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.727+0000 7ff82b400640 1 -- 172.21.15.46:0/4026282015 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217832633 0 0) 0x7ff82c0cd930 con 0x7ff82c145310 2024-04-25T08:43:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff830c00640 1 --2- 172.21.15.46:0/4026282015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0cf1b0 0x7ff82c15d090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff82c06d140 tx=0x7ff82401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fee2d129becb37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff82b400640 1 -- 172.21.15.46:0/4026282015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240b2070 con 0x7ff82c0cf1b0 2024-04-25T08:43:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff82b400640 1 -- 172.21.15.46:0/4026282015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8240a58d0 con 0x7ff82c0cf1b0 2024-04-25T08:43:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff82b400640 1 -- 172.21.15.46:0/4026282015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240a5ca0 con 0x7ff82c0cf1b0 2024-04-25T08:43:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 -- 172.21.15.46:0/4026282015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0cf1b0 msgr2=0x7ff82c15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 --2- 172.21.15.46:0/4026282015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0cf1b0 0x7ff82c15d090 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff82c06d140 tx=0x7ff82401c900 comp rx=0 tx=0).stop 2024-04-25T08:43:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 -- 172.21.15.46:0/4026282015 shutdown_connections 2024-04-25T08:43:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 --2- 172.21.15.46:0/4026282015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0cf1b0 0x7ff82c15d090 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 --2- 172.21.15.46:0/4026282015 >> v2:172.21.15.82:3300/0 conn(0x7ff82c06dc70 0x7ff82c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 -- 172.21.15.46:0/4026282015 >> 172.21.15.46:0/4026282015 conn(0x7ff82c06ac10 msgr2=0x7ff82c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 -- 172.21.15.46:0/4026282015 shutdown_connections 2024-04-25T08:43:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 -- 172.21.15.46:0/4026282015 wait complete. 2024-04-25T08:43:07.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 Processor -- start 2024-04-25T08:43:07.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.731+0000 7ff832c00640 1 -- start start 2024-04-25T08:43:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff832c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c0cf1b0 0x7ff82c0b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff832c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff82c145310 0x7ff82c0b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff832c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0b4500 0x7ff82c071270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff832c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1688a0 con 0x7ff82c0b4500 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff832c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c168720 con 0x7ff82c0cf1b0 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff832c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c168a20 con 0x7ff82c145310 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff830c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0b4500 0x7ff82c071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff830c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0b4500 0x7ff82c071270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43086/0 (socket says 172.21.15.46:43086) 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff830c00640 1 -- 172.21.15.46:0/3845296993 learned_addr learned my addr 172.21.15.46:0/3845296993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff823e00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff82c145310 0x7ff82c0b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff82be00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c0cf1b0 0x7ff82c0b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff830c00640 1 -- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff82c145310 msgr2=0x7ff82c0b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff830c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff82c145310 0x7ff82c0b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff830c00640 1 -- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c0cf1b0 msgr2=0x7ff82c0b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff830c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c0cf1b0 0x7ff82c0b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff830c00640 1 -- 172.21.15.46:0/3845296993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c1598b0 con 0x7ff82c0b4500 2024-04-25T08:43:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff830c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0b4500 0x7ff82c071270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff824019510 tx=0x7ff8240196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff828c00640 1 -- 172.21.15.46:0/3845296993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240b2050 con 0x7ff82c0b4500 2024-04-25T08:43:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff828c00640 1 -- 172.21.15.46:0/3845296993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8240b3040 con 0x7ff82c0b4500 2024-04-25T08:43:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff828c00640 1 -- 172.21.15.46:0/3845296993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240ad850 con 0x7ff82c0b4500 2024-04-25T08:43:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c0b2f20 con 0x7ff82c0b4500 2024-04-25T08:43:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c0b34e0 con 0x7ff82c0b4500 2024-04-25T08:43:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.735+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c069010 con 0x7ff82c0b4500 2024-04-25T08:43:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.739+0000 7ff828c00640 1 -- 172.21.15.46:0/3845296993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8240c0020 con 0x7ff82c0b4500 2024-04-25T08:43:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.739+0000 7ff828c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff808066f90 0x7ff808069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.739+0000 7ff828c00640 1 -- 172.21.15.46:0/3845296993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff82411fed0 con 0x7ff82c0b4500 2024-04-25T08:43:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.739+0000 7ff82be00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff808066f90 0x7ff808069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:07.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.739+0000 7ff82be00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff808066f90 0x7ff808069450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff81c000f70 tx=0x7ff81c007c10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:07.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.743+0000 7ff828c00640 1 -- 172.21.15.46:0/3845296993 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff82c1598b0 con 0x7ff82c0b4500 2024-04-25T08:43:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff82c0b0e60 con 0x7ff82c0b4500 2024-04-25T08:43:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff828c00640 1 -- 172.21.15.46:0/3845296993 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff824016520 con 0x7ff82c0b4500 2024-04-25T08:43:07.896 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff808066f90 msgr2=0x7ff808069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff808066f90 0x7ff808069450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff81c000f70 tx=0x7ff81c007c10 comp rx=0 tx=0).stop 2024-04-25T08:43:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0b4500 msgr2=0x7ff82c071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0b4500 0x7ff82c071270 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff824019510 tx=0x7ff8240196a0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 shutdown_connections 2024-04-25T08:43:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff808066f90 0x7ff808069450 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff82c0b4500 0x7ff82c071270 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff82c145310 0x7ff82c0b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 --2- 172.21.15.46:0/3845296993 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff82c0cf1b0 0x7ff82c0b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.903+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 >> 172.21.15.46:0/3845296993 conn(0x7ff82c06ac10 msgr2=0x7ff82c0b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.907+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 shutdown_connections 2024-04-25T08:43:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:07.907+0000 7ff832c00640 1 -- 172.21.15.46:0/3845296993 wait complete. 2024-04-25T08:43:07.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.067+0000 7feeca800640 1 Processor -- start 2024-04-25T08:43:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.067+0000 7feeca800640 1 -- start start 2024-04-25T08:43:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feeca800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec40d0100 0x7feec415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feeca800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feec415c010 0x7feec4099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feeca800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec40cc1b0 con 0x7feec415b6f0 2024-04-25T08:43:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feeca800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec40ccdc0 con 0x7feec415c010 2024-04-25T08:43:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feeca800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec40cbf50 con 0x7feec40d0100 2024-04-25T08:43:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feec9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feec415c010 0x7feec4099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feec9400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7feec415b6f0 0x7feec40cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60230/0 (socket says 172.21.15.46:60230) 2024-04-25T08:43:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feec9400640 1 -- 172.21.15.46:0/2757189960 learned_addr learned my addr 172.21.15.46:0/2757189960 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feec8a00640 1 --2- 172.21.15.46:0/2757189960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec40d0100 0x7feec415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feec9e00640 1 -- 172.21.15.46:0/2757189960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec40d0100 msgr2=0x7feec415bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feec9e00640 1 --2- 172.21.15.46:0/2757189960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec40d0100 0x7feec415bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feec9e00640 1 -- 172.21.15.46:0/2757189960 >> v1:172.21.15.46:6789/0 conn(0x7feec415b6f0 legacy=0x7feec40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feec9e00640 1 -- 172.21.15.46:0/2757189960 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40cc490 con 0x7feec415c010 2024-04-25T08:43:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.071+0000 7feec8a00640 1 --2- 172.21.15.46:0/2757189960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec40d0100 0x7feec415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feec9e00640 1 --2- 172.21.15.46:0/2757189960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec415c010 0x7feec4099a00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feebc009d90 tx=0x7feebc01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a253440587e32dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feec3e00640 1 -- 172.21.15.46:0/2757189960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0b2070 con 0x7feec415c010 2024-04-25T08:43:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feec3e00640 1 -- 172.21.15.46:0/2757189960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc0a5880 con 0x7feec415c010 2024-04-25T08:43:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feec3e00640 1 -- 172.21.15.46:0/2757189960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc0a5c50 con 0x7feec415c010 2024-04-25T08:43:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 -- 172.21.15.46:0/2757189960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec415c010 msgr2=0x7feec4099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 --2- 172.21.15.46:0/2757189960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec415c010 0x7feec4099a00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feebc009d90 tx=0x7feebc01c8b0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 -- 172.21.15.46:0/2757189960 shutdown_connections 2024-04-25T08:43:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 --2- 172.21.15.46:0/2757189960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec415c010 0x7feec4099a00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 --2- 172.21.15.46:0/2757189960 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec40d0100 0x7feec415bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 -- 172.21.15.46:0/2757189960 >> 172.21.15.46:0/2757189960 conn(0x7feec40b2f50 msgr2=0x7feec40b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:08.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 -- 172.21.15.46:0/2757189960 shutdown_connections 2024-04-25T08:43:08.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 -- 172.21.15.46:0/2757189960 wait complete. 2024-04-25T08:43:08.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 Processor -- start 2024-04-25T08:43:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 -- start start 2024-04-25T08:43:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec40d0100 0x7feec4154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec415b6f0 0x7feec4154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feec415c010 0x7feec40ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec409a8d0 con 0x7feec40d0100 2024-04-25T08:43:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec409a750 con 0x7feec415c010 2024-04-25T08:43:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feeca800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec41686d0 con 0x7feec415b6f0 2024-04-25T08:43:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feec9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec40d0100 0x7feec4154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feec9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec40d0100 0x7feec4154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51480/0 (socket says 172.21.15.46:51480) 2024-04-25T08:43:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.075+0000 7feec9400640 1 -- 172.21.15.46:0/3318031563 learned_addr learned my addr 172.21.15.46:0/3318031563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec9400640 1 -- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec415b6f0 msgr2=0x7feec4154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec9400640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec415b6f0 0x7feec4154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec9400640 1 -- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feec415c010 msgr2=0x7feec40ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec9400640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feec415c010 0x7feec40ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec9400640 1 -- 172.21.15.46:0/3318031563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40cb2d0 con 0x7feec40d0100 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec9400640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec40d0100 0x7feec4154300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c3a0 tx=0x7feeb800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec1600640 1 -- 172.21.15.46:0/3318031563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8017070 con 0x7feec40d0100 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec1600640 1 -- 172.21.15.46:0/3318031563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8005cf0 con 0x7feec40d0100 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec1600640 1 -- 172.21.15.46:0/3318031563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb80072d0 con 0x7feec40d0100 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec40c8740 con 0x7feec40d0100 2024-04-25T08:43:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec4154d80 con 0x7feec40d0100 2024-04-25T08:43:08.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec4069010 con 0x7feec40d0100 2024-04-25T08:43:08.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec1600640 1 -- 172.21.15.46:0/3318031563 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feeb801a070 con 0x7feec40d0100 2024-04-25T08:43:08.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec1600640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feea4066f60 0x7feea4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec1600640 1 -- 172.21.15.46:0/3318031563 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feeb8084630 con 0x7feec40d0100 2024-04-25T08:43:08.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.079+0000 7feec8a00640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feea4066f60 0x7feea4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.083+0000 7feec8a00640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feea4066f60 0x7feea4069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feec40c4dd0 tx=0x7feeb0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.083+0000 7feec1600640 1 -- 172.21.15.46:0/3318031563 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feeb8054be0 con 0x7feec40d0100 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feec40719f0 con 0x7feec40d0100 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feec1600640 1 -- 172.21.15.46:0/3318031563 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7feeb8058c80 con 0x7feec40d0100 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:08.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feea4066f60 msgr2=0x7feea4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feea4066f60 0x7feea4069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7feec40c4dd0 tx=0x7feeb0008040 comp rx=0 tx=0).stop 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec40d0100 msgr2=0x7feec4154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec40d0100 0x7feec4154300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c3a0 tx=0x7feeb800c860 comp rx=0 tx=0).stop 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 shutdown_connections 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feea4066f60 0x7feea4069420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feec415c010 0x7feec40ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feec415b6f0 0x7feec4154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 --2- 172.21.15.46:0/3318031563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feec40d0100 0x7feec4154300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 >> 172.21.15.46:0/3318031563 conn(0x7feec40b2f50 msgr2=0x7feec415ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 shutdown_connections 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.247+0000 7feeca800640 1 -- 172.21.15.46:0/3318031563 wait complete. 2024-04-25T08:43:08.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:08.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.411+0000 7f51ade00640 1 Processor -- start 2024-04-25T08:43:08.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.411+0000 7f51ade00640 1 -- start start 2024-04-25T08:43:08.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51ade00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51a806f4d0 0x7f51a80cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51ade00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a809a780 con 0x7f51a815dd50 2024-04-25T08:43:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51ade00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a809aa60 con 0x7f51a80d0450 2024-04-25T08:43:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51ade00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80708c0 con 0x7f51a806f4d0 2024-04-25T08:43:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f519fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f51a815dd50 0x7f51a80cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60242/0 (socket says 172.21.15.46:60242) 2024-04-25T08:43:08.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f519fe00640 1 -- 172.21.15.46:0/824288702 learned_addr learned my addr 172.21.15.46:0/824288702 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251769187 0 0) 0x7f51a809a780 con 0x7f51a815dd50 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f518c003530 con 0x7f51a815dd50 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6a00640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51a806f4d0 0x7f51a80cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1505490383 0 0) 0x7f518c003530 con 0x7f51a815dd50 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51a806f4d0 msgr2=0x7f51a80cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51a806f4d0 0x7f51a80cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 >> v1:172.21.15.82:6789/0 conn(0x7f51a80d0450 legacy=0x7f51a80cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a809ad40 con 0x7f51a815dd50 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f5190001680 con 0x7f51a815dd50 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 >> v1:172.21.15.46:6789/0 conn(0x7f51a815dd50 legacy=0x7f51a80cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f518c003a60 0x7f518c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c004810 0x7f518c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51a806f4d0 0x7f518c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80708c0 con 0x7f518c003a60 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c00bad0 con 0x7f518c004810 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c00bcd0 con 0x7f51a806f4d0 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6a00640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f518c003a60 0x7f518c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f519fe00640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c004810 0x7f518c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a7400640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51a806f4d0 0x7f518c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6a00640 1 -- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51a806f4d0 msgr2=0x7f518c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6a00640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51a806f4d0 0x7f518c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6a00640 1 -- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c004810 msgr2=0x7f518c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6a00640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c004810 0x7f518c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a6a00640 1 -- 172.21.15.46:0/824288702 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c003530 con 0x7f518c003a60 2024-04-25T08:43:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f51a7400640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51a806f4d0 0x7f518c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:08.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.415+0000 7f519fe00640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c004810 0x7f518c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:08.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51a6a00640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f518c003a60 0x7f518c003e80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5198009ae0 tx=0x7f519801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8384b000ed3868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:08.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980096b0 con 0x7f518c003a60 2024-04-25T08:43:08.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51980a6450 con 0x7f518c003a60 2024-04-25T08:43:08.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51a6000640 1 -- 172.21.15.46:0/824288702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980a6840 con 0x7f518c003a60 2024-04-25T08:43:08.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51ade00640 1 -- 172.21.15.46:0/824288702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f518c003a60 msgr2=0x7f518c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51ade00640 1 --2- 172.21.15.46:0/824288702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f518c003a60 0x7f518c003e80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5198009ae0 tx=0x7f519801ca40 comp rx=0 tx=0).stop 2024-04-25T08:43:08.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51a7400640 1 -- 172.21.15.46:0/824288702 reap_dead start 2024-04-25T08:43:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51ade00640 1 -- 172.21.15.46:0/824288702 shutdown_connections 2024-04-25T08:43:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51ade00640 1 -- 172.21.15.46:0/824288702 >> 172.21.15.46:0/824288702 conn(0x7f51a80b3160 msgr2=0x7f51a80b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51ade00640 1 -- 172.21.15.46:0/824288702 shutdown_connections 2024-04-25T08:43:08.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51ade00640 1 -- 172.21.15.46:0/824288702 wait complete. 2024-04-25T08:43:08.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51ade00640 1 Processor -- start 2024-04-25T08:43:08.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.419+0000 7f51ade00640 1 -- start start 2024-04-25T08:43:08.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51ade00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518c003a60 0x7f51a8155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51ade00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a8155410 0x7f51a80c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51ade00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80b22c0 0x7f51a80b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51ade00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80cd560 con 0x7f51a80b22c0 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51ade00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80cd3e0 con 0x7f51a8155410 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51ade00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80cd6e0 con 0x7f518c003a60 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80b22c0 0x7f51a80b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80b22c0 0x7f51a80b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51494/0 (socket says 172.21.15.46:51494) 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a7400640 1 -- 172.21.15.46:0/3603394285 learned_addr learned my addr 172.21.15.46:0/3603394285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a6a00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518c003a60 0x7f51a8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f519fe00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a8155410 0x7f51a80c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a7400640 1 -- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518c003a60 msgr2=0x7f51a8155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a7400640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518c003a60 0x7f51a8155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a7400640 1 -- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a8155410 msgr2=0x7f51a80c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a7400640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a8155410 0x7f51a80c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a7400640 1 -- 172.21.15.46:0/3603394285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a80b4730 con 0x7f51a80b22c0 2024-04-25T08:43:08.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51a7400640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80b22c0 0x7f51a80b2740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000f90 tx=0x7f51a000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:08.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f519f400640 1 -- 172.21.15.46:0/3603394285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0006b60 con 0x7f51a80b22c0 2024-04-25T08:43:08.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f519f400640 1 -- 172.21.15.46:0/3603394285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a0006d20 con 0x7f51a80b22c0 2024-04-25T08:43:08.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a80cb630 con 0x7f51a80b22c0 2024-04-25T08:43:08.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f519f400640 1 -- 172.21.15.46:0/3603394285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0003980 con 0x7f51a80b22c0 2024-04-25T08:43:08.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a80cae30 con 0x7f51a80b22c0 2024-04-25T08:43:08.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.423+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a8069010 con 0x7f51a80b22c0 2024-04-25T08:43:08.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.427+0000 7f519f400640 1 -- 172.21.15.46:0/3603394285 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51a001a020 con 0x7f51a80b22c0 2024-04-25T08:43:08.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.427+0000 7f519f400640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5184066f90 0x7f5184069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.427+0000 7f519f400640 1 -- 172.21.15.46:0/3603394285 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f51a0084940 con 0x7f51a80b22c0 2024-04-25T08:43:08.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.427+0000 7f51a6a00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5184066f90 0x7f5184069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.427+0000 7f51a6a00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5184066f90 0x7f5184069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51980ad000 tx=0x7f51980b2040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:08.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.431+0000 7f519f400640 1 -- 172.21.15.46:0/3603394285 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51a0054e60 con 0x7f51a80b22c0 2024-04-25T08:43:08.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f51a80b3610 con 0x7f51a80b22c0 2024-04-25T08:43:08.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f519f400640 1 -- 172.21.15.46:0/3603394285 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f51a0059e20 con 0x7f51a80b22c0 2024-04-25T08:43:08.588 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5184066f90 msgr2=0x7f5184069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5184066f90 0x7f5184069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51980ad000 tx=0x7f51980b2040 comp rx=0 tx=0).stop 2024-04-25T08:43:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80b22c0 msgr2=0x7f51a80b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80b22c0 0x7f51a80b2740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000f90 tx=0x7f51a000eb90 comp rx=0 tx=0).stop 2024-04-25T08:43:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 shutdown_connections 2024-04-25T08:43:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5184066f90 0x7f5184069450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51a80b22c0 0x7f51a80b2740 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a8155410 0x7f51a80c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 --2- 172.21.15.46:0/3603394285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518c003a60 0x7f51a8155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 >> 172.21.15.46:0/3603394285 conn(0x7f51a80b3160 msgr2=0x7f51a815cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 shutdown_connections 2024-04-25T08:43:08.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.595+0000 7f51ade00640 1 -- 172.21.15.46:0/3603394285 wait complete. 2024-04-25T08:43:08.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T08:43:08.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.755+0000 7f2cb2000640 1 Processor -- start 2024-04-25T08:43:08.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.755+0000 7f2cb2000640 1 -- start start 2024-04-25T08:43:08.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2cb2000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2cac15a750 0x7f2cac06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2cb2000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac06d5e0 con 0x7f2cac0aed10 2024-04-25T08:43:08.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2cb2000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac0ce720 con 0x7f2cac0cc9e0 2024-04-25T08:43:08.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2cb2000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac15ab30 con 0x7f2cac15a750 2024-04-25T08:43:08.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2ca3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2cac0aed10 0x7f2cac0b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60244/0 (socket says 172.21.15.46:60244) 2024-04-25T08:43:08.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2ca3e00640 1 -- 172.21.15.46:0/3584552624 learned_addr learned my addr 172.21.15.46:0/3584552624 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2cab400640 1 --2- 172.21.15.46:0/3584552624 >> v2:172.21.15.82:3300/0 conn(0x7f2cac15a750 0x7f2cac06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2caaa00640 1 -- 172.21.15.46:0/3584552624 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668563803 0 0) 0x7f2cac06d5e0 con 0x7f2cac0aed10 2024-04-25T08:43:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2caaa00640 1 -- 172.21.15.46:0/3584552624 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c90003530 con 0x7f2cac0aed10 2024-04-25T08:43:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2cab400640 1 -- 172.21.15.46:0/3584552624 >> v1:172.21.15.82:6789/0 conn(0x7f2cac0cc9e0 legacy=0x7f2cac0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2cab400640 1 -- 172.21.15.46:0/3584552624 >> v1:172.21.15.46:6789/0 conn(0x7f2cac0aed10 legacy=0x7f2cac0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.759+0000 7f2cab400640 1 -- 172.21.15.46:0/3584552624 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac0cea00 con 0x7f2cac15a750 2024-04-25T08:43:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cab400640 1 --2- 172.21.15.46:0/3584552624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac15a750 0x7f2cac06ec10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0097c0 tx=0x7f2c9c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e15dd8ce2c8e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2caaa00640 1 -- 172.21.15.46:0/3584552624 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0b0070 con 0x7f2cac15a750 2024-04-25T08:43:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2caaa00640 1 -- 172.21.15.46:0/3584552624 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c0ab600 con 0x7f2cac15a750 2024-04-25T08:43:08.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2caaa00640 1 -- 172.21.15.46:0/3584552624 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0ab9b0 con 0x7f2cac15a750 2024-04-25T08:43:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cb2000640 1 -- 172.21.15.46:0/3584552624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac15a750 msgr2=0x7f2cac06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cb2000640 1 --2- 172.21.15.46:0/3584552624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac15a750 0x7f2cac06ec10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0097c0 tx=0x7f2c9c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cb2000640 1 -- 172.21.15.46:0/3584552624 shutdown_connections 2024-04-25T08:43:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cb2000640 1 --2- 172.21.15.46:0/3584552624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac15a750 0x7f2cac06ec10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cb2000640 1 -- 172.21.15.46:0/3584552624 >> 172.21.15.46:0/3584552624 conn(0x7f2cac0b12a0 msgr2=0x7f2cac0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cb2000640 1 -- 172.21.15.46:0/3584552624 shutdown_connections 2024-04-25T08:43:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cb2000640 1 -- 172.21.15.46:0/3584552624 wait complete. 2024-04-25T08:43:08.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cb2000640 1 Processor -- start 2024-04-25T08:43:08.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.763+0000 7f2cb2000640 1 -- start start 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cb2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cac0aed10 0x7f2cac143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cb2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cac0cc9e0 0x7f2cac144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cb2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac15a750 0x7f2cac15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cb2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac168a50 con 0x7f2cac0aed10 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cb2000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac06f560 con 0x7f2cac15a750 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cb2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac168bd0 con 0x7f2cac0cc9e0 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cac0aed10 0x7f2cac143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cac0aed10 0x7f2cac143960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51510/0 (socket says 172.21.15.46:51510) 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cab400640 1 -- 172.21.15.46:0/3874269426 learned_addr learned my addr 172.21.15.46:0/3874269426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:08.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2ca3e00640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cac0cc9e0 0x7f2cac144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cabe00640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac15a750 0x7f2cac15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cab400640 1 -- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cac0cc9e0 msgr2=0x7f2cac144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cab400640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cac0cc9e0 0x7f2cac144050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cab400640 1 -- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac15a750 msgr2=0x7f2cac15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cab400640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac15a750 0x7f2cac15cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cab400640 1 -- 172.21.15.46:0/3874269426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac144b10 con 0x7f2cac0aed10 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cab400640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cac0aed10 0x7f2cac143960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0ae040 tx=0x7f2c9c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2ca3400640 1 -- 172.21.15.46:0/3874269426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0bc070 con 0x7f2cac0aed10 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2ca3400640 1 -- 172.21.15.46:0/3874269426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c0b1040 con 0x7f2cac0aed10 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac144df0 con 0x7f2cac0aed10 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2ca3400640 1 -- 172.21.15.46:0/3874269426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0b0050 con 0x7f2cac0aed10 2024-04-25T08:43:08.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac145c00 con 0x7f2cac0aed10 2024-04-25T08:43:08.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac069080 con 0x7f2cac0aed10 2024-04-25T08:43:08.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.767+0000 7f2ca3400640 1 -- 172.21.15.46:0/3874269426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c9c0bf070 con 0x7f2cac0aed10 2024-04-25T08:43:08.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.771+0000 7f2ca3400640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c88066f90 0x7f2c88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:08.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.771+0000 7f2ca3400640 1 -- 172.21.15.46:0/3874269426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2c9c11ef00 con 0x7f2cac0aed10 2024-04-25T08:43:08.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.771+0000 7f2ca3e00640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c88066f90 0x7f2c88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:08.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.771+0000 7f2ca3e00640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c88066f90 0x7f2c88069450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2cac14c4e0 tx=0x7f2c94009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:08.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.775+0000 7f2ca3400640 1 -- 172.21.15.46:0/3874269426 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c9c016090 con 0x7f2cac0aed10 2024-04-25T08:43:08.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:08.995+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f2cac15c190 con 0x7f2cac0aed10 2024-04-25T08:43:09.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.547+0000 7f2ca3400640 1 -- 172.21.15.46:0/3874269426 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v68) v1 ==== 145+0+0 (secure 0 0 0) 0x7f2c9c0efb00 con 0x7f2cac0aed10 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c88066f90 msgr2=0x7f2c88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c88066f90 0x7f2c88069450 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2cac14c4e0 tx=0x7f2c94009040 comp rx=0 tx=0).stop 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cac0aed10 msgr2=0x7f2cac143960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cac0aed10 0x7f2cac143960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0ae040 tx=0x7f2c9c01ace0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 shutdown_connections 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c88066f90 0x7f2c88069450 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cac15a750 0x7f2cac15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cac0cc9e0 0x7f2cac144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 --2- 172.21.15.46:0/3874269426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cac0aed10 0x7f2cac143960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 >> 172.21.15.46:0/3874269426 conn(0x7f2cac0b12a0 msgr2=0x7f2cac06e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 shutdown_connections 2024-04-25T08:43:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.551+0000 7f2cb2000640 1 -- 172.21.15.46:0/3874269426 wait complete. 2024-04-25T08:43:09.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:43:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.715+0000 7fa152800640 1 Processor -- start 2024-04-25T08:43:09.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.715+0000 7fa152800640 1 -- start start 2024-04-25T08:43:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa152800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa14c06c730 0x7fa14c06bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa152800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c09a410 0x7fa14c06cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa152800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14c09acb0 con 0x7fa14c0d0390 2024-04-25T08:43:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa152800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c0b3dc0 con 0x7fa14c06c730 2024-04-25T08:43:09.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa152800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0cf680 con 0x7fa14c09a410 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa151400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa14c06c730 0x7fa14c06bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa150a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c09a410 0x7fa14c06cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa151400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa14c06c730 0x7fa14c06bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:37668/0 (socket says 172.21.15.46:37668) 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa151e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa14c0d0390 0x7fa14c0cbab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:38168/0 (socket says 172.21.15.46:38168) 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa151400640 1 -- 172.21.15.46:0/3114393643 learned_addr learned my addr 172.21.15.46:0/3114393643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa150a00640 1 -- 172.21.15.46:0/3114393643 >> v2:172.21.15.82:3300/0 conn(0x7fa14c06c730 msgr2=0x7fa14c06bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa150a00640 1 --2- 172.21.15.46:0/3114393643 >> v2:172.21.15.82:3300/0 conn(0x7fa14c06c730 0x7fa14c06bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa150a00640 1 -- 172.21.15.46:0/3114393643 >> v1:172.21.15.82:6789/0 conn(0x7fa14c0d0390 legacy=0x7fa14c0cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.719+0000 7fa150a00640 1 -- 172.21.15.46:0/3114393643 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c09a9d0 con 0x7fa14c09a410 2024-04-25T08:43:09.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa150a00640 1 --2- 172.21.15.46:0/3114393643 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c09a410 0x7fa14c06cb10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1380097c0 tx=0x7fa13801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22692119be544477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa14be00640 1 -- 172.21.15.46:0/3114393643 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1380b2070 con 0x7fa14c09a410 2024-04-25T08:43:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa14be00640 1 -- 172.21.15.46:0/3114393643 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1380ad680 con 0x7fa14c09a410 2024-04-25T08:43:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa14be00640 1 -- 172.21.15.46:0/3114393643 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1380ada50 con 0x7fa14c09a410 2024-04-25T08:43:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 -- 172.21.15.46:0/3114393643 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c09a410 msgr2=0x7fa14c06cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 --2- 172.21.15.46:0/3114393643 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c09a410 0x7fa14c06cb10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa1380097c0 tx=0x7fa13801c470 comp rx=0 tx=0).stop 2024-04-25T08:43:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 -- 172.21.15.46:0/3114393643 shutdown_connections 2024-04-25T08:43:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 --2- 172.21.15.46:0/3114393643 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c09a410 0x7fa14c06cb10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 --2- 172.21.15.46:0/3114393643 >> v2:172.21.15.82:3300/0 conn(0x7fa14c06c730 0x7fa14c06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 -- 172.21.15.46:0/3114393643 >> 172.21.15.46:0/3114393643 conn(0x7fa14c0b2f50 msgr2=0x7fa14c0b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 -- 172.21.15.46:0/3114393643 shutdown_connections 2024-04-25T08:43:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 -- 172.21.15.46:0/3114393643 wait complete. 2024-04-25T08:43:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 Processor -- start 2024-04-25T08:43:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 -- start start 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c06c730 0x7fa14c157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa14c09a410 0x7fa14c1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14c0d0390 0x7fa14c0b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c06d050 con 0x7fa14c0d0390 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c15dd60 con 0x7fa14c09a410 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.723+0000 7fa152800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0cc1c0 con 0x7fa14c06c730 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14c0d0390 0x7fa14c0b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14c0d0390 0x7fa14c0b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51526/0 (socket says 172.21.15.46:51526) 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151e00640 1 -- 172.21.15.46:0/1148242289 learned_addr learned my addr 172.21.15.46:0/1148242289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa150a00640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa14c09a410 0x7fa14c1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151400640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c06c730 0x7fa14c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151e00640 1 -- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c06c730 msgr2=0x7fa14c157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151e00640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c06c730 0x7fa14c157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151e00640 1 -- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa14c09a410 msgr2=0x7fa14c1510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151e00640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa14c09a410 0x7fa14c1510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151e00640 1 -- 172.21.15.46:0/1148242289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c151620 con 0x7fa14c0d0390 2024-04-25T08:43:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa151e00640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14c0d0390 0x7fa14c0b1580 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa14400c700 tx=0x7fa14400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa149600640 1 -- 172.21.15.46:0/1148242289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144017070 con 0x7fa14c0d0390 2024-04-25T08:43:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa149600640 1 -- 172.21.15.46:0/1148242289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14400ce10 con 0x7fa14c0d0390 2024-04-25T08:43:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14c0ca7c0 con 0x7fa14c0d0390 2024-04-25T08:43:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa149600640 1 -- 172.21.15.46:0/1148242289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400a840 con 0x7fa14c0d0390 2024-04-25T08:43:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c0cad80 con 0x7fa14c0d0390 2024-04-25T08:43:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14c069010 con 0x7fa14c0d0390 2024-04-25T08:43:09.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.727+0000 7fa149600640 1 -- 172.21.15.46:0/1148242289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa14401a070 con 0x7fa14c0d0390 2024-04-25T08:43:09.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.731+0000 7fa149600640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa12c066f60 0x7fa12c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:09.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.731+0000 7fa149600640 1 -- 172.21.15.46:0/1148242289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa144084930 con 0x7fa14c0d0390 2024-04-25T08:43:09.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.731+0000 7fa151400640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa12c066f60 0x7fa12c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:09.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.731+0000 7fa151400640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa12c066f60 0x7fa12c069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa14c157610 tx=0x7fa140009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:09.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.735+0000 7fa149600640 1 -- 172.21.15.46:0/1148242289 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa144054e50 con 0x7fa14c0d0390 2024-04-25T08:43:09.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.899+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa14c06c510 con 0x7fa14c0d0390 2024-04-25T08:43:09.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa149600640 1 -- 172.21.15.46:0/1148242289 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa144058ef0 con 0x7fa14c0d0390 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa12c066f60 msgr2=0x7fa12c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa12c066f60 0x7fa12c069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa14c157610 tx=0x7fa140009240 comp rx=0 tx=0).stop 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14c0d0390 msgr2=0x7fa14c0b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14c0d0390 0x7fa14c0b1580 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa14400c700 tx=0x7fa14400cbc0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 shutdown_connections 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa12c066f60 0x7fa12c069420 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa14c0d0390 0x7fa14c0b1580 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa14c09a410 0x7fa14c1510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 --2- 172.21.15.46:0/1148242289 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa14c06c730 0x7fa14c157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 >> 172.21.15.46:0/1148242289 conn(0x7fa14c0b2f50 msgr2=0x7fa14c073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 shutdown_connections 2024-04-25T08:43:09.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:09.903+0000 7fa152800640 1 -- 172.21.15.46:0/1148242289 wait complete. 2024-04-25T08:43:09.915 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:43:09.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:43:09.996 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:43:10.053 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:43:10.053 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:10.053 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:10.061 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:10.062 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:10.114 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:10.114 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:10.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91c600640 1 Processor -- start 2024-04-25T08:43:10.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91c600640 1 -- start start 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe91415ef20 0x7fe9140cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91c600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe914099a00 0x7fe914099eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91c600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91415f300 con 0x7fe9140cd6d0 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91c600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9140cddb0 con 0x7fe914099a00 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140cc3d0 con 0x7fe91415ef20 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe914099a00 0x7fe914099eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe914099a00 0x7fe914099eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51530/0 (socket says 172.21.15.46:51530) 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91a600640 1 -- 172.21.15.46:0/3067341179 learned_addr learned my addr 172.21.15.46:0/3067341179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe919c00640 1 --2- 172.21.15.46:0/3067341179 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe91415ef20 0x7fe9140cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91a600640 1 -- 172.21.15.46:0/3067341179 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe91415ef20 msgr2=0x7fe9140cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91a600640 1 --2- 172.21.15.46:0/3067341179 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe91415ef20 0x7fe9140cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91a600640 1 -- 172.21.15.46:0/3067341179 >> v1:172.21.15.82:6789/0 conn(0x7fe9140cd6d0 legacy=0x7fe91409a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.359+0000 7fe91a600640 1 -- 172.21.15.46:0/3067341179 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140cc6b0 con 0x7fe914099a00 2024-04-25T08:43:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe91a600640 1 --2- 172.21.15.46:0/3067341179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe914099a00 0x7fe914099eb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009e50 tx=0x7fe90c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ba12027503848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe913e00640 1 -- 172.21.15.46:0/3067341179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0a57f0 con 0x7fe914099a00 2024-04-25T08:43:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe913e00640 1 -- 172.21.15.46:0/3067341179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c0a5990 con 0x7fe914099a00 2024-04-25T08:43:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe913e00640 1 -- 172.21.15.46:0/3067341179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0a5d60 con 0x7fe914099a00 2024-04-25T08:43:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe91c600640 1 -- 172.21.15.46:0/3067341179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe914099a00 msgr2=0x7fe914099eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe91c600640 1 --2- 172.21.15.46:0/3067341179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe914099a00 0x7fe914099eb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009e50 tx=0x7fe90c01ca30 comp rx=0 tx=0).stop 2024-04-25T08:43:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe91c600640 1 -- 172.21.15.46:0/3067341179 shutdown_connections 2024-04-25T08:43:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe91c600640 1 --2- 172.21.15.46:0/3067341179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe914099a00 0x7fe914099eb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe91c600640 1 --2- 172.21.15.46:0/3067341179 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe91415ef20 0x7fe9140cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe91c600640 1 -- 172.21.15.46:0/3067341179 >> 172.21.15.46:0/3067341179 conn(0x7fe9140b3120 msgr2=0x7fe9140b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe91c600640 1 -- 172.21.15.46:0/3067341179 shutdown_connections 2024-04-25T08:43:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.363+0000 7fe91c600640 1 -- 172.21.15.46:0/3067341179 wait complete. 2024-04-25T08:43:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 Processor -- start 2024-04-25T08:43:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 -- start start 2024-04-25T08:43:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe914099a00 0x7fe9140b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe91415ef20 0x7fe9140c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9141410a0 0x7fe91413e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91409aea0 con 0x7fe9141410a0 2024-04-25T08:43:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91409ad20 con 0x7fe91415ef20 2024-04-25T08:43:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe914168880 con 0x7fe914099a00 2024-04-25T08:43:10.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9141410a0 0x7fe91413e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9141410a0 0x7fe91413e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51546/0 (socket says 172.21.15.46:51546) 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91a600640 1 -- 172.21.15.46:0/658753833 learned_addr learned my addr 172.21.15.46:0/658753833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe919c00640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe914099a00 0x7fe9140b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe919200640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe91415ef20 0x7fe9140c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91a600640 1 -- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe914099a00 msgr2=0x7fe9140b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91a600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe914099a00 0x7fe9140b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91a600640 1 -- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe91415ef20 msgr2=0x7fe9140c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91a600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe91415ef20 0x7fe9140c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91a600640 1 -- 172.21.15.46:0/658753833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9141415a0 con 0x7fe9141410a0 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe919200640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe91415ef20 0x7fe9140c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91a600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9141410a0 0x7fe91413e600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe90c01a660 tx=0x7fe90c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe911600640 1 -- 172.21.15.46:0/658753833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0bc020 con 0x7fe9141410a0 2024-04-25T08:43:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe911600640 1 -- 172.21.15.46:0/658753833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90c0b8670 con 0x7fe9141410a0 2024-04-25T08:43:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe911600640 1 -- 172.21.15.46:0/658753833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0b8a20 con 0x7fe9141410a0 2024-04-25T08:43:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9140c87a0 con 0x7fe9141410a0 2024-04-25T08:43:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.367+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9140c8d60 con 0x7fe9141410a0 2024-04-25T08:43:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.371+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe914069080 con 0x7fe9141410a0 2024-04-25T08:43:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.371+0000 7fe911600640 1 -- 172.21.15.46:0/658753833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe90c0bf020 con 0x7fe9141410a0 2024-04-25T08:43:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.371+0000 7fe911600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe8f4066f60 0x7fe8f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.371+0000 7fe911600640 1 -- 172.21.15.46:0/658753833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe90c1205c0 con 0x7fe9141410a0 2024-04-25T08:43:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.371+0000 7fe919c00640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe8f4066f60 0x7fe8f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.371+0000 7fe919c00640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe8f4066f60 0x7fe8f4069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe908009b60 tx=0x7fe908009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.375+0000 7fe911600640 1 -- 172.21.15.46:0/658753833 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe90c0c6030 con 0x7fe9141410a0 2024-04-25T08:43:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.539+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9140d0700 con 0x7fe9141410a0 2024-04-25T08:43:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.539+0000 7fe911600640 1 -- 172.21.15.46:0/658753833 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7fe90c016620 con 0x7fe9141410a0 2024-04-25T08:43:10.534 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:10.534 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe8f4066f60 msgr2=0x7fe8f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe8f4066f60 0x7fe8f4069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe908009b60 tx=0x7fe908009190 comp rx=0 tx=0).stop 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9141410a0 msgr2=0x7fe91413e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9141410a0 0x7fe91413e600 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe90c01a660 tx=0x7fe90c0197d0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 shutdown_connections 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe8f4066f60 0x7fe8f4069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9141410a0 0x7fe91413e600 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe91415ef20 0x7fe9140c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 --2- 172.21.15.46:0/658753833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe914099a00 0x7fe9140b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 >> 172.21.15.46:0/658753833 conn(0x7fe9140b3120 msgr2=0x7fe9140cfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 shutdown_connections 2024-04-25T08:43:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.543+0000 7fe91c600640 1 -- 172.21.15.46:0/658753833 wait complete. 2024-04-25T08:43:10.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:10.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:10.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.707+0000 7ff976e00640 1 Processor -- start 2024-04-25T08:43:10.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.707+0000 7ff976e00640 1 -- start start 2024-04-25T08:43:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff976e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970145310 0x7ff97006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff976e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff97006dc70 0x7ff9700cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff976e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9700cd930 con 0x7ff9700cf1b0 2024-04-25T08:43:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff976e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9700cdbd0 con 0x7ff97006dc70 2024-04-25T08:43:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff976e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9700cdeb0 con 0x7ff970145310 2024-04-25T08:43:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff967400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff97006dc70 0x7ff9700cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970145310 0x7ff97006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff967400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff97006dc70 0x7ff9700cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:37684/0 (socket says 172.21.15.46:37684) 2024-04-25T08:43:10.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff967400640 1 -- 172.21.15.46:0/2757137494 learned_addr learned my addr 172.21.15.46:0/2757137494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:10.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff974e00640 1 --1- 172.21.15.46:0/2757137494 >> v1:172.21.15.82:6789/0 conn(0x7ff9700cf1b0 0x7ff97015d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:38182/0 (socket says 172.21.15.46:38182) 2024-04-25T08:43:10.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff967400640 1 -- 172.21.15.46:0/2757137494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970145310 msgr2=0x7ff97006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:10.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff967400640 1 --2- 172.21.15.46:0/2757137494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970145310 0x7ff97006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff967400640 1 -- 172.21.15.46:0/2757137494 >> v1:172.21.15.82:6789/0 conn(0x7ff9700cf1b0 legacy=0x7ff97015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.711+0000 7ff967400640 1 -- 172.21.15.46:0/2757137494 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9700ce190 con 0x7ff97006dc70 2024-04-25T08:43:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff967400640 1 --2- 172.21.15.46:0/2757137494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff97006dc70 0x7ff9700cec70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff958009870 tx=0x7ff95801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5fc8695b3e125c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff96f400640 1 -- 172.21.15.46:0/2757137494 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580a95f0 con 0x7ff97006dc70 2024-04-25T08:43:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff96f400640 1 -- 172.21.15.46:0/2757137494 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9580a9790 con 0x7ff97006dc70 2024-04-25T08:43:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff96f400640 1 -- 172.21.15.46:0/2757137494 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580a9b60 con 0x7ff97006dc70 2024-04-25T08:43:10.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 -- 172.21.15.46:0/2757137494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff97006dc70 msgr2=0x7ff9700cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 --2- 172.21.15.46:0/2757137494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff97006dc70 0x7ff9700cec70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff958009870 tx=0x7ff95801c530 comp rx=0 tx=0).stop 2024-04-25T08:43:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 -- 172.21.15.46:0/2757137494 shutdown_connections 2024-04-25T08:43:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 --2- 172.21.15.46:0/2757137494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff97006dc70 0x7ff9700cec70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 --2- 172.21.15.46:0/2757137494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970145310 0x7ff97006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 -- 172.21.15.46:0/2757137494 >> 172.21.15.46:0/2757137494 conn(0x7ff97006ac10 msgr2=0x7ff97006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 -- 172.21.15.46:0/2757137494 shutdown_connections 2024-04-25T08:43:10.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 -- 172.21.15.46:0/2757137494 wait complete. 2024-04-25T08:43:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 Processor -- start 2024-04-25T08:43:10.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.715+0000 7ff976e00640 1 -- start start 2024-04-25T08:43:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff976e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9700cf1b0 0x7ff9700aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff976e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff970145310 0x7ff970142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff976e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970143300 0x7ff9700b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff976e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97015d920 con 0x7ff9700cf1b0 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff976e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97015d7a0 con 0x7ff970145310 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff976e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff970168510 con 0x7ff970143300 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9700cf1b0 0x7ff9700aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9700cf1b0 0x7ff9700aa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51552/0 (socket says 172.21.15.46:51552) 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96fe00640 1 -- 172.21.15.46:0/3328880555 learned_addr learned my addr 172.21.15.46:0/3328880555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff974e00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970143300 0x7ff9700b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff967400640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff970145310 0x7ff970142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96fe00640 1 -- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970143300 msgr2=0x7ff9700b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96fe00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970143300 0x7ff9700b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96fe00640 1 -- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff970145310 msgr2=0x7ff970142dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96fe00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff970145310 0x7ff970142dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96fe00640 1 -- 172.21.15.46:0/3328880555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97013b630 con 0x7ff9700cf1b0 2024-04-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff967400640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff970145310 0x7ff970142dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96fe00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9700cf1b0 0x7ff9700aa170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e8a0 tx=0x7ff96000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96cc00640 1 -- 172.21.15.46:0/3328880555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff960017070 con 0x7ff9700cf1b0 2024-04-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96cc00640 1 -- 172.21.15.46:0/3328880555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96000c940 con 0x7ff9700cf1b0 2024-04-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96cc00640 1 -- 172.21.15.46:0/3328880555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96000ccf0 con 0x7ff9700cf1b0 2024-04-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff970139d80 con 0x7ff9700cf1b0 2024-04-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97013a020 con 0x7ff9700cf1b0 2024-04-25T08:43:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff970069010 con 0x7ff9700cf1b0 2024-04-25T08:43:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.719+0000 7ff96cc00640 1 -- 172.21.15.46:0/3328880555 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff96001a070 con 0x7ff9700cf1b0 2024-04-25T08:43:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.723+0000 7ff96cc00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff94c066f90 0x7ff94c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.723+0000 7ff96cc00640 1 -- 172.21.15.46:0/3328880555 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff960085240 con 0x7ff9700cf1b0 2024-04-25T08:43:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.723+0000 7ff967400640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff94c066f90 0x7ff94c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.723+0000 7ff967400640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff94c066f90 0x7ff94c069450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9580ae000 tx=0x7ff9580b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:10.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.727+0000 7ff96cc00640 1 -- 172.21.15.46:0/3328880555 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff97013b630 con 0x7ff9700cf1b0 2024-04-25T08:43:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.887+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff97013fb90 con 0x7ff9700cf1b0 2024-04-25T08:43:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.887+0000 7ff96cc00640 1 -- 172.21.15.46:0/3328880555 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff960055890 con 0x7ff9700cf1b0 2024-04-25T08:43:10.881 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff94c066f90 msgr2=0x7ff94c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff94c066f90 0x7ff94c069450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9580ae000 tx=0x7ff9580b3040 comp rx=0 tx=0).stop 2024-04-25T08:43:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9700cf1b0 msgr2=0x7ff9700aa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9700cf1b0 0x7ff9700aa170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff96000e8a0 tx=0x7ff96000eca0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 shutdown_connections 2024-04-25T08:43:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff94c066f90 0x7ff94c069450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff970143300 0x7ff9700b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff970145310 0x7ff970142dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 --2- 172.21.15.46:0/3328880555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9700cf1b0 0x7ff9700aa170 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 >> 172.21.15.46:0/3328880555 conn(0x7ff97006ac10 msgr2=0x7ff97015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 shutdown_connections 2024-04-25T08:43:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:10.891+0000 7ff976e00640 1 -- 172.21.15.46:0/3328880555 wait complete. 2024-04-25T08:43:10.902 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.051+0000 7f49ba800640 1 Processor -- start 2024-04-25T08:43:11.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.051+0000 7f49ba800640 1 -- start start 2024-04-25T08:43:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49ba800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49b4143990 0x7f49b40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49ba800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f49b41440d0 0x7f49b406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49ba800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b406ef80 0x7f49b40cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49ba800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b415ed30 con 0x7f49b4143990 2024-04-25T08:43:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49ba800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49b415e440 con 0x7f49b41440d0 2024-04-25T08:43:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49ba800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b415e6e0 con 0x7f49b406ef80 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49b4143990 0x7f49b40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49b4143990 0x7f49b40cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51564/0 (socket says 172.21.15.46:51564) 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3400640 1 -- 172.21.15.46:0/2717047958 learned_addr learned my addr 172.21.15.46:0/2717047958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49abe00640 1 --2- 172.21.15.46:0/2717047958 >> v2:172.21.15.82:3300/0 conn(0x7f49b41440d0 0x7f49b406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3400640 1 -- 172.21.15.46:0/2717047958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b406ef80 msgr2=0x7f49b40cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3e00640 1 --2- 172.21.15.46:0/2717047958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b406ef80 0x7f49b40cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3400640 1 --2- 172.21.15.46:0/2717047958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b406ef80 0x7f49b40cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3400640 1 -- 172.21.15.46:0/2717047958 >> v2:172.21.15.82:3300/0 conn(0x7f49b41440d0 msgr2=0x7f49b406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3400640 1 --2- 172.21.15.46:0/2717047958 >> v2:172.21.15.82:3300/0 conn(0x7f49b41440d0 0x7f49b406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3400640 1 -- 172.21.15.46:0/2717047958 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b415e9c0 con 0x7f49b4143990 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49b3e00640 1 --2- 172.21.15.46:0/2717047958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b406ef80 0x7f49b40cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.055+0000 7f49abe00640 1 --2- 172.21.15.46:0/2717047958 >> v2:172.21.15.82:3300/0 conn(0x7f49b41440d0 0x7f49b406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:11.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49b3400640 1 --2- 172.21.15.46:0/2717047958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b4143990 0x7f49b40cecc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f49b4070c80 tx=0x7f49a401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7351f0aa44169e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49b2a00640 1 -- 172.21.15.46:0/2717047958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a40b0070 con 0x7f49b4143990 2024-04-25T08:43:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49b2a00640 1 -- 172.21.15.46:0/2717047958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a40ab600 con 0x7f49b4143990 2024-04-25T08:43:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49b2a00640 1 -- 172.21.15.46:0/2717047958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a40ab9d0 con 0x7f49b4143990 2024-04-25T08:43:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 -- 172.21.15.46:0/2717047958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b4143990 msgr2=0x7f49b40cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 --2- 172.21.15.46:0/2717047958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b4143990 0x7f49b40cecc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f49b4070c80 tx=0x7f49a401c260 comp rx=0 tx=0).stop 2024-04-25T08:43:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 -- 172.21.15.46:0/2717047958 shutdown_connections 2024-04-25T08:43:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 --2- 172.21.15.46:0/2717047958 >> v2:172.21.15.82:3300/0 conn(0x7f49b41440d0 0x7f49b406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 --2- 172.21.15.46:0/2717047958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b406ef80 0x7f49b40cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 --2- 172.21.15.46:0/2717047958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b4143990 0x7f49b40cecc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 -- 172.21.15.46:0/2717047958 >> 172.21.15.46:0/2717047958 conn(0x7f49b406ac10 msgr2=0x7f49b406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:11.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 -- 172.21.15.46:0/2717047958 shutdown_connections 2024-04-25T08:43:11.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 -- 172.21.15.46:0/2717047958 wait complete. 2024-04-25T08:43:11.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 Processor -- start 2024-04-25T08:43:11.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.059+0000 7f49ba800640 1 -- start start 2024-04-25T08:43:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ba800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49b406ef80 0x7f49b40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ba800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b4143990 0x7f49b40b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ba800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b41440d0 0x7f49b415d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ba800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4143ef0 con 0x7f49b41440d0 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ba800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4143d70 con 0x7f49b406ef80 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ba800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b40b2720 con 0x7f49b4143990 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b41440d0 0x7f49b415d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b41440d0 0x7f49b415d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51572/0 (socket says 172.21.15.46:51572) 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3e00640 1 -- 172.21.15.46:0/408698734 learned_addr learned my addr 172.21.15.46:0/408698734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49abe00640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b4143990 0x7f49b40b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3400640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49b406ef80 0x7f49b40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3e00640 1 -- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b4143990 msgr2=0x7f49b40b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3e00640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b4143990 0x7f49b40b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3e00640 1 -- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49b406ef80 msgr2=0x7f49b40b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3e00640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49b406ef80 0x7f49b40b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3e00640 1 -- 172.21.15.46:0/408698734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b40b1570 con 0x7f49b41440d0 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49abe00640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b4143990 0x7f49b40b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3400640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49b406ef80 0x7f49b40b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49b3e00640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b41440d0 0x7f49b415d0d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00cef0 tx=0x7f49ac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ab400640 1 -- 172.21.15.46:0/408698734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac01a070 con 0x7f49b41440d0 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ab400640 1 -- 172.21.15.46:0/408698734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac00d860 con 0x7f49b41440d0 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ab400640 1 -- 172.21.15.46:0/408698734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00dc50 con 0x7f49b41440d0 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b40b1850 con 0x7f49b41440d0 2024-04-25T08:43:11.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b40b1e10 con 0x7f49b41440d0 2024-04-25T08:43:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b4069010 con 0x7f49b41440d0 2024-04-25T08:43:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.063+0000 7f49ab400640 1 -- 172.21.15.46:0/408698734 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49ac01d070 con 0x7f49b41440d0 2024-04-25T08:43:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.067+0000 7f49ab400640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4990066f60 0x7f4990069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.067+0000 7f49ab400640 1 -- 172.21.15.46:0/408698734 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f49ac085d80 con 0x7f49b41440d0 2024-04-25T08:43:11.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.067+0000 7f49b3400640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4990066f60 0x7f4990069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.067+0000 7f49b3400640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4990066f60 0x7f4990069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ad000 tx=0x7f49a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:11.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.071+0000 7f49ab400640 1 -- 172.21.15.46:0/408698734 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49ac0569b0 con 0x7f49b41440d0 2024-04-25T08:43:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49b413f970 con 0x7f49b41440d0 2024-04-25T08:43:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ab400640 1 -- 172.21.15.46:0/408698734 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49ac0562b0 con 0x7f49b41440d0 2024-04-25T08:43:11.224 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4990066f60 msgr2=0x7f4990069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4990066f60 0x7f4990069420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f49a40ad000 tx=0x7f49a4008040 comp rx=0 tx=0).stop 2024-04-25T08:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b41440d0 msgr2=0x7f49b415d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b41440d0 0x7f49b415d0d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00cef0 tx=0x7f49ac00a710 comp rx=0 tx=0).stop 2024-04-25T08:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 shutdown_connections 2024-04-25T08:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4990066f60 0x7f4990069420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49b41440d0 0x7f49b415d0d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49b4143990 0x7f49b40b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 --2- 172.21.15.46:0/408698734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49b406ef80 0x7f49b40b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 >> 172.21.15.46:0/408698734 conn(0x7f49b406ac10 msgr2=0x7f49b413a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 shutdown_connections 2024-04-25T08:43:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.231+0000 7f49ba800640 1 -- 172.21.15.46:0/408698734 wait complete. 2024-04-25T08:43:11.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:11.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.395+0000 7f2e30e00640 1 Processor -- start 2024-04-25T08:43:11.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.395+0000 7f2e30e00640 1 -- start start 2024-04-25T08:43:11.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.395+0000 7f2e30e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c145310 0x7f2e2c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.395+0000 7f2e30e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2e2c06dc70 0x7f2e2c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.395+0000 7f2e30e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c0cd930 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.395+0000 7f2e30e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c0cdbd0 con 0x7f2e2c06dc70 2024-04-25T08:43:11.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.395+0000 7f2e30e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0cdeb0 con 0x7f2e2c145310 2024-04-25T08:43:11.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e2a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2e2c0cf1b0 0x7f2e2c15d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60260/0 (socket says 172.21.15.46:60260) 2024-04-25T08:43:11.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e2a800640 1 -- 172.21.15.46:0/3039997086 learned_addr learned my addr 172.21.15.46:0/3039997086 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:11.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e29e00640 1 --2- 172.21.15.46:0/3039997086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c145310 0x7f2e2c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e21400640 1 --2- 172.21.15.46:0/3039997086 >> v2:172.21.15.82:3300/0 conn(0x7f2e2c06dc70 0x7f2e2c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e29400640 1 -- 172.21.15.46:0/3039997086 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468583624 0 0) 0x7f2e2c0cd930 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e29400640 1 -- 172.21.15.46:0/3039997086 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e10003530 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e29e00640 1 -- 172.21.15.46:0/3039997086 >> v2:172.21.15.82:3300/0 conn(0x7f2e2c06dc70 msgr2=0x7f2e2c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:11.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e29e00640 1 --2- 172.21.15.46:0/3039997086 >> v2:172.21.15.82:3300/0 conn(0x7f2e2c06dc70 0x7f2e2c0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e29e00640 1 -- 172.21.15.46:0/3039997086 >> v1:172.21.15.46:6789/0 conn(0x7f2e2c0cf1b0 legacy=0x7f2e2c15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e29e00640 1 -- 172.21.15.46:0/3039997086 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0ce190 con 0x7f2e2c145310 2024-04-25T08:43:11.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e21400640 1 --2- 172.21.15.46:0/3039997086 >> v2:172.21.15.82:3300/0 conn(0x7f2e2c06dc70 0x7f2e2c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:11.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.399+0000 7f2e29e00640 1 --2- 172.21.15.46:0/3039997086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c145310 0x7f2e2c06e0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0097c0 tx=0x7f2e1c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc1ebeb21c8c73c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:11.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e29400640 1 -- 172.21.15.46:0/3039997086 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0a7070 con 0x7f2e2c145310 2024-04-25T08:43:11.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e29400640 1 -- 172.21.15.46:0/3039997086 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0b0070 con 0x7f2e2c145310 2024-04-25T08:43:11.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e29400640 1 -- 172.21.15.46:0/3039997086 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0ab890 con 0x7f2e2c145310 2024-04-25T08:43:11.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 -- 172.21.15.46:0/3039997086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c145310 msgr2=0x7f2e2c06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 --2- 172.21.15.46:0/3039997086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c145310 0x7f2e2c06e0d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0097c0 tx=0x7f2e1c01c470 comp rx=0 tx=0).stop 2024-04-25T08:43:11.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 -- 172.21.15.46:0/3039997086 shutdown_connections 2024-04-25T08:43:11.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 --2- 172.21.15.46:0/3039997086 >> v2:172.21.15.82:3300/0 conn(0x7f2e2c06dc70 0x7f2e2c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 --2- 172.21.15.46:0/3039997086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c145310 0x7f2e2c06e0d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 -- 172.21.15.46:0/3039997086 >> 172.21.15.46:0/3039997086 conn(0x7f2e2c06ac10 msgr2=0x7f2e2c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:11.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 -- 172.21.15.46:0/3039997086 shutdown_connections 2024-04-25T08:43:11.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 -- 172.21.15.46:0/3039997086 wait complete. 2024-04-25T08:43:11.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 Processor -- start 2024-04-25T08:43:11.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 -- start start 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e2c0cf1b0 0x7f2e2c141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e2c145310 0x7f2e2c1422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c1f6970 0x7f2e2c0b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c15d920 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c15d7a0 con 0x7f2e2c145310 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e30e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c168510 con 0x7f2e2c1f6970 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e29e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e2c0cf1b0 0x7f2e2c141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e29e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e2c0cf1b0 0x7f2e2c141da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51582/0 (socket says 172.21.15.46:51582) 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e29e00640 1 -- 172.21.15.46:0/702943538 learned_addr learned my addr 172.21.15.46:0/702943538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e21400640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e2c145310 0x7f2e2c1422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.403+0000 7f2e2a800640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c1f6970 0x7f2e2c0b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e29e00640 1 -- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c1f6970 msgr2=0x7f2e2c0b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e29e00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c1f6970 0x7f2e2c0b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e29e00640 1 -- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e2c145310 msgr2=0x7f2e2c1422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e29e00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e2c145310 0x7f2e2c1422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e29e00640 1 -- 172.21.15.46:0/702943538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c0a8ad0 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e2a800640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c1f6970 0x7f2e2c0b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e29e00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e2c0cf1b0 0x7f2e2c141da0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0ad040 tx=0x7f2e1c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e22a00640 1 -- 172.21.15.46:0/702943538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0bf610 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e22a00640 1 -- 172.21.15.46:0/702943538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0bf7f0 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e22a00640 1 -- 172.21.15.46:0/702943538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0bfbe0 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c142980 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c142f40 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c069010 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e22a00640 1 -- 172.21.15.46:0/702943538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e1c0bd020 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e22a00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e08066f60 0x7f2e08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.407+0000 7f2e22a00640 1 -- 172.21.15.46:0/702943538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2e1c11f590 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.411+0000 7f2e21400640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e08066f60 0x7f2e08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.411+0000 7f2e21400640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e08066f60 0x7f2e08069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009580 tx=0x7f2e14009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:11.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.411+0000 7f2e22a00640 1 -- 172.21.15.46:0/702943538 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e1c0c6030 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e2c0b4680 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e22a00640 1 -- 172.21.15.46:0/702943538 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e1c016090 con 0x7f2e2c0cf1b0 2024-04-25T08:43:11.568 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e08066f60 msgr2=0x7f2e08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e08066f60 0x7f2e08069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009580 tx=0x7f2e14009190 comp rx=0 tx=0).stop 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e2c0cf1b0 msgr2=0x7f2e2c141da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e2c0cf1b0 0x7f2e2c141da0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0ad040 tx=0x7f2e1c01acb0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 shutdown_connections 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e08066f60 0x7f2e08069420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2e2c1f6970 0x7f2e2c0b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e2c145310 0x7f2e2c1422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 --2- 172.21.15.46:0/702943538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e2c0cf1b0 0x7f2e2c141da0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 >> 172.21.15.46:0/702943538 conn(0x7f2e2c06ac10 msgr2=0x7f2e2c15a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 shutdown_connections 2024-04-25T08:43:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.575+0000 7f2e30e00640 1 -- 172.21.15.46:0/702943538 wait complete. 2024-04-25T08:43:11.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:11.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f775cc00640 1 Processor -- start 2024-04-25T08:43:11.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f775cc00640 1 -- start start 2024-04-25T08:43:11.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f775cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580d31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f775cc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f77580ce5c0 0x7f77580ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f775cc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77580cef20 con 0x7f775815da60 2024-04-25T08:43:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f775cc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77580d2110 con 0x7f77580ce5c0 2024-04-25T08:43:11.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f775cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77580d23f0 con 0x7f77580b9410 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f7756800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f77580ce5c0 0x7f77580ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f774d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580d31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f7755e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f775815da60 0x7f77580d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:38196/0 (socket says 172.21.15.46:38196) 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f7755e00640 1 -- 172.21.15.46:0/3870755128 learned_addr learned my addr 172.21.15.46:0/3870755128 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f7755400640 1 -- 172.21.15.46:0/3870755128 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844811589 0 0) 0x7f77580cef20 con 0x7f775815da60 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f7755400640 1 -- 172.21.15.46:0/3870755128 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f773c003530 con 0x7f775815da60 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f7756800640 1 -- 172.21.15.46:0/3870755128 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 msgr2=0x7f77580d31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f7756800640 1 --2- 172.21.15.46:0/3870755128 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580d31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f7756800640 1 -- 172.21.15.46:0/3870755128 >> v1:172.21.15.82:6789/0 conn(0x7f775815da60 legacy=0x7f77580d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f7756800640 1 -- 172.21.15.46:0/3870755128 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580d26d0 con 0x7f77580ce5c0 2024-04-25T08:43:11.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.739+0000 7f774d400640 1 --2- 172.21.15.46:0/3870755128 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:11.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f7756800640 1 --2- 172.21.15.46:0/3870755128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77580ce5c0 0x7f77580ce9e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7750009d90 tx=0x7f775001f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207a90b01b7c1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:11.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f7755400640 1 -- 172.21.15.46:0/3870755128 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77500b4070 con 0x7f77580ce5c0 2024-04-25T08:43:11.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f7755400640 1 -- 172.21.15.46:0/3870755128 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77500af600 con 0x7f77580ce5c0 2024-04-25T08:43:11.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f7755400640 1 -- 172.21.15.46:0/3870755128 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77500af9d0 con 0x7f77580ce5c0 2024-04-25T08:43:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f775cc00640 1 -- 172.21.15.46:0/3870755128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77580ce5c0 msgr2=0x7f77580ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f775cc00640 1 --2- 172.21.15.46:0/3870755128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77580ce5c0 0x7f77580ce9e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7750009d90 tx=0x7f775001f150 comp rx=0 tx=0).stop 2024-04-25T08:43:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f775cc00640 1 -- 172.21.15.46:0/3870755128 shutdown_connections 2024-04-25T08:43:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f775cc00640 1 --2- 172.21.15.46:0/3870755128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77580ce5c0 0x7f77580ce9e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f775cc00640 1 --2- 172.21.15.46:0/3870755128 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580d31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f775cc00640 1 -- 172.21.15.46:0/3870755128 >> 172.21.15.46:0/3870755128 conn(0x7f7758072940 msgr2=0x7f775806be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f775cc00640 1 -- 172.21.15.46:0/3870755128 shutdown_connections 2024-04-25T08:43:11.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.743+0000 7f775cc00640 1 -- 172.21.15.46:0/3870755128 wait complete. 2024-04-25T08:43:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 Processor -- start 2024-04-25T08:43:11.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 -- start start 2024-04-25T08:43:11.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77580ce5c0 0x7f77580b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f775815da60 0x7f77580aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775815de40 con 0x7f77580ce5c0 2024-04-25T08:43:11.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580d2c90 con 0x7f775815da60 2024-04-25T08:43:11.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775815dfc0 con 0x7f77580b9410 2024-04-25T08:43:11.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77580ce5c0 0x7f77580b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77580ce5c0 0x7f77580b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51590/0 (socket says 172.21.15.46:51590) 2024-04-25T08:43:11.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774d400640 1 -- 172.21.15.46:0/3533138343 learned_addr learned my addr 172.21.15.46:0/3533138343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f7755e00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f7756800640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f775815da60 0x7f77580aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774d400640 1 -- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 msgr2=0x7f77580b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774d400640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774d400640 1 -- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f775815da60 msgr2=0x7f77580aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774d400640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f775815da60 0x7f77580aa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774d400640 1 -- 172.21.15.46:0/3533138343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580b3780 con 0x7f77580ce5c0 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f7755e00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f7756800640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f775815da60 0x7f77580aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:11.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774d400640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77580ce5c0 0x7f77580b3240 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f774000e980 tx=0x7f774000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774ea00640 1 -- 172.21.15.46:0/3533138343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740017070 con 0x7f77580ce5c0 2024-04-25T08:43:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774ea00640 1 -- 172.21.15.46:0/3533138343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774000ca00 con 0x7f77580ce5c0 2024-04-25T08:43:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f774ea00640 1 -- 172.21.15.46:0/3533138343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774000cdd0 con 0x7f77580ce5c0 2024-04-25T08:43:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775814adc0 con 0x7f77580ce5c0 2024-04-25T08:43:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.747+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775814b380 con 0x7f77580ce5c0 2024-04-25T08:43:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.751+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77580cdff0 con 0x7f77580ce5c0 2024-04-25T08:43:11.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.751+0000 7f774ea00640 1 -- 172.21.15.46:0/3533138343 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f774001a070 con 0x7f77580ce5c0 2024-04-25T08:43:11.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.751+0000 7f774ea00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7734066f60 0x7f7734069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:11.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.751+0000 7f774ea00640 1 -- 172.21.15.46:0/3533138343 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7740085400 con 0x7f77580ce5c0 2024-04-25T08:43:11.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.751+0000 7f7755e00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7734066f60 0x7f7734069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:11.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.751+0000 7f7755e00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7734066f60 0x7f7734069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7748005890 tx=0x7f7748009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:11.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.755+0000 7f774ea00640 1 -- 172.21.15.46:0/3533138343 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7740055f70 con 0x7f77580ce5c0 2024-04-25T08:43:11.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.915+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7758069010 con 0x7f77580ce5c0 2024-04-25T08:43:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.915+0000 7f774ea00640 1 -- 172.21.15.46:0/3533138343 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7740055870 con 0x7f77580ce5c0 2024-04-25T08:43:11.909 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:11.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7734066f60 msgr2=0x7f7734069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7734066f60 0x7f7734069420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7748005890 tx=0x7f7748009040 comp rx=0 tx=0).stop 2024-04-25T08:43:11.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77580ce5c0 msgr2=0x7f77580b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:11.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77580ce5c0 0x7f77580b3240 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f774000e980 tx=0x7f774000ee40 comp rx=0 tx=0).stop 2024-04-25T08:43:11.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 shutdown_connections 2024-04-25T08:43:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7734066f60 0x7f7734069420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f775815da60 0x7f77580aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77580ce5c0 0x7f77580b3240 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 --2- 172.21.15.46:0/3533138343 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f77580b9410 0x7f77580b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 >> 172.21.15.46:0/3533138343 conn(0x7f7758072940 msgr2=0x7f77580c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 shutdown_connections 2024-04-25T08:43:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:11.919+0000 7f775cc00640 1 -- 172.21.15.46:0/3533138343 wait complete. 2024-04-25T08:43:11.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:43:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95e9a00640 1 Processor -- start 2024-04-25T08:43:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95e9a00640 1 -- start start 2024-04-25T08:43:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95e9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 0x7f95e40b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95e9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95e40b0c30 0x7f95e4070c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95e9a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e40b5250 con 0x7f95e40711d0 2024-04-25T08:43:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95e9a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e40b5af0 con 0x7f95e40b0c30 2024-04-25T08:43:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95e9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40b5530 con 0x7f95e415dbd0 2024-04-25T08:43:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95dbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95e40b0c30 0x7f95e4070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95dbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95e40b0c30 0x7f95e4070c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51602/0 (socket says 172.21.15.46:51602) 2024-04-25T08:43:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95dbe00640 1 -- 172.21.15.46:0/1331915959 learned_addr learned my addr 172.21.15.46:0/1331915959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.079+0000 7f95e2a00640 1 --2- 172.21.15.46:0/1331915959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 0x7f95e40b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95dbe00640 1 -- 172.21.15.46:0/1331915959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 msgr2=0x7f95e40b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95dbe00640 1 --2- 172.21.15.46:0/1331915959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 0x7f95e40b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95dbe00640 1 -- 172.21.15.46:0/1331915959 >> v1:172.21.15.82:6789/0 conn(0x7f95e40711d0 legacy=0x7f95e415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95dbe00640 1 -- 172.21.15.46:0/1331915959 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40b5810 con 0x7f95e40b0c30 2024-04-25T08:43:12.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95dbe00640 1 --2- 172.21.15.46:0/1331915959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 0x7f95e4070c90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009870 tx=0x7f95cc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badd5bfd4a15722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:12.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e2000640 1 -- 172.21.15.46:0/1331915959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0b2070 con 0x7f95e40b0c30 2024-04-25T08:43:12.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e2000640 1 -- 172.21.15.46:0/1331915959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95cc0a9070 con 0x7f95e40b0c30 2024-04-25T08:43:12.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e2000640 1 -- 172.21.15.46:0/1331915959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0ad880 con 0x7f95e40b0c30 2024-04-25T08:43:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e9a00640 1 -- 172.21.15.46:0/1331915959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 msgr2=0x7f95e4070c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1331915959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 0x7f95e4070c90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009870 tx=0x7f95cc01c560 comp rx=0 tx=0).stop 2024-04-25T08:43:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e9a00640 1 -- 172.21.15.46:0/1331915959 shutdown_connections 2024-04-25T08:43:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1331915959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 0x7f95e4070c90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1331915959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 0x7f95e40b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e9a00640 1 -- 172.21.15.46:0/1331915959 >> 172.21.15.46:0/1331915959 conn(0x7f95e406f4f0 msgr2=0x7f95e406f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.083+0000 7f95e9a00640 1 -- 172.21.15.46:0/1331915959 shutdown_connections 2024-04-25T08:43:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 -- 172.21.15.46:0/1331915959 wait complete. 2024-04-25T08:43:12.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 Processor -- start 2024-04-25T08:43:12.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 -- start start 2024-04-25T08:43:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e40711d0 0x7f95e415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 0x7f95e413a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 0x7f95e4158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e416a850 con 0x7f95e40b0c30 2024-04-25T08:43:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40715e0 con 0x7f95e40711d0 2024-04-25T08:43:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e416a9d0 con 0x7f95e415dbd0 2024-04-25T08:43:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 0x7f95e413a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95dbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 0x7f95e413a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51608/0 (socket says 172.21.15.46:51608) 2024-04-25T08:43:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e40711d0 0x7f95e415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95dbe00640 1 -- 172.21.15.46:0/1628709928 learned_addr learned my addr 172.21.15.46:0/1628709928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e3400640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 0x7f95e4158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95dbe00640 1 -- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 msgr2=0x7f95e4158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95dbe00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 0x7f95e4158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95dbe00640 1 -- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e40711d0 msgr2=0x7f95e415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95dbe00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e40711d0 0x7f95e415a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95dbe00640 1 -- 172.21.15.46:0/1628709928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e415f0c0 con 0x7f95e40b0c30 2024-04-25T08:43:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95dbe00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 0x7f95e413a2e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0b0040 tx=0x7f95cc019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95daa00640 1 -- 172.21.15.46:0/1628709928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0bc070 con 0x7f95e40b0c30 2024-04-25T08:43:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95daa00640 1 -- 172.21.15.46:0/1628709928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95cc0b6670 con 0x7f95e40b0c30 2024-04-25T08:43:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e415a940 con 0x7f95e40b0c30 2024-04-25T08:43:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.087+0000 7f95daa00640 1 -- 172.21.15.46:0/1628709928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc0b2040 con 0x7f95e40b0c30 2024-04-25T08:43:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.091+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e413b190 con 0x7f95e40b0c30 2024-04-25T08:43:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.091+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e4069080 con 0x7f95e40b0c30 2024-04-25T08:43:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.091+0000 7f95daa00640 1 -- 172.21.15.46:0/1628709928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95cc0bf070 con 0x7f95e40b0c30 2024-04-25T08:43:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.091+0000 7f95daa00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95c0066f60 0x7f95c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.091+0000 7f95daa00640 1 -- 172.21.15.46:0/1628709928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f95cc11eb70 con 0x7f95e40b0c30 2024-04-25T08:43:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.091+0000 7f95e2a00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95c0066f60 0x7f95c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.091+0000 7f95e2a00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95c0066f60 0x7f95c0069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95d4007a60 tx=0x7f95d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.095+0000 7f95daa00640 1 -- 172.21.15.46:0/1628709928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95e415f0c0 con 0x7f95e40b0c30 2024-04-25T08:43:12.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f95e415e3c0 con 0x7f95e40b0c30 2024-04-25T08:43:12.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95daa00640 1 -- 172.21.15.46:0/1628709928 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f95cc016150 con 0x7f95e40b0c30 2024-04-25T08:43:12.263 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:43:12.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95c0066f60 msgr2=0x7f95c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95c0066f60 0x7f95c0069420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f95d4007a60 tx=0x7f95d4008040 comp rx=0 tx=0).stop 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 msgr2=0x7f95e413a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 0x7f95e413a2e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0b0040 tx=0x7f95cc019300 comp rx=0 tx=0).stop 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 shutdown_connections 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95c0066f60 0x7f95c0069420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95e415dbd0 0x7f95e4158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95e40b0c30 0x7f95e413a2e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 --2- 172.21.15.46:0/1628709928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e40711d0 0x7f95e415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 >> 172.21.15.46:0/1628709928 conn(0x7f95e406f4f0 msgr2=0x7f95e40b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 shutdown_connections 2024-04-25T08:43:12.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.271+0000 7f95e9a00640 1 -- 172.21.15.46:0/1628709928 wait complete. 2024-04-25T08:43:12.285 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:43:12.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:12.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T08:43:12.316 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:43:12.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:12.317 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:43:12.370 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:43:12.370 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:12.380 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:43:12.380 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:12.429 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:43:12.429 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:12.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4c05400640 1 Processor -- start 2024-04-25T08:43:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4c05400640 1 -- start start 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4c05400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c0009a610 0x7f4c0009aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4c05400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 0x7f4c00099e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4c05400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0015f300 con 0x7f4c0015ef20 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4c05400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c000cd8a0 con 0x7f4c0009a610 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4c05400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c000cddb0 con 0x7f4c00099a00 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bfe800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4c0015ef20 0x7f4c000cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60262/0 (socket says 172.21.15.46:60262) 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bfe800640 1 -- 172.21.15.46:0/3327431300 learned_addr learned my addr 172.21.15.46:0/3327431300 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bf5e00640 1 --2- 172.21.15.46:0/3327431300 >> v2:172.21.15.46:3300/0 conn(0x7f4c0009a610 0x7f4c0009aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bff200640 1 --2- 172.21.15.46:0/3327431300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 0x7f4c00099e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bfde00640 1 -- 172.21.15.46:0/3327431300 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41070559 0 0) 0x7f4c0015f300 con 0x7f4c0015ef20 2024-04-25T08:43:12.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bfde00640 1 -- 172.21.15.46:0/3327431300 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be4003530 con 0x7f4c0015ef20 2024-04-25T08:43:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bf5e00640 1 -- 172.21.15.46:0/3327431300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 msgr2=0x7f4c00099e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bf5e00640 1 --2- 172.21.15.46:0/3327431300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 0x7f4c00099e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bf5e00640 1 -- 172.21.15.46:0/3327431300 >> v1:172.21.15.46:6789/0 conn(0x7f4c0015ef20 legacy=0x7f4c000cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.635+0000 7f4bf5e00640 1 -- 172.21.15.46:0/3327431300 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c000cc3d0 con 0x7f4c0009a610 2024-04-25T08:43:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4bf5e00640 1 --2- 172.21.15.46:0/3327431300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 0x7f4c0009aa10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4be80097c0 tx=0x7f4be801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528ed871760e0097 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4bfde00640 1 -- 172.21.15.46:0/3327431300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be80b0070 con 0x7f4c0009a610 2024-04-25T08:43:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4bfde00640 1 -- 172.21.15.46:0/3327431300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be80ab600 con 0x7f4c0009a610 2024-04-25T08:43:12.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4bfde00640 1 -- 172.21.15.46:0/3327431300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be80ab9d0 con 0x7f4c0009a610 2024-04-25T08:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4c05400640 1 -- 172.21.15.46:0/3327431300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 msgr2=0x7f4c0009aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4c05400640 1 --2- 172.21.15.46:0/3327431300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 0x7f4c0009aa10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4be80097c0 tx=0x7f4be801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4c05400640 1 -- 172.21.15.46:0/3327431300 shutdown_connections 2024-04-25T08:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4c05400640 1 --2- 172.21.15.46:0/3327431300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 0x7f4c00099e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4c05400640 1 --2- 172.21.15.46:0/3327431300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 0x7f4c0009aa10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4c05400640 1 -- 172.21.15.46:0/3327431300 >> 172.21.15.46:0/3327431300 conn(0x7f4c000b3120 msgr2=0x7f4c000b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4c05400640 1 -- 172.21.15.46:0/3327431300 shutdown_connections 2024-04-25T08:43:12.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.639+0000 7f4c05400640 1 -- 172.21.15.46:0/3327431300 wait complete. 2024-04-25T08:43:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 Processor -- start 2024-04-25T08:43:12.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 -- start start 2024-04-25T08:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 0x7f4c0014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 0x7f4c0015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c0015ef20 0x7f4c0015dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c000cc190 con 0x7f4c0009a610 2024-04-25T08:43:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c000cc010 con 0x7f4c0015ef20 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c00168880 con 0x7f4c00099a00 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 0x7f4c0015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 0x7f4c0015d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51624/0 (socket says 172.21.15.46:51624) 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf5e00640 1 -- 172.21.15.46:0/3068009193 learned_addr learned my addr 172.21.15.46:0/3068009193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bff200640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c0015ef20 0x7f4c0015dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bfe800640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 0x7f4c0014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf5e00640 1 -- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 msgr2=0x7f4c0014e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf5e00640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 0x7f4c0014e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf5e00640 1 -- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c0015ef20 msgr2=0x7f4c0015dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf5e00640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c0015ef20 0x7f4c0015dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf5e00640 1 -- 172.21.15.46:0/3068009193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c001565a0 con 0x7f4c0009a610 2024-04-25T08:43:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf5e00640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 0x7f4c0015d570 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4be80ae040 tx=0x7f4be8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf7400640 1 -- 172.21.15.46:0/3068009193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be80b0040 con 0x7f4c0009a610 2024-04-25T08:43:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf7400640 1 -- 172.21.15.46:0/3068009193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be80b43f0 con 0x7f4c0009a610 2024-04-25T08:43:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4bf7400640 1 -- 172.21.15.46:0/3068009193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be80b4800 con 0x7f4c0009a610 2024-04-25T08:43:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c00159ce0 con 0x7f4c0009a610 2024-04-25T08:43:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0015a2a0 con 0x7f4c0009a610 2024-04-25T08:43:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.643+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c00069080 con 0x7f4c0009a610 2024-04-25T08:43:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.647+0000 7f4bf7400640 1 -- 172.21.15.46:0/3068009193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4be80b49a0 con 0x7f4c0009a610 2024-04-25T08:43:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.647+0000 7f4bf7400640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bdc066f60 0x7f4bdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.647+0000 7f4bf7400640 1 -- 172.21.15.46:0/3068009193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4be811eef0 con 0x7f4c0009a610 2024-04-25T08:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.647+0000 7f4bfe800640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bdc066f60 0x7f4bdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.647+0000 7f4bfe800640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bdc066f60 0x7f4bdc069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0005890 tx=0x7f4bf0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:12.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.651+0000 7f4bf7400640 1 -- 172.21.15.46:0/3068009193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4be8016090 con 0x7f4c0009a610 2024-04-25T08:43:12.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.815+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c0015a9d0 con 0x7f4c0009a610 2024-04-25T08:43:12.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.815+0000 7f4bf7400640 1 -- 172.21.15.46:0/3068009193 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7f4be80efb90 con 0x7f4c0009a610 2024-04-25T08:43:12.809 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:12.809 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bdc066f60 msgr2=0x7f4bdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bdc066f60 0x7f4bdc069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0005890 tx=0x7f4bf0009040 comp rx=0 tx=0).stop 2024-04-25T08:43:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 msgr2=0x7f4c0015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 0x7f4c0015d570 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4be80ae040 tx=0x7f4be8019240 comp rx=0 tx=0).stop 2024-04-25T08:43:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 shutdown_connections 2024-04-25T08:43:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bdc066f60 0x7f4bdc069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c0015ef20 0x7f4c0015dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c0009a610 0x7f4c0015d570 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 --2- 172.21.15.46:0/3068009193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4c00099a00 0x7f4c0014e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 >> 172.21.15.46:0/3068009193 conn(0x7f4c000b3120 msgr2=0x7f4c0006fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 shutdown_connections 2024-04-25T08:43:12.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.819+0000 7f4c05400640 1 -- 172.21.15.46:0/3068009193 wait complete. 2024-04-25T08:43:12.831 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:12.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:12.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.979+0000 7fd09b800640 1 Processor -- start 2024-04-25T08:43:12.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd09b800640 1 -- start start 2024-04-25T08:43:12.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd09b800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd094145310 0x7fd09406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd09b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd09406dc70 0x7fd0940cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd09b800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940cd930 con 0x7fd0940cf1b0 2024-04-25T08:43:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd09b800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0940cdbd0 con 0x7fd094145310 2024-04-25T08:43:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd09b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940cdeb0 con 0x7fd09406dc70 2024-04-25T08:43:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd098e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd094145310 0x7fd09406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd098e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd094145310 0x7fd09406e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51630/0 (socket says 172.21.15.46:51630) 2024-04-25T08:43:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd098e00640 1 -- 172.21.15.46:0/1776105047 learned_addr learned my addr 172.21.15.46:0/1776105047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:12.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd08be00640 1 --2- 172.21.15.46:0/1776105047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd09406dc70 0x7fd0940cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd098e00640 1 -- 172.21.15.46:0/1776105047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd09406dc70 msgr2=0x7fd0940cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:12.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd098e00640 1 --2- 172.21.15.46:0/1776105047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd09406dc70 0x7fd0940cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd098e00640 1 -- 172.21.15.46:0/1776105047 >> v1:172.21.15.82:6789/0 conn(0x7fd0940cf1b0 legacy=0x7fd09415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.983+0000 7fd098e00640 1 -- 172.21.15.46:0/1776105047 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940ce190 con 0x7fd094145310 2024-04-25T08:43:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd098e00640 1 --2- 172.21.15.46:0/1776105047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd094145310 0x7fd09406e0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd084009870 tx=0x7fd08401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81767a8f0cf8eaad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd093e00640 1 -- 172.21.15.46:0/1776105047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840a95f0 con 0x7fd094145310 2024-04-25T08:43:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd093e00640 1 -- 172.21.15.46:0/1776105047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0840a9790 con 0x7fd094145310 2024-04-25T08:43:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd093e00640 1 -- 172.21.15.46:0/1776105047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840a9b60 con 0x7fd094145310 2024-04-25T08:43:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 -- 172.21.15.46:0/1776105047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd094145310 msgr2=0x7fd09406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 --2- 172.21.15.46:0/1776105047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd094145310 0x7fd09406e0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd084009870 tx=0x7fd08401c530 comp rx=0 tx=0).stop 2024-04-25T08:43:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 -- 172.21.15.46:0/1776105047 shutdown_connections 2024-04-25T08:43:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 --2- 172.21.15.46:0/1776105047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd09406dc70 0x7fd0940cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 --2- 172.21.15.46:0/1776105047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd094145310 0x7fd09406e0d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 -- 172.21.15.46:0/1776105047 >> 172.21.15.46:0/1776105047 conn(0x7fd09406ac10 msgr2=0x7fd09406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 -- 172.21.15.46:0/1776105047 shutdown_connections 2024-04-25T08:43:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 -- 172.21.15.46:0/1776105047 wait complete. 2024-04-25T08:43:12.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 Processor -- start 2024-04-25T08:43:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.987+0000 7fd09b800640 1 -- start start 2024-04-25T08:43:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd09b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0940cf1b0 0x7fd0941289e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd09b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd094145310 0x7fd0940adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd09b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd09412b710 0x7fd09412acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd09b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09415d920 con 0x7fd09412b710 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd09b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09415d7a0 con 0x7fd0940cf1b0 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd09b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd094168510 con 0x7fd094145310 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd099800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd09412b710 0x7fd09412acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd099800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd09412b710 0x7fd09412acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51636/0 (socket says 172.21.15.46:51636) 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd099800640 1 -- 172.21.15.46:0/2198687538 learned_addr learned my addr 172.21.15.46:0/2198687538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd098e00640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0940cf1b0 0x7fd0941289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd08be00640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd094145310 0x7fd0940adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd099800640 1 -- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd094145310 msgr2=0x7fd0940adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd099800640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd094145310 0x7fd0940adce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd099800640 1 -- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0940cf1b0 msgr2=0x7fd0941289e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd099800640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0940cf1b0 0x7fd0941289e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd099800640 1 -- 172.21.15.46:0/2198687538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940ae5f0 con 0x7fd09412b710 2024-04-25T08:43:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd099800640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd09412b710 0x7fd09412acd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00c7a0 tx=0x7fd08c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd091600640 1 -- 172.21.15.46:0/2198687538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c003b80 con 0x7fd09412b710 2024-04-25T08:43:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd091600640 1 -- 172.21.15.46:0/2198687538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c003d20 con 0x7fd09412b710 2024-04-25T08:43:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd091600640 1 -- 172.21.15.46:0/2198687538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00aa30 con 0x7fd09412b710 2024-04-25T08:43:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0940b25f0 con 0x7fd09412b710 2024-04-25T08:43:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0940ae220 con 0x7fd09412b710 2024-04-25T08:43:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.991+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094069010 con 0x7fd09412b710 2024-04-25T08:43:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.995+0000 7fd091600640 1 -- 172.21.15.46:0/2198687538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd08c00abd0 con 0x7fd09412b710 2024-04-25T08:43:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.995+0000 7fd091600640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd070066f60 0x7fd070069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.995+0000 7fd091600640 1 -- 172.21.15.46:0/2198687538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd08c085300 con 0x7fd09412b710 2024-04-25T08:43:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.995+0000 7fd098e00640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd070066f60 0x7fd070069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.995+0000 7fd098e00640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd070066f60 0x7fd070069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd0840ae000 tx=0x7fd0840b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:12.999+0000 7fd091600640 1 -- 172.21.15.46:0/2198687538 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd08c055820 con 0x7fd09412b710 2024-04-25T08:43:13.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.159+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd09413c800 con 0x7fd09412b710 2024-04-25T08:43:13.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.159+0000 7fd091600640 1 -- 172.21.15.46:0/2198687538 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd08c0850c0 con 0x7fd09412b710 2024-04-25T08:43:13.153 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:13.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd070066f60 msgr2=0x7fd070069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd070066f60 0x7fd070069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd0840ae000 tx=0x7fd0840b3040 comp rx=0 tx=0).stop 2024-04-25T08:43:13.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd09412b710 msgr2=0x7fd09412acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd09412b710 0x7fd09412acd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00c7a0 tx=0x7fd08c00cc60 comp rx=0 tx=0).stop 2024-04-25T08:43:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 shutdown_connections 2024-04-25T08:43:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd070066f60 0x7fd070069420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd09412b710 0x7fd09412acd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd094145310 0x7fd0940adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 --2- 172.21.15.46:0/2198687538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0940cf1b0 0x7fd0941289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 >> 172.21.15.46:0/2198687538 conn(0x7fd09406ac10 msgr2=0x7fd0941442c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 shutdown_connections 2024-04-25T08:43:13.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.163+0000 7fd09b800640 1 -- 172.21.15.46:0/2198687538 wait complete. 2024-04-25T08:43:13.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:13.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.323+0000 7f2915c00640 1 Processor -- start 2024-04-25T08:43:13.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.323+0000 7f2915c00640 1 -- start start 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2915c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2910145310 0x7f291006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2915c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f291006dc70 0x7f29100cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2915c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100cf1b0 0x7f291015d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2915c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29100cd930 con 0x7f291006dc70 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2915c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29100cdbd0 con 0x7f2910145310 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2915c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29100cdeb0 con 0x7f29100cf1b0 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2906000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f291006dc70 0x7f29100cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2906000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f291006dc70 0x7f29100cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51652/0 (socket says 172.21.15.46:51652) 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2906000640 1 -- 172.21.15.46:0/2983426856 learned_addr learned my addr 172.21.15.46:0/2983426856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f290f400640 1 --2- 172.21.15.46:0/2983426856 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100cf1b0 0x7f291015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f290ea00640 1 --2- 172.21.15.46:0/2983426856 >> v2:172.21.15.82:3300/0 conn(0x7f2910145310 0x7f291006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2906000640 1 -- 172.21.15.46:0/2983426856 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100cf1b0 msgr2=0x7f291015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2906000640 1 --2- 172.21.15.46:0/2983426856 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100cf1b0 0x7f291015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2906000640 1 -- 172.21.15.46:0/2983426856 >> v2:172.21.15.82:3300/0 conn(0x7f2910145310 msgr2=0x7f291006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2906000640 1 --2- 172.21.15.46:0/2983426856 >> v2:172.21.15.82:3300/0 conn(0x7f2910145310 0x7f291006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2906000640 1 -- 172.21.15.46:0/2983426856 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29100ce190 con 0x7f291006dc70 2024-04-25T08:43:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.327+0000 7f2906000640 1 --2- 172.21.15.46:0/2983426856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291006dc70 0x7f29100cec70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2910072530 tx=0x7f28f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed43f643dcadc12b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f290e000640 1 -- 172.21.15.46:0/2983426856 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f80b1070 con 0x7f291006dc70 2024-04-25T08:43:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f290e000640 1 -- 172.21.15.46:0/2983426856 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f80ac620 con 0x7f291006dc70 2024-04-25T08:43:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f290e000640 1 -- 172.21.15.46:0/2983426856 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f80ac9f0 con 0x7f291006dc70 2024-04-25T08:43:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 -- 172.21.15.46:0/2983426856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291006dc70 msgr2=0x7f29100cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 --2- 172.21.15.46:0/2983426856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291006dc70 0x7f29100cec70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2910072530 tx=0x7f28f801c4f0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 -- 172.21.15.46:0/2983426856 shutdown_connections 2024-04-25T08:43:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 --2- 172.21.15.46:0/2983426856 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100cf1b0 0x7f291015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 --2- 172.21.15.46:0/2983426856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f291006dc70 0x7f29100cec70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 --2- 172.21.15.46:0/2983426856 >> v2:172.21.15.82:3300/0 conn(0x7f2910145310 0x7f291006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 -- 172.21.15.46:0/2983426856 >> 172.21.15.46:0/2983426856 conn(0x7f291006ac10 msgr2=0x7f291006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 -- 172.21.15.46:0/2983426856 shutdown_connections 2024-04-25T08:43:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 -- 172.21.15.46:0/2983426856 wait complete. 2024-04-25T08:43:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 Processor -- start 2024-04-25T08:43:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 -- start start 2024-04-25T08:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29100cf1b0 0x7f291013efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910145310 0x7f2910135a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100b3d50 0x7f2910155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29100713f0 con 0x7f29100cf1b0 2024-04-25T08:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2910071270 con 0x7f2910145310 2024-04-25T08:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2915c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2910071570 con 0x7f29100b3d50 2024-04-25T08:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f290ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29100cf1b0 0x7f291013efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f290ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29100cf1b0 0x7f291013efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51658/0 (socket says 172.21.15.46:51658) 2024-04-25T08:43:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f290ea00640 1 -- 172.21.15.46:0/3234924355 learned_addr learned my addr 172.21.15.46:0/3234924355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f2906000640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910145310 0x7f2910135a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.331+0000 7f290f400640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100b3d50 0x7f2910155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f290ea00640 1 -- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100b3d50 msgr2=0x7f2910155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f290ea00640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100b3d50 0x7f2910155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f290ea00640 1 -- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910145310 msgr2=0x7f2910135a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f290ea00640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910145310 0x7f2910135a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f290ea00640 1 -- 172.21.15.46:0/3234924355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291013f6c0 con 0x7f29100cf1b0 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2906000640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910145310 0x7f2910135a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f290ea00640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29100cf1b0 0x7f291013efc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f290000d5f0 tx=0x7f290000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2907400640 1 -- 172.21.15.46:0/3234924355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000125f0 con 0x7f29100cf1b0 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2907400640 1 -- 172.21.15.46:0/3234924355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2900019070 con 0x7f29100cf1b0 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291013f9a0 con 0x7f29100cf1b0 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2907400640 1 -- 172.21.15.46:0/3234924355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000129b0 con 0x7f29100cf1b0 2024-04-25T08:43:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2910156d10 con 0x7f29100cf1b0 2024-04-25T08:43:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2910069010 con 0x7f29100cf1b0 2024-04-25T08:43:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2907400640 1 -- 172.21.15.46:0/3234924355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2900012b50 con 0x7f29100cf1b0 2024-04-25T08:43:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2907400640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28ec066f60 0x7f28ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2907400640 1 -- 172.21.15.46:0/3234924355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2900085870 con 0x7f29100cf1b0 2024-04-25T08:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.335+0000 7f2906000640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28ec066f60 0x7f28ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.339+0000 7f2906000640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28ec066f60 0x7f28ec069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f28f80ae000 tx=0x7f28f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.339+0000 7f2907400640 1 -- 172.21.15.46:0/3234924355 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29000563e0 con 0x7f29100cf1b0 2024-04-25T08:43:13.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.503+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29100736e0 con 0x7f29100cf1b0 2024-04-25T08:43:13.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.503+0000 7f2907400640 1 -- 172.21.15.46:0/3234924355 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2900055ce0 con 0x7f29100cf1b0 2024-04-25T08:43:13.497 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28ec066f60 msgr2=0x7f28ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28ec066f60 0x7f28ec069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f28f80ae000 tx=0x7f28f8008040 comp rx=0 tx=0).stop 2024-04-25T08:43:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29100cf1b0 msgr2=0x7f291013efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29100cf1b0 0x7f291013efc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f290000d5f0 tx=0x7f290000dab0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 shutdown_connections 2024-04-25T08:43:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28ec066f60 0x7f28ec069420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29100b3d50 0x7f2910155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2910145310 0x7f2910135a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 --2- 172.21.15.46:0/3234924355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29100cf1b0 0x7f291013efc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 >> 172.21.15.46:0/3234924355 conn(0x7f291006ac10 msgr2=0x7f29100b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 shutdown_connections 2024-04-25T08:43:13.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.507+0000 7f2915c00640 1 -- 172.21.15.46:0/3234924355 wait complete. 2024-04-25T08:43:13.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:13.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f316b800640 1 Processor -- start 2024-04-25T08:43:13.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f316b800640 1 -- start start 2024-04-25T08:43:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f316b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f316406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f316b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f316406dc70 0x7f31640cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f316b800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31640cd930 con 0x7f31640cf1b0 2024-04-25T08:43:13.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f316b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31640cdbd0 con 0x7f316406dc70 2024-04-25T08:43:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f316b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31640cdeb0 con 0x7f3164145310 2024-04-25T08:43:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f3168e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f316406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f3163e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f316406dc70 0x7f31640cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f3168e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f316406e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:34316/0 (socket says 172.21.15.46:34316) 2024-04-25T08:43:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f3168e00640 1 -- 172.21.15.46:0/173930662 learned_addr learned my addr 172.21.15.46:0/173930662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f3168e00640 1 -- 172.21.15.46:0/173930662 >> v2:172.21.15.82:3300/0 conn(0x7f316406dc70 msgr2=0x7f31640cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f3168e00640 1 --2- 172.21.15.46:0/173930662 >> v2:172.21.15.82:3300/0 conn(0x7f316406dc70 0x7f31640cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f3168e00640 1 -- 172.21.15.46:0/173930662 >> v1:172.21.15.82:6789/0 conn(0x7f31640cf1b0 legacy=0x7f316415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f3168e00640 1 -- 172.21.15.46:0/173930662 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31640ce190 con 0x7f3164145310 2024-04-25T08:43:13.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.671+0000 7f3163e00640 1 --2- 172.21.15.46:0/173930662 >> v2:172.21.15.82:3300/0 conn(0x7f316406dc70 0x7f31640cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f3168e00640 1 --2- 172.21.15.46:0/173930662 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f316406e0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3158009870 tx=0x7f315801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3babf8fe406d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f3163400640 1 -- 172.21.15.46:0/173930662 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31580a95f0 con 0x7f3164145310 2024-04-25T08:43:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f3163400640 1 -- 172.21.15.46:0/173930662 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31580a9790 con 0x7f3164145310 2024-04-25T08:43:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f3163400640 1 -- 172.21.15.46:0/173930662 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31580a9b60 con 0x7f3164145310 2024-04-25T08:43:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f316b800640 1 -- 172.21.15.46:0/173930662 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 msgr2=0x7f316406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f316b800640 1 --2- 172.21.15.46:0/173930662 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f316406e0d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3158009870 tx=0x7f315801c530 comp rx=0 tx=0).stop 2024-04-25T08:43:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f316b800640 1 -- 172.21.15.46:0/173930662 shutdown_connections 2024-04-25T08:43:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f316b800640 1 --2- 172.21.15.46:0/173930662 >> v2:172.21.15.82:3300/0 conn(0x7f316406dc70 0x7f31640cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f316b800640 1 --2- 172.21.15.46:0/173930662 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f316406e0d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f316b800640 1 -- 172.21.15.46:0/173930662 >> 172.21.15.46:0/173930662 conn(0x7f316406ac10 msgr2=0x7f316406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f316b800640 1 -- 172.21.15.46:0/173930662 shutdown_connections 2024-04-25T08:43:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.675+0000 7f316b800640 1 -- 172.21.15.46:0/173930662 wait complete. 2024-04-25T08:43:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 Processor -- start 2024-04-25T08:43:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 -- start start 2024-04-25T08:43:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31640cf1b0 0x7f31640b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f31641590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31640b4020 0x7f31640b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316415d920 con 0x7f31640b4020 2024-04-25T08:43:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316415d7a0 con 0x7f31640cf1b0 2024-04-25T08:43:13.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3164168510 con 0x7f3164145310 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3169800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31640b4020 0x7f31640b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3169800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31640b4020 0x7f31640b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51664/0 (socket says 172.21.15.46:51664) 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3169800640 1 -- 172.21.15.46:0/1359978757 learned_addr learned my addr 172.21.15.46:0/1359978757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3163e00640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f31641590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3168e00640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31640cf1b0 0x7f31640b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3169800640 1 -- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 msgr2=0x7f31641590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3169800640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f31641590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3169800640 1 -- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31640cf1b0 msgr2=0x7f31640b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3169800640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31640cf1b0 0x7f31640b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3169800640 1 -- 172.21.15.46:0/1359978757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164159600 con 0x7f31640b4020 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3169800640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31640b4020 0x7f31640b44b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f315c00c7a0 tx=0x7f315c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3160c00640 1 -- 172.21.15.46:0/1359978757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c003b80 con 0x7f31640b4020 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3160c00640 1 -- 172.21.15.46:0/1359978757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c003d20 con 0x7f31640b4020 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f3160c00640 1 -- 172.21.15.46:0/1359978757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c00aa30 con 0x7f31640b4020 2024-04-25T08:43:13.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31641339e0 con 0x7f31640b4020 2024-04-25T08:43:13.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.679+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3164133fa0 con 0x7f31640b4020 2024-04-25T08:43:13.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.683+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3164069010 con 0x7f31640b4020 2024-04-25T08:43:13.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.683+0000 7f3160c00640 1 -- 172.21.15.46:0/1359978757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f315c00abd0 con 0x7f31640b4020 2024-04-25T08:43:13.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.683+0000 7f3160c00640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3144066f60 0x7f3144069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.683+0000 7f3160c00640 1 -- 172.21.15.46:0/1359978757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f315c085200 con 0x7f31640b4020 2024-04-25T08:43:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.683+0000 7f3168e00640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3144066f60 0x7f3144069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:13.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.683+0000 7f3168e00640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3144066f60 0x7f3144069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f31580ae000 tx=0x7f31580b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:13.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.687+0000 7f3160c00640 1 -- 172.21.15.46:0/1359978757 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f315c0557b0 con 0x7f31640b4020 2024-04-25T08:43:13.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.851+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f316413c770 con 0x7f31640b4020 2024-04-25T08:43:13.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f3160c00640 1 -- 172.21.15.46:0/1359978757 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f315c059850 con 0x7f31640b4020 2024-04-25T08:43:13.846 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3144066f60 msgr2=0x7f3144069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3144066f60 0x7f3144069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f31580ae000 tx=0x7f31580b3040 comp rx=0 tx=0).stop 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31640b4020 msgr2=0x7f31640b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31640b4020 0x7f31640b44b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f315c00c7a0 tx=0x7f315c00cc60 comp rx=0 tx=0).stop 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 shutdown_connections 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3144066f60 0x7f3144069420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31640b4020 0x7f31640b44b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3164145310 0x7f31641590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 --2- 172.21.15.46:0/1359978757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31640cf1b0 0x7f31640b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 >> 172.21.15.46:0/1359978757 conn(0x7f316406ac10 msgr2=0x7f3164141780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 shutdown_connections 2024-04-25T08:43:13.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:13.855+0000 7f316b800640 1 -- 172.21.15.46:0/1359978757 wait complete. 2024-04-25T08:43:13.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:14.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c8d200640 1 Processor -- start 2024-04-25T08:43:14.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c8d200640 1 -- start start 2024-04-25T08:43:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c8d200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0c88145310 0x7f0c8806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c8d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c8806dc70 0x7f0c880cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c8d200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c880cd930 con 0x7f0c880cf1b0 2024-04-25T08:43:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c8d200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880cdbd0 con 0x7f0c8806dc70 2024-04-25T08:43:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c8d200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c880cdeb0 con 0x7f0c88145310 2024-04-25T08:43:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c86800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0c880cf1b0 0x7f0c8815d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60268/0 (socket says 172.21.15.46:60268) 2024-04-25T08:43:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c86800640 1 -- 172.21.15.46:0/2705088431 learned_addr learned my addr 172.21.15.46:0/2705088431 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c7d400640 1 --2- 172.21.15.46:0/2705088431 >> v2:172.21.15.46:3300/0 conn(0x7f0c8806dc70 0x7f0c880cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.019+0000 7f0c85e00640 1 --2- 172.21.15.46:0/2705088431 >> v2:172.21.15.82:3300/0 conn(0x7f0c88145310 0x7f0c8806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85400640 1 -- 172.21.15.46:0/2705088431 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587712864 0 0) 0x7f0c880cd930 con 0x7f0c880cf1b0 2024-04-25T08:43:14.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85400640 1 -- 172.21.15.46:0/2705088431 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c6c003530 con 0x7f0c880cf1b0 2024-04-25T08:43:14.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85e00640 1 -- 172.21.15.46:0/2705088431 >> v2:172.21.15.46:3300/0 conn(0x7f0c8806dc70 msgr2=0x7f0c880cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:14.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85e00640 1 --2- 172.21.15.46:0/2705088431 >> v2:172.21.15.46:3300/0 conn(0x7f0c8806dc70 0x7f0c880cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85e00640 1 -- 172.21.15.46:0/2705088431 >> v1:172.21.15.46:6789/0 conn(0x7f0c880cf1b0 legacy=0x7f0c8815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85e00640 1 -- 172.21.15.46:0/2705088431 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c880ce190 con 0x7f0c88145310 2024-04-25T08:43:14.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c7d400640 1 --2- 172.21.15.46:0/2705088431 >> v2:172.21.15.46:3300/0 conn(0x7f0c8806dc70 0x7f0c880cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:14.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85e00640 1 --2- 172.21.15.46:0/2705088431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88145310 0x7f0c8806e0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c88072530 tx=0x7f0c7801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42e9e7a38926eaff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:14.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85400640 1 -- 172.21.15.46:0/2705088431 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c780ab650 con 0x7f0c88145310 2024-04-25T08:43:14.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85400640 1 -- 172.21.15.46:0/2705088431 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c780b0070 con 0x7f0c88145310 2024-04-25T08:43:14.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.023+0000 7f0c85400640 1 -- 172.21.15.46:0/2705088431 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c780ab9f0 con 0x7f0c88145310 2024-04-25T08:43:14.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 -- 172.21.15.46:0/2705088431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88145310 msgr2=0x7f0c8806e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2705088431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88145310 0x7f0c8806e0d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0c88072530 tx=0x7f0c7801c450 comp rx=0 tx=0).stop 2024-04-25T08:43:14.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 -- 172.21.15.46:0/2705088431 shutdown_connections 2024-04-25T08:43:14.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2705088431 >> v2:172.21.15.46:3300/0 conn(0x7f0c8806dc70 0x7f0c880cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2705088431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88145310 0x7f0c8806e0d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 -- 172.21.15.46:0/2705088431 >> 172.21.15.46:0/2705088431 conn(0x7f0c8806ac10 msgr2=0x7f0c8806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:14.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 -- 172.21.15.46:0/2705088431 shutdown_connections 2024-04-25T08:43:14.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 -- 172.21.15.46:0/2705088431 wait complete. 2024-04-25T08:43:14.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 Processor -- start 2024-04-25T08:43:14.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 -- start start 2024-04-25T08:43:14.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c880cf1b0 0x7f0c8813f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88145310 0x7f0c88158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88140ac0 0x7f0c8813be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8815d920 con 0x7f0c88145310 2024-04-25T08:43:14.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8815d7a0 con 0x7f0c88140ac0 2024-04-25T08:43:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c8d200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c88168510 con 0x7f0c880cf1b0 2024-04-25T08:43:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c7d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88145310 0x7f0c88158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c7d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88145310 0x7f0c88158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51678/0 (socket says 172.21.15.46:51678) 2024-04-25T08:43:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c7d400640 1 -- 172.21.15.46:0/2014726169 learned_addr learned my addr 172.21.15.46:0/2014726169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c86800640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88140ac0 0x7f0c8813be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c85e00640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c880cf1b0 0x7f0c8813f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c7d400640 1 -- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c880cf1b0 msgr2=0x7f0c8813f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c7d400640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c880cf1b0 0x7f0c8813f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c7d400640 1 -- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88140ac0 msgr2=0x7f0c8813be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c7d400640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88140ac0 0x7f0c8813be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c7d400640 1 -- 172.21.15.46:0/2014726169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8813e100 con 0x7f0c88145310 2024-04-25T08:43:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c86800640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88140ac0 0x7f0c8813be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c85e00640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c880cf1b0 0x7f0c8813f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.027+0000 7f0c7d400640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88145310 0x7f0c88158620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000ebe0 tx=0x7f0c7000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c7ea00640 1 -- 172.21.15.46:0/2014726169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70017070 con 0x7f0c88145310 2024-04-25T08:43:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c7ea00640 1 -- 172.21.15.46:0/2014726169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7000cc30 con 0x7f0c88145310 2024-04-25T08:43:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c7ea00640 1 -- 172.21.15.46:0/2014726169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70013470 con 0x7f0c88145310 2024-04-25T08:43:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8813e3e0 con 0x7f0c88145310 2024-04-25T08:43:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8813e9a0 con 0x7f0c88145310 2024-04-25T08:43:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c88069010 con 0x7f0c88145310 2024-04-25T08:43:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c7ea00640 1 -- 172.21.15.46:0/2014726169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c7001a070 con 0x7f0c88145310 2024-04-25T08:43:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c7ea00640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0c64066f60 0x7f0c64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c7ea00640 1 -- 172.21.15.46:0/2014726169 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0c700857d0 con 0x7f0c88145310 2024-04-25T08:43:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.031+0000 7f0c85e00640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0c64066f60 0x7f0c64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.035+0000 7f0c85e00640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0c64066f60 0x7f0c64069420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c780af000 tx=0x7f0c780a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.035+0000 7f0c7ea00640 1 -- 172.21.15.46:0/2014726169 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c70056340 con 0x7f0c88145310 2024-04-25T08:43:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0c8813f6e0 con 0x7f0c88145310 2024-04-25T08:43:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c7ea00640 1 -- 172.21.15.46:0/2014726169 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0c70055c40 con 0x7f0c88145310 2024-04-25T08:43:14.192 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0c64066f60 msgr2=0x7f0c64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0c64066f60 0x7f0c64069420 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c780af000 tx=0x7f0c780a7000 comp rx=0 tx=0).stop 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88145310 msgr2=0x7f0c88158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88145310 0x7f0c88158620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000ebe0 tx=0x7f0c7000c520 comp rx=0 tx=0).stop 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 shutdown_connections 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0c64066f60 0x7f0c64069420 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c88140ac0 0x7f0c8813be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c88145310 0x7f0c88158620 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 --2- 172.21.15.46:0/2014726169 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0c880cf1b0 0x7f0c8813f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 >> 172.21.15.46:0/2014726169 conn(0x7f0c8806ac10 msgr2=0x7f0c88144230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 shutdown_connections 2024-04-25T08:43:14.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.199+0000 7f0c8d200640 1 -- 172.21.15.46:0/2014726169 wait complete. 2024-04-25T08:43:14.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:43:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f8305e00640 1 Processor -- start 2024-04-25T08:43:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f8305e00640 1 -- start start 2024-04-25T08:43:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f8305e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f830015a750 0x7f830006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f8305e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83000cc9e0 0x7f830006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f8305e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830006d5e0 con 0x7f83000aed10 2024-04-25T08:43:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f8305e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830015ab30 con 0x7f830015a750 2024-04-25T08:43:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f8305e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000ce4f0 con 0x7f83000cc9e0 2024-04-25T08:43:14.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82f6000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f83000aed10 0x7f83000b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60284/0 (socket says 172.21.15.46:60284) 2024-04-25T08:43:14.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82f6000640 1 -- 172.21.15.46:0/485907390 learned_addr learned my addr 172.21.15.46:0/485907390 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887317882 0 0) 0x7f830006d5e0 con 0x7f83000aed10 2024-04-25T08:43:14.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fea00640 1 --2- 172.21.15.46:0/485907390 >> v2:172.21.15.82:3300/0 conn(0x7f830015a750 0x7f830006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e4003530 con 0x7f83000aed10 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82ff400640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83000cc9e0 0x7f830006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1453479761 0 0) 0x7f82e4003530 con 0x7f83000aed10 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83000cc9e0 msgr2=0x7f830006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83000cc9e0 0x7f830006f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 >> v2:172.21.15.82:3300/0 conn(0x7f830015a750 msgr2=0x7f830006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 --2- 172.21.15.46:0/485907390 >> v2:172.21.15.82:3300/0 conn(0x7f830015a750 0x7f830006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000ce720 con 0x7f83000aed10 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f82e8001680 con 0x7f83000aed10 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 >> v1:172.21.15.46:6789/0 conn(0x7f83000aed10 legacy=0x7f83000b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83000cc9e0 0x7f82e40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e4003f90 0x7f82e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82e4004d50 0x7f82e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e400b9d0 con 0x7f82e4003f90 2024-04-25T08:43:14.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e400bb50 con 0x7f82e4004d50 2024-04-25T08:43:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e400bcd0 con 0x7f83000cc9e0 2024-04-25T08:43:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82f6000640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e4003f90 0x7f82e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82fea00640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83000cc9e0 0x7f82e40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82ff400640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82e4004d50 0x7f82e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82f6000640 1 -- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83000cc9e0 msgr2=0x7f82e40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.363+0000 7f82f6000640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83000cc9e0 0x7f82e40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f82f6000640 1 -- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82e4004d50 msgr2=0x7f82e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f82f6000640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f82e4004d50 0x7f82e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f82f6000640 1 -- 172.21.15.46:0/485907390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e4003530 con 0x7f82e4003f90 2024-04-25T08:43:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f82f6000640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e4003f90 0x7f82e4004810 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f82e8007fc0 tx=0x7f82e8021520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8fef86892fce93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8007c90 con 0x7f82e4003f90 2024-04-25T08:43:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8005ae0 con 0x7f82e4003f90 2024-04-25T08:43:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f82fe000640 1 -- 172.21.15.46:0/485907390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8007070 con 0x7f82e4003f90 2024-04-25T08:43:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f8305e00640 1 -- 172.21.15.46:0/485907390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e4003f90 msgr2=0x7f82e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f8305e00640 1 --2- 172.21.15.46:0/485907390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e4003f90 0x7f82e4004810 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f82e8007fc0 tx=0x7f82e8021520 comp rx=0 tx=0).stop 2024-04-25T08:43:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f82ff400640 1 -- 172.21.15.46:0/485907390 reap_dead start 2024-04-25T08:43:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f8305e00640 1 -- 172.21.15.46:0/485907390 shutdown_connections 2024-04-25T08:43:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f8305e00640 1 -- 172.21.15.46:0/485907390 >> 172.21.15.46:0/485907390 conn(0x7f83000b12a0 msgr2=0x7f83000b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f8305e00640 1 -- 172.21.15.46:0/485907390 shutdown_connections 2024-04-25T08:43:14.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.367+0000 7f8305e00640 1 -- 172.21.15.46:0/485907390 wait complete. 2024-04-25T08:43:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f8305e00640 1 Processor -- start 2024-04-25T08:43:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f8305e00640 1 -- start start 2024-04-25T08:43:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f8305e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82e4003f90 0x7f8300159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f8305e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830014e0d0 0x7f8300157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f8305e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83000ad2e0 0x7f83000ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f8305e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000cece0 con 0x7f83000ad2e0 2024-04-25T08:43:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f8305e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830006d8c0 con 0x7f830014e0d0 2024-04-25T08:43:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f8305e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000cf200 con 0x7f82e4003f90 2024-04-25T08:43:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83000ad2e0 0x7f83000ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82fea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82e4003f90 0x7f8300159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83000ad2e0 0x7f83000ad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51684/0 (socket says 172.21.15.46:51684) 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82ff400640 1 -- 172.21.15.46:0/3005238345 learned_addr learned my addr 172.21.15.46:0/3005238345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82f6000640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830014e0d0 0x7f8300157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82ff400640 1 -- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82e4003f90 msgr2=0x7f8300159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82ff400640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82e4003f90 0x7f8300159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82ff400640 1 -- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830014e0d0 msgr2=0x7f8300157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82ff400640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830014e0d0 0x7f8300157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82ff400640 1 -- 172.21.15.46:0/3005238345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8300158070 con 0x7f83000ad2e0 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82ff400640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83000ad2e0 0x7f83000ad760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8300072260 tx=0x7f82f800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82f7e00640 1 -- 172.21.15.46:0/3005238345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800fd10 con 0x7f83000ad2e0 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82f7e00640 1 -- 172.21.15.46:0/3005238345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f8015430 con 0x7f83000ad2e0 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f82f7e00640 1 -- 172.21.15.46:0/3005238345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8015820 con 0x7f83000ad2e0 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.371+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83000c4e40 con 0x7f83000ad2e0 2024-04-25T08:43:14.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.375+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8300159000 con 0x7f83000ad2e0 2024-04-25T08:43:14.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.375+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8300069080 con 0x7f83000ad2e0 2024-04-25T08:43:14.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.375+0000 7f82f7e00640 1 -- 172.21.15.46:0/3005238345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82f80159c0 con 0x7f83000ad2e0 2024-04-25T08:43:14.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.375+0000 7f82f7e00640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82dc066f60 0x7f82dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.375+0000 7f82f7e00640 1 -- 172.21.15.46:0/3005238345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f82f8085ff0 con 0x7f83000ad2e0 2024-04-25T08:43:14.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.375+0000 7f82fea00640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82dc066f60 0x7f82dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.375+0000 7f82fea00640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82dc066f60 0x7f82dc069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009be0 tx=0x7f82f0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:14.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.379+0000 7f82f7e00640 1 -- 172.21.15.46:0/3005238345 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82f8019030 con 0x7f83000ad2e0 2024-04-25T08:43:14.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.551+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83000af810 con 0x7f83000ad2e0 2024-04-25T08:43:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.551+0000 7f82f7e00640 1 -- 172.21.15.46:0/3005238345 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f82f8010040 con 0x7f83000ad2e0 2024-04-25T08:43:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:43:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82dc066f60 msgr2=0x7f82dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82dc066f60 0x7f82dc069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009be0 tx=0x7f82f0009190 comp rx=0 tx=0).stop 2024-04-25T08:43:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83000ad2e0 msgr2=0x7f83000ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83000ad2e0 0x7f83000ad760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8300072260 tx=0x7f82f800f1b0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 shutdown_connections 2024-04-25T08:43:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f82dc066f60 0x7f82dc069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83000ad2e0 0x7f83000ad760 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f830014e0d0 0x7f8300157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 --2- 172.21.15.46:0/3005238345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82e4003f90 0x7f8300159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 >> 172.21.15.46:0/3005238345 conn(0x7f83000b12a0 msgr2=0x7f83000b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 shutdown_connections 2024-04-25T08:43:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.555+0000 7f8305e00640 1 -- 172.21.15.46:0/3005238345 wait complete. 2024-04-25T08:43:14.568 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T08:43:14.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:14.569 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T08:43:14.608 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:43:14.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:14.608 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:43:14.665 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:14.665 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:14.674 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:14.674 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:14.730 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:14.730 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:14.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:14.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c50e00640 1 Processor -- start 2024-04-25T08:43:14.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c50e00640 1 -- start start 2024-04-25T08:43:14.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c50e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 0x7f5c4c15f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c50e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c4c0cf3a0 0x7f5c4c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c50e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c06b7d0 con 0x7f5c4c0cf8a0 2024-04-25T08:43:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c50e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4c15be90 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c50e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c15c170 con 0x7f5c4c0cc1e0 2024-04-25T08:43:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c4a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c4c0cf3a0 0x7f5c4c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c4a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c4c0cf3a0 0x7f5c4c15f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51694/0 (socket says 172.21.15.46:51694) 2024-04-25T08:43:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c4a800640 1 -- 172.21.15.46:0/1488972658 learned_addr learned my addr 172.21.15.46:0/1488972658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:14.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c43e00640 1 --2- 172.21.15.46:0/1488972658 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 0x7f5c4c15f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c4a800640 1 -- 172.21.15.46:0/1488972658 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 msgr2=0x7f5c4c15f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c4a800640 1 --2- 172.21.15.46:0/1488972658 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 0x7f5c4c15f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c4a800640 1 -- 172.21.15.46:0/1488972658 >> v1:172.21.15.82:6789/0 conn(0x7f5c4c0cf8a0 legacy=0x7f5c4c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c4a800640 1 -- 172.21.15.46:0/1488972658 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c15c450 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.975+0000 7f5c43e00640 1 --2- 172.21.15.46:0/1488972658 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 0x7f5c4c15f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:14.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c4a800640 1 --2- 172.21.15.46:0/1488972658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 0x7f5c4c15f8b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009e50 tx=0x7f5c4401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18abfd0e2435eca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:14.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c49400640 1 -- 172.21.15.46:0/1488972658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440a5770 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c49400640 1 -- 172.21.15.46:0/1488972658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c440a5910 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c49400640 1 -- 172.21.15.46:0/1488972658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440a5ce0 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c50e00640 1 -- 172.21.15.46:0/1488972658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 msgr2=0x7f5c4c15f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c50e00640 1 --2- 172.21.15.46:0/1488972658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 0x7f5c4c15f8b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009e50 tx=0x7f5c4401c9a0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c50e00640 1 -- 172.21.15.46:0/1488972658 shutdown_connections 2024-04-25T08:43:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c50e00640 1 --2- 172.21.15.46:0/1488972658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 0x7f5c4c15f8b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c50e00640 1 --2- 172.21.15.46:0/1488972658 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 0x7f5c4c15f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c50e00640 1 -- 172.21.15.46:0/1488972658 >> 172.21.15.46:0/1488972658 conn(0x7f5c4c099a00 msgr2=0x7f5c4c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c50e00640 1 -- 172.21.15.46:0/1488972658 shutdown_connections 2024-04-25T08:43:14.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.979+0000 7f5c50e00640 1 -- 172.21.15.46:0/1488972658 wait complete. 2024-04-25T08:43:14.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 Processor -- start 2024-04-25T08:43:14.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 -- start start 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 0x7f5c4c0c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 0x7f5c4c0c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c0cf8a0 0x7f5c4c0c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0b2ab0 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0b2930 con 0x7f5c4c0cf8a0 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c168f20 con 0x7f5c4c0cc1e0 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c43e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 0x7f5c4c0c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c43e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 0x7f5c4c0c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51704/0 (socket says 172.21.15.46:51704) 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c43e00640 1 -- 172.21.15.46:0/3026256536 learned_addr learned my addr 172.21.15.46:0/3026256536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c49e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 0x7f5c4c0c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c4a800640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c0cf8a0 0x7f5c4c0c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c43e00640 1 -- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 msgr2=0x7f5c4c0c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c43e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 0x7f5c4c0c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c43e00640 1 -- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c0cf8a0 msgr2=0x7f5c4c0c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c43e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c0cf8a0 0x7f5c4c0c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c43e00640 1 -- 172.21.15.46:0/3026256536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c0c7490 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c43e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 0x7f5c4c0c5900 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e810 tx=0x7f5c3400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c42000640 1 -- 172.21.15.46:0/3026256536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34017070 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c42000640 1 -- 172.21.15.46:0/3026256536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3400c8d0 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c15ed90 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c42000640 1 -- 172.21.15.46:0/3026256536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400f040 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c0c6220 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.983+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c069080 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.987+0000 7f5c42000640 1 -- 172.21.15.46:0/3026256536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c3401a070 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.987+0000 7f5c42000640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c28066f60 0x7f5c28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:14.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.987+0000 7f5c42000640 1 -- 172.21.15.46:0/3026256536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5c34084ff0 con 0x7f5c4c0cf3a0 2024-04-25T08:43:14.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.987+0000 7f5c49e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c28066f60 0x7f5c28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:14.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.987+0000 7f5c49e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c28066f60 0x7f5c28069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c001770 tx=0x7f5c3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:14.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:14.991+0000 7f5c42000640 1 -- 172.21.15.46:0/3026256536 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c34055510 con 0x7f5c4c0cf3a0 2024-04-25T08:43:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.155+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c4c072e40 con 0x7f5c4c0cf3a0 2024-04-25T08:43:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.155+0000 7f5c42000640 1 -- 172.21.15.46:0/3026256536 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7f5c340595b0 con 0x7f5c4c0cf3a0 2024-04-25T08:43:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:15.149 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c28066f60 msgr2=0x7f5c28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c28066f60 0x7f5c28069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c001770 tx=0x7f5c3c009040 comp rx=0 tx=0).stop 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 msgr2=0x7f5c4c0c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 0x7f5c4c0c5900 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400e810 tx=0x7f5c3400ecd0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 shutdown_connections 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c28066f60 0x7f5c28069420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c0cf8a0 0x7f5c4c0c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4c0cf3a0 0x7f5c4c0c5900 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 --2- 172.21.15.46:0/3026256536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c4c0cc1e0 0x7f5c4c0c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 >> 172.21.15.46:0/3026256536 conn(0x7f5c4c099a00 msgr2=0x7f5c4c15a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 shutdown_connections 2024-04-25T08:43:15.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.159+0000 7f5c50e00640 1 -- 172.21.15.46:0/3026256536 wait complete. 2024-04-25T08:43:15.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:15.170 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.319+0000 7fa3ed800640 1 Processor -- start 2024-04-25T08:43:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3ed800640 1 -- start start 2024-04-25T08:43:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3ed800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3e815db80 0x7fa3e80cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3ed800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3e809a110 0x7fa3e806bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3ed800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e80cc420 con 0x7fa3e809a880 2024-04-25T08:43:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3ed800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e80cc6c0 con 0x7fa3e809a110 2024-04-25T08:43:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3ed800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e80cc9a0 con 0x7fa3e815db80 2024-04-25T08:43:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3e7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3e809a110 0x7fa3e806bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3e7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3e809a110 0x7fa3e806bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51720/0 (socket says 172.21.15.46:51720) 2024-04-25T08:43:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3e7400640 1 -- 172.21.15.46:0/1899761169 learned_addr learned my addr 172.21.15.46:0/1899761169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3e6a00640 1 --2- 172.21.15.46:0/1899761169 >> v2:172.21.15.82:3300/0 conn(0x7fa3e815db80 0x7fa3e80cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3e7400640 1 -- 172.21.15.46:0/1899761169 >> v2:172.21.15.82:3300/0 conn(0x7fa3e815db80 msgr2=0x7fa3e80cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3e7400640 1 --2- 172.21.15.46:0/1899761169 >> v2:172.21.15.82:3300/0 conn(0x7fa3e815db80 0x7fa3e80cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3e7400640 1 -- 172.21.15.46:0/1899761169 >> v1:172.21.15.82:6789/0 conn(0x7fa3e809a880 legacy=0x7fa3e8099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:15.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.323+0000 7fa3e7400640 1 -- 172.21.15.46:0/1899761169 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80ccc80 con 0x7fa3e809a110 2024-04-25T08:43:15.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3e7400640 1 --2- 172.21.15.46:0/1899761169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e809a110 0x7fa3e806bd20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8072cb0 tx=0x7fa3e001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60265f1b6dfca60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:15.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3e6000640 1 -- 172.21.15.46:0/1899761169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00aa430 con 0x7fa3e809a110 2024-04-25T08:43:15.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3e6000640 1 -- 172.21.15.46:0/1899761169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e00af070 con 0x7fa3e809a110 2024-04-25T08:43:15.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3e6000640 1 -- 172.21.15.46:0/1899761169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00aa7f0 con 0x7fa3e809a110 2024-04-25T08:43:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 -- 172.21.15.46:0/1899761169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e809a110 msgr2=0x7fa3e806bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 --2- 172.21.15.46:0/1899761169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e809a110 0x7fa3e806bd20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8072cb0 tx=0x7fa3e001c4b0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 -- 172.21.15.46:0/1899761169 shutdown_connections 2024-04-25T08:43:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 --2- 172.21.15.46:0/1899761169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e809a110 0x7fa3e806bd20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 --2- 172.21.15.46:0/1899761169 >> v2:172.21.15.82:3300/0 conn(0x7fa3e815db80 0x7fa3e80cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 -- 172.21.15.46:0/1899761169 >> 172.21.15.46:0/1899761169 conn(0x7fa3e80b3120 msgr2=0x7fa3e80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 -- 172.21.15.46:0/1899761169 shutdown_connections 2024-04-25T08:43:15.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 -- 172.21.15.46:0/1899761169 wait complete. 2024-04-25T08:43:15.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 Processor -- start 2024-04-25T08:43:15.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.327+0000 7fa3ed800640 1 -- start start 2024-04-25T08:43:15.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3ed800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3e809a110 0x7fa3e815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3ed800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3e809a880 0x7fa3e815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3ed800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e815db80 0x7fa3e8151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3ed800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80d0940 con 0x7fa3e815db80 2024-04-25T08:43:15.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3ed800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e80d07c0 con 0x7fa3e809a880 2024-04-25T08:43:15.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3ed800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e806c670 con 0x7fa3e809a110 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e815db80 0x7fa3e8151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e815db80 0x7fa3e8151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51730/0 (socket says 172.21.15.46:51730) 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e7400640 1 -- 172.21.15.46:0/3521453882 learned_addr learned my addr 172.21.15.46:0/3521453882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e6a00640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3e809a110 0x7fa3e815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3de000640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3e809a880 0x7fa3e815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e7400640 1 -- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3e809a110 msgr2=0x7fa3e815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e7400640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3e809a110 0x7fa3e815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e7400640 1 -- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3e809a880 msgr2=0x7fa3e815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e7400640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3e809a880 0x7fa3e815cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e7400640 1 -- 172.21.15.46:0/3521453882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80ca1b0 con 0x7fa3e815db80 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e6a00640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3e809a110 0x7fa3e815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3de000640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3e809a880 0x7fa3e815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3e7400640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e815db80 0x7fa3e8151a10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00ac040 tx=0x7fa3e001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3df400640 1 -- 172.21.15.46:0/3521453882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00bd5f0 con 0x7fa3e815db80 2024-04-25T08:43:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3df400640 1 -- 172.21.15.46:0/3521453882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e00bd790 con 0x7fa3e815db80 2024-04-25T08:43:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3df400640 1 -- 172.21.15.46:0/3521453882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00bdb80 con 0x7fa3e815db80 2024-04-25T08:43:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e815b740 con 0x7fa3e815db80 2024-04-25T08:43:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e815bd00 con 0x7fa3e815db80 2024-04-25T08:43:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e8069010 con 0x7fa3e815db80 2024-04-25T08:43:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.331+0000 7fa3df400640 1 -- 172.21.15.46:0/3521453882 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3e00b33f0 con 0x7fa3e815db80 2024-04-25T08:43:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.335+0000 7fa3df400640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa3c4066f60 0x7fa3c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.335+0000 7fa3df400640 1 -- 172.21.15.46:0/3521453882 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa3e011fc80 con 0x7fa3e815db80 2024-04-25T08:43:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.335+0000 7fa3e6a00640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa3c4066f60 0x7fa3c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.335+0000 7fa3e6a00640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa3c4066f60 0x7fa3c4069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa3e81546a0 tx=0x7fa3d800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:15.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.339+0000 7fa3df400640 1 -- 172.21.15.46:0/3521453882 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3e00160d0 con 0x7fa3e815db80 2024-04-25T08:43:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3e80ce9f0 con 0x7fa3e815db80 2024-04-25T08:43:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3df400640 1 -- 172.21.15.46:0/3521453882 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3e00f09a0 con 0x7fa3e815db80 2024-04-25T08:43:15.492 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa3c4066f60 msgr2=0x7fa3c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa3c4066f60 0x7fa3c4069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa3e81546a0 tx=0x7fa3d800b040 comp rx=0 tx=0).stop 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e815db80 msgr2=0x7fa3e8151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e815db80 0x7fa3e8151a10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00ac040 tx=0x7fa3e001acf0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 shutdown_connections 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa3c4066f60 0x7fa3c4069420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3e815db80 0x7fa3e8151a10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3e809a880 0x7fa3e815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 --2- 172.21.15.46:0/3521453882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3e809a110 0x7fa3e815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 >> 172.21.15.46:0/3521453882 conn(0x7fa3e80b3120 msgr2=0x7fa3e80cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 shutdown_connections 2024-04-25T08:43:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.499+0000 7fa3ed800640 1 -- 172.21.15.46:0/3521453882 wait complete. 2024-04-25T08:43:15.513 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:15.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4512c00640 1 Processor -- start 2024-04-25T08:43:15.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4512c00640 1 -- start start 2024-04-25T08:43:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4512c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f450c1440d0 0x7f450c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4512c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f450c06f150 0x7f450c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4512c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450c15e440 con 0x7f450c143990 2024-04-25T08:43:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4512c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450c15ed30 con 0x7f450c06f150 2024-04-25T08:43:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4512c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450c15e6e0 con 0x7f450c1440d0 2024-04-25T08:43:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4510c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f450c06f150 0x7f450c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4510c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f450c06f150 0x7f450c0b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51744/0 (socket says 172.21.15.46:51744) 2024-04-25T08:43:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4510c00640 1 -- 172.21.15.46:0/1049739658 learned_addr learned my addr 172.21.15.46:0/1049739658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4503400640 1 --2- 172.21.15.46:0/1049739658 >> v2:172.21.15.82:3300/0 conn(0x7f450c1440d0 0x7f450c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4510c00640 1 -- 172.21.15.46:0/1049739658 >> v2:172.21.15.82:3300/0 conn(0x7f450c1440d0 msgr2=0x7f450c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4510c00640 1 --2- 172.21.15.46:0/1049739658 >> v2:172.21.15.82:3300/0 conn(0x7f450c1440d0 0x7f450c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4510c00640 1 -- 172.21.15.46:0/1049739658 >> v1:172.21.15.82:6789/0 conn(0x7f450c143990 legacy=0x7f450c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4510c00640 1 -- 172.21.15.46:0/1049739658 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c15e9c0 con 0x7f450c06f150 2024-04-25T08:43:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.667+0000 7f4503400640 1 --2- 172.21.15.46:0/1049739658 >> v2:172.21.15.82:3300/0 conn(0x7f450c1440d0 0x7f450c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f4510c00640 1 --2- 172.21.15.46:0/1049739658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c06f150 0x7f450c0b2720 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f450c070c80 tx=0x7f450401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a94694ac0c9449a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f450b400640 1 -- 172.21.15.46:0/1049739658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45040a56c0 con 0x7f450c06f150 2024-04-25T08:43:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f450b400640 1 -- 172.21.15.46:0/1049739658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45040a5860 con 0x7f450c06f150 2024-04-25T08:43:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f450b400640 1 -- 172.21.15.46:0/1049739658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45040a5c30 con 0x7f450c06f150 2024-04-25T08:43:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f4512c00640 1 -- 172.21.15.46:0/1049739658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c06f150 msgr2=0x7f450c0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f4512c00640 1 --2- 172.21.15.46:0/1049739658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c06f150 0x7f450c0b2720 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f450c070c80 tx=0x7f450401c9f0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f4512c00640 1 -- 172.21.15.46:0/1049739658 shutdown_connections 2024-04-25T08:43:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f4512c00640 1 --2- 172.21.15.46:0/1049739658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c06f150 0x7f450c0b2720 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f4512c00640 1 --2- 172.21.15.46:0/1049739658 >> v2:172.21.15.82:3300/0 conn(0x7f450c1440d0 0x7f450c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f4512c00640 1 -- 172.21.15.46:0/1049739658 >> 172.21.15.46:0/1049739658 conn(0x7f450c06ac10 msgr2=0x7f450c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f4512c00640 1 -- 172.21.15.46:0/1049739658 shutdown_connections 2024-04-25T08:43:15.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.671+0000 7f4512c00640 1 -- 172.21.15.46:0/1049739658 wait complete. 2024-04-25T08:43:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 Processor -- start 2024-04-25T08:43:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 -- start start 2024-04-25T08:43:15.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f450c06f150 0x7f450c14bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c143990 0x7f450c13ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f450c1440d0 0x7f450c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0b35f0 con 0x7f450c143990 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0b3470 con 0x7f450c06f150 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0b3770 con 0x7f450c1440d0 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4503400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c143990 0x7f450c13ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4503400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c143990 0x7f450c13ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51746/0 (socket says 172.21.15.46:51746) 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4503400640 1 -- 172.21.15.46:0/4135021599 learned_addr learned my addr 172.21.15.46:0/4135021599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f450be00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f450c06f150 0x7f450c14bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4510c00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f450c1440d0 0x7f450c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4503400640 1 -- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f450c1440d0 msgr2=0x7f450c0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4503400640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f450c1440d0 0x7f450c0b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4503400640 1 -- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f450c06f150 msgr2=0x7f450c14bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4503400640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f450c06f150 0x7f450c14bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4503400640 1 -- 172.21.15.46:0/4135021599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c0b04e0 con 0x7f450c143990 2024-04-25T08:43:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4503400640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c143990 0x7f450c13ba80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f44f400d830 tx=0x7f44f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4508c00640 1 -- 172.21.15.46:0/4135021599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400b6c0 con 0x7f450c143990 2024-04-25T08:43:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4508c00640 1 -- 172.21.15.46:0/4135021599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f4019070 con 0x7f450c143990 2024-04-25T08:43:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c0b07c0 con 0x7f450c143990 2024-04-25T08:43:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4508c00640 1 -- 172.21.15.46:0/4135021599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f400baa0 con 0x7f450c143990 2024-04-25T08:43:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c0b0d80 con 0x7f450c143990 2024-04-25T08:43:15.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.675+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450c069010 con 0x7f450c143990 2024-04-25T08:43:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.679+0000 7f4508c00640 1 -- 172.21.15.46:0/4135021599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44f401e460 con 0x7f450c143990 2024-04-25T08:43:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.679+0000 7f4508c00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44e8066f60 0x7f44e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.679+0000 7f4508c00640 1 -- 172.21.15.46:0/4135021599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f44f4085a90 con 0x7f450c143990 2024-04-25T08:43:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.679+0000 7f450be00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44e8066f60 0x7f44e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.679+0000 7f450be00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44e8066f60 0x7f44e8069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000f70 tx=0x7f44fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:15.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.683+0000 7f4508c00640 1 -- 172.21.15.46:0/4135021599 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44f4056690 con 0x7f450c143990 2024-04-25T08:43:15.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.843+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f450c139b50 con 0x7f450c143990 2024-04-25T08:43:15.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.843+0000 7f4508c00640 1 -- 172.21.15.46:0/4135021599 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44f4055f90 con 0x7f450c143990 2024-04-25T08:43:15.838 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44e8066f60 msgr2=0x7f44e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44e8066f60 0x7f44e8069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000f70 tx=0x7f44fc009040 comp rx=0 tx=0).stop 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c143990 msgr2=0x7f450c13ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c143990 0x7f450c13ba80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f44f400d830 tx=0x7f44f400dcf0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 shutdown_connections 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44e8066f60 0x7f44e8069420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f450c1440d0 0x7f450c0b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f450c143990 0x7f450c13ba80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 --2- 172.21.15.46:0/4135021599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f450c06f150 0x7f450c14bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 >> 172.21.15.46:0/4135021599 conn(0x7f450c06ac10 msgr2=0x7f450c13a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 shutdown_connections 2024-04-25T08:43:15.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:15.847+0000 7f4512c00640 1 -- 172.21.15.46:0/4135021599 wait complete. 2024-04-25T08:43:15.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd2800640 1 Processor -- start 2024-04-25T08:43:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd2800640 1 -- start start 2024-04-25T08:43:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faccc06ac10 0x7faccc0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd2800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc06b0a0 con 0x7faccc0b2930 2024-04-25T08:43:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd2800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc06b360 con 0x7faccc0b11a0 2024-04-25T08:43:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc15cc50 con 0x7faccc06ac10 2024-04-25T08:43:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd0a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faccc0b2930 0x7faccc15c540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60296/0 (socket says 172.21.15.46:60296) 2024-04-25T08:43:16.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd0a00640 1 -- 172.21.15.46:0/2166089547 learned_addr learned my addr 172.21.15.46:0/2166089547 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd1e00640 1 --2- 172.21.15.46:0/2166089547 >> v2:172.21.15.82:3300/0 conn(0x7faccc06ac10 0x7faccc0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7faccbe00640 1 -- 172.21.15.46:0/2166089547 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698334942 0 0) 0x7faccc06b0a0 con 0x7faccc0b2930 2024-04-25T08:43:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7faccbe00640 1 -- 172.21.15.46:0/2166089547 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb4003530 con 0x7faccc0b2930 2024-04-25T08:43:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7faccbe00640 1 -- 172.21.15.46:0/2166089547 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680791137 0 0) 0x7faccc06b360 con 0x7faccc0b11a0 2024-04-25T08:43:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7faccbe00640 1 -- 172.21.15.46:0/2166089547 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faccc06b0a0 con 0x7faccc0b11a0 2024-04-25T08:43:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd1e00640 1 -- 172.21.15.46:0/2166089547 >> v1:172.21.15.82:6789/0 conn(0x7faccc0b11a0 legacy=0x7faccc0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd1e00640 1 -- 172.21.15.46:0/2166089547 >> v1:172.21.15.46:6789/0 conn(0x7faccc0b2930 legacy=0x7faccc15c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7faccbe00640 1 -- 172.21.15.46:0/2166089547 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1707345477 0 0) 0x7facb4003530 con 0x7faccc0b2930 2024-04-25T08:43:16.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.011+0000 7facd1e00640 1 -- 172.21.15.46:0/2166089547 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc06b640 con 0x7faccc06ac10 2024-04-25T08:43:16.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7facd1e00640 1 --2- 172.21.15.46:0/2166089547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 0x7faccc0afe30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7facc4009dc0 tx=0x7facc401c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7c7e6f64b2ff06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:16.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7faccbe00640 1 -- 172.21.15.46:0/2166089547 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40b2070 con 0x7faccc06ac10 2024-04-25T08:43:16.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7faccbe00640 1 -- 172.21.15.46:0/2166089547 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc40a5600 con 0x7faccc06ac10 2024-04-25T08:43:16.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7faccbe00640 1 -- 172.21.15.46:0/2166089547 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8001680 con 0x7faccc06ac10 2024-04-25T08:43:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7facd2800640 1 -- 172.21.15.46:0/2166089547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 msgr2=0x7faccc0afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7facd2800640 1 --2- 172.21.15.46:0/2166089547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 0x7faccc0afe30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7facc4009dc0 tx=0x7facc401c870 comp rx=0 tx=0).stop 2024-04-25T08:43:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7facd2800640 1 -- 172.21.15.46:0/2166089547 shutdown_connections 2024-04-25T08:43:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7facd2800640 1 --2- 172.21.15.46:0/2166089547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 0x7faccc0afe30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7facd2800640 1 -- 172.21.15.46:0/2166089547 >> 172.21.15.46:0/2166089547 conn(0x7faccc099a00 msgr2=0x7faccc0b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:16.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7facd2800640 1 -- 172.21.15.46:0/2166089547 shutdown_connections 2024-04-25T08:43:16.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.015+0000 7facd2800640 1 -- 172.21.15.46:0/2166089547 wait complete. 2024-04-25T08:43:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 Processor -- start 2024-04-25T08:43:16.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 -- start start 2024-04-25T08:43:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 0x7faccc06f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faccc0b11a0 0x7faccc1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faccc0b2930 0x7faccc153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc1594f0 con 0x7faccc0b11a0 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0b0780 con 0x7faccc06ac10 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc159670 con 0x7faccc0b2930 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faccc0b11a0 0x7faccc1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faccc0b11a0 0x7faccc1539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51754/0 (socket says 172.21.15.46:51754) 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 0x7faccc06f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd0a00640 1 -- 172.21.15.46:0/3763615476 learned_addr learned my addr 172.21.15.46:0/3763615476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd1e00640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faccc0b2930 0x7faccc153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd0a00640 1 -- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faccc0b2930 msgr2=0x7faccc153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd0a00640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faccc0b2930 0x7faccc153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd0a00640 1 -- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 msgr2=0x7faccc06f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd0a00640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 0x7faccc06f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd0a00640 1 -- 172.21.15.46:0/3763615476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc0b4f90 con 0x7faccc0b11a0 2024-04-25T08:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd1400640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 0x7faccc06f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd0a00640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faccc0b11a0 0x7faccc1539a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7facb800c730 tx=0x7facb800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facc9600640 1 -- 172.21.15.46:0/3763615476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8017070 con 0x7faccc0b11a0 2024-04-25T08:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facc9600640 1 -- 172.21.15.46:0/3763615476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb8005ae0 con 0x7faccc0b11a0 2024-04-25T08:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facc9600640 1 -- 172.21.15.46:0/3763615476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8007070 con 0x7faccc0b11a0 2024-04-25T08:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc0b5270 con 0x7faccc0b11a0 2024-04-25T08:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.019+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc0b5830 con 0x7faccc0b11a0 2024-04-25T08:43:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.023+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc069010 con 0x7faccc0b11a0 2024-04-25T08:43:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.023+0000 7facc9600640 1 -- 172.21.15.46:0/3763615476 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7facb801a070 con 0x7faccc0b11a0 2024-04-25T08:43:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.023+0000 7facc9600640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facac066f60 0x7facac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.023+0000 7facc9600640 1 -- 172.21.15.46:0/3763615476 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7facb8084820 con 0x7faccc0b11a0 2024-04-25T08:43:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.023+0000 7facd1400640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facac066f60 0x7facac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.023+0000 7facd1400640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facac066f60 0x7facac069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7facc00011f0 tx=0x7facc0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:16.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.027+0000 7facc9600640 1 -- 172.21.15.46:0/3763615476 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7facb8054d40 con 0x7faccc0b11a0 2024-04-25T08:43:16.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faccc0b2200 con 0x7faccc0b11a0 2024-04-25T08:43:16.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facc9600640 1 -- 172.21.15.46:0/3763615476 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7facb8058de0 con 0x7faccc0b11a0 2024-04-25T08:43:16.183 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facac066f60 msgr2=0x7facac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facac066f60 0x7facac069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7facc00011f0 tx=0x7facc0009040 comp rx=0 tx=0).stop 2024-04-25T08:43:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faccc0b11a0 msgr2=0x7faccc1539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faccc0b11a0 0x7faccc1539a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7facb800c730 tx=0x7facb800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 shutdown_connections 2024-04-25T08:43:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facac066f60 0x7facac069420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7faccc0b2930 0x7faccc153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faccc0b11a0 0x7faccc1539a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 --2- 172.21.15.46:0/3763615476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faccc06ac10 0x7faccc06f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 >> 172.21.15.46:0/3763615476 conn(0x7faccc099a00 msgr2=0x7faccc158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:16.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 shutdown_connections 2024-04-25T08:43:16.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.191+0000 7facd2800640 1 -- 172.21.15.46:0/3763615476 wait complete. 2024-04-25T08:43:16.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838fe00640 1 Processor -- start 2024-04-25T08:43:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838fe00640 1 -- start start 2024-04-25T08:43:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8390146a90 0x7f839006ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839006b150 0x7f83901674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83900b5210 con 0x7f83900b0ce0 2024-04-25T08:43:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83900b1ee0 con 0x7f8390146a90 2024-04-25T08:43:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83900b62e0 con 0x7f839006b150 2024-04-25T08:43:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8390146a90 0x7f839006ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f83900b0ce0 0x7f83900b17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60298/0 (socket says 172.21.15.46:60298) 2024-04-25T08:43:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838ea00640 1 -- 172.21.15.46:0/1633386334 learned_addr learned my addr 172.21.15.46:0/1633386334 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.355+0000 7f838f400640 1 --2- 172.21.15.46:0/1633386334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839006b150 0x7f83901674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838d600640 1 -- 172.21.15.46:0/1633386334 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218406280 0 0) 0x7f83900b5210 con 0x7f83900b0ce0 2024-04-25T08:43:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838d600640 1 -- 172.21.15.46:0/1633386334 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8374003530 con 0x7f83900b0ce0 2024-04-25T08:43:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838e000640 1 -- 172.21.15.46:0/1633386334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839006b150 msgr2=0x7f83901674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838e000640 1 --2- 172.21.15.46:0/1633386334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839006b150 0x7f83901674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838e000640 1 -- 172.21.15.46:0/1633386334 >> v1:172.21.15.46:6789/0 conn(0x7f83900b0ce0 legacy=0x7f83900b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838e000640 1 -- 172.21.15.46:0/1633386334 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83900b54f0 con 0x7f8390146a90 2024-04-25T08:43:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838e000640 1 --2- 172.21.15.46:0/1633386334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8390146a90 0x7f839006ac10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f837c0097f0 tx=0x7f837c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4d1fc74f93333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838d600640 1 -- 172.21.15.46:0/1633386334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c0b0070 con 0x7f8390146a90 2024-04-25T08:43:16.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838d600640 1 -- 172.21.15.46:0/1633386334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c0ab600 con 0x7f8390146a90 2024-04-25T08:43:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.359+0000 7f838d600640 1 -- 172.21.15.46:0/1633386334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c0ab9d0 con 0x7f8390146a90 2024-04-25T08:43:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 -- 172.21.15.46:0/1633386334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8390146a90 msgr2=0x7f839006ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 --2- 172.21.15.46:0/1633386334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8390146a90 0x7f839006ac10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f837c0097f0 tx=0x7f837c01c410 comp rx=0 tx=0).stop 2024-04-25T08:43:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 -- 172.21.15.46:0/1633386334 shutdown_connections 2024-04-25T08:43:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 --2- 172.21.15.46:0/1633386334 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f839006b150 0x7f83901674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 --2- 172.21.15.46:0/1633386334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8390146a90 0x7f839006ac10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 -- 172.21.15.46:0/1633386334 >> 172.21.15.46:0/1633386334 conn(0x7f8390099a00 msgr2=0x7f83900b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 -- 172.21.15.46:0/1633386334 shutdown_connections 2024-04-25T08:43:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 -- 172.21.15.46:0/1633386334 wait complete. 2024-04-25T08:43:16.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 Processor -- start 2024-04-25T08:43:16.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 -- start start 2024-04-25T08:43:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839006b150 0x7f839015af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83900b0ce0 0x7f83900ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8390146a90 0x7f83900adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839016a200 con 0x7f839006b150 2024-04-25T08:43:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83901455f0 con 0x7f8390146a90 2024-04-25T08:43:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839016a380 con 0x7f83900b0ce0 2024-04-25T08:43:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839006b150 0x7f839015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839006b150 0x7f839015af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51768/0 (socket says 172.21.15.46:51768) 2024-04-25T08:43:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838ea00640 1 -- 172.21.15.46:0/3516294511 learned_addr learned my addr 172.21.15.46:0/3516294511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838e000640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83900b0ce0 0x7f83900ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838f400640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8390146a90 0x7f83900adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838ea00640 1 -- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83900b0ce0 msgr2=0x7f83900ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838ea00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83900b0ce0 0x7f83900ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838ea00640 1 -- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8390146a90 msgr2=0x7f83900adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838ea00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8390146a90 0x7f83900adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838ea00640 1 -- 172.21.15.46:0/3516294511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83900affd0 con 0x7f839006b150 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.363+0000 7f838ea00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839006b150 0x7f839015af80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f838000c400 tx=0x7f838000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f8386a00640 1 -- 172.21.15.46:0/3516294511 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380017070 con 0x7f839006b150 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f8386a00640 1 -- 172.21.15.46:0/3516294511 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8380005d10 con 0x7f839006b150 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f8386a00640 1 -- 172.21.15.46:0/3516294511 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380007310 con 0x7f839006b150 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839015c490 con 0x7f839006b150 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839015ca50 con 0x7f839006b150 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839015c2a0 con 0x7f839006b150 2024-04-25T08:43:16.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f8386a00640 1 -- 172.21.15.46:0/3516294511 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f838001a070 con 0x7f839006b150 2024-04-25T08:43:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f8386a00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8370066f60 0x7f8370069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f8386a00640 1 -- 172.21.15.46:0/3516294511 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8380084810 con 0x7f839006b150 2024-04-25T08:43:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.367+0000 7f838e000640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8370066f60 0x7f8370069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.371+0000 7f838e000640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8370066f60 0x7f8370069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f837c0ad000 tx=0x7f837c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:16.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.371+0000 7f8386a00640 1 -- 172.21.15.46:0/3516294511 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8380055410 con 0x7f839006b150 2024-04-25T08:43:16.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.531+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8390069010 con 0x7f839006b150 2024-04-25T08:43:16.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.531+0000 7f8386a00640 1 -- 172.21.15.46:0/3516294511 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8380054d10 con 0x7f839006b150 2024-04-25T08:43:16.526 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8370066f60 msgr2=0x7f8370069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8370066f60 0x7f8370069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f837c0ad000 tx=0x7f837c0a6000 comp rx=0 tx=0).stop 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839006b150 msgr2=0x7f839015af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839006b150 0x7f839015af80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f838000c400 tx=0x7f838000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 shutdown_connections 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8370066f60 0x7f8370069420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8390146a90 0x7f83900adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83900b0ce0 0x7f83900ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 --2- 172.21.15.46:0/3516294511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f839006b150 0x7f839015af80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 >> 172.21.15.46:0/3516294511 conn(0x7f8390099a00 msgr2=0x7f83900b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 shutdown_connections 2024-04-25T08:43:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.535+0000 7f838fe00640 1 -- 172.21.15.46:0/3516294511 wait complete. 2024-04-25T08:43:16.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:43:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.695+0000 7fc357a00640 1 Processor -- start 2024-04-25T08:43:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.695+0000 7fc357a00640 1 -- start start 2024-04-25T08:43:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc357a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc3500cc150 0x7fc3500b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc357a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3500b25a0 con 0x7fc350157b00 2024-04-25T08:43:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc357a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3500cc6f0 con 0x7fc3500ce420 2024-04-25T08:43:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc357a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3500b2eb0 con 0x7fc3500cc150 2024-04-25T08:43:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc355000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc350157b00 0x7fc3500cdd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60302/0 (socket says 172.21.15.46:60302) 2024-04-25T08:43:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc355000640 1 -- 172.21.15.46:0/2497839425 learned_addr learned my addr 172.21.15.46:0/2497839425 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc347e00640 1 --2- 172.21.15.46:0/2497839425 >> v2:172.21.15.82:3300/0 conn(0x7fc3500cc150 0x7fc3500b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035409058 0 0) 0x7fc3500b25a0 con 0x7fc350157b00 2024-04-25T08:43:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc334003450 con 0x7fc350157b00 2024-04-25T08:43:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526781972 0 0) 0x7fc3500cc6f0 con 0x7fc3500ce420 2024-04-25T08:43:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3500b25a0 con 0x7fc3500ce420 2024-04-25T08:43:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029280862 0 0) 0x7fc334003450 con 0x7fc350157b00 2024-04-25T08:43:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 >> v2:172.21.15.82:3300/0 conn(0x7fc3500cc150 msgr2=0x7fc3500b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 --2- 172.21.15.46:0/2497839425 >> v2:172.21.15.82:3300/0 conn(0x7fc3500cc150 0x7fc3500b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 >> v1:172.21.15.82:6789/0 conn(0x7fc3500ce420 legacy=0x7fc35015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500b2860 con 0x7fc350157b00 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fc340001680 con 0x7fc350157b00 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc347e00640 1 --2- 172.21.15.46:0/2497839425 >> v2:172.21.15.82:3300/0 conn(0x7fc3500cc150 0x7fc3500b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 >> v1:172.21.15.46:6789/0 conn(0x7fc350157b00 legacy=0x7fc3500cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc334003c40 0x7fc334008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc334004170 0x7fc33400ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc33400b560 0x7fc33400da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500b2eb0 con 0x7fc33400b560 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33400e300 con 0x7fc334003c40 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33400e500 con 0x7fc334004170 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc355a00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc33400b560 0x7fc33400da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc355000640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc334003c40 0x7fc334008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc347e00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc334004170 0x7fc33400ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc355a00640 1 -- 172.21.15.46:0/2497839425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc334004170 msgr2=0x7fc33400ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc355a00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc334004170 0x7fc33400ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc355a00640 1 -- 172.21.15.46:0/2497839425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc334003c40 msgr2=0x7fc334008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc355a00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc334003c40 0x7fc334008860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.699+0000 7fc355a00640 1 -- 172.21.15.46:0/2497839425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc334003450 con 0x7fc33400b560 2024-04-25T08:43:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc355a00640 1 -- 172.21.15.46:0/2497839425 reap_dead start 2024-04-25T08:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc355a00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc33400b560 0x7fc33400da20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc348001fd0 tx=0x7fc34801f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f9db53cd77e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348001c20 con 0x7fc33400b560 2024-04-25T08:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3480a8c90 con 0x7fc33400b560 2024-04-25T08:43:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc34fe00640 1 -- 172.21.15.46:0/2497839425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348003bc0 con 0x7fc33400b560 2024-04-25T08:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc357a00640 1 -- 172.21.15.46:0/2497839425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc33400b560 msgr2=0x7fc33400da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc357a00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc33400b560 0x7fc33400da20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc348001fd0 tx=0x7fc34801f070 comp rx=0 tx=0).stop 2024-04-25T08:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc357a00640 1 -- 172.21.15.46:0/2497839425 shutdown_connections 2024-04-25T08:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc357a00640 1 --2- 172.21.15.46:0/2497839425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc33400b560 0x7fc33400da20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc357a00640 1 -- 172.21.15.46:0/2497839425 >> 172.21.15.46:0/2497839425 conn(0x7fc3500b12a0 msgr2=0x7fc3500b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc357a00640 1 -- 172.21.15.46:0/2497839425 shutdown_connections 2024-04-25T08:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc357a00640 1 -- 172.21.15.46:0/2497839425 wait complete. 2024-04-25T08:43:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc357a00640 1 Processor -- start 2024-04-25T08:43:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.703+0000 7fc357a00640 1 -- start start 2024-04-25T08:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc357a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc33400b560 0x7fc35014f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc357a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3500ad280 0x7fc35015c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc357a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc35015c9d0 0x7fc35015a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc357a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500ce870 con 0x7fc3500ad280 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc357a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500b4300 con 0x7fc33400b560 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc357a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3500ce9f0 con 0x7fc35015c9d0 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc347e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3500ad280 0x7fc35015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc347e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3500ad280 0x7fc35015c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51778/0 (socket says 172.21.15.46:51778) 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc347e00640 1 -- 172.21.15.46:0/2682937916 learned_addr learned my addr 172.21.15.46:0/2682937916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc355000640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc33400b560 0x7fc35014f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc355a00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc35015c9d0 0x7fc35015a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc347e00640 1 -- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc35015c9d0 msgr2=0x7fc35015a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc347e00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc35015c9d0 0x7fc35015a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc347e00640 1 -- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc33400b560 msgr2=0x7fc35014f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc347e00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc33400b560 0x7fc35014f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc347e00640 1 -- 172.21.15.46:0/2682937916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500ad840 con 0x7fc3500ad280 2024-04-25T08:43:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc355000640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc33400b560 0x7fc35014f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc347e00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3500ad280 0x7fc35015c490 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc338011cd0 tx=0x7fc33800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc34d600640 1 -- 172.21.15.46:0/2682937916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800fd20 con 0x7fc3500ad280 2024-04-25T08:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc34d600640 1 -- 172.21.15.46:0/2682937916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc338004020 con 0x7fc3500ad280 2024-04-25T08:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc34d600640 1 -- 172.21.15.46:0/2682937916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3380043f0 con 0x7fc3500ad280 2024-04-25T08:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3500adb20 con 0x7fc3500ad280 2024-04-25T08:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3500ae0e0 con 0x7fc3500ad280 2024-04-25T08:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.707+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc350069080 con 0x7fc3500ad280 2024-04-25T08:43:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.711+0000 7fc34d600640 1 -- 172.21.15.46:0/2682937916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc338012040 con 0x7fc3500ad280 2024-04-25T08:43:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.711+0000 7fc34d600640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc32c066f90 0x7fc32c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.711+0000 7fc34d600640 1 -- 172.21.15.46:0/2682937916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc338085920 con 0x7fc3500ad280 2024-04-25T08:43:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.711+0000 7fc355000640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc32c066f90 0x7fc32c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.711+0000 7fc355000640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc32c066f90 0x7fc32c069450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc340001210 tx=0x7fc340001e50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.715+0000 7fc34d600640 1 -- 172.21.15.46:0/2682937916 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc338019030 con 0x7fc3500ad280 2024-04-25T08:43:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.887+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc3500724a0 con 0x7fc3500ad280 2024-04-25T08:43:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.887+0000 7fc34d600640 1 -- 172.21.15.46:0/2682937916 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc338055e40 con 0x7fc3500ad280 2024-04-25T08:43:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:43:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc32c066f90 msgr2=0x7fc32c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc32c066f90 0x7fc32c069450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc340001210 tx=0x7fc340001e50 comp rx=0 tx=0).stop 2024-04-25T08:43:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3500ad280 msgr2=0x7fc35015c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3500ad280 0x7fc35015c490 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc338011cd0 tx=0x7fc33800f610 comp rx=0 tx=0).stop 2024-04-25T08:43:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 shutdown_connections 2024-04-25T08:43:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc32c066f90 0x7fc32c069450 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc35015c9d0 0x7fc35015a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3500ad280 0x7fc35015c490 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 --2- 172.21.15.46:0/2682937916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc33400b560 0x7fc35014f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 >> 172.21.15.46:0/2682937916 conn(0x7fc3500b12a0 msgr2=0x7fc3500b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 shutdown_connections 2024-04-25T08:43:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:16.891+0000 7fc357a00640 1 -- 172.21.15.46:0/2682937916 wait complete. 2024-04-25T08:43:16.903 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T08:43:16.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:16.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T08:43:16.927 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:43:16.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:16.927 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:43:16.982 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:43:16.982 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:16.992 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:43:16.992 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:17.041 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:43:17.041 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:17.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e40e00640 1 Processor -- start 2024-04-25T08:43:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e40e00640 1 -- start start 2024-04-25T08:43:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e40e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e3c099a00 0x7f7e3c099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e40e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 0x7f7e3c0b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e40e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c09a320 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e40e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c06db90 con 0x7f7e3c099a00 2024-04-25T08:43:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e40e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c06bdc0 con 0x7f7e3c0b2240 2024-04-25T08:43:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e33e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7e3c0ccde0 0x7f7e3c0cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60310/0 (socket says 172.21.15.46:60310) 2024-04-25T08:43:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e39e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e3c099a00 0x7f7e3c099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e33e00640 1 -- 172.21.15.46:0/2982742464 learned_addr learned my addr 172.21.15.46:0/2982742464 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e3a800640 1 --2- 172.21.15.46:0/2982742464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 0x7f7e3c0b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e39e00640 1 -- 172.21.15.46:0/2982742464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 msgr2=0x7f7e3c0b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e39e00640 1 --2- 172.21.15.46:0/2982742464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 0x7f7e3c0b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e39e00640 1 -- 172.21.15.46:0/2982742464 >> v1:172.21.15.46:6789/0 conn(0x7f7e3c0ccde0 legacy=0x7f7e3c0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e39e00640 1 -- 172.21.15.46:0/2982742464 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c06c0a0 con 0x7f7e3c099a00 2024-04-25T08:43:17.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.247+0000 7f7e39400640 1 -- 172.21.15.46:0/2982742464 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3861270449 0 0) 0x7f7e3c09a320 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e39e00640 1 --2- 172.21.15.46:0/2982742464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c099a00 0x7f7e3c099de0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0097c0 tx=0x7f7e2c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a327c150816cc751 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e39400640 1 -- 172.21.15.46:0/2982742464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0b0070 con 0x7f7e3c099a00 2024-04-25T08:43:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e39400640 1 -- 172.21.15.46:0/2982742464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c0ab600 con 0x7f7e3c099a00 2024-04-25T08:43:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e39400640 1 -- 172.21.15.46:0/2982742464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c0ab9d0 con 0x7f7e3c099a00 2024-04-25T08:43:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e40e00640 1 -- 172.21.15.46:0/2982742464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c099a00 msgr2=0x7f7e3c099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e40e00640 1 --2- 172.21.15.46:0/2982742464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c099a00 0x7f7e3c099de0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0097c0 tx=0x7f7e2c01c410 comp rx=0 tx=0).stop 2024-04-25T08:43:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e40e00640 1 -- 172.21.15.46:0/2982742464 shutdown_connections 2024-04-25T08:43:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e40e00640 1 --2- 172.21.15.46:0/2982742464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 0x7f7e3c0b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e40e00640 1 --2- 172.21.15.46:0/2982742464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c099a00 0x7f7e3c099de0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e40e00640 1 -- 172.21.15.46:0/2982742464 >> 172.21.15.46:0/2982742464 conn(0x7f7e3c0b3120 msgr2=0x7f7e3c0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e40e00640 1 -- 172.21.15.46:0/2982742464 shutdown_connections 2024-04-25T08:43:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.251+0000 7f7e40e00640 1 -- 172.21.15.46:0/2982742464 wait complete. 2024-04-25T08:43:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 Processor -- start 2024-04-25T08:43:17.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 -- start start 2024-04-25T08:43:17.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c099a00 0x7f7e3c15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 0x7f7e3c151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c0ccde0 0x7f7e3c1520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c15dd10 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0cdb80 con 0x7f7e3c099a00 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c15de90 con 0x7f7e3c0b2240 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e3a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c0ccde0 0x7f7e3c1520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e3a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c0ccde0 0x7f7e3c1520a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51802/0 (socket says 172.21.15.46:51802) 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e3a800640 1 -- 172.21.15.46:0/4202738865 learned_addr learned my addr 172.21.15.46:0/4202738865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e39e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c099a00 0x7f7e3c15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e33e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 0x7f7e3c151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e3a800640 1 -- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 msgr2=0x7f7e3c151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e3a800640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 0x7f7e3c151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e3a800640 1 -- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c099a00 msgr2=0x7f7e3c15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e3a800640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c099a00 0x7f7e3c15fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e3a800640 1 -- 172.21.15.46:0/4202738865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c150410 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e3a800640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c0ccde0 0x7f7e3c1520a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7e34007e60 tx=0x7f7e3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e32000640 1 -- 172.21.15.46:0/4202738865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34014420 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e32000640 1 -- 172.21.15.46:0/4202738865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e34019070 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c1528d0 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e32000640 1 -- 172.21.15.46:0/4202738865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34014820 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.255+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c152e90 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.259+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c069080 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.259+0000 7f7e32000640 1 -- 172.21.15.46:0/4202738865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e340149c0 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.259+0000 7f7e32000640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e18066f60 0x7f7e18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.259+0000 7f7e32000640 1 -- 172.21.15.46:0/4202738865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7e34085b50 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.259+0000 7f7e39e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e18066f60 0x7f7e18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.259+0000 7f7e39e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e18066f60 0x7f7e18069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0ad000 tx=0x7f7e2c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:17.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.263+0000 7f7e32000640 1 -- 172.21.15.46:0/4202738865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e34056100 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.431+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e3c067c50 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e32000640 1 -- 172.21.15.46:0/4202738865 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7f7e3405a1a0 con 0x7f7e3c0ccde0 2024-04-25T08:43:17.427 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:17.427 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e18066f60 msgr2=0x7f7e18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e18066f60 0x7f7e18069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0ad000 tx=0x7f7e2c0a6000 comp rx=0 tx=0).stop 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c0ccde0 msgr2=0x7f7e3c1520a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c0ccde0 0x7f7e3c1520a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7e34007e60 tx=0x7f7e3400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 shutdown_connections 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e18066f60 0x7f7e18069420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e3c0ccde0 0x7f7e3c1520a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e3c0b2240 0x7f7e3c151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 --2- 172.21.15.46:0/4202738865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e3c099a00 0x7f7e3c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 >> 172.21.15.46:0/4202738865 conn(0x7f7e3c0b3120 msgr2=0x7f7e3c155aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 shutdown_connections 2024-04-25T08:43:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.435+0000 7f7e40e00640 1 -- 172.21.15.46:0/4202738865 wait complete. 2024-04-25T08:43:17.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:17.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.599+0000 7fe60b000640 1 Processor -- start 2024-04-25T08:43:17.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.599+0000 7fe60b000640 1 -- start start 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe60b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe60406e620 0x7fe60406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe60b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 0x7fe60406c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe60b000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6040b3850 con 0x7fe60415dae0 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe60b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60406e190 con 0x7fe60406e620 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe60b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6040b3af0 con 0x7fe60406ea60 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe603400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe60406e620 0x7fe60406bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe603400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe60406e620 0x7fe60406bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51818/0 (socket says 172.21.15.46:51818) 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe603400640 1 -- 172.21.15.46:0/314717633 learned_addr learned my addr 172.21.15.46:0/314717633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe609000640 1 --2- 172.21.15.46:0/314717633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 0x7fe60406c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe603400640 1 -- 172.21.15.46:0/314717633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 msgr2=0x7fe60406c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe603400640 1 --2- 172.21.15.46:0/314717633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 0x7fe60406c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe603400640 1 -- 172.21.15.46:0/314717633 >> v1:172.21.15.82:6789/0 conn(0x7fe60415dae0 legacy=0x7fe604072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.603+0000 7fe603400640 1 -- 172.21.15.46:0/314717633 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6040b3dd0 con 0x7fe60406e620 2024-04-25T08:43:17.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe603400640 1 --2- 172.21.15.46:0/314717633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60406e620 0x7fe60406bc20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe60406e310 tx=0x7fe5f001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f4412b53f79884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:17.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe602a00640 1 -- 172.21.15.46:0/314717633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00aa670 con 0x7fe60406e620 2024-04-25T08:43:17.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe602a00640 1 -- 172.21.15.46:0/314717633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f00af070 con 0x7fe60406e620 2024-04-25T08:43:17.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe602a00640 1 -- 172.21.15.46:0/314717633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00aaa30 con 0x7fe60406e620 2024-04-25T08:43:17.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 -- 172.21.15.46:0/314717633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60406e620 msgr2=0x7fe60406bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 --2- 172.21.15.46:0/314717633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60406e620 0x7fe60406bc20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe60406e310 tx=0x7fe5f001c510 comp rx=0 tx=0).stop 2024-04-25T08:43:17.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 -- 172.21.15.46:0/314717633 shutdown_connections 2024-04-25T08:43:17.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 --2- 172.21.15.46:0/314717633 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 0x7fe60406c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 --2- 172.21.15.46:0/314717633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60406e620 0x7fe60406bc20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 -- 172.21.15.46:0/314717633 >> 172.21.15.46:0/314717633 conn(0x7fe604071ae0 msgr2=0x7fe604071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:17.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 -- 172.21.15.46:0/314717633 shutdown_connections 2024-04-25T08:43:17.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 -- 172.21.15.46:0/314717633 wait complete. 2024-04-25T08:43:17.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 Processor -- start 2024-04-25T08:43:17.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.607+0000 7fe60b000640 1 -- start start 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe60b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe60406e620 0x7fe604160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe60b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 0x7fe6041497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe60b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60415dae0 0x7fe60414d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe60b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60406f850 con 0x7fe60415dae0 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe60b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60406f6d0 con 0x7fe60406e620 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe60b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6040ad5c0 con 0x7fe60406ea60 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe609000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60415dae0 0x7fe60414d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe609000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60415dae0 0x7fe60414d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51832/0 (socket says 172.21.15.46:51832) 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe609000640 1 -- 172.21.15.46:0/1514827700 learned_addr learned my addr 172.21.15.46:0/1514827700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe603400640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 0x7fe6041497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe603e00640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe60406e620 0x7fe604160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe609000640 1 -- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 msgr2=0x7fe6041497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe609000640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 0x7fe6041497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe609000640 1 -- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe60406e620 msgr2=0x7fe604160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe609000640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe60406e620 0x7fe604160c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe609000640 1 -- 172.21.15.46:0/1514827700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe604149e50 con 0x7fe60415dae0 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe603400640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 0x7fe6041497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe609000640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60415dae0 0x7fe60414d3c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc007e60 tx=0x7fe5fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe5fbe00640 1 -- 172.21.15.46:0/1514827700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc014620 con 0x7fe60415dae0 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe5fbe00640 1 -- 172.21.15.46:0/1514827700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc019070 con 0x7fe60415dae0 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60414a130 con 0x7fe60415dae0 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe5fbe00640 1 -- 172.21.15.46:0/1514827700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc014a00 con 0x7fe60415dae0 2024-04-25T08:43:17.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60414a6f0 con 0x7fe60415dae0 2024-04-25T08:43:17.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe604069010 con 0x7fe60415dae0 2024-04-25T08:43:17.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.611+0000 7fe5fbe00640 1 -- 172.21.15.46:0/1514827700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5fc014ba0 con 0x7fe60415dae0 2024-04-25T08:43:17.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.615+0000 7fe5fbe00640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5e4066f60 0x7fe5e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.615+0000 7fe5fbe00640 1 -- 172.21.15.46:0/1514827700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe5fc085d40 con 0x7fe60415dae0 2024-04-25T08:43:17.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.615+0000 7fe603e00640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5e4066f60 0x7fe5e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.615+0000 7fe603e00640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5e4066f60 0x7fe5e4069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe60415fa60 tx=0x7fe5f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:17.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.619+0000 7fe5fbe00640 1 -- 172.21.15.46:0/1514827700 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5fc056950 con 0x7fe60415dae0 2024-04-25T08:43:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.779+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe604153ae0 con 0x7fe60415dae0 2024-04-25T08:43:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.779+0000 7fe5fbe00640 1 -- 172.21.15.46:0/1514827700 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5fc056250 con 0x7fe60415dae0 2024-04-25T08:43:17.773 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5e4066f60 msgr2=0x7fe5e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5e4066f60 0x7fe5e4069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe60415fa60 tx=0x7fe5f4009040 comp rx=0 tx=0).stop 2024-04-25T08:43:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60415dae0 msgr2=0x7fe60414d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60415dae0 0x7fe60414d3c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc007e60 tx=0x7fe5fc00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 shutdown_connections 2024-04-25T08:43:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5e4066f60 0x7fe5e4069420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe60415dae0 0x7fe60414d3c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe60406ea60 0x7fe6041497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 --2- 172.21.15.46:0/1514827700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe60406e620 0x7fe604160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 >> 172.21.15.46:0/1514827700 conn(0x7fe604071ae0 msgr2=0x7fe604152d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 shutdown_connections 2024-04-25T08:43:17.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.783+0000 7fe60b000640 1 -- 172.21.15.46:0/1514827700 wait complete. 2024-04-25T08:43:17.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29d1200640 1 Processor -- start 2024-04-25T08:43:17.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29d1200640 1 -- start start 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29d1200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f29cc145310 0x7f29cc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29d1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29cc06dc70 0x7f29cc0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29d1200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc0cd930 con 0x7f29cc0cf1b0 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29d1200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc0cdbd0 con 0x7f29cc06dc70 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29d1200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc0cdeb0 con 0x7f29cc145310 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29c3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29cc06dc70 0x7f29cc0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29c3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29cc06dc70 0x7f29cc0cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51846/0 (socket says 172.21.15.46:51846) 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29c3e00640 1 -- 172.21.15.46:0/487182406 learned_addr learned my addr 172.21.15.46:0/487182406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29c9e00640 1 --2- 172.21.15.46:0/487182406 >> v2:172.21.15.82:3300/0 conn(0x7f29cc145310 0x7f29cc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29c3e00640 1 -- 172.21.15.46:0/487182406 >> v2:172.21.15.82:3300/0 conn(0x7f29cc145310 msgr2=0x7f29cc06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29c3e00640 1 --2- 172.21.15.46:0/487182406 >> v2:172.21.15.82:3300/0 conn(0x7f29cc145310 0x7f29cc06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29c3e00640 1 -- 172.21.15.46:0/487182406 >> v1:172.21.15.82:6789/0 conn(0x7f29cc0cf1b0 legacy=0x7f29cc15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29c3e00640 1 -- 172.21.15.46:0/487182406 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc0ce190 con 0x7f29cc06dc70 2024-04-25T08:43:17.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.947+0000 7f29c9e00640 1 --2- 172.21.15.46:0/487182406 >> v2:172.21.15.82:3300/0 conn(0x7f29cc145310 0x7f29cc06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:17.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29c3e00640 1 --2- 172.21.15.46:0/487182406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc06dc70 0x7f29cc0cec70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f29cc072530 tx=0x7f29b401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd85b1bcd29c7af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29c9400640 1 -- 172.21.15.46:0/487182406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40aa650 con 0x7f29cc06dc70 2024-04-25T08:43:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29c9400640 1 -- 172.21.15.46:0/487182406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b40af070 con 0x7f29cc06dc70 2024-04-25T08:43:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29c9400640 1 -- 172.21.15.46:0/487182406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40aaa10 con 0x7f29cc06dc70 2024-04-25T08:43:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29d1200640 1 -- 172.21.15.46:0/487182406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc06dc70 msgr2=0x7f29cc0cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29d1200640 1 --2- 172.21.15.46:0/487182406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc06dc70 0x7f29cc0cec70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f29cc072530 tx=0x7f29b401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29d1200640 1 -- 172.21.15.46:0/487182406 shutdown_connections 2024-04-25T08:43:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29d1200640 1 --2- 172.21.15.46:0/487182406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc06dc70 0x7f29cc0cec70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29d1200640 1 --2- 172.21.15.46:0/487182406 >> v2:172.21.15.82:3300/0 conn(0x7f29cc145310 0x7f29cc06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29d1200640 1 -- 172.21.15.46:0/487182406 >> 172.21.15.46:0/487182406 conn(0x7f29cc06ac10 msgr2=0x7f29cc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29d1200640 1 -- 172.21.15.46:0/487182406 shutdown_connections 2024-04-25T08:43:17.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.951+0000 7f29d1200640 1 -- 172.21.15.46:0/487182406 wait complete. 2024-04-25T08:43:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 Processor -- start 2024-04-25T08:43:17.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 -- start start 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29cc0cf1b0 0x7f29cc13ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc145310 0x7f29cc13c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29cc137b00 0x7f29cc137fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc15d920 con 0x7f29cc145310 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc15d7a0 con 0x7f29cc0cf1b0 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc168510 con 0x7f29cc137b00 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc145310 0x7f29cc13c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc145310 0x7f29cc13c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51862/0 (socket says 172.21.15.46:51862) 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c3e00640 1 -- 172.21.15.46:0/65524680 learned_addr learned my addr 172.21.15.46:0/65524680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c9e00640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29cc0cf1b0 0x7f29cc13ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29ca800640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29cc137b00 0x7f29cc137fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c3e00640 1 -- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29cc137b00 msgr2=0x7f29cc137fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c3e00640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29cc137b00 0x7f29cc137fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c3e00640 1 -- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29cc0cf1b0 msgr2=0x7f29cc13ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c3e00640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29cc0cf1b0 0x7f29cc13ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c3e00640 1 -- 172.21.15.46:0/65524680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc13d400 con 0x7f29cc145310 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c3e00640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc145310 0x7f29cc13c100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f29b4019100 tx=0x7f29b401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c2000640 1 -- 172.21.15.46:0/65524680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40bd5f0 con 0x7f29cc145310 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c2000640 1 -- 172.21.15.46:0/65524680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b40a6040 con 0x7f29cc145310 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29c2000640 1 -- 172.21.15.46:0/65524680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40bd9d0 con 0x7f29cc145310 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc13f7f0 con 0x7f29cc145310 2024-04-25T08:43:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.955+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc13fdb0 con 0x7f29cc145310 2024-04-25T08:43:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.959+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc069010 con 0x7f29cc145310 2024-04-25T08:43:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.959+0000 7f29c2000640 1 -- 172.21.15.46:0/65524680 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29b40b33f0 con 0x7f29cc145310 2024-04-25T08:43:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.959+0000 7f29c2000640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29a8066f60 0x7f29a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.959+0000 7f29c2000640 1 -- 172.21.15.46:0/65524680 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f29b411f680 con 0x7f29cc145310 2024-04-25T08:43:17.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.959+0000 7f29c9e00640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29a8066f60 0x7f29a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.959+0000 7f29c9e00640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29a8066f60 0x7f29a8069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f29bc006e90 tx=0x7f29bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:17.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:17.963+0000 7f29c2000640 1 -- 172.21.15.46:0/65524680 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29b4016110 con 0x7f29cc145310 2024-04-25T08:43:18.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.123+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29cc0b1810 con 0x7f29cc145310 2024-04-25T08:43:18.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.123+0000 7f29c2000640 1 -- 172.21.15.46:0/65524680 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29b40efba0 con 0x7f29cc145310 2024-04-25T08:43:18.117 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29a8066f60 msgr2=0x7f29a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29a8066f60 0x7f29a8069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f29bc006e90 tx=0x7f29bc008040 comp rx=0 tx=0).stop 2024-04-25T08:43:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc145310 msgr2=0x7f29cc13c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc145310 0x7f29cc13c100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f29b4019100 tx=0x7f29b401ad30 comp rx=0 tx=0).stop 2024-04-25T08:43:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 shutdown_connections 2024-04-25T08:43:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29a8066f60 0x7f29a8069420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f29cc137b00 0x7f29cc137fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29cc145310 0x7f29cc13c100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 --2- 172.21.15.46:0/65524680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29cc0cf1b0 0x7f29cc13ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 >> 172.21.15.46:0/65524680 conn(0x7f29cc06ac10 msgr2=0x7f29cc0b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 shutdown_connections 2024-04-25T08:43:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.127+0000 7f29d1200640 1 -- 172.21.15.46:0/65524680 wait complete. 2024-04-25T08:43:18.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:18.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd18c00640 1 Processor -- start 2024-04-25T08:43:18.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd18c00640 1 -- start start 2024-04-25T08:43:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd18c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd140cd4d0 0x7fcd14168690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd18c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1409a780 con 0x7fcd1406f4d0 2024-04-25T08:43:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd18c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1409aa60 con 0x7fcd1415dd50 2024-04-25T08:43:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd18c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd140708c0 con 0x7fcd140cd4d0 2024-04-25T08:43:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcd1406f4d0 0x7fcd140cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52898/0 (socket says 172.21.15.46:52898) 2024-04-25T08:43:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11e00640 1 -- 172.21.15.46:0/2024069463 learned_addr learned my addr 172.21.15.46:0/2024069463 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967337262 0 0) 0x7fcd1409a780 con 0x7fcd1406f4d0 2024-04-25T08:43:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd12800640 1 --2- 172.21.15.46:0/2024069463 >> v2:172.21.15.82:3300/0 conn(0x7fcd140cd4d0 0x7fcd14168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccf8003530 con 0x7fcd1406f4d0 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1509815323 0 0) 0x7fccf8003530 con 0x7fcd1406f4d0 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 >> v2:172.21.15.82:3300/0 conn(0x7fcd140cd4d0 msgr2=0x7fcd14168690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 --2- 172.21.15.46:0/2024069463 >> v2:172.21.15.82:3300/0 conn(0x7fcd140cd4d0 0x7fcd14168690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 >> v1:172.21.15.82:6789/0 conn(0x7fcd1415dd50 legacy=0x7fcd140cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1409ad40 con 0x7fcd1406f4d0 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fcd04001680 con 0x7fcd1406f4d0 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 >> v1:172.21.15.46:6789/0 conn(0x7fcd1406f4d0 legacy=0x7fcd140cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccf8003a30 0x7fccf8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fccf8004810 0x7fccf8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf80051d0 0x7fccf800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140708c0 con 0x7fccf80051d0 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf800c100 con 0x7fccf8003a30 2024-04-25T08:43:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf800c300 con 0x7fccf8004810 2024-04-25T08:43:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd12800640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf80051d0 0x7fccf800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd09400640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fccf8004810 0x7fccf8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.291+0000 7fcd11e00640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccf8003a30 0x7fccf8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd12800640 1 -- 172.21.15.46:0/2024069463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fccf8004810 msgr2=0x7fccf8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd12800640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fccf8004810 0x7fccf8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd12800640 1 -- 172.21.15.46:0/2024069463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccf8003a30 msgr2=0x7fccf8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd12800640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccf8003a30 0x7fccf8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd12800640 1 -- 172.21.15.46:0/2024069463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf8003530 con 0x7fccf80051d0 2024-04-25T08:43:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd12800640 1 -- 172.21.15.46:0/2024069463 reap_dead start 2024-04-25T08:43:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd12800640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf80051d0 0x7fccf800b9d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009fc0 tx=0x7fcd0c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ad1db96e41e4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c009c10 con 0x7fccf80051d0 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c0a5d10 con 0x7fccf80051d0 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd11400640 1 -- 172.21.15.46:0/2024069463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c003be0 con 0x7fccf80051d0 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd18c00640 1 -- 172.21.15.46:0/2024069463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf80051d0 msgr2=0x7fccf800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd18c00640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf80051d0 0x7fccf800b9d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009fc0 tx=0x7fcd0c01ccc0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd18c00640 1 -- 172.21.15.46:0/2024069463 shutdown_connections 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd18c00640 1 --2- 172.21.15.46:0/2024069463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccf80051d0 0x7fccf800b9d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd18c00640 1 -- 172.21.15.46:0/2024069463 >> 172.21.15.46:0/2024069463 conn(0x7fcd140b3160 msgr2=0x7fcd140b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd18c00640 1 -- 172.21.15.46:0/2024069463 shutdown_connections 2024-04-25T08:43:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.295+0000 7fcd18c00640 1 -- 172.21.15.46:0/2024069463 wait complete. 2024-04-25T08:43:18.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 Processor -- start 2024-04-25T08:43:18.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 -- start start 2024-04-25T08:43:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccf80051d0 0x7fcd140c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd140c7820 0x7fcd140c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1415a6f0 0x7fcd1415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1409a410 con 0x7fcd1415a6f0 2024-04-25T08:43:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140cbfa0 con 0x7fccf80051d0 2024-04-25T08:43:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1409a590 con 0x7fcd140c7820 2024-04-25T08:43:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd12800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1415a6f0 0x7fcd1415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd12800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1415a6f0 0x7fcd1415ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56196/0 (socket says 172.21.15.46:56196) 2024-04-25T08:43:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd12800640 1 -- 172.21.15.46:0/957789811 learned_addr learned my addr 172.21.15.46:0/957789811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd09400640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd140c7820 0x7fcd140c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd11e00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccf80051d0 0x7fcd140c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd12800640 1 -- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd140c7820 msgr2=0x7fcd140c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd12800640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd140c7820 0x7fcd140c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd12800640 1 -- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccf80051d0 msgr2=0x7fcd140c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd12800640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccf80051d0 0x7fcd140c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd12800640 1 -- 172.21.15.46:0/957789811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1415b090 con 0x7fcd1415a6f0 2024-04-25T08:43:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd12800640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1415a6f0 0x7fcd1415ab50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0aeb40 tx=0x7fcd0c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd0aa00640 1 -- 172.21.15.46:0/957789811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c009c10 con 0x7fcd1415a6f0 2024-04-25T08:43:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd0aa00640 1 -- 172.21.15.46:0/957789811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c0bd070 con 0x7fcd1415a6f0 2024-04-25T08:43:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd0aa00640 1 -- 172.21.15.46:0/957789811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0039a0 con 0x7fcd1415a6f0 2024-04-25T08:43:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd140b4710 con 0x7fcd1415a6f0 2024-04-25T08:43:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.299+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd140b4cd0 con 0x7fcd1415a6f0 2024-04-25T08:43:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.303+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd14069010 con 0x7fcd1415a6f0 2024-04-25T08:43:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.303+0000 7fcd0aa00640 1 -- 172.21.15.46:0/957789811 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd0c003b40 con 0x7fcd1415a6f0 2024-04-25T08:43:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.303+0000 7fcd0aa00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fccf0066f60 0x7fccf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.303+0000 7fcd0aa00640 1 -- 172.21.15.46:0/957789811 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcd0c120f30 con 0x7fcd1415a6f0 2024-04-25T08:43:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.303+0000 7fcd11e00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fccf0066f60 0x7fccf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.303+0000 7fcd11e00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fccf0066f60 0x7fccf0069420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd04007fd0 tx=0x7fcd04007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.307+0000 7fcd0aa00640 1 -- 172.21.15.46:0/957789811 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd0c0169d0 con 0x7fcd1415a6f0 2024-04-25T08:43:18.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd1415c500 con 0x7fcd1415a6f0 2024-04-25T08:43:18.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd0aa00640 1 -- 172.21.15.46:0/957789811 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd0c0f1bc0 con 0x7fcd1415a6f0 2024-04-25T08:43:18.463 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fccf0066f60 msgr2=0x7fccf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fccf0066f60 0x7fccf0069420 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd04007fd0 tx=0x7fcd04007ca0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1415a6f0 msgr2=0x7fcd1415ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1415a6f0 0x7fcd1415ab50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0aeb40 tx=0x7fcd0c009750 comp rx=0 tx=0).stop 2024-04-25T08:43:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 shutdown_connections 2024-04-25T08:43:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fccf0066f60 0x7fccf0069420 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1415a6f0 0x7fcd1415ab50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd140c7820 0x7fcd140c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 --2- 172.21.15.46:0/957789811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccf80051d0 0x7fcd140c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 >> 172.21.15.46:0/957789811 conn(0x7fcd140b3160 msgr2=0x7fcd140cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 shutdown_connections 2024-04-25T08:43:18.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.471+0000 7fcd18c00640 1 -- 172.21.15.46:0/957789811 wait complete. 2024-04-25T08:43:18.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:18.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb527200640 1 Processor -- start 2024-04-25T08:43:18.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb527200640 1 -- start start 2024-04-25T08:43:18.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb527200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5201440d0 0x7fb5200cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb527200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb52006f150 0x7fb5200b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb527200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52015e440 con 0x7fb520143990 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb527200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52015ed30 con 0x7fb52006f150 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb527200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52015e6e0 con 0x7fb5201440d0 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb525200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb52006f150 0x7fb5200b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb525200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb52006f150 0x7fb5200b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56204/0 (socket says 172.21.15.46:56204) 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.639+0000 7fb525200640 1 -- 172.21.15.46:0/69062183 learned_addr learned my addr 172.21.15.46:0/69062183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb517400640 1 --2- 172.21.15.46:0/69062183 >> v2:172.21.15.82:3300/0 conn(0x7fb5201440d0 0x7fb5200cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb525200640 1 -- 172.21.15.46:0/69062183 >> v2:172.21.15.82:3300/0 conn(0x7fb5201440d0 msgr2=0x7fb5200cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb525200640 1 --2- 172.21.15.46:0/69062183 >> v2:172.21.15.82:3300/0 conn(0x7fb5201440d0 0x7fb5200cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb525200640 1 -- 172.21.15.46:0/69062183 >> v1:172.21.15.82:6789/0 conn(0x7fb520143990 legacy=0x7fb52006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:18.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb525200640 1 -- 172.21.15.46:0/69062183 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52015e9c0 con 0x7fb52006f150 2024-04-25T08:43:18.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb525200640 1 --2- 172.21.15.46:0/69062183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52006f150 0x7fb5200b2720 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb520070c80 tx=0x7fb51801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66263944165c0aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:18.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb51f400640 1 -- 172.21.15.46:0/69062183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180b0070 con 0x7fb52006f150 2024-04-25T08:43:18.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb51f400640 1 -- 172.21.15.46:0/69062183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5180ab600 con 0x7fb52006f150 2024-04-25T08:43:18.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb51f400640 1 -- 172.21.15.46:0/69062183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180ab9d0 con 0x7fb52006f150 2024-04-25T08:43:18.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb527200640 1 -- 172.21.15.46:0/69062183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52006f150 msgr2=0x7fb5200b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.643+0000 7fb527200640 1 --2- 172.21.15.46:0/69062183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52006f150 0x7fb5200b2720 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb520070c80 tx=0x7fb51801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 -- 172.21.15.46:0/69062183 shutdown_connections 2024-04-25T08:43:18.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 --2- 172.21.15.46:0/69062183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52006f150 0x7fb5200b2720 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 --2- 172.21.15.46:0/69062183 >> v2:172.21.15.82:3300/0 conn(0x7fb5201440d0 0x7fb5200cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 -- 172.21.15.46:0/69062183 >> 172.21.15.46:0/69062183 conn(0x7fb52006ac10 msgr2=0x7fb52006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:18.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 -- 172.21.15.46:0/69062183 shutdown_connections 2024-04-25T08:43:18.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 -- 172.21.15.46:0/69062183 wait complete. 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 Processor -- start 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 -- start start 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb52006f150 0x7fb52015da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb520143990 0x7fb520134780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5201440d0 0x7fb520155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200b35f0 con 0x7fb5201440d0 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200b3470 con 0x7fb520143990 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb527200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5200b3770 con 0x7fb52006f150 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb525200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5201440d0 0x7fb520155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb525200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5201440d0 0x7fb520155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56220/0 (socket says 172.21.15.46:56220) 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb525200640 1 -- 172.21.15.46:0/3180754272 learned_addr learned my addr 172.21.15.46:0/3180754272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb517400640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb520143990 0x7fb520134780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb51fe00640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb52006f150 0x7fb52015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb525200640 1 -- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb52006f150 msgr2=0x7fb52015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb525200640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb52006f150 0x7fb52015da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb525200640 1 -- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb520143990 msgr2=0x7fb520134780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb525200640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb520143990 0x7fb520134780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.647+0000 7fb525200640 1 -- 172.21.15.46:0/3180754272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520134cc0 con 0x7fb5201440d0 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb525200640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5201440d0 0x7fb520155f40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb5180ae040 tx=0x7fb51801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb51cc00640 1 -- 172.21.15.46:0/3180754272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180b0040 con 0x7fb5201440d0 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb51cc00640 1 -- 172.21.15.46:0/3180754272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5180bc070 con 0x7fb5201440d0 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb51cc00640 1 -- 172.21.15.46:0/3180754272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5180b6660 con 0x7fb5201440d0 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5201315f0 con 0x7fb5201440d0 2024-04-25T08:43:18.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb520131e30 con 0x7fb5201440d0 2024-04-25T08:43:18.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb520069010 con 0x7fb5201440d0 2024-04-25T08:43:18.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb51cc00640 1 -- 172.21.15.46:0/3180754272 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5180b6800 con 0x7fb5201440d0 2024-04-25T08:43:18.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb51cc00640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb51cc00640 1 -- 172.21.15.46:0/3180754272 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb51811f6f0 con 0x7fb5201440d0 2024-04-25T08:43:18.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.651+0000 7fb51fe00640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.655+0000 7fb51fe00640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb510005680 tx=0x7fb510003dd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:18.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.655+0000 7fb51cc00640 1 -- 172.21.15.46:0/3180754272 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5201315f0 con 0x7fb5201440d0 2024-04-25T08:43:18.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.815+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb52013a9b0 con 0x7fb5201440d0 2024-04-25T08:43:18.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb51cc00640 1 -- 172.21.15.46:0/3180754272 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb5180160d0 con 0x7fb5201440d0 2024-04-25T08:43:18.811 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:18.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 msgr2=0x7fb4fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb510005680 tx=0x7fb510003dd0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5201440d0 msgr2=0x7fb520155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5201440d0 0x7fb520155f40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb5180ae040 tx=0x7fb51801ad20 comp rx=0 tx=0).stop 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 shutdown_connections 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5201440d0 0x7fb520155f40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb520143990 0x7fb520134780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 --2- 172.21.15.46:0/3180754272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb52006f150 0x7fb52015da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 >> 172.21.15.46:0/3180754272 conn(0x7fb52006ac10 msgr2=0x7fb52006ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 shutdown_connections 2024-04-25T08:43:18.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.819+0000 7fb527200640 1 -- 172.21.15.46:0/3180754272 wait complete. 2024-04-25T08:43:18.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:43:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ffe400640 1 Processor -- start 2024-04-25T08:43:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ffe400640 1 -- start start 2024-04-25T08:43:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ffe400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ff8157b00 0x7f7ff80b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ffe400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ff80cc150 0x7f7ff80cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ffe400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff80b25a0 con 0x7f7ff80ce250 2024-04-25T08:43:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ffe400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff80b2eb0 con 0x7f7ff80cc150 2024-04-25T08:43:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ffe400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff80cc6f0 con 0x7f7ff8157b00 2024-04-25T08:43:18.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ff7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7ff80ce250 0x7f7ff815a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52904/0 (socket says 172.21.15.46:52904) 2024-04-25T08:43:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ff7e00640 1 -- 172.21.15.46:0/3307376965 learned_addr learned my addr 172.21.15.46:0/3307376965 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7feea00640 1 --2- 172.21.15.46:0/3307376965 >> v2:172.21.15.46:3300/0 conn(0x7f7ff80cc150 0x7f7ff80cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ff7400640 1 --2- 172.21.15.46:0/3307376965 >> v2:172.21.15.82:3300/0 conn(0x7f7ff8157b00 0x7f7ff80b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ff6a00640 1 -- 172.21.15.46:0/3307376965 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919080254 0 0) 0x7f7ff80b25a0 con 0x7f7ff80ce250 2024-04-25T08:43:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7ff6a00640 1 -- 172.21.15.46:0/3307376965 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fdc003530 con 0x7f7ff80ce250 2024-04-25T08:43:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7feea00640 1 -- 172.21.15.46:0/3307376965 >> v2:172.21.15.82:3300/0 conn(0x7f7ff8157b00 msgr2=0x7f7ff80b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7feea00640 1 --2- 172.21.15.46:0/3307376965 >> v2:172.21.15.82:3300/0 conn(0x7f7ff8157b00 0x7f7ff80b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7feea00640 1 -- 172.21.15.46:0/3307376965 >> v1:172.21.15.46:6789/0 conn(0x7f7ff80ce250 legacy=0x7f7ff815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.983+0000 7f7feea00640 1 -- 172.21.15.46:0/3307376965 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80b2860 con 0x7f7ff80cc150 2024-04-25T08:43:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7feea00640 1 --2- 172.21.15.46:0/3307376965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 0x7f7ff80cdd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7ff804ffb0 tx=0x7f7fe001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c64a343b4a46307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ff6a00640 1 -- 172.21.15.46:0/3307376965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00b0070 con 0x7f7ff80cc150 2024-04-25T08:43:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ff6a00640 1 -- 172.21.15.46:0/3307376965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe00ab600 con 0x7f7ff80cc150 2024-04-25T08:43:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ff6a00640 1 -- 172.21.15.46:0/3307376965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00ab9d0 con 0x7f7ff80cc150 2024-04-25T08:43:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 -- 172.21.15.46:0/3307376965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 msgr2=0x7f7ff80cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 --2- 172.21.15.46:0/3307376965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 0x7f7ff80cdd10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7ff804ffb0 tx=0x7f7fe001c330 comp rx=0 tx=0).stop 2024-04-25T08:43:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 -- 172.21.15.46:0/3307376965 shutdown_connections 2024-04-25T08:43:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 --2- 172.21.15.46:0/3307376965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 0x7f7ff80cdd10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 --2- 172.21.15.46:0/3307376965 >> v2:172.21.15.82:3300/0 conn(0x7f7ff8157b00 0x7f7ff80b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 -- 172.21.15.46:0/3307376965 >> 172.21.15.46:0/3307376965 conn(0x7f7ff80b12a0 msgr2=0x7f7ff80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 -- 172.21.15.46:0/3307376965 shutdown_connections 2024-04-25T08:43:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 -- 172.21.15.46:0/3307376965 wait complete. 2024-04-25T08:43:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 Processor -- start 2024-04-25T08:43:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.987+0000 7f7ffe400640 1 -- start start 2024-04-25T08:43:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ffe400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 0x7f7ff8166ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ffe400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ff80ce250 0x7f7ff8141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ffe400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ff8157b00 0x7f7ff8162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ffe400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff815ace0 con 0x7f7ff80cc150 2024-04-25T08:43:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ffe400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff815ab60 con 0x7f7ff8157b00 2024-04-25T08:43:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ffe400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff815ae60 con 0x7f7ff80ce250 2024-04-25T08:43:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 0x7f7ff8166ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 0x7f7ff8166ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56230/0 (socket says 172.21.15.46:56230) 2024-04-25T08:43:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7400640 1 -- 172.21.15.46:0/2716466681 learned_addr learned my addr 172.21.15.46:0/2716466681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7feea00640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ff80ce250 0x7f7ff8141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7e00640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ff8157b00 0x7f7ff8162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7400640 1 -- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ff80ce250 msgr2=0x7f7ff8141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7400640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ff80ce250 0x7f7ff8141ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7400640 1 -- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ff8157b00 msgr2=0x7f7ff8162ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7400640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ff8157b00 0x7f7ff8162ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7400640 1 -- 172.21.15.46:0/2716466681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff81fae70 con 0x7f7ff80cc150 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ff7400640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 0x7f7ff8166ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d620 tx=0x7f7fe800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7fefe00640 1 -- 172.21.15.46:0/2716466681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800b620 con 0x7f7ff80cc150 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7fefe00640 1 -- 172.21.15.46:0/2716466681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe8019070 con 0x7f7ff80cc150 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7fefe00640 1 -- 172.21.15.46:0/2716466681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800b9e0 con 0x7f7ff80cc150 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff8167430 con 0x7f7ff80cc150 2024-04-25T08:43:18.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff81fa390 con 0x7f7ff80cc150 2024-04-25T08:43:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff8069080 con 0x7f7ff80cc150 2024-04-25T08:43:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.991+0000 7f7fefe00640 1 -- 172.21.15.46:0/2716466681 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7fe800bb80 con 0x7f7ff80cc150 2024-04-25T08:43:18.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.995+0000 7f7fefe00640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd4066f60 0x7f7fd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.995+0000 7f7fefe00640 1 -- 172.21.15.46:0/2716466681 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7fe80857e0 con 0x7f7ff80cc150 2024-04-25T08:43:18.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.995+0000 7f7feea00640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd4066f60 0x7f7fd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:18.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.995+0000 7f7feea00640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd4066f60 0x7f7fd4069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00ad000 tx=0x7f7fe0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:18.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:18.999+0000 7f7fefe00640 1 -- 172.21.15.46:0/2716466681 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7fe80563e0 con 0x7f7ff80cc150 2024-04-25T08:43:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.171+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ff8157680 con 0x7f7ff80cc150 2024-04-25T08:43:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7fefe00640 1 -- 172.21.15.46:0/2716466681 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7fe8055ce0 con 0x7f7ff80cc150 2024-04-25T08:43:19.166 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:43:19.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd4066f60 msgr2=0x7f7fd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd4066f60 0x7f7fd4069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00ad000 tx=0x7f7fe0008040 comp rx=0 tx=0).stop 2024-04-25T08:43:19.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 msgr2=0x7f7ff8166ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 0x7f7ff8166ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d620 tx=0x7f7fe800dae0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 shutdown_connections 2024-04-25T08:43:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7fd4066f60 0x7f7fd4069420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ff8157b00 0x7f7ff8162ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ff80ce250 0x7f7ff8141ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 --2- 172.21.15.46:0/2716466681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ff80cc150 0x7f7ff8166ef0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 >> 172.21.15.46:0/2716466681 conn(0x7f7ff80b12a0 msgr2=0x7f7ff80b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 shutdown_connections 2024-04-25T08:43:19.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.175+0000 7f7ffe400640 1 -- 172.21.15.46:0/2716466681 wait complete. 2024-04-25T08:43:19.187 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T08:43:19.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:19.188 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T08:43:19.213 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:43:19.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:19.214 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:43:19.270 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:43:19.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c7800640 1 Processor -- start 2024-04-25T08:43:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c7800640 1 -- start start 2024-04-25T08:43:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c7800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27c015fa70 0x7f27c009a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c7800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c0162e00 con 0x7f27c00c4d10 2024-04-25T08:43:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c7800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c01630e0 con 0x7f27c0162360 2024-04-25T08:43:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c7800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c0099ba0 con 0x7f27c015fa70 2024-04-25T08:43:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27c015fa70 0x7f27c009a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c5800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f27c00c4d10 0x7f27c006bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52906/0 (socket says 172.21.15.46:52906) 2024-04-25T08:43:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c5800640 1 -- 172.21.15.46:0/3726146800 learned_addr learned my addr 172.21.15.46:0/3726146800 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c4e00640 1 -- 172.21.15.46:0/3726146800 >> v1:172.21.15.82:6789/0 conn(0x7f27c0162360 legacy=0x7f27c00c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c4e00640 1 -- 172.21.15.46:0/3726146800 >> v1:172.21.15.46:6789/0 conn(0x7f27c00c4d10 legacy=0x7f27c006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27c4e00640 1 -- 172.21.15.46:0/3726146800 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0099e80 con 0x7f27c015fa70 2024-04-25T08:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.419+0000 7f27bf400640 1 -- 172.21.15.46:0/3726146800 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216438189 0 0) 0x7f27c0162e00 con 0x7f27c00c4d10 2024-04-25T08:43:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27c4e00640 1 --2- 172.21.15.46:0/3726146800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c015fa70 0x7f27c009a3a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f27b40097c0 tx=0x7f27b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0756ea859b7b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27bf400640 1 -- 172.21.15.46:0/3726146800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b40aa620 con 0x7f27c015fa70 2024-04-25T08:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27bf400640 1 -- 172.21.15.46:0/3726146800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b40af070 con 0x7f27c015fa70 2024-04-25T08:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27bf400640 1 -- 172.21.15.46:0/3726146800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b40aa9e0 con 0x7f27c015fa70 2024-04-25T08:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27c7800640 1 -- 172.21.15.46:0/3726146800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c015fa70 msgr2=0x7f27c009a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27c7800640 1 --2- 172.21.15.46:0/3726146800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c015fa70 0x7f27c009a3a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f27b40097c0 tx=0x7f27b401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27c7800640 1 -- 172.21.15.46:0/3726146800 shutdown_connections 2024-04-25T08:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27c7800640 1 --2- 172.21.15.46:0/3726146800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c015fa70 0x7f27c009a3a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27c7800640 1 -- 172.21.15.46:0/3726146800 >> 172.21.15.46:0/3726146800 conn(0x7f27c00b3120 msgr2=0x7f27c00b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27c7800640 1 -- 172.21.15.46:0/3726146800 shutdown_connections 2024-04-25T08:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.423+0000 7f27c7800640 1 -- 172.21.15.46:0/3726146800 wait complete. 2024-04-25T08:43:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 Processor -- start 2024-04-25T08:43:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 -- start start 2024-04-25T08:43:19.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27c00c4d10 0x7f27c015c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27c015fa70 0x7f27c00c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c0162360 0x7f27c00c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c009aef0 con 0x7f27c0162360 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c0170330 con 0x7f27c00c4d10 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c015ca20 con 0x7f27c015fa70 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c0162360 0x7f27c00c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c0162360 0x7f27c00c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56244/0 (socket says 172.21.15.46:56244) 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c5800640 1 -- 172.21.15.46:0/151944738 learned_addr learned my addr 172.21.15.46:0/151944738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c4e00640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27c00c4d10 0x7f27c015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27bfe00640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27c015fa70 0x7f27c00c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c5800640 1 -- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27c015fa70 msgr2=0x7f27c00c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c5800640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27c015fa70 0x7f27c00c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c5800640 1 -- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27c00c4d10 msgr2=0x7f27c015c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c5800640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27c00c4d10 0x7f27c015c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c5800640 1 -- 172.21.15.46:0/151944738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c015ccc0 con 0x7f27c0162360 2024-04-25T08:43:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c5800640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c0162360 0x7f27c00c8c40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c780 tx=0x7f27b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27bcc00640 1 -- 172.21.15.46:0/151944738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8018070 con 0x7f27c0162360 2024-04-25T08:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27bcc00640 1 -- 172.21.15.46:0/151944738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b800a640 con 0x7f27c0162360 2024-04-25T08:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27bcc00640 1 -- 172.21.15.46:0/151944738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b800aab0 con 0x7f27c0162360 2024-04-25T08:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c015cfa0 con 0x7f27c0162360 2024-04-25T08:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.427+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c015d560 con 0x7f27c0162360 2024-04-25T08:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.431+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c0069080 con 0x7f27c0162360 2024-04-25T08:43:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.431+0000 7f27bcc00640 1 -- 172.21.15.46:0/151944738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27b800ac50 con 0x7f27c0162360 2024-04-25T08:43:19.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.431+0000 7f27bcc00640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27a0066f60 0x7f27a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.431+0000 7f27bcc00640 1 -- 172.21.15.46:0/151944738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f27b80849c0 con 0x7f27c0162360 2024-04-25T08:43:19.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.431+0000 7f27c4e00640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27a0066f60 0x7f27a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.431+0000 7f27c4e00640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27a0066f60 0x7f27a0069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27b40ae000 tx=0x7f27b40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:19.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.435+0000 7f27bcc00640 1 -- 172.21.15.46:0/151944738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27b8054f70 con 0x7f27c0162360 2024-04-25T08:43:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.631+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27c00b0950 con 0x7f27c0162360 2024-04-25T08:43:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27bcc00640 1 -- 172.21.15.46:0/151944738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f27b8059010 con 0x7f27c0162360 2024-04-25T08:43:19.626 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:19.627 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:38.824358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:43:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27a0066f60 msgr2=0x7f27a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27a0066f60 0x7f27a0069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27b40ae000 tx=0x7f27b40a7000 comp rx=0 tx=0).stop 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c0162360 msgr2=0x7f27c00c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c0162360 0x7f27c00c8c40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c780 tx=0x7f27b800cc40 comp rx=0 tx=0).stop 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 shutdown_connections 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27a0066f60 0x7f27a0069420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c0162360 0x7f27c00c8c40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27c015fa70 0x7f27c00c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 --2- 172.21.15.46:0/151944738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27c00c4d10 0x7f27c015c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 >> 172.21.15.46:0/151944738 conn(0x7f27c00b3120 msgr2=0x7f27c00b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 shutdown_connections 2024-04-25T08:43:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.635+0000 7f27c7800640 1 -- 172.21.15.46:0/151944738 wait complete. 2024-04-25T08:43:19.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.795+0000 7f7df1e00640 1 Processor -- start 2024-04-25T08:43:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.795+0000 7f7df1e00640 1 -- start start 2024-04-25T08:43:19.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7df1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7dec160f50 0x7f7dec15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7df1e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 0x7f7dec0ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7df1e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dec161330 con 0x7f7dec0adc50 2024-04-25T08:43:19.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7df1e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dec06d680 con 0x7f7dec160f50 2024-04-25T08:43:19.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7df1e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec0ae810 con 0x7f7dec06f110 2024-04-25T08:43:19.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7deaa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7dec160f50 0x7f7dec15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7deaa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7dec160f50 0x7f7dec15f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56252/0 (socket says 172.21.15.46:56252) 2024-04-25T08:43:19.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7deaa00640 1 -- 172.21.15.46:0/2803625735 learned_addr learned my addr 172.21.15.46:0/2803625735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:19.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7deb400640 1 --2- 172.21.15.46:0/2803625735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 0x7f7dec0ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7deaa00640 1 -- 172.21.15.46:0/2803625735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 msgr2=0x7f7dec0ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:19.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7deaa00640 1 --2- 172.21.15.46:0/2803625735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 0x7f7dec0ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7deaa00640 1 -- 172.21.15.46:0/2803625735 >> v1:172.21.15.82:6789/0 conn(0x7f7dec0adc50 legacy=0x7f7dec06ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.799+0000 7f7deaa00640 1 -- 172.21.15.46:0/2803625735 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0aeaf0 con 0x7f7dec160f50 2024-04-25T08:43:19.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7deaa00640 1 --2- 172.21.15.46:0/2803625735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 0x7f7dec15f1a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009870 tx=0x7f7ddc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56616629cf706558 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:19.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7de9600640 1 -- 172.21.15.46:0/2803625735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0a7070 con 0x7f7dec160f50 2024-04-25T08:43:19.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7de9600640 1 -- 172.21.15.46:0/2803625735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc0b0070 con 0x7f7dec160f50 2024-04-25T08:43:19.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7de9600640 1 -- 172.21.15.46:0/2803625735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0ab890 con 0x7f7dec160f50 2024-04-25T08:43:19.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 -- 172.21.15.46:0/2803625735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 msgr2=0x7f7dec15f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 --2- 172.21.15.46:0/2803625735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 0x7f7dec15f1a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009870 tx=0x7f7ddc01c560 comp rx=0 tx=0).stop 2024-04-25T08:43:19.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 -- 172.21.15.46:0/2803625735 shutdown_connections 2024-04-25T08:43:19.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 --2- 172.21.15.46:0/2803625735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 0x7f7dec0ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 --2- 172.21.15.46:0/2803625735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 0x7f7dec15f1a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 -- 172.21.15.46:0/2803625735 >> 172.21.15.46:0/2803625735 conn(0x7f7dec0b12a0 msgr2=0x7f7dec0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:19.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 -- 172.21.15.46:0/2803625735 shutdown_connections 2024-04-25T08:43:19.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 -- 172.21.15.46:0/2803625735 wait complete. 2024-04-25T08:43:19.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 Processor -- start 2024-04-25T08:43:19.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 -- start start 2024-04-25T08:43:19.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 0x7f7dec140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dec0adc50 0x7f7dec0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 0x7f7dec13e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec06e4a0 con 0x7f7dec160f50 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec06e320 con 0x7f7dec0adc50 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.803+0000 7f7df1e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec06e620 con 0x7f7dec06f110 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 0x7f7dec13e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 0x7f7dec13e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56258/0 (socket says 172.21.15.46:56258) 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deb400640 1 -- 172.21.15.46:0/3788417293 learned_addr learned my addr 172.21.15.46:0/3788417293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7dea000640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dec0adc50 0x7f7dec0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deaa00640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 0x7f7dec140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deb400640 1 -- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 msgr2=0x7f7dec140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deb400640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 0x7f7dec140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deb400640 1 -- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dec0adc50 msgr2=0x7f7dec0cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deb400640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dec0adc50 0x7f7dec0cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deb400640 1 -- 172.21.15.46:0/3788417293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0cdff0 con 0x7f7dec160f50 2024-04-25T08:43:19.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7deb400640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 0x7f7dec13e400 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7de4007de0 tx=0x7f7de400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:19.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7de3400640 1 -- 172.21.15.46:0/3788417293 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4014620 con 0x7f7dec160f50 2024-04-25T08:43:19.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7de3400640 1 -- 172.21.15.46:0/3788417293 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de4019070 con 0x7f7dec160f50 2024-04-25T08:43:19.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7de3400640 1 -- 172.21.15.46:0/3788417293 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4014a20 con 0x7f7dec160f50 2024-04-25T08:43:19.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dec13e940 con 0x7f7dec160f50 2024-04-25T08:43:19.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dec0ce550 con 0x7f7dec160f50 2024-04-25T08:43:19.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dec141aa0 con 0x7f7dec160f50 2024-04-25T08:43:19.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.807+0000 7f7de3400640 1 -- 172.21.15.46:0/3788417293 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7de4014bc0 con 0x7f7dec160f50 2024-04-25T08:43:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.811+0000 7f7de3400640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dcc066f90 0x7f7dcc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.811+0000 7f7de3400640 1 -- 172.21.15.46:0/3788417293 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7de4085c90 con 0x7f7dec160f50 2024-04-25T08:43:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.811+0000 7f7deaa00640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dcc066f90 0x7f7dcc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.811+0000 7f7deaa00640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dcc066f90 0x7f7dcc069450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0af000 tx=0x7f7ddc0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:19.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.815+0000 7f7de3400640 1 -- 172.21.15.46:0/3788417293 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dec13e940 con 0x7f7dec160f50 2024-04-25T08:43:19.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.975+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dec0b23c0 con 0x7f7dec160f50 2024-04-25T08:43:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.975+0000 7f7de3400640 1 -- 172.21.15.46:0/3788417293 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7f7dec0cdff0 con 0x7f7dec160f50 2024-04-25T08:43:19.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:19.970 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dcc066f90 msgr2=0x7f7dcc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dcc066f90 0x7f7dcc069450 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0af000 tx=0x7f7ddc0a7000 comp rx=0 tx=0).stop 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 msgr2=0x7f7dec13e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 0x7f7dec13e400 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7de4007de0 tx=0x7f7de400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 shutdown_connections 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dcc066f90 0x7f7dcc069450 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec160f50 0x7f7dec13e400 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7dec0adc50 0x7f7dec0cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 --2- 172.21.15.46:0/3788417293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7dec06f110 0x7f7dec140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 >> 172.21.15.46:0/3788417293 conn(0x7f7dec0b12a0 msgr2=0x7f7dec0b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 shutdown_connections 2024-04-25T08:43:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:19.979+0000 7f7df1e00640 1 -- 172.21.15.46:0/3788417293 wait complete. 2024-04-25T08:43:19.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:18.424873+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:19.729069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:42:23.413095+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:43:19.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:20.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:20.102 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbfa7e00640 1 Processor -- start 2024-04-25T08:43:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbfa7e00640 1 -- start start 2024-04-25T08:43:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbfa7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbfa806ac10 0x7fbfa806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbfa7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa80b0370 0x7fbfa80b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbfa7e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa815e010 con 0x7fbfa8160cd0 2024-04-25T08:43:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbfa7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa815e2b0 con 0x7fbfa806ac10 2024-04-25T08:43:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbfa7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa815e590 con 0x7fbfa80b0370 2024-04-25T08:43:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbf9fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbfa806ac10 0x7fbfa806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbf9fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbfa806ac10 0x7fbfa806b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56268/0 (socket says 172.21.15.46:56268) 2024-04-25T08:43:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbf9fe00640 1 -- 172.21.15.46:0/846845346 learned_addr learned my addr 172.21.15.46:0/846845346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.143+0000 7fbfa7400640 1 --2- 172.21.15.46:0/846845346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa80b0370 0x7fbfa80b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.147+0000 7fbf9fe00640 1 -- 172.21.15.46:0/846845346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa80b0370 msgr2=0x7fbfa80b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.147+0000 7fbf9fe00640 1 --2- 172.21.15.46:0/846845346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa80b0370 0x7fbfa80b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.147+0000 7fbf9fe00640 1 -- 172.21.15.46:0/846845346 >> v1:172.21.15.82:6789/0 conn(0x7fbfa8160cd0 legacy=0x7fbfa80b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.147+0000 7fbf9fe00640 1 -- 172.21.15.46:0/846845346 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa815e870 con 0x7fbfa806ac10 2024-04-25T08:43:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.147+0000 7fbf9fe00640 1 --2- 172.21.15.46:0/846845346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa806ac10 0x7fbfa806b0c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009870 tx=0x7fbf9001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407d9234e61e696c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.147+0000 7fbfa6000640 1 -- 172.21.15.46:0/846845346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900b2070 con 0x7fbfa806ac10 2024-04-25T08:43:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.147+0000 7fbfa6000640 1 -- 172.21.15.46:0/846845346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf900a9070 con 0x7fbfa806ac10 2024-04-25T08:43:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.147+0000 7fbfa6000640 1 -- 172.21.15.46:0/846845346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf900ad880 con 0x7fbfa806ac10 2024-04-25T08:43:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 -- 172.21.15.46:0/846845346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa806ac10 msgr2=0x7fbfa806b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 --2- 172.21.15.46:0/846845346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa806ac10 0x7fbfa806b0c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbf90009870 tx=0x7fbf9001c560 comp rx=0 tx=0).stop 2024-04-25T08:43:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 -- 172.21.15.46:0/846845346 shutdown_connections 2024-04-25T08:43:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 --2- 172.21.15.46:0/846845346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa80b0370 0x7fbfa80b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 --2- 172.21.15.46:0/846845346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa806ac10 0x7fbfa806b0c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 -- 172.21.15.46:0/846845346 >> 172.21.15.46:0/846845346 conn(0x7fbfa80b5aa0 msgr2=0x7fbfa8099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 -- 172.21.15.46:0/846845346 shutdown_connections 2024-04-25T08:43:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 -- 172.21.15.46:0/846845346 wait complete. 2024-04-25T08:43:20.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 Processor -- start 2024-04-25T08:43:20.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 -- start start 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa806ac10 0x7fbfa80afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa80b0370 0x7fbfa80ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa8160cd0 0x7fbfa81fa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa8072aa0 con 0x7fbfa8160cd0 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa8072920 con 0x7fbfa80b0370 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa815f8f0 con 0x7fbfa806ac10 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa8160cd0 0x7fbfa81fa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa6a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa806ac10 0x7fbfa80afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa8160cd0 0x7fbfa81fa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56284/0 (socket says 172.21.15.46:56284) 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7400640 1 -- 172.21.15.46:0/530167392 learned_addr learned my addr 172.21.15.46:0/530167392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbf9fe00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa80b0370 0x7fbfa80ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7400640 1 -- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa806ac10 msgr2=0x7fbfa80afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7400640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa806ac10 0x7fbfa80afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7400640 1 -- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa80b0370 msgr2=0x7fbfa80ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7400640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa80b0370 0x7fbfa80ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.151+0000 7fbfa7400640 1 -- 172.21.15.46:0/530167392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa81facd0 con 0x7fbfa8160cd0 2024-04-25T08:43:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbfa7400640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa8160cd0 0x7fbfa81fa230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009dd0 tx=0x7fbfa000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbf9ea00640 1 -- 172.21.15.46:0/530167392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0018020 con 0x7fbfa8160cd0 2024-04-25T08:43:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbf9ea00640 1 -- 172.21.15.46:0/530167392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa0014480 con 0x7fbfa8160cd0 2024-04-25T08:43:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbf9ea00640 1 -- 172.21.15.46:0/530167392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0014870 con 0x7fbfa8160cd0 2024-04-25T08:43:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa81fafb0 con 0x7fbfa8160cd0 2024-04-25T08:43:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa80af770 con 0x7fbfa8160cd0 2024-04-25T08:43:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa8069010 con 0x7fbfa8160cd0 2024-04-25T08:43:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbf9ea00640 1 -- 172.21.15.46:0/530167392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfa001b020 con 0x7fbfa8160cd0 2024-04-25T08:43:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbf9ea00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf84066f60 0x7fbf84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbf9ea00640 1 -- 172.21.15.46:0/530167392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbfa0085ea0 con 0x7fbfa8160cd0 2024-04-25T08:43:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.155+0000 7fbfa6a00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf84066f60 0x7fbf84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.159+0000 7fbfa6a00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf84066f60 0x7fbf84069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf98001770 tx=0x7fbf98009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:20.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.159+0000 7fbf9ea00640 1 -- 172.21.15.46:0/530167392 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfa0022070 con 0x7fbfa8160cd0 2024-04-25T08:43:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbfa8072700 con 0x7fbfa8160cd0 2024-04-25T08:43:20.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbf9ea00640 1 -- 172.21.15.46:0/530167392 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbfa0010040 con 0x7fbfa8160cd0 2024-04-25T08:43:20.315 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf84066f60 msgr2=0x7fbf84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf84066f60 0x7fbf84069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf98001770 tx=0x7fbf98009040 comp rx=0 tx=0).stop 2024-04-25T08:43:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa8160cd0 msgr2=0x7fbfa81fa230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa8160cd0 0x7fbfa81fa230 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009dd0 tx=0x7fbfa000c490 comp rx=0 tx=0).stop 2024-04-25T08:43:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 shutdown_connections 2024-04-25T08:43:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf84066f60 0x7fbf84069420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfa8160cd0 0x7fbfa81fa230 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfa80b0370 0x7fbfa80ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 --2- 172.21.15.46:0/530167392 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbfa806ac10 0x7fbfa80afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 >> 172.21.15.46:0/530167392 conn(0x7fbfa80b5aa0 msgr2=0x7fbfa815c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 shutdown_connections 2024-04-25T08:43:20.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.323+0000 7fbfa7e00640 1 -- 172.21.15.46:0/530167392 wait complete. 2024-04-25T08:43:20.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64ffe00640 1 Processor -- start 2024-04-25T08:43:20.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64ffe00640 1 -- start start 2024-04-25T08:43:20.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64ffe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64f8071980 0x7f64f815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64ffe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f815eab0 con 0x7f64f813d0b0 2024-04-25T08:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64ffe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f813d490 con 0x7f64f8071980 2024-04-25T08:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64ffe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f815e500 con 0x7f64f815db50 2024-04-25T08:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64fde00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64fca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64f8071980 0x7f64f815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64fde00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f815dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:46414/0 (socket says 172.21.15.46:46414) 2024-04-25T08:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64fde00640 1 -- 172.21.15.46:0/1517572096 learned_addr learned my addr 172.21.15.46:0/1517572096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64fde00640 1 -- 172.21.15.46:0/1517572096 >> v2:172.21.15.82:3300/0 conn(0x7f64f8071980 msgr2=0x7f64f815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64fde00640 1 --2- 172.21.15.46:0/1517572096 >> v2:172.21.15.82:3300/0 conn(0x7f64f8071980 0x7f64f815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64fde00640 1 -- 172.21.15.46:0/1517572096 >> v1:172.21.15.82:6789/0 conn(0x7f64f813d0b0 legacy=0x7f64f8071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64fde00640 1 -- 172.21.15.46:0/1517572096 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f815e7d0 con 0x7f64f815db50 2024-04-25T08:43:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.487+0000 7f64fca00640 1 --2- 172.21.15.46:0/1517572096 >> v2:172.21.15.82:3300/0 conn(0x7f64f8071980 0x7f64f815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64fde00640 1 --2- 172.21.15.46:0/1517572096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f815dfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009e50 tx=0x7f64f001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2be734c080d4717a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64f7e00640 1 -- 172.21.15.46:0/1517572096 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f00a5770 con 0x7f64f815db50 2024-04-25T08:43:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64f7e00640 1 -- 172.21.15.46:0/1517572096 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f00a5910 con 0x7f64f815db50 2024-04-25T08:43:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64f7e00640 1 -- 172.21.15.46:0/1517572096 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f00a5ce0 con 0x7f64f815db50 2024-04-25T08:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64ffe00640 1 -- 172.21.15.46:0/1517572096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 msgr2=0x7f64f815dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64ffe00640 1 --2- 172.21.15.46:0/1517572096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f815dfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009e50 tx=0x7f64f001ca30 comp rx=0 tx=0).stop 2024-04-25T08:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64ffe00640 1 -- 172.21.15.46:0/1517572096 shutdown_connections 2024-04-25T08:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64ffe00640 1 --2- 172.21.15.46:0/1517572096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f815dfc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64ffe00640 1 --2- 172.21.15.46:0/1517572096 >> v2:172.21.15.82:3300/0 conn(0x7f64f8071980 0x7f64f815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64ffe00640 1 -- 172.21.15.46:0/1517572096 >> 172.21.15.46:0/1517572096 conn(0x7f64f806ac10 msgr2=0x7f64f806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64ffe00640 1 -- 172.21.15.46:0/1517572096 shutdown_connections 2024-04-25T08:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.491+0000 7f64ffe00640 1 -- 172.21.15.46:0/1517572096 wait complete. 2024-04-25T08:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 Processor -- start 2024-04-25T08:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 -- start start 2024-04-25T08:43:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8071980 0x7f64f8138760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f813d0b0 0x7f64f8164060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f8157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80b28a0 con 0x7f64f8071980 2024-04-25T08:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80b2720 con 0x7f64f813d0b0 2024-04-25T08:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f80b2a20 con 0x7f64f815db50 2024-04-25T08:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8071980 0x7f64f8138760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8071980 0x7f64f8138760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56290/0 (socket says 172.21.15.46:56290) 2024-04-25T08:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fd400640 1 -- 172.21.15.46:0/2349735353 learned_addr learned my addr 172.21.15.46:0/2349735353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:20.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fca00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f813d0b0 0x7f64f8164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fde00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f8157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fd400640 1 -- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 msgr2=0x7f64f8157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fd400640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f8157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fd400640 1 -- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f813d0b0 msgr2=0x7f64f8164060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fd400640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f813d0b0 0x7f64f8164060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fd400640 1 -- 172.21.15.46:0/2349735353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f81645a0 con 0x7f64f8071980 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fde00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f8157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64fd400640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8071980 0x7f64f8138760 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c420 tx=0x7f64ec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64f5600640 1 -- 172.21.15.46:0/2349735353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0052d0 con 0x7f64f8071980 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64f5600640 1 -- 172.21.15.46:0/2349735353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec005470 con 0x7f64f8071980 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64f5600640 1 -- 172.21.15.46:0/2349735353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec007260 con 0x7f64f8071980 2024-04-25T08:43:20.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f8138df0 con 0x7f64f8071980 2024-04-25T08:43:20.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.495+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f81393b0 con 0x7f64f8071980 2024-04-25T08:43:20.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.499+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f8069010 con 0x7f64f8071980 2024-04-25T08:43:20.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.499+0000 7f64f5600640 1 -- 172.21.15.46:0/2349735353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64ec007400 con 0x7f64f8071980 2024-04-25T08:43:20.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.499+0000 7f64f5600640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64d8066f60 0x7f64d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.499+0000 7f64f5600640 1 -- 172.21.15.46:0/2349735353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f64ec084a00 con 0x7f64f8071980 2024-04-25T08:43:20.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.499+0000 7f64fca00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64d8066f60 0x7f64d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.499+0000 7f64fca00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64d8066f60 0x7f64d8069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f64e400a720 tx=0x7f64e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:20.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.503+0000 7f64f5600640 1 -- 172.21.15.46:0/2349735353 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64ec055570 con 0x7f64f8071980 2024-04-25T08:43:20.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.667+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64f80cce90 con 0x7f64f8071980 2024-04-25T08:43:20.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.667+0000 7f64f5600640 1 -- 172.21.15.46:0/2349735353 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64ec054e70 con 0x7f64f8071980 2024-04-25T08:43:20.662 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:20.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64d8066f60 msgr2=0x7f64d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64d8066f60 0x7f64d8069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f64e400a720 tx=0x7f64e4008040 comp rx=0 tx=0).stop 2024-04-25T08:43:20.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8071980 msgr2=0x7f64f8138760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8071980 0x7f64f8138760 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c420 tx=0x7f64ec00c8e0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 shutdown_connections 2024-04-25T08:43:20.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64d8066f60 0x7f64d8069420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f815db50 0x7f64f8157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f813d0b0 0x7f64f8164060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 --2- 172.21.15.46:0/2349735353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f8071980 0x7f64f8138760 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 >> 172.21.15.46:0/2349735353 conn(0x7f64f806ac10 msgr2=0x7f64f806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:20.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 shutdown_connections 2024-04-25T08:43:20.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.671+0000 7f64ffe00640 1 -- 172.21.15.46:0/2349735353 wait complete. 2024-04-25T08:43:20.684 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:20.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3fd200640 1 Processor -- start 2024-04-25T08:43:20.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3fd200640 1 -- start start 2024-04-25T08:43:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3fd200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 0x7fe3f806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3fd200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3f80d0390 0x7fe3f80cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3fd200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f809acb0 con 0x7fe3f809a410 2024-04-25T08:43:20.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3fd200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f80b3dc0 con 0x7fe3f80d0390 2024-04-25T08:43:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3fd200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80cf680 con 0x7fe3f806c730 2024-04-25T08:43:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3f80d0390 0x7fe3f80cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe3f809a410 0x7fe3f806cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52922/0 (socket says 172.21.15.46:52922) 2024-04-25T08:43:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f6a00640 1 -- 172.21.15.46:0/3810245666 learned_addr learned my addr 172.21.15.46:0/3810245666 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3f80d0390 0x7fe3f80cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56304/0 (socket says 172.21.15.46:56304) 2024-04-25T08:43:20.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f7400640 1 --2- 172.21.15.46:0/3810245666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 0x7fe3f806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f7e00640 1 -- 172.21.15.46:0/3810245666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 msgr2=0x7fe3f806bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f7e00640 1 --2- 172.21.15.46:0/3810245666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 0x7fe3f806bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f7e00640 1 -- 172.21.15.46:0/3810245666 >> v1:172.21.15.46:6789/0 conn(0x7fe3f809a410 legacy=0x7fe3f806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f7e00640 1 -- 172.21.15.46:0/3810245666 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f809a9d0 con 0x7fe3f80d0390 2024-04-25T08:43:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f7400640 1 --2- 172.21.15.46:0/3810245666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 0x7fe3f806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:20.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.835+0000 7fe3f6000640 1 -- 172.21.15.46:0/3810245666 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680663929 0 0) 0x7fe3f809acb0 con 0x7fe3f809a410 2024-04-25T08:43:20.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3f7e00640 1 --2- 172.21.15.46:0/3810245666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 0x7fe3f80cbab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009d90 tx=0x7fe3f001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c81e99718182651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:20.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3f6000640 1 -- 172.21.15.46:0/3810245666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00b2070 con 0x7fe3f80d0390 2024-04-25T08:43:20.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3f6000640 1 -- 172.21.15.46:0/3810245666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f00a5a20 con 0x7fe3f80d0390 2024-04-25T08:43:20.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3f6000640 1 -- 172.21.15.46:0/3810245666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00a5df0 con 0x7fe3f80d0390 2024-04-25T08:43:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3fd200640 1 -- 172.21.15.46:0/3810245666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 msgr2=0x7fe3f80cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3fd200640 1 --2- 172.21.15.46:0/3810245666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 0x7fe3f80cbab0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009d90 tx=0x7fe3f001c940 comp rx=0 tx=0).stop 2024-04-25T08:43:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3fd200640 1 -- 172.21.15.46:0/3810245666 shutdown_connections 2024-04-25T08:43:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3fd200640 1 --2- 172.21.15.46:0/3810245666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 0x7fe3f80cbab0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3fd200640 1 --2- 172.21.15.46:0/3810245666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 0x7fe3f806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3fd200640 1 -- 172.21.15.46:0/3810245666 >> 172.21.15.46:0/3810245666 conn(0x7fe3f80b2f50 msgr2=0x7fe3f80b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3fd200640 1 -- 172.21.15.46:0/3810245666 shutdown_connections 2024-04-25T08:43:20.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.839+0000 7fe3fd200640 1 -- 172.21.15.46:0/3810245666 wait complete. 2024-04-25T08:43:20.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3fd200640 1 Processor -- start 2024-04-25T08:43:20.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3fd200640 1 -- start start 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3fd200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 0x7fe3f81f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3fd200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3f809a410 0x7fe3f814acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3fd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 0x7fe3f814b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3fd200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f815dd60 con 0x7fe3f80d0390 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3fd200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f80cc400 con 0x7fe3f809a410 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3fd200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f816b040 con 0x7fe3f806c730 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 0x7fe3f814b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 0x7fe3f814b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56314/0 (socket says 172.21.15.46:56314) 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7e00640 1 -- 172.21.15.46:0/1789590454 learned_addr learned my addr 172.21.15.46:0/1789590454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7400640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 0x7fe3f81f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f6a00640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3f809a410 0x7fe3f814acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7e00640 1 -- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 msgr2=0x7fe3f81f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7e00640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 0x7fe3f81f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7e00640 1 -- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3f809a410 msgr2=0x7fe3f814acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7e00640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3f809a410 0x7fe3f814acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7e00640 1 -- 172.21.15.46:0/1789590454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f814a230 con 0x7fe3f80d0390 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f6a00640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3f809a410 0x7fe3f814acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3f7e00640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 0x7fe3f814b230 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009d90 tx=0x7fe3f00196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3ef400640 1 -- 172.21.15.46:0/1789590454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00b2040 con 0x7fe3f80d0390 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3ef400640 1 -- 172.21.15.46:0/1789590454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f00ad580 con 0x7fe3f80d0390 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3ef400640 1 -- 172.21.15.46:0/1789590454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00ad930 con 0x7fe3f80d0390 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.843+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f814ceb0 con 0x7fe3f80d0390 2024-04-25T08:43:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.847+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f814a4f0 con 0x7fe3f80d0390 2024-04-25T08:43:20.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.847+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f8069010 con 0x7fe3f80d0390 2024-04-25T08:43:20.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.847+0000 7fe3ef400640 1 -- 172.21.15.46:0/1789590454 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3f00c1020 con 0x7fe3f80d0390 2024-04-25T08:43:20.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.847+0000 7fe3ef400640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3d8066f60 0x7fe3d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:20.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.847+0000 7fe3ef400640 1 -- 172.21.15.46:0/1789590454 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe3f011ff60 con 0x7fe3f80d0390 2024-04-25T08:43:20.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.847+0000 7fe3f7400640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3d8066f60 0x7fe3d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:20.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.851+0000 7fe3f7400640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3d8066f60 0x7fe3d8069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8006fd0 tx=0x7fe3e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:20.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:20.851+0000 7fe3ef400640 1 -- 172.21.15.46:0/1789590454 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3f0016560 con 0x7fe3f80d0390 2024-04-25T08:43:21.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.011+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3f806c510 con 0x7fe3f80d0390 2024-04-25T08:43:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.011+0000 7fe3ef400640 1 -- 172.21.15.46:0/1789590454 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3f00a59f0 con 0x7fe3f80d0390 2024-04-25T08:43:21.006 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3d8066f60 msgr2=0x7fe3d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3d8066f60 0x7fe3d8069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8006fd0 tx=0x7fe3e8008040 comp rx=0 tx=0).stop 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 msgr2=0x7fe3f814b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 0x7fe3f814b230 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009d90 tx=0x7fe3f00196e0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 shutdown_connections 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3d8066f60 0x7fe3d8069420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f80d0390 0x7fe3f814b230 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3f809a410 0x7fe3f814acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 --2- 172.21.15.46:0/1789590454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3f806c730 0x7fe3f81f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 >> 172.21.15.46:0/1789590454 conn(0x7fe3f80b2f50 msgr2=0x7fe3f80700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:21.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 shutdown_connections 2024-04-25T08:43:21.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.015+0000 7fe3fd200640 1 -- 172.21.15.46:0/1789590454 wait complete. 2024-04-25T08:43:21.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:43:21.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b8600640 1 Processor -- start 2024-04-25T08:43:21.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b8600640 1 -- start start 2024-04-25T08:43:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b8600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95b015b870 0x7f95b006d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b8600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f95b00cdb70 0x7f95b00cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00ce020 0x7f95b00cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b8600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b00ce5f0 con 0x7f95b015b870 2024-04-25T08:43:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b8600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b00ce8d0 con 0x7f95b00cdb70 2024-04-25T08:43:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00cebb0 con 0x7f95b00ce020 2024-04-25T08:43:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95b015b870 0x7f95b006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95b015b870 0x7f95b006d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56322/0 (socket says 172.21.15.46:56322) 2024-04-25T08:43:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b5c00640 1 -- 172.21.15.46:0/4270485179 learned_addr learned my addr 172.21.15.46:0/4270485179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b6600640 1 --2- 172.21.15.46:0/4270485179 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00ce020 0x7f95b00cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b5200640 1 --2- 172.21.15.46:0/4270485179 >> v2:172.21.15.82:3300/0 conn(0x7f95b00cdb70 0x7f95b00cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b5c00640 1 -- 172.21.15.46:0/4270485179 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00ce020 msgr2=0x7f95b00cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b5c00640 1 --2- 172.21.15.46:0/4270485179 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00ce020 0x7f95b00cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b5c00640 1 -- 172.21.15.46:0/4270485179 >> v2:172.21.15.82:3300/0 conn(0x7f95b00cdb70 msgr2=0x7f95b00cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b5c00640 1 --2- 172.21.15.46:0/4270485179 >> v2:172.21.15.82:3300/0 conn(0x7f95b00cdb70 0x7f95b00cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.179+0000 7f95b5c00640 1 -- 172.21.15.46:0/4270485179 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00b2900 con 0x7f95b015b870 2024-04-25T08:43:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b5c00640 1 --2- 172.21.15.46:0/4270485179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b015b870 0x7f95b006d870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f95b00ae990 tx=0x7f95a401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de873e3e07f56e51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95afe00640 1 -- 172.21.15.46:0/4270485179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40b2070 con 0x7f95b015b870 2024-04-25T08:43:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95afe00640 1 -- 172.21.15.46:0/4270485179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a40a9070 con 0x7f95b015b870 2024-04-25T08:43:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95afe00640 1 -- 172.21.15.46:0/4270485179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40ad880 con 0x7f95b015b870 2024-04-25T08:43:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b8600640 1 -- 172.21.15.46:0/4270485179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b015b870 msgr2=0x7f95b006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b8600640 1 --2- 172.21.15.46:0/4270485179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b015b870 0x7f95b006d870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f95b00ae990 tx=0x7f95a401c490 comp rx=0 tx=0).stop 2024-04-25T08:43:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b8600640 1 -- 172.21.15.46:0/4270485179 shutdown_connections 2024-04-25T08:43:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b8600640 1 --2- 172.21.15.46:0/4270485179 >> v2:172.21.15.82:3300/0 conn(0x7f95b00cdb70 0x7f95b00cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b8600640 1 --2- 172.21.15.46:0/4270485179 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00ce020 0x7f95b00cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b8600640 1 --2- 172.21.15.46:0/4270485179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b015b870 0x7f95b006d870 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b8600640 1 -- 172.21.15.46:0/4270485179 >> 172.21.15.46:0/4270485179 conn(0x7f95b006ac10 msgr2=0x7f95b006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b8600640 1 -- 172.21.15.46:0/4270485179 shutdown_connections 2024-04-25T08:43:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.183+0000 7f95b8600640 1 -- 172.21.15.46:0/4270485179 wait complete. 2024-04-25T08:43:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 Processor -- start 2024-04-25T08:43:21.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 -- start start 2024-04-25T08:43:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00cdb70 0x7f95b00b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b00ce020 0x7f95b00b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b015b870 0x7f95b0139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0071e10 con 0x7f95b00ce020 2024-04-25T08:43:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0071c90 con 0x7f95b015b870 2024-04-25T08:43:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0071f90 con 0x7f95b00cdb70 2024-04-25T08:43:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b00ce020 0x7f95b00b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b00ce020 0x7f95b00b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56328/0 (socket says 172.21.15.46:56328) 2024-04-25T08:43:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5200640 1 -- 172.21.15.46:0/731418281 learned_addr learned my addr 172.21.15.46:0/731418281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5c00640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00cdb70 0x7f95b00b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b6600640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b015b870 0x7f95b0139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5200640 1 -- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00cdb70 msgr2=0x7f95b00b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5200640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00cdb70 0x7f95b00b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5200640 1 -- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b015b870 msgr2=0x7f95b0139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5200640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b015b870 0x7f95b0139dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5200640 1 -- 172.21.15.46:0/731418281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b014a300 con 0x7f95b00ce020 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b6600640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b015b870 0x7f95b0139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b5200640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b00ce020 0x7f95b00b1eb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f959c00d800 tx=0x7f959c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95ad600640 1 -- 172.21.15.46:0/731418281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00b6c0 con 0x7f95b00ce020 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95ad600640 1 -- 172.21.15.46:0/731418281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c019070 con 0x7f95b00ce020 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95ad600640 1 -- 172.21.15.46:0/731418281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00ba60 con 0x7f95b00ce020 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b014a5e0 con 0x7f95b00ce020 2024-04-25T08:43:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.187+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b014aba0 con 0x7f95b00ce020 2024-04-25T08:43:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.191+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b0069010 con 0x7f95b00ce020 2024-04-25T08:43:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.191+0000 7f95ad600640 1 -- 172.21.15.46:0/731418281 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f959c00bc00 con 0x7f95b00ce020 2024-04-25T08:43:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.191+0000 7f95ad600640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9590066f60 0x7f9590069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.191+0000 7f95ad600640 1 -- 172.21.15.46:0/731418281 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f959c085910 con 0x7f95b00ce020 2024-04-25T08:43:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.191+0000 7f95b5c00640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9590066f60 0x7f9590069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.191+0000 7f95b5c00640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9590066f60 0x7f9590069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95a40af000 tx=0x7f95a40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.195+0000 7f95ad600640 1 -- 172.21.15.46:0/731418281 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f959c056480 con 0x7f95b00ce020 2024-04-25T08:43:21.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.355+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f95b014afa0 con 0x7f95b00ce020 2024-04-25T08:43:21.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.355+0000 7f95ad600640 1 -- 172.21.15.46:0/731418281 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f959c055d80 con 0x7f95b00ce020 2024-04-25T08:43:21.349 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:21.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9590066f60 msgr2=0x7f9590069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9590066f60 0x7f9590069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f95a40af000 tx=0x7f95a40a7000 comp rx=0 tx=0).stop 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b00ce020 msgr2=0x7f95b00b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b00ce020 0x7f95b00b1eb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f959c00d800 tx=0x7f959c00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 shutdown_connections 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9590066f60 0x7f9590069420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95b015b870 0x7f95b0139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95b00ce020 0x7f95b00b1eb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 --2- 172.21.15.46:0/731418281 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f95b00cdb70 0x7f95b00b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 >> 172.21.15.46:0/731418281 conn(0x7f95b006ac10 msgr2=0x7f95b01522a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 shutdown_connections 2024-04-25T08:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.359+0000 7f95b8600640 1 -- 172.21.15.46:0/731418281 wait complete. 2024-04-25T08:43:21.371 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.519+0000 7fd79be00640 1 Processor -- start 2024-04-25T08:43:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.519+0000 7fd79be00640 1 -- start start 2024-04-25T08:43:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd79be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd79415dbd0 0x7fd7940b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd79be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7940b0c30 0x7fd794070c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd79be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd7940711d0 0x7fd79415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd79be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7940b5af0 con 0x7fd79415dbd0 2024-04-25T08:43:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd79be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7940b5250 con 0x7fd7940711d0 2024-04-25T08:43:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd79be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b5530 con 0x7fd7940b0c30 2024-04-25T08:43:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd799400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd79415dbd0 0x7fd7940b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd799400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd79415dbd0 0x7fd7940b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56330/0 (socket says 172.21.15.46:56330) 2024-04-25T08:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd799400640 1 -- 172.21.15.46:0/3363698068 learned_addr learned my addr 172.21.15.46:0/3363698068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd798a00640 1 --2- 172.21.15.46:0/3363698068 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7940b0c30 0x7fd794070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd799e00640 1 --2- 172.21.15.46:0/3363698068 >> v2:172.21.15.82:3300/0 conn(0x7fd7940711d0 0x7fd79415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd799400640 1 -- 172.21.15.46:0/3363698068 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7940b0c30 msgr2=0x7fd794070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd799400640 1 --2- 172.21.15.46:0/3363698068 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7940b0c30 0x7fd794070c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd799400640 1 -- 172.21.15.46:0/3363698068 >> v2:172.21.15.82:3300/0 conn(0x7fd7940711d0 msgr2=0x7fd79415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd799400640 1 --2- 172.21.15.46:0/3363698068 >> v2:172.21.15.82:3300/0 conn(0x7fd7940711d0 0x7fd79415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd799400640 1 -- 172.21.15.46:0/3363698068 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940b5810 con 0x7fd79415dbd0 2024-04-25T08:43:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.523+0000 7fd798a00640 1 --2- 172.21.15.46:0/3363698068 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7940b0c30 0x7fd794070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:21.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd799400640 1 --2- 172.21.15.46:0/3363698068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd79415dbd0 0x7fd7940b06f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd79406bba0 tx=0x7fd78801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3b3e3419f754f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd793e00640 1 -- 172.21.15.46:0/3363698068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880b0070 con 0x7fd79415dbd0 2024-04-25T08:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd793e00640 1 -- 172.21.15.46:0/3363698068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7880ab600 con 0x7fd79415dbd0 2024-04-25T08:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd793e00640 1 -- 172.21.15.46:0/3363698068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880ab9d0 con 0x7fd79415dbd0 2024-04-25T08:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 -- 172.21.15.46:0/3363698068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd79415dbd0 msgr2=0x7fd7940b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 --2- 172.21.15.46:0/3363698068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd79415dbd0 0x7fd7940b06f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd79406bba0 tx=0x7fd78801c520 comp rx=0 tx=0).stop 2024-04-25T08:43:21.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 -- 172.21.15.46:0/3363698068 shutdown_connections 2024-04-25T08:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 --2- 172.21.15.46:0/3363698068 >> v2:172.21.15.82:3300/0 conn(0x7fd7940711d0 0x7fd79415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 --2- 172.21.15.46:0/3363698068 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7940b0c30 0x7fd794070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 --2- 172.21.15.46:0/3363698068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd79415dbd0 0x7fd7940b06f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 -- 172.21.15.46:0/3363698068 >> 172.21.15.46:0/3363698068 conn(0x7fd79406f4f0 msgr2=0x7fd79406f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 -- 172.21.15.46:0/3363698068 shutdown_connections 2024-04-25T08:43:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 -- 172.21.15.46:0/3363698068 wait complete. 2024-04-25T08:43:21.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 Processor -- start 2024-04-25T08:43:21.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 -- start start 2024-04-25T08:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7940711d0 0x7fd7940d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.527+0000 7fd79be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7940b0c30 0x7fd7940cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd79be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd79415dbd0 0x7fd79414f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd79be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79406cda0 con 0x7fd7940711d0 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd79be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940b1010 con 0x7fd7940b0c30 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd79be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79406cf20 con 0x7fd79415dbd0 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7940711d0 0x7fd7940d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7940711d0 0x7fd7940d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56346/0 (socket says 172.21.15.46:56346) 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799400640 1 -- 172.21.15.46:0/3039826462 learned_addr learned my addr 172.21.15.46:0/3039826462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799e00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd79415dbd0 0x7fd79414f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd798a00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7940b0c30 0x7fd7940cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799400640 1 -- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd79415dbd0 msgr2=0x7fd79414f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799400640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd79415dbd0 0x7fd79414f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799400640 1 -- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7940b0c30 msgr2=0x7fd7940cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799400640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7940b0c30 0x7fd7940cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799400640 1 -- 172.21.15.46:0/3039826462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940d1080 con 0x7fd7940711d0 2024-04-25T08:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd799400640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7940711d0 0x7fd7940d0b40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd7880ae040 tx=0x7fd78801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd791600640 1 -- 172.21.15.46:0/3039826462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880b0040 con 0x7fd7940711d0 2024-04-25T08:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd791600640 1 -- 172.21.15.46:0/3039826462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7880bc070 con 0x7fd7940711d0 2024-04-25T08:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd791600640 1 -- 172.21.15.46:0/3039826462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880b6880 con 0x7fd7940711d0 2024-04-25T08:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7940cb2c0 con 0x7fd7940711d0 2024-04-25T08:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7940cad00 con 0x7fd7940711d0 2024-04-25T08:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd794069080 con 0x7fd7940711d0 2024-04-25T08:43:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.531+0000 7fd791600640 1 -- 172.21.15.46:0/3039826462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7880b6a20 con 0x7fd7940711d0 2024-04-25T08:43:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.535+0000 7fd791600640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd774066f60 0x7fd774069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.535+0000 7fd791600640 1 -- 172.21.15.46:0/3039826462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd78811fa30 con 0x7fd7940711d0 2024-04-25T08:43:21.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.535+0000 7fd798a00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd774066f60 0x7fd774069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.535+0000 7fd798a00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd774066f60 0x7fd774069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd79415f530 tx=0x7fd780008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.535+0000 7fd791600640 1 -- 172.21.15.46:0/3039826462 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7940cb2c0 con 0x7fd7940711d0 2024-04-25T08:43:21.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd794157bd0 con 0x7fd7940711d0 2024-04-25T08:43:21.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd791600640 1 -- 172.21.15.46:0/3039826462 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fd788016110 con 0x7fd7940711d0 2024-04-25T08:43:21.728 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:21.728 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:38.824358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:43:21.729 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-25T08:43:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd774066f60 msgr2=0x7fd774069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd774066f60 0x7fd774069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd79415f530 tx=0x7fd780008040 comp rx=0 tx=0).stop 2024-04-25T08:43:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7940711d0 msgr2=0x7fd7940d0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7940711d0 0x7fd7940d0b40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd7880ae040 tx=0x7fd78801ad60 comp rx=0 tx=0).stop 2024-04-25T08:43:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 shutdown_connections 2024-04-25T08:43:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd774066f60 0x7fd774069420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd79415dbd0 0x7fd79414f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7940b0c30 0x7fd7940cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 --2- 172.21.15.46:0/3039826462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7940711d0 0x7fd7940d0b40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.735+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 >> 172.21.15.46:0/3039826462 conn(0x7fd79406f4f0 msgr2=0x7fd7940b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.739+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 shutdown_connections 2024-04-25T08:43:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.739+0000 7fd79be00640 1 -- 172.21.15.46:0/3039826462 wait complete. 2024-04-25T08:43:21.750 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a8ae00640 1 Processor -- start 2024-04-25T08:43:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a8ae00640 1 -- start start 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a8ae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a8415ef20 0x7f6a840cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a8ae00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6a840cd6d0 0x7f6a8409a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a8ae00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 0x7f6a84099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a8ae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8415f300 con 0x7f6a8415ef20 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a8ae00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a840cddb0 con 0x7f6a840cd6d0 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a8ae00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a840cc3d0 con 0x7f6a8409ab50 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a83e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a8415ef20 0x7f6a840cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a83e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a8415ef20 0x7f6a840cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56358/0 (socket says 172.21.15.46:56358) 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a83e00640 1 -- 172.21.15.46:0/897460278 learned_addr learned my addr 172.21.15.46:0/897460278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a7be00640 1 --2- 172.21.15.46:0/897460278 >> v2:172.21.15.82:3300/0 conn(0x7f6a840cd6d0 0x7f6a8409a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a88e00640 1 --2- 172.21.15.46:0/897460278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 0x7f6a84099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a83e00640 1 -- 172.21.15.46:0/897460278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 msgr2=0x7f6a84099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a83e00640 1 --2- 172.21.15.46:0/897460278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 0x7f6a84099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a83e00640 1 -- 172.21.15.46:0/897460278 >> v2:172.21.15.82:3300/0 conn(0x7f6a840cd6d0 msgr2=0x7f6a8409a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a83e00640 1 --2- 172.21.15.46:0/897460278 >> v2:172.21.15.82:3300/0 conn(0x7f6a840cd6d0 0x7f6a8409a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.899+0000 7f6a83e00640 1 -- 172.21.15.46:0/897460278 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a840cc6b0 con 0x7f6a8415ef20 2024-04-25T08:43:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a83e00640 1 --2- 172.21.15.46:0/897460278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 0x7f6a840cd190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6a840726a0 tx=0x7f6a7401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386b5c37bf3a0d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a83400640 1 -- 172.21.15.46:0/897460278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a740aa430 con 0x7f6a8415ef20 2024-04-25T08:43:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a83400640 1 -- 172.21.15.46:0/897460278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a740af070 con 0x7f6a8415ef20 2024-04-25T08:43:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a83400640 1 -- 172.21.15.46:0/897460278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a740aa7f0 con 0x7f6a8415ef20 2024-04-25T08:43:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a8ae00640 1 -- 172.21.15.46:0/897460278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 msgr2=0x7f6a840cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/897460278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 0x7f6a840cd190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6a840726a0 tx=0x7f6a7401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a8ae00640 1 -- 172.21.15.46:0/897460278 shutdown_connections 2024-04-25T08:43:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/897460278 >> v2:172.21.15.82:3300/0 conn(0x7f6a840cd6d0 0x7f6a8409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/897460278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 0x7f6a84099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/897460278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 0x7f6a840cd190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a8ae00640 1 -- 172.21.15.46:0/897460278 >> 172.21.15.46:0/897460278 conn(0x7f6a840b3120 msgr2=0x7f6a840b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a8ae00640 1 -- 172.21.15.46:0/897460278 shutdown_connections 2024-04-25T08:43:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.903+0000 7f6a8ae00640 1 -- 172.21.15.46:0/897460278 wait complete. 2024-04-25T08:43:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 Processor -- start 2024-04-25T08:43:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 -- start start 2024-04-25T08:43:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 0x7f6a840c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 0x7f6a840b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a8415a000 0x7f6a840c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8415f540 con 0x7f6a8415ef20 2024-04-25T08:43:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8409a350 con 0x7f6a8415a000 2024-04-25T08:43:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8415f6c0 con 0x7f6a8409ab50 2024-04-25T08:43:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 0x7f6a840b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 0x7f6a840b6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56368/0 (socket says 172.21.15.46:56368) 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a7be00640 1 -- 172.21.15.46:0/2300477482 learned_addr learned my addr 172.21.15.46:0/2300477482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a83e00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 0x7f6a840c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a88e00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a8415a000 0x7f6a840c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a7be00640 1 -- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 msgr2=0x7f6a840c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a7be00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 0x7f6a840c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a7be00640 1 -- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a8415a000 msgr2=0x7f6a840c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a7be00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a8415a000 0x7f6a840c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a7be00640 1 -- 172.21.15.46:0/2300477482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a840b6f50 con 0x7f6a8415ef20 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a7be00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 0x7f6a840b6a10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00e930 tx=0x7f6a6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a80c00640 1 -- 172.21.15.46:0/2300477482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c017070 con 0x7f6a8415ef20 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a80c00640 1 -- 172.21.15.46:0/2300477482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c00ca00 con 0x7f6a8415ef20 2024-04-25T08:43:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a80c00640 1 -- 172.21.15.46:0/2300477482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00cdb0 con 0x7f6a8415ef20 2024-04-25T08:43:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a840c8ce0 con 0x7f6a8415ef20 2024-04-25T08:43:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.907+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a840c92a0 con 0x7f6a8415ef20 2024-04-25T08:43:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.911+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a84069080 con 0x7f6a8415ef20 2024-04-25T08:43:21.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.911+0000 7f6a80c00640 1 -- 172.21.15.46:0/2300477482 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a6c01a070 con 0x7f6a8415ef20 2024-04-25T08:43:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.911+0000 7f6a80c00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a60066f60 0x7f6a60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.911+0000 7f6a80c00640 1 -- 172.21.15.46:0/2300477482 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6a6c085470 con 0x7f6a8415ef20 2024-04-25T08:43:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.911+0000 7f6a83e00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a60066f60 0x7f6a60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:21.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.911+0000 7f6a83e00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a60066f60 0x7f6a60069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6a740ae000 tx=0x7f6a740a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:21.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:21.915+0000 7f6a80c00640 1 -- 172.21.15.46:0/2300477482 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a6c056130 con 0x7f6a8415ef20 2024-04-25T08:43:22.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.079+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a840c78e0 con 0x7f6a8415ef20 2024-04-25T08:43:22.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.079+0000 7f6a80c00640 1 -- 172.21.15.46:0/2300477482 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7f6a6c055a30 con 0x7f6a8415ef20 2024-04-25T08:43:22.072 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:22.072 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a60066f60 msgr2=0x7f6a60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a60066f60 0x7f6a60069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6a740ae000 tx=0x7f6a740a7000 comp rx=0 tx=0).stop 2024-04-25T08:43:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 msgr2=0x7f6a840b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 0x7f6a840b6a10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00e930 tx=0x7f6a6c00edf0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 shutdown_connections 2024-04-25T08:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a60066f60 0x7f6a60069420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a8415a000 0x7f6a840c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a8415ef20 0x7f6a840b6a10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 --2- 172.21.15.46:0/2300477482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a8409ab50 0x7f6a840c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 >> 172.21.15.46:0/2300477482 conn(0x7f6a840b3120 msgr2=0x7f6a84070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 shutdown_connections 2024-04-25T08:43:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.083+0000 7f6a8ae00640 1 -- 172.21.15.46:0/2300477482 wait complete. 2024-04-25T08:43:22.094 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:43:22.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:22.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c9b400640 1 Processor -- start 2024-04-25T08:43:22.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c9b400640 1 -- start start 2024-04-25T08:43:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c9b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c94159230 0x7f2c9415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c9b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c940cbeb0 0x7f2c940cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c9b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2c940cda80 0x7f2c9415f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c9b400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940cc700 con 0x7f2c940cbeb0 2024-04-25T08:43:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c9b400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c940cc9e0 con 0x7f2c940cda80 2024-04-25T08:43:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c9b400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c940cccc0 con 0x7f2c94159230 2024-04-25T08:43:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c93e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c940cbeb0 0x7f2c940cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c93e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c940cbeb0 0x7f2c940cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56380/0 (socket says 172.21.15.46:56380) 2024-04-25T08:43:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c93e00640 1 -- 172.21.15.46:0/375090980 learned_addr learned my addr 172.21.15.46:0/375090980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c99400640 1 --2- 172.21.15.46:0/375090980 >> v2:172.21.15.82:3300/0 conn(0x7f2c940cda80 0x7f2c9415f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c98a00640 1 --2- 172.21.15.46:0/375090980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c94159230 0x7f2c9415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c93e00640 1 -- 172.21.15.46:0/375090980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c94159230 msgr2=0x7f2c9415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c93e00640 1 --2- 172.21.15.46:0/375090980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c94159230 0x7f2c9415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c93e00640 1 -- 172.21.15.46:0/375090980 >> v2:172.21.15.82:3300/0 conn(0x7f2c940cda80 msgr2=0x7f2c9415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c93e00640 1 --2- 172.21.15.46:0/375090980 >> v2:172.21.15.82:3300/0 conn(0x7f2c940cda80 0x7f2c9415f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.243+0000 7f2c93e00640 1 -- 172.21.15.46:0/375090980 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940ccfa0 con 0x7f2c940cbeb0 2024-04-25T08:43:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c93e00640 1 --2- 172.21.15.46:0/375090980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c940cbeb0 0x7f2c940cd540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2c9415c2f0 tx=0x7f2c8001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdaa20bae1fbc2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c93400640 1 -- 172.21.15.46:0/375090980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800b0070 con 0x7f2c940cbeb0 2024-04-25T08:43:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c93400640 1 -- 172.21.15.46:0/375090980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c800ab600 con 0x7f2c940cbeb0 2024-04-25T08:43:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c93400640 1 -- 172.21.15.46:0/375090980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800ab9d0 con 0x7f2c940cbeb0 2024-04-25T08:43:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c9b400640 1 -- 172.21.15.46:0/375090980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c940cbeb0 msgr2=0x7f2c940cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c9b400640 1 --2- 172.21.15.46:0/375090980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c940cbeb0 0x7f2c940cd540 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2c9415c2f0 tx=0x7f2c8001c1d0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c9b400640 1 -- 172.21.15.46:0/375090980 shutdown_connections 2024-04-25T08:43:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c9b400640 1 --2- 172.21.15.46:0/375090980 >> v2:172.21.15.82:3300/0 conn(0x7f2c940cda80 0x7f2c9415f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c9b400640 1 --2- 172.21.15.46:0/375090980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c940cbeb0 0x7f2c940cd540 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c9b400640 1 --2- 172.21.15.46:0/375090980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c94159230 0x7f2c9415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c9b400640 1 -- 172.21.15.46:0/375090980 >> 172.21.15.46:0/375090980 conn(0x7f2c940b12a0 msgr2=0x7f2c940b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c9b400640 1 -- 172.21.15.46:0/375090980 shutdown_connections 2024-04-25T08:43:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.247+0000 7f2c9b400640 1 -- 172.21.15.46:0/375090980 wait complete. 2024-04-25T08:43:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 Processor -- start 2024-04-25T08:43:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 -- start start 2024-04-25T08:43:22.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c940cbeb0 0x7f2c941580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c940cda80 0x7f2c94144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c94159230 0x7f2c9412c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415d380 con 0x7f2c94159230 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415d200 con 0x7f2c940cbeb0 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9415d500 con 0x7f2c940cda80 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c99400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c94159230 0x7f2c9412c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c99400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c94159230 0x7f2c9412c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56392/0 (socket says 172.21.15.46:56392) 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c99400640 1 -- 172.21.15.46:0/3283474650 learned_addr learned my addr 172.21.15.46:0/3283474650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c98a00640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c940cbeb0 0x7f2c941580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c93e00640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c940cda80 0x7f2c94144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c99400640 1 -- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c940cda80 msgr2=0x7f2c94144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c99400640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c940cda80 0x7f2c94144c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c99400640 1 -- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c940cbeb0 msgr2=0x7f2c941580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c99400640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c940cbeb0 0x7f2c941580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c99400640 1 -- 172.21.15.46:0/3283474650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c94159820 con 0x7f2c94159230 2024-04-25T08:43:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c98a00640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c940cbeb0 0x7f2c941580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c99400640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c94159230 0x7f2c9412c4c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c007e60 tx=0x7f2c8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c90c00640 1 -- 172.21.15.46:0/3283474650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c014620 con 0x7f2c94159230 2024-04-25T08:43:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c90c00640 1 -- 172.21.15.46:0/3283474650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c019070 con 0x7f2c94159230 2024-04-25T08:43:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c90c00640 1 -- 172.21.15.46:0/3283474650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c014a20 con 0x7f2c94159230 2024-04-25T08:43:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c94159b00 con 0x7f2c94159230 2024-04-25T08:43:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9415a0c0 con 0x7f2c94159230 2024-04-25T08:43:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.251+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c94069080 con 0x7f2c94159230 2024-04-25T08:43:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.255+0000 7f2c90c00640 1 -- 172.21.15.46:0/3283474650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c8c014bc0 con 0x7f2c94159230 2024-04-25T08:43:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.255+0000 7f2c90c00640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c74066f90 0x7f2c74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.255+0000 7f2c90c00640 1 -- 172.21.15.46:0/3283474650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2c8c085bd0 con 0x7f2c94159230 2024-04-25T08:43:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.255+0000 7f2c98a00640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c74066f90 0x7f2c74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.255+0000 7f2c98a00640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c74066f90 0x7f2c74069450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b9f0 tx=0x7f2c88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.259+0000 7f2c90c00640 1 -- 172.21.15.46:0/3283474650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c8c0567d0 con 0x7f2c94159230 2024-04-25T08:43:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c94166550 con 0x7f2c94159230 2024-04-25T08:43:22.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c90c00640 1 -- 172.21.15.46:0/3283474650 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f2c8c0560d0 con 0x7f2c94159230 2024-04-25T08:43:22.447 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:22.448 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:38.824358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:43:22.448 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-25T08:43:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c74066f90 msgr2=0x7f2c74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c74066f90 0x7f2c74069450 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b9f0 tx=0x7f2c88008040 comp rx=0 tx=0).stop 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c94159230 msgr2=0x7f2c9412c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c94159230 0x7f2c9412c4c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c007e60 tx=0x7f2c8c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 shutdown_connections 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c74066f90 0x7f2c74069450 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c94159230 0x7f2c9412c4c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c940cda80 0x7f2c94144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 --2- 172.21.15.46:0/3283474650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c940cbeb0 0x7f2c941580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 >> 172.21.15.46:0/3283474650 conn(0x7f2c940b12a0 msgr2=0x7f2c9406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 shutdown_connections 2024-04-25T08:43:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.455+0000 7f2c9b400640 1 -- 172.21.15.46:0/3283474650 wait complete. 2024-04-25T08:43:22.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:22.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f2800640 1 Processor -- start 2024-04-25T08:43:22.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f2800640 1 -- start start 2024-04-25T08:43:22.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 0x7f22ec15b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22ec09a040 0x7f22ec15b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f2800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec0cc080 con 0x7f22ec099c60 2024-04-25T08:43:22.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f2800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec15bb50 con 0x7f22ec09a040 2024-04-25T08:43:22.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0cc360 con 0x7f22ec0b2b20 2024-04-25T08:43:22.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22ec09a040 0x7f22ec15b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22ec09a040 0x7f22ec15b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56400/0 (socket says 172.21.15.46:56400) 2024-04-25T08:43:22.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f1e00640 1 -- 172.21.15.46:0/1923081535 learned_addr learned my addr 172.21.15.46:0/1923081535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:22.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f0a00640 1 --2- 172.21.15.46:0/1923081535 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 0x7f22ec15b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f1e00640 1 -- 172.21.15.46:0/1923081535 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 msgr2=0x7f22ec15b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:22.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f1e00640 1 --2- 172.21.15.46:0/1923081535 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 0x7f22ec15b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f1e00640 1 -- 172.21.15.46:0/1923081535 >> v1:172.21.15.82:6789/0 conn(0x7f22ec099c60 legacy=0x7f22ec0b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:22.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.619+0000 7f22f1e00640 1 -- 172.21.15.46:0/1923081535 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0cc640 con 0x7f22ec09a040 2024-04-25T08:43:22.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f1e00640 1 --2- 172.21.15.46:0/1923081535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec09a040 0x7f22ec15b610 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009e50 tx=0x7f22e401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e1ce08adb5a3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:22.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22ebe00640 1 -- 172.21.15.46:0/1923081535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e40b4070 con 0x7f22ec09a040 2024-04-25T08:43:22.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22ebe00640 1 -- 172.21.15.46:0/1923081535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e40af400 con 0x7f22ec09a040 2024-04-25T08:43:22.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22ebe00640 1 -- 172.21.15.46:0/1923081535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e40af7d0 con 0x7f22ec09a040 2024-04-25T08:43:22.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 -- 172.21.15.46:0/1923081535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec09a040 msgr2=0x7f22ec15b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 --2- 172.21.15.46:0/1923081535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec09a040 0x7f22ec15b610 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009e50 tx=0x7f22e401f210 comp rx=0 tx=0).stop 2024-04-25T08:43:22.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 -- 172.21.15.46:0/1923081535 shutdown_connections 2024-04-25T08:43:22.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 --2- 172.21.15.46:0/1923081535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec09a040 0x7f22ec15b610 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 --2- 172.21.15.46:0/1923081535 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 0x7f22ec15b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 -- 172.21.15.46:0/1923081535 >> 172.21.15.46:0/1923081535 conn(0x7f22ec0d1480 msgr2=0x7f22ec0d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:22.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 -- 172.21.15.46:0/1923081535 shutdown_connections 2024-04-25T08:43:22.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 -- 172.21.15.46:0/1923081535 wait complete. 2024-04-25T08:43:22.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 Processor -- start 2024-04-25T08:43:22.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.623+0000 7f22f2800640 1 -- start start 2024-04-25T08:43:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec099c60 0x7f22ec0b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ec09a040 0x7f22ec0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 0x7f22ec15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0cbce0 con 0x7f22ec099c60 2024-04-25T08:43:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0ccc00 con 0x7f22ec09a040 2024-04-25T08:43:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0cbe60 con 0x7f22ec0b2b20 2024-04-25T08:43:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec099c60 0x7f22ec0b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec099c60 0x7f22ec0b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56414/0 (socket says 172.21.15.46:56414) 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1400640 1 -- 172.21.15.46:0/654855494 learned_addr learned my addr 172.21.15.46:0/654855494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1e00640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 0x7f22ec15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f0a00640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ec09a040 0x7f22ec0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1400640 1 -- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 msgr2=0x7f22ec15a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1400640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 0x7f22ec15a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1400640 1 -- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ec09a040 msgr2=0x7f22ec0ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1400640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ec09a040 0x7f22ec0ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1400640 1 -- 172.21.15.46:0/654855494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec15ab90 con 0x7f22ec099c60 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1e00640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 0x7f22ec15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f0a00640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ec09a040 0x7f22ec0ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f1400640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec099c60 0x7f22ec0b6c10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c420 tx=0x7f22e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22e9600640 1 -- 172.21.15.46:0/654855494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e00052d0 con 0x7f22ec099c60 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22e9600640 1 -- 172.21.15.46:0/654855494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0005470 con 0x7f22ec099c60 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22e9600640 1 -- 172.21.15.46:0/654855494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0007260 con 0x7f22ec099c60 2024-04-25T08:43:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec0cad70 con 0x7f22ec099c60 2024-04-25T08:43:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec0cb330 con 0x7f22ec099c60 2024-04-25T08:43:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.627+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec069010 con 0x7f22ec099c60 2024-04-25T08:43:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.631+0000 7f22e9600640 1 -- 172.21.15.46:0/654855494 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22e0007400 con 0x7f22ec099c60 2024-04-25T08:43:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.631+0000 7f22e9600640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22cc066f90 0x7f22cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.631+0000 7f22e9600640 1 -- 172.21.15.46:0/654855494 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f22e00848f0 con 0x7f22ec099c60 2024-04-25T08:43:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.631+0000 7f22f0a00640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22cc066f90 0x7f22cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.631+0000 7f22f0a00640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22cc066f90 0x7f22cc069450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22ec09f540 tx=0x7f22d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:22.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.635+0000 7f22e9600640 1 -- 172.21.15.46:0/654855494 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22e00554f0 con 0x7f22ec099c60 2024-04-25T08:43:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22ec0b0940 con 0x7f22ec099c60 2024-04-25T08:43:22.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22e9600640 1 -- 172.21.15.46:0/654855494 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f22e0054df0 con 0x7f22ec099c60 2024-04-25T08:43:22.824 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:22.824 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3712529634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3712529634},{"type":"v1","addr":"172.21.15.46:6835","nonce":3712529634}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1808938057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1808938057},{"type":"v1","addr":"172.21.15.82:6836","nonce":1808938057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1625964283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1625964283},{"type":"v1","addr":"172.21.15.46:6833","nonce":1625964283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:20.723239+0000","modified":"2024-04-25T08:42:38.824358+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3580064760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3580064760},{"type":"v1","addr":"172.21.15.82:6837","nonce":3580064760}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:43:22.824 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-04-25T08:43:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22cc066f90 msgr2=0x7f22cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22cc066f90 0x7f22cc069450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f22ec09f540 tx=0x7f22d8008040 comp rx=0 tx=0).stop 2024-04-25T08:43:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec099c60 msgr2=0x7f22ec0b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec099c60 0x7f22ec0b6c10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c420 tx=0x7f22e000c8e0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 shutdown_connections 2024-04-25T08:43:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22cc066f90 0x7f22cc069450 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22ec0b2b20 0x7f22ec15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ec09a040 0x7f22ec0ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.831+0000 7f22f2800640 1 --2- 172.21.15.46:0/654855494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22ec099c60 0x7f22ec0b6c10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.835+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 >> 172.21.15.46:0/654855494 conn(0x7f22ec0d1480 msgr2=0x7f22ec06be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.835+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 shutdown_connections 2024-04-25T08:43:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.835+0000 7f22f2800640 1 -- 172.21.15.46:0/654855494 wait complete. 2024-04-25T08:43:22.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae9d400640 1 Processor -- start 2024-04-25T08:43:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae9d400640 1 -- start start 2024-04-25T08:43:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae9d400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae98099a00 0x7fae98099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae9d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae980ccde0 0x7fae980cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae9d400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9809a320 con 0x7fae980b2240 2024-04-25T08:43:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae9d400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae9806db90 con 0x7fae98099a00 2024-04-25T08:43:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae9d400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9806bdc0 con 0x7fae980ccde0 2024-04-25T08:43:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae96800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae98099a00 0x7fae98099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae8fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae980ccde0 0x7fae980cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae97200640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fae980b2240 0x7fae980b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:42176/0 (socket says 172.21.15.46:42176) 2024-04-25T08:43:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae96800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae98099a00 0x7fae98099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:42210/0 (socket says 172.21.15.46:42210) 2024-04-25T08:43:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae97200640 1 -- 172.21.15.46:0/1254799252 learned_addr learned my addr 172.21.15.46:0/1254799252 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae96800640 1 -- 172.21.15.46:0/1254799252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae980ccde0 msgr2=0x7fae980cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae96800640 1 --2- 172.21.15.46:0/1254799252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae980ccde0 0x7fae980cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae96800640 1 -- 172.21.15.46:0/1254799252 >> v1:172.21.15.82:6789/0 conn(0x7fae980b2240 legacy=0x7fae980b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae96800640 1 -- 172.21.15.46:0/1254799252 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9806c0a0 con 0x7fae98099a00 2024-04-25T08:43:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.995+0000 7fae95e00640 1 -- 172.21.15.46:0/1254799252 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504746087 0 0) 0x7fae9809a320 con 0x7fae980b2240 2024-04-25T08:43:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae96800640 1 --2- 172.21.15.46:0/1254799252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98099a00 0x7fae98099de0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fae880097c0 tx=0x7fae8801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c473c6c6883262e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae95e00640 1 -- 172.21.15.46:0/1254799252 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880b1070 con 0x7fae98099a00 2024-04-25T08:43:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae95e00640 1 -- 172.21.15.46:0/1254799252 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae880a8050 con 0x7fae98099a00 2024-04-25T08:43:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae95e00640 1 -- 172.21.15.46:0/1254799252 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae880ac860 con 0x7fae98099a00 2024-04-25T08:43:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae9d400640 1 -- 172.21.15.46:0/1254799252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98099a00 msgr2=0x7fae98099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae9d400640 1 --2- 172.21.15.46:0/1254799252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98099a00 0x7fae98099de0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fae880097c0 tx=0x7fae8801c3e0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae9d400640 1 -- 172.21.15.46:0/1254799252 shutdown_connections 2024-04-25T08:43:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae9d400640 1 --2- 172.21.15.46:0/1254799252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae980ccde0 0x7fae980cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae9d400640 1 --2- 172.21.15.46:0/1254799252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae98099a00 0x7fae98099de0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae9d400640 1 -- 172.21.15.46:0/1254799252 >> 172.21.15.46:0/1254799252 conn(0x7fae980b3120 msgr2=0x7fae980b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae9d400640 1 -- 172.21.15.46:0/1254799252 shutdown_connections 2024-04-25T08:43:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:22.999+0000 7fae9d400640 1 -- 172.21.15.46:0/1254799252 wait complete. 2024-04-25T08:43:22.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 Processor -- start 2024-04-25T08:43:22.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 -- start start 2024-04-25T08:43:22.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98099a00 0x7fae980b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae980b2240 0x7fae98158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae980ccde0 0x7fae981593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae980cd7a0 con 0x7fae980ccde0 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae980cdb80 con 0x7fae980b2240 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae980cd920 con 0x7fae98099a00 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae97200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae980ccde0 0x7fae981593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae97200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae980ccde0 0x7fae981593b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56422/0 (socket says 172.21.15.46:56422) 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae97200640 1 -- 172.21.15.46:0/1687681955 learned_addr learned my addr 172.21.15.46:0/1687681955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae96800640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98099a00 0x7fae980b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae8fe00640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae980b2240 0x7fae98158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae97200640 1 -- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98099a00 msgr2=0x7fae980b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae97200640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98099a00 0x7fae980b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae97200640 1 -- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae980b2240 msgr2=0x7fae98158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae97200640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae980b2240 0x7fae98158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae97200640 1 -- 172.21.15.46:0/1687681955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae981598f0 con 0x7fae980ccde0 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae8fe00640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae980b2240 0x7fae98158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae97200640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae980ccde0 0x7fae981593b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae9000c780 tx=0x7fae9000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae8ea00640 1 -- 172.21.15.46:0/1687681955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90018070 con 0x7fae980ccde0 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae8ea00640 1 -- 172.21.15.46:0/1687681955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9000a640 con 0x7fae980ccde0 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae8ea00640 1 -- 172.21.15.46:0/1687681955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000aa30 con 0x7fae980ccde0 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae98154740 con 0x7fae980ccde0 2024-04-25T08:43:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae98154d00 con 0x7fae980ccde0 2024-04-25T08:43:22.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.003+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae98069080 con 0x7fae980ccde0 2024-04-25T08:43:22.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.007+0000 7fae8ea00640 1 -- 172.21.15.46:0/1687681955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae9000abd0 con 0x7fae980ccde0 2024-04-25T08:43:22.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.007+0000 7fae8ea00640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fae74066f60 0x7fae74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:22.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.007+0000 7fae8ea00640 1 -- 172.21.15.46:0/1687681955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fae900848f0 con 0x7fae980ccde0 2024-04-25T08:43:23.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.007+0000 7fae96800640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fae74066f60 0x7fae74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.007+0000 7fae96800640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fae74066f60 0x7fae74069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae880ae000 tx=0x7fae880a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.011+0000 7fae8ea00640 1 -- 172.21.15.46:0/1687681955 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae90054ea0 con 0x7fae980ccde0 2024-04-25T08:43:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.171+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae98154ed0 con 0x7fae980ccde0 2024-04-25T08:43:23.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae8ea00640 1 -- 172.21.15.46:0/1687681955 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+18855 (secure 0 0 0) 0x7fae98154ed0 con 0x7fae980ccde0 2024-04-25T08:43:23.167 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:23.167 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:01.503109+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:18.424873+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:19.729069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:42:23.413095+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fae74066f60 msgr2=0x7fae74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fae74066f60 0x7fae74069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae880ae000 tx=0x7fae880a7000 comp rx=0 tx=0).stop 2024-04-25T08:43:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae980ccde0 msgr2=0x7fae981593b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae980ccde0 0x7fae981593b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae9000c780 tx=0x7fae9000cc40 comp rx=0 tx=0).stop 2024-04-25T08:43:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 shutdown_connections 2024-04-25T08:43:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fae74066f60 0x7fae74069420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae980ccde0 0x7fae981593b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae980b2240 0x7fae98158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 --2- 172.21.15.46:0/1687681955 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fae98099a00 0x7fae980b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 >> 172.21.15.46:0/1687681955 conn(0x7fae980b3120 msgr2=0x7fae980cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 shutdown_connections 2024-04-25T08:43:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.175+0000 7fae9d400640 1 -- 172.21.15.46:0/1687681955 wait complete. 2024-04-25T08:43:23.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:43:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.335+0000 7f59d4600640 1 Processor -- start 2024-04-25T08:43:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.335+0000 7f59d4600640 1 -- start start 2024-04-25T08:43:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d4600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59cc1654c0 0x7f59cc163400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d4600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc162330 0x7f59cc06e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d4600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc06e9d0 con 0x7f59cc06d270 2024-04-25T08:43:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d4600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc06ec70 con 0x7f59cc1654c0 2024-04-25T08:43:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d4600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc06ef50 con 0x7f59cc162330 2024-04-25T08:43:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59cc1654c0 0x7f59cc163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59cc1654c0 0x7f59cc163400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56432/0 (socket says 172.21.15.46:56432) 2024-04-25T08:43:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d1c00640 1 -- 172.21.15.46:0/3172119126 learned_addr learned my addr 172.21.15.46:0/3172119126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d1200640 1 --2- 172.21.15.46:0/3172119126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc162330 0x7f59cc06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d1c00640 1 -- 172.21.15.46:0/3172119126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc162330 msgr2=0x7f59cc06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d1c00640 1 --2- 172.21.15.46:0/3172119126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc162330 0x7f59cc06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d1c00640 1 -- 172.21.15.46:0/3172119126 >> v1:172.21.15.82:6789/0 conn(0x7f59cc06d270 legacy=0x7f59cc0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d1c00640 1 -- 172.21.15.46:0/3172119126 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc06f230 con 0x7f59cc1654c0 2024-04-25T08:43:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.339+0000 7f59d1200640 1 --2- 172.21.15.46:0/3172119126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc162330 0x7f59cc06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:23.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d1c00640 1 --2- 172.21.15.46:0/3172119126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc1654c0 0x7f59cc163400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009870 tx=0x7f59c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca1307c42691f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:23.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59cbe00640 1 -- 172.21.15.46:0/3172119126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00b0070 con 0x7f59cc1654c0 2024-04-25T08:43:23.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59cbe00640 1 -- 172.21.15.46:0/3172119126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c00ab600 con 0x7f59cc1654c0 2024-04-25T08:43:23.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59cbe00640 1 -- 172.21.15.46:0/3172119126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00ab9d0 con 0x7f59cc1654c0 2024-04-25T08:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 -- 172.21.15.46:0/3172119126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc1654c0 msgr2=0x7f59cc163400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 --2- 172.21.15.46:0/3172119126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc1654c0 0x7f59cc163400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009870 tx=0x7f59c001c520 comp rx=0 tx=0).stop 2024-04-25T08:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 -- 172.21.15.46:0/3172119126 shutdown_connections 2024-04-25T08:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 --2- 172.21.15.46:0/3172119126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc162330 0x7f59cc06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 --2- 172.21.15.46:0/3172119126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc1654c0 0x7f59cc163400 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 -- 172.21.15.46:0/3172119126 >> 172.21.15.46:0/3172119126 conn(0x7f59cc0b12a0 msgr2=0x7f59cc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 -- 172.21.15.46:0/3172119126 shutdown_connections 2024-04-25T08:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 -- 172.21.15.46:0/3172119126 wait complete. 2024-04-25T08:43:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 Processor -- start 2024-04-25T08:43:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.343+0000 7f59d4600640 1 -- start start 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d4600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc06d270 0x7f59cc15efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d4600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59cc162330 0x7f59cc0b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d4600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc0b4630 0x7f59cc155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d4600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc16ee20 con 0x7f59cc06d270 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d4600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc16eca0 con 0x7f59cc162330 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d4600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc16efa0 con 0x7f59cc0b4630 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc06d270 0x7f59cc15efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc06d270 0x7f59cc15efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56446/0 (socket says 172.21.15.46:56446) 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1c00640 1 -- 172.21.15.46:0/3307100864 learned_addr learned my addr 172.21.15.46:0/3307100864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1200640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59cc162330 0x7f59cc0b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d2600640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc0b4630 0x7f59cc155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1c00640 1 -- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc0b4630 msgr2=0x7f59cc155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1c00640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc0b4630 0x7f59cc155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1c00640 1 -- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59cc162330 msgr2=0x7f59cc0b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1c00640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59cc162330 0x7f59cc0b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1c00640 1 -- 172.21.15.46:0/3307100864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc156650 con 0x7f59cc06d270 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1200640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59cc162330 0x7f59cc0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d1c00640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc06d270 0x7f59cc15efb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f59c00ae040 tx=0x7f59c001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59c9600640 1 -- 172.21.15.46:0/3307100864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00b0040 con 0x7f59cc06d270 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59c9600640 1 -- 172.21.15.46:0/3307100864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c00bc070 con 0x7f59cc06d270 2024-04-25T08:43:23.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59c9600640 1 -- 172.21.15.46:0/3307100864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00b6880 con 0x7f59cc06d270 2024-04-25T08:43:23.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc156930 con 0x7f59cc06d270 2024-04-25T08:43:23.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc156ef0 con 0x7f59cc06d270 2024-04-25T08:43:23.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc069080 con 0x7f59cc06d270 2024-04-25T08:43:23.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.347+0000 7f59c9600640 1 -- 172.21.15.46:0/3307100864 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59c00b6a20 con 0x7f59cc06d270 2024-04-25T08:43:23.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.351+0000 7f59c9600640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f59ac066f90 0x7f59ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.351+0000 7f59c9600640 1 -- 172.21.15.46:0/3307100864 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f59c011f860 con 0x7f59cc06d270 2024-04-25T08:43:23.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.351+0000 7f59d1200640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f59ac066f90 0x7f59ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.351+0000 7f59d1200640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f59ac066f90 0x7f59ac069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59cc1fe470 tx=0x7f59b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:23.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.355+0000 7f59c9600640 1 -- 172.21.15.46:0/3307100864 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59c0016110 con 0x7f59cc06d270 2024-04-25T08:43:23.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.547+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f59cc06c250 con 0x7f59cc06d270 2024-04-25T08:43:23.542 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:43:23.551+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:43:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.551+0000 7f59c9600640 1 -- 172.21.15.46:0/3307100864 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f59c00f0490 con 0x7f59cc06d270 2024-04-25T08:43:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f59ac066f90 msgr2=0x7f59ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f59ac066f90 0x7f59ac069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59cc1fe470 tx=0x7f59b8008040 comp rx=0 tx=0).stop 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc06d270 msgr2=0x7f59cc15efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc06d270 0x7f59cc15efb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f59c00ae040 tx=0x7f59c001ad60 comp rx=0 tx=0).stop 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 shutdown_connections 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f59ac066f90 0x7f59ac069450 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f59cc0b4630 0x7f59cc155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59cc162330 0x7f59cc0b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 --2- 172.21.15.46:0/3307100864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59cc06d270 0x7f59cc15efb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 >> 172.21.15.46:0/3307100864 conn(0x7f59cc0b12a0 msgr2=0x7f59cc071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 shutdown_connections 2024-04-25T08:43:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.555+0000 7f59d4600640 1 -- 172.21.15.46:0/3307100864 wait complete. 2024-04-25T08:43:23.558 INFO:tasks.ceph.mds.b.smithi082.stderr: -5209> 2024-04-25T08:42:24.656+0000 7f34ad400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:43:23.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:43:23.609 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-04-25T08:43:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e0b800640 1 Processor -- start 2024-04-25T08:43:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e0b800640 1 -- start start 2024-04-25T08:43:23.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e0b800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e041440d0 0x7f4e040cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e0b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4e0406f150 0x7f4e040b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e0b800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0415e440 con 0x7f4e04143990 2024-04-25T08:43:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e0b800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0415ed30 con 0x7f4e041440d0 2024-04-25T08:43:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e0b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0415e6e0 con 0x7f4e0406f150 2024-04-25T08:43:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e041440d0 0x7f4e040cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e09800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4e0406f150 0x7f4e040b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e041440d0 0x7f4e040cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56458/0 (socket says 172.21.15.46:56458) 2024-04-25T08:43:23.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e03e00640 1 -- 172.21.15.46:0/2567337463 learned_addr learned my addr 172.21.15.46:0/2567337463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:23.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e09800640 1 -- 172.21.15.46:0/2567337463 >> v2:172.21.15.46:3300/0 conn(0x7f4e041440d0 msgr2=0x7f4e040cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:23.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e09800640 1 --2- 172.21.15.46:0/2567337463 >> v2:172.21.15.46:3300/0 conn(0x7f4e041440d0 0x7f4e040cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e09800640 1 -- 172.21.15.46:0/2567337463 >> v1:172.21.15.82:6789/0 conn(0x7f4e04143990 legacy=0x7f4e0406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e09800640 1 -- 172.21.15.46:0/2567337463 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0415e9c0 con 0x7f4e0406f150 2024-04-25T08:43:23.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e03e00640 1 --2- 172.21.15.46:0/2567337463 >> v2:172.21.15.46:3300/0 conn(0x7f4e041440d0 0x7f4e040cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:23.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.719+0000 7f4e03400640 1 -- 172.21.15.46:0/2567337463 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026912667 0 0) 0x7f4e0415e440 con 0x7f4e04143990 2024-04-25T08:43:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e09800640 1 --2- 172.21.15.46:0/2567337463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e0406f150 0x7f4e040b2720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4e040af320 tx=0x7f4dfc01c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fa7ab92f344339d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e03400640 1 -- 172.21.15.46:0/2567337463 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc0b2070 con 0x7f4e0406f150 2024-04-25T08:43:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e03400640 1 -- 172.21.15.46:0/2567337463 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc0a57f0 con 0x7f4e0406f150 2024-04-25T08:43:23.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e03400640 1 -- 172.21.15.46:0/2567337463 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc0a5bc0 con 0x7f4e0406f150 2024-04-25T08:43:23.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e0b800640 1 -- 172.21.15.46:0/2567337463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e0406f150 msgr2=0x7f4e040b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e0b800640 1 --2- 172.21.15.46:0/2567337463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e0406f150 0x7f4e040b2720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4e040af320 tx=0x7f4dfc01c930 comp rx=0 tx=0).stop 2024-04-25T08:43:23.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e0b800640 1 -- 172.21.15.46:0/2567337463 shutdown_connections 2024-04-25T08:43:23.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e0b800640 1 --2- 172.21.15.46:0/2567337463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e0406f150 0x7f4e040b2720 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e0b800640 1 --2- 172.21.15.46:0/2567337463 >> v2:172.21.15.46:3300/0 conn(0x7f4e041440d0 0x7f4e040cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e0b800640 1 -- 172.21.15.46:0/2567337463 >> 172.21.15.46:0/2567337463 conn(0x7f4e0406ac10 msgr2=0x7f4e0406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:23.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e0b800640 1 -- 172.21.15.46:0/2567337463 shutdown_connections 2024-04-25T08:43:23.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e0b800640 1 -- 172.21.15.46:0/2567337463 wait complete. 2024-04-25T08:43:23.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.723+0000 7f4e0b800640 1 Processor -- start 2024-04-25T08:43:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 -- start start 2024-04-25T08:43:23.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e0406f150 0x7f4e0413eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e04143990 0x7f4e04159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4e041440d0 0x7f4e0413fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e040b35f0 con 0x7f4e04143990 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e040b3470 con 0x7f4e0406f150 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e040b3770 con 0x7f4e041440d0 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e04143990 0x7f4e04159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e04143990 0x7f4e04159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56466/0 (socket says 172.21.15.46:56466) 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e03e00640 1 -- 172.21.15.46:0/2963231649 learned_addr learned my addr 172.21.15.46:0/2963231649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e09800640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4e041440d0 0x7f4e0413fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e08e00640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e0406f150 0x7f4e0413eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e03e00640 1 -- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4e041440d0 msgr2=0x7f4e0413fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e03e00640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4e041440d0 0x7f4e0413fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e03e00640 1 -- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e0406f150 msgr2=0x7f4e0413eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e03e00640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e0406f150 0x7f4e0413eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e03e00640 1 -- 172.21.15.46:0/2963231649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0413d310 con 0x7f4e04143990 2024-04-25T08:43:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e03e00640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e04143990 0x7f4e04159300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4df000e930 tx=0x7f4df000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e00c00640 1 -- 172.21.15.46:0/2963231649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0017070 con 0x7f4e04143990 2024-04-25T08:43:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e00c00640 1 -- 172.21.15.46:0/2963231649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df000ca00 con 0x7f4e04143990 2024-04-25T08:43:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e00c00640 1 -- 172.21.15.46:0/2963231649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df000cdd0 con 0x7f4e04143990 2024-04-25T08:43:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0413d5f0 con 0x7f4e04143990 2024-04-25T08:43:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0413dbb0 con 0x7f4e04143990 2024-04-25T08:43:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.727+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e04069010 con 0x7f4e04143990 2024-04-25T08:43:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.731+0000 7f4e00c00640 1 -- 172.21.15.46:0/2963231649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4df001a070 con 0x7f4e04143990 2024-04-25T08:43:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.731+0000 7f4e00c00640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4de4066f60 0x7f4de4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.731+0000 7f4e00c00640 1 -- 172.21.15.46:0/2963231649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f4df0085680 con 0x7f4e04143990 2024-04-25T08:43:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.731+0000 7f4e08e00640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4de4066f60 0x7f4de4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:23.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.731+0000 7f4e08e00640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4de4066f60 0x7f4de4069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4df8000f70 tx=0x7f4df8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:23.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.735+0000 7f4e00c00640 1 -- 172.21.15.46:0/2963231649 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4df0055bd0 con 0x7f4e04143990 2024-04-25T08:43:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:23.931+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4e040ae810 con 0x7f4e04143990 2024-04-25T08:43:24.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.599+0000 7f4e00c00640 1 -- 172.21.15.46:0/2963231649 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4df0059c70 con 0x7f4e04143990 2024-04-25T08:43:24.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4de4066f60 msgr2=0x7f4de4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:24.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4de4066f60 0x7f4de4069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4df8000f70 tx=0x7f4df8009040 comp rx=0 tx=0).stop 2024-04-25T08:43:24.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e04143990 msgr2=0x7f4e04159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:24.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e04143990 0x7f4e04159300 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4df000e930 tx=0x7f4df000edf0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 shutdown_connections 2024-04-25T08:43:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4de4066f60 0x7f4de4069420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4e041440d0 0x7f4e0413fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e04143990 0x7f4e04159300 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 --2- 172.21.15.46:0/2963231649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4e0406f150 0x7f4e0413eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 >> 172.21.15.46:0/2963231649 conn(0x7f4e0406ac10 msgr2=0x7f4e04164c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 shutdown_connections 2024-04-25T08:43:24.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.603+0000 7f4e0b800640 1 -- 172.21.15.46:0/2963231649 wait complete. 2024-04-25T08:43:24.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:43:24.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.775+0000 7ff99ba00640 1 Processor -- start 2024-04-25T08:43:24.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.775+0000 7ff99ba00640 1 -- start start 2024-04-25T08:43:24.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff99ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff99406bc20 0x7ff99406c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:24.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff99ba00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99406c540 con 0x7ff9940ae1a0 2024-04-25T08:43:24.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff99ba00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99406ee30 con 0x7ff99415dce0 2024-04-25T08:43:24.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff99ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99406f110 con 0x7ff99406bc20 2024-04-25T08:43:24.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9940ae1a0 0x7ff99406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52936/0 (socket says 172.21.15.46:52936) 2024-04-25T08:43:24.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999a00640 1 -- 172.21.15.46:0/662264026 learned_addr learned my addr 172.21.15.46:0/662264026 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999000640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff99406bc20 0x7ff99406c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891155271 0 0) 0x7ff99406c540 con 0x7ff9940ae1a0 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff978003530 con 0x7ff9940ae1a0 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531191235 0 0) 0x7ff99406ee30 con 0x7ff99415dce0 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff99406c540 con 0x7ff99415dce0 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 347956643 0 0) 0x7ff978003530 con 0x7ff9940ae1a0 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff99406bc20 msgr2=0x7ff99406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff99406bc20 0x7ff99406c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 >> v1:172.21.15.82:6789/0 conn(0x7ff99415dce0 legacy=0x7ff99406db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99406f3f0 con 0x7ff9940ae1a0 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7ff98c001a00 con 0x7ff9940ae1a0 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 >> v1:172.21.15.46:6789/0 conn(0x7ff9940ae1a0 legacy=0x7ff99406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff978003b80 0x7ff978003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff978004810 0x7ff978004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9780051d0 0x7ff97800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99406f110 con 0x7ff9780051d0 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97800bff0 con 0x7ff978004810 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97800c1f0 con 0x7ff978003b80 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999000640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff99406bc20 0x7ff99406c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999a00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9780051d0 0x7ff97800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff98be00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff978004810 0x7ff978004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999000640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff978003b80 0x7ff978003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999a00640 1 -- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff978003b80 msgr2=0x7ff978003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999a00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff978003b80 0x7ff978003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999a00640 1 -- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff978004810 msgr2=0x7ff978004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999a00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff978004810 0x7ff978004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.779+0000 7ff999a00640 1 -- 172.21.15.46:0/662264026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff978003530 con 0x7ff9780051d0 2024-04-25T08:43:24.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff999a00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9780051d0 0x7ff97800b9d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00a700 tx=0x7ff98c01f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b3977caa703321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:24.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00a350 con 0x7ff9780051d0 2024-04-25T08:43:24.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c0b1070 con 0x7ff9780051d0 2024-04-25T08:43:24.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff993e00640 1 -- 172.21.15.46:0/662264026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0ac8b0 con 0x7ff9780051d0 2024-04-25T08:43:24.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff99ba00640 1 -- 172.21.15.46:0/662264026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9780051d0 msgr2=0x7ff97800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:24.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff99ba00640 1 --2- 172.21.15.46:0/662264026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9780051d0 0x7ff97800b9d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00a700 tx=0x7ff98c01f960 comp rx=0 tx=0).stop 2024-04-25T08:43:24.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff999a00640 1 -- 172.21.15.46:0/662264026 reap_dead start 2024-04-25T08:43:24.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff99ba00640 1 -- 172.21.15.46:0/662264026 shutdown_connections 2024-04-25T08:43:24.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff99ba00640 1 -- 172.21.15.46:0/662264026 >> 172.21.15.46:0/662264026 conn(0x7ff994071ae0 msgr2=0x7ff994071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:24.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff99ba00640 1 -- 172.21.15.46:0/662264026 shutdown_connections 2024-04-25T08:43:24.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff99ba00640 1 -- 172.21.15.46:0/662264026 wait complete. 2024-04-25T08:43:24.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff99ba00640 1 Processor -- start 2024-04-25T08:43:24.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.783+0000 7ff99ba00640 1 -- start start 2024-04-25T08:43:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff99ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9780051d0 0x7ff99414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff99ba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff994152bd0 0x7ff994153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff99ba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9941f6bd0 0x7ff9941f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff99ba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940b3b10 con 0x7ff9941f6bd0 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff99ba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940cfad0 con 0x7ff994152bd0 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff99ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940b3c90 con 0x7ff9780051d0 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9941f6bd0 0x7ff9941f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9941f6bd0 0x7ff9941f7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56488/0 (socket says 172.21.15.46:56488) 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999a00640 1 -- 172.21.15.46:0/403419073 learned_addr learned my addr 172.21.15.46:0/403419073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999a00640 1 -- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9780051d0 msgr2=0x7ff99414eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999000640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9780051d0 0x7ff99414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff98be00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff994152bd0 0x7ff994153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999a00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9780051d0 0x7ff99414eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999a00640 1 -- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff994152bd0 msgr2=0x7ff994153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999a00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff994152bd0 0x7ff994153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999a00640 1 -- 172.21.15.46:0/403419073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99414f410 con 0x7ff9941f6bd0 2024-04-25T08:43:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999000640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9780051d0 0x7ff99414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff98be00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff994152bd0 0x7ff994153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff999a00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9941f6bd0 0x7ff9941f7050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00a700 tx=0x7ff978005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff991600640 1 -- 172.21.15.46:0/403419073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00a350 con 0x7ff9941f6bd0 2024-04-25T08:43:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff991600640 1 -- 172.21.15.46:0/403419073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c0b8420 con 0x7ff9941f6bd0 2024-04-25T08:43:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9941f7a20 con 0x7ff9941f6bd0 2024-04-25T08:43:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff991600640 1 -- 172.21.15.46:0/403419073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c001a00 con 0x7ff9941f6bd0 2024-04-25T08:43:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99415e710 con 0x7ff9941f6bd0 2024-04-25T08:43:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.787+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff994069080 con 0x7ff9941f6bd0 2024-04-25T08:43:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.791+0000 7ff991600640 1 -- 172.21.15.46:0/403419073 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff98c0b87d0 con 0x7ff9941f6bd0 2024-04-25T08:43:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.791+0000 7ff991600640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff970066f60 0x7ff970069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.791+0000 7ff991600640 1 -- 172.21.15.46:0/403419073 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6821+0+0 (secure 0 0 0) 0x7ff98c1236e0 con 0x7ff9941f6bd0 2024-04-25T08:43:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.791+0000 7ff999000640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff970066f60 0x7ff970069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:24.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.791+0000 7ff999000640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff970066f60 0x7ff970069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9940679f0 tx=0x7ff984008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:24.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.795+0000 7ff991600640 1 -- 172.21.15.46:0/403419073 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff98c019670 con 0x7ff9941f6bd0 2024-04-25T08:43:24.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:24.959+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff99415b320 con 0x7ff9941f6bd0 2024-04-25T08:43:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.603+0000 7ff991600640 1 -- 172.21.15.46:0/403419073 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff98c0f4160 con 0x7ff9941f6bd0 2024-04-25T08:43:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:43:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.603+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff970066f60 msgr2=0x7ff970069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.603+0000 7ff99ba00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff970066f60 0x7ff970069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9940679f0 tx=0x7ff984008040 comp rx=0 tx=0).stop 2024-04-25T08:43:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.603+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9941f6bd0 msgr2=0x7ff9941f7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.603+0000 7ff99ba00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9941f6bd0 0x7ff9941f7050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00a700 tx=0x7ff978005840 comp rx=0 tx=0).stop 2024-04-25T08:43:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.607+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 shutdown_connections 2024-04-25T08:43:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.607+0000 7ff99ba00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff970066f60 0x7ff970069420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.607+0000 7ff99ba00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9941f6bd0 0x7ff9941f7050 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.607+0000 7ff99ba00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff994152bd0 0x7ff994153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.607+0000 7ff99ba00640 1 --2- 172.21.15.46:0/403419073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff9780051d0 0x7ff99414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.607+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 >> 172.21.15.46:0/403419073 conn(0x7ff994071ae0 msgr2=0x7ff99415a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.607+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 shutdown_connections 2024-04-25T08:43:25.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.607+0000 7ff99ba00640 1 -- 172.21.15.46:0/403419073 wait complete. 2024-04-25T08:43:25.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:43:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc448c00640 1 Processor -- start 2024-04-25T08:43:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc448c00640 1 -- start start 2024-04-25T08:43:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc448c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc4440cd5b0 0x7fc44406c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc448c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc444099a00 0x7fc444099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc448c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4440d0160 con 0x7fc4440d1200 2024-04-25T08:43:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc448c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44409a390 con 0x7fc444099a00 2024-04-25T08:43:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc448c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4440d0440 con 0x7fc4440cd5b0 2024-04-25T08:43:25.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc442800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc444099a00 0x7fc444099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc442800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc444099a00 0x7fc444099e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56498/0 (socket says 172.21.15.46:56498) 2024-04-25T08:43:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc442800640 1 -- 172.21.15.46:0/1400510093 learned_addr learned my addr 172.21.15.46:0/1400510093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc441e00640 1 --2- 172.21.15.46:0/1400510093 >> v2:172.21.15.82:3300/0 conn(0x7fc4440cd5b0 0x7fc44406c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc442800640 1 -- 172.21.15.46:0/1400510093 >> v2:172.21.15.82:3300/0 conn(0x7fc4440cd5b0 msgr2=0x7fc44406c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc442800640 1 --2- 172.21.15.46:0/1400510093 >> v2:172.21.15.82:3300/0 conn(0x7fc4440cd5b0 0x7fc44406c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc442800640 1 -- 172.21.15.46:0/1400510093 >> v1:172.21.15.82:6789/0 conn(0x7fc4440d1200 legacy=0x7fc4440cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.787+0000 7fc442800640 1 -- 172.21.15.46:0/1400510093 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4440d0720 con 0x7fc444099a00 2024-04-25T08:43:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc442800640 1 --2- 172.21.15.46:0/1400510093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444099a00 0x7fc444099e50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc4440b3d90 tx=0x7fc43c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcb893f18168d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc441400640 1 -- 172.21.15.46:0/1400510093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0b4070 con 0x7fc444099a00 2024-04-25T08:43:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc441400640 1 -- 172.21.15.46:0/1400510093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c0af400 con 0x7fc444099a00 2024-04-25T08:43:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc441400640 1 -- 172.21.15.46:0/1400510093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c0af7d0 con 0x7fc444099a00 2024-04-25T08:43:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc448c00640 1 -- 172.21.15.46:0/1400510093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444099a00 msgr2=0x7fc444099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc448c00640 1 --2- 172.21.15.46:0/1400510093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444099a00 0x7fc444099e50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc4440b3d90 tx=0x7fc43c01f1e0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc448c00640 1 -- 172.21.15.46:0/1400510093 shutdown_connections 2024-04-25T08:43:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc448c00640 1 --2- 172.21.15.46:0/1400510093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444099a00 0x7fc444099e50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc448c00640 1 --2- 172.21.15.46:0/1400510093 >> v2:172.21.15.82:3300/0 conn(0x7fc4440cd5b0 0x7fc44406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc448c00640 1 -- 172.21.15.46:0/1400510093 >> 172.21.15.46:0/1400510093 conn(0x7fc4440b31c0 msgr2=0x7fc4440b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc448c00640 1 -- 172.21.15.46:0/1400510093 shutdown_connections 2024-04-25T08:43:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.791+0000 7fc448c00640 1 -- 172.21.15.46:0/1400510093 wait complete. 2024-04-25T08:43:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 Processor -- start 2024-04-25T08:43:25.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 -- start start 2024-04-25T08:43:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc444099a00 0x7fc4441f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440cd5b0 0x7fc4441f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4440cae20 0x7fc4440cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440cd990 con 0x7fc4440cd5b0 2024-04-25T08:43:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440d0ce0 con 0x7fc4440cae20 2024-04-25T08:43:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440cdb10 con 0x7fc444099a00 2024-04-25T08:43:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc439400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440cd5b0 0x7fc4441f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc439400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440cd5b0 0x7fc4441f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56500/0 (socket says 172.21.15.46:56500) 2024-04-25T08:43:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc439400640 1 -- 172.21.15.46:0/1556026100 learned_addr learned my addr 172.21.15.46:0/1556026100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc441e00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc444099a00 0x7fc4441f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc442800640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4440cae20 0x7fc4440cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc439400640 1 -- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc444099a00 msgr2=0x7fc4441f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc439400640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc444099a00 0x7fc4441f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc439400640 1 -- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4440cae20 msgr2=0x7fc4440cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc439400640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4440cae20 0x7fc4440cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc439400640 1 -- 172.21.15.46:0/1556026100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4440c8840 con 0x7fc4440cd5b0 2024-04-25T08:43:25.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc442800640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4440cae20 0x7fc4440cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc439400640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440cd5b0 0x7fc4441f4fe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00c450 tx=0x7fc42c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc43aa00640 1 -- 172.21.15.46:0/1556026100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0052d0 con 0x7fc4440cd5b0 2024-04-25T08:43:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc43aa00640 1 -- 172.21.15.46:0/1556026100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c005470 con 0x7fc4440cd5b0 2024-04-25T08:43:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4440c8b20 con 0x7fc4440cd5b0 2024-04-25T08:43:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc43aa00640 1 -- 172.21.15.46:0/1556026100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c007260 con 0x7fc4440cd5b0 2024-04-25T08:43:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.795+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4440c90e0 con 0x7fc4440cd5b0 2024-04-25T08:43:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.799+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc444069080 con 0x7fc4440cd5b0 2024-04-25T08:43:25.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.799+0000 7fc43aa00640 1 -- 172.21.15.46:0/1556026100 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc42c007400 con 0x7fc4440cd5b0 2024-04-25T08:43:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.799+0000 7fc43aa00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc420066f90 0x7fc420069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.799+0000 7fc43aa00640 1 -- 172.21.15.46:0/1556026100 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6383+0+0 (secure 0 0 0) 0x7fc42c084720 con 0x7fc4440cd5b0 2024-04-25T08:43:25.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.799+0000 7fc441e00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc420066f90 0x7fc420069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:25.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.799+0000 7fc441e00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc420066f90 0x7fc420069450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc434009b60 tx=0x7fc434009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:25.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.803+0000 7fc43aa00640 1 -- 172.21.15.46:0/1556026100 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc42c054e30 con 0x7fc4440cd5b0 2024-04-25T08:43:25.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:25.967+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc4440680e0 con 0x7fc4440cd5b0 2024-04-25T08:43:26.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.615+0000 7fc43aa00640 1 -- 172.21.15.46:0/1556026100 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc42c058ed0 con 0x7fc4440cd5b0 2024-04-25T08:43:26.608 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc420066f90 msgr2=0x7fc420069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc420066f90 0x7fc420069450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc434009b60 tx=0x7fc434009190 comp rx=0 tx=0).stop 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440cd5b0 msgr2=0x7fc4441f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440cd5b0 0x7fc4441f4fe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00c450 tx=0x7fc42c00c910 comp rx=0 tx=0).stop 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 shutdown_connections 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc420066f90 0x7fc420069450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4440cae20 0x7fc4440cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4440cd5b0 0x7fc4441f4fe0 secure :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00c450 tx=0x7fc42c00c910 comp rx=0 tx=0).stop 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 --2- 172.21.15.46:0/1556026100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc444099a00 0x7fc4441f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 >> 172.21.15.46:0/1556026100 conn(0x7fc4440b31c0 msgr2=0x7fc44406f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 shutdown_connections 2024-04-25T08:43:26.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.619+0000 7fc448c00640 1 -- 172.21.15.46:0/1556026100 wait complete. 2024-04-25T08:43:26.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T08:43:26.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.795+0000 7feadba00640 1 Processor -- start 2024-04-25T08:43:26.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7feadba00640 1 -- start start 2024-04-25T08:43:26.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7feadba00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fead40718b0 0x7fead406f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:26.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7feadba00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead406e4d0 con 0x7fead4071c90 2024-04-25T08:43:26.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7feadba00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead40b3a00 con 0x7fead40b3260 2024-04-25T08:43:26.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7feadba00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead40ad070 con 0x7fead40718b0 2024-04-25T08:43:26.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fead4071c90 0x7fead406d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52950/0 (socket says 172.21.15.46:52950) 2024-04-25T08:43:26.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3e00640 1 -- 172.21.15.46:0/1167776694 learned_addr learned my addr 172.21.15.46:0/1167776694 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:26.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead9000640 1 --2- 172.21.15.46:0/1167776694 >> v2:172.21.15.82:3300/0 conn(0x7fead40718b0 0x7fead406f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377596944 0 0) 0x7fead406e4d0 con 0x7fead4071c90 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feabc003530 con 0x7fead4071c90 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1029432614 0 0) 0x7feabc003530 con 0x7fead4071c90 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 >> v2:172.21.15.82:3300/0 conn(0x7fead40718b0 msgr2=0x7fead406f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 --2- 172.21.15.46:0/1167776694 >> v2:172.21.15.82:3300/0 conn(0x7fead40718b0 0x7fead406f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 >> v1:172.21.15.82:6789/0 conn(0x7fead40b3260 legacy=0x7fead406ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead40ad350 con 0x7fead4071c90 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7feac0001680 con 0x7fead4071c90 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 >> v1:172.21.15.46:6789/0 conn(0x7fead4071c90 legacy=0x7fead406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc0039c0 0x7feabc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc0043c0 0x7feabc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feabc005130 0x7feabc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40ad070 con 0x7feabc0043c0 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc00c070 con 0x7feabc005130 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc00c270 con 0x7feabc0039c0 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3e00640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc0043c0 0x7feabc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead9000640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc0039c0 0x7feabc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead9a00640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feabc005130 0x7feabc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3e00640 1 -- 172.21.15.46:0/1167776694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc0039c0 msgr2=0x7feabc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3e00640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc0039c0 0x7feabc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3e00640 1 -- 172.21.15.46:0/1167776694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feabc005130 msgr2=0x7feabc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3e00640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feabc005130 0x7feabc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead3e00640 1 -- 172.21.15.46:0/1167776694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc003530 con 0x7feabc0043c0 2024-04-25T08:43:26.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.799+0000 7fead9a00640 1 -- 172.21.15.46:0/1167776694 reap_dead start 2024-04-25T08:43:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7fead3e00640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc0043c0 0x7feabc004890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7feac000c240 tx=0x7feac001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747f0266aaf2e4de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0007d20 con 0x7feabc0043c0 2024-04-25T08:43:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac00a9020 con 0x7feabc0043c0 2024-04-25T08:43:26.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7fead3400640 1 -- 172.21.15.46:0/1167776694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0005ae0 con 0x7feabc0043c0 2024-04-25T08:43:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7feadba00640 1 -- 172.21.15.46:0/1167776694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc0043c0 msgr2=0x7feabc004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7feadba00640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc0043c0 0x7feabc004890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7feac000c240 tx=0x7feac001cc60 comp rx=0 tx=0).stop 2024-04-25T08:43:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7feadba00640 1 -- 172.21.15.46:0/1167776694 shutdown_connections 2024-04-25T08:43:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7feadba00640 1 --2- 172.21.15.46:0/1167776694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc0043c0 0x7feabc004890 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7feadba00640 1 -- 172.21.15.46:0/1167776694 >> 172.21.15.46:0/1167776694 conn(0x7fead40b16e0 msgr2=0x7fead40b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7feadba00640 1 -- 172.21.15.46:0/1167776694 shutdown_connections 2024-04-25T08:43:26.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.803+0000 7feadba00640 1 -- 172.21.15.46:0/1167776694 wait complete. 2024-04-25T08:43:26.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 Processor -- start 2024-04-25T08:43:26.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 -- start start 2024-04-25T08:43:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc0043c0 0x7fead41534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fead4155f70 0x7fead4159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fead415a0a0 0x7fead415eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:26.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40acd00 con 0x7fead4155f70 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40b3710 con 0x7fead415a0a0 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead406f140 con 0x7feabc0043c0 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fead4155f70 0x7fead4159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fead4155f70 0x7fead4159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56520/0 (socket says 172.21.15.46:56520) 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead3e00640 1 -- 172.21.15.46:0/1739595301 learned_addr learned my addr 172.21.15.46:0/1739595301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead9000640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc0043c0 0x7fead41534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead9a00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fead415a0a0 0x7fead415eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead3e00640 1 -- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc0043c0 msgr2=0x7fead41534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead3e00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc0043c0 0x7fead41534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead3e00640 1 -- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fead415a0a0 msgr2=0x7fead415eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead3e00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fead415a0a0 0x7fead415eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead3e00640 1 -- 172.21.15.46:0/1739595301 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead4153a10 con 0x7fead4155f70 2024-04-25T08:43:26.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead9a00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fead415a0a0 0x7fead415eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:26.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead3e00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fead4155f70 0x7fead4159b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7feac00ab040 tx=0x7feabc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:26.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead0c00640 1 -- 172.21.15.46:0/1739595301 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0001c60 con 0x7fead4155f70 2024-04-25T08:43:26.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead0c00640 1 -- 172.21.15.46:0/1739595301 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac00b2070 con 0x7fead4155f70 2024-04-25T08:43:26.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7fead0c00640 1 -- 172.21.15.46:0/1739595301 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac0004bf0 con 0x7fead4155f70 2024-04-25T08:43:26.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead41551f0 con 0x7fead4155f70 2024-04-25T08:43:26.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.807+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead4155770 con 0x7fead4155f70 2024-04-25T08:43:26.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.811+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead4069080 con 0x7fead4155f70 2024-04-25T08:43:26.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.811+0000 7fead0c00640 1 -- 172.21.15.46:0/1739595301 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feac00ae430 con 0x7fead4155f70 2024-04-25T08:43:26.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.811+0000 7fead0c00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feab4066f90 0x7feab4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:26.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.811+0000 7fead0c00640 1 -- 172.21.15.46:0/1739595301 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5853+0+0 (secure 0 0 0) 0x7feac011e830 con 0x7fead4155f70 2024-04-25T08:43:26.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.811+0000 7fead9000640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feab4066f90 0x7feab4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:26.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.811+0000 7fead9000640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feab4066f90 0x7feab4069450 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fead4159020 tx=0x7feac8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:26.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.815+0000 7fead0c00640 1 -- 172.21.15.46:0/1739595301 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feac0016970 con 0x7fead4155f70 2024-04-25T08:43:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:26.987+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fead4160910 con 0x7fead4155f70 2024-04-25T08:43:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.623+0000 7fead0c00640 1 -- 172.21.15.46:0/1739595301 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) v1 ==== 178+0+0 (secure 0 0 0) 0x7feac00ef7b0 con 0x7fead4155f70 2024-04-25T08:43:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data_ec' removed 2024-04-25T08:43:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feab4066f90 msgr2=0x7feab4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feab4066f90 0x7feab4069450 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fead4159020 tx=0x7feac8008040 comp rx=0 tx=0).stop 2024-04-25T08:43:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fead4155f70 msgr2=0x7fead4159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fead4155f70 0x7fead4159b60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7feac00ab040 tx=0x7feabc005840 comp rx=0 tx=0).stop 2024-04-25T08:43:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 shutdown_connections 2024-04-25T08:43:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feab4066f90 0x7feab4069450 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fead415a0a0 0x7fead415eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fead4155f70 0x7fead4159b60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 --2- 172.21.15.46:0/1739595301 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc0043c0 0x7fead41534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 >> 172.21.15.46:0/1739595301 conn(0x7fead40b16e0 msgr2=0x7fead40d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 shutdown_connections 2024-04-25T08:43:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.627+0000 7feadba00640 1 -- 172.21.15.46:0/1739595301 wait complete. 2024-04-25T08:43:27.647 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:43:27.647 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:43:27.647 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:43:27.647 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:27.647 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:43:27.648 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:43:27.648 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:43:27.648 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:27.648 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:43:27.648 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:43:27.648 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:43:27.648 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:27.648 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:43:27.649 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:43:27.649 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:43:27.649 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:27.661 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:43:27.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:43:27.662 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:43:27.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:43:27.663 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:43:27.663 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:43:27.664 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:43:27.664 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:43:27.708 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:43:27.709 DEBUG:teuthology.parallel:result is None 2024-04-25T08:43:27.710 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:43:27.710 DEBUG:teuthology.parallel:result is None 2024-04-25T08:43:27.712 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:43:27.712 DEBUG:teuthology.parallel:result is None 2024-04-25T08:43:27.714 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:43:27.714 DEBUG:teuthology.parallel:result is None 2024-04-25T08:43:27.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:43:27.800 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-04-25T08:43:27.800 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-04-25T08:43:27.802 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:43:27.802 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:43:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f840ce00640 1 Processor -- start 2024-04-25T08:43:27.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f840ce00640 1 -- start start 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f840ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8408070c90 0x7f8408071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f840ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f840806c380 0x7f840806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f840ce00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84080b5250 con 0x7f840815dbd0 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f840ce00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84080b5af0 con 0x7f8408070c90 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f840ce00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84080b5530 con 0x7f840806c380 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f83fd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8408070c90 0x7f8408071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f83fd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8408070c90 0x7f8408071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56554/0 (socket says 172.21.15.46:56554) 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f83fd400640 1 -- 172.21.15.46:0/2433105740 learned_addr learned my addr 172.21.15.46:0/2433105740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.871+0000 7f8406800640 1 --2- 172.21.15.46:0/2433105740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f840806c380 0x7f840806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:27.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f83fd400640 1 -- 172.21.15.46:0/2433105740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f840806c380 msgr2=0x7f840806c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:27.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f83fd400640 1 --2- 172.21.15.46:0/2433105740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f840806c380 0x7f840806c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:27.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f83fd400640 1 -- 172.21.15.46:0/2433105740 >> v1:172.21.15.82:6789/0 conn(0x7f840815dbd0 legacy=0x7f84080b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:27.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f83fd400640 1 -- 172.21.15.46:0/2433105740 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84080b5810 con 0x7f8408070c90 2024-04-25T08:43:27.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f83fd400640 1 --2- 172.21.15.46:0/2433105740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8408070c90 0x7f8408071110 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009e50 tx=0x7f83f001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28bd3f189e392d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:27.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f8405400640 1 -- 172.21.15.46:0/2433105740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f00a5700 con 0x7f8408070c90 2024-04-25T08:43:27.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f8405400640 1 -- 172.21.15.46:0/2433105740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f00a58a0 con 0x7f8408070c90 2024-04-25T08:43:27.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f8405400640 1 -- 172.21.15.46:0/2433105740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f00a5c70 con 0x7f8408070c90 2024-04-25T08:43:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f840ce00640 1 -- 172.21.15.46:0/2433105740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8408070c90 msgr2=0x7f8408071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f840ce00640 1 --2- 172.21.15.46:0/2433105740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8408070c90 0x7f8408071110 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009e50 tx=0x7f83f001ca30 comp rx=0 tx=0).stop 2024-04-25T08:43:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f840ce00640 1 -- 172.21.15.46:0/2433105740 shutdown_connections 2024-04-25T08:43:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f840ce00640 1 --2- 172.21.15.46:0/2433105740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f840806c380 0x7f840806c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f840ce00640 1 --2- 172.21.15.46:0/2433105740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8408070c90 0x7f8408071110 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f840ce00640 1 -- 172.21.15.46:0/2433105740 >> 172.21.15.46:0/2433105740 conn(0x7f840806f4f0 msgr2=0x7f840806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f840ce00640 1 -- 172.21.15.46:0/2433105740 shutdown_connections 2024-04-25T08:43:27.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.875+0000 7f840ce00640 1 -- 172.21.15.46:0/2433105740 wait complete. 2024-04-25T08:43:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 Processor -- start 2024-04-25T08:43:27.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 -- start start 2024-04-25T08:43:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840806c380 0x7f840815f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:27.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8408070c90 0x7f840815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f840815dbd0 0x7f8408155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840816a850 con 0x7f840806c380 2024-04-25T08:43:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84080b1010 con 0x7f840815dbd0 2024-04-25T08:43:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840816a9d0 con 0x7f8408070c90 2024-04-25T08:43:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8405e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840806c380 0x7f840815f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8405e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840806c380 0x7f840815f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56570/0 (socket says 172.21.15.46:56570) 2024-04-25T08:43:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8405e00640 1 -- 172.21.15.46:0/2419467075 learned_addr learned my addr 172.21.15.46:0/2419467075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8406800640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f840815dbd0 0x7f8408155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:27.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f83fd400640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8408070c90 0x7f840815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:27.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8405e00640 1 -- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8408070c90 msgr2=0x7f840815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:27.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8405e00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8408070c90 0x7f840815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:27.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8405e00640 1 -- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f840815dbd0 msgr2=0x7f8408155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:27.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8405e00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f840815dbd0 0x7f8408155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:27.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8405e00640 1 -- 172.21.15.46:0/2419467075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840815f630 con 0x7f840806c380 2024-04-25T08:43:27.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8406800640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f840815dbd0 0x7f8408155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:27.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f8405e00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840806c380 0x7f840815f0f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f83f800c3f0 tx=0x7f83f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:27.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f83fea00640 1 -- 172.21.15.46:0/2419467075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8017070 con 0x7f840806c380 2024-04-25T08:43:27.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f83fea00640 1 -- 172.21.15.46:0/2419467075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f80052d0 con 0x7f840806c380 2024-04-25T08:43:27.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f83fea00640 1 -- 172.21.15.46:0/2419467075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f8003840 con 0x7f840806c380 2024-04-25T08:43:27.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840815e050 con 0x7f840806c380 2024-04-25T08:43:27.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.879+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840815e610 con 0x7f840806c380 2024-04-25T08:43:27.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.883+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8408069080 con 0x7f840806c380 2024-04-25T08:43:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.883+0000 7f83fea00640 1 -- 172.21.15.46:0/2419467075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83f801a070 con 0x7f840806c380 2024-04-25T08:43:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.883+0000 7f83fea00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f83e4066f60 0x7f83e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:27.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.883+0000 7f83fea00640 1 -- 172.21.15.46:0/2419467075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f83f80841a0 con 0x7f840806c380 2024-04-25T08:43:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.883+0000 7f83fd400640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f83e4066f60 0x7f83e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:27.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.883+0000 7f83fd400640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f83e4066f60 0x7f83e4069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83f00ae000 tx=0x7f83f00a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:27.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:27.887+0000 7f83fea00640 1 -- 172.21.15.46:0/2419467075 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83f8005480 con 0x7f840806c380 2024-04-25T08:43:28.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.059+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8408073db0 con 0x7f840806c380 2024-04-25T08:43:28.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f83fea00640 1 -- 172.21.15.46:0/2419467075 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) v1 ==== 97+0+0 (secure 0 0 0) 0x7f83f8054cc0 con 0x7f840806c380 2024-04-25T08:43:28.627 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-25T08:43:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f83e4066f60 msgr2=0x7f83e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f83e4066f60 0x7f83e4069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f83f00ae000 tx=0x7f83f00a7070 comp rx=0 tx=0).stop 2024-04-25T08:43:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840806c380 msgr2=0x7f840815f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840806c380 0x7f840815f0f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f83f800c3f0 tx=0x7f83f800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:43:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 shutdown_connections 2024-04-25T08:43:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f83e4066f60 0x7f83e4069420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f840815dbd0 0x7f8408155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:28.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8408070c90 0x7f840815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 --2- 172.21.15.46:0/2419467075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f840806c380 0x7f840815f0f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 >> 172.21.15.46:0/2419467075 conn(0x7f840806f4f0 msgr2=0x7f8408073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 shutdown_connections 2024-04-25T08:43:28.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.635+0000 7f840ce00640 1 -- 172.21.15.46:0/2419467075 wait complete. 2024-04-25T08:43:28.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:43:28.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60db000640 1 Processor -- start 2024-04-25T08:43:28.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60db000640 1 -- start start 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60db000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60d40aed10 0x7f60d4165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60db000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d40b35b0 con 0x7f60d406f320 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60db000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d406d5e0 con 0x7f60d415a750 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60db000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d415ab30 con 0x7f60d40aed10 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60cb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f60d406f320 0x7f60d4162200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44002/0 (socket says 172.21.15.46:44002) 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60cb400640 1 -- 172.21.15.46:0/172964131 learned_addr learned my addr 172.21.15.46:0/172964131 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60d9000640 1 --2- 172.21.15.46:0/172964131 >> v2:172.21.15.46:3300/0 conn(0x7f60d40aed10 0x7f60d4165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.803+0000 7f60d3400640 1 -- 172.21.15.46:0/172964131 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851171220 0 0) 0x7f60d40b35b0 con 0x7f60d406f320 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60d3400640 1 -- 172.21.15.46:0/172964131 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60b8003530 con 0x7f60d406f320 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60d9000640 1 -- 172.21.15.46:0/172964131 >> v1:172.21.15.82:6789/0 conn(0x7f60d415a750 legacy=0x7f60d406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:28.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60d9000640 1 -- 172.21.15.46:0/172964131 >> v1:172.21.15.46:6789/0 conn(0x7f60d406f320 legacy=0x7f60d4162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:28.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60d9000640 1 -- 172.21.15.46:0/172964131 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d40ce720 con 0x7f60d40aed10 2024-04-25T08:43:28.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60d9000640 1 --2- 172.21.15.46:0/172964131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d40aed10 0x7f60d4165930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0097c0 tx=0x7f60cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc654cd44ba6c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:28.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60d3400640 1 -- 172.21.15.46:0/172964131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0b0070 con 0x7f60d40aed10 2024-04-25T08:43:28.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60d3400640 1 -- 172.21.15.46:0/172964131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60cc0ab600 con 0x7f60d40aed10 2024-04-25T08:43:28.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60d3400640 1 -- 172.21.15.46:0/172964131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0ab9d0 con 0x7f60d40aed10 2024-04-25T08:43:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60db000640 1 -- 172.21.15.46:0/172964131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d40aed10 msgr2=0x7f60d4165930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60db000640 1 --2- 172.21.15.46:0/172964131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d40aed10 0x7f60d4165930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0097c0 tx=0x7f60cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:43:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60db000640 1 -- 172.21.15.46:0/172964131 shutdown_connections 2024-04-25T08:43:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60db000640 1 --2- 172.21.15.46:0/172964131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d40aed10 0x7f60d4165930 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60db000640 1 -- 172.21.15.46:0/172964131 >> 172.21.15.46:0/172964131 conn(0x7f60d40b12a0 msgr2=0x7f60d40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60db000640 1 -- 172.21.15.46:0/172964131 shutdown_connections 2024-04-25T08:43:28.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.807+0000 7f60db000640 1 -- 172.21.15.46:0/172964131 wait complete. 2024-04-25T08:43:28.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 Processor -- start 2024-04-25T08:43:28.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 -- start start 2024-04-25T08:43:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60d406f320 0x7f60d415d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60d40aed10 0x7f60d414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d415a750 0x7f60d415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4168690 con 0x7f60d415a750 2024-04-25T08:43:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d406d270 con 0x7f60d406f320 2024-04-25T08:43:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d4168810 con 0x7f60d40aed10 2024-04-25T08:43:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d415a750 0x7f60d415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d415a750 0x7f60d415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33974/0 (socket says 172.21.15.46:33974) 2024-04-25T08:43:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d9000640 1 -- 172.21.15.46:0/2831011485 learned_addr learned my addr 172.21.15.46:0/2831011485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d3e00640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60d406f320 0x7f60d415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60cb400640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60d40aed10 0x7f60d414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d9000640 1 -- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60d40aed10 msgr2=0x7f60d414b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d9000640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60d40aed10 0x7f60d414b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d9000640 1 -- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60d406f320 msgr2=0x7f60d415d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d9000640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60d406f320 0x7f60d415d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d9000640 1 -- 172.21.15.46:0/2831011485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d40cd500 con 0x7f60d415a750 2024-04-25T08:43:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d9000640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d415a750 0x7f60d415b4d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0ae040 tx=0x7f60cc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d0c00640 1 -- 172.21.15.46:0/2831011485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0b8020 con 0x7f60d415a750 2024-04-25T08:43:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d0c00640 1 -- 172.21.15.46:0/2831011485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60cc0b4410 con 0x7f60d415a750 2024-04-25T08:43:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60d0c00640 1 -- 172.21.15.46:0/2831011485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0b0040 con 0x7f60d415a750 2024-04-25T08:43:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d415adf0 con 0x7f60d415a750 2024-04-25T08:43:28.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.811+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d414c740 con 0x7f60d415a750 2024-04-25T08:43:28.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.815+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d4069080 con 0x7f60d415a750 2024-04-25T08:43:28.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.815+0000 7f60d0c00640 1 -- 172.21.15.46:0/2831011485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60cc0bb020 con 0x7f60d415a750 2024-04-25T08:43:28.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.815+0000 7f60d0c00640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60b0066f60 0x7f60b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.815+0000 7f60d0c00640 1 -- 172.21.15.46:0/2831011485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f60cc11ead0 con 0x7f60d415a750 2024-04-25T08:43:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.815+0000 7f60d3e00640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60b0066f60 0x7f60b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:28.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.815+0000 7f60d3e00640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60b0066f60 0x7f60b0069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60c4000f70 tx=0x7f60c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:28.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:28.819+0000 7f60d0c00640 1 -- 172.21.15.46:0/2831011485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60cc0c2030 con 0x7f60d415a750 2024-04-25T08:43:29.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.035+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f60d4156ed0 con 0x7f60d415a750 2024-04-25T08:43:29.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60d0c00640 1 -- 172.21.15.46:0/2831011485 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f60cc016090 con 0x7f60d415a750 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmq+gDGRAAtB5ObM/zelMmCZ4cJPghdQ==", 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmyIaAJRAA9zodtgftP7faNth4UPyCTA==", 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmFahWIBAAORHmzDLXPjuANF+/W2wKpQ==", 2024-04-25T08:43:29.032 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmRhrPLBAATXOHl1jGJYmIlnYk/JPcOw==", 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA0FypmaHiqEhAAYzasCl8t+pls5IHe9tCP4g==", 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA3FypmOGTuGhAA37F+qRI/+pj5dkLc3TBIUQ==", 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.034 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA6Fypm/YbNJRAAP1QyYDLY/xFvRz63R+H/Pw==", 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA9FypmiT8ALhAAX/2CJJeJ+2SGNhwah8NTBQ==", 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBDFypmyG88JBAAq1065gzLIIR/6fYaw0bBTg==", 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:43:29.036 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBGFypmNir3KxAAszCXJJ1KFtCusDZhBHAZSQ==", 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBJFypm4r7LNxAA42A07ntI6OOgblk+4i1Yiw==", 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBNFypmG1fOBhAAOKRgw04jIo8H3u2hWIZe2g==", 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.038 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q==", 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmxPyHOhAADBuigZlq3GHURsJkB0nnyA==", 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:43:29.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypm2PP0NhAAixV/kkY7eFPXUW/nJDOCOA==", 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:43:29.040 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAzFypmaVDtBBAAKnWk9CxZzg2r6T/IycuFEg==", 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.041 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAxFypm2W2MOBAAN7TDQiZXFV6+pWMTar95sg==", 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mds", 2024-04-25T08:43:29.042 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBTFypmByDfBRAA6QgkdviBlC/A3FKwVOl4VA==", 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBTFypmtirfBRAA5GXRsX/u+Vxw/vl9fWVg4g==", 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-osd", 2024-04-25T08:43:29.043 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBTFypmKDPfBRAAEICvC9ExuxdFLw20CqOdKw==", 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBTFypmWjvfBRAAxfPvkjGEW21VYsU/0X2+dQ==", 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBTFypmTkXfBRAAt9LHHhOiAgfQgkzEbWhxDg==", 2024-04-25T08:43:29.044 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBTFypm703fBRAALKWcQQFf+yPV45HQlqryrA==", 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.045 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmzhptFBAAlFQZQn1GSseEguy5zfw2CA==", 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmAY6CEBAA2qHNALNTRKmCJRZatDqAvw==", 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:43:29.046 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:43:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:29.047 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-25T08:43:29.047 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60b0066f60 msgr2=0x7f60b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60b0066f60 0x7f60b0069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f60c4000f70 tx=0x7f60c4009040 comp rx=0 tx=0).stop 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d415a750 msgr2=0x7f60d415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d415a750 0x7f60d415b4d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0ae040 tx=0x7f60cc019240 comp rx=0 tx=0).stop 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 shutdown_connections 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60b0066f60 0x7f60b0069420 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60d415a750 0x7f60d415b4d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60d40aed10 0x7f60d414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 --2- 172.21.15.46:0/2831011485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60d406f320 0x7f60d415d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 >> 172.21.15.46:0/2831011485 conn(0x7f60d40b12a0 msgr2=0x7f60d40b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 shutdown_connections 2024-04-25T08:43:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.039+0000 7f60db000640 1 -- 172.21.15.46:0/2831011485 wait complete. 2024-04-25T08:43:29.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T08:43:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe12d400640 1 Processor -- start 2024-04-25T08:43:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe12d400640 1 -- start start 2024-04-25T08:43:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe12d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1280ccde0 0x7fe1280cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe12d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 0x7fe1280b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe12d400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12806bdc0 con 0x7fe128099a00 2024-04-25T08:43:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe12d400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12806db90 con 0x7fe1280ccde0 2024-04-25T08:43:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe12d400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12806c0a0 con 0x7fe1280b2240 2024-04-25T08:43:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe11fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1280ccde0 0x7fe1280cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe11fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1280ccde0 0x7fe1280cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33990/0 (socket says 172.21.15.46:33990) 2024-04-25T08:43:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe11fe00640 1 -- 172.21.15.46:0/1981647410 learned_addr learned my addr 172.21.15.46:0/1981647410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe127200640 1 --2- 172.21.15.46:0/1981647410 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 0x7fe1280b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe11fe00640 1 -- 172.21.15.46:0/1981647410 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 msgr2=0x7fe1280b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe11fe00640 1 --2- 172.21.15.46:0/1981647410 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 0x7fe1280b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe11fe00640 1 -- 172.21.15.46:0/1981647410 >> v1:172.21.15.82:6789/0 conn(0x7fe128099a00 legacy=0x7fe128099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.203+0000 7fe11fe00640 1 -- 172.21.15.46:0/1981647410 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12806c380 con 0x7fe1280ccde0 2024-04-25T08:43:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe11fe00640 1 --2- 172.21.15.46:0/1981647410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1280ccde0 0x7fe1280cd290 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe1100098a0 tx=0x7fe11001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b29ad0f0d9c0708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe125e00640 1 -- 172.21.15.46:0/1981647410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1100ab610 con 0x7fe1280ccde0 2024-04-25T08:43:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe125e00640 1 -- 172.21.15.46:0/1981647410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1100b0070 con 0x7fe1280ccde0 2024-04-25T08:43:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe125e00640 1 -- 172.21.15.46:0/1981647410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1100ab9d0 con 0x7fe1280ccde0 2024-04-25T08:43:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe12d400640 1 -- 172.21.15.46:0/1981647410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1280ccde0 msgr2=0x7fe1280cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe12d400640 1 --2- 172.21.15.46:0/1981647410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1280ccde0 0x7fe1280cd290 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe1100098a0 tx=0x7fe11001c480 comp rx=0 tx=0).stop 2024-04-25T08:43:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe12d400640 1 -- 172.21.15.46:0/1981647410 shutdown_connections 2024-04-25T08:43:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe12d400640 1 --2- 172.21.15.46:0/1981647410 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 0x7fe1280b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe12d400640 1 --2- 172.21.15.46:0/1981647410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1280ccde0 0x7fe1280cd290 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe12d400640 1 -- 172.21.15.46:0/1981647410 >> 172.21.15.46:0/1981647410 conn(0x7fe1280b3120 msgr2=0x7fe1280b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe12d400640 1 -- 172.21.15.46:0/1981647410 shutdown_connections 2024-04-25T08:43:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.207+0000 7fe12d400640 1 -- 172.21.15.46:0/1981647410 wait complete. 2024-04-25T08:43:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 Processor -- start 2024-04-25T08:43:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 -- start start 2024-04-25T08:43:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128099a00 0x7fe128156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 0x7fe128154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1280ccde0 0x7fe12806f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280cd7d0 con 0x7fe128099a00 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12815dd10 con 0x7fe1280ccde0 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe128155180 con 0x7fe1280b2240 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe126800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128099a00 0x7fe128156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe126800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128099a00 0x7fe128156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33992/0 (socket says 172.21.15.46:33992) 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe126800640 1 -- 172.21.15.46:0/2330332793 learned_addr learned my addr 172.21.15.46:0/2330332793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe127200640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1280ccde0 0x7fe12806f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe11fe00640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 0x7fe128154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe126800640 1 -- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 msgr2=0x7fe128154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe126800640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 0x7fe128154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe126800640 1 -- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1280ccde0 msgr2=0x7fe12806f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe126800640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1280ccde0 0x7fe12806f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe126800640 1 -- 172.21.15.46:0/2330332793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128150520 con 0x7fe128099a00 2024-04-25T08:43:29.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe126800640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128099a00 0x7fe128156e90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c370 tx=0x7fe11800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe11ea00640 1 -- 172.21.15.46:0/2330332793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1180052d0 con 0x7fe128099a00 2024-04-25T08:43:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe11ea00640 1 -- 172.21.15.46:0/2330332793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe118005470 con 0x7fe128099a00 2024-04-25T08:43:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe11ea00640 1 -- 172.21.15.46:0/2330332793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118007260 con 0x7fe128099a00 2024-04-25T08:43:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1281507c0 con 0x7fe128099a00 2024-04-25T08:43:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.211+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe128150d80 con 0x7fe128099a00 2024-04-25T08:43:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.215+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe128069080 con 0x7fe128099a00 2024-04-25T08:43:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.215+0000 7fe11ea00640 1 -- 172.21.15.46:0/2330332793 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe118007400 con 0x7fe128099a00 2024-04-25T08:43:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.215+0000 7fe11ea00640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe104066f60 0x7fe104069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.215+0000 7fe11ea00640 1 -- 172.21.15.46:0/2330332793 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe1180842c0 con 0x7fe128099a00 2024-04-25T08:43:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.215+0000 7fe11fe00640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe104066f60 0x7fe104069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.215+0000 7fe11fe00640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe104066f60 0x7fe104069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1100af000 tx=0x7fe1100a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.219+0000 7fe11ea00640 1 -- 172.21.15.46:0/2330332793 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe118054da0 con 0x7fe128099a00 2024-04-25T08:43:29.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.439+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe128099ef0 con 0x7fe128099a00 2024-04-25T08:43:29.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.439+0000 7fe11ea00640 1 -- 172.21.15.46:0/2330332793 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe118084080 con 0x7fe128099a00 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe104066f60 msgr2=0x7fe104069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe104066f60 0x7fe104069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe1100af000 tx=0x7fe1100a7000 comp rx=0 tx=0).stop 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128099a00 msgr2=0x7fe128156e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128099a00 0x7fe128156e90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c370 tx=0x7fe11800c830 comp rx=0 tx=0).stop 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 shutdown_connections 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe104066f60 0x7fe104069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1280ccde0 0x7fe12806f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1280b2240 0x7fe128154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 --2- 172.21.15.46:0/2330332793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128099a00 0x7fe128156e90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 >> 172.21.15.46:0/2330332793 conn(0x7fe1280b3120 msgr2=0x7fe1280cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 shutdown_connections 2024-04-25T08:43:29.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.443+0000 7fe12d400640 1 -- 172.21.15.46:0/2330332793 wait complete. 2024-04-25T08:43:29.455 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T08:43:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d9800640 1 Processor -- start 2024-04-25T08:43:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d9800640 1 -- start start 2024-04-25T08:43:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d9800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5d4157b00 0x7fe5d40b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40cc150 0x7fe5d40cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d9800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d40b25a0 con 0x7fe5d40ce250 2024-04-25T08:43:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d9800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40b2eb0 con 0x7fe5d4157b00 2024-04-25T08:43:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d40cc6f0 con 0x7fe5d40cc150 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5d4157b00 0x7fe5d40b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5d4157b00 0x7fe5d40b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34004/0 (socket says 172.21.15.46:34004) 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d2a00640 1 -- 172.21.15.46:0/3392035131 learned_addr learned my addr 172.21.15.46:0/3392035131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.603+0000 7fe5d3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe5d40ce250 0x7fe5d415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44016/0 (socket says 172.21.15.46:44016) 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5ca000640 1 --2- 172.21.15.46:0/3392035131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40cc150 0x7fe5d40cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d2a00640 1 -- 172.21.15.46:0/3392035131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40cc150 msgr2=0x7fe5d40cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d2a00640 1 --2- 172.21.15.46:0/3392035131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40cc150 0x7fe5d40cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d2a00640 1 -- 172.21.15.46:0/3392035131 >> v1:172.21.15.46:6789/0 conn(0x7fe5d40ce250 legacy=0x7fe5d415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d2a00640 1 -- 172.21.15.46:0/3392035131 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40b2860 con 0x7fe5d4157b00 2024-04-25T08:43:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d2000640 1 -- 172.21.15.46:0/3392035131 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190170760 0 0) 0x7fe5d40b25a0 con 0x7fe5d40ce250 2024-04-25T08:43:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d2a00640 1 --2- 172.21.15.46:0/3392035131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d4157b00 0x7fe5d40b3340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009e50 tx=0x7fe5c401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5903cc8d2cc6eb4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d2000640 1 -- 172.21.15.46:0/3392035131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40a56c0 con 0x7fe5d4157b00 2024-04-25T08:43:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d2000640 1 -- 172.21.15.46:0/3392035131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c40a5860 con 0x7fe5d4157b00 2024-04-25T08:43:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d2000640 1 -- 172.21.15.46:0/3392035131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40a5c30 con 0x7fe5d4157b00 2024-04-25T08:43:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d9800640 1 -- 172.21.15.46:0/3392035131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d4157b00 msgr2=0x7fe5d40b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.607+0000 7fe5d9800640 1 --2- 172.21.15.46:0/3392035131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d4157b00 0x7fe5d40b3340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009e50 tx=0x7fe5c401c970 comp rx=0 tx=0).stop 2024-04-25T08:43:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 -- 172.21.15.46:0/3392035131 shutdown_connections 2024-04-25T08:43:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 --2- 172.21.15.46:0/3392035131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40cc150 0x7fe5d40cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 --2- 172.21.15.46:0/3392035131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d4157b00 0x7fe5d40b3340 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 -- 172.21.15.46:0/3392035131 >> 172.21.15.46:0/3392035131 conn(0x7fe5d40b12a0 msgr2=0x7fe5d40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 -- 172.21.15.46:0/3392035131 shutdown_connections 2024-04-25T08:43:29.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 -- 172.21.15.46:0/3392035131 wait complete. 2024-04-25T08:43:29.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 Processor -- start 2024-04-25T08:43:29.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 -- start start 2024-04-25T08:43:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d40cc150 0x7fe5d41638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40ce250 0x7fe5d41537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5d4157b00 0x7fe5d4151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d415aef0 con 0x7fe5d40cc150 2024-04-25T08:43:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d415ad70 con 0x7fe5d4157b00 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d415b070 con 0x7fe5d40ce250 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d40cc150 0x7fe5d41638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d40cc150 0x7fe5d41638c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34016/0 (socket says 172.21.15.46:34016) 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d2a00640 1 -- 172.21.15.46:0/1646882638 learned_addr learned my addr 172.21.15.46:0/1646882638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5ca000640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40ce250 0x7fe5d41537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d3400640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5d4157b00 0x7fe5d4151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d2a00640 1 -- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40ce250 msgr2=0x7fe5d41537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d2a00640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40ce250 0x7fe5d41537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d2a00640 1 -- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5d4157b00 msgr2=0x7fe5d4151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d2a00640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5d4157b00 0x7fe5d4151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d2a00640 1 -- 172.21.15.46:0/1646882638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4154d90 con 0x7fe5d40cc150 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d3400640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5d4157b00 0x7fe5d4151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5d2a00640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d40cc150 0x7fe5d41638c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe5c401a5a0 tx=0x7fe5c4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:29.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.611+0000 7fe5cb400640 1 -- 172.21.15.46:0/1646882638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40bc020 con 0x7fe5d40cc150 2024-04-25T08:43:29.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.615+0000 7fe5cb400640 1 -- 172.21.15.46:0/1646882638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c40b8670 con 0x7fe5d40cc150 2024-04-25T08:43:29.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.615+0000 7fe5cb400640 1 -- 172.21.15.46:0/1646882638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40b8a60 con 0x7fe5d40cc150 2024-04-25T08:43:29.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.615+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d41554a0 con 0x7fe5d40cc150 2024-04-25T08:43:29.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.615+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d4163e00 con 0x7fe5d40cc150 2024-04-25T08:43:29.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.615+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d4069080 con 0x7fe5d40cc150 2024-04-25T08:43:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.615+0000 7fe5cb400640 1 -- 172.21.15.46:0/1646882638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5c40bf020 con 0x7fe5d40cc150 2024-04-25T08:43:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.615+0000 7fe5cb400640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5b0066f60 0x7fe5b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.615+0000 7fe5cb400640 1 -- 172.21.15.46:0/1646882638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe5c411fbf0 con 0x7fe5d40cc150 2024-04-25T08:43:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.615+0000 7fe5ca000640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5b0066f60 0x7fe5b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.619+0000 7fe5ca000640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5b0066f60 0x7fe5b0069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5d41644a0 tx=0x7fe5bc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:29.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.619+0000 7fe5cb400640 1 -- 172.21.15.46:0/1646882638 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5c40c6030 con 0x7fe5d40cc150 2024-04-25T08:43:29.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.839+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fe5d40724a0 con 0x7fe5d40cc150 2024-04-25T08:43:29.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.843+0000 7fe5cb400640 1 -- 172.21.15.46:0/1646882638 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe5c4016620 con 0x7fe5d40cc150 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5b0066f60 msgr2=0x7fe5b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5b0066f60 0x7fe5b0069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5d41644a0 tx=0x7fe5bc009190 comp rx=0 tx=0).stop 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d40cc150 msgr2=0x7fe5d41638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d40cc150 0x7fe5d41638c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe5c401a5a0 tx=0x7fe5c4019710 comp rx=0 tx=0).stop 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 shutdown_connections 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe5b0066f60 0x7fe5b0069420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5d4157b00 0x7fe5d4151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe5d40ce250 0x7fe5d41537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 --2- 172.21.15.46:0/1646882638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5d40cc150 0x7fe5d41638c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 >> 172.21.15.46:0/1646882638 conn(0x7fe5d40b12a0 msgr2=0x7fe5d40af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 shutdown_connections 2024-04-25T08:43:29.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:29.847+0000 7fe5d9800640 1 -- 172.21.15.46:0/1646882638 wait complete. 2024-04-25T08:43:29.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T08:43:29.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466e00640 1 Processor -- start 2024-04-25T08:43:29.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466e00640 1 -- start start 2024-04-25T08:43:30.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600cdd00 0x7f54600ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f54600b10f0 0x7f54600b1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54600cf6d0 con 0x7f546015d7a0 2024-04-25T08:43:30.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54600cf4a0 con 0x7f54600b10f0 2024-04-25T08:43:30.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546006ce40 con 0x7f54600cdd00 2024-04-25T08:43:30.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5465a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f546015d7a0 0x7f546006c730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44022/0 (socket says 172.21.15.46:44022) 2024-04-25T08:43:30.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5465a00640 1 -- 172.21.15.46:0/3567437533 learned_addr learned my addr 172.21.15.46:0/3567437533 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466400640 1 --2- 172.21.15.46:0/3567437533 >> v2:172.21.15.46:3300/0 conn(0x7f54600b10f0 0x7f54600b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5465000640 1 --2- 172.21.15.46:0/3567437533 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600cdd00 0x7f54600ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f545fe00640 1 -- 172.21.15.46:0/3567437533 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 381575156 0 0) 0x7f54600cf6d0 con 0x7f546015d7a0 2024-04-25T08:43:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f545fe00640 1 -- 172.21.15.46:0/3567437533 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5448003530 con 0x7f546015d7a0 2024-04-25T08:43:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466400640 1 -- 172.21.15.46:0/3567437533 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600cdd00 msgr2=0x7f54600ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466400640 1 --2- 172.21.15.46:0/3567437533 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600cdd00 0x7f54600ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466400640 1 -- 172.21.15.46:0/3567437533 >> v1:172.21.15.46:6789/0 conn(0x7f546015d7a0 legacy=0x7f546006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5466400640 1 -- 172.21.15.46:0/3567437533 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600cf9b0 con 0x7f54600b10f0 2024-04-25T08:43:30.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.007+0000 7f5465000640 1 --2- 172.21.15.46:0/3567437533 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600cdd00 0x7f54600ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:30.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f5466400640 1 --2- 172.21.15.46:0/3567437533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600b10f0 0x7f54600b1560 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5458009d90 tx=0x7f545801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04978f46e258aba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:30.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f545fe00640 1 -- 172.21.15.46:0/3567437533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580b2070 con 0x7f54600b10f0 2024-04-25T08:43:30.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f545fe00640 1 -- 172.21.15.46:0/3567437533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54580a5920 con 0x7f54600b10f0 2024-04-25T08:43:30.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f545fe00640 1 -- 172.21.15.46:0/3567437533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580a5cf0 con 0x7f54600b10f0 2024-04-25T08:43:30.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f5466e00640 1 -- 172.21.15.46:0/3567437533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600b10f0 msgr2=0x7f54600b1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f5466e00640 1 --2- 172.21.15.46:0/3567437533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600b10f0 0x7f54600b1560 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5458009d90 tx=0x7f545801c970 comp rx=0 tx=0).stop 2024-04-25T08:43:30.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f5466e00640 1 -- 172.21.15.46:0/3567437533 shutdown_connections 2024-04-25T08:43:30.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f5466e00640 1 --2- 172.21.15.46:0/3567437533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54600b10f0 0x7f54600b1560 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f5466e00640 1 --2- 172.21.15.46:0/3567437533 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600cdd00 0x7f54600ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f5466e00640 1 -- 172.21.15.46:0/3567437533 >> 172.21.15.46:0/3567437533 conn(0x7f54600d1480 msgr2=0x7f54600d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:30.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f5466e00640 1 -- 172.21.15.46:0/3567437533 shutdown_connections 2024-04-25T08:43:30.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.011+0000 7f5466e00640 1 -- 172.21.15.46:0/3567437533 wait complete. 2024-04-25T08:43:30.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 Processor -- start 2024-04-25T08:43:30.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 -- start start 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600b10f0 0x7f546014a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54600cdd00 0x7f54600cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546015d7a0 0x7f5460099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54600d0770 con 0x7f546015d7a0 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546015db80 con 0x7f54600cdd00 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54600b5480 con 0x7f54600b10f0 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546015d7a0 0x7f5460099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546015d7a0 0x7f5460099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34028/0 (socket says 172.21.15.46:34028) 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466400640 1 -- 172.21.15.46:0/2373797738 learned_addr learned my addr 172.21.15.46:0/2373797738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5465000640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54600cdd00 0x7f54600cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5465a00640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600b10f0 0x7f546014a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466400640 1 -- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600b10f0 msgr2=0x7f546014a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466400640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600b10f0 0x7f546014a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466400640 1 -- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54600cdd00 msgr2=0x7f54600cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466400640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54600cdd00 0x7f54600cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466400640 1 -- 172.21.15.46:0/2373797738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54600b43e0 con 0x7f546015d7a0 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466400640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546015d7a0 0x7f5460099a00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f54580b0040 tx=0x7f5458019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f545d600640 1 -- 172.21.15.46:0/2373797738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580b2050 con 0x7f546015d7a0 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f545d600640 1 -- 172.21.15.46:0/2373797738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54580be070 con 0x7f546015d7a0 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f545d600640 1 -- 172.21.15.46:0/2373797738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580ada50 con 0x7f546015d7a0 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54600b46c0 con 0x7f546015d7a0 2024-04-25T08:43:30.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.015+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54600b4c80 con 0x7f546015d7a0 2024-04-25T08:43:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.019+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5460069010 con 0x7f546015d7a0 2024-04-25T08:43:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.019+0000 7f545d600640 1 -- 172.21.15.46:0/2373797738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54580adbf0 con 0x7f546015d7a0 2024-04-25T08:43:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.019+0000 7f545d600640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5440066f60 0x7f5440069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.019+0000 7f545d600640 1 -- 172.21.15.46:0/2373797738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5458120080 con 0x7f546015d7a0 2024-04-25T08:43:30.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.019+0000 7f5465a00640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5440066f60 0x7f5440069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.019+0000 7f5465a00640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5440066f60 0x7f5440069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5454005890 tx=0x7f5454009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:30.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.023+0000 7f545d600640 1 -- 172.21.15.46:0/2373797738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5458016560 con 0x7f546015d7a0 2024-04-25T08:43:30.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.243+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f546006f690 con 0x7f546015d7a0 2024-04-25T08:43:30.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f545d600640 1 -- 172.21.15.46:0/2373797738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f54580ab340 con 0x7f546015d7a0 2024-04-25T08:43:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5440066f60 msgr2=0x7f5440069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5440066f60 0x7f5440069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5454005890 tx=0x7f5454009040 comp rx=0 tx=0).stop 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546015d7a0 msgr2=0x7f5460099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546015d7a0 0x7f5460099a00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f54580b0040 tx=0x7f5458019710 comp rx=0 tx=0).stop 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 shutdown_connections 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5440066f60 0x7f5440069420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f546015d7a0 0x7f5460099a00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54600cdd00 0x7f54600cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 --2- 172.21.15.46:0/2373797738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54600b10f0 0x7f546014a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 >> 172.21.15.46:0/2373797738 conn(0x7f54600d1480 msgr2=0x7f54600cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 shutdown_connections 2024-04-25T08:43:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.247+0000 7f5466e00640 1 -- 172.21.15.46:0/2373797738 wait complete. 2024-04-25T08:43:30.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T08:43:30.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6614800640 1 Processor -- start 2024-04-25T08:43:30.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6614800640 1 -- start start 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6614800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f660c15a750 0x7f660c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6614800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f660c0aed10 0x7f660c06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6614800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660c06d5e0 con 0x7f660c0cc9e0 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6614800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c15ab30 con 0x7f660c0aed10 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6614800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660c0ce4f0 con 0x7f660c15a750 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6611400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f660c0aed10 0x7f660c06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6612800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f660c0cc9e0 0x7f660c0b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44036/0 (socket says 172.21.15.46:44036) 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6611400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f660c0aed10 0x7f660c06f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34036/0 (socket says 172.21.15.46:34036) 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6612800640 1 -- 172.21.15.46:0/1989785609 learned_addr learned my addr 172.21.15.46:0/1989785609 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6611e00640 1 --2- 172.21.15.46:0/1989785609 >> v2:172.21.15.82:3300/0 conn(0x7f660c15a750 0x7f660c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6611400640 1 -- 172.21.15.46:0/1989785609 >> v2:172.21.15.82:3300/0 conn(0x7f660c15a750 msgr2=0x7f660c06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6611400640 1 --2- 172.21.15.46:0/1989785609 >> v2:172.21.15.82:3300/0 conn(0x7f660c15a750 0x7f660c06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6611400640 1 -- 172.21.15.46:0/1989785609 >> v1:172.21.15.46:6789/0 conn(0x7f660c0cc9e0 legacy=0x7f660c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6611400640 1 -- 172.21.15.46:0/1989785609 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0ce720 con 0x7f660c0aed10 2024-04-25T08:43:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6610a00640 1 -- 172.21.15.46:0/1989785609 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093720247 0 0) 0x7f660c06d5e0 con 0x7f660c0cc9e0 2024-04-25T08:43:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.411+0000 7f6611e00640 1 --2- 172.21.15.46:0/1989785609 >> v2:172.21.15.82:3300/0 conn(0x7f660c15a750 0x7f660c06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6611400640 1 --2- 172.21.15.46:0/1989785609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0aed10 0x7f660c06f150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f660c04ffd0 tx=0x7f65fc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b974e11c6afd7329 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6610a00640 1 -- 172.21.15.46:0/1989785609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0b0070 con 0x7f660c0aed10 2024-04-25T08:43:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6610a00640 1 -- 172.21.15.46:0/1989785609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc0ab600 con 0x7f660c0aed10 2024-04-25T08:43:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6610a00640 1 -- 172.21.15.46:0/1989785609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0ab9d0 con 0x7f660c0aed10 2024-04-25T08:43:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 -- 172.21.15.46:0/1989785609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0aed10 msgr2=0x7f660c06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 --2- 172.21.15.46:0/1989785609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0aed10 0x7f660c06f150 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f660c04ffd0 tx=0x7f65fc01c480 comp rx=0 tx=0).stop 2024-04-25T08:43:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 -- 172.21.15.46:0/1989785609 shutdown_connections 2024-04-25T08:43:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 --2- 172.21.15.46:0/1989785609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0aed10 0x7f660c06f150 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 --2- 172.21.15.46:0/1989785609 >> v2:172.21.15.82:3300/0 conn(0x7f660c15a750 0x7f660c06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 -- 172.21.15.46:0/1989785609 >> 172.21.15.46:0/1989785609 conn(0x7f660c0b12a0 msgr2=0x7f660c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 -- 172.21.15.46:0/1989785609 shutdown_connections 2024-04-25T08:43:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 -- 172.21.15.46:0/1989785609 wait complete. 2024-04-25T08:43:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 Processor -- start 2024-04-25T08:43:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.415+0000 7f6614800640 1 -- start start 2024-04-25T08:43:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6614800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f660c0aed10 0x7f660c154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6614800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0cc9e0 0x7f660c143020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6614800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f660c15a750 0x7f660c0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6614800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c06db00 con 0x7f660c0cc9e0 2024-04-25T08:43:30.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6614800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660c06d270 con 0x7f660c15a750 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6614800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660c06dc80 con 0x7f660c0aed10 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0cc9e0 0x7f660c143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0cc9e0 0x7f660c143020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34040/0 (socket says 172.21.15.46:34040) 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611400640 1 -- 172.21.15.46:0/2338569129 learned_addr learned my addr 172.21.15.46:0/2338569129 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611e00640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f660c0aed10 0x7f660c154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6612800640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f660c15a750 0x7f660c0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611400640 1 -- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f660c0aed10 msgr2=0x7f660c154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611400640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f660c0aed10 0x7f660c154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611400640 1 -- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f660c15a750 msgr2=0x7f660c0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611400640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f660c15a750 0x7f660c0ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611400640 1 -- 172.21.15.46:0/2338569129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c143560 con 0x7f660c0cc9e0 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6612800640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f660c15a750 0x7f660c0ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6611400640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0cc9e0 0x7f660c143020 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65fc01a0b0 tx=0x7f65fc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:30.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6606000640 1 -- 172.21.15.46:0/2338569129 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0b0050 con 0x7f660c0cc9e0 2024-04-25T08:43:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6606000640 1 -- 172.21.15.46:0/2338569129 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc0b1040 con 0x7f660c0cc9e0 2024-04-25T08:43:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660c154ee0 con 0x7f660c0cc9e0 2024-04-25T08:43:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6606000640 1 -- 172.21.15.46:0/2338569129 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc004220 con 0x7f660c0cc9e0 2024-04-25T08:43:30.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660c155180 con 0x7f660c0cc9e0 2024-04-25T08:43:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.419+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c069080 con 0x7f660c0cc9e0 2024-04-25T08:43:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.423+0000 7f6606000640 1 -- 172.21.15.46:0/2338569129 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65fc0be060 con 0x7f660c0cc9e0 2024-04-25T08:43:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.423+0000 7f6606000640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65f0066f60 0x7f65f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.423+0000 7f6606000640 1 -- 172.21.15.46:0/2338569129 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f65fc11e820 con 0x7f660c0cc9e0 2024-04-25T08:43:30.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.423+0000 7f6611e00640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65f0066f60 0x7f65f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.423+0000 7f6611e00640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65f0066f60 0x7f65f0069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f660c0ae380 tx=0x7f6600008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:30.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.427+0000 7f6606000640 1 -- 172.21.15.46:0/2338569129 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65fc016070 con 0x7f660c0cc9e0 2024-04-25T08:43:30.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.647+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f660c071d90 con 0x7f660c0cc9e0 2024-04-25T08:43:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6606000640 1 -- 172.21.15.46:0/2338569129 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f65fc0ef9e0 con 0x7f660c0cc9e0 2024-04-25T08:43:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65f0066f60 msgr2=0x7f65f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65f0066f60 0x7f65f0069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f660c0ae380 tx=0x7f6600008040 comp rx=0 tx=0).stop 2024-04-25T08:43:30.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0cc9e0 msgr2=0x7f660c143020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0cc9e0 0x7f660c143020 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65fc01a0b0 tx=0x7f65fc01acc0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 shutdown_connections 2024-04-25T08:43:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65f0066f60 0x7f65f0069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f660c15a750 0x7f660c0ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c0cc9e0 0x7f660c143020 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 --2- 172.21.15.46:0/2338569129 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f660c0aed10 0x7f660c154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 >> 172.21.15.46:0/2338569129 conn(0x7f660c0b12a0 msgr2=0x7f660c1505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 shutdown_connections 2024-04-25T08:43:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.651+0000 7f6614800640 1 -- 172.21.15.46:0/2338569129 wait complete. 2024-04-25T08:43:30.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T08:43:30.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.811+0000 7f45f9c00640 1 Processor -- start 2024-04-25T08:43:30.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.811+0000 7f45f9c00640 1 -- start start 2024-04-25T08:43:30.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f9c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf140 0x7f45f415d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f9c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f406ed70 con 0x7f45f40afd40 2024-04-25T08:43:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f9c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f40cde00 con 0x7f45f40cf520 2024-04-25T08:43:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f9c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40afb40 con 0x7f45f40cf140 2024-04-25T08:43:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45ea000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f45f40afd40 0x7f45f40ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44046/0 (socket says 172.21.15.46:44046) 2024-04-25T08:43:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45ea000640 1 -- 172.21.15.46:0/3429158412 learned_addr learned my addr 172.21.15.46:0/3429158412 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:30.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f2a00640 1 --2- 172.21.15.46:0/3429158412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf140 0x7f45f415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f2000640 1 -- 172.21.15.46:0/3429158412 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34222842 0 0) 0x7f45dc001120 con 0x7f45f40afd40 2024-04-25T08:43:30.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f2000640 1 -- 172.21.15.46:0/3429158412 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45d8003530 con 0x7f45f40afd40 2024-04-25T08:43:30.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f2000640 1 -- 172.21.15.46:0/3429158412 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000234970 0 0) 0x7f45f40cde00 con 0x7f45f40cf520 2024-04-25T08:43:30.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f2a00640 1 -- 172.21.15.46:0/3429158412 >> v1:172.21.15.82:6789/0 conn(0x7f45f40cf520 legacy=0x7f45f40cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f2a00640 1 -- 172.21.15.46:0/3429158412 >> v1:172.21.15.46:6789/0 conn(0x7f45f40afd40 legacy=0x7f45f40ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.815+0000 7f45f2a00640 1 -- 172.21.15.46:0/3429158412 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40d13f0 con 0x7f45f40cf140 2024-04-25T08:43:30.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f2a00640 1 --2- 172.21.15.46:0/3429158412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf140 0x7f45f415d590 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45e40097f0 tx=0x7f45e401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed53b28c6edb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f2000640 1 -- 172.21.15.46:0/3429158412 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e40b1070 con 0x7f45f40cf140 2024-04-25T08:43:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f2000640 1 -- 172.21.15.46:0/3429158412 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e40ac620 con 0x7f45f40cf140 2024-04-25T08:43:30.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f2000640 1 -- 172.21.15.46:0/3429158412 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e40ac9d0 con 0x7f45f40cf140 2024-04-25T08:43:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f9c00640 1 -- 172.21.15.46:0/3429158412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf140 msgr2=0x7f45f415d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f9c00640 1 --2- 172.21.15.46:0/3429158412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf140 0x7f45f415d590 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f45e40097f0 tx=0x7f45e401c3c0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f9c00640 1 -- 172.21.15.46:0/3429158412 shutdown_connections 2024-04-25T08:43:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f9c00640 1 --2- 172.21.15.46:0/3429158412 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf140 0x7f45f415d590 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f9c00640 1 -- 172.21.15.46:0/3429158412 >> 172.21.15.46:0/3429158412 conn(0x7f45f40728c0 msgr2=0x7f45f4099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f9c00640 1 -- 172.21.15.46:0/3429158412 shutdown_connections 2024-04-25T08:43:30.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f9c00640 1 -- 172.21.15.46:0/3429158412 wait complete. 2024-04-25T08:43:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f9c00640 1 Processor -- start 2024-04-25T08:43:30.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.819+0000 7f45f9c00640 1 -- start start 2024-04-25T08:43:30.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40afd40 0x7f45f415f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f9c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f40cf140 0x7f45f4145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f9c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf520 0x7f45f40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4162010 con 0x7f45f40afd40 2024-04-25T08:43:30.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f9c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4161d30 con 0x7f45f40cf140 2024-04-25T08:43:30.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f9c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f415dee0 con 0x7f45f40cf520 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40afd40 0x7f45f415f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40afd40 0x7f45f415f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34046/0 (socket says 172.21.15.46:34046) 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f2a00640 1 -- 172.21.15.46:0/3676977171 learned_addr learned my addr 172.21.15.46:0/3676977171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45ea000640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f40cf140 0x7f45f4145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f3400640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf520 0x7f45f40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f2a00640 1 -- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf520 msgr2=0x7f45f40b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f2a00640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf520 0x7f45f40b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f2a00640 1 -- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f40cf140 msgr2=0x7f45f4145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f2a00640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f40cf140 0x7f45f4145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:30.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f2a00640 1 -- 172.21.15.46:0/3676977171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4160f10 con 0x7f45f40afd40 2024-04-25T08:43:30.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f3400640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf520 0x7f45f40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:30.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f2a00640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40afd40 0x7f45f415f570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f45e40af040 tx=0x7f45e401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:30.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45eb400640 1 -- 172.21.15.46:0/3676977171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e40b1050 con 0x7f45f40afd40 2024-04-25T08:43:30.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45eb400640 1 -- 172.21.15.46:0/3676977171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e40b2040 con 0x7f45f40afd40 2024-04-25T08:43:30.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f415fe70 con 0x7f45f40afd40 2024-04-25T08:43:30.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45eb400640 1 -- 172.21.15.46:0/3676977171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e40b8450 con 0x7f45f40afd40 2024-04-25T08:43:30.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f4160430 con 0x7f45f40afd40 2024-04-25T08:43:30.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f406ccb0 con 0x7f45f40afd40 2024-04-25T08:43:30.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45eb400640 1 -- 172.21.15.46:0/3676977171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45e40bf030 con 0x7f45f40afd40 2024-04-25T08:43:30.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.823+0000 7f45eb400640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45d0066f60 0x7f45d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:30.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.827+0000 7f45eb400640 1 -- 172.21.15.46:0/3676977171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f45e411e950 con 0x7f45f40afd40 2024-04-25T08:43:30.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.827+0000 7f45ea000640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45d0066f60 0x7f45d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.827+0000 7f45ea000640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45d0066f60 0x7f45d0069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f45f40b58b0 tx=0x7f45dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:30.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:30.827+0000 7f45eb400640 1 -- 172.21.15.46:0/3676977171 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45e40c6060 con 0x7f45f40afd40 2024-04-25T08:43:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.047+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f45f40af880 con 0x7f45f40afd40 2024-04-25T08:43:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.047+0000 7f45eb400640 1 -- 172.21.15.46:0/3676977171 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f45e40160d0 con 0x7f45f40afd40 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45d0066f60 msgr2=0x7f45d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45d0066f60 0x7f45d0069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f45f40b58b0 tx=0x7f45dc009040 comp rx=0 tx=0).stop 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40afd40 msgr2=0x7f45f415f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40afd40 0x7f45f415f570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f45e40af040 tx=0x7f45e401ac00 comp rx=0 tx=0).stop 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 shutdown_connections 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f45d0066f60 0x7f45d0069420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f45f40cf520 0x7f45f40b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45f40cf140 0x7f45f4145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 --2- 172.21.15.46:0/3676977171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45f40afd40 0x7f45f415f570 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 >> 172.21.15.46:0/3676977171 conn(0x7f45f40728c0 msgr2=0x7f45f4152240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 shutdown_connections 2024-04-25T08:43:31.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.051+0000 7f45f9c00640 1 -- 172.21.15.46:0/3676977171 wait complete. 2024-04-25T08:43:31.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T08:43:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeefe00640 1 Processor -- start 2024-04-25T08:43:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeefe00640 1 -- start start 2024-04-25T08:43:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeefe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feee815bdb0 0x7feee80cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeefe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feee80cda80 0x7feee815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeefe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee80cc700 con 0x7feee8159230 2024-04-25T08:43:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeefe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee80cded0 con 0x7feee815bdb0 2024-04-25T08:43:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeefe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee80cc9e0 con 0x7feee80cda80 2024-04-25T08:43:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeeca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feee815bdb0 0x7feee80cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeed400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7feee8159230 0x7feee80cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44050/0 (socket says 172.21.15.46:44050) 2024-04-25T08:43:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeed400640 1 -- 172.21.15.46:0/3566857359 learned_addr learned my addr 172.21.15.46:0/3566857359 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.211+0000 7feeede00640 1 --2- 172.21.15.46:0/3566857359 >> v2:172.21.15.82:3300/0 conn(0x7feee80cda80 0x7feee815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeeca00640 1 -- 172.21.15.46:0/3566857359 >> v2:172.21.15.82:3300/0 conn(0x7feee80cda80 msgr2=0x7feee815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeeca00640 1 --2- 172.21.15.46:0/3566857359 >> v2:172.21.15.82:3300/0 conn(0x7feee80cda80 0x7feee815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeeca00640 1 -- 172.21.15.46:0/3566857359 >> v1:172.21.15.46:6789/0 conn(0x7feee8159230 legacy=0x7feee80cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeeca00640 1 -- 172.21.15.46:0/3566857359 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80cccc0 con 0x7feee815bdb0 2024-04-25T08:43:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeede00640 1 --2- 172.21.15.46:0/3566857359 >> v2:172.21.15.82:3300/0 conn(0x7feee80cda80 0x7feee815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feee7e00640 1 -- 172.21.15.46:0/3566857359 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431596407 0 0) 0x7feee80cc700 con 0x7feee8159230 2024-04-25T08:43:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeeca00640 1 --2- 172.21.15.46:0/3566857359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee815bdb0 0x7feee80cd540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feee80b2340 tx=0x7feed401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217d462ec920ba97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feee7e00640 1 -- 172.21.15.46:0/3566857359 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40b0070 con 0x7feee815bdb0 2024-04-25T08:43:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feee7e00640 1 -- 172.21.15.46:0/3566857359 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed40ab600 con 0x7feee815bdb0 2024-04-25T08:43:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feee7e00640 1 -- 172.21.15.46:0/3566857359 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40ab9d0 con 0x7feee815bdb0 2024-04-25T08:43:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeefe00640 1 -- 172.21.15.46:0/3566857359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee815bdb0 msgr2=0x7feee80cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeefe00640 1 --2- 172.21.15.46:0/3566857359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee815bdb0 0x7feee80cd540 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feee80b2340 tx=0x7feed401c3f0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeefe00640 1 -- 172.21.15.46:0/3566857359 shutdown_connections 2024-04-25T08:43:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeefe00640 1 --2- 172.21.15.46:0/3566857359 >> v2:172.21.15.82:3300/0 conn(0x7feee80cda80 0x7feee815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeefe00640 1 --2- 172.21.15.46:0/3566857359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee815bdb0 0x7feee80cd540 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeefe00640 1 -- 172.21.15.46:0/3566857359 >> 172.21.15.46:0/3566857359 conn(0x7feee80b12a0 msgr2=0x7feee80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeefe00640 1 -- 172.21.15.46:0/3566857359 shutdown_connections 2024-04-25T08:43:31.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.215+0000 7feeefe00640 1 -- 172.21.15.46:0/3566857359 wait complete. 2024-04-25T08:43:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeefe00640 1 Processor -- start 2024-04-25T08:43:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeefe00640 1 -- start start 2024-04-25T08:43:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeefe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80cda80 0x7feee813f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeefe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee8159230 0x7feee806e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeefe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feee815bdb0 0x7feee8143980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeefe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee815e0d0 con 0x7feee8159230 2024-04-25T08:43:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeefe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee815df50 con 0x7feee80cda80 2024-04-25T08:43:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeefe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee815e250 con 0x7feee815bdb0 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeeca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee8159230 0x7feee806e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeeca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee8159230 0x7feee806e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34066/0 (socket says 172.21.15.46:34066) 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeeca00640 1 -- 172.21.15.46:0/3611386844 learned_addr learned my addr 172.21.15.46:0/3611386844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeed400640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80cda80 0x7feee813f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeede00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feee815bdb0 0x7feee8143980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeeca00640 1 -- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feee815bdb0 msgr2=0x7feee8143980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeeca00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feee815bdb0 0x7feee8143980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeeca00640 1 -- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80cda80 msgr2=0x7feee813f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeeca00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80cda80 0x7feee813f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeeca00640 1 -- 172.21.15.46:0/3611386844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee815a230 con 0x7feee8159230 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeede00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feee815bdb0 0x7feee8143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeeca00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee8159230 0x7feee806e3d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7feed4019000 tx=0x7feed4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feee5600640 1 -- 172.21.15.46:0/3611386844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40b0040 con 0x7feee8159230 2024-04-25T08:43:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feee5600640 1 -- 172.21.15.46:0/3611386844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed40bc070 con 0x7feee8159230 2024-04-25T08:43:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feee5600640 1 -- 172.21.15.46:0/3611386844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed40b5660 con 0x7feee8159230 2024-04-25T08:43:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.219+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee815a510 con 0x7feee8159230 2024-04-25T08:43:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.223+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee815aad0 con 0x7feee8159230 2024-04-25T08:43:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.223+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee8069080 con 0x7feee8159230 2024-04-25T08:43:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.223+0000 7feee5600640 1 -- 172.21.15.46:0/3611386844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feed40b3070 con 0x7feee8159230 2024-04-25T08:43:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.223+0000 7feee5600640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feec8066f60 0x7feec8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.223+0000 7feee5600640 1 -- 172.21.15.46:0/3611386844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7feed411f120 con 0x7feee8159230 2024-04-25T08:43:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.223+0000 7feeed400640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feec8066f60 0x7feec8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.223+0000 7feeed400640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feec8066f60 0x7feec8069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feedc000f70 tx=0x7feedc007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:31.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.227+0000 7feee5600640 1 -- 172.21.15.46:0/3611386844 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feed4016070 con 0x7feee8159230 2024-04-25T08:43:31.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.447+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7feee806eb00 con 0x7feee8159230 2024-04-25T08:43:31.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.447+0000 7feee5600640 1 -- 172.21.15.46:0/3611386844 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7feed40f0370 con 0x7feee8159230 2024-04-25T08:43:31.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feec8066f60 msgr2=0x7feec8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feec8066f60 0x7feec8069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feedc000f70 tx=0x7feedc007b60 comp rx=0 tx=0).stop 2024-04-25T08:43:31.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee8159230 msgr2=0x7feee806e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee8159230 0x7feee806e3d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7feed4019000 tx=0x7feed4019190 comp rx=0 tx=0).stop 2024-04-25T08:43:31.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 shutdown_connections 2024-04-25T08:43:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7feec8066f60 0x7feec8069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feee815bdb0 0x7feee8143980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feee8159230 0x7feee806e3d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 --2- 172.21.15.46:0/3611386844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feee80cda80 0x7feee813f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 >> 172.21.15.46:0/3611386844 conn(0x7feee80b12a0 msgr2=0x7feee80b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 shutdown_connections 2024-04-25T08:43:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.451+0000 7feeefe00640 1 -- 172.21.15.46:0/3611386844 wait complete. 2024-04-25T08:43:31.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:31.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.611+0000 7f832b800640 1 Processor -- start 2024-04-25T08:43:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.611+0000 7f832b800640 1 -- start start 2024-04-25T08:43:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.611+0000 7f832b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832409a110 0x7f832409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.611+0000 7f832b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f832415f1e0 0x7f8324070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.611+0000 7f832b800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83240cfd00 con 0x7f832415ee00 2024-04-25T08:43:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.611+0000 7f832b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8324071560 con 0x7f832415f1e0 2024-04-25T08:43:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.611+0000 7f832b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240ccb00 con 0x7f832409a110 2024-04-25T08:43:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.611+0000 7f8328e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f832415ee00 0x7f8324099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44064/0 (socket says 172.21.15.46:44064) 2024-04-25T08:43:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.611+0000 7f8328e00640 1 -- 172.21.15.46:0/95756541 learned_addr learned my addr 172.21.15.46:0/95756541 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:31.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f831be00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832409a110 0x7f832409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8329800640 1 --2- 172.21.15.46:0/95756541 >> v2:172.21.15.82:3300/0 conn(0x7f832415f1e0 0x7f8324070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621474756 0 0) 0x7f83240cfd00 con 0x7f832415ee00 2024-04-25T08:43:31.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8308003530 con 0x7f832415ee00 2024-04-25T08:43:31.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2280029443 0 0) 0x7f8308003530 con 0x7f832415ee00 2024-04-25T08:43:31.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832409a110 msgr2=0x7f832409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:31.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832409a110 0x7f832409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 >> v2:172.21.15.82:3300/0 conn(0x7f832415f1e0 msgr2=0x7f8324070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:31.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 --2- 172.21.15.46:0/95756541 >> v2:172.21.15.82:3300/0 conn(0x7f832415f1e0 0x7f8324070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240cffa0 con 0x7f832415ee00 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f8314001680 con 0x7f832415ee00 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 >> v1:172.21.15.46:6789/0 conn(0x7f832415ee00 legacy=0x7f8324099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832409a110 0x7f83080039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308003f40 0x7f8308004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8308004d50 0x7f83080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240ccb00 con 0x7f8308003f40 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830800b9d0 con 0x7f8308004d50 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830800bb50 con 0x7f832409a110 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f831be00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308003f40 0x7f8308004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8328e00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832409a110 0x7f83080039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f8329800640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8308004d50 0x7f83080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f831be00640 1 -- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832409a110 msgr2=0x7f83080039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:31.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f831be00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832409a110 0x7f83080039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f831be00640 1 -- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8308004d50 msgr2=0x7f83080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:31.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f831be00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8308004d50 0x7f83080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.615+0000 7f831be00640 1 -- 172.21.15.46:0/95756541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308003530 con 0x7f8308003f40 2024-04-25T08:43:31.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f831be00640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308003f40 0x7f8308004810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f830c009ae0 tx=0x7f830c01c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cb1b1991959181 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:31.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c0b1070 con 0x7f8308003f40 2024-04-25T08:43:31.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c0a4a90 con 0x7f8308003f40 2024-04-25T08:43:31.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f8323e00640 1 -- 172.21.15.46:0/95756541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c0096b0 con 0x7f8308003f40 2024-04-25T08:43:31.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f832b800640 1 -- 172.21.15.46:0/95756541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308003f40 msgr2=0x7f8308004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f832b800640 1 --2- 172.21.15.46:0/95756541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8308003f40 0x7f8308004810 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f830c009ae0 tx=0x7f830c01c6c0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f8329800640 1 -- 172.21.15.46:0/95756541 reap_dead start 2024-04-25T08:43:31.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f832b800640 1 -- 172.21.15.46:0/95756541 shutdown_connections 2024-04-25T08:43:31.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f832b800640 1 -- 172.21.15.46:0/95756541 >> 172.21.15.46:0/95756541 conn(0x7f83240b3170 msgr2=0x7f83240b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:31.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f832b800640 1 -- 172.21.15.46:0/95756541 shutdown_connections 2024-04-25T08:43:31.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f832b800640 1 -- 172.21.15.46:0/95756541 wait complete. 2024-04-25T08:43:31.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f832b800640 1 Processor -- start 2024-04-25T08:43:31.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.619+0000 7f832b800640 1 -- start start 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f832b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8308003f40 0x7f83240c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f832b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240b71a0 0x7f83241f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f832b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240ca9c0 0x7f83240cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f832b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240cd520 con 0x7f83240b71a0 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f832b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83240cd3a0 con 0x7f83240ca9c0 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f832b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83240cd6a0 con 0x7f8308003f40 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f831be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240b71a0 0x7f83241f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f831be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240b71a0 0x7f83241f4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34080/0 (socket says 172.21.15.46:34080) 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f831be00640 1 -- 172.21.15.46:0/1032842959 learned_addr learned my addr 172.21.15.46:0/1032842959 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f8329800640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240ca9c0 0x7f83240cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f8328e00640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8308003f40 0x7f83240c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f831be00640 1 -- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8308003f40 msgr2=0x7f83240c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f831be00640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8308003f40 0x7f83240c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f831be00640 1 -- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240ca9c0 msgr2=0x7f83240cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f831be00640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240ca9c0 0x7f83240cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f831be00640 1 -- 172.21.15.46:0/1032842959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240d0800 con 0x7f83240b71a0 2024-04-25T08:43:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f831be00640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240b71a0 0x7f83241f4cf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f830c01c4f0 tx=0x7f8308005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f8321600640 1 -- 172.21.15.46:0/1032842959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c0bc020 con 0x7f83240b71a0 2024-04-25T08:43:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f8321600640 1 -- 172.21.15.46:0/1032842959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830c0aca60 con 0x7f83240b71a0 2024-04-25T08:43:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f8321600640 1 -- 172.21.15.46:0/1032842959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830c0b1040 con 0x7f83240b71a0 2024-04-25T08:43:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83241f5230 con 0x7f83240b71a0 2024-04-25T08:43:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83240c9010 con 0x7f83240b71a0 2024-04-25T08:43:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83240d09d0 con 0x7f83240b71a0 2024-04-25T08:43:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.623+0000 7f8321600640 1 -- 172.21.15.46:0/1032842959 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f830c004020 con 0x7f83240b71a0 2024-04-25T08:43:31.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.627+0000 7f8321600640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f90 0x7f8300069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.627+0000 7f8321600640 1 -- 172.21.15.46:0/1032842959 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f830c11df90 con 0x7f83240b71a0 2024-04-25T08:43:31.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.627+0000 7f8328e00640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f90 0x7f8300069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.627+0000 7f8328e00640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f90 0x7f8300069450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8314001060 tx=0x7f8314007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:31.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.631+0000 7f8321600640 1 -- 172.21.15.46:0/1032842959 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83241f5230 con 0x7f83240b71a0 2024-04-25T08:43:31.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.791+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8324069080 con 0x7f83240b71a0 2024-04-25T08:43:31.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f8321600640 1 -- 172.21.15.46:0/1032842959 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+13116 (secure 0 0 0) 0x7f830c016780 con 0x7f83240b71a0 2024-04-25T08:43:31.786 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:31.787 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:28.643116+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f90 msgr2=0x7f8300069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f90 0x7f8300069450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8314001060 tx=0x7f8314007900 comp rx=0 tx=0).stop 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240b71a0 msgr2=0x7f83241f4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240b71a0 0x7f83241f4cf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f830c01c4f0 tx=0x7f8308005840 comp rx=0 tx=0).stop 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 shutdown_connections 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f90 0x7f8300069450 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240ca9c0 0x7f83240cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240b71a0 0x7f83241f4cf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 --2- 172.21.15.46:0/1032842959 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8308003f40 0x7f83240c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 >> 172.21.15.46:0/1032842959 conn(0x7f83240b3170 msgr2=0x7f8324099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 shutdown_connections 2024-04-25T08:43:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.795+0000 7f832b800640 1 -- 172.21.15.46:0/1032842959 wait complete. 2024-04-25T08:43:31.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:43:31.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9d5000640 1 Processor -- start 2024-04-25T08:43:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9d5000640 1 -- start start 2024-04-25T08:43:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9d5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d006f150 0x7fe9d00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9d5000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d015e440 con 0x7fe9d0143990 2024-04-25T08:43:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9d5000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d015e6e0 con 0x7fe9d01440d0 2024-04-25T08:43:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9d5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d015ed30 con 0x7fe9d006f150 2024-04-25T08:43:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9cde00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe9d0143990 0x7fe9d006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44076/0 (socket says 172.21.15.46:44076) 2024-04-25T08:43:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9cde00640 1 -- 172.21.15.46:0/1672637530 learned_addr learned my addr 172.21.15.46:0/1672637530 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9ce800640 1 --2- 172.21.15.46:0/1672637530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d006f150 0x7fe9d00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9cca00640 1 -- 172.21.15.46:0/1672637530 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851879105 0 0) 0x7fe9d015e440 con 0x7fe9d0143990 2024-04-25T08:43:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9cca00640 1 -- 172.21.15.46:0/1672637530 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b4003530 con 0x7fe9d0143990 2024-04-25T08:43:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9cca00640 1 -- 172.21.15.46:0/1672637530 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2830683975 0 0) 0x7fe9b4003530 con 0x7fe9d0143990 2024-04-25T08:43:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9ce800640 1 -- 172.21.15.46:0/1672637530 >> v1:172.21.15.82:6789/0 conn(0x7fe9d01440d0 legacy=0x7fe9d00b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9ce800640 1 -- 172.21.15.46:0/1672637530 >> v1:172.21.15.46:6789/0 conn(0x7fe9d0143990 legacy=0x7fe9d006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.959+0000 7fe9ce800640 1 -- 172.21.15.46:0/1672637530 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d015e9c0 con 0x7fe9d006f150 2024-04-25T08:43:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9ce800640 1 --2- 172.21.15.46:0/1672637530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d006f150 0x7fe9d00cecc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9d015eeb0 tx=0x7fe9c801c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c974025218bc00d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9cca00640 1 -- 172.21.15.46:0/1672637530 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80ac070 con 0x7fe9d006f150 2024-04-25T08:43:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9cca00640 1 -- 172.21.15.46:0/1672637530 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c80a6430 con 0x7fe9d006f150 2024-04-25T08:43:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9cca00640 1 -- 172.21.15.46:0/1672637530 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0001680 con 0x7fe9d006f150 2024-04-25T08:43:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9d5000640 1 -- 172.21.15.46:0/1672637530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d006f150 msgr2=0x7fe9d00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9d5000640 1 --2- 172.21.15.46:0/1672637530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d006f150 0x7fe9d00cecc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9d015eeb0 tx=0x7fe9c801c760 comp rx=0 tx=0).stop 2024-04-25T08:43:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9d5000640 1 -- 172.21.15.46:0/1672637530 shutdown_connections 2024-04-25T08:43:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9d5000640 1 --2- 172.21.15.46:0/1672637530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d006f150 0x7fe9d00cecc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9d5000640 1 -- 172.21.15.46:0/1672637530 >> 172.21.15.46:0/1672637530 conn(0x7fe9d006ac10 msgr2=0x7fe9d006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9d5000640 1 -- 172.21.15.46:0/1672637530 shutdown_connections 2024-04-25T08:43:31.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.963+0000 7fe9d5000640 1 -- 172.21.15.46:0/1672637530 wait complete. 2024-04-25T08:43:31.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 Processor -- start 2024-04-25T08:43:31.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 -- start start 2024-04-25T08:43:31.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d006f150 0x7fe9d015a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d0143990 0x7fe9d013da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d01440d0 0x7fe9d013a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d01686a0 con 0x7fe9d006f150 2024-04-25T08:43:31.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0168520 con 0x7fe9d0143990 2024-04-25T08:43:31.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0168820 con 0x7fe9d01440d0 2024-04-25T08:43:31.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d006f150 0x7fe9d015a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d006f150 0x7fe9d015a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34084/0 (socket says 172.21.15.46:34084) 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cde00640 1 -- 172.21.15.46:0/2705708952 learned_addr learned my addr 172.21.15.46:0/2705708952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9ce800640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d01440d0 0x7fe9d013a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cd400640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d0143990 0x7fe9d013da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cde00640 1 -- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d01440d0 msgr2=0x7fe9d013a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cde00640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d01440d0 0x7fe9d013a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cde00640 1 -- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d0143990 msgr2=0x7fe9d013da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cde00640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d0143990 0x7fe9d013da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cde00640 1 -- 172.21.15.46:0/2705708952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d00acb70 con 0x7fe9d006f150 2024-04-25T08:43:31.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9cde00640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d006f150 0x7fe9d015a000 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c500 tx=0x7fe9c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:31.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9c6000640 1 -- 172.21.15.46:0/2705708952 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0018070 con 0x7fe9d006f150 2024-04-25T08:43:31.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9c6000640 1 -- 172.21.15.46:0/2705708952 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c0007560 con 0x7fe9d006f150 2024-04-25T08:43:31.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9c6000640 1 -- 172.21.15.46:0/2705708952 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0004a30 con 0x7fe9d006f150 2024-04-25T08:43:31.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d00ace50 con 0x7fe9d006f150 2024-04-25T08:43:31.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.967+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d00ad410 con 0x7fe9d006f150 2024-04-25T08:43:31.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.971+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d0069010 con 0x7fe9d006f150 2024-04-25T08:43:31.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.971+0000 7fe9c6000640 1 -- 172.21.15.46:0/2705708952 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9c0004bd0 con 0x7fe9d006f150 2024-04-25T08:43:31.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.971+0000 7fe9c6000640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:31.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.971+0000 7fe9c6000640 1 -- 172.21.15.46:0/2705708952 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe9c00841a0 con 0x7fe9d006f150 2024-04-25T08:43:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.971+0000 7fe9cd400640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.971+0000 7fe9cd400640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe9d013c0f0 tx=0x7fe9bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:31.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:31.975+0000 7fe9c6000640 1 -- 172.21.15.46:0/2705708952 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9c0054d10 con 0x7fe9d006f150 2024-04-25T08:43:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9d0138cc0 con 0x7fe9d006f150 2024-04-25T08:43:32.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9c6000640 1 -- 172.21.15.46:0/2705708952 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9c0058db0 con 0x7fe9d006f150 2024-04-25T08:43:32.131 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b0066f60 msgr2=0x7fe9b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe9d013c0f0 tx=0x7fe9bc009040 comp rx=0 tx=0).stop 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d006f150 msgr2=0x7fe9d015a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d006f150 0x7fe9d015a000 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c500 tx=0x7fe9c000c9c0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 shutdown_connections 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9d01440d0 0x7fe9d013a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9d0143990 0x7fe9d013da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 --2- 172.21.15.46:0/2705708952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d006f150 0x7fe9d015a000 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 >> 172.21.15.46:0/2705708952 conn(0x7fe9d006ac10 msgr2=0x7fe9d01394c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 shutdown_connections 2024-04-25T08:43:32.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.139+0000 7fe9d5000640 1 -- 172.21.15.46:0/2705708952 wait complete. 2024-04-25T08:43:32.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:43:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9608200640 1 Processor -- start 2024-04-25T08:43:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9608200640 1 -- start start 2024-04-25T08:43:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9608200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96000cf9d0 0x7f96000b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9608200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960015dd30 0x7f960015e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9608200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960006b460 con 0x7f96000cfdb0 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9608200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9600161d90 con 0x7f96000cf9d0 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9608200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9600161740 con 0x7f960015dd30 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9605800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96000cf9d0 0x7f96000b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9605800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96000cf9d0 0x7f96000b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34086/0 (socket says 172.21.15.46:34086) 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9605800640 1 -- 172.21.15.46:0/4244025735 learned_addr learned my addr 172.21.15.46:0/4244025735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9606200640 1 --2- 172.21.15.46:0/4244025735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960015dd30 0x7f960015e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9605800640 1 -- 172.21.15.46:0/4244025735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960015dd30 msgr2=0x7f960015e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9605800640 1 --2- 172.21.15.46:0/4244025735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960015dd30 0x7f960015e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9605800640 1 -- 172.21.15.46:0/4244025735 >> v1:172.21.15.82:6789/0 conn(0x7f96000cfdb0 legacy=0x7f960006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.303+0000 7f9605800640 1 -- 172.21.15.46:0/4244025735 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9600161a20 con 0x7f96000cf9d0 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f9605800640 1 --2- 172.21.15.46:0/4244025735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 0x7f96000b2bd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009870 tx=0x7f95f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b8ae9c6abce398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f95ffe00640 1 -- 172.21.15.46:0/4244025735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f40ab5f0 con 0x7f96000cf9d0 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f95ffe00640 1 -- 172.21.15.46:0/4244025735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f40b0070 con 0x7f96000cf9d0 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f95ffe00640 1 -- 172.21.15.46:0/4244025735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f40ab9b0 con 0x7f96000cf9d0 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f9608200640 1 -- 172.21.15.46:0/4244025735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 msgr2=0x7f96000b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f9608200640 1 --2- 172.21.15.46:0/4244025735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 0x7f96000b2bd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009870 tx=0x7f95f401c520 comp rx=0 tx=0).stop 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f9608200640 1 -- 172.21.15.46:0/4244025735 shutdown_connections 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f9608200640 1 --2- 172.21.15.46:0/4244025735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960015dd30 0x7f960015e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f9608200640 1 --2- 172.21.15.46:0/4244025735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 0x7f96000b2bd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f9608200640 1 -- 172.21.15.46:0/4244025735 >> 172.21.15.46:0/4244025735 conn(0x7f9600099a00 msgr2=0x7f96000b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f9608200640 1 -- 172.21.15.46:0/4244025735 shutdown_connections 2024-04-25T08:43:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.307+0000 7f9608200640 1 -- 172.21.15.46:0/4244025735 wait complete. 2024-04-25T08:43:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 Processor -- start 2024-04-25T08:43:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 -- start start 2024-04-25T08:43:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 0x7f96000b57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96000cfdb0 0x7f9600154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960015dd30 0x7f960015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960016aac0 con 0x7f96000cf9d0 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9600162430 con 0x7f960015dd30 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960016ac40 con 0x7f96000cfdb0 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9605800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 0x7f96000b57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9605800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 0x7f96000b57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34100/0 (socket says 172.21.15.46:34100) 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9605800640 1 -- 172.21.15.46:0/1260635526 learned_addr learned my addr 172.21.15.46:0/1260635526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9604e00640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96000cfdb0 0x7f9600154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9606200640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960015dd30 0x7f960015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9605800640 1 -- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96000cfdb0 msgr2=0x7f9600154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9605800640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96000cfdb0 0x7f9600154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9605800640 1 -- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960015dd30 msgr2=0x7f960015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9605800640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960015dd30 0x7f960015c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9605800640 1 -- 172.21.15.46:0/1260635526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96001f74e0 con 0x7f96000cf9d0 2024-04-25T08:43:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9604e00640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96000cfdb0 0x7f9600154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9605800640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 0x7f96000b57a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f95f401cdc0 tx=0x7f95f401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f95fd600640 1 -- 172.21.15.46:0/1260635526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f40be5f0 con 0x7f96000cf9d0 2024-04-25T08:43:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f95fd600640 1 -- 172.21.15.46:0/1260635526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f40be790 con 0x7f96000cf9d0 2024-04-25T08:43:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96001f77c0 con 0x7f96000cf9d0 2024-04-25T08:43:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f95fd600640 1 -- 172.21.15.46:0/1260635526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f40beb80 con 0x7f96000cf9d0 2024-04-25T08:43:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96001f7d80 con 0x7f96000cf9d0 2024-04-25T08:43:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.311+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9600069080 con 0x7f96000cf9d0 2024-04-25T08:43:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.315+0000 7f95fd600640 1 -- 172.21.15.46:0/1260635526 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95f40b43f0 con 0x7f96000cf9d0 2024-04-25T08:43:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.315+0000 7f95fd600640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95e0066f90 0x7f95e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.315+0000 7f95fd600640 1 -- 172.21.15.46:0/1260635526 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f95f411f840 con 0x7f96000cf9d0 2024-04-25T08:43:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.315+0000 7f9604e00640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95e0066f90 0x7f95e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.315+0000 7f9604e00640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95e0066f90 0x7f95e0069450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f960014d830 tx=0x7f95ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.319+0000 7f95fd600640 1 -- 172.21.15.46:0/1260635526 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95f40c4060 con 0x7f96000cf9d0 2024-04-25T08:43:32.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f960006b810 con 0x7f96000cf9d0 2024-04-25T08:43:32.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f95fd600640 1 -- 172.21.15.46:0/1260635526 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+65 (secure 0 0 0) 0x7f95f4016110 con 0x7f96000cf9d0 2024-04-25T08:43:32.475 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:32.475 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:43:32.475 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:43:32.475 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-25T08:43:32.476 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-25T08:43:32.476 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:43:32.476 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:43:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95e0066f90 msgr2=0x7f95e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95e0066f90 0x7f95e0069450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f960014d830 tx=0x7f95ec008040 comp rx=0 tx=0).stop 2024-04-25T08:43:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 msgr2=0x7f96000b57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 0x7f96000b57a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f95f401cdc0 tx=0x7f95f401ad60 comp rx=0 tx=0).stop 2024-04-25T08:43:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 shutdown_connections 2024-04-25T08:43:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95e0066f90 0x7f95e0069450 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960015dd30 0x7f960015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96000cfdb0 0x7f9600154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 --2- 172.21.15.46:0/1260635526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96000cf9d0 0x7f96000b57a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 >> 172.21.15.46:0/1260635526 conn(0x7f9600099a00 msgr2=0x7f9600156990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:32.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 shutdown_connections 2024-04-25T08:43:32.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.483+0000 7f9608200640 1 -- 172.21.15.46:0/1260635526 wait complete. 2024-04-25T08:43:32.495 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:43:32.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:43:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.647+0000 7f9149a00640 1 Processor -- start 2024-04-25T08:43:32.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.647+0000 7f9149a00640 1 -- start start 2024-04-25T08:43:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.647+0000 7f9149a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f914415db80 0x7f9144099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.647+0000 7f9149a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f91440b2230 0x7f9144099f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.647+0000 7f9149a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91440b2820 con 0x7f914409a740 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.647+0000 7f9149a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91440d05e0 con 0x7f91440b2230 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.647+0000 7f9149a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91440b2b00 con 0x7f914415db80 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.647+0000 7f9149000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f91440b2230 0x7f9144099f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9149000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f91440b2230 0x7f9144099f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34116/0 (socket says 172.21.15.46:34116) 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9149000640 1 -- 172.21.15.46:0/338712171 learned_addr learned my addr 172.21.15.46:0/338712171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9143e00640 1 --2- 172.21.15.46:0/338712171 >> v2:172.21.15.82:3300/0 conn(0x7f914415db80 0x7f9144099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9149000640 1 -- 172.21.15.46:0/338712171 >> v2:172.21.15.82:3300/0 conn(0x7f914415db80 msgr2=0x7f9144099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9149000640 1 --2- 172.21.15.46:0/338712171 >> v2:172.21.15.82:3300/0 conn(0x7f914415db80 0x7f9144099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9149000640 1 -- 172.21.15.46:0/338712171 >> v1:172.21.15.82:6789/0 conn(0x7f914409a740 legacy=0x7f914406bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9149000640 1 -- 172.21.15.46:0/338712171 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440d0270 con 0x7f91440b2230 2024-04-25T08:43:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9149000640 1 --2- 172.21.15.46:0/338712171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91440b2230 0x7f9144099f40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9144072350 tx=0x7f913c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806578d38a2b76e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9143400640 1 -- 172.21.15.46:0/338712171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c0b4070 con 0x7f91440b2230 2024-04-25T08:43:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9143400640 1 -- 172.21.15.46:0/338712171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c0af400 con 0x7f91440b2230 2024-04-25T08:43:32.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.651+0000 7f9143400640 1 -- 172.21.15.46:0/338712171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c0af7d0 con 0x7f91440b2230 2024-04-25T08:43:32.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 -- 172.21.15.46:0/338712171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91440b2230 msgr2=0x7f9144099f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 --2- 172.21.15.46:0/338712171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91440b2230 0x7f9144099f40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9144072350 tx=0x7f913c01f1e0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 -- 172.21.15.46:0/338712171 shutdown_connections 2024-04-25T08:43:32.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 --2- 172.21.15.46:0/338712171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91440b2230 0x7f9144099f40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 --2- 172.21.15.46:0/338712171 >> v2:172.21.15.82:3300/0 conn(0x7f914415db80 0x7f9144099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 -- 172.21.15.46:0/338712171 >> 172.21.15.46:0/338712171 conn(0x7f91440b3020 msgr2=0x7f91440b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:32.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 -- 172.21.15.46:0/338712171 shutdown_connections 2024-04-25T08:43:32.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 -- 172.21.15.46:0/338712171 wait complete. 2024-04-25T08:43:32.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 Processor -- start 2024-04-25T08:43:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 -- start start 2024-04-25T08:43:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f914409a740 0x7f9144158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f91440b2230 0x7f91440ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914415db80 0x7f91440af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440b1be0 con 0x7f914409a740 2024-04-25T08:43:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914415df60 con 0x7f914415db80 2024-04-25T08:43:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91440b1d60 con 0x7f91440b2230 2024-04-25T08:43:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9143e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f914409a740 0x7f9144158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9143e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f914409a740 0x7f9144158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34126/0 (socket says 172.21.15.46:34126) 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9143e00640 1 -- 172.21.15.46:0/1215910093 learned_addr learned my addr 172.21.15.46:0/1215910093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f913be00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f91440b2230 0x7f91440ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9149000640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914415db80 0x7f91440af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9143e00640 1 -- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f91440b2230 msgr2=0x7f91440ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9143e00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f91440b2230 0x7f91440ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9143e00640 1 -- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914415db80 msgr2=0x7f91440af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9143e00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914415db80 0x7f91440af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9143e00640 1 -- 172.21.15.46:0/1215910093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440c6cc0 con 0x7f914409a740 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.655+0000 7f9143e00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f914409a740 0x7f9144158ba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f913400e870 tx=0x7f913400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f9140c00640 1 -- 172.21.15.46:0/1215910093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134017070 con 0x7f914409a740 2024-04-25T08:43:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f9140c00640 1 -- 172.21.15.46:0/1215910093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913400c940 con 0x7f914409a740 2024-04-25T08:43:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f9140c00640 1 -- 172.21.15.46:0/1215910093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913400ccf0 con 0x7f914409a740 2024-04-25T08:43:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91440c6fa0 con 0x7f914409a740 2024-04-25T08:43:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91440c7560 con 0x7f914409a740 2024-04-25T08:43:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9144069010 con 0x7f914409a740 2024-04-25T08:43:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f9140c00640 1 -- 172.21.15.46:0/1215910093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f913401a070 con 0x7f914409a740 2024-04-25T08:43:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f9140c00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9120066f60 0x7f9120069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f9140c00640 1 -- 172.21.15.46:0/1215910093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9134085080 con 0x7f914409a740 2024-04-25T08:43:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f913be00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9120066f60 0x7f9120069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.659+0000 7f913be00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9120066f60 0x7f9120069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f91440c6ad0 tx=0x7f912c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.663+0000 7f9140c00640 1 -- 172.21.15.46:0/1215910093 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9134055b60 con 0x7f914409a740 2024-04-25T08:43:32.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:32.827+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9144070d20 con 0x7f914409a740 2024-04-25T08:43:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.651+0000 7f9140c00640 1 -- 172.21.15.46:0/1215910093 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9134059c00 con 0x7f914409a740 2024-04-25T08:43:33.644 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-25T08:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9120066f60 msgr2=0x7f9120069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9120066f60 0x7f9120069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f91440c6ad0 tx=0x7f912c009040 comp rx=0 tx=0).stop 2024-04-25T08:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f914409a740 msgr2=0x7f9144158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f914409a740 0x7f9144158ba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f913400e870 tx=0x7f913400ec70 comp rx=0 tx=0).stop 2024-04-25T08:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 shutdown_connections 2024-04-25T08:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9120066f60 0x7f9120069420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914415db80 0x7f91440af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f91440b2230 0x7f91440ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 --2- 172.21.15.46:0/1215910093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f914409a740 0x7f9144158ba0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 >> 172.21.15.46:0/1215910093 conn(0x7f91440b3020 msgr2=0x7f9144070600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 shutdown_connections 2024-04-25T08:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.655+0000 7f9149a00640 1 -- 172.21.15.46:0/1215910093 wait complete. 2024-04-25T08:43:33.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:43:33.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.831+0000 7f61f7e00640 1 Processor -- start 2024-04-25T08:43:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.831+0000 7f61f7e00640 1 -- start start 2024-04-25T08:43:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f806c730 0x7f61f806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:33.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f7e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f809acb0 con 0x7f61f809a410 2024-04-25T08:43:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f7e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f809a9d0 con 0x7f61f80d0390 2024-04-25T08:43:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80b3dc0 con 0x7f61f806c730 2024-04-25T08:43:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61efe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f61f809a410 0x7f61f806cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44082/0 (socket says 172.21.15.46:44082) 2024-04-25T08:43:33.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61efe00640 1 -- 172.21.15.46:0/770450979 learned_addr learned my addr 172.21.15.46:0/770450979 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f6a00640 1 --2- 172.21.15.46:0/770450979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f806c730 0x7f61f806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f6000640 1 -- 172.21.15.46:0/770450979 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016823733 0 0) 0x7f61f809acb0 con 0x7f61f809a410 2024-04-25T08:43:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f6000640 1 -- 172.21.15.46:0/770450979 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61dc003530 con 0x7f61f809a410 2024-04-25T08:43:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f6000640 1 -- 172.21.15.46:0/770450979 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205559573 0 0) 0x7f61f809a9d0 con 0x7f61f80d0390 2024-04-25T08:43:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f6000640 1 -- 172.21.15.46:0/770450979 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f809acb0 con 0x7f61f80d0390 2024-04-25T08:43:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f6a00640 1 -- 172.21.15.46:0/770450979 >> v1:172.21.15.82:6789/0 conn(0x7f61f80d0390 legacy=0x7f61f80cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f6a00640 1 -- 172.21.15.46:0/770450979 >> v1:172.21.15.46:6789/0 conn(0x7f61f809a410 legacy=0x7f61f806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:33.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.835+0000 7f61f6a00640 1 -- 172.21.15.46:0/770450979 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f80cf8b0 con 0x7f61f806c730 2024-04-25T08:43:33.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f6a00640 1 --2- 172.21.15.46:0/770450979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f806c730 0x7f61f806bf30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61e80097f0 tx=0x7f61e801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ce6390f49c0e348 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:33.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f6000640 1 -- 172.21.15.46:0/770450979 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80b0070 con 0x7f61f806c730 2024-04-25T08:43:33.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f6000640 1 -- 172.21.15.46:0/770450979 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e80ab600 con 0x7f61f806c730 2024-04-25T08:43:33.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f6000640 1 -- 172.21.15.46:0/770450979 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80ab9d0 con 0x7f61f806c730 2024-04-25T08:43:33.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f7e00640 1 -- 172.21.15.46:0/770450979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f806c730 msgr2=0x7f61f806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:33.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f7e00640 1 --2- 172.21.15.46:0/770450979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f806c730 0x7f61f806bf30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f61e80097f0 tx=0x7f61e801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:43:33.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f7e00640 1 -- 172.21.15.46:0/770450979 shutdown_connections 2024-04-25T08:43:33.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f7e00640 1 --2- 172.21.15.46:0/770450979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f806c730 0x7f61f806bf30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:33.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f7e00640 1 -- 172.21.15.46:0/770450979 >> 172.21.15.46:0/770450979 conn(0x7f61f80b2f50 msgr2=0x7f61f80b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:33.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f7e00640 1 -- 172.21.15.46:0/770450979 shutdown_connections 2024-04-25T08:43:33.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f7e00640 1 -- 172.21.15.46:0/770450979 wait complete. 2024-04-25T08:43:33.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f7e00640 1 Processor -- start 2024-04-25T08:43:33.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.839+0000 7f61f7e00640 1 -- start start 2024-04-25T08:43:33.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f806c730 0x7f61f8147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:33.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f809a410 0x7f61f81483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:33.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f80d0390 0x7f61f814d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:33.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80cc340 con 0x7f61f806c730 2024-04-25T08:43:33.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f80cc1c0 con 0x7f61f80d0390 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f816c460 con 0x7f61f809a410 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f806c730 0x7f61f8147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f806c730 0x7f61f8147e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34142/0 (socket says 172.21.15.46:34142) 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f6a00640 1 -- 172.21.15.46:0/3102570927 learned_addr learned my addr 172.21.15.46:0/3102570927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7400640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f80d0390 0x7f61f814d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61efe00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f809a410 0x7f61f81483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f6a00640 1 -- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f809a410 msgr2=0x7f61f81483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f6a00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f809a410 0x7f61f81483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f6a00640 1 -- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f80d0390 msgr2=0x7f61f814d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f6a00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f80d0390 0x7f61f814d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f6a00640 1 -- 172.21.15.46:0/3102570927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f8155f50 con 0x7f61f806c730 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7400640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f80d0390 0x7f61f814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:33.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61efe00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f809a410 0x7f61f81483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:33.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f6a00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f806c730 0x7f61f8147e80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f61e80190f0 tx=0x7f61e801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:33.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61eea00640 1 -- 172.21.15.46:0/3102570927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80b0050 con 0x7f61f806c730 2024-04-25T08:43:33.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61eea00640 1 -- 172.21.15.46:0/3102570927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e80bc070 con 0x7f61f806c730 2024-04-25T08:43:33.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61eea00640 1 -- 172.21.15.46:0/3102570927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8006e20 con 0x7f61f806c730 2024-04-25T08:43:33.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8155c70 con 0x7f61f806c730 2024-04-25T08:43:33.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f80cd370 con 0x7f61f806c730 2024-04-25T08:43:33.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f8069010 con 0x7f61f806c730 2024-04-25T08:43:33.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.843+0000 7f61eea00640 1 -- 172.21.15.46:0/3102570927 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61e80b5870 con 0x7f61f806c730 2024-04-25T08:43:33.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.847+0000 7f61eea00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f61d4066f60 0x7f61d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:33.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.847+0000 7f61eea00640 1 -- 172.21.15.46:0/3102570927 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f61e811ff30 con 0x7f61f806c730 2024-04-25T08:43:33.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.847+0000 7f61efe00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f61d4066f60 0x7f61d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:33.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.847+0000 7f61efe00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f61d4066f60 0x7f61d4069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f61f81490f0 tx=0x7f61e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:33.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:33.851+0000 7f61eea00640 1 -- 172.21.15.46:0/3102570927 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61e80160d0 con 0x7f61f806c730 2024-04-25T08:43:34.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.015+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f61f806c510 con 0x7f61f806c730 2024-04-25T08:43:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.659+0000 7f61eea00640 1 -- 172.21.15.46:0/3102570927 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f61e80f1100 con 0x7f61f806c730 2024-04-25T08:43:34.652 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-25T08:43:34.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f61d4066f60 msgr2=0x7f61d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:34.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f61d4066f60 0x7f61d4069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f61f81490f0 tx=0x7f61e0009040 comp rx=0 tx=0).stop 2024-04-25T08:43:34.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f806c730 msgr2=0x7f61f8147e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:34.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f806c730 0x7f61f8147e80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f61e80190f0 tx=0x7f61e801ad20 comp rx=0 tx=0).stop 2024-04-25T08:43:34.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 shutdown_connections 2024-04-25T08:43:34.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f61d4066f60 0x7f61d4069420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:34.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f80d0390 0x7f61f814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61f809a410 0x7f61f81483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 --2- 172.21.15.46:0/3102570927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f806c730 0x7f61f8147e80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 >> 172.21.15.46:0/3102570927 conn(0x7f61f80b2f50 msgr2=0x7f61f8152090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 shutdown_connections 2024-04-25T08:43:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.663+0000 7f61f7e00640 1 -- 172.21.15.46:0/3102570927 wait complete. 2024-04-25T08:43:34.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:43:34.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a24e00640 1 Processor -- start 2024-04-25T08:43:34.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a24e00640 1 -- start start 2024-04-25T08:43:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a24e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4a2015dd50 0x7f4a200cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a24e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2009a780 con 0x7f4a2006f4d0 2024-04-25T08:43:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a24e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2009aa60 con 0x7f4a200d0450 2024-04-25T08:43:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a24e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a200708c0 con 0x7f4a2015dd50 2024-04-25T08:43:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a1de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4a2006f4d0 0x7f4a200cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44088/0 (socket says 172.21.15.46:44088) 2024-04-25T08:43:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a1de00640 1 -- 172.21.15.46:0/333314203 learned_addr learned my addr 172.21.15.46:0/333314203 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a15400640 1 --2- 172.21.15.46:0/333314203 >> v2:172.21.15.82:3300/0 conn(0x7f4a2015dd50 0x7f4a200cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a1d400640 1 -- 172.21.15.46:0/333314203 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157985933 0 0) 0x7f4a2009a780 con 0x7f4a2006f4d0 2024-04-25T08:43:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a1d400640 1 -- 172.21.15.46:0/333314203 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a04003530 con 0x7f4a2006f4d0 2024-04-25T08:43:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a15400640 1 -- 172.21.15.46:0/333314203 >> v1:172.21.15.82:6789/0 conn(0x7f4a200d0450 legacy=0x7f4a200cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a15400640 1 -- 172.21.15.46:0/333314203 >> v1:172.21.15.46:6789/0 conn(0x7f4a2006f4d0 legacy=0x7f4a200cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.855+0000 7f4a15400640 1 -- 172.21.15.46:0/333314203 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2009ad40 con 0x7f4a2015dd50 2024-04-25T08:43:34.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a15400640 1 --2- 172.21.15.46:0/333314203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2015dd50 0x7f4a200cff10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a080097c0 tx=0x7f4a0801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d83bc10d175c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:34.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a1d400640 1 -- 172.21.15.46:0/333314203 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a080a7070 con 0x7f4a2015dd50 2024-04-25T08:43:34.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a1d400640 1 -- 172.21.15.46:0/333314203 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a080b0070 con 0x7f4a2015dd50 2024-04-25T08:43:34.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a1d400640 1 -- 172.21.15.46:0/333314203 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a080ab870 con 0x7f4a2015dd50 2024-04-25T08:43:34.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a24e00640 1 -- 172.21.15.46:0/333314203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2015dd50 msgr2=0x7f4a200cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a24e00640 1 --2- 172.21.15.46:0/333314203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2015dd50 0x7f4a200cff10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4a080097c0 tx=0x7f4a0801c470 comp rx=0 tx=0).stop 2024-04-25T08:43:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a24e00640 1 -- 172.21.15.46:0/333314203 shutdown_connections 2024-04-25T08:43:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a24e00640 1 --2- 172.21.15.46:0/333314203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2015dd50 0x7f4a200cff10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a24e00640 1 -- 172.21.15.46:0/333314203 >> 172.21.15.46:0/333314203 conn(0x7f4a200b3160 msgr2=0x7f4a200b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a24e00640 1 -- 172.21.15.46:0/333314203 shutdown_connections 2024-04-25T08:43:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.859+0000 7f4a24e00640 1 -- 172.21.15.46:0/333314203 wait complete. 2024-04-25T08:43:34.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 Processor -- start 2024-04-25T08:43:34.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 -- start start 2024-04-25T08:43:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2006f4d0 0x7f4a20155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a200d0450 0x7f4a20154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2015dd50 0x7f4a200b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a200cd560 con 0x7f4a2015dd50 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a200cd3e0 con 0x7f4a2006f4d0 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a200cd6e0 con 0x7f4a200d0450 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2015dd50 0x7f4a200b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2015dd50 0x7f4a200b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34144/0 (socket says 172.21.15.46:34144) 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1e800640 1 -- 172.21.15.46:0/416488328 learned_addr learned my addr 172.21.15.46:0/416488328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a15400640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a200d0450 0x7f4a20154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1de00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2006f4d0 0x7f4a20155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1e800640 1 -- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a200d0450 msgr2=0x7f4a20154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1e800640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a200d0450 0x7f4a20154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1e800640 1 -- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2006f4d0 msgr2=0x7f4a20155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:34.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1e800640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2006f4d0 0x7f4a20155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:34.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1e800640 1 -- 172.21.15.46:0/416488328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2015eaf0 con 0x7f4a2015dd50 2024-04-25T08:43:34.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1de00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2006f4d0 0x7f4a20155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:34.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a1e800640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2015dd50 0x7f4a200b5ab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c710 tx=0x7f4a1800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:34.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a16a00640 1 -- 172.21.15.46:0/416488328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18016020 con 0x7f4a2015dd50 2024-04-25T08:43:34.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a16a00640 1 -- 172.21.15.46:0/416488328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a18006d60 con 0x7f4a2015dd50 2024-04-25T08:43:34.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a16a00640 1 -- 172.21.15.46:0/416488328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a18003bc0 con 0x7f4a2015dd50 2024-04-25T08:43:34.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a200cb4c0 con 0x7f4a2015dd50 2024-04-25T08:43:34.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a20155100 con 0x7f4a2015dd50 2024-04-25T08:43:34.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.863+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a20069010 con 0x7f4a2015dd50 2024-04-25T08:43:34.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.867+0000 7f4a16a00640 1 -- 172.21.15.46:0/416488328 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a1800a850 con 0x7f4a2015dd50 2024-04-25T08:43:34.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.867+0000 7f4a16a00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49fc066f60 0x7f49fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:34.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.867+0000 7f4a16a00640 1 -- 172.21.15.46:0/416488328 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f4a18084d90 con 0x7f4a2015dd50 2024-04-25T08:43:34.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.867+0000 7f4a1de00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49fc066f60 0x7f49fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:34.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.867+0000 7f4a1de00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49fc066f60 0x7f49fc069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a100011f0 tx=0x7f4a10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:34.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:34.871+0000 7f4a16a00640 1 -- 172.21.15.46:0/416488328 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a18020020 con 0x7f4a2015dd50 2024-04-25T08:43:35.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.067+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4a20073cf0 con 0x7f4a2015dd50 2024-04-25T08:43:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a16a00640 1 -- 172.21.15.46:0/416488328 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4a18055c60 con 0x7f4a2015dd50 2024-04-25T08:43:35.654 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T08:43:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49fc066f60 msgr2=0x7f49fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49fc066f60 0x7f49fc069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a100011f0 tx=0x7f4a10009040 comp rx=0 tx=0).stop 2024-04-25T08:43:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2015dd50 msgr2=0x7f4a200b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2015dd50 0x7f4a200b5ab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4a1800c710 tx=0x7f4a1800cbd0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 shutdown_connections 2024-04-25T08:43:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49fc066f60 0x7f49fc069420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a2015dd50 0x7f4a200b5ab0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a200d0450 0x7f4a20154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 --2- 172.21.15.46:0/416488328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a2006f4d0 0x7f4a20155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.663+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 >> 172.21.15.46:0/416488328 conn(0x7f4a200b3160 msgr2=0x7f4a20158970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.667+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 shutdown_connections 2024-04-25T08:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.667+0000 7f4a24e00640 1 -- 172.21.15.46:0/416488328 wait complete. 2024-04-25T08:43:35.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.863+0000 7f3fe6000640 1 Processor -- start 2024-04-25T08:43:35.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.863+0000 7f3fe6000640 1 -- start start 2024-04-25T08:43:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fe6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe0159230 0x7f3fe015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fe6000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3fe00cbeb0 0x7f3fe00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fe6000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00cc700 con 0x7f3fe00cda80 2024-04-25T08:43:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fe6000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00cc9e0 con 0x7f3fe00cbeb0 2024-04-25T08:43:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fe6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00cccc0 con 0x7f3fe0159230 2024-04-25T08:43:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fdfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3fe00cda80 0x7f3fe015f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44098/0 (socket says 172.21.15.46:44098) 2024-04-25T08:43:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fdfe00640 1 -- 172.21.15.46:0/1807158303 learned_addr learned my addr 172.21.15.46:0/1807158303 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fdf400640 1 --2- 172.21.15.46:0/1807158303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe0159230 0x7f3fe015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:35.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fd6a00640 1 --2- 172.21.15.46:0/1807158303 >> v2:172.21.15.82:3300/0 conn(0x7f3fe00cbeb0 0x7f3fe00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fdea00640 1 -- 172.21.15.46:0/1807158303 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148175363 0 0) 0x7f3fe00cc700 con 0x7f3fe00cda80 2024-04-25T08:43:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fdea00640 1 -- 172.21.15.46:0/1807158303 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc4003530 con 0x7f3fe00cda80 2024-04-25T08:43:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fd6a00640 1 -- 172.21.15.46:0/1807158303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe0159230 msgr2=0x7f3fe015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fd6a00640 1 --2- 172.21.15.46:0/1807158303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe0159230 0x7f3fe015bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fd6a00640 1 -- 172.21.15.46:0/1807158303 >> v1:172.21.15.46:6789/0 conn(0x7f3fe00cda80 legacy=0x7f3fe015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fd6a00640 1 -- 172.21.15.46:0/1807158303 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00ccfa0 con 0x7f3fe00cbeb0 2024-04-25T08:43:35.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.867+0000 7f3fdf400640 1 --2- 172.21.15.46:0/1807158303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe0159230 0x7f3fe015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fd6a00640 1 --2- 172.21.15.46:0/1807158303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fe00cbeb0 0x7f3fe00cd540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80097c0 tx=0x7f3fc801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42f16f3a65b62007 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fdea00640 1 -- 172.21.15.46:0/1807158303 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80b0070 con 0x7f3fe00cbeb0 2024-04-25T08:43:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fdea00640 1 -- 172.21.15.46:0/1807158303 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc80ab600 con 0x7f3fe00cbeb0 2024-04-25T08:43:35.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fdea00640 1 -- 172.21.15.46:0/1807158303 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc80ab9b0 con 0x7f3fe00cbeb0 2024-04-25T08:43:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 -- 172.21.15.46:0/1807158303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fe00cbeb0 msgr2=0x7f3fe00cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 --2- 172.21.15.46:0/1807158303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fe00cbeb0 0x7f3fe00cd540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80097c0 tx=0x7f3fc801c410 comp rx=0 tx=0).stop 2024-04-25T08:43:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 -- 172.21.15.46:0/1807158303 shutdown_connections 2024-04-25T08:43:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 --2- 172.21.15.46:0/1807158303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fe00cbeb0 0x7f3fe00cd540 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 --2- 172.21.15.46:0/1807158303 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe0159230 0x7f3fe015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 -- 172.21.15.46:0/1807158303 >> 172.21.15.46:0/1807158303 conn(0x7f3fe00b12a0 msgr2=0x7f3fe00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 -- 172.21.15.46:0/1807158303 shutdown_connections 2024-04-25T08:43:35.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 -- 172.21.15.46:0/1807158303 wait complete. 2024-04-25T08:43:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 Processor -- start 2024-04-25T08:43:35.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.871+0000 7f3fe6000640 1 -- start start 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fe6000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fe00cbeb0 0x7f3fe0157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fe6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe00cda80 0x7f3fe00af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fe6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe0159230 0x7f3fe00af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fe6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0168900 con 0x7f3fe0159230 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fe6000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0168780 con 0x7f3fe00cbeb0 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fe6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0168a80 con 0x7f3fe00cda80 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe0159230 0x7f3fe00af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe0159230 0x7f3fe00af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34152/0 (socket says 172.21.15.46:34152) 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdfe00640 1 -- 172.21.15.46:0/1039566637 learned_addr learned my addr 172.21.15.46:0/1039566637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdf400640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fe00cbeb0 0x7f3fe0157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:35.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fd6a00640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe00cda80 0x7f3fe00af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdfe00640 1 -- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe00cda80 msgr2=0x7f3fe00af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdfe00640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe00cda80 0x7f3fe00af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdfe00640 1 -- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fe00cbeb0 msgr2=0x7f3fe0157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdfe00640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fe00cbeb0 0x7f3fe0157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdfe00640 1 -- 172.21.15.46:0/1039566637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00b0440 con 0x7f3fe0159230 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fdfe00640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe0159230 0x7f3fe00af900 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800caa0 tx=0x7f3fd800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fd7e00640 1 -- 172.21.15.46:0/1039566637 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8017070 con 0x7f3fe0159230 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fd7e00640 1 -- 172.21.15.46:0/1039566637 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd800ac00 con 0x7f3fe0159230 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe013cb00 con 0x7f3fe0159230 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fd7e00640 1 -- 172.21.15.46:0/1039566637 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8012400 con 0x7f3fe0159230 2024-04-25T08:43:35.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe0158190 con 0x7f3fe0159230 2024-04-25T08:43:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.875+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe013c910 con 0x7f3fe0159230 2024-04-25T08:43:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.879+0000 7f3fd7e00640 1 -- 172.21.15.46:0/1039566637 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3fd801a070 con 0x7f3fe0159230 2024-04-25T08:43:35.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.879+0000 7f3fd7e00640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fbc066f60 0x7f3fbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.879+0000 7f3fd7e00640 1 -- 172.21.15.46:0/1039566637 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3fd8084f90 con 0x7f3fe0159230 2024-04-25T08:43:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.879+0000 7f3fdf400640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fbc066f60 0x7f3fbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:35.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.879+0000 7f3fdf400640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fbc066f60 0x7f3fbc069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009580 tx=0x7f3fd0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:35.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:35.883+0000 7f3fd7e00640 1 -- 172.21.15.46:0/1039566637 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3fd8055720 con 0x7f3fe0159230 2024-04-25T08:43:36.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.047+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fe0069080 con 0x7f3fe0159230 2024-04-25T08:43:36.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.047+0000 7f3fd7e00640 1 -- 172.21.15.46:0/1039566637 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16907 (secure 0 0 0) 0x7f3fd80597c0 con 0x7f3fe0159230 2024-04-25T08:43:36.041 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:36.041 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:35.070342+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:43:32.873190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:43:34.064607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fbc066f60 msgr2=0x7f3fbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fbc066f60 0x7f3fbc069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009580 tx=0x7f3fd0009190 comp rx=0 tx=0).stop 2024-04-25T08:43:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe0159230 msgr2=0x7f3fe00af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe0159230 0x7f3fe00af900 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800caa0 tx=0x7f3fd800cf60 comp rx=0 tx=0).stop 2024-04-25T08:43:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 shutdown_connections 2024-04-25T08:43:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fbc066f60 0x7f3fbc069420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fe0159230 0x7f3fe00af900 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3fe00cda80 0x7f3fe00af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 --2- 172.21.15.46:0/1039566637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fe00cbeb0 0x7f3fe0157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 >> 172.21.15.46:0/1039566637 conn(0x7f3fe00b12a0 msgr2=0x7f3fe0144e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 shutdown_connections 2024-04-25T08:43:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.051+0000 7f3fe6000640 1 -- 172.21.15.46:0/1039566637 wait complete. 2024-04-25T08:43:36.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a26400640 1 Processor -- start 2024-04-25T08:43:36.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a26400640 1 -- start start 2024-04-25T08:43:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a26400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a2015bdb0 0x7f6a200cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a26400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 0x7f6a2015d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a26400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a200cc700 con 0x7f6a20159230 2024-04-25T08:43:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a26400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a200cc9e0 con 0x7f6a2015bdb0 2024-04-25T08:43:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a26400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a200cccc0 con 0x7f6a200cda80 2024-04-25T08:43:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a16a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6a2015bdb0 0x7f6a200cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a1f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6a20159230 0x7f6a200cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44108/0 (socket says 172.21.15.46:44108) 2024-04-25T08:43:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a1f400640 1 -- 172.21.15.46:0/967706322 learned_addr learned my addr 172.21.15.46:0/967706322 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a1fe00640 1 --2- 172.21.15.46:0/967706322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 0x7f6a2015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a16a00640 1 -- 172.21.15.46:0/967706322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 msgr2=0x7f6a2015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a16a00640 1 --2- 172.21.15.46:0/967706322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 0x7f6a2015d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a16a00640 1 -- 172.21.15.46:0/967706322 >> v1:172.21.15.46:6789/0 conn(0x7f6a20159230 legacy=0x7f6a200cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a16a00640 1 -- 172.21.15.46:0/967706322 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200ccfa0 con 0x7f6a2015bdb0 2024-04-25T08:43:36.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a1ea00640 1 -- 172.21.15.46:0/967706322 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806638003 0 0) 0x7f6a200cc700 con 0x7f6a20159230 2024-04-25T08:43:36.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.211+0000 7f6a1fe00640 1 --2- 172.21.15.46:0/967706322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 0x7f6a2015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:36.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a16a00640 1 --2- 172.21.15.46:0/967706322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 0x7f6a200cd540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6a080097c0 tx=0x7f6a0801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417b6f19178931fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:36.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a1ea00640 1 -- 172.21.15.46:0/967706322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a080b0070 con 0x7f6a2015bdb0 2024-04-25T08:43:36.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a1ea00640 1 -- 172.21.15.46:0/967706322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a080ab600 con 0x7f6a2015bdb0 2024-04-25T08:43:36.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a1ea00640 1 -- 172.21.15.46:0/967706322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a080ab9d0 con 0x7f6a2015bdb0 2024-04-25T08:43:36.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a26400640 1 -- 172.21.15.46:0/967706322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 msgr2=0x7f6a200cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a26400640 1 --2- 172.21.15.46:0/967706322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 0x7f6a200cd540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6a080097c0 tx=0x7f6a0801c410 comp rx=0 tx=0).stop 2024-04-25T08:43:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a26400640 1 -- 172.21.15.46:0/967706322 shutdown_connections 2024-04-25T08:43:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a26400640 1 --2- 172.21.15.46:0/967706322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 0x7f6a2015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a26400640 1 --2- 172.21.15.46:0/967706322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 0x7f6a200cd540 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a26400640 1 -- 172.21.15.46:0/967706322 >> 172.21.15.46:0/967706322 conn(0x7f6a200b12a0 msgr2=0x7f6a200b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a26400640 1 -- 172.21.15.46:0/967706322 shutdown_connections 2024-04-25T08:43:36.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.215+0000 7f6a26400640 1 -- 172.21.15.46:0/967706322 wait complete. 2024-04-25T08:43:36.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 Processor -- start 2024-04-25T08:43:36.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 -- start start 2024-04-25T08:43:36.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 0x7f6a201584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20159230 0x7f6a2006ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 0x7f6a2006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2015e320 con 0x7f6a2015bdb0 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2015e1a0 con 0x7f6a20159230 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2015e4a0 con 0x7f6a200cda80 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 0x7f6a2006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 0x7f6a2006f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34168/0 (socket says 172.21.15.46:34168) 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1fe00640 1 -- 172.21.15.46:0/4108827768 learned_addr learned my addr 172.21.15.46:0/4108827768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1f400640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 0x7f6a201584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a16a00640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20159230 0x7f6a2006ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1fe00640 1 -- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 msgr2=0x7f6a201584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1fe00640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 0x7f6a201584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1fe00640 1 -- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20159230 msgr2=0x7f6a2006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1fe00640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20159230 0x7f6a2006ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1fe00640 1 -- 172.21.15.46:0/4108827768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a200b2820 con 0x7f6a2015bdb0 2024-04-25T08:43:36.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a1fe00640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 0x7f6a2006f110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6a18007df0 tx=0x7f6a1800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a17e00640 1 -- 172.21.15.46:0/4108827768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1801a070 con 0x7f6a2015bdb0 2024-04-25T08:43:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a17e00640 1 -- 172.21.15.46:0/4108827768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a180039a0 con 0x7f6a2015bdb0 2024-04-25T08:43:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a17e00640 1 -- 172.21.15.46:0/4108827768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18003d90 con 0x7f6a2015bdb0 2024-04-25T08:43:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2013faf0 con 0x7f6a2015bdb0 2024-04-25T08:43:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a200b2dc0 con 0x7f6a2015bdb0 2024-04-25T08:43:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.219+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a20069080 con 0x7f6a2015bdb0 2024-04-25T08:43:36.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.223+0000 7f6a17e00640 1 -- 172.21.15.46:0/4108827768 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a1801d070 con 0x7f6a2015bdb0 2024-04-25T08:43:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.223+0000 7f6a17e00640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69fc066f60 0x7f69fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.223+0000 7f6a17e00640 1 -- 172.21.15.46:0/4108827768 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a180853d0 con 0x7f6a2015bdb0 2024-04-25T08:43:36.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.223+0000 7f6a1f400640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69fc066f60 0x7f69fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.223+0000 7f6a1f400640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69fc066f60 0x7f69fc069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a10000f70 tx=0x7f6a10007df0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:36.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.227+0000 7f6a17e00640 1 -- 172.21.15.46:0/4108827768 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a18010070 con 0x7f6a2015bdb0 2024-04-25T08:43:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.387+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a200723d0 con 0x7f6a2015bdb0 2024-04-25T08:43:36.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a17e00640 1 -- 172.21.15.46:0/4108827768 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16907 (secure 0 0 0) 0x7f6a18055c00 con 0x7f6a2015bdb0 2024-04-25T08:43:36.384 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:36.384 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":39,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:35.070342+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:43:32.873190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:43:34.064607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69fc066f60 msgr2=0x7f69fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69fc066f60 0x7f69fc069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a10000f70 tx=0x7f6a10007df0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 msgr2=0x7f6a2006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 0x7f6a2006f110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6a18007df0 tx=0x7f6a1800d4a0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 shutdown_connections 2024-04-25T08:43:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69fc066f60 0x7f69fc069420 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a2015bdb0 0x7f6a2006f110 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a20159230 0x7f6a2006ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 --2- 172.21.15.46:0/4108827768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a200cda80 0x7f6a201584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 >> 172.21.15.46:0/4108827768 conn(0x7f6a200b12a0 msgr2=0x7f6a201422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 shutdown_connections 2024-04-25T08:43:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.391+0000 7f6a26400640 1 -- 172.21.15.46:0/4108827768 wait complete. 2024-04-25T08:43:36.405 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:43:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f71c00640 1 Processor -- start 2024-04-25T08:43:36.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f71c00640 1 -- start start 2024-04-25T08:43:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f71c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c15d900 0x7f6f6c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f71c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f6c06b470 0x7f6f6c0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f71c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6c0b0040 con 0x7f6f6c0afc20 2024-04-25T08:43:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f71c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6c06fca0 con 0x7f6f6c06b470 2024-04-25T08:43:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f71c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c06ff80 con 0x7f6f6c15d900 2024-04-25T08:43:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f6b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f6c06b470 0x7f6f6c0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f62000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6f6c0afc20 0x7f6f6c06ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44110/0 (socket says 172.21.15.46:44110) 2024-04-25T08:43:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f62000640 1 -- 172.21.15.46:0/4165690729 learned_addr learned my addr 172.21.15.46:0/4165690729 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.555+0000 7f6f6aa00640 1 --2- 172.21.15.46:0/4165690729 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c15d900 0x7f6f6c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6b400640 1 -- 172.21.15.46:0/4165690729 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c15d900 msgr2=0x7f6f6c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6b400640 1 --2- 172.21.15.46:0/4165690729 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c15d900 0x7f6f6c0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6b400640 1 -- 172.21.15.46:0/4165690729 >> v1:172.21.15.46:6789/0 conn(0x7f6f6c0afc20 legacy=0x7f6f6c06ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6b400640 1 -- 172.21.15.46:0/4165690729 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c070260 con 0x7f6f6c06b470 2024-04-25T08:43:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6aa00640 1 --2- 172.21.15.46:0/4165690729 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c15d900 0x7f6f6c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:36.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6a000640 1 -- 172.21.15.46:0/4165690729 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231502708 0 0) 0x7f6f6c0b0040 con 0x7f6f6c0afc20 2024-04-25T08:43:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6b400640 1 --2- 172.21.15.46:0/4165690729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 0x7f6f6c0d0210 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009d90 tx=0x7f6f6401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9992f0a495a0261f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6a000640 1 -- 172.21.15.46:0/4165690729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f640b2070 con 0x7f6f6c06b470 2024-04-25T08:43:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6a000640 1 -- 172.21.15.46:0/4165690729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f640a5890 con 0x7f6f6c06b470 2024-04-25T08:43:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f6a000640 1 -- 172.21.15.46:0/4165690729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f640a5c60 con 0x7f6f6c06b470 2024-04-25T08:43:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f71c00640 1 -- 172.21.15.46:0/4165690729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 msgr2=0x7f6f6c0d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f71c00640 1 --2- 172.21.15.46:0/4165690729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 0x7f6f6c0d0210 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009d90 tx=0x7f6f6401c8e0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f71c00640 1 -- 172.21.15.46:0/4165690729 shutdown_connections 2024-04-25T08:43:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f71c00640 1 --2- 172.21.15.46:0/4165690729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 0x7f6f6c0d0210 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f71c00640 1 --2- 172.21.15.46:0/4165690729 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c15d900 0x7f6f6c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.559+0000 7f6f71c00640 1 -- 172.21.15.46:0/4165690729 >> 172.21.15.46:0/4165690729 conn(0x7f6f6c06e3e0 msgr2=0x7f6f6c06e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 -- 172.21.15.46:0/4165690729 shutdown_connections 2024-04-25T08:43:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 -- 172.21.15.46:0/4165690729 wait complete. 2024-04-25T08:43:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 Processor -- start 2024-04-25T08:43:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 -- start start 2024-04-25T08:43:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 0x7f6f6c14c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c0afc20 0x7f6f6c14d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f6c15d900 0x7f6f6c0c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0d10e0 con 0x7f6f6c06b470 2024-04-25T08:43:36.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0d0f60 con 0x7f6f6c15d900 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0d1260 con 0x7f6f6c0afc20 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 0x7f6f6c14c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 0x7f6f6c14c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34178/0 (socket says 172.21.15.46:34178) 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6aa00640 1 -- 172.21.15.46:0/3019502181 learned_addr learned my addr 172.21.15.46:0/3019502181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6aa00640 1 -- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c0afc20 msgr2=0x7f6f6c14d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f62000640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c0afc20 0x7f6f6c14d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6b400640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f6c15d900 0x7f6f6c0c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6aa00640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c0afc20 0x7f6f6c14d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6aa00640 1 -- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f6c15d900 msgr2=0x7f6f6c0c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6aa00640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f6c15d900 0x7f6f6c0c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6aa00640 1 -- 172.21.15.46:0/3019502181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c148ef0 con 0x7f6f6c06b470 2024-04-25T08:43:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6b400640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f6c15d900 0x7f6f6c0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f62000640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c0afc20 0x7f6f6c14d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f6aa00640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 0x7f6f6c14c8b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00d800 tx=0x7f6f5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:36.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f63400640 1 -- 172.21.15.46:0/3019502181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c018020 con 0x7f6f6c06b470 2024-04-25T08:43:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f63400640 1 -- 172.21.15.46:0/3019502181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0096c0 con 0x7f6f6c06b470 2024-04-25T08:43:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f63400640 1 -- 172.21.15.46:0/3019502181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c009ab0 con 0x7f6f6c06b470 2024-04-25T08:43:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.563+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6c14f500 con 0x7f6f6c06b470 2024-04-25T08:43:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.567+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c14fac0 con 0x7f6f6c06b470 2024-04-25T08:43:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.567+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6c069010 con 0x7f6f6c06b470 2024-04-25T08:43:36.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.567+0000 7f6f63400640 1 -- 172.21.15.46:0/3019502181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f5c01b020 con 0x7f6f6c06b470 2024-04-25T08:43:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.567+0000 7f6f63400640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f48066f60 0x7f6f48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.567+0000 7f6f63400640 1 -- 172.21.15.46:0/3019502181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f5c0856d0 con 0x7f6f6c06b470 2024-04-25T08:43:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.567+0000 7f6f62000640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f48066f60 0x7f6f48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:36.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.567+0000 7f6f62000640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f48066f60 0x7f6f48069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0af760 tx=0x7f6f54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:36.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.571+0000 7f6f63400640 1 -- 172.21.15.46:0/3019502181 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f5c022020 con 0x7f6f6c06b470 2024-04-25T08:43:36.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:36.771+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6f6c072890 con 0x7f6f6c06b470 2024-04-25T08:43:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f63400640 1 -- 172.21.15.46:0/3019502181 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6f6c148ef0 con 0x7f6f6c06b470 2024-04-25T08:43:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f48066f60 msgr2=0x7f6f48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f48066f60 0x7f6f48069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0af760 tx=0x7f6f54009040 comp rx=0 tx=0).stop 2024-04-25T08:43:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 msgr2=0x7f6f6c14c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 0x7f6f6c14c8b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00d800 tx=0x7f6f5c00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 shutdown_connections 2024-04-25T08:43:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f48066f60 0x7f6f48069420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f6c15d900 0x7f6f6c0c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f6c0afc20 0x7f6f6c14d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 --2- 172.21.15.46:0/3019502181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c06b470 0x7f6f6c14c8b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 >> 172.21.15.46:0/3019502181 conn(0x7f6f6c06e3e0 msgr2=0x7f6f6c0b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 shutdown_connections 2024-04-25T08:43:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.687+0000 7f6f71c00640 1 -- 172.21.15.46:0/3019502181 wait complete. 2024-04-25T08:43:37.706 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:43:37.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:43:37.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f031d200640 1 Processor -- start 2024-04-25T08:43:37.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f031d200640 1 -- start start 2024-04-25T08:43:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f031d200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 0x7f0318072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f031d200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f031806e620 0x7f031806bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f031d200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03180b3850 con 0x7f031806ea60 2024-04-25T08:43:37.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f031d200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031806e190 con 0x7f031806e620 2024-04-25T08:43:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f031d200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03180b3af0 con 0x7f031815dae0 2024-04-25T08:43:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f0316800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f031806ea60 0x7f03180acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44120/0 (socket says 172.21.15.46:44120) 2024-04-25T08:43:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f0316800640 1 -- 172.21.15.46:0/458325545 learned_addr learned my addr 172.21.15.46:0/458325545 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f030fe00640 1 --2- 172.21.15.46:0/458325545 >> v2:172.21.15.82:3300/0 conn(0x7f031806e620 0x7f031806bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f0315e00640 1 --2- 172.21.15.46:0/458325545 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 0x7f0318072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f0315400640 1 -- 172.21.15.46:0/458325545 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465281005 0 0) 0x7f03180b3850 con 0x7f031806ea60 2024-04-25T08:43:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f0315400640 1 -- 172.21.15.46:0/458325545 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02fc003530 con 0x7f031806ea60 2024-04-25T08:43:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f030fe00640 1 -- 172.21.15.46:0/458325545 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 msgr2=0x7f0318072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f030fe00640 1 --2- 172.21.15.46:0/458325545 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 0x7f0318072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f030fe00640 1 -- 172.21.15.46:0/458325545 >> v1:172.21.15.46:6789/0 conn(0x7f031806ea60 legacy=0x7f03180acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.863+0000 7f030fe00640 1 -- 172.21.15.46:0/458325545 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180b3dd0 con 0x7f031806e620 2024-04-25T08:43:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f030fe00640 1 --2- 172.21.15.46:0/458325545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031806e620 0x7f031806bc20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f031806e310 tx=0x7f030001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c09021c76aff29e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f0315400640 1 -- 172.21.15.46:0/458325545 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03000b2070 con 0x7f031806e620 2024-04-25T08:43:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f0315400640 1 -- 172.21.15.46:0/458325545 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03000ad680 con 0x7f031806e620 2024-04-25T08:43:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f0315400640 1 -- 172.21.15.46:0/458325545 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03000ada30 con 0x7f031806e620 2024-04-25T08:43:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f031d200640 1 -- 172.21.15.46:0/458325545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031806e620 msgr2=0x7f031806bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f031d200640 1 --2- 172.21.15.46:0/458325545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031806e620 0x7f031806bc20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f031806e310 tx=0x7f030001c470 comp rx=0 tx=0).stop 2024-04-25T08:43:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f031d200640 1 -- 172.21.15.46:0/458325545 shutdown_connections 2024-04-25T08:43:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f031d200640 1 --2- 172.21.15.46:0/458325545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031806e620 0x7f031806bc20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f031d200640 1 --2- 172.21.15.46:0/458325545 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 0x7f0318072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f031d200640 1 -- 172.21.15.46:0/458325545 >> 172.21.15.46:0/458325545 conn(0x7f0318071ae0 msgr2=0x7f0318071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f031d200640 1 -- 172.21.15.46:0/458325545 shutdown_connections 2024-04-25T08:43:37.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.867+0000 7f031d200640 1 -- 172.21.15.46:0/458325545 wait complete. 2024-04-25T08:43:37.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 Processor -- start 2024-04-25T08:43:37.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 -- start start 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031806e620 0x7f0318145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031806ea60 0x7f03181525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 0x7f03181357f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031815dec0 con 0x7f031806e620 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031806c570 con 0x7f031806ea60 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031806f6d0 con 0x7f031815dae0 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0315e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031806e620 0x7f0318145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0315e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031806e620 0x7f0318145030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34184/0 (socket says 172.21.15.46:34184) 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0315e00640 1 -- 172.21.15.46:0/595003324 learned_addr learned my addr 172.21.15.46:0/595003324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f030fe00640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031806ea60 0x7f03181525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0316800640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 0x7f03181357f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0315e00640 1 -- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 msgr2=0x7f03181357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0315e00640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 0x7f03181357f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0315e00640 1 -- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031806ea60 msgr2=0x7f03181525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0315e00640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031806ea60 0x7f03181525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0315e00640 1 -- 172.21.15.46:0/595003324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318067d60 con 0x7f031806e620 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0316800640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 0x7f03181357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f0315e00640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031806e620 0x7f0318145030 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f030800e8a0 tx=0x7f030800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f030e000640 1 -- 172.21.15.46:0/595003324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308017070 con 0x7f031806e620 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f030e000640 1 -- 172.21.15.46:0/595003324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030800c940 con 0x7f031806e620 2024-04-25T08:43:37.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f030e000640 1 -- 172.21.15.46:0/595003324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030800ccf0 con 0x7f031806e620 2024-04-25T08:43:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03180b44a0 con 0x7f031806e620 2024-04-25T08:43:37.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.871+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0318160690 con 0x7f031806e620 2024-04-25T08:43:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.875+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0318069010 con 0x7f031806e620 2024-04-25T08:43:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.875+0000 7f030e000640 1 -- 172.21.15.46:0/595003324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f030801a070 con 0x7f031806e620 2024-04-25T08:43:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.875+0000 7f030e000640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02f4066f60 0x7f02f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.875+0000 7f030e000640 1 -- 172.21.15.46:0/595003324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0308085190 con 0x7f031806e620 2024-04-25T08:43:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.875+0000 7f030fe00640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02f4066f60 0x7f02f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.875+0000 7f030fe00640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02f4066f60 0x7f02f4069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f03000af000 tx=0x7f03000a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:37.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:37.879+0000 7f030e000640 1 -- 172.21.15.46:0/595003324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0308055f70 con 0x7f031806e620 2024-04-25T08:43:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.079+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f03180cf060 con 0x7f031806e620 2024-04-25T08:43:38.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.691+0000 7f030e000640 1 -- 172.21.15.46:0/595003324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0308055870 con 0x7f031806e620 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02f4066f60 msgr2=0x7f02f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02f4066f60 0x7f02f4069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f03000af000 tx=0x7f03000a7000 comp rx=0 tx=0).stop 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031806e620 msgr2=0x7f0318145030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031806e620 0x7f0318145030 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f030800e8a0 tx=0x7f030800eca0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 shutdown_connections 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02f4066f60 0x7f02f4069420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f031815dae0 0x7f03181357f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031806ea60 0x7f03181525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 --2- 172.21.15.46:0/595003324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f031806e620 0x7f0318145030 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 >> 172.21.15.46:0/595003324 conn(0x7f0318071ae0 msgr2=0x7f03180b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:38.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 shutdown_connections 2024-04-25T08:43:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.695+0000 7f031d200640 1 -- 172.21.15.46:0/595003324 wait complete. 2024-04-25T08:43:38.709 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:43:38.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd956600640 1 Processor -- start 2024-04-25T08:43:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd956600640 1 -- start start 2024-04-25T08:43:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd956600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd950099a00 0x7fd950099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd956600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd9500ccde0 0x7fd9500cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd956600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500b2240 0x7fd9500b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd956600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd95009a320 con 0x7fd950099a00 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd956600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd95006bdc0 con 0x7fd9500ccde0 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd956600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95006c0a0 con 0x7fd9500b2240 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd94f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd950099a00 0x7fd950099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd94f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd950099a00 0x7fd950099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46356/0 (socket says 172.21.15.46:46356) 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd94f400640 1 -- 172.21.15.46:0/1152916453 learned_addr learned my addr 172.21.15.46:0/1152916453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd946a00640 1 --2- 172.21.15.46:0/1152916453 >> v2:172.21.15.82:3300/0 conn(0x7fd9500ccde0 0x7fd9500cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd94fe00640 1 --2- 172.21.15.46:0/1152916453 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500b2240 0x7fd9500b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd94f400640 1 -- 172.21.15.46:0/1152916453 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500b2240 msgr2=0x7fd9500b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd94f400640 1 --2- 172.21.15.46:0/1152916453 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500b2240 0x7fd9500b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd94f400640 1 -- 172.21.15.46:0/1152916453 >> v2:172.21.15.82:3300/0 conn(0x7fd9500ccde0 msgr2=0x7fd9500cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd94f400640 1 --2- 172.21.15.46:0/1152916453 >> v2:172.21.15.82:3300/0 conn(0x7fd9500ccde0 0x7fd9500cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.863+0000 7fd94f400640 1 -- 172.21.15.46:0/1152916453 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95006c380 con 0x7fd950099a00 2024-04-25T08:43:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd94f400640 1 --2- 172.21.15.46:0/1152916453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 0x7fd950099de0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd950072be0 tx=0x7fd94001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2782b554ce9353c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd94ea00640 1 -- 172.21.15.46:0/1152916453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9400ab640 con 0x7fd950099a00 2024-04-25T08:43:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd94ea00640 1 -- 172.21.15.46:0/1152916453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9400b0070 con 0x7fd950099a00 2024-04-25T08:43:38.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd94ea00640 1 -- 172.21.15.46:0/1152916453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9400aba00 con 0x7fd950099a00 2024-04-25T08:43:38.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd956600640 1 -- 172.21.15.46:0/1152916453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 msgr2=0x7fd950099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:38.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd956600640 1 --2- 172.21.15.46:0/1152916453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 0x7fd950099de0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd950072be0 tx=0x7fd94001c460 comp rx=0 tx=0).stop 2024-04-25T08:43:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd956600640 1 -- 172.21.15.46:0/1152916453 shutdown_connections 2024-04-25T08:43:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd956600640 1 --2- 172.21.15.46:0/1152916453 >> v2:172.21.15.82:3300/0 conn(0x7fd9500ccde0 0x7fd9500cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd956600640 1 --2- 172.21.15.46:0/1152916453 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500b2240 0x7fd9500b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd956600640 1 --2- 172.21.15.46:0/1152916453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 0x7fd950099de0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd956600640 1 -- 172.21.15.46:0/1152916453 >> 172.21.15.46:0/1152916453 conn(0x7fd9500b3120 msgr2=0x7fd9500b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd956600640 1 -- 172.21.15.46:0/1152916453 shutdown_connections 2024-04-25T08:43:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.867+0000 7fd956600640 1 -- 172.21.15.46:0/1152916453 wait complete. 2024-04-25T08:43:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 Processor -- start 2024-04-25T08:43:38.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 -- start start 2024-04-25T08:43:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 0x7fd9500c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9500b2240 0x7fd950152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500ccde0 0x7fd950153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500cd7d0 con 0x7fd950099a00 2024-04-25T08:43:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95015dd10 con 0x7fd9500b2240 2024-04-25T08:43:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9500b43b0 con 0x7fd9500ccde0 2024-04-25T08:43:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 0x7fd9500c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 0x7fd9500c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46360/0 (socket says 172.21.15.46:46360) 2024-04-25T08:43:38.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94f400640 1 -- 172.21.15.46:0/1382512674 learned_addr learned my addr 172.21.15.46:0/1382512674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:38.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd946a00640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9500b2240 0x7fd950152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:38.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94fe00640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500ccde0 0x7fd950153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:38.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94f400640 1 -- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500ccde0 msgr2=0x7fd950153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:38.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94f400640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500ccde0 0x7fd950153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94f400640 1 -- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9500b2240 msgr2=0x7fd950152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:38.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94f400640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9500b2240 0x7fd950152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94f400640 1 -- 172.21.15.46:0/1382512674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9500b4690 con 0x7fd950099a00 2024-04-25T08:43:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd94f400640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 0x7fd9500c7bd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd94001ab10 tx=0x7fd94001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd947e00640 1 -- 172.21.15.46:0/1382512674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9400abdf0 con 0x7fd950099a00 2024-04-25T08:43:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd947e00640 1 -- 172.21.15.46:0/1382512674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9400be3f0 con 0x7fd950099a00 2024-04-25T08:43:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd947e00640 1 -- 172.21.15.46:0/1382512674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9400be7c0 con 0x7fd950099a00 2024-04-25T08:43:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9500b4970 con 0x7fd950099a00 2024-04-25T08:43:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.871+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9500b4f30 con 0x7fd950099a00 2024-04-25T08:43:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.875+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd950069080 con 0x7fd950099a00 2024-04-25T08:43:38.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.875+0000 7fd947e00640 1 -- 172.21.15.46:0/1382512674 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9400c7440 con 0x7fd950099a00 2024-04-25T08:43:38.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.875+0000 7fd947e00640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd92c066f60 0x7fd92c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:38.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.875+0000 7fd947e00640 1 -- 172.21.15.46:0/1382512674 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd94011f760 con 0x7fd950099a00 2024-04-25T08:43:38.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.875+0000 7fd946a00640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd92c066f60 0x7fd92c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:38.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.875+0000 7fd946a00640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd92c066f60 0x7fd92c069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd9500af6c0 tx=0x7fd938008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:38.879+0000 7fd947e00640 1 -- 172.21.15.46:0/1382512674 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd940016110 con 0x7fd950099a00 2024-04-25T08:43:39.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.075+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd950159cb0 con 0x7fd950099a00 2024-04-25T08:43:39.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd947e00640 1 -- 172.21.15.46:0/1382512674 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fd9400f06f0 con 0x7fd950099a00 2024-04-25T08:43:39.070 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:39.071 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1079752579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1079752579},{"type":"v1","addr":"172.21.15.82:6836","nonce":1079752579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/106980078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":106980078},{"type":"v1","addr":"172.21.15.82:6837","nonce":106980078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1318225485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1318225485},{"type":"v1","addr":"172.21.15.46:6834","nonce":1318225485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:43:35.070722+0000","modified":"2024-04-25T08:43:38.080210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2025854544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2025854544},{"type":"v1","addr":"172.21.15.46:6835","nonce":2025854544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:43:39.071 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-25T08:43:39.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd92c066f60 msgr2=0x7fd92c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd92c066f60 0x7fd92c069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd9500af6c0 tx=0x7fd938008040 comp rx=0 tx=0).stop 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 msgr2=0x7fd9500c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 0x7fd9500c7bd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd94001ab10 tx=0x7fd94001aca0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 shutdown_connections 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd92c066f60 0x7fd92c069420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9500ccde0 0x7fd950153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9500b2240 0x7fd950152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 --2- 172.21.15.46:0/1382512674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd950099a00 0x7fd9500c7bd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 >> 172.21.15.46:0/1382512674 conn(0x7fd9500b3120 msgr2=0x7fd9500cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 shutdown_connections 2024-04-25T08:43:39.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.079+0000 7fd956600640 1 -- 172.21.15.46:0/1382512674 wait complete. 2024-04-25T08:43:39.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:39.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.239+0000 7f0311200640 1 Processor -- start 2024-04-25T08:43:39.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.239+0000 7f0311200640 1 -- start start 2024-04-25T08:43:39.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.239+0000 7f0311200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f030c0cebb0 0x7f030c0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.239+0000 7f0311200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f030c161590 0x7f030c161a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.239+0000 7f0311200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c06e9a0 0x7f030c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.239+0000 7f0311200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030c161f40 con 0x7f030c0cebb0 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.239+0000 7f0311200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030c0cef90 con 0x7f030c161590 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.239+0000 7f0311200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030c06f720 con 0x7f030c06e9a0 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f030c0cebb0 0x7f030c0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f030c0cebb0 0x7f030c0b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46374/0 (socket says 172.21.15.46:46374) 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309e00640 1 -- 172.21.15.46:0/1679061512 learned_addr learned my addr 172.21.15.46:0/1679061512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f030a800640 1 --2- 172.21.15.46:0/1679061512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c06e9a0 0x7f030c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309400640 1 --2- 172.21.15.46:0/1679061512 >> v2:172.21.15.82:3300/0 conn(0x7f030c161590 0x7f030c161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309e00640 1 -- 172.21.15.46:0/1679061512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c06e9a0 msgr2=0x7f030c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309e00640 1 --2- 172.21.15.46:0/1679061512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c06e9a0 0x7f030c06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309e00640 1 -- 172.21.15.46:0/1679061512 >> v2:172.21.15.82:3300/0 conn(0x7f030c161590 msgr2=0x7f030c161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309e00640 1 --2- 172.21.15.46:0/1679061512 >> v2:172.21.15.82:3300/0 conn(0x7f030c161590 0x7f030c161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309e00640 1 -- 172.21.15.46:0/1679061512 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c06fa00 con 0x7f030c0cebb0 2024-04-25T08:43:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f030a800640 1 --2- 172.21.15.46:0/1679061512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c06e9a0 0x7f030c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0309e00640 1 --2- 172.21.15.46:0/1679061512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c0cebb0 0x7f030c0b25a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f030c050090 tx=0x7f02fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b76352cad632ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0308a00640 1 -- 172.21.15.46:0/1679061512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0b1070 con 0x7f030c0cebb0 2024-04-25T08:43:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0308a00640 1 -- 172.21.15.46:0/1679061512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc0ac630 con 0x7f030c0cebb0 2024-04-25T08:43:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.243+0000 7f0308a00640 1 -- 172.21.15.46:0/1679061512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0a8050 con 0x7f030c0cebb0 2024-04-25T08:43:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 -- 172.21.15.46:0/1679061512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c0cebb0 msgr2=0x7f030c0b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 --2- 172.21.15.46:0/1679061512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c0cebb0 0x7f030c0b25a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f030c050090 tx=0x7f02fc01c520 comp rx=0 tx=0).stop 2024-04-25T08:43:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 -- 172.21.15.46:0/1679061512 shutdown_connections 2024-04-25T08:43:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 --2- 172.21.15.46:0/1679061512 >> v2:172.21.15.82:3300/0 conn(0x7f030c161590 0x7f030c161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 --2- 172.21.15.46:0/1679061512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c06e9a0 0x7f030c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 --2- 172.21.15.46:0/1679061512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c0cebb0 0x7f030c0b25a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 -- 172.21.15.46:0/1679061512 >> 172.21.15.46:0/1679061512 conn(0x7f030c0b12a0 msgr2=0x7f030c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 -- 172.21.15.46:0/1679061512 shutdown_connections 2024-04-25T08:43:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 -- 172.21.15.46:0/1679061512 wait complete. 2024-04-25T08:43:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 Processor -- start 2024-04-25T08:43:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 -- start start 2024-04-25T08:43:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c06e9a0 0x7f030c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0cebb0 0x7f030c1455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c161590 0x7f030c15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0adba0 con 0x7f030c06e9a0 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0ada20 con 0x7f030c0cebb0 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0311200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030c0add20 con 0x7f030c161590 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0309e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c06e9a0 0x7f030c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0309e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c06e9a0 0x7f030c157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46388/0 (socket says 172.21.15.46:46388) 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0309e00640 1 -- 172.21.15.46:0/714897752 learned_addr learned my addr 172.21.15.46:0/714897752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f030a800640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c161590 0x7f030c15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0309400640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0cebb0 0x7f030c1455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0309e00640 1 -- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c161590 msgr2=0x7f030c15ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0309e00640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c161590 0x7f030c15ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0309e00640 1 -- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0cebb0 msgr2=0x7f030c1455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0309e00640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0cebb0 0x7f030c1455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.247+0000 7f0309e00640 1 -- 172.21.15.46:0/714897752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c161b90 con 0x7f030c06e9a0 2024-04-25T08:43:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0309e00640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c06e9a0 0x7f030c157520 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f02fc000c10 tx=0x7f02fc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0302000640 1 -- 172.21.15.46:0/714897752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0bc070 con 0x7f030c06e9a0 2024-04-25T08:43:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0302000640 1 -- 172.21.15.46:0/714897752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc0ab5e0 con 0x7f030c06e9a0 2024-04-25T08:43:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0302000640 1 -- 172.21.15.46:0/714897752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0b1070 con 0x7f030c06e9a0 2024-04-25T08:43:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030c072300 con 0x7f030c06e9a0 2024-04-25T08:43:39.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030c145e30 con 0x7f030c06e9a0 2024-04-25T08:43:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030c069080 con 0x7f030c06e9a0 2024-04-25T08:43:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0302000640 1 -- 172.21.15.46:0/714897752 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02fc0ab990 con 0x7f030c06e9a0 2024-04-25T08:43:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0302000640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02ec066f90 0x7f02ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0302000640 1 -- 172.21.15.46:0/714897752 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02fc11ebf0 con 0x7f030c06e9a0 2024-04-25T08:43:39.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.251+0000 7f0309400640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02ec066f90 0x7f02ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.255+0000 7f0309400640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02ec066f90 0x7f02ec069450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f030c1415e0 tx=0x7f02f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:39.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.255+0000 7f0302000640 1 -- 172.21.15.46:0/714897752 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f02fc016110 con 0x7f030c06e9a0 2024-04-25T08:43:39.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.423+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f030c0b42b0 con 0x7f030c06e9a0 2024-04-25T08:43:39.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.423+0000 7f0302000640 1 -- 172.21.15.46:0/714897752 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f02fc0ef420 con 0x7f030c06e9a0 2024-04-25T08:43:39.417 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":40,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:36.675414+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:43:32.873190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:43:34.064607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02ec066f90 msgr2=0x7f02ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02ec066f90 0x7f02ec069450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f030c1415e0 tx=0x7f02f8008040 comp rx=0 tx=0).stop 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c06e9a0 msgr2=0x7f030c157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c06e9a0 0x7f030c157520 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f02fc000c10 tx=0x7f02fc0192c0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 shutdown_connections 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f02ec066f90 0x7f02ec069450 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f030c161590 0x7f030c15ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f030c0cebb0 0x7f030c1455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 --2- 172.21.15.46:0/714897752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030c06e9a0 0x7f030c157520 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 >> 172.21.15.46:0/714897752 conn(0x7f030c0b12a0 msgr2=0x7f030c15ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 shutdown_connections 2024-04-25T08:43:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.427+0000 7f0311200640 1 -- 172.21.15.46:0/714897752 wait complete. 2024-04-25T08:43:39.439 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:43:39.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:43:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.587+0000 7fd401a00640 1 Processor -- start 2024-04-25T08:43:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.587+0000 7fd401a00640 1 -- start start 2024-04-25T08:43:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd401a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc15dbd0 0x7fd3fc0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd401a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3fc0b0c30 0x7fd3fc070c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd401a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3fc0b5250 con 0x7fd3fc0711d0 2024-04-25T08:43:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd401a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3fc0b5af0 con 0x7fd3fc0b0c30 2024-04-25T08:43:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd401a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0b5530 con 0x7fd3fc15dbd0 2024-04-25T08:43:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3fb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd3fc0711d0 0x7fd3fc15dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58324/0 (socket says 172.21.15.46:58324) 2024-04-25T08:43:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3fb400640 1 -- 172.21.15.46:0/966210433 learned_addr learned my addr 172.21.15.46:0/966210433 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3faa00640 1 --2- 172.21.15.46:0/966210433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc15dbd0 0x7fd3fc0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3fa000640 1 --2- 172.21.15.46:0/966210433 >> v2:172.21.15.82:3300/0 conn(0x7fd3fc0b0c30 0x7fd3fc070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3f9600640 1 -- 172.21.15.46:0/966210433 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094108018 0 0) 0x7fd3fc0b5250 con 0x7fd3fc0711d0 2024-04-25T08:43:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3f9600640 1 -- 172.21.15.46:0/966210433 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3e0003530 con 0x7fd3fc0711d0 2024-04-25T08:43:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3fa000640 1 -- 172.21.15.46:0/966210433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc15dbd0 msgr2=0x7fd3fc0b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3fa000640 1 --2- 172.21.15.46:0/966210433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc15dbd0 0x7fd3fc0b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3fa000640 1 -- 172.21.15.46:0/966210433 >> v1:172.21.15.46:6789/0 conn(0x7fd3fc0711d0 legacy=0x7fd3fc15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.591+0000 7fd3fa000640 1 -- 172.21.15.46:0/966210433 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc0b5810 con 0x7fd3fc0b0c30 2024-04-25T08:43:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd3fa000640 1 --2- 172.21.15.46:0/966210433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc0b0c30 0x7fd3fc070c90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80097c0 tx=0x7fd3e801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74ee736651fc8f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd3f9600640 1 -- 172.21.15.46:0/966210433 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80b0070 con 0x7fd3fc0b0c30 2024-04-25T08:43:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd3f9600640 1 -- 172.21.15.46:0/966210433 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e80ab600 con 0x7fd3fc0b0c30 2024-04-25T08:43:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd3f9600640 1 -- 172.21.15.46:0/966210433 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80ab9b0 con 0x7fd3fc0b0c30 2024-04-25T08:43:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 -- 172.21.15.46:0/966210433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc0b0c30 msgr2=0x7fd3fc070c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 --2- 172.21.15.46:0/966210433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc0b0c30 0x7fd3fc070c90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80097c0 tx=0x7fd3e801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 -- 172.21.15.46:0/966210433 shutdown_connections 2024-04-25T08:43:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 --2- 172.21.15.46:0/966210433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc0b0c30 0x7fd3fc070c90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 --2- 172.21.15.46:0/966210433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc15dbd0 0x7fd3fc0b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 -- 172.21.15.46:0/966210433 >> 172.21.15.46:0/966210433 conn(0x7fd3fc06f4f0 msgr2=0x7fd3fc06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 -- 172.21.15.46:0/966210433 shutdown_connections 2024-04-25T08:43:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 -- 172.21.15.46:0/966210433 wait complete. 2024-04-25T08:43:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 Processor -- start 2024-04-25T08:43:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 -- start start 2024-04-25T08:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3fc0711d0 0x7fd3fc160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc0b0c30 0x7fd3fc1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc15dbd0 0x7fd3fc14f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc16a850 con 0x7fd3fc0711d0 2024-04-25T08:43:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc0715e0 con 0x7fd3fc15dbd0 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd401a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3fc16a9d0 con 0x7fd3fc0b0c30 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd3faa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3fc0711d0 0x7fd3fc160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd3faa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3fc0711d0 0x7fd3fc160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46398/0 (socket says 172.21.15.46:46398) 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.595+0000 7fd3faa00640 1 -- 172.21.15.46:0/4250115690 learned_addr learned my addr 172.21.15.46:0/4250115690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3fa000640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc0b0c30 0x7fd3fc1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3fb400640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc15dbd0 0x7fd3fc14f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3faa00640 1 -- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc0b0c30 msgr2=0x7fd3fc1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3faa00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc0b0c30 0x7fd3fc1542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3faa00640 1 -- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc15dbd0 msgr2=0x7fd3fc14f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3faa00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc15dbd0 0x7fd3fc14f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3faa00640 1 -- 172.21.15.46:0/4250115690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3fc15f1f0 con 0x7fd3fc0711d0 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3fa000640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc0b0c30 0x7fd3fc1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3fb400640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc15dbd0 0x7fd3fc14f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3faa00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3fc0711d0 0x7fd3fc160290 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00e960 tx=0x7fd3ec00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3f2a00640 1 -- 172.21.15.46:0/4250115690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec017070 con 0x7fd3fc0711d0 2024-04-25T08:43:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3f2a00640 1 -- 172.21.15.46:0/4250115690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec00ca00 con 0x7fd3fc0711d0 2024-04-25T08:43:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3f2a00640 1 -- 172.21.15.46:0/4250115690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec00cdb0 con 0x7fd3fc0711d0 2024-04-25T08:43:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3fc15f4d0 con 0x7fd3fc0711d0 2024-04-25T08:43:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3fc15fa90 con 0x7fd3fc0711d0 2024-04-25T08:43:39.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3fc069080 con 0x7fd3fc0711d0 2024-04-25T08:43:39.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3f2a00640 1 -- 172.21.15.46:0/4250115690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3ec01a070 con 0x7fd3fc0711d0 2024-04-25T08:43:39.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3f2a00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3dc066f60 0x7fd3dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.599+0000 7fd3f2a00640 1 -- 172.21.15.46:0/4250115690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3ec085540 con 0x7fd3fc0711d0 2024-04-25T08:43:39.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.603+0000 7fd3fa000640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3dc066f60 0x7fd3dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.603+0000 7fd3fa000640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3dc066f60 0x7fd3dc069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80ad000 tx=0x7fd3e80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.603+0000 7fd3f2a00640 1 -- 172.21.15.46:0/4250115690 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3ec056450 con 0x7fd3fc0711d0 2024-04-25T08:43:39.634 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:43:39.643+0000 7f5f36800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:43:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.767+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd3fc06d830 con 0x7fd3dc066f60 2024-04-25T08:43:39.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.775+0000 7fd3f2a00640 1 -- 172.21.15.46:0/4250115690 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251934 (secure 0 0 0) 0x7fd3fc06d830 con 0x7fd3dc066f60 2024-04-25T08:43:39.770 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:39.771 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:43:39.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3dc066f60 msgr2=0x7fd3dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3dc066f60 0x7fd3dc069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80ad000 tx=0x7fd3e80a6000 comp rx=0 tx=0).stop 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3fc0711d0 msgr2=0x7fd3fc160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3fc0711d0 0x7fd3fc160290 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00e960 tx=0x7fd3ec00ee20 comp rx=0 tx=0).stop 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 shutdown_connections 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3dc066f60 0x7fd3dc069420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3fc15dbd0 0x7fd3fc14f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3fc0b0c30 0x7fd3fc1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 --2- 172.21.15.46:0/4250115690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3fc0711d0 0x7fd3fc160290 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 >> 172.21.15.46:0/4250115690 conn(0x7fd3fc06f4f0 msgr2=0x7fd3fc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 shutdown_connections 2024-04-25T08:43:39.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.779+0000 7fd401a00640 1 -- 172.21.15.46:0/4250115690 wait complete. 2024-04-25T08:43:39.791 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-04-25T08:43:38.272225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237584,"kb_used_data":4756,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737136,"statfs":{"total":773094113280,"available":772850827264,"internally_reserved":0,"allocated":4870144,"data_stored":2753400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.619001"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684394+0000","last_change":"2024-04-25T08:43:35.700911+0000","last_active":"2024-04-25T08:43:36.684394+0000","last_peered":"2024-04-25T08:43:36.684394+0000","last_clean":"2024-04-25T08:43:36.684394+0000","last_became_active":"2024-04-25T08:43:35.700661+0000","last_became_peered":"2024-04-25T08:43:35.700661+0000","last_unstale":"2024-04-25T08:43:36.684394+0000","last_undegraded":"2024-04-25T08:43:36.684394+0000","last_fullsized":"2024-04-25T08:43:36.684394+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:04.137354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684164+0000","last_change":"2024-04-25T08:43:35.689804+0000","last_active":"2024-04-25T08:43:36.684164+0000","last_peered":"2024-04-25T08:43:36.684164+0000","last_clean":"2024-04-25T08:43:36.684164+0000","last_became_active":"2024-04-25T08:43:35.689431+0000","last_became_peered":"2024-04-25T08:43:35.689431+0000","last_unstale":"2024-04-25T08:43:36.684164+0000","last_undegraded":"2024-04-25T08:43:36.684164+0000","last_fullsized":"2024-04-25T08:43:36.684164+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:45.015217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712542+0000","last_change":"2024-04-25T08:43:35.710788+0000","last_active":"2024-04-25T08:43:35.712542+0000","last_peered":"2024-04-25T08:43:35.712542+0000","last_clean":"2024-04-25T08:43:35.712542+0000","last_became_active":"2024-04-25T08:43:35.708973+0000","last_became_peered":"2024-04-25T08:43:35.708973+0000","last_unstale":"2024-04-25T08:43:35.712542+0000","last_undegraded":"2024-04-25T08:43:35.712542+0000","last_fullsized":"2024-04-25T08:43:35.712542+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:28.903230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683566+0000","last_change":"2024-04-25T08:43:35.702570+0000","last_active":"2024-04-25T08:43:36.683566+0000","last_peered":"2024-04-25T08:43:36.683566+0000","last_clean":"2024-04-25T08:43:36.683566+0000","last_became_active":"2024-04-25T08:43:35.701978+0000","last_became_peered":"2024-04-25T08:43:35.701978+0000","last_unstale":"2024-04-25T08:43:36.683566+0000","last_undegraded":"2024-04-25T08:43:36.683566+0000","last_fullsized":"2024-04-25T08:43:36.683566+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:27.282709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682910+0000","last_change":"2024-04-25T08:43:35.701716+0000","last_active":"2024-04-25T08:43:36.682910+0000","last_peered":"2024-04-25T08:43:36.682910+0000","last_clean":"2024-04-25T08:43:36.682910+0000","last_became_active":"2024-04-25T08:43:35.701421+0000","last_became_peered":"2024-04-25T08:43:35.701421+0000","last_unstale":"2024-04-25T08:43:36.682910+0000","last_undegraded":"2024-04-25T08:43:36.682910+0000","last_fullsized":"2024-04-25T08:43:36.682910+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:53.704185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820335+0000","last_change":"2024-04-25T08:43:35.693975+0000","last_active":"2024-04-25T08:43:36.820335+0000","last_peered":"2024-04-25T08:43:36.820335+0000","last_clean":"2024-04-25T08:43:36.820335+0000","last_became_active":"2024-04-25T08:43:35.693675+0000","last_became_peered":"2024-04-25T08:43:35.693675+0000","last_unstale":"2024-04-25T08:43:36.820335+0000","last_undegraded":"2024-04-25T08:43:36.820335+0000","last_fullsized":"2024-04-25T08:43:36.820335+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:18.095625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.709098+0000","last_change":"2024-04-25T08:43:35.700568+0000","last_active":"2024-04-25T08:43:35.709098+0000","last_peered":"2024-04-25T08:43:35.709098+0000","last_clean":"2024-04-25T08:43:35.709098+0000","last_became_active":"2024-04-25T08:43:35.697855+0000","last_became_peered":"2024-04-25T08:43:35.697855+0000","last_unstale":"2024-04-25T08:43:35.709098+0000","last_undegraded":"2024-04-25T08:43:35.709098+0000","last_fullsized":"2024-04-25T08:43:35.709098+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:43.958244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682438+0000","last_change":"2024-04-25T08:43:35.708245+0000","last_active":"2024-04-25T08:43:36.682438+0000","last_peered":"2024-04-25T08:43:36.682438+0000","last_clean":"2024-04-25T08:43:36.682438+0000","last_became_active":"2024-04-25T08:43:35.705002+0000","last_became_peered":"2024-04-25T08:43:35.705002+0000","last_unstale":"2024-04-25T08:43:36.682438+0000","last_undegraded":"2024-04-25T08:43:36.682438+0000","last_fullsized":"2024-04-25T08:43:36.682438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:47.203771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712600+0000","last_change":"2024-04-25T08:43:35.711391+0000","last_active":"2024-04-25T08:43:35.712600+0000","last_peered":"2024-04-25T08:43:35.712600+0000","last_clean":"2024-04-25T08:43:35.712600+0000","last_became_active":"2024-04-25T08:43:35.711121+0000","last_became_peered":"2024-04-25T08:43:35.711121+0000","last_unstale":"2024-04-25T08:43:35.712600+0000","last_undegraded":"2024-04-25T08:43:35.712600+0000","last_fullsized":"2024-04-25T08:43:35.712600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:41.662334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708803+0000","last_change":"2024-04-25T08:43:35.703259+0000","last_active":"2024-04-25T08:43:35.708803+0000","last_peered":"2024-04-25T08:43:35.708803+0000","last_clean":"2024-04-25T08:43:35.708803+0000","last_became_active":"2024-04-25T08:43:35.702959+0000","last_became_peered":"2024-04-25T08:43:35.702959+0000","last_unstale":"2024-04-25T08:43:35.708803+0000","last_undegraded":"2024-04-25T08:43:35.708803+0000","last_fullsized":"2024-04-25T08:43:35.708803+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.145256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.710417+0000","last_change":"2024-04-25T08:43:35.705545+0000","last_active":"2024-04-25T08:43:35.710417+0000","last_peered":"2024-04-25T08:43:35.710417+0000","last_clean":"2024-04-25T08:43:35.710417+0000","last_became_active":"2024-04-25T08:43:35.705052+0000","last_became_peered":"2024-04-25T08:43:35.705052+0000","last_unstale":"2024-04-25T08:43:35.710417+0000","last_undegraded":"2024-04-25T08:43:35.710417+0000","last_fullsized":"2024-04-25T08:43:35.710417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:54.574871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.710772+0000","last_change":"2024-04-25T08:43:35.697643+0000","last_active":"2024-04-25T08:43:35.710772+0000","last_peered":"2024-04-25T08:43:35.710772+0000","last_clean":"2024-04-25T08:43:35.710772+0000","last_became_active":"2024-04-25T08:43:35.697335+0000","last_became_peered":"2024-04-25T08:43:35.697335+0000","last_unstale":"2024-04-25T08:43:35.710772+0000","last_undegraded":"2024-04-25T08:43:35.710772+0000","last_fullsized":"2024-04-25T08:43:35.710772+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:26.000357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708863+0000","last_change":"2024-04-25T08:43:35.704208+0000","last_active":"2024-04-25T08:43:35.708863+0000","last_peered":"2024-04-25T08:43:35.708863+0000","last_clean":"2024-04-25T08:43:35.708863+0000","last_became_active":"2024-04-25T08:43:35.703869+0000","last_became_peered":"2024-04-25T08:43:35.703869+0000","last_unstale":"2024-04-25T08:43:35.708863+0000","last_undegraded":"2024-04-25T08:43:35.708863+0000","last_fullsized":"2024-04-25T08:43:35.708863+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:28.610365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683607+0000","last_change":"2024-04-25T08:43:35.697841+0000","last_active":"2024-04-25T08:43:36.683607+0000","last_peered":"2024-04-25T08:43:36.683607+0000","last_clean":"2024-04-25T08:43:36.683607+0000","last_became_active":"2024-04-25T08:43:35.697533+0000","last_became_peered":"2024-04-25T08:43:35.697533+0000","last_unstale":"2024-04-25T08:43:36.683607+0000","last_undegraded":"2024-04-25T08:43:36.683607+0000","last_fullsized":"2024-04-25T08:43:36.683607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:32.121504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.709115+0000","last_change":"2024-04-25T08:43:35.701080+0000","last_active":"2024-04-25T08:43:35.709115+0000","last_peered":"2024-04-25T08:43:35.709115+0000","last_clean":"2024-04-25T08:43:35.709115+0000","last_became_active":"2024-04-25T08:43:35.700652+0000","last_became_peered":"2024-04-25T08:43:35.700652+0000","last_unstale":"2024-04-25T08:43:35.709115+0000","last_undegraded":"2024-04-25T08:43:35.709115+0000","last_fullsized":"2024-04-25T08:43:35.709115+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:18.095625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683265+0000","last_change":"2024-04-25T08:43:35.705915+0000","last_active":"2024-04-25T08:43:36.683265+0000","last_peered":"2024-04-25T08:43:36.683265+0000","last_clean":"2024-04-25T08:43:36.683265+0000","last_became_active":"2024-04-25T08:43:35.705653+0000","last_became_peered":"2024-04-25T08:43:35.705653+0000","last_unstale":"2024-04-25T08:43:36.683265+0000","last_undegraded":"2024-04-25T08:43:36.683265+0000","last_fullsized":"2024-04-25T08:43:36.683265+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:20.851391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683374+0000","last_change":"2024-04-25T08:43:35.698575+0000","last_active":"2024-04-25T08:43:36.683374+0000","last_peered":"2024-04-25T08:43:36.683374+0000","last_clean":"2024-04-25T08:43:36.683374+0000","last_became_active":"2024-04-25T08:43:35.698303+0000","last_became_peered":"2024-04-25T08:43:35.698303+0000","last_unstale":"2024-04-25T08:43:36.683374+0000","last_undegraded":"2024-04-25T08:43:36.683374+0000","last_fullsized":"2024-04-25T08:43:36.683374+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:06.474434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.710059+0000","last_change":"2024-04-25T08:43:35.683195+0000","last_active":"2024-04-25T08:43:36.710059+0000","last_peered":"2024-04-25T08:43:36.710059+0000","last_clean":"2024-04-25T08:43:36.710059+0000","last_became_active":"2024-04-25T08:43:35.682917+0000","last_became_peered":"2024-04-25T08:43:35.682917+0000","last_unstale":"2024-04-25T08:43:36.710059+0000","last_undegraded":"2024-04-25T08:43:36.710059+0000","last_fullsized":"2024-04-25T08:43:36.710059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:15.426531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683779+0000","last_change":"2024-04-25T08:43:35.686127+0000","last_active":"2024-04-25T08:43:36.683779+0000","last_peered":"2024-04-25T08:43:36.683779+0000","last_clean":"2024-04-25T08:43:36.683779+0000","last_became_active":"2024-04-25T08:43:35.685838+0000","last_became_peered":"2024-04-25T08:43:35.685838+0000","last_unstale":"2024-04-25T08:43:36.683779+0000","last_undegraded":"2024-04-25T08:43:36.683779+0000","last_fullsized":"2024-04-25T08:43:36.683779+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709938+0000","last_change":"2024-04-25T08:43:35.686152+0000","last_active":"2024-04-25T08:43:36.709938+0000","last_peered":"2024-04-25T08:43:36.709938+0000","last_clean":"2024-04-25T08:43:36.709938+0000","last_became_active":"2024-04-25T08:43:35.683503+0000","last_became_peered":"2024-04-25T08:43:35.683503+0000","last_unstale":"2024-04-25T08:43:36.709938+0000","last_undegraded":"2024-04-25T08:43:36.709938+0000","last_fullsized":"2024-04-25T08:43:36.709938+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:00.202978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708037+0000","last_change":"2024-04-25T08:43:35.699208+0000","last_active":"2024-04-25T08:43:35.708037+0000","last_peered":"2024-04-25T08:43:35.708037+0000","last_clean":"2024-04-25T08:43:35.708037+0000","last_became_active":"2024-04-25T08:43:35.698926+0000","last_became_peered":"2024-04-25T08:43:35.698926+0000","last_unstale":"2024-04-25T08:43:35.708037+0000","last_undegraded":"2024-04-25T08:43:35.708037+0000","last_fullsized":"2024-04-25T08:43:35.708037+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:41.662334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684489+0000","last_change":"2024-04-25T08:43:35.692630+0000","last_active":"2024-04-25T08:43:36.684489+0000","last_peered":"2024-04-25T08:43:36.684489+0000","last_clean":"2024-04-25T08:43:36.684489+0000","last_became_active":"2024-04-25T08:43:35.692311+0000","last_became_peered":"2024-04-25T08:43:35.692311+0000","last_unstale":"2024-04-25T08:43:36.684489+0000","last_undegraded":"2024-04-25T08:43:36.684489+0000","last_fullsized":"2024-04-25T08:43:36.684489+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:20.851391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683388+0000","last_change":"2024-04-25T08:43:35.695519+0000","last_active":"2024-04-25T08:43:36.683388+0000","last_peered":"2024-04-25T08:43:36.683388+0000","last_clean":"2024-04-25T08:43:36.683388+0000","last_became_active":"2024-04-25T08:43:35.694946+0000","last_became_peered":"2024-04-25T08:43:35.694946+0000","last_unstale":"2024-04-25T08:43:36.683388+0000","last_undegraded":"2024-04-25T08:43:36.683388+0000","last_fullsized":"2024-04-25T08:43:36.683388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:12.608221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682914+0000","last_change":"2024-04-25T08:43:35.700634+0000","last_active":"2024-04-25T08:43:36.682914+0000","last_peered":"2024-04-25T08:43:36.682914+0000","last_clean":"2024-04-25T08:43:36.682914+0000","last_became_active":"2024-04-25T08:43:35.700263+0000","last_became_peered":"2024-04-25T08:43:35.700263+0000","last_unstale":"2024-04-25T08:43:36.682914+0000","last_undegraded":"2024-04-25T08:43:36.682914+0000","last_fullsized":"2024-04-25T08:43:36.682914+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:02.189663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.710052+0000","last_change":"2024-04-25T08:43:35.698882+0000","last_active":"2024-04-25T08:43:36.710052+0000","last_peered":"2024-04-25T08:43:36.710052+0000","last_clean":"2024-04-25T08:43:36.710052+0000","last_became_active":"2024-04-25T08:43:35.698469+0000","last_became_peered":"2024-04-25T08:43:35.698469+0000","last_unstale":"2024-04-25T08:43:36.710052+0000","last_undegraded":"2024-04-25T08:43:36.710052+0000","last_fullsized":"2024-04-25T08:43:36.710052+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.357301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820613+0000","last_change":"2024-04-25T08:43:35.706450+0000","last_active":"2024-04-25T08:43:36.820613+0000","last_peered":"2024-04-25T08:43:36.820613+0000","last_clean":"2024-04-25T08:43:36.820613+0000","last_became_active":"2024-04-25T08:43:35.706141+0000","last_became_peered":"2024-04-25T08:43:35.706141+0000","last_unstale":"2024-04-25T08:43:36.820613+0000","last_undegraded":"2024-04-25T08:43:36.820613+0000","last_fullsized":"2024-04-25T08:43:36.820613+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:06.474434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709991+0000","last_change":"2024-04-25T08:43:35.687674+0000","last_active":"2024-04-25T08:43:36.709991+0000","last_peered":"2024-04-25T08:43:36.709991+0000","last_clean":"2024-04-25T08:43:36.709991+0000","last_became_active":"2024-04-25T08:43:35.687374+0000","last_became_peered":"2024-04-25T08:43:35.687374+0000","last_unstale":"2024-04-25T08:43:36.709991+0000","last_undegraded":"2024-04-25T08:43:36.709991+0000","last_fullsized":"2024-04-25T08:43:36.709991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:28.903230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708080+0000","last_change":"2024-04-25T08:43:35.700376+0000","last_active":"2024-04-25T08:43:35.708080+0000","last_peered":"2024-04-25T08:43:35.708080+0000","last_clean":"2024-04-25T08:43:35.708080+0000","last_became_active":"2024-04-25T08:43:35.700115+0000","last_became_peered":"2024-04-25T08:43:35.700115+0000","last_unstale":"2024-04-25T08:43:35.708080+0000","last_undegraded":"2024-04-25T08:43:35.708080+0000","last_fullsized":"2024-04-25T08:43:35.708080+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:32.121504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709857+0000","last_change":"2024-04-25T08:43:35.691080+0000","last_active":"2024-04-25T08:43:36.709857+0000","last_peered":"2024-04-25T08:43:36.709857+0000","last_clean":"2024-04-25T08:43:36.709857+0000","last_became_active":"2024-04-25T08:43:35.690764+0000","last_became_peered":"2024-04-25T08:43:35.690764+0000","last_unstale":"2024-04-25T08:43:36.709857+0000","last_undegraded":"2024-04-25T08:43:36.709857+0000","last_fullsized":"2024-04-25T08:43:36.709857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:10.828438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709756+0000","last_change":"2024-04-25T08:43:35.678455+0000","last_active":"2024-04-25T08:43:36.709756+0000","last_peered":"2024-04-25T08:43:36.709756+0000","last_clean":"2024-04-25T08:43:36.709756+0000","last_became_active":"2024-04-25T08:43:35.677876+0000","last_became_peered":"2024-04-25T08:43:35.677876+0000","last_unstale":"2024-04-25T08:43:36.709756+0000","last_undegraded":"2024-04-25T08:43:36.709756+0000","last_fullsized":"2024-04-25T08:43:36.709756+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:50.574954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683646+0000","last_change":"2024-04-25T08:43:35.695297+0000","last_active":"2024-04-25T08:43:36.683646+0000","last_peered":"2024-04-25T08:43:36.683646+0000","last_clean":"2024-04-25T08:43:36.683646+0000","last_became_active":"2024-04-25T08:43:35.694973+0000","last_became_peered":"2024-04-25T08:43:35.694973+0000","last_unstale":"2024-04-25T08:43:36.683646+0000","last_undegraded":"2024-04-25T08:43:36.683646+0000","last_fullsized":"2024-04-25T08:43:36.683646+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:28.610365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820257+0000","last_change":"2024-04-25T08:43:35.686285+0000","last_active":"2024-04-25T08:43:36.820257+0000","last_peered":"2024-04-25T08:43:36.820257+0000","last_clean":"2024-04-25T08:43:36.820257+0000","last_became_active":"2024-04-25T08:43:35.685962+0000","last_became_peered":"2024-04-25T08:43:35.685962+0000","last_unstale":"2024-04-25T08:43:36.820257+0000","last_undegraded":"2024-04-25T08:43:36.820257+0000","last_fullsized":"2024-04-25T08:43:36.820257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:32.121504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683439+0000","last_change":"2024-04-25T08:43:35.708776+0000","last_active":"2024-04-25T08:43:36.683439+0000","last_peered":"2024-04-25T08:43:36.683439+0000","last_clean":"2024-04-25T08:43:36.683439+0000","last_became_active":"2024-04-25T08:43:35.708465+0000","last_became_peered":"2024-04-25T08:43:35.708465+0000","last_unstale":"2024-04-25T08:43:36.683439+0000","last_undegraded":"2024-04-25T08:43:36.683439+0000","last_fullsized":"2024-04-25T08:43:36.683439+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.145256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.710116+0000","last_change":"2024-04-25T08:43:34.676582+0000","last_active":"2024-04-25T08:43:36.710116+0000","last_peered":"2024-04-25T08:43:36.710116+0000","last_clean":"2024-04-25T08:43:36.710116+0000","last_became_active":"2024-04-25T08:43:34.676311+0000","last_became_peered":"2024-04-25T08:43:34.676311+0000","last_unstale":"2024-04-25T08:43:36.710116+0000","last_undegraded":"2024-04-25T08:43:36.710116+0000","last_fullsized":"2024-04-25T08:43:36.710116+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:12.964771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820628+0000","last_change":"2024-04-25T08:43:35.703304+0000","last_active":"2024-04-25T08:43:36.820628+0000","last_peered":"2024-04-25T08:43:36.820628+0000","last_clean":"2024-04-25T08:43:36.820628+0000","last_became_active":"2024-04-25T08:43:35.702930+0000","last_became_peered":"2024-04-25T08:43:35.702930+0000","last_unstale":"2024-04-25T08:43:36.820628+0000","last_undegraded":"2024-04-25T08:43:36.820628+0000","last_fullsized":"2024-04-25T08:43:36.820628+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:12.608221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.706742+0000","last_change":"2024-04-25T08:43:34.692652+0000","last_active":"2024-04-25T08:43:35.706742+0000","last_peered":"2024-04-25T08:43:35.706742+0000","last_clean":"2024-04-25T08:43:35.706742+0000","last_became_active":"2024-04-25T08:43:34.692360+0000","last_became_peered":"2024-04-25T08:43:34.692360+0000","last_unstale":"2024-04-25T08:43:35.706742+0000","last_undegraded":"2024-04-25T08:43:35.706742+0000","last_fullsized":"2024-04-25T08:43:35.706742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:36.494719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684995+0000","last_change":"2024-04-25T08:43:35.683970+0000","last_active":"2024-04-25T08:43:36.684995+0000","last_peered":"2024-04-25T08:43:36.684995+0000","last_clean":"2024-04-25T08:43:36.684995+0000","last_became_active":"2024-04-25T08:43:35.683684+0000","last_became_peered":"2024-04-25T08:43:35.683684+0000","last_unstale":"2024-04-25T08:43:36.684995+0000","last_undegraded":"2024-04-25T08:43:36.684995+0000","last_fullsized":"2024-04-25T08:43:36.684995+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.145256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820667+0000","last_change":"2024-04-25T08:43:34.676639+0000","last_active":"2024-04-25T08:43:36.820667+0000","last_peered":"2024-04-25T08:43:36.820667+0000","last_clean":"2024-04-25T08:43:36.820667+0000","last_became_active":"2024-04-25T08:43:34.676341+0000","last_became_peered":"2024-04-25T08:43:34.676341+0000","last_unstale":"2024-04-25T08:43:36.820667+0000","last_undegraded":"2024-04-25T08:43:36.820667+0000","last_fullsized":"2024-04-25T08:43:36.820667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:51.225447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684342+0000","last_change":"2024-04-25T08:43:35.689654+0000","last_active":"2024-04-25T08:43:36.684342+0000","last_peered":"2024-04-25T08:43:36.684342+0000","last_clean":"2024-04-25T08:43:36.684342+0000","last_became_active":"2024-04-25T08:43:35.688996+0000","last_became_peered":"2024-04-25T08:43:35.688996+0000","last_unstale":"2024-04-25T08:43:36.684342+0000","last_undegraded":"2024-04-25T08:43:36.684342+0000","last_fullsized":"2024-04-25T08:43:36.684342+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:45.102390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684616+0000","last_change":"2024-04-25T08:43:34.688559+0000","last_active":"2024-04-25T08:43:36.684616+0000","last_peered":"2024-04-25T08:43:36.684616+0000","last_clean":"2024-04-25T08:43:36.684616+0000","last_became_active":"2024-04-25T08:43:34.688187+0000","last_became_peered":"2024-04-25T08:43:34.688187+0000","last_unstale":"2024-04-25T08:43:36.684616+0000","last_undegraded":"2024-04-25T08:43:36.684616+0000","last_fullsized":"2024-04-25T08:43:36.684616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:36.494719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712219+0000","last_change":"2024-04-25T08:43:35.708325+0000","last_active":"2024-04-25T08:43:35.712219+0000","last_peered":"2024-04-25T08:43:35.712219+0000","last_clean":"2024-04-25T08:43:35.712219+0000","last_became_active":"2024-04-25T08:43:35.707610+0000","last_became_peered":"2024-04-25T08:43:35.707610+0000","last_unstale":"2024-04-25T08:43:35.712219+0000","last_undegraded":"2024-04-25T08:43:35.712219+0000","last_fullsized":"2024-04-25T08:43:35.712219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:00.202978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.704053+0000","last_change":"2024-04-25T08:43:34.685574+0000","last_active":"2024-04-25T08:43:35.704053+0000","last_peered":"2024-04-25T08:43:35.704053+0000","last_clean":"2024-04-25T08:43:35.704053+0000","last_became_active":"2024-04-25T08:43:34.685241+0000","last_became_peered":"2024-04-25T08:43:34.685241+0000","last_unstale":"2024-04-25T08:43:35.704053+0000","last_undegraded":"2024-04-25T08:43:35.704053+0000","last_fullsized":"2024-04-25T08:43:35.704053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.567248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.707999+0000","last_change":"2024-04-25T08:43:35.697541+0000","last_active":"2024-04-25T08:43:35.707999+0000","last_peered":"2024-04-25T08:43:35.707999+0000","last_clean":"2024-04-25T08:43:35.707999+0000","last_became_active":"2024-04-25T08:43:35.696604+0000","last_became_peered":"2024-04-25T08:43:35.696604+0000","last_unstale":"2024-04-25T08:43:35.707999+0000","last_undegraded":"2024-04-25T08:43:35.707999+0000","last_fullsized":"2024-04-25T08:43:35.707999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:15.426531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683261+0000","last_change":"2024-04-25T08:43:34.685281+0000","last_active":"2024-04-25T08:43:36.683261+0000","last_peered":"2024-04-25T08:43:36.683261+0000","last_clean":"2024-04-25T08:43:36.683261+0000","last_became_active":"2024-04-25T08:43:34.685012+0000","last_became_peered":"2024-04-25T08:43:34.685012+0000","last_unstale":"2024-04-25T08:43:36.683261+0000","last_undegraded":"2024-04-25T08:43:36.683261+0000","last_fullsized":"2024-04-25T08:43:36.683261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:51.225447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684298+0000","last_change":"2024-04-25T08:43:35.703989+0000","last_active":"2024-04-25T08:43:36.684298+0000","last_peered":"2024-04-25T08:43:36.684298+0000","last_clean":"2024-04-25T08:43:36.684298+0000","last_became_active":"2024-04-25T08:43:35.703719+0000","last_became_peered":"2024-04-25T08:43:35.703719+0000","last_unstale":"2024-04-25T08:43:36.684298+0000","last_undegraded":"2024-04-25T08:43:36.684298+0000","last_fullsized":"2024-04-25T08:43:36.684298+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:18.095625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820335+0000","last_change":"2024-04-25T08:43:34.685569+0000","last_active":"2024-04-25T08:43:36.820335+0000","last_peered":"2024-04-25T08:43:36.820335+0000","last_clean":"2024-04-25T08:43:36.820335+0000","last_became_active":"2024-04-25T08:43:34.685258+0000","last_became_peered":"2024-04-25T08:43:34.685258+0000","last_unstale":"2024-04-25T08:43:36.820335+0000","last_undegraded":"2024-04-25T08:43:36.820335+0000","last_fullsized":"2024-04-25T08:43:36.820335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:36.494719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683350+0000","last_change":"2024-04-25T08:43:35.695143+0000","last_active":"2024-04-25T08:43:36.683350+0000","last_peered":"2024-04-25T08:43:36.683350+0000","last_clean":"2024-04-25T08:43:36.683350+0000","last_became_active":"2024-04-25T08:43:35.694868+0000","last_became_peered":"2024-04-25T08:43:35.694868+0000","last_unstale":"2024-04-25T08:43:36.683350+0000","last_undegraded":"2024-04-25T08:43:36.683350+0000","last_fullsized":"2024-04-25T08:43:36.683350+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:44.020550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682987+0000","last_change":"2024-04-25T08:43:34.677461+0000","last_active":"2024-04-25T08:43:36.682987+0000","last_peered":"2024-04-25T08:43:36.682987+0000","last_clean":"2024-04-25T08:43:36.682987+0000","last_became_active":"2024-04-25T08:43:34.677148+0000","last_became_peered":"2024-04-25T08:43:34.677148+0000","last_unstale":"2024-04-25T08:43:36.682987+0000","last_undegraded":"2024-04-25T08:43:36.682987+0000","last_fullsized":"2024-04-25T08:43:36.682987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.992651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683703+0000","last_change":"2024-04-25T08:43:35.702343+0000","last_active":"2024-04-25T08:43:36.683703+0000","last_peered":"2024-04-25T08:43:36.683703+0000","last_clean":"2024-04-25T08:43:36.683703+0000","last_became_active":"2024-04-25T08:43:35.701631+0000","last_became_peered":"2024-04-25T08:43:35.701631+0000","last_unstale":"2024-04-25T08:43:36.683703+0000","last_undegraded":"2024-04-25T08:43:36.683703+0000","last_fullsized":"2024-04-25T08:43:36.683703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:07.955819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684397+0000","last_change":"2024-04-25T08:43:34.682629+0000","last_active":"2024-04-25T08:43:36.684397+0000","last_peered":"2024-04-25T08:43:36.684397+0000","last_clean":"2024-04-25T08:43:36.684397+0000","last_became_active":"2024-04-25T08:43:34.682289+0000","last_became_peered":"2024-04-25T08:43:34.682289+0000","last_unstale":"2024-04-25T08:43:36.684397+0000","last_undegraded":"2024-04-25T08:43:36.684397+0000","last_fullsized":"2024-04-25T08:43:36.684397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:40.654628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683671+0000","last_change":"2024-04-25T08:43:35.695103+0000","last_active":"2024-04-25T08:43:36.683671+0000","last_peered":"2024-04-25T08:43:36.683671+0000","last_clean":"2024-04-25T08:43:36.683671+0000","last_became_active":"2024-04-25T08:43:35.694309+0000","last_became_peered":"2024-04-25T08:43:35.694309+0000","last_unstale":"2024-04-25T08:43:36.683671+0000","last_undegraded":"2024-04-25T08:43:36.683671+0000","last_fullsized":"2024-04-25T08:43:36.683671+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:54.574871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.686693+0000","last_change":"2024-04-25T08:43:34.690955+0000","last_active":"2024-04-25T08:43:35.686693+0000","last_peered":"2024-04-25T08:43:35.686693+0000","last_clean":"2024-04-25T08:43:35.686693+0000","last_became_active":"2024-04-25T08:43:34.690678+0000","last_became_peered":"2024-04-25T08:43:34.690678+0000","last_unstale":"2024-04-25T08:43:35.686693+0000","last_undegraded":"2024-04-25T08:43:35.686693+0000","last_fullsized":"2024-04-25T08:43:35.686693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:27.895523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.711204+0000","last_change":"2024-04-25T08:43:35.707226+0000","last_active":"2024-04-25T08:43:35.711204+0000","last_peered":"2024-04-25T08:43:35.711204+0000","last_clean":"2024-04-25T08:43:35.711204+0000","last_became_active":"2024-04-25T08:43:35.706881+0000","last_became_peered":"2024-04-25T08:43:35.706881+0000","last_unstale":"2024-04-25T08:43:35.711204+0000","last_undegraded":"2024-04-25T08:43:35.711204+0000","last_fullsized":"2024-04-25T08:43:35.711204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:10.828438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.707692+0000","last_change":"2024-04-25T08:43:34.690935+0000","last_active":"2024-04-25T08:43:35.707692+0000","last_peered":"2024-04-25T08:43:35.707692+0000","last_clean":"2024-04-25T08:43:35.707692+0000","last_became_active":"2024-04-25T08:43:34.690284+0000","last_became_peered":"2024-04-25T08:43:34.690284+0000","last_unstale":"2024-04-25T08:43:35.707692+0000","last_undegraded":"2024-04-25T08:43:35.707692+0000","last_fullsized":"2024-04-25T08:43:35.707692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:09.820732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684827+0000","last_change":"2024-04-25T08:43:35.691933+0000","last_active":"2024-04-25T08:43:36.684827+0000","last_peered":"2024-04-25T08:43:36.684827+0000","last_clean":"2024-04-25T08:43:36.684827+0000","last_became_active":"2024-04-25T08:43:35.691645+0000","last_became_peered":"2024-04-25T08:43:35.691645+0000","last_unstale":"2024-04-25T08:43:36.684827+0000","last_undegraded":"2024-04-25T08:43:36.684827+0000","last_fullsized":"2024-04-25T08:43:36.684827+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:12.608221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683954+0000","last_change":"2024-04-25T08:43:34.681753+0000","last_active":"2024-04-25T08:43:36.683954+0000","last_peered":"2024-04-25T08:43:36.683954+0000","last_clean":"2024-04-25T08:43:36.683954+0000","last_became_active":"2024-04-25T08:43:34.680960+0000","last_became_peered":"2024-04-25T08:43:34.680960+0000","last_unstale":"2024-04-25T08:43:36.683954+0000","last_undegraded":"2024-04-25T08:43:36.683954+0000","last_fullsized":"2024-04-25T08:43:36.683954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:14.418825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712634+0000","last_change":"2024-04-25T08:43:35.711374+0000","last_active":"2024-04-25T08:43:35.712634+0000","last_peered":"2024-04-25T08:43:35.712634+0000","last_clean":"2024-04-25T08:43:35.712634+0000","last_became_active":"2024-04-25T08:43:35.711083+0000","last_became_peered":"2024-04-25T08:43:35.711083+0000","last_unstale":"2024-04-25T08:43:35.712634+0000","last_undegraded":"2024-04-25T08:43:35.712634+0000","last_fullsized":"2024-04-25T08:43:35.712634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:15.426531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683625+0000","last_change":"2024-04-25T08:43:34.686447+0000","last_active":"2024-04-25T08:43:36.683625+0000","last_peered":"2024-04-25T08:43:36.683625+0000","last_clean":"2024-04-25T08:43:36.683625+0000","last_became_active":"2024-04-25T08:43:34.685801+0000","last_became_peered":"2024-04-25T08:43:34.685801+0000","last_unstale":"2024-04-25T08:43:36.683625+0000","last_undegraded":"2024-04-25T08:43:36.683625+0000","last_fullsized":"2024-04-25T08:43:36.683625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:06.349595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682638+0000","last_change":"2024-04-25T08:43:35.703698+0000","last_active":"2024-04-25T08:43:36.682638+0000","last_peered":"2024-04-25T08:43:36.682638+0000","last_clean":"2024-04-25T08:43:36.682638+0000","last_became_active":"2024-04-25T08:43:35.703402+0000","last_became_peered":"2024-04-25T08:43:35.703402+0000","last_unstale":"2024-04-25T08:43:36.682638+0000","last_undegraded":"2024-04-25T08:43:36.682638+0000","last_fullsized":"2024-04-25T08:43:36.682638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:42.497662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820272+0000","last_change":"2024-04-25T08:43:34.681941+0000","last_active":"2024-04-25T08:43:36.820272+0000","last_peered":"2024-04-25T08:43:36.820272+0000","last_clean":"2024-04-25T08:43:36.820272+0000","last_became_active":"2024-04-25T08:43:34.681666+0000","last_became_peered":"2024-04-25T08:43:34.681666+0000","last_unstale":"2024-04-25T08:43:36.820272+0000","last_undegraded":"2024-04-25T08:43:36.820272+0000","last_fullsized":"2024-04-25T08:43:36.820272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.992651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683964+0000","last_change":"2024-04-25T08:43:35.693452+0000","last_active":"2024-04-25T08:43:36.683964+0000","last_peered":"2024-04-25T08:43:36.683964+0000","last_clean":"2024-04-25T08:43:36.683964+0000","last_became_active":"2024-04-25T08:43:35.693105+0000","last_became_peered":"2024-04-25T08:43:35.693105+0000","last_unstale":"2024-04-25T08:43:36.683964+0000","last_undegraded":"2024-04-25T08:43:36.683964+0000","last_fullsized":"2024-04-25T08:43:36.683964+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:42.497662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709680+0000","last_change":"2024-04-25T08:43:34.676749+0000","last_active":"2024-04-25T08:43:36.709680+0000","last_peered":"2024-04-25T08:43:36.709680+0000","last_clean":"2024-04-25T08:43:36.709680+0000","last_became_active":"2024-04-25T08:43:34.676193+0000","last_became_peered":"2024-04-25T08:43:34.676193+0000","last_unstale":"2024-04-25T08:43:36.709680+0000","last_undegraded":"2024-04-25T08:43:36.709680+0000","last_fullsized":"2024-04-25T08:43:36.709680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:30.462862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683345+0000","last_change":"2024-04-25T08:43:35.702991+0000","last_active":"2024-04-25T08:43:36.683345+0000","last_peered":"2024-04-25T08:43:36.683345+0000","last_clean":"2024-04-25T08:43:36.683345+0000","last_became_active":"2024-04-25T08:43:35.702728+0000","last_became_peered":"2024-04-25T08:43:35.702728+0000","last_unstale":"2024-04-25T08:43:36.683345+0000","last_undegraded":"2024-04-25T08:43:36.683345+0000","last_fullsized":"2024-04-25T08:43:36.683345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:30.859525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683386+0000","last_change":"2024-04-25T08:43:34.686860+0000","last_active":"2024-04-25T08:43:36.683386+0000","last_peered":"2024-04-25T08:43:36.683386+0000","last_clean":"2024-04-25T08:43:36.683386+0000","last_became_active":"2024-04-25T08:43:34.686567+0000","last_became_peered":"2024-04-25T08:43:34.686567+0000","last_unstale":"2024-04-25T08:43:36.683386+0000","last_undegraded":"2024-04-25T08:43:36.683386+0000","last_fullsized":"2024-04-25T08:43:36.683386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:52.768221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683855+0000","last_change":"2024-04-25T08:43:35.675041+0000","last_active":"2024-04-25T08:43:36.683855+0000","last_peered":"2024-04-25T08:43:36.683855+0000","last_clean":"2024-04-25T08:43:36.683855+0000","last_became_active":"2024-04-25T08:43:35.674671+0000","last_became_peered":"2024-04-25T08:43:35.674671+0000","last_unstale":"2024-04-25T08:43:36.683855+0000","last_undegraded":"2024-04-25T08:43:36.683855+0000","last_fullsized":"2024-04-25T08:43:36.683855+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:10.178193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708650+0000","last_change":"2024-04-25T08:43:34.684965+0000","last_active":"2024-04-25T08:43:35.708650+0000","last_peered":"2024-04-25T08:43:35.708650+0000","last_clean":"2024-04-25T08:43:35.708650+0000","last_became_active":"2024-04-25T08:43:34.684707+0000","last_became_peered":"2024-04-25T08:43:34.684707+0000","last_unstale":"2024-04-25T08:43:35.708650+0000","last_undegraded":"2024-04-25T08:43:35.708650+0000","last_fullsized":"2024-04-25T08:43:35.708650+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:59.195272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708602+0000","last_change":"2024-04-25T08:43:35.701347+0000","last_active":"2024-04-25T08:43:35.708602+0000","last_peered":"2024-04-25T08:43:35.708602+0000","last_clean":"2024-04-25T08:43:35.708602+0000","last_became_active":"2024-04-25T08:43:35.701051+0000","last_became_peered":"2024-04-25T08:43:35.701051+0000","last_unstale":"2024-04-25T08:43:35.708602+0000","last_undegraded":"2024-04-25T08:43:35.708602+0000","last_fullsized":"2024-04-25T08:43:35.708602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:20.851391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684899+0000","last_change":"2024-04-25T08:43:34.685675+0000","last_active":"2024-04-25T08:43:36.684899+0000","last_peered":"2024-04-25T08:43:36.684899+0000","last_clean":"2024-04-25T08:43:36.684899+0000","last_became_active":"2024-04-25T08:43:34.684712+0000","last_became_peered":"2024-04-25T08:43:34.684712+0000","last_unstale":"2024-04-25T08:43:36.684899+0000","last_undegraded":"2024-04-25T08:43:36.684899+0000","last_fullsized":"2024-04-25T08:43:36.684899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:51.225447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684813+0000","last_change":"2024-04-25T08:43:35.689495+0000","last_active":"2024-04-25T08:43:36.684813+0000","last_peered":"2024-04-25T08:43:36.684813+0000","last_clean":"2024-04-25T08:43:36.684813+0000","last_became_active":"2024-04-25T08:43:35.688991+0000","last_became_peered":"2024-04-25T08:43:35.688991+0000","last_unstale":"2024-04-25T08:43:36.684813+0000","last_undegraded":"2024-04-25T08:43:36.684813+0000","last_fullsized":"2024-04-25T08:43:36.684813+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:06.474434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683831+0000","last_change":"2024-04-25T08:43:34.691860+0000","last_active":"2024-04-25T08:43:36.683831+0000","last_peered":"2024-04-25T08:43:36.683831+0000","last_clean":"2024-04-25T08:43:36.683831+0000","last_became_active":"2024-04-25T08:43:34.691554+0000","last_became_peered":"2024-04-25T08:43:34.691554+0000","last_unstale":"2024-04-25T08:43:36.683831+0000","last_undegraded":"2024-04-25T08:43:36.683831+0000","last_fullsized":"2024-04-25T08:43:36.683831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:02.177817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683992+0000","last_change":"2024-04-25T08:43:35.686878+0000","last_active":"2024-04-25T08:43:36.683992+0000","last_peered":"2024-04-25T08:43:36.683992+0000","last_clean":"2024-04-25T08:43:36.683992+0000","last_became_active":"2024-04-25T08:43:35.686621+0000","last_became_peered":"2024-04-25T08:43:35.686621+0000","last_unstale":"2024-04-25T08:43:36.683992+0000","last_undegraded":"2024-04-25T08:43:36.683992+0000","last_fullsized":"2024-04-25T08:43:36.683992+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:31.470568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820452+0000","last_change":"2024-04-25T08:43:34.677144+0000","last_active":"2024-04-25T08:43:36.820452+0000","last_peered":"2024-04-25T08:43:36.820452+0000","last_clean":"2024-04-25T08:43:36.820452+0000","last_became_active":"2024-04-25T08:43:34.676233+0000","last_became_peered":"2024-04-25T08:43:34.676233+0000","last_unstale":"2024-04-25T08:43:36.820452+0000","last_undegraded":"2024-04-25T08:43:36.820452+0000","last_fullsized":"2024-04-25T08:43:36.820452+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:52.768221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683743+0000","last_change":"2024-04-25T08:43:35.698244+0000","last_active":"2024-04-25T08:43:36.683743+0000","last_peered":"2024-04-25T08:43:36.683743+0000","last_clean":"2024-04-25T08:43:36.683743+0000","last_became_active":"2024-04-25T08:43:35.697920+0000","last_became_peered":"2024-04-25T08:43:35.697920+0000","last_unstale":"2024-04-25T08:43:36.683743+0000","last_undegraded":"2024-04-25T08:43:36.683743+0000","last_fullsized":"2024-04-25T08:43:36.683743+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:09.195652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683021+0000","last_change":"2024-04-25T08:43:34.691363+0000","last_active":"2024-04-25T08:43:36.683021+0000","last_peered":"2024-04-25T08:43:36.683021+0000","last_clean":"2024-04-25T08:43:36.683021+0000","last_became_active":"2024-04-25T08:43:34.691032+0000","last_became_peered":"2024-04-25T08:43:34.691032+0000","last_unstale":"2024-04-25T08:43:36.683021+0000","last_undegraded":"2024-04-25T08:43:36.683021+0000","last_fullsized":"2024-04-25T08:43:36.683021+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:58.410307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712605+0000","last_change":"2024-04-25T08:43:35.710875+0000","last_active":"2024-04-25T08:43:35.712605+0000","last_peered":"2024-04-25T08:43:35.712605+0000","last_clean":"2024-04-25T08:43:35.712605+0000","last_became_active":"2024-04-25T08:43:35.708402+0000","last_became_peered":"2024-04-25T08:43:35.708402+0000","last_unstale":"2024-04-25T08:43:35.712605+0000","last_undegraded":"2024-04-25T08:43:35.712605+0000","last_fullsized":"2024-04-25T08:43:35.712605+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.357301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683213+0000","last_change":"2024-04-25T08:43:34.690466+0000","last_active":"2024-04-25T08:43:36.683213+0000","last_peered":"2024-04-25T08:43:36.683213+0000","last_clean":"2024-04-25T08:43:36.683213+0000","last_became_active":"2024-04-25T08:43:34.690193+0000","last_became_peered":"2024-04-25T08:43:34.690193+0000","last_unstale":"2024-04-25T08:43:36.683213+0000","last_undegraded":"2024-04-25T08:43:36.683213+0000","last_fullsized":"2024-04-25T08:43:36.683213+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:48.869636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708574+0000","last_change":"2024-04-25T08:43:35.701972+0000","last_active":"2024-04-25T08:43:35.708574+0000","last_peered":"2024-04-25T08:43:35.708574+0000","last_clean":"2024-04-25T08:43:35.708574+0000","last_became_active":"2024-04-25T08:43:35.697873+0000","last_became_peered":"2024-04-25T08:43:35.697873+0000","last_unstale":"2024-04-25T08:43:35.708574+0000","last_undegraded":"2024-04-25T08:43:35.708574+0000","last_fullsized":"2024-04-25T08:43:35.708574+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:12.608221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683777+0000","last_change":"2024-04-25T08:43:34.685145+0000","last_active":"2024-04-25T08:43:36.683777+0000","last_peered":"2024-04-25T08:43:36.683777+0000","last_clean":"2024-04-25T08:43:36.683777+0000","last_became_active":"2024-04-25T08:43:34.684876+0000","last_became_peered":"2024-04-25T08:43:34.684876+0000","last_unstale":"2024-04-25T08:43:36.683777+0000","last_undegraded":"2024-04-25T08:43:36.683777+0000","last_fullsized":"2024-04-25T08:43:36.683777+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:36.494719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683526+0000","last_change":"2024-04-25T08:43:35.698345+0000","last_active":"2024-04-25T08:43:36.683526+0000","last_peered":"2024-04-25T08:43:36.683526+0000","last_clean":"2024-04-25T08:43:36.683526+0000","last_became_active":"2024-04-25T08:43:35.698029+0000","last_became_peered":"2024-04-25T08:43:35.698029+0000","last_unstale":"2024-04-25T08:43:36.683526+0000","last_undegraded":"2024-04-25T08:43:36.683526+0000","last_fullsized":"2024-04-25T08:43:36.683526+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:48.007401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682858+0000","last_change":"2024-04-25T08:43:34.692862+0000","last_active":"2024-04-25T08:43:36.682858+0000","last_peered":"2024-04-25T08:43:36.682858+0000","last_clean":"2024-04-25T08:43:36.682858+0000","last_became_active":"2024-04-25T08:43:34.692617+0000","last_became_peered":"2024-04-25T08:43:34.692617+0000","last_unstale":"2024-04-25T08:43:36.682858+0000","last_undegraded":"2024-04-25T08:43:36.682858+0000","last_fullsized":"2024-04-25T08:43:36.682858+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:46.405208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820505+0000","last_change":"2024-04-25T08:43:35.709834+0000","last_active":"2024-04-25T08:43:36.820505+0000","last_peered":"2024-04-25T08:43:36.820505+0000","last_clean":"2024-04-25T08:43:36.820505+0000","last_became_active":"2024-04-25T08:43:35.709601+0000","last_became_peered":"2024-04-25T08:43:35.709601+0000","last_unstale":"2024-04-25T08:43:36.820505+0000","last_undegraded":"2024-04-25T08:43:36.820505+0000","last_fullsized":"2024-04-25T08:43:36.820505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:20.851391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708030+0000","last_change":"2024-04-25T08:43:34.690387+0000","last_active":"2024-04-25T08:43:35.708030+0000","last_peered":"2024-04-25T08:43:35.708030+0000","last_clean":"2024-04-25T08:43:35.708030+0000","last_became_active":"2024-04-25T08:43:34.690071+0000","last_became_peered":"2024-04-25T08:43:34.690071+0000","last_unstale":"2024-04-25T08:43:35.708030+0000","last_undegraded":"2024-04-25T08:43:35.708030+0000","last_fullsized":"2024-04-25T08:43:35.708030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:06.349595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683825+0000","last_change":"2024-04-25T08:43:35.696198+0000","last_active":"2024-04-25T08:43:36.683825+0000","last_peered":"2024-04-25T08:43:36.683825+0000","last_clean":"2024-04-25T08:43:36.683825+0000","last_became_active":"2024-04-25T08:43:35.693991+0000","last_became_peered":"2024-04-25T08:43:35.693991+0000","last_unstale":"2024-04-25T08:43:36.683825+0000","last_undegraded":"2024-04-25T08:43:36.683825+0000","last_fullsized":"2024-04-25T08:43:36.683825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:56.146218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683500+0000","last_change":"2024-04-25T08:43:34.688617+0000","last_active":"2024-04-25T08:43:36.683500+0000","last_peered":"2024-04-25T08:43:36.683500+0000","last_clean":"2024-04-25T08:43:36.683500+0000","last_became_active":"2024-04-25T08:43:34.688365+0000","last_became_peered":"2024-04-25T08:43:34.688365+0000","last_unstale":"2024-04-25T08:43:36.683500+0000","last_undegraded":"2024-04-25T08:43:36.683500+0000","last_fullsized":"2024-04-25T08:43:36.683500+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:10.079374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684190+0000","last_change":"2024-04-25T08:43:35.694335+0000","last_active":"2024-04-25T08:43:36.684190+0000","last_peered":"2024-04-25T08:43:36.684190+0000","last_clean":"2024-04-25T08:43:36.684190+0000","last_became_active":"2024-04-25T08:43:35.693837+0000","last_became_peered":"2024-04-25T08:43:35.693837+0000","last_unstale":"2024-04-25T08:43:36.684190+0000","last_undegraded":"2024-04-25T08:43:36.684190+0000","last_fullsized":"2024-04-25T08:43:36.684190+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.761157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.692614+0000","last_change":"2024-04-25T08:43:34.690787+0000","last_active":"2024-04-25T08:43:35.692614+0000","last_peered":"2024-04-25T08:43:35.692614+0000","last_clean":"2024-04-25T08:43:35.692614+0000","last_became_active":"2024-04-25T08:43:34.690512+0000","last_became_peered":"2024-04-25T08:43:34.690512+0000","last_unstale":"2024-04-25T08:43:35.692614+0000","last_undegraded":"2024-04-25T08:43:35.692614+0000","last_fullsized":"2024-04-25T08:43:35.692614+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:52.768221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708328+0000","last_change":"2024-04-25T08:43:35.697804+0000","last_active":"2024-04-25T08:43:35.708328+0000","last_peered":"2024-04-25T08:43:35.708328+0000","last_clean":"2024-04-25T08:43:35.708328+0000","last_became_active":"2024-04-25T08:43:35.697433+0000","last_became_peered":"2024-04-25T08:43:35.697433+0000","last_unstale":"2024-04-25T08:43:35.708328+0000","last_undegraded":"2024-04-25T08:43:35.708328+0000","last_fullsized":"2024-04-25T08:43:35.708328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:52.233153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.698119+0000","last_change":"2024-04-25T08:43:34.686768+0000","last_active":"2024-04-25T08:43:35.698119+0000","last_peered":"2024-04-25T08:43:35.698119+0000","last_clean":"2024-04-25T08:43:35.698119+0000","last_became_active":"2024-04-25T08:43:34.686453+0000","last_became_peered":"2024-04-25T08:43:34.686453+0000","last_unstale":"2024-04-25T08:43:35.698119+0000","last_undegraded":"2024-04-25T08:43:35.698119+0000","last_fullsized":"2024-04-25T08:43:35.698119+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.567248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820330+0000","last_change":"2024-04-25T08:43:35.693888+0000","last_active":"2024-04-25T08:43:36.820330+0000","last_peered":"2024-04-25T08:43:36.820330+0000","last_clean":"2024-04-25T08:43:36.820330+0000","last_became_active":"2024-04-25T08:43:35.693586+0000","last_became_peered":"2024-04-25T08:43:35.693586+0000","last_unstale":"2024-04-25T08:43:36.820330+0000","last_undegraded":"2024-04-25T08:43:36.820330+0000","last_fullsized":"2024-04-25T08:43:36.820330+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:43.958244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709705+0000","last_change":"2024-04-25T08:43:34.683378+0000","last_active":"2024-04-25T08:43:36.709705+0000","last_peered":"2024-04-25T08:43:36.709705+0000","last_clean":"2024-04-25T08:43:36.709705+0000","last_became_active":"2024-04-25T08:43:34.681953+0000","last_became_peered":"2024-04-25T08:43:34.681953+0000","last_unstale":"2024-04-25T08:43:36.709705+0000","last_undegraded":"2024-04-25T08:43:36.709705+0000","last_fullsized":"2024-04-25T08:43:36.709705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:39.155803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684290+0000","last_change":"2024-04-25T08:43:35.687330+0000","last_active":"2024-04-25T08:43:36.684290+0000","last_peered":"2024-04-25T08:43:36.684290+0000","last_clean":"2024-04-25T08:43:36.684290+0000","last_became_active":"2024-04-25T08:43:35.687067+0000","last_became_peered":"2024-04-25T08:43:35.687067+0000","last_unstale":"2024-04-25T08:43:36.684290+0000","last_undegraded":"2024-04-25T08:43:36.684290+0000","last_fullsized":"2024-04-25T08:43:36.684290+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:43.958244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682946+0000","last_change":"2024-04-25T08:43:34.674852+0000","last_active":"2024-04-25T08:43:36.682946+0000","last_peered":"2024-04-25T08:43:36.682946+0000","last_clean":"2024-04-25T08:43:36.682946+0000","last_became_active":"2024-04-25T08:43:34.672852+0000","last_became_peered":"2024-04-25T08:43:34.672852+0000","last_unstale":"2024-04-25T08:43:36.682946+0000","last_undegraded":"2024-04-25T08:43:36.682946+0000","last_fullsized":"2024-04-25T08:43:36.682946+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:52.768221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683867+0000","last_change":"2024-04-25T08:43:35.694141+0000","last_active":"2024-04-25T08:43:36.683867+0000","last_peered":"2024-04-25T08:43:36.683867+0000","last_clean":"2024-04-25T08:43:36.683867+0000","last_became_active":"2024-04-25T08:43:35.693828+0000","last_became_peered":"2024-04-25T08:43:35.693828+0000","last_unstale":"2024-04-25T08:43:36.683867+0000","last_undegraded":"2024-04-25T08:43:36.683867+0000","last_fullsized":"2024-04-25T08:43:36.683867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:47.203771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682462+0000","last_change":"2024-04-25T08:43:34.684624+0000","last_active":"2024-04-25T08:43:36.682462+0000","last_peered":"2024-04-25T08:43:36.682462+0000","last_clean":"2024-04-25T08:43:36.682462+0000","last_became_active":"2024-04-25T08:43:34.683796+0000","last_became_peered":"2024-04-25T08:43:34.683796+0000","last_unstale":"2024-04-25T08:43:36.682462+0000","last_undegraded":"2024-04-25T08:43:36.682462+0000","last_fullsized":"2024-04-25T08:43:36.682462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:27.895523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.710117+0000","last_change":"2024-04-25T08:43:35.701576+0000","last_active":"2024-04-25T08:43:35.710117+0000","last_peered":"2024-04-25T08:43:35.710117+0000","last_clean":"2024-04-25T08:43:35.710117+0000","last_became_active":"2024-04-25T08:43:35.697618+0000","last_became_peered":"2024-04-25T08:43:35.697618+0000","last_unstale":"2024-04-25T08:43:35.710117+0000","last_undegraded":"2024-04-25T08:43:35.710117+0000","last_fullsized":"2024-04-25T08:43:35.710117+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:06.474434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683297+0000","last_change":"2024-04-25T08:43:34.684309+0000","last_active":"2024-04-25T08:43:36.683297+0000","last_peered":"2024-04-25T08:43:36.683297+0000","last_clean":"2024-04-25T08:43:36.683297+0000","last_became_active":"2024-04-25T08:43:34.684035+0000","last_became_peered":"2024-04-25T08:43:34.684035+0000","last_unstale":"2024-04-25T08:43:36.683297+0000","last_undegraded":"2024-04-25T08:43:36.683297+0000","last_fullsized":"2024-04-25T08:43:36.683297+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.992651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683557+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:43:36.683557+0000","last_peered":"2024-04-25T08:43:36.683557+0000","last_clean":"2024-04-25T08:43:36.683557+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:43:36.683557+0000","last_undegraded":"2024-04-25T08:43:36.683557+0000","last_fullsized":"2024-04-25T08:43:36.683557+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":787194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":195726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":195256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":195748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":479232,"data_stored":203482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":195214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":195234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":785546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:43:39.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:43:39.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.979+0000 7f048f600640 1 Processor -- start 2024-04-25T08:43:39.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.979+0000 7f048f600640 1 -- start start 2024-04-25T08:43:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048f600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048815bdb0 0x7f04880cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048f600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f04880cda80 0x7f048815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048f600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04880cc700 con 0x7f0488159230 2024-04-25T08:43:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048f600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04880cc9e0 con 0x7f04880cda80 2024-04-25T08:43:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048f600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880cccc0 con 0x7f048815bdb0 2024-04-25T08:43:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0488159230 0x7f04880cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58338/0 (socket says 172.21.15.46:58338) 2024-04-25T08:43:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048cc00640 1 -- 172.21.15.46:0/3867907473 learned_addr learned my addr 172.21.15.46:0/3867907473 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725327394 0 0) 0x7f04880cc700 con 0x7f0488159230 2024-04-25T08:43:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0470003530 con 0x7f0488159230 2024-04-25T08:43:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048d600640 1 --2- 172.21.15.46:0/3867907473 >> v2:172.21.15.82:3300/0 conn(0x7f04880cda80 0x7f048815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487e00640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048815bdb0 0x7f04880cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 171854943 0 0) 0x7f0470003530 con 0x7f0488159230 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048815bdb0 msgr2=0x7f04880cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048815bdb0 0x7f04880cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 >> v2:172.21.15.82:3300/0 conn(0x7f04880cda80 msgr2=0x7f048815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 --2- 172.21.15.46:0/3867907473 >> v2:172.21.15.82:3300/0 conn(0x7f04880cda80 0x7f048815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880ccfa0 con 0x7f0488159230 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f047c001680 con 0x7f0488159230 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 >> v1:172.21.15.46:6789/0 conn(0x7f0488159230 legacy=0x7f04880cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04700039c0 0x7f0470003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0470004810 0x7f0470004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700051d0 0x7f047000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880cc9e0 con 0x7f04700051d0 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880cccc0 con 0x7f0470004810 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047000c0f0 con 0x7f04700039c0 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487e00640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048815bdb0 0x7f04880cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048d600640 1 --2- 172.21.15.46:0/3867907473 >> v2:172.21.15.82:3300/0 conn(0x7f04880cda80 0x7f048815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048d600640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700051d0 0x7f047000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048cc00640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04700039c0 0x7f0470003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f0487e00640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0470004810 0x7f0470004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048d600640 1 -- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04700039c0 msgr2=0x7f0470003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048d600640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04700039c0 0x7f0470003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048d600640 1 -- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0470004810 msgr2=0x7f0470004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048d600640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0470004810 0x7f0470004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.983+0000 7f048d600640 1 -- 172.21.15.46:0/3867907473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0470003530 con 0x7f04700051d0 2024-04-25T08:43:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f048d600640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700051d0 0x7f047000b9d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f04800073e0 tx=0x7f048001cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cd19367732916e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04800b2070 con 0x7f04700051d0 2024-04-25T08:43:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04800a5e40 con 0x7f04700051d0 2024-04-25T08:43:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f0487400640 1 -- 172.21.15.46:0/3867907473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480009c70 con 0x7f04700051d0 2024-04-25T08:43:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f048f600640 1 -- 172.21.15.46:0/3867907473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700051d0 msgr2=0x7f047000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f048f600640 1 --2- 172.21.15.46:0/3867907473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04700051d0 0x7f047000b9d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f04800073e0 tx=0x7f048001cf40 comp rx=0 tx=0).stop 2024-04-25T08:43:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f048d600640 1 -- 172.21.15.46:0/3867907473 reap_dead start 2024-04-25T08:43:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f048f600640 1 -- 172.21.15.46:0/3867907473 shutdown_connections 2024-04-25T08:43:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f048f600640 1 -- 172.21.15.46:0/3867907473 >> 172.21.15.46:0/3867907473 conn(0x7f04880b12a0 msgr2=0x7f04880b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f048f600640 1 -- 172.21.15.46:0/3867907473 shutdown_connections 2024-04-25T08:43:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.987+0000 7f048f600640 1 -- 172.21.15.46:0/3867907473 wait complete. 2024-04-25T08:43:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 Processor -- start 2024-04-25T08:43:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 -- start start 2024-04-25T08:43:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04881580a0 0x7f048813ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048813c040 0x7f04880b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04880b3d80 0x7f04880aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0488159790 con 0x7f048813c040 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0488159610 con 0x7f04881580a0 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880cd540 con 0x7f04880b3d80 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0487e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048813c040 0x7f04880b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0487e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048813c040 0x7f04880b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46426/0 (socket says 172.21.15.46:46426) 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0487e00640 1 -- 172.21.15.46:0/768354188 learned_addr learned my addr 172.21.15.46:0/768354188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048d600640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04880b3d80 0x7f04880aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048cc00640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04881580a0 0x7f048813ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0487e00640 1 -- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04880b3d80 msgr2=0x7f04880aca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0487e00640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04880b3d80 0x7f04880aca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0487e00640 1 -- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04881580a0 msgr2=0x7f048813ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0487e00640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04881580a0 0x7f048813ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0487e00640 1 -- 172.21.15.46:0/768354188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488159ba0 con 0x7f048813c040 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048d600640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04880b3d80 0x7f04880aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048cc00640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04881580a0 0x7f048813ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0487e00640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048813c040 0x7f04880b3840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0474011540 tx=0x7f0474011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0484c00640 1 -- 172.21.15.46:0/768354188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474019070 con 0x7f048813c040 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0484c00640 1 -- 172.21.15.46:0/768354188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0474004220 con 0x7f048813c040 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f0484c00640 1 -- 172.21.15.46:0/768354188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047400f040 con 0x7f048813c040 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048813f550 con 0x7f048813c040 2024-04-25T08:43:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048813fb10 con 0x7f048813c040 2024-04-25T08:43:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.991+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0488069080 con 0x7f048813c040 2024-04-25T08:43:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.995+0000 7f0484c00640 1 -- 172.21.15.46:0/768354188 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0474015670 con 0x7f048813c040 2024-04-25T08:43:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.995+0000 7f0484c00640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.995+0000 7f0484c00640 1 -- 172.21.15.46:0/768354188 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0474085410 con 0x7f048813c040 2024-04-25T08:43:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.995+0000 7f048cc00640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.995+0000 7f048cc00640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f047c001060 tx=0x7f047c001e00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:39.999+0000 7f0484c00640 1 -- 172.21.15.46:0/768354188 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0474056200 con 0x7f048813c040 2024-04-25T08:43:40.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.155+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0488068520 con 0x7f0468066f60 2024-04-25T08:43:40.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.163+0000 7f0484c00640 1 -- 172.21.15.46:0/768354188 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251934 (secure 0 0 0) 0x7f047c013950 con 0x7f0468066f60 2024-04-25T08:43:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:40.161 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 msgr2=0x7f0468069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f047c001060 tx=0x7f047c001e00 comp rx=0 tx=0).stop 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048813c040 msgr2=0x7f04880b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048813c040 0x7f04880b3840 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0474011540 tx=0x7f0474011a00 comp rx=0 tx=0).stop 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 shutdown_connections 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04880b3d80 0x7f04880aca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048813c040 0x7f04880b3840 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 --2- 172.21.15.46:0/768354188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04881580a0 0x7f048813ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 >> 172.21.15.46:0/768354188 conn(0x7f04880b12a0 msgr2=0x7f04880b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 shutdown_connections 2024-04-25T08:43:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.167+0000 7f048f600640 1 -- 172.21.15.46:0/768354188 wait complete. 2024-04-25T08:43:40.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-04-25T08:43:38.272225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237584,"kb_used_data":4756,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737136,"statfs":{"total":773094113280,"available":772850827264,"internally_reserved":0,"allocated":4870144,"data_stored":2753400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.619001"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684394+0000","last_change":"2024-04-25T08:43:35.700911+0000","last_active":"2024-04-25T08:43:36.684394+0000","last_peered":"2024-04-25T08:43:36.684394+0000","last_clean":"2024-04-25T08:43:36.684394+0000","last_became_active":"2024-04-25T08:43:35.700661+0000","last_became_peered":"2024-04-25T08:43:35.700661+0000","last_unstale":"2024-04-25T08:43:36.684394+0000","last_undegraded":"2024-04-25T08:43:36.684394+0000","last_fullsized":"2024-04-25T08:43:36.684394+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:04.137354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684164+0000","last_change":"2024-04-25T08:43:35.689804+0000","last_active":"2024-04-25T08:43:36.684164+0000","last_peered":"2024-04-25T08:43:36.684164+0000","last_clean":"2024-04-25T08:43:36.684164+0000","last_became_active":"2024-04-25T08:43:35.689431+0000","last_became_peered":"2024-04-25T08:43:35.689431+0000","last_unstale":"2024-04-25T08:43:36.684164+0000","last_undegraded":"2024-04-25T08:43:36.684164+0000","last_fullsized":"2024-04-25T08:43:36.684164+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:45.015217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712542+0000","last_change":"2024-04-25T08:43:35.710788+0000","last_active":"2024-04-25T08:43:35.712542+0000","last_peered":"2024-04-25T08:43:35.712542+0000","last_clean":"2024-04-25T08:43:35.712542+0000","last_became_active":"2024-04-25T08:43:35.708973+0000","last_became_peered":"2024-04-25T08:43:35.708973+0000","last_unstale":"2024-04-25T08:43:35.712542+0000","last_undegraded":"2024-04-25T08:43:35.712542+0000","last_fullsized":"2024-04-25T08:43:35.712542+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:28.903230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683566+0000","last_change":"2024-04-25T08:43:35.702570+0000","last_active":"2024-04-25T08:43:36.683566+0000","last_peered":"2024-04-25T08:43:36.683566+0000","last_clean":"2024-04-25T08:43:36.683566+0000","last_became_active":"2024-04-25T08:43:35.701978+0000","last_became_peered":"2024-04-25T08:43:35.701978+0000","last_unstale":"2024-04-25T08:43:36.683566+0000","last_undegraded":"2024-04-25T08:43:36.683566+0000","last_fullsized":"2024-04-25T08:43:36.683566+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:27.282709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682910+0000","last_change":"2024-04-25T08:43:35.701716+0000","last_active":"2024-04-25T08:43:36.682910+0000","last_peered":"2024-04-25T08:43:36.682910+0000","last_clean":"2024-04-25T08:43:36.682910+0000","last_became_active":"2024-04-25T08:43:35.701421+0000","last_became_peered":"2024-04-25T08:43:35.701421+0000","last_unstale":"2024-04-25T08:43:36.682910+0000","last_undegraded":"2024-04-25T08:43:36.682910+0000","last_fullsized":"2024-04-25T08:43:36.682910+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:53.704185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820335+0000","last_change":"2024-04-25T08:43:35.693975+0000","last_active":"2024-04-25T08:43:36.820335+0000","last_peered":"2024-04-25T08:43:36.820335+0000","last_clean":"2024-04-25T08:43:36.820335+0000","last_became_active":"2024-04-25T08:43:35.693675+0000","last_became_peered":"2024-04-25T08:43:35.693675+0000","last_unstale":"2024-04-25T08:43:36.820335+0000","last_undegraded":"2024-04-25T08:43:36.820335+0000","last_fullsized":"2024-04-25T08:43:36.820335+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:18.095625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.709098+0000","last_change":"2024-04-25T08:43:35.700568+0000","last_active":"2024-04-25T08:43:35.709098+0000","last_peered":"2024-04-25T08:43:35.709098+0000","last_clean":"2024-04-25T08:43:35.709098+0000","last_became_active":"2024-04-25T08:43:35.697855+0000","last_became_peered":"2024-04-25T08:43:35.697855+0000","last_unstale":"2024-04-25T08:43:35.709098+0000","last_undegraded":"2024-04-25T08:43:35.709098+0000","last_fullsized":"2024-04-25T08:43:35.709098+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:43.958244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682438+0000","last_change":"2024-04-25T08:43:35.708245+0000","last_active":"2024-04-25T08:43:36.682438+0000","last_peered":"2024-04-25T08:43:36.682438+0000","last_clean":"2024-04-25T08:43:36.682438+0000","last_became_active":"2024-04-25T08:43:35.705002+0000","last_became_peered":"2024-04-25T08:43:35.705002+0000","last_unstale":"2024-04-25T08:43:36.682438+0000","last_undegraded":"2024-04-25T08:43:36.682438+0000","last_fullsized":"2024-04-25T08:43:36.682438+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:47.203771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712600+0000","last_change":"2024-04-25T08:43:35.711391+0000","last_active":"2024-04-25T08:43:35.712600+0000","last_peered":"2024-04-25T08:43:35.712600+0000","last_clean":"2024-04-25T08:43:35.712600+0000","last_became_active":"2024-04-25T08:43:35.711121+0000","last_became_peered":"2024-04-25T08:43:35.711121+0000","last_unstale":"2024-04-25T08:43:35.712600+0000","last_undegraded":"2024-04-25T08:43:35.712600+0000","last_fullsized":"2024-04-25T08:43:35.712600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:41.662334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708803+0000","last_change":"2024-04-25T08:43:35.703259+0000","last_active":"2024-04-25T08:43:35.708803+0000","last_peered":"2024-04-25T08:43:35.708803+0000","last_clean":"2024-04-25T08:43:35.708803+0000","last_became_active":"2024-04-25T08:43:35.702959+0000","last_became_peered":"2024-04-25T08:43:35.702959+0000","last_unstale":"2024-04-25T08:43:35.708803+0000","last_undegraded":"2024-04-25T08:43:35.708803+0000","last_fullsized":"2024-04-25T08:43:35.708803+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.145256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.710417+0000","last_change":"2024-04-25T08:43:35.705545+0000","last_active":"2024-04-25T08:43:35.710417+0000","last_peered":"2024-04-25T08:43:35.710417+0000","last_clean":"2024-04-25T08:43:35.710417+0000","last_became_active":"2024-04-25T08:43:35.705052+0000","last_became_peered":"2024-04-25T08:43:35.705052+0000","last_unstale":"2024-04-25T08:43:35.710417+0000","last_undegraded":"2024-04-25T08:43:35.710417+0000","last_fullsized":"2024-04-25T08:43:35.710417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:54.574871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.710772+0000","last_change":"2024-04-25T08:43:35.697643+0000","last_active":"2024-04-25T08:43:35.710772+0000","last_peered":"2024-04-25T08:43:35.710772+0000","last_clean":"2024-04-25T08:43:35.710772+0000","last_became_active":"2024-04-25T08:43:35.697335+0000","last_became_peered":"2024-04-25T08:43:35.697335+0000","last_unstale":"2024-04-25T08:43:35.710772+0000","last_undegraded":"2024-04-25T08:43:35.710772+0000","last_fullsized":"2024-04-25T08:43:35.710772+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:26.000357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708863+0000","last_change":"2024-04-25T08:43:35.704208+0000","last_active":"2024-04-25T08:43:35.708863+0000","last_peered":"2024-04-25T08:43:35.708863+0000","last_clean":"2024-04-25T08:43:35.708863+0000","last_became_active":"2024-04-25T08:43:35.703869+0000","last_became_peered":"2024-04-25T08:43:35.703869+0000","last_unstale":"2024-04-25T08:43:35.708863+0000","last_undegraded":"2024-04-25T08:43:35.708863+0000","last_fullsized":"2024-04-25T08:43:35.708863+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:28.610365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683607+0000","last_change":"2024-04-25T08:43:35.697841+0000","last_active":"2024-04-25T08:43:36.683607+0000","last_peered":"2024-04-25T08:43:36.683607+0000","last_clean":"2024-04-25T08:43:36.683607+0000","last_became_active":"2024-04-25T08:43:35.697533+0000","last_became_peered":"2024-04-25T08:43:35.697533+0000","last_unstale":"2024-04-25T08:43:36.683607+0000","last_undegraded":"2024-04-25T08:43:36.683607+0000","last_fullsized":"2024-04-25T08:43:36.683607+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:32.121504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.709115+0000","last_change":"2024-04-25T08:43:35.701080+0000","last_active":"2024-04-25T08:43:35.709115+0000","last_peered":"2024-04-25T08:43:35.709115+0000","last_clean":"2024-04-25T08:43:35.709115+0000","last_became_active":"2024-04-25T08:43:35.700652+0000","last_became_peered":"2024-04-25T08:43:35.700652+0000","last_unstale":"2024-04-25T08:43:35.709115+0000","last_undegraded":"2024-04-25T08:43:35.709115+0000","last_fullsized":"2024-04-25T08:43:35.709115+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:18.095625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683265+0000","last_change":"2024-04-25T08:43:35.705915+0000","last_active":"2024-04-25T08:43:36.683265+0000","last_peered":"2024-04-25T08:43:36.683265+0000","last_clean":"2024-04-25T08:43:36.683265+0000","last_became_active":"2024-04-25T08:43:35.705653+0000","last_became_peered":"2024-04-25T08:43:35.705653+0000","last_unstale":"2024-04-25T08:43:36.683265+0000","last_undegraded":"2024-04-25T08:43:36.683265+0000","last_fullsized":"2024-04-25T08:43:36.683265+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:20.851391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683374+0000","last_change":"2024-04-25T08:43:35.698575+0000","last_active":"2024-04-25T08:43:36.683374+0000","last_peered":"2024-04-25T08:43:36.683374+0000","last_clean":"2024-04-25T08:43:36.683374+0000","last_became_active":"2024-04-25T08:43:35.698303+0000","last_became_peered":"2024-04-25T08:43:35.698303+0000","last_unstale":"2024-04-25T08:43:36.683374+0000","last_undegraded":"2024-04-25T08:43:36.683374+0000","last_fullsized":"2024-04-25T08:43:36.683374+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:06.474434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.710059+0000","last_change":"2024-04-25T08:43:35.683195+0000","last_active":"2024-04-25T08:43:36.710059+0000","last_peered":"2024-04-25T08:43:36.710059+0000","last_clean":"2024-04-25T08:43:36.710059+0000","last_became_active":"2024-04-25T08:43:35.682917+0000","last_became_peered":"2024-04-25T08:43:35.682917+0000","last_unstale":"2024-04-25T08:43:36.710059+0000","last_undegraded":"2024-04-25T08:43:36.710059+0000","last_fullsized":"2024-04-25T08:43:36.710059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:15.426531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683779+0000","last_change":"2024-04-25T08:43:35.686127+0000","last_active":"2024-04-25T08:43:36.683779+0000","last_peered":"2024-04-25T08:43:36.683779+0000","last_clean":"2024-04-25T08:43:36.683779+0000","last_became_active":"2024-04-25T08:43:35.685838+0000","last_became_peered":"2024-04-25T08:43:35.685838+0000","last_unstale":"2024-04-25T08:43:36.683779+0000","last_undegraded":"2024-04-25T08:43:36.683779+0000","last_fullsized":"2024-04-25T08:43:36.683779+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.367077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709938+0000","last_change":"2024-04-25T08:43:35.686152+0000","last_active":"2024-04-25T08:43:36.709938+0000","last_peered":"2024-04-25T08:43:36.709938+0000","last_clean":"2024-04-25T08:43:36.709938+0000","last_became_active":"2024-04-25T08:43:35.683503+0000","last_became_peered":"2024-04-25T08:43:35.683503+0000","last_unstale":"2024-04-25T08:43:36.709938+0000","last_undegraded":"2024-04-25T08:43:36.709938+0000","last_fullsized":"2024-04-25T08:43:36.709938+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:00.202978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708037+0000","last_change":"2024-04-25T08:43:35.699208+0000","last_active":"2024-04-25T08:43:35.708037+0000","last_peered":"2024-04-25T08:43:35.708037+0000","last_clean":"2024-04-25T08:43:35.708037+0000","last_became_active":"2024-04-25T08:43:35.698926+0000","last_became_peered":"2024-04-25T08:43:35.698926+0000","last_unstale":"2024-04-25T08:43:35.708037+0000","last_undegraded":"2024-04-25T08:43:35.708037+0000","last_fullsized":"2024-04-25T08:43:35.708037+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:41.662334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684489+0000","last_change":"2024-04-25T08:43:35.692630+0000","last_active":"2024-04-25T08:43:36.684489+0000","last_peered":"2024-04-25T08:43:36.684489+0000","last_clean":"2024-04-25T08:43:36.684489+0000","last_became_active":"2024-04-25T08:43:35.692311+0000","last_became_peered":"2024-04-25T08:43:35.692311+0000","last_unstale":"2024-04-25T08:43:36.684489+0000","last_undegraded":"2024-04-25T08:43:36.684489+0000","last_fullsized":"2024-04-25T08:43:36.684489+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:20.851391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683388+0000","last_change":"2024-04-25T08:43:35.695519+0000","last_active":"2024-04-25T08:43:36.683388+0000","last_peered":"2024-04-25T08:43:36.683388+0000","last_clean":"2024-04-25T08:43:36.683388+0000","last_became_active":"2024-04-25T08:43:35.694946+0000","last_became_peered":"2024-04-25T08:43:35.694946+0000","last_unstale":"2024-04-25T08:43:36.683388+0000","last_undegraded":"2024-04-25T08:43:36.683388+0000","last_fullsized":"2024-04-25T08:43:36.683388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:12.608221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682914+0000","last_change":"2024-04-25T08:43:35.700634+0000","last_active":"2024-04-25T08:43:36.682914+0000","last_peered":"2024-04-25T08:43:36.682914+0000","last_clean":"2024-04-25T08:43:36.682914+0000","last_became_active":"2024-04-25T08:43:35.700263+0000","last_became_peered":"2024-04-25T08:43:35.700263+0000","last_unstale":"2024-04-25T08:43:36.682914+0000","last_undegraded":"2024-04-25T08:43:36.682914+0000","last_fullsized":"2024-04-25T08:43:36.682914+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:02.189663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.710052+0000","last_change":"2024-04-25T08:43:35.698882+0000","last_active":"2024-04-25T08:43:36.710052+0000","last_peered":"2024-04-25T08:43:36.710052+0000","last_clean":"2024-04-25T08:43:36.710052+0000","last_became_active":"2024-04-25T08:43:35.698469+0000","last_became_peered":"2024-04-25T08:43:35.698469+0000","last_unstale":"2024-04-25T08:43:36.710052+0000","last_undegraded":"2024-04-25T08:43:36.710052+0000","last_fullsized":"2024-04-25T08:43:36.710052+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.357301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820613+0000","last_change":"2024-04-25T08:43:35.706450+0000","last_active":"2024-04-25T08:43:36.820613+0000","last_peered":"2024-04-25T08:43:36.820613+0000","last_clean":"2024-04-25T08:43:36.820613+0000","last_became_active":"2024-04-25T08:43:35.706141+0000","last_became_peered":"2024-04-25T08:43:35.706141+0000","last_unstale":"2024-04-25T08:43:36.820613+0000","last_undegraded":"2024-04-25T08:43:36.820613+0000","last_fullsized":"2024-04-25T08:43:36.820613+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:06.474434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709991+0000","last_change":"2024-04-25T08:43:35.687674+0000","last_active":"2024-04-25T08:43:36.709991+0000","last_peered":"2024-04-25T08:43:36.709991+0000","last_clean":"2024-04-25T08:43:36.709991+0000","last_became_active":"2024-04-25T08:43:35.687374+0000","last_became_peered":"2024-04-25T08:43:35.687374+0000","last_unstale":"2024-04-25T08:43:36.709991+0000","last_undegraded":"2024-04-25T08:43:36.709991+0000","last_fullsized":"2024-04-25T08:43:36.709991+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:28.903230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708080+0000","last_change":"2024-04-25T08:43:35.700376+0000","last_active":"2024-04-25T08:43:35.708080+0000","last_peered":"2024-04-25T08:43:35.708080+0000","last_clean":"2024-04-25T08:43:35.708080+0000","last_became_active":"2024-04-25T08:43:35.700115+0000","last_became_peered":"2024-04-25T08:43:35.700115+0000","last_unstale":"2024-04-25T08:43:35.708080+0000","last_undegraded":"2024-04-25T08:43:35.708080+0000","last_fullsized":"2024-04-25T08:43:35.708080+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:32.121504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709857+0000","last_change":"2024-04-25T08:43:35.691080+0000","last_active":"2024-04-25T08:43:36.709857+0000","last_peered":"2024-04-25T08:43:36.709857+0000","last_clean":"2024-04-25T08:43:36.709857+0000","last_became_active":"2024-04-25T08:43:35.690764+0000","last_became_peered":"2024-04-25T08:43:35.690764+0000","last_unstale":"2024-04-25T08:43:36.709857+0000","last_undegraded":"2024-04-25T08:43:36.709857+0000","last_fullsized":"2024-04-25T08:43:36.709857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:10.828438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709756+0000","last_change":"2024-04-25T08:43:35.678455+0000","last_active":"2024-04-25T08:43:36.709756+0000","last_peered":"2024-04-25T08:43:36.709756+0000","last_clean":"2024-04-25T08:43:36.709756+0000","last_became_active":"2024-04-25T08:43:35.677876+0000","last_became_peered":"2024-04-25T08:43:35.677876+0000","last_unstale":"2024-04-25T08:43:36.709756+0000","last_undegraded":"2024-04-25T08:43:36.709756+0000","last_fullsized":"2024-04-25T08:43:36.709756+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:50.574954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683646+0000","last_change":"2024-04-25T08:43:35.695297+0000","last_active":"2024-04-25T08:43:36.683646+0000","last_peered":"2024-04-25T08:43:36.683646+0000","last_clean":"2024-04-25T08:43:36.683646+0000","last_became_active":"2024-04-25T08:43:35.694973+0000","last_became_peered":"2024-04-25T08:43:35.694973+0000","last_unstale":"2024-04-25T08:43:36.683646+0000","last_undegraded":"2024-04-25T08:43:36.683646+0000","last_fullsized":"2024-04-25T08:43:36.683646+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:28.610365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820257+0000","last_change":"2024-04-25T08:43:35.686285+0000","last_active":"2024-04-25T08:43:36.820257+0000","last_peered":"2024-04-25T08:43:36.820257+0000","last_clean":"2024-04-25T08:43:36.820257+0000","last_became_active":"2024-04-25T08:43:35.685962+0000","last_became_peered":"2024-04-25T08:43:35.685962+0000","last_unstale":"2024-04-25T08:43:36.820257+0000","last_undegraded":"2024-04-25T08:43:36.820257+0000","last_fullsized":"2024-04-25T08:43:36.820257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:32.121504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683439+0000","last_change":"2024-04-25T08:43:35.708776+0000","last_active":"2024-04-25T08:43:36.683439+0000","last_peered":"2024-04-25T08:43:36.683439+0000","last_clean":"2024-04-25T08:43:36.683439+0000","last_became_active":"2024-04-25T08:43:35.708465+0000","last_became_peered":"2024-04-25T08:43:35.708465+0000","last_unstale":"2024-04-25T08:43:36.683439+0000","last_undegraded":"2024-04-25T08:43:36.683439+0000","last_fullsized":"2024-04-25T08:43:36.683439+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.145256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.710116+0000","last_change":"2024-04-25T08:43:34.676582+0000","last_active":"2024-04-25T08:43:36.710116+0000","last_peered":"2024-04-25T08:43:36.710116+0000","last_clean":"2024-04-25T08:43:36.710116+0000","last_became_active":"2024-04-25T08:43:34.676311+0000","last_became_peered":"2024-04-25T08:43:34.676311+0000","last_unstale":"2024-04-25T08:43:36.710116+0000","last_undegraded":"2024-04-25T08:43:36.710116+0000","last_fullsized":"2024-04-25T08:43:36.710116+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:12.964771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820628+0000","last_change":"2024-04-25T08:43:35.703304+0000","last_active":"2024-04-25T08:43:36.820628+0000","last_peered":"2024-04-25T08:43:36.820628+0000","last_clean":"2024-04-25T08:43:36.820628+0000","last_became_active":"2024-04-25T08:43:35.702930+0000","last_became_peered":"2024-04-25T08:43:35.702930+0000","last_unstale":"2024-04-25T08:43:36.820628+0000","last_undegraded":"2024-04-25T08:43:36.820628+0000","last_fullsized":"2024-04-25T08:43:36.820628+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:12.608221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.706742+0000","last_change":"2024-04-25T08:43:34.692652+0000","last_active":"2024-04-25T08:43:35.706742+0000","last_peered":"2024-04-25T08:43:35.706742+0000","last_clean":"2024-04-25T08:43:35.706742+0000","last_became_active":"2024-04-25T08:43:34.692360+0000","last_became_peered":"2024-04-25T08:43:34.692360+0000","last_unstale":"2024-04-25T08:43:35.706742+0000","last_undegraded":"2024-04-25T08:43:35.706742+0000","last_fullsized":"2024-04-25T08:43:35.706742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:36.494719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684995+0000","last_change":"2024-04-25T08:43:35.683970+0000","last_active":"2024-04-25T08:43:36.684995+0000","last_peered":"2024-04-25T08:43:36.684995+0000","last_clean":"2024-04-25T08:43:36.684995+0000","last_became_active":"2024-04-25T08:43:35.683684+0000","last_became_peered":"2024-04-25T08:43:35.683684+0000","last_unstale":"2024-04-25T08:43:36.684995+0000","last_undegraded":"2024-04-25T08:43:36.684995+0000","last_fullsized":"2024-04-25T08:43:36.684995+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:50.145256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820667+0000","last_change":"2024-04-25T08:43:34.676639+0000","last_active":"2024-04-25T08:43:36.820667+0000","last_peered":"2024-04-25T08:43:36.820667+0000","last_clean":"2024-04-25T08:43:36.820667+0000","last_became_active":"2024-04-25T08:43:34.676341+0000","last_became_peered":"2024-04-25T08:43:34.676341+0000","last_unstale":"2024-04-25T08:43:36.820667+0000","last_undegraded":"2024-04-25T08:43:36.820667+0000","last_fullsized":"2024-04-25T08:43:36.820667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:51.225447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684342+0000","last_change":"2024-04-25T08:43:35.689654+0000","last_active":"2024-04-25T08:43:36.684342+0000","last_peered":"2024-04-25T08:43:36.684342+0000","last_clean":"2024-04-25T08:43:36.684342+0000","last_became_active":"2024-04-25T08:43:35.688996+0000","last_became_peered":"2024-04-25T08:43:35.688996+0000","last_unstale":"2024-04-25T08:43:36.684342+0000","last_undegraded":"2024-04-25T08:43:36.684342+0000","last_fullsized":"2024-04-25T08:43:36.684342+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:45.102390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684616+0000","last_change":"2024-04-25T08:43:34.688559+0000","last_active":"2024-04-25T08:43:36.684616+0000","last_peered":"2024-04-25T08:43:36.684616+0000","last_clean":"2024-04-25T08:43:36.684616+0000","last_became_active":"2024-04-25T08:43:34.688187+0000","last_became_peered":"2024-04-25T08:43:34.688187+0000","last_unstale":"2024-04-25T08:43:36.684616+0000","last_undegraded":"2024-04-25T08:43:36.684616+0000","last_fullsized":"2024-04-25T08:43:36.684616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:36.494719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712219+0000","last_change":"2024-04-25T08:43:35.708325+0000","last_active":"2024-04-25T08:43:35.712219+0000","last_peered":"2024-04-25T08:43:35.712219+0000","last_clean":"2024-04-25T08:43:35.712219+0000","last_became_active":"2024-04-25T08:43:35.707610+0000","last_became_peered":"2024-04-25T08:43:35.707610+0000","last_unstale":"2024-04-25T08:43:35.712219+0000","last_undegraded":"2024-04-25T08:43:35.712219+0000","last_fullsized":"2024-04-25T08:43:35.712219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:00.202978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.704053+0000","last_change":"2024-04-25T08:43:34.685574+0000","last_active":"2024-04-25T08:43:35.704053+0000","last_peered":"2024-04-25T08:43:35.704053+0000","last_clean":"2024-04-25T08:43:35.704053+0000","last_became_active":"2024-04-25T08:43:34.685241+0000","last_became_peered":"2024-04-25T08:43:34.685241+0000","last_unstale":"2024-04-25T08:43:35.704053+0000","last_undegraded":"2024-04-25T08:43:35.704053+0000","last_fullsized":"2024-04-25T08:43:35.704053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.567248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.707999+0000","last_change":"2024-04-25T08:43:35.697541+0000","last_active":"2024-04-25T08:43:35.707999+0000","last_peered":"2024-04-25T08:43:35.707999+0000","last_clean":"2024-04-25T08:43:35.707999+0000","last_became_active":"2024-04-25T08:43:35.696604+0000","last_became_peered":"2024-04-25T08:43:35.696604+0000","last_unstale":"2024-04-25T08:43:35.707999+0000","last_undegraded":"2024-04-25T08:43:35.707999+0000","last_fullsized":"2024-04-25T08:43:35.707999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:15.426531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683261+0000","last_change":"2024-04-25T08:43:34.685281+0000","last_active":"2024-04-25T08:43:36.683261+0000","last_peered":"2024-04-25T08:43:36.683261+0000","last_clean":"2024-04-25T08:43:36.683261+0000","last_became_active":"2024-04-25T08:43:34.685012+0000","last_became_peered":"2024-04-25T08:43:34.685012+0000","last_unstale":"2024-04-25T08:43:36.683261+0000","last_undegraded":"2024-04-25T08:43:36.683261+0000","last_fullsized":"2024-04-25T08:43:36.683261+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:51.225447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684298+0000","last_change":"2024-04-25T08:43:35.703989+0000","last_active":"2024-04-25T08:43:36.684298+0000","last_peered":"2024-04-25T08:43:36.684298+0000","last_clean":"2024-04-25T08:43:36.684298+0000","last_became_active":"2024-04-25T08:43:35.703719+0000","last_became_peered":"2024-04-25T08:43:35.703719+0000","last_unstale":"2024-04-25T08:43:36.684298+0000","last_undegraded":"2024-04-25T08:43:36.684298+0000","last_fullsized":"2024-04-25T08:43:36.684298+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:18.095625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820335+0000","last_change":"2024-04-25T08:43:34.685569+0000","last_active":"2024-04-25T08:43:36.820335+0000","last_peered":"2024-04-25T08:43:36.820335+0000","last_clean":"2024-04-25T08:43:36.820335+0000","last_became_active":"2024-04-25T08:43:34.685258+0000","last_became_peered":"2024-04-25T08:43:34.685258+0000","last_unstale":"2024-04-25T08:43:36.820335+0000","last_undegraded":"2024-04-25T08:43:36.820335+0000","last_fullsized":"2024-04-25T08:43:36.820335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:36.494719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683350+0000","last_change":"2024-04-25T08:43:35.695143+0000","last_active":"2024-04-25T08:43:36.683350+0000","last_peered":"2024-04-25T08:43:36.683350+0000","last_clean":"2024-04-25T08:43:36.683350+0000","last_became_active":"2024-04-25T08:43:35.694868+0000","last_became_peered":"2024-04-25T08:43:35.694868+0000","last_unstale":"2024-04-25T08:43:36.683350+0000","last_undegraded":"2024-04-25T08:43:36.683350+0000","last_fullsized":"2024-04-25T08:43:36.683350+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:44.020550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682987+0000","last_change":"2024-04-25T08:43:34.677461+0000","last_active":"2024-04-25T08:43:36.682987+0000","last_peered":"2024-04-25T08:43:36.682987+0000","last_clean":"2024-04-25T08:43:36.682987+0000","last_became_active":"2024-04-25T08:43:34.677148+0000","last_became_peered":"2024-04-25T08:43:34.677148+0000","last_unstale":"2024-04-25T08:43:36.682987+0000","last_undegraded":"2024-04-25T08:43:36.682987+0000","last_fullsized":"2024-04-25T08:43:36.682987+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.992651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683703+0000","last_change":"2024-04-25T08:43:35.702343+0000","last_active":"2024-04-25T08:43:36.683703+0000","last_peered":"2024-04-25T08:43:36.683703+0000","last_clean":"2024-04-25T08:43:36.683703+0000","last_became_active":"2024-04-25T08:43:35.701631+0000","last_became_peered":"2024-04-25T08:43:35.701631+0000","last_unstale":"2024-04-25T08:43:36.683703+0000","last_undegraded":"2024-04-25T08:43:36.683703+0000","last_fullsized":"2024-04-25T08:43:36.683703+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:07.955819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684397+0000","last_change":"2024-04-25T08:43:34.682629+0000","last_active":"2024-04-25T08:43:36.684397+0000","last_peered":"2024-04-25T08:43:36.684397+0000","last_clean":"2024-04-25T08:43:36.684397+0000","last_became_active":"2024-04-25T08:43:34.682289+0000","last_became_peered":"2024-04-25T08:43:34.682289+0000","last_unstale":"2024-04-25T08:43:36.684397+0000","last_undegraded":"2024-04-25T08:43:36.684397+0000","last_fullsized":"2024-04-25T08:43:36.684397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:40.654628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683671+0000","last_change":"2024-04-25T08:43:35.695103+0000","last_active":"2024-04-25T08:43:36.683671+0000","last_peered":"2024-04-25T08:43:36.683671+0000","last_clean":"2024-04-25T08:43:36.683671+0000","last_became_active":"2024-04-25T08:43:35.694309+0000","last_became_peered":"2024-04-25T08:43:35.694309+0000","last_unstale":"2024-04-25T08:43:36.683671+0000","last_undegraded":"2024-04-25T08:43:36.683671+0000","last_fullsized":"2024-04-25T08:43:36.683671+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:54.574871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.686693+0000","last_change":"2024-04-25T08:43:34.690955+0000","last_active":"2024-04-25T08:43:35.686693+0000","last_peered":"2024-04-25T08:43:35.686693+0000","last_clean":"2024-04-25T08:43:35.686693+0000","last_became_active":"2024-04-25T08:43:34.690678+0000","last_became_peered":"2024-04-25T08:43:34.690678+0000","last_unstale":"2024-04-25T08:43:35.686693+0000","last_undegraded":"2024-04-25T08:43:35.686693+0000","last_fullsized":"2024-04-25T08:43:35.686693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:27.895523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.711204+0000","last_change":"2024-04-25T08:43:35.707226+0000","last_active":"2024-04-25T08:43:35.711204+0000","last_peered":"2024-04-25T08:43:35.711204+0000","last_clean":"2024-04-25T08:43:35.711204+0000","last_became_active":"2024-04-25T08:43:35.706881+0000","last_became_peered":"2024-04-25T08:43:35.706881+0000","last_unstale":"2024-04-25T08:43:35.711204+0000","last_undegraded":"2024-04-25T08:43:35.711204+0000","last_fullsized":"2024-04-25T08:43:35.711204+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:10.828438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.707692+0000","last_change":"2024-04-25T08:43:34.690935+0000","last_active":"2024-04-25T08:43:35.707692+0000","last_peered":"2024-04-25T08:43:35.707692+0000","last_clean":"2024-04-25T08:43:35.707692+0000","last_became_active":"2024-04-25T08:43:34.690284+0000","last_became_peered":"2024-04-25T08:43:34.690284+0000","last_unstale":"2024-04-25T08:43:35.707692+0000","last_undegraded":"2024-04-25T08:43:35.707692+0000","last_fullsized":"2024-04-25T08:43:35.707692+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:09.820732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684827+0000","last_change":"2024-04-25T08:43:35.691933+0000","last_active":"2024-04-25T08:43:36.684827+0000","last_peered":"2024-04-25T08:43:36.684827+0000","last_clean":"2024-04-25T08:43:36.684827+0000","last_became_active":"2024-04-25T08:43:35.691645+0000","last_became_peered":"2024-04-25T08:43:35.691645+0000","last_unstale":"2024-04-25T08:43:36.684827+0000","last_undegraded":"2024-04-25T08:43:36.684827+0000","last_fullsized":"2024-04-25T08:43:36.684827+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:12.608221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":20,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683954+0000","last_change":"2024-04-25T08:43:34.681753+0000","last_active":"2024-04-25T08:43:36.683954+0000","last_peered":"2024-04-25T08:43:36.683954+0000","last_clean":"2024-04-25T08:43:36.683954+0000","last_became_active":"2024-04-25T08:43:34.680960+0000","last_became_peered":"2024-04-25T08:43:34.680960+0000","last_unstale":"2024-04-25T08:43:36.683954+0000","last_undegraded":"2024-04-25T08:43:36.683954+0000","last_fullsized":"2024-04-25T08:43:36.683954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:14.418825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712634+0000","last_change":"2024-04-25T08:43:35.711374+0000","last_active":"2024-04-25T08:43:35.712634+0000","last_peered":"2024-04-25T08:43:35.712634+0000","last_clean":"2024-04-25T08:43:35.712634+0000","last_became_active":"2024-04-25T08:43:35.711083+0000","last_became_peered":"2024-04-25T08:43:35.711083+0000","last_unstale":"2024-04-25T08:43:35.712634+0000","last_undegraded":"2024-04-25T08:43:35.712634+0000","last_fullsized":"2024-04-25T08:43:35.712634+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:15.426531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683625+0000","last_change":"2024-04-25T08:43:34.686447+0000","last_active":"2024-04-25T08:43:36.683625+0000","last_peered":"2024-04-25T08:43:36.683625+0000","last_clean":"2024-04-25T08:43:36.683625+0000","last_became_active":"2024-04-25T08:43:34.685801+0000","last_became_peered":"2024-04-25T08:43:34.685801+0000","last_unstale":"2024-04-25T08:43:36.683625+0000","last_undegraded":"2024-04-25T08:43:36.683625+0000","last_fullsized":"2024-04-25T08:43:36.683625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:06.349595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682638+0000","last_change":"2024-04-25T08:43:35.703698+0000","last_active":"2024-04-25T08:43:36.682638+0000","last_peered":"2024-04-25T08:43:36.682638+0000","last_clean":"2024-04-25T08:43:36.682638+0000","last_became_active":"2024-04-25T08:43:35.703402+0000","last_became_peered":"2024-04-25T08:43:35.703402+0000","last_unstale":"2024-04-25T08:43:36.682638+0000","last_undegraded":"2024-04-25T08:43:36.682638+0000","last_fullsized":"2024-04-25T08:43:36.682638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:42.497662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820272+0000","last_change":"2024-04-25T08:43:34.681941+0000","last_active":"2024-04-25T08:43:36.820272+0000","last_peered":"2024-04-25T08:43:36.820272+0000","last_clean":"2024-04-25T08:43:36.820272+0000","last_became_active":"2024-04-25T08:43:34.681666+0000","last_became_peered":"2024-04-25T08:43:34.681666+0000","last_unstale":"2024-04-25T08:43:36.820272+0000","last_undegraded":"2024-04-25T08:43:36.820272+0000","last_fullsized":"2024-04-25T08:43:36.820272+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.992651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683964+0000","last_change":"2024-04-25T08:43:35.693452+0000","last_active":"2024-04-25T08:43:36.683964+0000","last_peered":"2024-04-25T08:43:36.683964+0000","last_clean":"2024-04-25T08:43:36.683964+0000","last_became_active":"2024-04-25T08:43:35.693105+0000","last_became_peered":"2024-04-25T08:43:35.693105+0000","last_unstale":"2024-04-25T08:43:36.683964+0000","last_undegraded":"2024-04-25T08:43:36.683964+0000","last_fullsized":"2024-04-25T08:43:36.683964+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:42.497662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709680+0000","last_change":"2024-04-25T08:43:34.676749+0000","last_active":"2024-04-25T08:43:36.709680+0000","last_peered":"2024-04-25T08:43:36.709680+0000","last_clean":"2024-04-25T08:43:36.709680+0000","last_became_active":"2024-04-25T08:43:34.676193+0000","last_became_peered":"2024-04-25T08:43:34.676193+0000","last_unstale":"2024-04-25T08:43:36.709680+0000","last_undegraded":"2024-04-25T08:43:36.709680+0000","last_fullsized":"2024-04-25T08:43:36.709680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:30.462862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683345+0000","last_change":"2024-04-25T08:43:35.702991+0000","last_active":"2024-04-25T08:43:36.683345+0000","last_peered":"2024-04-25T08:43:36.683345+0000","last_clean":"2024-04-25T08:43:36.683345+0000","last_became_active":"2024-04-25T08:43:35.702728+0000","last_became_peered":"2024-04-25T08:43:35.702728+0000","last_unstale":"2024-04-25T08:43:36.683345+0000","last_undegraded":"2024-04-25T08:43:36.683345+0000","last_fullsized":"2024-04-25T08:43:36.683345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:30.859525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":22,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683386+0000","last_change":"2024-04-25T08:43:34.686860+0000","last_active":"2024-04-25T08:43:36.683386+0000","last_peered":"2024-04-25T08:43:36.683386+0000","last_clean":"2024-04-25T08:43:36.683386+0000","last_became_active":"2024-04-25T08:43:34.686567+0000","last_became_peered":"2024-04-25T08:43:34.686567+0000","last_unstale":"2024-04-25T08:43:36.683386+0000","last_undegraded":"2024-04-25T08:43:36.683386+0000","last_fullsized":"2024-04-25T08:43:36.683386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:52.768221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683855+0000","last_change":"2024-04-25T08:43:35.675041+0000","last_active":"2024-04-25T08:43:36.683855+0000","last_peered":"2024-04-25T08:43:36.683855+0000","last_clean":"2024-04-25T08:43:36.683855+0000","last_became_active":"2024-04-25T08:43:35.674671+0000","last_became_peered":"2024-04-25T08:43:35.674671+0000","last_unstale":"2024-04-25T08:43:36.683855+0000","last_undegraded":"2024-04-25T08:43:36.683855+0000","last_fullsized":"2024-04-25T08:43:36.683855+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:10.178193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708650+0000","last_change":"2024-04-25T08:43:34.684965+0000","last_active":"2024-04-25T08:43:35.708650+0000","last_peered":"2024-04-25T08:43:35.708650+0000","last_clean":"2024-04-25T08:43:35.708650+0000","last_became_active":"2024-04-25T08:43:34.684707+0000","last_became_peered":"2024-04-25T08:43:34.684707+0000","last_unstale":"2024-04-25T08:43:35.708650+0000","last_undegraded":"2024-04-25T08:43:35.708650+0000","last_fullsized":"2024-04-25T08:43:35.708650+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:59.195272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708602+0000","last_change":"2024-04-25T08:43:35.701347+0000","last_active":"2024-04-25T08:43:35.708602+0000","last_peered":"2024-04-25T08:43:35.708602+0000","last_clean":"2024-04-25T08:43:35.708602+0000","last_became_active":"2024-04-25T08:43:35.701051+0000","last_became_peered":"2024-04-25T08:43:35.701051+0000","last_unstale":"2024-04-25T08:43:35.708602+0000","last_undegraded":"2024-04-25T08:43:35.708602+0000","last_fullsized":"2024-04-25T08:43:35.708602+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:20.851391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684899+0000","last_change":"2024-04-25T08:43:34.685675+0000","last_active":"2024-04-25T08:43:36.684899+0000","last_peered":"2024-04-25T08:43:36.684899+0000","last_clean":"2024-04-25T08:43:36.684899+0000","last_became_active":"2024-04-25T08:43:34.684712+0000","last_became_peered":"2024-04-25T08:43:34.684712+0000","last_unstale":"2024-04-25T08:43:36.684899+0000","last_undegraded":"2024-04-25T08:43:36.684899+0000","last_fullsized":"2024-04-25T08:43:36.684899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:51.225447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684813+0000","last_change":"2024-04-25T08:43:35.689495+0000","last_active":"2024-04-25T08:43:36.684813+0000","last_peered":"2024-04-25T08:43:36.684813+0000","last_clean":"2024-04-25T08:43:36.684813+0000","last_became_active":"2024-04-25T08:43:35.688991+0000","last_became_peered":"2024-04-25T08:43:35.688991+0000","last_unstale":"2024-04-25T08:43:36.684813+0000","last_undegraded":"2024-04-25T08:43:36.684813+0000","last_fullsized":"2024-04-25T08:43:36.684813+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:06.474434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683831+0000","last_change":"2024-04-25T08:43:34.691860+0000","last_active":"2024-04-25T08:43:36.683831+0000","last_peered":"2024-04-25T08:43:36.683831+0000","last_clean":"2024-04-25T08:43:36.683831+0000","last_became_active":"2024-04-25T08:43:34.691554+0000","last_became_peered":"2024-04-25T08:43:34.691554+0000","last_unstale":"2024-04-25T08:43:36.683831+0000","last_undegraded":"2024-04-25T08:43:36.683831+0000","last_fullsized":"2024-04-25T08:43:36.683831+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:02.177817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683992+0000","last_change":"2024-04-25T08:43:35.686878+0000","last_active":"2024-04-25T08:43:36.683992+0000","last_peered":"2024-04-25T08:43:36.683992+0000","last_clean":"2024-04-25T08:43:36.683992+0000","last_became_active":"2024-04-25T08:43:35.686621+0000","last_became_peered":"2024-04-25T08:43:35.686621+0000","last_unstale":"2024-04-25T08:43:36.683992+0000","last_undegraded":"2024-04-25T08:43:36.683992+0000","last_fullsized":"2024-04-25T08:43:36.683992+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:31.470568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820452+0000","last_change":"2024-04-25T08:43:34.677144+0000","last_active":"2024-04-25T08:43:36.820452+0000","last_peered":"2024-04-25T08:43:36.820452+0000","last_clean":"2024-04-25T08:43:36.820452+0000","last_became_active":"2024-04-25T08:43:34.676233+0000","last_became_peered":"2024-04-25T08:43:34.676233+0000","last_unstale":"2024-04-25T08:43:36.820452+0000","last_undegraded":"2024-04-25T08:43:36.820452+0000","last_fullsized":"2024-04-25T08:43:36.820452+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:52.768221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683743+0000","last_change":"2024-04-25T08:43:35.698244+0000","last_active":"2024-04-25T08:43:36.683743+0000","last_peered":"2024-04-25T08:43:36.683743+0000","last_clean":"2024-04-25T08:43:36.683743+0000","last_became_active":"2024-04-25T08:43:35.697920+0000","last_became_peered":"2024-04-25T08:43:35.697920+0000","last_unstale":"2024-04-25T08:43:36.683743+0000","last_undegraded":"2024-04-25T08:43:36.683743+0000","last_fullsized":"2024-04-25T08:43:36.683743+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:09.195652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683021+0000","last_change":"2024-04-25T08:43:34.691363+0000","last_active":"2024-04-25T08:43:36.683021+0000","last_peered":"2024-04-25T08:43:36.683021+0000","last_clean":"2024-04-25T08:43:36.683021+0000","last_became_active":"2024-04-25T08:43:34.691032+0000","last_became_peered":"2024-04-25T08:43:34.691032+0000","last_unstale":"2024-04-25T08:43:36.683021+0000","last_undegraded":"2024-04-25T08:43:36.683021+0000","last_fullsized":"2024-04-25T08:43:36.683021+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:58.410307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.712605+0000","last_change":"2024-04-25T08:43:35.710875+0000","last_active":"2024-04-25T08:43:35.712605+0000","last_peered":"2024-04-25T08:43:35.712605+0000","last_clean":"2024-04-25T08:43:35.712605+0000","last_became_active":"2024-04-25T08:43:35.708402+0000","last_became_peered":"2024-04-25T08:43:35.708402+0000","last_unstale":"2024-04-25T08:43:35.712605+0000","last_undegraded":"2024-04-25T08:43:35.712605+0000","last_fullsized":"2024-04-25T08:43:35.712605+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:07.357301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683213+0000","last_change":"2024-04-25T08:43:34.690466+0000","last_active":"2024-04-25T08:43:36.683213+0000","last_peered":"2024-04-25T08:43:36.683213+0000","last_clean":"2024-04-25T08:43:36.683213+0000","last_became_active":"2024-04-25T08:43:34.690193+0000","last_became_peered":"2024-04-25T08:43:34.690193+0000","last_unstale":"2024-04-25T08:43:36.683213+0000","last_undegraded":"2024-04-25T08:43:36.683213+0000","last_fullsized":"2024-04-25T08:43:36.683213+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:48.869636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708574+0000","last_change":"2024-04-25T08:43:35.701972+0000","last_active":"2024-04-25T08:43:35.708574+0000","last_peered":"2024-04-25T08:43:35.708574+0000","last_clean":"2024-04-25T08:43:35.708574+0000","last_became_active":"2024-04-25T08:43:35.697873+0000","last_became_peered":"2024-04-25T08:43:35.697873+0000","last_unstale":"2024-04-25T08:43:35.708574+0000","last_undegraded":"2024-04-25T08:43:35.708574+0000","last_fullsized":"2024-04-25T08:43:35.708574+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:12.608221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683777+0000","last_change":"2024-04-25T08:43:34.685145+0000","last_active":"2024-04-25T08:43:36.683777+0000","last_peered":"2024-04-25T08:43:36.683777+0000","last_clean":"2024-04-25T08:43:36.683777+0000","last_became_active":"2024-04-25T08:43:34.684876+0000","last_became_peered":"2024-04-25T08:43:34.684876+0000","last_unstale":"2024-04-25T08:43:36.683777+0000","last_undegraded":"2024-04-25T08:43:36.683777+0000","last_fullsized":"2024-04-25T08:43:36.683777+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:36.494719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683526+0000","last_change":"2024-04-25T08:43:35.698345+0000","last_active":"2024-04-25T08:43:36.683526+0000","last_peered":"2024-04-25T08:43:36.683526+0000","last_clean":"2024-04-25T08:43:36.683526+0000","last_became_active":"2024-04-25T08:43:35.698029+0000","last_became_peered":"2024-04-25T08:43:35.698029+0000","last_unstale":"2024-04-25T08:43:36.683526+0000","last_undegraded":"2024-04-25T08:43:36.683526+0000","last_fullsized":"2024-04-25T08:43:36.683526+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:48.007401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682858+0000","last_change":"2024-04-25T08:43:34.692862+0000","last_active":"2024-04-25T08:43:36.682858+0000","last_peered":"2024-04-25T08:43:36.682858+0000","last_clean":"2024-04-25T08:43:36.682858+0000","last_became_active":"2024-04-25T08:43:34.692617+0000","last_became_peered":"2024-04-25T08:43:34.692617+0000","last_unstale":"2024-04-25T08:43:36.682858+0000","last_undegraded":"2024-04-25T08:43:36.682858+0000","last_fullsized":"2024-04-25T08:43:36.682858+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:46.405208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820505+0000","last_change":"2024-04-25T08:43:35.709834+0000","last_active":"2024-04-25T08:43:36.820505+0000","last_peered":"2024-04-25T08:43:36.820505+0000","last_clean":"2024-04-25T08:43:36.820505+0000","last_became_active":"2024-04-25T08:43:35.709601+0000","last_became_peered":"2024-04-25T08:43:35.709601+0000","last_unstale":"2024-04-25T08:43:36.820505+0000","last_undegraded":"2024-04-25T08:43:36.820505+0000","last_fullsized":"2024-04-25T08:43:36.820505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:20.851391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708030+0000","last_change":"2024-04-25T08:43:34.690387+0000","last_active":"2024-04-25T08:43:35.708030+0000","last_peered":"2024-04-25T08:43:35.708030+0000","last_clean":"2024-04-25T08:43:35.708030+0000","last_became_active":"2024-04-25T08:43:34.690071+0000","last_became_peered":"2024-04-25T08:43:34.690071+0000","last_unstale":"2024-04-25T08:43:35.708030+0000","last_undegraded":"2024-04-25T08:43:35.708030+0000","last_fullsized":"2024-04-25T08:43:35.708030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:06.349595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683825+0000","last_change":"2024-04-25T08:43:35.696198+0000","last_active":"2024-04-25T08:43:36.683825+0000","last_peered":"2024-04-25T08:43:36.683825+0000","last_clean":"2024-04-25T08:43:36.683825+0000","last_became_active":"2024-04-25T08:43:35.693991+0000","last_became_peered":"2024-04-25T08:43:35.693991+0000","last_unstale":"2024-04-25T08:43:36.683825+0000","last_undegraded":"2024-04-25T08:43:36.683825+0000","last_fullsized":"2024-04-25T08:43:36.683825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:56.146218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683500+0000","last_change":"2024-04-25T08:43:34.688617+0000","last_active":"2024-04-25T08:43:36.683500+0000","last_peered":"2024-04-25T08:43:36.683500+0000","last_clean":"2024-04-25T08:43:36.683500+0000","last_became_active":"2024-04-25T08:43:34.688365+0000","last_became_peered":"2024-04-25T08:43:34.688365+0000","last_unstale":"2024-04-25T08:43:36.683500+0000","last_undegraded":"2024-04-25T08:43:36.683500+0000","last_fullsized":"2024-04-25T08:43:36.683500+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:10.079374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684190+0000","last_change":"2024-04-25T08:43:35.694335+0000","last_active":"2024-04-25T08:43:36.684190+0000","last_peered":"2024-04-25T08:43:36.684190+0000","last_clean":"2024-04-25T08:43:36.684190+0000","last_became_active":"2024-04-25T08:43:35.693837+0000","last_became_peered":"2024-04-25T08:43:35.693837+0000","last_unstale":"2024-04-25T08:43:36.684190+0000","last_undegraded":"2024-04-25T08:43:36.684190+0000","last_fullsized":"2024-04-25T08:43:36.684190+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.761157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.692614+0000","last_change":"2024-04-25T08:43:34.690787+0000","last_active":"2024-04-25T08:43:35.692614+0000","last_peered":"2024-04-25T08:43:35.692614+0000","last_clean":"2024-04-25T08:43:35.692614+0000","last_became_active":"2024-04-25T08:43:34.690512+0000","last_became_peered":"2024-04-25T08:43:34.690512+0000","last_unstale":"2024-04-25T08:43:35.692614+0000","last_undegraded":"2024-04-25T08:43:35.692614+0000","last_fullsized":"2024-04-25T08:43:35.692614+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:52.768221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.708328+0000","last_change":"2024-04-25T08:43:35.697804+0000","last_active":"2024-04-25T08:43:35.708328+0000","last_peered":"2024-04-25T08:43:35.708328+0000","last_clean":"2024-04-25T08:43:35.708328+0000","last_became_active":"2024-04-25T08:43:35.697433+0000","last_became_peered":"2024-04-25T08:43:35.697433+0000","last_unstale":"2024-04-25T08:43:35.708328+0000","last_undegraded":"2024-04-25T08:43:35.708328+0000","last_fullsized":"2024-04-25T08:43:35.708328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:52.233153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.698119+0000","last_change":"2024-04-25T08:43:34.686768+0000","last_active":"2024-04-25T08:43:35.698119+0000","last_peered":"2024-04-25T08:43:35.698119+0000","last_clean":"2024-04-25T08:43:35.698119+0000","last_became_active":"2024-04-25T08:43:34.686453+0000","last_became_peered":"2024-04-25T08:43:34.686453+0000","last_unstale":"2024-04-25T08:43:35.698119+0000","last_undegraded":"2024-04-25T08:43:35.698119+0000","last_fullsized":"2024-04-25T08:43:35.698119+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.567248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.820330+0000","last_change":"2024-04-25T08:43:35.693888+0000","last_active":"2024-04-25T08:43:36.820330+0000","last_peered":"2024-04-25T08:43:36.820330+0000","last_clean":"2024-04-25T08:43:36.820330+0000","last_became_active":"2024-04-25T08:43:35.693586+0000","last_became_peered":"2024-04-25T08:43:35.693586+0000","last_unstale":"2024-04-25T08:43:36.820330+0000","last_undegraded":"2024-04-25T08:43:36.820330+0000","last_fullsized":"2024-04-25T08:43:36.820330+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:43.958244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.709705+0000","last_change":"2024-04-25T08:43:34.683378+0000","last_active":"2024-04-25T08:43:36.709705+0000","last_peered":"2024-04-25T08:43:36.709705+0000","last_clean":"2024-04-25T08:43:36.709705+0000","last_became_active":"2024-04-25T08:43:34.681953+0000","last_became_peered":"2024-04-25T08:43:34.681953+0000","last_unstale":"2024-04-25T08:43:36.709705+0000","last_undegraded":"2024-04-25T08:43:36.709705+0000","last_fullsized":"2024-04-25T08:43:36.709705+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:39.155803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.684290+0000","last_change":"2024-04-25T08:43:35.687330+0000","last_active":"2024-04-25T08:43:36.684290+0000","last_peered":"2024-04-25T08:43:36.684290+0000","last_clean":"2024-04-25T08:43:36.684290+0000","last_became_active":"2024-04-25T08:43:35.687067+0000","last_became_peered":"2024-04-25T08:43:35.687067+0000","last_unstale":"2024-04-25T08:43:36.684290+0000","last_undegraded":"2024-04-25T08:43:36.684290+0000","last_fullsized":"2024-04-25T08:43:36.684290+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:43.958244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682946+0000","last_change":"2024-04-25T08:43:34.674852+0000","last_active":"2024-04-25T08:43:36.682946+0000","last_peered":"2024-04-25T08:43:36.682946+0000","last_clean":"2024-04-25T08:43:36.682946+0000","last_became_active":"2024-04-25T08:43:34.672852+0000","last_became_peered":"2024-04-25T08:43:34.672852+0000","last_unstale":"2024-04-25T08:43:36.682946+0000","last_undegraded":"2024-04-25T08:43:36.682946+0000","last_fullsized":"2024-04-25T08:43:36.682946+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:52.768221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683867+0000","last_change":"2024-04-25T08:43:35.694141+0000","last_active":"2024-04-25T08:43:36.683867+0000","last_peered":"2024-04-25T08:43:36.683867+0000","last_clean":"2024-04-25T08:43:36.683867+0000","last_became_active":"2024-04-25T08:43:35.693828+0000","last_became_peered":"2024-04-25T08:43:35.693828+0000","last_unstale":"2024-04-25T08:43:36.683867+0000","last_undegraded":"2024-04-25T08:43:36.683867+0000","last_fullsized":"2024-04-25T08:43:36.683867+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:47.203771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.682462+0000","last_change":"2024-04-25T08:43:34.684624+0000","last_active":"2024-04-25T08:43:36.682462+0000","last_peered":"2024-04-25T08:43:36.682462+0000","last_clean":"2024-04-25T08:43:36.682462+0000","last_became_active":"2024-04-25T08:43:34.683796+0000","last_became_peered":"2024-04-25T08:43:34.683796+0000","last_unstale":"2024-04-25T08:43:36.682462+0000","last_undegraded":"2024-04-25T08:43:36.682462+0000","last_fullsized":"2024-04-25T08:43:36.682462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:27.895523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T08:43:35.710117+0000","last_change":"2024-04-25T08:43:35.701576+0000","last_active":"2024-04-25T08:43:35.710117+0000","last_peered":"2024-04-25T08:43:35.710117+0000","last_clean":"2024-04-25T08:43:35.710117+0000","last_became_active":"2024-04-25T08:43:35.697618+0000","last_became_peered":"2024-04-25T08:43:35.697618+0000","last_unstale":"2024-04-25T08:43:35.710117+0000","last_undegraded":"2024-04-25T08:43:35.710117+0000","last_fullsized":"2024-04-25T08:43:35.710117+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:34.653790+0000","last_clean_scrub_stamp":"2024-04-25T08:43:34.653790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:06.474434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683297+0000","last_change":"2024-04-25T08:43:34.684309+0000","last_active":"2024-04-25T08:43:36.683297+0000","last_peered":"2024-04-25T08:43:36.683297+0000","last_clean":"2024-04-25T08:43:36.683297+0000","last_became_active":"2024-04-25T08:43:34.684035+0000","last_became_peered":"2024-04-25T08:43:34.684035+0000","last_unstale":"2024-04-25T08:43:36.683297+0000","last_undegraded":"2024-04-25T08:43:36.683297+0000","last_fullsized":"2024-04-25T08:43:36.683297+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:43:33.646083+0000","last_clean_scrub_stamp":"2024-04-25T08:43:33.646083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:24.992651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:43:36.683557+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:43:36.683557+0000","last_peered":"2024-04-25T08:43:36.683557+0000","last_clean":"2024-04-25T08:43:36.683557+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:43:36.683557+0000","last_undegraded":"2024-04-25T08:43:36.683557+0000","last_fullsized":"2024-04-25T08:43:36.683557+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":787194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":195726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":195256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":195748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":479232,"data_stored":203482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":195214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":195234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":785546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:43:40.184 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:43:40.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.379+0000 7f2127800640 1 Processor -- start 2024-04-25T08:43:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.379+0000 7f2127800640 1 -- start start 2024-04-25T08:43:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f2127800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21201440d0 0x7f21200cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f2127800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f212006f150 0x7f21200b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f2127800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f212015e440 con 0x7f2120143990 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f2127800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212015ed30 con 0x7f21201440d0 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f2127800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212015e6e0 con 0x7f212006f150 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f211fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21201440d0 0x7f21200cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f211fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21201440d0 0x7f21200cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46438/0 (socket says 172.21.15.46:46438) 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f211fe00640 1 -- 172.21.15.46:0/2764333349 learned_addr learned my addr 172.21.15.46:0/2764333349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f2124e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2120143990 0x7f212006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58354/0 (socket says 172.21.15.46:58354) 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f2125800640 1 --2- 172.21.15.46:0/2764333349 >> v2:172.21.15.82:3300/0 conn(0x7f212006f150 0x7f21200b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f211fe00640 1 -- 172.21.15.46:0/2764333349 >> v2:172.21.15.82:3300/0 conn(0x7f212006f150 msgr2=0x7f21200b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f211fe00640 1 --2- 172.21.15.46:0/2764333349 >> v2:172.21.15.82:3300/0 conn(0x7f212006f150 0x7f21200b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f211fe00640 1 -- 172.21.15.46:0/2764333349 >> v1:172.21.15.46:6789/0 conn(0x7f2120143990 legacy=0x7f212006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f211fe00640 1 -- 172.21.15.46:0/2764333349 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212015e9c0 con 0x7f21201440d0 2024-04-25T08:43:40.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f211fe00640 1 --2- 172.21.15.46:0/2764333349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21201440d0 0x7f21200cecc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f21200af320 tx=0x7f210c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b7df1dca564255 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:40.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.383+0000 7f211f400640 1 -- 172.21.15.46:0/2764333349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c0b0070 con 0x7f21201440d0 2024-04-25T08:43:40.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f211f400640 1 -- 172.21.15.46:0/2764333349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c0ab600 con 0x7f21201440d0 2024-04-25T08:43:40.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f211f400640 1 -- 172.21.15.46:0/2764333349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c0ab9d0 con 0x7f21201440d0 2024-04-25T08:43:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 -- 172.21.15.46:0/2764333349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21201440d0 msgr2=0x7f21200cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 --2- 172.21.15.46:0/2764333349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21201440d0 0x7f21200cecc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f21200af320 tx=0x7f210c01c520 comp rx=0 tx=0).stop 2024-04-25T08:43:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 -- 172.21.15.46:0/2764333349 shutdown_connections 2024-04-25T08:43:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 --2- 172.21.15.46:0/2764333349 >> v2:172.21.15.82:3300/0 conn(0x7f212006f150 0x7f21200b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 --2- 172.21.15.46:0/2764333349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21201440d0 0x7f21200cecc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 -- 172.21.15.46:0/2764333349 >> 172.21.15.46:0/2764333349 conn(0x7f212006ac10 msgr2=0x7f212006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 -- 172.21.15.46:0/2764333349 shutdown_connections 2024-04-25T08:43:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 -- 172.21.15.46:0/2764333349 wait complete. 2024-04-25T08:43:40.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 Processor -- start 2024-04-25T08:43:40.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 -- start start 2024-04-25T08:43:40.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212006f150 0x7f2120159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2120143990 0x7f2120133410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21201440d0 0x7f212015d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21200b35f0 con 0x7f212006f150 2024-04-25T08:43:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21200b3470 con 0x7f2120143990 2024-04-25T08:43:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2127800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21200b3770 con 0x7f21201440d0 2024-04-25T08:43:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2124e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212006f150 0x7f2120159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2124e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212006f150 0x7f2120159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46442/0 (socket says 172.21.15.46:46442) 2024-04-25T08:43:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.387+0000 7f2124e00640 1 -- 172.21.15.46:0/1151873681 learned_addr learned my addr 172.21.15.46:0/1151873681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f2124e00640 1 -- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21201440d0 msgr2=0x7f212015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f2124e00640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21201440d0 0x7f212015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f211fe00640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2120143990 0x7f2120133410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f2124e00640 1 -- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2120143990 msgr2=0x7f2120133410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f2124e00640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2120143990 0x7f2120133410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f2124e00640 1 -- 172.21.15.46:0/1151873681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2120133c60 con 0x7f212006f150 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f211fe00640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2120143990 0x7f2120133410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f2124e00640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212006f150 0x7f2120159940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f211400c420 tx=0x7f211400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f211cc00640 1 -- 172.21.15.46:0/1151873681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21140074e0 con 0x7f212006f150 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f211cc00640 1 -- 172.21.15.46:0/1151873681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2114007680 con 0x7f212006f150 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f211cc00640 1 -- 172.21.15.46:0/1151873681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114004a30 con 0x7f212006f150 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21200af510 con 0x7f212006f150 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21200afad0 con 0x7f212006f150 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2120069010 con 0x7f212006f150 2024-04-25T08:43:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f211cc00640 1 -- 172.21.15.46:0/1151873681 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f211401a020 con 0x7f212006f150 2024-04-25T08:43:40.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f211cc00640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2100066f60 0x7f2100069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f211cc00640 1 -- 172.21.15.46:0/1151873681 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2114084ab0 con 0x7f212006f150 2024-04-25T08:43:40.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.391+0000 7f211fe00640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2100066f60 0x7f2100069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.395+0000 7f211fe00640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2100066f60 0x7f2100069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f210c0ad000 tx=0x7f210c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:40.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.395+0000 7f211cc00640 1 -- 172.21.15.46:0/1151873681 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2114055890 con 0x7f212006f150 2024-04-25T08:43:40.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.619+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f212015a540 con 0x7f212006f150 2024-04-25T08:43:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.619+0000 7f211cc00640 1 -- 172.21.15.46:0/1151873681 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2114055190 con 0x7f212006f150 2024-04-25T08:43:40.614 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-25T08:43:40.614 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q== 2024-04-25T08:43:40.614 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-25T08:43:40.614 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-25T08:43:40.614 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:43:40.614 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2100066f60 msgr2=0x7f2100069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2100066f60 0x7f2100069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f210c0ad000 tx=0x7f210c008040 comp rx=0 tx=0).stop 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212006f150 msgr2=0x7f2120159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212006f150 0x7f2120159940 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f211400c420 tx=0x7f211400c8e0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 shutdown_connections 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2100066f60 0x7f2100069420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f21201440d0 0x7f212015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2120143990 0x7f2120133410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 --2- 172.21.15.46:0/1151873681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212006f150 0x7f2120159940 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 >> 172.21.15.46:0/1151873681 conn(0x7f212006ac10 msgr2=0x7f21200b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 shutdown_connections 2024-04-25T08:43:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.623+0000 7f2127800640 1 -- 172.21.15.46:0/1151873681 wait complete. 2024-04-25T08:43:40.635 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:40.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.783+0000 7f9216e00640 1 Processor -- start 2024-04-25T08:43:40.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.783+0000 7f9216e00640 1 -- start start 2024-04-25T08:43:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f9216e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f921015dbd0 0x7f92100b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f9216e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92100b5250 con 0x7f92100713a0 2024-04-25T08:43:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f9216e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92100b5530 con 0x7f92100b0c30 2024-04-25T08:43:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f9216e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92100b5af0 con 0x7f921015dbd0 2024-04-25T08:43:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f920fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f921015dbd0 0x7f92100b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f9214e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f92100713a0 0x7f921006c380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58368/0 (socket says 172.21.15.46:58368) 2024-04-25T08:43:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f9214e00640 1 -- 172.21.15.46:0/1045745569 learned_addr learned my addr 172.21.15.46:0/1045745569 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f920fe00640 1 -- 172.21.15.46:0/1045745569 >> v1:172.21.15.82:6789/0 conn(0x7f92100b0c30 legacy=0x7f9210070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f920fe00640 1 -- 172.21.15.46:0/1045745569 >> v1:172.21.15.46:6789/0 conn(0x7f92100713a0 legacy=0x7f921006c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f920fe00640 1 -- 172.21.15.46:0/1045745569 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100b5810 con 0x7f921015dbd0 2024-04-25T08:43:40.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.787+0000 7f920f400640 1 -- 172.21.15.46:0/1045745569 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355889876 0 0) 0x7f92100b5250 con 0x7f92100713a0 2024-04-25T08:43:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f920fe00640 1 --2- 172.21.15.46:0/1045745569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921015dbd0 0x7f92100b06f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f92000097c0 tx=0x7f920001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7e182f8cd5faf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:40.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f920f400640 1 -- 172.21.15.46:0/1045745569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000b0070 con 0x7f921015dbd0 2024-04-25T08:43:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f920f400640 1 -- 172.21.15.46:0/1045745569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92000ab600 con 0x7f921015dbd0 2024-04-25T08:43:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f920f400640 1 -- 172.21.15.46:0/1045745569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000ab9d0 con 0x7f921015dbd0 2024-04-25T08:43:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 -- 172.21.15.46:0/1045745569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921015dbd0 msgr2=0x7f92100b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 --2- 172.21.15.46:0/1045745569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921015dbd0 0x7f92100b06f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f92000097c0 tx=0x7f920001c460 comp rx=0 tx=0).stop 2024-04-25T08:43:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 -- 172.21.15.46:0/1045745569 shutdown_connections 2024-04-25T08:43:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 --2- 172.21.15.46:0/1045745569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921015dbd0 0x7f92100b06f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 -- 172.21.15.46:0/1045745569 >> 172.21.15.46:0/1045745569 conn(0x7f921006f4f0 msgr2=0x7f921006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 -- 172.21.15.46:0/1045745569 shutdown_connections 2024-04-25T08:43:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 -- 172.21.15.46:0/1045745569 wait complete. 2024-04-25T08:43:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 Processor -- start 2024-04-25T08:43:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 -- start start 2024-04-25T08:43:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.791+0000 7f9216e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92100713a0 0x7f92101609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9216e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92100b0c30 0x7f921015fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9216e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921015dbd0 0x7f9210160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9216e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921016b650 con 0x7f92100713a0 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9216e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921006d570 con 0x7f92100b0c30 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9216e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921016b7d0 con 0x7f921015dbd0 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92100713a0 0x7f92101609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92100713a0 0x7f92101609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46458/0 (socket says 172.21.15.46:46458) 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920fe00640 1 -- 172.21.15.46:0/42517695 learned_addr learned my addr 172.21.15.46:0/42517695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9207e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92100b0c30 0x7f921015fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9214e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921015dbd0 0x7f9210160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920fe00640 1 -- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921015dbd0 msgr2=0x7f9210160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920fe00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921015dbd0 0x7f9210160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920fe00640 1 -- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92100b0c30 msgr2=0x7f921015fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920fe00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92100b0c30 0x7f921015fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920fe00640 1 -- 172.21.15.46:0/42517695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921014fb50 con 0x7f92100713a0 2024-04-25T08:43:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9207e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92100b0c30 0x7f921015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9214e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921015dbd0 0x7f9210160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920fe00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92100713a0 0x7f92101609a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f92000ae040 tx=0x7f920001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920cc00640 1 -- 172.21.15.46:0/42517695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000bd070 con 0x7f92100713a0 2024-04-25T08:43:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920cc00640 1 -- 172.21.15.46:0/42517695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9200004230 con 0x7f92100713a0 2024-04-25T08:43:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920cc00640 1 -- 172.21.15.46:0/42517695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000b0040 con 0x7f92100713a0 2024-04-25T08:43:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921015f560 con 0x7f92100713a0 2024-04-25T08:43:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921015f280 con 0x7f92100713a0 2024-04-25T08:43:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9210069080 con 0x7f92100713a0 2024-04-25T08:43:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.795+0000 7f920cc00640 1 -- 172.21.15.46:0/42517695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92000c0070 con 0x7f92100713a0 2024-04-25T08:43:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.799+0000 7f920cc00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91ec066f60 0x7f91ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.799+0000 7f920cc00640 1 -- 172.21.15.46:0/42517695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f920011ef00 con 0x7f92100713a0 2024-04-25T08:43:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.799+0000 7f9207e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91ec066f60 0x7f91ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:40.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.799+0000 7f9207e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91ec066f60 0x7f91ec069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92100d04e0 tx=0x7f91f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.803+0000 7f920cc00640 1 -- 172.21.15.46:0/42517695 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9200016050 con 0x7f92100713a0 2024-04-25T08:43:40.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.999+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9210160ee0 con 0x7f92100713a0 2024-04-25T08:43:40.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:40.999+0000 7f920cc00640 1 -- 172.21.15.46:0/42517695 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f92000efce0 con 0x7f92100713a0 2024-04-25T08:43:40.994 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:40.994 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1079752579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1079752579},{"type":"v1","addr":"172.21.15.82:6836","nonce":1079752579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/106980078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":106980078},{"type":"v1","addr":"172.21.15.82:6837","nonce":106980078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1318225485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1318225485},{"type":"v1","addr":"172.21.15.46:6834","nonce":1318225485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:43:35.070722+0000","modified":"2024-04-25T08:43:38.080210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2025854544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2025854544},{"type":"v1","addr":"172.21.15.46:6835","nonce":2025854544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:43:40.994 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91ec066f60 msgr2=0x7f91ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91ec066f60 0x7f91ec069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92100d04e0 tx=0x7f91f8009040 comp rx=0 tx=0).stop 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92100713a0 msgr2=0x7f92101609a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92100713a0 0x7f92101609a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f92000ae040 tx=0x7f920001aca0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 shutdown_connections 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91ec066f60 0x7f91ec069420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921015dbd0 0x7f9210160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92100b0c30 0x7f921015fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 --2- 172.21.15.46:0/42517695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92100713a0 0x7f92101609a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 >> 172.21.15.46:0/42517695 conn(0x7f921006f4f0 msgr2=0x7f9210152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:40.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 shutdown_connections 2024-04-25T08:43:40.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.003+0000 7f9216e00640 1 -- 172.21.15.46:0/42517695 wait complete. 2024-04-25T08:43:41.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:43:35.070722+0000', 'modified': '2024-04-25T08:43:38.080210+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5295': {'gid': 5295, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/2025854544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2025854544}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2025854544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:43:41.417 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:43:41.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:43:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe080000640 1 Processor -- start 2024-04-25T08:43:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe080000640 1 -- start start 2024-04-25T08:43:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe080000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe078159230 0x7fe07815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe080000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cbeb0 0x7fe0780cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe080000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe0780cda80 0x7fe07815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe080000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780cc700 con 0x7fe078159230 2024-04-25T08:43:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe080000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780cc9e0 con 0x7fe0780cda80 2024-04-25T08:43:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe080000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780cccc0 con 0x7fe0780cbeb0 2024-04-25T08:43:41.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe078159230 0x7fe07815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe078159230 0x7fe07815bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46474/0 (socket says 172.21.15.46:46474) 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07d600640 1 -- 172.21.15.46:0/1506527581 learned_addr learned my addr 172.21.15.46:0/1506527581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07e000640 1 --2- 172.21.15.46:0/1506527581 >> v2:172.21.15.82:3300/0 conn(0x7fe0780cda80 0x7fe07815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07cc00640 1 --2- 172.21.15.46:0/1506527581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cbeb0 0x7fe0780cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07d600640 1 -- 172.21.15.46:0/1506527581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cbeb0 msgr2=0x7fe0780cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07d600640 1 --2- 172.21.15.46:0/1506527581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cbeb0 0x7fe0780cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07d600640 1 -- 172.21.15.46:0/1506527581 >> v2:172.21.15.82:3300/0 conn(0x7fe0780cda80 msgr2=0x7fe07815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07d600640 1 --2- 172.21.15.46:0/1506527581 >> v2:172.21.15.82:3300/0 conn(0x7fe0780cda80 0x7fe07815f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07d600640 1 -- 172.21.15.46:0/1506527581 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780ccfa0 con 0x7fe078159230 2024-04-25T08:43:41.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.571+0000 7fe07e000640 1 --2- 172.21.15.46:0/1506527581 >> v2:172.21.15.82:3300/0 conn(0x7fe0780cda80 0x7fe07815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe07d600640 1 --2- 172.21.15.46:0/1506527581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe078159230 0x7fe07815bdb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe07815c2f0 tx=0x7fe06c01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77f3c03e680d2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe077e00640 1 -- 172.21.15.46:0/1506527581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0b0070 con 0x7fe078159230 2024-04-25T08:43:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe077e00640 1 -- 172.21.15.46:0/1506527581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c0ab600 con 0x7fe078159230 2024-04-25T08:43:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe077e00640 1 -- 172.21.15.46:0/1506527581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0ab9d0 con 0x7fe078159230 2024-04-25T08:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe080000640 1 -- 172.21.15.46:0/1506527581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe078159230 msgr2=0x7fe07815bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe080000640 1 --2- 172.21.15.46:0/1506527581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe078159230 0x7fe07815bdb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe07815c2f0 tx=0x7fe06c01c1d0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe080000640 1 -- 172.21.15.46:0/1506527581 shutdown_connections 2024-04-25T08:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe080000640 1 --2- 172.21.15.46:0/1506527581 >> v2:172.21.15.82:3300/0 conn(0x7fe0780cda80 0x7fe07815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe080000640 1 --2- 172.21.15.46:0/1506527581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cbeb0 0x7fe0780cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe080000640 1 --2- 172.21.15.46:0/1506527581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe078159230 0x7fe07815bdb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe080000640 1 -- 172.21.15.46:0/1506527581 >> 172.21.15.46:0/1506527581 conn(0x7fe0780b12a0 msgr2=0x7fe0780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe080000640 1 -- 172.21.15.46:0/1506527581 shutdown_connections 2024-04-25T08:43:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.575+0000 7fe080000640 1 -- 172.21.15.46:0/1506527581 wait complete. 2024-04-25T08:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 Processor -- start 2024-04-25T08:43:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 -- start start 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cbeb0 0x7fe07813be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cda80 0x7fe07815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe078159230 0x7fe07813d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07815d200 con 0x7fe0780cbeb0 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780cc4a0 con 0x7fe078159230 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07815d380 con 0x7fe0780cda80 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cbeb0 0x7fe07813be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cbeb0 0x7fe07813be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46478/0 (socket says 172.21.15.46:46478) 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07d600640 1 -- 172.21.15.46:0/3085421017 learned_addr learned my addr 172.21.15.46:0/3085421017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07cc00640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cda80 0x7fe07815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07e000640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe078159230 0x7fe07813d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07d600640 1 -- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cda80 msgr2=0x7fe07815b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07d600640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cda80 0x7fe07815b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07d600640 1 -- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe078159230 msgr2=0x7fe07813d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07d600640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe078159230 0x7fe07813d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07d600640 1 -- 172.21.15.46:0/3085421017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe078157d10 con 0x7fe0780cbeb0 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07cc00640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cda80 0x7fe07815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe07d600640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cbeb0 0x7fe07813be90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe06c01ca70 tx=0x7fe06c01aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe075600640 1 -- 172.21.15.46:0/3085421017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0ba020 con 0x7fe0780cbeb0 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe075600640 1 -- 172.21.15.46:0/3085421017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c0b6640 con 0x7fe0780cbeb0 2024-04-25T08:43:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe075600640 1 -- 172.21.15.46:0/3085421017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0b0040 con 0x7fe0780cbeb0 2024-04-25T08:43:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 -- 172.21.15.46:0/3085421017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe078157ff0 con 0x7fe0780cbeb0 2024-04-25T08:43:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.579+0000 7fe080000640 1 -- 172.21.15.46:0/3085421017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07813c3d0 con 0x7fe0780cbeb0 2024-04-25T08:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.583+0000 7fe075600640 1 -- 172.21.15.46:0/3085421017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe06c0bd020 con 0x7fe0780cbeb0 2024-04-25T08:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.583+0000 7fe075600640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe058066f60 0x7fe058069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.583+0000 7fe075600640 1 -- 172.21.15.46:0/3085421017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe06c11f4f0 con 0x7fe0780cbeb0 2024-04-25T08:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.583+0000 7fe07cc00640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe058066f60 0x7fe058069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.583+0000 7fe0833e5000 1 Processor -- start 2024-04-25T08:43:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.583+0000 7fe0833e5000 1 -- start start 2024-04-25T08:43:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.583+0000 7fe07cc00640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe058066f60 0x7fe058069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe064004660 tx=0x7fe064008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cf7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 0x55efc6cade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6cae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6b915f0 con 0x55efc6cf7b70 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6af8da0 con 0x55efc6cae340 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efc6a81150 con 0x55efc6cf8490 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cf7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cf7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46490/0 (socket says 172.21.15.46:46490) 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07d600640 1 -- 172.21.15.46:0/2997269056 learned_addr learned my addr 172.21.15.46:0/2997269056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07cc00640 1 --2- 172.21.15.46:0/2997269056 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 0x55efc6cade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07e000640 1 --2- 172.21.15.46:0/2997269056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6cae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07d600640 1 -- 172.21.15.46:0/2997269056 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 msgr2=0x55efc6cade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07d600640 1 --2- 172.21.15.46:0/2997269056 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 0x55efc6cade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07d600640 1 -- 172.21.15.46:0/2997269056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 msgr2=0x55efc6cae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07d600640 1 --2- 172.21.15.46:0/2997269056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6cae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07d600640 1 -- 172.21.15.46:0/2997269056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efc6a9aca0 con 0x55efc6cf7b70 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe07d600640 1 --2- 172.21.15.46:0/2997269056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cf7f50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0efb60 tx=0x7fe06c0ebfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c2c2e7120a4fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe06a000640 1 -- 172.21.15.46:0/2997269056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0f3d20 con 0x55efc6cf7b70 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe06a000640 1 -- 172.21.15.46:0/2997269056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c0f82e0 con 0x55efc6cf7b70 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe06a000640 1 -- 172.21.15.46:0/2997269056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0ab4c0 con 0x55efc6cf7b70 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- 172.21.15.46:0/2997269056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 msgr2=0x55efc6cf7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- 172.21.15.46:0/2997269056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cf7f50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0efb60 tx=0x7fe06c0ebfa0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- 172.21.15.46:0/2997269056 shutdown_connections 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- 172.21.15.46:0/2997269056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6cae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- 172.21.15.46:0/2997269056 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 0x55efc6cade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- 172.21.15.46:0/2997269056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cf7f50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- 172.21.15.46:0/2997269056 >> 172.21.15.46:0/2997269056 conn(0x55efc6cb02a0 msgr2=0x55efc6ca82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- 172.21.15.46:0/2997269056 shutdown_connections 2024-04-25T08:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- 172.21.15.46:0/2997269056 wait complete. 2024-04-25T08:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- start start 2024-04-25T08:43:41.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 Processor -- start 2024-04-25T08:43:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6cfeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 0x55efc6d03d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6ab3990 con 0x55efc6cf7b70 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6af4e40 con 0x55efc6cae340 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.587+0000 7fe0833e5000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efc6a84900 con 0x55efc6cf8490 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 10 client.0 _ll_register_callbacks cb 0x55efc6bac1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cff410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46498/0 (socket says 172.21.15.46:46498) 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07cc00640 1 -- 172.21.15.46:0/543039452 learned_addr learned my addr 172.21.15.46:0/543039452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07d600640 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6cfeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07e000640 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 0x55efc6d03d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07cc00640 1 -- 172.21.15.46:0/543039452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 msgr2=0x55efc6d03d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07cc00640 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 0x55efc6d03d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07cc00640 1 -- 172.21.15.46:0/543039452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 msgr2=0x55efc6cfeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07cc00640 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6cfeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07cc00640 1 -- 172.21.15.46:0/543039452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efc6ab1630 con 0x55efc6cf7b70 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07e000640 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 0x55efc6d03d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07cc00640 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cff410 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe064014460 tx=0x7fe064004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7b51f44298382e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 1 -- 172.21.15.46:0/543039452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064025020 con 0x55efc6cf7b70 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 1 -- 172.21.15.46:0/543039452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe064021830 con 0x55efc6cf7b70 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 -- 172.21.15.46:0/543039452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe06c0f82e0 con 0x55efc6cf7b70 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 1 -- 172.21.15.46:0/543039452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe06402f050 con 0x55efc6cf7b70 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 10 client.5478 fetch_fsmap learned FSMap version 18 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 -- 172.21.15.46:0/543039452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55efc6a813a0 con 0x55efc6cf7b70 2024-04-25T08:43:41.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 1 -- 172.21.15.46:0/543039452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c0f82e0 con 0x55efc6cf7b70 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 1 -- 172.21.15.46:0/543039452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe064032020 con 0x55efc6cf7b70 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 10 client.5478 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 10 client.5478 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 20 client.5478 populate_metadata read hostname 'smithi046' 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] conn(0x55efc6d06aa0 0x55efc6d08e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 4 client.5478 mds_command: new command op to 5295 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 -- 172.21.15.46:0/543039452 --> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efc6a94a50 con 0x55efc6d06aa0 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07d600640 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] conn(0x55efc6d06aa0 0x55efc6d08e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe07d600640 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] conn(0x55efc6d06aa0 0x55efc6d08e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80d2f93d32af4b4 server_cookie=efc339f2b5199fab in_seq=0 out_seq=0 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 10 client.5478 ms_handle_connect on v2:172.21.15.46:6833/2025854544 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 1 -- 172.21.15.46:0/543039452 <== mds.0 v2:172.21.15.46:6833/2025854544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55efc6a94a50 con 0x55efc6d06aa0 2024-04-25T08:43:41.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 10 client.5478 handle_command_reply: tid=0 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 client.5478 shutdown 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 -- 172.21.15.46:0/543039452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 msgr2=0x55efc6cff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cff410 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe064014460 tx=0x7fe064004740 comp rx=0 tx=0).stop 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 -- 172.21.15.46:0/543039452 shutdown_connections 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] conn(0x55efc6d06aa0 0x55efc6d08e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf8490 0x55efc6d03d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cf7b70 0x55efc6cff410 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe05ea00640 0 client.5478 ms_handle_reset on v2:172.21.15.46:6833/2025854544 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 --2- 172.21.15.46:0/543039452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6cfeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 -- 172.21.15.46:0/543039452 >> 172.21.15.46:0/543039452 conn(0x55efc6cb02a0 msgr2=0x55efc6caf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 -- 172.21.15.46:0/543039452 shutdown_connections 2024-04-25T08:43:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 1 -- 172.21.15.46:0/543039452 wait complete. 2024-04-25T08:43:41.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.591+0000 7fe0833e5000 20 client.5478 trim_cache size 0 max 16384 2024-04-25T08:43:41.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.607+0000 7fe0833e5000 1 Processor -- start 2024-04-25T08:43:41.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.607+0000 7fe0833e5000 1 -- start start 2024-04-25T08:43:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cb02a0 0x55efc6d54140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cae340 0x55efc6d54680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cf7b70 0x55efc6d54bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780b3ac0 con 0x55efc6cae340 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe078161970 con 0x55efc6cf7b70 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe078161f70 con 0x55efc6cb02a0 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe07cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cae340 0x55efc6d54680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe07cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cae340 0x55efc6d54680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46500/0 (socket says 172.21.15.46:46500) 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe07cc00640 1 -- 172.21.15.46:0/580261023 learned_addr learned my addr 172.21.15.46:0/580261023 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe07cc00640 1 -- 172.21.15.46:0/580261023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cb02a0 msgr2=0x55efc6d54140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe07cc00640 1 --2- 172.21.15.46:0/580261023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cb02a0 0x55efc6d54140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe07cc00640 1 -- 172.21.15.46:0/580261023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cf7b70 msgr2=0x55efc6d54bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe07cc00640 1 --2- 172.21.15.46:0/580261023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cf7b70 0x55efc6d54bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe07cc00640 1 -- 172.21.15.46:0/580261023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efc6b7a600 con 0x55efc6cae340 2024-04-25T08:43:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe07cc00640 1 --2- 172.21.15.46:0/580261023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cae340 0x55efc6d54680 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe064021d30 tx=0x7fe064020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f453cc13855d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe069600640 1 -- 172.21.15.46:0/580261023 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064025050 con 0x55efc6cae340 2024-04-25T08:43:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe069600640 1 -- 172.21.15.46:0/580261023 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06401c040 con 0x55efc6cae340 2024-04-25T08:43:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe069600640 1 -- 172.21.15.46:0/580261023 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06401b070 con 0x55efc6cae340 2024-04-25T08:43:41.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 -- 172.21.15.46:0/580261023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cae340 msgr2=0x55efc6d54680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 --2- 172.21.15.46:0/580261023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cae340 0x55efc6d54680 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe064021d30 tx=0x7fe064020000 comp rx=0 tx=0).stop 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 -- 172.21.15.46:0/580261023 shutdown_connections 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 --2- 172.21.15.46:0/580261023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cf7b70 0x55efc6d54bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 --2- 172.21.15.46:0/580261023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cae340 0x55efc6d54680 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 --2- 172.21.15.46:0/580261023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cb02a0 0x55efc6d54140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 -- 172.21.15.46:0/580261023 >> 172.21.15.46:0/580261023 conn(0x55efc6cfee60 msgr2=0x55efc6ca8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 -- 172.21.15.46:0/580261023 shutdown_connections 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 -- 172.21.15.46:0/580261023 wait complete. 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 -- start start 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.611+0000 7fe0833e5000 1 Processor -- start 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6d3c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cb02a0 0x55efc6d3cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf7b70 0x55efc6d3d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6afd3d0 con 0x55efc6cb02a0 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc6bb4d50 con 0x55efc6cae340 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55efc6a93980 con 0x55efc6cf7b70 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 10 client.0 _ll_register_callbacks cb 0x7fe06c0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cb02a0 0x55efc6d3cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cb02a0 0x55efc6d3cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46506/0 (socket says 172.21.15.46:46506) 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07cc00640 1 -- 172.21.15.46:0/1519350390 learned_addr learned my addr 172.21.15.46:0/1519350390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07d600640 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6d3c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07e000640 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf7b70 0x55efc6d3d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07cc00640 1 -- 172.21.15.46:0/1519350390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf7b70 msgr2=0x55efc6d3d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07cc00640 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf7b70 0x55efc6d3d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07cc00640 1 -- 172.21.15.46:0/1519350390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 msgr2=0x55efc6d3c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07cc00640 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6d3c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07cc00640 1 -- 172.21.15.46:0/1519350390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efc6a8d9b0 con 0x55efc6cb02a0 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07e000640 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf7b70 0x55efc6d3d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07cc00640 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cb02a0 0x55efc6d3cf10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe064008010 tx=0x7fe064014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743c3ce4ec9488ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 1 -- 172.21.15.46:0/1519350390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064025040 con 0x55efc6cb02a0 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- 172.21.15.46:0/1519350390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe06401c040 con 0x55efc6cb02a0 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 1 -- 172.21.15.46:0/1519350390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe06403f990 con 0x55efc6cb02a0 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 1 -- 172.21.15.46:0/1519350390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe06400fbf0 con 0x55efc6cb02a0 2024-04-25T08:43:41.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 10 client.5484 fetch_fsmap learned FSMap version 18 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- 172.21.15.46:0/1519350390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55efc6aa00b0 con 0x55efc6cb02a0 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 1 -- 172.21.15.46:0/1519350390 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06401c040 con 0x55efc6cb02a0 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 1 -- 172.21.15.46:0/1519350390 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe064032040 con 0x55efc6cb02a0 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 10 client.5484 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 10 client.5484 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 20 client.5484 populate_metadata read hostname 'smithi046' 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] conn(0x55efc6cf8490 0x55efc6d3d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 4 client.5484 mds_command: new command op to 5295 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- 172.21.15.46:0/1519350390 --> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55efc6cae170 con 0x55efc6cf8490 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07d600640 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] conn(0x55efc6cf8490 0x55efc6d3d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe07d600640 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] conn(0x55efc6cf8490 0x55efc6d3d990 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d5c9c7c60db5e72 server_cookie=ce13b947a98854a in_seq=0 out_seq=0 2024-04-25T08:43:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 10 client.5484 ms_handle_connect on v2:172.21.15.46:6833/2025854544 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 1 -- 172.21.15.46:0/1519350390 <== mds.0 v2:172.21.15.46:6833/2025854544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55efc6cae170 con 0x55efc6cf8490 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 10 client.5484 handle_command_reply: tid=0 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 client.5484 shutdown 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- 172.21.15.46:0/1519350390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cb02a0 msgr2=0x55efc6d3cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cb02a0 0x55efc6d3cf10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe064008010 tx=0x7fe064014000 comp rx=0 tx=0).stop 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- 172.21.15.46:0/1519350390 shutdown_connections 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.46:6833/2025854544,v1:172.21.15.46:6835/2025854544] conn(0x55efc6cf8490 0x55efc6d3d990 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55efc6cf7b70 0x55efc6d3d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efc6cb02a0 0x55efc6d3cf10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe05ea00640 0 client.5484 ms_handle_reset on v2:172.21.15.46:6833/2025854544 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 --2- 172.21.15.46:0/1519350390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55efc6cae340 0x55efc6d3c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- 172.21.15.46:0/1519350390 >> 172.21.15.46:0/1519350390 conn(0x55efc6cfee60 msgr2=0x55efc6d06e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- 172.21.15.46:0/1519350390 shutdown_connections 2024-04-25T08:43:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 1 -- 172.21.15.46:0/1519350390 wait complete. 2024-04-25T08:43:41.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.615+0000 7fe0833e5000 20 client.5484 trim_cache size 0 max 16384 2024-04-25T08:43:41.615 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:43:41.615 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:43:41.615 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:43:41.615 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5295, 2024-04-25T08:43:41.615 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:43:41.616 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:43:41.616 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:43:41.616 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.9535839849999999, 2024-04-25T08:43:41.616 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 18, 2024-04-25T08:43:41.616 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 40, 2024-04-25T08:43:41.616 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 40, 2024-04-25T08:43:41.616 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 13.802913475 2024-04-25T08:43:41.616 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:43:41.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 -- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe058066f60 msgr2=0x7fe058069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe058066f60 0x7fe058069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe064004660 tx=0x7fe064008040 comp rx=0 tx=0).stop 2024-04-25T08:43:41.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 -- 172.21.15.46:0/3085421017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cbeb0 msgr2=0x7fe07813be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cbeb0 0x7fe07813be90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe06c01ca70 tx=0x7fe06c01aa10 comp rx=0 tx=0).stop 2024-04-25T08:43:41.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 -- 172.21.15.46:0/3085421017 shutdown_connections 2024-04-25T08:43:41.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe058066f60 0x7fe058069420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe078159230 0x7fe07813d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0780cda80 0x7fe07815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 --2- 172.21.15.46:0/3085421017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0780cbeb0 0x7fe07813be90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 -- 172.21.15.46:0/3085421017 >> 172.21.15.46:0/3085421017 conn(0x7fe0780b12a0 msgr2=0x7fe0780af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:41.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 -- 172.21.15.46:0/3085421017 shutdown_connections 2024-04-25T08:43:41.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.619+0000 7fe080000640 1 -- 172.21.15.46:0/3085421017 wait complete. 2024-04-25T08:43:41.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:43:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518f800640 1 Processor -- start 2024-04-25T08:43:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518f800640 1 -- start start 2024-04-25T08:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518f800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f51880ae360 0x7f518815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 0x7f518806ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518f800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5188161330 con 0x7f5188160f50 2024-04-25T08:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518f800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518806f390 con 0x7f51880ae360 2024-04-25T08:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518806d680 con 0x7f518806ea00 2024-04-25T08:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518ce00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5188160f50 0x7f51880adc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58382/0 (socket says 172.21.15.46:58382) 2024-04-25T08:43:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518ce00640 1 -- 172.21.15.46:0/3220098576 learned_addr learned my addr 172.21.15.46:0/3220098576 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f517fe00640 1 --2- 172.21.15.46:0/3220098576 >> v2:172.21.15.82:3300/0 conn(0x7f51880ae360 0x7f518815f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f518d800640 1 --2- 172.21.15.46:0/3220098576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 0x7f518806ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.779+0000 7f5187e00640 1 -- 172.21.15.46:0/3220098576 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415928041 0 0) 0x7f5188161330 con 0x7f5188160f50 2024-04-25T08:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f5187e00640 1 -- 172.21.15.46:0/3220098576 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f516c003530 con 0x7f5188160f50 2024-04-25T08:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f517fe00640 1 -- 172.21.15.46:0/3220098576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 msgr2=0x7f518806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f517fe00640 1 --2- 172.21.15.46:0/3220098576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 0x7f518806ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f517fe00640 1 -- 172.21.15.46:0/3220098576 >> v1:172.21.15.46:6789/0 conn(0x7f5188160f50 legacy=0x7f51880adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f517fe00640 1 -- 172.21.15.46:0/3220098576 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880ae810 con 0x7f51880ae360 2024-04-25T08:43:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f517fe00640 1 --2- 172.21.15.46:0/3220098576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51880ae360 0x7f518815f1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51700097c0 tx=0x7f517001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a5b3c66ee74f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f5187e00640 1 -- 172.21.15.46:0/3220098576 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51700a7070 con 0x7f51880ae360 2024-04-25T08:43:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f5187e00640 1 -- 172.21.15.46:0/3220098576 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51700b0070 con 0x7f51880ae360 2024-04-25T08:43:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f5187e00640 1 -- 172.21.15.46:0/3220098576 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51700ab890 con 0x7f51880ae360 2024-04-25T08:43:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f518f800640 1 -- 172.21.15.46:0/3220098576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51880ae360 msgr2=0x7f518815f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.783+0000 7f518f800640 1 --2- 172.21.15.46:0/3220098576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51880ae360 0x7f518815f1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f51700097c0 tx=0x7f517001c470 comp rx=0 tx=0).stop 2024-04-25T08:43:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 -- 172.21.15.46:0/3220098576 shutdown_connections 2024-04-25T08:43:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 --2- 172.21.15.46:0/3220098576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 0x7f518806ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 --2- 172.21.15.46:0/3220098576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51880ae360 0x7f518815f1a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 -- 172.21.15.46:0/3220098576 >> 172.21.15.46:0/3220098576 conn(0x7f51880b12a0 msgr2=0x7f51880b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 -- 172.21.15.46:0/3220098576 shutdown_connections 2024-04-25T08:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 -- 172.21.15.46:0/3220098576 wait complete. 2024-04-25T08:43:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 Processor -- start 2024-04-25T08:43:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 -- start start 2024-04-25T08:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 0x7f51881590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51880ae360 0x7f5188140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5188160f50 0x7f51881408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518806e300 con 0x7f5188160f50 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51880aedd0 con 0x7f51880ae360 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518806e480 con 0x7f518806ea00 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5188160f50 0x7f51881408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5188160f50 0x7f51881408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46512/0 (socket says 172.21.15.46:46512) 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518d800640 1 -- 172.21.15.46:0/1517388637 learned_addr learned my addr 172.21.15.46:0/1517388637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518ce00640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 0x7f51881590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f517fe00640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51880ae360 0x7f5188140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518d800640 1 -- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 msgr2=0x7f51881590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518d800640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 0x7f51881590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518d800640 1 -- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51880ae360 msgr2=0x7f5188140370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518d800640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51880ae360 0x7f5188140370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518d800640 1 -- 172.21.15.46:0/1517388637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880cded0 con 0x7f5188160f50 2024-04-25T08:43:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.787+0000 7f518d800640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5188160f50 0x7f51881408b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f518000efc0 tx=0x7f518000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f5185600640 1 -- 172.21.15.46:0/1517388637 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180018020 con 0x7f5188160f50 2024-04-25T08:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f5185600640 1 -- 172.21.15.46:0/1517388637 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5180014480 con 0x7f5188160f50 2024-04-25T08:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f5185600640 1 -- 172.21.15.46:0/1517388637 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180014870 con 0x7f5188160f50 2024-04-25T08:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51880ce1b0 con 0x7f5188160f50 2024-04-25T08:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51880ce770 con 0x7f5188160f50 2024-04-25T08:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5188069080 con 0x7f5188160f50 2024-04-25T08:43:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f5185600640 1 -- 172.21.15.46:0/1517388637 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f518001b020 con 0x7f5188160f50 2024-04-25T08:43:41.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f5185600640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5164066f90 0x7f5164069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:41.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f5185600640 1 -- 172.21.15.46:0/1517388637 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5180085ac0 con 0x7f5188160f50 2024-04-25T08:43:41.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.791+0000 7f518ce00640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5164066f90 0x7f5164069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:41.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.795+0000 7f518ce00640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5164066f90 0x7f5164069450 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5178007fd0 tx=0x7f5178009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:41.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.795+0000 7f5185600640 1 -- 172.21.15.46:0/1517388637 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5180022070 con 0x7f5188160f50 2024-04-25T08:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f51880b0e80 con 0x7f5188160f50 2024-04-25T08:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f5185600640 1 -- 172.21.15.46:0/1517388637 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5180010040 con 0x7f5188160f50 2024-04-25T08:43:41.991 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5164066f90 msgr2=0x7f5164069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5164066f90 0x7f5164069450 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5178007fd0 tx=0x7f5178009040 comp rx=0 tx=0).stop 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5188160f50 msgr2=0x7f51881408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5188160f50 0x7f51881408b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f518000efc0 tx=0x7f518000c490 comp rx=0 tx=0).stop 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 shutdown_connections 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5164066f90 0x7f5164069450 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5188160f50 0x7f51881408b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51880ae360 0x7f5188140370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 --2- 172.21.15.46:0/1517388637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f518806ea00 0x7f51881590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 >> 172.21.15.46:0/1517388637 conn(0x7f51880b12a0 msgr2=0x7f51881438f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 shutdown_connections 2024-04-25T08:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:41.999+0000 7f518f800640 1 -- 172.21.15.46:0/1517388637 wait complete. 2024-04-25T08:43:42.011 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:43:42.011 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:43:42.012 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:43:42.012 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:43:42.012 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:43:42.012 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:43:42.012 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:43:42.012 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:43:42.012 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:43:42.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.012 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:43:42.021 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:43:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:43:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:43:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:43:42.021 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42381sec preferred_lft 42381sec 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:43:42.022 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:43:42.023 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:43:42.024 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:43:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:43:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:43:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:43:42.024 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:43:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:43:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:43:42.024 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:43:42.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.025 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:43:42.083 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:43:42.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.085 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:43:42.138 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:43:42.139 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:43:42.139 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:43:42.139 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:43:42.139 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:43:42.139 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:43:42.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.139 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:43:42.139 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:43:42.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:43:42.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:43:42.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:43:42.140 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:43:42.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.235 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:43:42.244 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:43:42.244 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:43:42.244 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:43:42.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:43:42.306 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:43:42.306 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:43:42.357 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:43:42.357 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:43:42.357 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:43:42.405 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:43:42.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.405 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:43:42.495 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:43:42.495 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:43:42.495 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:43:42.510 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:43:42.510 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:43:42.511 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:43:42.511 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:43:42.511 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:43:42.522 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:43:42.523 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:43:42.523 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:43:42.523 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:43:42.523 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:43:42.523 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:43:42.523 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:43:42.523 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:43:42.526 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:43:42.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.526 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:43:42.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.548 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:43:42.607 INFO:teuthology.orchestra.run.smithi046.stdout:client5105 2024-04-25T08:43:42.607 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:42.607 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:42.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.658 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-25T08:43:42.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.721 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-25T08:43:42.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.772 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-25T08:43:42.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.828 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-25T08:43:42.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.887 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-25T08:43:42.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:42.939 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T08:43:42.997 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-04-25T08:43:42.997 INFO:teuthology.orchestra.run.smithi046.stdout:./rfile 2024-04-25T08:43:42.997 INFO:teuthology.orchestra.run.smithi046.stdout:./parentdir 2024-04-25T08:43:42.997 INFO:teuthology.orchestra.run.smithi046.stdout:./parentdir/pfile 2024-04-25T08:43:42.997 INFO:teuthology.orchestra.run.smithi046.stdout:./parentdir/childdir 2024-04-25T08:43:42.997 INFO:teuthology.orchestra.run.smithi046.stdout:./parentdir/childdir/cfile 2024-04-25T08:43:42.999 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:43:42.999 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:43:42.999 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:43:42.999 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:43:42.999 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T08:43:42.999 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:43:43.108 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:43:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:43:43.119 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:43:43.119 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:43:43.119 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:43:43.119 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:43:43.119 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-25T08:43:43.119 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:43:43.227 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-04-25T08:43:43.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:43:43.236 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:43:43.236 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:43:43.236 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:43:43.236 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:43:43.236 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-25T08:43:43.236 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:43:43.347 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-25T08:43:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:43:43.356 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:43:43.356 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:43:43.356 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:43:43.356 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:43:43.357 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-25T08:43:43.357 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:43:43.463 INFO:teuthology.orchestra.run.smithi046.stdout:1099511628279 2024-04-25T08:43:43.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:43:43.472 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:43:43.472 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:43:43.473 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:43:43.473 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:43:43.473 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-25T08:43:43.473 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:43:43.579 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-04-25T08:43:43.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:43:43.588 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:43:43.588 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:43:43.588 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:43:43.589 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:43:43.589 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-25T08:43:43.589 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:43:43.702 INFO:teuthology.orchestra.run.smithi046.stdout:1099511628280 2024-04-25T08:43:43.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:43:43.714 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:43.714 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:43.770 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:43.770 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:43.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.011+0000 7fd9aee00640 1 Processor -- start 2024-04-25T08:43:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd9aee00640 1 -- start start 2024-04-25T08:43:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd9aee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9a806f320 0x7fd9a8162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd9aee00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd9a80aed10 0x7fd9a8164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd9aee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a80b35b0 con 0x7fd9a815a750 2024-04-25T08:43:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd9aee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a815ab30 con 0x7fd9a806f320 2024-04-25T08:43:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd9aee00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a80ce4f0 con 0x7fd9a80aed10 2024-04-25T08:43:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd99f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9a806f320 0x7fd9a8162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd99f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9a806f320 0x7fd9a8162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46522/0 (socket says 172.21.15.46:46522) 2024-04-25T08:43:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd99f400640 1 -- 172.21.15.46:0/3974377297 learned_addr learned my addr 172.21.15.46:0/3974377297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd9ace00640 1 --2- 172.21.15.46:0/3974377297 >> v2:172.21.15.82:3300/0 conn(0x7fd9a80aed10 0x7fd9a8164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd99f400640 1 -- 172.21.15.46:0/3974377297 >> v2:172.21.15.82:3300/0 conn(0x7fd9a80aed10 msgr2=0x7fd9a8164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd99f400640 1 --2- 172.21.15.46:0/3974377297 >> v2:172.21.15.82:3300/0 conn(0x7fd9a80aed10 0x7fd9a8164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd99f400640 1 -- 172.21.15.46:0/3974377297 >> v1:172.21.15.82:6789/0 conn(0x7fd9a815a750 legacy=0x7fd9a806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd99f400640 1 -- 172.21.15.46:0/3974377297 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a806d5e0 con 0x7fd9a806f320 2024-04-25T08:43:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.015+0000 7fd9ace00640 1 --2- 172.21.15.46:0/3974377297 >> v2:172.21.15.82:3300/0 conn(0x7fd9a80aed10 0x7fd9a8164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:44.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd99f400640 1 --2- 172.21.15.46:0/3974377297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 0x7fd9a8162200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8070b00 tx=0x7fd99001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a242477cd4784f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:44.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9a7400640 1 -- 172.21.15.46:0/3974377297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9900b1070 con 0x7fd9a806f320 2024-04-25T08:43:44.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9a7400640 1 -- 172.21.15.46:0/3974377297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9900ac630 con 0x7fd9a806f320 2024-04-25T08:43:44.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9a7400640 1 -- 172.21.15.46:0/3974377297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9900a8050 con 0x7fd9a806f320 2024-04-25T08:43:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 -- 172.21.15.46:0/3974377297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 msgr2=0x7fd9a8162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 --2- 172.21.15.46:0/3974377297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 0x7fd9a8162200 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8070b00 tx=0x7fd99001c4e0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 -- 172.21.15.46:0/3974377297 shutdown_connections 2024-04-25T08:43:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 --2- 172.21.15.46:0/3974377297 >> v2:172.21.15.82:3300/0 conn(0x7fd9a80aed10 0x7fd9a8164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 --2- 172.21.15.46:0/3974377297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 0x7fd9a8162200 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 -- 172.21.15.46:0/3974377297 >> 172.21.15.46:0/3974377297 conn(0x7fd9a80b12a0 msgr2=0x7fd9a80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 -- 172.21.15.46:0/3974377297 shutdown_connections 2024-04-25T08:43:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 -- 172.21.15.46:0/3974377297 wait complete. 2024-04-25T08:43:44.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 Processor -- start 2024-04-25T08:43:44.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.019+0000 7fd9aee00640 1 -- start start 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9aee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 0x7fd9a815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9aee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9a80aed10 0x7fd9a8142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9aee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9a815a750 0x7fd9a815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9aee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a806d270 con 0x7fd9a806f320 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9aee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a80cf200 con 0x7fd9a80aed10 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9aee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81674b0 con 0x7fd9a815a750 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 0x7fd9a815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 0x7fd9a815d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46530/0 (socket says 172.21.15.46:46530) 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a7e00640 1 -- 172.21.15.46:0/1490775600 learned_addr learned my addr 172.21.15.46:0/1490775600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd99f400640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9a80aed10 0x7fd9a8142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9ace00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9a815a750 0x7fd9a815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a7e00640 1 -- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9a815a750 msgr2=0x7fd9a815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a7e00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9a815a750 0x7fd9a815bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a7e00640 1 -- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9a80aed10 msgr2=0x7fd9a8142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a7e00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9a80aed10 0x7fd9a8142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a7e00640 1 -- 172.21.15.46:0/1490775600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a8143340 con 0x7fd9a806f320 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9ace00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9a815a750 0x7fd9a815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd99f400640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9a80aed10 0x7fd9a8142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a7e00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 0x7fd9a815d4d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd99800b410 tx=0x7fd99800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a4c00640 1 -- 172.21.15.46:0/1490775600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998015030 con 0x7fd9a806f320 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a4c00640 1 -- 172.21.15.46:0/1490775600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd998004020 con 0x7fd9a806f320 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a4c00640 1 -- 172.21.15.46:0/1490775600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998004450 con 0x7fd9a806f320 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a815ae60 con 0x7fd9a806f320 2024-04-25T08:43:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a815b420 con 0x7fd9a806f320 2024-04-25T08:43:44.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a8069080 con 0x7fd9a806f320 2024-04-25T08:43:44.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.023+0000 7fd9a4c00640 1 -- 172.21.15.46:0/1490775600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd998018070 con 0x7fd9a806f320 2024-04-25T08:43:44.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.027+0000 7fd9a4c00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd984066f60 0x7fd984069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.027+0000 7fd9a4c00640 1 -- 172.21.15.46:0/1490775600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd998084d50 con 0x7fd9a806f320 2024-04-25T08:43:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.027+0000 7fd99f400640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd984066f60 0x7fd984069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.027+0000 7fd99f400640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd984066f60 0x7fd984069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd9900ae000 tx=0x7fd990008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.031+0000 7fd9a4c00640 1 -- 172.21.15.46:0/1490775600 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd998088030 con 0x7fd9a806f320 2024-04-25T08:43:44.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9a81643e0 con 0x7fd9a806f320 2024-04-25T08:43:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9a4c00640 1 -- 172.21.15.46:0/1490775600 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16894 (secure 0 0 0) 0x7fd998055bd0 con 0x7fd9a806f320 2024-04-25T08:43:44.187 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:44.187 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":40,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:36.675414+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:43:32.873190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:43:34.064607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:44.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd984066f60 msgr2=0x7fd984069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd984066f60 0x7fd984069420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd9900ae000 tx=0x7fd990008040 comp rx=0 tx=0).stop 2024-04-25T08:43:44.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 msgr2=0x7fd9a815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 0x7fd9a815d4d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd99800b410 tx=0x7fd99800b8d0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 shutdown_connections 2024-04-25T08:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd984066f60 0x7fd984069420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9a815a750 0x7fd9a815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9a80aed10 0x7fd9a8142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 --2- 172.21.15.46:0/1490775600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a806f320 0x7fd9a815d4d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 >> 172.21.15.46:0/1490775600 conn(0x7fd9a80b12a0 msgr2=0x7fd9a80b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 shutdown_connections 2024-04-25T08:43:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.195+0000 7fd9aee00640 1 -- 172.21.15.46:0/1490775600 wait complete. 2024-04-25T08:43:44.208 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:43:32.873190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:43:34.064607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:43:44.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:44.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.359+0000 7f25eee00640 1 Processor -- start 2024-04-25T08:43:44.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.359+0000 7f25eee00640 1 -- start start 2024-04-25T08:43:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.359+0000 7f25eee00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25e806e620 0x7f25e806bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.359+0000 7f25eee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e806ea60 0x7f25e806c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.359+0000 7f25eee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e80b3850 con 0x7f25e815dae0 2024-04-25T08:43:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.359+0000 7f25eee00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e806e190 con 0x7f25e806e620 2024-04-25T08:43:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.359+0000 7f25eee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80b3af0 con 0x7f25e806ea60 2024-04-25T08:43:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25dfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25e806e620 0x7f25e806bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25ece00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e806ea60 0x7f25e806c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25dfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25e806e620 0x7f25e806bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:56532/0 (socket says 172.21.15.46:56532) 2024-04-25T08:43:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25dfe00640 1 -- 172.21.15.46:0/4251050335 learned_addr learned my addr 172.21.15.46:0/4251050335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25ece00640 1 -- 172.21.15.46:0/4251050335 >> v2:172.21.15.82:3300/0 conn(0x7f25e806e620 msgr2=0x7f25e806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25ece00640 1 --2- 172.21.15.46:0/4251050335 >> v2:172.21.15.82:3300/0 conn(0x7f25e806e620 0x7f25e806bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25ece00640 1 -- 172.21.15.46:0/4251050335 >> v1:172.21.15.82:6789/0 conn(0x7f25e815dae0 legacy=0x7f25e8072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25ece00640 1 -- 172.21.15.46:0/4251050335 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e80b3dd0 con 0x7f25e806ea60 2024-04-25T08:43:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25dfe00640 1 --2- 172.21.15.46:0/4251050335 >> v2:172.21.15.82:3300/0 conn(0x7f25e806e620 0x7f25e806bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25ece00640 1 --2- 172.21.15.46:0/4251050335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e806ea60 0x7f25e806c160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25e806e310 tx=0x7f25e001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=687997f9ebec8fad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25e7400640 1 -- 172.21.15.46:0/4251050335 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e00b2070 con 0x7f25e806ea60 2024-04-25T08:43:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.363+0000 7f25e7400640 1 -- 172.21.15.46:0/4251050335 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e00a59e0 con 0x7f25e806ea60 2024-04-25T08:43:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25e7400640 1 -- 172.21.15.46:0/4251050335 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e00a5db0 con 0x7f25e806ea60 2024-04-25T08:43:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 -- 172.21.15.46:0/4251050335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e806ea60 msgr2=0x7f25e806c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 --2- 172.21.15.46:0/4251050335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e806ea60 0x7f25e806c160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f25e806e310 tx=0x7f25e001c920 comp rx=0 tx=0).stop 2024-04-25T08:43:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 -- 172.21.15.46:0/4251050335 shutdown_connections 2024-04-25T08:43:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 --2- 172.21.15.46:0/4251050335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e806ea60 0x7f25e806c160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 --2- 172.21.15.46:0/4251050335 >> v2:172.21.15.82:3300/0 conn(0x7f25e806e620 0x7f25e806bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 -- 172.21.15.46:0/4251050335 >> 172.21.15.46:0/4251050335 conn(0x7f25e8071ae0 msgr2=0x7f25e8071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 -- 172.21.15.46:0/4251050335 shutdown_connections 2024-04-25T08:43:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 -- 172.21.15.46:0/4251050335 wait complete. 2024-04-25T08:43:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 Processor -- start 2024-04-25T08:43:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 -- start start 2024-04-25T08:43:44.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e806e620 0x7f25e814b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e806ea60 0x7f25e815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e815dae0 0x7f25e8151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e806f850 con 0x7f25e806ea60 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e806f6d0 con 0x7f25e806e620 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25eee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80ad5c0 con 0x7f25e815dae0 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e806ea60 0x7f25e815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e806ea60 0x7f25e815fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46542/0 (socket says 172.21.15.46:46542) 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25dfe00640 1 -- 172.21.15.46:0/3818575913 learned_addr learned my addr 172.21.15.46:0/3818575913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25e7e00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e806e620 0x7f25e814b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25ece00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e815dae0 0x7f25e8151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25dfe00640 1 -- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e815dae0 msgr2=0x7f25e8151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.367+0000 7f25dfe00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e815dae0 0x7f25e8151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25dfe00640 1 -- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e806e620 msgr2=0x7f25e814b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25dfe00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e806e620 0x7f25e814b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25dfe00640 1 -- 172.21.15.46:0/3818575913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e814bba0 con 0x7f25e806ea60 2024-04-25T08:43:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25e7e00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e806e620 0x7f25e814b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25dfe00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e806ea60 0x7f25e815fa20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f25d000e930 tx=0x7f25d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25e4c00640 1 -- 172.21.15.46:0/3818575913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0017070 con 0x7f25e806ea60 2024-04-25T08:43:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25e4c00640 1 -- 172.21.15.46:0/3818575913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d000ca00 con 0x7f25e806ea60 2024-04-25T08:43:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25e4c00640 1 -- 172.21.15.46:0/3818575913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000cdd0 con 0x7f25e806ea60 2024-04-25T08:43:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e814be80 con 0x7f25e806ea60 2024-04-25T08:43:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e814c440 con 0x7f25e806ea60 2024-04-25T08:43:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e8069010 con 0x7f25e806ea60 2024-04-25T08:43:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25e4c00640 1 -- 172.21.15.46:0/3818575913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25d001a070 con 0x7f25e806ea60 2024-04-25T08:43:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25e4c00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25c4066f60 0x7f25c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25e4c00640 1 -- 172.21.15.46:0/3818575913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f25d00854a0 con 0x7f25e806ea60 2024-04-25T08:43:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.371+0000 7f25e7e00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25c4066f60 0x7f25c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.375+0000 7f25e7e00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25c4066f60 0x7f25c4069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000f70 tx=0x7f25d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.375+0000 7f25e4c00640 1 -- 172.21.15.46:0/3818575913 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25d0056280 con 0x7f25e806ea60 2024-04-25T08:43:44.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25e80aedb0 con 0x7f25e806ea60 2024-04-25T08:43:44.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25e4c00640 1 -- 172.21.15.46:0/3818575913 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25d0055b80 con 0x7f25e806ea60 2024-04-25T08:43:44.531 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25c4066f60 msgr2=0x7f25c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25c4066f60 0x7f25c4069420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000f70 tx=0x7f25d8009040 comp rx=0 tx=0).stop 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e806ea60 msgr2=0x7f25e815fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e806ea60 0x7f25e815fa20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f25d000e930 tx=0x7f25d000edf0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 shutdown_connections 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25c4066f60 0x7f25c4069420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25e815dae0 0x7f25e8151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25e806ea60 0x7f25e815fa20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 --2- 172.21.15.46:0/3818575913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25e806e620 0x7f25e814b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 >> 172.21.15.46:0/3818575913 conn(0x7f25e8071ae0 msgr2=0x7f25e80af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 shutdown_connections 2024-04-25T08:43:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.539+0000 7f25eee00640 1 -- 172.21.15.46:0/3818575913 wait complete. 2024-04-25T08:43:44.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:44.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.703+0000 7f4a1d600640 1 Processor -- start 2024-04-25T08:43:44.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.703+0000 7f4a1d600640 1 -- start start 2024-04-25T08:43:44.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a1d600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4a1806c730 0x7f4a1806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a1d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a1809a410 0x7f4a1806cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a1d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a180d0390 0x7f4a180cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a1d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a180b3dc0 con 0x7f4a1809a410 2024-04-25T08:43:44.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a1d600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a180cf680 con 0x7f4a1806c730 2024-04-25T08:43:44.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a1d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1809acb0 con 0x7f4a180d0390 2024-04-25T08:43:44.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a0fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a1809a410 0x7f4a1806cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a0fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a1809a410 0x7f4a1806cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46554/0 (socket says 172.21.15.46:46554) 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a0fe00640 1 -- 172.21.15.46:0/2098449327 learned_addr learned my addr 172.21.15.46:0/2098449327 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a17e00640 1 --2- 172.21.15.46:0/2098449327 >> v2:172.21.15.82:3300/0 conn(0x7f4a1806c730 0x7f4a1806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a1cc00640 1 --2- 172.21.15.46:0/2098449327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a180d0390 0x7f4a180cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a0fe00640 1 -- 172.21.15.46:0/2098449327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a180d0390 msgr2=0x7f4a180cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a0fe00640 1 --2- 172.21.15.46:0/2098449327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a180d0390 0x7f4a180cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a0fe00640 1 -- 172.21.15.46:0/2098449327 >> v2:172.21.15.82:3300/0 conn(0x7f4a1806c730 msgr2=0x7f4a1806bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a0fe00640 1 --2- 172.21.15.46:0/2098449327 >> v2:172.21.15.82:3300/0 conn(0x7f4a1806c730 0x7f4a1806bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a0fe00640 1 -- 172.21.15.46:0/2098449327 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1809a9d0 con 0x7f4a1809a410 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a17e00640 1 --2- 172.21.15.46:0/2098449327 >> v2:172.21.15.82:3300/0 conn(0x7f4a1806c730 0x7f4a1806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:44.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.707+0000 7f4a1cc00640 1 --2- 172.21.15.46:0/2098449327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a180d0390 0x7f4a180cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:44.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a0fe00640 1 --2- 172.21.15.46:0/2098449327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1809a410 0x7f4a1806cb10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4a1806eb50 tx=0x7f4a0001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b020e928c0d622b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:44.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a17400640 1 -- 172.21.15.46:0/2098449327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000b1070 con 0x7f4a1809a410 2024-04-25T08:43:44.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a17400640 1 -- 172.21.15.46:0/2098449327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000ac620 con 0x7f4a1809a410 2024-04-25T08:43:44.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a17400640 1 -- 172.21.15.46:0/2098449327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000ac9f0 con 0x7f4a1809a410 2024-04-25T08:43:44.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 -- 172.21.15.46:0/2098449327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1809a410 msgr2=0x7f4a1806cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 --2- 172.21.15.46:0/2098449327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1809a410 0x7f4a1806cb10 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4a1806eb50 tx=0x7f4a0001c4b0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 -- 172.21.15.46:0/2098449327 shutdown_connections 2024-04-25T08:43:44.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 --2- 172.21.15.46:0/2098449327 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a180d0390 0x7f4a180cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 --2- 172.21.15.46:0/2098449327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1809a410 0x7f4a1806cb10 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 --2- 172.21.15.46:0/2098449327 >> v2:172.21.15.82:3300/0 conn(0x7f4a1806c730 0x7f4a1806bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 -- 172.21.15.46:0/2098449327 >> 172.21.15.46:0/2098449327 conn(0x7f4a180b2f50 msgr2=0x7f4a180b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:44.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 -- 172.21.15.46:0/2098449327 shutdown_connections 2024-04-25T08:43:44.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 -- 172.21.15.46:0/2098449327 wait complete. 2024-04-25T08:43:44.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 Processor -- start 2024-04-25T08:43:44.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.711+0000 7f4a1d600640 1 -- start start 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a1806c730 0x7f4a180ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1809a410 0x7f4a180af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a180d0390 0x7f4a180af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1815dd60 con 0x7f4a180d0390 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1d600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180cc400 con 0x7f4a1809a410 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1806d050 con 0x7f4a1806c730 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a180d0390 0x7f4a180af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a17e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a1806c730 0x7f4a180ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a0fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1809a410 0x7f4a180af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a17e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a1806c730 0x7f4a180ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:50150/0 (socket says 172.21.15.46:50150) 2024-04-25T08:43:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a17e00640 1 -- 172.21.15.46:0/3277332994 learned_addr learned my addr 172.21.15.46:0/3277332994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:44.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1cc00640 1 -- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a1806c730 msgr2=0x7f4a180ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1cc00640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a1806c730 0x7f4a180ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1cc00640 1 -- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1809a410 msgr2=0x7f4a180af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1cc00640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1809a410 0x7f4a180af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1cc00640 1 -- 172.21.15.46:0/3277332994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180afeb0 con 0x7f4a180d0390 2024-04-25T08:43:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1cc00640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a180d0390 0x7f4a180af720 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4a10007d40 tx=0x7f4a1000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a14c00640 1 -- 172.21.15.46:0/3277332994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1001a070 con 0x7f4a180d0390 2024-04-25T08:43:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a14c00640 1 -- 172.21.15.46:0/3277332994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a100039a0 con 0x7f4a180d0390 2024-04-25T08:43:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a14c00640 1 -- 172.21.15.46:0/3277332994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10003d90 con 0x7f4a180d0390 2024-04-25T08:43:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a180b0190 con 0x7f4a180d0390 2024-04-25T08:43:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a181569a0 con 0x7f4a180d0390 2024-04-25T08:43:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a18156ba0 con 0x7f4a180d0390 2024-04-25T08:43:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.715+0000 7f4a14c00640 1 -- 172.21.15.46:0/3277332994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a1001d070 con 0x7f4a180d0390 2024-04-25T08:43:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.719+0000 7f4a14c00640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49f4066f60 0x7f49f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.719+0000 7f4a14c00640 1 -- 172.21.15.46:0/3277332994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a10085390 con 0x7f4a180d0390 2024-04-25T08:43:44.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.719+0000 7f4a17e00640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49f4066f60 0x7f49f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:44.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.719+0000 7f4a17e00640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49f4066f60 0x7f49f4069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4a180b3910 tx=0x7f4a08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:44.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.723+0000 7f4a14c00640 1 -- 172.21.15.46:0/3277332994 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a18156ba0 con 0x7f4a180d0390 2024-04-25T08:43:44.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a180b0360 con 0x7f4a180d0390 2024-04-25T08:43:44.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a14c00640 1 -- 172.21.15.46:0/3277332994 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a180b0360 con 0x7f4a180d0390 2024-04-25T08:43:44.880 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49f4066f60 msgr2=0x7f49f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49f4066f60 0x7f49f4069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4a180b3910 tx=0x7f4a08008040 comp rx=0 tx=0).stop 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a180d0390 msgr2=0x7f4a180af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a180d0390 0x7f4a180af720 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4a10007d40 tx=0x7f4a1000d4a0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 shutdown_connections 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f49f4066f60 0x7f49f4069420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a180d0390 0x7f4a180af720 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1809a410 0x7f4a180af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 --2- 172.21.15.46:0/3277332994 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4a1806c730 0x7f4a180ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 >> 172.21.15.46:0/3277332994 conn(0x7f4a180b2f50 msgr2=0x7f4a1806bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 shutdown_connections 2024-04-25T08:43:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:44.887+0000 7f4a1d600640 1 -- 172.21.15.46:0/3277332994 wait complete. 2024-04-25T08:43:44.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:45.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.051+0000 7ff1d5e00640 1 Processor -- start 2024-04-25T08:43:45.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.051+0000 7ff1d5e00640 1 -- start start 2024-04-25T08:43:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1d5e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff1d0143990 0x7ff1d00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1d5e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d006f150 0x7ff1d00b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1d5e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d015e440 con 0x7ff1d01440d0 2024-04-25T08:43:45.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1d5e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d015ed30 con 0x7ff1d0143990 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1d5e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d015e6e0 con 0x7ff1d006f150 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1c6000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff1d01440d0 0x7ff1d006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58386/0 (socket says 172.21.15.46:58386) 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1c6000640 1 -- 172.21.15.46:0/3367401561 learned_addr learned my addr 172.21.15.46:0/3367401561 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1cea00640 1 --2- 172.21.15.46:0/3367401561 >> v2:172.21.15.82:3300/0 conn(0x7ff1d0143990 0x7ff1d00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1cf400640 1 --2- 172.21.15.46:0/3367401561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d006f150 0x7ff1d00b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1ce000640 1 -- 172.21.15.46:0/3367401561 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997450276 0 0) 0x7ff1d015e440 con 0x7ff1d01440d0 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1ce000640 1 -- 172.21.15.46:0/3367401561 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1b4003530 con 0x7ff1d01440d0 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1cea00640 1 -- 172.21.15.46:0/3367401561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d006f150 msgr2=0x7ff1d00b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1cea00640 1 --2- 172.21.15.46:0/3367401561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d006f150 0x7ff1d00b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1cea00640 1 -- 172.21.15.46:0/3367401561 >> v1:172.21.15.46:6789/0 conn(0x7ff1d01440d0 legacy=0x7ff1d006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.055+0000 7ff1cea00640 1 -- 172.21.15.46:0/3367401561 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d015e9c0 con 0x7ff1d0143990 2024-04-25T08:43:45.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1cea00640 1 --2- 172.21.15.46:0/3367401561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0143990 0x7ff1d00cecc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1d015eeb0 tx=0x7ff1c001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5294d47907c0b148 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1ce000640 1 -- 172.21.15.46:0/3367401561 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c00b2070 con 0x7ff1d0143990 2024-04-25T08:43:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1ce000640 1 -- 172.21.15.46:0/3367401561 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c00a5810 con 0x7ff1d0143990 2024-04-25T08:43:45.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1ce000640 1 -- 172.21.15.46:0/3367401561 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c00a5bc0 con 0x7ff1d0143990 2024-04-25T08:43:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 -- 172.21.15.46:0/3367401561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0143990 msgr2=0x7ff1d00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 --2- 172.21.15.46:0/3367401561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0143990 0x7ff1d00cecc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff1d015eeb0 tx=0x7ff1c001c950 comp rx=0 tx=0).stop 2024-04-25T08:43:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 -- 172.21.15.46:0/3367401561 shutdown_connections 2024-04-25T08:43:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 --2- 172.21.15.46:0/3367401561 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d006f150 0x7ff1d00b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 --2- 172.21.15.46:0/3367401561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d0143990 0x7ff1d00cecc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 -- 172.21.15.46:0/3367401561 >> 172.21.15.46:0/3367401561 conn(0x7ff1d006ac10 msgr2=0x7ff1d006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 -- 172.21.15.46:0/3367401561 shutdown_connections 2024-04-25T08:43:45.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 -- 172.21.15.46:0/3367401561 wait complete. 2024-04-25T08:43:45.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 Processor -- start 2024-04-25T08:43:45.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.059+0000 7ff1d5e00640 1 -- start start 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1d5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d006f150 0x7ff1d0159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1d5e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d0143990 0x7ff1d01592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1d5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d01440d0 0x7ff1d01f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1d5e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d00b35f0 con 0x7ff1d006f150 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1d5e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d00b3470 con 0x7ff1d01440d0 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1d5e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d00b3770 con 0x7ff1d0143990 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d006f150 0x7ff1d0159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d006f150 0x7ff1d0159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46578/0 (socket says 172.21.15.46:46578) 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cea00640 1 -- 172.21.15.46:0/890390238 learned_addr learned my addr 172.21.15.46:0/890390238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cea00640 1 -- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d0143990 msgr2=0x7ff1d01592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1c6000640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d0143990 0x7ff1d01592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cf400640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d01440d0 0x7ff1d01f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cea00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d0143990 0x7ff1d01592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cea00640 1 -- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d01440d0 msgr2=0x7ff1d01f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cea00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d01440d0 0x7ff1d01f8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cea00640 1 -- 172.21.15.46:0/890390238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d015cd30 con 0x7ff1d006f150 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cf400640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d01440d0 0x7ff1d01f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1c6000640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d0143990 0x7ff1d01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1cea00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d006f150 0x7ff1d0159990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0000f90 tx=0x7ff1c00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1c7e00640 1 -- 172.21.15.46:0/890390238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c00b2040 con 0x7ff1d006f150 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1c7e00640 1 -- 172.21.15.46:0/890390238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c00bf070 con 0x7ff1d006f150 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d015d010 con 0x7ff1d006f150 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1c7e00640 1 -- 172.21.15.46:0/890390238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0003bc0 con 0x7ff1d006f150 2024-04-25T08:43:45.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d0133210 con 0x7ff1d006f150 2024-04-25T08:43:45.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d0069010 con 0x7ff1d006f150 2024-04-25T08:43:45.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.063+0000 7ff1c7e00640 1 -- 172.21.15.46:0/890390238 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1c0004040 con 0x7ff1d006f150 2024-04-25T08:43:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.067+0000 7ff1c7e00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1ac066f60 0x7ff1ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.067+0000 7ff1c7e00640 1 -- 172.21.15.46:0/890390238 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1c0120170 con 0x7ff1d006f150 2024-04-25T08:43:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.067+0000 7ff1c6000640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1ac066f60 0x7ff1ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.067+0000 7ff1c6000640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1ac066f60 0x7ff1ac069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00adfe0 tx=0x7ff1b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:45.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.071+0000 7ff1c7e00640 1 -- 172.21.15.46:0/890390238 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1c0016540 con 0x7ff1d006f150 2024-04-25T08:43:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1d00ae2a0 con 0x7ff1d006f150 2024-04-25T08:43:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1c7e00640 1 -- 172.21.15.46:0/890390238 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff1c00f0ff0 con 0x7ff1d006f150 2024-04-25T08:43:45.227 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1ac066f60 msgr2=0x7ff1ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1ac066f60 0x7ff1ac069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00adfe0 tx=0x7ff1b8009040 comp rx=0 tx=0).stop 2024-04-25T08:43:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d006f150 msgr2=0x7ff1d0159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d006f150 0x7ff1d0159990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0000f90 tx=0x7ff1c00196f0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 shutdown_connections 2024-04-25T08:43:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1ac066f60 0x7ff1ac069420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1d01440d0 0x7ff1d01f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1d0143990 0x7ff1d01592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 --2- 172.21.15.46:0/890390238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1d006f150 0x7ff1d0159990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 >> 172.21.15.46:0/890390238 conn(0x7ff1d006ac10 msgr2=0x7ff1d00b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 shutdown_connections 2024-04-25T08:43:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.235+0000 7ff1d5e00640 1 -- 172.21.15.46:0/890390238 wait complete. 2024-04-25T08:43:45.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:43:45.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f29600640 1 Processor -- start 2024-04-25T08:43:45.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f29600640 1 -- start start 2024-04-25T08:43:45.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f29600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2f240cf9e0 0x7f2f240cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f29600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2409a680 0x7f2f240cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f29600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f240cf560 con 0x7f2f240d0300 2024-04-25T08:43:45.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f29600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f240cd3d0 con 0x7f2f240cf9e0 2024-04-25T08:43:45.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f29600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24099ba0 con 0x7f2f2409a680 2024-04-25T08:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f23400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2f240d0300 0x7f2f2415ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58402/0 (socket says 172.21.15.46:58402) 2024-04-25T08:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f23400640 1 -- 172.21.15.46:0/2664994875 learned_addr learned my addr 172.21.15.46:0/2664994875 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f22000640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2409a680 0x7f2f240cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f22a00640 1 --2- 172.21.15.46:0/2664994875 >> v2:172.21.15.82:3300/0 conn(0x7f2f240cf9e0 0x7f2f240cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819811668 0 0) 0x7f2f240cf560 con 0x7f2f240d0300 2024-04-25T08:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f08003530 con 0x7f2f240d0300 2024-04-25T08:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 49500706 0 0) 0x7f2f08003530 con 0x7f2f240d0300 2024-04-25T08:43:45.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2409a680 msgr2=0x7f2f240cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f21600640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2409a680 0x7f2f240cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 >> v2:172.21.15.82:3300/0 conn(0x7f2f240cf9e0 msgr2=0x7f2f240cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f21600640 1 --2- 172.21.15.46:0/2664994875 >> v2:172.21.15.82:3300/0 conn(0x7f2f240cf9e0 0x7f2f240cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24099e80 con 0x7f2f240d0300 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.399+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f2f1c001a00 con 0x7f2f240d0300 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 >> v1:172.21.15.46:6789/0 conn(0x7f2f240d0300 legacy=0x7f2f2415ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2409a680 0x7f2f080039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f08004d50 0x7f2f080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240cd3d0 con 0x7f2f08003f40 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24099ba0 con 0x7f2f08004d50 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0800b9d0 con 0x7f2f2409a680 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f22000640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f22a00640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2409a680 0x7f2f080039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f23400640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f08004d50 0x7f2f080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f22000640 1 -- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2409a680 msgr2=0x7f2f080039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f22000640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2409a680 0x7f2f080039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f22000640 1 -- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f08004d50 msgr2=0x7f2f080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f22000640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f08004d50 0x7f2f080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f22000640 1 -- 172.21.15.46:0/2664994875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08003530 con 0x7f2f08003f40 2024-04-25T08:43:45.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f22000640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f08004810 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009ae0 tx=0x7f2f1001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5d0057b61084ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:45.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f100b2070 con 0x7f2f08003f40 2024-04-25T08:43:45.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f100a5550 con 0x7f2f08003f40 2024-04-25T08:43:45.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.403+0000 7f2f21600640 1 -- 172.21.15.46:0/2664994875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f100096b0 con 0x7f2f08003f40 2024-04-25T08:43:45.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 -- 172.21.15.46:0/2664994875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 msgr2=0x7f2f08004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 --2- 172.21.15.46:0/2664994875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f08004810 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009ae0 tx=0x7f2f1001ca10 comp rx=0 tx=0).stop 2024-04-25T08:43:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f23400640 1 -- 172.21.15.46:0/2664994875 reap_dead start 2024-04-25T08:43:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 -- 172.21.15.46:0/2664994875 shutdown_connections 2024-04-25T08:43:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 -- 172.21.15.46:0/2664994875 >> 172.21.15.46:0/2664994875 conn(0x7f2f240d1700 msgr2=0x7f2f240d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 -- 172.21.15.46:0/2664994875 shutdown_connections 2024-04-25T08:43:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 -- 172.21.15.46:0/2664994875 wait complete. 2024-04-25T08:43:45.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 Processor -- start 2024-04-25T08:43:45.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 -- start start 2024-04-25T08:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f2415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f2415acd0 0x7f2f2414a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2414ad60 0x7f2f2414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2409ac60 con 0x7f2f08003f40 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2409aae0 con 0x7f2f2415acd0 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f29600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2409ade0 con 0x7f2f2414ad60 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f2415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f2415a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46594/0 (socket says 172.21.15.46:46594) 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f22a00640 1 -- 172.21.15.46:0/3996099337 learned_addr learned my addr 172.21.15.46:0/3996099337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f22000640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f2415acd0 0x7f2f2414a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f23400640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2414ad60 0x7f2f2414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f22a00640 1 -- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2414ad60 msgr2=0x7f2f2414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f22a00640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2414ad60 0x7f2f2414b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f22a00640 1 -- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f2415acd0 msgr2=0x7f2f2414a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f22a00640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f2415acd0 0x7f2f2414a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.407+0000 7f2f22a00640 1 -- 172.21.15.46:0/3996099337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2415be80 con 0x7f2f08003f40 2024-04-25T08:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f22a00640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f2415a790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2f14011ad0 tx=0x7f2f14011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f1aa00640 1 -- 172.21.15.46:0/3996099337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400fbb0 con 0x7f2f08003f40 2024-04-25T08:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f1aa00640 1 -- 172.21.15.46:0/3996099337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1400fd50 con 0x7f2f08003f40 2024-04-25T08:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f24148480 con 0x7f2f08003f40 2024-04-25T08:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f1aa00640 1 -- 172.21.15.46:0/3996099337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14014680 con 0x7f2f08003f40 2024-04-25T08:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2415b550 con 0x7f2f08003f40 2024-04-25T08:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f24069080 con 0x7f2f08003f40 2024-04-25T08:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f1aa00640 1 -- 172.21.15.46:0/3996099337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f14012040 con 0x7f2f08003f40 2024-04-25T08:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f1aa00640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2f04066f90 0x7f2f04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f1aa00640 1 -- 172.21.15.46:0/3996099337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f140857a0 con 0x7f2f08003f40 2024-04-25T08:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.411+0000 7f2f22000640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2f04066f90 0x7f2f04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.415+0000 7f2f22000640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2f04066f90 0x7f2f04069450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2f100af000 tx=0x7f2f100046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:45.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.415+0000 7f2f1aa00640 1 -- 172.21.15.46:0/3996099337 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f14019030 con 0x7f2f08003f40 2024-04-25T08:43:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2f240d2530 con 0x7f2f08003f40 2024-04-25T08:43:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f1aa00640 1 -- 172.21.15.46:0/3996099337 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2f140566b0 con 0x7f2f08003f40 2024-04-25T08:43:45.583 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2f04066f90 msgr2=0x7f2f04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2f04066f90 0x7f2f04069450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2f100af000 tx=0x7f2f100046d0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 msgr2=0x7f2f2415a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f2415a790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2f14011ad0 tx=0x7f2f14011f90 comp rx=0 tx=0).stop 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 shutdown_connections 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2f04066f90 0x7f2f04069450 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f2414ad60 0x7f2f2414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f2415acd0 0x7f2f2414a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 --2- 172.21.15.46:0/3996099337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f08003f40 0x7f2f2415a790 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 >> 172.21.15.46:0/3996099337 conn(0x7f2f240d1700 msgr2=0x7f2f240c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 shutdown_connections 2024-04-25T08:43:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.591+0000 7f2f29600640 1 -- 172.21.15.46:0/3996099337 wait complete. 2024-04-25T08:43:45.605 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:43:45.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:45.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:43:45.665 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:43:45.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:43:45.666 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:43:45.684 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:45.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93b1200640 1 Processor -- start 2024-04-25T08:43:45.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93b1200640 1 -- start start 2024-04-25T08:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93b1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93ac159230 0x7f93ac15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93b1200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f93ac0cdc50 0x7f93ac15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93b1200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ac0cc700 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93b1200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ac0cc9e0 con 0x7f93ac159230 2024-04-25T08:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93b1200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ac0cccc0 con 0x7f93ac0cdc50 2024-04-25T08:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93a9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93ac159230 0x7f93ac15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93a9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93ac159230 0x7f93ac15bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46596/0 (socket says 172.21.15.46:46596) 2024-04-25T08:43:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93a9e00640 1 -- 172.21.15.46:0/1090669486 learned_addr learned my addr 172.21.15.46:0/1090669486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93aa800640 1 --2- 172.21.15.46:0/1090669486 >> v2:172.21.15.82:3300/0 conn(0x7f93ac0cdc50 0x7f93ac15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93a9e00640 1 -- 172.21.15.46:0/1090669486 >> v2:172.21.15.82:3300/0 conn(0x7f93ac0cdc50 msgr2=0x7f93ac15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93a9e00640 1 --2- 172.21.15.46:0/1090669486 >> v2:172.21.15.82:3300/0 conn(0x7f93ac0cdc50 0x7f93ac15d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93a9e00640 1 -- 172.21.15.46:0/1090669486 >> v1:172.21.15.82:6789/0 conn(0x7f93ac0cbeb0 legacy=0x7f93ac0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.879+0000 7f93a9e00640 1 -- 172.21.15.46:0/1090669486 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac0ccfa0 con 0x7f93ac159230 2024-04-25T08:43:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93a9e00640 1 --2- 172.21.15.46:0/1090669486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac159230 0x7f93ac15bdb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0b2340 tx=0x7f939c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412f73259fa4c2cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93a8a00640 1 -- 172.21.15.46:0/1090669486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0ab680 con 0x7f93ac159230 2024-04-25T08:43:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93a8a00640 1 -- 172.21.15.46:0/1090669486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c0b0070 con 0x7f93ac159230 2024-04-25T08:43:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93a8a00640 1 -- 172.21.15.46:0/1090669486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0aba40 con 0x7f93ac159230 2024-04-25T08:43:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 -- 172.21.15.46:0/1090669486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac159230 msgr2=0x7f93ac15bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 --2- 172.21.15.46:0/1090669486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac159230 0x7f93ac15bdb0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0b2340 tx=0x7f939c01c460 comp rx=0 tx=0).stop 2024-04-25T08:43:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 -- 172.21.15.46:0/1090669486 shutdown_connections 2024-04-25T08:43:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 --2- 172.21.15.46:0/1090669486 >> v2:172.21.15.82:3300/0 conn(0x7f93ac0cdc50 0x7f93ac15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 --2- 172.21.15.46:0/1090669486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac159230 0x7f93ac15bdb0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 -- 172.21.15.46:0/1090669486 >> 172.21.15.46:0/1090669486 conn(0x7f93ac0b12a0 msgr2=0x7f93ac0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:45.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 -- 172.21.15.46:0/1090669486 shutdown_connections 2024-04-25T08:43:45.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 -- 172.21.15.46:0/1090669486 wait complete. 2024-04-25T08:43:45.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 Processor -- start 2024-04-25T08:43:45.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.883+0000 7f93b1200640 1 -- start start 2024-04-25T08:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93b1200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac0cbeb0 0x7f93ac13f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93b1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac0cdc50 0x7f93ac1423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93b1200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93ac159230 0x7f93ac143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93b1200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac15df50 con 0x7f93ac0cdc50 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93b1200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac0cc4e0 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93b1200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac15e0d0 con 0x7f93ac159230 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac0cdc50 0x7f93ac1423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac0cbeb0 0x7f93ac13f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac0cdc50 0x7f93ac1423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46604/0 (socket says 172.21.15.46:46604) 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9400640 1 -- 172.21.15.46:0/92860309 learned_addr learned my addr 172.21.15.46:0/92860309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93aa800640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93ac159230 0x7f93ac143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9e00640 1 -- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93ac159230 msgr2=0x7f93ac143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9e00640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93ac159230 0x7f93ac143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9e00640 1 -- 172.21.15.46:0/92860309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac0cdc50 msgr2=0x7f93ac1423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9e00640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac0cdc50 0x7f93ac1423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9e00640 1 -- 172.21.15.46:0/92860309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac1f75f0 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9400640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac0cdc50 0x7f93ac1423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a9e00640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac0cbeb0 0x7f93ac13f750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f939c0ac000 tx=0x7f939c01aca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a2000640 1 -- 172.21.15.46:0/92860309 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0be5f0 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a2000640 1 -- 172.21.15.46:0/92860309 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939c0a7070 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ac1f7c80 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93a2000640 1 -- 172.21.15.46:0/92860309 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c0be9d0 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac1f8240 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.887+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac069080 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.891+0000 7f93a2000640 1 -- 172.21.15.46:0/92860309 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f939c0b43f0 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.891+0000 7f93a2000640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f938c066f60 0x7f938c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.891+0000 7f93a2000640 1 -- 172.21.15.46:0/92860309 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f939c11fba0 con 0x7f93ac0cbeb0 2024-04-25T08:43:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.891+0000 7f93a9400640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f938c066f60 0x7f938c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.891+0000 7f93a9400640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f938c066f60 0x7f938c069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f93ac067f50 tx=0x7f9398009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:45.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:45.895+0000 7f93a2000640 1 -- 172.21.15.46:0/92860309 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f939c016110 con 0x7f93ac0cbeb0 2024-04-25T08:43:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93ac0723d0 con 0x7f93ac0cbeb0 2024-04-25T08:43:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93a2000640 1 -- 172.21.15.46:0/92860309 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f939c0f0330 con 0x7f93ac0cbeb0 2024-04-25T08:43:46.083 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:46.083 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1079752579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1079752579},{"type":"v1","addr":"172.21.15.82:6836","nonce":1079752579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/106980078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":106980078},{"type":"v1","addr":"172.21.15.82:6837","nonce":106980078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1318225485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1318225485},{"type":"v1","addr":"172.21.15.46:6834","nonce":1318225485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:43:35.070722+0000","modified":"2024-04-25T08:43:38.080210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2025854544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2025854544},{"type":"v1","addr":"172.21.15.46:6835","nonce":2025854544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:43:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-25T08:43:46.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f938c066f60 msgr2=0x7f938c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:46.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f938c066f60 0x7f938c069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f93ac067f50 tx=0x7f9398009040 comp rx=0 tx=0).stop 2024-04-25T08:43:46.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac0cbeb0 msgr2=0x7f93ac13f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:46.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac0cbeb0 0x7f93ac13f750 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f939c0ac000 tx=0x7f939c01aca0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 shutdown_connections 2024-04-25T08:43:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f938c066f60 0x7f938c069420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f93ac159230 0x7f93ac143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93ac0cdc50 0x7f93ac1423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 --2- 172.21.15.46:0/92860309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93ac0cbeb0 0x7f93ac13f750 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 >> 172.21.15.46:0/92860309 conn(0x7f93ac0b12a0 msgr2=0x7f93ac164900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 shutdown_connections 2024-04-25T08:43:46.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.091+0000 7f93b1200640 1 -- 172.21.15.46:0/92860309 wait complete. 2024-04-25T08:43:46.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:43:46.279 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-25T08:43:46.280 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:43:46.280 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.427+0000 7f7197c00640 1 Processor -- start 2024-04-25T08:43:46.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.427+0000 7f7197c00640 1 -- start start 2024-04-25T08:43:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7197c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f719015dbd0 0x7f71900b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7197c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f71900b0c30 0x7f7190070c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7197c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71900b5250 con 0x7f71900711d0 2024-04-25T08:43:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7197c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71900b5af0 con 0x7f71900b0c30 2024-04-25T08:43:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7197c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71900b5530 con 0x7f719015dbd0 2024-04-25T08:43:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7187e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f71900b0c30 0x7f7190070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7195200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f719015dbd0 0x7f71900b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7195c00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f71900711d0 0x7f719015dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:47140/0 (socket says 172.21.15.46:47140) 2024-04-25T08:43:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7195c00640 1 -- 172.21.15.46:0/680459692 learned_addr learned my addr 172.21.15.46:0/680459692 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7187e00640 1 -- 172.21.15.46:0/680459692 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f719015dbd0 msgr2=0x7f71900b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7187e00640 1 --2- 172.21.15.46:0/680459692 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f719015dbd0 0x7f71900b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7187e00640 1 -- 172.21.15.46:0/680459692 >> v1:172.21.15.82:6789/0 conn(0x7f71900711d0 legacy=0x7f719015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f7187e00640 1 -- 172.21.15.46:0/680459692 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71900b5810 con 0x7f71900b0c30 2024-04-25T08:43:46.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.431+0000 7f718fe00640 1 -- 172.21.15.46:0/680459692 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896262740 0 0) 0x7f71900b5250 con 0x7f71900711d0 2024-04-25T08:43:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7187e00640 1 --2- 172.21.15.46:0/680459692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71900b0c30 0x7f7190070c90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f71780097c0 tx=0x7f717801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64443ed8e7e4a32c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f718fe00640 1 -- 172.21.15.46:0/680459692 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71780b0070 con 0x7f71900b0c30 2024-04-25T08:43:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f718fe00640 1 -- 172.21.15.46:0/680459692 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71780ab600 con 0x7f71900b0c30 2024-04-25T08:43:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f718fe00640 1 -- 172.21.15.46:0/680459692 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71780ab9d0 con 0x7f71900b0c30 2024-04-25T08:43:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 -- 172.21.15.46:0/680459692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71900b0c30 msgr2=0x7f7190070c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 --2- 172.21.15.46:0/680459692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71900b0c30 0x7f7190070c90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f71780097c0 tx=0x7f717801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 -- 172.21.15.46:0/680459692 shutdown_connections 2024-04-25T08:43:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 --2- 172.21.15.46:0/680459692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71900b0c30 0x7f7190070c90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 --2- 172.21.15.46:0/680459692 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f719015dbd0 0x7f71900b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 -- 172.21.15.46:0/680459692 >> 172.21.15.46:0/680459692 conn(0x7f719006f4f0 msgr2=0x7f719006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 -- 172.21.15.46:0/680459692 shutdown_connections 2024-04-25T08:43:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 -- 172.21.15.46:0/680459692 wait complete. 2024-04-25T08:43:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 Processor -- start 2024-04-25T08:43:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.435+0000 7f7197c00640 1 -- start start 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7197c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71900711d0 0x7f719014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7197c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71900b0c30 0x7f719014b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7197c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719015dbd0 0x7f719015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7197c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719016a850 con 0x7f719015dbd0 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7197c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7190071620 con 0x7f71900711d0 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7197c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719016a9d0 con 0x7f71900b0c30 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719015dbd0 0x7f719015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719015dbd0 0x7f719015f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46618/0 (socket says 172.21.15.46:46618) 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195c00640 1 -- 172.21.15.46:0/2750413607 learned_addr learned my addr 172.21.15.46:0/2750413607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195200640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71900711d0 0x7f719014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:46.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7187e00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71900b0c30 0x7f719014b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195c00640 1 -- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71900b0c30 msgr2=0x7f719014b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195c00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71900b0c30 0x7f719014b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195c00640 1 -- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71900711d0 msgr2=0x7f719014ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195c00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71900711d0 0x7f719014ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195c00640 1 -- 172.21.15.46:0/2750413607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190154fd0 con 0x7f719015dbd0 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7195c00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719015dbd0 0x7f719015f0c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f718800c7d0 tx=0x7f718800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f718d600640 1 -- 172.21.15.46:0/2750413607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188018070 con 0x7f719015dbd0 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f718d600640 1 -- 172.21.15.46:0/2750413607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718800a640 con 0x7f719015dbd0 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f718d600640 1 -- 172.21.15.46:0/2750413607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718800aa70 con 0x7f719015dbd0 2024-04-25T08:43:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71901552b0 con 0x7f719015dbd0 2024-04-25T08:43:46.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719014b680 con 0x7f719015dbd0 2024-04-25T08:43:46.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7190069080 con 0x7f719015dbd0 2024-04-25T08:43:46.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.439+0000 7f718d600640 1 -- 172.21.15.46:0/2750413607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f718800ac10 con 0x7f719015dbd0 2024-04-25T08:43:46.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.443+0000 7f718d600640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f716c066f60 0x7f716c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:46.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.443+0000 7f718d600640 1 -- 172.21.15.46:0/2750413607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7188084750 con 0x7f719015dbd0 2024-04-25T08:43:46.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.443+0000 7f7195200640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f716c066f60 0x7f716c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:46.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.443+0000 7f7195200640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f716c066f60 0x7f716c069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f71900cd790 tx=0x7f7180009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.447+0000 7f718d600640 1 -- 172.21.15.46:0/2750413607 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7188054f70 con 0x7f719015dbd0 2024-04-25T08:43:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.643+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71900b0820 con 0x7f719015dbd0 2024-04-25T08:43:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.643+0000 7f718d600640 1 -- 172.21.15.46:0/2750413607 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f7188059010 con 0x7f719015dbd0 2024-04-25T08:43:46.638 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:46.638 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1079752579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1079752579},{"type":"v1","addr":"172.21.15.82:6836","nonce":1079752579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/106980078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":106980078},{"type":"v1","addr":"172.21.15.82:6837","nonce":106980078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1318225485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1318225485},{"type":"v1","addr":"172.21.15.46:6834","nonce":1318225485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:43:35.070722+0000","modified":"2024-04-25T08:43:38.080210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2025854544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2025854544},{"type":"v1","addr":"172.21.15.46:6835","nonce":2025854544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:43:46.638 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-25T08:43:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f716c066f60 msgr2=0x7f716c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f716c066f60 0x7f716c069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f71900cd790 tx=0x7f7180009190 comp rx=0 tx=0).stop 2024-04-25T08:43:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719015dbd0 msgr2=0x7f719015f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719015dbd0 0x7f719015f0c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f718800c7d0 tx=0x7f718800cc90 comp rx=0 tx=0).stop 2024-04-25T08:43:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 shutdown_connections 2024-04-25T08:43:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f716c066f60 0x7f716c069420 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f719015dbd0 0x7f719015f0c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f71900b0c30 0x7f719014b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 --2- 172.21.15.46:0/2750413607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71900711d0 0x7f719014ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 >> 172.21.15.46:0/2750413607 conn(0x7f719006f4f0 msgr2=0x7f71900d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 shutdown_connections 2024-04-25T08:43:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:46.647+0000 7f7197c00640 1 -- 172.21.15.46:0/2750413607 wait complete. 2024-04-25T08:43:46.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-04-25T08:43:46.822 INFO:teuthology.orchestra.run.smithi046.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-25T08:43:46.823 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-25T08:43:50.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:50.165 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:56.824 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-25T08:43:56.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c65db080 1 Processor -- start 2024-04-25T08:43:56.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c65db080 1 -- start start 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c65db080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705560 0x55bf2868d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c65db080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705db0 0x55bf28724f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c65db080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf28725890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c65db080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf28708180 con 0x55bf287254b0 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c65db080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf28708000 con 0x55bf28705db0 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c65db080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf28708300 con 0x55bf28705560 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c4200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf28725890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c4200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf28725890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51098/0 (socket says 172.21.15.46:51098) 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c4200640 1 -- 172.21.15.46:0/1913597002 learned_addr learned my addr 172.21.15.46:0/1913597002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c3800640 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705560 0x55bf2868d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c2e00640 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705db0 0x55bf28724f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c4200640 1 -- 172.21.15.46:0/1913597002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705560 msgr2=0x55bf2868d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c4200640 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705560 0x55bf2868d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c4200640 1 -- 172.21.15.46:0/1913597002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705db0 msgr2=0x55bf28724f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.883+0000 7f92c4200640 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705db0 0x55bf28724f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c4200640 1 -- 172.21.15.46:0/1913597002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf28704b40 con 0x55bf287254b0 2024-04-25T08:43:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c2e00640 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705db0 0x55bf28724f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:56.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c4200640 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf28725890 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00b600 tx=0x7f92bc00bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f76d89de9613d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c1a00640 1 -- 172.21.15.46:0/1913597002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc009810 con 0x55bf287254b0 2024-04-25T08:43:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c1a00640 1 -- 172.21.15.46:0/1913597002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92bc015070 con 0x55bf287254b0 2024-04-25T08:43:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c1a00640 1 -- 172.21.15.46:0/1913597002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92bc009bb0 con 0x55bf287254b0 2024-04-25T08:43:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 -- 172.21.15.46:0/1913597002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 msgr2=0x55bf28725890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf28725890 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00b600 tx=0x7f92bc00bac0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 -- 172.21.15.46:0/1913597002 shutdown_connections 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf28725890 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705db0 0x55bf28724f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 --2- 172.21.15.46:0/1913597002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705560 0x55bf2868d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 -- 172.21.15.46:0/1913597002 >> 172.21.15.46:0/1913597002 conn(0x55bf2864d230 msgr2=0x55bf287087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 -- 172.21.15.46:0/1913597002 shutdown_connections 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 -- 172.21.15.46:0/1913597002 wait complete. 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 Processor -- start 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 -- start start 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705560 0x55bf287bbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705db0 0x55bf28703ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf287bd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf287051c0 con 0x55bf287254b0 2024-04-25T08:43:56.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf28704f20 con 0x55bf28705560 2024-04-25T08:43:56.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c65db080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf287221f0 con 0x55bf28705db0 2024-04-25T08:43:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c3800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705560 0x55bf287bbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c4200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf287bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c3800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705560 0x55bf287bbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:47488/0 (socket says 172.21.15.46:47488) 2024-04-25T08:43:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c3800640 1 -- 172.21.15.46:0/1437064135 learned_addr learned my addr 172.21.15.46:0/1437064135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.887+0000 7f92c2e00640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705db0 0x55bf28703ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c3800640 1 -- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705db0 msgr2=0x55bf28703ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c3800640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705db0 0x55bf28703ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c3800640 1 -- 172.21.15.46:0/1437064135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 msgr2=0x55bf287bd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c3800640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf287bd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c3800640 1 -- 172.21.15.46:0/1437064135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf287bc710 con 0x55bf28705560 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c4200640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf287254b0 0x55bf287bd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c2e00640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bf28705db0 0x55bf28703ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c3800640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705560 0x55bf287bbe40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f92b4019b00 tx=0x7f92b40181f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92bbe00640 1 -- 172.21.15.46:0/1437064135 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40af040 con 0x55bf28705560 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf287bc9b0 con 0x55bf28705560 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92bbe00640 1 -- 172.21.15.46:0/1437064135 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b40b6630 con 0x55bf28705560 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92bbe00640 1 -- 172.21.15.46:0/1437064135 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40b69e0 con 0x55bf28705560 2024-04-25T08:43:56.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf287bcef0 con 0x55bf28705560 2024-04-25T08:43:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.891+0000 7f92bbe00640 1 -- 172.21.15.46:0/1437064135 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92b40be020 con 0x55bf28705560 2024-04-25T08:43:56.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92bbe00640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92a4065940 0x7f92a4067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92bbe00640 1 -- 172.21.15.46:0/1437064135 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92b411eef0 con 0x55bf28705560 2024-04-25T08:43:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55bf2870e810 0x55bf287bdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x55bf28704420 con 0x55bf2870e810 2024-04-25T08:43:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c2e00640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92a4065940 0x7f92a4067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c4200640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55bf2870e810 0x55bf287bdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c2e00640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92a4065940 0x7f92a4067e00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f92b0005fd0 tx=0x7f92b0005b90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c4200640 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55bf2870e810 0x55bf287bdb70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c4200640 1 -- 172.21.15.46:0/1437064135 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f92bc009ce0 con 0x55bf2870e810 2024-04-25T08:43:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55bf2870e810 msgr2=0x55bf287bdb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55bf2870e810 0x55bf287bdb70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92a4065940 msgr2=0x7f92a4067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92a4065940 0x7f92a4067e00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f92b0005fd0 tx=0x7f92b0005b90 comp rx=0 tx=0).stop 2024-04-25T08:43:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705560 msgr2=0x55bf287bbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 --2- 172.21.15.46:0/1437064135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf28705560 0x55bf287bbe40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f92b4019b00 tx=0x7f92b40181f0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c4200640 1 -- 172.21.15.46:0/1437064135 reap_dead start 2024-04-25T08:43:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 shutdown_connections 2024-04-25T08:43:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 >> 172.21.15.46:0/1437064135 conn(0x55bf2864d230 msgr2=0x55bf287087c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 shutdown_connections 2024-04-25T08:43:56.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.895+0000 7f92c65db080 1 -- 172.21.15.46:0/1437064135 wait complete. 2024-04-25T08:43:56.893 INFO:teuthology.orchestra.run.smithi046.stdout:mytag 2024-04-25T08:43:56.893 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-25T08:43:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.947+0000 7f3b74a4b080 1 Processor -- start 2024-04-25T08:43:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.947+0000 7f3b74a4b080 1 -- start start 2024-04-25T08:43:56.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c0bf9b73d0 0x55c0bfa56e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c0bfa5e5c0 0x55c0bf9895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5ef10 0x55c0bf989b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bf92d9f0 con 0x55c0bfa5ef10 2024-04-25T08:43:56.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bf931950 con 0x55c0bf9b73d0 2024-04-25T08:43:56.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0bfa5de80 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5ef10 0x55c0bf989b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5ef10 0x55c0bf989b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51106/0 (socket says 172.21.15.46:51106) 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71c00640 1 -- 172.21.15.46:0/1839375871 learned_addr learned my addr 172.21.15.46:0/1839375871 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71200640 1 --2- 172.21.15.46:0/1839375871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c0bf9b73d0 0x55c0bfa56e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b6be00640 1 --2- 172.21.15.46:0/1839375871 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c0bfa5e5c0 0x55c0bf9895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71c00640 1 -- 172.21.15.46:0/1839375871 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c0bfa5e5c0 msgr2=0x55c0bf9895e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71c00640 1 --2- 172.21.15.46:0/1839375871 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c0bfa5e5c0 0x55c0bf9895e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71c00640 1 -- 172.21.15.46:0/1839375871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c0bf9b73d0 msgr2=0x55c0bfa56e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71c00640 1 --2- 172.21.15.46:0/1839375871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c0bf9b73d0 0x55c0bfa56e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71c00640 1 -- 172.21.15.46:0/1839375871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0bfa63d30 con 0x55c0bfa5ef10 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b71c00640 1 --2- 172.21.15.46:0/1839375871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5ef10 0x55c0bf989b20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c001680 tx=0x7f3b6c001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919e7e29b2621568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b6aa00640 1 -- 172.21.15.46:0/1839375871 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c003660 con 0x55c0bfa5ef10 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b6aa00640 1 -- 172.21.15.46:0/1839375871 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c003800 con 0x55c0bfa5ef10 2024-04-25T08:43:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b6aa00640 1 -- 172.21.15.46:0/1839375871 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c003bf0 con 0x55c0bfa5ef10 2024-04-25T08:43:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- 172.21.15.46:0/1839375871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5ef10 msgr2=0x55c0bf989b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- 172.21.15.46:0/1839375871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5ef10 0x55c0bf989b20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c001680 tx=0x7f3b6c001b40 comp rx=0 tx=0).stop 2024-04-25T08:43:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- 172.21.15.46:0/1839375871 shutdown_connections 2024-04-25T08:43:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- 172.21.15.46:0/1839375871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5ef10 0x55c0bf989b20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- 172.21.15.46:0/1839375871 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c0bfa5e5c0 0x55c0bf9895e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- 172.21.15.46:0/1839375871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c0bf9b73d0 0x55c0bfa56e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- 172.21.15.46:0/1839375871 >> 172.21.15.46:0/1839375871 conn(0x55c0bf9b1d30 msgr2=0x55c0bfa662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- 172.21.15.46:0/1839375871 shutdown_connections 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- 172.21.15.46:0/1839375871 wait complete. 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 Processor -- start 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- start start 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c0bf9b73d0 0x55c0bfa5c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5e5c0 0x55c0bfa3d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c0bfa5ef10 0x55c0bfa3dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bfa573d0 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0bfa63fd0 con 0x55c0bf9b73d0 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.951+0000 7f3b74a4b080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c0bfa57670 con 0x55c0bfa5ef10 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5e5c0 0x55c0bfa3d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5e5c0 0x55c0bfa3d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51108/0 (socket says 172.21.15.46:51108) 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b6be00640 1 -- 172.21.15.46:0/763403853 learned_addr learned my addr 172.21.15.46:0/763403853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b71200640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c0bf9b73d0 0x55c0bfa5c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b71c00640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c0bfa5ef10 0x55c0bfa3dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b6be00640 1 -- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c0bfa5ef10 msgr2=0x55c0bfa3dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b6be00640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c0bfa5ef10 0x55c0bfa3dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b6be00640 1 -- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c0bf9b73d0 msgr2=0x55c0bfa5c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b6be00640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c0bf9b73d0 0x55c0bfa5c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b6be00640 1 -- 172.21.15.46:0/763403853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0bf9b5340 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b6be00640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5e5c0 0x55c0bfa3d770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c000c10 tx=0x7f3b5c018150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b69600640 1 -- 172.21.15.46:0/763403853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0af040 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b69600640 1 -- 172.21.15.46:0/763403853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c0be070 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b69600640 1 -- 172.21.15.46:0/763403853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0b55f0 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c0bf9b55e0 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0bf9b5b20 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.955+0000 7f3b69600640 1 -- 172.21.15.46:0/763403853 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b5c0b5790 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b69600640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3b50065940 0x7f3b50067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b69600640 1 -- 172.21.15.46:0/763403853 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b5c11edf0 con 0x55c0bfa5e5c0 2024-04-25T08:43:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b71200640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3b50065940 0x7f3b50067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55c0bf9b5fc0 0x55c0bf9b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x55c0bf9b68e0 con 0x55c0bf9b5fc0 2024-04-25T08:43:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b71c00640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55c0bf9b5fc0 0x55c0bf9b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b71200640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3b50065940 0x7f3b50067e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b64000c10 tx=0x7f3b64001820 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b71c00640 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55c0bf9b5fc0 0x55c0bf9b63a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b71c00640 1 -- 172.21.15.46:0/763403853 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f3b6c01b040 con 0x55c0bf9b5fc0 2024-04-25T08:43:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55c0bf9b5fc0 msgr2=0x55c0bf9b63a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55c0bf9b5fc0 0x55c0bf9b63a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3b50065940 msgr2=0x7f3b50067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3b50065940 0x7f3b50067e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b64000c10 tx=0x7f3b64001820 comp rx=0 tx=0).stop 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5e5c0 msgr2=0x55c0bfa3d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 --2- 172.21.15.46:0/763403853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c0bfa5e5c0 0x55c0bfa3d770 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c000c10 tx=0x7f3b5c018150 comp rx=0 tx=0).stop 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b71c00640 1 -- 172.21.15.46:0/763403853 reap_dead start 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 shutdown_connections 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 >> 172.21.15.46:0/763403853 conn(0x55c0bf9b1d30 msgr2=0x55c0bfa661e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 shutdown_connections 2024-04-25T08:43:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:56.959+0000 7f3b74a4b080 1 -- 172.21.15.46:0/763403853 wait complete. 2024-04-25T08:43:56.957 INFO:teuthology.orchestra.run.smithi046.stdout:mytag 2024-04-25T08:43:56.957 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T08:43:57.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff712dff080 1 Processor -- start 2024-04-25T08:43:57.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff712dff080 1 -- start start 2024-04-25T08:43:57.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff712dff080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268124c0 0x557f268e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff712dff080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f268e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff712dff080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268c8560 0x557f268e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff712dff080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f268c9530 con 0x557f26859f20 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff712dff080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f267b69f0 con 0x557f268124c0 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff712dff080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f268c97d0 con 0x557f268c8560 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff70f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f268e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff70f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f268e52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51120/0 (socket says 172.21.15.46:51120) 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff70f600640 1 -- 172.21.15.46:0/1925290622 learned_addr learned my addr 172.21.15.46:0/1925290622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff710000640 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268124c0 0x557f268e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff710a00640 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268c8560 0x557f268e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff70f600640 1 -- 172.21.15.46:0/1925290622 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268c8560 msgr2=0x557f268e5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff70f600640 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268c8560 0x557f268e5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff70f600640 1 -- 172.21.15.46:0/1925290622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268124c0 msgr2=0x557f268e6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff70f600640 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268124c0 0x557f268e6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff70f600640 1 -- 172.21.15.46:0/1925290622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f268e73f0 con 0x557f26859f20 2024-04-25T08:43:57.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.051+0000 7ff710a00640 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268c8560 0x557f268e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:57.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70f600640 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f268e52c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00a500 tx=0x7ff6fc003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6121b9b8d5cac4b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70e200640 1 -- 172.21.15.46:0/1925290622 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0af040 con 0x557f26859f20 2024-04-25T08:43:57.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70e200640 1 -- 172.21.15.46:0/1925290622 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc0b9070 con 0x557f26859f20 2024-04-25T08:43:57.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70e200640 1 -- 172.21.15.46:0/1925290622 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0b4410 con 0x557f26859f20 2024-04-25T08:43:57.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 -- 172.21.15.46:0/1925290622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 msgr2=0x557f268e52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f268e52c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00a500 tx=0x7ff6fc003030 comp rx=0 tx=0).stop 2024-04-25T08:43:57.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 -- 172.21.15.46:0/1925290622 shutdown_connections 2024-04-25T08:43:57.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268c8560 0x557f268e5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f268e52c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 --2- 172.21.15.46:0/1925290622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268124c0 0x557f268e6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 -- 172.21.15.46:0/1925290622 >> 172.21.15.46:0/1925290622 conn(0x557f26810230 msgr2=0x557f26859290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 -- 172.21.15.46:0/1925290622 shutdown_connections 2024-04-25T08:43:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 -- 172.21.15.46:0/1925290622 wait complete. 2024-04-25T08:43:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 Processor -- start 2024-04-25T08:43:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 -- start start 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268124c0 0x557f268e39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f269778e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268c8560 0x557f26977e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f268e7690 con 0x557f26859f20 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f268e7930 con 0x557f268c8560 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff712dff080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f268e7bd0 con 0x557f268124c0 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f269778e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f269778e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51122/0 (socket says 172.21.15.46:51122) 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70f600640 1 -- 172.21.15.46:0/1947742732 learned_addr learned my addr 172.21.15.46:0/1947742732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff710a00640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268c8560 0x557f26977e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff710000640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268124c0 0x557f268e39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70f600640 1 -- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268124c0 msgr2=0x557f268e39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70f600640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557f268124c0 0x557f268e39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70f600640 1 -- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268c8560 msgr2=0x557f26977e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70f600640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268c8560 0x557f26977e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff70f600640 1 -- 172.21.15.46:0/1947742732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f268e3ef0 con 0x557f26859f20 2024-04-25T08:43:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.055+0000 7ff710a00640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557f268c8560 0x557f26977e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff70f600640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f269778e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0b6040 tx=0x7ff6fc018150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff70ce00640 1 -- 172.21.15.46:0/1947742732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0af050 con 0x557f26859f20 2024-04-25T08:43:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff70ce00640 1 -- 172.21.15.46:0/1947742732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc0c5070 con 0x557f26859f20 2024-04-25T08:43:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff70ce00640 1 -- 172.21.15.46:0/1947742732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0b4860 con 0x557f26859f20 2024-04-25T08:43:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557f26978360 con 0x557f26859f20 2024-04-25T08:43:57.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f26978890 con 0x557f26859f20 2024-04-25T08:43:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff70ce00640 1 -- 172.21.15.46:0/1947742732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6fc0b4a00 con 0x557f26859f20 2024-04-25T08:43:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff70ce00640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6f0065940 0x7ff6f0067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff70ce00640 1 -- 172.21.15.46:0/1947742732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6fc11f9d0 con 0x557f26859f20 2024-04-25T08:43:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff712dff080 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557f26858530 0x557f268c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x557f268c58b0 con 0x557f26858530 2024-04-25T08:43:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff710000640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6f0065940 0x7ff6f0067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.059+0000 7ff710a00640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557f26858530 0x557f268c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff710000640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6f0065940 0x7ff6f0067e00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff700004180 tx=0x7ff700010040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff710a00640 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557f26858530 0x557f268c5370 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff710a00640 1 -- 172.21.15.46:0/1947742732 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7ff708014420 con 0x557f26858530 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557f26858530 msgr2=0x557f268c5370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557f26858530 0x557f268c5370 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6f0065940 msgr2=0x7ff6f0067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6f0065940 0x7ff6f0067e00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff700004180 tx=0x7ff700010040 comp rx=0 tx=0).stop 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 msgr2=0x557f269778e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 --2- 172.21.15.46:0/1947742732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557f26859f20 0x557f269778e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0b6040 tx=0x7ff6fc018150 comp rx=0 tx=0).stop 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff710a00640 1 -- 172.21.15.46:0/1947742732 reap_dead start 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 shutdown_connections 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 >> 172.21.15.46:0/1947742732 conn(0x557f26810230 msgr2=0x557f26859490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 shutdown_connections 2024-04-25T08:43:57.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.063+0000 7ff712dff080 1 -- 172.21.15.46:0/1947742732 wait complete. 2024-04-25T08:43:57.060 INFO:teuthology.orchestra.run.smithi046.stdout:mytag 2024-04-25T08:43:57.060 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T08:43:57.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f8ade8080 1 Processor -- start 2024-04-25T08:43:57.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f8ade8080 1 -- start start 2024-04-25T08:43:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f8ade8080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3a697ed30 0x55e3a699e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f8ade8080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a69816f0 0x55e3a68f4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f8ade8080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a6981ad0 0x55e3a6909530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f8ade8080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a697f240 con 0x55e3a69816f0 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f8ade8080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a686d9f0 con 0x55e3a697ed30 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f8ade8080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3a697f4e0 con 0x55e3a6981ad0 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f87600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a69816f0 0x55e3a68f4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f87600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a69816f0 0x55e3a68f4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51148/0 (socket says 172.21.15.46:51148) 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f87600640 1 -- 172.21.15.46:0/1342499596 learned_addr learned my addr 172.21.15.46:0/1342499596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f88000640 1 --2- 172.21.15.46:0/1342499596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3a697ed30 0x55e3a699e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f88a00640 1 --2- 172.21.15.46:0/1342499596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a6981ad0 0x55e3a6909530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f87600640 1 -- 172.21.15.46:0/1342499596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a6981ad0 msgr2=0x55e3a6909530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f87600640 1 --2- 172.21.15.46:0/1342499596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a6981ad0 0x55e3a6909530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f87600640 1 -- 172.21.15.46:0/1342499596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3a697ed30 msgr2=0x55e3a699e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f87600640 1 --2- 172.21.15.46:0/1342499596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3a697ed30 0x55e3a699e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f87600640 1 -- 172.21.15.46:0/1342499596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3a699a780 con 0x55e3a69816f0 2024-04-25T08:43:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.159+0000 7f0f87600640 1 --2- 172.21.15.46:0/1342499596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a69816f0 0x55e3a68f4dc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0f740b2040 tx=0x7f0f74003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f4de263a7958a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f86200640 1 -- 172.21.15.46:0/1342499596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f740b4070 con 0x55e3a69816f0 2024-04-25T08:43:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f86200640 1 -- 172.21.15.46:0/1342499596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f740be070 con 0x55e3a69816f0 2024-04-25T08:43:57.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f86200640 1 -- 172.21.15.46:0/1342499596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f740b98a0 con 0x55e3a69816f0 2024-04-25T08:43:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1342499596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a69816f0 msgr2=0x55e3a68f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 --2- 172.21.15.46:0/1342499596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a69816f0 0x55e3a68f4dc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f0f740b2040 tx=0x7f0f74003030 comp rx=0 tx=0).stop 2024-04-25T08:43:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1342499596 shutdown_connections 2024-04-25T08:43:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 --2- 172.21.15.46:0/1342499596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a6981ad0 0x55e3a6909530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 --2- 172.21.15.46:0/1342499596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a69816f0 0x55e3a68f4dc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 --2- 172.21.15.46:0/1342499596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3a697ed30 0x55e3a699e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1342499596 >> 172.21.15.46:0/1342499596 conn(0x55e3a68c7230 msgr2=0x55e3a68f1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1342499596 shutdown_connections 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1342499596 wait complete. 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 Processor -- start 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 -- start start 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a697ed30 0x55e3a68f0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3a69816f0 0x55e3a6908160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a6981ad0 0x55e3a69086a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a699aa20 con 0x55e3a6981ad0 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3a68f32e0 con 0x55e3a69816f0 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f8ade8080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3a699acc0 con 0x55e3a697ed30 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a6981ad0 0x55e3a69086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a697ed30 0x55e3a68f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f87600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3a69816f0 0x55e3a6908160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a697ed30 0x55e3a68f0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:40834/0 (socket says 172.21.15.46:40834) 2024-04-25T08:43:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a6981ad0 0x55e3a69086a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51154/0 (socket says 172.21.15.46:51154) 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88000640 1 -- 172.21.15.46:0/1058805409 learned_addr learned my addr 172.21.15.46:0/1058805409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88a00640 1 -- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a697ed30 msgr2=0x55e3a68f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88a00640 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e3a697ed30 0x55e3a68f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88a00640 1 -- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3a69816f0 msgr2=0x55e3a6908160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88a00640 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e3a69816f0 0x55e3a6908160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88a00640 1 -- 172.21.15.46:0/1058805409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3a690bf70 con 0x55e3a6981ad0 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.163+0000 7f0f88a00640 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a6981ad0 0x55e3a69086a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000d1b0 tx=0x7f0f8000d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f84e00640 1 -- 172.21.15.46:0/1058805409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80012420 con 0x55e3a6981ad0 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f84e00640 1 -- 172.21.15.46:0/1058805409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f80017070 con 0x55e3a6981ad0 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f84e00640 1 -- 172.21.15.46:0/1058805409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f800127e0 con 0x55e3a6981ad0 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e3a690c210 con 0x55e3a6981ad0 2024-04-25T08:43:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3a690c750 con 0x55e3a6981ad0 2024-04-25T08:43:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f84e00640 1 -- 172.21.15.46:0/1058805409 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f80012b90 con 0x55e3a6981ad0 2024-04-25T08:43:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f84e00640 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f68065970 0x7f0f68067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f84e00640 1 -- 172.21.15.46:0/1058805409 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f80083b50 con 0x55e3a6981ad0 2024-04-25T08:43:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f8ade8080 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55e3a6a34790 0x55e3a6a34b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f88000640 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f68065970 0x7f0f68067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x55e3a6a350b0 con 0x55e3a6a34790 2024-04-25T08:43:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f87600640 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55e3a6a34790 0x55e3a6a34b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.167+0000 7f0f88000640 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f68065970 0x7f0f68067e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f780055e0 tx=0x7f0f7800c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f87600640 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55e3a6a34790 0x55e3a6a34b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f87600640 1 -- 172.21.15.46:0/1058805409 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f0f740b7070 con 0x55e3a6a34790 2024-04-25T08:43:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55e3a6a34790 msgr2=0x55e3a6a34b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55e3a6a34790 0x55e3a6a34b70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f68065970 msgr2=0x7f0f68067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f68065970 0x7f0f68067e30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f780055e0 tx=0x7f0f7800c040 comp rx=0 tx=0).stop 2024-04-25T08:43:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a6981ad0 msgr2=0x55e3a69086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 --2- 172.21.15.46:0/1058805409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3a6981ad0 0x55e3a69086a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000d1b0 tx=0x7f0f8000d670 comp rx=0 tx=0).stop 2024-04-25T08:43:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f88a00640 1 -- 172.21.15.46:0/1058805409 reap_dead start 2024-04-25T08:43:57.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 shutdown_connections 2024-04-25T08:43:57.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 >> 172.21.15.46:0/1058805409 conn(0x55e3a68c7230 msgr2=0x55e3a68f67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:57.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 shutdown_connections 2024-04-25T08:43:57.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.171+0000 7f0f8ade8080 1 -- 172.21.15.46:0/1058805409 wait complete. 2024-04-25T08:43:57.167 INFO:teuthology.orchestra.run.smithi046.stdout:mytag 2024-04-25T08:43:57.167 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T08:43:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe4355a0080 1 Processor -- start 2024-04-25T08:43:57.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe4355a0080 1 -- start start 2024-04-25T08:43:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe4355a0080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6df99c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe4355a0080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557a6e00d560 0x557a6e029b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe4355a0080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00ddb0 0x557a6e02a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe4355a0080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6e00ed10 con 0x557a6df574c0 2024-04-25T08:43:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe4355a0080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6defb9f0 con 0x557a6e00d560 2024-04-25T08:43:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe4355a0080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a6e00efb0 con 0x557a6e00ddb0 2024-04-25T08:43:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe432800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6df99c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe432800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6df99c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51172/0 (socket says 172.21.15.46:51172) 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe432800640 1 -- 172.21.15.46:0/3648751915 learned_addr learned my addr 172.21.15.46:0/3648751915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe433200640 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00ddb0 0x557a6e02a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe431e00640 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557a6e00d560 0x557a6e029b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe432800640 1 -- 172.21.15.46:0/3648751915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00ddb0 msgr2=0x557a6e02a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe432800640 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00ddb0 0x557a6e02a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe432800640 1 -- 172.21.15.46:0/3648751915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557a6e00d560 msgr2=0x557a6e029b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe432800640 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557a6e00d560 0x557a6e029b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe432800640 1 -- 172.21.15.46:0/3648751915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a6e02ad70 con 0x557a6df574c0 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe433200640 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00ddb0 0x557a6e02a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe432800640 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6df99c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe4240ad040 tx=0x7fe424003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb074676492bceca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.263+0000 7fe430a00640 1 -- 172.21.15.46:0/3648751915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240af040 con 0x557a6df574c0 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe430a00640 1 -- 172.21.15.46:0/3648751915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240b9070 con 0x557a6df574c0 2024-04-25T08:43:57.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe430a00640 1 -- 172.21.15.46:0/3648751915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240b4410 con 0x557a6df574c0 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- 172.21.15.46:0/3648751915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 msgr2=0x557a6df99c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6df99c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe4240ad040 tx=0x7fe424003030 comp rx=0 tx=0).stop 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- 172.21.15.46:0/3648751915 shutdown_connections 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00ddb0 0x557a6e02a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557a6e00d560 0x557a6e029b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 --2- 172.21.15.46:0/3648751915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6df99c90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- 172.21.15.46:0/3648751915 >> 172.21.15.46:0/3648751915 conn(0x557a6df55230 msgr2=0x557a6df80a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- 172.21.15.46:0/3648751915 shutdown_connections 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- 172.21.15.46:0/3648751915 wait complete. 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 Processor -- start 2024-04-25T08:43:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- start start 2024-04-25T08:43:57.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6e02de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00d560 0x557a6df80760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557a6e00ddb0 0x557a6e00ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6e02b010 con 0x557a6df574c0 2024-04-25T08:43:57.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a6e025e90 con 0x557a6e00ddb0 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a6df9e680 con 0x557a6e00d560 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe432800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6e02de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe432800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6e02de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51188/0 (socket says 172.21.15.46:51188) 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe432800640 1 -- 172.21.15.46:0/689123653 learned_addr learned my addr 172.21.15.46:0/689123653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe431e00640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00d560 0x557a6df80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe433200640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557a6e00ddb0 0x557a6e00ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe432800640 1 -- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00d560 msgr2=0x557a6df80760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe432800640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00d560 0x557a6df80760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe432800640 1 -- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557a6e00ddb0 msgr2=0x557a6e00ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe432800640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557a6e00ddb0 0x557a6e00ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe432800640 1 -- 172.21.15.46:0/689123653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a6e0271a0 con 0x557a6df574c0 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe431e00640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557a6e00d560 0x557a6df80760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:57.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe432800640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6e02de80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe424096dd0 tx=0x7fe424018190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe42b400640 1 -- 172.21.15.46:0/689123653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240af040 con 0x557a6df574c0 2024-04-25T08:43:57.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe42b400640 1 -- 172.21.15.46:0/689123653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240b4820 con 0x557a6df574c0 2024-04-25T08:43:57.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe42b400640 1 -- 172.21.15.46:0/689123653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240b4c10 con 0x557a6df574c0 2024-04-25T08:43:57.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557a6e027440 con 0x557a6df574c0 2024-04-25T08:43:57.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.267+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a6e027980 con 0x557a6df574c0 2024-04-25T08:43:57.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.271+0000 7fe42b400640 1 -- 172.21.15.46:0/689123653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4240c5420 con 0x557a6df574c0 2024-04-25T08:43:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.271+0000 7fe42b400640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe414065970 0x7fe414067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.271+0000 7fe42b400640 1 -- 172.21.15.46:0/689123653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe424120350 con 0x557a6df574c0 2024-04-25T08:43:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.271+0000 7fe4355a0080 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557a6e027e20 0x557a6e026010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.271+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v9 -- 0x557a6e026550 con 0x557a6e027e20 2024-04-25T08:43:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.271+0000 7fe431e00640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe414065970 0x7fe414067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.271+0000 7fe433200640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557a6e027e20 0x557a6e026010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.271+0000 7fe431e00640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe414065970 0x7fe414067e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe42000c600 tx=0x7fe420005830 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.271+0000 7fe433200640 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557a6e027e20 0x557a6e026010 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe433200640 1 -- 172.21.15.46:0/689123653 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe42c00d9b0 con 0x557a6e027e20 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557a6e027e20 msgr2=0x557a6e026010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557a6e027e20 0x557a6e026010 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe414065970 msgr2=0x7fe414067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe414065970 0x7fe414067e30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe42000c600 tx=0x7fe420005830 comp rx=0 tx=0).stop 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 msgr2=0x557a6e02de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 --2- 172.21.15.46:0/689123653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a6df574c0 0x557a6e02de80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe424096dd0 tx=0x7fe424018190 comp rx=0 tx=0).stop 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe433200640 1 -- 172.21.15.46:0/689123653 reap_dead start 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 shutdown_connections 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 >> 172.21.15.46:0/689123653 conn(0x557a6df55230 msgr2=0x557a6df9b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 shutdown_connections 2024-04-25T08:43:57.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.275+0000 7fe4355a0080 1 -- 172.21.15.46:0/689123653 wait complete. 2024-04-25T08:43:57.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:43:57.271 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:43:57.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:43:57.400 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:43:57.484 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:43:57.484 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:57.484 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:57.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:43:57.493 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:43:57.493 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:57.502 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:43:57.502 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:43:57.502 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:57.545 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:43:57.545 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:43:57.545 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:43:57.554 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:43:57.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04efe00640 1 Processor -- start 2024-04-25T08:43:57.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04efe00640 1 -- start start 2024-04-25T08:43:57.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04efe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f04f009a110 0x7f04f015c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04efe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f00d0530 con 0x7f04f015ec80 2024-04-25T08:43:57.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04efe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04f00cc020 con 0x7f04f00cf4d0 2024-04-25T08:43:57.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04efe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04f015f060 con 0x7f04f009a110 2024-04-25T08:43:57.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04eea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f04f015ec80 0x7f04f015fc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56988/0 (socket says 172.21.15.46:56988) 2024-04-25T08:43:57.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04eea00640 1 -- 172.21.15.46:0/2151322307 learned_addr learned my addr 172.21.15.46:0/2151322307 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:57.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04ef400640 1 --2- 172.21.15.46:0/2151322307 >> v2:172.21.15.82:3300/0 conn(0x7f04f009a110 0x7f04f015c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04ef400640 1 -- 172.21.15.46:0/2151322307 >> v1:172.21.15.82:6789/0 conn(0x7f04f00cf4d0 legacy=0x7f04f0099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04ef400640 1 -- 172.21.15.46:0/2151322307 >> v1:172.21.15.46:6789/0 conn(0x7f04f015ec80 legacy=0x7f04f015fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.739+0000 7f04ef400640 1 -- 172.21.15.46:0/2151322307 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f009a9f0 con 0x7f04f009a110 2024-04-25T08:43:57.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04ef400640 1 --2- 172.21.15.46:0/2151322307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f009a110 0x7f04f015c100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009d90 tx=0x7f04e801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8b0ede48d041e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04ed600640 1 -- 172.21.15.46:0/2151322307 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e80b2070 con 0x7f04f009a110 2024-04-25T08:43:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04ed600640 1 -- 172.21.15.46:0/2151322307 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e80a5970 con 0x7f04f009a110 2024-04-25T08:43:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04ed600640 1 -- 172.21.15.46:0/2151322307 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e80a5d40 con 0x7f04f009a110 2024-04-25T08:43:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04efe00640 1 -- 172.21.15.46:0/2151322307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f009a110 msgr2=0x7f04f015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04efe00640 1 --2- 172.21.15.46:0/2151322307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f009a110 0x7f04f015c100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009d90 tx=0x7f04e801c8b0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04efe00640 1 -- 172.21.15.46:0/2151322307 shutdown_connections 2024-04-25T08:43:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04efe00640 1 --2- 172.21.15.46:0/2151322307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f009a110 0x7f04f015c100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04efe00640 1 -- 172.21.15.46:0/2151322307 >> 172.21.15.46:0/2151322307 conn(0x7f04f00b2f90 msgr2=0x7f04f00b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04efe00640 1 -- 172.21.15.46:0/2151322307 shutdown_connections 2024-04-25T08:43:57.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.743+0000 7f04efe00640 1 -- 172.21.15.46:0/2151322307 wait complete. 2024-04-25T08:43:57.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 Processor -- start 2024-04-25T08:43:57.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 -- start start 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04f009a110 0x7f04f006d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f00cf4d0 0x7f04f006c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f015ec80 0x7f04f0166d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f015ca50 con 0x7f04f00cf4d0 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f00d0250 con 0x7f04f015ec80 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04f015cbd0 con 0x7f04f009a110 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f00cf4d0 0x7f04f006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ee000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f00cf4d0 0x7f04f006c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51196/0 (socket says 172.21.15.46:51196) 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ee000640 1 -- 172.21.15.46:0/1271347598 learned_addr learned my addr 172.21.15.46:0/1271347598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04eea00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04f009a110 0x7f04f006d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ef400640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f015ec80 0x7f04f0166d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ee000640 1 -- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04f009a110 msgr2=0x7f04f006d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ee000640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04f009a110 0x7f04f006d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ee000640 1 -- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f015ec80 msgr2=0x7f04f0166d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ee000640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f015ec80 0x7f04f0166d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ee000640 1 -- 172.21.15.46:0/1271347598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f006bdc0 con 0x7f04f00cf4d0 2024-04-25T08:43:57.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04ee000640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f00cf4d0 0x7f04f006c860 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00c500 tx=0x7f04dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04e6a00640 1 -- 172.21.15.46:0/1271347598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc016020 con 0x7f04f00cf4d0 2024-04-25T08:43:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04e6a00640 1 -- 172.21.15.46:0/1271347598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04dc005220 con 0x7f04f00cf4d0 2024-04-25T08:43:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f006d9d0 con 0x7f04f00cf4d0 2024-04-25T08:43:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04e6a00640 1 -- 172.21.15.46:0/1271347598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04dc007030 con 0x7f04f00cf4d0 2024-04-25T08:43:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f006d6c0 con 0x7f04f00cf4d0 2024-04-25T08:43:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.747+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f0069010 con 0x7f04f00cf4d0 2024-04-25T08:43:57.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.751+0000 7f04e6a00640 1 -- 172.21.15.46:0/1271347598 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04dc0071f0 con 0x7f04f00cf4d0 2024-04-25T08:43:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.751+0000 7f04e6a00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f04d0066f90 0x7f04d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.751+0000 7f04e6a00640 1 -- 172.21.15.46:0/1271347598 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04dc084a10 con 0x7f04f00cf4d0 2024-04-25T08:43:57.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.751+0000 7f04eea00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f04d0066f90 0x7f04d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:57.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.751+0000 7f04eea00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f04d0066f90 0x7f04d0069450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f04e0005890 tx=0x7f04e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:57.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.755+0000 7f04e6a00640 1 -- 172.21.15.46:0/1271347598 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04dc020020 con 0x7f04f00cf4d0 2024-04-25T08:43:57.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04f009a530 con 0x7f04f00cf4d0 2024-04-25T08:43:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04e6a00640 1 -- 172.21.15.46:0/1271347598 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f04dc055250 con 0x7f04f00cf4d0 2024-04-25T08:43:57.943 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:57.943 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1079752579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1079752579},{"type":"v1","addr":"172.21.15.82:6836","nonce":1079752579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/106980078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":106980078},{"type":"v1","addr":"172.21.15.82:6837","nonce":106980078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1318225485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1318225485},{"type":"v1","addr":"172.21.15.46:6834","nonce":1318225485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:43:35.070722+0000","modified":"2024-04-25T08:43:38.080210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2025854544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2025854544},{"type":"v1","addr":"172.21.15.46:6835","nonce":2025854544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:43:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-25T08:43:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f04d0066f90 msgr2=0x7f04d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f04d0066f90 0x7f04d0069450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f04e0005890 tx=0x7f04e0009040 comp rx=0 tx=0).stop 2024-04-25T08:43:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f00cf4d0 msgr2=0x7f04f006c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f00cf4d0 0x7f04f006c860 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00c500 tx=0x7f04dc00c9c0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 shutdown_connections 2024-04-25T08:43:57.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f04d0066f90 0x7f04d0069450 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04f015ec80 0x7f04f0166d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04f00cf4d0 0x7f04f006c860 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 --2- 172.21.15.46:0/1271347598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f04f009a110 0x7f04f006d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 >> 172.21.15.46:0/1271347598 conn(0x7f04f00b2f90 msgr2=0x7f04f00701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 shutdown_connections 2024-04-25T08:43:57.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:57.951+0000 7f04efe00640 1 -- 172.21.15.46:0/1271347598 wait complete. 2024-04-25T08:43:57.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.111+0000 7fd5a4000640 1 Processor -- start 2024-04-25T08:43:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.111+0000 7fd5a4000640 1 -- start start 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a4000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf140 0x7fd59c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a4000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd59c0afd40 0x7fd59c15dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a4000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59c06ed70 con 0x7fd59c0cf520 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a4000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59c0afb40 con 0x7fd59c0afd40 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a4000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0d13f0 con 0x7fd59c0cf140 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a1600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf140 0x7fd59c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a0c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd59c0afd40 0x7fd59c15dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a1600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf140 0x7fd59c15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:45304/0 (socket says 172.21.15.46:45304) 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a1600640 1 -- 172.21.15.46:0/1334082777 learned_addr learned my addr 172.21.15.46:0/1334082777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a1600640 1 -- 172.21.15.46:0/1334082777 >> v2:172.21.15.82:3300/0 conn(0x7fd59c0afd40 msgr2=0x7fd59c15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a1600640 1 --2- 172.21.15.46:0/1334082777 >> v2:172.21.15.82:3300/0 conn(0x7fd59c0afd40 0x7fd59c15dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a1600640 1 -- 172.21.15.46:0/1334082777 >> v1:172.21.15.82:6789/0 conn(0x7fd59c0cf520 legacy=0x7fd59c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.115+0000 7fd5a1600640 1 -- 172.21.15.46:0/1334082777 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c162530 con 0x7fd59c0cf140 2024-04-25T08:43:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a1600640 1 --2- 172.21.15.46:0/1334082777 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf140 0x7fd59c15d590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd590009870 tx=0x7fd59001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d2df6adb48620f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd59be00640 1 -- 172.21.15.46:0/1334082777 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900b0070 con 0x7fd59c0cf140 2024-04-25T08:43:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd59be00640 1 -- 172.21.15.46:0/1334082777 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5900ab600 con 0x7fd59c0cf140 2024-04-25T08:43:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd59be00640 1 -- 172.21.15.46:0/1334082777 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900ab9d0 con 0x7fd59c0cf140 2024-04-25T08:43:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 -- 172.21.15.46:0/1334082777 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf140 msgr2=0x7fd59c15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 --2- 172.21.15.46:0/1334082777 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf140 0x7fd59c15d590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd590009870 tx=0x7fd59001c440 comp rx=0 tx=0).stop 2024-04-25T08:43:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 -- 172.21.15.46:0/1334082777 shutdown_connections 2024-04-25T08:43:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 --2- 172.21.15.46:0/1334082777 >> v2:172.21.15.82:3300/0 conn(0x7fd59c0afd40 0x7fd59c15dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 --2- 172.21.15.46:0/1334082777 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf140 0x7fd59c15d590 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 -- 172.21.15.46:0/1334082777 >> 172.21.15.46:0/1334082777 conn(0x7fd59c0728c0 msgr2=0x7fd59c099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 -- 172.21.15.46:0/1334082777 shutdown_connections 2024-04-25T08:43:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 -- 172.21.15.46:0/1334082777 wait complete. 2024-04-25T08:43:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 Processor -- start 2024-04-25T08:43:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.119+0000 7fd5a4000640 1 -- start start 2024-04-25T08:43:58.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a4000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c0afd40 0x7fd59c0ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a4000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd59c0cf140 0x7fd59c0b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a4000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf520 0x7fd59c1609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a4000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c0af7f0 con 0x7fd59c0cf140 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a4000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c162010 con 0x7fd59c0afd40 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a4000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59c06b620 con 0x7fd59c0cf520 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd59c0cf140 0x7fd59c0b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd59c0cf140 0x7fd59c0b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45414/0 (socket says 172.21.15.46:45414) 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a0c00640 1 -- 172.21.15.46:0/83148462 learned_addr learned my addr 172.21.15.46:0/83148462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a1600640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c0afd40 0x7fd59c0ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a2000640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf520 0x7fd59c1609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a0c00640 1 -- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf520 msgr2=0x7fd59c1609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a0c00640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf520 0x7fd59c1609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a0c00640 1 -- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c0afd40 msgr2=0x7fd59c0ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a0c00640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c0afd40 0x7fd59c0ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a0c00640 1 -- 172.21.15.46:0/83148462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59c15ebb0 con 0x7fd59c0cf140 2024-04-25T08:43:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a1600640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c0afd40 0x7fd59c0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a0c00640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd59c0cf140 0x7fd59c0b0b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd58800e8a0 tx=0x7fd58800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd599600640 1 -- 172.21.15.46:0/83148462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588017070 con 0x7fd59c0cf140 2024-04-25T08:43:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd599600640 1 -- 172.21.15.46:0/83148462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58800c940 con 0x7fd59c0cf140 2024-04-25T08:43:58.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c0ae760 con 0x7fd59c0cf140 2024-04-25T08:43:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd599600640 1 -- 172.21.15.46:0/83148462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58800ccf0 con 0x7fd59c0cf140 2024-04-25T08:43:58.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59c0aed20 con 0x7fd59c0cf140 2024-04-25T08:43:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59c069080 con 0x7fd59c0cf140 2024-04-25T08:43:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.123+0000 7fd599600640 1 -- 172.21.15.46:0/83148462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd58801a070 con 0x7fd59c0cf140 2024-04-25T08:43:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.127+0000 7fd599600640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd57c066f60 0x7fd57c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.127+0000 7fd599600640 1 -- 172.21.15.46:0/83148462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd588085100 con 0x7fd59c0cf140 2024-04-25T08:43:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.127+0000 7fd5a1600640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd57c066f60 0x7fd57c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.127+0000 7fd5a1600640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd57c066f60 0x7fd57c069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ad000 tx=0x7fd5900a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.131+0000 7fd599600640 1 -- 172.21.15.46:0/83148462 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd588055f70 con 0x7fd59c0cf140 2024-04-25T08:43:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd59c070330 con 0x7fd59c0cf140 2024-04-25T08:43:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd599600640 1 -- 172.21.15.46:0/83148462 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16894 (secure 0 0 0) 0x7fd588055870 con 0x7fd59c0cf140 2024-04-25T08:43:58.287 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:58.287 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":40,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:36.675414+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:43:32.873190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:43:34.064607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:43:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd57c066f60 msgr2=0x7fd57c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd57c066f60 0x7fd57c069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd5900ad000 tx=0x7fd5900a6000 comp rx=0 tx=0).stop 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd59c0cf140 msgr2=0x7fd59c0b0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd59c0cf140 0x7fd59c0b0b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd58800e8a0 tx=0x7fd58800eca0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 shutdown_connections 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd57c066f60 0x7fd57c069420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd59c0cf520 0x7fd59c1609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd59c0cf140 0x7fd59c0b0b30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 --2- 172.21.15.46:0/83148462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd59c0afd40 0x7fd59c0ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 >> 172.21.15.46:0/83148462 conn(0x7fd59c0728c0 msgr2=0x7fd59c156290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 shutdown_connections 2024-04-25T08:43:58.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.295+0000 7fd5a4000640 1 -- 172.21.15.46:0/83148462 wait complete. 2024-04-25T08:43:58.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:43:32.873190+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:43:34.064607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:43:58.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:43:58.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa937400640 1 Processor -- start 2024-04-25T08:43:58.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa937400640 1 -- start start 2024-04-25T08:43:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa937400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa930143990 0x7fa9300cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa937400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa93006f150 0x7fa9300b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa937400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93015e440 con 0x7fa9301440d0 2024-04-25T08:43:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa937400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93015ed30 con 0x7fa930143990 2024-04-25T08:43:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa937400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93015e6e0 con 0x7fa93006f150 2024-04-25T08:43:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa934a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa930143990 0x7fa9300cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa934a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa930143990 0x7fa9300cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45420/0 (socket says 172.21.15.46:45420) 2024-04-25T08:43:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa934a00640 1 -- 172.21.15.46:0/25997188 learned_addr learned my addr 172.21.15.46:0/25997188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:58.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa935400640 1 --2- 172.21.15.46:0/25997188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa93006f150 0x7fa9300b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa934a00640 1 -- 172.21.15.46:0/25997188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa93006f150 msgr2=0x7fa9300b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:58.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa934a00640 1 --2- 172.21.15.46:0/25997188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa93006f150 0x7fa9300b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa934a00640 1 -- 172.21.15.46:0/25997188 >> v1:172.21.15.46:6789/0 conn(0x7fa9301440d0 legacy=0x7fa93006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa934a00640 1 -- 172.21.15.46:0/25997188 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93015e9c0 con 0x7fa930143990 2024-04-25T08:43:58.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.463+0000 7fa935400640 1 --2- 172.21.15.46:0/25997188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa93006f150 0x7fa9300b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa934a00640 1 --2- 172.21.15.46:0/25997188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 0x7fa9300cecc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa93015eeb0 tx=0x7fa92401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08f0d6abd3b292c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa92f400640 1 -- 172.21.15.46:0/25997188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240b0070 con 0x7fa930143990 2024-04-25T08:43:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa92f400640 1 -- 172.21.15.46:0/25997188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9240ab600 con 0x7fa930143990 2024-04-25T08:43:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa92f400640 1 -- 172.21.15.46:0/25997188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240ab9d0 con 0x7fa930143990 2024-04-25T08:43:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa937400640 1 -- 172.21.15.46:0/25997188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 msgr2=0x7fa9300cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa937400640 1 --2- 172.21.15.46:0/25997188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 0x7fa9300cecc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa93015eeb0 tx=0x7fa92401c540 comp rx=0 tx=0).stop 2024-04-25T08:43:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa937400640 1 -- 172.21.15.46:0/25997188 shutdown_connections 2024-04-25T08:43:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa937400640 1 --2- 172.21.15.46:0/25997188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa93006f150 0x7fa9300b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa937400640 1 --2- 172.21.15.46:0/25997188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 0x7fa9300cecc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa937400640 1 -- 172.21.15.46:0/25997188 >> 172.21.15.46:0/25997188 conn(0x7fa93006ac10 msgr2=0x7fa93006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa937400640 1 -- 172.21.15.46:0/25997188 shutdown_connections 2024-04-25T08:43:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa937400640 1 -- 172.21.15.46:0/25997188 wait complete. 2024-04-25T08:43:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.467+0000 7fa937400640 1 Processor -- start 2024-04-25T08:43:58.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 -- start start 2024-04-25T08:43:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa93006f150 0x7fa930133920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 0x7fa9300b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9301440d0 0x7fa930135b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300b35f0 con 0x7fa930143990 2024-04-25T08:43:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300b3470 con 0x7fa93006f150 2024-04-25T08:43:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300b3770 con 0x7fa9301440d0 2024-04-25T08:43:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 0x7fa9300b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 0x7fa9300b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45434/0 (socket says 172.21.15.46:45434) 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92fe00640 1 -- 172.21.15.46:0/78233566 learned_addr learned my addr 172.21.15.46:0/78233566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa935400640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9301440d0 0x7fa930135b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa934a00640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa93006f150 0x7fa930133920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92fe00640 1 -- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9301440d0 msgr2=0x7fa930135b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92fe00640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9301440d0 0x7fa930135b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92fe00640 1 -- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa93006f150 msgr2=0x7fa930133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92fe00640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa93006f150 0x7fa930133920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92fe00640 1 -- 172.21.15.46:0/78233566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300b4a70 con 0x7fa930143990 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa935400640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9301440d0 0x7fa930135b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa934a00640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa93006f150 0x7fa930133920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92fe00640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 0x7fa9300b0e00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00c370 tx=0x7fa91c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92cc00640 1 -- 172.21.15.46:0/78233566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0052d0 con 0x7fa930143990 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92cc00640 1 -- 172.21.15.46:0/78233566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c005470 con 0x7fa930143990 2024-04-25T08:43:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa92cc00640 1 -- 172.21.15.46:0/78233566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c007260 con 0x7fa930143990 2024-04-25T08:43:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa930133e60 con 0x7fa930143990 2024-04-25T08:43:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9301640c0 con 0x7fa930143990 2024-04-25T08:43:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.471+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa930069010 con 0x7fa930143990 2024-04-25T08:43:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.475+0000 7fa92cc00640 1 -- 172.21.15.46:0/78233566 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa91c007400 con 0x7fa930143990 2024-04-25T08:43:58.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.475+0000 7fa92cc00640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa910066f60 0x7fa910069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.475+0000 7fa92cc00640 1 -- 172.21.15.46:0/78233566 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa91c084780 con 0x7fa930143990 2024-04-25T08:43:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.475+0000 7fa934a00640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa910066f60 0x7fa910069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.475+0000 7fa934a00640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa910066f60 0x7fa910069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa930070c80 tx=0x7fa924009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:58.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.479+0000 7fa92cc00640 1 -- 172.21.15.46:0/78233566 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa91c0554d0 con 0x7fa930143990 2024-04-25T08:43:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.643+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa930139bc0 con 0x7fa930143990 2024-04-25T08:43:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.643+0000 7fa92cc00640 1 -- 172.21.15.46:0/78233566 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa91c054dd0 con 0x7fa930143990 2024-04-25T08:43:58.637 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa910066f60 msgr2=0x7fa910069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa910066f60 0x7fa910069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa930070c80 tx=0x7fa924009040 comp rx=0 tx=0).stop 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 msgr2=0x7fa9300b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 0x7fa9300b0e00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00c370 tx=0x7fa91c00c830 comp rx=0 tx=0).stop 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 shutdown_connections 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa910066f60 0x7fa910069420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa9301440d0 0x7fa930135b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa930143990 0x7fa9300b0e00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 --2- 172.21.15.46:0/78233566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa93006f150 0x7fa930133920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 >> 172.21.15.46:0/78233566 conn(0x7fa93006ac10 msgr2=0x7fa930071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 shutdown_connections 2024-04-25T08:43:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.647+0000 7fa937400640 1 -- 172.21.15.46:0/78233566 wait complete. 2024-04-25T08:43:58.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:43:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.811+0000 7f99b8600640 1 Processor -- start 2024-04-25T08:43:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.811+0000 7f99b8600640 1 -- start start 2024-04-25T08:43:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b8600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99b0143990 0x7f99b00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b01440d0 0x7f99b006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b8600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b015e440 con 0x7f99b006ef80 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b8600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b015ed30 con 0x7f99b0143990 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b00b3730 con 0x7f99b01440d0 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99b0143990 0x7f99b00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99b0143990 0x7f99b00cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45440/0 (socket says 172.21.15.46:45440) 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b5c00640 1 -- 172.21.15.46:0/436747297 learned_addr learned my addr 172.21.15.46:0/436747297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b5200640 1 --2- 172.21.15.46:0/436747297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b01440d0 0x7f99b006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b5c00640 1 -- 172.21.15.46:0/436747297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b01440d0 msgr2=0x7f99b006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b5c00640 1 --2- 172.21.15.46:0/436747297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b01440d0 0x7f99b006ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b5c00640 1 -- 172.21.15.46:0/436747297 >> v1:172.21.15.46:6789/0 conn(0x7f99b006ef80 legacy=0x7f99b00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b5c00640 1 -- 172.21.15.46:0/436747297 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b015e6e0 con 0x7f99b0143990 2024-04-25T08:43:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99b5c00640 1 --2- 172.21.15.46:0/436747297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b0143990 0x7f99b00cecc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009870 tx=0x7f99a401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f301fd4cc3356aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.815+0000 7f99afe00640 1 -- 172.21.15.46:0/436747297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40b0070 con 0x7f99b0143990 2024-04-25T08:43:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99afe00640 1 -- 172.21.15.46:0/436747297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a40ab600 con 0x7f99b0143990 2024-04-25T08:43:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99afe00640 1 -- 172.21.15.46:0/436747297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40ab9d0 con 0x7f99b0143990 2024-04-25T08:43:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 -- 172.21.15.46:0/436747297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b0143990 msgr2=0x7f99b00cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 --2- 172.21.15.46:0/436747297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b0143990 0x7f99b00cecc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009870 tx=0x7f99a401c560 comp rx=0 tx=0).stop 2024-04-25T08:43:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 -- 172.21.15.46:0/436747297 shutdown_connections 2024-04-25T08:43:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 --2- 172.21.15.46:0/436747297 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b01440d0 0x7f99b006ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 --2- 172.21.15.46:0/436747297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b0143990 0x7f99b00cecc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 -- 172.21.15.46:0/436747297 >> 172.21.15.46:0/436747297 conn(0x7f99b006ac10 msgr2=0x7f99b006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 -- 172.21.15.46:0/436747297 shutdown_connections 2024-04-25T08:43:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 -- 172.21.15.46:0/436747297 wait complete. 2024-04-25T08:43:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 Processor -- start 2024-04-25T08:43:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 -- start start 2024-04-25T08:43:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b006ef80 0x7f99b0158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b0143990 0x7f99b0164280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b01440d0 0x7f99b00afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b0143d70 con 0x7f99b006ef80 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b006f390 con 0x7f99b01440d0 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b0143ef0 con 0x7f99b0143990 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b006ef80 0x7f99b0158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b006ef80 0x7f99b0158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45442/0 (socket says 172.21.15.46:45442) 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b5c00640 1 -- 172.21.15.46:0/573479001 learned_addr learned my addr 172.21.15.46:0/573479001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b5200640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b0143990 0x7f99b0164280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b6600640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b01440d0 0x7f99b00afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b5c00640 1 -- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b0143990 msgr2=0x7f99b0164280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b5c00640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b0143990 0x7f99b0164280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b5c00640 1 -- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b01440d0 msgr2=0x7f99b00afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b5c00640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b01440d0 0x7f99b00afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.819+0000 7f99b5c00640 1 -- 172.21.15.46:0/573479001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b0164b80 con 0x7f99b006ef80 2024-04-25T08:43:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99b5c00640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b006ef80 0x7f99b0158b00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f99a40ae040 tx=0x7f99a401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99ad600640 1 -- 172.21.15.46:0/573479001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40ba020 con 0x7f99b006ef80 2024-04-25T08:43:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99ad600640 1 -- 172.21.15.46:0/573479001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a40b6640 con 0x7f99b006ef80 2024-04-25T08:43:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99ad600640 1 -- 172.21.15.46:0/573479001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a40b0040 con 0x7f99b006ef80 2024-04-25T08:43:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b0164e60 con 0x7f99b006ef80 2024-04-25T08:43:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b0137da0 con 0x7f99b006ef80 2024-04-25T08:43:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b0069010 con 0x7f99b006ef80 2024-04-25T08:43:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99ad600640 1 -- 172.21.15.46:0/573479001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99a40bd020 con 0x7f99b006ef80 2024-04-25T08:43:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99ad600640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9990066f60 0x7f9990069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99ad600640 1 -- 172.21.15.46:0/573479001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99a411f400 con 0x7f99b006ef80 2024-04-25T08:43:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.823+0000 7f99b5200640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9990066f60 0x7f9990069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.827+0000 7f99b5200640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9990066f60 0x7f9990069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f99b00afb80 tx=0x7f999c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:58.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.827+0000 7f99ad600640 1 -- 172.21.15.46:0/573479001 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99a40c4030 con 0x7f99b006ef80 2024-04-25T08:43:58.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f99b006b290 con 0x7f99b006ef80 2024-04-25T08:43:58.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99ad600640 1 -- 172.21.15.46:0/573479001 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f99a4016150 con 0x7f99b006ef80 2024-04-25T08:43:58.984 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:43:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9990066f60 msgr2=0x7f9990069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99b8600640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9990066f60 0x7f9990069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f99b00afb80 tx=0x7f999c008040 comp rx=0 tx=0).stop 2024-04-25T08:43:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b006ef80 msgr2=0x7f99b0158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99b8600640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b006ef80 0x7f99b0158b00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f99a40ae040 tx=0x7f99a401ada0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 shutdown_connections 2024-04-25T08:43:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99b8600640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9990066f60 0x7f9990069420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99b8600640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b01440d0 0x7f99b00afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.991+0000 7f99b8600640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f99b0143990 0x7f99b0164280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.995+0000 7f99b8600640 1 --2- 172.21.15.46:0/573479001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99b006ef80 0x7f99b0158b00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:58.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.995+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 >> 172.21.15.46:0/573479001 conn(0x7f99b006ac10 msgr2=0x7f99b00cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:58.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.995+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 shutdown_connections 2024-04-25T08:43:58.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:58.995+0000 7f99b8600640 1 -- 172.21.15.46:0/573479001 wait complete. 2024-04-25T08:43:59.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:43:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.155+0000 7f9d6be00640 1 Processor -- start 2024-04-25T08:43:59.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6be00640 1 -- start start 2024-04-25T08:43:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 0x7f9d6c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c15def0 con 0x7f9d6c099a00 2024-04-25T08:43:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d6c0d0b60 con 0x7f9d6c070430 2024-04-25T08:43:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0cbaf0 con 0x7f9d6c15db10 2024-04-25T08:43:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d62000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9d6c099a00 0x7f9d6c0cc540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58072/0 (socket says 172.21.15.46:58072) 2024-04-25T08:43:59.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d62000640 1 -- 172.21.15.46:0/3210113586 learned_addr learned my addr 172.21.15.46:0/3210113586 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6aa00640 1 --2- 172.21.15.46:0/3210113586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 0x7f9d6c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6a000640 1 -- 172.21.15.46:0/3210113586 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687560472 0 0) 0x7f9d6c15def0 con 0x7f9d6c099a00 2024-04-25T08:43:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6a000640 1 -- 172.21.15.46:0/3210113586 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d50003530 con 0x7f9d6c099a00 2024-04-25T08:43:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6a000640 1 -- 172.21.15.46:0/3210113586 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445299287 0 0) 0x7f9d6c0d0b60 con 0x7f9d6c070430 2024-04-25T08:43:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6a000640 1 -- 172.21.15.46:0/3210113586 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d6c15def0 con 0x7f9d6c070430 2024-04-25T08:43:59.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6a000640 1 -- 172.21.15.46:0/3210113586 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3165185340 0 0) 0x7f9d50003530 con 0x7f9d6c099a00 2024-04-25T08:43:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6aa00640 1 -- 172.21.15.46:0/3210113586 >> v1:172.21.15.82:6789/0 conn(0x7f9d6c070430 legacy=0x7f9d6c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6aa00640 1 -- 172.21.15.46:0/3210113586 >> v1:172.21.15.46:6789/0 conn(0x7f9d6c099a00 legacy=0x7f9d6c0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.159+0000 7f9d6aa00640 1 -- 172.21.15.46:0/3210113586 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0d0e40 con 0x7f9d6c15db10 2024-04-25T08:43:59.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6aa00640 1 --2- 172.21.15.46:0/3210113586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 0x7f9d6c0cf7d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009dc0 tx=0x7f9d5c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=881b5e75594183f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6a000640 1 -- 172.21.15.46:0/3210113586 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54001600 con 0x7f9d6c15db10 2024-04-25T08:43:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6a000640 1 -- 172.21.15.46:0/3210113586 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c0a6500 con 0x7f9d6c15db10 2024-04-25T08:43:59.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6a000640 1 -- 172.21.15.46:0/3210113586 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0a68d0 con 0x7f9d6c15db10 2024-04-25T08:43:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6be00640 1 -- 172.21.15.46:0/3210113586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 msgr2=0x7f9d6c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6be00640 1 --2- 172.21.15.46:0/3210113586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 0x7f9d6c0cf7d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009dc0 tx=0x7f9d5c01c8b0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6be00640 1 -- 172.21.15.46:0/3210113586 shutdown_connections 2024-04-25T08:43:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6be00640 1 --2- 172.21.15.46:0/3210113586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 0x7f9d6c0cf7d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6be00640 1 -- 172.21.15.46:0/3210113586 >> 172.21.15.46:0/3210113586 conn(0x7f9d6c0b2f90 msgr2=0x7f9d6c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6be00640 1 -- 172.21.15.46:0/3210113586 shutdown_connections 2024-04-25T08:43:59.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6be00640 1 -- 172.21.15.46:0/3210113586 wait complete. 2024-04-25T08:43:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6be00640 1 Processor -- start 2024-04-25T08:43:59.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.163+0000 7f9d6be00640 1 -- start start 2024-04-25T08:43:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d6c070430 0x7f9d6c0b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c099a00 0x7f9d6c0c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 0x7f9d6c0c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c16be60 con 0x7f9d6c070430 2024-04-25T08:43:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c0d07f0 con 0x7f9d6c099a00 2024-04-25T08:43:59.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c16bfe0 con 0x7f9d6c15db10 2024-04-25T08:43:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d6c070430 0x7f9d6c0b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d6c070430 0x7f9d6c0b67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45456/0 (socket says 172.21.15.46:45456) 2024-04-25T08:43:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6aa00640 1 -- 172.21.15.46:0/3152970072 learned_addr learned my addr 172.21.15.46:0/3152970072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6aa00640 1 -- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 msgr2=0x7f9d6c0c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6aa00640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 0x7f9d6c0c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6aa00640 1 -- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c099a00 msgr2=0x7f9d6c0c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6aa00640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c099a00 0x7f9d6c0c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6aa00640 1 -- 172.21.15.46:0/3152970072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c0b3bd0 con 0x7f9d6c070430 2024-04-25T08:43:59.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6aa00640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d6c070430 0x7f9d6c0b67d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c01af60 tx=0x7f9d5c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d63400640 1 -- 172.21.15.46:0/3152970072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c009450 con 0x7f9d6c070430 2024-04-25T08:43:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d63400640 1 -- 172.21.15.46:0/3152970072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5c0a6e40 con 0x7f9d6c070430 2024-04-25T08:43:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c154140 con 0x7f9d6c070430 2024-04-25T08:43:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d63400640 1 -- 172.21.15.46:0/3152970072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0b8880 con 0x7f9d6c070430 2024-04-25T08:43:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c154700 con 0x7f9d6c070430 2024-04-25T08:43:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.167+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c069010 con 0x7f9d6c070430 2024-04-25T08:43:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.171+0000 7f9d63400640 1 -- 172.21.15.46:0/3152970072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d5c0c3020 con 0x7f9d6c070430 2024-04-25T08:43:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.171+0000 7f9d63400640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d48066f60 0x7f9d48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.171+0000 7f9d63400640 1 -- 172.21.15.46:0/3152970072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9d5c11fb30 con 0x7f9d6c070430 2024-04-25T08:43:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.171+0000 7f9d62000640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d48066f60 0x7f9d48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.171+0000 7f9d62000640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d48066f60 0x7f9d48069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c1612f0 tx=0x7f9d54001240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.175+0000 7f9d63400640 1 -- 172.21.15.46:0/3152970072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d5c0ca020 con 0x7f9d6c070430 2024-04-25T08:43:59.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.335+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d6c0b0b40 con 0x7f9d6c070430 2024-04-25T08:43:59.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.335+0000 7f9d63400640 1 -- 172.21.15.46:0/3152970072 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d5c0164a0 con 0x7f9d6c070430 2024-04-25T08:43:59.329 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:43:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d48066f60 msgr2=0x7f9d48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d48066f60 0x7f9d48069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c1612f0 tx=0x7f9d54001240 comp rx=0 tx=0).stop 2024-04-25T08:43:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d6c070430 msgr2=0x7f9d6c0b67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d6c070430 0x7f9d6c0b67d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c01af60 tx=0x7f9d5c019650 comp rx=0 tx=0).stop 2024-04-25T08:43:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 shutdown_connections 2024-04-25T08:43:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d48066f60 0x7f9d48069420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d6c15db10 0x7f9d6c0c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c099a00 0x7f9d6c0c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 --2- 172.21.15.46:0/3152970072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d6c070430 0x7f9d6c0b67d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 >> 172.21.15.46:0/3152970072 conn(0x7f9d6c0b2f90 msgr2=0x7f9d6c158580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 shutdown_connections 2024-04-25T08:43:59.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.339+0000 7f9d6be00640 1 -- 172.21.15.46:0/3152970072 wait complete. 2024-04-25T08:43:59.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:59.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd9a00640 1 Processor -- start 2024-04-25T08:43:59.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd9a00640 1 -- start start 2024-04-25T08:43:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4157b00 0x7f5dd40b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd9a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd40b25a0 con 0x7f5dd40cc150 2024-04-25T08:43:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd9a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd40cc6f0 con 0x7f5dd40ce420 2024-04-25T08:43:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40b2eb0 con 0x7f5dd4157b00 2024-04-25T08:43:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dca000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5dd40cc150 0x7f5dd40cdd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58088/0 (socket says 172.21.15.46:58088) 2024-04-25T08:43:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dca000640 1 -- 172.21.15.46:0/3225176309 learned_addr learned my addr 172.21.15.46:0/3225176309 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:43:59.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2a00640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4157b00 0x7f5dd40b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812831791 0 0) 0x7f5dd40b25a0 con 0x7f5dd40cc150 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db8003530 con 0x7f5dd40cc150 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620574551 0 0) 0x7f5dd40cc6f0 con 0x7f5dd40ce420 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dd40b25a0 con 0x7f5dd40ce420 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1361504076 0 0) 0x7f5db8003530 con 0x7f5dd40cc150 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4157b00 msgr2=0x7f5dd40b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4157b00 0x7f5dd40b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 >> v1:172.21.15.82:6789/0 conn(0x7f5dd40ce420 legacy=0x7f5dd415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40b2860 con 0x7f5dd40cc150 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f5dbc001600 con 0x7f5dd40cc150 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2a00640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4157b00 0x7f5dd40b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:43:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 >> v1:172.21.15.46:6789/0 conn(0x7f5dd40cc150 legacy=0x7f5dd40cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4157b00 0x7f5db8003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004110 0x7f5db8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db8004d50 0x7f5db80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40b2eb0 con 0x7f5db8004d50 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db800bae0 con 0x7f5db8004110 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db800bce0 con 0x7f5dd4157b00 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd3400640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db8004d50 0x7f5db80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd2a00640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4157b00 0x7f5db8003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dca000640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004110 0x7f5db8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd3400640 1 -- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4157b00 msgr2=0x7f5db8003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd3400640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4157b00 0x7f5db8003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd3400640 1 -- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004110 msgr2=0x7f5db8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd3400640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004110 0x7f5db8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dd3400640 1 -- 172.21.15.46:0/3225176309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8003530 con 0x7f5db8004d50 2024-04-25T08:43:59.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.499+0000 7f5dca000640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004110 0x7f5db8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd3400640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db8004d50 0x7f5db80051d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc007fd0 tx=0x7f5dcc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5b2e43af09f583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc001c20 con 0x7f5db8004d50 2024-04-25T08:43:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc003ba0 con 0x7f5db8004d50 2024-04-25T08:43:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd2000640 1 -- 172.21.15.46:0/3225176309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0050d0 con 0x7f5db8004d50 2024-04-25T08:43:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd9a00640 1 -- 172.21.15.46:0/3225176309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db8004d50 msgr2=0x7f5db80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd9a00640 1 --2- 172.21.15.46:0/3225176309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db8004d50 0x7f5db80051d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc007fd0 tx=0x7f5dcc01c8e0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd3400640 1 -- 172.21.15.46:0/3225176309 reap_dead start 2024-04-25T08:43:59.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd9a00640 1 -- 172.21.15.46:0/3225176309 shutdown_connections 2024-04-25T08:43:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd9a00640 1 -- 172.21.15.46:0/3225176309 >> 172.21.15.46:0/3225176309 conn(0x7f5dd40b12a0 msgr2=0x7f5dd40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd9a00640 1 -- 172.21.15.46:0/3225176309 shutdown_connections 2024-04-25T08:43:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.503+0000 7f5dd9a00640 1 -- 172.21.15.46:0/3225176309 wait complete. 2024-04-25T08:43:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 Processor -- start 2024-04-25T08:43:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 -- start start 2024-04-25T08:43:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004d50 0x7f5dd4159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd41481c0 0x7f5dd40a8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4148640 0x7f5dd40a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40ce8b0 con 0x7f5dd41481c0 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40b4300 con 0x7f5db8004d50 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40cea30 con 0x7f5dd4148640 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dca000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd41481c0 0x7f5dd40a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dca000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd41481c0 0x7f5dd40a8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45474/0 (socket says 172.21.15.46:45474) 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dca000640 1 -- 172.21.15.46:0/161730760 learned_addr learned my addr 172.21.15.46:0/161730760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd3400640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4148640 0x7f5dd40a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd2a00640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004d50 0x7f5dd4159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dca000640 1 -- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4148640 msgr2=0x7f5dd40a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dca000640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4148640 0x7f5dd40a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dca000640 1 -- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004d50 msgr2=0x7f5dd4159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dca000640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004d50 0x7f5dd4159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dca000640 1 -- 172.21.15.46:0/161730760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd414a7d0 con 0x7f5dd41481c0 2024-04-25T08:43:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dca000640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd41481c0 0x7f5dd40a8850 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc011820 tx=0x7f5dbc011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dcb400640 1 -- 172.21.15.46:0/161730760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc010040 con 0x7f5dd41481c0 2024-04-25T08:43:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dcb400640 1 -- 172.21.15.46:0/161730760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc01a070 con 0x7f5dd41481c0 2024-04-25T08:43:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dcb400640 1 -- 172.21.15.46:0/161730760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00fd60 con 0x7f5dd41481c0 2024-04-25T08:43:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd414aab0 con 0x7f5dd41481c0 2024-04-25T08:43:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.507+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd41597b0 con 0x7f5dd41481c0 2024-04-25T08:43:59.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.511+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd4069080 con 0x7f5dd41481c0 2024-04-25T08:43:59.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.511+0000 7f5dcb400640 1 -- 172.21.15.46:0/161730760 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dbc01f460 con 0x7f5dd41481c0 2024-04-25T08:43:59.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.511+0000 7f5dcb400640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066f90 0x7f5db0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.511+0000 7f5dcb400640 1 -- 172.21.15.46:0/161730760 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5dbc088240 con 0x7f5dd41481c0 2024-04-25T08:43:59.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.511+0000 7f5dd2a00640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066f90 0x7f5db0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.511+0000 7f5dd2a00640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066f90 0x7f5db0069450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40b0dd0 tx=0x7f5dc4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:59.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.515+0000 7f5dcb400640 1 -- 172.21.15.46:0/161730760 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5dbc058b00 con 0x7f5dd41481c0 2024-04-25T08:43:59.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dd40724a0 con 0x7f5dd41481c0 2024-04-25T08:43:59.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dcb400640 1 -- 172.21.15.46:0/161730760 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f5dbc05cba0 con 0x7f5dd41481c0 2024-04-25T08:43:59.707 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:43:59.707 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1079752579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1079752579},{"type":"v1","addr":"172.21.15.82:6836","nonce":1079752579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/106980078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":106980078},{"type":"v1","addr":"172.21.15.82:6837","nonce":106980078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1318225485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1318225485},{"type":"v1","addr":"172.21.15.46:6834","nonce":1318225485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:43:35.070722+0000","modified":"2024-04-25T08:43:38.080210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2025854544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2025854544},{"type":"v1","addr":"172.21.15.46:6835","nonce":2025854544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:43:59.707 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-25T08:43:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066f90 msgr2=0x7f5db0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066f90 0x7f5db0069450 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40b0dd0 tx=0x7f5dc4009190 comp rx=0 tx=0).stop 2024-04-25T08:43:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd41481c0 msgr2=0x7f5dd40a8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd41481c0 0x7f5dd40a8850 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc011820 tx=0x7f5dbc011ce0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 shutdown_connections 2024-04-25T08:43:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5db0066f90 0x7f5db0069450 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5dd4148640 0x7f5dd40a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dd41481c0 0x7f5dd40a8850 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 --2- 172.21.15.46:0/161730760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db8004d50 0x7f5dd4159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 >> 172.21.15.46:0/161730760 conn(0x7f5dd40b12a0 msgr2=0x7f5dd40cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:59.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 shutdown_connections 2024-04-25T08:43:59.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.715+0000 7f5dd9a00640 1 -- 172.21.15.46:0/161730760 wait complete. 2024-04-25T08:43:59.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:43:59.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.875+0000 7f5ef3e00640 1 Processor -- start 2024-04-25T08:43:59.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.875+0000 7f5ef3e00640 1 -- start start 2024-04-25T08:43:59.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5ef3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ef40cf4d0 0x7f5ef4099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5ef3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ef4099f40 0x7f5ef415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5ef3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef40d0530 con 0x7f5ef415ec80 2024-04-25T08:43:59.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5ef3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef415f060 con 0x7f5ef40cf4d0 2024-04-25T08:43:59.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5ef3e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef409a370 con 0x7f5ef4099f40 2024-04-25T08:43:59.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5eebe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ef40cf4d0 0x7f5ef4099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5eebe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ef40cf4d0 0x7f5ef4099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45480/0 (socket says 172.21.15.46:45480) 2024-04-25T08:43:59.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5eebe00640 1 -- 172.21.15.46:0/3428363081 learned_addr learned my addr 172.21.15.46:0/3428363081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5ef3400640 1 --2- 172.21.15.46:0/3428363081 >> v2:172.21.15.82:3300/0 conn(0x7f5ef4099f40 0x7f5ef415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5eebe00640 1 -- 172.21.15.46:0/3428363081 >> v2:172.21.15.82:3300/0 conn(0x7f5ef4099f40 msgr2=0x7f5ef415b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:59.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5eebe00640 1 --2- 172.21.15.46:0/3428363081 >> v2:172.21.15.82:3300/0 conn(0x7f5ef4099f40 0x7f5ef415b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5eebe00640 1 -- 172.21.15.46:0/3428363081 >> v1:172.21.15.82:6789/0 conn(0x7f5ef415ec80 legacy=0x7f5ef415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:59.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.879+0000 7f5eebe00640 1 -- 172.21.15.46:0/3428363081 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40cc020 con 0x7f5ef40cf4d0 2024-04-25T08:43:59.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5eebe00640 1 --2- 172.21.15.46:0/3428363081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef40cf4d0 0x7f5ef4099a00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4073170 tx=0x7f5edc01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db12aa6a609d337d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef2000640 1 -- 172.21.15.46:0/3428363081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0b0070 con 0x7f5ef40cf4d0 2024-04-25T08:43:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef2000640 1 -- 172.21.15.46:0/3428363081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc0ab600 con 0x7f5ef40cf4d0 2024-04-25T08:43:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef2000640 1 -- 172.21.15.46:0/3428363081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0ab9d0 con 0x7f5ef40cf4d0 2024-04-25T08:43:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 -- 172.21.15.46:0/3428363081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef40cf4d0 msgr2=0x7f5ef4099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 --2- 172.21.15.46:0/3428363081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef40cf4d0 0x7f5ef4099a00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4073170 tx=0x7f5edc01c240 comp rx=0 tx=0).stop 2024-04-25T08:43:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 -- 172.21.15.46:0/3428363081 shutdown_connections 2024-04-25T08:43:59.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 --2- 172.21.15.46:0/3428363081 >> v2:172.21.15.82:3300/0 conn(0x7f5ef4099f40 0x7f5ef415b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 --2- 172.21.15.46:0/3428363081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef40cf4d0 0x7f5ef4099a00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 -- 172.21.15.46:0/3428363081 >> 172.21.15.46:0/3428363081 conn(0x7f5ef40b2f90 msgr2=0x7f5ef40b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 -- 172.21.15.46:0/3428363081 shutdown_connections 2024-04-25T08:43:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 -- 172.21.15.46:0/3428363081 wait complete. 2024-04-25T08:43:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 Processor -- start 2024-04-25T08:43:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.883+0000 7f5ef3e00640 1 -- start start 2024-04-25T08:43:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef4099f40 0x7f5ef4099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ef40cf4d0 0x7f5ef41522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef415ec80 0x7f5ef406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40d0250 con 0x7f5ef415ec80 2024-04-25T08:43:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40cff70 con 0x7f5ef4099f40 2024-04-25T08:43:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef415d080 con 0x7f5ef40cf4d0 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef415ec80 0x7f5ef406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef415ec80 0x7f5ef406d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45496/0 (socket says 172.21.15.46:45496) 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3400640 1 -- 172.21.15.46:0/806850883 learned_addr learned my addr 172.21.15.46:0/806850883 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef2a00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef4099f40 0x7f5ef4099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5eebe00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ef40cf4d0 0x7f5ef41522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3400640 1 -- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ef40cf4d0 msgr2=0x7f5ef41522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3400640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ef40cf4d0 0x7f5ef41522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3400640 1 -- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef4099f40 msgr2=0x7f5ef4099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3400640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef4099f40 0x7f5ef4099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3400640 1 -- 172.21.15.46:0/806850883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef4153230 con 0x7f5ef415ec80 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3400640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef415ec80 0x7f5ef406d6b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5eec007e60 tx=0x7f5eec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5eeaa00640 1 -- 172.21.15.46:0/806850883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec014420 con 0x7f5ef415ec80 2024-04-25T08:43:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5eeaa00640 1 -- 172.21.15.46:0/806850883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec019070 con 0x7f5ef415ec80 2024-04-25T08:43:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5eeaa00640 1 -- 172.21.15.46:0/806850883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec014820 con 0x7f5ef415ec80 2024-04-25T08:43:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef4153510 con 0x7f5ef415ec80 2024-04-25T08:43:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef4153ad0 con 0x7f5ef415ec80 2024-04-25T08:43:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef4069010 con 0x7f5ef415ec80 2024-04-25T08:43:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.887+0000 7f5eeaa00640 1 -- 172.21.15.46:0/806850883 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5eec0149c0 con 0x7f5ef415ec80 2024-04-25T08:43:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.891+0000 7f5eeaa00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ed0066f60 0x7f5ed0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.891+0000 7f5eeaa00640 1 -- 172.21.15.46:0/806850883 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5eec085910 con 0x7f5ef415ec80 2024-04-25T08:43:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.891+0000 7f5ef2a00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ed0066f60 0x7f5ed0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:59.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.891+0000 7f5ef2a00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ed0066f60 0x7f5ed0069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4156590 tx=0x7f5ee4007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:59.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:43:59.895+0000 7f5eeaa00640 1 -- 172.21.15.46:0/806850883 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5eec056130 con 0x7f5ef415ec80 2024-04-25T08:44:00.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.055+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ef4153ca0 con 0x7f5ef415ec80 2024-04-25T08:44:00.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.055+0000 7f5eeaa00640 1 -- 172.21.15.46:0/806850883 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f5eec05a1d0 con 0x7f5ef415ec80 2024-04-25T08:44:00.048 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:00.048 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":40,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:43:36.675414+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:43:32.873190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:43:34.064607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ed0066f60 msgr2=0x7f5ed0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ed0066f60 0x7f5ed0069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4156590 tx=0x7f5ee4007e40 comp rx=0 tx=0).stop 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef415ec80 msgr2=0x7f5ef406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef415ec80 0x7f5ef406d6b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5eec007e60 tx=0x7f5eec00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 shutdown_connections 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ed0066f60 0x7f5ed0069420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ef415ec80 0x7f5ef406d6b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ef40cf4d0 0x7f5ef41522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 --2- 172.21.15.46:0/806850883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ef4099f40 0x7f5ef4099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 >> 172.21.15.46:0/806850883 conn(0x7f5ef40b2f90 msgr2=0x7f5ef415f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 shutdown_connections 2024-04-25T08:44:00.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.059+0000 7f5ef3e00640 1 -- 172.21.15.46:0/806850883 wait complete. 2024-04-25T08:44:00.070 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:44:00.070 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f9e800640 1 Processor -- start 2024-04-25T08:44:00.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f9e800640 1 -- start start 2024-04-25T08:44:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f9e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5f9815bdb0 0x7f5f980cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f9e800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f980cda80 0x7f5f9815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f9e800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f980cc700 con 0x7f5f98159230 2024-04-25T08:44:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f9e800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f980cded0 con 0x7f5f9815bdb0 2024-04-25T08:44:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f9e800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f980cc9e0 con 0x7f5f980cda80 2024-04-25T08:44:00.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f8fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5f9815bdb0 0x7f5f980cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f8fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5f9815bdb0 0x7f5f980cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45506/0 (socket says 172.21.15.46:45506) 2024-04-25T08:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f8fe00640 1 -- 172.21.15.46:0/3810017841 learned_addr learned my addr 172.21.15.46:0/3810017841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f97400640 1 --1- 172.21.15.46:0/3810017841 >> v1:172.21.15.82:6789/0 conn(0x7f5f98159230 0x7f5f980cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:60688/0 (socket says 172.21.15.46:60688) 2024-04-25T08:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.219+0000 7f5f97e00640 1 --2- 172.21.15.46:0/3810017841 >> v2:172.21.15.82:3300/0 conn(0x7f5f980cda80 0x7f5f9815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f8fe00640 1 -- 172.21.15.46:0/3810017841 >> v2:172.21.15.82:3300/0 conn(0x7f5f980cda80 msgr2=0x7f5f9815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:00.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f8fe00640 1 --2- 172.21.15.46:0/3810017841 >> v2:172.21.15.82:3300/0 conn(0x7f5f980cda80 0x7f5f9815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f96a00640 1 -- 172.21.15.46:0/3810017841 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134801386 0 0) 0x7f5f980cc700 con 0x7f5f98159230 2024-04-25T08:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f8fe00640 1 -- 172.21.15.46:0/3810017841 >> v1:172.21.15.82:6789/0 conn(0x7f5f98159230 legacy=0x7f5f980cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f8fe00640 1 -- 172.21.15.46:0/3810017841 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f980cccc0 con 0x7f5f9815bdb0 2024-04-25T08:44:00.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f8fe00640 1 --2- 172.21.15.46:0/3810017841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 0x7f5f980cd540 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5f980b2340 tx=0x7f5f8001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf91df2e056220a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f96a00640 1 -- 172.21.15.46:0/3810017841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f800b2070 con 0x7f5f9815bdb0 2024-04-25T08:44:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f96a00640 1 -- 172.21.15.46:0/3810017841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f800ad680 con 0x7f5f9815bdb0 2024-04-25T08:44:00.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f96a00640 1 -- 172.21.15.46:0/3810017841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f800ada50 con 0x7f5f9815bdb0 2024-04-25T08:44:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f9e800640 1 -- 172.21.15.46:0/3810017841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 msgr2=0x7f5f980cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f9e800640 1 --2- 172.21.15.46:0/3810017841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 0x7f5f980cd540 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5f980b2340 tx=0x7f5f8001c490 comp rx=0 tx=0).stop 2024-04-25T08:44:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f9e800640 1 -- 172.21.15.46:0/3810017841 shutdown_connections 2024-04-25T08:44:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f9e800640 1 --2- 172.21.15.46:0/3810017841 >> v2:172.21.15.82:3300/0 conn(0x7f5f980cda80 0x7f5f9815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f9e800640 1 --2- 172.21.15.46:0/3810017841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 0x7f5f980cd540 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f9e800640 1 -- 172.21.15.46:0/3810017841 >> 172.21.15.46:0/3810017841 conn(0x7f5f980b12a0 msgr2=0x7f5f980b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f9e800640 1 -- 172.21.15.46:0/3810017841 shutdown_connections 2024-04-25T08:44:00.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.223+0000 7f5f9e800640 1 -- 172.21.15.46:0/3810017841 wait complete. 2024-04-25T08:44:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 Processor -- start 2024-04-25T08:44:00.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 -- start start 2024-04-25T08:44:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f980cda80 0x7f5f980a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f98159230 0x7f5f98140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 0x7f5f98165830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9815e0d0 con 0x7f5f9815bdb0 2024-04-25T08:44:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9815df50 con 0x7f5f980cda80 2024-04-25T08:44:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9815e250 con 0x7f5f98159230 2024-04-25T08:44:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 0x7f5f98165830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 0x7f5f98165830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45512/0 (socket says 172.21.15.46:45512) 2024-04-25T08:44:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97e00640 1 -- 172.21.15.46:0/1635473420 learned_addr learned my addr 172.21.15.46:0/1635473420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97400640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f980cda80 0x7f5f980a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f8fe00640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f98159230 0x7f5f98140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97e00640 1 -- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f98159230 msgr2=0x7f5f98140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97e00640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f98159230 0x7f5f98140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97e00640 1 -- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f980cda80 msgr2=0x7f5f980a8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97e00640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f980cda80 0x7f5f980a8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97e00640 1 -- 172.21.15.46:0/1635473420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f98166580 con 0x7f5f9815bdb0 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f97e00640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 0x7f5f98165830 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000efc0 tx=0x7f5f9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f8f400640 1 -- 172.21.15.46:0/1635473420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90018020 con 0x7f5f9815bdb0 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f8f400640 1 -- 172.21.15.46:0/1635473420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f90014480 con 0x7f5f9815bdb0 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f98144490 con 0x7f5f9815bdb0 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f8f400640 1 -- 172.21.15.46:0/1635473420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90014870 con 0x7f5f9815bdb0 2024-04-25T08:44:00.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.227+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f98140860 con 0x7f5f9815bdb0 2024-04-25T08:44:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.231+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f98069080 con 0x7f5f9815bdb0 2024-04-25T08:44:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.231+0000 7f5f8f400640 1 -- 172.21.15.46:0/1635473420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f9001b020 con 0x7f5f9815bdb0 2024-04-25T08:44:00.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.231+0000 7f5f8f400640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f74066f60 0x7f5f74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.231+0000 7f5f8f400640 1 -- 172.21.15.46:0/1635473420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f90085b00 con 0x7f5f9815bdb0 2024-04-25T08:44:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.231+0000 7f5f97400640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f74066f60 0x7f5f74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.231+0000 7f5f97400640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f74066f60 0x7f5f74069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5f88007a90 tx=0x7f5f880079e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:00.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.235+0000 7f5f8f400640 1 -- 172.21.15.46:0/1635473420 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f90022070 con 0x7f5f9815bdb0 2024-04-25T08:44:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.431+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f980723d0 con 0x7f5f9815bdb0 2024-04-25T08:44:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.431+0000 7f5f8f400640 1 -- 172.21.15.46:0/1635473420 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f5f90010040 con 0x7f5f9815bdb0 2024-04-25T08:44:00.424 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:00.425 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1079752579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1079752579},{"type":"v1","addr":"172.21.15.82:6836","nonce":1079752579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/106980078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":106980078},{"type":"v1","addr":"172.21.15.82:6837","nonce":106980078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1318225485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1318225485},{"type":"v1","addr":"172.21.15.46:6834","nonce":1318225485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:43:35.070722+0000","modified":"2024-04-25T08:43:38.080210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2025854544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2025854544},{"type":"v1","addr":"172.21.15.46:6835","nonce":2025854544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:44:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-25T08:44:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f74066f60 msgr2=0x7f5f74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f74066f60 0x7f5f74069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5f88007a90 tx=0x7f5f880079e0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 msgr2=0x7f5f98165830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 0x7f5f98165830 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000efc0 tx=0x7f5f9000c490 comp rx=0 tx=0).stop 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 shutdown_connections 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f74066f60 0x7f5f74069420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f9815bdb0 0x7f5f98165830 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f98159230 0x7f5f98140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 --2- 172.21.15.46:0/1635473420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f980cda80 0x7f5f980a8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 >> 172.21.15.46:0/1635473420 conn(0x7f5f980b12a0 msgr2=0x7f5f980b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 shutdown_connections 2024-04-25T08:44:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.435+0000 7f5f9e800640 1 -- 172.21.15.46:0/1635473420 wait complete. 2024-04-25T08:44:00.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:00.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd7a600640 1 Processor -- start 2024-04-25T08:44:00.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd7a600640 1 -- start start 2024-04-25T08:44:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd7a600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd74159230 0x7fcd7415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd7a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 0x7fcd740cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd7a600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd740cc700 con 0x7fcd740cda80 2024-04-25T08:44:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd7a600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd740cc9e0 con 0x7fcd74159230 2024-04-25T08:44:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd7a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740cccc0 con 0x7fcd740cbeb0 2024-04-25T08:44:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd73400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd74159230 0x7fcd7415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd72a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 0x7fcd740cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd73e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcd740cda80 0x7fcd7415f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:60700/0 (socket says 172.21.15.46:60700) 2024-04-25T08:44:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd73e00640 1 -- 172.21.15.46:0/4109451483 learned_addr learned my addr 172.21.15.46:0/4109451483 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd73400640 1 -- 172.21.15.46:0/4109451483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 msgr2=0x7fcd740cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd73400640 1 --2- 172.21.15.46:0/4109451483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 0x7fcd740cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd73400640 1 -- 172.21.15.46:0/4109451483 >> v1:172.21.15.82:6789/0 conn(0x7fcd740cda80 legacy=0x7fcd7415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd72000640 1 -- 172.21.15.46:0/4109451483 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443557187 0 0) 0x7fcd740cc700 con 0x7fcd740cda80 2024-04-25T08:44:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.595+0000 7fcd73400640 1 -- 172.21.15.46:0/4109451483 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd740ccfa0 con 0x7fcd74159230 2024-04-25T08:44:00.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd73400640 1 --2- 172.21.15.46:0/4109451483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd74159230 0x7fcd7415bdb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcd640097c0 tx=0x7fcd6401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aea525fcfe46404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:00.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd72000640 1 -- 172.21.15.46:0/4109451483 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640b0070 con 0x7fcd74159230 2024-04-25T08:44:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd72000640 1 -- 172.21.15.46:0/4109451483 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd640ab600 con 0x7fcd74159230 2024-04-25T08:44:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd72000640 1 -- 172.21.15.46:0/4109451483 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd640ab9d0 con 0x7fcd74159230 2024-04-25T08:44:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd7a600640 1 -- 172.21.15.46:0/4109451483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd74159230 msgr2=0x7fcd7415bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd7a600640 1 --2- 172.21.15.46:0/4109451483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd74159230 0x7fcd7415bdb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcd640097c0 tx=0x7fcd6401c410 comp rx=0 tx=0).stop 2024-04-25T08:44:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd7a600640 1 -- 172.21.15.46:0/4109451483 shutdown_connections 2024-04-25T08:44:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd7a600640 1 --2- 172.21.15.46:0/4109451483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 0x7fcd740cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd7a600640 1 --2- 172.21.15.46:0/4109451483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd74159230 0x7fcd7415bdb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd7a600640 1 -- 172.21.15.46:0/4109451483 >> 172.21.15.46:0/4109451483 conn(0x7fcd740b12a0 msgr2=0x7fcd740b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd7a600640 1 -- 172.21.15.46:0/4109451483 shutdown_connections 2024-04-25T08:44:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.599+0000 7fcd7a600640 1 -- 172.21.15.46:0/4109451483 wait complete. 2024-04-25T08:44:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 Processor -- start 2024-04-25T08:44:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 -- start start 2024-04-25T08:44:00.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 0x7fcd7413d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd740cda80 0x7fcd74144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd74159230 0x7fcd7413b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd74168780 con 0x7fcd740cda80 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740cc4a0 con 0x7fcd74159230 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd74168900 con 0x7fcd740cbeb0 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd740cda80 0x7fcd74144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd72a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd740cda80 0x7fcd74144f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45528/0 (socket says 172.21.15.46:45528) 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd72a00640 1 -- 172.21.15.46:0/1267565426 learned_addr learned my addr 172.21.15.46:0/1267565426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd73e00640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd74159230 0x7fcd7413b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd73400640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 0x7fcd7413d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd72a00640 1 -- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 msgr2=0x7fcd7413d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd72a00640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 0x7fcd7413d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd72a00640 1 -- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd74159230 msgr2=0x7fcd7413b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd72a00640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd74159230 0x7fcd7413b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd72a00640 1 -- 172.21.15.46:0/1267565426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd740b4130 con 0x7fcd740cda80 2024-04-25T08:44:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd72a00640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd740cda80 0x7fcd74144f00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000e8a0 tx=0x7fcd6000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd6b400640 1 -- 172.21.15.46:0/1267565426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60017070 con 0x7fcd740cda80 2024-04-25T08:44:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd6b400640 1 -- 172.21.15.46:0/1267565426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6000c940 con 0x7fcd740cda80 2024-04-25T08:44:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd6b400640 1 -- 172.21.15.46:0/1267565426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6000ccf0 con 0x7fcd740cda80 2024-04-25T08:44:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd740b4410 con 0x7fcd740cda80 2024-04-25T08:44:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7413bd10 con 0x7fcd740cda80 2024-04-25T08:44:00.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.603+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd74069080 con 0x7fcd740cda80 2024-04-25T08:44:00.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.607+0000 7fcd6b400640 1 -- 172.21.15.46:0/1267565426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd6001a070 con 0x7fcd740cda80 2024-04-25T08:44:00.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.607+0000 7fcd6b400640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd54066f90 0x7fcd54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.607+0000 7fcd6b400640 1 -- 172.21.15.46:0/1267565426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd60085350 con 0x7fcd740cda80 2024-04-25T08:44:00.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.607+0000 7fcd73400640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd54066f90 0x7fcd54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.607+0000 7fcd73400640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd54066f90 0x7fcd54069450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcd640ad000 tx=0x7fcd64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:00.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.611+0000 7fcd6b400640 1 -- 172.21.15.46:0/1267565426 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd60055b70 con 0x7fcd740cda80 2024-04-25T08:44:00.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd740cef30 con 0x7fcd740cda80 2024-04-25T08:44:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd6b400640 1 -- 172.21.15.46:0/1267565426 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fcd60059c10 con 0x7fcd740cda80 2024-04-25T08:44:00.803 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:00.803 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1079752579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1079752579},{"type":"v1","addr":"172.21.15.82:6836","nonce":1079752579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/106980078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":106980078},{"type":"v1","addr":"172.21.15.82:6837","nonce":106980078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1318225485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1318225485},{"type":"v1","addr":"172.21.15.46:6834","nonce":1318225485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:43:35.070722+0000","modified":"2024-04-25T08:43:38.080210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/2025854544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2025854544},{"type":"v1","addr":"172.21.15.46:6835","nonce":2025854544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:44:00.803 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-04-25T08:44:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd54066f90 msgr2=0x7fcd54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd54066f90 0x7fcd54069450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcd640ad000 tx=0x7fcd64008040 comp rx=0 tx=0).stop 2024-04-25T08:44:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd740cda80 msgr2=0x7fcd74144f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd740cda80 0x7fcd74144f00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000e8a0 tx=0x7fcd6000eca0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 shutdown_connections 2024-04-25T08:44:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd54066f90 0x7fcd54069450 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd74159230 0x7fcd7413b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd740cda80 0x7fcd74144f00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 --2- 172.21.15.46:0/1267565426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd740cbeb0 0x7fcd7413d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 >> 172.21.15.46:0/1267565426 conn(0x7fcd740b12a0 msgr2=0x7fcd7406ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 shutdown_connections 2024-04-25T08:44:00.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.811+0000 7fcd7a600640 1 -- 172.21.15.46:0/1267565426 wait complete. 2024-04-25T08:44:00.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:00.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2eb3000640 1 Processor -- start 2024-04-25T08:44:00.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2eb3000640 1 -- start start 2024-04-25T08:44:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2eb3000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2eac159230 0x7f2eac15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2eb3000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac0cc700 con 0x7f2eac0cbeb0 2024-04-25T08:44:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2eb3000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac0cc9e0 con 0x7f2eac0cdc50 2024-04-25T08:44:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2eb3000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac0cccc0 con 0x7f2eac159230 2024-04-25T08:44:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2eabe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2eac159230 0x7f2eac15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2ea3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2eac0cbeb0 0x7f2eac0cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58104/0 (socket says 172.21.15.46:58104) 2024-04-25T08:44:00.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2eabe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2eac159230 0x7f2eac15bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45540/0 (socket says 172.21.15.46:45540) 2024-04-25T08:44:00.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.975+0000 7f2ea3400640 1 -- 172.21.15.46:0/2022712802 learned_addr learned my addr 172.21.15.46:0/2022712802 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:00.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.979+0000 7f2eabe00640 1 -- 172.21.15.46:0/2022712802 >> v1:172.21.15.82:6789/0 conn(0x7f2eac0cdc50 legacy=0x7f2eac15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:00.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.979+0000 7f2eabe00640 1 -- 172.21.15.46:0/2022712802 >> v1:172.21.15.46:6789/0 conn(0x7f2eac0cbeb0 legacy=0x7f2eac0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.979+0000 7f2eabe00640 1 -- 172.21.15.46:0/2022712802 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac0ccfa0 con 0x7f2eac159230 2024-04-25T08:44:00.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.979+0000 7f2eabe00640 1 --2- 172.21.15.46:0/2022712802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 0x7f2eac15bdb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0097c0 tx=0x7f2e9c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859fcf6267536cef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:00.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.979+0000 7f2eab400640 1 -- 172.21.15.46:0/2022712802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0b1070 con 0x7f2eac159230 2024-04-25T08:44:00.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.979+0000 7f2eab400640 1 -- 172.21.15.46:0/2022712802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c0ac620 con 0x7f2eac159230 2024-04-25T08:44:00.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.979+0000 7f2eab400640 1 -- 172.21.15.46:0/2022712802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0ac9f0 con 0x7f2eac159230 2024-04-25T08:44:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.979+0000 7f2eb3000640 1 -- 172.21.15.46:0/2022712802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 msgr2=0x7f2eac15bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.979+0000 7f2eb3000640 1 --2- 172.21.15.46:0/2022712802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 0x7f2eac15bdb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0097c0 tx=0x7f2e9c01c3a0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 -- 172.21.15.46:0/2022712802 shutdown_connections 2024-04-25T08:44:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 --2- 172.21.15.46:0/2022712802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 0x7f2eac15bdb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 -- 172.21.15.46:0/2022712802 >> 172.21.15.46:0/2022712802 conn(0x7f2eac0b12a0 msgr2=0x7f2eac0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 -- 172.21.15.46:0/2022712802 shutdown_connections 2024-04-25T08:44:00.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 -- 172.21.15.46:0/2022712802 wait complete. 2024-04-25T08:44:00.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 Processor -- start 2024-04-25T08:44:00.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 -- start start 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eac0cbeb0 0x7f2eac13c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2eac0cdc50 0x7f2eac0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 0x7f2eac0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac15e120 con 0x7f2eac159230 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0cc4a0 con 0x7f2eac0cbeb0 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb3000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac15e2a0 con 0x7f2eac0cdc50 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 0x7f2eac0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 0x7f2eac0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45548/0 (socket says 172.21.15.46:45548) 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb1000640 1 -- 172.21.15.46:0/1996293324 learned_addr learned my addr 172.21.15.46:0/1996293324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eabe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eac0cbeb0 0x7f2eac13c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2ea3400640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2eac0cdc50 0x7f2eac0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb1000640 1 -- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2eac0cdc50 msgr2=0x7f2eac0b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb1000640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2eac0cdc50 0x7f2eac0b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb1000640 1 -- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eac0cbeb0 msgr2=0x7f2eac13c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb1000640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eac0cbeb0 0x7f2eac13c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb1000640 1 -- 172.21.15.46:0/1996293324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac13d1c0 con 0x7f2eac159230 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2eb1000640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 0x7f2eac0b3560 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400c7b0 tx=0x7f2ea400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.983+0000 7f2ea8c00640 1 -- 172.21.15.46:0/1996293324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4017070 con 0x7f2eac159230 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2ea8c00640 1 -- 172.21.15.46:0/1996293324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea4003db0 con 0x7f2eac159230 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2ea8c00640 1 -- 172.21.15.46:0/1996293324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea400ce60 con 0x7f2eac159230 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac13d4a0 con 0x7f2eac159230 2024-04-25T08:44:00.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac13da60 con 0x7f2eac159230 2024-04-25T08:44:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eac069080 con 0x7f2eac159230 2024-04-25T08:44:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2ea8c00640 1 -- 172.21.15.46:0/1996293324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ea40052b0 con 0x7f2eac159230 2024-04-25T08:44:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2ea8c00640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e88066f60 0x7f2e88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:00.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2ea8c00640 1 -- 172.21.15.46:0/1996293324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2ea4084730 con 0x7f2eac159230 2024-04-25T08:44:00.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2eabe00640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e88066f60 0x7f2e88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:00.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.987+0000 7f2eabe00640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e88066f60 0x7f2e88069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0ae000 tx=0x7f2e9c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:00.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:00.991+0000 7f2ea8c00640 1 -- 172.21.15.46:0/1996293324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ea4054eb0 con 0x7f2eac159230 2024-04-25T08:44:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2eac167560 con 0x7f2eac159230 2024-04-25T08:44:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2ea8c00640 1 -- 172.21.15.46:0/1996293324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16942 (secure 0 0 0) 0x7f2ea4058f50 con 0x7f2eac159230 2024-04-25T08:44:01.151 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:01.151 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":41,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:00.748961+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:43:32.873190+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:43:34.064607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.4","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:01.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e88066f60 msgr2=0x7f2e88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e88066f60 0x7f2e88069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c0ae000 tx=0x7f2e9c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:44:01.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 msgr2=0x7f2eac0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 0x7f2eac0b3560 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400c7b0 tx=0x7f2ea400cc70 comp rx=0 tx=0).stop 2024-04-25T08:44:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 shutdown_connections 2024-04-25T08:44:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e88066f60 0x7f2e88069420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eac159230 0x7f2eac0b3560 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2eac0cdc50 0x7f2eac0b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 --2- 172.21.15.46:0/1996293324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eac0cbeb0 0x7f2eac13c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 >> 172.21.15.46:0/1996293324 conn(0x7f2eac0b12a0 msgr2=0x7f2eac0b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 shutdown_connections 2024-04-25T08:44:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.159+0000 7f2eb3000640 1 -- 172.21.15.46:0/1996293324 wait complete. 2024-04-25T08:44:01.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:44:01.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9f400640 1 Processor -- start 2024-04-25T08:44:01.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9f400640 1 -- start start 2024-04-25T08:44:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f98157b00 0x7f3f980b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9f400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f980cc150 0x7f3f980cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9f400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f980b25a0 con 0x7f3f980ce250 2024-04-25T08:44:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9f400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f980b2eb0 con 0x7f3f980cc150 2024-04-25T08:44:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9f400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980cc6f0 con 0x7f3f98157b00 2024-04-25T08:44:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f98157b00 0x7f3f980b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f98157b00 0x7f3f980b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:45444/0 (socket says 172.21.15.46:45444) 2024-04-25T08:44:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9ca00640 1 -- 172.21.15.46:0/351681782 learned_addr learned my addr 172.21.15.46:0/351681782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f97e00640 1 --2- 172.21.15.46:0/351681782 >> v2:172.21.15.82:3300/0 conn(0x7f3f980cc150 0x7f3f980cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9ca00640 1 -- 172.21.15.46:0/351681782 >> v2:172.21.15.82:3300/0 conn(0x7f3f980cc150 msgr2=0x7f3f980cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9ca00640 1 --2- 172.21.15.46:0/351681782 >> v2:172.21.15.82:3300/0 conn(0x7f3f980cc150 0x7f3f980cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9ca00640 1 -- 172.21.15.46:0/351681782 >> v1:172.21.15.82:6789/0 conn(0x7f3f980ce250 legacy=0x7f3f9815a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.323+0000 7f3f9ca00640 1 -- 172.21.15.46:0/351681782 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980b2860 con 0x7f3f98157b00 2024-04-25T08:44:01.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f9ca00640 1 --2- 172.21.15.46:0/351681782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f98157b00 0x7f3f980b3340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00bb10 tx=0x7f3f8c01eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d77e90940eb5441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f97400640 1 -- 172.21.15.46:0/351681782 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0b4070 con 0x7f3f98157b00 2024-04-25T08:44:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f97400640 1 -- 172.21.15.46:0/351681782 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c0a7be0 con 0x7f3f98157b00 2024-04-25T08:44:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f97400640 1 -- 172.21.15.46:0/351681782 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c0af440 con 0x7f3f98157b00 2024-04-25T08:44:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f9f400640 1 -- 172.21.15.46:0/351681782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f98157b00 msgr2=0x7f3f980b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f9f400640 1 --2- 172.21.15.46:0/351681782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f98157b00 0x7f3f980b3340 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00bb10 tx=0x7f3f8c01eb70 comp rx=0 tx=0).stop 2024-04-25T08:44:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f9f400640 1 -- 172.21.15.46:0/351681782 shutdown_connections 2024-04-25T08:44:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f9f400640 1 --2- 172.21.15.46:0/351681782 >> v2:172.21.15.82:3300/0 conn(0x7f3f980cc150 0x7f3f980cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f9f400640 1 --2- 172.21.15.46:0/351681782 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f98157b00 0x7f3f980b3340 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f9f400640 1 -- 172.21.15.46:0/351681782 >> 172.21.15.46:0/351681782 conn(0x7f3f980b12a0 msgr2=0x7f3f980b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f9f400640 1 -- 172.21.15.46:0/351681782 shutdown_connections 2024-04-25T08:44:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.327+0000 7f3f9f400640 1 -- 172.21.15.46:0/351681782 wait complete. 2024-04-25T08:44:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 Processor -- start 2024-04-25T08:44:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 -- start start 2024-04-25T08:44:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f980cc150 0x7f3f98151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f980ce250 0x7f3f980afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f98157b00 0x7f3f9806db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9815ace0 con 0x7f3f98157b00 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9815ab60 con 0x7f3f980ce250 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9815ae60 con 0x7f3f980cc150 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f98157b00 0x7f3f9806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f98157b00 0x7f3f9806db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45560/0 (socket says 172.21.15.46:45560) 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9d400640 1 -- 172.21.15.46:0/1330922165 learned_addr learned my addr 172.21.15.46:0/1330922165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9ca00640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f980cc150 0x7f3f98151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f97e00640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f980ce250 0x7f3f980afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9d400640 1 -- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f980cc150 msgr2=0x7f3f98151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9d400640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f980cc150 0x7f3f98151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9d400640 1 -- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f980ce250 msgr2=0x7f3f980afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9d400640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f980ce250 0x7f3f980afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9d400640 1 -- 172.21.15.46:0/1330922165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f981517d0 con 0x7f3f98157b00 2024-04-25T08:44:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9d400640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f98157b00 0x7f3f9806db00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000c7a0 tx=0x7f3f9000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f94c00640 1 -- 172.21.15.46:0/1330922165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90003b80 con 0x7f3f98157b00 2024-04-25T08:44:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f94c00640 1 -- 172.21.15.46:0/1330922165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f90003d20 con 0x7f3f98157b00 2024-04-25T08:44:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f94c00640 1 -- 172.21.15.46:0/1330922165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9000aa90 con 0x7f3f98157b00 2024-04-25T08:44:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f98157f80 con 0x7f3f98157b00 2024-04-25T08:44:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f98151d20 con 0x7f3f98157b00 2024-04-25T08:44:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.331+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f98069080 con 0x7f3f98157b00 2024-04-25T08:44:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.335+0000 7f3f94c00640 1 -- 172.21.15.46:0/1330922165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f9000ac30 con 0x7f3f98157b00 2024-04-25T08:44:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.335+0000 7f3f94c00640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f78066f60 0x7f3f78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.335+0000 7f3f94c00640 1 -- 172.21.15.46:0/1330922165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f3f90085040 con 0x7f3f98157b00 2024-04-25T08:44:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.335+0000 7f3f9ca00640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f78066f60 0x7f3f78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.335+0000 7f3f9ca00640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f78066f60 0x7f3f78069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0b1000 tx=0x7f3f8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:01.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.339+0000 7f3f94c00640 1 -- 172.21.15.46:0/1330922165 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f981517d0 con 0x7f3f98157b00 2024-04-25T08:44:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.535+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3f98071660 con 0x7f3f98157b00 2024-04-25T08:44:01.528 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:44:01.535+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:44:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f94c00640 1 -- 172.21.15.46:0/1330922165 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3f900557b0 con 0x7f3f98157b00 2024-04-25T08:44:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:44:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f78066f60 msgr2=0x7f3f78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f78066f60 0x7f3f78069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0b1000 tx=0x7f3f8c008040 comp rx=0 tx=0).stop 2024-04-25T08:44:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f98157b00 msgr2=0x7f3f9806db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f98157b00 0x7f3f9806db00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3f9000c7a0 tx=0x7f3f9000cc60 comp rx=0 tx=0).stop 2024-04-25T08:44:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 shutdown_connections 2024-04-25T08:44:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f78066f60 0x7f3f78069420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f98157b00 0x7f3f9806db00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f980ce250 0x7f3f980afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 --2- 172.21.15.46:0/1330922165 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f980cc150 0x7f3f98151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 >> 172.21.15.46:0/1330922165 conn(0x7f3f980b12a0 msgr2=0x7f3f980b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.539+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 shutdown_connections 2024-04-25T08:44:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.543+0000 7f3f9f400640 1 -- 172.21.15.46:0/1330922165 wait complete. 2024-04-25T08:44:01.546 INFO:tasks.ceph.mds.c.smithi046.stderr: -6447> 2024-04-25T08:43:39.643+0000 7f5f36800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:44:01.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:44:01.596 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:44:01.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.707+0000 7fdb4cc00640 1 Processor -- start 2024-04-25T08:44:01.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.707+0000 7fdb4cc00640 1 -- start start 2024-04-25T08:44:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb4cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 0x7fdb480cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb4cc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdb481440d0 0x7fdb4806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb4cc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4815e440 con 0x7fdb4806ef80 2024-04-25T08:44:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb4cc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4815ed30 con 0x7fdb481440d0 2024-04-25T08:44:01.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb4cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb480b3730 con 0x7fdb48143990 2024-04-25T08:44:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb46800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdb4806ef80 0x7fdb480cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58110/0 (socket says 172.21.15.46:58110) 2024-04-25T08:44:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb46800640 1 -- 172.21.15.46:0/1835735780 learned_addr learned my addr 172.21.15.46:0/1835735780 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb45e00640 1 --2- 172.21.15.46:0/1835735780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 0x7fdb480cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb3d400640 1 --2- 172.21.15.46:0/1835735780 >> v2:172.21.15.82:3300/0 conn(0x7fdb481440d0 0x7fdb4806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb45400640 1 -- 172.21.15.46:0/1835735780 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527750717 0 0) 0x7fdb4815e440 con 0x7fdb4806ef80 2024-04-25T08:44:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb45400640 1 -- 172.21.15.46:0/1835735780 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb2c003530 con 0x7fdb4806ef80 2024-04-25T08:44:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb45e00640 1 -- 172.21.15.46:0/1835735780 >> v2:172.21.15.82:3300/0 conn(0x7fdb481440d0 msgr2=0x7fdb4806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb45e00640 1 --2- 172.21.15.46:0/1835735780 >> v2:172.21.15.82:3300/0 conn(0x7fdb481440d0 0x7fdb4806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb45e00640 1 -- 172.21.15.46:0/1835735780 >> v1:172.21.15.46:6789/0 conn(0x7fdb4806ef80 legacy=0x7fdb480cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb45e00640 1 -- 172.21.15.46:0/1835735780 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4815e6e0 con 0x7fdb48143990 2024-04-25T08:44:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.711+0000 7fdb3d400640 1 --2- 172.21.15.46:0/1835735780 >> v2:172.21.15.82:3300/0 conn(0x7fdb481440d0 0x7fdb4806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb45e00640 1 --2- 172.21.15.46:0/1835735780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 0x7fdb480cecc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097c0 tx=0x7fdb3801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab8322a6aea5f4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb45400640 1 -- 172.21.15.46:0/1835735780 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380b0070 con 0x7fdb48143990 2024-04-25T08:44:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb45400640 1 -- 172.21.15.46:0/1835735780 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb380a4e50 con 0x7fdb48143990 2024-04-25T08:44:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb45400640 1 -- 172.21.15.46:0/1835735780 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380ab820 con 0x7fdb48143990 2024-04-25T08:44:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 -- 172.21.15.46:0/1835735780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 msgr2=0x7fdb480cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 --2- 172.21.15.46:0/1835735780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 0x7fdb480cecc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097c0 tx=0x7fdb3801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 -- 172.21.15.46:0/1835735780 shutdown_connections 2024-04-25T08:44:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 --2- 172.21.15.46:0/1835735780 >> v2:172.21.15.82:3300/0 conn(0x7fdb481440d0 0x7fdb4806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 --2- 172.21.15.46:0/1835735780 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 0x7fdb480cecc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 -- 172.21.15.46:0/1835735780 >> 172.21.15.46:0/1835735780 conn(0x7fdb4806ac10 msgr2=0x7fdb4806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 -- 172.21.15.46:0/1835735780 shutdown_connections 2024-04-25T08:44:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 -- 172.21.15.46:0/1835735780 wait complete. 2024-04-25T08:44:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 Processor -- start 2024-04-25T08:44:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 -- start start 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb4806ef80 0x7fdb4815f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 0x7fdb481593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb481440d0 0x7fdb480b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48143d70 con 0x7fdb4806ef80 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4806f390 con 0x7fdb481440d0 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb4cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48143ef0 con 0x7fdb48143990 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb45e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb4806ef80 0x7fdb4815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb45e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb4806ef80 0x7fdb4815f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45608/0 (socket says 172.21.15.46:45608) 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb45e00640 1 -- 172.21.15.46:0/4248192312 learned_addr learned my addr 172.21.15.46:0/4248192312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb3d400640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 0x7fdb481593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.715+0000 7fdb46800640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb481440d0 0x7fdb480b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb45e00640 1 -- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 msgr2=0x7fdb481593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb45e00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 0x7fdb481593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb45e00640 1 -- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb481440d0 msgr2=0x7fdb480b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb45e00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb481440d0 0x7fdb480b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:01.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb45e00640 1 -- 172.21.15.46:0/4248192312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb48159fc0 con 0x7fdb4806ef80 2024-04-25T08:44:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb45e00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb4806ef80 0x7fdb4815f460 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdb3801a0d0 tx=0x7fdb3801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb3ea00640 1 -- 172.21.15.46:0/4248192312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380b0050 con 0x7fdb4806ef80 2024-04-25T08:44:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb3ea00640 1 -- 172.21.15.46:0/4248192312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb380bc070 con 0x7fdb4806ef80 2024-04-25T08:44:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb3ea00640 1 -- 172.21.15.46:0/4248192312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380b7690 con 0x7fdb4806ef80 2024-04-25T08:44:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb480b13a0 con 0x7fdb4806ef80 2024-04-25T08:44:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb48137ad0 con 0x7fdb4806ef80 2024-04-25T08:44:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb48069010 con 0x7fdb4806ef80 2024-04-25T08:44:01.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb3ea00640 1 -- 172.21.15.46:0/4248192312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb380b7830 con 0x7fdb4806ef80 2024-04-25T08:44:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.719+0000 7fdb3ea00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdb24066f60 0x7fdb24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.723+0000 7fdb3ea00640 1 -- 172.21.15.46:0/4248192312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fdb3811f670 con 0x7fdb4806ef80 2024-04-25T08:44:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.723+0000 7fdb3d400640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdb24066f60 0x7fdb24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.723+0000 7fdb3d400640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdb24066f60 0x7fdb24069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009580 tx=0x7fdb30009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.723+0000 7fdb3ea00640 1 -- 172.21.15.46:0/4248192312 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb48159fc0 con 0x7fdb4806ef80 2024-04-25T08:44:01.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:01.923+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdb480ced70 con 0x7fdb4806ef80 2024-04-25T08:44:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.539+0000 7fdb3ea00640 1 -- 172.21.15.46:0/4248192312 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdb38016090 con 0x7fdb4806ef80 2024-04-25T08:44:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdb24066f60 msgr2=0x7fdb24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdb24066f60 0x7fdb24069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009580 tx=0x7fdb30009190 comp rx=0 tx=0).stop 2024-04-25T08:44:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb4806ef80 msgr2=0x7fdb4815f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb4806ef80 0x7fdb4815f460 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdb3801a0d0 tx=0x7fdb3801ace0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 shutdown_connections 2024-04-25T08:44:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdb24066f60 0x7fdb24069420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb481440d0 0x7fdb480b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdb48143990 0x7fdb481593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 --2- 172.21.15.46:0/4248192312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb4806ef80 0x7fdb4815f460 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 >> 172.21.15.46:0/4248192312 conn(0x7fdb4806ac10 msgr2=0x7fdb480cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 shutdown_connections 2024-04-25T08:44:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.543+0000 7fdb4cc00640 1 -- 172.21.15.46:0/4248192312 wait complete. 2024-04-25T08:44:02.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:44:02.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.763+0000 7f7b52000640 1 Processor -- start 2024-04-25T08:44:02.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.763+0000 7f7b52000640 1 -- start start 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b52000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b4c0ce190 0x7f7b4c0d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b52000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b4c0d0660 0x7f7b4c0b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b52000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c0b6bb0 con 0x7f7b4c15dc40 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b52000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4c0d0a70 con 0x7f7b4c0d0660 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b52000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4c15e020 con 0x7f7b4c0ce190 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b51600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b4c0d0660 0x7f7b4c0b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b51600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b4c0d0660 0x7f7b4c0b7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45612/0 (socket says 172.21.15.46:45612) 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b51600640 1 -- 172.21.15.46:0/2416286258 learned_addr learned my addr 172.21.15.46:0/2416286258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b43e00640 1 --2- 172.21.15.46:0/2416286258 >> v2:172.21.15.82:3300/0 conn(0x7f7b4c0ce190 0x7f7b4c0d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b51600640 1 -- 172.21.15.46:0/2416286258 >> v2:172.21.15.82:3300/0 conn(0x7f7b4c0ce190 msgr2=0x7f7b4c0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b51600640 1 --2- 172.21.15.46:0/2416286258 >> v2:172.21.15.82:3300/0 conn(0x7f7b4c0ce190 0x7f7b4c0d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b51600640 1 -- 172.21.15.46:0/2416286258 >> v1:172.21.15.82:6789/0 conn(0x7f7b4c15dc40 legacy=0x7f7b4c0d1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:02.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.767+0000 7f7b51600640 1 -- 172.21.15.46:0/2416286258 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c0d23f0 con 0x7f7b4c0d0660 2024-04-25T08:44:02.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b51600640 1 --2- 172.21.15.46:0/2416286258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c0d0660 0x7f7b4c0b7070 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0b6980 tx=0x7f7b4401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6482b609f11b38f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b4be00640 1 -- 172.21.15.46:0/2416286258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b440a5620 con 0x7f7b4c0d0660 2024-04-25T08:44:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b4be00640 1 -- 172.21.15.46:0/2416286258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b440a57c0 con 0x7f7b4c0d0660 2024-04-25T08:44:02.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b4be00640 1 -- 172.21.15.46:0/2416286258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b440a5b90 con 0x7f7b4c0d0660 2024-04-25T08:44:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 -- 172.21.15.46:0/2416286258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c0d0660 msgr2=0x7f7b4c0b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 --2- 172.21.15.46:0/2416286258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c0d0660 0x7f7b4c0b7070 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0b6980 tx=0x7f7b4401c960 comp rx=0 tx=0).stop 2024-04-25T08:44:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 -- 172.21.15.46:0/2416286258 shutdown_connections 2024-04-25T08:44:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 --2- 172.21.15.46:0/2416286258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c0d0660 0x7f7b4c0b7070 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 --2- 172.21.15.46:0/2416286258 >> v2:172.21.15.82:3300/0 conn(0x7f7b4c0ce190 0x7f7b4c0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 -- 172.21.15.46:0/2416286258 >> 172.21.15.46:0/2416286258 conn(0x7f7b4c0b5220 msgr2=0x7f7b4c0b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 -- 172.21.15.46:0/2416286258 shutdown_connections 2024-04-25T08:44:02.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 -- 172.21.15.46:0/2416286258 wait complete. 2024-04-25T08:44:02.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 Processor -- start 2024-04-25T08:44:02.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 -- start start 2024-04-25T08:44:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b4c0ce190 0x7f7b4c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7b4c0d0660 0x7f7b4c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c15dc40 0x7f7b4c0c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c0b7f40 con 0x7f7b4c15dc40 2024-04-25T08:44:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c0b7dc0 con 0x7f7b4c0ce190 2024-04-25T08:44:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b52000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c0b80c0 con 0x7f7b4c0d0660 2024-04-25T08:44:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b51600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c15dc40 0x7f7b4c0c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.771+0000 7f7b51600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c15dc40 0x7f7b4c0c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45618/0 (socket says 172.21.15.46:45618) 2024-04-25T08:44:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b51600640 1 -- 172.21.15.46:0/4204535162 learned_addr learned my addr 172.21.15.46:0/4204535162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b50c00640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b4c0ce190 0x7f7b4c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b43e00640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7b4c0d0660 0x7f7b4c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b51600640 1 -- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7b4c0d0660 msgr2=0x7f7b4c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b51600640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7b4c0d0660 0x7f7b4c0c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b51600640 1 -- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b4c0ce190 msgr2=0x7f7b4c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b51600640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b4c0ce190 0x7f7b4c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b51600640 1 -- 172.21.15.46:0/4204535162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c15e820 con 0x7f7b4c15dc40 2024-04-25T08:44:02.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b51600640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c15dc40 0x7f7b4c0c6320 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7b44019570 tx=0x7f7b44019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:02.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b49600640 1 -- 172.21.15.46:0/4204535162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b440bc020 con 0x7f7b4c15dc40 2024-04-25T08:44:02.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b49600640 1 -- 172.21.15.46:0/4204535162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b440b86b0 con 0x7f7b4c15dc40 2024-04-25T08:44:02.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4c15eb00 con 0x7f7b4c15dc40 2024-04-25T08:44:02.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b49600640 1 -- 172.21.15.46:0/4204535162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44009510 con 0x7f7b4c15dc40 2024-04-25T08:44:02.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c15f0c0 con 0x7f7b4c15dc40 2024-04-25T08:44:02.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c069010 con 0x7f7b4c15dc40 2024-04-25T08:44:02.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b49600640 1 -- 172.21.15.46:0/4204535162 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b440bf020 con 0x7f7b4c15dc40 2024-04-25T08:44:02.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.775+0000 7f7b49600640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7b28066f60 0x7f7b28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:02.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.779+0000 7f7b49600640 1 -- 172.21.15.46:0/4204535162 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f7b4411ff70 con 0x7f7b4c15dc40 2024-04-25T08:44:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.779+0000 7f7b50c00640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7b28066f60 0x7f7b28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.779+0000 7f7b50c00640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7b28066f60 0x7f7b28069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c007fd0 tx=0x7f7b3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:02.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.779+0000 7f7b49600640 1 -- 172.21.15.46:0/4204535162 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b440c6070 con 0x7f7b4c15dc40 2024-04-25T08:44:02.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:02.943+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b4c156ce0 con 0x7f7b4c15dc40 2024-04-25T08:44:03.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.767+0000 7f7b49600640 1 -- 172.21.15.46:0/4204535162 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7b440165e0 con 0x7f7b4c15dc40 2024-04-25T08:44:03.759 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:44:03.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7b28066f60 msgr2=0x7f7b28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7b28066f60 0x7f7b28069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c007fd0 tx=0x7f7b3c009040 comp rx=0 tx=0).stop 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c15dc40 msgr2=0x7f7b4c0c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c15dc40 0x7f7b4c0c6320 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7b44019570 tx=0x7f7b44019700 comp rx=0 tx=0).stop 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 shutdown_connections 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7b28066f60 0x7f7b28069420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b4c15dc40 0x7f7b4c0c6320 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7b4c0d0660 0x7f7b4c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 --2- 172.21.15.46:0/4204535162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b4c0ce190 0x7f7b4c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 >> 172.21.15.46:0/4204535162 conn(0x7f7b4c0b5220 msgr2=0x7f7b4c159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:03.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 shutdown_connections 2024-04-25T08:44:03.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.771+0000 7f7b52000640 1 -- 172.21.15.46:0/4204535162 wait complete. 2024-04-25T08:44:03.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:44:03.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01df000640 1 Processor -- start 2024-04-25T08:44:03.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01df000640 1 -- start start 2024-04-25T08:44:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01df000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01d815dad0 0x7f01d80d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01df000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01d809a110 0x7f01d81622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01df000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d80cf750 con 0x7f01d815deb0 2024-04-25T08:44:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01df000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80d0320 con 0x7f01d809a110 2024-04-25T08:44:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01df000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80cf9f0 con 0x7f01d815dad0 2024-04-25T08:44:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01de600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01d809a110 0x7f01d81622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01de600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01d809a110 0x7f01d81622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45628/0 (socket says 172.21.15.46:45628) 2024-04-25T08:44:03.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01de600640 1 -- 172.21.15.46:0/2715427761 learned_addr learned my addr 172.21.15.46:0/2715427761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:03.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01ddc00640 1 --2- 172.21.15.46:0/2715427761 >> v2:172.21.15.82:3300/0 conn(0x7f01d815dad0 0x7f01d80d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:03.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01de600640 1 -- 172.21.15.46:0/2715427761 >> v2:172.21.15.82:3300/0 conn(0x7f01d815dad0 msgr2=0x7f01d80d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:03.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01de600640 1 --2- 172.21.15.46:0/2715427761 >> v2:172.21.15.82:3300/0 conn(0x7f01d815dad0 0x7f01d80d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:03.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01de600640 1 -- 172.21.15.46:0/2715427761 >> v1:172.21.15.82:6789/0 conn(0x7f01d815deb0 legacy=0x7f01d8099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:03.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.951+0000 7f01de600640 1 -- 172.21.15.46:0/2715427761 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80cfcd0 con 0x7f01d809a110 2024-04-25T08:44:03.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01de600640 1 --2- 172.21.15.46:0/2715427761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d809a110 0x7f01d81622a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f01d8072580 tx=0x7f01d001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20e5980aff55410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:03.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01d7e00640 1 -- 172.21.15.46:0/2715427761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00a5860 con 0x7f01d809a110 2024-04-25T08:44:03.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01d7e00640 1 -- 172.21.15.46:0/2715427761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00a5a00 con 0x7f01d809a110 2024-04-25T08:44:03.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01d7e00640 1 -- 172.21.15.46:0/2715427761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00a5dd0 con 0x7f01d809a110 2024-04-25T08:44:03.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01df000640 1 -- 172.21.15.46:0/2715427761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d809a110 msgr2=0x7f01d81622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:03.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01df000640 1 --2- 172.21.15.46:0/2715427761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d809a110 0x7f01d81622a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f01d8072580 tx=0x7f01d001c980 comp rx=0 tx=0).stop 2024-04-25T08:44:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01df000640 1 -- 172.21.15.46:0/2715427761 shutdown_connections 2024-04-25T08:44:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01df000640 1 --2- 172.21.15.46:0/2715427761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d809a110 0x7f01d81622a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01df000640 1 --2- 172.21.15.46:0/2715427761 >> v2:172.21.15.82:3300/0 conn(0x7f01d815dad0 0x7f01d80d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01df000640 1 -- 172.21.15.46:0/2715427761 >> 172.21.15.46:0/2715427761 conn(0x7f01d80b2f70 msgr2=0x7f01d80b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01df000640 1 -- 172.21.15.46:0/2715427761 shutdown_connections 2024-04-25T08:44:03.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.955+0000 7f01df000640 1 -- 172.21.15.46:0/2715427761 wait complete. 2024-04-25T08:44:03.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 Processor -- start 2024-04-25T08:44:03.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 -- start start 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d809a110 0x7f01d815a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d815dad0 0x7f01d80c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01d815deb0 0x7f01d80c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80cc670 con 0x7f01d815dad0 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80cc4f0 con 0x7f01d809a110 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80cc7f0 con 0x7f01d815deb0 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01dd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d815dad0 0x7f01d80c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01dd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d815dad0 0x7f01d80c57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45640/0 (socket says 172.21.15.46:45640) 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01dd200640 1 -- 172.21.15.46:0/1581690507 learned_addr learned my addr 172.21.15.46:0/1581690507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01ddc00640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d809a110 0x7f01d815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01de600640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01d815deb0 0x7f01d80c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:03.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01dd200640 1 -- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01d815deb0 msgr2=0x7f01d80c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01dd200640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01d815deb0 0x7f01d80c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01dd200640 1 -- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d809a110 msgr2=0x7f01d815a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01dd200640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d809a110 0x7f01d815a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01dd200640 1 -- 172.21.15.46:0/1581690507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d815f100 con 0x7f01d815dad0 2024-04-25T08:44:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01dd200640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d815dad0 0x7f01d80c57a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b410 tx=0x7f01c400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:03.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01d5600640 1 -- 172.21.15.46:0/1581690507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4015030 con 0x7f01d815dad0 2024-04-25T08:44:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01d5600640 1 -- 172.21.15.46:0/1581690507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c4004020 con 0x7f01d815dad0 2024-04-25T08:44:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01d5600640 1 -- 172.21.15.46:0/1581690507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4004410 con 0x7f01d815dad0 2024-04-25T08:44:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d815f3e0 con 0x7f01d815dad0 2024-04-25T08:44:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d815f9a0 con 0x7f01d815dad0 2024-04-25T08:44:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.959+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d8069010 con 0x7f01d815dad0 2024-04-25T08:44:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.963+0000 7f01d5600640 1 -- 172.21.15.46:0/1581690507 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01c4018070 con 0x7f01d815dad0 2024-04-25T08:44:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.963+0000 7f01d5600640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f01b8066f60 0x7f01b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.963+0000 7f01d5600640 1 -- 172.21.15.46:0/1581690507 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f01c4084c90 con 0x7f01d815dad0 2024-04-25T08:44:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.963+0000 7f01ddc00640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f01b8066f60 0x7f01b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.963+0000 7f01ddc00640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f01b8066f60 0x7f01b8069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f01d81603e0 tx=0x7f01cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:03.967+0000 7f01d5600640 1 -- 172.21.15.46:0/1581690507 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01c4055bb0 con 0x7f01d815dad0 2024-04-25T08:44:04.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.131+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01d81610f0 con 0x7f01d815dad0 2024-04-25T08:44:04.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.771+0000 7f01d5600640 1 -- 172.21.15.46:0/1581690507 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) v1 ==== 169+0+0 (secure 0 0 0) 0x7f01c40554b0 con 0x7f01d815dad0 2024-04-25T08:44:04.766 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-25T08:44:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f01b8066f60 msgr2=0x7f01b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f01b8066f60 0x7f01b8069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f01d81603e0 tx=0x7f01cc008040 comp rx=0 tx=0).stop 2024-04-25T08:44:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d815dad0 msgr2=0x7f01d80c57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d815dad0 0x7f01d80c57a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b410 tx=0x7f01c400b8d0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 shutdown_connections 2024-04-25T08:44:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f01b8066f60 0x7f01b8069420 secure :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f01d81603e0 tx=0x7f01cc008040 comp rx=0 tx=0).stop 2024-04-25T08:44:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f01d815deb0 0x7f01d80c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d815dad0 0x7f01d80c57a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 --2- 172.21.15.46:0/1581690507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01d809a110 0x7f01d815a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 >> 172.21.15.46:0/1581690507 conn(0x7f01d80b2f70 msgr2=0x7f01d8070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 shutdown_connections 2024-04-25T08:44:04.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.779+0000 7f01df000640 1 -- 172.21.15.46:0/1581690507 wait complete. 2024-04-25T08:44:04.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:44:04.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.951+0000 7f190b000640 1 Processor -- start 2024-04-25T08:44:04.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f190b000640 1 -- start start 2024-04-25T08:44:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f190b000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f190406b320 0x7f190406fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f190b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19041596e0 con 0x7f190415bd50 2024-04-25T08:44:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f190b000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19040cd3d0 con 0x7f1904159300 2024-04-25T08:44:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f190b000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19040cfda0 con 0x7f190406b320 2024-04-25T08:44:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f18fbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f190415bd50 0x7f190406ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58118/0 (socket says 172.21.15.46:58118) 2024-04-25T08:44:04.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f18fbe00640 1 -- 172.21.15.46:0/1922759839 learned_addr learned my addr 172.21.15.46:0/1922759839 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1909000640 1 --2- 172.21.15.46:0/1922759839 >> v2:172.21.15.82:3300/0 conn(0x7f190406b320 0x7f190406fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683500479 0 0) 0x7f19041596e0 con 0x7f190415bd50 2024-04-25T08:44:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e8003530 con 0x7f190415bd50 2024-04-25T08:44:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1319242377 0 0) 0x7f18e8003530 con 0x7f190415bd50 2024-04-25T08:44:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 >> v2:172.21.15.82:3300/0 conn(0x7f190406b320 msgr2=0x7f190406fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 --2- 172.21.15.46:0/1922759839 >> v2:172.21.15.82:3300/0 conn(0x7f190406b320 0x7f190406fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 >> v1:172.21.15.82:6789/0 conn(0x7f1904159300 legacy=0x7f19040cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19040cd6b0 con 0x7f190415bd50 2024-04-25T08:44:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f18ec0016a0 con 0x7f190415bd50 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 >> v1:172.21.15.46:6789/0 conn(0x7f190415bd50 legacy=0x7f190406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18e8003a30 0x7f18e8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8004810 0x7f18e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18e80051d0 0x7f18e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19040cfda0 con 0x7f18e8003a30 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e800c100 con 0x7f18e8004810 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e800c300 con 0x7f18e80051d0 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903e00640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18e8003a30 0x7f18e8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f18fbe00640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8004810 0x7f18e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1909000640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18e80051d0 0x7f18e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903e00640 1 -- 172.21.15.46:0/1922759839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18e80051d0 msgr2=0x7f18e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903e00640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18e80051d0 0x7f18e800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903e00640 1 -- 172.21.15.46:0/1922759839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8004810 msgr2=0x7f18e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903e00640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e8004810 0x7f18e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1903e00640 1 -- 172.21.15.46:0/1922759839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e8003530 con 0x7f18e8003a30 2024-04-25T08:44:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.955+0000 7f1909000640 1 -- 172.21.15.46:0/1922759839 reap_dead start 2024-04-25T08:44:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f1903e00640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18e8003a30 0x7f18e8003e50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f18f4001780 tx=0x7f18f401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa139a8231bdeb39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f40013a0 con 0x7f18e8003a30 2024-04-25T08:44:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f40052f0 con 0x7f18e8003a30 2024-04-25T08:44:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f1903400640 1 -- 172.21.15.46:0/1922759839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4003840 con 0x7f18e8003a30 2024-04-25T08:44:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f190b000640 1 -- 172.21.15.46:0/1922759839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18e8003a30 msgr2=0x7f18e8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f190b000640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18e8003a30 0x7f18e8003e50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f18f4001780 tx=0x7f18f401c3d0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f190b000640 1 -- 172.21.15.46:0/1922759839 shutdown_connections 2024-04-25T08:44:04.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f190b000640 1 --2- 172.21.15.46:0/1922759839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18e8003a30 0x7f18e8003e50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f190b000640 1 -- 172.21.15.46:0/1922759839 >> 172.21.15.46:0/1922759839 conn(0x7f190406e3e0 msgr2=0x7f190406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f190b000640 1 -- 172.21.15.46:0/1922759839 shutdown_connections 2024-04-25T08:44:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.959+0000 7f190b000640 1 -- 172.21.15.46:0/1922759839 wait complete. 2024-04-25T08:44:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 Processor -- start 2024-04-25T08:44:04.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 -- start start 2024-04-25T08:44:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18e8003a30 0x7f1904155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19040b1260 0x7f19040cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19040b16e0 0x7f19040cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190406fb00 con 0x7f19040b16e0 2024-04-25T08:44:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19040cd190 con 0x7f19040b1260 2024-04-25T08:44:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190406fc80 con 0x7f18e8003a30 2024-04-25T08:44:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1909000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19040b16e0 0x7f19040cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1909000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19040b16e0 0x7f19040cbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45660/0 (socket says 172.21.15.46:45660) 2024-04-25T08:44:04.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1909000640 1 -- 172.21.15.46:0/156316859 learned_addr learned my addr 172.21.15.46:0/156316859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f18fbe00640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19040b1260 0x7f19040cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1903e00640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18e8003a30 0x7f1904155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1909000640 1 -- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18e8003a30 msgr2=0x7f1904155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1909000640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18e8003a30 0x7f1904155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1909000640 1 -- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19040b1260 msgr2=0x7f19040cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1909000640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19040b1260 0x7f19040cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1909000640 1 -- 172.21.15.46:0/156316859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19040afee0 con 0x7f19040b16e0 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1909000640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19040b16e0 0x7f19040cbc40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f190415ce50 tx=0x7f18fc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1900c00640 1 -- 172.21.15.46:0/156316859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc011db0 con 0x7f19040b16e0 2024-04-25T08:44:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1900c00640 1 -- 172.21.15.46:0/156316859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc01b070 con 0x7f19040b16e0 2024-04-25T08:44:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f1900c00640 1 -- 172.21.15.46:0/156316859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0039a0 con 0x7f19040b16e0 2024-04-25T08:44:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19040b01c0 con 0x7f19040b16e0 2024-04-25T08:44:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.963+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19040b0780 con 0x7f19040b16e0 2024-04-25T08:44:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.967+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1904069010 con 0x7f19040b16e0 2024-04-25T08:44:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.967+0000 7f1900c00640 1 -- 172.21.15.46:0/156316859 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18fc003b40 con 0x7f19040b16e0 2024-04-25T08:44:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.967+0000 7f1900c00640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18e0066f90 0x7f18e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.967+0000 7f1900c00640 1 -- 172.21.15.46:0/156316859 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f18fc085520 con 0x7f19040b16e0 2024-04-25T08:44:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.967+0000 7f1903e00640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18e0066f90 0x7f18e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.967+0000 7f1903e00640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18e0066f90 0x7f18e0069450 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18f4001750 tx=0x7f18f40011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:04.971+0000 7f1900c00640 1 -- 172.21.15.46:0/156316859 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18fc055fb0 con 0x7f19040b16e0 2024-04-25T08:44:05.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.135+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f190415ac50 con 0x7f19040b16e0 2024-04-25T08:44:05.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f1900c00640 1 -- 172.21.15.46:0/156316859 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f18fc05a050 con 0x7f19040b16e0 2024-04-25T08:44:05.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18e0066f90 msgr2=0x7f18e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18e0066f90 0x7f18e0069450 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f18f4001750 tx=0x7f18f40011a0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19040b16e0 msgr2=0x7f19040cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19040b16e0 0x7f19040cbc40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f190415ce50 tx=0x7f18fc0115a0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 shutdown_connections 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18e0066f90 0x7f18e0069450 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19040b16e0 0x7f19040cbc40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19040b1260 0x7f19040cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 --2- 172.21.15.46:0/156316859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18e8003a30 0x7f1904155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 >> 172.21.15.46:0/156316859 conn(0x7f190406e3e0 msgr2=0x7f1904159c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 shutdown_connections 2024-04-25T08:44:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.139+0000 7f190b000640 1 -- 172.21.15.46:0/156316859 wait complete. 2024-04-25T08:44:05.153 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T08:44:05.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.307+0000 7fb2d5800640 1 Processor -- start 2024-04-25T08:44:05.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.307+0000 7fb2d5800640 1 -- start start 2024-04-25T08:44:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2d5800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f320 0x7fb2d0162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2d5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2d00aed10 0x7fb2d0164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2d5800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d00b35b0 con 0x7fb2d015a750 2024-04-25T08:44:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2d5800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d015ab30 con 0x7fb2d00aed10 2024-04-25T08:44:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2d5800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00ce4f0 con 0x7fb2d006f320 2024-04-25T08:44:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2cf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2d00aed10 0x7fb2d0164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2cf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2d00aed10 0x7fb2d0164750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45670/0 (socket says 172.21.15.46:45670) 2024-04-25T08:44:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2cf400640 1 -- 172.21.15.46:0/3233677368 learned_addr learned my addr 172.21.15.46:0/3233677368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:05.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2c6000640 1 --2- 172.21.15.46:0/3233677368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f320 0x7fb2d0162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2cf400640 1 -- 172.21.15.46:0/3233677368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f320 msgr2=0x7fb2d0162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2cf400640 1 --2- 172.21.15.46:0/3233677368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f320 0x7fb2d0162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2cf400640 1 -- 172.21.15.46:0/3233677368 >> v1:172.21.15.82:6789/0 conn(0x7fb2d015a750 legacy=0x7fb2d006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:05.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.311+0000 7fb2cf400640 1 -- 172.21.15.46:0/3233677368 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d006d5e0 con 0x7fb2d00aed10 2024-04-25T08:44:05.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2cf400640 1 --2- 172.21.15.46:0/3233677368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00aed10 0x7fb2d0164750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800d150 tx=0x7fb2c801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40acd4b6a2d02931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2ce000640 1 -- 172.21.15.46:0/3233677368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c800b2d0 con 0x7fb2d00aed10 2024-04-25T08:44:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2ce000640 1 -- 172.21.15.46:0/3233677368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c800b470 con 0x7fb2d00aed10 2024-04-25T08:44:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2ce000640 1 -- 172.21.15.46:0/3233677368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80a7b10 con 0x7fb2d00aed10 2024-04-25T08:44:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 -- 172.21.15.46:0/3233677368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00aed10 msgr2=0x7fb2d0164750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 --2- 172.21.15.46:0/3233677368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00aed10 0x7fb2d0164750 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800d150 tx=0x7fb2c801ec30 comp rx=0 tx=0).stop 2024-04-25T08:44:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 -- 172.21.15.46:0/3233677368 shutdown_connections 2024-04-25T08:44:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 --2- 172.21.15.46:0/3233677368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00aed10 0x7fb2d0164750 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 --2- 172.21.15.46:0/3233677368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f320 0x7fb2d0162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 -- 172.21.15.46:0/3233677368 >> 172.21.15.46:0/3233677368 conn(0x7fb2d00b12a0 msgr2=0x7fb2d00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:05.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 -- 172.21.15.46:0/3233677368 shutdown_connections 2024-04-25T08:44:05.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 -- 172.21.15.46:0/3233677368 wait complete. 2024-04-25T08:44:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 Processor -- start 2024-04-25T08:44:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.315+0000 7fb2d5800640 1 -- start start 2024-04-25T08:44:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2d5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f320 0x7fb2d014a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2d5800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d00aed10 0x7fb2d0157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2d5800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d015a750 0x7fb2d0157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2d5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d006d270 con 0x7fb2d006f320 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2d5800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00cf200 con 0x7fb2d015a750 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2d5800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d006e310 con 0x7fb2d00aed10 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f320 0x7fb2d014a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f320 0x7fb2d014a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45680/0 (socket says 172.21.15.46:45680) 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cea00640 1 -- 172.21.15.46:0/500771227 learned_addr learned my addr 172.21.15.46:0/500771227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cf400640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d015a750 0x7fb2d0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cea00640 1 -- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d00aed10 msgr2=0x7fb2d0157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cea00640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d00aed10 0x7fb2d0157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cea00640 1 -- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d015a750 msgr2=0x7fb2d0157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cea00640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d015a750 0x7fb2d0157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cea00640 1 -- 172.21.15.46:0/500771227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d014e5e0 con 0x7fb2d006f320 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cf400640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d015a750 0x7fb2d0157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2cea00640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f320 0x7fb2d014a5a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c370 tx=0x7fb2c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2c7400640 1 -- 172.21.15.46:0/500771227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00052d0 con 0x7fb2d006f320 2024-04-25T08:44:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2c7400640 1 -- 172.21.15.46:0/500771227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c0005470 con 0x7fb2d006f320 2024-04-25T08:44:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d014e8c0 con 0x7fb2d006f320 2024-04-25T08:44:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2c7400640 1 -- 172.21.15.46:0/500771227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0007240 con 0x7fb2d006f320 2024-04-25T08:44:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d014ee80 con 0x7fb2d006f320 2024-04-25T08:44:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d0069080 con 0x7fb2d006f320 2024-04-25T08:44:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.319+0000 7fb2c7400640 1 -- 172.21.15.46:0/500771227 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2c00073e0 con 0x7fb2d006f320 2024-04-25T08:44:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.323+0000 7fb2c7400640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2ac066f60 0x7fb2ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.323+0000 7fb2c7400640 1 -- 172.21.15.46:0/500771227 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb2c0084350 con 0x7fb2d006f320 2024-04-25T08:44:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.323+0000 7fb2c6000640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2ac066f60 0x7fb2ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.323+0000 7fb2c6000640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2ac066f60 0x7fb2ac069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0156c90 tx=0x7fb2b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:05.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.327+0000 7fb2c7400640 1 -- 172.21.15.46:0/500771227 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2c0055560 con 0x7fb2d006f320 2024-04-25T08:44:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.543+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fb2d0152450 con 0x7fb2d006f320 2024-04-25T08:44:05.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.775+0000 7fb2c7400640 1 -- 172.21.15.46:0/500771227 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v119) v1 ==== 142+0+0 (secure 0 0 0) 0x7fb2c0054e60 con 0x7fb2d006f320 2024-04-25T08:44:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2ac066f60 msgr2=0x7fb2ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2ac066f60 0x7fb2ac069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0156c90 tx=0x7fb2b8008040 comp rx=0 tx=0).stop 2024-04-25T08:44:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f320 msgr2=0x7fb2d014a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f320 0x7fb2d014a5a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000c370 tx=0x7fb2c000c830 comp rx=0 tx=0).stop 2024-04-25T08:44:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 shutdown_connections 2024-04-25T08:44:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2ac066f60 0x7fb2ac069420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d015a750 0x7fb2d0157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d00aed10 0x7fb2d0157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 --2- 172.21.15.46:0/500771227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f320 0x7fb2d014a5a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 >> 172.21.15.46:0/500771227 conn(0x7fb2d00b12a0 msgr2=0x7fb2d0153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 shutdown_connections 2024-04-25T08:44:05.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.779+0000 7fb2d5800640 1 -- 172.21.15.46:0/500771227 wait complete. 2024-04-25T08:44:05.791 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T08:44:05.792 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-25T08:44:05.792 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:44:05.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:05.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f3672200640 1 Processor -- start 2024-04-25T08:44:06.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f3672200640 1 -- start start 2024-04-25T08:44:06.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f3672200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 0x7f366c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f3672200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366c06d5e0 con 0x7f366c0aed10 2024-04-25T08:44:06.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f3672200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366c0ce720 con 0x7f366c0cc9e0 2024-04-25T08:44:06.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f3672200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366c15ab30 con 0x7f366c15a750 2024-04-25T08:44:06.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f366c0aed10 0x7f366c0b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58126/0 (socket says 172.21.15.46:58126) 2024-04-25T08:44:06.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366aa00640 1 -- 172.21.15.46:0/3239670213 learned_addr learned my addr 172.21.15.46:0/3239670213 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:06.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366b400640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 0x7f366c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201514808 0 0) 0x7f366c06d5e0 con 0x7f366c0aed10 2024-04-25T08:44:06.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3650003530 con 0x7f366c0aed10 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102909366 0 0) 0x7f366c0ce720 con 0x7f366c0cc9e0 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f366c06d5e0 con 0x7f366c0cc9e0 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2607209499 0 0) 0x7f3650003530 con 0x7f366c0aed10 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 msgr2=0x7f366c06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 0x7f366c06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 >> v1:172.21.15.82:6789/0 conn(0x7f366c0cc9e0 legacy=0x7f366c0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c0cea00 con 0x7f366c0aed10 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f3658001680 con 0x7f366c0aed10 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366b400640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 0x7f366c06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 >> v1:172.21.15.46:6789/0 conn(0x7f366c0aed10 legacy=0x7f366c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3650003b80 0x7f3650003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 0x7f3650004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004d50 0x7f36500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365000ba80 con 0x7f3650003b80 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365000bc00 con 0x7f3650004d50 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365000be00 con 0x7f366c15a750 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366b400640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3650003b80 0x7f3650003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366aa00640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 0x7f3650004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366be00640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004d50 0x7f36500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366b400640 1 -- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 msgr2=0x7f3650004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366b400640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 0x7f3650004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366b400640 1 -- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004d50 msgr2=0x7f36500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366b400640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004d50 0x7f36500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366b400640 1 -- 172.21.15.46:0/3239670213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3650003530 con 0x7f3650003b80 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366aa00640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f366c15a750 0x7f3650004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.943+0000 7f366be00640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004d50 0x7f36500051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f366b400640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3650003b80 0x7f3650003f80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f365c009cb0 tx=0x7f365c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27263c93620d66f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0b2070 con 0x7f3650003b80 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c0a56b0 con 0x7f3650003b80 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f366a000640 1 -- 172.21.15.46:0/3239670213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c009880 con 0x7f3650003b80 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f3672200640 1 -- 172.21.15.46:0/3239670213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3650003b80 msgr2=0x7f3650003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f3672200640 1 --2- 172.21.15.46:0/3239670213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3650003b80 0x7f3650003f80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f365c009cb0 tx=0x7f365c01cc10 comp rx=0 tx=0).stop 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f366be00640 1 -- 172.21.15.46:0/3239670213 reap_dead start 2024-04-25T08:44:06.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f3672200640 1 -- 172.21.15.46:0/3239670213 shutdown_connections 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f3672200640 1 -- 172.21.15.46:0/3239670213 >> 172.21.15.46:0/3239670213 conn(0x7f366c0b12a0 msgr2=0x7f366c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f3672200640 1 -- 172.21.15.46:0/3239670213 shutdown_connections 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.947+0000 7f3672200640 1 -- 172.21.15.46:0/3239670213 wait complete. 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 Processor -- start 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 -- start start 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3650003b80 0x7f366c14f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f366c143790 0x7f366c157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c1576a0 0x7f366c15cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c06e490 con 0x7f366c143790 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c06e310 con 0x7f366c1576a0 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366c06e610 con 0x7f3650003b80 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f366c143790 0x7f366c157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f366c143790 0x7f366c157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45702/0 (socket says 172.21.15.46:45702) 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366aa00640 1 -- 172.21.15.46:0/3362712419 learned_addr learned my addr 172.21.15.46:0/3362712419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366b400640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3650003b80 0x7f366c14f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366be00640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c1576a0 0x7f366c15cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366aa00640 1 -- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3650003b80 msgr2=0x7f366c14f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366aa00640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3650003b80 0x7f366c14f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366aa00640 1 -- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c1576a0 msgr2=0x7f366c15cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366aa00640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c1576a0 0x7f366c15cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366aa00640 1 -- 172.21.15.46:0/3362712419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c14e4e0 con 0x7f366c143790 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366b400640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3650003b80 0x7f366c14f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366be00640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c1576a0 0x7f366c15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f366aa00640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f366c143790 0x7f366c157160 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f365800e7b0 tx=0x7f365800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3663400640 1 -- 172.21.15.46:0/3362712419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800ee60 con 0x7f366c143790 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3663400640 1 -- 172.21.15.46:0/3362712419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3658018070 con 0x7f366c143790 2024-04-25T08:44:06.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3663400640 1 -- 172.21.15.46:0/3362712419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800cad0 con 0x7f366c143790 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c14e7c0 con 0x7f366c143790 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.951+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366c14ed80 con 0x7f366c143790 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.955+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c069080 con 0x7f366c143790 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.955+0000 7f3663400640 1 -- 172.21.15.46:0/3362712419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f365800cc70 con 0x7f366c143790 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.955+0000 7f3663400640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f364c066f60 0x7f364c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.955+0000 7f3663400640 1 -- 172.21.15.46:0/3362712419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3658084e00 con 0x7f366c143790 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.955+0000 7f366b400640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f364c066f60 0x7f364c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.955+0000 7f366b400640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f364c066f60 0x7f364c069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f365c0af000 tx=0x7f365c0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:05.959+0000 7f3663400640 1 -- 172.21.15.46:0/3362712419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3658055ee0 con 0x7f366c143790 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f366c14fd50 con 0x7f366c143790 2024-04-25T08:44:06.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3663400640 1 -- 172.21.15.46:0/3362712419 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+13245 (secure 0 0 0) 0x7f36580557e0 con 0x7f366c143790 2024-04-25T08:44:06.159 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:06.159 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":44,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:04.769148+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2025854544":"2024-04-26T08:44:01.537766+0000","172.21.15.46:6833/2025854544":"2024-04-26T08:44:01.537766+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f364c066f60 msgr2=0x7f364c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f364c066f60 0x7f364c069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f365c0af000 tx=0x7f365c0046d0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f366c143790 msgr2=0x7f366c157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f366c143790 0x7f366c157160 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f365800e7b0 tx=0x7f365800ec70 comp rx=0 tx=0).stop 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 shutdown_connections 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f364c066f60 0x7f364c069420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366c1576a0 0x7f366c15cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f366c143790 0x7f366c157160 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 --2- 172.21.15.46:0/3362712419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3650003b80 0x7f366c14f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 >> 172.21.15.46:0/3362712419 conn(0x7f366c0b12a0 msgr2=0x7f366c14b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.123+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 shutdown_connections 2024-04-25T08:44:06.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.127+0000 7f3672200640 1 -- 172.21.15.46:0/3362712419 wait complete. 2024-04-25T08:44:06.162 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:44:06.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:44:06.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e69c00640 1 Processor -- start 2024-04-25T08:44:06.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e69c00640 1 -- start start 2024-04-25T08:44:06.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e69c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e64143990 0x7f3e640cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e69c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3e6406f150 0x7f3e640b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e69c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6415e440 con 0x7f3e641440d0 2024-04-25T08:44:06.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e69c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6415ed30 con 0x7f3e64143990 2024-04-25T08:44:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e69c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6415e6e0 con 0x7f3e6406f150 2024-04-25T08:44:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e62a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e64143990 0x7f3e640cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e62a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e64143990 0x7f3e640cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45718/0 (socket says 172.21.15.46:45718) 2024-04-25T08:44:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e62a00640 1 -- 172.21.15.46:0/2544863687 learned_addr learned my addr 172.21.15.46:0/2544863687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e62000640 1 --1- 172.21.15.46:0/2544863687 >> v1:172.21.15.46:6789/0 conn(0x7f3e641440d0 0x7f3e6406ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58130/0 (socket says 172.21.15.46:58130) 2024-04-25T08:44:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e63400640 1 --2- 172.21.15.46:0/2544863687 >> v2:172.21.15.82:3300/0 conn(0x7f3e6406f150 0x7f3e640b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e62a00640 1 -- 172.21.15.46:0/2544863687 >> v2:172.21.15.82:3300/0 conn(0x7f3e6406f150 msgr2=0x7f3e640b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:06.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e62a00640 1 --2- 172.21.15.46:0/2544863687 >> v2:172.21.15.82:3300/0 conn(0x7f3e6406f150 0x7f3e640b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e62a00640 1 -- 172.21.15.46:0/2544863687 >> v1:172.21.15.46:6789/0 conn(0x7f3e641440d0 legacy=0x7f3e6406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e62a00640 1 -- 172.21.15.46:0/2544863687 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6415e9c0 con 0x7f3e64143990 2024-04-25T08:44:06.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.315+0000 7f3e61600640 1 -- 172.21.15.46:0/2544863687 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931158699 0 0) 0x7f3e6415e440 con 0x7f3e641440d0 2024-04-25T08:44:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e62a00640 1 --2- 172.21.15.46:0/2544863687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 0x7f3e640cecc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3e640af320 tx=0x7f3e5401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3175710664c9b569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e61600640 1 -- 172.21.15.46:0/2544863687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540b0070 con 0x7f3e64143990 2024-04-25T08:44:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e61600640 1 -- 172.21.15.46:0/2544863687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540ab600 con 0x7f3e64143990 2024-04-25T08:44:06.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e61600640 1 -- 172.21.15.46:0/2544863687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540ab9d0 con 0x7f3e64143990 2024-04-25T08:44:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e69c00640 1 -- 172.21.15.46:0/2544863687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 msgr2=0x7f3e640cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e69c00640 1 --2- 172.21.15.46:0/2544863687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 0x7f3e640cecc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3e640af320 tx=0x7f3e5401c520 comp rx=0 tx=0).stop 2024-04-25T08:44:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e69c00640 1 -- 172.21.15.46:0/2544863687 shutdown_connections 2024-04-25T08:44:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e69c00640 1 --2- 172.21.15.46:0/2544863687 >> v2:172.21.15.82:3300/0 conn(0x7f3e6406f150 0x7f3e640b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e69c00640 1 --2- 172.21.15.46:0/2544863687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 0x7f3e640cecc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e69c00640 1 -- 172.21.15.46:0/2544863687 >> 172.21.15.46:0/2544863687 conn(0x7f3e6406ac10 msgr2=0x7f3e6406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e69c00640 1 -- 172.21.15.46:0/2544863687 shutdown_connections 2024-04-25T08:44:06.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e69c00640 1 -- 172.21.15.46:0/2544863687 wait complete. 2024-04-25T08:44:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.319+0000 7f3e69c00640 1 Processor -- start 2024-04-25T08:44:06.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 -- start start 2024-04-25T08:44:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6406f150 0x7f3e640b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 0x7f3e64159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e641440d0 0x7f3e6415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e640b35f0 con 0x7f3e64143990 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e640b3470 con 0x7f3e6406f150 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e640b3770 con 0x7f3e641440d0 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 0x7f3e64159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 0x7f3e64159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45724/0 (socket says 172.21.15.46:45724) 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62000640 1 -- 172.21.15.46:0/2409538382 learned_addr learned my addr 172.21.15.46:0/2409538382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62a00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6406f150 0x7f3e640b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e63400640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e641440d0 0x7f3e6415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62000640 1 -- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e641440d0 msgr2=0x7f3e6415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62000640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e641440d0 0x7f3e6415c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62000640 1 -- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6406f150 msgr2=0x7f3e640b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62000640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6406f150 0x7f3e640b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62000640 1 -- 172.21.15.46:0/2409538382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e640b0b80 con 0x7f3e64143990 2024-04-25T08:44:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e62000640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 0x7f3e64159b90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000c4b0 tx=0x7f3e5000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e5aa00640 1 -- 172.21.15.46:0/2409538382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50018070 con 0x7f3e64143990 2024-04-25T08:44:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e5aa00640 1 -- 172.21.15.46:0/2409538382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e50007500 con 0x7f3e64143990 2024-04-25T08:44:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e5aa00640 1 -- 172.21.15.46:0/2409538382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50004a30 con 0x7f3e64143990 2024-04-25T08:44:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e640b0e60 con 0x7f3e64143990 2024-04-25T08:44:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e640b1420 con 0x7f3e64143990 2024-04-25T08:44:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.323+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e64069010 con 0x7f3e64143990 2024-04-25T08:44:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.327+0000 7f3e5aa00640 1 -- 172.21.15.46:0/2409538382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e50004bd0 con 0x7f3e64143990 2024-04-25T08:44:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.327+0000 7f3e5aa00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e44066f60 0x7f3e44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.327+0000 7f3e5aa00640 1 -- 172.21.15.46:0/2409538382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3e50084080 con 0x7f3e64143990 2024-04-25T08:44:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.327+0000 7f3e62a00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e44066f60 0x7f3e44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.327+0000 7f3e62a00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e44066f60 0x7f3e44069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3e540ad000 tx=0x7f3e540a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.331+0000 7f3e5aa00640 1 -- 172.21.15.46:0/2409538382 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e50054ba0 con 0x7f3e64143990 2024-04-25T08:44:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.491+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e6406deb0 con 0x7f3e64143990 2024-04-25T08:44:06.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.491+0000 7f3e5aa00640 1 -- 172.21.15.46:0/2409538382 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e50058c40 con 0x7f3e64143990 2024-04-25T08:44:06.485 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:44:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e44066f60 msgr2=0x7f3e44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e44066f60 0x7f3e44069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3e540ad000 tx=0x7f3e540a6000 comp rx=0 tx=0).stop 2024-04-25T08:44:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 msgr2=0x7f3e64159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 0x7f3e64159b90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000c4b0 tx=0x7f3e5000c970 comp rx=0 tx=0).stop 2024-04-25T08:44:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 shutdown_connections 2024-04-25T08:44:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e44066f60 0x7f3e44069420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e641440d0 0x7f3e6415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e64143990 0x7f3e64159b90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 --2- 172.21.15.46:0/2409538382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6406f150 0x7f3e640b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 >> 172.21.15.46:0/2409538382 conn(0x7f3e6406ac10 msgr2=0x7f3e641391a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 shutdown_connections 2024-04-25T08:44:06.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.495+0000 7f3e69c00640 1 -- 172.21.15.46:0/2409538382 wait complete. 2024-04-25T08:44:06.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T08:44:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd8c00640 1 Processor -- start 2024-04-25T08:44:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd8c00640 1 -- start start 2024-04-25T08:44:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd8c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd415a750 0x7f1bd406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd8c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd406d5e0 con 0x7f1bd40cc9e0 2024-04-25T08:44:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd8c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bd40ce720 con 0x7f1bd40aed10 2024-04-25T08:44:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd8c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd415ab30 con 0x7f1bd415a750 2024-04-25T08:44:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1bd40cc9e0 0x7f1bd40b2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58132/0 (socket says 172.21.15.46:58132) 2024-04-25T08:44:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd2800640 1 -- 172.21.15.46:0/4272615487 learned_addr learned my addr 172.21.15.46:0/4272615487 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd1e00640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd415a750 0x7f1bd406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208192082 0 0) 0x7f1bd406d5e0 con 0x7f1bd40cc9e0 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.655+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bb8003530 con 0x7f1bd40cc9e0 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037767194 0 0) 0x7f1bd40ce720 con 0x7f1bd40aed10 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bd406d5e0 con 0x7f1bd40aed10 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4121069235 0 0) 0x7f1bb8003530 con 0x7f1bd40cc9e0 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd415a750 msgr2=0x7f1bd406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd415a750 0x7f1bd406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 >> v1:172.21.15.82:6789/0 conn(0x7f1bd40aed10 legacy=0x7f1bd40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd40cea00 con 0x7f1bd40cc9e0 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f1bcc001a00 con 0x7f1bd40cc9e0 2024-04-25T08:44:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1e00640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd415a750 0x7f1bd406ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 >> v1:172.21.15.46:6789/0 conn(0x7f1bd40cc9e0 legacy=0x7f1bd40b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb8003b80 0x7f1bb8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb8004810 0x7f1bb8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb80051d0 0x7f1bb800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb800e1d0 con 0x7f1bb8003b80 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb800e380 con 0x7f1bb80051d0 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb800e580 con 0x7f1bb8004810 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1e00640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb8003b80 0x7f1bb8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd2800640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb80051d0 0x7f1bb800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bcbe00640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb8004810 0x7f1bb8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1e00640 1 -- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb8004810 msgr2=0x7f1bb8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1e00640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bb8004810 0x7f1bb8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1e00640 1 -- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb80051d0 msgr2=0x7f1bb800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1e00640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb80051d0 0x7f1bb800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1e00640 1 -- 172.21.15.46:0/4272615487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8003530 con 0x7f1bb8003b80 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1e00640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb8003b80 0x7f1bb8003f80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40700e0 tx=0x7f1bc401caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52f86dcd1c32bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.659+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4009810 con 0x7f1bb8003b80 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc4004030 con 0x7f1bb8003b80 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd1400640 1 -- 172.21.15.46:0/4272615487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40043e0 con 0x7f1bb8003b80 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 -- 172.21.15.46:0/4272615487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb8003b80 msgr2=0x7f1bb8003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 --2- 172.21.15.46:0/4272615487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bb8003b80 0x7f1bb8003f80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40700e0 tx=0x7f1bc401caf0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd2800640 1 -- 172.21.15.46:0/4272615487 reap_dead start 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 -- 172.21.15.46:0/4272615487 shutdown_connections 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 -- 172.21.15.46:0/4272615487 >> 172.21.15.46:0/4272615487 conn(0x7f1bd40b12a0 msgr2=0x7f1bd40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 -- 172.21.15.46:0/4272615487 shutdown_connections 2024-04-25T08:44:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 -- 172.21.15.46:0/4272615487 wait complete. 2024-04-25T08:44:06.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 Processor -- start 2024-04-25T08:44:06.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 -- start start 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8003b80 0x7f1bd414e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd4150d60 0x7f1bd4140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd414cca0 0x7f1bd414d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd406e490 con 0x7f1bd4150d60 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd406e310 con 0x7f1bb8003b80 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd8c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd406e610 con 0x7f1bd414cca0 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd4150d60 0x7f1bd4140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd4150d60 0x7f1bd4140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45742/0 (socket says 172.21.15.46:45742) 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bcbe00640 1 -- 172.21.15.46:0/1352498677 learned_addr learned my addr 172.21.15.46:0/1352498677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:06.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd1e00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8003b80 0x7f1bd414e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.663+0000 7f1bd2800640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd414cca0 0x7f1bd414d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bcbe00640 1 -- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd414cca0 msgr2=0x7f1bd414d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bcbe00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd414cca0 0x7f1bd414d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bcbe00640 1 -- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8003b80 msgr2=0x7f1bd414e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bcbe00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8003b80 0x7f1bd414e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bcbe00640 1 -- 172.21.15.46:0/1352498677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd41f5c90 con 0x7f1bd4150d60 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bcbe00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd4150d60 0x7f1bd4140e50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e7b0 tx=0x7f1bbc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bca000640 1 -- 172.21.15.46:0/1352498677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0050d0 con 0x7f1bd4150d60 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bca000640 1 -- 172.21.15.46:0/1352498677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc018070 con 0x7f1bd4150d60 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd41f5f70 con 0x7f1bd4150d60 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bca000640 1 -- 172.21.15.46:0/1352498677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00c6c0 con 0x7f1bd4150d60 2024-04-25T08:44:06.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd41f6530 con 0x7f1bd4150d60 2024-04-25T08:44:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bd4069080 con 0x7f1bd4150d60 2024-04-25T08:44:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bca000640 1 -- 172.21.15.46:0/1352498677 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bbc00c860 con 0x7f1bd4150d60 2024-04-25T08:44:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bca000640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1bb0066f90 0x7f1bb0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:06.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bca000640 1 -- 172.21.15.46:0/1352498677 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1bbc084cd0 con 0x7f1bd4150d60 2024-04-25T08:44:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.667+0000 7f1bd1e00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1bb0066f90 0x7f1bb0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:06.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.671+0000 7f1bd1e00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1bb0066f90 0x7f1bb0069450 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40ae000 tx=0x7f1bc40046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:06.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.671+0000 7f1bca000640 1 -- 172.21.15.46:0/1352498677 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1bbc055820 con 0x7f1bd4150d60 2024-04-25T08:44:06.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:06.891+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f1bd40afe70 con 0x7f1bd4150d60 2024-04-25T08:44:07.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.783+0000 7f1bca000640 1 -- 172.21.15.46:0/1352498677 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v121) v1 ==== 152+0+0 (secure 0 0 0) 0x7f1bbc0598c0 con 0x7f1bd4150d60 2024-04-25T08:44:07.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1bb0066f90 msgr2=0x7f1bb0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:07.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1bb0066f90 0x7f1bb0069450 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1bc40ae000 tx=0x7f1bc40046d0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd4150d60 msgr2=0x7f1bd4140e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd4150d60 0x7f1bd4140e50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e7b0 tx=0x7f1bbc00ec70 comp rx=0 tx=0).stop 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 shutdown_connections 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1bb0066f90 0x7f1bb0069450 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1bd414cca0 0x7f1bd414d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bd4150d60 0x7f1bd4140e50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 --2- 172.21.15.46:0/1352498677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bb8003b80 0x7f1bd414e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 >> 172.21.15.46:0/1352498677 conn(0x7f1bd40b12a0 msgr2=0x7f1bd4073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 shutdown_connections 2024-04-25T08:44:07.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.787+0000 7f1bd8c00640 1 -- 172.21.15.46:0/1352498677 wait complete. 2024-04-25T08:44:07.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:44:07.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdca7e00640 1 Processor -- start 2024-04-25T08:44:07.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdca7e00640 1 -- start start 2024-04-25T08:44:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdca7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdca809a410 0x7fdca806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdca7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca80d0390 0x7fdca80cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdca7e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca809acb0 con 0x7fdca806c730 2024-04-25T08:44:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdca7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca80b3dc0 con 0x7fdca809a410 2024-04-25T08:44:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdca7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80cf680 con 0x7fdca80d0390 2024-04-25T08:44:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdc9e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdca809a410 0x7fdca806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdc9e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdca809a410 0x7fdca806bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45750/0 (socket says 172.21.15.46:45750) 2024-04-25T08:44:07.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdc9e000640 1 -- 172.21.15.46:0/1528193220 learned_addr learned my addr 172.21.15.46:0/1528193220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdc9e000640 1 -- 172.21.15.46:0/1528193220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca80d0390 msgr2=0x7fdca80cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdc9e000640 1 --2- 172.21.15.46:0/1528193220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca80d0390 0x7fdca80cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdc9e000640 1 -- 172.21.15.46:0/1528193220 >> v1:172.21.15.82:6789/0 conn(0x7fdca806c730 legacy=0x7fdca806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:07.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.959+0000 7fdc9e000640 1 -- 172.21.15.46:0/1528193220 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca809a9d0 con 0x7fdca809a410 2024-04-25T08:44:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdc9e000640 1 --2- 172.21.15.46:0/1528193220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca809a410 0x7fdca806bf30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdc900098a0 tx=0x7fdc9001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d18642d2451c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:07.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca6000640 1 -- 172.21.15.46:0/1528193220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900aa5f0 con 0x7fdca809a410 2024-04-25T08:44:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca6000640 1 -- 172.21.15.46:0/1528193220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc900af070 con 0x7fdca809a410 2024-04-25T08:44:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca6000640 1 -- 172.21.15.46:0/1528193220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900aa9b0 con 0x7fdca809a410 2024-04-25T08:44:07.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca7e00640 1 -- 172.21.15.46:0/1528193220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca809a410 msgr2=0x7fdca806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca7e00640 1 --2- 172.21.15.46:0/1528193220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca809a410 0x7fdca806bf30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdc900098a0 tx=0x7fdc9001c4d0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca7e00640 1 -- 172.21.15.46:0/1528193220 shutdown_connections 2024-04-25T08:44:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca7e00640 1 --2- 172.21.15.46:0/1528193220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca80d0390 0x7fdca80cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca7e00640 1 --2- 172.21.15.46:0/1528193220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca809a410 0x7fdca806bf30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca7e00640 1 -- 172.21.15.46:0/1528193220 >> 172.21.15.46:0/1528193220 conn(0x7fdca80b2f50 msgr2=0x7fdca80b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca7e00640 1 -- 172.21.15.46:0/1528193220 shutdown_connections 2024-04-25T08:44:07.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.963+0000 7fdca7e00640 1 -- 172.21.15.46:0/1528193220 wait complete. 2024-04-25T08:44:07.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 Processor -- start 2024-04-25T08:44:07.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 -- start start 2024-04-25T08:44:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca806c730 0x7fdca815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca809a410 0x7fdca81527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca80d0390 0x7fdca81558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca815dd60 con 0x7fdca80d0390 2024-04-25T08:44:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca80cc400 con 0x7fdca809a410 2024-04-25T08:44:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca816b040 con 0x7fdca806c730 2024-04-25T08:44:07.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca80d0390 0x7fdca81558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca80d0390 0x7fdca81558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45754/0 (socket says 172.21.15.46:45754) 2024-04-25T08:44:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7400640 1 -- 172.21.15.46:0/1113771164 learned_addr learned my addr 172.21.15.46:0/1113771164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdc9e000640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca809a410 0x7fdca81527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:07.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca6a00640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca806c730 0x7fdca815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7400640 1 -- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca806c730 msgr2=0x7fdca815bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7400640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca806c730 0x7fdca815bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7400640 1 -- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca809a410 msgr2=0x7fdca81527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7400640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca809a410 0x7fdca81527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7400640 1 -- 172.21.15.46:0/1113771164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca8152d10 con 0x7fdca80d0390 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7400640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca80d0390 0x7fdca81558d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdca000b970 tx=0x7fdca000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdc9f400640 1 -- 172.21.15.46:0/1113771164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0019070 con 0x7fdca80d0390 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdc9f400640 1 -- 172.21.15.46:0/1113771164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca000cb60 con 0x7fdca80d0390 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdc9f400640 1 -- 172.21.15.46:0/1113771164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0014440 con 0x7fdca80d0390 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca815c3e0 con 0x7fdca80d0390 2024-04-25T08:44:07.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca815d120 con 0x7fdca80d0390 2024-04-25T08:44:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.967+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8069010 con 0x7fdca80d0390 2024-04-25T08:44:07.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.971+0000 7fdc9f400640 1 -- 172.21.15.46:0/1113771164 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdca001c070 con 0x7fdca80d0390 2024-04-25T08:44:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.971+0000 7fdc9f400640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc84066f60 0x7fdc84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.971+0000 7fdc9f400640 1 -- 172.21.15.46:0/1113771164 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdca0084b80 con 0x7fdca80d0390 2024-04-25T08:44:07.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.971+0000 7fdca6a00640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc84066f60 0x7fdc84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:07.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.971+0000 7fdca6a00640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc84066f60 0x7fdc84069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdca80cda70 tx=0x7fdc98009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:07.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:07.975+0000 7fdc9f400640 1 -- 172.21.15.46:0/1113771164 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdca8152d10 con 0x7fdca80d0390 2024-04-25T08:44:08.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.139+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdca80719f0 con 0x7fdca80d0390 2024-04-25T08:44:08.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdc9f400640 1 -- 172.21.15.46:0/1113771164 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdca00556a0 con 0x7fdca80d0390 2024-04-25T08:44:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc84066f60 msgr2=0x7fdc84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc84066f60 0x7fdc84069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdca80cda70 tx=0x7fdc98009060 comp rx=0 tx=0).stop 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca80d0390 msgr2=0x7fdca81558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca80d0390 0x7fdca81558d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdca000b970 tx=0x7fdca000be30 comp rx=0 tx=0).stop 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 shutdown_connections 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc84066f60 0x7fdc84069420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdca80d0390 0x7fdca81558d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdca809a410 0x7fdca81527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 --2- 172.21.15.46:0/1113771164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdca806c730 0x7fdca815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 >> 172.21.15.46:0/1113771164 conn(0x7fdca80b2f50 msgr2=0x7fdca8070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 shutdown_connections 2024-04-25T08:44:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.143+0000 7fdca7e00640 1 -- 172.21.15.46:0/1113771164 wait complete. 2024-04-25T08:44:08.156 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:44:08.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:44:08.244 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:44:08.324 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:44:08.324 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:44:08.325 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:08.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:44:08.333 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:44:08.334 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:08.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:44:08.343 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:44:08.343 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:08.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:44:08.386 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:44:08.386 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:08.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:44:08.397 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:44:08.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f08200640 1 Processor -- start 2024-04-25T08:44:08.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f08200640 1 -- start start 2024-04-25T08:44:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f08200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f00159230 0x7f2f0015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:08.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f08200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2f000cbeb0 0x7f2f000cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f08200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f000cc700 con 0x7f2f000cda80 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f08200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f000cded0 con 0x7f2f00159230 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f08200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f000cc9e0 con 0x7f2f000cbeb0 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f05800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f00159230 0x7f2f0015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f06200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2f000cda80 0x7f2f0015f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39662/0 (socket says 172.21.15.46:39662) 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f05800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f00159230 0x7f2f0015bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47974/0 (socket says 172.21.15.46:47974) 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f06200640 1 -- 172.21.15.46:0/2298351257 learned_addr learned my addr 172.21.15.46:0/2298351257 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f04e00640 1 --2- 172.21.15.46:0/2298351257 >> v2:172.21.15.82:3300/0 conn(0x7f2f000cbeb0 0x7f2f000cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f05800640 1 -- 172.21.15.46:0/2298351257 >> v2:172.21.15.82:3300/0 conn(0x7f2f000cbeb0 msgr2=0x7f2f000cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:08.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f05800640 1 --2- 172.21.15.46:0/2298351257 >> v2:172.21.15.82:3300/0 conn(0x7f2f000cbeb0 0x7f2f000cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f05800640 1 -- 172.21.15.46:0/2298351257 >> v1:172.21.15.46:6789/0 conn(0x7f2f000cda80 legacy=0x7f2f0015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.579+0000 7f2f05800640 1 -- 172.21.15.46:0/2298351257 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f000cccc0 con 0x7f2f00159230 2024-04-25T08:44:08.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2f05800640 1 --2- 172.21.15.46:0/2298351257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f00159230 0x7f2f0015bdb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2f00050000 tx=0x7f2ef401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ff49b112813a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2effe00640 1 -- 172.21.15.46:0/2298351257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40b1070 con 0x7f2f00159230 2024-04-25T08:44:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2effe00640 1 -- 172.21.15.46:0/2298351257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef40ac620 con 0x7f2f00159230 2024-04-25T08:44:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2effe00640 1 -- 172.21.15.46:0/2298351257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40ac9f0 con 0x7f2f00159230 2024-04-25T08:44:08.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2f08200640 1 -- 172.21.15.46:0/2298351257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f00159230 msgr2=0x7f2f0015bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2f08200640 1 --2- 172.21.15.46:0/2298351257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f00159230 0x7f2f0015bdb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2f00050000 tx=0x7f2ef401c3c0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2f08200640 1 -- 172.21.15.46:0/2298351257 shutdown_connections 2024-04-25T08:44:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2f08200640 1 --2- 172.21.15.46:0/2298351257 >> v2:172.21.15.82:3300/0 conn(0x7f2f000cbeb0 0x7f2f000cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2f08200640 1 --2- 172.21.15.46:0/2298351257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f00159230 0x7f2f0015bdb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2f08200640 1 -- 172.21.15.46:0/2298351257 >> 172.21.15.46:0/2298351257 conn(0x7f2f000b12a0 msgr2=0x7f2f000b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2f08200640 1 -- 172.21.15.46:0/2298351257 shutdown_connections 2024-04-25T08:44:08.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.583+0000 7f2f08200640 1 -- 172.21.15.46:0/2298351257 wait complete. 2024-04-25T08:44:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 Processor -- start 2024-04-25T08:44:08.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 -- start start 2024-04-25T08:44:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f000cbeb0 0x7f2f0013d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f000cda80 0x7f2f00140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f00159230 0x7f2f00140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000cc410 con 0x7f2f000cbeb0 2024-04-25T08:44:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f000cc290 con 0x7f2f000cda80 2024-04-25T08:44:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00168780 con 0x7f2f00159230 2024-04-25T08:44:08.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f05800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f000cbeb0 0x7f2f0013d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f05800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f000cbeb0 0x7f2f0013d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47982/0 (socket says 172.21.15.46:47982) 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f05800640 1 -- 172.21.15.46:0/3283959245 learned_addr learned my addr 172.21.15.46:0/3283959245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f05800640 1 -- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f00159230 msgr2=0x7f2f00140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f05800640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f00159230 0x7f2f00140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f05800640 1 -- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f000cda80 msgr2=0x7f2f00140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f05800640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f000cda80 0x7f2f00140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f05800640 1 -- 172.21.15.46:0/3283959245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f000b42f0 con 0x7f2f000cbeb0 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f05800640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f000cbeb0 0x7f2f0013d740 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40af040 tx=0x7f2ef4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2efd600640 1 -- 172.21.15.46:0/3283959245 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40b9070 con 0x7f2f000cbeb0 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2efd600640 1 -- 172.21.15.46:0/3283959245 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef40b5410 con 0x7f2f000cbeb0 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2efd600640 1 -- 172.21.15.46:0/3283959245 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40b1040 con 0x7f2f000cbeb0 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0013fa40 con 0x7f2f000cbeb0 2024-04-25T08:44:08.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f001402e0 con 0x7f2f000cbeb0 2024-04-25T08:44:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.587+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f0013fc10 con 0x7f2f000cbeb0 2024-04-25T08:44:08.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.591+0000 7f2efd600640 1 -- 172.21.15.46:0/3283959245 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ef40bc070 con 0x7f2f000cbeb0 2024-04-25T08:44:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.591+0000 7f2efd600640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ee0066f60 0x7f2ee0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.591+0000 7f2efd600640 1 -- 172.21.15.46:0/3283959245 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2ef411edf0 con 0x7f2f000cbeb0 2024-04-25T08:44:08.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.591+0000 7f2f04e00640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ee0066f60 0x7f2ee0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:08.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.591+0000 7f2f04e00640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ee0066f60 0x7f2ee0069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2f001424d0 tx=0x7f2eec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:08.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.595+0000 7f2efd600640 1 -- 172.21.15.46:0/3283959245 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ef4016070 con 0x7f2f000cbeb0 2024-04-25T08:44:08.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.799+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f00069080 con 0x7f2f000cbeb0 2024-04-25T08:44:08.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.799+0000 7f2efd600640 1 -- 172.21.15.46:0/3283959245 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f2ef40efed0 con 0x7f2f000cbeb0 2024-04-25T08:44:08.792 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:08.792 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1079752579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1079752579},{"type":"v1","addr":"172.21.15.82:6836","nonce":1079752579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/106980078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":106980078},{"type":"v1","addr":"172.21.15.82:6837","nonce":106980078}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5292,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1318225485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1318225485},{"type":"v1","addr":"172.21.15.46:6834","nonce":1318225485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5643,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1060195791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1060195791},{"type":"v1","addr":"172.21.15.46:6835","nonce":1060195791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2024-04-25T08:44:08.793 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 20 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ee0066f60 msgr2=0x7f2ee0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ee0066f60 0x7f2ee0069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2f001424d0 tx=0x7f2eec008040 comp rx=0 tx=0).stop 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f000cbeb0 msgr2=0x7f2f0013d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f000cbeb0 0x7f2f0013d740 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40af040 tx=0x7f2ef4019160 comp rx=0 tx=0).stop 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 shutdown_connections 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ee0066f60 0x7f2ee0069420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f00159230 0x7f2f00140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f000cda80 0x7f2f00140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 --2- 172.21.15.46:0/3283959245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f000cbeb0 0x7f2f0013d740 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 >> 172.21.15.46:0/3283959245 conn(0x7f2f000b12a0 msgr2=0x7f2f0006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 shutdown_connections 2024-04-25T08:44:08.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:08.803+0000 7f2f08200640 1 -- 172.21.15.46:0/3283959245 wait complete. 2024-04-25T08:44:08.815 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:44:08.815 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:44:08.815 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:44:08.815 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:44:08.816 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:44:08.816 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:44:08.816 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:44:08.816 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:44:08.816 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:44:08.816 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:44:08.816 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:44:08.816 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:44:08.817 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:44:08.817 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:44:08.817 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:44:08.817 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:44:08.830 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:44:08.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:44:08.832 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:44:08.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:44:08.834 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:44:08.834 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:44:08.835 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:44:08.835 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:44:08.880 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:44:08.880 DEBUG:teuthology.parallel:result is None 2024-04-25T08:44:08.882 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:44:08.882 DEBUG:teuthology.parallel:result is None 2024-04-25T08:44:08.884 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:44:08.884 DEBUG:teuthology.parallel:result is None 2024-04-25T08:44:08.886 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:44:08.886 DEBUG:teuthology.parallel:result is None 2024-04-25T08:44:08.886 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:44:08.967 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:44:08.967 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:44:08.976 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-04-25T08:44:08.976 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-04-25T08:44:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.039+0000 7f4d39a00640 1 Processor -- start 2024-04-25T08:44:09.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.039+0000 7f4d39a00640 1 -- start start 2024-04-25T08:44:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d39a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4d34159230 0x7f4d3415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d39a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cbeb0 0x7f4d340cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d39a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4d340cda80 0x7f4d3415f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:09.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d39a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d340cc700 con 0x7f4d340cda80 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d39a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d340cc9e0 con 0x7f4d34159230 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d39a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340cccc0 con 0x7f4d340cbeb0 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d33400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4d340cda80 0x7f4d3415f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d33400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4d340cda80 0x7f4d3415f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48020/0 (socket says 172.21.15.46:48020) 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d33400640 1 -- 172.21.15.46:0/650560041 learned_addr learned my addr 172.21.15.46:0/650560041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d2be00640 1 --2- 172.21.15.46:0/650560041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cbeb0 0x7f4d340cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d32a00640 1 --2- 172.21.15.46:0/650560041 >> v2:172.21.15.82:3300/0 conn(0x7f4d34159230 0x7f4d3415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d33400640 1 -- 172.21.15.46:0/650560041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cbeb0 msgr2=0x7f4d340cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d33400640 1 --2- 172.21.15.46:0/650560041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cbeb0 0x7f4d340cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d33400640 1 -- 172.21.15.46:0/650560041 >> v2:172.21.15.82:3300/0 conn(0x7f4d34159230 msgr2=0x7f4d3415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d33400640 1 --2- 172.21.15.46:0/650560041 >> v2:172.21.15.82:3300/0 conn(0x7f4d34159230 0x7f4d3415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d33400640 1 -- 172.21.15.46:0/650560041 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d340ccfa0 con 0x7f4d340cda80 2024-04-25T08:44:09.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d2be00640 1 --2- 172.21.15.46:0/650560041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cbeb0 0x7f4d340cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:09.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.043+0000 7f4d32a00640 1 --2- 172.21.15.46:0/650560041 >> v2:172.21.15.82:3300/0 conn(0x7f4d34159230 0x7f4d3415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:09.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d33400640 1 --2- 172.21.15.46:0/650560041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cda80 0x7f4d3415f3e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4d34072a90 tx=0x7f4d2c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f731d96ff6b348f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:09.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d32000640 1 -- 172.21.15.46:0/650560041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0ac020 con 0x7f4d340cda80 2024-04-25T08:44:09.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d32000640 1 -- 172.21.15.46:0/650560041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c0a6440 con 0x7f4d340cda80 2024-04-25T08:44:09.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d32000640 1 -- 172.21.15.46:0/650560041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0a6810 con 0x7f4d340cda80 2024-04-25T08:44:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 -- 172.21.15.46:0/650560041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cda80 msgr2=0x7f4d3415f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 --2- 172.21.15.46:0/650560041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cda80 0x7f4d3415f3e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4d34072a90 tx=0x7f4d2c01c6a0 comp rx=0 tx=0).stop 2024-04-25T08:44:09.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 -- 172.21.15.46:0/650560041 shutdown_connections 2024-04-25T08:44:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 --2- 172.21.15.46:0/650560041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cda80 0x7f4d3415f3e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 --2- 172.21.15.46:0/650560041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cbeb0 0x7f4d340cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 --2- 172.21.15.46:0/650560041 >> v2:172.21.15.82:3300/0 conn(0x7f4d34159230 0x7f4d3415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 -- 172.21.15.46:0/650560041 >> 172.21.15.46:0/650560041 conn(0x7f4d340b12a0 msgr2=0x7f4d340b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 -- 172.21.15.46:0/650560041 shutdown_connections 2024-04-25T08:44:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 -- 172.21.15.46:0/650560041 wait complete. 2024-04-25T08:44:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 Processor -- start 2024-04-25T08:44:09.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 -- start start 2024-04-25T08:44:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cbeb0 0x7f4d3413ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cda80 0x7f4d3413bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d34159230 0x7f4d3406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3415d200 con 0x7f4d340cbeb0 2024-04-25T08:44:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d340cc4a0 con 0x7f4d34159230 2024-04-25T08:44:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.047+0000 7f4d39a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3415d380 con 0x7f4d340cda80 2024-04-25T08:44:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d32a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cbeb0 0x7f4d3413ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:09.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d32a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cbeb0 0x7f4d3413ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48028/0 (socket says 172.21.15.46:48028) 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d32a00640 1 -- 172.21.15.46:0/3407135551 learned_addr learned my addr 172.21.15.46:0/3407135551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d2be00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cda80 0x7f4d3413bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d33400640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d34159230 0x7f4d3406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d32a00640 1 -- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cda80 msgr2=0x7f4d3413bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d32a00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cda80 0x7f4d3413bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d32a00640 1 -- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d34159230 msgr2=0x7f4d3406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d32a00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d34159230 0x7f4d3406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d32a00640 1 -- 172.21.15.46:0/3407135551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d340b3a90 con 0x7f4d340cbeb0 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d32a00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cbeb0 0x7f4d3413ceb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400d800 tx=0x7f4d2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d2aa00640 1 -- 172.21.15.46:0/3407135551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24018020 con 0x7f4d340cbeb0 2024-04-25T08:44:09.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d2aa00640 1 -- 172.21.15.46:0/3407135551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d240096c0 con 0x7f4d340cbeb0 2024-04-25T08:44:09.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d2aa00640 1 -- 172.21.15.46:0/3407135551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24009ab0 con 0x7f4d340cbeb0 2024-04-25T08:44:09.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3413c110 con 0x7f4d340cbeb0 2024-04-25T08:44:09.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d341f7500 con 0x7f4d340cbeb0 2024-04-25T08:44:09.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d34069080 con 0x7f4d340cbeb0 2024-04-25T08:44:09.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.051+0000 7f4d2aa00640 1 -- 172.21.15.46:0/3407135551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d2401b020 con 0x7f4d340cbeb0 2024-04-25T08:44:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.055+0000 7f4d2aa00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d10066f60 0x7f4d10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.055+0000 7f4d2aa00640 1 -- 172.21.15.46:0/3407135551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4d24085290 con 0x7f4d340cbeb0 2024-04-25T08:44:09.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.055+0000 7f4d2be00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d10066f60 0x7f4d10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:09.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.055+0000 7f4d2be00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d10066f60 0x7f4d10069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d3415a640 tx=0x7f4d1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:09.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.059+0000 7f4d2aa00640 1 -- 172.21.15.46:0/3407135551 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d24022020 con 0x7f4d340cbeb0 2024-04-25T08:44:09.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:09.239+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4d34158560 con 0x7f4d340cbeb0 2024-04-25T08:44:10.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d2aa00640 1 -- 172.21.15.46:0/3407135551 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4d24055db0 con 0x7f4d340cbeb0 2024-04-25T08:44:10.150 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-25T08:44:10.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d10066f60 msgr2=0x7f4d10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d10066f60 0x7f4d10069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4d3415a640 tx=0x7f4d1c008040 comp rx=0 tx=0).stop 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cbeb0 msgr2=0x7f4d3413ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cbeb0 0x7f4d3413ceb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400d800 tx=0x7f4d2400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 shutdown_connections 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d10066f60 0x7f4d10069420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d34159230 0x7f4d3406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d340cda80 0x7f4d3413bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 --2- 172.21.15.46:0/3407135551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d340cbeb0 0x7f4d3413ceb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 >> 172.21.15.46:0/3407135551 conn(0x7f4d340b12a0 msgr2=0x7f4d34141cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 shutdown_connections 2024-04-25T08:44:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.159+0000 7f4d39a00640 1 -- 172.21.15.46:0/3407135551 wait complete. 2024-04-25T08:44:10.177 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:44:10.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd8b800640 1 Processor -- start 2024-04-25T08:44:10.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd8b800640 1 -- start start 2024-04-25T08:44:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd8b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd8415bdb0 0x7fcd840cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd8b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd840cda80 0x7fcd8415d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd8b800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd840cc700 con 0x7fcd84159230 2024-04-25T08:44:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd8b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd840cc9e0 con 0x7fcd840cda80 2024-04-25T08:44:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd8b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840cccc0 con 0x7fcd8415bdb0 2024-04-25T08:44:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd88e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcd84159230 0x7fcd840cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39696/0 (socket says 172.21.15.46:39696) 2024-04-25T08:44:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd88e00640 1 -- 172.21.15.46:0/4291322064 learned_addr learned my addr 172.21.15.46:0/4291322064 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83e00640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd8415bdb0 0x7fcd840cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd89800640 1 --2- 172.21.15.46:0/4291322064 >> v2:172.21.15.82:3300/0 conn(0x7fcd840cda80 0x7fcd8415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350393929 0 0) 0x7fcd840cc700 con 0x7fcd84159230 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd6c003530 con 0x7fcd84159230 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2251879257 0 0) 0x7fcd6c003530 con 0x7fcd84159230 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd8415bdb0 msgr2=0x7fcd840cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd8415bdb0 0x7fcd840cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 >> v2:172.21.15.82:3300/0 conn(0x7fcd840cda80 msgr2=0x7fcd8415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 --2- 172.21.15.46:0/4291322064 >> v2:172.21.15.82:3300/0 conn(0x7fcd840cda80 0x7fcd8415d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd840ccfa0 con 0x7fcd84159230 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fcd78001680 con 0x7fcd84159230 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 >> v1:172.21.15.46:6789/0 conn(0x7fcd84159230 legacy=0x7fcd840cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd8415bdb0 0x7fcd6c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c003f40 0x7fcd6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c004d50 0x7fcd6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840cc9e0 con 0x7fcd6c004d50 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840cccc0 con 0x7fcd6c003f40 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.331+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6c00ba20 con 0x7fcd8415bdb0 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd89800640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c004d50 0x7fcd6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83e00640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c003f40 0x7fcd6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd88e00640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd8415bdb0 0x7fcd6c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83e00640 1 -- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd8415bdb0 msgr2=0x7fcd6c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83e00640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd8415bdb0 0x7fcd6c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83e00640 1 -- 172.21.15.46:0/4291322064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c004d50 msgr2=0x7fcd6c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:10.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83e00640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c004d50 0x7fcd6c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83e00640 1 -- 172.21.15.46:0/4291322064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c003530 con 0x7fcd6c003f40 2024-04-25T08:44:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd89800640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c004d50 0x7fcd6c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83e00640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c003f40 0x7fcd6c004810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009ae0 tx=0x7fcd7001ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8d1b67fdd059f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd700b2070 con 0x7fcd6c003f40 2024-04-25T08:44:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd700a5660 con 0x7fcd6c003f40 2024-04-25T08:44:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.335+0000 7fcd83400640 1 -- 172.21.15.46:0/4291322064 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd700096b0 con 0x7fcd6c003f40 2024-04-25T08:44:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 -- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c003f40 msgr2=0x7fcd6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 --2- 172.21.15.46:0/4291322064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd6c003f40 0x7fcd6c004810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009ae0 tx=0x7fcd7001ca70 comp rx=0 tx=0).stop 2024-04-25T08:44:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd89800640 1 -- 172.21.15.46:0/4291322064 reap_dead start 2024-04-25T08:44:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 -- 172.21.15.46:0/4291322064 shutdown_connections 2024-04-25T08:44:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 -- 172.21.15.46:0/4291322064 >> 172.21.15.46:0/4291322064 conn(0x7fcd840b12a0 msgr2=0x7fcd840b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 -- 172.21.15.46:0/4291322064 shutdown_connections 2024-04-25T08:44:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 -- 172.21.15.46:0/4291322064 wait complete. 2024-04-25T08:44:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 Processor -- start 2024-04-25T08:44:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 -- start start 2024-04-25T08:44:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c003f40 0x7fcd84158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd84158970 0x7fcd8415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd841e8d00 0x7fcd8415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd84159790 con 0x7fcd6c003f40 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd84159610 con 0x7fcd84158970 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd8b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840cd540 con 0x7fcd841e8d00 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd88e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c003f40 0x7fcd84158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd88e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c003f40 0x7fcd84158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48046/0 (socket says 172.21.15.46:48046) 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd88e00640 1 -- 172.21.15.46:0/1694265197 learned_addr learned my addr 172.21.15.46:0/1694265197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd89800640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd841e8d00 0x7fcd8415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd83e00640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd84158970 0x7fcd8415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd88e00640 1 -- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd841e8d00 msgr2=0x7fcd8415a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.339+0000 7fcd88e00640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd841e8d00 0x7fcd8415a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd88e00640 1 -- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd84158970 msgr2=0x7fcd8415a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd88e00640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd84158970 0x7fcd8415a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd88e00640 1 -- 172.21.15.46:0/1694265197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd841e5a30 con 0x7fcd6c003f40 2024-04-25T08:44:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd83e00640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd84158970 0x7fcd8415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd88e00640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c003f40 0x7fcd84158430 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd78012510 tx=0x7fcd780129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd80c00640 1 -- 172.21.15.46:0/1694265197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7801c070 con 0x7fcd6c003f40 2024-04-25T08:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd80c00640 1 -- 172.21.15.46:0/1694265197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd78014830 con 0x7fcd6c003f40 2024-04-25T08:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd80c00640 1 -- 172.21.15.46:0/1694265197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78014c20 con 0x7fcd6c003f40 2024-04-25T08:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd841e91d0 con 0x7fcd6c003f40 2024-04-25T08:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd841e9790 con 0x7fcd6c003f40 2024-04-25T08:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd84069080 con 0x7fcd6c003f40 2024-04-25T08:44:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd80c00640 1 -- 172.21.15.46:0/1694265197 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd780153e0 con 0x7fcd6c003f40 2024-04-25T08:44:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd80c00640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd64066f60 0x7fcd64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd80c00640 1 -- 172.21.15.46:0/1694265197 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcd780878b0 con 0x7fcd6c003f40 2024-04-25T08:44:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.343+0000 7fcd83e00640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd64066f60 0x7fcd64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.347+0000 7fcd83e00640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd64066f60 0x7fcd64069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009ae0 tx=0x7fcd700046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.351+0000 7fcd80c00640 1 -- 172.21.15.46:0/1694265197 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd841e5a30 con 0x7fcd6c003f40 2024-04-25T08:44:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.567+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcd840723d0 con 0x7fcd6c003f40 2024-04-25T08:44:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd80c00640 1 -- 172.21.15.46:0/1694265197 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcd780589e0 con 0x7fcd6c003f40 2024-04-25T08:44:10.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:10.563 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:44:10.563 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-25T08:44:10.563 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.563 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-25T08:44:10.563 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmq+gDGRAAtB5ObM/zelMmCZ4cJPghdQ==", 2024-04-25T08:44:10.563 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.563 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmyIaAJRAA9zodtgftP7faNth4UPyCTA==", 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.564 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmFahWIBAAORHmzDLXPjuANF+/W2wKpQ==", 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.565 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmRhrPLBAATXOHl1jGJYmIlnYk/JPcOw==", 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA0FypmaHiqEhAAYzasCl8t+pls5IHe9tCP4g==", 2024-04-25T08:44:10.566 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA3FypmOGTuGhAA37F+qRI/+pj5dkLc3TBIUQ==", 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:44:10.567 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA6Fypm/YbNJRAAP1QyYDLY/xFvRz63R+H/Pw==", 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-25T08:44:10.568 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA9FypmiT8ALhAAX/2CJJeJ+2SGNhwah8NTBQ==", 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBDFypmyG88JBAAq1065gzLIIR/6fYaw0bBTg==", 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.569 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBGFypmNir3KxAAszCXJJ1KFtCusDZhBHAZSQ==", 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-25T08:44:10.570 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBJFypm4r7LNxAA42A07ntI6OOgblk+4i1Yiw==", 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBNFypmG1fOBhAAOKRgw04jIo8H3u2hWIZe2g==", 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.571 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q==", 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmxPyHOhAADBuigZlq3GHURsJkB0nnyA==", 2024-04-25T08:44:10.572 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypm2PP0NhAAixV/kkY7eFPXUW/nJDOCOA==", 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:44:10.573 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAzFypmaVDtBBAAKnWk9CxZzg2r6T/IycuFEg==", 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.574 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAxFypm2W2MOBAAN7TDQiZXFV6+pWMTar95sg==", 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-25T08:44:10.575 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmzhptFBAAlFQZQn1GSseEguy5zfw2CA==", 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmAY6CEBAA2qHNALNTRKmCJRZatDqAvw==", 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:44:10.576 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:44:10.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:10.577 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-25T08:44:10.577 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:44:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd64066f60 msgr2=0x7fcd64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd64066f60 0x7fcd64069420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009ae0 tx=0x7fcd700046d0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c003f40 msgr2=0x7fcd84158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c003f40 0x7fcd84158430 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd78012510 tx=0x7fcd780129d0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 shutdown_connections 2024-04-25T08:44:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd64066f60 0x7fcd64069420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd841e8d00 0x7fcd8415a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd84158970 0x7fcd8415a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 --2- 172.21.15.46:0/1694265197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd6c003f40 0x7fcd84158430 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 >> 172.21.15.46:0/1694265197 conn(0x7fcd840b12a0 msgr2=0x7fcd84143850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 shutdown_connections 2024-04-25T08:44:10.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.571+0000 7fcd8b800640 1 -- 172.21.15.46:0/1694265197 wait complete. 2024-04-25T08:44:10.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.731+0000 7f8d80600640 1 Processor -- start 2024-04-25T08:44:10.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.731+0000 7f8d80600640 1 -- start start 2024-04-25T08:44:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d80600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 0x7f8d7815f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d80600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8d7806f110 0x7f8d780ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d80600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d78161330 con 0x7f8d780adc50 2024-04-25T08:44:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d80600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d780ae5f0 con 0x7f8d7806f110 2024-04-25T08:44:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d80600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7806d680 con 0x7f8d78160f50 2024-04-25T08:44:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8d7806f110 0x7f8d780ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7d200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8d780adc50 0x7f8d7806ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39704/0 (socket says 172.21.15.46:39704) 2024-04-25T08:44:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7d200640 1 -- 172.21.15.46:0/1347757427 learned_addr learned my addr 172.21.15.46:0/1347757427 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7dc00640 1 --2- 172.21.15.46:0/1347757427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 0x7f8d7815f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7e600640 1 -- 172.21.15.46:0/1347757427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 msgr2=0x7f8d7815f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7e600640 1 --2- 172.21.15.46:0/1347757427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 0x7f8d7815f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7e600640 1 -- 172.21.15.46:0/1347757427 >> v1:172.21.15.46:6789/0 conn(0x7f8d780adc50 legacy=0x7f8d7806ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7e600640 1 -- 172.21.15.46:0/1347757427 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d780ae810 con 0x7f8d7806f110 2024-04-25T08:44:10.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d77e00640 1 -- 172.21.15.46:0/1347757427 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541024571 0 0) 0x7f8d78161330 con 0x7f8d780adc50 2024-04-25T08:44:10.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7dc00640 1 --2- 172.21.15.46:0/1347757427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 0x7f8d7815f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:10.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.735+0000 7f8d7e600640 1 --2- 172.21.15.46:0/1347757427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7806f110 0x7f8d780ae0b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009d90 tx=0x7f8d7001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b343e1de8e36eae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:10.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d77e00640 1 -- 172.21.15.46:0/1347757427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700b2070 con 0x7f8d7806f110 2024-04-25T08:44:10.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d77e00640 1 -- 172.21.15.46:0/1347757427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d700a5a00 con 0x7f8d7806f110 2024-04-25T08:44:10.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d77e00640 1 -- 172.21.15.46:0/1347757427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700a5dd0 con 0x7f8d7806f110 2024-04-25T08:44:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 -- 172.21.15.46:0/1347757427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7806f110 msgr2=0x7f8d780ae0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 --2- 172.21.15.46:0/1347757427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7806f110 0x7f8d780ae0b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009d90 tx=0x7f8d7001c940 comp rx=0 tx=0).stop 2024-04-25T08:44:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 -- 172.21.15.46:0/1347757427 shutdown_connections 2024-04-25T08:44:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 --2- 172.21.15.46:0/1347757427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7806f110 0x7f8d780ae0b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 --2- 172.21.15.46:0/1347757427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 0x7f8d7815f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 -- 172.21.15.46:0/1347757427 >> 172.21.15.46:0/1347757427 conn(0x7f8d780b12a0 msgr2=0x7f8d780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 -- 172.21.15.46:0/1347757427 shutdown_connections 2024-04-25T08:44:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 -- 172.21.15.46:0/1347757427 wait complete. 2024-04-25T08:44:10.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 Processor -- start 2024-04-25T08:44:10.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 -- start start 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7806f110 0x7f8d78130200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d780adc50 0x7f8d781320e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 0x7f8d78132620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7806e300 con 0x7f8d780adc50 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780aedd0 con 0x7f8d7806f110 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d80600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7806e480 con 0x7f8d78160f50 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d7d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d780adc50 0x7f8d781320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d7d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d780adc50 0x7f8d781320e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48058/0 (socket says 172.21.15.46:48058) 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.739+0000 7f8d7d200640 1 -- 172.21.15.46:0/768527134 learned_addr learned my addr 172.21.15.46:0/768527134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:10.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d7e600640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 0x7f8d78132620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d7dc00640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7806f110 0x7f8d78130200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d7d200640 1 -- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 msgr2=0x7f8d78132620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d7d200640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 0x7f8d78132620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d7d200640 1 -- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7806f110 msgr2=0x7f8d78130200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d7d200640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7806f110 0x7f8d78130200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d7d200640 1 -- 172.21.15.46:0/768527134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7812f160 con 0x7f8d780adc50 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d7d200640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d780adc50 0x7f8d781320e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8d640011f0 tx=0x7f8d6400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d75600640 1 -- 172.21.15.46:0/768527134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64018070 con 0x7f8d780adc50 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d75600640 1 -- 172.21.15.46:0/768527134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d640074e0 con 0x7f8d780adc50 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d75600640 1 -- 172.21.15.46:0/768527134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64004a10 con 0x7f8d780adc50 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7812f440 con 0x7f8d780adc50 2024-04-25T08:44:10.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7812fa00 con 0x7f8d780adc50 2024-04-25T08:44:10.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d78069080 con 0x7f8d780adc50 2024-04-25T08:44:10.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d75600640 1 -- 172.21.15.46:0/768527134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d64004bb0 con 0x7f8d780adc50 2024-04-25T08:44:10.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d75600640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d58066f60 0x7f8d58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:10.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.743+0000 7f8d75600640 1 -- 172.21.15.46:0/768527134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8d64084160 con 0x7f8d780adc50 2024-04-25T08:44:10.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.747+0000 7f8d7dc00640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d58066f60 0x7f8d58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:10.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.747+0000 7f8d7dc00640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d58066f60 0x7f8d58069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d78131d80 tx=0x7f8d6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:10.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.747+0000 7f8d75600640 1 -- 172.21.15.46:0/768527134 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d64054c40 con 0x7f8d780adc50 2024-04-25T08:44:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.915+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d780b27f0 con 0x7f8d780adc50 2024-04-25T08:44:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.915+0000 7f8d75600640 1 -- 172.21.15.46:0/768527134 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+13116 (secure 0 0 0) 0x7f8d64058ce0 con 0x7f8d780adc50 2024-04-25T08:44:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:10.909 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:10.157682+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d58066f60 msgr2=0x7f8d58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d58066f60 0x7f8d58069420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d78131d80 tx=0x7f8d6c008040 comp rx=0 tx=0).stop 2024-04-25T08:44:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d780adc50 msgr2=0x7f8d781320e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:10.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d780adc50 0x7f8d781320e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8d640011f0 tx=0x7f8d6400c9b0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 shutdown_connections 2024-04-25T08:44:10.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d58066f60 0x7f8d58069420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d78160f50 0x7f8d78132620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d780adc50 0x7f8d781320e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 --2- 172.21.15.46:0/768527134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7806f110 0x7f8d78130200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:10.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 >> 172.21.15.46:0/768527134 conn(0x7f8d780b12a0 msgr2=0x7f8d780b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:10.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 shutdown_connections 2024-04-25T08:44:10.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:10.919+0000 7f8d80600640 1 -- 172.21.15.46:0/768527134 wait complete. 2024-04-25T08:44:10.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:44:10.931 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:44:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51f7200640 1 Processor -- start 2024-04-25T08:44:11.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51f7200640 1 -- start start 2024-04-25T08:44:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51f7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51f006ed90 0x7f51f0147070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51f7200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f51f0071d80 0x7f51f015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51f7200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51f015e460 con 0x7f51f01475b0 2024-04-25T08:44:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51f7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f015e740 con 0x7f51f006ed90 2024-04-25T08:44:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51f7200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51f015ea20 con 0x7f51f0071d80 2024-04-25T08:44:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51e7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f51f01475b0 0x7f51f0071670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39712/0 (socket says 172.21.15.46:39712) 2024-04-25T08:44:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51e7e00640 1 -- 172.21.15.46:0/3870292770 learned_addr learned my addr 172.21.15.46:0/3870292770 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51efe00640 1 --2- 172.21.15.46:0/3870292770 >> v2:172.21.15.46:3300/0 conn(0x7f51f006ed90 0x7f51f0147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51f5200640 1 --2- 172.21.15.46:0/3870292770 >> v2:172.21.15.82:3300/0 conn(0x7f51f0071d80 0x7f51f015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.083+0000 7f51ef400640 1 -- 172.21.15.46:0/3870292770 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549758552 0 0) 0x7f51f015e460 con 0x7f51f01475b0 2024-04-25T08:44:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51efe00640 1 -- 172.21.15.46:0/3870292770 >> v2:172.21.15.82:3300/0 conn(0x7f51f0071d80 msgr2=0x7f51f015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51efe00640 1 --2- 172.21.15.46:0/3870292770 >> v2:172.21.15.82:3300/0 conn(0x7f51f0071d80 0x7f51f015f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51efe00640 1 -- 172.21.15.46:0/3870292770 >> v1:172.21.15.46:6789/0 conn(0x7f51f01475b0 legacy=0x7f51f0071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51efe00640 1 -- 172.21.15.46:0/3870292770 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f015ed00 con 0x7f51f006ed90 2024-04-25T08:44:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51efe00640 1 --2- 172.21.15.46:0/3870292770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f006ed90 0x7f51f0147070 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f51f00cf390 tx=0x7f51e001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2417322a4d60a675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51ef400640 1 -- 172.21.15.46:0/3870292770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00ab430 con 0x7f51f006ed90 2024-04-25T08:44:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51ef400640 1 -- 172.21.15.46:0/3870292770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e00b0070 con 0x7f51f006ed90 2024-04-25T08:44:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51ef400640 1 -- 172.21.15.46:0/3870292770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00ab7f0 con 0x7f51f006ed90 2024-04-25T08:44:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51f7200640 1 -- 172.21.15.46:0/3870292770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f006ed90 msgr2=0x7f51f0147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51f7200640 1 --2- 172.21.15.46:0/3870292770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f006ed90 0x7f51f0147070 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f51f00cf390 tx=0x7f51e001c410 comp rx=0 tx=0).stop 2024-04-25T08:44:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51f7200640 1 -- 172.21.15.46:0/3870292770 shutdown_connections 2024-04-25T08:44:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51f7200640 1 --2- 172.21.15.46:0/3870292770 >> v2:172.21.15.82:3300/0 conn(0x7f51f0071d80 0x7f51f015f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51f7200640 1 --2- 172.21.15.46:0/3870292770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f006ed90 0x7f51f0147070 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51f7200640 1 -- 172.21.15.46:0/3870292770 >> 172.21.15.46:0/3870292770 conn(0x7f51f00b1670 msgr2=0x7f51f00b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51f7200640 1 -- 172.21.15.46:0/3870292770 shutdown_connections 2024-04-25T08:44:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.087+0000 7f51f7200640 1 -- 172.21.15.46:0/3870292770 wait complete. 2024-04-25T08:44:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 Processor -- start 2024-04-25T08:44:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 -- start start 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51f006ed90 0x7f51f015b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f0071d80 0x7f51f00acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51f015bbc0 0x7f51f015c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f01605c0 con 0x7f51f0071d80 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f0160440 con 0x7f51f015bbc0 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f0160740 con 0x7f51f006ed90 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f0071d80 0x7f51f00acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f0071d80 0x7f51f00acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48084/0 (socket says 172.21.15.46:48084) 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51e7e00640 1 -- 172.21.15.46:0/2572989637 learned_addr learned my addr 172.21.15.46:0/2572989637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f5200640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51f015bbc0 0x7f51f015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51efe00640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51f006ed90 0x7f51f015b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51e7e00640 1 -- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51f006ed90 msgr2=0x7f51f015b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51e7e00640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51f006ed90 0x7f51f015b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51e7e00640 1 -- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51f015bbc0 msgr2=0x7f51f015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51e7e00640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51f015bbc0 0x7f51f015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51e7e00640 1 -- 172.21.15.46:0/2572989637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f013b910 con 0x7f51f0071d80 2024-04-25T08:44:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51efe00640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51f006ed90 0x7f51f015b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51e7e00640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f0071d80 0x7f51f00acda0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c3e0 tx=0x7f51d800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51ecc00640 1 -- 172.21.15.46:0/2572989637 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d80074e0 con 0x7f51f0071d80 2024-04-25T08:44:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51ecc00640 1 -- 172.21.15.46:0/2572989637 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d80047f0 con 0x7f51f0071d80 2024-04-25T08:44:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51ecc00640 1 -- 172.21.15.46:0/2572989637 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8007680 con 0x7f51f0071d80 2024-04-25T08:44:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f013c8d0 con 0x7f51f0071d80 2024-04-25T08:44:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.091+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f013ce90 con 0x7f51f0071d80 2024-04-25T08:44:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.095+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f0069010 con 0x7f51f0071d80 2024-04-25T08:44:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.095+0000 7f51ecc00640 1 -- 172.21.15.46:0/2572989637 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51d8011070 con 0x7f51f0071d80 2024-04-25T08:44:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.095+0000 7f51ecc00640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f51cc066f60 0x7f51cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.095+0000 7f51ecc00640 1 -- 172.21.15.46:0/2572989637 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f51d8084a70 con 0x7f51f0071d80 2024-04-25T08:44:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.095+0000 7f51efe00640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f51cc066f60 0x7f51cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.095+0000 7f51efe00640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f51cc066f60 0x7f51cc069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f51e00af000 tx=0x7f51e00a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.099+0000 7f51ecc00640 1 -- 172.21.15.46:0/2572989637 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51d8055c30 con 0x7f51f0071d80 2024-04-25T08:44:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.263+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51f01686c0 con 0x7f51f0071d80 2024-04-25T08:44:11.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51ecc00640 1 -- 172.21.15.46:0/2572989637 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51d8055530 con 0x7f51f0071d80 2024-04-25T08:44:11.258 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:44:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f51cc066f60 msgr2=0x7f51cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f51cc066f60 0x7f51cc069420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f51e00af000 tx=0x7f51e00a7000 comp rx=0 tx=0).stop 2024-04-25T08:44:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f0071d80 msgr2=0x7f51f00acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f0071d80 0x7f51f00acda0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c3e0 tx=0x7f51d800c8a0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 shutdown_connections 2024-04-25T08:44:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f51cc066f60 0x7f51cc069420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51f015bbc0 0x7f51f015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51f0071d80 0x7f51f00acda0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 --2- 172.21.15.46:0/2572989637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51f006ed90 0x7f51f015b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 >> 172.21.15.46:0/2572989637 conn(0x7f51f00b1670 msgr2=0x7f51f00b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 shutdown_connections 2024-04-25T08:44:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.267+0000 7f51f7200640 1 -- 172.21.15.46:0/2572989637 wait complete. 2024-04-25T08:44:11.280 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:44:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.427+0000 7f27f4e00640 1 Processor -- start 2024-04-25T08:44:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.427+0000 7f27f4e00640 1 -- start start 2024-04-25T08:44:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.427+0000 7f27f4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27f0159230 0x7f27f015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.427+0000 7f27f4e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f00cc700 con 0x7f27f00cdc50 2024-04-25T08:44:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.427+0000 7f27f4e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f00cc9e0 con 0x7f27f00cbeb0 2024-04-25T08:44:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.427+0000 7f27f4e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f00cccc0 con 0x7f27f0159230 2024-04-25T08:44:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ee800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f27f00cdc50 0x7f27f015d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39722/0 (socket says 172.21.15.46:39722) 2024-04-25T08:44:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ee800640 1 -- 172.21.15.46:0/272364296 learned_addr learned my addr 172.21.15.46:0/272364296 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ede00640 1 --2- 172.21.15.46:0/272364296 >> v2:172.21.15.46:3300/0 conn(0x7f27f0159230 0x7f27f015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ed400640 1 -- 172.21.15.46:0/272364296 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3851939651 0 0) 0x7f27f00cc700 con 0x7f27f00cdc50 2024-04-25T08:44:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ed400640 1 -- 172.21.15.46:0/272364296 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d4003530 con 0x7f27f00cdc50 2024-04-25T08:44:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ede00640 1 -- 172.21.15.46:0/272364296 >> v1:172.21.15.82:6789/0 conn(0x7f27f00cbeb0 legacy=0x7f27f00cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ede00640 1 -- 172.21.15.46:0/272364296 >> v1:172.21.15.46:6789/0 conn(0x7f27f00cdc50 legacy=0x7f27f015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ede00640 1 -- 172.21.15.46:0/272364296 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f00ccfa0 con 0x7f27f0159230 2024-04-25T08:44:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ede00640 1 --2- 172.21.15.46:0/272364296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f0159230 0x7f27f015bdb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f27e00097c0 tx=0x7f27e001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17559e0b8ed95cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ed400640 1 -- 172.21.15.46:0/272364296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00ab600 con 0x7f27f0159230 2024-04-25T08:44:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ed400640 1 -- 172.21.15.46:0/272364296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00b0070 con 0x7f27f0159230 2024-04-25T08:44:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.431+0000 7f27ed400640 1 -- 172.21.15.46:0/272364296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00ab9c0 con 0x7f27f0159230 2024-04-25T08:44:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 -- 172.21.15.46:0/272364296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f0159230 msgr2=0x7f27f015bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 --2- 172.21.15.46:0/272364296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f0159230 0x7f27f015bdb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f27e00097c0 tx=0x7f27e001c3e0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 -- 172.21.15.46:0/272364296 shutdown_connections 2024-04-25T08:44:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 --2- 172.21.15.46:0/272364296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f0159230 0x7f27f015bdb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 -- 172.21.15.46:0/272364296 >> 172.21.15.46:0/272364296 conn(0x7f27f00b12a0 msgr2=0x7f27f00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 -- 172.21.15.46:0/272364296 shutdown_connections 2024-04-25T08:44:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 -- 172.21.15.46:0/272364296 wait complete. 2024-04-25T08:44:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 Processor -- start 2024-04-25T08:44:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 -- start start 2024-04-25T08:44:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f00cbeb0 0x7f27f013e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27f00cdc50 0x7f27f006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27f0159230 0x7f27f0142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f015e120 con 0x7f27f00cbeb0 2024-04-25T08:44:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00cc4a0 con 0x7f27f00cdc50 2024-04-25T08:44:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27f4e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f015e2a0 con 0x7f27f0159230 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f00cbeb0 0x7f27f013e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f00cbeb0 0x7f27f013e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48094/0 (socket says 172.21.15.46:48094) 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ede00640 1 -- 172.21.15.46:0/2342666311 learned_addr learned my addr 172.21.15.46:0/2342666311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27e5400640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27f00cdc50 0x7f27f006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ede00640 1 -- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27f0159230 msgr2=0x7f27f0142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ee800640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27f0159230 0x7f27f0142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ede00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27f0159230 0x7f27f0142bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ede00640 1 -- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27f00cdc50 msgr2=0x7f27f006d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ede00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27f00cdc50 0x7f27f006d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ede00640 1 -- 172.21.15.46:0/2342666311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f006d920 con 0x7f27f00cbeb0 2024-04-25T08:44:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.435+0000 7f27ee800640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27f0159230 0x7f27f0142bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27ede00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f00cbeb0 0x7f27f013e7c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f27e0018ff0 tx=0x7f27e001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27e6a00640 1 -- 172.21.15.46:0/2342666311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00bf610 con 0x7f27f00cbeb0 2024-04-25T08:44:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27e6a00640 1 -- 172.21.15.46:0/2342666311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00bf7d0 con 0x7f27f00cbeb0 2024-04-25T08:44:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f006dc00 con 0x7f27f00cbeb0 2024-04-25T08:44:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27e6a00640 1 -- 172.21.15.46:0/2342666311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00bfbc0 con 0x7f27f00cbeb0 2024-04-25T08:44:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f006e440 con 0x7f27f00cbeb0 2024-04-25T08:44:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f0069080 con 0x7f27f00cbeb0 2024-04-25T08:44:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27e6a00640 1 -- 172.21.15.46:0/2342666311 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27e00bd020 con 0x7f27f00cbeb0 2024-04-25T08:44:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27e6a00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27cc066f60 0x7f27cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27e6a00640 1 -- 172.21.15.46:0/2342666311 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f27e011f860 con 0x7f27f00cbeb0 2024-04-25T08:44:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.439+0000 7f27e5400640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27cc066f60 0x7f27cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.443+0000 7f27e5400640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27cc066f60 0x7f27cc069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f27f006d300 tx=0x7f27d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.443+0000 7f27e6a00640 1 -- 172.21.15.46:0/2342666311 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27f006dc00 con 0x7f27f00cbeb0 2024-04-25T08:44:11.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.607+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f27f00723d0 con 0x7f27f00cbeb0 2024-04-25T08:44:11.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27e6a00640 1 -- 172.21.15.46:0/2342666311 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v49) v1 ==== 84+0+65 (secure 0 0 0) 0x7f27e00c4060 con 0x7f27f00cbeb0 2024-04-25T08:44:11.605 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:11.605 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:44:11.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-25T08:44:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-25T08:44:11.606 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:11.606 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:44:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27cc066f60 msgr2=0x7f27cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27cc066f60 0x7f27cc069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f27f006d300 tx=0x7f27d8009040 comp rx=0 tx=0).stop 2024-04-25T08:44:11.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f00cbeb0 msgr2=0x7f27f013e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f00cbeb0 0x7f27f013e7c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f27e0018ff0 tx=0x7f27e001ac20 comp rx=0 tx=0).stop 2024-04-25T08:44:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 shutdown_connections 2024-04-25T08:44:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27cc066f60 0x7f27cc069420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27f0159230 0x7f27f0142bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27f00cdc50 0x7f27f006d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 --2- 172.21.15.46:0/2342666311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f00cbeb0 0x7f27f013e7c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 >> 172.21.15.46:0/2342666311 conn(0x7f27f00b12a0 msgr2=0x7f27f015d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 shutdown_connections 2024-04-25T08:44:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.611+0000 7f27f4e00640 1 -- 172.21.15.46:0/2342666311 wait complete. 2024-04-25T08:44:11.624 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:44:11.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:44:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.775+0000 7f87bae00640 1 Processor -- start 2024-04-25T08:44:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.775+0000 7f87bae00640 1 -- start start 2024-04-25T08:44:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87bae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87b413e6b0 0x7f87b40711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87bae00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 0x7f87b406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87bae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b406d330 con 0x7f87b4071720 2024-04-25T08:44:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87bae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b415cce0 con 0x7f87b413e6b0 2024-04-25T08:44:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87bae00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b406d610 con 0x7f87b415c680 2024-04-25T08:44:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87abe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87b4071720 0x7f87b415bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39738/0 (socket says 172.21.15.46:39738) 2024-04-25T08:44:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87abe00640 1 -- 172.21.15.46:0/3760233864 learned_addr learned my addr 172.21.15.46:0/3760233864 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87b3e00640 1 --2- 172.21.15.46:0/3760233864 >> v2:172.21.15.46:3300/0 conn(0x7f87b413e6b0 0x7f87b40711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87b8e00640 1 --2- 172.21.15.46:0/3760233864 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 0x7f87b406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87b3e00640 1 -- 172.21.15.46:0/3760233864 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 msgr2=0x7f87b406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87b3e00640 1 --2- 172.21.15.46:0/3760233864 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 0x7f87b406e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87b3e00640 1 -- 172.21.15.46:0/3760233864 >> v1:172.21.15.46:6789/0 conn(0x7f87b4071720 legacy=0x7f87b415bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87b3400640 1 -- 172.21.15.46:0/3760233864 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719492110 0 0) 0x7f87b406d330 con 0x7f87b4071720 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.779+0000 7f87b3e00640 1 -- 172.21.15.46:0/3760233864 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b406d8f0 con 0x7f87b413e6b0 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87b3e00640 1 --2- 172.21.15.46:0/3760233864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b413e6b0 0x7f87b40711e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f87a40097c0 tx=0x7f87a401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b0035ba49c93f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87b3400640 1 -- 172.21.15.46:0/3760233864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40b0070 con 0x7f87b413e6b0 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87b3400640 1 -- 172.21.15.46:0/3760233864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a40ab600 con 0x7f87b413e6b0 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87b3400640 1 -- 172.21.15.46:0/3760233864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40ab9d0 con 0x7f87b413e6b0 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 -- 172.21.15.46:0/3760233864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b413e6b0 msgr2=0x7f87b40711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 --2- 172.21.15.46:0/3760233864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b413e6b0 0x7f87b40711e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f87a40097c0 tx=0x7f87a401c380 comp rx=0 tx=0).stop 2024-04-25T08:44:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 -- 172.21.15.46:0/3760233864 shutdown_connections 2024-04-25T08:44:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 --2- 172.21.15.46:0/3760233864 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 0x7f87b406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 --2- 172.21.15.46:0/3760233864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b413e6b0 0x7f87b40711e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 -- 172.21.15.46:0/3760233864 >> 172.21.15.46:0/3760233864 conn(0x7f87b4067050 msgr2=0x7f87b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 -- 172.21.15.46:0/3760233864 shutdown_connections 2024-04-25T08:44:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 -- 172.21.15.46:0/3760233864 wait complete. 2024-04-25T08:44:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 Processor -- start 2024-04-25T08:44:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.783+0000 7f87bae00640 1 -- start start 2024-04-25T08:44:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87bae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b4071720 0x7f87b415fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87bae00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87b413e6b0 0x7f87b415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87bae00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 0x7f87b4138c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87bae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4071b70 con 0x7f87b4071720 2024-04-25T08:44:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87bae00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b415ca90 con 0x7f87b413e6b0 2024-04-25T08:44:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87bae00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4168590 con 0x7f87b415c680 2024-04-25T08:44:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b4071720 0x7f87b415fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b4071720 0x7f87b415fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48108/0 (socket says 172.21.15.46:48108) 2024-04-25T08:44:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b3e00640 1 -- 172.21.15.46:0/4042481504 learned_addr learned my addr 172.21.15.46:0/4042481504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87abe00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87b413e6b0 0x7f87b415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b8e00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 0x7f87b4138c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b3e00640 1 -- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 msgr2=0x7f87b4138c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b3e00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 0x7f87b4138c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b3e00640 1 -- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87b413e6b0 msgr2=0x7f87b415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b3e00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87b413e6b0 0x7f87b415f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b3e00640 1 -- 172.21.15.46:0/4042481504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b41603e0 con 0x7f87b4071720 2024-04-25T08:44:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87abe00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87b413e6b0 0x7f87b415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b3e00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b4071720 0x7f87b415fd00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f87a4019fb0 tx=0x7f87a401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b0c00640 1 -- 172.21.15.46:0/4042481504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40b0050 con 0x7f87b4071720 2024-04-25T08:44:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b0c00640 1 -- 172.21.15.46:0/4042481504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a40bc070 con 0x7f87b4071720 2024-04-25T08:44:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b0c00640 1 -- 172.21.15.46:0/4042481504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40b5830 con 0x7f87b4071720 2024-04-25T08:44:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b41606c0 con 0x7f87b4071720 2024-04-25T08:44:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b413be80 con 0x7f87b4071720 2024-04-25T08:44:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b4069080 con 0x7f87b4071720 2024-04-25T08:44:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.787+0000 7f87b0c00640 1 -- 172.21.15.46:0/4042481504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87a40b59d0 con 0x7f87b4071720 2024-04-25T08:44:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.791+0000 7f87b0c00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8790066f60 0x7f8790069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.791+0000 7f87b0c00640 1 -- 172.21.15.46:0/4042481504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f87a411f730 con 0x7f87b4071720 2024-04-25T08:44:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.791+0000 7f87abe00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8790066f60 0x7f8790069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.791+0000 7f87abe00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8790066f60 0x7f8790069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f87b41655b0 tx=0x7f879c007cb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:11.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.795+0000 7f87b0c00640 1 -- 172.21.15.46:0/4042481504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87a4016090 con 0x7f87b4071720 2024-04-25T08:44:11.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:11.975+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f87b40cf240 con 0x7f87b4071720 2024-04-25T08:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.803+0000 7f87b0c00640 1 -- 172.21.15.46:0/4042481504 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v50) v1 ==== 138+0+0 (secure 0 0 0) 0x7f87a40f08f0 con 0x7f87b4071720 2024-04-25T08:44:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-25T08:44:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8790066f60 msgr2=0x7f8790069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8790066f60 0x7f8790069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f87b41655b0 tx=0x7f879c007cb0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b4071720 msgr2=0x7f87b415fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b4071720 0x7f87b415fd00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f87a4019fb0 tx=0x7f87a401abc0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 shutdown_connections 2024-04-25T08:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8790066f60 0x7f8790069420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87b415c680 0x7f87b4138c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87b413e6b0 0x7f87b415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 --2- 172.21.15.46:0/4042481504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87b4071720 0x7f87b415fd00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 >> 172.21.15.46:0/4042481504 conn(0x7f87b4067050 msgr2=0x7f87b415b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 shutdown_connections 2024-04-25T08:44:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.807+0000 7f87bae00640 1 -- 172.21.15.46:0/4042481504 wait complete. 2024-04-25T08:44:12.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:44:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f1104c00640 1 Processor -- start 2024-04-25T08:44:12.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f1104c00640 1 -- start start 2024-04-25T08:44:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f1104c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f110006b230 0x7f11000cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:12.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f1104c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1100162380 0x7f11000cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:12.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f1104c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110006b610 con 0x7f11000ce220 2024-04-25T08:44:12.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f1104c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110006f980 con 0x7f1100162380 2024-04-25T08:44:12.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f1104c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110006fc60 con 0x7f110006b230 2024-04-25T08:44:12.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10f5400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f11000ce220 0x7f1100161c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39750/0 (socket says 172.21.15.46:39750) 2024-04-25T08:44:12.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10f5400640 1 -- 172.21.15.46:0/1033130912 learned_addr learned my addr 172.21.15.46:0/1033130912 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:12.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10fde00640 1 --2- 172.21.15.46:0/1033130912 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f110006b230 0x7f11000cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:12.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10fe800640 1 --2- 172.21.15.46:0/1033130912 >> v2:172.21.15.82:3300/0 conn(0x7f1100162380 0x7f11000cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10fd400640 1 -- 172.21.15.46:0/1033130912 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233662983 0 0) 0x7f110006b610 con 0x7f11000ce220 2024-04-25T08:44:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10fd400640 1 -- 172.21.15.46:0/1033130912 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e4003530 con 0x7f11000ce220 2024-04-25T08:44:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10fde00640 1 -- 172.21.15.46:0/1033130912 >> v2:172.21.15.82:3300/0 conn(0x7f1100162380 msgr2=0x7f11000cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10fde00640 1 --2- 172.21.15.46:0/1033130912 >> v2:172.21.15.82:3300/0 conn(0x7f1100162380 0x7f11000cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10fde00640 1 -- 172.21.15.46:0/1033130912 >> v1:172.21.15.46:6789/0 conn(0x7f11000ce220 legacy=0x7f1100161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10fde00640 1 -- 172.21.15.46:0/1033130912 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110006ff40 con 0x7f110006b230 2024-04-25T08:44:12.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.987+0000 7f10fe800640 1 --2- 172.21.15.46:0/1033130912 >> v2:172.21.15.82:3300/0 conn(0x7f1100162380 0x7f11000cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:12.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f10fde00640 1 --2- 172.21.15.46:0/1033130912 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f110006b230 0x7f11000cccf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097c0 tx=0x7f10f001c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=145abffd527a8e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f10fd400640 1 -- 172.21.15.46:0/1033130912 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00b0070 con 0x7f110006b230 2024-04-25T08:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f10fd400640 1 -- 172.21.15.46:0/1033130912 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f00ab600 con 0x7f110006b230 2024-04-25T08:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f10fd400640 1 -- 172.21.15.46:0/1033130912 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f00ab9d0 con 0x7f110006b230 2024-04-25T08:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f1104c00640 1 -- 172.21.15.46:0/1033130912 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f110006b230 msgr2=0x7f11000cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f1104c00640 1 --2- 172.21.15.46:0/1033130912 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f110006b230 0x7f11000cccf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f10f00097c0 tx=0x7f10f001c350 comp rx=0 tx=0).stop 2024-04-25T08:44:12.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f1104c00640 1 -- 172.21.15.46:0/1033130912 shutdown_connections 2024-04-25T08:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f1104c00640 1 --2- 172.21.15.46:0/1033130912 >> v2:172.21.15.82:3300/0 conn(0x7f1100162380 0x7f11000cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f1104c00640 1 --2- 172.21.15.46:0/1033130912 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f110006b230 0x7f11000cccf0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f1104c00640 1 -- 172.21.15.46:0/1033130912 >> 172.21.15.46:0/1033130912 conn(0x7f1100099a00 msgr2=0x7f11000b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f1104c00640 1 -- 172.21.15.46:0/1033130912 shutdown_connections 2024-04-25T08:44:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.991+0000 7f1104c00640 1 -- 172.21.15.46:0/1033130912 wait complete. 2024-04-25T08:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 Processor -- start 2024-04-25T08:44:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 -- start start 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110006b230 0x7f11000b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100162380 0x7f11000b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11001605e0 0x7f1100158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000cff50 con 0x7f1100162380 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000cfdd0 con 0x7f110006b230 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11000d00d0 con 0x7f11001605e0 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100162380 0x7f11000b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100162380 0x7f11000b58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48112/0 (socket says 172.21.15.46:48112) 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f5400640 1 -- 172.21.15.46:0/734807209 learned_addr learned my addr 172.21.15.46:0/734807209 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10fde00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110006b230 0x7f11000b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10fe800640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11001605e0 0x7f1100158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f5400640 1 -- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11001605e0 msgr2=0x7f1100158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f5400640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11001605e0 0x7f1100158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f5400640 1 -- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110006b230 msgr2=0x7f11000b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f5400640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110006b230 0x7f11000b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f5400640 1 -- 172.21.15.46:0/734807209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1100157dc0 con 0x7f1100162380 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10fe800640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11001605e0 0x7f1100158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f5400640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100162380 0x7f11000b58d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c3f0 tx=0x7f10e800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f6a00640 1 -- 172.21.15.46:0/734807209 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8017070 con 0x7f1100162380 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f6a00640 1 -- 172.21.15.46:0/734807209 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e8005cf0 con 0x7f1100162380 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f10f6a00640 1 -- 172.21.15.46:0/734807209 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80072f0 con 0x7f1100162380 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11001580a0 con 0x7f1100162380 2024-04-25T08:44:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.995+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1100158660 con 0x7f1100162380 2024-04-25T08:44:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.999+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1100069010 con 0x7f1100162380 2024-04-25T08:44:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.999+0000 7f10f6a00640 1 -- 172.21.15.46:0/734807209 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10e801a070 con 0x7f1100162380 2024-04-25T08:44:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.999+0000 7f10f6a00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f10dc066f60 0x7f10dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.999+0000 7f10f6a00640 1 -- 172.21.15.46:0/734807209 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f10e80844a0 con 0x7f1100162380 2024-04-25T08:44:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.999+0000 7f10fde00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f10dc066f60 0x7f10dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:12.999+0000 7f10fde00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f10dc066f60 0x7f10dc069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f10f00ad000 tx=0x7f10f00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.003+0000 7f10f6a00640 1 -- 172.21.15.46:0/734807209 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10e8054d80 con 0x7f1100162380 2024-04-25T08:44:13.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.167+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f110006b810 con 0x7f1100162380 2024-04-25T08:44:13.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.806+0000 7f10f6a00640 1 -- 172.21.15.46:0/734807209 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v51) v1 ==== 170+0+0 (secure 0 0 0) 0x7f10e8058e20 con 0x7f1100162380 2024-04-25T08:44:13.799 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-25T08:44:13.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f10dc066f60 msgr2=0x7f10dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f10dc066f60 0x7f10dc069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f10f00ad000 tx=0x7f10f00a6000 comp rx=0 tx=0).stop 2024-04-25T08:44:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100162380 msgr2=0x7f11000b58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100162380 0x7f11000b58d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f10e800c3f0 tx=0x7f10e800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:44:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 shutdown_connections 2024-04-25T08:44:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f10dc066f60 0x7f10dc069420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11001605e0 0x7f1100158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1100162380 0x7f11000b58d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 --2- 172.21.15.46:0/734807209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110006b230 0x7f11000b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:13.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 >> 172.21.15.46:0/734807209 conn(0x7f1100099a00 msgr2=0x7f1100161c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:13.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 shutdown_connections 2024-04-25T08:44:13.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:13.810+0000 7f1104c00640 1 -- 172.21.15.46:0/734807209 wait complete. 2024-04-25T08:44:13.847 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:44:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f015da00640 1 Processor -- start 2024-04-25T08:44:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f015da00640 1 -- start start 2024-04-25T08:44:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f015da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015806d870 0x7f01580cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f015da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01580cc620 0x7f01580b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f015da00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01580ce5f0 con 0x7f015815b870 2024-04-25T08:44:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f015da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01580ccab0 con 0x7f01580cc620 2024-04-25T08:44:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f015da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01580ce8d0 con 0x7f015806d870 2024-04-25T08:44:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f0157400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01580cc620 0x7f01580b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f0157400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01580cc620 0x7f01580b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48118/0 (socket says 172.21.15.46:48118) 2024-04-25T08:44:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f0157400640 1 -- 172.21.15.46:0/938859447 learned_addr learned my addr 172.21.15.46:0/938859447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f014fe00640 1 --2- 172.21.15.46:0/938859447 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015806d870 0x7f01580cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:13.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f0157400640 1 -- 172.21.15.46:0/938859447 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015806d870 msgr2=0x7f01580cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:13.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f0157400640 1 --2- 172.21.15.46:0/938859447 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015806d870 0x7f01580cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:13.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f0157400640 1 -- 172.21.15.46:0/938859447 >> v1:172.21.15.82:6789/0 conn(0x7f015815b870 legacy=0x7f01580cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:13.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.002+0000 7f0157400640 1 -- 172.21.15.46:0/938859447 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01580cebb0 con 0x7f01580cc620 2024-04-25T08:44:13.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f0157400640 1 --2- 172.21.15.46:0/938859447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 0x7f01580b2e20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0150009e50 tx=0x7f015001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ea524a29d90410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f0156000640 1 -- 172.21.15.46:0/938859447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500a5660 con 0x7f01580cc620 2024-04-25T08:44:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f0156000640 1 -- 172.21.15.46:0/938859447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01500a5800 con 0x7f01580cc620 2024-04-25T08:44:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f0156000640 1 -- 172.21.15.46:0/938859447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500a5bd0 con 0x7f01580cc620 2024-04-25T08:44:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 -- 172.21.15.46:0/938859447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 msgr2=0x7f01580b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 --2- 172.21.15.46:0/938859447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 0x7f01580b2e20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0150009e50 tx=0x7f015001c9a0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 -- 172.21.15.46:0/938859447 shutdown_connections 2024-04-25T08:44:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 --2- 172.21.15.46:0/938859447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 0x7f01580b2e20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 --2- 172.21.15.46:0/938859447 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015806d870 0x7f01580cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 -- 172.21.15.46:0/938859447 >> 172.21.15.46:0/938859447 conn(0x7f015806ac10 msgr2=0x7f015806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 -- 172.21.15.46:0/938859447 shutdown_connections 2024-04-25T08:44:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 -- 172.21.15.46:0/938859447 wait complete. 2024-04-25T08:44:14.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 Processor -- start 2024-04-25T08:44:14.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.006+0000 7f015da00640 1 -- start start 2024-04-25T08:44:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f015da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f015806d870 0x7f0158144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f015da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 0x7f015814ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f015da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015815b870 0x7f0158160280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f015da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01580b3cf0 con 0x7f01580cc620 2024-04-25T08:44:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f015da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01580b3b70 con 0x7f015806d870 2024-04-25T08:44:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f015da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015816a6d0 con 0x7f015815b870 2024-04-25T08:44:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 0x7f015814ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 0x7f015814ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48130/0 (socket says 172.21.15.46:48130) 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014fe00640 1 -- 172.21.15.46:0/358922356 learned_addr learned my addr 172.21.15.46:0/358922356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f0156a00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f015806d870 0x7f0158144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f0157400640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015815b870 0x7f0158160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014fe00640 1 -- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015815b870 msgr2=0x7f0158160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014fe00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015815b870 0x7f0158160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014fe00640 1 -- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f015806d870 msgr2=0x7f0158144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014fe00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f015806d870 0x7f0158144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014fe00640 1 -- 172.21.15.46:0/358922356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01580b15c0 con 0x7f01580cc620 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014fe00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 0x7f015814ec30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f014000d800 tx=0x7f014000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014ea00640 1 -- 172.21.15.46:0/358922356 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014000b6c0 con 0x7f01580cc620 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014ea00640 1 -- 172.21.15.46:0/358922356 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0140019070 con 0x7f01580cc620 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f014ea00640 1 -- 172.21.15.46:0/358922356 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014000ba60 con 0x7f01580cc620 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01580b18a0 con 0x7f01580cc620 2024-04-25T08:44:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0158154500 con 0x7f01580cc620 2024-04-25T08:44:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.010+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0158069010 con 0x7f01580cc620 2024-04-25T08:44:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.014+0000 7f014ea00640 1 -- 172.21.15.46:0/358922356 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f014000bc00 con 0x7f01580cc620 2024-04-25T08:44:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.014+0000 7f014ea00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0134066f60 0x7f0134069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.014+0000 7f014ea00640 1 -- 172.21.15.46:0/358922356 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f0140085620 con 0x7f01580cc620 2024-04-25T08:44:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.014+0000 7f0156a00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0134066f60 0x7f0134069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.014+0000 7f0156a00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0134066f60 0x7f0134069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0148001770 tx=0x7f0148009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:14.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.018+0000 7f014ea00640 1 -- 172.21.15.46:0/358922356 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0140055d80 con 0x7f01580cc620 2024-04-25T08:44:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.214+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f01580b1a70 con 0x7f01580cc620 2024-04-25T08:44:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.842+0000 7f014ea00640 1 -- 172.21.15.46:0/358922356 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0140059e20 con 0x7f01580cc620 2024-04-25T08:44:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T08:44:14.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0134066f60 msgr2=0x7f0134069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:14.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0134066f60 0x7f0134069420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0148001770 tx=0x7f0148009040 comp rx=0 tx=0).stop 2024-04-25T08:44:14.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 msgr2=0x7f015814ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:14.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 0x7f015814ec30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f014000d800 tx=0x7f014000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 shutdown_connections 2024-04-25T08:44:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0134066f60 0x7f0134069420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f015815b870 0x7f0158160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580cc620 0x7f015814ec30 secure :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f014000d800 tx=0x7f014000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 --2- 172.21.15.46:0/358922356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f015806d870 0x7f0158144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.850+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 >> 172.21.15.46:0/358922356 conn(0x7f015806ac10 msgr2=0x7f015815a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:14.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.854+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 shutdown_connections 2024-04-25T08:44:14.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:14.854+0000 7f015da00640 1 -- 172.21.15.46:0/358922356 wait complete. 2024-04-25T08:44:14.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:15.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f502ea00640 1 Processor -- start 2024-04-25T08:44:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f502ea00640 1 -- start start 2024-04-25T08:44:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f502ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502815ef20 0x7f50280cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f502ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f50280cd6d0 0x7f502809a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f502ea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502815f300 con 0x7f502809ab50 2024-04-25T08:44:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f502ea00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50280cddb0 con 0x7f50280cd6d0 2024-04-25T08:44:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f502ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50280cc3d0 con 0x7f502815ef20 2024-04-25T08:44:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f502ca00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f502809ab50 0x7f5028099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39756/0 (socket says 172.21.15.46:39756) 2024-04-25T08:44:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f502ca00640 1 -- 172.21.15.46:0/116865974 learned_addr learned my addr 172.21.15.46:0/116865974 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f501f400640 1 --2- 172.21.15.46:0/116865974 >> v2:172.21.15.82:3300/0 conn(0x7f50280cd6d0 0x7f502809a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f5027e00640 1 --2- 172.21.15.46:0/116865974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502815ef20 0x7f50280cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f5027400640 1 -- 172.21.15.46:0/116865974 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265362134 0 0) 0x7f502815f300 con 0x7f502809ab50 2024-04-25T08:44:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f5027400640 1 -- 172.21.15.46:0/116865974 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f500c003530 con 0x7f502809ab50 2024-04-25T08:44:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f501f400640 1 -- 172.21.15.46:0/116865974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502815ef20 msgr2=0x7f50280cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:15.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f501f400640 1 --2- 172.21.15.46:0/116865974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502815ef20 0x7f50280cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f501f400640 1 -- 172.21.15.46:0/116865974 >> v1:172.21.15.46:6789/0 conn(0x7f502809ab50 legacy=0x7f5028099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f501f400640 1 -- 172.21.15.46:0/116865974 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280cc6b0 con 0x7f50280cd6d0 2024-04-25T08:44:15.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.030+0000 7f5027e00640 1 --2- 172.21.15.46:0/116865974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502815ef20 0x7f50280cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f501f400640 1 --2- 172.21.15.46:0/116865974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50280cd6d0 0x7f502809a610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f50100097c0 tx=0x7f501001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f80c070ddb2cbd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f5027400640 1 -- 172.21.15.46:0/116865974 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50100a7070 con 0x7f50280cd6d0 2024-04-25T08:44:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f5027400640 1 -- 172.21.15.46:0/116865974 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50100b0070 con 0x7f50280cd6d0 2024-04-25T08:44:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f5027400640 1 -- 172.21.15.46:0/116865974 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50100ab890 con 0x7f50280cd6d0 2024-04-25T08:44:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f502ea00640 1 -- 172.21.15.46:0/116865974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50280cd6d0 msgr2=0x7f502809a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f502ea00640 1 --2- 172.21.15.46:0/116865974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50280cd6d0 0x7f502809a610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f50100097c0 tx=0x7f501001c470 comp rx=0 tx=0).stop 2024-04-25T08:44:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f502ea00640 1 -- 172.21.15.46:0/116865974 shutdown_connections 2024-04-25T08:44:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f502ea00640 1 --2- 172.21.15.46:0/116865974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50280cd6d0 0x7f502809a610 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f502ea00640 1 --2- 172.21.15.46:0/116865974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502815ef20 0x7f50280cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f502ea00640 1 -- 172.21.15.46:0/116865974 >> 172.21.15.46:0/116865974 conn(0x7f50280b3120 msgr2=0x7f50280b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f502ea00640 1 -- 172.21.15.46:0/116865974 shutdown_connections 2024-04-25T08:44:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.034+0000 7f502ea00640 1 -- 172.21.15.46:0/116865974 wait complete. 2024-04-25T08:44:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 Processor -- start 2024-04-25T08:44:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 -- start start 2024-04-25T08:44:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502809ab50 0x7f5028155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f502815ef20 0x7f5028152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50280b4ee0 0x7f502815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280c5b10 con 0x7f50280b4ee0 2024-04-25T08:44:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280c5990 con 0x7f502815ef20 2024-04-25T08:44:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502809a110 con 0x7f502809ab50 2024-04-25T08:44:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50280b4ee0 0x7f502815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50280b4ee0 0x7f502815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48146/0 (socket says 172.21.15.46:48146) 2024-04-25T08:44:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ca00640 1 -- 172.21.15.46:0/820023419 learned_addr learned my addr 172.21.15.46:0/820023419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f501f400640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f502815ef20 0x7f5028152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f5027e00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502809ab50 0x7f5028155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ca00640 1 -- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502809ab50 msgr2=0x7f5028155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ca00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502809ab50 0x7f5028155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ca00640 1 -- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f502815ef20 msgr2=0x7f5028152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ca00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f502815ef20 0x7f5028152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ca00640 1 -- 172.21.15.46:0/820023419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50281543a0 con 0x7f50280b4ee0 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ca00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50280b4ee0 0x7f502815f780 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f502000cb40 tx=0x7f502000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f5024c00640 1 -- 172.21.15.46:0/820023419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020017070 con 0x7f50280b4ee0 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f5024c00640 1 -- 172.21.15.46:0/820023419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502000aca0 con 0x7f50280b4ee0 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f5024c00640 1 -- 172.21.15.46:0/820023419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020012620 con 0x7f50280b4ee0 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50280b53c0 con 0x7f50280b4ee0 2024-04-25T08:44:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.038+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50280b5980 con 0x7f50280b4ee0 2024-04-25T08:44:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.042+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5028069080 con 0x7f50280b4ee0 2024-04-25T08:44:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.042+0000 7f5024c00640 1 -- 172.21.15.46:0/820023419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f502001a070 con 0x7f50280b4ee0 2024-04-25T08:44:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.042+0000 7f5024c00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5004066f60 0x7f5004069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.042+0000 7f5024c00640 1 -- 172.21.15.46:0/820023419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5020084fc0 con 0x7f50280b4ee0 2024-04-25T08:44:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.042+0000 7f5027e00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5004066f60 0x7f5004069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.042+0000 7f5027e00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5004066f60 0x7f5004069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5018009580 tx=0x7f5018009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.046+0000 7f5024c00640 1 -- 172.21.15.46:0/820023419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5020055750 con 0x7f50280b4ee0 2024-04-25T08:44:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.210+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50280d0700 con 0x7f50280b4ee0 2024-04-25T08:44:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.210+0000 7f5024c00640 1 -- 172.21.15.46:0/820023419 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16907 (secure 0 0 0) 0x7f50200597f0 con 0x7f50280b4ee0 2024-04-25T08:44:15.204 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:15.204 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":52,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:14.217718+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:44:12.022357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:44:13.215069+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5004066f60 msgr2=0x7f5004069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5004066f60 0x7f5004069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5018009580 tx=0x7f5018009190 comp rx=0 tx=0).stop 2024-04-25T08:44:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50280b4ee0 msgr2=0x7f502815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50280b4ee0 0x7f502815f780 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f502000cb40 tx=0x7f502000a060 comp rx=0 tx=0).stop 2024-04-25T08:44:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 shutdown_connections 2024-04-25T08:44:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5004066f60 0x7f5004069420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50280b4ee0 0x7f502815f780 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f502815ef20 0x7f5028152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 --2- 172.21.15.46:0/820023419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f502809ab50 0x7f5028155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 >> 172.21.15.46:0/820023419 conn(0x7f50280b3120 msgr2=0x7f50280cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 shutdown_connections 2024-04-25T08:44:15.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.214+0000 7f502ea00640 1 -- 172.21.15.46:0/820023419 wait complete. 2024-04-25T08:44:15.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d1da00640 1 Processor -- start 2024-04-25T08:44:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d1da00640 1 -- start start 2024-04-25T08:44:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d1da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d1815dba0 0x7f2d1806fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d1da00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2d180d0000 0x7f2d18161610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d1da00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d180700d0 con 0x7f2d180b2ce0 2024-04-25T08:44:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d1da00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d1815df80 con 0x7f2d180d0000 2024-04-25T08:44:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d1da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d180cd1a0 con 0x7f2d1815dba0 2024-04-25T08:44:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d0e000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2d180b2ce0 0x7f2d180cf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39770/0 (socket says 172.21.15.46:39770) 2024-04-25T08:44:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d0e000640 1 -- 172.21.15.46:0/1355298054 learned_addr learned my addr 172.21.15.46:0/1355298054 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d16a00640 1 --2- 172.21.15.46:0/1355298054 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d1815dba0 0x7f2d1806fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d17400640 1 --2- 172.21.15.46:0/1355298054 >> v2:172.21.15.82:3300/0 conn(0x7f2d180d0000 0x7f2d18161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d16000640 1 -- 172.21.15.46:0/1355298054 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3776568561 0 0) 0x7f2d180700d0 con 0x7f2d180b2ce0 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d16000640 1 -- 172.21.15.46:0/1355298054 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cfc003530 con 0x7f2d180b2ce0 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d16000640 1 -- 172.21.15.46:0/1355298054 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 506343205 0 0) 0x7f2cfc003530 con 0x7f2d180b2ce0 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d16a00640 1 -- 172.21.15.46:0/1355298054 >> v2:172.21.15.82:3300/0 conn(0x7f2d180d0000 msgr2=0x7f2d18161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d16a00640 1 --2- 172.21.15.46:0/1355298054 >> v2:172.21.15.82:3300/0 conn(0x7f2d180d0000 0x7f2d18161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d16a00640 1 -- 172.21.15.46:0/1355298054 >> v1:172.21.15.46:6789/0 conn(0x7f2d180b2ce0 legacy=0x7f2d180cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d16a00640 1 -- 172.21.15.46:0/1355298054 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d180cd480 con 0x7f2d1815dba0 2024-04-25T08:44:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.374+0000 7f2d17400640 1 --2- 172.21.15.46:0/1355298054 >> v2:172.21.15.82:3300/0 conn(0x7f2d180d0000 0x7f2d18161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:15.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d16a00640 1 --2- 172.21.15.46:0/1355298054 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d1815dba0 0x7f2d1806fad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d080097c0 tx=0x7f2d0801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b3276184f062123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d16000640 1 -- 172.21.15.46:0/1355298054 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d080b0070 con 0x7f2d1815dba0 2024-04-25T08:44:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d16000640 1 -- 172.21.15.46:0/1355298054 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d080ab400 con 0x7f2d1815dba0 2024-04-25T08:44:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d16000640 1 -- 172.21.15.46:0/1355298054 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00001680 con 0x7f2d1815dba0 2024-04-25T08:44:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d1da00640 1 -- 172.21.15.46:0/1355298054 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d1815dba0 msgr2=0x7f2d1806fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d1da00640 1 --2- 172.21.15.46:0/1355298054 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d1815dba0 0x7f2d1806fad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d080097c0 tx=0x7f2d0801c330 comp rx=0 tx=0).stop 2024-04-25T08:44:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d1da00640 1 -- 172.21.15.46:0/1355298054 shutdown_connections 2024-04-25T08:44:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d1da00640 1 --2- 172.21.15.46:0/1355298054 >> v2:172.21.15.82:3300/0 conn(0x7f2d180d0000 0x7f2d18161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d1da00640 1 --2- 172.21.15.46:0/1355298054 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d1815dba0 0x7f2d1806fad0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d1da00640 1 -- 172.21.15.46:0/1355298054 >> 172.21.15.46:0/1355298054 conn(0x7f2d18099a00 msgr2=0x7f2d180b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d1da00640 1 -- 172.21.15.46:0/1355298054 shutdown_connections 2024-04-25T08:44:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.378+0000 7f2d1da00640 1 -- 172.21.15.46:0/1355298054 wait complete. 2024-04-25T08:44:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 Processor -- start 2024-04-25T08:44:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 -- start start 2024-04-25T08:44:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180b2ce0 0x7f2d1814cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d180d0000 0x7f2d180c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1815dba0 0x7f2d1815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1816aad0 con 0x7f2d1815dba0 2024-04-25T08:44:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1816a950 con 0x7f2d180b2ce0 2024-04-25T08:44:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1816ac50 con 0x7f2d180d0000 2024-04-25T08:44:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1815dba0 0x7f2d1815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1815dba0 0x7f2d1815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48154/0 (socket says 172.21.15.46:48154) 2024-04-25T08:44:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d17400640 1 -- 172.21.15.46:0/492503072 learned_addr learned my addr 172.21.15.46:0/492503072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d0e000640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d180d0000 0x7f2d180c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d16a00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180b2ce0 0x7f2d1814cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d17400640 1 -- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d180d0000 msgr2=0x7f2d180c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d17400640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d180d0000 0x7f2d180c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d17400640 1 -- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180b2ce0 msgr2=0x7f2d1814cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d17400640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180b2ce0 0x7f2d1814cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d17400640 1 -- 172.21.15.46:0/492503072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d180c5340 con 0x7f2d1815dba0 2024-04-25T08:44:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d16a00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180b2ce0 0x7f2d1814cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d0e000640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d180d0000 0x7f2d180c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d17400640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1815dba0 0x7f2d1815bee0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009f00 tx=0x7f2d1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d0f400640 1 -- 172.21.15.46:0/492503072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10018030 con 0x7f2d1815dba0 2024-04-25T08:44:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d0f400640 1 -- 172.21.15.46:0/492503072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1000ce50 con 0x7f2d1815dba0 2024-04-25T08:44:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d0f400640 1 -- 172.21.15.46:0/492503072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d10003bc0 con 0x7f2d1815dba0 2024-04-25T08:44:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d180c5620 con 0x7f2d1815dba0 2024-04-25T08:44:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.382+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d180c5be0 con 0x7f2d1815dba0 2024-04-25T08:44:15.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.386+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d18069080 con 0x7f2d1815dba0 2024-04-25T08:44:15.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.386+0000 7f2d0f400640 1 -- 172.21.15.46:0/492503072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d1001b070 con 0x7f2d1815dba0 2024-04-25T08:44:15.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.386+0000 7f2d0f400640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2cf4066f60 0x7f2cf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.386+0000 7f2d0f400640 1 -- 172.21.15.46:0/492503072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d10085c70 con 0x7f2d1815dba0 2024-04-25T08:44:15.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.386+0000 7f2d16a00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2cf4066f60 0x7f2cf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.386+0000 7f2d16a00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2cf4066f60 0x7f2cf4069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2d080ad000 tx=0x7f2d080a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:15.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.390+0000 7f2d0f400640 1 -- 172.21.15.46:0/492503072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d10010040 con 0x7f2d1815dba0 2024-04-25T08:44:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.554+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d18168720 con 0x7f2d1815dba0 2024-04-25T08:44:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.554+0000 7f2d0f400640 1 -- 172.21.15.46:0/492503072 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16907 (secure 0 0 0) 0x7f2d10056ae0 con 0x7f2d1815dba0 2024-04-25T08:44:15.549 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:15.549 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":52,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:14.217718+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:44:12.022357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:44:13.215069+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2cf4066f60 msgr2=0x7f2cf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2cf4066f60 0x7f2cf4069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2d080ad000 tx=0x7f2d080a6000 comp rx=0 tx=0).stop 2024-04-25T08:44:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1815dba0 msgr2=0x7f2d1815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1815dba0 0x7f2d1815bee0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2d10009f00 tx=0x7f2d1000c490 comp rx=0 tx=0).stop 2024-04-25T08:44:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 shutdown_connections 2024-04-25T08:44:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2cf4066f60 0x7f2cf4069420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d1815dba0 0x7f2d1815bee0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d180d0000 0x7f2d180c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 --2- 172.21.15.46:0/492503072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180b2ce0 0x7f2d1814cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 >> 172.21.15.46:0/492503072 conn(0x7f2d18099a00 msgr2=0x7f2d180adcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 shutdown_connections 2024-04-25T08:44:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.558+0000 7f2d1da00640 1 -- 172.21.15.46:0/492503072 wait complete. 2024-04-25T08:44:15.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:44:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ac6c00640 1 Processor -- start 2024-04-25T08:44:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ac6c00640 1 -- start start 2024-04-25T08:44:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ac6c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ac0143990 0x7f5ac00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ac6c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ac01440d0 0x7f5ac006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ac6c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac015e440 con 0x7f5ac006ef80 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ac6c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac015ed30 con 0x7f5ac01440d0 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ac6c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac00b3730 con 0x7f5ac0143990 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ab7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ac01440d0 0x7f5ac006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ab7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5ac01440d0 0x7f5ac006ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48166/0 (socket says 172.21.15.46:48166) 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ab7e00640 1 -- 172.21.15.46:0/1413574775 learned_addr learned my addr 172.21.15.46:0/1413574775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5abfe00640 1 --2- 172.21.15.46:0/1413574775 >> v2:172.21.15.82:3300/0 conn(0x7f5ac0143990 0x7f5ac00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ab7e00640 1 -- 172.21.15.46:0/1413574775 >> v2:172.21.15.82:3300/0 conn(0x7f5ac0143990 msgr2=0x7f5ac00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ab7e00640 1 --2- 172.21.15.46:0/1413574775 >> v2:172.21.15.82:3300/0 conn(0x7f5ac0143990 0x7f5ac00cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ab7e00640 1 -- 172.21.15.46:0/1413574775 >> v1:172.21.15.82:6789/0 conn(0x7f5ac006ef80 legacy=0x7f5ac00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5ab7e00640 1 -- 172.21.15.46:0/1413574775 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac015e6e0 con 0x7f5ac01440d0 2024-04-25T08:44:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.722+0000 7f5abfe00640 1 --2- 172.21.15.46:0/1413574775 >> v2:172.21.15.82:3300/0 conn(0x7f5ac0143990 0x7f5ac00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5ab7e00640 1 --2- 172.21.15.46:0/1413574775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac01440d0 0x7f5ac006ea40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5ac006b9d0 tx=0x7f5aa801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda852f835209930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5abf400640 1 -- 172.21.15.46:0/1413574775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80b0070 con 0x7f5ac01440d0 2024-04-25T08:44:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5abf400640 1 -- 172.21.15.46:0/1413574775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa8004230 con 0x7f5ac01440d0 2024-04-25T08:44:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5abf400640 1 -- 172.21.15.46:0/1413574775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8003620 con 0x7f5ac01440d0 2024-04-25T08:44:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1413574775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac01440d0 msgr2=0x7f5ac006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5ac6c00640 1 --2- 172.21.15.46:0/1413574775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac01440d0 0x7f5ac006ea40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5ac006b9d0 tx=0x7f5aa801c520 comp rx=0 tx=0).stop 2024-04-25T08:44:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1413574775 shutdown_connections 2024-04-25T08:44:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5ac6c00640 1 --2- 172.21.15.46:0/1413574775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac01440d0 0x7f5ac006ea40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5ac6c00640 1 --2- 172.21.15.46:0/1413574775 >> v2:172.21.15.82:3300/0 conn(0x7f5ac0143990 0x7f5ac00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1413574775 >> 172.21.15.46:0/1413574775 conn(0x7f5ac006ac10 msgr2=0x7f5ac006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1413574775 shutdown_connections 2024-04-25T08:44:15.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.726+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1413574775 wait complete. 2024-04-25T08:44:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 Processor -- start 2024-04-25T08:44:15.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 -- start start 2024-04-25T08:44:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac006ef80 0x7f5ac0164280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ac0143990 0x7f5ac0159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ac01440d0 0x7f5ac01f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac0143d70 con 0x7f5ac006ef80 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac006f390 con 0x7f5ac01440d0 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac0143ef0 con 0x7f5ac0143990 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac006ef80 0x7f5ac0164280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac006ef80 0x7f5ac0164280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48180/0 (socket says 172.21.15.46:48180) 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abfe00640 1 -- 172.21.15.46:0/1572327604 learned_addr learned my addr 172.21.15.46:0/1572327604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac4c00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ac01440d0 0x7f5ac01f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ab7e00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ac0143990 0x7f5ac0159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abfe00640 1 -- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ac0143990 msgr2=0x7f5ac0159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abfe00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ac0143990 0x7f5ac0159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abfe00640 1 -- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ac01440d0 msgr2=0x7f5ac01f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abfe00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ac01440d0 0x7f5ac01f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abfe00640 1 -- 172.21.15.46:0/1572327604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac015c700 con 0x7f5ac006ef80 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ab7e00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ac0143990 0x7f5ac0159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abfe00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac006ef80 0x7f5ac0164280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000e840 tx=0x7f5ab000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abcc00640 1 -- 172.21.15.46:0/1572327604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0017070 con 0x7f5ac006ef80 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abcc00640 1 -- 172.21.15.46:0/1572327604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab000c8d0 con 0x7f5ac006ef80 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5abcc00640 1 -- 172.21.15.46:0/1572327604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000f040 con 0x7f5ac006ef80 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac0139340 con 0x7f5ac006ef80 2024-04-25T08:44:15.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.730+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac015c9a0 con 0x7f5ac006ef80 2024-04-25T08:44:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.734+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac0069010 con 0x7f5ac006ef80 2024-04-25T08:44:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.734+0000 7f5abcc00640 1 -- 172.21.15.46:0/1572327604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ab001a070 con 0x7f5ac006ef80 2024-04-25T08:44:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.734+0000 7f5abcc00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a9c066f60 0x7f5a9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.734+0000 7f5abcc00640 1 -- 172.21.15.46:0/1572327604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ab0084e20 con 0x7f5ac006ef80 2024-04-25T08:44:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.734+0000 7f5ab7e00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a9c066f60 0x7f5a9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.734+0000 7f5ab7e00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a9c066f60 0x7f5a9c069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80ad000 tx=0x7f5aa80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:15.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.738+0000 7f5abcc00640 1 -- 172.21.15.46:0/1572327604 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ab0055c00 con 0x7f5ac006ef80 2024-04-25T08:44:15.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:15.938+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5ac006da40 con 0x7f5ac006ef80 2024-04-25T08:44:16.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.850+0000 7f5abcc00640 1 -- 172.21.15.46:0/1572327604 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5ab0055500 con 0x7f5ac006ef80 2024-04-25T08:44:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a9c066f60 msgr2=0x7f5a9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a9c066f60 0x7f5a9c069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80ad000 tx=0x7f5aa80a6000 comp rx=0 tx=0).stop 2024-04-25T08:44:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac006ef80 msgr2=0x7f5ac0164280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac006ef80 0x7f5ac0164280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000e840 tx=0x7f5ab000ed00 comp rx=0 tx=0).stop 2024-04-25T08:44:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 shutdown_connections 2024-04-25T08:44:16.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a9c066f60 0x7f5a9c069420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ac01440d0 0x7f5ac01f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ac0143990 0x7f5ac0159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 --2- 172.21.15.46:0/1572327604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac006ef80 0x7f5ac0164280 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 >> 172.21.15.46:0/1572327604 conn(0x7f5ac006ac10 msgr2=0x7f5ac006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 shutdown_connections 2024-04-25T08:44:16.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:16.854+0000 7f5ac6c00640 1 -- 172.21.15.46:0/1572327604 wait complete. 2024-04-25T08:44:16.870 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:44:16.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:44:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a2b800640 1 Processor -- start 2024-04-25T08:44:17.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a2b800640 1 -- start start 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a2b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3a2415dae0 0x7f3a24072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a2b800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a2406e620 0x7f3a2406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a2b800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a240b3850 con 0x7f3a2406ea60 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a2b800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2406e190 con 0x7f3a2406e620 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a2b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a240b3af0 con 0x7f3a2415dae0 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a1be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a2406e620 0x7f3a2406bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a1be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a2406e620 0x7f3a2406bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48188/0 (socket says 172.21.15.46:48188) 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a1be00640 1 -- 172.21.15.46:0/3355924048 learned_addr learned my addr 172.21.15.46:0/3355924048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a28e00640 1 --2- 172.21.15.46:0/3355924048 >> v2:172.21.15.82:3300/0 conn(0x7f3a2415dae0 0x7f3a24072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a1be00640 1 -- 172.21.15.46:0/3355924048 >> v2:172.21.15.82:3300/0 conn(0x7f3a2415dae0 msgr2=0x7f3a24072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a1be00640 1 --2- 172.21.15.46:0/3355924048 >> v2:172.21.15.82:3300/0 conn(0x7f3a2415dae0 0x7f3a24072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a1be00640 1 -- 172.21.15.46:0/3355924048 >> v1:172.21.15.82:6789/0 conn(0x7f3a2406ea60 legacy=0x7f3a240acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.026+0000 7f3a1be00640 1 -- 172.21.15.46:0/3355924048 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a240b3dd0 con 0x7f3a2406e620 2024-04-25T08:44:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a1be00640 1 --2- 172.21.15.46:0/3355924048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406e620 0x7f3a2406bc20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3a2406e310 tx=0x7f3a0c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ec898a8ec2469b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a23e00640 1 -- 172.21.15.46:0/3355924048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0ab600 con 0x7f3a2406e620 2024-04-25T08:44:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a23e00640 1 -- 172.21.15.46:0/3355924048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c0b0070 con 0x7f3a2406e620 2024-04-25T08:44:17.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a23e00640 1 -- 172.21.15.46:0/3355924048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0ab9c0 con 0x7f3a2406e620 2024-04-25T08:44:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a2b800640 1 -- 172.21.15.46:0/3355924048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406e620 msgr2=0x7f3a2406bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a2b800640 1 --2- 172.21.15.46:0/3355924048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406e620 0x7f3a2406bc20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3a2406e310 tx=0x7f3a0c01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a2b800640 1 -- 172.21.15.46:0/3355924048 shutdown_connections 2024-04-25T08:44:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a2b800640 1 --2- 172.21.15.46:0/3355924048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406e620 0x7f3a2406bc20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a2b800640 1 --2- 172.21.15.46:0/3355924048 >> v2:172.21.15.82:3300/0 conn(0x7f3a2415dae0 0x7f3a24072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a2b800640 1 -- 172.21.15.46:0/3355924048 >> 172.21.15.46:0/3355924048 conn(0x7f3a24071ae0 msgr2=0x7f3a24071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a2b800640 1 -- 172.21.15.46:0/3355924048 shutdown_connections 2024-04-25T08:44:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.030+0000 7f3a2b800640 1 -- 172.21.15.46:0/3355924048 wait complete. 2024-04-25T08:44:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 Processor -- start 2024-04-25T08:44:17.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 -- start start 2024-04-25T08:44:17.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2406e620 0x7f3a2414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406ea60 0x7f3a2415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a2415dae0 0x7f3a2415bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2415dec0 con 0x7f3a2406ea60 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2406c570 con 0x7f3a2406e620 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2406f6d0 con 0x7f3a2415dae0 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406ea60 0x7f3a2415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406ea60 0x7f3a2415eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48192/0 (socket says 172.21.15.46:48192) 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a1be00640 1 -- 172.21.15.46:0/338409853 learned_addr learned my addr 172.21.15.46:0/338409853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a1be00640 1 -- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a2415dae0 msgr2=0x7f3a2415bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a28e00640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2406e620 0x7f3a2414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a29800640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a2415dae0 0x7f3a2415bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a1be00640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a2415dae0 0x7f3a2415bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a1be00640 1 -- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2406e620 msgr2=0x7f3a2414d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:17.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a1be00640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2406e620 0x7f3a2414d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a1be00640 1 -- 172.21.15.46:0/338409853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2406f500 con 0x7f3a2406ea60 2024-04-25T08:44:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a28e00640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2406e620 0x7f3a2414d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a29800640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a2415dae0 0x7f3a2415bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a1be00640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406ea60 0x7f3a2415eee0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c019100 tx=0x7f3a0c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a21600640 1 -- 172.21.15.46:0/338409853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0be5f0 con 0x7f3a2406ea60 2024-04-25T08:44:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a21600640 1 -- 172.21.15.46:0/338409853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c0be790 con 0x7f3a2406ea60 2024-04-25T08:44:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a21600640 1 -- 172.21.15.46:0/338409853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c0beb40 con 0x7f3a2406ea60 2024-04-25T08:44:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2415f630 con 0x7f3a2406ea60 2024-04-25T08:44:17.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.034+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2414f1e0 con 0x7f3a2406ea60 2024-04-25T08:44:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.038+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a24069010 con 0x7f3a2406ea60 2024-04-25T08:44:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.038+0000 7f3a21600640 1 -- 172.21.15.46:0/338409853 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a0c0aa070 con 0x7f3a2406ea60 2024-04-25T08:44:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.038+0000 7f3a21600640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a00066f60 0x7f3a00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:17.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.038+0000 7f3a21600640 1 -- 172.21.15.46:0/338409853 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a0c11f0b0 con 0x7f3a2406ea60 2024-04-25T08:44:17.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.038+0000 7f3a28e00640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a00066f60 0x7f3a00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:17.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.038+0000 7f3a28e00640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a00066f60 0x7f3a00069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3a140097a0 tx=0x7f3a14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:17.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.042+0000 7f3a21600640 1 -- 172.21.15.46:0/338409853 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a0c016110 con 0x7f3a2406ea60 2024-04-25T08:44:17.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.242+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3a2415e660 con 0x7f3a2406ea60 2024-04-25T08:44:17.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.850+0000 7f3a21600640 1 -- 172.21.15.46:0/338409853 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3a0c0ef8d0 con 0x7f3a2406ea60 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a00066f60 msgr2=0x7f3a00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a00066f60 0x7f3a00069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3a140097a0 tx=0x7f3a14008040 comp rx=0 tx=0).stop 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406ea60 msgr2=0x7f3a2415eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406ea60 0x7f3a2415eee0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c019100 tx=0x7f3a0c01ad30 comp rx=0 tx=0).stop 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 shutdown_connections 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a00066f60 0x7f3a00069420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a2415dae0 0x7f3a2415bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a2406ea60 0x7f3a2415eee0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 --2- 172.21.15.46:0/338409853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a2406e620 0x7f3a2414d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 >> 172.21.15.46:0/338409853 conn(0x7f3a24071ae0 msgr2=0x7f3a24159bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 shutdown_connections 2024-04-25T08:44:17.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:17.854+0000 7f3a2b800640 1 -- 172.21.15.46:0/338409853 wait complete. 2024-04-25T08:44:17.870 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:44:17.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:18.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb47a00640 1 Processor -- start 2024-04-25T08:44:18.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb47a00640 1 -- start start 2024-04-25T08:44:18.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb47a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb4009a610 0x7ffb4009aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb47a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb40099a00 0x7ffb40099e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb47a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4015f300 con 0x7ffb4015ef20 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb47a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb400cd8a0 con 0x7ffb40099a00 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb47a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400cddb0 con 0x7ffb4009a610 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb45000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffb4015ef20 0x7ffb400cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39774/0 (socket says 172.21.15.46:39774) 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb45000640 1 -- 172.21.15.46:0/2489508693 learned_addr learned my addr 172.21.15.46:0/2489508693 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb45a00640 1 --2- 172.21.15.46:0/2489508693 >> v2:172.21.15.46:3300/0 conn(0x7ffb40099a00 0x7ffb40099e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb37e00640 1 --2- 172.21.15.46:0/2489508693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb4009a610 0x7ffb4009aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb3fe00640 1 -- 172.21.15.46:0/2489508693 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372959386 0 0) 0x7ffb4015f300 con 0x7ffb4015ef20 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb3fe00640 1 -- 172.21.15.46:0/2489508693 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb24003530 con 0x7ffb4015ef20 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb45a00640 1 -- 172.21.15.46:0/2489508693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb4009a610 msgr2=0x7ffb4009aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb45a00640 1 --2- 172.21.15.46:0/2489508693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb4009a610 0x7ffb4009aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb45a00640 1 -- 172.21.15.46:0/2489508693 >> v1:172.21.15.46:6789/0 conn(0x7ffb4015ef20 legacy=0x7ffb400cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.022+0000 7ffb45a00640 1 -- 172.21.15.46:0/2489508693 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400cc3d0 con 0x7ffb40099a00 2024-04-25T08:44:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb45a00640 1 --2- 172.21.15.46:0/2489508693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb40099a00 0x7ffb40099e90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009d90 tx=0x7ffb3801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173dfc09efe45262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb3fe00640 1 -- 172.21.15.46:0/2489508693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380b2070 con 0x7ffb40099a00 2024-04-25T08:44:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb3fe00640 1 -- 172.21.15.46:0/2489508693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb380a5830 con 0x7ffb40099a00 2024-04-25T08:44:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb3fe00640 1 -- 172.21.15.46:0/2489508693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380a5be0 con 0x7ffb40099a00 2024-04-25T08:44:18.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb47a00640 1 -- 172.21.15.46:0/2489508693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb40099a00 msgr2=0x7ffb40099e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb47a00640 1 --2- 172.21.15.46:0/2489508693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb40099a00 0x7ffb40099e90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009d90 tx=0x7ffb3801c970 comp rx=0 tx=0).stop 2024-04-25T08:44:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb47a00640 1 -- 172.21.15.46:0/2489508693 shutdown_connections 2024-04-25T08:44:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb47a00640 1 --2- 172.21.15.46:0/2489508693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb40099a00 0x7ffb40099e90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb47a00640 1 --2- 172.21.15.46:0/2489508693 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb4009a610 0x7ffb4009aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb47a00640 1 -- 172.21.15.46:0/2489508693 >> 172.21.15.46:0/2489508693 conn(0x7ffb400b3120 msgr2=0x7ffb400b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb47a00640 1 -- 172.21.15.46:0/2489508693 shutdown_connections 2024-04-25T08:44:18.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.026+0000 7ffb47a00640 1 -- 172.21.15.46:0/2489508693 wait complete. 2024-04-25T08:44:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 Processor -- start 2024-04-25T08:44:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 -- start start 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb40099a00 0x7ffb40150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4009a610 0x7ffb4015d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4015ef20 0x7ffb4015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400cc190 con 0x7ffb4015ef20 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400cc010 con 0x7ffb4009a610 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40168880 con 0x7ffb40099a00 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4015ef20 0x7ffb4015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4015ef20 0x7ffb4015d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48212/0 (socket says 172.21.15.46:48212) 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45a00640 1 -- 172.21.15.46:0/3849071973 learned_addr learned my addr 172.21.15.46:0/3849071973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45000640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb40099a00 0x7ffb40150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb37e00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4009a610 0x7ffb4015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45a00640 1 -- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb40099a00 msgr2=0x7ffb40150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45a00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb40099a00 0x7ffb40150ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45a00640 1 -- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4009a610 msgr2=0x7ffb4015d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45a00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4009a610 0x7ffb4015d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45a00640 1 -- 172.21.15.46:0/3849071973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40159be0 con 0x7ffb4015ef20 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb45a00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4015ef20 0x7ffb4015d6b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffb380b0040 tx=0x7ffb38019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb3d600640 1 -- 172.21.15.46:0/3849071973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380b2050 con 0x7ffb4015ef20 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb3d600640 1 -- 172.21.15.46:0/3849071973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb380be070 con 0x7ffb4015ef20 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb40159ec0 con 0x7ffb4015ef20 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb3d600640 1 -- 172.21.15.46:0/3849071973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380ad890 con 0x7ffb4015ef20 2024-04-25T08:44:18.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4015a480 con 0x7ffb4015ef20 2024-04-25T08:44:18.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.030+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40069080 con 0x7ffb4015ef20 2024-04-25T08:44:18.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.034+0000 7ffb3d600640 1 -- 172.21.15.46:0/3849071973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb380ada30 con 0x7ffb4015ef20 2024-04-25T08:44:18.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.034+0000 7ffb3d600640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffb1c066f60 0x7ffb1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.034+0000 7ffb3d600640 1 -- 172.21.15.46:0/3849071973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb381200c0 con 0x7ffb4015ef20 2024-04-25T08:44:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.034+0000 7ffb45000640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffb1c066f60 0x7ffb1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.034+0000 7ffb45000640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffb1c066f60 0x7ffb1c069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffb30007fd0 tx=0x7ffb30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:18.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.038+0000 7ffb3d600640 1 -- 172.21.15.46:0/3849071973 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb38016560 con 0x7ffb4015ef20 2024-04-25T08:44:18.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.234+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb400d0700 con 0x7ffb4015ef20 2024-04-25T08:44:18.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.234+0000 7ffb3d600640 1 -- 172.21.15.46:0/3849071973 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4935 (secure 0 0 0) 0x7ffb380f0850 con 0x7ffb4015ef20 2024-04-25T08:44:18.228 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:18.228 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/850914154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":850914154},{"type":"v1","addr":"172.21.15.46:6835","nonce":850914154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1115060038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1115060038},{"type":"v1","addr":"172.21.15.82:6836","nonce":1115060038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5620,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1383788710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1383788710},{"type":"v1","addr":"172.21.15.82:6837","nonce":1383788710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:44:14.218049+0000","modified":"2024-04-25T08:44:17.246143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/3843568539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3843568539},{"type":"v1","addr":"172.21.15.46:6834","nonce":3843568539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:44:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-04-25T08:44:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffb1c066f60 msgr2=0x7ffb1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffb1c066f60 0x7ffb1c069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffb30007fd0 tx=0x7ffb30009040 comp rx=0 tx=0).stop 2024-04-25T08:44:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4015ef20 msgr2=0x7ffb4015d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4015ef20 0x7ffb4015d6b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffb380b0040 tx=0x7ffb38019710 comp rx=0 tx=0).stop 2024-04-25T08:44:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 shutdown_connections 2024-04-25T08:44:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffb1c066f60 0x7ffb1c069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb4015ef20 0x7ffb4015d6b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4009a610 0x7ffb4015d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 --2- 172.21.15.46:0/3849071973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffb40099a00 0x7ffb40150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 >> 172.21.15.46:0/3849071973 conn(0x7ffb400b3120 msgr2=0x7ffb400c74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 shutdown_connections 2024-04-25T08:44:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.238+0000 7ffb47a00640 1 -- 172.21.15.46:0/3849071973 wait complete. 2024-04-25T08:44:18.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.394+0000 7f5edf600640 1 Processor -- start 2024-04-25T08:44:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.394+0000 7f5edf600640 1 -- start start 2024-04-25T08:44:18.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edf600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ed80aed10 0x7f5ed8165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edf600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed80b35b0 con 0x7f5ed815a750 2024-04-25T08:44:18.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edf600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed806d5e0 con 0x7f5ed806f320 2024-04-25T08:44:18.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edf600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed815ab30 con 0x7f5ed80aed10 2024-04-25T08:44:18.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edcc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5ed815a750 0x7f5ed806ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46286/0 (socket says 172.21.15.46:46286) 2024-04-25T08:44:18.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edcc00640 1 -- 172.21.15.46:0/3332381443 learned_addr learned my addr 172.21.15.46:0/3332381443 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:18.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edd600640 1 --2- 172.21.15.46:0/3332381443 >> v2:172.21.15.82:3300/0 conn(0x7f5ed80aed10 0x7f5ed8165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576467397 0 0) 0x7f5ed80b35b0 con 0x7f5ed815a750 2024-04-25T08:44:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ebc005740 con 0x7f5ed815a750 2024-04-25T08:44:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541881110 0 0) 0x7f5ed806d5e0 con 0x7f5ed806f320 2024-04-25T08:44:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed80b35b0 con 0x7f5ed806f320 2024-04-25T08:44:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 514441525 0 0) 0x7f5ebc005740 con 0x7f5ed815a750 2024-04-25T08:44:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 >> v2:172.21.15.82:3300/0 conn(0x7f5ed80aed10 msgr2=0x7f5ed8165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 --2- 172.21.15.46:0/3332381443 >> v2:172.21.15.82:3300/0 conn(0x7f5ed80aed10 0x7f5ed8165930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 >> v1:172.21.15.82:6789/0 conn(0x7f5ed806f320 legacy=0x7f5ed8162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed80ce720 con 0x7f5ed815a750 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f5ec8001680 con 0x7f5ed815a750 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 >> v1:172.21.15.46:6789/0 conn(0x7f5ed815a750 legacy=0x7f5ed806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ebc005db0 0x7f5ebc0061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ebc006a20 0x7f5ebc006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc0073e0 0x7f5ebc00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc00e2b0 con 0x7f5ebc005db0 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc00e430 con 0x7f5ebc0073e0 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ebc00e630 con 0x7f5ebc006a20 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edcc00640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ebc005db0 0x7f5ebc0061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5ecfe00640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ebc006a20 0x7f5ebc006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edd600640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc0073e0 0x7f5ebc00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edcc00640 1 -- 172.21.15.46:0/3332381443 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ebc006a20 msgr2=0x7f5ebc006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edcc00640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ebc006a20 0x7f5ebc006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edcc00640 1 -- 172.21.15.46:0/3332381443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc0073e0 msgr2=0x7f5ebc00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edcc00640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc0073e0 0x7f5ebc00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edd600640 1 -- 172.21.15.46:0/3332381443 reap_dead start 2024-04-25T08:44:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.398+0000 7f5edcc00640 1 -- 172.21.15.46:0/3332381443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ebc005740 con 0x7f5ebc005db0 2024-04-25T08:44:18.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5edcc00640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ebc005db0 0x7f5ebc0061b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800c110 tx=0x7f5ec801cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37d14d203d2d3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8007c10 con 0x7f5ebc005db0 2024-04-25T08:44:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec80a9070 con 0x7f5ebc005db0 2024-04-25T08:44:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5ed7e00640 1 -- 172.21.15.46:0/3332381443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8005d20 con 0x7f5ebc005db0 2024-04-25T08:44:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5edf600640 1 -- 172.21.15.46:0/3332381443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ebc005db0 msgr2=0x7f5ebc0061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5edf600640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ebc005db0 0x7f5ebc0061b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800c110 tx=0x7f5ec801cb60 comp rx=0 tx=0).stop 2024-04-25T08:44:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5edf600640 1 -- 172.21.15.46:0/3332381443 shutdown_connections 2024-04-25T08:44:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5edf600640 1 --2- 172.21.15.46:0/3332381443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ebc005db0 0x7f5ebc0061b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5edf600640 1 -- 172.21.15.46:0/3332381443 >> 172.21.15.46:0/3332381443 conn(0x7f5ed80b12a0 msgr2=0x7f5ed80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5edf600640 1 -- 172.21.15.46:0/3332381443 shutdown_connections 2024-04-25T08:44:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.402+0000 7f5edf600640 1 -- 172.21.15.46:0/3332381443 wait complete. 2024-04-25T08:44:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 Processor -- start 2024-04-25T08:44:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 -- start start 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc005db0 0x7f5ed814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ed814f7e0 0x7f5ed815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ed8150d40 0x7f5ed806db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80b2db0 con 0x7f5ed814f7e0 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed806d270 con 0x7f5ebc005db0 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80b2f30 con 0x7f5ed8150d40 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ed814f7e0 0x7f5ed815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ecfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ed814f7e0 0x7f5ed815bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36802/0 (socket says 172.21.15.46:36802) 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ecfe00640 1 -- 172.21.15.46:0/1247011372 learned_addr learned my addr 172.21.15.46:0/1247011372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edcc00640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc005db0 0x7f5ed814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edd600640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ed8150d40 0x7f5ed806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ecfe00640 1 -- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ed8150d40 msgr2=0x7f5ed806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ecfe00640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ed8150d40 0x7f5ed806db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ecfe00640 1 -- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc005db0 msgr2=0x7f5ed814f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ecfe00640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc005db0 0x7f5ed814f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ecfe00640 1 -- 172.21.15.46:0/1247011372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed814f5a0 con 0x7f5ed814f7e0 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edcc00640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc005db0 0x7f5ed814f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ecfe00640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ed814f7e0 0x7f5ed815bbc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000f7a0 tx=0x7f5ec000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ed5600640 1 -- 172.21.15.46:0/1247011372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec001a070 con 0x7f5ed814f7e0 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ed5600640 1 -- 172.21.15.46:0/1247011372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec000fe50 con 0x7f5ed814f7e0 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed814fca0 con 0x7f5ed814f7e0 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5ed5600640 1 -- 172.21.15.46:0/1247011372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0007030 con 0x7f5ed814f7e0 2024-04-25T08:44:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.406+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed8150260 con 0x7f5ed814f7e0 2024-04-25T08:44:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.410+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed8069080 con 0x7f5ed814f7e0 2024-04-25T08:44:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.410+0000 7f5ed5600640 1 -- 172.21.15.46:0/1247011372 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ec001d070 con 0x7f5ed814f7e0 2024-04-25T08:44:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.410+0000 7f5ed5600640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5eb4066f90 0x7f5eb4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.410+0000 7f5ed5600640 1 -- 172.21.15.46:0/1247011372 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ec00849e0 con 0x7f5ed814f7e0 2024-04-25T08:44:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.410+0000 7f5edcc00640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5eb4066f90 0x7f5eb4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.410+0000 7f5edcc00640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5eb4066f90 0x7f5eb4069450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80ab000 tx=0x7f5ec80b0040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:18.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.414+0000 7f5ed5600640 1 -- 172.21.15.46:0/1247011372 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ec00557c0 con 0x7f5ed814f7e0 2024-04-25T08:44:18.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.578+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ed80b0220 con 0x7f5ed814f7e0 2024-04-25T08:44:18.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.578+0000 7f5ed5600640 1 -- 172.21.15.46:0/1247011372 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f5ec00550c0 con 0x7f5ed814f7e0 2024-04-25T08:44:18.573 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:18.573 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:15.829540+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:44:12.022357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:44:13.215069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5eb4066f90 msgr2=0x7f5eb4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5eb4066f90 0x7f5eb4069450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80ab000 tx=0x7f5ec80b0040 comp rx=0 tx=0).stop 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ed814f7e0 msgr2=0x7f5ed815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ed814f7e0 0x7f5ed815bbc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000f7a0 tx=0x7f5ec000fc60 comp rx=0 tx=0).stop 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 shutdown_connections 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5eb4066f90 0x7f5eb4069450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5ed8150d40 0x7f5ed806db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ed814f7e0 0x7f5ed815bbc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 --2- 172.21.15.46:0/1247011372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ebc005db0 0x7f5ed814f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 >> 172.21.15.46:0/1247011372 conn(0x7f5ed80b12a0 msgr2=0x7f5ed81e8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 shutdown_connections 2024-04-25T08:44:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.582+0000 7f5edf600640 1 -- 172.21.15.46:0/1247011372 wait complete. 2024-04-25T08:44:18.594 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:44:18.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:44:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17eb200640 1 Processor -- start 2024-04-25T08:44:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17eb200640 1 -- start start 2024-04-25T08:44:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17eb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f17e40b2240 0x7f17e40b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17eb200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e406bdc0 con 0x7f17e40ccde0 2024-04-25T08:44:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17eb200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e406c0a0 con 0x7f17e4099a00 2024-04-25T08:44:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17eb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e406db90 con 0x7f17e40b2240 2024-04-25T08:44:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17e9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f17e40b2240 0x7f17e40b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17db400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f17e40ccde0 0x7f17e40cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46288/0 (socket says 172.21.15.46:46288) 2024-04-25T08:44:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17db400640 1 -- 172.21.15.46:0/1980406284 learned_addr learned my addr 172.21.15.46:0/1980406284 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17e9200640 1 -- 172.21.15.46:0/1980406284 >> v1:172.21.15.82:6789/0 conn(0x7f17e4099a00 legacy=0x7f17e4099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17e9200640 1 -- 172.21.15.46:0/1980406284 >> v1:172.21.15.46:6789/0 conn(0x7f17e40ccde0 legacy=0x7f17e40cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.742+0000 7f17e9200640 1 -- 172.21.15.46:0/1980406284 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e406c380 con 0x7f17e40b2240 2024-04-25T08:44:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17e9200640 1 --2- 172.21.15.46:0/1980406284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40b2240 0x7f17e40b2690 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009dc0 tx=0x7f17dc01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60e5f885b0cecd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17e3400640 1 -- 172.21.15.46:0/1980406284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0b2070 con 0x7f17e40b2240 2024-04-25T08:44:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17e3400640 1 -- 172.21.15.46:0/1980406284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc0a5780 con 0x7f17e40b2240 2024-04-25T08:44:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17e3400640 1 -- 172.21.15.46:0/1980406284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0a5b50 con 0x7f17e40b2240 2024-04-25T08:44:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17eb200640 1 -- 172.21.15.46:0/1980406284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40b2240 msgr2=0x7f17e40b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17eb200640 1 --2- 172.21.15.46:0/1980406284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40b2240 0x7f17e40b2690 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009dc0 tx=0x7f17dc01c8c0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17eb200640 1 -- 172.21.15.46:0/1980406284 shutdown_connections 2024-04-25T08:44:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17eb200640 1 --2- 172.21.15.46:0/1980406284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40b2240 0x7f17e40b2690 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17eb200640 1 -- 172.21.15.46:0/1980406284 >> 172.21.15.46:0/1980406284 conn(0x7f17e40b3120 msgr2=0x7f17e40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:18.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17eb200640 1 -- 172.21.15.46:0/1980406284 shutdown_connections 2024-04-25T08:44:18.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.746+0000 7f17eb200640 1 -- 172.21.15.46:0/1980406284 wait complete. 2024-04-25T08:44:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 Processor -- start 2024-04-25T08:44:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 -- start start 2024-04-25T08:44:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17e4099a00 0x7f17e414ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17e40b2240 0x7f17e414f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40ccde0 0x7f17e414fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e415de90 con 0x7f17e40ccde0 2024-04-25T08:44:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e415dd10 con 0x7f17e4099a00 2024-04-25T08:44:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e416be10 con 0x7f17e40b2240 2024-04-25T08:44:18.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40ccde0 0x7f17e414fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40ccde0 0x7f17e414fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36826/0 (socket says 172.21.15.46:36826) 2024-04-25T08:44:18.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e9200640 1 -- 172.21.15.46:0/3694560862 learned_addr learned my addr 172.21.15.46:0/3694560862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:18.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e3e00640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17e4099a00 0x7f17e414ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17db400640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17e40b2240 0x7f17e414f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e9200640 1 -- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17e40b2240 msgr2=0x7f17e414f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e9200640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17e40b2240 0x7f17e414f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e9200640 1 -- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17e4099a00 msgr2=0x7f17e414ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e9200640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17e4099a00 0x7f17e414ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e9200640 1 -- 172.21.15.46:0/3694560862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e414f280 con 0x7f17e40ccde0 2024-04-25T08:44:18.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e9200640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40ccde0 0x7f17e414fbc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f17dc01a4f0 tx=0x7f17dc019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:18.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e0c00640 1 -- 172.21.15.46:0/3694560862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0bd020 con 0x7f17e40ccde0 2024-04-25T08:44:18.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e0c00640 1 -- 172.21.15.46:0/3694560862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc0b8670 con 0x7f17e40ccde0 2024-04-25T08:44:18.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17e0c00640 1 -- 172.21.15.46:0/3694560862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0b2040 con 0x7f17e40ccde0 2024-04-25T08:44:18.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e41587e0 con 0x7f17e40ccde0 2024-04-25T08:44:18.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e4158da0 con 0x7f17e40ccde0 2024-04-25T08:44:18.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.750+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e4069080 con 0x7f17e40ccde0 2024-04-25T08:44:18.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.754+0000 7f17e0c00640 1 -- 172.21.15.46:0/3694560862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17dc0c0020 con 0x7f17e40ccde0 2024-04-25T08:44:18.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.754+0000 7f17e0c00640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17c0066f60 0x7f17c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:18.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.754+0000 7f17e0c00640 1 -- 172.21.15.46:0/3694560862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f17dc11ffa0 con 0x7f17e40ccde0 2024-04-25T08:44:18.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.754+0000 7f17e3e00640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17c0066f60 0x7f17c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:18.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.754+0000 7f17e3e00640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17c0066f60 0x7f17c0069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f17d4001770 tx=0x7f17d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:18.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.758+0000 7f17e0c00640 1 -- 172.21.15.46:0/3694560862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17dc0c7030 con 0x7f17e40ccde0 2024-04-25T08:44:18.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.918+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f17e4070940 con 0x7f17c0066f60 2024-04-25T08:44:18.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.926+0000 7f17e0c00640 1 -- 172.21.15.46:0/3694560862 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251957 (secure 0 0 0) 0x7f17e4070940 con 0x7f17c0066f60 2024-04-25T08:44:18.922 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:18.923 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:44:18.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17c0066f60 msgr2=0x7f17c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17c0066f60 0x7f17c0069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f17d4001770 tx=0x7f17d4009040 comp rx=0 tx=0).stop 2024-04-25T08:44:18.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40ccde0 msgr2=0x7f17e414fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:18.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40ccde0 0x7f17e414fbc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f17dc01a4f0 tx=0x7f17dc019660 comp rx=0 tx=0).stop 2024-04-25T08:44:18.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 shutdown_connections 2024-04-25T08:44:18.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17c0066f60 0x7f17c0069420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17e40ccde0 0x7f17e414fbc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17e40b2240 0x7f17e414f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 --2- 172.21.15.46:0/3694560862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17e4099a00 0x7f17e414ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:18.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 >> 172.21.15.46:0/3694560862 conn(0x7f17e40b3120 msgr2=0x7f17e4073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:18.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 shutdown_connections 2024-04-25T08:44:18.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:18.930+0000 7f17eb200640 1 -- 172.21.15.46:0/3694560862 wait complete. 2024-04-25T08:44:18.942 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-04-25T08:44:18.282320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238956,"kb_used_data":6140,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735764,"statfs":{"total":773094113280,"available":772849422336,"internally_reserved":0,"allocated":6287360,"data_stored":3430840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.477845"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834351+0000","last_change":"2024-04-25T08:44:14.845382+0000","last_active":"2024-04-25T08:44:15.834351+0000","last_peered":"2024-04-25T08:44:15.834351+0000","last_clean":"2024-04-25T08:44:15.834351+0000","last_became_active":"2024-04-25T08:44:14.845069+0000","last_became_peered":"2024-04-25T08:44:14.845069+0000","last_unstale":"2024-04-25T08:44:15.834351+0000","last_undegraded":"2024-04-25T08:44:15.834351+0000","last_fullsized":"2024-04-25T08:44:15.834351+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:35.493623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867585+0000","last_change":"2024-04-25T08:44:14.863335+0000","last_active":"2024-04-25T08:44:15.867585+0000","last_peered":"2024-04-25T08:44:15.867585+0000","last_clean":"2024-04-25T08:44:15.867585+0000","last_became_active":"2024-04-25T08:44:14.863088+0000","last_became_peered":"2024-04-25T08:44:14.863088+0000","last_unstale":"2024-04-25T08:44:15.867585+0000","last_undegraded":"2024-04-25T08:44:15.867585+0000","last_fullsized":"2024-04-25T08:44:15.867585+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:02.535112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865000+0000","last_change":"2024-04-25T08:44:14.877002+0000","last_active":"2024-04-25T08:44:15.865000+0000","last_peered":"2024-04-25T08:44:15.865000+0000","last_clean":"2024-04-25T08:44:15.865000+0000","last_became_active":"2024-04-25T08:44:14.871920+0000","last_became_peered":"2024-04-25T08:44:14.871920+0000","last_unstale":"2024-04-25T08:44:15.865000+0000","last_undegraded":"2024-04-25T08:44:15.865000+0000","last_fullsized":"2024-04-25T08:44:15.865000+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:29.576117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867260+0000","last_change":"2024-04-25T08:44:14.858692+0000","last_active":"2024-04-25T08:44:15.867260+0000","last_peered":"2024-04-25T08:44:15.867260+0000","last_clean":"2024-04-25T08:44:15.867260+0000","last_became_active":"2024-04-25T08:44:14.858365+0000","last_became_peered":"2024-04-25T08:44:14.858365+0000","last_unstale":"2024-04-25T08:44:15.867260+0000","last_undegraded":"2024-04-25T08:44:15.867260+0000","last_fullsized":"2024-04-25T08:44:15.867260+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:59.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317749+0000","last_change":"2024-04-25T08:44:14.856622+0000","last_active":"2024-04-25T08:44:16.317749+0000","last_peered":"2024-04-25T08:44:16.317749+0000","last_clean":"2024-04-25T08:44:16.317749+0000","last_became_active":"2024-04-25T08:44:14.856350+0000","last_became_peered":"2024-04-25T08:44:14.856350+0000","last_unstale":"2024-04-25T08:44:16.317749+0000","last_undegraded":"2024-04-25T08:44:16.317749+0000","last_fullsized":"2024-04-25T08:44:16.317749+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.335902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833719+0000","last_change":"2024-04-25T08:44:14.861575+0000","last_active":"2024-04-25T08:44:15.833719+0000","last_peered":"2024-04-25T08:44:15.833719+0000","last_clean":"2024-04-25T08:44:15.833719+0000","last_became_active":"2024-04-25T08:44:14.861255+0000","last_became_peered":"2024-04-25T08:44:14.861255+0000","last_unstale":"2024-04-25T08:44:15.833719+0000","last_undegraded":"2024-04-25T08:44:15.833719+0000","last_fullsized":"2024-04-25T08:44:15.833719+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:03.763484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864420+0000","last_change":"2024-04-25T08:44:14.876807+0000","last_active":"2024-04-25T08:44:15.864420+0000","last_peered":"2024-04-25T08:44:15.864420+0000","last_clean":"2024-04-25T08:44:15.864420+0000","last_became_active":"2024-04-25T08:44:14.865415+0000","last_became_peered":"2024-04-25T08:44:14.865415+0000","last_unstale":"2024-04-25T08:44:15.864420+0000","last_undegraded":"2024-04-25T08:44:15.864420+0000","last_fullsized":"2024-04-25T08:44:15.864420+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:47.737372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319381+0000","last_change":"2024-04-25T08:44:14.854113+0000","last_active":"2024-04-25T08:44:16.319381+0000","last_peered":"2024-04-25T08:44:16.319381+0000","last_clean":"2024-04-25T08:44:16.319381+0000","last_became_active":"2024-04-25T08:44:14.853682+0000","last_became_peered":"2024-04-25T08:44:14.853682+0000","last_unstale":"2024-04-25T08:44:16.319381+0000","last_undegraded":"2024-04-25T08:44:16.319381+0000","last_fullsized":"2024-04-25T08:44:16.319381+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:54.441005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319524+0000","last_change":"2024-04-25T08:44:14.867724+0000","last_active":"2024-04-25T08:44:16.319524+0000","last_peered":"2024-04-25T08:44:16.319524+0000","last_clean":"2024-04-25T08:44:16.319524+0000","last_became_active":"2024-04-25T08:44:14.867478+0000","last_became_peered":"2024-04-25T08:44:14.867478+0000","last_unstale":"2024-04-25T08:44:16.319524+0000","last_undegraded":"2024-04-25T08:44:16.319524+0000","last_fullsized":"2024-04-25T08:44:16.319524+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:25.078541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834497+0000","last_change":"2024-04-25T08:44:14.854352+0000","last_active":"2024-04-25T08:44:15.834497+0000","last_peered":"2024-04-25T08:44:15.834497+0000","last_clean":"2024-04-25T08:44:15.834497+0000","last_became_active":"2024-04-25T08:44:14.854089+0000","last_became_peered":"2024-04-25T08:44:14.854089+0000","last_unstale":"2024-04-25T08:44:15.834497+0000","last_undegraded":"2024-04-25T08:44:15.834497+0000","last_fullsized":"2024-04-25T08:44:15.834497+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:02.535112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319097+0000","last_change":"2024-04-25T08:44:14.859492+0000","last_active":"2024-04-25T08:44:16.319097+0000","last_peered":"2024-04-25T08:44:16.319097+0000","last_clean":"2024-04-25T08:44:16.319097+0000","last_became_active":"2024-04-25T08:44:14.859265+0000","last_became_peered":"2024-04-25T08:44:14.859265+0000","last_unstale":"2024-04-25T08:44:16.319097+0000","last_undegraded":"2024-04-25T08:44:16.319097+0000","last_fullsized":"2024-04-25T08:44:16.319097+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:23.108998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319607+0000","last_change":"2024-04-25T08:44:14.858476+0000","last_active":"2024-04-25T08:44:16.319607+0000","last_peered":"2024-04-25T08:44:16.319607+0000","last_clean":"2024-04-25T08:44:16.319607+0000","last_became_active":"2024-04-25T08:44:14.858099+0000","last_became_peered":"2024-04-25T08:44:14.858099+0000","last_unstale":"2024-04-25T08:44:16.319607+0000","last_undegraded":"2024-04-25T08:44:16.319607+0000","last_fullsized":"2024-04-25T08:44:16.319607+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:29.576117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833902+0000","last_change":"2024-04-25T08:44:14.853647+0000","last_active":"2024-04-25T08:44:15.833902+0000","last_peered":"2024-04-25T08:44:15.833902+0000","last_clean":"2024-04-25T08:44:15.833902+0000","last_became_active":"2024-04-25T08:44:14.853358+0000","last_became_peered":"2024-04-25T08:44:14.853358+0000","last_unstale":"2024-04-25T08:44:15.833902+0000","last_undegraded":"2024-04-25T08:44:15.833902+0000","last_fullsized":"2024-04-25T08:44:15.833902+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:06.020349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864675+0000","last_change":"2024-04-25T08:44:14.864541+0000","last_active":"2024-04-25T08:44:15.864675+0000","last_peered":"2024-04-25T08:44:15.864675+0000","last_clean":"2024-04-25T08:44:15.864675+0000","last_became_active":"2024-04-25T08:44:14.864269+0000","last_became_peered":"2024-04-25T08:44:14.864269+0000","last_unstale":"2024-04-25T08:44:15.864675+0000","last_undegraded":"2024-04-25T08:44:15.864675+0000","last_fullsized":"2024-04-25T08:44:15.864675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.707169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867170+0000","last_change":"2024-04-25T08:44:14.860298+0000","last_active":"2024-04-25T08:44:15.867170+0000","last_peered":"2024-04-25T08:44:15.867170+0000","last_clean":"2024-04-25T08:44:15.867170+0000","last_became_active":"2024-04-25T08:44:14.860094+0000","last_became_peered":"2024-04-25T08:44:14.860094+0000","last_unstale":"2024-04-25T08:44:15.867170+0000","last_undegraded":"2024-04-25T08:44:15.867170+0000","last_fullsized":"2024-04-25T08:44:15.867170+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:06.020349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833682+0000","last_change":"2024-04-25T08:44:14.847970+0000","last_active":"2024-04-25T08:44:15.833682+0000","last_peered":"2024-04-25T08:44:15.833682+0000","last_clean":"2024-04-25T08:44:15.833682+0000","last_became_active":"2024-04-25T08:44:14.847662+0000","last_became_peered":"2024-04-25T08:44:14.847662+0000","last_unstale":"2024-04-25T08:44:15.833682+0000","last_undegraded":"2024-04-25T08:44:15.833682+0000","last_fullsized":"2024-04-25T08:44:15.833682+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:30.256964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865251+0000","last_change":"2024-04-25T08:44:14.872220+0000","last_active":"2024-04-25T08:44:15.865251+0000","last_peered":"2024-04-25T08:44:15.865251+0000","last_clean":"2024-04-25T08:44:15.865251+0000","last_became_active":"2024-04-25T08:44:14.862940+0000","last_became_peered":"2024-04-25T08:44:14.862940+0000","last_unstale":"2024-04-25T08:44:15.865251+0000","last_undegraded":"2024-04-25T08:44:15.865251+0000","last_fullsized":"2024-04-25T08:44:15.865251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:38.860125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867583+0000","last_change":"2024-04-25T08:44:14.865300+0000","last_active":"2024-04-25T08:44:15.867583+0000","last_peered":"2024-04-25T08:44:15.867583+0000","last_clean":"2024-04-25T08:44:15.867583+0000","last_became_active":"2024-04-25T08:44:14.865051+0000","last_became_peered":"2024-04-25T08:44:14.865051+0000","last_unstale":"2024-04-25T08:44:15.867583+0000","last_undegraded":"2024-04-25T08:44:15.867583+0000","last_fullsized":"2024-04-25T08:44:15.867583+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:34.481308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865000+0000","last_change":"2024-04-25T08:44:14.872768+0000","last_active":"2024-04-25T08:44:15.865000+0000","last_peered":"2024-04-25T08:44:15.865000+0000","last_clean":"2024-04-25T08:44:15.865000+0000","last_became_active":"2024-04-25T08:44:14.871620+0000","last_became_peered":"2024-04-25T08:44:14.871620+0000","last_unstale":"2024-04-25T08:44:15.865000+0000","last_undegraded":"2024-04-25T08:44:15.865000+0000","last_fullsized":"2024-04-25T08:44:15.865000+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:25.078541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835111+0000","last_change":"2024-04-25T08:44:14.862318+0000","last_active":"2024-04-25T08:44:15.835111+0000","last_peered":"2024-04-25T08:44:15.835111+0000","last_clean":"2024-04-25T08:44:15.835111+0000","last_became_active":"2024-04-25T08:44:14.861429+0000","last_became_peered":"2024-04-25T08:44:14.861429+0000","last_unstale":"2024-04-25T08:44:15.835111+0000","last_undegraded":"2024-04-25T08:44:15.835111+0000","last_fullsized":"2024-04-25T08:44:15.835111+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:23.634635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835041+0000","last_change":"2024-04-25T08:44:14.854726+0000","last_active":"2024-04-25T08:44:15.835041+0000","last_peered":"2024-04-25T08:44:15.835041+0000","last_clean":"2024-04-25T08:44:15.835041+0000","last_became_active":"2024-04-25T08:44:14.854465+0000","last_became_peered":"2024-04-25T08:44:14.854465+0000","last_unstale":"2024-04-25T08:44:15.835041+0000","last_undegraded":"2024-04-25T08:44:15.835041+0000","last_fullsized":"2024-04-25T08:44:15.835041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.335902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319068+0000","last_change":"2024-04-25T08:44:14.857014+0000","last_active":"2024-04-25T08:44:16.319068+0000","last_peered":"2024-04-25T08:44:16.319068+0000","last_clean":"2024-04-25T08:44:16.319068+0000","last_became_active":"2024-04-25T08:44:14.856677+0000","last_became_peered":"2024-04-25T08:44:14.856677+0000","last_unstale":"2024-04-25T08:44:16.319068+0000","last_undegraded":"2024-04-25T08:44:16.319068+0000","last_fullsized":"2024-04-25T08:44:16.319068+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.707169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.318972+0000","last_change":"2024-04-25T08:44:14.857256+0000","last_active":"2024-04-25T08:44:16.318972+0000","last_peered":"2024-04-25T08:44:16.318972+0000","last_clean":"2024-04-25T08:44:16.318972+0000","last_became_active":"2024-04-25T08:44:14.857005+0000","last_became_peered":"2024-04-25T08:44:14.857005+0000","last_unstale":"2024-04-25T08:44:16.318972+0000","last_undegraded":"2024-04-25T08:44:16.318972+0000","last_fullsized":"2024-04-25T08:44:16.318972+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:11.272258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864455+0000","last_change":"2024-04-25T08:44:14.853262+0000","last_active":"2024-04-25T08:44:15.864455+0000","last_peered":"2024-04-25T08:44:15.864455+0000","last_clean":"2024-04-25T08:44:15.864455+0000","last_became_active":"2024-04-25T08:44:14.852958+0000","last_became_peered":"2024-04-25T08:44:14.852958+0000","last_unstale":"2024-04-25T08:44:15.864455+0000","last_undegraded":"2024-04-25T08:44:15.864455+0000","last_fullsized":"2024-04-25T08:44:15.864455+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:19.081585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865255+0000","last_change":"2024-04-25T08:44:14.872344+0000","last_active":"2024-04-25T08:44:15.865255+0000","last_peered":"2024-04-25T08:44:15.865255+0000","last_clean":"2024-04-25T08:44:15.865255+0000","last_became_active":"2024-04-25T08:44:14.859356+0000","last_became_peered":"2024-04-25T08:44:14.859356+0000","last_unstale":"2024-04-25T08:44:15.865255+0000","last_undegraded":"2024-04-25T08:44:15.865255+0000","last_fullsized":"2024-04-25T08:44:15.865255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:02.277771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319765+0000","last_change":"2024-04-25T08:44:14.843823+0000","last_active":"2024-04-25T08:44:16.319765+0000","last_peered":"2024-04-25T08:44:16.319765+0000","last_clean":"2024-04-25T08:44:16.319765+0000","last_became_active":"2024-04-25T08:44:14.843554+0000","last_became_peered":"2024-04-25T08:44:14.843554+0000","last_unstale":"2024-04-25T08:44:16.319765+0000","last_undegraded":"2024-04-25T08:44:16.319765+0000","last_fullsized":"2024-04-25T08:44:16.319765+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:27.767834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834332+0000","last_change":"2024-04-25T08:44:14.873232+0000","last_active":"2024-04-25T08:44:15.834332+0000","last_peered":"2024-04-25T08:44:15.834332+0000","last_clean":"2024-04-25T08:44:15.834332+0000","last_became_active":"2024-04-25T08:44:14.870478+0000","last_became_peered":"2024-04-25T08:44:14.870478+0000","last_unstale":"2024-04-25T08:44:15.834332+0000","last_undegraded":"2024-04-25T08:44:15.834332+0000","last_fullsized":"2024-04-25T08:44:15.834332+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835156+0000","last_change":"2024-04-25T08:44:14.858192+0000","last_active":"2024-04-25T08:44:15.835156+0000","last_peered":"2024-04-25T08:44:15.835156+0000","last_clean":"2024-04-25T08:44:15.835156+0000","last_became_active":"2024-04-25T08:44:14.857938+0000","last_became_peered":"2024-04-25T08:44:14.857938+0000","last_unstale":"2024-04-25T08:44:15.835156+0000","last_undegraded":"2024-04-25T08:44:15.835156+0000","last_fullsized":"2024-04-25T08:44:15.835156+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:42.271056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864726+0000","last_change":"2024-04-25T08:44:14.857342+0000","last_active":"2024-04-25T08:44:15.864726+0000","last_peered":"2024-04-25T08:44:15.864726+0000","last_clean":"2024-04-25T08:44:15.864726+0000","last_became_active":"2024-04-25T08:44:14.857003+0000","last_became_peered":"2024-04-25T08:44:14.857003+0000","last_unstale":"2024-04-25T08:44:15.864726+0000","last_undegraded":"2024-04-25T08:44:15.864726+0000","last_fullsized":"2024-04-25T08:44:15.864726+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:54.441005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867340+0000","last_change":"2024-04-25T08:44:14.862768+0000","last_active":"2024-04-25T08:44:15.867340+0000","last_peered":"2024-04-25T08:44:15.867340+0000","last_clean":"2024-04-25T08:44:15.867340+0000","last_became_active":"2024-04-25T08:44:14.862384+0000","last_became_peered":"2024-04-25T08:44:14.862384+0000","last_unstale":"2024-04-25T08:44:15.867340+0000","last_undegraded":"2024-04-25T08:44:15.867340+0000","last_fullsized":"2024-04-25T08:44:15.867340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864530+0000","last_change":"2024-04-25T08:44:14.865687+0000","last_active":"2024-04-25T08:44:15.864530+0000","last_peered":"2024-04-25T08:44:15.864530+0000","last_clean":"2024-04-25T08:44:15.864530+0000","last_became_active":"2024-04-25T08:44:14.865004+0000","last_became_peered":"2024-04-25T08:44:14.865004+0000","last_unstale":"2024-04-25T08:44:15.864530+0000","last_undegraded":"2024-04-25T08:44:15.864530+0000","last_fullsized":"2024-04-25T08:44:15.864530+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:26.314438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834186+0000","last_change":"2024-04-25T08:44:14.846903+0000","last_active":"2024-04-25T08:44:15.834186+0000","last_peered":"2024-04-25T08:44:15.834186+0000","last_clean":"2024-04-25T08:44:15.834186+0000","last_became_active":"2024-04-25T08:44:14.846530+0000","last_became_peered":"2024-04-25T08:44:14.846530+0000","last_unstale":"2024-04-25T08:44:15.834186+0000","last_undegraded":"2024-04-25T08:44:15.834186+0000","last_fullsized":"2024-04-25T08:44:15.834186+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:35.493623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.866935+0000","last_change":"2024-04-25T08:44:14.864066+0000","last_active":"2024-04-25T08:44:15.866935+0000","last_peered":"2024-04-25T08:44:15.866935+0000","last_clean":"2024-04-25T08:44:15.866935+0000","last_became_active":"2024-04-25T08:44:14.863817+0000","last_became_peered":"2024-04-25T08:44:14.863817+0000","last_unstale":"2024-04-25T08:44:15.866935+0000","last_undegraded":"2024-04-25T08:44:15.866935+0000","last_fullsized":"2024-04-25T08:44:15.866935+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:42.806945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835269+0000","last_change":"2024-04-25T08:44:13.832203+0000","last_active":"2024-04-25T08:44:15.835269+0000","last_peered":"2024-04-25T08:44:15.835269+0000","last_clean":"2024-04-25T08:44:15.835269+0000","last_became_active":"2024-04-25T08:44:13.831889+0000","last_became_peered":"2024-04-25T08:44:13.831889+0000","last_unstale":"2024-04-25T08:44:15.835269+0000","last_undegraded":"2024-04-25T08:44:15.835269+0000","last_fullsized":"2024-04-25T08:44:15.835269+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:46.675483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834271+0000","last_change":"2024-04-25T08:44:14.861591+0000","last_active":"2024-04-25T08:44:15.834271+0000","last_peered":"2024-04-25T08:44:15.834271+0000","last_clean":"2024-04-25T08:44:15.834271+0000","last_became_active":"2024-04-25T08:44:14.861186+0000","last_became_peered":"2024-04-25T08:44:14.861186+0000","last_unstale":"2024-04-25T08:44:15.834271+0000","last_undegraded":"2024-04-25T08:44:15.834271+0000","last_fullsized":"2024-04-25T08:44:15.834271+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:02.535112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834313+0000","last_change":"2024-04-25T08:44:13.840737+0000","last_active":"2024-04-25T08:44:15.834313+0000","last_peered":"2024-04-25T08:44:15.834313+0000","last_clean":"2024-04-25T08:44:15.834313+0000","last_became_active":"2024-04-25T08:44:13.840436+0000","last_became_peered":"2024-04-25T08:44:13.840436+0000","last_unstale":"2024-04-25T08:44:15.834313+0000","last_undegraded":"2024-04-25T08:44:15.834313+0000","last_fullsized":"2024-04-25T08:44:15.834313+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:40.924903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834893+0000","last_change":"2024-04-25T08:44:14.857569+0000","last_active":"2024-04-25T08:44:15.834893+0000","last_peered":"2024-04-25T08:44:15.834893+0000","last_clean":"2024-04-25T08:44:15.834893+0000","last_became_active":"2024-04-25T08:44:14.857289+0000","last_became_peered":"2024-04-25T08:44:14.857289+0000","last_unstale":"2024-04-25T08:44:15.834893+0000","last_undegraded":"2024-04-25T08:44:15.834893+0000","last_fullsized":"2024-04-25T08:44:15.834893+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:48.314480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834457+0000","last_change":"2024-04-25T08:44:13.836577+0000","last_active":"2024-04-25T08:44:15.834457+0000","last_peered":"2024-04-25T08:44:15.834457+0000","last_clean":"2024-04-25T08:44:15.834457+0000","last_became_active":"2024-04-25T08:44:13.836262+0000","last_became_peered":"2024-04-25T08:44:13.836262+0000","last_unstale":"2024-04-25T08:44:15.834457+0000","last_undegraded":"2024-04-25T08:44:15.834457+0000","last_fullsized":"2024-04-25T08:44:15.834457+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.410453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.318884+0000","last_change":"2024-04-25T08:44:14.845059+0000","last_active":"2024-04-25T08:44:16.318884+0000","last_peered":"2024-04-25T08:44:16.318884+0000","last_clean":"2024-04-25T08:44:16.318884+0000","last_became_active":"2024-04-25T08:44:14.844799+0000","last_became_peered":"2024-04-25T08:44:14.844799+0000","last_unstale":"2024-04-25T08:44:16.318884+0000","last_undegraded":"2024-04-25T08:44:16.318884+0000","last_fullsized":"2024-04-25T08:44:16.318884+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:26.314438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834466+0000","last_change":"2024-04-25T08:44:13.830937+0000","last_active":"2024-04-25T08:44:15.834466+0000","last_peered":"2024-04-25T08:44:15.834466+0000","last_clean":"2024-04-25T08:44:15.834466+0000","last_became_active":"2024-04-25T08:44:13.830630+0000","last_became_peered":"2024-04-25T08:44:13.830630+0000","last_unstale":"2024-04-25T08:44:15.834466+0000","last_undegraded":"2024-04-25T08:44:15.834466+0000","last_fullsized":"2024-04-25T08:44:15.834466+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.329695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317907+0000","last_change":"2024-04-25T08:44:14.857454+0000","last_active":"2024-04-25T08:44:16.317907+0000","last_peered":"2024-04-25T08:44:16.317907+0000","last_clean":"2024-04-25T08:44:16.317907+0000","last_became_active":"2024-04-25T08:44:14.857216+0000","last_became_peered":"2024-04-25T08:44:14.857216+0000","last_unstale":"2024-04-25T08:44:16.317907+0000","last_undegraded":"2024-04-25T08:44:16.317907+0000","last_fullsized":"2024-04-25T08:44:16.317907+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:23.634635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864688+0000","last_change":"2024-04-25T08:44:13.832624+0000","last_active":"2024-04-25T08:44:15.864688+0000","last_peered":"2024-04-25T08:44:15.864688+0000","last_clean":"2024-04-25T08:44:15.864688+0000","last_became_active":"2024-04-25T08:44:13.832310+0000","last_became_peered":"2024-04-25T08:44:13.832310+0000","last_unstale":"2024-04-25T08:44:15.864688+0000","last_undegraded":"2024-04-25T08:44:15.864688+0000","last_fullsized":"2024-04-25T08:44:15.864688+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:15.487629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867430+0000","last_change":"2024-04-25T08:44:14.863226+0000","last_active":"2024-04-25T08:44:15.867430+0000","last_peered":"2024-04-25T08:44:15.867430+0000","last_clean":"2024-04-25T08:44:15.867430+0000","last_became_active":"2024-04-25T08:44:14.862726+0000","last_became_peered":"2024-04-25T08:44:14.862726+0000","last_unstale":"2024-04-25T08:44:15.867430+0000","last_undegraded":"2024-04-25T08:44:15.867430+0000","last_fullsized":"2024-04-25T08:44:15.867430+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317632+0000","last_change":"2024-04-25T08:44:13.829202+0000","last_active":"2024-04-25T08:44:16.317632+0000","last_peered":"2024-04-25T08:44:16.317632+0000","last_clean":"2024-04-25T08:44:16.317632+0000","last_became_active":"2024-04-25T08:44:13.828896+0000","last_became_peered":"2024-04-25T08:44:13.828896+0000","last_unstale":"2024-04-25T08:44:16.317632+0000","last_undegraded":"2024-04-25T08:44:16.317632+0000","last_fullsized":"2024-04-25T08:44:16.317632+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:40.924903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319918+0000","last_change":"2024-04-25T08:44:14.843658+0000","last_active":"2024-04-25T08:44:16.319918+0000","last_peered":"2024-04-25T08:44:16.319918+0000","last_clean":"2024-04-25T08:44:16.319918+0000","last_became_active":"2024-04-25T08:44:14.843203+0000","last_became_peered":"2024-04-25T08:44:14.843203+0000","last_unstale":"2024-04-25T08:44:16.319918+0000","last_undegraded":"2024-04-25T08:44:16.319918+0000","last_fullsized":"2024-04-25T08:44:16.319918+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:38.860125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834857+0000","last_change":"2024-04-25T08:44:13.833544+0000","last_active":"2024-04-25T08:44:15.834857+0000","last_peered":"2024-04-25T08:44:15.834857+0000","last_clean":"2024-04-25T08:44:15.834857+0000","last_became_active":"2024-04-25T08:44:13.833024+0000","last_became_peered":"2024-04-25T08:44:13.833024+0000","last_unstale":"2024-04-25T08:44:15.834857+0000","last_undegraded":"2024-04-25T08:44:15.834857+0000","last_fullsized":"2024-04-25T08:44:15.834857+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.410453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867536+0000","last_change":"2024-04-25T08:44:14.865485+0000","last_active":"2024-04-25T08:44:15.867536+0000","last_peered":"2024-04-25T08:44:15.867536+0000","last_clean":"2024-04-25T08:44:15.867536+0000","last_became_active":"2024-04-25T08:44:14.865231+0000","last_became_peered":"2024-04-25T08:44:14.865231+0000","last_unstale":"2024-04-25T08:44:15.867536+0000","last_undegraded":"2024-04-25T08:44:15.867536+0000","last_fullsized":"2024-04-25T08:44:15.867536+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:35.493623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864474+0000","last_change":"2024-04-25T08:44:13.831048+0000","last_active":"2024-04-25T08:44:15.864474+0000","last_peered":"2024-04-25T08:44:15.864474+0000","last_clean":"2024-04-25T08:44:15.864474+0000","last_became_active":"2024-04-25T08:44:13.830737+0000","last_became_peered":"2024-04-25T08:44:13.830737+0000","last_unstale":"2024-04-25T08:44:15.864474+0000","last_undegraded":"2024-04-25T08:44:15.864474+0000","last_fullsized":"2024-04-25T08:44:15.864474+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.933448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317555+0000","last_change":"2024-04-25T08:44:14.852785+0000","last_active":"2024-04-25T08:44:16.317555+0000","last_peered":"2024-04-25T08:44:16.317555+0000","last_clean":"2024-04-25T08:44:16.317555+0000","last_became_active":"2024-04-25T08:44:14.852472+0000","last_became_peered":"2024-04-25T08:44:14.852472+0000","last_unstale":"2024-04-25T08:44:16.317555+0000","last_undegraded":"2024-04-25T08:44:16.317555+0000","last_fullsized":"2024-04-25T08:44:16.317555+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:08.143188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867450+0000","last_change":"2024-04-25T08:44:13.828624+0000","last_active":"2024-04-25T08:44:15.867450+0000","last_peered":"2024-04-25T08:44:15.867450+0000","last_clean":"2024-04-25T08:44:15.867450+0000","last_became_active":"2024-04-25T08:44:13.828054+0000","last_became_peered":"2024-04-25T08:44:13.828054+0000","last_unstale":"2024-04-25T08:44:15.867450+0000","last_undegraded":"2024-04-25T08:44:15.867450+0000","last_fullsized":"2024-04-25T08:44:15.867450+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.933448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834104+0000","last_change":"2024-04-25T08:44:14.854260+0000","last_active":"2024-04-25T08:44:15.834104+0000","last_peered":"2024-04-25T08:44:15.834104+0000","last_clean":"2024-04-25T08:44:15.834104+0000","last_became_active":"2024-04-25T08:44:14.853525+0000","last_became_peered":"2024-04-25T08:44:14.853525+0000","last_unstale":"2024-04-25T08:44:15.834104+0000","last_undegraded":"2024-04-25T08:44:15.834104+0000","last_fullsized":"2024-04-25T08:44:15.834104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317918+0000","last_change":"2024-04-25T08:44:13.834681+0000","last_active":"2024-04-25T08:44:16.317918+0000","last_peered":"2024-04-25T08:44:16.317918+0000","last_clean":"2024-04-25T08:44:16.317918+0000","last_became_active":"2024-04-25T08:44:13.834347+0000","last_became_peered":"2024-04-25T08:44:13.834347+0000","last_unstale":"2024-04-25T08:44:16.317918+0000","last_undegraded":"2024-04-25T08:44:16.317918+0000","last_fullsized":"2024-04-25T08:44:16.317918+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:23.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864694+0000","last_change":"2024-04-25T08:44:14.857952+0000","last_active":"2024-04-25T08:44:15.864694+0000","last_peered":"2024-04-25T08:44:15.864694+0000","last_clean":"2024-04-25T08:44:15.864694+0000","last_became_active":"2024-04-25T08:44:14.857694+0000","last_became_peered":"2024-04-25T08:44:14.857694+0000","last_unstale":"2024-04-25T08:44:15.864694+0000","last_undegraded":"2024-04-25T08:44:15.864694+0000","last_fullsized":"2024-04-25T08:44:15.864694+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.693702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867497+0000","last_change":"2024-04-25T08:44:13.839415+0000","last_active":"2024-04-25T08:44:15.867497+0000","last_peered":"2024-04-25T08:44:15.867497+0000","last_clean":"2024-04-25T08:44:15.867497+0000","last_became_active":"2024-04-25T08:44:13.839068+0000","last_became_peered":"2024-04-25T08:44:13.839068+0000","last_unstale":"2024-04-25T08:44:15.867497+0000","last_undegraded":"2024-04-25T08:44:15.867497+0000","last_fullsized":"2024-04-25T08:44:15.867497+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:20.939359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317927+0000","last_change":"2024-04-25T08:44:14.859416+0000","last_active":"2024-04-25T08:44:16.317927+0000","last_peered":"2024-04-25T08:44:16.317927+0000","last_clean":"2024-04-25T08:44:16.317927+0000","last_became_active":"2024-04-25T08:44:14.859112+0000","last_became_peered":"2024-04-25T08:44:14.859112+0000","last_unstale":"2024-04-25T08:44:16.317927+0000","last_undegraded":"2024-04-25T08:44:16.317927+0000","last_fullsized":"2024-04-25T08:44:16.317927+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:42.271056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319605+0000","last_change":"2024-04-25T08:44:13.822261+0000","last_active":"2024-04-25T08:44:16.319605+0000","last_peered":"2024-04-25T08:44:16.319605+0000","last_clean":"2024-04-25T08:44:16.319605+0000","last_became_active":"2024-04-25T08:44:13.821468+0000","last_became_peered":"2024-04-25T08:44:13.821468+0000","last_unstale":"2024-04-25T08:44:16.319605+0000","last_undegraded":"2024-04-25T08:44:16.319605+0000","last_fullsized":"2024-04-25T08:44:16.319605+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:05.014142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834139+0000","last_change":"2024-04-25T08:44:14.848680+0000","last_active":"2024-04-25T08:44:15.834139+0000","last_peered":"2024-04-25T08:44:15.834139+0000","last_clean":"2024-04-25T08:44:15.834139+0000","last_became_active":"2024-04-25T08:44:14.847901+0000","last_became_peered":"2024-04-25T08:44:14.847901+0000","last_unstale":"2024-04-25T08:44:15.834139+0000","last_undegraded":"2024-04-25T08:44:15.834139+0000","last_fullsized":"2024-04-25T08:44:15.834139+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864816+0000","last_change":"2024-04-25T08:44:13.832989+0000","last_active":"2024-04-25T08:44:15.864816+0000","last_peered":"2024-04-25T08:44:15.864816+0000","last_clean":"2024-04-25T08:44:15.864816+0000","last_became_active":"2024-04-25T08:44:13.832689+0000","last_became_peered":"2024-04-25T08:44:13.832689+0000","last_unstale":"2024-04-25T08:44:15.864816+0000","last_undegraded":"2024-04-25T08:44:15.864816+0000","last_fullsized":"2024-04-25T08:44:15.864816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:31.375263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865174+0000","last_change":"2024-04-25T08:44:14.863881+0000","last_active":"2024-04-25T08:44:15.865174+0000","last_peered":"2024-04-25T08:44:15.865174+0000","last_clean":"2024-04-25T08:44:15.865174+0000","last_became_active":"2024-04-25T08:44:14.859108+0000","last_became_peered":"2024-04-25T08:44:14.859108+0000","last_unstale":"2024-04-25T08:44:15.865174+0000","last_undegraded":"2024-04-25T08:44:15.865174+0000","last_fullsized":"2024-04-25T08:44:15.865174+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:48.153045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835103+0000","last_change":"2024-04-25T08:44:13.834677+0000","last_active":"2024-04-25T08:44:15.835103+0000","last_peered":"2024-04-25T08:44:15.835103+0000","last_clean":"2024-04-25T08:44:15.835103+0000","last_became_active":"2024-04-25T08:44:13.834384+0000","last_became_peered":"2024-04-25T08:44:13.834384+0000","last_unstale":"2024-04-25T08:44:15.835103+0000","last_undegraded":"2024-04-25T08:44:15.835103+0000","last_fullsized":"2024-04-25T08:44:15.835103+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:23.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319791+0000","last_change":"2024-04-25T08:44:14.861080+0000","last_active":"2024-04-25T08:44:16.319791+0000","last_peered":"2024-04-25T08:44:16.319791+0000","last_clean":"2024-04-25T08:44:16.319791+0000","last_became_active":"2024-04-25T08:44:14.860713+0000","last_became_peered":"2024-04-25T08:44:14.860713+0000","last_unstale":"2024-04-25T08:44:16.319791+0000","last_undegraded":"2024-04-25T08:44:16.319791+0000","last_fullsized":"2024-04-25T08:44:16.319791+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:48.153045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867262+0000","last_change":"2024-04-25T08:44:13.827274+0000","last_active":"2024-04-25T08:44:15.867262+0000","last_peered":"2024-04-25T08:44:15.867262+0000","last_clean":"2024-04-25T08:44:15.867262+0000","last_became_active":"2024-04-25T08:44:13.827020+0000","last_became_peered":"2024-04-25T08:44:13.827020+0000","last_unstale":"2024-04-25T08:44:15.867262+0000","last_undegraded":"2024-04-25T08:44:15.867262+0000","last_fullsized":"2024-04-25T08:44:15.867262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:06.183174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865123+0000","last_change":"2024-04-25T08:44:14.873838+0000","last_active":"2024-04-25T08:44:15.865123+0000","last_peered":"2024-04-25T08:44:15.865123+0000","last_clean":"2024-04-25T08:44:15.865123+0000","last_became_active":"2024-04-25T08:44:14.873570+0000","last_became_peered":"2024-04-25T08:44:14.873570+0000","last_unstale":"2024-04-25T08:44:15.865123+0000","last_undegraded":"2024-04-25T08:44:15.865123+0000","last_fullsized":"2024-04-25T08:44:15.865123+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:27.767834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":22,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319534+0000","last_change":"2024-04-25T08:44:13.823663+0000","last_active":"2024-04-25T08:44:16.319534+0000","last_peered":"2024-04-25T08:44:16.319534+0000","last_clean":"2024-04-25T08:44:16.319534+0000","last_became_active":"2024-04-25T08:44:13.822970+0000","last_became_peered":"2024-04-25T08:44:13.822970+0000","last_unstale":"2024-04-25T08:44:16.319534+0000","last_undegraded":"2024-04-25T08:44:16.319534+0000","last_fullsized":"2024-04-25T08:44:16.319534+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:57.170270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319401+0000","last_change":"2024-04-25T08:44:14.847871+0000","last_active":"2024-04-25T08:44:16.319401+0000","last_peered":"2024-04-25T08:44:16.319401+0000","last_clean":"2024-04-25T08:44:16.319401+0000","last_became_active":"2024-04-25T08:44:14.847511+0000","last_became_peered":"2024-04-25T08:44:14.847511+0000","last_unstale":"2024-04-25T08:44:16.319401+0000","last_undegraded":"2024-04-25T08:44:16.319401+0000","last_fullsized":"2024-04-25T08:44:16.319401+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:36.380477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865340+0000","last_change":"2024-04-25T08:44:13.831815+0000","last_active":"2024-04-25T08:44:15.865340+0000","last_peered":"2024-04-25T08:44:15.865340+0000","last_clean":"2024-04-25T08:44:15.865340+0000","last_became_active":"2024-04-25T08:44:13.831549+0000","last_became_peered":"2024-04-25T08:44:13.831549+0000","last_unstale":"2024-04-25T08:44:15.865340+0000","last_undegraded":"2024-04-25T08:44:15.865340+0000","last_fullsized":"2024-04-25T08:44:15.865340+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:07.136981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833701+0000","last_change":"2024-04-25T08:44:14.856849+0000","last_active":"2024-04-25T08:44:15.833701+0000","last_peered":"2024-04-25T08:44:15.833701+0000","last_clean":"2024-04-25T08:44:15.833701+0000","last_became_active":"2024-04-25T08:44:14.856511+0000","last_became_peered":"2024-04-25T08:44:14.856511+0000","last_unstale":"2024-04-25T08:44:15.833701+0000","last_undegraded":"2024-04-25T08:44:15.833701+0000","last_fullsized":"2024-04-25T08:44:15.833701+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:58.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319434+0000","last_change":"2024-04-25T08:44:13.827580+0000","last_active":"2024-04-25T08:44:16.319434+0000","last_peered":"2024-04-25T08:44:16.319434+0000","last_clean":"2024-04-25T08:44:16.319434+0000","last_became_active":"2024-04-25T08:44:13.827081+0000","last_became_peered":"2024-04-25T08:44:13.827081+0000","last_unstale":"2024-04-25T08:44:16.319434+0000","last_undegraded":"2024-04-25T08:44:16.319434+0000","last_fullsized":"2024-04-25T08:44:16.319434+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.047776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317593+0000","last_change":"2024-04-25T08:44:14.874041+0000","last_active":"2024-04-25T08:44:16.317593+0000","last_peered":"2024-04-25T08:44:16.317593+0000","last_clean":"2024-04-25T08:44:16.317593+0000","last_became_active":"2024-04-25T08:44:14.872918+0000","last_became_peered":"2024-04-25T08:44:14.872918+0000","last_unstale":"2024-04-25T08:44:16.317593+0000","last_undegraded":"2024-04-25T08:44:16.317593+0000","last_fullsized":"2024-04-25T08:44:16.317593+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:48.314480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835597+0000","last_change":"2024-04-25T08:44:13.836322+0000","last_active":"2024-04-25T08:44:15.835597+0000","last_peered":"2024-04-25T08:44:15.835597+0000","last_clean":"2024-04-25T08:44:15.835597+0000","last_became_active":"2024-04-25T08:44:13.836019+0000","last_became_peered":"2024-04-25T08:44:13.836019+0000","last_unstale":"2024-04-25T08:44:15.835597+0000","last_undegraded":"2024-04-25T08:44:15.835597+0000","last_fullsized":"2024-04-25T08:44:15.835597+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:14.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834043+0000","last_change":"2024-04-25T08:44:14.845540+0000","last_active":"2024-04-25T08:44:15.834043+0000","last_peered":"2024-04-25T08:44:15.834043+0000","last_clean":"2024-04-25T08:44:15.834043+0000","last_became_active":"2024-04-25T08:44:14.845247+0000","last_became_peered":"2024-04-25T08:44:14.845247+0000","last_unstale":"2024-04-25T08:44:15.834043+0000","last_undegraded":"2024-04-25T08:44:15.834043+0000","last_fullsized":"2024-04-25T08:44:15.834043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:59.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835216+0000","last_change":"2024-04-25T08:44:13.826386+0000","last_active":"2024-04-25T08:44:15.835216+0000","last_peered":"2024-04-25T08:44:15.835216+0000","last_clean":"2024-04-25T08:44:15.835216+0000","last_became_active":"2024-04-25T08:44:13.826048+0000","last_became_peered":"2024-04-25T08:44:13.826048+0000","last_unstale":"2024-04-25T08:44:15.835216+0000","last_undegraded":"2024-04-25T08:44:15.835216+0000","last_fullsized":"2024-04-25T08:44:15.835216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:59.861264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834074+0000","last_change":"2024-04-25T08:44:14.854319+0000","last_active":"2024-04-25T08:44:15.834074+0000","last_peered":"2024-04-25T08:44:15.834074+0000","last_clean":"2024-04-25T08:44:15.834074+0000","last_became_active":"2024-04-25T08:44:14.853844+0000","last_became_peered":"2024-04-25T08:44:14.853844+0000","last_unstale":"2024-04-25T08:44:15.834074+0000","last_undegraded":"2024-04-25T08:44:15.834074+0000","last_fullsized":"2024-04-25T08:44:15.834074+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:59.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834185+0000","last_change":"2024-04-25T08:44:13.836241+0000","last_active":"2024-04-25T08:44:15.834185+0000","last_peered":"2024-04-25T08:44:15.834185+0000","last_clean":"2024-04-25T08:44:15.834185+0000","last_became_active":"2024-04-25T08:44:13.835917+0000","last_became_peered":"2024-04-25T08:44:13.835917+0000","last_unstale":"2024-04-25T08:44:15.834185+0000","last_undegraded":"2024-04-25T08:44:15.834185+0000","last_fullsized":"2024-04-25T08:44:15.834185+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:50.768133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835630+0000","last_change":"2024-04-25T08:44:14.855917+0000","last_active":"2024-04-25T08:44:15.835630+0000","last_peered":"2024-04-25T08:44:15.835630+0000","last_clean":"2024-04-25T08:44:15.835630+0000","last_became_active":"2024-04-25T08:44:14.855652+0000","last_became_peered":"2024-04-25T08:44:14.855652+0000","last_unstale":"2024-04-25T08:44:15.835630+0000","last_undegraded":"2024-04-25T08:44:15.835630+0000","last_fullsized":"2024-04-25T08:44:15.835630+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:30.256964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319101+0000","last_change":"2024-04-25T08:44:13.817252+0000","last_active":"2024-04-25T08:44:16.319101+0000","last_peered":"2024-04-25T08:44:16.319101+0000","last_clean":"2024-04-25T08:44:16.319101+0000","last_became_active":"2024-04-25T08:44:13.816979+0000","last_became_peered":"2024-04-25T08:44:13.816979+0000","last_unstale":"2024-04-25T08:44:16.319101+0000","last_undegraded":"2024-04-25T08:44:16.319101+0000","last_fullsized":"2024-04-25T08:44:16.319101+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:29.250756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865051+0000","last_change":"2024-04-25T08:44:14.872100+0000","last_active":"2024-04-25T08:44:15.865051+0000","last_peered":"2024-04-25T08:44:15.865051+0000","last_clean":"2024-04-25T08:44:15.865051+0000","last_became_active":"2024-04-25T08:44:14.871824+0000","last_became_peered":"2024-04-25T08:44:14.871824+0000","last_unstale":"2024-04-25T08:44:15.865051+0000","last_undegraded":"2024-04-25T08:44:15.865051+0000","last_fullsized":"2024-04-25T08:44:15.865051+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:44.521500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864485+0000","last_change":"2024-04-25T08:44:13.838856+0000","last_active":"2024-04-25T08:44:15.864485+0000","last_peered":"2024-04-25T08:44:15.864485+0000","last_clean":"2024-04-25T08:44:15.864485+0000","last_became_active":"2024-04-25T08:44:13.838556+0000","last_became_peered":"2024-04-25T08:44:13.838556+0000","last_unstale":"2024-04-25T08:44:15.864485+0000","last_undegraded":"2024-04-25T08:44:15.864485+0000","last_fullsized":"2024-04-25T08:44:15.864485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:20.939359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834338+0000","last_change":"2024-04-25T08:44:14.840865+0000","last_active":"2024-04-25T08:44:15.834338+0000","last_peered":"2024-04-25T08:44:15.834338+0000","last_clean":"2024-04-25T08:44:15.834338+0000","last_became_active":"2024-04-25T08:44:14.840241+0000","last_became_peered":"2024-04-25T08:44:14.840241+0000","last_unstale":"2024-04-25T08:44:15.834338+0000","last_undegraded":"2024-04-25T08:44:15.834338+0000","last_fullsized":"2024-04-25T08:44:15.834338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319618+0000","last_change":"2024-04-25T08:44:13.821334+0000","last_active":"2024-04-25T08:44:16.319618+0000","last_peered":"2024-04-25T08:44:16.319618+0000","last_clean":"2024-04-25T08:44:16.319618+0000","last_became_active":"2024-04-25T08:44:13.821047+0000","last_became_peered":"2024-04-25T08:44:13.821047+0000","last_unstale":"2024-04-25T08:44:16.319618+0000","last_undegraded":"2024-04-25T08:44:16.319618+0000","last_fullsized":"2024-04-25T08:44:16.319618+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:38.347524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834290+0000","last_change":"2024-04-25T08:44:14.840948+0000","last_active":"2024-04-25T08:44:15.834290+0000","last_peered":"2024-04-25T08:44:15.834290+0000","last_clean":"2024-04-25T08:44:15.834290+0000","last_became_active":"2024-04-25T08:44:14.840622+0000","last_became_peered":"2024-04-25T08:44:14.840622+0000","last_unstale":"2024-04-25T08:44:15.834290+0000","last_undegraded":"2024-04-25T08:44:15.834290+0000","last_fullsized":"2024-04-25T08:44:15.834290+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:34.481308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835346+0000","last_change":"2024-04-25T08:44:13.835312+0000","last_active":"2024-04-25T08:44:15.835346+0000","last_peered":"2024-04-25T08:44:15.835346+0000","last_clean":"2024-04-25T08:44:15.835346+0000","last_became_active":"2024-04-25T08:44:13.833473+0000","last_became_peered":"2024-04-25T08:44:13.833473+0000","last_unstale":"2024-04-25T08:44:15.835346+0000","last_undegraded":"2024-04-25T08:44:15.835346+0000","last_fullsized":"2024-04-25T08:44:15.835346+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:50.768133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833790+0000","last_change":"2024-04-25T08:44:14.846108+0000","last_active":"2024-04-25T08:44:15.833790+0000","last_peered":"2024-04-25T08:44:15.833790+0000","last_clean":"2024-04-25T08:44:15.833790+0000","last_became_active":"2024-04-25T08:44:14.845602+0000","last_became_peered":"2024-04-25T08:44:14.845602+0000","last_unstale":"2024-04-25T08:44:15.833790+0000","last_undegraded":"2024-04-25T08:44:15.833790+0000","last_fullsized":"2024-04-25T08:44:15.833790+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:42.806945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867536+0000","last_change":"2024-04-25T08:44:13.832138+0000","last_active":"2024-04-25T08:44:15.867536+0000","last_peered":"2024-04-25T08:44:15.867536+0000","last_clean":"2024-04-25T08:44:15.867536+0000","last_became_active":"2024-04-25T08:44:13.831825+0000","last_became_peered":"2024-04-25T08:44:13.831825+0000","last_unstale":"2024-04-25T08:44:15.867536+0000","last_undegraded":"2024-04-25T08:44:15.867536+0000","last_fullsized":"2024-04-25T08:44:15.867536+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:15.487629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319338+0000","last_change":"2024-04-25T08:44:14.854033+0000","last_active":"2024-04-25T08:44:16.319338+0000","last_peered":"2024-04-25T08:44:16.319338+0000","last_clean":"2024-04-25T08:44:16.319338+0000","last_became_active":"2024-04-25T08:44:14.853752+0000","last_became_peered":"2024-04-25T08:44:14.853752+0000","last_unstale":"2024-04-25T08:44:16.319338+0000","last_undegraded":"2024-04-25T08:44:16.319338+0000","last_fullsized":"2024-04-25T08:44:16.319338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.693702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864816+0000","last_change":"2024-04-25T08:44:13.828110+0000","last_active":"2024-04-25T08:44:15.864816+0000","last_peered":"2024-04-25T08:44:15.864816+0000","last_clean":"2024-04-25T08:44:15.864816+0000","last_became_active":"2024-04-25T08:44:13.827771+0000","last_became_peered":"2024-04-25T08:44:13.827771+0000","last_unstale":"2024-04-25T08:44:15.864816+0000","last_undegraded":"2024-04-25T08:44:15.864816+0000","last_fullsized":"2024-04-25T08:44:15.864816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:47.308273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864790+0000","last_change":"2024-04-25T08:44:14.856737+0000","last_active":"2024-04-25T08:44:15.864790+0000","last_peered":"2024-04-25T08:44:15.864790+0000","last_clean":"2024-04-25T08:44:15.864790+0000","last_became_active":"2024-04-25T08:44:14.856420+0000","last_became_peered":"2024-04-25T08:44:14.856420+0000","last_unstale":"2024-04-25T08:44:15.864790+0000","last_undegraded":"2024-04-25T08:44:15.864790+0000","last_fullsized":"2024-04-25T08:44:15.864790+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:36.380477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319439+0000","last_change":"2024-04-25T08:44:13.818771+0000","last_active":"2024-04-25T08:44:16.319439+0000","last_peered":"2024-04-25T08:44:16.319439+0000","last_clean":"2024-04-25T08:44:16.319439+0000","last_became_active":"2024-04-25T08:44:13.818399+0000","last_became_peered":"2024-04-25T08:44:13.818399+0000","last_unstale":"2024-04-25T08:44:16.319439+0000","last_undegraded":"2024-04-25T08:44:16.319439+0000","last_fullsized":"2024-04-25T08:44:16.319439+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:41.800738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834324+0000","last_change":"2024-04-25T08:44:14.855120+0000","last_active":"2024-04-25T08:44:15.834324+0000","last_peered":"2024-04-25T08:44:15.834324+0000","last_clean":"2024-04-25T08:44:15.834324+0000","last_became_active":"2024-04-25T08:44:14.854759+0000","last_became_peered":"2024-04-25T08:44:14.854759+0000","last_unstale":"2024-04-25T08:44:15.834324+0000","last_undegraded":"2024-04-25T08:44:15.834324+0000","last_fullsized":"2024-04-25T08:44:15.834324+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:45.942204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319171+0000","last_change":"2024-04-25T08:44:13.838954+0000","last_active":"2024-04-25T08:44:16.319171+0000","last_peered":"2024-04-25T08:44:16.319171+0000","last_clean":"2024-04-25T08:44:16.319171+0000","last_became_active":"2024-04-25T08:44:13.838640+0000","last_became_peered":"2024-04-25T08:44:13.838640+0000","last_unstale":"2024-04-25T08:44:16.319171+0000","last_undegraded":"2024-04-25T08:44:16.319171+0000","last_fullsized":"2024-04-25T08:44:16.319171+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:02.757277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865418+0000","last_change":"2024-04-25T08:44:14.865620+0000","last_active":"2024-04-25T08:44:15.865418+0000","last_peered":"2024-04-25T08:44:15.865418+0000","last_clean":"2024-04-25T08:44:15.865418+0000","last_became_active":"2024-04-25T08:44:14.865355+0000","last_became_peered":"2024-04-25T08:44:14.865355+0000","last_unstale":"2024-04-25T08:44:15.865418+0000","last_undegraded":"2024-04-25T08:44:15.865418+0000","last_fullsized":"2024-04-25T08:44:15.865418+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:44.684383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317511+0000","last_change":"2024-04-25T08:44:13.816637+0000","last_active":"2024-04-25T08:44:16.317511+0000","last_peered":"2024-04-25T08:44:16.317511+0000","last_clean":"2024-04-25T08:44:16.317511+0000","last_became_active":"2024-04-25T08:44:13.816317+0000","last_became_peered":"2024-04-25T08:44:13.816317+0000","last_unstale":"2024-04-25T08:44:16.317511+0000","last_undegraded":"2024-04-25T08:44:16.317511+0000","last_fullsized":"2024-04-25T08:44:16.317511+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.410453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319741+0000","last_change":"2024-04-25T08:44:14.842679+0000","last_active":"2024-04-25T08:44:16.319741+0000","last_peered":"2024-04-25T08:44:16.319741+0000","last_clean":"2024-04-25T08:44:16.319741+0000","last_became_active":"2024-04-25T08:44:14.842301+0000","last_became_peered":"2024-04-25T08:44:14.842301+0000","last_unstale":"2024-04-25T08:44:16.319741+0000","last_undegraded":"2024-04-25T08:44:16.319741+0000","last_fullsized":"2024-04-25T08:44:16.319741+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:44.521500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834318+0000","last_change":"2024-04-25T08:44:13.840482+0000","last_active":"2024-04-25T08:44:15.834318+0000","last_peered":"2024-04-25T08:44:15.834318+0000","last_clean":"2024-04-25T08:44:15.834318+0000","last_became_active":"2024-04-25T08:44:13.840204+0000","last_became_peered":"2024-04-25T08:44:13.840204+0000","last_unstale":"2024-04-25T08:44:15.834318+0000","last_undegraded":"2024-04-25T08:44:15.834318+0000","last_fullsized":"2024-04-25T08:44:15.834318+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:47.308273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834893+0000","last_change":"2024-04-25T08:44:14.853120+0000","last_active":"2024-04-25T08:44:15.834893+0000","last_peered":"2024-04-25T08:44:15.834893+0000","last_clean":"2024-04-25T08:44:15.834893+0000","last_became_active":"2024-04-25T08:44:14.852845+0000","last_became_peered":"2024-04-25T08:44:14.852845+0000","last_unstale":"2024-04-25T08:44:15.834893+0000","last_undegraded":"2024-04-25T08:44:15.834893+0000","last_fullsized":"2024-04-25T08:44:15.834893+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:08.143188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.866983+0000","last_change":"2024-04-25T08:44:13.832756+0000","last_active":"2024-04-25T08:44:15.866983+0000","last_peered":"2024-04-25T08:44:15.866983+0000","last_clean":"2024-04-25T08:44:15.866983+0000","last_became_active":"2024-04-25T08:44:13.832461+0000","last_became_peered":"2024-04-25T08:44:13.832461+0000","last_unstale":"2024-04-25T08:44:15.866983+0000","last_undegraded":"2024-04-25T08:44:15.866983+0000","last_fullsized":"2024-04-25T08:44:15.866983+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.329695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.866941+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:44:15.866941+0000","last_peered":"2024-04-25T08:44:15.866941+0000","last_clean":"2024-04-25T08:44:15.866941+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:44:15.866941+0000","last_undegraded":"2024-04-25T08:44:15.866941+0000","last_fullsized":"2024-04-25T08:44:15.866941+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":871989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":282149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":281497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":275387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":647168,"data_stored":283131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":643072,"data_stored":283109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":638976,"data_stored":281657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":871921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:44:18.947 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:44:19.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f8152400640 1 Processor -- start 2024-04-25T08:44:19.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f8152400640 1 -- start start 2024-04-25T08:44:19.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f8152400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f814c0ccde0 0x7f814c0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f8152400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c06bdc0 con 0x7f814c099a00 2024-04-25T08:44:19.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f8152400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814c06c0a0 con 0x7f814c0b2240 2024-04-25T08:44:19.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f8152400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814c06db90 con 0x7f814c0ccde0 2024-04-25T08:44:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f814c099a00 0x7f814c099de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46300/0 (socket says 172.21.15.46:46300) 2024-04-25T08:44:19.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814b400640 1 -- 172.21.15.46:0/341907418 learned_addr learned my addr 172.21.15.46:0/341907418 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814aa00640 1 --2- 172.21.15.46:0/341907418 >> v2:172.21.15.82:3300/0 conn(0x7f814c0ccde0 0x7f814c0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885435388 0 0) 0x7f814c06bdc0 con 0x7f814c099a00 2024-04-25T08:44:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8130003530 con 0x7f814c099a00 2024-04-25T08:44:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4031321910 0 0) 0x7f8130003530 con 0x7f814c099a00 2024-04-25T08:44:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 >> v2:172.21.15.82:3300/0 conn(0x7f814c0ccde0 msgr2=0x7f814c0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 --2- 172.21.15.46:0/341907418 >> v2:172.21.15.82:3300/0 conn(0x7f814c0ccde0 0x7f814c0cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 >> v1:172.21.15.82:6789/0 conn(0x7f814c0b2240 legacy=0x7f814c0b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c06c380 con 0x7f814c099a00 2024-04-25T08:44:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f813c001680 con 0x7f814c099a00 2024-04-25T08:44:19.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 >> v1:172.21.15.46:6789/0 conn(0x7f814c099a00 legacy=0x7f814c099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81300039f0 0x7f8130003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8130004810 0x7f8130004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130005220 0x7f813000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c06db90 con 0x7f8130004810 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813000c150 con 0x7f8130005220 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813000c350 con 0x7f81300039f0 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814aa00640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8130004810 0x7f8130004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814b400640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81300039f0 0x7f8130003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814be00640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130005220 0x7f813000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814aa00640 1 -- 172.21.15.46:0/341907418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81300039f0 msgr2=0x7f8130003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814aa00640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f81300039f0 0x7f8130003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814aa00640 1 -- 172.21.15.46:0/341907418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130005220 msgr2=0x7f813000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814aa00640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130005220 0x7f813000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814aa00640 1 -- 172.21.15.46:0/341907418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8130003530 con 0x7f8130004810 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814be00640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130005220 0x7f813000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:19.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.134+0000 7f814be00640 1 -- 172.21.15.46:0/341907418 reap_dead start 2024-04-25T08:44:19.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f814aa00640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8130004810 0x7f8130004c90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8138009ae0 tx=0x7f813801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28804dbf5cdfe30d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380b2070 con 0x7f8130004810 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81380a55a0 con 0x7f8130004810 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f814a000640 1 -- 172.21.15.46:0/341907418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380096b0 con 0x7f8130004810 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f8152400640 1 -- 172.21.15.46:0/341907418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8130004810 msgr2=0x7f8130004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f8152400640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8130004810 0x7f8130004c90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8138009ae0 tx=0x7f813801ca40 comp rx=0 tx=0).stop 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f8152400640 1 -- 172.21.15.46:0/341907418 shutdown_connections 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f8152400640 1 --2- 172.21.15.46:0/341907418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8130004810 0x7f8130004c90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f8152400640 1 -- 172.21.15.46:0/341907418 >> 172.21.15.46:0/341907418 conn(0x7f814c0b3120 msgr2=0x7f814c0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f8152400640 1 -- 172.21.15.46:0/341907418 shutdown_connections 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.138+0000 7f8152400640 1 -- 172.21.15.46:0/341907418 wait complete. 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 Processor -- start 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 -- start start 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130004810 0x7f814c157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c158550 0x7f814c15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c0ce2a0 0x7f814c0ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c09a370 con 0x7f814c0ce2a0 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c09a1f0 con 0x7f8130004810 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814c15dd10 con 0x7f814c158550 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c0ce2a0 0x7f814c0ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c0ce2a0 0x7f814c0ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36854/0 (socket says 172.21.15.46:36854) 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814be00640 1 -- 172.21.15.46:0/344214564 learned_addr learned my addr 172.21.15.46:0/344214564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814aa00640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c158550 0x7f814c15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814b400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130004810 0x7f814c157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814be00640 1 -- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c158550 msgr2=0x7f814c15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814be00640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c158550 0x7f814c15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814be00640 1 -- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130004810 msgr2=0x7f814c157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814be00640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130004810 0x7f814c157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814be00640 1 -- 172.21.15.46:0/344214564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c1ea410 con 0x7f814c0ce2a0 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814aa00640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c158550 0x7f814c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814b400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130004810 0x7f814c157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f814be00640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c0ce2a0 0x7f814c0ce720 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f814c072650 tx=0x7f814400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8143400640 1 -- 172.21.15.46:0/344214564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144003980 con 0x7f814c0ce2a0 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8143400640 1 -- 172.21.15.46:0/344214564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8144018070 con 0x7f814c0ce2a0 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c1ea6f0 con 0x7f814c0ce2a0 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8143400640 1 -- 172.21.15.46:0/344214564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144003d20 con 0x7f814c0ce2a0 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.142+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c1eacb0 con 0x7f814c0ce2a0 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.146+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814c069080 con 0x7f814c0ce2a0 2024-04-25T08:44:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.146+0000 7f8143400640 1 -- 172.21.15.46:0/344214564 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f81440050d0 con 0x7f814c0ce2a0 2024-04-25T08:44:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.146+0000 7f8143400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f812c066f60 0x7f812c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.146+0000 7f8143400640 1 -- 172.21.15.46:0/344214564 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8144084e30 con 0x7f814c0ce2a0 2024-04-25T08:44:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.146+0000 7f814b400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f812c066f60 0x7f812c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.146+0000 7f814b400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f812c066f60 0x7f812c069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f813c001210 tx=0x7f813c001120 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.150+0000 7f8143400640 1 -- 172.21.15.46:0/344214564 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8144055ca0 con 0x7f814c0ce2a0 2024-04-25T08:44:19.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.314+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f814c15aca0 con 0x7f812c066f60 2024-04-25T08:44:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.318+0000 7f8143400640 1 -- 172.21.15.46:0/344214564 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251957 (secure 0 0 0) 0x7f814c15aca0 con 0x7f812c066f60 2024-04-25T08:44:19.315 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:19.317 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:44:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f812c066f60 msgr2=0x7f812c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f812c066f60 0x7f812c069420 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f813c001210 tx=0x7f813c001120 comp rx=0 tx=0).stop 2024-04-25T08:44:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c0ce2a0 msgr2=0x7f814c0ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c0ce2a0 0x7f814c0ce720 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f814c072650 tx=0x7f814400eb90 comp rx=0 tx=0).stop 2024-04-25T08:44:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 shutdown_connections 2024-04-25T08:44:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f812c066f60 0x7f812c069420 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c0ce2a0 0x7f814c0ce720 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f814c158550 0x7f814c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 --2- 172.21.15.46:0/344214564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8130004810 0x7f814c157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 >> 172.21.15.46:0/344214564 conn(0x7f814c0b3120 msgr2=0x7f814c073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:19.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 shutdown_connections 2024-04-25T08:44:19.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.322+0000 7f8152400640 1 -- 172.21.15.46:0/344214564 wait complete. 2024-04-25T08:44:19.336 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-04-25T08:44:18.282320+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238956,"kb_used_data":6140,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735764,"statfs":{"total":773094113280,"available":772849422336,"internally_reserved":0,"allocated":6287360,"data_stored":3430840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.477845"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834351+0000","last_change":"2024-04-25T08:44:14.845382+0000","last_active":"2024-04-25T08:44:15.834351+0000","last_peered":"2024-04-25T08:44:15.834351+0000","last_clean":"2024-04-25T08:44:15.834351+0000","last_became_active":"2024-04-25T08:44:14.845069+0000","last_became_peered":"2024-04-25T08:44:14.845069+0000","last_unstale":"2024-04-25T08:44:15.834351+0000","last_undegraded":"2024-04-25T08:44:15.834351+0000","last_fullsized":"2024-04-25T08:44:15.834351+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:35.493623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867585+0000","last_change":"2024-04-25T08:44:14.863335+0000","last_active":"2024-04-25T08:44:15.867585+0000","last_peered":"2024-04-25T08:44:15.867585+0000","last_clean":"2024-04-25T08:44:15.867585+0000","last_became_active":"2024-04-25T08:44:14.863088+0000","last_became_peered":"2024-04-25T08:44:14.863088+0000","last_unstale":"2024-04-25T08:44:15.867585+0000","last_undegraded":"2024-04-25T08:44:15.867585+0000","last_fullsized":"2024-04-25T08:44:15.867585+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:02.535112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865000+0000","last_change":"2024-04-25T08:44:14.877002+0000","last_active":"2024-04-25T08:44:15.865000+0000","last_peered":"2024-04-25T08:44:15.865000+0000","last_clean":"2024-04-25T08:44:15.865000+0000","last_became_active":"2024-04-25T08:44:14.871920+0000","last_became_peered":"2024-04-25T08:44:14.871920+0000","last_unstale":"2024-04-25T08:44:15.865000+0000","last_undegraded":"2024-04-25T08:44:15.865000+0000","last_fullsized":"2024-04-25T08:44:15.865000+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:29.576117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867260+0000","last_change":"2024-04-25T08:44:14.858692+0000","last_active":"2024-04-25T08:44:15.867260+0000","last_peered":"2024-04-25T08:44:15.867260+0000","last_clean":"2024-04-25T08:44:15.867260+0000","last_became_active":"2024-04-25T08:44:14.858365+0000","last_became_peered":"2024-04-25T08:44:14.858365+0000","last_unstale":"2024-04-25T08:44:15.867260+0000","last_undegraded":"2024-04-25T08:44:15.867260+0000","last_fullsized":"2024-04-25T08:44:15.867260+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:59.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317749+0000","last_change":"2024-04-25T08:44:14.856622+0000","last_active":"2024-04-25T08:44:16.317749+0000","last_peered":"2024-04-25T08:44:16.317749+0000","last_clean":"2024-04-25T08:44:16.317749+0000","last_became_active":"2024-04-25T08:44:14.856350+0000","last_became_peered":"2024-04-25T08:44:14.856350+0000","last_unstale":"2024-04-25T08:44:16.317749+0000","last_undegraded":"2024-04-25T08:44:16.317749+0000","last_fullsized":"2024-04-25T08:44:16.317749+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.335902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833719+0000","last_change":"2024-04-25T08:44:14.861575+0000","last_active":"2024-04-25T08:44:15.833719+0000","last_peered":"2024-04-25T08:44:15.833719+0000","last_clean":"2024-04-25T08:44:15.833719+0000","last_became_active":"2024-04-25T08:44:14.861255+0000","last_became_peered":"2024-04-25T08:44:14.861255+0000","last_unstale":"2024-04-25T08:44:15.833719+0000","last_undegraded":"2024-04-25T08:44:15.833719+0000","last_fullsized":"2024-04-25T08:44:15.833719+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:03.763484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864420+0000","last_change":"2024-04-25T08:44:14.876807+0000","last_active":"2024-04-25T08:44:15.864420+0000","last_peered":"2024-04-25T08:44:15.864420+0000","last_clean":"2024-04-25T08:44:15.864420+0000","last_became_active":"2024-04-25T08:44:14.865415+0000","last_became_peered":"2024-04-25T08:44:14.865415+0000","last_unstale":"2024-04-25T08:44:15.864420+0000","last_undegraded":"2024-04-25T08:44:15.864420+0000","last_fullsized":"2024-04-25T08:44:15.864420+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:47.737372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319381+0000","last_change":"2024-04-25T08:44:14.854113+0000","last_active":"2024-04-25T08:44:16.319381+0000","last_peered":"2024-04-25T08:44:16.319381+0000","last_clean":"2024-04-25T08:44:16.319381+0000","last_became_active":"2024-04-25T08:44:14.853682+0000","last_became_peered":"2024-04-25T08:44:14.853682+0000","last_unstale":"2024-04-25T08:44:16.319381+0000","last_undegraded":"2024-04-25T08:44:16.319381+0000","last_fullsized":"2024-04-25T08:44:16.319381+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:54.441005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319524+0000","last_change":"2024-04-25T08:44:14.867724+0000","last_active":"2024-04-25T08:44:16.319524+0000","last_peered":"2024-04-25T08:44:16.319524+0000","last_clean":"2024-04-25T08:44:16.319524+0000","last_became_active":"2024-04-25T08:44:14.867478+0000","last_became_peered":"2024-04-25T08:44:14.867478+0000","last_unstale":"2024-04-25T08:44:16.319524+0000","last_undegraded":"2024-04-25T08:44:16.319524+0000","last_fullsized":"2024-04-25T08:44:16.319524+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:25.078541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834497+0000","last_change":"2024-04-25T08:44:14.854352+0000","last_active":"2024-04-25T08:44:15.834497+0000","last_peered":"2024-04-25T08:44:15.834497+0000","last_clean":"2024-04-25T08:44:15.834497+0000","last_became_active":"2024-04-25T08:44:14.854089+0000","last_became_peered":"2024-04-25T08:44:14.854089+0000","last_unstale":"2024-04-25T08:44:15.834497+0000","last_undegraded":"2024-04-25T08:44:15.834497+0000","last_fullsized":"2024-04-25T08:44:15.834497+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:02.535112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319097+0000","last_change":"2024-04-25T08:44:14.859492+0000","last_active":"2024-04-25T08:44:16.319097+0000","last_peered":"2024-04-25T08:44:16.319097+0000","last_clean":"2024-04-25T08:44:16.319097+0000","last_became_active":"2024-04-25T08:44:14.859265+0000","last_became_peered":"2024-04-25T08:44:14.859265+0000","last_unstale":"2024-04-25T08:44:16.319097+0000","last_undegraded":"2024-04-25T08:44:16.319097+0000","last_fullsized":"2024-04-25T08:44:16.319097+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:23.108998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319607+0000","last_change":"2024-04-25T08:44:14.858476+0000","last_active":"2024-04-25T08:44:16.319607+0000","last_peered":"2024-04-25T08:44:16.319607+0000","last_clean":"2024-04-25T08:44:16.319607+0000","last_became_active":"2024-04-25T08:44:14.858099+0000","last_became_peered":"2024-04-25T08:44:14.858099+0000","last_unstale":"2024-04-25T08:44:16.319607+0000","last_undegraded":"2024-04-25T08:44:16.319607+0000","last_fullsized":"2024-04-25T08:44:16.319607+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:29.576117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833902+0000","last_change":"2024-04-25T08:44:14.853647+0000","last_active":"2024-04-25T08:44:15.833902+0000","last_peered":"2024-04-25T08:44:15.833902+0000","last_clean":"2024-04-25T08:44:15.833902+0000","last_became_active":"2024-04-25T08:44:14.853358+0000","last_became_peered":"2024-04-25T08:44:14.853358+0000","last_unstale":"2024-04-25T08:44:15.833902+0000","last_undegraded":"2024-04-25T08:44:15.833902+0000","last_fullsized":"2024-04-25T08:44:15.833902+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:06.020349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864675+0000","last_change":"2024-04-25T08:44:14.864541+0000","last_active":"2024-04-25T08:44:15.864675+0000","last_peered":"2024-04-25T08:44:15.864675+0000","last_clean":"2024-04-25T08:44:15.864675+0000","last_became_active":"2024-04-25T08:44:14.864269+0000","last_became_peered":"2024-04-25T08:44:14.864269+0000","last_unstale":"2024-04-25T08:44:15.864675+0000","last_undegraded":"2024-04-25T08:44:15.864675+0000","last_fullsized":"2024-04-25T08:44:15.864675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.707169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867170+0000","last_change":"2024-04-25T08:44:14.860298+0000","last_active":"2024-04-25T08:44:15.867170+0000","last_peered":"2024-04-25T08:44:15.867170+0000","last_clean":"2024-04-25T08:44:15.867170+0000","last_became_active":"2024-04-25T08:44:14.860094+0000","last_became_peered":"2024-04-25T08:44:14.860094+0000","last_unstale":"2024-04-25T08:44:15.867170+0000","last_undegraded":"2024-04-25T08:44:15.867170+0000","last_fullsized":"2024-04-25T08:44:15.867170+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:06.020349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833682+0000","last_change":"2024-04-25T08:44:14.847970+0000","last_active":"2024-04-25T08:44:15.833682+0000","last_peered":"2024-04-25T08:44:15.833682+0000","last_clean":"2024-04-25T08:44:15.833682+0000","last_became_active":"2024-04-25T08:44:14.847662+0000","last_became_peered":"2024-04-25T08:44:14.847662+0000","last_unstale":"2024-04-25T08:44:15.833682+0000","last_undegraded":"2024-04-25T08:44:15.833682+0000","last_fullsized":"2024-04-25T08:44:15.833682+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:30.256964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865251+0000","last_change":"2024-04-25T08:44:14.872220+0000","last_active":"2024-04-25T08:44:15.865251+0000","last_peered":"2024-04-25T08:44:15.865251+0000","last_clean":"2024-04-25T08:44:15.865251+0000","last_became_active":"2024-04-25T08:44:14.862940+0000","last_became_peered":"2024-04-25T08:44:14.862940+0000","last_unstale":"2024-04-25T08:44:15.865251+0000","last_undegraded":"2024-04-25T08:44:15.865251+0000","last_fullsized":"2024-04-25T08:44:15.865251+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:38.860125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867583+0000","last_change":"2024-04-25T08:44:14.865300+0000","last_active":"2024-04-25T08:44:15.867583+0000","last_peered":"2024-04-25T08:44:15.867583+0000","last_clean":"2024-04-25T08:44:15.867583+0000","last_became_active":"2024-04-25T08:44:14.865051+0000","last_became_peered":"2024-04-25T08:44:14.865051+0000","last_unstale":"2024-04-25T08:44:15.867583+0000","last_undegraded":"2024-04-25T08:44:15.867583+0000","last_fullsized":"2024-04-25T08:44:15.867583+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:34.481308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865000+0000","last_change":"2024-04-25T08:44:14.872768+0000","last_active":"2024-04-25T08:44:15.865000+0000","last_peered":"2024-04-25T08:44:15.865000+0000","last_clean":"2024-04-25T08:44:15.865000+0000","last_became_active":"2024-04-25T08:44:14.871620+0000","last_became_peered":"2024-04-25T08:44:14.871620+0000","last_unstale":"2024-04-25T08:44:15.865000+0000","last_undegraded":"2024-04-25T08:44:15.865000+0000","last_fullsized":"2024-04-25T08:44:15.865000+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:25.078541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835111+0000","last_change":"2024-04-25T08:44:14.862318+0000","last_active":"2024-04-25T08:44:15.835111+0000","last_peered":"2024-04-25T08:44:15.835111+0000","last_clean":"2024-04-25T08:44:15.835111+0000","last_became_active":"2024-04-25T08:44:14.861429+0000","last_became_peered":"2024-04-25T08:44:14.861429+0000","last_unstale":"2024-04-25T08:44:15.835111+0000","last_undegraded":"2024-04-25T08:44:15.835111+0000","last_fullsized":"2024-04-25T08:44:15.835111+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:23.634635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835041+0000","last_change":"2024-04-25T08:44:14.854726+0000","last_active":"2024-04-25T08:44:15.835041+0000","last_peered":"2024-04-25T08:44:15.835041+0000","last_clean":"2024-04-25T08:44:15.835041+0000","last_became_active":"2024-04-25T08:44:14.854465+0000","last_became_peered":"2024-04-25T08:44:14.854465+0000","last_unstale":"2024-04-25T08:44:15.835041+0000","last_undegraded":"2024-04-25T08:44:15.835041+0000","last_fullsized":"2024-04-25T08:44:15.835041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.335902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319068+0000","last_change":"2024-04-25T08:44:14.857014+0000","last_active":"2024-04-25T08:44:16.319068+0000","last_peered":"2024-04-25T08:44:16.319068+0000","last_clean":"2024-04-25T08:44:16.319068+0000","last_became_active":"2024-04-25T08:44:14.856677+0000","last_became_peered":"2024-04-25T08:44:14.856677+0000","last_unstale":"2024-04-25T08:44:16.319068+0000","last_undegraded":"2024-04-25T08:44:16.319068+0000","last_fullsized":"2024-04-25T08:44:16.319068+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.707169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.318972+0000","last_change":"2024-04-25T08:44:14.857256+0000","last_active":"2024-04-25T08:44:16.318972+0000","last_peered":"2024-04-25T08:44:16.318972+0000","last_clean":"2024-04-25T08:44:16.318972+0000","last_became_active":"2024-04-25T08:44:14.857005+0000","last_became_peered":"2024-04-25T08:44:14.857005+0000","last_unstale":"2024-04-25T08:44:16.318972+0000","last_undegraded":"2024-04-25T08:44:16.318972+0000","last_fullsized":"2024-04-25T08:44:16.318972+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:11.272258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864455+0000","last_change":"2024-04-25T08:44:14.853262+0000","last_active":"2024-04-25T08:44:15.864455+0000","last_peered":"2024-04-25T08:44:15.864455+0000","last_clean":"2024-04-25T08:44:15.864455+0000","last_became_active":"2024-04-25T08:44:14.852958+0000","last_became_peered":"2024-04-25T08:44:14.852958+0000","last_unstale":"2024-04-25T08:44:15.864455+0000","last_undegraded":"2024-04-25T08:44:15.864455+0000","last_fullsized":"2024-04-25T08:44:15.864455+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:19.081585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865255+0000","last_change":"2024-04-25T08:44:14.872344+0000","last_active":"2024-04-25T08:44:15.865255+0000","last_peered":"2024-04-25T08:44:15.865255+0000","last_clean":"2024-04-25T08:44:15.865255+0000","last_became_active":"2024-04-25T08:44:14.859356+0000","last_became_peered":"2024-04-25T08:44:14.859356+0000","last_unstale":"2024-04-25T08:44:15.865255+0000","last_undegraded":"2024-04-25T08:44:15.865255+0000","last_fullsized":"2024-04-25T08:44:15.865255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:02.277771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319765+0000","last_change":"2024-04-25T08:44:14.843823+0000","last_active":"2024-04-25T08:44:16.319765+0000","last_peered":"2024-04-25T08:44:16.319765+0000","last_clean":"2024-04-25T08:44:16.319765+0000","last_became_active":"2024-04-25T08:44:14.843554+0000","last_became_peered":"2024-04-25T08:44:14.843554+0000","last_unstale":"2024-04-25T08:44:16.319765+0000","last_undegraded":"2024-04-25T08:44:16.319765+0000","last_fullsized":"2024-04-25T08:44:16.319765+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:27.767834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834332+0000","last_change":"2024-04-25T08:44:14.873232+0000","last_active":"2024-04-25T08:44:15.834332+0000","last_peered":"2024-04-25T08:44:15.834332+0000","last_clean":"2024-04-25T08:44:15.834332+0000","last_became_active":"2024-04-25T08:44:14.870478+0000","last_became_peered":"2024-04-25T08:44:14.870478+0000","last_unstale":"2024-04-25T08:44:15.834332+0000","last_undegraded":"2024-04-25T08:44:15.834332+0000","last_fullsized":"2024-04-25T08:44:15.834332+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835156+0000","last_change":"2024-04-25T08:44:14.858192+0000","last_active":"2024-04-25T08:44:15.835156+0000","last_peered":"2024-04-25T08:44:15.835156+0000","last_clean":"2024-04-25T08:44:15.835156+0000","last_became_active":"2024-04-25T08:44:14.857938+0000","last_became_peered":"2024-04-25T08:44:14.857938+0000","last_unstale":"2024-04-25T08:44:15.835156+0000","last_undegraded":"2024-04-25T08:44:15.835156+0000","last_fullsized":"2024-04-25T08:44:15.835156+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:42.271056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864726+0000","last_change":"2024-04-25T08:44:14.857342+0000","last_active":"2024-04-25T08:44:15.864726+0000","last_peered":"2024-04-25T08:44:15.864726+0000","last_clean":"2024-04-25T08:44:15.864726+0000","last_became_active":"2024-04-25T08:44:14.857003+0000","last_became_peered":"2024-04-25T08:44:14.857003+0000","last_unstale":"2024-04-25T08:44:15.864726+0000","last_undegraded":"2024-04-25T08:44:15.864726+0000","last_fullsized":"2024-04-25T08:44:15.864726+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:54.441005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867340+0000","last_change":"2024-04-25T08:44:14.862768+0000","last_active":"2024-04-25T08:44:15.867340+0000","last_peered":"2024-04-25T08:44:15.867340+0000","last_clean":"2024-04-25T08:44:15.867340+0000","last_became_active":"2024-04-25T08:44:14.862384+0000","last_became_peered":"2024-04-25T08:44:14.862384+0000","last_unstale":"2024-04-25T08:44:15.867340+0000","last_undegraded":"2024-04-25T08:44:15.867340+0000","last_fullsized":"2024-04-25T08:44:15.867340+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864530+0000","last_change":"2024-04-25T08:44:14.865687+0000","last_active":"2024-04-25T08:44:15.864530+0000","last_peered":"2024-04-25T08:44:15.864530+0000","last_clean":"2024-04-25T08:44:15.864530+0000","last_became_active":"2024-04-25T08:44:14.865004+0000","last_became_peered":"2024-04-25T08:44:14.865004+0000","last_unstale":"2024-04-25T08:44:15.864530+0000","last_undegraded":"2024-04-25T08:44:15.864530+0000","last_fullsized":"2024-04-25T08:44:15.864530+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:26.314438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834186+0000","last_change":"2024-04-25T08:44:14.846903+0000","last_active":"2024-04-25T08:44:15.834186+0000","last_peered":"2024-04-25T08:44:15.834186+0000","last_clean":"2024-04-25T08:44:15.834186+0000","last_became_active":"2024-04-25T08:44:14.846530+0000","last_became_peered":"2024-04-25T08:44:14.846530+0000","last_unstale":"2024-04-25T08:44:15.834186+0000","last_undegraded":"2024-04-25T08:44:15.834186+0000","last_fullsized":"2024-04-25T08:44:15.834186+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:35.493623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.866935+0000","last_change":"2024-04-25T08:44:14.864066+0000","last_active":"2024-04-25T08:44:15.866935+0000","last_peered":"2024-04-25T08:44:15.866935+0000","last_clean":"2024-04-25T08:44:15.866935+0000","last_became_active":"2024-04-25T08:44:14.863817+0000","last_became_peered":"2024-04-25T08:44:14.863817+0000","last_unstale":"2024-04-25T08:44:15.866935+0000","last_undegraded":"2024-04-25T08:44:15.866935+0000","last_fullsized":"2024-04-25T08:44:15.866935+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:42.806945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835269+0000","last_change":"2024-04-25T08:44:13.832203+0000","last_active":"2024-04-25T08:44:15.835269+0000","last_peered":"2024-04-25T08:44:15.835269+0000","last_clean":"2024-04-25T08:44:15.835269+0000","last_became_active":"2024-04-25T08:44:13.831889+0000","last_became_peered":"2024-04-25T08:44:13.831889+0000","last_unstale":"2024-04-25T08:44:15.835269+0000","last_undegraded":"2024-04-25T08:44:15.835269+0000","last_fullsized":"2024-04-25T08:44:15.835269+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:46.675483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834271+0000","last_change":"2024-04-25T08:44:14.861591+0000","last_active":"2024-04-25T08:44:15.834271+0000","last_peered":"2024-04-25T08:44:15.834271+0000","last_clean":"2024-04-25T08:44:15.834271+0000","last_became_active":"2024-04-25T08:44:14.861186+0000","last_became_peered":"2024-04-25T08:44:14.861186+0000","last_unstale":"2024-04-25T08:44:15.834271+0000","last_undegraded":"2024-04-25T08:44:15.834271+0000","last_fullsized":"2024-04-25T08:44:15.834271+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:02.535112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834313+0000","last_change":"2024-04-25T08:44:13.840737+0000","last_active":"2024-04-25T08:44:15.834313+0000","last_peered":"2024-04-25T08:44:15.834313+0000","last_clean":"2024-04-25T08:44:15.834313+0000","last_became_active":"2024-04-25T08:44:13.840436+0000","last_became_peered":"2024-04-25T08:44:13.840436+0000","last_unstale":"2024-04-25T08:44:15.834313+0000","last_undegraded":"2024-04-25T08:44:15.834313+0000","last_fullsized":"2024-04-25T08:44:15.834313+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:40.924903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834893+0000","last_change":"2024-04-25T08:44:14.857569+0000","last_active":"2024-04-25T08:44:15.834893+0000","last_peered":"2024-04-25T08:44:15.834893+0000","last_clean":"2024-04-25T08:44:15.834893+0000","last_became_active":"2024-04-25T08:44:14.857289+0000","last_became_peered":"2024-04-25T08:44:14.857289+0000","last_unstale":"2024-04-25T08:44:15.834893+0000","last_undegraded":"2024-04-25T08:44:15.834893+0000","last_fullsized":"2024-04-25T08:44:15.834893+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:48.314480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834457+0000","last_change":"2024-04-25T08:44:13.836577+0000","last_active":"2024-04-25T08:44:15.834457+0000","last_peered":"2024-04-25T08:44:15.834457+0000","last_clean":"2024-04-25T08:44:15.834457+0000","last_became_active":"2024-04-25T08:44:13.836262+0000","last_became_peered":"2024-04-25T08:44:13.836262+0000","last_unstale":"2024-04-25T08:44:15.834457+0000","last_undegraded":"2024-04-25T08:44:15.834457+0000","last_fullsized":"2024-04-25T08:44:15.834457+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.410453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.318884+0000","last_change":"2024-04-25T08:44:14.845059+0000","last_active":"2024-04-25T08:44:16.318884+0000","last_peered":"2024-04-25T08:44:16.318884+0000","last_clean":"2024-04-25T08:44:16.318884+0000","last_became_active":"2024-04-25T08:44:14.844799+0000","last_became_peered":"2024-04-25T08:44:14.844799+0000","last_unstale":"2024-04-25T08:44:16.318884+0000","last_undegraded":"2024-04-25T08:44:16.318884+0000","last_fullsized":"2024-04-25T08:44:16.318884+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:26.314438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834466+0000","last_change":"2024-04-25T08:44:13.830937+0000","last_active":"2024-04-25T08:44:15.834466+0000","last_peered":"2024-04-25T08:44:15.834466+0000","last_clean":"2024-04-25T08:44:15.834466+0000","last_became_active":"2024-04-25T08:44:13.830630+0000","last_became_peered":"2024-04-25T08:44:13.830630+0000","last_unstale":"2024-04-25T08:44:15.834466+0000","last_undegraded":"2024-04-25T08:44:15.834466+0000","last_fullsized":"2024-04-25T08:44:15.834466+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.329695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317907+0000","last_change":"2024-04-25T08:44:14.857454+0000","last_active":"2024-04-25T08:44:16.317907+0000","last_peered":"2024-04-25T08:44:16.317907+0000","last_clean":"2024-04-25T08:44:16.317907+0000","last_became_active":"2024-04-25T08:44:14.857216+0000","last_became_peered":"2024-04-25T08:44:14.857216+0000","last_unstale":"2024-04-25T08:44:16.317907+0000","last_undegraded":"2024-04-25T08:44:16.317907+0000","last_fullsized":"2024-04-25T08:44:16.317907+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:23.634635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864688+0000","last_change":"2024-04-25T08:44:13.832624+0000","last_active":"2024-04-25T08:44:15.864688+0000","last_peered":"2024-04-25T08:44:15.864688+0000","last_clean":"2024-04-25T08:44:15.864688+0000","last_became_active":"2024-04-25T08:44:13.832310+0000","last_became_peered":"2024-04-25T08:44:13.832310+0000","last_unstale":"2024-04-25T08:44:15.864688+0000","last_undegraded":"2024-04-25T08:44:15.864688+0000","last_fullsized":"2024-04-25T08:44:15.864688+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:15.487629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867430+0000","last_change":"2024-04-25T08:44:14.863226+0000","last_active":"2024-04-25T08:44:15.867430+0000","last_peered":"2024-04-25T08:44:15.867430+0000","last_clean":"2024-04-25T08:44:15.867430+0000","last_became_active":"2024-04-25T08:44:14.862726+0000","last_became_peered":"2024-04-25T08:44:14.862726+0000","last_unstale":"2024-04-25T08:44:15.867430+0000","last_undegraded":"2024-04-25T08:44:15.867430+0000","last_fullsized":"2024-04-25T08:44:15.867430+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317632+0000","last_change":"2024-04-25T08:44:13.829202+0000","last_active":"2024-04-25T08:44:16.317632+0000","last_peered":"2024-04-25T08:44:16.317632+0000","last_clean":"2024-04-25T08:44:16.317632+0000","last_became_active":"2024-04-25T08:44:13.828896+0000","last_became_peered":"2024-04-25T08:44:13.828896+0000","last_unstale":"2024-04-25T08:44:16.317632+0000","last_undegraded":"2024-04-25T08:44:16.317632+0000","last_fullsized":"2024-04-25T08:44:16.317632+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:40.924903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319918+0000","last_change":"2024-04-25T08:44:14.843658+0000","last_active":"2024-04-25T08:44:16.319918+0000","last_peered":"2024-04-25T08:44:16.319918+0000","last_clean":"2024-04-25T08:44:16.319918+0000","last_became_active":"2024-04-25T08:44:14.843203+0000","last_became_peered":"2024-04-25T08:44:14.843203+0000","last_unstale":"2024-04-25T08:44:16.319918+0000","last_undegraded":"2024-04-25T08:44:16.319918+0000","last_fullsized":"2024-04-25T08:44:16.319918+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:38.860125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834857+0000","last_change":"2024-04-25T08:44:13.833544+0000","last_active":"2024-04-25T08:44:15.834857+0000","last_peered":"2024-04-25T08:44:15.834857+0000","last_clean":"2024-04-25T08:44:15.834857+0000","last_became_active":"2024-04-25T08:44:13.833024+0000","last_became_peered":"2024-04-25T08:44:13.833024+0000","last_unstale":"2024-04-25T08:44:15.834857+0000","last_undegraded":"2024-04-25T08:44:15.834857+0000","last_fullsized":"2024-04-25T08:44:15.834857+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.410453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867536+0000","last_change":"2024-04-25T08:44:14.865485+0000","last_active":"2024-04-25T08:44:15.867536+0000","last_peered":"2024-04-25T08:44:15.867536+0000","last_clean":"2024-04-25T08:44:15.867536+0000","last_became_active":"2024-04-25T08:44:14.865231+0000","last_became_peered":"2024-04-25T08:44:14.865231+0000","last_unstale":"2024-04-25T08:44:15.867536+0000","last_undegraded":"2024-04-25T08:44:15.867536+0000","last_fullsized":"2024-04-25T08:44:15.867536+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:35.493623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864474+0000","last_change":"2024-04-25T08:44:13.831048+0000","last_active":"2024-04-25T08:44:15.864474+0000","last_peered":"2024-04-25T08:44:15.864474+0000","last_clean":"2024-04-25T08:44:15.864474+0000","last_became_active":"2024-04-25T08:44:13.830737+0000","last_became_peered":"2024-04-25T08:44:13.830737+0000","last_unstale":"2024-04-25T08:44:15.864474+0000","last_undegraded":"2024-04-25T08:44:15.864474+0000","last_fullsized":"2024-04-25T08:44:15.864474+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.933448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317555+0000","last_change":"2024-04-25T08:44:14.852785+0000","last_active":"2024-04-25T08:44:16.317555+0000","last_peered":"2024-04-25T08:44:16.317555+0000","last_clean":"2024-04-25T08:44:16.317555+0000","last_became_active":"2024-04-25T08:44:14.852472+0000","last_became_peered":"2024-04-25T08:44:14.852472+0000","last_unstale":"2024-04-25T08:44:16.317555+0000","last_undegraded":"2024-04-25T08:44:16.317555+0000","last_fullsized":"2024-04-25T08:44:16.317555+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:08.143188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867450+0000","last_change":"2024-04-25T08:44:13.828624+0000","last_active":"2024-04-25T08:44:15.867450+0000","last_peered":"2024-04-25T08:44:15.867450+0000","last_clean":"2024-04-25T08:44:15.867450+0000","last_became_active":"2024-04-25T08:44:13.828054+0000","last_became_peered":"2024-04-25T08:44:13.828054+0000","last_unstale":"2024-04-25T08:44:15.867450+0000","last_undegraded":"2024-04-25T08:44:15.867450+0000","last_fullsized":"2024-04-25T08:44:15.867450+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.933448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834104+0000","last_change":"2024-04-25T08:44:14.854260+0000","last_active":"2024-04-25T08:44:15.834104+0000","last_peered":"2024-04-25T08:44:15.834104+0000","last_clean":"2024-04-25T08:44:15.834104+0000","last_became_active":"2024-04-25T08:44:14.853525+0000","last_became_peered":"2024-04-25T08:44:14.853525+0000","last_unstale":"2024-04-25T08:44:15.834104+0000","last_undegraded":"2024-04-25T08:44:15.834104+0000","last_fullsized":"2024-04-25T08:44:15.834104+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:52.173685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317918+0000","last_change":"2024-04-25T08:44:13.834681+0000","last_active":"2024-04-25T08:44:16.317918+0000","last_peered":"2024-04-25T08:44:16.317918+0000","last_clean":"2024-04-25T08:44:16.317918+0000","last_became_active":"2024-04-25T08:44:13.834347+0000","last_became_peered":"2024-04-25T08:44:13.834347+0000","last_unstale":"2024-04-25T08:44:16.317918+0000","last_undegraded":"2024-04-25T08:44:16.317918+0000","last_fullsized":"2024-04-25T08:44:16.317918+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:23.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864694+0000","last_change":"2024-04-25T08:44:14.857952+0000","last_active":"2024-04-25T08:44:15.864694+0000","last_peered":"2024-04-25T08:44:15.864694+0000","last_clean":"2024-04-25T08:44:15.864694+0000","last_became_active":"2024-04-25T08:44:14.857694+0000","last_became_peered":"2024-04-25T08:44:14.857694+0000","last_unstale":"2024-04-25T08:44:15.864694+0000","last_undegraded":"2024-04-25T08:44:15.864694+0000","last_fullsized":"2024-04-25T08:44:15.864694+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.693702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867497+0000","last_change":"2024-04-25T08:44:13.839415+0000","last_active":"2024-04-25T08:44:15.867497+0000","last_peered":"2024-04-25T08:44:15.867497+0000","last_clean":"2024-04-25T08:44:15.867497+0000","last_became_active":"2024-04-25T08:44:13.839068+0000","last_became_peered":"2024-04-25T08:44:13.839068+0000","last_unstale":"2024-04-25T08:44:15.867497+0000","last_undegraded":"2024-04-25T08:44:15.867497+0000","last_fullsized":"2024-04-25T08:44:15.867497+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:20.939359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317927+0000","last_change":"2024-04-25T08:44:14.859416+0000","last_active":"2024-04-25T08:44:16.317927+0000","last_peered":"2024-04-25T08:44:16.317927+0000","last_clean":"2024-04-25T08:44:16.317927+0000","last_became_active":"2024-04-25T08:44:14.859112+0000","last_became_peered":"2024-04-25T08:44:14.859112+0000","last_unstale":"2024-04-25T08:44:16.317927+0000","last_undegraded":"2024-04-25T08:44:16.317927+0000","last_fullsized":"2024-04-25T08:44:16.317927+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:42.271056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319605+0000","last_change":"2024-04-25T08:44:13.822261+0000","last_active":"2024-04-25T08:44:16.319605+0000","last_peered":"2024-04-25T08:44:16.319605+0000","last_clean":"2024-04-25T08:44:16.319605+0000","last_became_active":"2024-04-25T08:44:13.821468+0000","last_became_peered":"2024-04-25T08:44:13.821468+0000","last_unstale":"2024-04-25T08:44:16.319605+0000","last_undegraded":"2024-04-25T08:44:16.319605+0000","last_fullsized":"2024-04-25T08:44:16.319605+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:05.014142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834139+0000","last_change":"2024-04-25T08:44:14.848680+0000","last_active":"2024-04-25T08:44:15.834139+0000","last_peered":"2024-04-25T08:44:15.834139+0000","last_clean":"2024-04-25T08:44:15.834139+0000","last_became_active":"2024-04-25T08:44:14.847901+0000","last_became_peered":"2024-04-25T08:44:14.847901+0000","last_unstale":"2024-04-25T08:44:15.834139+0000","last_undegraded":"2024-04-25T08:44:15.834139+0000","last_fullsized":"2024-04-25T08:44:15.834139+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864816+0000","last_change":"2024-04-25T08:44:13.832989+0000","last_active":"2024-04-25T08:44:15.864816+0000","last_peered":"2024-04-25T08:44:15.864816+0000","last_clean":"2024-04-25T08:44:15.864816+0000","last_became_active":"2024-04-25T08:44:13.832689+0000","last_became_peered":"2024-04-25T08:44:13.832689+0000","last_unstale":"2024-04-25T08:44:15.864816+0000","last_undegraded":"2024-04-25T08:44:15.864816+0000","last_fullsized":"2024-04-25T08:44:15.864816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:31.375263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865174+0000","last_change":"2024-04-25T08:44:14.863881+0000","last_active":"2024-04-25T08:44:15.865174+0000","last_peered":"2024-04-25T08:44:15.865174+0000","last_clean":"2024-04-25T08:44:15.865174+0000","last_became_active":"2024-04-25T08:44:14.859108+0000","last_became_peered":"2024-04-25T08:44:14.859108+0000","last_unstale":"2024-04-25T08:44:15.865174+0000","last_undegraded":"2024-04-25T08:44:15.865174+0000","last_fullsized":"2024-04-25T08:44:15.865174+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:48.153045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835103+0000","last_change":"2024-04-25T08:44:13.834677+0000","last_active":"2024-04-25T08:44:15.835103+0000","last_peered":"2024-04-25T08:44:15.835103+0000","last_clean":"2024-04-25T08:44:15.835103+0000","last_became_active":"2024-04-25T08:44:13.834384+0000","last_became_peered":"2024-04-25T08:44:13.834384+0000","last_unstale":"2024-04-25T08:44:15.835103+0000","last_undegraded":"2024-04-25T08:44:15.835103+0000","last_fullsized":"2024-04-25T08:44:15.835103+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:23.471644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319791+0000","last_change":"2024-04-25T08:44:14.861080+0000","last_active":"2024-04-25T08:44:16.319791+0000","last_peered":"2024-04-25T08:44:16.319791+0000","last_clean":"2024-04-25T08:44:16.319791+0000","last_became_active":"2024-04-25T08:44:14.860713+0000","last_became_peered":"2024-04-25T08:44:14.860713+0000","last_unstale":"2024-04-25T08:44:16.319791+0000","last_undegraded":"2024-04-25T08:44:16.319791+0000","last_fullsized":"2024-04-25T08:44:16.319791+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:48.153045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867262+0000","last_change":"2024-04-25T08:44:13.827274+0000","last_active":"2024-04-25T08:44:15.867262+0000","last_peered":"2024-04-25T08:44:15.867262+0000","last_clean":"2024-04-25T08:44:15.867262+0000","last_became_active":"2024-04-25T08:44:13.827020+0000","last_became_peered":"2024-04-25T08:44:13.827020+0000","last_unstale":"2024-04-25T08:44:15.867262+0000","last_undegraded":"2024-04-25T08:44:15.867262+0000","last_fullsized":"2024-04-25T08:44:15.867262+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:15:06.183174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865123+0000","last_change":"2024-04-25T08:44:14.873838+0000","last_active":"2024-04-25T08:44:15.865123+0000","last_peered":"2024-04-25T08:44:15.865123+0000","last_clean":"2024-04-25T08:44:15.865123+0000","last_became_active":"2024-04-25T08:44:14.873570+0000","last_became_peered":"2024-04-25T08:44:14.873570+0000","last_unstale":"2024-04-25T08:44:15.865123+0000","last_undegraded":"2024-04-25T08:44:15.865123+0000","last_fullsized":"2024-04-25T08:44:15.865123+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:27.767834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":22,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319534+0000","last_change":"2024-04-25T08:44:13.823663+0000","last_active":"2024-04-25T08:44:16.319534+0000","last_peered":"2024-04-25T08:44:16.319534+0000","last_clean":"2024-04-25T08:44:16.319534+0000","last_became_active":"2024-04-25T08:44:13.822970+0000","last_became_peered":"2024-04-25T08:44:13.822970+0000","last_unstale":"2024-04-25T08:44:16.319534+0000","last_undegraded":"2024-04-25T08:44:16.319534+0000","last_fullsized":"2024-04-25T08:44:16.319534+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:57.170270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319401+0000","last_change":"2024-04-25T08:44:14.847871+0000","last_active":"2024-04-25T08:44:16.319401+0000","last_peered":"2024-04-25T08:44:16.319401+0000","last_clean":"2024-04-25T08:44:16.319401+0000","last_became_active":"2024-04-25T08:44:14.847511+0000","last_became_peered":"2024-04-25T08:44:14.847511+0000","last_unstale":"2024-04-25T08:44:16.319401+0000","last_undegraded":"2024-04-25T08:44:16.319401+0000","last_fullsized":"2024-04-25T08:44:16.319401+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:36.380477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865340+0000","last_change":"2024-04-25T08:44:13.831815+0000","last_active":"2024-04-25T08:44:15.865340+0000","last_peered":"2024-04-25T08:44:15.865340+0000","last_clean":"2024-04-25T08:44:15.865340+0000","last_became_active":"2024-04-25T08:44:13.831549+0000","last_became_peered":"2024-04-25T08:44:13.831549+0000","last_unstale":"2024-04-25T08:44:15.865340+0000","last_undegraded":"2024-04-25T08:44:15.865340+0000","last_fullsized":"2024-04-25T08:44:15.865340+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:07.136981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833701+0000","last_change":"2024-04-25T08:44:14.856849+0000","last_active":"2024-04-25T08:44:15.833701+0000","last_peered":"2024-04-25T08:44:15.833701+0000","last_clean":"2024-04-25T08:44:15.833701+0000","last_became_active":"2024-04-25T08:44:14.856511+0000","last_became_peered":"2024-04-25T08:44:14.856511+0000","last_unstale":"2024-04-25T08:44:15.833701+0000","last_undegraded":"2024-04-25T08:44:15.833701+0000","last_fullsized":"2024-04-25T08:44:15.833701+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:58.176477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319434+0000","last_change":"2024-04-25T08:44:13.827580+0000","last_active":"2024-04-25T08:44:16.319434+0000","last_peered":"2024-04-25T08:44:16.319434+0000","last_clean":"2024-04-25T08:44:16.319434+0000","last_became_active":"2024-04-25T08:44:13.827081+0000","last_became_peered":"2024-04-25T08:44:13.827081+0000","last_unstale":"2024-04-25T08:44:16.319434+0000","last_undegraded":"2024-04-25T08:44:16.319434+0000","last_fullsized":"2024-04-25T08:44:16.319434+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.047776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317593+0000","last_change":"2024-04-25T08:44:14.874041+0000","last_active":"2024-04-25T08:44:16.317593+0000","last_peered":"2024-04-25T08:44:16.317593+0000","last_clean":"2024-04-25T08:44:16.317593+0000","last_became_active":"2024-04-25T08:44:14.872918+0000","last_became_peered":"2024-04-25T08:44:14.872918+0000","last_unstale":"2024-04-25T08:44:16.317593+0000","last_undegraded":"2024-04-25T08:44:16.317593+0000","last_fullsized":"2024-04-25T08:44:16.317593+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:48.314480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835597+0000","last_change":"2024-04-25T08:44:13.836322+0000","last_active":"2024-04-25T08:44:15.835597+0000","last_peered":"2024-04-25T08:44:15.835597+0000","last_clean":"2024-04-25T08:44:15.835597+0000","last_became_active":"2024-04-25T08:44:13.836019+0000","last_became_peered":"2024-04-25T08:44:13.836019+0000","last_unstale":"2024-04-25T08:44:15.835597+0000","last_undegraded":"2024-04-25T08:44:15.835597+0000","last_fullsized":"2024-04-25T08:44:15.835597+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:14.593484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834043+0000","last_change":"2024-04-25T08:44:14.845540+0000","last_active":"2024-04-25T08:44:15.834043+0000","last_peered":"2024-04-25T08:44:15.834043+0000","last_clean":"2024-04-25T08:44:15.834043+0000","last_became_active":"2024-04-25T08:44:14.845247+0000","last_became_peered":"2024-04-25T08:44:14.845247+0000","last_unstale":"2024-04-25T08:44:15.834043+0000","last_undegraded":"2024-04-25T08:44:15.834043+0000","last_fullsized":"2024-04-25T08:44:15.834043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:59.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835216+0000","last_change":"2024-04-25T08:44:13.826386+0000","last_active":"2024-04-25T08:44:15.835216+0000","last_peered":"2024-04-25T08:44:15.835216+0000","last_clean":"2024-04-25T08:44:15.835216+0000","last_became_active":"2024-04-25T08:44:13.826048+0000","last_became_peered":"2024-04-25T08:44:13.826048+0000","last_unstale":"2024-04-25T08:44:15.835216+0000","last_undegraded":"2024-04-25T08:44:15.835216+0000","last_fullsized":"2024-04-25T08:44:15.835216+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:59.861264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834074+0000","last_change":"2024-04-25T08:44:14.854319+0000","last_active":"2024-04-25T08:44:15.834074+0000","last_peered":"2024-04-25T08:44:15.834074+0000","last_clean":"2024-04-25T08:44:15.834074+0000","last_became_active":"2024-04-25T08:44:14.853844+0000","last_became_peered":"2024-04-25T08:44:14.853844+0000","last_unstale":"2024-04-25T08:44:15.834074+0000","last_undegraded":"2024-04-25T08:44:15.834074+0000","last_fullsized":"2024-04-25T08:44:15.834074+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:59.969200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834185+0000","last_change":"2024-04-25T08:44:13.836241+0000","last_active":"2024-04-25T08:44:15.834185+0000","last_peered":"2024-04-25T08:44:15.834185+0000","last_clean":"2024-04-25T08:44:15.834185+0000","last_became_active":"2024-04-25T08:44:13.835917+0000","last_became_peered":"2024-04-25T08:44:13.835917+0000","last_unstale":"2024-04-25T08:44:15.834185+0000","last_undegraded":"2024-04-25T08:44:15.834185+0000","last_fullsized":"2024-04-25T08:44:15.834185+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:50.768133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835630+0000","last_change":"2024-04-25T08:44:14.855917+0000","last_active":"2024-04-25T08:44:15.835630+0000","last_peered":"2024-04-25T08:44:15.835630+0000","last_clean":"2024-04-25T08:44:15.835630+0000","last_became_active":"2024-04-25T08:44:14.855652+0000","last_became_peered":"2024-04-25T08:44:14.855652+0000","last_unstale":"2024-04-25T08:44:15.835630+0000","last_undegraded":"2024-04-25T08:44:15.835630+0000","last_fullsized":"2024-04-25T08:44:15.835630+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:30.256964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319101+0000","last_change":"2024-04-25T08:44:13.817252+0000","last_active":"2024-04-25T08:44:16.319101+0000","last_peered":"2024-04-25T08:44:16.319101+0000","last_clean":"2024-04-25T08:44:16.319101+0000","last_became_active":"2024-04-25T08:44:13.816979+0000","last_became_peered":"2024-04-25T08:44:13.816979+0000","last_unstale":"2024-04-25T08:44:16.319101+0000","last_undegraded":"2024-04-25T08:44:16.319101+0000","last_fullsized":"2024-04-25T08:44:16.319101+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:29.250756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865051+0000","last_change":"2024-04-25T08:44:14.872100+0000","last_active":"2024-04-25T08:44:15.865051+0000","last_peered":"2024-04-25T08:44:15.865051+0000","last_clean":"2024-04-25T08:44:15.865051+0000","last_became_active":"2024-04-25T08:44:14.871824+0000","last_became_peered":"2024-04-25T08:44:14.871824+0000","last_unstale":"2024-04-25T08:44:15.865051+0000","last_undegraded":"2024-04-25T08:44:15.865051+0000","last_fullsized":"2024-04-25T08:44:15.865051+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:44.521500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864485+0000","last_change":"2024-04-25T08:44:13.838856+0000","last_active":"2024-04-25T08:44:15.864485+0000","last_peered":"2024-04-25T08:44:15.864485+0000","last_clean":"2024-04-25T08:44:15.864485+0000","last_became_active":"2024-04-25T08:44:13.838556+0000","last_became_peered":"2024-04-25T08:44:13.838556+0000","last_unstale":"2024-04-25T08:44:15.864485+0000","last_undegraded":"2024-04-25T08:44:15.864485+0000","last_fullsized":"2024-04-25T08:44:15.864485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:20.939359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834338+0000","last_change":"2024-04-25T08:44:14.840865+0000","last_active":"2024-04-25T08:44:15.834338+0000","last_peered":"2024-04-25T08:44:15.834338+0000","last_clean":"2024-04-25T08:44:15.834338+0000","last_became_active":"2024-04-25T08:44:14.840241+0000","last_became_peered":"2024-04-25T08:44:14.840241+0000","last_unstale":"2024-04-25T08:44:15.834338+0000","last_undegraded":"2024-04-25T08:44:15.834338+0000","last_fullsized":"2024-04-25T08:44:15.834338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:19.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319618+0000","last_change":"2024-04-25T08:44:13.821334+0000","last_active":"2024-04-25T08:44:16.319618+0000","last_peered":"2024-04-25T08:44:16.319618+0000","last_clean":"2024-04-25T08:44:16.319618+0000","last_became_active":"2024-04-25T08:44:13.821047+0000","last_became_peered":"2024-04-25T08:44:13.821047+0000","last_unstale":"2024-04-25T08:44:16.319618+0000","last_undegraded":"2024-04-25T08:44:16.319618+0000","last_fullsized":"2024-04-25T08:44:16.319618+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:38.347524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834290+0000","last_change":"2024-04-25T08:44:14.840948+0000","last_active":"2024-04-25T08:44:15.834290+0000","last_peered":"2024-04-25T08:44:15.834290+0000","last_clean":"2024-04-25T08:44:15.834290+0000","last_became_active":"2024-04-25T08:44:14.840622+0000","last_became_peered":"2024-04-25T08:44:14.840622+0000","last_unstale":"2024-04-25T08:44:15.834290+0000","last_undegraded":"2024-04-25T08:44:15.834290+0000","last_fullsized":"2024-04-25T08:44:15.834290+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:34.481308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.835346+0000","last_change":"2024-04-25T08:44:13.835312+0000","last_active":"2024-04-25T08:44:15.835346+0000","last_peered":"2024-04-25T08:44:15.835346+0000","last_clean":"2024-04-25T08:44:15.835346+0000","last_became_active":"2024-04-25T08:44:13.833473+0000","last_became_peered":"2024-04-25T08:44:13.833473+0000","last_unstale":"2024-04-25T08:44:15.835346+0000","last_undegraded":"2024-04-25T08:44:15.835346+0000","last_fullsized":"2024-04-25T08:44:15.835346+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:50.768133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.833790+0000","last_change":"2024-04-25T08:44:14.846108+0000","last_active":"2024-04-25T08:44:15.833790+0000","last_peered":"2024-04-25T08:44:15.833790+0000","last_clean":"2024-04-25T08:44:15.833790+0000","last_became_active":"2024-04-25T08:44:14.845602+0000","last_became_peered":"2024-04-25T08:44:14.845602+0000","last_unstale":"2024-04-25T08:44:15.833790+0000","last_undegraded":"2024-04-25T08:44:15.833790+0000","last_fullsized":"2024-04-25T08:44:15.833790+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:42.806945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.867536+0000","last_change":"2024-04-25T08:44:13.832138+0000","last_active":"2024-04-25T08:44:15.867536+0000","last_peered":"2024-04-25T08:44:15.867536+0000","last_clean":"2024-04-25T08:44:15.867536+0000","last_became_active":"2024-04-25T08:44:13.831825+0000","last_became_peered":"2024-04-25T08:44:13.831825+0000","last_unstale":"2024-04-25T08:44:15.867536+0000","last_undegraded":"2024-04-25T08:44:15.867536+0000","last_fullsized":"2024-04-25T08:44:15.867536+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:15.487629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319338+0000","last_change":"2024-04-25T08:44:14.854033+0000","last_active":"2024-04-25T08:44:16.319338+0000","last_peered":"2024-04-25T08:44:16.319338+0000","last_clean":"2024-04-25T08:44:16.319338+0000","last_became_active":"2024-04-25T08:44:14.853752+0000","last_became_peered":"2024-04-25T08:44:14.853752+0000","last_unstale":"2024-04-25T08:44:16.319338+0000","last_undegraded":"2024-04-25T08:44:16.319338+0000","last_fullsized":"2024-04-25T08:44:16.319338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:31.693702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864816+0000","last_change":"2024-04-25T08:44:13.828110+0000","last_active":"2024-04-25T08:44:15.864816+0000","last_peered":"2024-04-25T08:44:15.864816+0000","last_clean":"2024-04-25T08:44:15.864816+0000","last_became_active":"2024-04-25T08:44:13.827771+0000","last_became_peered":"2024-04-25T08:44:13.827771+0000","last_unstale":"2024-04-25T08:44:15.864816+0000","last_undegraded":"2024-04-25T08:44:15.864816+0000","last_fullsized":"2024-04-25T08:44:15.864816+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:47.308273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.864790+0000","last_change":"2024-04-25T08:44:14.856737+0000","last_active":"2024-04-25T08:44:15.864790+0000","last_peered":"2024-04-25T08:44:15.864790+0000","last_clean":"2024-04-25T08:44:15.864790+0000","last_became_active":"2024-04-25T08:44:14.856420+0000","last_became_peered":"2024-04-25T08:44:14.856420+0000","last_unstale":"2024-04-25T08:44:15.864790+0000","last_undegraded":"2024-04-25T08:44:15.864790+0000","last_fullsized":"2024-04-25T08:44:15.864790+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:36.380477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319439+0000","last_change":"2024-04-25T08:44:13.818771+0000","last_active":"2024-04-25T08:44:16.319439+0000","last_peered":"2024-04-25T08:44:16.319439+0000","last_clean":"2024-04-25T08:44:16.319439+0000","last_became_active":"2024-04-25T08:44:13.818399+0000","last_became_peered":"2024-04-25T08:44:13.818399+0000","last_unstale":"2024-04-25T08:44:16.319439+0000","last_undegraded":"2024-04-25T08:44:16.319439+0000","last_fullsized":"2024-04-25T08:44:16.319439+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:41.800738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834324+0000","last_change":"2024-04-25T08:44:14.855120+0000","last_active":"2024-04-25T08:44:15.834324+0000","last_peered":"2024-04-25T08:44:15.834324+0000","last_clean":"2024-04-25T08:44:15.834324+0000","last_became_active":"2024-04-25T08:44:14.854759+0000","last_became_peered":"2024-04-25T08:44:14.854759+0000","last_unstale":"2024-04-25T08:44:15.834324+0000","last_undegraded":"2024-04-25T08:44:15.834324+0000","last_fullsized":"2024-04-25T08:44:15.834324+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:45.942204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319171+0000","last_change":"2024-04-25T08:44:13.838954+0000","last_active":"2024-04-25T08:44:16.319171+0000","last_peered":"2024-04-25T08:44:16.319171+0000","last_clean":"2024-04-25T08:44:16.319171+0000","last_became_active":"2024-04-25T08:44:13.838640+0000","last_became_peered":"2024-04-25T08:44:13.838640+0000","last_unstale":"2024-04-25T08:44:16.319171+0000","last_undegraded":"2024-04-25T08:44:16.319171+0000","last_fullsized":"2024-04-25T08:44:16.319171+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:02.757277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.865418+0000","last_change":"2024-04-25T08:44:14.865620+0000","last_active":"2024-04-25T08:44:15.865418+0000","last_peered":"2024-04-25T08:44:15.865418+0000","last_clean":"2024-04-25T08:44:15.865418+0000","last_became_active":"2024-04-25T08:44:14.865355+0000","last_became_peered":"2024-04-25T08:44:14.865355+0000","last_unstale":"2024-04-25T08:44:15.865418+0000","last_undegraded":"2024-04-25T08:44:15.865418+0000","last_fullsized":"2024-04-25T08:44:15.865418+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:44.684383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.317511+0000","last_change":"2024-04-25T08:44:13.816637+0000","last_active":"2024-04-25T08:44:16.317511+0000","last_peered":"2024-04-25T08:44:16.317511+0000","last_clean":"2024-04-25T08:44:16.317511+0000","last_became_active":"2024-04-25T08:44:13.816317+0000","last_became_peered":"2024-04-25T08:44:13.816317+0000","last_unstale":"2024-04-25T08:44:16.317511+0000","last_undegraded":"2024-04-25T08:44:16.317511+0000","last_fullsized":"2024-04-25T08:44:16.317511+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:25.410453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:16.319741+0000","last_change":"2024-04-25T08:44:14.842679+0000","last_active":"2024-04-25T08:44:16.319741+0000","last_peered":"2024-04-25T08:44:16.319741+0000","last_clean":"2024-04-25T08:44:16.319741+0000","last_became_active":"2024-04-25T08:44:14.842301+0000","last_became_peered":"2024-04-25T08:44:14.842301+0000","last_unstale":"2024-04-25T08:44:16.319741+0000","last_undegraded":"2024-04-25T08:44:16.319741+0000","last_fullsized":"2024-04-25T08:44:16.319741+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:44.521500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834318+0000","last_change":"2024-04-25T08:44:13.840482+0000","last_active":"2024-04-25T08:44:15.834318+0000","last_peered":"2024-04-25T08:44:15.834318+0000","last_clean":"2024-04-25T08:44:15.834318+0000","last_became_active":"2024-04-25T08:44:13.840204+0000","last_became_peered":"2024-04-25T08:44:13.840204+0000","last_unstale":"2024-04-25T08:44:15.834318+0000","last_undegraded":"2024-04-25T08:44:15.834318+0000","last_fullsized":"2024-04-25T08:44:15.834318+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:47.308273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.834893+0000","last_change":"2024-04-25T08:44:14.853120+0000","last_active":"2024-04-25T08:44:15.834893+0000","last_peered":"2024-04-25T08:44:15.834893+0000","last_clean":"2024-04-25T08:44:15.834893+0000","last_became_active":"2024-04-25T08:44:14.852845+0000","last_became_peered":"2024-04-25T08:44:14.852845+0000","last_unstale":"2024-04-25T08:44:15.834893+0000","last_undegraded":"2024-04-25T08:44:15.834893+0000","last_fullsized":"2024-04-25T08:44:15.834893+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:13.804543+0000","last_clean_scrub_stamp":"2024-04-25T08:44:13.804543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:08.143188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.866983+0000","last_change":"2024-04-25T08:44:13.832756+0000","last_active":"2024-04-25T08:44:15.866983+0000","last_peered":"2024-04-25T08:44:15.866983+0000","last_clean":"2024-04-25T08:44:15.866983+0000","last_became_active":"2024-04-25T08:44:13.832461+0000","last_became_peered":"2024-04-25T08:44:13.832461+0000","last_unstale":"2024-04-25T08:44:15.866983+0000","last_undegraded":"2024-04-25T08:44:15.866983+0000","last_fullsized":"2024-04-25T08:44:15.866983+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:44:12.798336+0000","last_clean_scrub_stamp":"2024-04-25T08:44:12.798336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.329695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":230,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:44:15.866941+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:44:15.866941+0000","last_peered":"2024-04-25T08:44:15.866941+0000","last_clean":"2024-04-25T08:44:15.866941+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:44:15.866941+0000","last_undegraded":"2024-04-25T08:44:15.866941+0000","last_fullsized":"2024-04-25T08:44:15.866941+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343612,"statfs":{"total":96636764160,"available":96607858688,"internally_reserved":0,"allocated":1232896,"data_stored":871989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":638976,"data_stored":282149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":281497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":622592,"data_stored":275387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":647168,"data_stored":283131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":643072,"data_stored":283109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31808,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":638976,"data_stored":281657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":871921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:44:19.340 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:44:19.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:44:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.530+0000 7fb809a00640 1 Processor -- start 2024-04-25T08:44:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.530+0000 7fb809a00640 1 -- start start 2024-04-25T08:44:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.530+0000 7fb809a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8041440d0 0x7fb8040cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.530+0000 7fb809a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb80406f150 0x7fb8040b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.530+0000 7fb809a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80415e440 con 0x7fb804143990 2024-04-25T08:44:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.530+0000 7fb809a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80415ed30 con 0x7fb80406f150 2024-04-25T08:44:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.530+0000 7fb809a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415e6e0 con 0x7fb8041440d0 2024-04-25T08:44:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.530+0000 7fb802a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb804143990 0x7fb80406ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46310/0 (socket says 172.21.15.46:46310) 2024-04-25T08:44:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.530+0000 7fb802a00640 1 -- 172.21.15.46:0/2719676861 learned_addr learned my addr 172.21.15.46:0/2719676861 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb803400640 1 --2- 172.21.15.46:0/2719676861 >> v2:172.21.15.82:3300/0 conn(0x7fb80406f150 0x7fb8040b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb7fa000640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8041440d0 0x7fb8040cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305902488 0 0) 0x7fb80415e440 con 0x7fb804143990 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7e8003530 con 0x7fb804143990 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527135807 0 0) 0x7fb7e8003530 con 0x7fb804143990 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8041440d0 msgr2=0x7fb8040cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8041440d0 0x7fb8040cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 >> v2:172.21.15.82:3300/0 conn(0x7fb80406f150 msgr2=0x7fb8040b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 --2- 172.21.15.46:0/2719676861 >> v2:172.21.15.82:3300/0 conn(0x7fb80406f150 0x7fb8040b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80415e9c0 con 0x7fb804143990 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fb7ec001680 con 0x7fb804143990 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb803400640 1 --2- 172.21.15.46:0/2719676861 >> v2:172.21.15.82:3300/0 conn(0x7fb80406f150 0x7fb8040b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 >> v1:172.21.15.46:6789/0 conn(0x7fb804143990 legacy=0x7fb80406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8041440d0 0x7fb7e8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8003fb0 0x7fb7e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e8004d50 0x7fb7e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415ed30 con 0x7fb7e8004d50 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415e6e0 con 0x7fb7e8003fb0 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e800ba80 con 0x7fb8041440d0 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb803400640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e8004d50 0x7fb7e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb802a00640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8041440d0 0x7fb7e8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb7fa000640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8003fb0 0x7fb7e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb803400640 1 -- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8041440d0 msgr2=0x7fb7e8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb803400640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8041440d0 0x7fb7e8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb803400640 1 -- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8003fb0 msgr2=0x7fb7e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb803400640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8003fb0 0x7fb7e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.534+0000 7fb803400640 1 -- 172.21.15.46:0/2719676861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e8003530 con 0x7fb7e8004d50 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb803400640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e8004d50 0x7fb7e80051d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e350 tx=0x7fb7fc01cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa2c5ea50579ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0b0070 con 0x7fb7e8004d50 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0ab600 con 0x7fb7e8004d50 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb802000640 1 -- 172.21.15.46:0/2719676861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc009de0 con 0x7fb7e8004d50 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 -- 172.21.15.46:0/2719676861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e8004d50 msgr2=0x7fb7e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 --2- 172.21.15.46:0/2719676861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7e8004d50 0x7fb7e80051d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e350 tx=0x7fb7fc01cb10 comp rx=0 tx=0).stop 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb803400640 1 -- 172.21.15.46:0/2719676861 reap_dead start 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 -- 172.21.15.46:0/2719676861 shutdown_connections 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 -- 172.21.15.46:0/2719676861 >> 172.21.15.46:0/2719676861 conn(0x7fb80406ac10 msgr2=0x7fb80406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 -- 172.21.15.46:0/2719676861 shutdown_connections 2024-04-25T08:44:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 -- 172.21.15.46:0/2719676861 wait complete. 2024-04-25T08:44:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 Processor -- start 2024-04-25T08:44:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 -- start start 2024-04-25T08:44:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8004d50 0x7fb8040b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8040af9c0 0x7fb8041582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8040cbb50 0x7fb8040cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb804143ef0 con 0x7fb8040cbb50 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb804143d70 con 0x7fb7e8004d50 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.538+0000 7fb809a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040b3070 con 0x7fb8040af9c0 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb803400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8040cbb50 0x7fb8040cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb803400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8040cbb50 0x7fb8040cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36864/0 (socket says 172.21.15.46:36864) 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb803400640 1 -- 172.21.15.46:0/1870234136 learned_addr learned my addr 172.21.15.46:0/1870234136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb7fa000640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8040af9c0 0x7fb8041582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb802a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8004d50 0x7fb8040b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb803400640 1 -- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8040af9c0 msgr2=0x7fb8041582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb803400640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8040af9c0 0x7fb8041582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb803400640 1 -- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8004d50 msgr2=0x7fb8040b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb803400640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8004d50 0x7fb8040b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb803400640 1 -- 172.21.15.46:0/1870234136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804158830 con 0x7fb8040cbb50 2024-04-25T08:44:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb802a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8004d50 0x7fb8040b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb7fa000640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8040af9c0 0x7fb8041582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb803400640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8040cbb50 0x7fb8040cbfd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e350 tx=0x7fb7fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb7fb400640 1 -- 172.21.15.46:0/1870234136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc0b0040 con 0x7fb8040cbb50 2024-04-25T08:44:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb7fb400640 1 -- 172.21.15.46:0/1870234136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc0ab400 con 0x7fb8040cbb50 2024-04-25T08:44:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb804135600 con 0x7fb8040cbb50 2024-04-25T08:44:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb7fb400640 1 -- 172.21.15.46:0/1870234136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc009de0 con 0x7fb8040cbb50 2024-04-25T08:44:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb804135bc0 con 0x7fb8040cbb50 2024-04-25T08:44:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb804069010 con 0x7fb8040cbb50 2024-04-25T08:44:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.542+0000 7fb7fb400640 1 -- 172.21.15.46:0/1870234136 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7fc0bf030 con 0x7fb8040cbb50 2024-04-25T08:44:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.546+0000 7fb7fb400640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e0066f60 0x7fb7e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.546+0000 7fb7fb400640 1 -- 172.21.15.46:0/1870234136 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb7fc11eee0 con 0x7fb8040cbb50 2024-04-25T08:44:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.546+0000 7fb802a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e0066f60 0x7fb7e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.546+0000 7fb802a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e0066f60 0x7fb7e0069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000c10 tx=0x7fb7ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.550+0000 7fb7fb400640 1 -- 172.21.15.46:0/1870234136 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7fc0c6060 con 0x7fb8040cbb50 2024-04-25T08:44:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.770+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb804138bb0 con 0x7fb8040cbb50 2024-04-25T08:44:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.774+0000 7fb7fb400640 1 -- 172.21.15.46:0/1870234136 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fb7fc016b90 con 0x7fb8040cbb50 2024-04-25T08:44:19.768 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-25T08:44:19.768 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q== 2024-04-25T08:44:19.768 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-25T08:44:19.768 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-25T08:44:19.768 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:44:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-25T08:44:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e0066f60 msgr2=0x7fb7e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e0066f60 0x7fb7e0069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000c10 tx=0x7fb7ec009040 comp rx=0 tx=0).stop 2024-04-25T08:44:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8040cbb50 msgr2=0x7fb8040cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8040cbb50 0x7fb8040cbfd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e350 tx=0x7fb7fc009750 comp rx=0 tx=0).stop 2024-04-25T08:44:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 shutdown_connections 2024-04-25T08:44:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e0066f60 0x7fb7e0069420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8040cbb50 0x7fb8040cbfd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8040af9c0 0x7fb8041582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 --2- 172.21.15.46:0/1870234136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e8004d50 0x7fb8040b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 >> 172.21.15.46:0/1870234136 conn(0x7fb80406ac10 msgr2=0x7fb8040adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 shutdown_connections 2024-04-25T08:44:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.778+0000 7fb809a00640 1 -- 172.21.15.46:0/1870234136 wait complete. 2024-04-25T08:44:19.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:19.793 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-25T08:44:19.798+0000 7fb8b6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:44:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5ff400640 1 Processor -- start 2024-04-25T08:44:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5ff400640 1 -- start start 2024-04-25T08:44:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5ff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5f806e9e0 0x7fd5f815edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5ff400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd5f806eee0 0x7fd5f80cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5ff400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f806fa90 con 0x7fd5f80cd550 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5ff400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f80cafe0 con 0x7fd5f806e9e0 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5ff400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f815f300 con 0x7fd5f806eee0 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5efe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5f806e9e0 0x7fd5f815edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5efe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5f806e9e0 0x7fd5f815edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36880/0 (socket says 172.21.15.46:36880) 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5efe00640 1 -- 172.21.15.46:0/770095184 learned_addr learned my addr 172.21.15.46:0/770095184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5fd400640 1 --2- 172.21.15.46:0/770095184 >> v2:172.21.15.82:3300/0 conn(0x7fd5f806eee0 0x7fd5f80cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5efe00640 1 -- 172.21.15.46:0/770095184 >> v2:172.21.15.82:3300/0 conn(0x7fd5f806eee0 msgr2=0x7fd5f80cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5efe00640 1 --2- 172.21.15.46:0/770095184 >> v2:172.21.15.82:3300/0 conn(0x7fd5f806eee0 0x7fd5f80cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5efe00640 1 -- 172.21.15.46:0/770095184 >> v1:172.21.15.82:6789/0 conn(0x7fd5f80cd550 legacy=0x7fd5f806e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5efe00640 1 -- 172.21.15.46:0/770095184 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f80cb210 con 0x7fd5f806e9e0 2024-04-25T08:44:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.938+0000 7fd5fd400640 1 --2- 172.21.15.46:0/770095184 >> v2:172.21.15.82:3300/0 conn(0x7fd5f806eee0 0x7fd5f80cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5efe00640 1 --2- 172.21.15.46:0/770095184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806e9e0 0x7fd5f815edc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80c4080 tx=0x7fd5e001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6398026b93f977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5f7e00640 1 -- 172.21.15.46:0/770095184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00b4070 con 0x7fd5f806e9e0 2024-04-25T08:44:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5f7e00640 1 -- 172.21.15.46:0/770095184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e00a7ac0 con 0x7fd5f806e9e0 2024-04-25T08:44:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5f7e00640 1 -- 172.21.15.46:0/770095184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00af440 con 0x7fd5f806e9e0 2024-04-25T08:44:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5ff400640 1 -- 172.21.15.46:0/770095184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806e9e0 msgr2=0x7fd5f815edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5ff400640 1 --2- 172.21.15.46:0/770095184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806e9e0 0x7fd5f815edc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80c4080 tx=0x7fd5e001ebf0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5ff400640 1 -- 172.21.15.46:0/770095184 shutdown_connections 2024-04-25T08:44:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5ff400640 1 --2- 172.21.15.46:0/770095184 >> v2:172.21.15.82:3300/0 conn(0x7fd5f806eee0 0x7fd5f80cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5ff400640 1 --2- 172.21.15.46:0/770095184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806e9e0 0x7fd5f815edc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5ff400640 1 -- 172.21.15.46:0/770095184 >> 172.21.15.46:0/770095184 conn(0x7fd5f8099a00 msgr2=0x7fd5f80b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5ff400640 1 -- 172.21.15.46:0/770095184 shutdown_connections 2024-04-25T08:44:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.942+0000 7fd5ff400640 1 -- 172.21.15.46:0/770095184 wait complete. 2024-04-25T08:44:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 Processor -- start 2024-04-25T08:44:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 -- start start 2024-04-25T08:44:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f806e9e0 0x7fd5f8157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806eee0 0x7fd5f81fa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5f80cd550 0x7fd5f80b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80cc8a0 con 0x7fd5f806eee0 2024-04-25T08:44:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80cc720 con 0x7fd5f806e9e0 2024-04-25T08:44:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80cca20 con 0x7fd5f80cd550 2024-04-25T08:44:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806eee0 0x7fd5f81fa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806eee0 0x7fd5f81fa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36882/0 (socket says 172.21.15.46:36882) 2024-04-25T08:44:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5efe00640 1 -- 172.21.15.46:0/3280265630 learned_addr learned my addr 172.21.15.46:0/3280265630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5fca00640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f806e9e0 0x7fd5f8157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5fd400640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5f80cd550 0x7fd5f80b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5efe00640 1 -- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5f80cd550 msgr2=0x7fd5f80b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5efe00640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5f80cd550 0x7fd5f80b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5efe00640 1 -- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f806e9e0 msgr2=0x7fd5f8157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5efe00640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f806e9e0 0x7fd5f8157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5efe00640 1 -- 172.21.15.46:0/3280265630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f80c88a0 con 0x7fd5f806eee0 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5fd400640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5f80cd550 0x7fd5f80b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5fca00640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f806e9e0 0x7fd5f8157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5efe00640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806eee0 0x7fd5f81fa7c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000c10 tx=0x7fd5e001b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5f5600640 1 -- 172.21.15.46:0/3280265630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00b4030 con 0x7fd5f806eee0 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5f5600640 1 -- 172.21.15.46:0/3280265630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e00b6070 con 0x7fd5f806eee0 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5f5600640 1 -- 172.21.15.46:0/3280265630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00ad550 con 0x7fd5f806eee0 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f80c8b80 con 0x7fd5f806eee0 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f80c9140 con 0x7fd5f806eee0 2024-04-25T08:44:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.946+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f8069080 con 0x7fd5f806eee0 2024-04-25T08:44:19.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.950+0000 7fd5f5600640 1 -- 172.21.15.46:0/3280265630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5e00c1060 con 0x7fd5f806eee0 2024-04-25T08:44:19.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.950+0000 7fd5f5600640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd5d4066f60 0x7fd5d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:19.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.950+0000 7fd5f5600640 1 -- 172.21.15.46:0/3280265630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd5e011f6c0 con 0x7fd5f806eee0 2024-04-25T08:44:19.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.950+0000 7fd5fca00640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd5d4066f60 0x7fd5d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:19.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.950+0000 7fd5fca00640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd5d4066f60 0x7fd5d4069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8001650 tx=0x7fd5e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:19.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:19.954+0000 7fd5f5600640 1 -- 172.21.15.46:0/3280265630 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5e00187e0 con 0x7fd5f806eee0 2024-04-25T08:44:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.150+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5f80cdca0 con 0x7fd5f806eee0 2024-04-25T08:44:20.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.150+0000 7fd5f5600640 1 -- 172.21.15.46:0/3280265630 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fd5e00f0530 con 0x7fd5f806eee0 2024-04-25T08:44:20.144 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:20.144 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/850914154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":850914154},{"type":"v1","addr":"172.21.15.46:6835","nonce":850914154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1115060038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1115060038},{"type":"v1","addr":"172.21.15.82:6836","nonce":1115060038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5620,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1383788710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1383788710},{"type":"v1","addr":"172.21.15.82:6837","nonce":1383788710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:44:14.218049+0000","modified":"2024-04-25T08:44:17.246143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/3843568539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3843568539},{"type":"v1","addr":"172.21.15.46:6834","nonce":3843568539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:44:20.145 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-04-25T08:44:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd5d4066f60 msgr2=0x7fd5d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd5d4066f60 0x7fd5d4069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8001650 tx=0x7fd5e8009040 comp rx=0 tx=0).stop 2024-04-25T08:44:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806eee0 msgr2=0x7fd5f81fa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806eee0 0x7fd5f81fa7c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000c10 tx=0x7fd5e001b990 comp rx=0 tx=0).stop 2024-04-25T08:44:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 shutdown_connections 2024-04-25T08:44:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd5d4066f60 0x7fd5d4069420 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5f80cd550 0x7fd5f80b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5f806eee0 0x7fd5f81fa7c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 --2- 172.21.15.46:0/3280265630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5f806e9e0 0x7fd5f8157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 >> 172.21.15.46:0/3280265630 conn(0x7fd5f8099a00 msgr2=0x7fd5f80b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 shutdown_connections 2024-04-25T08:44:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.154+0000 7fd5ff400640 1 -- 172.21.15.46:0/3280265630 wait complete. 2024-04-25T08:44:20.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:44:20.213 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:44:20.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:44:14.218049+0000', 'modified': '2024-04-25T08:44:17.246143+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5733}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5733': {'gid': 5733, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6834/3843568539', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 3843568539}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 3843568539}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:44:20.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:44:20.569 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:44:20.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.718+0000 7fc945000640 1 Processor -- start 2024-04-25T08:44:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.718+0000 7fc945000640 1 -- start start 2024-04-25T08:44:20.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc945000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9400af3f0 0x7fc9400b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc945000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc94006f630 0x7fc9400b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc945000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc94015e180 con 0x7fc94006db40 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc945000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9400b3ca0 con 0x7fc94006f630 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc945000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94015e460 con 0x7fc9400af3f0 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc94006f630 0x7fc9400b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc94006f630 0x7fc9400b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36894/0 (socket says 172.21.15.46:36894) 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93e800640 1 -- 172.21.15.46:0/2161356291 learned_addr learned my addr 172.21.15.46:0/2161356291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93de00640 1 --2- 172.21.15.46:0/2161356291 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9400af3f0 0x7fc9400b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93e800640 1 -- 172.21.15.46:0/2161356291 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9400af3f0 msgr2=0x7fc9400b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93e800640 1 --2- 172.21.15.46:0/2161356291 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9400af3f0 0x7fc9400b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93e800640 1 -- 172.21.15.46:0/2161356291 >> v1:172.21.15.82:6789/0 conn(0x7fc94006db40 legacy=0x7fc94006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93e800640 1 -- 172.21.15.46:0/2161356291 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94015e740 con 0x7fc94006f630 2024-04-25T08:44:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93e800640 1 --2- 172.21.15.46:0/2161356291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc94006f630 0x7fc9400b3380 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc938009e50 tx=0x7fc93801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db418b8deb682a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93d400640 1 -- 172.21.15.46:0/2161356291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380a5610 con 0x7fc94006f630 2024-04-25T08:44:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93d400640 1 -- 172.21.15.46:0/2161356291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9380a57b0 con 0x7fc94006f630 2024-04-25T08:44:20.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.722+0000 7fc93d400640 1 -- 172.21.15.46:0/2161356291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380a5b80 con 0x7fc94006f630 2024-04-25T08:44:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 -- 172.21.15.46:0/2161356291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc94006f630 msgr2=0x7fc9400b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 --2- 172.21.15.46:0/2161356291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc94006f630 0x7fc9400b3380 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc938009e50 tx=0x7fc93801ca30 comp rx=0 tx=0).stop 2024-04-25T08:44:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 -- 172.21.15.46:0/2161356291 shutdown_connections 2024-04-25T08:44:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 --2- 172.21.15.46:0/2161356291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc94006f630 0x7fc9400b3380 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 --2- 172.21.15.46:0/2161356291 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc9400af3f0 0x7fc9400b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 -- 172.21.15.46:0/2161356291 >> 172.21.15.46:0/2161356291 conn(0x7fc9400cfd90 msgr2=0x7fc9400d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 -- 172.21.15.46:0/2161356291 shutdown_connections 2024-04-25T08:44:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 -- 172.21.15.46:0/2161356291 wait complete. 2024-04-25T08:44:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 Processor -- start 2024-04-25T08:44:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 -- start start 2024-04-25T08:44:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc94006db40 0x7fc940148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc94006f630 0x7fc94014ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9400af3f0 0x7fc940149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94006bda0 con 0x7fc9400af3f0 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94006bc20 con 0x7fc94006f630 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc945000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94016a6f0 con 0x7fc94006db40 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc93e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9400af3f0 0x7fc940149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc93e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9400af3f0 0x7fc940149ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36896/0 (socket says 172.21.15.46:36896) 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc93de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc94006db40 0x7fc940148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc93e800640 1 -- 172.21.15.46:0/178807600 learned_addr learned my addr 172.21.15.46:0/178807600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc935400640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc94006f630 0x7fc94014ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc93e800640 1 -- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc94006db40 msgr2=0x7fc940148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc93e800640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc94006db40 0x7fc940148040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc93e800640 1 -- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc94006f630 msgr2=0x7fc94014ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc93e800640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc94006f630 0x7fc94014ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.726+0000 7fc93e800640 1 -- 172.21.15.46:0/178807600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94014d890 con 0x7fc9400af3f0 2024-04-25T08:44:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc93e800640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9400af3f0 0x7fc940149ce0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc93801a660 tx=0x7fc9380197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc936a00640 1 -- 172.21.15.46:0/178807600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380bc020 con 0x7fc9400af3f0 2024-04-25T08:44:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc936a00640 1 -- 172.21.15.46:0/178807600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9380b8670 con 0x7fc9400af3f0 2024-04-25T08:44:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc936a00640 1 -- 172.21.15.46:0/178807600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938009510 con 0x7fc9400af3f0 2024-04-25T08:44:20.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc945000640 1 -- 172.21.15.46:0/178807600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94014db70 con 0x7fc9400af3f0 2024-04-25T08:44:20.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc945000640 1 -- 172.21.15.46:0/178807600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94014e410 con 0x7fc9400af3f0 2024-04-25T08:44:20.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc936a00640 1 -- 172.21.15.46:0/178807600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9380bf020 con 0x7fc9400af3f0 2024-04-25T08:44:20.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc936a00640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc91c066f60 0x7fc91c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc936a00640 1 -- 172.21.15.46:0/178807600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc9381201f0 con 0x7fc9400af3f0 2024-04-25T08:44:20.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc9485af000 1 Processor -- start 2024-04-25T08:44:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc93de00640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc91c066f60 0x7fc91c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.730+0000 7fc9485af000 1 -- start start 2024-04-25T08:44:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93de00640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc91c066f60 0x7fc91c069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9401f99e0 tx=0x7fc930009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b400d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 0x55c46b3b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b3b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46b29e5f0 con 0x55c46b3b6f30 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46b205da0 con 0x55c46b400920 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46b18e150 con 0x55c46b401240 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b3b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b3b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36906/0 (socket says 172.21.15.46:36906) 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93e800640 1 -- 172.21.15.46:0/4112221306 learned_addr learned my addr 172.21.15.46:0/4112221306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93de00640 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b400d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc935400640 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 0x55c46b3b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93e800640 1 -- 172.21.15.46:0/4112221306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 msgr2=0x55c46b3b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93e800640 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 0x55c46b3b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93e800640 1 -- 172.21.15.46:0/4112221306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 msgr2=0x55c46b400d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93e800640 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b400d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93e800640 1 -- 172.21.15.46:0/4112221306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c46b1a7ca0 con 0x55c46b3b6f30 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93de00640 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b400d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc93e800640 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b3b7310 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc9380f8c90 tx=0x7fc938009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da46740b25c2de36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc92ea00640 1 -- 172.21.15.46:0/4112221306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380f91c0 con 0x55c46b3b6f30 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc92ea00640 1 -- 172.21.15.46:0/4112221306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938117e50 con 0x55c46b3b6f30 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc92ea00640 1 -- 172.21.15.46:0/4112221306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9380a5690 con 0x55c46b3b6f30 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 -- 172.21.15.46:0/4112221306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 msgr2=0x55c46b3b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b3b7310 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc9380f8c90 tx=0x7fc938009810 comp rx=0 tx=0).stop 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 -- 172.21.15.46:0/4112221306 shutdown_connections 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b3b7310 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 0x55c46b3b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 --2- 172.21.15.46:0/4112221306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b400d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 -- 172.21.15.46:0/4112221306 >> 172.21.15.46:0/4112221306 conn(0x55c46b3bd2a0 msgr2=0x55c46b3b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 -- 172.21.15.46:0/4112221306 shutdown_connections 2024-04-25T08:44:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 -- 172.21.15.46:0/4112221306 wait complete. 2024-04-25T08:44:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 -- start start 2024-04-25T08:44:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.734+0000 7fc9485af000 1 Processor -- start 2024-04-25T08:44:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b407c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b4081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 0x55c46b3baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46b201e40 con 0x55c46b3b6f30 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46b191900 con 0x55c46b400920 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46b29e5f0 con 0x55c46b401240 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 10 client.0 _ll_register_callbacks cb 0x55c46b2b91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b407c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b407c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36920/0 (socket says 172.21.15.46:36920) 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93de00640 1 -- 172.21.15.46:0/1123843674 learned_addr learned my addr 172.21.15.46:0/1123843674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc935400640 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b4081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93e800640 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 0x55c46b3baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93de00640 1 -- 172.21.15.46:0/1123843674 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 msgr2=0x55c46b3baaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93de00640 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 0x55c46b3baaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93de00640 1 -- 172.21.15.46:0/1123843674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 msgr2=0x55c46b4081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93de00640 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b4081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93de00640 1 -- 172.21.15.46:0/1123843674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c46b1be630 con 0x55c46b3b6f30 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc93de00640 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b407c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc930014a40 tx=0x7fc9300046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8aa07cf03529b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc923400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc923400640 1 -- 172.21.15.46:0/1123843674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc930026030 con 0x55c46b3b6f30 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 -- 172.21.15.46:0/1123843674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc938117e50 con 0x55c46b3b6f30 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc923400640 1 -- 172.21.15.46:0/1123843674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc93000f040 con 0x55c46b3b6f30 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc923400640 1 -- 172.21.15.46:0/1123843674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc930032050 con 0x55c46b3b6f30 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 10 client.5880 fetch_fsmap learned FSMap version 25 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 -- 172.21.15.46:0/1123843674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c46b18e3a0 con 0x55c46b3b6f30 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc923400640 1 -- 172.21.15.46:0/1123843674 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938117e50 con 0x55c46b3b6f30 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc923400640 1 -- 172.21.15.46:0/1123843674 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc930035020 con 0x55c46b3b6f30 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 10 client.5880 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 10 client.5880 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:44:20.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 20 client.5880 populate_metadata read hostname 'smithi046' 2024-04-25T08:44:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c46b3bc160 0x55c46b411ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 4 client.5880 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:44:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 -- 172.21.15.46:0/1123843674 --> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c46b1a1a50 con 0x55c46b3bc160 2024-04-25T08:44:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc935400640 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c46b3bc160 0x55c46b411ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc923400640 10 client.5880 ms_handle_connect on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc935400640 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c46b3bc160 0x55c46b411ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1883dc3e92610 server_cookie=7c3a820059719d51 in_seq=0 out_seq=0 2024-04-25T08:44:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc923400640 1 -- 172.21.15.46:0/1123843674 <== mds.0 v2:172.21.15.46:6832/3843568539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c46b1a1a50 con 0x55c46b3bc160 2024-04-25T08:44:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc923400640 10 client.5880 handle_command_reply: tid=0 2024-04-25T08:44:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.738+0000 7fc9485af000 1 client.5880 shutdown 2024-04-25T08:44:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 -- 172.21.15.46:0/1123843674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 msgr2=0x55c46b407c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b407c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc930014a40 tx=0x7fc9300046d0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 -- 172.21.15.46:0/1123843674 shutdown_connections 2024-04-25T08:44:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c46b3bc160 0x55c46b411ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b401240 0x55c46b3baaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b4081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 --2- 172.21.15.46:0/1123843674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b407c80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 -- 172.21.15.46:0/1123843674 >> 172.21.15.46:0/1123843674 conn(0x55c46b3bd2a0 msgr2=0x55c46b3b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc923400640 0 client.5880 ms_handle_reset on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 -- 172.21.15.46:0/1123843674 shutdown_connections 2024-04-25T08:44:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 1 -- 172.21.15.46:0/1123843674 wait complete. 2024-04-25T08:44:20.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.742+0000 7fc9485af000 20 client.5880 trim_cache size 0 max 16384 2024-04-25T08:44:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 Processor -- start 2024-04-25T08:44:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- start start 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b3bd2a0 0x55c46b45c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b45cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b400920 0x55c46b45d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc940202270 con 0x55c46b3b6f30 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc940152960 con 0x55c46b3bd2a0 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc940201cd0 con 0x55c46b400920 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc935400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b45cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc935400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b45cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36924/0 (socket says 172.21.15.46:36924) 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc935400640 1 -- 172.21.15.46:0/1886369497 learned_addr learned my addr 172.21.15.46:0/1886369497 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc93de00640 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b3bd2a0 0x55c46b45c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc93e800640 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b400920 0x55c46b45d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc935400640 1 -- 172.21.15.46:0/1886369497 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b400920 msgr2=0x55c46b45d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc935400640 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b400920 0x55c46b45d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc935400640 1 -- 172.21.15.46:0/1886369497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b3bd2a0 msgr2=0x55c46b45c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc935400640 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b3bd2a0 0x55c46b45c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc935400640 1 -- 172.21.15.46:0/1886369497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c46b287600 con 0x55c46b3b6f30 2024-04-25T08:44:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc93e800640 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b400920 0x55c46b45d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc935400640 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b45cf10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55c46b3bc5d0 tx=0x7fc928009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d519b113c04a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc92e000640 1 -- 172.21.15.46:0/1886369497 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928004e90 con 0x55c46b3b6f30 2024-04-25T08:44:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc92e000640 1 -- 172.21.15.46:0/1886369497 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92801e020 con 0x55c46b3b6f30 2024-04-25T08:44:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc92e000640 1 -- 172.21.15.46:0/1886369497 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92801a570 con 0x55c46b3b6f30 2024-04-25T08:44:20.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- 172.21.15.46:0/1886369497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 msgr2=0x55c46b45cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b45cf10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55c46b3bc5d0 tx=0x7fc928009040 comp rx=0 tx=0).stop 2024-04-25T08:44:20.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- 172.21.15.46:0/1886369497 shutdown_connections 2024-04-25T08:44:20.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b400920 0x55c46b45d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3b6f30 0x55c46b45cf10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 --2- 172.21.15.46:0/1886369497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b3bd2a0 0x55c46b45c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- 172.21.15.46:0/1886369497 >> 172.21.15.46:0/1886369497 conn(0x55c46b407f60 msgr2=0x55c46b3ba220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:20.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- 172.21.15.46:0/1886369497 shutdown_connections 2024-04-25T08:44:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- 172.21.15.46:0/1886369497 wait complete. 2024-04-25T08:44:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.758+0000 7fc9485af000 1 -- start start 2024-04-25T08:44:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 Processor -- start 2024-04-25T08:44:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b3b6f30 0x55c46b462100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3bd2a0 0x55c46b462640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b445d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46b20a3d0 con 0x55c46b3bd2a0 2024-04-25T08:44:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c46b2c1d50 con 0x55c46b400920 2024-04-25T08:44:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c46b1a0980 con 0x55c46b3b6f30 2024-04-25T08:44:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 10 client.0 _ll_register_callbacks cb 0x7fc938009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:44:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc935400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3bd2a0 0x55c46b462640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc935400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3bd2a0 0x55c46b462640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36926/0 (socket says 172.21.15.46:36926) 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc935400640 1 -- 172.21.15.46:0/422049070 learned_addr learned my addr 172.21.15.46:0/422049070 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc93de00640 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b3b6f30 0x55c46b462100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc93e800640 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b445d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc935400640 1 -- 172.21.15.46:0/422049070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b3b6f30 msgr2=0x55c46b462100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc935400640 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b3b6f30 0x55c46b462100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc935400640 1 -- 172.21.15.46:0/422049070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 msgr2=0x55c46b445d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc935400640 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b445d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc935400640 1 -- 172.21.15.46:0/422049070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c46b19a9b0 con 0x55c46b3bd2a0 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc935400640 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3bd2a0 0x55c46b462640 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc928000fe0 tx=0x7fc928000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab3fff8b4057a8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc923400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc923400640 1 -- 172.21.15.46:0/422049070 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928007450 con 0x55c46b3bd2a0 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc923400640 1 -- 172.21.15.46:0/422049070 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc9280075f0 con 0x55c46b3bd2a0 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 -- 172.21.15.46:0/422049070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc92801e020 con 0x55c46b3bd2a0 2024-04-25T08:44:20.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc923400640 1 -- 172.21.15.46:0/422049070 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc928031050 con 0x55c46b3bd2a0 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 10 client.5886 fetch_fsmap learned FSMap version 25 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 -- 172.21.15.46:0/422049070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c46b1ad0b0 con 0x55c46b3bd2a0 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc923400640 1 -- 172.21.15.46:0/422049070 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92801e020 con 0x55c46b3bd2a0 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc923400640 1 -- 172.21.15.46:0/422049070 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc928034020 con 0x55c46b3bd2a0 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 10 client.5886 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 10 client.5886 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 20 client.5886 populate_metadata read hostname 'smithi046' 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c46b401240 0x55c46b4462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 4 client.5886 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 -- 172.21.15.46:0/422049070 --> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c46b2132e0 con 0x55c46b401240 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc93de00640 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c46b401240 0x55c46b4462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc93de00640 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c46b401240 0x55c46b4462a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1000c079037800 server_cookie=8af55d70b9bfee19 in_seq=0 out_seq=0 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc923400640 10 client.5886 ms_handle_connect on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc923400640 1 -- 172.21.15.46:0/422049070 <== mds.0 v2:172.21.15.46:6832/3843568539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c46b2132e0 con 0x55c46b401240 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc923400640 10 client.5886 handle_command_reply: tid=0 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.762+0000 7fc9485af000 1 client.5886 shutdown 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 -- 172.21.15.46:0/422049070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3bd2a0 msgr2=0x55c46b462640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3bd2a0 0x55c46b462640 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc928000fe0 tx=0x7fc928000bd0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 -- 172.21.15.46:0/422049070 shutdown_connections 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c46b401240 0x55c46b4462a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c46b400920 0x55c46b445d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c46b3bd2a0 0x55c46b462640 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 --2- 172.21.15.46:0/422049070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c46b3b6f30 0x55c46b462100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc923400640 0 client.5886 ms_handle_reset on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 -- 172.21.15.46:0/422049070 >> 172.21.15.46:0/422049070 conn(0x55c46b407f60 msgr2=0x55c46b408750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 -- 172.21.15.46:0/422049070 shutdown_connections 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 1 -- 172.21.15.46:0/422049070 wait complete. 2024-04-25T08:44:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc9485af000 20 client.5886 trim_cache size 0 max 16384 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5733, 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.9200608959999999, 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 25, 2024-04-25T08:44:20.762 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 53, 2024-04-25T08:44:20.763 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 53, 2024-04-25T08:44:20.763 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 11.787020331000001 2024-04-25T08:44:20.763 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:44:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 -- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc91c066f60 msgr2=0x7fc91c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc91c066f60 0x7fc91c069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9401f99e0 tx=0x7fc930009190 comp rx=0 tx=0).stop 2024-04-25T08:44:20.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 -- 172.21.15.46:0/178807600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9400af3f0 msgr2=0x7fc940149ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9400af3f0 0x7fc940149ce0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc93801a660 tx=0x7fc9380197d0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 -- 172.21.15.46:0/178807600 shutdown_connections 2024-04-25T08:44:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc91c066f60 0x7fc91c069420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9400af3f0 0x7fc940149ce0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc94006f630 0x7fc94014ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 --2- 172.21.15.46:0/178807600 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc94006db40 0x7fc940148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 -- 172.21.15.46:0/178807600 >> 172.21.15.46:0/178807600 conn(0x7fc9400cfd90 msgr2=0x7fc94015ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 -- 172.21.15.46:0/178807600 shutdown_connections 2024-04-25T08:44:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.766+0000 7fc945000640 1 -- 172.21.15.46:0/178807600 wait complete. 2024-04-25T08:44:20.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:44:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33be800640 1 Processor -- start 2024-04-25T08:44:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33be800640 1 -- start start 2024-04-25T08:44:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33be800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b8099a00 0x7f33b8099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33be800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f33b80ccde0 0x7f33b80cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33be800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b809a320 con 0x7f33b80b2240 2024-04-25T08:44:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33be800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b806db90 con 0x7f33b80ccde0 2024-04-25T08:44:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33be800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b806bdc0 con 0x7f33b8099a00 2024-04-25T08:44:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f33b80b2240 0x7f33b80b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46322/0 (socket says 172.21.15.46:46322) 2024-04-25T08:44:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b7e00640 1 -- 172.21.15.46:0/1784051251 learned_addr learned my addr 172.21.15.46:0/1784051251 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b7400640 1 --2- 172.21.15.46:0/1784051251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b8099a00 0x7f33b8099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33afe00640 1 --2- 172.21.15.46:0/1784051251 >> v2:172.21.15.82:3300/0 conn(0x7f33b80ccde0 0x7f33b80cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b6a00640 1 -- 172.21.15.46:0/1784051251 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395637296 0 0) 0x7f33b809a320 con 0x7f33b80b2240 2024-04-25T08:44:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b6a00640 1 -- 172.21.15.46:0/1784051251 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f339c003530 con 0x7f33b80b2240 2024-04-25T08:44:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b6a00640 1 -- 172.21.15.46:0/1784051251 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1984303819 0 0) 0x7f339c003530 con 0x7f33b80b2240 2024-04-25T08:44:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b7400640 1 -- 172.21.15.46:0/1784051251 >> v2:172.21.15.82:3300/0 conn(0x7f33b80ccde0 msgr2=0x7f33b80cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b7400640 1 --2- 172.21.15.46:0/1784051251 >> v2:172.21.15.82:3300/0 conn(0x7f33b80ccde0 0x7f33b80cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b7400640 1 -- 172.21.15.46:0/1784051251 >> v1:172.21.15.46:6789/0 conn(0x7f33b80b2240 legacy=0x7f33b80b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33b7400640 1 -- 172.21.15.46:0/1784051251 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b806c0a0 con 0x7f33b8099a00 2024-04-25T08:44:20.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.926+0000 7f33afe00640 1 --2- 172.21.15.46:0/1784051251 >> v2:172.21.15.82:3300/0 conn(0x7f33b80ccde0 0x7f33b80cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:20.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33b7400640 1 --2- 172.21.15.46:0/1784051251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b8099a00 0x7f33b8099de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f33a80097c0 tx=0x7f33a801c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead255da82bf63aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33b6a00640 1 -- 172.21.15.46:0/1784051251 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a80b0070 con 0x7f33b8099a00 2024-04-25T08:44:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33b6a00640 1 -- 172.21.15.46:0/1784051251 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a80ab400 con 0x7f33b8099a00 2024-04-25T08:44:20.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33b6a00640 1 -- 172.21.15.46:0/1784051251 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0001a00 con 0x7f33b8099a00 2024-04-25T08:44:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33be800640 1 -- 172.21.15.46:0/1784051251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b8099a00 msgr2=0x7f33b8099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33be800640 1 --2- 172.21.15.46:0/1784051251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b8099a00 0x7f33b8099de0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f33a80097c0 tx=0x7f33a801c300 comp rx=0 tx=0).stop 2024-04-25T08:44:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33be800640 1 -- 172.21.15.46:0/1784051251 shutdown_connections 2024-04-25T08:44:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33be800640 1 --2- 172.21.15.46:0/1784051251 >> v2:172.21.15.82:3300/0 conn(0x7f33b80ccde0 0x7f33b80cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33be800640 1 --2- 172.21.15.46:0/1784051251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b8099a00 0x7f33b8099de0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.930+0000 7f33be800640 1 -- 172.21.15.46:0/1784051251 >> 172.21.15.46:0/1784051251 conn(0x7f33b80b3120 msgr2=0x7f33b80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 -- 172.21.15.46:0/1784051251 shutdown_connections 2024-04-25T08:44:20.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 -- 172.21.15.46:0/1784051251 wait complete. 2024-04-25T08:44:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 Processor -- start 2024-04-25T08:44:20.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 -- start start 2024-04-25T08:44:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33b8099a00 0x7f33b8141200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b80b2240 0x7f33b80c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b80ccde0 0x7f33b80c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80cd7a0 con 0x7f33b80b2240 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80cdb80 con 0x7f33b8099a00 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80cd920 con 0x7f33b80ccde0 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b80b2240 0x7f33b80c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b80b2240 0x7f33b80c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36934/0 (socket says 172.21.15.46:36934) 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33afe00640 1 -- 172.21.15.46:0/3299726501 learned_addr learned my addr 172.21.15.46:0/3299726501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33b7e00640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b80ccde0 0x7f33b80c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33b7400640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33b8099a00 0x7f33b8141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33afe00640 1 -- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b80ccde0 msgr2=0x7f33b80c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33afe00640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b80ccde0 0x7f33b80c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33afe00640 1 -- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33b8099a00 msgr2=0x7f33b8141200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33afe00640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33b8099a00 0x7f33b8141200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33afe00640 1 -- 172.21.15.46:0/3299726501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b80c9780 con 0x7f33b80b2240 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33afe00640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b80b2240 0x7f33b80c8f40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f33a000ebe0 tx=0x7f33a000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33af400640 1 -- 172.21.15.46:0/3299726501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0017070 con 0x7f33b80b2240 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33af400640 1 -- 172.21.15.46:0/3299726501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a000cc30 con 0x7f33b80b2240 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b80c9a60 con 0x7f33b80b2240 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33af400640 1 -- 172.21.15.46:0/3299726501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0013470 con 0x7f33b80b2240 2024-04-25T08:44:20.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.934+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b80ca020 con 0x7f33b80b2240 2024-04-25T08:44:20.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.938+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b8069080 con 0x7f33b80b2240 2024-04-25T08:44:20.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.938+0000 7f33af400640 1 -- 172.21.15.46:0/3299726501 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33a001a070 con 0x7f33b80b2240 2024-04-25T08:44:20.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.938+0000 7f33af400640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3394066f60 0x7f3394069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:20.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.938+0000 7f33af400640 1 -- 172.21.15.46:0/3299726501 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f33a00855c0 con 0x7f33b80b2240 2024-04-25T08:44:20.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.938+0000 7f33b7400640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3394066f60 0x7f3394069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:20.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.938+0000 7f33b7400640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3394066f60 0x7f3394069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f33a80ad000 tx=0x7f33a80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:20.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:20.942+0000 7f33af400640 1 -- 172.21.15.46:0/3299726501 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33a00563b0 con 0x7f33b80b2240 2024-04-25T08:44:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f33b8156350 con 0x7f33b80b2240 2024-04-25T08:44:21.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33af400640 1 -- 172.21.15.46:0/3299726501 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f33a0055cb0 con 0x7f33b80b2240 2024-04-25T08:44:21.140 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:44:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3394066f60 msgr2=0x7f3394069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3394066f60 0x7f3394069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f33a80ad000 tx=0x7f33a80a6000 comp rx=0 tx=0).stop 2024-04-25T08:44:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b80b2240 msgr2=0x7f33b80c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b80b2240 0x7f33b80c8f40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f33a000ebe0 tx=0x7f33a000c520 comp rx=0 tx=0).stop 2024-04-25T08:44:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 shutdown_connections 2024-04-25T08:44:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3394066f60 0x7f3394069420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33b80ccde0 0x7f33b80c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b80b2240 0x7f33b80c8f40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 --2- 172.21.15.46:0/3299726501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33b8099a00 0x7f33b8141200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 >> 172.21.15.46:0/3299726501 conn(0x7f33b80b3120 msgr2=0x7f33b81555c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 shutdown_connections 2024-04-25T08:44:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:21.146+0000 7f33be800640 1 -- 172.21.15.46:0/3299726501 wait complete. 2024-04-25T08:44:21.160 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:44:21.160 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:44:21.161 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:44:21.161 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:44:21.161 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:44:21.161 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:44:21.161 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:44:21.161 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:44:21.162 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:44:21.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.162 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:44:21.171 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:44:21.172 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42341sec preferred_lft 42341sec 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:44:21.173 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:44:21.174 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:44:21.175 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:44:21.175 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:44:21.175 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:44:21.175 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:44:21.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.176 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:44:21.235 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:44:21.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.237 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:44:21.290 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:44:21.290 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:44:21.290 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:44:21.290 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:44:21.290 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:44:21.290 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:44:21.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.291 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:44:21.291 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:44:21.291 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:44:21.291 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:44:21.291 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:44:21.291 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:44:21.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.383 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:44:21.392 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:44:21.393 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:44:21.393 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:44:21.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:44:21.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:44:21.457 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:44:21.513 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:44:21.514 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:44:21.514 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:44:21.564 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:44:21.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.565 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:44:21.654 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:44:21.654 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:44:21.654 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:44:21.669 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:44:21.669 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:44:21.669 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:44:21.670 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:44:21.670 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:44:21.679 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:44:21.679 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:44:21.679 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:44:21.679 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:44:21.679 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:44:21.679 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:44:21.679 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:44:21.679 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:44:21.683 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:44:21.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.683 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:44:21.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.703 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:44:21.761 INFO:teuthology.orchestra.run.smithi046.stdout:client5752 2024-04-25T08:44:21.761 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:44:21.762 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:21.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.814 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-25T08:44:21.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:21.871 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-25T08:44:21.927 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:44:21.927 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:44:21.927 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:44:21.928 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:44:21.928 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-25T08:44:21.928 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:44:22.034 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-04-25T08:44:22.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:44:22.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a90c00640 1 Processor -- start 2024-04-25T08:44:22.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a90c00640 1 -- start start 2024-04-25T08:44:22.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c15b810 0x7f3a8c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a90c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a8c0669a0 0x7f3a8c0ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a90c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3a8c0adf60 0x7f3a8c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a90c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8c072c70 con 0x7f3a8c0669a0 2024-04-25T08:44:22.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a90c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8c15f490 con 0x7f3a8c0adf60 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a90c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c15f1b0 con 0x7f3a8c15b810 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a8c0669a0 0x7f3a8c0ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a8c0669a0 0x7f3a8c0ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36946/0 (socket says 172.21.15.46:36946) 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8aa00640 1 -- 172.21.15.46:0/189925781 learned_addr learned my addr 172.21.15.46:0/189925781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8be00640 1 --2- 172.21.15.46:0/189925781 >> v2:172.21.15.82:3300/0 conn(0x7f3a8c0adf60 0x7f3a8c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8b400640 1 --2- 172.21.15.46:0/189925781 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c15b810 0x7f3a8c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8aa00640 1 -- 172.21.15.46:0/189925781 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c15b810 msgr2=0x7f3a8c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8aa00640 1 --2- 172.21.15.46:0/189925781 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c15b810 0x7f3a8c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8aa00640 1 -- 172.21.15.46:0/189925781 >> v2:172.21.15.82:3300/0 conn(0x7f3a8c0adf60 msgr2=0x7f3a8c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8aa00640 1 --2- 172.21.15.46:0/189925781 >> v2:172.21.15.82:3300/0 conn(0x7f3a8c0adf60 0x7f3a8c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.238+0000 7f3a8aa00640 1 -- 172.21.15.46:0/189925781 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c06d940 con 0x7f3a8c0669a0 2024-04-25T08:44:22.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a8aa00640 1 --2- 172.21.15.46:0/189925781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c0669a0 0x7f3a8c0ccf50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009870 tx=0x7f3a7801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94319bc83913124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a8a000640 1 -- 172.21.15.46:0/189925781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a780b0070 con 0x7f3a8c0669a0 2024-04-25T08:44:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a8a000640 1 -- 172.21.15.46:0/189925781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a780ab600 con 0x7f3a8c0669a0 2024-04-25T08:44:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a8a000640 1 -- 172.21.15.46:0/189925781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a780ab9d0 con 0x7f3a8c0669a0 2024-04-25T08:44:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a90c00640 1 -- 172.21.15.46:0/189925781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c0669a0 msgr2=0x7f3a8c0ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a90c00640 1 --2- 172.21.15.46:0/189925781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c0669a0 0x7f3a8c0ccf50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009870 tx=0x7f3a7801c490 comp rx=0 tx=0).stop 2024-04-25T08:44:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a90c00640 1 -- 172.21.15.46:0/189925781 shutdown_connections 2024-04-25T08:44:22.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a90c00640 1 --2- 172.21.15.46:0/189925781 >> v2:172.21.15.82:3300/0 conn(0x7f3a8c0adf60 0x7f3a8c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a90c00640 1 --2- 172.21.15.46:0/189925781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c0669a0 0x7f3a8c0ccf50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a90c00640 1 --2- 172.21.15.46:0/189925781 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c15b810 0x7f3a8c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a90c00640 1 -- 172.21.15.46:0/189925781 >> 172.21.15.46:0/189925781 conn(0x7f3a8c072710 msgr2=0x7f3a8c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a90c00640 1 -- 172.21.15.46:0/189925781 shutdown_connections 2024-04-25T08:44:22.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.242+0000 7f3a90c00640 1 -- 172.21.15.46:0/189925781 wait complete. 2024-04-25T08:44:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 Processor -- start 2024-04-25T08:44:22.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 -- start start 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a8c0669a0 0x7f3a8c142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c0adf60 0x7f3a8c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c15ad50 0x7f3a8c0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c0cd610 con 0x7f3a8c15ad50 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c0cd490 con 0x7f3a8c0669a0 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c06ef10 con 0x7f3a8c0adf60 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c15ad50 0x7f3a8c0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c15ad50 0x7f3a8c0b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36954/0 (socket says 172.21.15.46:36954) 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8be00640 1 -- 172.21.15.46:0/408114244 learned_addr learned my addr 172.21.15.46:0/408114244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8aa00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c0adf60 0x7f3a8c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8b400640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a8c0669a0 0x7f3a8c142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8be00640 1 -- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c0adf60 msgr2=0x7f3a8c15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8be00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c0adf60 0x7f3a8c15a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8be00640 1 -- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a8c0669a0 msgr2=0x7f3a8c142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8be00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a8c0669a0 0x7f3a8c142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8be00640 1 -- 172.21.15.46:0/408114244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c14ff60 con 0x7f3a8c15ad50 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a8be00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c15ad50 0x7f3a8c0b3b80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3a84007e60 tx=0x7f3a8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a83400640 1 -- 172.21.15.46:0/408114244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84014620 con 0x7f3a8c15ad50 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a83400640 1 -- 172.21.15.46:0/408114244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84019070 con 0x7f3a8c15ad50 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a83400640 1 -- 172.21.15.46:0/408114244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84014a20 con 0x7f3a8c15ad50 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c150200 con 0x7f3a8c15ad50 2024-04-25T08:44:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c1507c0 con 0x7f3a8c15ad50 2024-04-25T08:44:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.246+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8c15d250 con 0x7f3a8c15ad50 2024-04-25T08:44:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.250+0000 7f3a83400640 1 -- 172.21.15.46:0/408114244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a84014bc0 con 0x7f3a8c15ad50 2024-04-25T08:44:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.250+0000 7f3a83400640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a6c066f60 0x7f3a6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.250+0000 7f3a83400640 1 -- 172.21.15.46:0/408114244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a840859a0 con 0x7f3a8c15ad50 2024-04-25T08:44:22.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.250+0000 7f3a8b400640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a6c066f60 0x7f3a6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.250+0000 7f3a8b400640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a6c066f60 0x7f3a6c069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00b930 tx=0x7f3a7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.254+0000 7f3a83400640 1 -- 172.21.15.46:0/408114244 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a8c15d250 con 0x7f3a8c15ad50 2024-04-25T08:44:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.454+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a8c069010 con 0x7f3a8c15ad50 2024-04-25T08:44:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a83400640 1 -- 172.21.15.46:0/408114244 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f3a840568b0 con 0x7f3a8c15ad50 2024-04-25T08:44:22.450 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:22.450 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/850914154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":850914154},{"type":"v1","addr":"172.21.15.46:6835","nonce":850914154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1115060038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1115060038},{"type":"v1","addr":"172.21.15.82:6836","nonce":1115060038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5620,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1383788710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1383788710},{"type":"v1","addr":"172.21.15.82:6837","nonce":1383788710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:44:14.218049+0000","modified":"2024-04-25T08:44:17.246143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/3843568539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3843568539},{"type":"v1","addr":"172.21.15.46:6834","nonce":3843568539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:44:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a6c066f60 msgr2=0x7f3a6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a6c066f60 0x7f3a6c069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00b930 tx=0x7f3a7c008040 comp rx=0 tx=0).stop 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c15ad50 msgr2=0x7f3a8c0b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c15ad50 0x7f3a8c0b3b80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3a84007e60 tx=0x7f3a8400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 shutdown_connections 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a6c066f60 0x7f3a6c069420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a8c15ad50 0x7f3a8c0b3b80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a8c0adf60 0x7f3a8c15a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 --2- 172.21.15.46:0/408114244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a8c0669a0 0x7f3a8c142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 >> 172.21.15.46:0/408114244 conn(0x7f3a8c072710 msgr2=0x7f3a8c0aed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:22.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 shutdown_connections 2024-04-25T08:44:22.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.458+0000 7f3a90c00640 1 -- 172.21.15.46:0/408114244 wait complete. 2024-04-25T08:44:22.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T08:44:22.642 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-25T08:44:22.643 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:44:22.643 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T08:44:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.694+0000 7ff677679080 1 Processor -- start 2024-04-25T08:44:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.694+0000 7ff677679080 1 -- start start 2024-04-25T08:44:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558629636560 0x558629635fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 0x558629655640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586295c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558629639180 con 0x558629580020 2024-04-25T08:44:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558629639000 con 0x558629636560 2024-04-25T08:44:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x558629639300 con 0x558629636db0 2024-04-25T08:44:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff675200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586295c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff675200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586295c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36978/0 (socket says 172.21.15.46:36978) 2024-04-25T08:44:22.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff675200640 1 -- 172.21.15.46:0/1794696369 learned_addr learned my addr 172.21.15.46:0/1794696369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff674800640 1 --2- 172.21.15.46:0/1794696369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558629636560 0x558629635fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff673e00640 1 --2- 172.21.15.46:0/1794696369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 0x558629655640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff675200640 1 -- 172.21.15.46:0/1794696369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 msgr2=0x558629655640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff675200640 1 --2- 172.21.15.46:0/1794696369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 0x558629655640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff675200640 1 -- 172.21.15.46:0/1794696369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558629636560 msgr2=0x558629635fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff675200640 1 --2- 172.21.15.46:0/1794696369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558629636560 0x558629635fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff675200640 1 -- 172.21.15.46:0/1794696369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558629655d80 con 0x558629580020 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff675200640 1 --2- 172.21.15.46:0/1794696369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586295c2b70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff66c01ba40 tx=0x7ff66c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6766395f331e76c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff672a00640 1 -- 172.21.15.46:0/1794696369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0aab60 con 0x558629580020 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff672a00640 1 -- 172.21.15.46:0/1794696369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c0aad00 con 0x558629580020 2024-04-25T08:44:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff672a00640 1 -- 172.21.15.46:0/1794696369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0b4680 con 0x558629580020 2024-04-25T08:44:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 -- 172.21.15.46:0/1794696369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 msgr2=0x5586295c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- 172.21.15.46:0/1794696369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586295c2b70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff66c01ba40 tx=0x7ff66c0033b0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 -- 172.21.15.46:0/1794696369 shutdown_connections 2024-04-25T08:44:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- 172.21.15.46:0/1794696369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586295c2b70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- 172.21.15.46:0/1794696369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 0x558629655640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- 172.21.15.46:0/1794696369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558629636560 0x558629635fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 -- 172.21.15.46:0/1794696369 >> 172.21.15.46:0/1794696369 conn(0x55862957e230 msgr2=0x5586296540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 -- 172.21.15.46:0/1794696369 shutdown_connections 2024-04-25T08:44:22.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 -- 172.21.15.46:0/1794696369 wait complete. 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 Processor -- start 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 -- start start 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586296f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558629636560 0x5586296e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.698+0000 7ff677679080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 0x5586296e5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff677679080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586296562c0 con 0x558629580020 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff677679080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558629656020 con 0x558629636560 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff677679080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x558629656560 con 0x558629636db0 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff674800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586296f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff674800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586296f1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36988/0 (socket says 172.21.15.46:36988) 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff674800640 1 -- 172.21.15.46:0/1779926144 learned_addr learned my addr 172.21.15.46:0/1779926144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff673e00640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558629636560 0x5586296e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff675200640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 0x5586296e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff674800640 1 -- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 msgr2=0x5586296e5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff674800640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 0x5586296e5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff674800640 1 -- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558629636560 msgr2=0x5586296e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff674800640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558629636560 0x5586296e53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff674800640 1 -- 172.21.15.46:0/1779926144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586295c3850 con 0x558629580020 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff675200640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558629636db0 0x5586296e5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff674800640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586296f1a00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff668008740 tx=0x7ff668008b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff671600640 1 -- 172.21.15.46:0/1779926144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668006b10 con 0x558629580020 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff671600640 1 -- 172.21.15.46:0/1779926144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff668019070 con 0x558629580020 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff671600640 1 -- 172.21.15.46:0/1779926144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6680143f0 con 0x558629580020 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5586295c3af0 con 0x558629580020 2024-04-25T08:44:22.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586295c4030 con 0x558629580020 2024-04-25T08:44:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff671600640 1 -- 172.21.15.46:0/1779926144 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff668014590 con 0x558629580020 2024-04-25T08:44:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.702+0000 7ff671600640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff654065970 0x7ff654067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff671600640 1 -- 172.21.15.46:0/1779926144 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff668084380 con 0x558629580020 2024-04-25T08:44:22.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5586296efec0 0x558629657840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53) v9 -- 0x558629657d80 con 0x5586296efec0 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff673e00640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff654065970 0x7ff654067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff675200640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5586296efec0 0x558629657840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff675200640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5586296efec0 0x558629657840 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff673e00640 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff654065970 0x7ff654067e30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff660009770 tx=0x7ff660005ae0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff675200640 1 -- 172.21.15.46:0/1779926144 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7ff66c0ba070 con 0x5586296efec0 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5586296efec0 msgr2=0x558629657840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5586296efec0 0x558629657840 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff654065970 msgr2=0x7ff654067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff654065970 0x7ff654067e30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff660009770 tx=0x7ff660005ae0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 msgr2=0x5586296f1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 --2- 172.21.15.46:0/1779926144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558629580020 0x5586296f1a00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff668008740 tx=0x7ff668008b40 comp rx=0 tx=0).stop 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff675200640 1 -- 172.21.15.46:0/1779926144 reap_dead start 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 shutdown_connections 2024-04-25T08:44:22.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 >> 172.21.15.46:0/1779926144 conn(0x55862957e230 msgr2=0x55862957c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:22.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 shutdown_connections 2024-04-25T08:44:22.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.706+0000 7ff677679080 1 -- 172.21.15.46:0/1779926144 wait complete. 2024-04-25T08:44:22.704 INFO:teuthology.orchestra.run.smithi046.stdout:Yalphaparent_a 2024-04-25T08:44:22.704 DEBUG:teuthology.orchestra.run.smithi046:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T08:44:22.798 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:44:22.798 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "ancestors": [ 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1099511627776, 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "alpha", 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 2 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1, 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "parent_a", 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 4 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-25T08:44:22.799 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 8, 2024-04-25T08:44:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: "old_pools": [] 2024-04-25T08:44:22.800 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:44:22.800 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:22.803 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-25T08:44:22.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a6c4e080 1 Processor -- start 2024-04-25T08:44:22.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a6c4e080 1 -- start start 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a6c4e080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08ef7970 0x55be08ec5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a6c4e080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 0x55be08fa5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a6c4e080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be08f9c2b0 0x55be08fa5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a6c4e080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be08ef7d50 con 0x55be08ef7970 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a6c4e080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be08e6d9f0 con 0x55be08f9c2b0 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a6c4e080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be08f9db10 con 0x55be08f103f0 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08ef7970 0x55be08ec5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08ef7970 0x55be08ec5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36994/0 (socket says 172.21.15.46:36994) 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3e00640 1 -- 172.21.15.46:0/3018408281 learned_addr learned my addr 172.21.15.46:0/3018408281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a4800640 1 --2- 172.21.15.46:0/3018408281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be08f9c2b0 0x55be08fa5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3400640 1 --2- 172.21.15.46:0/3018408281 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 0x55be08fa5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3e00640 1 -- 172.21.15.46:0/3018408281 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 msgr2=0x55be08fa5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3e00640 1 --2- 172.21.15.46:0/3018408281 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 0x55be08fa5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3e00640 1 -- 172.21.15.46:0/3018408281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be08f9c2b0 msgr2=0x55be08fa5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3e00640 1 --2- 172.21.15.46:0/3018408281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be08f9c2b0 0x55be08fa5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3e00640 1 -- 172.21.15.46:0/3018408281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be08ef7500 con 0x55be08ef7970 2024-04-25T08:44:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a3e00640 1 --2- 172.21.15.46:0/3018408281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08ef7970 0x55be08ec5790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52940ac040 tx=0x7f5294003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5772baeb7cd1eadf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a2000640 1 -- 172.21.15.46:0/3018408281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940aa740 con 0x55be08ef7970 2024-04-25T08:44:22.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a2000640 1 -- 172.21.15.46:0/3018408281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52940a6070 con 0x55be08ef7970 2024-04-25T08:44:22.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.902+0000 7f52a2000640 1 -- 172.21.15.46:0/3018408281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940aab40 con 0x55be08ef7970 2024-04-25T08:44:22.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- 172.21.15.46:0/3018408281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08ef7970 msgr2=0x55be08ec5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 --2- 172.21.15.46:0/3018408281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08ef7970 0x55be08ec5790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f52940ac040 tx=0x7f5294003030 comp rx=0 tx=0).stop 2024-04-25T08:44:22.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- 172.21.15.46:0/3018408281 shutdown_connections 2024-04-25T08:44:22.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 --2- 172.21.15.46:0/3018408281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be08f9c2b0 0x55be08fa5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 --2- 172.21.15.46:0/3018408281 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 0x55be08fa5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 --2- 172.21.15.46:0/3018408281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08ef7970 0x55be08ec5790 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- 172.21.15.46:0/3018408281 >> 172.21.15.46:0/3018408281 conn(0x55be08ef2d70 msgr2=0x55be08f9fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:22.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- 172.21.15.46:0/3018408281 shutdown_connections 2024-04-25T08:44:22.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- 172.21.15.46:0/3018408281 wait complete. 2024-04-25T08:44:22.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 Processor -- start 2024-04-25T08:44:22.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- start start 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be08ef7970 0x55be08f969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 0x55be08fa0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08f9c2b0 0x55be08fa0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be08fa6870 con 0x55be08f9c2b0 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be08fa65d0 con 0x55be08ef7970 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be08fa6b10 con 0x55be08f103f0 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a4800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08f9c2b0 0x55be08fa0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a4800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08f9c2b0 0x55be08fa0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37002/0 (socket says 172.21.15.46:37002) 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a4800640 1 -- 172.21.15.46:0/2544446244 learned_addr learned my addr 172.21.15.46:0/2544446244 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:22.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a3400640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 0x55be08fa0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a3e00640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be08ef7970 0x55be08f969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a4800640 1 -- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 msgr2=0x55be08fa0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a4800640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 0x55be08fa0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a4800640 1 -- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be08ef7970 msgr2=0x55be08f969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a4800640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55be08ef7970 0x55be08f969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a4800640 1 -- 172.21.15.46:0/2544446244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be08f9b230 con 0x55be08f9c2b0 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a3400640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55be08f103f0 0x55be08fa0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a4800640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08f9c2b0 0x55be08fa0c60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f529c0106d0 tx=0x7f529c010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a0c00640 1 -- 172.21.15.46:0/2544446244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c018070 con 0x55be08f9c2b0 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a0c00640 1 -- 172.21.15.46:0/2544446244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c00e740 con 0x55be08f9c2b0 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a0c00640 1 -- 172.21.15.46:0/2544446244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c00eb30 con 0x55be08f9c2b0 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55be08f9a7b0 con 0x55be08f9c2b0 2024-04-25T08:44:22.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.906+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be08f9acf0 con 0x55be08f9c2b0 2024-04-25T08:44:22.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.910+0000 7f52a0c00640 1 -- 172.21.15.46:0/2544446244 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f529c01b070 con 0x55be08f9c2b0 2024-04-25T08:44:22.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.910+0000 7f52a0c00640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5284065940 0x7f5284067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.910+0000 7f52a0c00640 1 -- 172.21.15.46:0/2544446244 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f529c0843d0 con 0x55be08f9c2b0 2024-04-25T08:44:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.910+0000 7f52a6c4e080 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55be090387b0 0x55be090439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.910+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53) v9 -- 0x55be08f96f30 con 0x55be090387b0 2024-04-25T08:44:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.910+0000 7f52a3e00640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5284065940 0x7f5284067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.910+0000 7f52a3400640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55be090387b0 0x55be090439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.910+0000 7f52a3400640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55be090387b0 0x55be090439e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.910+0000 7f52a3e00640 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5284065940 0x7f5284067e00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f52940bd000 tx=0x7f52940a8000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a3400640 1 -- 172.21.15.46:0/2544446244 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v53'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f529000f570 con 0x55be090387b0 2024-04-25T08:44:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55be090387b0 msgr2=0x55be090439e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55be090387b0 0x55be090439e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5284065940 msgr2=0x7f5284067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5284065940 0x7f5284067e00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f52940bd000 tx=0x7f52940a8000 comp rx=0 tx=0).stop 2024-04-25T08:44:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08f9c2b0 msgr2=0x55be08fa0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 --2- 172.21.15.46:0/2544446244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be08f9c2b0 0x55be08fa0c60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f529c0106d0 tx=0x7f529c010b90 comp rx=0 tx=0).stop 2024-04-25T08:44:22.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a4800640 1 -- 172.21.15.46:0/2544446244 reap_dead start 2024-04-25T08:44:22.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 shutdown_connections 2024-04-25T08:44:22.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 >> 172.21.15.46:0/2544446244 conn(0x55be08ef2d70 msgr2=0x55be08f109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:22.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 shutdown_connections 2024-04-25T08:44:22.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:22.914+0000 7f52a6c4e080 1 -- 172.21.15.46:0/2544446244 wait complete. 2024-04-25T08:44:22.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T08:44:22.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:22.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-25T08:44:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f8595e00640 1 Processor -- start 2024-04-25T08:44:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f8595e00640 1 -- start start 2024-04-25T08:44:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f8595e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f859006ea20 0x7f85900ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f8595e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8590141580 0x7f85900cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f8595e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85900b2e60 con 0x7f8590140ee0 2024-04-25T08:44:23.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f8595e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85901412c0 con 0x7f8590141580 2024-04-25T08:44:23.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f8595e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85900cef50 con 0x7f859006ea20 2024-04-25T08:44:23.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8590140ee0 0x7f859006e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46326/0 (socket says 172.21.15.46:46326) 2024-04-25T08:44:23.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858ea00640 1 -- 172.21.15.46:0/8993354 learned_addr learned my addr 172.21.15.46:0/8993354 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:23.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858f400640 1 --2- 172.21.15.46:0/8993354 >> v2:172.21.15.46:3300/0 conn(0x7f8590141580 0x7f85900cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858e000640 1 --2- 172.21.15.46:0/8993354 >> v2:172.21.15.82:3300/0 conn(0x7f859006ea20 0x7f85900ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858d600640 1 -- 172.21.15.46:0/8993354 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252918921 0 0) 0x7f85900b2e60 con 0x7f8590140ee0 2024-04-25T08:44:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858d600640 1 -- 172.21.15.46:0/8993354 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8574003530 con 0x7f8590140ee0 2024-04-25T08:44:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858f400640 1 -- 172.21.15.46:0/8993354 >> v2:172.21.15.82:3300/0 conn(0x7f859006ea20 msgr2=0x7f85900ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858f400640 1 --2- 172.21.15.46:0/8993354 >> v2:172.21.15.82:3300/0 conn(0x7f859006ea20 0x7f85900ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858f400640 1 -- 172.21.15.46:0/8993354 >> v1:172.21.15.46:6789/0 conn(0x7f8590140ee0 legacy=0x7f859006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.078+0000 7f858f400640 1 -- 172.21.15.46:0/8993354 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85900b30d0 con 0x7f8590141580 2024-04-25T08:44:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f858f400640 1 --2- 172.21.15.46:0/8993354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8590141580 0x7f85900cea10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f85900cd6c0 tx=0x7f858801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece8fdfdb555e8ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f858d600640 1 -- 172.21.15.46:0/8993354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85880b2070 con 0x7f8590141580 2024-04-25T08:44:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f858d600640 1 -- 172.21.15.46:0/8993354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85880a59f0 con 0x7f8590141580 2024-04-25T08:44:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f858d600640 1 -- 172.21.15.46:0/8993354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85880a5da0 con 0x7f8590141580 2024-04-25T08:44:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f8595e00640 1 -- 172.21.15.46:0/8993354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8590141580 msgr2=0x7f85900cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f8595e00640 1 --2- 172.21.15.46:0/8993354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8590141580 0x7f85900cea10 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f85900cd6c0 tx=0x7f858801c930 comp rx=0 tx=0).stop 2024-04-25T08:44:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f8595e00640 1 -- 172.21.15.46:0/8993354 shutdown_connections 2024-04-25T08:44:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f8595e00640 1 --2- 172.21.15.46:0/8993354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8590141580 0x7f85900cea10 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f8595e00640 1 --2- 172.21.15.46:0/8993354 >> v2:172.21.15.82:3300/0 conn(0x7f859006ea20 0x7f85900ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f8595e00640 1 -- 172.21.15.46:0/8993354 >> 172.21.15.46:0/8993354 conn(0x7f8590067050 msgr2=0x7f859006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f8595e00640 1 -- 172.21.15.46:0/8993354 shutdown_connections 2024-04-25T08:44:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f8595e00640 1 -- 172.21.15.46:0/8993354 wait complete. 2024-04-25T08:44:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.082+0000 7f8595e00640 1 Processor -- start 2024-04-25T08:44:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8595e00640 1 -- start start 2024-04-25T08:44:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8595e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859006ea20 0x7f859013d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8595e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8590140ee0 0x7f85900ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8595e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8590141580 0x7f85900addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8595e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85900b4540 con 0x7f859006ea20 2024-04-25T08:44:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8595e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85900b43c0 con 0x7f8590140ee0 2024-04-25T08:44:23.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8595e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85900b46c0 con 0x7f8590141580 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859006ea20 0x7f859013d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859006ea20 0x7f859013d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37016/0 (socket says 172.21.15.46:37016) 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858ea00640 1 -- 172.21.15.46:0/3512367017 learned_addr learned my addr 172.21.15.46:0/3512367017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858f400640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8590141580 0x7f85900addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858e000640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8590140ee0 0x7f85900ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858ea00640 1 -- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8590141580 msgr2=0x7f85900addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858ea00640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8590141580 0x7f85900addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858ea00640 1 -- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8590140ee0 msgr2=0x7f85900ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858ea00640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8590140ee0 0x7f85900ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858ea00640 1 -- 172.21.15.46:0/3512367017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85900abcf0 con 0x7f859006ea20 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858e000640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8590140ee0 0x7f85900ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f858ea00640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859006ea20 0x7f859013d8d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f858000c480 tx=0x7f858000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8587400640 1 -- 172.21.15.46:0/3512367017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580017070 con 0x7f859006ea20 2024-04-25T08:44:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8587400640 1 -- 172.21.15.46:0/3512367017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8580005cf0 con 0x7f859006ea20 2024-04-25T08:44:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8587400640 1 -- 172.21.15.46:0/3512367017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580007290 con 0x7f859006ea20 2024-04-25T08:44:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8595e00640 1 -- 172.21.15.46:0/3512367017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85900abfd0 con 0x7f859006ea20 2024-04-25T08:44:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.086+0000 7f8595e00640 1 -- 172.21.15.46:0/3512367017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85900a9270 con 0x7f859006ea20 2024-04-25T08:44:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f8587400640 1 -- 172.21.15.46:0/3512367017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f858001a070 con 0x7f859006ea20 2024-04-25T08:44:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f8587400640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8570066f60 0x7f8570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7ff293e00640 1 Processor -- start 2024-04-25T08:44:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f8587400640 1 -- 172.21.15.46:0/3512367017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8580084560 con 0x7f859006ea20 2024-04-25T08:44:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7ff293e00640 1 -- start start 2024-04-25T08:44:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f85992b0000 1 Processor -- start 2024-04-25T08:44:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f858e000640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8570066f60 0x7f8570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f85992b0000 1 -- start start 2024-04-25T08:44:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f858e000640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8570066f60 0x7f8570069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f85900b0470 tx=0x7f857c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7ff293e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff28c0b2d10 0x7ff28c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f85992b0000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588ddc90 0x55c4588de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f85992b0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588de5b0 0x55c458893d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7ff293e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff28c15f4c0 0x7ff28c099f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f85992b0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c458894290 0x55c458894670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f85992b0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45877af40 con 0x55c458894290 2024-04-25T08:44:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f85992b0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4586e2070 con 0x55c4588de5b0 2024-04-25T08:44:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7f85992b0000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45866a1a0 con 0x55c4588ddc90 2024-04-25T08:44:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.090+0000 7ff293e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff28c06d1e0 con 0x7ff28c06bf30 2024-04-25T08:44:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff293e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff28c0cf9e0 con 0x7ff28c15f4c0 2024-04-25T08:44:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff293e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c06cc20 con 0x7ff28c0b2d10 2024-04-25T08:44:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff290a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff28c15f4c0 0x7ff28c099f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c458894290 0x55c458894670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff290a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff28c15f4c0 0x7ff28c099f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37024/0 (socket says 172.21.15.46:37024) 2024-04-25T08:44:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff290a00640 1 -- 172.21.15.46:0/3093406826 learned_addr learned my addr 172.21.15.46:0/3093406826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588de5b0 0x55c458893d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c458894290 0x55c458894670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37034/0 (socket says 172.21.15.46:37034) 2024-04-25T08:44:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858f400640 1 -- 172.21.15.46:0/2824641914 learned_addr learned my addr 172.21.15.46:0/2824641914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff291400640 1 --2- 172.21.15.46:0/3093406826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff28c0b2d10 0x7ff28c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858ea00640 1 --2- 172.21.15.46:0/2824641914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588ddc90 0x55c4588de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858f400640 1 -- 172.21.15.46:0/2824641914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588ddc90 msgr2=0x55c4588de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858f400640 1 --2- 172.21.15.46:0/2824641914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588ddc90 0x55c4588de070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858f400640 1 -- 172.21.15.46:0/2824641914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588de5b0 msgr2=0x55c458893d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858f400640 1 --2- 172.21.15.46:0/2824641914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588de5b0 0x55c458893d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858f400640 1 -- 172.21.15.46:0/2824641914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c458668510 con 0x55c458894290 2024-04-25T08:44:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858e000640 1 --2- 172.21.15.46:0/2824641914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588de5b0 0x55c458893d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff290a00640 1 -- 172.21.15.46:0/3093406826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff28c0b2d10 msgr2=0x7ff28c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff290a00640 1 --2- 172.21.15.46:0/3093406826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff28c0b2d10 0x7ff28c099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff290a00640 1 -- 172.21.15.46:0/3093406826 >> v1:172.21.15.46:6789/0 conn(0x7ff28c06bf30 legacy=0x7ff28c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff290a00640 1 -- 172.21.15.46:0/3093406826 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c06cf00 con 0x7ff28c15f4c0 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f858f400640 1 --2- 172.21.15.46:0/2824641914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c458894290 0x55c458894670 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f858801c970 tx=0x7f8588004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3605e378a4f10d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f857be00640 1 -- 172.21.15.46:0/2824641914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85880be070 con 0x55c458894290 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f857be00640 1 -- 172.21.15.46:0/2824641914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85880b3040 con 0x55c458894290 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f857be00640 1 -- 172.21.15.46:0/2824641914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85880b2040 con 0x55c458894290 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 -- 172.21.15.46:0/2824641914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c458894290 msgr2=0x55c458894670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 --2- 172.21.15.46:0/2824641914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c458894290 0x55c458894670 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f858801c970 tx=0x7f8588004ac0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 -- 172.21.15.46:0/2824641914 shutdown_connections 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 --2- 172.21.15.46:0/2824641914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c458894290 0x55c458894670 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 --2- 172.21.15.46:0/2824641914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588de5b0 0x55c458893d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 --2- 172.21.15.46:0/2824641914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588ddc90 0x55c4588de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 -- 172.21.15.46:0/2824641914 >> 172.21.15.46:0/2824641914 conn(0x55c458890240 msgr2=0x55c458892630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 -- 172.21.15.46:0/2824641914 shutdown_connections 2024-04-25T08:44:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 -- 172.21.15.46:0/2824641914 wait complete. 2024-04-25T08:44:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 -- start start 2024-04-25T08:44:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7f85992b0000 1 Processor -- start 2024-04-25T08:44:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.094+0000 7ff290a00640 1 --2- 172.21.15.46:0/3093406826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c15f4c0 0x7ff28c099f40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff27801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a601021c8066c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff28be00640 1 -- 172.21.15.46:0/3093406826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2780a95f0 con 0x7ff28c15f4c0 2024-04-25T08:44:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff28be00640 1 -- 172.21.15.46:0/3093406826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2780a9790 con 0x7ff28c15f4c0 2024-04-25T08:44:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff28be00640 1 -- 172.21.15.46:0/3093406826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2780a9b60 con 0x7ff28c15f4c0 2024-04-25T08:44:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 0x55c45889ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 0x55c45889f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c45889f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45869c6c0 con 0x55c4588de5b0 2024-04-25T08:44:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4586de110 con 0x55c4588ddc90 2024-04-25T08:44:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45889d200 con 0x55c458894290 2024-04-25T08:44:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 10 client.0 _ll_register_callbacks cb 0x55c45887f570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:44:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 -- 172.21.15.46:0/3093406826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c15f4c0 msgr2=0x7ff28c099f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 --2- 172.21.15.46:0/3093406826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c15f4c0 0x7ff28c099f40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff27801c530 comp rx=0 tx=0).stop 2024-04-25T08:44:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c45889f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 0x55c45889f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c45889f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37050/0 (socket says 172.21.15.46:37050) 2024-04-25T08:44:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858f400640 1 -- 172.21.15.46:0/289625978 learned_addr learned my addr 172.21.15.46:0/289625978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 -- 172.21.15.46:0/3093406826 shutdown_connections 2024-04-25T08:44:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 --2- 172.21.15.46:0/3093406826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c15f4c0 0x7ff28c099f40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 --2- 172.21.15.46:0/3093406826 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff28c0b2d10 0x7ff28c099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 -- 172.21.15.46:0/3093406826 >> 172.21.15.46:0/3093406826 conn(0x7ff28c0b3160 msgr2=0x7ff28c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858ea00640 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 0x55c45889ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 -- 172.21.15.46:0/3093406826 shutdown_connections 2024-04-25T08:44:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858e000640 1 -- 172.21.15.46:0/289625978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 msgr2=0x55c45889ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858e000640 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 0x55c45889ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858e000640 1 -- 172.21.15.46:0/289625978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 msgr2=0x55c45889f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858e000640 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c45889f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858e000640 1 -- 172.21.15.46:0/289625978 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c45866a3b0 con 0x55c4588ddc90 2024-04-25T08:44:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 -- 172.21.15.46:0/3093406826 wait complete. 2024-04-25T08:44:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858e000640 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 0x55c45889f370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f857c011b20 tx=0x7f857c0046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=930be4e417933848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f8578c00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T08:44:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f8578c00640 1 -- 172.21.15.46:0/289625978 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c027070 con 0x55c4588ddc90 2024-04-25T08:44:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f8578c00640 1 -- 172.21.15.46:0/289625978 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f857c014710 con 0x55c4588ddc90 2024-04-25T08:44:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 -- 172.21.15.46:0/289625978 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85880b3040 con 0x55c4588ddc90 2024-04-25T08:44:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f8578c00640 1 -- 172.21.15.46:0/289625978 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f857c01d040 con 0x55c4588ddc90 2024-04-25T08:44:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 10 client.5791 fetch_fsmap learned FSMap version 25 2024-04-25T08:44:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 -- 172.21.15.46:0/289625978 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c458763e00 con 0x55c4588ddc90 2024-04-25T08:44:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f8578c00640 1 -- 172.21.15.46:0/289625978 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85880b3040 con 0x55c4588ddc90 2024-04-25T08:44:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 Processor -- start 2024-04-25T08:44:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f8578c00640 1 -- 172.21.15.46:0/289625978 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f857c0059a0 con 0x55c4588ddc90 2024-04-25T08:44:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7ff293e00640 1 -- start start 2024-04-25T08:44:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 10 client.5791 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:44:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 10 client.5791 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:44:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 20 client.5791 populate_metadata read hostname 'smithi046' 2024-04-25T08:44:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c4588a0120 0x55c4588ef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 4 client.5791 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:44:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f85992b0000 1 -- 172.21.15.46:0/289625978 --> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c45867dd60 con 0x55c4588a0120 2024-04-25T08:44:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.098+0000 7f858ea00640 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c4588a0120 0x55c4588ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f858ea00640 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c4588a0120 0x55c4588ef410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7725aa32b98ed3 server_cookie=7f8af031f714cf86 in_seq=0 out_seq=0 2024-04-25T08:44:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f8578c00640 10 client.5791 ms_handle_connect on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff293e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff28c06bf30 0x7ff28c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff293e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0b2d10 0x7ff28c0b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff293e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c15f4c0 0x7ff28c152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff293e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c15dc70 con 0x7ff28c0b2d10 2024-04-25T08:44:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff293e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c15daf0 con 0x7ff28c15f4c0 2024-04-25T08:44:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff293e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28c15ddf0 con 0x7ff28c06bf30 2024-04-25T08:44:23.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f8578c00640 1 -- 172.21.15.46:0/289625978 <== mds.0 v2:172.21.15.46:6832/3843568539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c45867dd60 con 0x55c4588a0120 2024-04-25T08:44:23.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f8578c00640 10 client.5791 handle_command_reply: tid=0 2024-04-25T08:44:23.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 client.5791 shutdown 2024-04-25T08:44:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff290a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0b2d10 0x7ff28c0b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff290a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0b2d10 0x7ff28c0b6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37060/0 (socket says 172.21.15.46:37060) 2024-04-25T08:44:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff290a00640 1 -- 172.21.15.46:0/372006715 learned_addr learned my addr 172.21.15.46:0/372006715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff291400640 1 --2- 172.21.15.46:0/372006715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff28c06bf30 0x7ff28c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff291e00640 1 --2- 172.21.15.46:0/372006715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c15f4c0 0x7ff28c152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 -- 172.21.15.46:0/289625978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 msgr2=0x55c45889f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 0x55c45889f370 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f857c011b20 tx=0x7f857c0046d0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 -- 172.21.15.46:0/289625978 shutdown_connections 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c4588a0120 0x55c4588ef410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c45889f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 0x55c45889f370 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 --2- 172.21.15.46:0/289625978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 0x55c45889ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 -- 172.21.15.46:0/289625978 >> 172.21.15.46:0/289625978 conn(0x55c458890240 msgr2=0x55c458892ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f8578c00640 0 client.5791 ms_handle_reset on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 -- 172.21.15.46:0/289625978 shutdown_connections 2024-04-25T08:44:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 1 -- 172.21.15.46:0/289625978 wait complete. 2024-04-25T08:44:23.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff291400640 1 -- 172.21.15.46:0/372006715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c15f4c0 msgr2=0x7ff28c152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff291400640 1 --2- 172.21.15.46:0/372006715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff28c15f4c0 0x7ff28c152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff291400640 1 -- 172.21.15.46:0/372006715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0b2d10 msgr2=0x7ff28c0b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff291400640 1 --2- 172.21.15.46:0/372006715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0b2d10 0x7ff28c0b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff291400640 1 -- 172.21.15.46:0/372006715 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c1567f0 con 0x7ff28c06bf30 2024-04-25T08:44:23.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff290a00640 1 --2- 172.21.15.46:0/372006715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff28c0b2d10 0x7ff28c0b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7f85992b0000 20 client.5791 trim_cache size 0 max 16384 2024-04-25T08:44:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff291400640 1 --2- 172.21.15.46:0/372006715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff28c06bf30 0x7ff28c1597f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff28000e870 tx=0x7ff28000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff289600640 1 -- 172.21.15.46:0/372006715 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280017070 con 0x7ff28c06bf30 2024-04-25T08:44:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff289600640 1 -- 172.21.15.46:0/372006715 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28000c940 con 0x7ff28c06bf30 2024-04-25T08:44:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff289600640 1 -- 172.21.15.46:0/372006715 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000ccf0 con 0x7ff28c06bf30 2024-04-25T08:44:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff293e00640 1 -- 172.21.15.46:0/372006715 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28c156ad0 con 0x7ff28c06bf30 2024-04-25T08:44:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff293e00640 1 -- 172.21.15.46:0/372006715 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28c157090 con 0x7ff28c06bf30 2024-04-25T08:44:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.102+0000 7ff293e00640 1 -- 172.21.15.46:0/372006715 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff28c069010 con 0x7ff28c06bf30 2024-04-25T08:44:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.106+0000 7ff289600640 1 -- 172.21.15.46:0/372006715 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff28001a070 con 0x7ff28c06bf30 2024-04-25T08:44:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.106+0000 7ff289600640 1 --2- 172.21.15.46:0/372006715 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff26c066f60 0x7ff26c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.106+0000 7ff289600640 1 -- 172.21.15.46:0/372006715 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff280085120 con 0x7ff28c06bf30 2024-04-25T08:44:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.106+0000 7ff289600640 1 -- 172.21.15.46:0/372006715 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7ff2800876c0 con 0x7ff28c06bf30 2024-04-25T08:44:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.106+0000 7ff290a00640 1 --2- 172.21.15.46:0/372006715 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff26c066f60 0x7ff26c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.106+0000 7ff293e00640 1 -- 172.21.15.46:0/372006715 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ff28c09a530 con 0x7ff28c06bf30 2024-04-25T08:44:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.106+0000 7ff290a00640 1 --2- 172.21.15.46:0/372006715 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff26c066f60 0x7ff26c069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff27801c570 tx=0x7ff2780b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f85992b0000 1 Processor -- start 2024-04-25T08:44:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f85992b0000 1 -- start start 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f85992b0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c458894290 0x55c458893640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f85992b0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588ddc90 0x55c458893b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f85992b0000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588de5b0 0x55c4588ef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f85992b0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8590202d60 con 0x55c4588ddc90 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f85992b0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85900a9db0 con 0x55c458894290 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f85992b0000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85901fa080 con 0x55c4588de5b0 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588ddc90 0x55c458893b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588ddc90 0x55c458893b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37066/0 (socket says 172.21.15.46:37066) 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858e000640 1 -- 172.21.15.46:0/3960533440 learned_addr learned my addr 172.21.15.46:0/3960533440 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858f400640 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588de5b0 0x55c4588ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858ea00640 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c458894290 0x55c458893640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858e000640 1 -- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588de5b0 msgr2=0x55c4588ef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858e000640 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588de5b0 0x55c4588ef410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858e000640 1 -- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c458894290 msgr2=0x55c458893640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858e000640 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c458894290 0x55c458893640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858e000640 1 -- 172.21.15.46:0/3960533440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c458739c50 con 0x55c4588ddc90 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858f400640 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588de5b0 0x55c4588ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.118+0000 7f858ea00640 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c458894290 0x55c458893640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858e000640 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588ddc90 0x55c458893b80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f857c004710 tx=0x7f857c024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d771d2e78633f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f857b400640 1 -- 172.21.15.46:0/3960533440 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c027050 con 0x55c4588ddc90 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f857b400640 1 -- 172.21.15.46:0/3960533440 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c01e040 con 0x55c4588ddc90 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f857b400640 1 -- 172.21.15.46:0/3960533440 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c014910 con 0x55c4588ddc90 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 -- 172.21.15.46:0/3960533440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588ddc90 msgr2=0x55c458893b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588ddc90 0x55c458893b80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f857c004710 tx=0x7f857c024000 comp rx=0 tx=0).stop 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 -- 172.21.15.46:0/3960533440 shutdown_connections 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c4588de5b0 0x55c4588ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588ddc90 0x55c458893b80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 --2- 172.21.15.46:0/3960533440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c458894290 0x55c458893640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 -- 172.21.15.46:0/3960533440 >> 172.21.15.46:0/3960533440 conn(0x55c458890240 msgr2=0x55c458898d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 -- 172.21.15.46:0/3960533440 shutdown_connections 2024-04-25T08:44:23.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 -- 172.21.15.46:0/3960533440 wait complete. 2024-04-25T08:44:23.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 -- start start 2024-04-25T08:44:23.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 Processor -- start 2024-04-25T08:44:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 0x55c4588f1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 0x55c4588f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c4588fa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c458684bd0 con 0x55c4588de5b0 2024-04-25T08:44:23.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85900a97e0 con 0x55c4588ddc90 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c458798f20 con 0x55c458894290 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f85992b0000 10 client.0 _ll_register_callbacks cb 0x7f8574003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c4588fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c4588fa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37072/0 (socket says 172.21.15.46:37072) 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 0x55c4588f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858f400640 1 -- 172.21.15.46:0/895347764 learned_addr learned my addr 172.21.15.46:0/895347764 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858ea00640 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 0x55c4588f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858f400640 1 -- 172.21.15.46:0/895347764 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 msgr2=0x55c4588f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858f400640 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 0x55c4588f1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858f400640 1 -- 172.21.15.46:0/895347764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 msgr2=0x55c4588f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858f400640 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 0x55c4588f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858f400640 1 -- 172.21.15.46:0/895347764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4586aa120 con 0x55c4588de5b0 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f858f400640 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c4588fa420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f858801a560 tx=0x7f8588004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aba84b64bd42dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.122+0000 7f8578c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8578c00640 1 -- 172.21.15.46:0/895347764 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85880b2040 con 0x55c4588de5b0 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8578c00640 1 -- 172.21.15.46:0/895347764 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8588003c80 con 0x55c4588de5b0 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 -- 172.21.15.46:0/895347764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f857c01e040 con 0x55c4588de5b0 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8578c00640 1 -- 172.21.15.46:0/895347764 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85880d4050 con 0x55c4588de5b0 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 10 client.5943 fetch_fsmap learned FSMap version 25 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 -- 172.21.15.46:0/895347764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c458689170 con 0x55c4588de5b0 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8578c00640 1 -- 172.21.15.46:0/895347764 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c01e040 con 0x55c4588de5b0 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8578c00640 1 -- 172.21.15.46:0/895347764 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85880d7020 con 0x55c4588de5b0 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 10 client.5943 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 10 client.5943 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 20 client.5943 populate_metadata read hostname 'smithi046' 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c45893df70 0x55c45893e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 4 client.5943 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T08:44:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 -- 172.21.15.46:0/895347764 --> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55c4586ef5b0 con 0x55c45893df70 2024-04-25T08:44:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f858ea00640 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c45893df70 0x55c45893e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f858ea00640 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c45893df70 0x55c45893e380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c168874fe578225b server_cookie=fc0f9b7f5919060b in_seq=0 out_seq=0 2024-04-25T08:44:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8578c00640 10 client.5943 ms_handle_connect on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8578c00640 1 -- 172.21.15.46:0/895347764 <== mds.0 v2:172.21.15.46:6832/3843568539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c4586ef5b0 con 0x55c45893df70 2024-04-25T08:44:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8578c00640 10 client.5943 handle_command_reply: tid=0 2024-04-25T08:44:23.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 client.5943 shutdown 2024-04-25T08:44:23.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 -- 172.21.15.46:0/895347764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 msgr2=0x55c4588fa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c4588fa420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f858801a560 tx=0x7f8588004a50 comp rx=0 tx=0).stop 2024-04-25T08:44:23.123 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-25T08:44:23.126+0000 7fb8b1c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5752=0-4194304@1} caps={5752=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5752 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560e37de1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 -- 172.21.15.46:0/895347764 shutdown_connections 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55c45893df70 0x55c45893e380 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c4588de5b0 0x55c4588fa420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c4588ddc90 0x55c4588f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 --2- 172.21.15.46:0/895347764 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c458894290 0x55c4588f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 -- 172.21.15.46:0/895347764 >> 172.21.15.46:0/895347764 conn(0x55c458890240 msgr2=0x55c4588994a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8578c00640 0 client.5943 ms_handle_reset on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 -- 172.21.15.46:0/895347764 shutdown_connections 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 1 -- 172.21.15.46:0/895347764 wait complete. 2024-04-25T08:44:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f85992b0000 20 client.5943 trim_cache size 0 max 16384 2024-04-25T08:44:23.125 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:44:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:44:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "e62a7666-8eee-4334-a4e0-d8933af7b8b2", 2024-04-25T08:44:23.125 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:44:23.125 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 -- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8570066f60 msgr2=0x7f8570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8570066f60 0x7f8570069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f85900b0470 tx=0x7f857c009190 comp rx=0 tx=0).stop 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 -- 172.21.15.46:0/3512367017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859006ea20 msgr2=0x7f859013d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859006ea20 0x7f859013d8d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f858000c480 tx=0x7f858000c940 comp rx=0 tx=0).stop 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 -- 172.21.15.46:0/3512367017 shutdown_connections 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8570066f60 0x7f8570069420 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8590141580 0x7f85900addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8590140ee0 0x7f85900ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 --2- 172.21.15.46:0/3512367017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f859006ea20 0x7f859013d8d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 -- 172.21.15.46:0/3512367017 >> 172.21.15.46:0/3512367017 conn(0x7f8590067050 msgr2=0x7f859006d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 -- 172.21.15.46:0/3512367017 shutdown_connections 2024-04-25T08:44:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.126+0000 7f8595e00640 1 -- 172.21.15.46:0/3512367017 wait complete. 2024-04-25T08:44:23.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:23.141 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:44:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.290+0000 7fbb7fa00640 1 Processor -- start 2024-04-25T08:44:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.290+0000 7fbb7fa00640 1 -- start start 2024-04-25T08:44:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.290+0000 7fbb7fa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb780cd4d0 0x7fbb78168690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.290+0000 7fbb7fa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7809a780 con 0x7fbb7815dd50 2024-04-25T08:44:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.290+0000 7fbb7fa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7809aa60 con 0x7fbb7806f4d0 2024-04-25T08:44:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.290+0000 7fbb7fa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb780708c0 con 0x7fbb780cd4d0 2024-04-25T08:44:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb6fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbb7815dd50 0x7fbb780cff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46348/0 (socket says 172.21.15.46:46348) 2024-04-25T08:44:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb6fe00640 1 -- 172.21.15.46:0/1837184679 learned_addr learned my addr 172.21.15.46:0/1837184679 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb7da00640 1 --2- 172.21.15.46:0/1837184679 >> v2:172.21.15.46:3300/0 conn(0x7fbb780cd4d0 0x7fbb78168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb77e00640 1 -- 172.21.15.46:0/1837184679 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537195006 0 0) 0x7fbb7809a780 con 0x7fbb7815dd50 2024-04-25T08:44:23.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb77e00640 1 -- 172.21.15.46:0/1837184679 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb5c003530 con 0x7fbb7815dd50 2024-04-25T08:44:23.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb7da00640 1 -- 172.21.15.46:0/1837184679 >> v1:172.21.15.82:6789/0 conn(0x7fbb7806f4d0 legacy=0x7fbb780cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb7da00640 1 -- 172.21.15.46:0/1837184679 >> v1:172.21.15.46:6789/0 conn(0x7fbb7815dd50 legacy=0x7fbb780cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb7da00640 1 -- 172.21.15.46:0/1837184679 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7809ad40 con 0x7fbb780cd4d0 2024-04-25T08:44:23.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb7da00640 1 --2- 172.21.15.46:0/1837184679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb780cd4d0 0x7fbb78168690 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009d90 tx=0x7fbb7001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec57ee4d5083b30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb77e00640 1 -- 172.21.15.46:0/1837184679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700b2070 con 0x7fbb780cd4d0 2024-04-25T08:44:23.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb77e00640 1 -- 172.21.15.46:0/1837184679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb700a5a00 con 0x7fbb780cd4d0 2024-04-25T08:44:23.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.294+0000 7fbb77e00640 1 -- 172.21.15.46:0/1837184679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700a5dd0 con 0x7fbb780cd4d0 2024-04-25T08:44:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1837184679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb780cd4d0 msgr2=0x7fbb78168690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 --2- 172.21.15.46:0/1837184679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb780cd4d0 0x7fbb78168690 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009d90 tx=0x7fbb7001c940 comp rx=0 tx=0).stop 2024-04-25T08:44:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1837184679 shutdown_connections 2024-04-25T08:44:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 --2- 172.21.15.46:0/1837184679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb780cd4d0 0x7fbb78168690 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1837184679 >> 172.21.15.46:0/1837184679 conn(0x7fbb780b3160 msgr2=0x7fbb780b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1837184679 shutdown_connections 2024-04-25T08:44:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1837184679 wait complete. 2024-04-25T08:44:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 Processor -- start 2024-04-25T08:44:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 -- start start 2024-04-25T08:44:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7806f4d0 0x7fbb780c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb780cd4d0 0x7fbb78157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb7815dd50 0x7fbb7814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7816b800 con 0x7fbb7806f4d0 2024-04-25T08:44:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb780d0620 con 0x7fbb780cd4d0 2024-04-25T08:44:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7fa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7816b980 con 0x7fbb7815dd50 2024-04-25T08:44:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7806f4d0 0x7fbb780c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7806f4d0 0x7fbb780c91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37092/0 (socket says 172.21.15.46:37092) 2024-04-25T08:44:23.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7d000640 1 -- 172.21.15.46:0/1565728577 learned_addr learned my addr 172.21.15.46:0/1565728577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb780cd4d0 0x7fbb78157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7da00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb7815dd50 0x7fbb7814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7d000640 1 -- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb7815dd50 msgr2=0x7fbb7814fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7d000640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb7815dd50 0x7fbb7814fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7d000640 1 -- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb780cd4d0 msgr2=0x7fbb78157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7d000640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb780cd4d0 0x7fbb78157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7d000640 1 -- 172.21.15.46:0/1565728577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb781565c0 con 0x7fbb7806f4d0 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.298+0000 7fbb7da00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb7815dd50 0x7fbb7814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.302+0000 7fbb7d000640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7806f4d0 0x7fbb780c91c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c420 tx=0x7fbb6800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.302+0000 7fbb75600640 1 -- 172.21.15.46:0/1565728577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb680052d0 con 0x7fbb7806f4d0 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.302+0000 7fbb75600640 1 -- 172.21.15.46:0/1565728577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb68005470 con 0x7fbb7806f4d0 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.302+0000 7fbb75600640 1 -- 172.21.15.46:0/1565728577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68007260 con 0x7fbb7806f4d0 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.302+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1565728577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb781568a0 con 0x7fbb7806f4d0 2024-04-25T08:44:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.302+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1565728577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb78156e60 con 0x7fbb7806f4d0 2024-04-25T08:44:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.302+0000 7fbb75600640 1 -- 172.21.15.46:0/1565728577 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb68007400 con 0x7fbb7806f4d0 2024-04-25T08:44:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.302+0000 7fbb75600640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbb54066f90 0x7fbb54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb75600640 1 -- 172.21.15.46:0/1565728577 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb68084710 con 0x7fbb7806f4d0 2024-04-25T08:44:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbb54066f90 0x7fbb54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 Processor -- start 2024-04-25T08:44:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 -- start start 2024-04-25T08:44:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbb54066f90 0x7fbb54069450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb7815c9c0 tx=0x7fbb60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 0x55f2e8052db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8008b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80090a0 0x55f2e8009480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e7e55ff0 con 0x55f2e80532f0 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e7dde150 con 0x55f2e80090a0 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2e7e52090 con 0x55f2e80529d0 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8008b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8008b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37096/0 (socket says 172.21.15.46:37096) 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 -- 172.21.15.46:0/4145394714 learned_addr learned my addr 172.21.15.46:0/4145394714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb7d000640 1 --2- 172.21.15.46:0/4145394714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 0x55f2e8052db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb7da00640 1 --2- 172.21.15.46:0/4145394714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80090a0 0x55f2e8009480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 -- 172.21.15.46:0/4145394714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 msgr2=0x55f2e8052db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/4145394714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 0x55f2e8052db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 -- 172.21.15.46:0/4145394714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80090a0 msgr2=0x55f2e8009480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/4145394714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80090a0 0x55f2e8009480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 -- 172.21.15.46:0/4145394714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2e7fec0d0 con 0x55f2e80532f0 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/4145394714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8008b60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbb60001e90 tx=0x7fbb60011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f305312967ac3de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6d600640 1 -- 172.21.15.46:0/4145394714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60005cf0 con 0x55f2e80532f0 2024-04-25T08:44:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6d600640 1 -- 172.21.15.46:0/4145394714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6000e750 con 0x55f2e80532f0 2024-04-25T08:44:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb6d600640 1 -- 172.21.15.46:0/4145394714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000eb80 con 0x55f2e80532f0 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 -- 172.21.15.46:0/4145394714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 msgr2=0x55f2e8008b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 --2- 172.21.15.46:0/4145394714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8008b60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbb60001e90 tx=0x7fbb60011000 comp rx=0 tx=0).stop 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 -- 172.21.15.46:0/4145394714 shutdown_connections 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 --2- 172.21.15.46:0/4145394714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80090a0 0x55f2e8009480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 --2- 172.21.15.46:0/4145394714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8008b60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 --2- 172.21.15.46:0/4145394714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 0x55f2e8052db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 -- 172.21.15.46:0/4145394714 >> 172.21.15.46:0/4145394714 conn(0x55f2e8005020 msgr2=0x55f2e8007440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 -- 172.21.15.46:0/4145394714 shutdown_connections 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.306+0000 7fbb82e43000 1 -- 172.21.15.46:0/4145394714 wait complete. 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 -- start start 2024-04-25T08:44:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 Processor -- start 2024-04-25T08:44:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 0x55f2e80377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80529d0 0x55f2e8037d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 0x55f2e8038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e7e1a250 con 0x55f2e80529d0 2024-04-25T08:44:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e7e10690 con 0x55f2e80532f0 2024-04-25T08:44:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2e7ddea40 con 0x55f2e80090a0 2024-04-25T08:44:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 10 client.0 _ll_register_callbacks cb 0x55f2e7ff3e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80529d0 0x55f2e8037d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb6fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80529d0 0x55f2e8037d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37108/0 (socket says 172.21.15.46:37108) 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb6fe00640 1 -- 172.21.15.46:0/3218329443 learned_addr learned my addr 172.21.15.46:0/3218329443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb7d000640 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 0x55f2e80377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb7da00640 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 0x55f2e8038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb6fe00640 1 -- 172.21.15.46:0/3218329443 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 msgr2=0x55f2e80377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 0x55f2e80377f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb6fe00640 1 -- 172.21.15.46:0/3218329443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 msgr2=0x55f2e8038270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 0x55f2e8038270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb6fe00640 1 -- 172.21.15.46:0/3218329443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2e7f031c0 con 0x55f2e80529d0 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80529d0 0x55f2e8037d30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55f2e7e123d0 tx=0x7fbb60013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8409918f1045658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb66000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb66000640 1 -- 172.21.15.46:0/3218329443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb60013420 con 0x55f2e80529d0 2024-04-25T08:44:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb66000640 1 -- 172.21.15.46:0/3218329443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb60010050 con 0x55f2e80529d0 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 -- 172.21.15.46:0/3218329443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb6000e750 con 0x55f2e80529d0 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb66000640 1 -- 172.21.15.46:0/3218329443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb600135c0 con 0x55f2e80529d0 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 10 client.5958 fetch_fsmap learned FSMap version 25 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 -- 172.21.15.46:0/3218329443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f2e7eff170 con 0x55f2e80529d0 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb66000640 1 -- 172.21.15.46:0/3218329443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6000e750 con 0x55f2e80529d0 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb66000640 1 -- 172.21.15.46:0/3218329443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb60013760 con 0x55f2e80529d0 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 10 client.5958 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 10 client.5958 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 20 client.5958 populate_metadata read hostname 'smithi046' 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55f2e8062c80 0x55f2e8065070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 4 client.5958 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.310+0000 7fbb82e43000 1 -- 172.21.15.46:0/3218329443 --> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2e7df1c60 con 0x55f2e8062c80 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb7d000640 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55f2e8062c80 0x55f2e8065070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb7d000640 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55f2e8062c80 0x55f2e8065070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b1254c354e5a53 server_cookie=304097117ffa545a in_seq=0 out_seq=0 2024-04-25T08:44:23.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb66000640 10 client.5958 ms_handle_connect on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb66000640 1 -- 172.21.15.46:0/3218329443 <== mds.0 v2:172.21.15.46:6832/3843568539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2e7df1c60 con 0x55f2e8062c80 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb66000640 10 client.5958 handle_command_reply: tid=0 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 client.5958 shutdown 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 -- 172.21.15.46:0/3218329443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80529d0 msgr2=0x55f2e8037d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80529d0 0x55f2e8037d30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55f2e7e123d0 tx=0x7fbb60013000 comp rx=0 tx=0).stop 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 -- 172.21.15.46:0/3218329443 shutdown_connections 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55f2e8062c80 0x55f2e8065070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 0x55f2e8038270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80529d0 0x55f2e8037d30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3218329443 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 0x55f2e80377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb66000640 0 client.5958 ms_handle_reset on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 -- 172.21.15.46:0/3218329443 >> 172.21.15.46:0/3218329443 conn(0x55f2e8005020 msgr2=0x55f2e8007ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 -- 172.21.15.46:0/3218329443 shutdown_connections 2024-04-25T08:44:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 1 -- 172.21.15.46:0/3218329443 wait complete. 2024-04-25T08:44:23.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.314+0000 7fbb82e43000 20 client.5958 trim_cache size 0 max 16384 2024-04-25T08:44:23.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 Processor -- start 2024-04-25T08:44:23.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 -- start start 2024-04-25T08:44:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 0x55f2e8008a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80529d0 0x55f2e805fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8060380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78203400 con 0x55f2e80532f0 2024-04-25T08:44:23.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78202580 con 0x55f2e80529d0 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7806d400 con 0x55f2e80090a0 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8060380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8060380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37120/0 (socket says 172.21.15.46:37120) 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7da00640 1 -- 172.21.15.46:0/850657538 learned_addr learned my addr 172.21.15.46:0/850657538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/850657538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80529d0 0x55f2e805fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7d000640 1 --2- 172.21.15.46:0/850657538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 0x55f2e8008a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7da00640 1 -- 172.21.15.46:0/850657538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 msgr2=0x55f2e8008a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7da00640 1 --2- 172.21.15.46:0/850657538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 0x55f2e8008a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7da00640 1 -- 172.21.15.46:0/850657538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80529d0 msgr2=0x55f2e805fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7da00640 1 --2- 172.21.15.46:0/850657538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80529d0 0x55f2e805fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7da00640 1 -- 172.21.15.46:0/850657538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2e7e9cde0 con 0x55f2e80532f0 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb7da00640 1 --2- 172.21.15.46:0/850657538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8060380 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009d90 tx=0x7fbb70004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e518093a05af15e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb6cc00640 1 -- 172.21.15.46:0/850657538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700b2040 con 0x55f2e80532f0 2024-04-25T08:44:23.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb6cc00640 1 -- 172.21.15.46:0/850657538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb700a5bb0 con 0x55f2e80532f0 2024-04-25T08:44:23.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb6cc00640 1 -- 172.21.15.46:0/850657538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700ad440 con 0x55f2e80532f0 2024-04-25T08:44:23.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 -- 172.21.15.46:0/850657538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 msgr2=0x55f2e8060380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 --2- 172.21.15.46:0/850657538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8060380 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009d90 tx=0x7fbb70004ac0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 -- 172.21.15.46:0/850657538 shutdown_connections 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 --2- 172.21.15.46:0/850657538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80532f0 0x55f2e8060380 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 --2- 172.21.15.46:0/850657538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80529d0 0x55f2e805fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 --2- 172.21.15.46:0/850657538 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80090a0 0x55f2e8008a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 -- 172.21.15.46:0/850657538 >> 172.21.15.46:0/850657538 conn(0x55f2e8005020 msgr2=0x55f2e805f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 -- 172.21.15.46:0/850657538 shutdown_connections 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.330+0000 7fbb82e43000 1 -- 172.21.15.46:0/850657538 wait complete. 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 -- start start 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 Processor -- start 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80090a0 0x55f2e80b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 0x55f2e80b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 0x55f2e8064a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2e7df0e00 con 0x55f2e80090a0 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb78201f80 con 0x55f2e80532f0 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2e7e55300 con 0x55f2e80529d0 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 10 client.0 _ll_register_callbacks cb 0x55f2e7fd4040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80090a0 0x55f2e80b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80090a0 0x55f2e80b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37126/0 (socket says 172.21.15.46:37126) 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7d000640 1 -- 172.21.15.46:0/3793455088 learned_addr learned my addr 172.21.15.46:0/3793455088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 0x55f2e80b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7da00640 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 0x55f2e8064a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7d000640 1 -- 172.21.15.46:0/3793455088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 msgr2=0x55f2e80b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7d000640 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 0x55f2e80b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7d000640 1 -- 172.21.15.46:0/3793455088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 msgr2=0x55f2e8064a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7d000640 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 0x55f2e8064a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7d000640 1 -- 172.21.15.46:0/3793455088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2e7df78e0 con 0x55f2e80090a0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb7d000640 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80090a0 0x55f2e80b2720 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbb68054af0 tx=0x7fbb68000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556a237800d606e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb66000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb66000640 1 -- 172.21.15.46:0/3793455088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68059e60 con 0x55f2e80090a0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb66000640 1 -- 172.21.15.46:0/3793455088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb6806cb80 con 0x55f2e80090a0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 -- 172.21.15.46:0/3793455088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb700a5bb0 con 0x55f2e80090a0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb66000640 1 -- 172.21.15.46:0/3793455088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb68082050 con 0x55f2e80090a0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 10 client.5964 fetch_fsmap learned FSMap version 25 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 -- 172.21.15.46:0/3793455088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55f2e7eaf8f0 con 0x55f2e80090a0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb66000640 1 -- 172.21.15.46:0/3793455088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb700a5bb0 con 0x55f2e80090a0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb66000640 1 -- 172.21.15.46:0/3793455088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbb68054240 con 0x55f2e80090a0 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 10 client.5964 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 10 client.5964 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 20 client.5964 populate_metadata read hostname 'smithi046' 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55f2e8065790 0x55f2e8065ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 4 client.5964 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:44:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb82e43000 1 -- 172.21.15.46:0/3793455088 --> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f2e7e63530 con 0x55f2e8065790 2024-04-25T08:44:23.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55f2e8065790 0x55f2e8065ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb6fe00640 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55f2e8065790 0x55f2e8065ba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=823683d8093bf58c server_cookie=1eefab7c1608a8e0 in_seq=0 out_seq=0 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.334+0000 7fbb66000640 10 client.5964 ms_handle_connect on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb66000640 1 -- 172.21.15.46:0/3793455088 <== mds.0 v2:172.21.15.46:6832/3843568539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55f2e7e63530 con 0x55f2e8065790 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb66000640 10 client.5964 handle_command_reply: tid=0 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 client.5964 shutdown 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 -- 172.21.15.46:0/3793455088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80090a0 msgr2=0x55f2e80b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80090a0 0x55f2e80b2720 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbb68054af0 tx=0x7fbb68000ee0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 -- 172.21.15.46:0/3793455088 shutdown_connections 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x55f2e8065790 0x55f2e8065ba0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f2e80532f0 0x55f2e8064a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55f2e80529d0 0x55f2e80b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 --2- 172.21.15.46:0/3793455088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2e80090a0 0x55f2e80b2720 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb66000640 0 client.5964 ms_handle_reset on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 -- 172.21.15.46:0/3793455088 >> 172.21.15.46:0/3793455088 conn(0x55f2e8005020 msgr2=0x55f2e805ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 -- 172.21.15.46:0/3793455088 shutdown_connections 2024-04-25T08:44:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 1 -- 172.21.15.46:0/3793455088 wait complete. 2024-04-25T08:44:23.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb82e43000 20 client.5964 trim_cache size 0 max 16384 2024-04-25T08:44:23.334 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:44:23.335 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:44:23.335 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:44:23.335 INFO:teuthology.orchestra.run.smithi046.stdout: "e62a7666-8eee-4334-a4e0-d8933af7b8b2": { 2024-04-25T08:44:23.335 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-25T08:44:23.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "e62a7666-8eee-4334-a4e0-d8933af7b8b2", 2024-04-25T08:44:23.335 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-04-25T08:44:23.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:23.335 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:23.335 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:44:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbb54066f90 msgr2=0x7fbb54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbb54066f90 0x7fbb54069450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbb7815c9c0 tx=0x7fbb60009040 comp rx=0 tx=0).stop 2024-04-25T08:44:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1565728577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7806f4d0 msgr2=0x7fbb780c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7806f4d0 0x7fbb780c91c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbb6800c420 tx=0x7fbb6800c8e0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1565728577 shutdown_connections 2024-04-25T08:44:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbb54066f90 0x7fbb54069450 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbb7815dd50 0x7fbb7814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb780cd4d0 0x7fbb78157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 --2- 172.21.15.46:0/1565728577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb7806f4d0 0x7fbb780c91c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1565728577 >> 172.21.15.46:0/1565728577 conn(0x7fbb780b3160 msgr2=0x7fbb780cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1565728577 shutdown_connections 2024-04-25T08:44:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:23.338+0000 7fbb7fa00640 1 -- 172.21.15.46:0/1565728577 wait complete. 2024-04-25T08:44:23.350 INFO:tasks.cephfs.filesystem:scrub status for tag:e62a7666-8eee-4334-a4e0-d8933af7b8b2 - {'path': '/', 'tag': 'e62a7666-8eee-4334-a4e0-d8933af7b8b2', 'options': 'recursive,repair'} 2024-04-25T08:44:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:24.882+0000 7ff289600640 1 -- 172.21.15.46:0/372006715 <== mon.2 v2:172.21.15.82:3301/0 7 ==== log(6 entries from seq 1 at 2024-04-25T08:44:23.129523+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7ff280053950 con 0x7ff28c06bf30 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: cluster: 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: id: 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: health: HEALTH_OK 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: services: 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: mgr: x(active, since 2m), standbys: y 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-25T08:44:24.877 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: data: 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: volumes: 1/1 healthy 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: pools: 3 pools, 97 pgs 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: objects: 24 objects, 579 KiB 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: pgs: 97 active+clean 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: io: 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:44:23.129523+0000 mds.a [INF] scrub queued for path: / 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:44:23.129526+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:44:23.129530+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-25T08:44:24.878 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:44:23.130366+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-25T08:44:24.879 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:44:23.130378+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-25T08:44:24.879 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:44:23.130995+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T08:44:25.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:25.882+0000 7ff289600640 1 -- 172.21.15.46:0/372006715 <== mon.2 v2:172.21.15.82:3301/0 8 ==== log(1 entries from seq 7 at 2024-04-25T08:44:23.983807+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff280053bb0 con 0x7ff28c06bf30 2024-04-25T08:44:25.876 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:44:23.983807+0000 mds.a [INF] scrub summary: idle 2024-04-25T08:44:50.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:44:50.280 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:44:53.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:53.351 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:44:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac2e00640 1 Processor -- start 2024-04-25T08:44:53.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac2e00640 1 -- start start 2024-04-25T08:44:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac2e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feabc1440d0 0x7feabc0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac2e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feabc06f150 0x7feabc0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac2e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc15e440 con 0x7feabc143990 2024-04-25T08:44:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac2e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feabc15ed30 con 0x7feabc06f150 2024-04-25T08:44:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac2e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feabc15e6e0 con 0x7feabc1440d0 2024-04-25T08:44:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac0e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7feabc06f150 0x7feabc0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feabbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7feabc143990 0x7feabc06ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58188/0 (socket says 172.21.15.46:58188) 2024-04-25T08:44:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feabbe00640 1 -- 172.21.15.46:0/793612740 learned_addr learned my addr 172.21.15.46:0/793612740 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:53.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feab3400640 1 --2- 172.21.15.46:0/793612740 >> v2:172.21.15.82:3300/0 conn(0x7feabc1440d0 0x7feabc0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac0e00640 1 -- 172.21.15.46:0/793612740 >> v2:172.21.15.82:3300/0 conn(0x7feabc1440d0 msgr2=0x7feabc0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac0e00640 1 --2- 172.21.15.46:0/793612740 >> v2:172.21.15.82:3300/0 conn(0x7feabc1440d0 0x7feabc0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac0e00640 1 -- 172.21.15.46:0/793612740 >> v1:172.21.15.46:6789/0 conn(0x7feabc143990 legacy=0x7feabc06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feac0e00640 1 -- 172.21.15.46:0/793612740 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc15e9c0 con 0x7feabc06f150 2024-04-25T08:44:53.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.514+0000 7feabb400640 1 -- 172.21.15.46:0/793612740 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945882876 0 0) 0x7feabc15e440 con 0x7feabc143990 2024-04-25T08:44:53.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac0e00640 1 --2- 172.21.15.46:0/793612740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc06f150 0x7feabc0b2720 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7feabc070c80 tx=0x7feab401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e94bfc5352349ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feabb400640 1 -- 172.21.15.46:0/793612740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40b2070 con 0x7feabc06f150 2024-04-25T08:44:53.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feabb400640 1 -- 172.21.15.46:0/793612740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab40a57f0 con 0x7feabc06f150 2024-04-25T08:44:53.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feabb400640 1 -- 172.21.15.46:0/793612740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40a5bc0 con 0x7feabc06f150 2024-04-25T08:44:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 -- 172.21.15.46:0/793612740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc06f150 msgr2=0x7feabc0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 --2- 172.21.15.46:0/793612740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc06f150 0x7feabc0b2720 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7feabc070c80 tx=0x7feab401c930 comp rx=0 tx=0).stop 2024-04-25T08:44:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 -- 172.21.15.46:0/793612740 shutdown_connections 2024-04-25T08:44:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 --2- 172.21.15.46:0/793612740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc06f150 0x7feabc0b2720 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 --2- 172.21.15.46:0/793612740 >> v2:172.21.15.82:3300/0 conn(0x7feabc1440d0 0x7feabc0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 -- 172.21.15.46:0/793612740 >> 172.21.15.46:0/793612740 conn(0x7feabc06ac10 msgr2=0x7feabc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 -- 172.21.15.46:0/793612740 shutdown_connections 2024-04-25T08:44:53.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 -- 172.21.15.46:0/793612740 wait complete. 2024-04-25T08:44:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 Processor -- start 2024-04-25T08:44:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.518+0000 7feac2e00640 1 -- start start 2024-04-25T08:44:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feac2e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc06f150 0x7feabc1595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feac2e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc143990 0x7feabc1356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feac2e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feabc1440d0 0x7feabc135c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feac2e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0b35f0 con 0x7feabc143990 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feac2e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0b3470 con 0x7feabc1440d0 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feac2e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0b3770 con 0x7feabc06f150 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc143990 0x7feabc1356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc143990 0x7feabc1356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60178/0 (socket says 172.21.15.46:60178) 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab3400640 1 -- 172.21.15.46:0/655702424 learned_addr learned my addr 172.21.15.46:0/655702424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feabbe00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc06f150 0x7feabc1595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feac0e00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feabc1440d0 0x7feabc135c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab3400640 1 -- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc06f150 msgr2=0x7feabc1595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab3400640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc06f150 0x7feabc1595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab3400640 1 -- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feabc1440d0 msgr2=0x7feabc135c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab3400640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feabc1440d0 0x7feabc135c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab3400640 1 -- 172.21.15.46:0/655702424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc136d50 con 0x7feabc143990 2024-04-25T08:44:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feabbe00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc06f150 0x7feabc1595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab3400640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc143990 0x7feabc1356d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7feaa400d5f0 tx=0x7feaa400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab8c00640 1 -- 172.21.15.46:0/655702424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa40125f0 con 0x7feabc143990 2024-04-25T08:44:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab8c00640 1 -- 172.21.15.46:0/655702424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa4019070 con 0x7feabc143990 2024-04-25T08:44:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feab8c00640 1 -- 172.21.15.46:0/655702424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa40129b0 con 0x7feabc143990 2024-04-25T08:44:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feac2e00640 1 -- 172.21.15.46:0/655702424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc13c280 con 0x7feabc143990 2024-04-25T08:44:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.522+0000 7feac2e00640 1 -- 172.21.15.46:0/655702424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc136550 con 0x7feabc143990 2024-04-25T08:44:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab8c00640 1 -- 172.21.15.46:0/655702424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feaa4012b50 con 0x7feabc143990 2024-04-25T08:44:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab8c00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fea98066f60 0x7fea98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab8c00640 1 -- 172.21.15.46:0/655702424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feaa4085540 con 0x7feabc143990 2024-04-25T08:44:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feabbe00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fea98066f60 0x7fea98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac6208000 1 Processor -- start 2024-04-25T08:44:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac6208000 1 -- start start 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feabbe00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fea98066f60 0x7fea98069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feaac007fd0 tx=0x7feaac007c20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac6208000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a153210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac6208000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a106440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac6208000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a106980 0x56197a106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac6208000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561979f51ff0 con 0x56197a153750 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac6208000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561979eda150 con 0x56197a152e30 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac6208000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561979f4e090 con 0x56197a106980 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a106440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60188/0 (socket says 172.21.15.46:60188) 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab3400640 1 -- 172.21.15.46:0/1038835108 learned_addr learned my addr 172.21.15.46:0/1038835108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feabbe00640 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac0e00640 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a106980 0x56197a106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab3400640 1 -- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a106980 msgr2=0x56197a106d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab3400640 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a106980 0x56197a106d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab3400640 1 -- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 msgr2=0x56197a153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab3400640 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab3400640 1 -- 172.21.15.46:0/1038835108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56197a0e80d0 con 0x56197a153750 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feabbe00640 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feac0e00640 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a106980 0x56197a106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab3400640 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a106440 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feaa4052110 tx=0x7feaa4056470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb72121eae42397d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab0c00640 1 -- 172.21.15.46:0/1038835108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa405c0c0 con 0x56197a153750 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab0c00640 1 -- 172.21.15.46:0/1038835108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa405e3c0 con 0x56197a153750 2024-04-25T08:44:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.526+0000 7feab0c00640 1 -- 172.21.15.46:0/1038835108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4053fb0 con 0x56197a153750 2024-04-25T08:44:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- 172.21.15.46:0/1038835108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 msgr2=0x56197a106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a106440 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7feaa4052110 tx=0x7feaa4056470 comp rx=0 tx=0).stop 2024-04-25T08:44:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- 172.21.15.46:0/1038835108 shutdown_connections 2024-04-25T08:44:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a106980 0x56197a106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a106440 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 --2- 172.21.15.46:0/1038835108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- 172.21.15.46:0/1038835108 >> 172.21.15.46:0/1038835108 conn(0x56197a1339c0 msgr2=0x56197a135db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- 172.21.15.46:0/1038835108 shutdown_connections 2024-04-25T08:44:53.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- 172.21.15.46:0/1038835108 wait complete. 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- start start 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 Processor -- start 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a15a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a152e30 0x56197a15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a10f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561979f16250 con 0x56197a153750 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561979f0c690 con 0x56197a106980 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561979edaa40 con 0x56197a152e30 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 10 client.0 _ll_register_callbacks cb 0x56197a0efe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a10f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a10f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60200/0 (socket says 172.21.15.46:60200) 2024-04-25T08:44:53.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac0e00640 1 -- 172.21.15.46:0/1952873589 learned_addr learned my addr 172.21.15.46:0/1952873589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feabbe00640 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feab3400640 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a152e30 0x56197a15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac0e00640 1 -- 172.21.15.46:0/1952873589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a152e30 msgr2=0x56197a15a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac0e00640 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a152e30 0x56197a15a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac0e00640 1 -- 172.21.15.46:0/1952873589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 msgr2=0x56197a15a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac0e00640 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a15a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac0e00640 1 -- 172.21.15.46:0/1952873589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561979fff1c0 con 0x56197a153750 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feabbe00640 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac0e00640 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a10f360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feab401ce10 tx=0x7feab401b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edebe89b7e3c95a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feaa9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feaa9600640 1 -- 172.21.15.46:0/1952873589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40bf070 con 0x56197a153750 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feaa9600640 1 -- 172.21.15.46:0/1952873589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feab40a5ad0 con 0x56197a153750 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- 172.21.15.46:0/1952873589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feaa405e3c0 con 0x56197a153750 2024-04-25T08:44:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feaa9600640 1 -- 172.21.15.46:0/1952873589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab40ca050 con 0x56197a153750 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 10 client.5979 fetch_fsmap learned FSMap version 25 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feac6208000 1 -- 172.21.15.46:0/1952873589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x561979ffb170 con 0x56197a153750 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.530+0000 7feaa9600640 1 -- 172.21.15.46:0/1952873589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa405e3c0 con 0x56197a153750 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feaa9600640 1 -- 172.21.15.46:0/1952873589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feab40cd020 con 0x56197a153750 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 10 client.5979 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 10 client.5979 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 20 client.5979 populate_metadata read hostname 'smithi046' 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x56197a1100a0 0x56197a110480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 4 client.5979 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 -- 172.21.15.46:0/1952873589 --> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561979eedc60 con 0x56197a1100a0 2024-04-25T08:44:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feabbe00640 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x56197a1100a0 0x56197a110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feabbe00640 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x56197a1100a0 0x56197a110480 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588bda4529200a04 server_cookie=8e1b7dfc256cbc27 in_seq=0 out_seq=0 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feaa9600640 10 client.5979 ms_handle_connect on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feaa9600640 1 -- 172.21.15.46:0/1952873589 <== mds.0 v2:172.21.15.46:6832/3843568539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561979eedc60 con 0x56197a1100a0 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feaa9600640 10 client.5979 handle_command_reply: tid=0 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 client.5979 shutdown 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 -- 172.21.15.46:0/1952873589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 msgr2=0x56197a10f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a10f360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7feab401ce10 tx=0x7feab401b170 comp rx=0 tx=0).stop 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 -- 172.21.15.46:0/1952873589 shutdown_connections 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x56197a1100a0 0x56197a110480 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a10f360 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a152e30 0x56197a15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 --2- 172.21.15.46:0/1952873589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feaa9600640 0 client.5979 ms_handle_reset on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 -- 172.21.15.46:0/1952873589 >> 172.21.15.46:0/1952873589 conn(0x56197a1339c0 msgr2=0x56197a135db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 -- 172.21.15.46:0/1952873589 shutdown_connections 2024-04-25T08:44:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 1 -- 172.21.15.46:0/1952873589 wait complete. 2024-04-25T08:44:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.534+0000 7feac6208000 20 client.5979 trim_cache size 0 max 16384 2024-04-25T08:44:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 Processor -- start 2024-04-25T08:44:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 -- start start 2024-04-25T08:44:53.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a1af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a152e30 0x56197a1af590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a1afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc1fec10 con 0x56197a153750 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0b07b0 con 0x56197a106980 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc13b960 con 0x56197a152e30 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a1afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a1afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60216/0 (socket says 172.21.15.46:60216) 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac0e00640 1 -- 172.21.15.46:0/2064125870 learned_addr learned my addr 172.21.15.46:0/2064125870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feabbe00640 1 --2- 172.21.15.46:0/2064125870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a1af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac0e00640 1 -- 172.21.15.46:0/2064125870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a152e30 msgr2=0x56197a1af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac0e00640 1 --2- 172.21.15.46:0/2064125870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a152e30 0x56197a1af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac0e00640 1 -- 172.21.15.46:0/2064125870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 msgr2=0x56197a1af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac0e00640 1 --2- 172.21.15.46:0/2064125870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a1af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac0e00640 1 -- 172.21.15.46:0/2064125870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561979f98de0 con 0x56197a153750 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feabbe00640 1 --2- 172.21.15.46:0/2064125870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a1af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:53.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac0e00640 1 --2- 172.21.15.46:0/2064125870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a1afad0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feab4002240 tx=0x7feab40b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1dacef101559e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feaabe00640 1 -- 172.21.15.46:0/2064125870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40a5e60 con 0x56197a153750 2024-04-25T08:44:53.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feaabe00640 1 -- 172.21.15.46:0/2064125870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab40b3040 con 0x56197a153750 2024-04-25T08:44:53.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feaabe00640 1 -- 172.21.15.46:0/2064125870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40bf040 con 0x56197a153750 2024-04-25T08:44:53.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 -- 172.21.15.46:0/2064125870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 msgr2=0x56197a1afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 --2- 172.21.15.46:0/2064125870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a1afad0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feab4002240 tx=0x7feab40b9000 comp rx=0 tx=0).stop 2024-04-25T08:44:53.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 -- 172.21.15.46:0/2064125870 shutdown_connections 2024-04-25T08:44:53.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 --2- 172.21.15.46:0/2064125870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a153750 0x56197a1afad0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 --2- 172.21.15.46:0/2064125870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a152e30 0x56197a1af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 --2- 172.21.15.46:0/2064125870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a106980 0x56197a1af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 -- 172.21.15.46:0/2064125870 >> 172.21.15.46:0/2064125870 conn(0x56197a1339c0 msgr2=0x56197a120500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 -- 172.21.15.46:0/2064125870 shutdown_connections 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.550+0000 7feac6208000 1 -- 172.21.15.46:0/2064125870 wait complete. 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 -- start start 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 Processor -- start 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a106980 0x56197a152530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a157e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a153750 0x56197a1583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561979eece00 con 0x56197a106980 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc13bf60 con 0x56197a152e30 2024-04-25T08:44:53.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561979f51300 con 0x56197a153750 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 10 client.0 _ll_register_callbacks cb 0x56197a0d0040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feabbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a106980 0x56197a152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feabbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a106980 0x56197a152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60228/0 (socket says 172.21.15.46:60228) 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feabbe00640 1 -- 172.21.15.46:0/3690178863 learned_addr learned my addr 172.21.15.46:0/3690178863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feab3400640 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac0e00640 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a153750 0x56197a1583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feabbe00640 1 -- 172.21.15.46:0/3690178863 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a153750 msgr2=0x56197a1583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feabbe00640 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a153750 0x56197a1583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feabbe00640 1 -- 172.21.15.46:0/3690178863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 msgr2=0x56197a157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feabbe00640 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feabbe00640 1 -- 172.21.15.46:0/3690178863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561979ef38e0 con 0x56197a106980 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feab3400640 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feabbe00640 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a106980 0x56197a152530 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feaac001d90 tx=0x7feaac000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8833c07e5a7875ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feaa9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:44:53.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feaa9600640 1 -- 172.21.15.46:0/3690178863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac014930 con 0x56197a106980 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feaa9600640 1 -- 172.21.15.46:0/3690178863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feaac02dc80 con 0x56197a106980 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 -- 172.21.15.46:0/3690178863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feab40b3040 con 0x56197a106980 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feaa9600640 1 -- 172.21.15.46:0/3690178863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feaac014ad0 con 0x56197a106980 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 10 client.5985 fetch_fsmap learned FSMap version 25 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 -- 172.21.15.46:0/3690178863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x561979fab8f0 con 0x56197a106980 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feaa9600640 1 -- 172.21.15.46:0/3690178863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab40b3040 con 0x56197a106980 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feaa9600640 1 -- 172.21.15.46:0/3690178863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feaac034050 con 0x56197a106980 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 10 client.5985 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 10 client.5985 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 20 client.5985 populate_metadata read hostname 'smithi046' 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x56197a1b2c70 0x56197a1b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 4 client.5985 mds_command: new command op to 5733 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feac6208000 1 -- 172.21.15.46:0/3690178863 --> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561979f5f530 con 0x56197a1b2c70 2024-04-25T08:44:53.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.554+0000 7feab3400640 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x56197a1b2c70 0x56197a1b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feab3400640 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x56197a1b2c70 0x56197a1b3080 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8fb377e06064f5b server_cookie=bafa81423c9131b5 in_seq=0 out_seq=0 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feaa9600640 10 client.5985 ms_handle_connect on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feaa9600640 1 -- 172.21.15.46:0/3690178863 <== mds.0 v2:172.21.15.46:6832/3843568539 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561979f5f530 con 0x56197a1b2c70 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feaa9600640 10 client.5985 handle_command_reply: tid=0 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 client.5985 shutdown 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 -- 172.21.15.46:0/3690178863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a106980 msgr2=0x56197a152530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a106980 0x56197a152530 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feaac001d90 tx=0x7feaac000ee0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 -- 172.21.15.46:0/3690178863 shutdown_connections 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.46:6832/3843568539,v1:172.21.15.46:6834/3843568539] conn(0x56197a1b2c70 0x56197a1b3080 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56197a153750 0x56197a1583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56197a152e30 0x56197a157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 --2- 172.21.15.46:0/3690178863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56197a106980 0x56197a152530 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feaa9600640 0 client.5985 ms_handle_reset on v2:172.21.15.46:6832/3843568539 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 -- 172.21.15.46:0/3690178863 >> 172.21.15.46:0/3690178863 conn(0x56197a1339c0 msgr2=0x56197a15ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 -- 172.21.15.46:0/3690178863 shutdown_connections 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 1 -- 172.21.15.46:0/3690178863 wait complete. 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac6208000 20 client.5985 trim_cache size 0 max 16384 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:44:53.555 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:44:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 -- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fea98066f60 msgr2=0x7fea98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fea98066f60 0x7fea98069420 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7feaac007fd0 tx=0x7feaac007c20 comp rx=0 tx=0).stop 2024-04-25T08:44:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 -- 172.21.15.46:0/655702424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc143990 msgr2=0x7feabc1356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc143990 0x7feabc1356d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7feaa400d5f0 tx=0x7feaa400dab0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 -- 172.21.15.46:0/655702424 shutdown_connections 2024-04-25T08:44:53.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fea98066f60 0x7fea98069420 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feabc1440d0 0x7feabc135c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feabc143990 0x7feabc1356d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 --2- 172.21.15.46:0/655702424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7feabc06f150 0x7feabc1595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 -- 172.21.15.46:0/655702424 >> 172.21.15.46:0/655702424 conn(0x7feabc06ac10 msgr2=0x7feabc13d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:53.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 -- 172.21.15.46:0/655702424 shutdown_connections 2024-04-25T08:44:53.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.558+0000 7feac2e00640 1 -- 172.21.15.46:0/655702424 wait complete. 2024-04-25T08:44:53.570 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:44:53.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.818+0000 7f6f99a00640 1 Processor -- start 2024-04-25T08:44:53.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.818+0000 7f6f99a00640 1 -- start start 2024-04-25T08:44:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.818+0000 7f6f99a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f9415db60 0x7f6f940cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.818+0000 7f6f99a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f940b06a0 0x7f6f940cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.818+0000 7f6f99a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9409af70 con 0x7f6f940cfe70 2024-04-25T08:44:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.818+0000 7f6f99a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9406d730 con 0x7f6f940b06a0 2024-04-25T08:44:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.818+0000 7f6f99a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9406ce90 con 0x7f6f9415db60 2024-04-25T08:44:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f8b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f940b06a0 0x7f6f940cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f8b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f940b06a0 0x7f6f940cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60234/0 (socket says 172.21.15.46:60234) 2024-04-25T08:44:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f8b400640 1 -- 172.21.15.46:0/1779327476 learned_addr learned my addr 172.21.15.46:0/1779327476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:53.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f93e00640 1 --2- 172.21.15.46:0/1779327476 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f9415db60 0x7f6f940cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f8b400640 1 -- 172.21.15.46:0/1779327476 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f9415db60 msgr2=0x7f6f940cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f8b400640 1 --2- 172.21.15.46:0/1779327476 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f9415db60 0x7f6f940cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f8b400640 1 -- 172.21.15.46:0/1779327476 >> v1:172.21.15.82:6789/0 conn(0x7f6f940cfe70 legacy=0x7f6f94099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f8b400640 1 -- 172.21.15.46:0/1779327476 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9406d170 con 0x7f6f940b06a0 2024-04-25T08:44:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f93e00640 1 --2- 172.21.15.46:0/1779327476 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f9415db60 0x7f6f940cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:53.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f8b400640 1 --2- 172.21.15.46:0/1779327476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 0x7f6f940cf930 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009870 tx=0x7f6f7c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5930ec24d0fc93c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f93400640 1 -- 172.21.15.46:0/1779327476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0b2070 con 0x7f6f940b06a0 2024-04-25T08:44:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f93400640 1 -- 172.21.15.46:0/1779327476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7c0a9070 con 0x7f6f940b06a0 2024-04-25T08:44:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.822+0000 7f6f93400640 1 -- 172.21.15.46:0/1779327476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0ad880 con 0x7f6f940b06a0 2024-04-25T08:44:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 -- 172.21.15.46:0/1779327476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 msgr2=0x7f6f940cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 --2- 172.21.15.46:0/1779327476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 0x7f6f940cf930 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009870 tx=0x7f6f7c01c560 comp rx=0 tx=0).stop 2024-04-25T08:44:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 -- 172.21.15.46:0/1779327476 shutdown_connections 2024-04-25T08:44:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 --2- 172.21.15.46:0/1779327476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 0x7f6f940cf930 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 --2- 172.21.15.46:0/1779327476 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f9415db60 0x7f6f940cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 -- 172.21.15.46:0/1779327476 >> 172.21.15.46:0/1779327476 conn(0x7f6f940d14f0 msgr2=0x7f6f940d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 -- 172.21.15.46:0/1779327476 shutdown_connections 2024-04-25T08:44:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 -- 172.21.15.46:0/1779327476 wait complete. 2024-04-25T08:44:53.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 Processor -- start 2024-04-25T08:44:53.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 -- start start 2024-04-25T08:44:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 0x7f6f94154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f940cfe70 0x7f6f94154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9415db60 0x7f6f9415f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9416aa40 con 0x7f6f940b06a0 2024-04-25T08:44:53.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9416abc0 con 0x7f6f9415db60 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9416ad40 con 0x7f6f940cfe70 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 0x7f6f94154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 0x7f6f94154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60246/0 (socket says 172.21.15.46:60246) 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f93e00640 1 -- 172.21.15.46:0/1034650553 learned_addr learned my addr 172.21.15.46:0/1034650553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f8b400640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f940cfe70 0x7f6f94154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f99000640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9415db60 0x7f6f9415f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f93e00640 1 -- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f940cfe70 msgr2=0x7f6f94154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f93e00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f940cfe70 0x7f6f94154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f93e00640 1 -- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9415db60 msgr2=0x7f6f9415f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.826+0000 7f6f93e00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9415db60 0x7f6f9415f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f93e00640 1 -- 172.21.15.46:0/1034650553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f94155510 con 0x7f6f940b06a0 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f99000640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9415db60 0x7f6f9415f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f93e00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 0x7f6f94154f30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400d800 tx=0x7f6f8400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f90c00640 1 -- 172.21.15.46:0/1034650553 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400b6c0 con 0x7f6f940b06a0 2024-04-25T08:44:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f90c00640 1 -- 172.21.15.46:0/1034650553 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f84019070 con 0x7f6f940b06a0 2024-04-25T08:44:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f90c00640 1 -- 172.21.15.46:0/1034650553 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8400baa0 con 0x7f6f940b06a0 2024-04-25T08:44:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f941557f0 con 0x7f6f940b06a0 2024-04-25T08:44:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f94155db0 con 0x7f6f940b06a0 2024-04-25T08:44:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f94069010 con 0x7f6f940b06a0 2024-04-25T08:44:53.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f90c00640 1 -- 172.21.15.46:0/1034650553 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f8400bc40 con 0x7f6f940b06a0 2024-04-25T08:44:53.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f90c00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f70066f60 0x7f6f70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:53.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f90c00640 1 -- 172.21.15.46:0/1034650553 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6f84085530 con 0x7f6f940b06a0 2024-04-25T08:44:53.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.830+0000 7f6f8b400640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f70066f60 0x7f6f70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:53.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.834+0000 7f6f8b400640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f70066f60 0x7f6f70069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0af000 tx=0x7f6f7c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:53.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:53.838+0000 7f6f90c00640 1 -- 172.21.15.46:0/1034650553 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f840563a0 con 0x7f6f940b06a0 2024-04-25T08:44:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.030+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f94073d90 con 0x7f6f940b06a0 2024-04-25T08:44:54.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.030+0000 7f6f90c00640 1 -- 172.21.15.46:0/1034650553 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f6f84055ca0 con 0x7f6f940b06a0 2024-04-25T08:44:54.025 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:54.025 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/850914154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":850914154},{"type":"v1","addr":"172.21.15.46:6835","nonce":850914154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1115060038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1115060038},{"type":"v1","addr":"172.21.15.82:6836","nonce":1115060038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5620,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1383788710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1383788710},{"type":"v1","addr":"172.21.15.82:6837","nonce":1383788710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:44:14.218049+0000","modified":"2024-04-25T08:44:17.246143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/3843568539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3843568539},{"type":"v1","addr":"172.21.15.46:6834","nonce":3843568539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:44:54.025 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f70066f60 msgr2=0x7f6f70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f70066f60 0x7f6f70069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0af000 tx=0x7f6f7c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 msgr2=0x7f6f94154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 0x7f6f94154f30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400d800 tx=0x7f6f8400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 shutdown_connections 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6f70066f60 0x7f6f70069420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9415db60 0x7f6f9415f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6f940cfe70 0x7f6f94154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 --2- 172.21.15.46:0/1034650553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f940b06a0 0x7f6f94154f30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 >> 172.21.15.46:0/1034650553 conn(0x7f6f940d14f0 msgr2=0x7f6f9406fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:54.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 shutdown_connections 2024-04-25T08:44:54.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.034+0000 7f6f99a00640 1 -- 172.21.15.46:0/1034650553 wait complete. 2024-04-25T08:44:54.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T08:44:54.213 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-25T08:44:54.214 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:44:54.214 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T08:44:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.266+0000 7f2e59cf0080 1 Processor -- start 2024-04-25T08:44:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.266+0000 7f2e59cf0080 1 -- start start 2024-04-25T08:44:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a2683a4c0 0x555a2690c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a268f0560 0x555a26881500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a269a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a268f1a70 con 0x555a268f0db0 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a267de9f0 con 0x555a268f0560 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a268f1d10 con 0x555a2683a4c0 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a269a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a269a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60266/0 (socket says 172.21.15.46:60266) 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57a00640 1 -- 172.21.15.46:0/717274799 learned_addr learned my addr 172.21.15.46:0/717274799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57000640 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a2683a4c0 0x555a2690c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e56600640 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a268f0560 0x555a26881500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57a00640 1 -- 172.21.15.46:0/717274799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a2683a4c0 msgr2=0x555a2690c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57a00640 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a2683a4c0 0x555a2690c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57a00640 1 -- 172.21.15.46:0/717274799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a268f0560 msgr2=0x555a26881500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57a00640 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a268f0560 0x555a26881500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57a00640 1 -- 172.21.15.46:0/717274799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a2690d560 con 0x555a268f0db0 2024-04-25T08:44:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e56600640 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a268f0560 0x555a26881500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e57a00640 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a269a14e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000a560 tx=0x7f2e500033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ce435122129698 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e55200640 1 -- 172.21.15.46:0/717274799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500b9070 con 0x555a268f0db0 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e55200640 1 -- 172.21.15.46:0/717274799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e500ab550 con 0x555a268f0db0 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e55200640 1 -- 172.21.15.46:0/717274799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500ab900 con 0x555a268f0db0 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- 172.21.15.46:0/717274799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 msgr2=0x555a269a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a269a14e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000a560 tx=0x7f2e500033b0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- 172.21.15.46:0/717274799 shutdown_connections 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a269a14e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a268f0560 0x555a26881500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- 172.21.15.46:0/717274799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a2683a4c0 0x555a2690c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- 172.21.15.46:0/717274799 >> 172.21.15.46:0/717274799 conn(0x555a26838230 msgr2=0x555a26863a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- 172.21.15.46:0/717274799 shutdown_connections 2024-04-25T08:44:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- 172.21.15.46:0/717274799 wait complete. 2024-04-25T08:44:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 Processor -- start 2024-04-25T08:44:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- start start 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a2683a4c0 0x555a269176e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a268f0560 0x555a26910ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a26836520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2690f6d0 con 0x555a268f0db0 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a2690f470 con 0x555a2683a4c0 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.270+0000 7f2e59cf0080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a2690f970 con 0x555a268f0560 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a26836520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a26836520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60280/0 (socket says 172.21.15.46:60280) 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57a00640 1 -- 172.21.15.46:0/1170783271 learned_addr learned my addr 172.21.15.46:0/1170783271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57000640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a2683a4c0 0x555a269176e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e56600640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a268f0560 0x555a26910ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57a00640 1 -- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a268f0560 msgr2=0x555a26910ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57a00640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a268f0560 0x555a26910ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57a00640 1 -- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a2683a4c0 msgr2=0x555a269176e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57a00640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a2683a4c0 0x555a269176e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57a00640 1 -- 172.21.15.46:0/1170783271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a268804c0 con 0x555a268f0db0 2024-04-25T08:44:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e57a00640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a26836520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e5001ba20 tx=0x7f2e50017f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e4f400640 1 -- 172.21.15.46:0/1170783271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500c5070 con 0x555a268f0db0 2024-04-25T08:44:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e4f400640 1 -- 172.21.15.46:0/1170783271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e500ba040 con 0x555a268f0db0 2024-04-25T08:44:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e4f400640 1 -- 172.21.15.46:0/1170783271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500b9070 con 0x555a268f0db0 2024-04-25T08:44:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555a26880760 con 0x555a268f0db0 2024-04-25T08:44:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a26880ca0 con 0x555a268f0db0 2024-04-25T08:44:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e4f400640 1 -- 172.21.15.46:0/1170783271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e500c8070 con 0x555a268f0db0 2024-04-25T08:44:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e4f400640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e38065940 0x7f2e38067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.274+0000 7f2e4f400640 1 -- 172.21.15.46:0/1170783271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2e5011ec50 con 0x555a268f0db0 2024-04-25T08:44:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e57000640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e38065940 0x7f2e38067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x555a26911430 0x555a26913120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e53) v9 -- 0x555a26913660 con 0x555a26911430 2024-04-25T08:44:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e56600640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x555a26911430 0x555a26913120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e57000640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e38065940 0x7f2e38067e00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2e48007750 tx=0x7f2e48005b00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e56600640 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x555a26911430 0x555a26913120 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e56600640 1 -- 172.21.15.46:0/1170783271 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f2e4400c230 con 0x555a26911430 2024-04-25T08:44:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x555a26911430 msgr2=0x555a26913120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x555a26911430 0x555a26913120 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e38065940 msgr2=0x7f2e38067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2e38065940 0x7f2e38067e00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2e48007750 tx=0x7f2e48005b00 comp rx=0 tx=0).stop 2024-04-25T08:44:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 msgr2=0x555a26836520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 --2- 172.21.15.46:0/1170783271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a268f0db0 0x555a26836520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2e5001ba20 tx=0x7f2e50017f20 comp rx=0 tx=0).stop 2024-04-25T08:44:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e57a00640 1 -- 172.21.15.46:0/1170783271 reap_dead start 2024-04-25T08:44:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 shutdown_connections 2024-04-25T08:44:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 >> 172.21.15.46:0/1170783271 conn(0x555a26838230 msgr2=0x555a2690c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 shutdown_connections 2024-04-25T08:44:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.278+0000 7f2e59cf0080 1 -- 172.21.15.46:0/1170783271 wait complete. 2024-04-25T08:44:54.276 INFO:teuthology.orchestra.run.smithi046.stdout:Yalphaparent_a 2024-04-25T08:44:54.276 DEBUG:teuthology.orchestra.run.smithi046:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T08:44:54.366 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:44:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-04-25T08:44:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "ancestors": [ 2024-04-25T08:44:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1099511627776, 2024-04-25T08:44:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "alpha", 2024-04-25T08:44:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 2 2024-04-25T08:44:54.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1, 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "parent_a", 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 4 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 8, 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: "old_pools": [] 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:44:54.367 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:54.372 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:44:54.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:44:54.488 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:44:54.580 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:44:54.580 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:44:54.580 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:54.589 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:44:54.589 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:54.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f947de00640 1 Processor -- start 2024-04-25T08:44:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f947de00640 1 -- start start 2024-04-25T08:44:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f947de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9478159230 0x7f947815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f947de00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94780cdc50 0x7f947815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f947de00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94780cc700 con 0x7f94780cbeb0 2024-04-25T08:44:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f947de00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780cc9e0 con 0x7f9478159230 2024-04-25T08:44:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f947de00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780cccc0 con 0x7f94780cdc50 2024-04-25T08:44:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f9476a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9478159230 0x7f947815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f946fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f94780cbeb0 0x7f94780cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58194/0 (socket says 172.21.15.46:58194) 2024-04-25T08:44:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f946fe00640 1 -- 172.21.15.46:0/2657889527 learned_addr learned my addr 172.21.15.46:0/2657889527 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f9477400640 1 --2- 172.21.15.46:0/2657889527 >> v2:172.21.15.82:3300/0 conn(0x7f94780cdc50 0x7f947815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f9476a00640 1 -- 172.21.15.46:0/2657889527 >> v2:172.21.15.82:3300/0 conn(0x7f94780cdc50 msgr2=0x7f947815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f9476a00640 1 --2- 172.21.15.46:0/2657889527 >> v2:172.21.15.82:3300/0 conn(0x7f94780cdc50 0x7f947815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f9476a00640 1 -- 172.21.15.46:0/2657889527 >> v1:172.21.15.46:6789/0 conn(0x7f94780cbeb0 legacy=0x7f94780cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.834+0000 7f9476a00640 1 -- 172.21.15.46:0/2657889527 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780ccfa0 con 0x7f9478159230 2024-04-25T08:44:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f9476a00640 1 --2- 172.21.15.46:0/2657889527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 0x7f947815bdb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f94780b2340 tx=0x7f946801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff6f03c5a533916 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f9476000640 1 -- 172.21.15.46:0/2657889527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680b1070 con 0x7f9478159230 2024-04-25T08:44:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f9476000640 1 -- 172.21.15.46:0/2657889527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94680a8050 con 0x7f9478159230 2024-04-25T08:44:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f9476000640 1 -- 172.21.15.46:0/2657889527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680ac860 con 0x7f9478159230 2024-04-25T08:44:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f947de00640 1 -- 172.21.15.46:0/2657889527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 msgr2=0x7f947815bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f947de00640 1 --2- 172.21.15.46:0/2657889527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 0x7f947815bdb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f94780b2340 tx=0x7f946801c3c0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f947de00640 1 -- 172.21.15.46:0/2657889527 shutdown_connections 2024-04-25T08:44:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f947de00640 1 --2- 172.21.15.46:0/2657889527 >> v2:172.21.15.82:3300/0 conn(0x7f94780cdc50 0x7f947815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f947de00640 1 --2- 172.21.15.46:0/2657889527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 0x7f947815bdb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f947de00640 1 -- 172.21.15.46:0/2657889527 >> 172.21.15.46:0/2657889527 conn(0x7f94780b12a0 msgr2=0x7f94780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:54.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f947de00640 1 -- 172.21.15.46:0/2657889527 shutdown_connections 2024-04-25T08:44:54.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.838+0000 7f947de00640 1 -- 172.21.15.46:0/2657889527 wait complete. 2024-04-25T08:44:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 Processor -- start 2024-04-25T08:44:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 -- start start 2024-04-25T08:44:54.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94780cbeb0 0x7f9478165610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94780cdc50 0x7f94780a8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 0x7f947813e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947815df50 con 0x7f9478159230 2024-04-25T08:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780cc4e0 con 0x7f94780cbeb0 2024-04-25T08:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947815e0d0 con 0x7f94780cdc50 2024-04-25T08:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9477400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 0x7f947813e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9477400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 0x7f947813e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60290/0 (socket says 172.21.15.46:60290) 2024-04-25T08:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9477400640 1 -- 172.21.15.46:0/4024400543 learned_addr learned my addr 172.21.15.46:0/4024400543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9476a00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94780cbeb0 0x7f9478165610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f946fe00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94780cdc50 0x7f94780a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9477400640 1 -- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94780cdc50 msgr2=0x7f94780a8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9477400640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94780cdc50 0x7f94780a8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9477400640 1 -- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94780cbeb0 msgr2=0x7f9478165610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9477400640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94780cbeb0 0x7f9478165610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9477400640 1 -- 172.21.15.46:0/4024400543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947815af00 con 0x7f9478159230 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f9477400640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 0x7f947813e3c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9470007e60 tx=0x7f947000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f946f400640 1 -- 172.21.15.46:0/4024400543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470014620 con 0x7f9478159230 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f946f400640 1 -- 172.21.15.46:0/4024400543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9470019070 con 0x7f9478159230 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947815b1e0 con 0x7f9478159230 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f946f400640 1 -- 172.21.15.46:0/4024400543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470014a00 con 0x7f9478159230 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.842+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947815b7a0 con 0x7f9478159230 2024-04-25T08:44:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.846+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9478069080 con 0x7f9478159230 2024-04-25T08:44:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.846+0000 7f946f400640 1 -- 172.21.15.46:0/4024400543 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9470014ba0 con 0x7f9478159230 2024-04-25T08:44:54.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.846+0000 7f946f400640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9454066f60 0x7f9454069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.846+0000 7f946f400640 1 -- 172.21.15.46:0/4024400543 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9470085a20 con 0x7f9478159230 2024-04-25T08:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.846+0000 7f9476a00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9454066f60 0x7f9454069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:54.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.846+0000 7f9476a00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9454066f60 0x7f9454069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f94680ae000 tx=0x7f94680a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:54.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:54.850+0000 7f946f400640 1 -- 172.21.15.46:0/4024400543 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94700561b0 con 0x7f9478159230 2024-04-25T08:44:55.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94780723d0 con 0x7f9478159230 2024-04-25T08:44:55.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f946f400640 1 -- 172.21.15.46:0/4024400543 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f947005a250 con 0x7f9478159230 2024-04-25T08:44:55.011 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:55.011 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:15.829540+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:44:12.022357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:44:13.215069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:55.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9454066f60 msgr2=0x7f9454069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9454066f60 0x7f9454069420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f94680ae000 tx=0x7f94680a7000 comp rx=0 tx=0).stop 2024-04-25T08:44:55.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 msgr2=0x7f947813e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 0x7f947813e3c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9470007e60 tx=0x7f947000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 shutdown_connections 2024-04-25T08:44:55.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9454066f60 0x7f9454069420 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478159230 0x7f947813e3c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94780cdc50 0x7f94780a8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 --2- 172.21.15.46:0/4024400543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94780cbeb0 0x7f9478165610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 >> 172.21.15.46:0/4024400543 conn(0x7f94780b12a0 msgr2=0x7f947815f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:55.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 shutdown_connections 2024-04-25T08:44:55.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.018+0000 7f947de00640 1 -- 172.21.15.46:0/4024400543 wait complete. 2024-04-25T08:44:55.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:44:12.022357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:44:13.215069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:44:55.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:44:55.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.182+0000 7f0d51000640 1 Processor -- start 2024-04-25T08:44:55.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.182+0000 7f0d51000640 1 -- start start 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d51000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d4c15dd50 0x7f0d4c0cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d51000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c09a780 con 0x7f0d4c0d0450 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d51000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c09aa60 con 0x7f0d4c06f4d0 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d51000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c0708c0 con 0x7f0d4c15dd50 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d4a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0d4c0d0450 0x7f0d4c0cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58206/0 (socket says 172.21.15.46:58206) 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d4a800640 1 -- 172.21.15.46:0/3511896831 learned_addr learned my addr 172.21.15.46:0/3511896831 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d41400640 1 --2- 172.21.15.46:0/3511896831 >> v2:172.21.15.46:3300/0 conn(0x7f0d4c15dd50 0x7f0d4c0cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d49400640 1 -- 172.21.15.46:0/3511896831 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493935453 0 0) 0x7f0d4c09a780 con 0x7f0d4c0d0450 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d49400640 1 -- 172.21.15.46:0/3511896831 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d30003530 con 0x7f0d4c0d0450 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d41400640 1 -- 172.21.15.46:0/3511896831 >> v1:172.21.15.82:6789/0 conn(0x7f0d4c06f4d0 legacy=0x7f0d4c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d41400640 1 -- 172.21.15.46:0/3511896831 >> v1:172.21.15.46:6789/0 conn(0x7f0d4c0d0450 legacy=0x7f0d4c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.186+0000 7f0d41400640 1 -- 172.21.15.46:0/3511896831 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c09ad40 con 0x7f0d4c15dd50 2024-04-25T08:44:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d41400640 1 --2- 172.21.15.46:0/3511896831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c15dd50 0x7f0d4c0cff10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d340097c0 tx=0x7f0d3401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146f0c56bb6a6cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d49400640 1 -- 172.21.15.46:0/3511896831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340a7070 con 0x7f0d4c15dd50 2024-04-25T08:44:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d49400640 1 -- 172.21.15.46:0/3511896831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d340b0070 con 0x7f0d4c15dd50 2024-04-25T08:44:55.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d49400640 1 -- 172.21.15.46:0/3511896831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340ab890 con 0x7f0d4c15dd50 2024-04-25T08:44:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 -- 172.21.15.46:0/3511896831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c15dd50 msgr2=0x7f0d4c0cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 --2- 172.21.15.46:0/3511896831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c15dd50 0x7f0d4c0cff10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d340097c0 tx=0x7f0d3401c470 comp rx=0 tx=0).stop 2024-04-25T08:44:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 -- 172.21.15.46:0/3511896831 shutdown_connections 2024-04-25T08:44:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 --2- 172.21.15.46:0/3511896831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c15dd50 0x7f0d4c0cff10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 -- 172.21.15.46:0/3511896831 >> 172.21.15.46:0/3511896831 conn(0x7f0d4c0b3160 msgr2=0x7f0d4c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 -- 172.21.15.46:0/3511896831 shutdown_connections 2024-04-25T08:44:55.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 -- 172.21.15.46:0/3511896831 wait complete. 2024-04-25T08:44:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 Processor -- start 2024-04-25T08:44:55.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 -- start start 2024-04-25T08:44:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c06f4d0 0x7f0d4c154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c0d0450 0x7f0d4c1fa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d4c15dd50 0x7f0d4c0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0cd560 con 0x7f0d4c0d0450 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0cd3e0 con 0x7f0d4c06f4d0 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d51000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0cd6e0 con 0x7f0d4c15dd50 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d41400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c0d0450 0x7f0d4c1fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d41400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c0d0450 0x7f0d4c1fa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60314/0 (socket says 172.21.15.46:60314) 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.190+0000 7f0d41400640 1 -- 172.21.15.46:0/1582508981 learned_addr learned my addr 172.21.15.46:0/1582508981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d4a800640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d4c15dd50 0x7f0d4c0b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d49e00640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c06f4d0 0x7f0d4c154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d41400640 1 -- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d4c15dd50 msgr2=0x7f0d4c0b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d41400640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d4c15dd50 0x7f0d4c0b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d41400640 1 -- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c06f4d0 msgr2=0x7f0d4c154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d41400640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c06f4d0 0x7f0d4c154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d41400640 1 -- 172.21.15.46:0/1582508981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c1fa8b0 con 0x7f0d4c0d0450 2024-04-25T08:44:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d41400640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c0d0450 0x7f0d4c1fa370 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0d340ad040 tx=0x7f0d3401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d42a00640 1 -- 172.21.15.46:0/1582508981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340bf610 con 0x7f0d4c0d0450 2024-04-25T08:44:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d42a00640 1 -- 172.21.15.46:0/1582508981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d340bf7f0 con 0x7f0d4c0d0450 2024-04-25T08:44:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d42a00640 1 -- 172.21.15.46:0/1582508981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d340bfbe0 con 0x7f0d4c0d0450 2024-04-25T08:44:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c15a750 con 0x7f0d4c0d0450 2024-04-25T08:44:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c15ad10 con 0x7f0d4c0d0450 2024-04-25T08:44:55.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c069010 con 0x7f0d4c0d0450 2024-04-25T08:44:55.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d42a00640 1 -- 172.21.15.46:0/1582508981 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d340bd020 con 0x7f0d4c0d0450 2024-04-25T08:44:55.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d42a00640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d28066f90 0x7f0d28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.194+0000 7f0d42a00640 1 -- 172.21.15.46:0/1582508981 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d3411f830 con 0x7f0d4c0d0450 2024-04-25T08:44:55.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.198+0000 7f0d49e00640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d28066f90 0x7f0d28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.198+0000 7f0d49e00640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d28066f90 0x7f0d28069450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0012f0 tx=0x7f0d3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.202+0000 7f0d42a00640 1 -- 172.21.15.46:0/1582508981 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d4c1fa8b0 con 0x7f0d4c0d0450 2024-04-25T08:44:55.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.362+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d4c0b74f0 con 0x7f0d4c0d0450 2024-04-25T08:44:55.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.362+0000 7f0d42a00640 1 -- 172.21.15.46:0/1582508981 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d34016090 con 0x7f0d4c0d0450 2024-04-25T08:44:55.357 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:44:55.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d28066f90 msgr2=0x7f0d28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d28066f90 0x7f0d28069450 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0012f0 tx=0x7f0d3c009040 comp rx=0 tx=0).stop 2024-04-25T08:44:55.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c0d0450 msgr2=0x7f0d4c1fa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c0d0450 0x7f0d4c1fa370 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0d340ad040 tx=0x7f0d3401acb0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 shutdown_connections 2024-04-25T08:44:55.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d28066f90 0x7f0d28069450 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d4c15dd50 0x7f0d4c0b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d4c0d0450 0x7f0d4c1fa370 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 --2- 172.21.15.46:0/1582508981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d4c06f4d0 0x7f0d4c154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 >> 172.21.15.46:0/1582508981 conn(0x7f0d4c0b3160 msgr2=0x7f0d4c157d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:55.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 shutdown_connections 2024-04-25T08:44:55.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.366+0000 7f0d51000640 1 -- 172.21.15.46:0/1582508981 wait complete. 2024-04-25T08:44:55.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:44:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c03a00640 1 Processor -- start 2024-04-25T08:44:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c03a00640 1 -- start start 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c03a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4bfc06f6d0 0x7f4bfc06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c03a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bfc0af120 0x7f4bfc06c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c03a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc072600 0x7f4bfc15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c03a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc1563e0 con 0x7f4bfc0af120 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c03a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc15c0f0 con 0x7f4bfc06f6d0 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c03a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15c3d0 con 0x7f4bfc072600 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4bf3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bfc0af120 0x7f4bfc06c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4bf3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4bfc0af120 0x7f4bfc06c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60316/0 (socket says 172.21.15.46:60316) 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4bf3e00640 1 -- 172.21.15.46:0/1415502367 learned_addr learned my addr 172.21.15.46:0/1415502367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c01000640 1 --2- 172.21.15.46:0/1415502367 >> v2:172.21.15.82:3300/0 conn(0x7f4bfc06f6d0 0x7f4bfc06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4c01a00640 1 --2- 172.21.15.46:0/1415502367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc072600 0x7f4bfc15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4bf3e00640 1 -- 172.21.15.46:0/1415502367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc072600 msgr2=0x7f4bfc15f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4bf3e00640 1 --2- 172.21.15.46:0/1415502367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc072600 0x7f4bfc15f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4bf3e00640 1 -- 172.21.15.46:0/1415502367 >> v2:172.21.15.82:3300/0 conn(0x7f4bfc06f6d0 msgr2=0x7f4bfc06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:55.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4bf3e00640 1 --2- 172.21.15.46:0/1415502367 >> v2:172.21.15.82:3300/0 conn(0x7f4bfc06f6d0 0x7f4bfc06bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.530+0000 7f4bf3e00640 1 -- 172.21.15.46:0/1415502367 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc15c6b0 con 0x7f4bfc0af120 2024-04-25T08:44:55.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4bf3e00640 1 --2- 172.21.15.46:0/1415502367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 0x7f4bfc06c160 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0b2c00 tx=0x7f4be401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f69af3fe93b2cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:55.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4bfbe00640 1 -- 172.21.15.46:0/1415502367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40b0070 con 0x7f4bfc0af120 2024-04-25T08:44:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4bfbe00640 1 -- 172.21.15.46:0/1415502367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be40ab600 con 0x7f4bfc0af120 2024-04-25T08:44:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4bfbe00640 1 -- 172.21.15.46:0/1415502367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40ab9d0 con 0x7f4bfc0af120 2024-04-25T08:44:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4c03a00640 1 -- 172.21.15.46:0/1415502367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 msgr2=0x7f4bfc06c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4c03a00640 1 --2- 172.21.15.46:0/1415502367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 0x7f4bfc06c160 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0b2c00 tx=0x7f4be401c520 comp rx=0 tx=0).stop 2024-04-25T08:44:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4c03a00640 1 -- 172.21.15.46:0/1415502367 shutdown_connections 2024-04-25T08:44:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4c03a00640 1 --2- 172.21.15.46:0/1415502367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc072600 0x7f4bfc15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4c03a00640 1 --2- 172.21.15.46:0/1415502367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 0x7f4bfc06c160 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4c03a00640 1 --2- 172.21.15.46:0/1415502367 >> v2:172.21.15.82:3300/0 conn(0x7f4bfc06f6d0 0x7f4bfc06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4c03a00640 1 -- 172.21.15.46:0/1415502367 >> 172.21.15.46:0/1415502367 conn(0x7f4bfc071ae0 msgr2=0x7f4bfc071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4c03a00640 1 -- 172.21.15.46:0/1415502367 shutdown_connections 2024-04-25T08:44:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.534+0000 7f4c03a00640 1 -- 172.21.15.46:0/1415502367 wait complete. 2024-04-25T08:44:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 Processor -- start 2024-04-25T08:44:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 -- start start 2024-04-25T08:44:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc06f6d0 0x7f4bfc1392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bfc072600 0x7f4bfc155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 0x7f4bfc152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc06ded0 con 0x7f4bfc0af120 2024-04-25T08:44:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc06dd50 con 0x7f4bfc072600 2024-04-25T08:44:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc06e050 con 0x7f4bfc06f6d0 2024-04-25T08:44:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 0x7f4bfc152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 0x7f4bfc152c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60322/0 (socket says 172.21.15.46:60322) 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01a00640 1 -- 172.21.15.46:0/3759787180 learned_addr learned my addr 172.21.15.46:0/3759787180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01000640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc06f6d0 0x7f4bfc1392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4bf3e00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bfc072600 0x7f4bfc155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01a00640 1 -- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc06f6d0 msgr2=0x7f4bfc1392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01a00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc06f6d0 0x7f4bfc1392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01a00640 1 -- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bfc072600 msgr2=0x7f4bfc155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01a00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bfc072600 0x7f4bfc155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01a00640 1 -- 172.21.15.46:0/3759787180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc13be70 con 0x7f4bfc0af120 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01000640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc06f6d0 0x7f4bfc1392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4bf3e00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bfc072600 0x7f4bfc155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c01a00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 0x7f4bfc152c70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4007e40 tx=0x7f4bf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4bf9600640 1 -- 172.21.15.46:0/3759787180 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4014620 con 0x7f4bfc0af120 2024-04-25T08:44:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4bf9600640 1 -- 172.21.15.46:0/3759787180 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf4019070 con 0x7f4bfc0af120 2024-04-25T08:44:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc13c150 con 0x7f4bfc0af120 2024-04-25T08:44:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4bf9600640 1 -- 172.21.15.46:0/3759787180 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4014a20 con 0x7f4bfc0af120 2024-04-25T08:44:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc13c710 con 0x7f4bfc0af120 2024-04-25T08:44:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.538+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc13ca40 con 0x7f4bfc0af120 2024-04-25T08:44:55.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.542+0000 7f4bf9600640 1 -- 172.21.15.46:0/3759787180 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4bf4014bc0 con 0x7f4bfc0af120 2024-04-25T08:44:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.542+0000 7f4bf9600640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bd8066f90 0x7f4bd8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.542+0000 7f4bf9600640 1 -- 172.21.15.46:0/3759787180 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4bf4085aa0 con 0x7f4bfc0af120 2024-04-25T08:44:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.542+0000 7f4c01000640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bd8066f90 0x7f4bd8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.542+0000 7f4c01000640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bd8066f90 0x7f4bd8069450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009b30 tx=0x7f4bec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:55.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.546+0000 7f4bf9600640 1 -- 172.21.15.46:0/3759787180 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4bf4011260 con 0x7f4bfc0af120 2024-04-25T08:44:55.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.706+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4bfc069010 con 0x7f4bfc0af120 2024-04-25T08:44:55.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4bf9600640 1 -- 172.21.15.46:0/3759787180 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4bf4056880 con 0x7f4bfc0af120 2024-04-25T08:44:55.702 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bd8066f90 msgr2=0x7f4bd8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bd8066f90 0x7f4bd8069450 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009b30 tx=0x7f4bec008040 comp rx=0 tx=0).stop 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 msgr2=0x7f4bfc152c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 0x7f4bfc152c70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4007e40 tx=0x7f4bf400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 shutdown_connections 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4bd8066f90 0x7f4bd8069450 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4bfc0af120 0x7f4bfc152c70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bfc072600 0x7f4bfc155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 --2- 172.21.15.46:0/3759787180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4bfc06f6d0 0x7f4bfc1392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 >> 172.21.15.46:0/3759787180 conn(0x7f4bfc071ae0 msgr2=0x7f4bfc15d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 shutdown_connections 2024-04-25T08:44:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.710+0000 7f4c03a00640 1 -- 172.21.15.46:0/3759787180 wait complete. 2024-04-25T08:44:55.723 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:44:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f5d200640 1 Processor -- start 2024-04-25T08:44:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f5d200640 1 -- start start 2024-04-25T08:44:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f5d200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f580cf210 0x7f5f58160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f5d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5f5809a040 0x7f5f5815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f5d200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f580cc530 con 0x7f5f58099c60 2024-04-25T08:44:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f5d200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f580cc300 con 0x7f5f5809a040 2024-04-25T08:44:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f5d200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f580cc810 con 0x7f5f580cf210 2024-04-25T08:44:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f56800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5f5809a040 0x7f5f5815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f56800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5f5809a040 0x7f5f5815eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60324/0 (socket says 172.21.15.46:60324) 2024-04-25T08:44:55.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f56800640 1 -- 172.21.15.46:0/1709774374 learned_addr learned my addr 172.21.15.46:0/1709774374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f4fe00640 1 --2- 172.21.15.46:0/1709774374 >> v2:172.21.15.82:3300/0 conn(0x7f5f580cf210 0x7f5f58160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f56800640 1 -- 172.21.15.46:0/1709774374 >> v2:172.21.15.82:3300/0 conn(0x7f5f580cf210 msgr2=0x7f5f58160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f56800640 1 --2- 172.21.15.46:0/1709774374 >> v2:172.21.15.82:3300/0 conn(0x7f5f580cf210 0x7f5f58160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f56800640 1 -- 172.21.15.46:0/1709774374 >> v1:172.21.15.82:6789/0 conn(0x7f5f58099c60 legacy=0x7f5f580cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f56800640 1 -- 172.21.15.46:0/1709774374 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f580ccaf0 con 0x7f5f5809a040 2024-04-25T08:44:55.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.874+0000 7f5f4fe00640 1 --2- 172.21.15.46:0/1709774374 >> v2:172.21.15.82:3300/0 conn(0x7f5f580cf210 0x7f5f58160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f56800640 1 --2- 172.21.15.46:0/1709774374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f5809a040 0x7f5f5815eb60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5f580730c0 tx=0x7f5f5001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28098128efd7fd38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f55400640 1 -- 172.21.15.46:0/1709774374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f500a5780 con 0x7f5f5809a040 2024-04-25T08:44:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f55400640 1 -- 172.21.15.46:0/1709774374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f500a5920 con 0x7f5f5809a040 2024-04-25T08:44:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f55400640 1 -- 172.21.15.46:0/1709774374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f500a5cf0 con 0x7f5f5809a040 2024-04-25T08:44:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f5d200640 1 -- 172.21.15.46:0/1709774374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f5809a040 msgr2=0x7f5f5815eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f5d200640 1 --2- 172.21.15.46:0/1709774374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f5809a040 0x7f5f5815eb60 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5f580730c0 tx=0x7f5f5001c9b0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f5d200640 1 -- 172.21.15.46:0/1709774374 shutdown_connections 2024-04-25T08:44:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f5d200640 1 --2- 172.21.15.46:0/1709774374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f5809a040 0x7f5f5815eb60 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f5d200640 1 --2- 172.21.15.46:0/1709774374 >> v2:172.21.15.82:3300/0 conn(0x7f5f580cf210 0x7f5f58160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f5d200640 1 -- 172.21.15.46:0/1709774374 >> 172.21.15.46:0/1709774374 conn(0x7f5f580b3130 msgr2=0x7f5f580b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f5d200640 1 -- 172.21.15.46:0/1709774374 shutdown_connections 2024-04-25T08:44:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.878+0000 7f5f5d200640 1 -- 172.21.15.46:0/1709774374 wait complete. 2024-04-25T08:44:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 Processor -- start 2024-04-25T08:44:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 -- start start 2024-04-25T08:44:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f58099c60 0x7f5f5806bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5809a040 0x7f5f58154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f580cf210 0x7f5f5806d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f580b1930 con 0x7f5f58099c60 2024-04-25T08:44:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f580b17b0 con 0x7f5f5809a040 2024-04-25T08:44:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f580b1ab0 con 0x7f5f580cf210 2024-04-25T08:44:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f58099c60 0x7f5f5806bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f58099c60 0x7f5f5806bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60338/0 (socket says 172.21.15.46:60338) 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f55e00640 1 -- 172.21.15.46:0/2090921943 learned_addr learned my addr 172.21.15.46:0/2090921943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f56800640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f580cf210 0x7f5f5806d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f4fe00640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5809a040 0x7f5f58154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f55e00640 1 -- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f580cf210 msgr2=0x7f5f5806d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f55e00640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f580cf210 0x7f5f5806d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f55e00640 1 -- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5809a040 msgr2=0x7f5f58154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f55e00640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5809a040 0x7f5f58154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f55e00640 1 -- 172.21.15.46:0/2090921943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58164e10 con 0x7f5f58099c60 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f56800640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f580cf210 0x7f5f5806d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f4fe00640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5809a040 0x7f5f58154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f55e00640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f58099c60 0x7f5f5806bd20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800b410 tx=0x7f5f4800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f4e000640 1 -- 172.21.15.46:0/2090921943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48015030 con 0x7f5f58099c60 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f4e000640 1 -- 172.21.15.46:0/2090921943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f48004020 con 0x7f5f58099c60 2024-04-25T08:44:55.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f4e000640 1 -- 172.21.15.46:0/2090921943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f48004450 con 0x7f5f58099c60 2024-04-25T08:44:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f581650f0 con 0x7f5f58099c60 2024-04-25T08:44:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f581656b0 con 0x7f5f58099c60 2024-04-25T08:44:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.882+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f58069010 con 0x7f5f58099c60 2024-04-25T08:44:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.886+0000 7f5f4e000640 1 -- 172.21.15.46:0/2090921943 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f48018070 con 0x7f5f58099c60 2024-04-25T08:44:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.886+0000 7f5f4e000640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f34066f90 0x7f5f34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.886+0000 7f5f4e000640 1 -- 172.21.15.46:0/2090921943 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5f48084d70 con 0x7f5f58099c60 2024-04-25T08:44:55.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.886+0000 7f5f4fe00640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f34066f90 0x7f5f34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:55.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.886+0000 7f5f4fe00640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f34066f90 0x7f5f34069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f40006fd0 tx=0x7f5f40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:55.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:55.890+0000 7f5f4e000640 1 -- 172.21.15.46:0/2090921943 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f48055be0 con 0x7f5f58099c60 2024-04-25T08:44:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.050+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5f580b3f60 con 0x7f5f58099c60 2024-04-25T08:44:56.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.050+0000 7f5f4e000640 1 -- 172.21.15.46:0/2090921943 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f480554e0 con 0x7f5f58099c60 2024-04-25T08:44:56.045 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:44:56.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f34066f90 msgr2=0x7f5f34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f34066f90 0x7f5f34069450 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5f40006fd0 tx=0x7f5f40008040 comp rx=0 tx=0).stop 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f58099c60 msgr2=0x7f5f5806bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f58099c60 0x7f5f5806bd20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800b410 tx=0x7f5f4800b8d0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 shutdown_connections 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f34066f90 0x7f5f34069450 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5f580cf210 0x7f5f5806d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5809a040 0x7f5f58154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 --2- 172.21.15.46:0/2090921943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f58099c60 0x7f5f5806bd20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 >> 172.21.15.46:0/2090921943 conn(0x7f5f580b3130 msgr2=0x7f5f580ae6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 shutdown_connections 2024-04-25T08:44:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.054+0000 7f5f5d200640 1 -- 172.21.15.46:0/2090921943 wait complete. 2024-04-25T08:44:56.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:44:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08cd600640 1 Processor -- start 2024-04-25T08:44:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08cd600640 1 -- start start 2024-04-25T08:44:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08cd600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f08c8159230 0x7f08c815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08cd600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08c80cbeb0 0x7f08c80cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08cd600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c80cc700 con 0x7f08c80cda80 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08cd600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c80cded0 con 0x7f08c80cbeb0 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08cd600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c80cc9e0 con 0x7f08c8159230 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08bfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08c80cbeb0 0x7f08c80cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08bfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08c80cbeb0 0x7f08c80cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60340/0 (socket says 172.21.15.46:60340) 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08bfe00640 1 -- 172.21.15.46:0/2392488071 learned_addr learned my addr 172.21.15.46:0/2392488071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.214+0000 7f08c6a00640 1 --2- 172.21.15.46:0/2392488071 >> v2:172.21.15.82:3300/0 conn(0x7f08c8159230 0x7f08c815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08bfe00640 1 -- 172.21.15.46:0/2392488071 >> v2:172.21.15.82:3300/0 conn(0x7f08c8159230 msgr2=0x7f08c815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08bfe00640 1 --2- 172.21.15.46:0/2392488071 >> v2:172.21.15.82:3300/0 conn(0x7f08c8159230 0x7f08c815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08bfe00640 1 -- 172.21.15.46:0/2392488071 >> v1:172.21.15.82:6789/0 conn(0x7f08c80cda80 legacy=0x7f08c815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08bfe00640 1 -- 172.21.15.46:0/2392488071 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c80cccc0 con 0x7f08c80cbeb0 2024-04-25T08:44:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08bfe00640 1 --2- 172.21.15.46:0/2392488071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cbeb0 0x7f08c80cd540 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f08c8072a90 tx=0x7f08b001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e364fa386870a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08c6000640 1 -- 172.21.15.46:0/2392488071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b00ab660 con 0x7f08c80cbeb0 2024-04-25T08:44:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08c6000640 1 -- 172.21.15.46:0/2392488071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b00b0070 con 0x7f08c80cbeb0 2024-04-25T08:44:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08c6000640 1 -- 172.21.15.46:0/2392488071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b00aba20 con 0x7f08c80cbeb0 2024-04-25T08:44:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08cd600640 1 -- 172.21.15.46:0/2392488071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cbeb0 msgr2=0x7f08c80cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08cd600640 1 --2- 172.21.15.46:0/2392488071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cbeb0 0x7f08c80cd540 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f08c8072a90 tx=0x7f08b001c460 comp rx=0 tx=0).stop 2024-04-25T08:44:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08cd600640 1 -- 172.21.15.46:0/2392488071 shutdown_connections 2024-04-25T08:44:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08cd600640 1 --2- 172.21.15.46:0/2392488071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cbeb0 0x7f08c80cd540 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08cd600640 1 --2- 172.21.15.46:0/2392488071 >> v2:172.21.15.82:3300/0 conn(0x7f08c8159230 0x7f08c815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.218+0000 7f08cd600640 1 -- 172.21.15.46:0/2392488071 >> 172.21.15.46:0/2392488071 conn(0x7f08c80b12a0 msgr2=0x7f08c80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 -- 172.21.15.46:0/2392488071 shutdown_connections 2024-04-25T08:44:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 -- 172.21.15.46:0/2392488071 wait complete. 2024-04-25T08:44:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 Processor -- start 2024-04-25T08:44:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 -- start start 2024-04-25T08:44:56.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08c80cbeb0 0x7f08c8157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cda80 0x7f08c8164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08c8159230 0x7f08c815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c8159790 con 0x7f08c80cda80 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c8159610 con 0x7f08c80cbeb0 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08cd600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c8168780 con 0x7f08c8159230 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cda80 0x7f08c8164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cda80 0x7f08c8164900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60344/0 (socket says 172.21.15.46:60344) 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08bfe00640 1 -- 172.21.15.46:0/134658602 learned_addr learned my addr 172.21.15.46:0/134658602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08c6a00640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08c80cbeb0 0x7f08c8157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08c7400640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08c8159230 0x7f08c815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08bfe00640 1 -- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08c8159230 msgr2=0x7f08c815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08bfe00640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08c8159230 0x7f08c815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08bfe00640 1 -- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08c80cbeb0 msgr2=0x7f08c8157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08bfe00640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08c80cbeb0 0x7f08c8157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08bfe00640 1 -- 172.21.15.46:0/134658602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8067ec0 con 0x7f08c80cda80 2024-04-25T08:44:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.222+0000 7f08bfe00640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cda80 0x7f08c8164900 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f08b00ad040 tx=0x7f08b001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08bea00640 1 -- 172.21.15.46:0/134658602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b00be5f0 con 0x7f08c80cda80 2024-04-25T08:44:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08bea00640 1 -- 172.21.15.46:0/134658602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b00be790 con 0x7f08c80cda80 2024-04-25T08:44:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c8157e60 con 0x7f08c80cda80 2024-04-25T08:44:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08bea00640 1 -- 172.21.15.46:0/134658602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b00beb80 con 0x7f08c80cda80 2024-04-25T08:44:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c8165630 con 0x7f08c80cda80 2024-04-25T08:44:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c8069080 con 0x7f08c80cda80 2024-04-25T08:44:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08bea00640 1 -- 172.21.15.46:0/134658602 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08b00b43f0 con 0x7f08c80cda80 2024-04-25T08:44:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08bea00640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08a4066f90 0x7f08a4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08bea00640 1 -- 172.21.15.46:0/134658602 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08b011fe70 con 0x7f08c80cda80 2024-04-25T08:44:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.226+0000 7f08c6a00640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08a4066f90 0x7f08a4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.230+0000 7f08c6a00640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08a4066f90 0x7f08a4069450 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b9d0 tx=0x7f08b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.230+0000 7f08bea00640 1 -- 172.21.15.46:0/134658602 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08b00c4060 con 0x7f08c80cda80 2024-04-25T08:44:56.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.402+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f08c80723d0 con 0x7f08c80cda80 2024-04-25T08:44:56.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08bea00640 1 -- 172.21.15.46:0/134658602 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7f08b0016110 con 0x7f08c80cda80 2024-04-25T08:44:56.398 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08a4066f90 msgr2=0x7f08a4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08a4066f90 0x7f08a4069450 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f08b800b9d0 tx=0x7f08b8008040 comp rx=0 tx=0).stop 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cda80 msgr2=0x7f08c8164900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cda80 0x7f08c8164900 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f08b00ad040 tx=0x7f08b001aca0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 shutdown_connections 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08a4066f90 0x7f08a4069450 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08c8159230 0x7f08c815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08c80cda80 0x7f08c8164900 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 --2- 172.21.15.46:0/134658602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08c80cbeb0 0x7f08c8157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 >> 172.21.15.46:0/134658602 conn(0x7f08c80b12a0 msgr2=0x7f08c806d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 shutdown_connections 2024-04-25T08:44:56.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.406+0000 7f08cd600640 1 -- 172.21.15.46:0/134658602 wait complete. 2024-04-25T08:44:56.419 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:44:56.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:56.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:44:56.476 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:44:56.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:44:56.476 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:44:56.493 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:44:56.493 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:56.503 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:44:56.503 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:44:56.503 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:56.541 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:44:56.541 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:44:56.541 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:44:56.557 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:44:56.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db4400640 1 Processor -- start 2024-04-25T08:44:56.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db4400640 1 -- start start 2024-04-25T08:44:56.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db4400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4dac15dbd0 0x7f4dac0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db4400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dac0b5250 con 0x7f4dac0b0c30 2024-04-25T08:44:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db4400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dac0b5530 con 0x7f4dac0713a0 2024-04-25T08:44:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db4400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dac0b5af0 con 0x7f4dac15dbd0 2024-04-25T08:44:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4dac15dbd0 0x7f4dac0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db1000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4dac0b0c30 0x7f4dac070c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58208/0 (socket says 172.21.15.46:58208) 2024-04-25T08:44:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db1000640 1 -- 172.21.15.46:0/817012926 learned_addr learned my addr 172.21.15.46:0/817012926 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db1a00640 1 -- 172.21.15.46:0/817012926 >> v1:172.21.15.82:6789/0 conn(0x7f4dac0713a0 legacy=0x7f4dac06c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db1a00640 1 -- 172.21.15.46:0/817012926 >> v1:172.21.15.46:6789/0 conn(0x7f4dac0b0c30 legacy=0x7f4dac070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4dabe00640 1 -- 172.21.15.46:0/817012926 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276026576 0 0) 0x7f4dac0b5250 con 0x7f4dac0b0c30 2024-04-25T08:44:56.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.734+0000 7f4db1a00640 1 -- 172.21.15.46:0/817012926 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac0b5810 con 0x7f4dac15dbd0 2024-04-25T08:44:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4db1a00640 1 --2- 172.21.15.46:0/817012926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac15dbd0 0x7f4dac0b06f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4da00097c0 tx=0x7f4da001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f35a02b7676f0bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4dabe00640 1 -- 172.21.15.46:0/817012926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da00b0070 con 0x7f4dac15dbd0 2024-04-25T08:44:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4dabe00640 1 -- 172.21.15.46:0/817012926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da00ab600 con 0x7f4dac15dbd0 2024-04-25T08:44:56.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4dabe00640 1 -- 172.21.15.46:0/817012926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da00ab9d0 con 0x7f4dac15dbd0 2024-04-25T08:44:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4db4400640 1 -- 172.21.15.46:0/817012926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac15dbd0 msgr2=0x7f4dac0b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4db4400640 1 --2- 172.21.15.46:0/817012926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac15dbd0 0x7f4dac0b06f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4da00097c0 tx=0x7f4da001c460 comp rx=0 tx=0).stop 2024-04-25T08:44:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4db4400640 1 -- 172.21.15.46:0/817012926 shutdown_connections 2024-04-25T08:44:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4db4400640 1 --2- 172.21.15.46:0/817012926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac15dbd0 0x7f4dac0b06f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4db4400640 1 -- 172.21.15.46:0/817012926 >> 172.21.15.46:0/817012926 conn(0x7f4dac06f4f0 msgr2=0x7f4dac06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4db4400640 1 -- 172.21.15.46:0/817012926 shutdown_connections 2024-04-25T08:44:56.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.738+0000 7f4db4400640 1 -- 172.21.15.46:0/817012926 wait complete. 2024-04-25T08:44:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 Processor -- start 2024-04-25T08:44:56.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 -- start start 2024-04-25T08:44:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4dac0713a0 0x7f4dac0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac0b0c30 0x7f4dac151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac15dbd0 0x7f4dac1522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac16b650 con 0x7f4dac0b0c30 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac06d570 con 0x7f4dac15dbd0 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac16b7d0 con 0x7f4dac0713a0 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac0b0c30 0x7f4dac151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac0b0c30 0x7f4dac151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60362/0 (socket says 172.21.15.46:60362) 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1000640 1 -- 172.21.15.46:0/3504013901 learned_addr learned my addr 172.21.15.46:0/3504013901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1a00640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4dac0713a0 0x7f4dac0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db2400640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac15dbd0 0x7f4dac1522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1000640 1 -- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4dac0713a0 msgr2=0x7f4dac0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1000640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4dac0713a0 0x7f4dac0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1000640 1 -- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac15dbd0 msgr2=0x7f4dac1522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1000640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac15dbd0 0x7f4dac1522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1000640 1 -- 172.21.15.46:0/3504013901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dac150530 con 0x7f4dac0b0c30 2024-04-25T08:44:56.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db1000640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac0b0c30 0x7f4dac151db0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4d980011f0 tx=0x7f4d9800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4da9600640 1 -- 172.21.15.46:0/3504013901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98018070 con 0x7f4dac0b0c30 2024-04-25T08:44:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4da9600640 1 -- 172.21.15.46:0/3504013901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d980074e0 con 0x7f4dac0b0c30 2024-04-25T08:44:56.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4da9600640 1 -- 172.21.15.46:0/3504013901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98004a10 con 0x7f4dac0b0c30 2024-04-25T08:44:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dac0ce530 con 0x7f4dac0b0c30 2024-04-25T08:44:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.742+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dac0ceaf0 con 0x7f4dac0b0c30 2024-04-25T08:44:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.746+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dac0cf3e0 con 0x7f4dac0b0c30 2024-04-25T08:44:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.746+0000 7f4da9600640 1 -- 172.21.15.46:0/3504013901 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d98004bb0 con 0x7f4dac0b0c30 2024-04-25T08:44:56.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.746+0000 7f4da9600640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d8c066f60 0x7f4d8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.746+0000 7f4da9600640 1 -- 172.21.15.46:0/3504013901 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d98084530 con 0x7f4dac0b0c30 2024-04-25T08:44:56.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.746+0000 7f4db1a00640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d8c066f60 0x7f4d8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:56.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.746+0000 7f4db1a00640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d8c066f60 0x7f4d8c069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4da00ad000 tx=0x7f4da00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:56.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.750+0000 7f4da9600640 1 -- 172.21.15.46:0/3504013901 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d98055310 con 0x7f4dac0b0c30 2024-04-25T08:44:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.950+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dac069080 con 0x7f4dac0b0c30 2024-04-25T08:44:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.950+0000 7f4da9600640 1 -- 172.21.15.46:0/3504013901 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f4d98054c10 con 0x7f4dac0b0c30 2024-04-25T08:44:56.945 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:56.945 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/850914154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":850914154},{"type":"v1","addr":"172.21.15.46:6835","nonce":850914154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1115060038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1115060038},{"type":"v1","addr":"172.21.15.82:6836","nonce":1115060038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5620,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1383788710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1383788710},{"type":"v1","addr":"172.21.15.82:6837","nonce":1383788710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:44:14.218049+0000","modified":"2024-04-25T08:44:17.246143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/3843568539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3843568539},{"type":"v1","addr":"172.21.15.46:6834","nonce":3843568539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:44:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-04-25T08:44:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d8c066f60 msgr2=0x7f4d8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d8c066f60 0x7f4d8c069420 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4da00ad000 tx=0x7f4da00a6000 comp rx=0 tx=0).stop 2024-04-25T08:44:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac0b0c30 msgr2=0x7f4dac151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac0b0c30 0x7f4dac151db0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4d980011f0 tx=0x7f4d9800c9b0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 shutdown_connections 2024-04-25T08:44:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d8c066f60 0x7f4d8c069420 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dac15dbd0 0x7f4dac1522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4dac0b0c30 0x7f4dac151db0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 --2- 172.21.15.46:0/3504013901 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4dac0713a0 0x7f4dac0cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 >> 172.21.15.46:0/3504013901 conn(0x7f4dac06f4f0 msgr2=0x7f4dac0b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 shutdown_connections 2024-04-25T08:44:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:56.954+0000 7f4db4400640 1 -- 172.21.15.46:0/3504013901 wait complete. 2024-04-25T08:44:56.965 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:57.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3fe00640 1 Processor -- start 2024-04-25T08:44:57.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3fe00640 1 -- start start 2024-04-25T08:44:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fde3806f320 0x7fde38162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde380aed10 0x7fde38164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde380b35b0 con 0x7fde3815a750 2024-04-25T08:44:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3fe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3815ab30 con 0x7fde3806f320 2024-04-25T08:44:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde380ce4f0 con 0x7fde380aed10 2024-04-25T08:44:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fde3815a750 0x7fde3806ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58218/0 (socket says 172.21.15.46:58218) 2024-04-25T08:44:57.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3d400640 1 -- 172.21.15.46:0/614311847 learned_addr learned my addr 172.21.15.46:0/614311847 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3de00640 1 --2- 172.21.15.46:0/614311847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde380aed10 0x7fde38164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3ca00640 1 --2- 172.21.15.46:0/614311847 >> v2:172.21.15.82:3300/0 conn(0x7fde3806f320 0x7fde38162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde37e00640 1 -- 172.21.15.46:0/614311847 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767427365 0 0) 0x7fde380b35b0 con 0x7fde3815a750 2024-04-25T08:44:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde37e00640 1 -- 172.21.15.46:0/614311847 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde20003530 con 0x7fde3815a750 2024-04-25T08:44:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3de00640 1 -- 172.21.15.46:0/614311847 >> v2:172.21.15.82:3300/0 conn(0x7fde3806f320 msgr2=0x7fde38162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3de00640 1 --2- 172.21.15.46:0/614311847 >> v2:172.21.15.82:3300/0 conn(0x7fde3806f320 0x7fde38162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde37e00640 1 -- 172.21.15.46:0/614311847 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3393241839 0 0) 0x7fde20003530 con 0x7fde3815a750 2024-04-25T08:44:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3de00640 1 -- 172.21.15.46:0/614311847 >> v1:172.21.15.46:6789/0 conn(0x7fde3815a750 legacy=0x7fde3806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.114+0000 7fde3de00640 1 -- 172.21.15.46:0/614311847 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3806d5e0 con 0x7fde380aed10 2024-04-25T08:44:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde3de00640 1 --2- 172.21.15.46:0/614311847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde380aed10 0x7fde38164750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde30009d90 tx=0x7fde3001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d7e1169495d999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde37e00640 1 -- 172.21.15.46:0/614311847 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde300b2070 con 0x7fde380aed10 2024-04-25T08:44:57.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde37e00640 1 -- 172.21.15.46:0/614311847 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde300a5580 con 0x7fde380aed10 2024-04-25T08:44:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde37e00640 1 -- 172.21.15.46:0/614311847 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c001680 con 0x7fde380aed10 2024-04-25T08:44:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde3fe00640 1 -- 172.21.15.46:0/614311847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde380aed10 msgr2=0x7fde38164750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde3fe00640 1 --2- 172.21.15.46:0/614311847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde380aed10 0x7fde38164750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fde30009d90 tx=0x7fde3001c890 comp rx=0 tx=0).stop 2024-04-25T08:44:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde3fe00640 1 -- 172.21.15.46:0/614311847 shutdown_connections 2024-04-25T08:44:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde3fe00640 1 --2- 172.21.15.46:0/614311847 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde380aed10 0x7fde38164750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde3fe00640 1 --2- 172.21.15.46:0/614311847 >> v2:172.21.15.82:3300/0 conn(0x7fde3806f320 0x7fde38162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde3fe00640 1 -- 172.21.15.46:0/614311847 >> 172.21.15.46:0/614311847 conn(0x7fde380b12a0 msgr2=0x7fde380b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde3fe00640 1 -- 172.21.15.46:0/614311847 shutdown_connections 2024-04-25T08:44:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.118+0000 7fde3fe00640 1 -- 172.21.15.46:0/614311847 wait complete. 2024-04-25T08:44:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 Processor -- start 2024-04-25T08:44:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 -- start start 2024-04-25T08:44:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde3806f320 0x7fde38159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380aed10 0x7fde38153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde3815a750 0x7fde38151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3806d270 con 0x7fde380aed10 2024-04-25T08:44:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde380cf200 con 0x7fde3815a750 2024-04-25T08:44:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde381654a0 con 0x7fde3806f320 2024-04-25T08:44:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380aed10 0x7fde38153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380aed10 0x7fde38153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60378/0 (socket says 172.21.15.46:60378) 2024-04-25T08:44:57.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3ca00640 1 -- 172.21.15.46:0/601752958 learned_addr learned my addr 172.21.15.46:0/601752958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:57.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3d400640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde3806f320 0x7fde38159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3de00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde3815a750 0x7fde38151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3ca00640 1 -- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde3806f320 msgr2=0x7fde38159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3ca00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde3806f320 0x7fde38159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3ca00640 1 -- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde3815a750 msgr2=0x7fde38151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3ca00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde3815a750 0x7fde38151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3ca00640 1 -- 172.21.15.46:0/601752958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde38152080 con 0x7fde380aed10 2024-04-25T08:44:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3ca00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380aed10 0x7fde38153120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fde2400e930 tx=0x7fde2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde35600640 1 -- 172.21.15.46:0/601752958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde24017070 con 0x7fde380aed10 2024-04-25T08:44:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde35600640 1 -- 172.21.15.46:0/601752958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2400c9a0 con 0x7fde380aed10 2024-04-25T08:44:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde38152360 con 0x7fde380aed10 2024-04-25T08:44:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde35600640 1 -- 172.21.15.46:0/601752958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2400cd50 con 0x7fde380aed10 2024-04-25T08:44:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.122+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde38152920 con 0x7fde380aed10 2024-04-25T08:44:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.126+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde38069080 con 0x7fde380aed10 2024-04-25T08:44:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.126+0000 7fde35600640 1 -- 172.21.15.46:0/601752958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde2401a070 con 0x7fde380aed10 2024-04-25T08:44:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.126+0000 7fde35600640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde18066f90 0x7fde18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.126+0000 7fde35600640 1 -- 172.21.15.46:0/601752958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fde24085060 con 0x7fde380aed10 2024-04-25T08:44:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.126+0000 7fde3d400640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde18066f90 0x7fde18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.126+0000 7fde3d400640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde18066f90 0x7fde18069450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0011f0 tx=0x7fde2c0016e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.130+0000 7fde35600640 1 -- 172.21.15.46:0/601752958 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde240559e0 con 0x7fde380aed10 2024-04-25T08:44:57.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.294+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde380b0820 con 0x7fde380aed10 2024-04-25T08:44:57.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.294+0000 7fde35600640 1 -- 172.21.15.46:0/601752958 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16894 (secure 0 0 0) 0x7fde24059a80 con 0x7fde380aed10 2024-04-25T08:44:57.288 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:57.288 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:15.829540+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:44:12.022357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:44:13.215069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde18066f90 msgr2=0x7fde18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde18066f90 0x7fde18069450 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0011f0 tx=0x7fde2c0016e0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380aed10 msgr2=0x7fde38153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380aed10 0x7fde38153120 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fde2400e930 tx=0x7fde2400edf0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 shutdown_connections 2024-04-25T08:44:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde18066f90 0x7fde18069450 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde3815a750 0x7fde38151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde380aed10 0x7fde38153120 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 --2- 172.21.15.46:0/601752958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde3806f320 0x7fde38159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 >> 172.21.15.46:0/601752958 conn(0x7fde380b12a0 msgr2=0x7fde380afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 shutdown_connections 2024-04-25T08:44:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.298+0000 7fde3fe00640 1 -- 172.21.15.46:0/601752958 wait complete. 2024-04-25T08:44:57.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:44:12.022357+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:44:13.215069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:44:57.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:44:57.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb524000640 1 Processor -- start 2024-04-25T08:44:57.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb524000640 1 -- start start 2024-04-25T08:44:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb524000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 0x7fb51c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb524000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb51c15d090 0x7fb51c15d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb524000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0cd930 con 0x7fb51c06dc70 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb524000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c0cdbd0 con 0x7fb51c15d090 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb524000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0cdeb0 con 0x7fb51c145310 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb520c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb51c06dc70 0x7fb51c0cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58228/0 (socket says 172.21.15.46:58228) 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb522000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb51c15d090 0x7fb51c15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb520c00640 1 -- 172.21.15.46:0/3257532996 learned_addr learned my addr 172.21.15.46:0/3257532996 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb521600640 1 --2- 172.21.15.46:0/3257532996 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 0x7fb51c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb51be00640 1 -- 172.21.15.46:0/3257532996 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456358318 0 0) 0x7fb51c0cd930 con 0x7fb51c06dc70 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb522000640 1 -- 172.21.15.46:0/3257532996 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 msgr2=0x7fb51c06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb522000640 1 --2- 172.21.15.46:0/3257532996 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 0x7fb51c06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb522000640 1 -- 172.21.15.46:0/3257532996 >> v1:172.21.15.46:6789/0 conn(0x7fb51c06dc70 legacy=0x7fb51c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb522000640 1 -- 172.21.15.46:0/3257532996 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0ce190 con 0x7fb51c15d090 2024-04-25T08:44:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.462+0000 7fb521600640 1 --2- 172.21.15.46:0/3257532996 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 0x7fb51c06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb522000640 1 --2- 172.21.15.46:0/3257532996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 0x7fb51c15d540 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb514009d90 tx=0x7fb51401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbece151c9eae02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb51be00640 1 -- 172.21.15.46:0/3257532996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5140b2070 con 0x7fb51c15d090 2024-04-25T08:44:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb51be00640 1 -- 172.21.15.46:0/3257532996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5140a5920 con 0x7fb51c15d090 2024-04-25T08:44:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb51be00640 1 -- 172.21.15.46:0/3257532996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5140a5cf0 con 0x7fb51c15d090 2024-04-25T08:44:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb524000640 1 -- 172.21.15.46:0/3257532996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 msgr2=0x7fb51c15d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb524000640 1 --2- 172.21.15.46:0/3257532996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 0x7fb51c15d540 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb514009d90 tx=0x7fb51401c970 comp rx=0 tx=0).stop 2024-04-25T08:44:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb524000640 1 -- 172.21.15.46:0/3257532996 shutdown_connections 2024-04-25T08:44:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb524000640 1 --2- 172.21.15.46:0/3257532996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 0x7fb51c15d540 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb524000640 1 --2- 172.21.15.46:0/3257532996 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 0x7fb51c06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb524000640 1 -- 172.21.15.46:0/3257532996 >> 172.21.15.46:0/3257532996 conn(0x7fb51c06ac10 msgr2=0x7fb51c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb524000640 1 -- 172.21.15.46:0/3257532996 shutdown_connections 2024-04-25T08:44:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.466+0000 7fb524000640 1 -- 172.21.15.46:0/3257532996 wait complete. 2024-04-25T08:44:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 Processor -- start 2024-04-25T08:44:57.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 -- start start 2024-04-25T08:44:57.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 0x7fb51c139030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 0x7fb51c13ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51c15e1d0 0x7fb51c156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c168510 con 0x7fb51c15d090 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0cf380 con 0x7fb51c15e1d0 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c168690 con 0x7fb51c145310 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb520c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 0x7fb51c13ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb520c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 0x7fb51c13ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60394/0 (socket says 172.21.15.46:60394) 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb520c00640 1 -- 172.21.15.46:0/1437637349 learned_addr learned my addr 172.21.15.46:0/1437637349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb521600640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 0x7fb51c139030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb522000640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51c15e1d0 0x7fb51c156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb520c00640 1 -- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 msgr2=0x7fb51c139030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb520c00640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 0x7fb51c139030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb520c00640 1 -- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51c15e1d0 msgr2=0x7fb51c156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb520c00640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51c15e1d0 0x7fb51c156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb520c00640 1 -- 172.21.15.46:0/1437637349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c0b41a0 con 0x7fb51c15d090 2024-04-25T08:44:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb520c00640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 0x7fb51c13ce20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb5080011f0 tx=0x7fb50800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb519600640 1 -- 172.21.15.46:0/1437637349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508018070 con 0x7fb51c15d090 2024-04-25T08:44:57.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb519600640 1 -- 172.21.15.46:0/1437637349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5080074e0 con 0x7fb51c15d090 2024-04-25T08:44:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb519600640 1 -- 172.21.15.46:0/1437637349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb508004a10 con 0x7fb51c15d090 2024-04-25T08:44:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c0b4480 con 0x7fb51c15d090 2024-04-25T08:44:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c0b29e0 con 0x7fb51c15d090 2024-04-25T08:44:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.470+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c069010 con 0x7fb51c15d090 2024-04-25T08:44:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.474+0000 7fb519600640 1 -- 172.21.15.46:0/1437637349 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb508004bb0 con 0x7fb51c15d090 2024-04-25T08:44:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.474+0000 7fb519600640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.474+0000 7fb519600640 1 -- 172.21.15.46:0/1437637349 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb508084470 con 0x7fb51c15d090 2024-04-25T08:44:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.474+0000 7fb521600640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.474+0000 7fb521600640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0b1e80 tx=0x7fb510008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:57.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.478+0000 7fb519600640 1 -- 172.21.15.46:0/1437637349 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb508054c80 con 0x7fb51c15d090 2024-04-25T08:44:57.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.642+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb51c06c250 con 0x7fb51c15d090 2024-04-25T08:44:57.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.642+0000 7fb519600640 1 -- 172.21.15.46:0/1437637349 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb508058d20 con 0x7fb51c15d090 2024-04-25T08:44:57.637 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:44:57.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 msgr2=0x7fb4fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb51c0b1e80 tx=0x7fb510008040 comp rx=0 tx=0).stop 2024-04-25T08:44:57.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 msgr2=0x7fb51c13ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 0x7fb51c13ce20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb5080011f0 tx=0x7fb50800c9b0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 shutdown_connections 2024-04-25T08:44:57.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb4fc066f60 0x7fb4fc069420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51c15e1d0 0x7fb51c156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb51c15d090 0x7fb51c13ce20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 --2- 172.21.15.46:0/1437637349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb51c145310 0x7fb51c139030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 >> 172.21.15.46:0/1437637349 conn(0x7fb51c06ac10 msgr2=0x7fb51c144230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:57.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 shutdown_connections 2024-04-25T08:44:57.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.646+0000 7fb524000640 1 -- 172.21.15.46:0/1437637349 wait complete. 2024-04-25T08:44:57.658 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:44:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.806+0000 7f58b0a00640 1 Processor -- start 2024-04-25T08:44:57.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58b0a00640 1 -- start start 2024-04-25T08:44:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58b0a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58ac06f4d0 0x7f58ac0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58b0a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 0x7f58ac0cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58b0a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58ac0d06f0 con 0x7f58ac0d0280 2024-04-25T08:44:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58b0a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58ac09a780 con 0x7f58ac06f4d0 2024-04-25T08:44:57.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58b0a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac09aa60 con 0x7f58ac15dd50 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58a9e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58ac06f4d0 0x7f58ac0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58a3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 0x7f58ac0cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58a9e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58ac06f4d0 0x7f58ac0cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:47112/0 (socket says 172.21.15.46:47112) 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58a9e00640 1 -- 172.21.15.46:0/2394361247 learned_addr learned my addr 172.21.15.46:0/2394361247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58a3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 0x7f58ac0cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:51090/0 (socket says 172.21.15.46:51090) 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58aa800640 1 --1- 172.21.15.46:0/2394361247 >> v1:172.21.15.82:6789/0 conn(0x7f58ac0d0280 0x7f58ac06f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:34518/0 (socket says 172.21.15.46:34518) 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58a9e00640 1 -- 172.21.15.46:0/2394361247 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 msgr2=0x7f58ac0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58a9e00640 1 --2- 172.21.15.46:0/2394361247 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 0x7f58ac0cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58a9e00640 1 -- 172.21.15.46:0/2394361247 >> v1:172.21.15.82:6789/0 conn(0x7f58ac0d0280 legacy=0x7f58ac06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.810+0000 7f58a9e00640 1 -- 172.21.15.46:0/2394361247 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac09ad40 con 0x7f58ac06f4d0 2024-04-25T08:44:57.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58a9e00640 1 --2- 172.21.15.46:0/2394361247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac06f4d0 0x7f58ac0cf800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f589c009870 tx=0x7f589c01c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3588ef550c0e67c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58a9400640 1 -- 172.21.15.46:0/2394361247 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0a95f0 con 0x7f58ac06f4d0 2024-04-25T08:44:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58a9400640 1 -- 172.21.15.46:0/2394361247 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c0a9790 con 0x7f58ac06f4d0 2024-04-25T08:44:57.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58a9400640 1 -- 172.21.15.46:0/2394361247 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0a9b60 con 0x7f58ac06f4d0 2024-04-25T08:44:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 -- 172.21.15.46:0/2394361247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac06f4d0 msgr2=0x7f58ac0cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 --2- 172.21.15.46:0/2394361247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac06f4d0 0x7f58ac0cf800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f589c009870 tx=0x7f589c01c530 comp rx=0 tx=0).stop 2024-04-25T08:44:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 -- 172.21.15.46:0/2394361247 shutdown_connections 2024-04-25T08:44:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 --2- 172.21.15.46:0/2394361247 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 0x7f58ac0cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 --2- 172.21.15.46:0/2394361247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac06f4d0 0x7f58ac0cf800 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 -- 172.21.15.46:0/2394361247 >> 172.21.15.46:0/2394361247 conn(0x7f58ac0b3160 msgr2=0x7f58ac0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 -- 172.21.15.46:0/2394361247 shutdown_connections 2024-04-25T08:44:57.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 -- 172.21.15.46:0/2394361247 wait complete. 2024-04-25T08:44:57.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 Processor -- start 2024-04-25T08:44:57.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.814+0000 7f58b0a00640 1 -- start start 2024-04-25T08:44:57.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58b0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac06f4d0 0x7f58ac15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58b0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac0d0280 0x7f58ac15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58b0a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 0x7f58ac15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58b0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac16a820 con 0x7f58ac0d0280 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58b0a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac16a6a0 con 0x7f58ac06f4d0 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58b0a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac16a9a0 con 0x7f58ac15dd50 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac0d0280 0x7f58ac15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac0d0280 0x7f58ac15ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60396/0 (socket says 172.21.15.46:60396) 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a3e00640 1 -- 172.21.15.46:0/2055366495 learned_addr learned my addr 172.21.15.46:0/2055366495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a9e00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac06f4d0 0x7f58ac15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58aa800640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 0x7f58ac15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a3e00640 1 -- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 msgr2=0x7f58ac15bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a3e00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 0x7f58ac15bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a3e00640 1 -- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac06f4d0 msgr2=0x7f58ac15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a3e00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac06f4d0 0x7f58ac15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a3e00640 1 -- 172.21.15.46:0/2055366495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0b1d30 con 0x7f58ac0d0280 2024-04-25T08:44:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a3e00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac0d0280 0x7f58ac15ba40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f589400e8a0 tx=0x7f589400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a2000640 1 -- 172.21.15.46:0/2055366495 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894017070 con 0x7f58ac0d0280 2024-04-25T08:44:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a2000640 1 -- 172.21.15.46:0/2055366495 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589400c940 con 0x7f58ac0d0280 2024-04-25T08:44:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a2000640 1 -- 172.21.15.46:0/2055366495 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589400ccf0 con 0x7f58ac0d0280 2024-04-25T08:44:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac0b2010 con 0x7f58ac0d0280 2024-04-25T08:44:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac153750 con 0x7f58ac0d0280 2024-04-25T08:44:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ac153a70 con 0x7f58ac0d0280 2024-04-25T08:44:57.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.818+0000 7f58a2000640 1 -- 172.21.15.46:0/2055366495 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f589401a070 con 0x7f58ac0d0280 2024-04-25T08:44:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.822+0000 7f58a2000640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5888066f60 0x7f5888069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.822+0000 7f58a2000640 1 -- 172.21.15.46:0/2055366495 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58940853e0 con 0x7f58ac0d0280 2024-04-25T08:44:57.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.822+0000 7f58a9e00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5888066f60 0x7f5888069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:57.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.822+0000 7f58a9e00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5888066f60 0x7f5888069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f589c0ae000 tx=0x7f589c0b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:57.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.826+0000 7f58a2000640 1 -- 172.21.15.46:0/2055366495 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58ac153a70 con 0x7f58ac0d0280 2024-04-25T08:44:57.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.986+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58ac069010 con 0x7f58ac0d0280 2024-04-25T08:44:57.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.986+0000 7f58a2000640 1 -- 172.21.15.46:0/2055366495 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5894055b70 con 0x7f58ac0d0280 2024-04-25T08:44:57.982 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:44:57.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5888066f60 msgr2=0x7f5888069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5888066f60 0x7f5888069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f589c0ae000 tx=0x7f589c0b3040 comp rx=0 tx=0).stop 2024-04-25T08:44:57.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac0d0280 msgr2=0x7f58ac15ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:57.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac0d0280 0x7f58ac15ba40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f589400e8a0 tx=0x7f589400eca0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 shutdown_connections 2024-04-25T08:44:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5888066f60 0x7f5888069420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58ac15dd50 0x7f58ac15bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58ac0d0280 0x7f58ac15ba40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 --2- 172.21.15.46:0/2055366495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58ac06f4d0 0x7f58ac15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:57.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 >> 172.21.15.46:0/2055366495 conn(0x7f58ac0b3160 msgr2=0x7f58ac157020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 shutdown_connections 2024-04-25T08:44:57.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:57.990+0000 7f58b0a00640 1 -- 172.21.15.46:0/2055366495 wait complete. 2024-04-25T08:44:58.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:44:58.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.150+0000 7f94a4e00640 1 Processor -- start 2024-04-25T08:44:58.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.150+0000 7f94a4e00640 1 -- start start 2024-04-25T08:44:58.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f94a4e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94a006dc70 0x7f94a00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f94a4e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00cd930 con 0x7f94a0145310 2024-04-25T08:44:58.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f94a4e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00cdbd0 con 0x7f94a00cf1b0 2024-04-25T08:44:58.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f94a4e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a00cdeb0 con 0x7f94a006dc70 2024-04-25T08:44:58.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f94a0145310 0x7f94a006e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48462/0 (socket says 172.21.15.46:48462) 2024-04-25T08:44:58.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949de00640 1 -- 172.21.15.46:0/1683484875 learned_addr learned my addr 172.21.15.46:0/1683484875 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f9495400640 1 --2- 172.21.15.46:0/1683484875 >> v2:172.21.15.82:3300/0 conn(0x7f94a006dc70 0x7f94a00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181544166 0 0) 0x7f94a00cd930 con 0x7f94a0145310 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9484003530 con 0x7f94a0145310 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 241471328 0 0) 0x7f9484003530 con 0x7f94a0145310 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 >> v2:172.21.15.82:3300/0 conn(0x7f94a006dc70 msgr2=0x7f94a00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 --2- 172.21.15.46:0/1683484875 >> v2:172.21.15.82:3300/0 conn(0x7f94a006dc70 0x7f94a00cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 >> v1:172.21.15.82:6789/0 conn(0x7f94a00cf1b0 legacy=0x7f94a015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00ce190 con 0x7f94a0145310 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f9490001680 con 0x7f94a0145310 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f9495400640 1 --2- 172.21.15.46:0/1683484875 >> v2:172.21.15.82:3300/0 conn(0x7f94a006dc70 0x7f94a00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 >> v1:172.21.15.46:6789/0 conn(0x7f94a0145310 legacy=0x7f94a006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484003a30 0x7f9484003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9484004810 0x7f9484004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94840051d0 0x7f948400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00cdeb0 con 0x7f9484004810 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948400c100 con 0x7f9484003a30 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948400c300 con 0x7f94840051d0 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f9495400640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9484004810 0x7f9484004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949de00640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484003a30 0x7f9484003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949e800640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94840051d0 0x7f948400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f9495400640 1 -- 172.21.15.46:0/1683484875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94840051d0 msgr2=0x7f948400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f9495400640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94840051d0 0x7f948400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f9495400640 1 -- 172.21.15.46:0/1683484875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484003a30 msgr2=0x7f9484003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f9495400640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484003a30 0x7f9484003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f9495400640 1 -- 172.21.15.46:0/1683484875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484003530 con 0x7f9484004810 2024-04-25T08:44:58.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.154+0000 7f949e800640 1 -- 172.21.15.46:0/1683484875 reap_dead start 2024-04-25T08:44:58.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f9495400640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9484004810 0x7f9484004c90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9488009cb0 tx=0x7f948801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61127cc38531c6d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:58.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94880b2070 con 0x7f9484004810 2024-04-25T08:44:58.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94880a57c0 con 0x7f9484004810 2024-04-25T08:44:58.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f949d400640 1 -- 172.21.15.46:0/1683484875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488009880 con 0x7f9484004810 2024-04-25T08:44:58.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f94a4e00640 1 -- 172.21.15.46:0/1683484875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9484004810 msgr2=0x7f9484004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f94a4e00640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9484004810 0x7f9484004c90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9488009cb0 tx=0x7f948801cc10 comp rx=0 tx=0).stop 2024-04-25T08:44:58.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f94a4e00640 1 -- 172.21.15.46:0/1683484875 shutdown_connections 2024-04-25T08:44:58.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f94a4e00640 1 --2- 172.21.15.46:0/1683484875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9484004810 0x7f9484004c90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f94a4e00640 1 -- 172.21.15.46:0/1683484875 >> 172.21.15.46:0/1683484875 conn(0x7f94a006ac10 msgr2=0x7f94a006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:58.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f94a4e00640 1 -- 172.21.15.46:0/1683484875 shutdown_connections 2024-04-25T08:44:58.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f94a4e00640 1 -- 172.21.15.46:0/1683484875 wait complete. 2024-04-25T08:44:58.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f94a4e00640 1 Processor -- start 2024-04-25T08:44:58.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.158+0000 7f94a4e00640 1 -- start start 2024-04-25T08:44:58.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f94a4e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484004810 0x7f94a01396b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f94a4e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94a014d5b0 0x7f94a0136f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f94a4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a0137700 0x7f94a0137b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f94a4e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00cec70 con 0x7f94a0137700 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f94a4e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00ce750 con 0x7f9484004810 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f94a4e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00cedf0 con 0x7f94a014d5b0 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a0137700 0x7f94a0137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a0137700 0x7f94a0137b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56608/0 (socket says 172.21.15.46:56608) 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949e800640 1 -- 172.21.15.46:0/1876105272 learned_addr learned my addr 172.21.15.46:0/1876105272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949de00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484004810 0x7f94a01396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f9495400640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94a014d5b0 0x7f94a0136f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949e800640 1 -- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94a014d5b0 msgr2=0x7f94a0136f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949e800640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94a014d5b0 0x7f94a0136f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949e800640 1 -- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484004810 msgr2=0x7f94a01396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949e800640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484004810 0x7f94a01396b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949e800640 1 -- 172.21.15.46:0/1876105272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a01380c0 con 0x7f94a0137700 2024-04-25T08:44:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949de00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484004810 0x7f94a01396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f949e800640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a0137700 0x7f94a0137b80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f949800f8b0 tx=0x7f949800fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:58.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f9496a00640 1 -- 172.21.15.46:0/1876105272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498003b90 con 0x7f94a0137700 2024-04-25T08:44:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f9496a00640 1 -- 172.21.15.46:0/1876105272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498018070 con 0x7f94a0137700 2024-04-25T08:44:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f9496a00640 1 -- 172.21.15.46:0/1876105272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94980050d0 con 0x7f94a0137700 2024-04-25T08:44:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a0138360 con 0x7f94a0137700 2024-04-25T08:44:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a0138920 con 0x7f94a0137700 2024-04-25T08:44:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a0069010 con 0x7f94a0137700 2024-04-25T08:44:58.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.162+0000 7f9496a00640 1 -- 172.21.15.46:0/1876105272 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9498005270 con 0x7f94a0137700 2024-04-25T08:44:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.166+0000 7f9496a00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f947c066f90 0x7f947c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.166+0000 7f9496a00640 1 -- 172.21.15.46:0/1876105272 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9498084fb0 con 0x7f94a0137700 2024-04-25T08:44:58.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.166+0000 7f949de00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f947c066f90 0x7f947c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.166+0000 7f949de00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f947c066f90 0x7f947c069450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9490007d10 tx=0x7f9490009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:58.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.170+0000 7f9496a00640 1 -- 172.21.15.46:0/1876105272 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9498055d90 con 0x7f94a0137700 2024-04-25T08:44:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.330+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94a01374c0 con 0x7f94a0137700 2024-04-25T08:44:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.330+0000 7f9496a00640 1 -- 172.21.15.46:0/1876105272 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9498055690 con 0x7f94a0137700 2024-04-25T08:44:58.324 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:44:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f947c066f90 msgr2=0x7f947c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f947c066f90 0x7f947c069450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9490007d10 tx=0x7f9490009040 comp rx=0 tx=0).stop 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a0137700 msgr2=0x7f94a0137b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a0137700 0x7f94a0137b80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f949800f8b0 tx=0x7f949800fd70 comp rx=0 tx=0).stop 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 shutdown_connections 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f947c066f90 0x7f947c069450 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a0137700 0x7f94a0137b80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94a014d5b0 0x7f94a0136f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 --2- 172.21.15.46:0/1876105272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9484004810 0x7f94a01396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 >> 172.21.15.46:0/1876105272 conn(0x7f94a006ac10 msgr2=0x7f94a015b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 shutdown_connections 2024-04-25T08:44:58.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.334+0000 7f94a4e00640 1 -- 172.21.15.46:0/1876105272 wait complete. 2024-04-25T08:44:58.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:58.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd11200640 1 Processor -- start 2024-04-25T08:44:58.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd11200640 1 -- start start 2024-04-25T08:44:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd11200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd0c15a750 0x7efd0c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd11200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efd0c0aed10 0x7efd0c06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd11200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c06d5e0 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd11200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0c15ab30 con 0x7efd0c15a750 2024-04-25T08:44:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd11200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0c0ce4f0 con 0x7efd0c0aed10 2024-04-25T08:44:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd09e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd0c15a750 0x7efd0c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd09e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd0c15a750 0x7efd0c06ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56616/0 (socket says 172.21.15.46:56616) 2024-04-25T08:44:58.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd09e00640 1 -- 172.21.15.46:0/1140976957 learned_addr learned my addr 172.21.15.46:0/1140976957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:58.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd01400640 1 --2- 172.21.15.46:0/1140976957 >> v2:172.21.15.82:3300/0 conn(0x7efd0c0aed10 0x7efd0c06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd09e00640 1 -- 172.21.15.46:0/1140976957 >> v2:172.21.15.82:3300/0 conn(0x7efd0c0aed10 msgr2=0x7efd0c06f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:58.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd09e00640 1 --2- 172.21.15.46:0/1140976957 >> v2:172.21.15.82:3300/0 conn(0x7efd0c0aed10 0x7efd0c06f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd09e00640 1 -- 172.21.15.46:0/1140976957 >> v1:172.21.15.82:6789/0 conn(0x7efd0c0cc9e0 legacy=0x7efd0c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:58.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd09e00640 1 -- 172.21.15.46:0/1140976957 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c0ce720 con 0x7efd0c15a750 2024-04-25T08:44:58.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.494+0000 7efd01400640 1 --2- 172.21.15.46:0/1140976957 >> v2:172.21.15.82:3300/0 conn(0x7efd0c0aed10 0x7efd0c06f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:58.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd09e00640 1 --2- 172.21.15.46:0/1140976957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c15a750 0x7efd0c06ec10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0cdca0 tx=0x7efcfc01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd2731de5bef62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:58.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd09400640 1 -- 172.21.15.46:0/1140976957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0a5760 con 0x7efd0c15a750 2024-04-25T08:44:58.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd09400640 1 -- 172.21.15.46:0/1140976957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc0a5900 con 0x7efd0c15a750 2024-04-25T08:44:58.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd09400640 1 -- 172.21.15.46:0/1140976957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0a5cd0 con 0x7efd0c15a750 2024-04-25T08:44:58.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd11200640 1 -- 172.21.15.46:0/1140976957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c15a750 msgr2=0x7efd0c06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd11200640 1 --2- 172.21.15.46:0/1140976957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c15a750 0x7efd0c06ec10 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0cdca0 tx=0x7efcfc01c9b0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd11200640 1 -- 172.21.15.46:0/1140976957 shutdown_connections 2024-04-25T08:44:58.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd11200640 1 --2- 172.21.15.46:0/1140976957 >> v2:172.21.15.82:3300/0 conn(0x7efd0c0aed10 0x7efd0c06f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd11200640 1 --2- 172.21.15.46:0/1140976957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c15a750 0x7efd0c06ec10 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd11200640 1 -- 172.21.15.46:0/1140976957 >> 172.21.15.46:0/1140976957 conn(0x7efd0c0b12a0 msgr2=0x7efd0c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:58.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd11200640 1 -- 172.21.15.46:0/1140976957 shutdown_connections 2024-04-25T08:44:58.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.498+0000 7efd11200640 1 -- 172.21.15.46:0/1140976957 wait complete. 2024-04-25T08:44:58.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 Processor -- start 2024-04-25T08:44:58.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 -- start start 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd0c0aed10 0x7efd0c148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c0cc9e0 0x7efd0c1f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd0c15a750 0x7efd0c14a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c167230 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c06d270 con 0x7efd0c0aed10 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c1673b0 con 0x7efd0c15a750 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c0cc9e0 0x7efd0c1f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd01400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c0cc9e0 0x7efd0c1f67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56630/0 (socket says 172.21.15.46:56630) 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd01400640 1 -- 172.21.15.46:0/871576232 learned_addr learned my addr 172.21.15.46:0/871576232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd09e00640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd0c0aed10 0x7efd0c148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd0a800640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd0c15a750 0x7efd0c14a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd01400640 1 -- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd0c15a750 msgr2=0x7efd0c14a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd01400640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd0c15a750 0x7efd0c14a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd01400640 1 -- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd0c0aed10 msgr2=0x7efd0c148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd01400640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd0c0aed10 0x7efd0c148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd01400640 1 -- 172.21.15.46:0/871576232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c149620 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd0a800640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd0c15a750 0x7efd0c14a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd01400640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c0cc9e0 0x7efd0c1f67b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c820 tx=0x7efcf400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd02a00640 1 -- 172.21.15.46:0/871576232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4015070 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd02a00640 1 -- 172.21.15.46:0/871576232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf4007a30 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd02a00640 1 -- 172.21.15.46:0/871576232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4007e00 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c149900 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.502+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c149ec0 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.506+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0c069080 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.506+0000 7efd02a00640 1 -- 172.21.15.46:0/871576232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcf4018070 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.506+0000 7efd02a00640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efce8066f60 0x7efce8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.506+0000 7efd02a00640 1 -- 172.21.15.46:0/871576232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efcf4085190 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.506+0000 7efd09e00640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efce8066f60 0x7efce8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.506+0000 7efd09e00640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efce8066f60 0x7efce8069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0ae000 tx=0x7efcfc0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:58.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.510+0000 7efd02a00640 1 -- 172.21.15.46:0/871576232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcf4056010 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd0c154c00 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd02a00640 1 -- 172.21.15.46:0/871576232 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4935 (secure 0 0 0) 0x7efcf4055910 con 0x7efd0c0cc9e0 2024-04-25T08:44:58.699 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:58.699 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/850914154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":850914154},{"type":"v1","addr":"172.21.15.46:6835","nonce":850914154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1115060038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1115060038},{"type":"v1","addr":"172.21.15.82:6836","nonce":1115060038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5620,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1383788710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1383788710},{"type":"v1","addr":"172.21.15.82:6837","nonce":1383788710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:44:14.218049+0000","modified":"2024-04-25T08:44:17.246143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/3843568539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3843568539},{"type":"v1","addr":"172.21.15.46:6834","nonce":3843568539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:44:58.699 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-04-25T08:44:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efce8066f60 msgr2=0x7efce8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efce8066f60 0x7efce8069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0ae000 tx=0x7efcfc0a7070 comp rx=0 tx=0).stop 2024-04-25T08:44:58.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c0cc9e0 msgr2=0x7efd0c1f67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c0cc9e0 0x7efd0c1f67b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efcf400c820 tx=0x7efcf400cce0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 shutdown_connections 2024-04-25T08:44:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efce8066f60 0x7efce8069420 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd0c15a750 0x7efd0c14a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd0c0cc9e0 0x7efd0c1f67b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 --2- 172.21.15.46:0/871576232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd0c0aed10 0x7efd0c148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 >> 172.21.15.46:0/871576232 conn(0x7efd0c0b12a0 msgr2=0x7efd0c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 shutdown_connections 2024-04-25T08:44:58.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.706+0000 7efd11200640 1 -- 172.21.15.46:0/871576232 wait complete. 2024-04-25T08:44:58.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca4600640 1 Processor -- start 2024-04-25T08:44:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca4600640 1 -- start start 2024-04-25T08:44:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca4600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c9c159230 0x7f5c9c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca4600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c9c0cbeb0 0x7f5c9c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca4600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cda80 0x7f5c9c15f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca4600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c0cc700 con 0x7f5c9c0cbeb0 2024-04-25T08:44:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca4600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c0cc9e0 con 0x7f5c9c159230 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca4600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0cccc0 con 0x7f5c9c0cda80 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c9c0cbeb0 0x7f5c9c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c9c0cbeb0 0x7f5c9c0cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56632/0 (socket says 172.21.15.46:56632) 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca1c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c9c159230 0x7f5c9c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca1200640 1 -- 172.21.15.46:0/1420022050 learned_addr learned my addr 172.21.15.46:0/1420022050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca2600640 1 --2- 172.21.15.46:0/1420022050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cda80 0x7f5c9c15f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca1200640 1 -- 172.21.15.46:0/1420022050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cda80 msgr2=0x7f5c9c15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca1200640 1 --2- 172.21.15.46:0/1420022050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cda80 0x7f5c9c15f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca1200640 1 -- 172.21.15.46:0/1420022050 >> v2:172.21.15.82:3300/0 conn(0x7f5c9c159230 msgr2=0x7f5c9c15bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca1200640 1 --2- 172.21.15.46:0/1420022050 >> v2:172.21.15.82:3300/0 conn(0x7f5c9c159230 0x7f5c9c15bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.870+0000 7f5ca1200640 1 -- 172.21.15.46:0/1420022050 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c0ccfa0 con 0x7f5c9c0cbeb0 2024-04-25T08:44:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca1200640 1 --2- 172.21.15.46:0/1420022050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c0cbeb0 0x7f5c9c0cd540 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c072a90 tx=0x7f5c8801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1043537da72fad1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5c9be00640 1 -- 172.21.15.46:0/1420022050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c880b0070 con 0x7f5c9c0cbeb0 2024-04-25T08:44:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5c9be00640 1 -- 172.21.15.46:0/1420022050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c880ab600 con 0x7f5c9c0cbeb0 2024-04-25T08:44:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5c9be00640 1 -- 172.21.15.46:0/1420022050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c880ab9d0 con 0x7f5c9c0cbeb0 2024-04-25T08:44:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca4600640 1 -- 172.21.15.46:0/1420022050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c0cbeb0 msgr2=0x7f5c9c0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca4600640 1 --2- 172.21.15.46:0/1420022050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c0cbeb0 0x7f5c9c0cd540 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c072a90 tx=0x7f5c8801c1d0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca4600640 1 -- 172.21.15.46:0/1420022050 shutdown_connections 2024-04-25T08:44:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca4600640 1 --2- 172.21.15.46:0/1420022050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cda80 0x7f5c9c15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca4600640 1 --2- 172.21.15.46:0/1420022050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c0cbeb0 0x7f5c9c0cd540 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca4600640 1 --2- 172.21.15.46:0/1420022050 >> v2:172.21.15.82:3300/0 conn(0x7f5c9c159230 0x7f5c9c15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca4600640 1 -- 172.21.15.46:0/1420022050 >> 172.21.15.46:0/1420022050 conn(0x7f5c9c0b12a0 msgr2=0x7f5c9c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca4600640 1 -- 172.21.15.46:0/1420022050 shutdown_connections 2024-04-25T08:44:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.874+0000 7f5ca4600640 1 -- 172.21.15.46:0/1420022050 wait complete. 2024-04-25T08:44:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 Processor -- start 2024-04-25T08:44:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 -- start start 2024-04-25T08:44:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cbeb0 0x7f5c9c161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c9c0cda80 0x7f5c9c165b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c159230 0x7f5c9c0af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c15d380 con 0x7f5c9c159230 2024-04-25T08:44:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c15d200 con 0x7f5c9c0cda80 2024-04-25T08:44:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c15d500 con 0x7f5c9c0cbeb0 2024-04-25T08:44:58.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c159230 0x7f5c9c0af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c159230 0x7f5c9c0af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56636/0 (socket says 172.21.15.46:56636) 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca2600640 1 -- 172.21.15.46:0/2320720415 learned_addr learned my addr 172.21.15.46:0/2320720415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca1200640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c9c0cda80 0x7f5c9c165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca1c00640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cbeb0 0x7f5c9c161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca2600640 1 -- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cbeb0 msgr2=0x7f5c9c161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca2600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cbeb0 0x7f5c9c161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca2600640 1 -- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c9c0cda80 msgr2=0x7f5c9c165b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca2600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c9c0cda80 0x7f5c9c165b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca2600640 1 -- 172.21.15.46:0/2320720415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c160b30 con 0x7f5c9c159230 2024-04-25T08:44:58.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca2600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c159230 0x7f5c9c0af0d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400eed0 tx=0x7f5c9400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5c99600640 1 -- 172.21.15.46:0/2320720415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94018020 con 0x7f5c9c159230 2024-04-25T08:44:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5c99600640 1 -- 172.21.15.46:0/2320720415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c94014400 con 0x7f5c9c159230 2024-04-25T08:44:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c160e10 con 0x7f5c9c159230 2024-04-25T08:44:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5c99600640 1 -- 172.21.15.46:0/2320720415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c940147f0 con 0x7f5c9c159230 2024-04-25T08:44:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.878+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c1613d0 con 0x7f5c9c159230 2024-04-25T08:44:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.882+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c069080 con 0x7f5c9c159230 2024-04-25T08:44:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.882+0000 7f5c99600640 1 -- 172.21.15.46:0/2320720415 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c9401b020 con 0x7f5c9c159230 2024-04-25T08:44:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.882+0000 7f5c99600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c7c066f60 0x7f5c7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.882+0000 7f5c99600640 1 -- 172.21.15.46:0/2320720415 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c94085ad0 con 0x7f5c9c159230 2024-04-25T08:44:58.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.882+0000 7f5ca1c00640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c7c066f60 0x7f5c7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:58.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.882+0000 7f5ca1c00640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c7c066f60 0x7f5c7c069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c1663e0 tx=0x7f5c90009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:58.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:58.886+0000 7f5c99600640 1 -- 172.21.15.46:0/2320720415 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c94022030 con 0x7f5c9c159230 2024-04-25T08:44:59.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.050+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c9c0723d0 con 0x7f5c9c159230 2024-04-25T08:44:59.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5c99600640 1 -- 172.21.15.46:0/2320720415 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f5c94056260 con 0x7f5c9c159230 2024-04-25T08:44:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:59.046 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:15.829540+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:44:12.022357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:44:13.215069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c7c066f60 msgr2=0x7f5c7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c7c066f60 0x7f5c7c069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c1663e0 tx=0x7f5c90009190 comp rx=0 tx=0).stop 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c159230 msgr2=0x7f5c9c0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c159230 0x7f5c9c0af0d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400eed0 tx=0x7f5c9400b480 comp rx=0 tx=0).stop 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 shutdown_connections 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5c7c066f60 0x7f5c7c069420 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c9c159230 0x7f5c9c0af0d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c9c0cda80 0x7f5c9c165b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 --2- 172.21.15.46:0/2320720415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5c9c0cbeb0 0x7f5c9c161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 >> 172.21.15.46:0/2320720415 conn(0x7f5c9c0b12a0 msgr2=0x7f5c9c0b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 shutdown_connections 2024-04-25T08:44:59.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.054+0000 7f5ca4600640 1 -- 172.21.15.46:0/2320720415 wait complete. 2024-04-25T08:44:59.067 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:44:59.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:59.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.214+0000 7fdee9a00640 1 Processor -- start 2024-04-25T08:44:59.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.214+0000 7fdee9a00640 1 -- start start 2024-04-25T08:44:59.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.214+0000 7fdee9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee40afd40 0x7fdee40d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.214+0000 7fdee9a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee406ed70 con 0x7fdee415dca0 2024-04-25T08:44:59.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.214+0000 7fdee9a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee4162530 con 0x7fdee40cf140 2024-04-25T08:44:59.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.214+0000 7fdee9a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee40afb40 con 0x7fdee40afd40 2024-04-25T08:44:59.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.214+0000 7fdee3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee40afd40 0x7fdee40d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.214+0000 7fdedbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdee415dca0 0x7fdee40d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48472/0 (socket says 172.21.15.46:48472) 2024-04-25T08:44:59.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.214+0000 7fdedbe00640 1 -- 172.21.15.46:0/2705194061 learned_addr learned my addr 172.21.15.46:0/2705194061 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:59.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee3400640 1 -- 172.21.15.46:0/2705194061 >> v1:172.21.15.82:6789/0 conn(0x7fdee40cf140 legacy=0x7fdee415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee3400640 1 -- 172.21.15.46:0/2705194061 >> v1:172.21.15.46:6789/0 conn(0x7fdee415dca0 legacy=0x7fdee40d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee3400640 1 -- 172.21.15.46:0/2705194061 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4161770 con 0x7fdee40afd40 2024-04-25T08:44:59.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee2000640 1 -- 172.21.15.46:0/2705194061 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1434953248 0 0) 0x7fdecc001120 con 0x7fdee415dca0 2024-04-25T08:44:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee3400640 1 --2- 172.21.15.46:0/2705194061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 0x7fdee40d0af0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009d90 tx=0x7fdedc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54db8d8660e41a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee2000640 1 -- 172.21.15.46:0/2705194061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc0b2070 con 0x7fdee40afd40 2024-04-25T08:44:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee2000640 1 -- 172.21.15.46:0/2705194061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc0a59a0 con 0x7fdee40afd40 2024-04-25T08:44:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee2000640 1 -- 172.21.15.46:0/2705194061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc0a5d70 con 0x7fdee40afd40 2024-04-25T08:44:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee9a00640 1 -- 172.21.15.46:0/2705194061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 msgr2=0x7fdee40d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.218+0000 7fdee9a00640 1 --2- 172.21.15.46:0/2705194061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 0x7fdee40d0af0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009d90 tx=0x7fdedc01c8e0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 -- 172.21.15.46:0/2705194061 shutdown_connections 2024-04-25T08:44:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 --2- 172.21.15.46:0/2705194061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 0x7fdee40d0af0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 -- 172.21.15.46:0/2705194061 >> 172.21.15.46:0/2705194061 conn(0x7fdee40728c0 msgr2=0x7fdee4099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 -- 172.21.15.46:0/2705194061 shutdown_connections 2024-04-25T08:44:59.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 -- 172.21.15.46:0/2705194061 wait complete. 2024-04-25T08:44:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 Processor -- start 2024-04-25T08:44:59.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 -- start start 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 0x7fdee415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40cf140 0x7fdee415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee415dca0 0x7fdee41609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40af7f0 con 0x7fdee40afd40 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40d1440 con 0x7fdee415dca0 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee406b600 con 0x7fdee40cf140 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 0x7fdee415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 0x7fdee415e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56658/0 (socket says 172.21.15.46:56658) 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee2a00640 1 -- 172.21.15.46:0/1038353140 learned_addr learned my addr 172.21.15.46:0/1038353140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdedbe00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40cf140 0x7fdee415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee3400640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee415dca0 0x7fdee41609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee2a00640 1 -- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40cf140 msgr2=0x7fdee415f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee2a00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40cf140 0x7fdee415f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee2a00640 1 -- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee415dca0 msgr2=0x7fdee41609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee2a00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee415dca0 0x7fdee41609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee2a00640 1 -- 172.21.15.46:0/1038353140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee415e720 con 0x7fdee40afd40 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.222+0000 7fdee2a00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 0x7fdee415e140 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fded400c530 tx=0x7fded400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdedaa00640 1 -- 172.21.15.46:0/1038353140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4016020 con 0x7fdee40afd40 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdedaa00640 1 -- 172.21.15.46:0/1038353140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded4005220 con 0x7fdee40afd40 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdedaa00640 1 -- 172.21.15.46:0/1038353140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded4007030 con 0x7fdee40afd40 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee415ea00 con 0x7fdee40afd40 2024-04-25T08:44:59.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee415efc0 con 0x7fdee40afd40 2024-04-25T08:44:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee406ccb0 con 0x7fdee40afd40 2024-04-25T08:44:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdedaa00640 1 -- 172.21.15.46:0/1038353140 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fded40071d0 con 0x7fdee40afd40 2024-04-25T08:44:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdedaa00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec0066f60 0x7fdec0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdedaa00640 1 -- 172.21.15.46:0/1038353140 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fded4084aa0 con 0x7fdee40afd40 2024-04-25T08:44:59.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.226+0000 7fdedbe00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec0066f60 0x7fdec0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.230+0000 7fdedbe00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec0066f60 0x7fdec0069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdee40b58c0 tx=0x7fdecc007b10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:59.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.230+0000 7fdedaa00640 1 -- 172.21.15.46:0/1038353140 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fded4020020 con 0x7fdee40afd40 2024-04-25T08:44:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.430+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdee415abb0 con 0x7fdee40afd40 2024-04-25T08:44:59.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.430+0000 7fdedaa00640 1 -- 172.21.15.46:0/1038353140 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4935 (secure 0 0 0) 0x7fded40074c0 con 0x7fdee40afd40 2024-04-25T08:44:59.424 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:59.424 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/850914154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":850914154},{"type":"v1","addr":"172.21.15.46:6835","nonce":850914154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1115060038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1115060038},{"type":"v1","addr":"172.21.15.82:6836","nonce":1115060038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5620,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1383788710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1383788710},{"type":"v1","addr":"172.21.15.82:6837","nonce":1383788710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:44:14.218049+0000","modified":"2024-04-25T08:44:17.246143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/3843568539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3843568539},{"type":"v1","addr":"172.21.15.46:6834","nonce":3843568539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:44:59.424 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-04-25T08:44:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec0066f60 msgr2=0x7fdec0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec0066f60 0x7fdec0069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdee40b58c0 tx=0x7fdecc007b10 comp rx=0 tx=0).stop 2024-04-25T08:44:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 msgr2=0x7fdee415e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 0x7fdee415e140 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fded400c530 tx=0x7fded400c9f0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 shutdown_connections 2024-04-25T08:44:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec0066f60 0x7fdec0069420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee415dca0 0x7fdee41609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee40cf140 0x7fdee415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 --2- 172.21.15.46:0/1038353140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee40afd40 0x7fdee415e140 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 >> 172.21.15.46:0/1038353140 conn(0x7fdee40728c0 msgr2=0x7fdee406ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:59.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 shutdown_connections 2024-04-25T08:44:59.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.434+0000 7fdee9a00640 1 -- 172.21.15.46:0/1038353140 wait complete. 2024-04-25T08:44:59.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:44:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.590+0000 7f276be00640 1 Processor -- start 2024-04-25T08:44:59.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.590+0000 7f276be00640 1 -- start start 2024-04-25T08:44:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f276be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f276406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f276be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276406d5e0 con 0x7f27640cc9e0 2024-04-25T08:44:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f276be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27640ce720 con 0x7f27640aed10 2024-04-25T08:44:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f276be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276415ab30 con 0x7f276415a750 2024-04-25T08:44:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2769e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f27640cc9e0 0x7f27640b2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48486/0 (socket says 172.21.15.46:48486) 2024-04-25T08:44:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2769e00640 1 -- 172.21.15.46:0/3520119608 learned_addr learned my addr 172.21.15.46:0/3520119608 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2769400640 1 --2- 172.21.15.46:0/3520119608 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f276406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2763e00640 1 -- 172.21.15.46:0/3520119608 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615315351 0 0) 0x7f276406d5e0 con 0x7f27640cc9e0 2024-04-25T08:44:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2763e00640 1 -- 172.21.15.46:0/3520119608 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f274c003530 con 0x7f27640cc9e0 2024-04-25T08:44:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2763e00640 1 -- 172.21.15.46:0/3520119608 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373107763 0 0) 0x7f27640ce720 con 0x7f27640aed10 2024-04-25T08:44:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2763e00640 1 -- 172.21.15.46:0/3520119608 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f276406d5e0 con 0x7f27640aed10 2024-04-25T08:44:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2769400640 1 -- 172.21.15.46:0/3520119608 >> v1:172.21.15.82:6789/0 conn(0x7f27640aed10 legacy=0x7f27640b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2769400640 1 -- 172.21.15.46:0/3520119608 >> v1:172.21.15.46:6789/0 conn(0x7f27640cc9e0 legacy=0x7f27640b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.594+0000 7f2769400640 1 -- 172.21.15.46:0/3520119608 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640cea00 con 0x7f276415a750 2024-04-25T08:44:59.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f2769400640 1 --2- 172.21.15.46:0/3520119608 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f276406ec10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f27580097f0 tx=0x7f275801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40205eba19d5d824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f2763e00640 1 -- 172.21.15.46:0/3520119608 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27580b0070 con 0x7f276415a750 2024-04-25T08:44:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f2763e00640 1 -- 172.21.15.46:0/3520119608 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27580ab600 con 0x7f276415a750 2024-04-25T08:44:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f2763e00640 1 -- 172.21.15.46:0/3520119608 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27580ab9d0 con 0x7f276415a750 2024-04-25T08:44:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f276be00640 1 -- 172.21.15.46:0/3520119608 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 msgr2=0x7f276406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f276be00640 1 --2- 172.21.15.46:0/3520119608 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f276406ec10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f27580097f0 tx=0x7f275801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f276be00640 1 -- 172.21.15.46:0/3520119608 shutdown_connections 2024-04-25T08:44:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f276be00640 1 --2- 172.21.15.46:0/3520119608 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f276406ec10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f276be00640 1 -- 172.21.15.46:0/3520119608 >> 172.21.15.46:0/3520119608 conn(0x7f27640b12a0 msgr2=0x7f27640b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f276be00640 1 -- 172.21.15.46:0/3520119608 shutdown_connections 2024-04-25T08:44:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f276be00640 1 -- 172.21.15.46:0/3520119608 wait complete. 2024-04-25T08:44:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f276be00640 1 Processor -- start 2024-04-25T08:44:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.598+0000 7f276be00640 1 -- start start 2024-04-25T08:44:59.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f276be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27640aed10 0x7f2764137de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f276be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27640cc9e0 0x7f276414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f276be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f27640adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f276be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2764168a50 con 0x7f27640cc9e0 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f276be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276406f560 con 0x7f27640aed10 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f276be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2764168bd0 con 0x7f276415a750 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2768a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27640cc9e0 0x7f276414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2768a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27640cc9e0 0x7f276414ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56662/0 (socket says 172.21.15.46:56662) 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2768a00640 1 -- 172.21.15.46:0/2697745248 learned_addr learned my addr 172.21.15.46:0/2697745248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2769e00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f27640adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2769400640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27640aed10 0x7f2764137de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2768a00640 1 -- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 msgr2=0x7f27640adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2768a00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f27640adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2768a00640 1 -- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27640aed10 msgr2=0x7f2764137de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2768a00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27640aed10 0x7f2764137de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2768a00640 1 -- 172.21.15.46:0/2697745248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27640ac870 con 0x7f27640cc9e0 2024-04-25T08:44:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2769400640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27640aed10 0x7f2764137de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2769e00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f27640adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:44:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2768a00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27640cc9e0 0x7f276414ce90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f275000c430 tx=0x7f275000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2761600640 1 -- 172.21.15.46:0/2697745248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750017070 con 0x7f27640cc9e0 2024-04-25T08:44:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2761600640 1 -- 172.21.15.46:0/2697745248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27500052f0 con 0x7f27640cc9e0 2024-04-25T08:44:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276414d3d0 con 0x7f27640cc9e0 2024-04-25T08:44:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2761600640 1 -- 172.21.15.46:0/2697745248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750003840 con 0x7f27640cc9e0 2024-04-25T08:44:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27640acdf0 con 0x7f27640cc9e0 2024-04-25T08:44:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2764069080 con 0x7f27640cc9e0 2024-04-25T08:44:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.602+0000 7f2761600640 1 -- 172.21.15.46:0/2697745248 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f275001a070 con 0x7f27640cc9e0 2024-04-25T08:44:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.606+0000 7f2761600640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2744066f60 0x7f2744069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.606+0000 7f2761600640 1 -- 172.21.15.46:0/2697745248 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27500845c0 con 0x7f27640cc9e0 2024-04-25T08:44:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.606+0000 7f2769400640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2744066f60 0x7f2744069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.606+0000 7f2769400640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2744066f60 0x7f2744069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f27580ad000 tx=0x7f27580a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:59.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.610+0000 7f2761600640 1 -- 172.21.15.46:0/2697745248 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2750055450 con 0x7f27640cc9e0 2024-04-25T08:44:59.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.802+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27640b3ad0 con 0x7f27640cc9e0 2024-04-25T08:44:59.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f2761600640 1 -- 172.21.15.46:0/2697745248 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4935 (secure 0 0 0) 0x7f2750054d50 con 0x7f27640cc9e0 2024-04-25T08:44:59.798 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:44:59.798 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/850914154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":850914154},{"type":"v1","addr":"172.21.15.46:6835","nonce":850914154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1115060038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1115060038},{"type":"v1","addr":"172.21.15.82:6836","nonce":1115060038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5620,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1383788710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1383788710},{"type":"v1","addr":"172.21.15.82:6837","nonce":1383788710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:44:14.218049+0000","modified":"2024-04-25T08:44:17.246143+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5733},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5733":{"gid":5733,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6834/3843568539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3843568539},{"type":"v1","addr":"172.21.15.46:6834","nonce":3843568539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:44:59.798 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2744066f60 msgr2=0x7f2744069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2744066f60 0x7f2744069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f27580ad000 tx=0x7f27580a6000 comp rx=0 tx=0).stop 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27640cc9e0 msgr2=0x7f276414ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27640cc9e0 0x7f276414ce90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f275000c430 tx=0x7f275000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 shutdown_connections 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2744066f60 0x7f2744069420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f276415a750 0x7f27640adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27640cc9e0 0x7f276414ce90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 --2- 172.21.15.46:0/2697745248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27640aed10 0x7f2764137de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 >> 172.21.15.46:0/2697745248 conn(0x7f27640b12a0 msgr2=0x7f276415ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 shutdown_connections 2024-04-25T08:44:59.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.806+0000 7f276be00640 1 -- 172.21.15.46:0/2697745248 wait complete. 2024-04-25T08:44:59.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:44:59.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f463aa00640 1 Processor -- start 2024-04-25T08:44:59.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f463aa00640 1 -- start start 2024-04-25T08:44:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f463aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f463415dbd0 0x7f46340b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f463aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46340713a0 0x7f463406c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f463aa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46340b5250 con 0x7f46340b0c30 2024-04-25T08:44:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f463aa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340b5af0 con 0x7f463415dbd0 2024-04-25T08:44:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f463aa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340b5530 con 0x7f46340713a0 2024-04-25T08:44:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f462be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f46340b0c30 0x7f4634070c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48502/0 (socket says 172.21.15.46:48502) 2024-04-25T08:44:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f462be00640 1 -- 172.21.15.46:0/1020393251 learned_addr learned my addr 172.21.15.46:0/1020393251 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:44:59.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f4633e00640 1 --2- 172.21.15.46:0/1020393251 >> v2:172.21.15.46:3300/0 conn(0x7f463415dbd0 0x7f46340b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.966+0000 7f4638a00640 1 --2- 172.21.15.46:0/1020393251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46340713a0 0x7f463406c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4633e00640 1 -- 172.21.15.46:0/1020393251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46340713a0 msgr2=0x7f463406c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:44:59.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4633400640 1 -- 172.21.15.46:0/1020393251 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490017617 0 0) 0x7f46340b5250 con 0x7f46340b0c30 2024-04-25T08:44:59.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4633e00640 1 --2- 172.21.15.46:0/1020393251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46340713a0 0x7f463406c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4633e00640 1 -- 172.21.15.46:0/1020393251 >> v1:172.21.15.46:6789/0 conn(0x7f46340b0c30 legacy=0x7f4634070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4633e00640 1 -- 172.21.15.46:0/1020393251 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340b5810 con 0x7f463415dbd0 2024-04-25T08:44:59.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4638a00640 1 --2- 172.21.15.46:0/1020393251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46340713a0 0x7f463406c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:44:59.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4633e00640 1 --2- 172.21.15.46:0/1020393251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463415dbd0 0x7f46340b06f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f46240097c0 tx=0x7f462401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83428f6500d796d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:59.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4633400640 1 -- 172.21.15.46:0/1020393251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46240b1070 con 0x7f463415dbd0 2024-04-25T08:44:59.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4633400640 1 -- 172.21.15.46:0/1020393251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46240ac410 con 0x7f463415dbd0 2024-04-25T08:44:59.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f4633400640 1 -- 172.21.15.46:0/1020393251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46240ac7e0 con 0x7f463415dbd0 2024-04-25T08:44:59.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f463aa00640 1 -- 172.21.15.46:0/1020393251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463415dbd0 msgr2=0x7f46340b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f463aa00640 1 --2- 172.21.15.46:0/1020393251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463415dbd0 0x7f46340b06f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f46240097c0 tx=0x7f462401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f463aa00640 1 -- 172.21.15.46:0/1020393251 shutdown_connections 2024-04-25T08:44:59.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f463aa00640 1 --2- 172.21.15.46:0/1020393251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f46340713a0 0x7f463406c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f463aa00640 1 --2- 172.21.15.46:0/1020393251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463415dbd0 0x7f46340b06f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f463aa00640 1 -- 172.21.15.46:0/1020393251 >> 172.21.15.46:0/1020393251 conn(0x7f463406f4f0 msgr2=0x7f463406f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:44:59.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.970+0000 7f463aa00640 1 -- 172.21.15.46:0/1020393251 shutdown_connections 2024-04-25T08:44:59.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 -- 172.21.15.46:0/1020393251 wait complete. 2024-04-25T08:44:59.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 Processor -- start 2024-04-25T08:44:59.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 -- start start 2024-04-25T08:44:59.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340713a0 0x7f46340caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46340b0c30 0x7f46340caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463415dbd0 0x7f463414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463406d4e0 con 0x7f46340713a0 2024-04-25T08:44:59.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463406d360 con 0x7f46340b0c30 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463406d660 con 0x7f463415dbd0 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4633e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340713a0 0x7f46340caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4633e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340713a0 0x7f46340caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56684/0 (socket says 172.21.15.46:56684) 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4633e00640 1 -- 172.21.15.46:0/3455339636 learned_addr learned my addr 172.21.15.46:0/3455339636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f462be00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46340b0c30 0x7f46340caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4638a00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463415dbd0 0x7f463414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4633e00640 1 -- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463415dbd0 msgr2=0x7f463414a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4633e00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463415dbd0 0x7f463414a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4633e00640 1 -- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46340b0c30 msgr2=0x7f46340caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4633e00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46340b0c30 0x7f46340caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4633e00640 1 -- 172.21.15.46:0/3455339636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340d12b0 con 0x7f46340713a0 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4633e00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340713a0 0x7f46340caa20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f462401a0d0 tx=0x7f4624019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4630c00640 1 -- 172.21.15.46:0/3455339636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46240b9020 con 0x7f46340713a0 2024-04-25T08:44:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4630c00640 1 -- 172.21.15.46:0/3455339636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46240b5410 con 0x7f46340713a0 2024-04-25T08:44:59.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f4630c00640 1 -- 172.21.15.46:0/3455339636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46240b1040 con 0x7f46340713a0 2024-04-25T08:44:59.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.974+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46340ccda0 con 0x7f46340713a0 2024-04-25T08:44:59.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.978+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46340cd360 con 0x7f46340713a0 2024-04-25T08:44:59.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.978+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4634069080 con 0x7f46340713a0 2024-04-25T08:44:59.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.978+0000 7f4630c00640 1 -- 172.21.15.46:0/3455339636 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46240bc020 con 0x7f46340713a0 2024-04-25T08:44:59.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.978+0000 7f4630c00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4610066f90 0x7f4610069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:44:59.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.978+0000 7f4630c00640 1 -- 172.21.15.46:0/3455339636 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f462411ed20 con 0x7f46340713a0 2024-04-25T08:44:59.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.978+0000 7f462be00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4610066f90 0x7f4610069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:44:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.978+0000 7f462be00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4610066f90 0x7f4610069450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f461c000f70 tx=0x7f461c007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:44:59.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:44:59.982+0000 7f4630c00640 1 -- 172.21.15.46:0/3455339636 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46240c3030 con 0x7f46340713a0 2024-04-25T08:45:00.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.150+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46340ce780 con 0x7f46340713a0 2024-04-25T08:45:00.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.150+0000 7f4630c00640 1 -- 172.21.15.46:0/3455339636 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16894 (secure 0 0 0) 0x7f46340d12b0 con 0x7f46340713a0 2024-04-25T08:45:00.144 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:00.144 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:44:15.829540+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:44:12.022357+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:44:13.215069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:45:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4610066f90 msgr2=0x7f4610069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4610066f90 0x7f4610069450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f461c000f70 tx=0x7f461c007b60 comp rx=0 tx=0).stop 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340713a0 msgr2=0x7f46340caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340713a0 0x7f46340caa20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f462401a0d0 tx=0x7f4624019240 comp rx=0 tx=0).stop 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 shutdown_connections 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4610066f90 0x7f4610069450 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463415dbd0 0x7f463414a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46340b0c30 0x7f46340caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 --2- 172.21.15.46:0/3455339636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46340713a0 0x7f46340caa20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 >> 172.21.15.46:0/3455339636 conn(0x7f463406f4f0 msgr2=0x7f463415c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 shutdown_connections 2024-04-25T08:45:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.154+0000 7f463aa00640 1 -- 172.21.15.46:0/3455339636 wait complete. 2024-04-25T08:45:00.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:45:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a1ca00640 1 Processor -- start 2024-04-25T08:45:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a1ca00640 1 -- start start 2024-04-25T08:45:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a1ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a1815bdb0 0x7f2a180cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a1ca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a180cda80 0x7f2a1815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a1ca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a180cc700 con 0x7f2a18159230 2024-04-25T08:45:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a1ca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a180cded0 con 0x7f2a1815bdb0 2024-04-25T08:45:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a1ca00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a180cc9e0 con 0x7f2a180cda80 2024-04-25T08:45:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a15400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a1815bdb0 0x7f2a180cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a15e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2a18159230 0x7f2a180cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48510/0 (socket says 172.21.15.46:48510) 2024-04-25T08:45:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a15e00640 1 -- 172.21.15.46:0/1912946885 learned_addr learned my addr 172.21.15.46:0/1912946885 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.314+0000 7f2a16800640 1 --2- 172.21.15.46:0/1912946885 >> v2:172.21.15.82:3300/0 conn(0x7f2a180cda80 0x7f2a1815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a15400640 1 -- 172.21.15.46:0/1912946885 >> v2:172.21.15.82:3300/0 conn(0x7f2a180cda80 msgr2=0x7f2a1815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a15400640 1 --2- 172.21.15.46:0/1912946885 >> v2:172.21.15.82:3300/0 conn(0x7f2a180cda80 0x7f2a1815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a15400640 1 -- 172.21.15.46:0/1912946885 >> v1:172.21.15.46:6789/0 conn(0x7f2a18159230 legacy=0x7f2a180cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a14a00640 1 -- 172.21.15.46:0/1912946885 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368020038 0 0) 0x7f2a180cc700 con 0x7f2a18159230 2024-04-25T08:45:00.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a15400640 1 -- 172.21.15.46:0/1912946885 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a180cccc0 con 0x7f2a1815bdb0 2024-04-25T08:45:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a15400640 1 --2- 172.21.15.46:0/1912946885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a1815bdb0 0x7f2a180cd540 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2a180b2340 tx=0x7f2a0401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6c0c5f5f92e12d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a14a00640 1 -- 172.21.15.46:0/1912946885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a040b0070 con 0x7f2a1815bdb0 2024-04-25T08:45:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a14a00640 1 -- 172.21.15.46:0/1912946885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a040ab600 con 0x7f2a1815bdb0 2024-04-25T08:45:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a14a00640 1 -- 172.21.15.46:0/1912946885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a040ab9d0 con 0x7f2a1815bdb0 2024-04-25T08:45:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1912946885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a1815bdb0 msgr2=0x7f2a180cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.318+0000 7f2a1ca00640 1 --2- 172.21.15.46:0/1912946885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a1815bdb0 0x7f2a180cd540 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2a180b2340 tx=0x7f2a0401c3f0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1912946885 shutdown_connections 2024-04-25T08:45:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 --2- 172.21.15.46:0/1912946885 >> v2:172.21.15.82:3300/0 conn(0x7f2a180cda80 0x7f2a1815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 --2- 172.21.15.46:0/1912946885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a1815bdb0 0x7f2a180cd540 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1912946885 >> 172.21.15.46:0/1912946885 conn(0x7f2a180b12a0 msgr2=0x7f2a180b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1912946885 shutdown_connections 2024-04-25T08:45:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1912946885 wait complete. 2024-04-25T08:45:00.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 Processor -- start 2024-04-25T08:45:00.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 -- start start 2024-04-25T08:45:00.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cda80 0x7f2a18166150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a18159230 0x7f2a1812b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a1815bdb0 0x7f2a181643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1815e0d0 con 0x7f2a180cda80 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1815df50 con 0x7f2a18159230 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a1ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1815e250 con 0x7f2a1815bdb0 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cda80 0x7f2a18166150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cda80 0x7f2a18166150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56692/0 (socket says 172.21.15.46:56692) 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15e00640 1 -- 172.21.15.46:0/1726438698 learned_addr learned my addr 172.21.15.46:0/1726438698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a16800640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a1815bdb0 0x7f2a181643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15400640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a18159230 0x7f2a1812b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15e00640 1 -- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a1815bdb0 msgr2=0x7f2a181643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15e00640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a1815bdb0 0x7f2a181643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15e00640 1 -- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a18159230 msgr2=0x7f2a1812b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15e00640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a18159230 0x7f2a1812b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15e00640 1 -- 172.21.15.46:0/1726438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1812bed0 con 0x7f2a180cda80 2024-04-25T08:45:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a15e00640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cda80 0x7f2a18166150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2a080011f0 tx=0x7f2a0800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a0e000640 1 -- 172.21.15.46:0/1726438698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08018070 con 0x7f2a180cda80 2024-04-25T08:45:00.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a0e000640 1 -- 172.21.15.46:0/1726438698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a080074e0 con 0x7f2a180cda80 2024-04-25T08:45:00.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.322+0000 7f2a0e000640 1 -- 172.21.15.46:0/1726438698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08004a10 con 0x7f2a180cda80 2024-04-25T08:45:00.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.326+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a18164910 con 0x7f2a180cda80 2024-04-25T08:45:00.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.326+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1812e4e0 con 0x7f2a180cda80 2024-04-25T08:45:00.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.326+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1813faa0 con 0x7f2a180cda80 2024-04-25T08:45:00.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.326+0000 7f2a0e000640 1 -- 172.21.15.46:0/1726438698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a08004bb0 con 0x7f2a180cda80 2024-04-25T08:45:00.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.326+0000 7f2a0e000640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29f8066f60 0x7f29f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.326+0000 7f2a0e000640 1 -- 172.21.15.46:0/1726438698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a080845a0 con 0x7f2a180cda80 2024-04-25T08:45:00.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.326+0000 7f2a15400640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29f8066f60 0x7f29f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.326+0000 7f2a15400640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29f8066f60 0x7f29f8069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a040ad000 tx=0x7f2a040a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.330+0000 7f2a0e000640 1 -- 172.21.15.46:0/1726438698 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a08054d30 con 0x7f2a180cda80 2024-04-25T08:45:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.526+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2a18069080 con 0x7f2a180cda80 2024-04-25T08:45:00.520 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:45:00.526+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:45:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a0e000640 1 -- 172.21.15.46:0/1726438698 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2a0805b080 con 0x7f2a180cda80 2024-04-25T08:45:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:45:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29f8066f60 msgr2=0x7f29f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29f8066f60 0x7f29f8069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a040ad000 tx=0x7f2a040a6000 comp rx=0 tx=0).stop 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cda80 msgr2=0x7f2a18166150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cda80 0x7f2a18166150 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2a080011f0 tx=0x7f2a0800c9b0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 shutdown_connections 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f29f8066f60 0x7f29f8069420 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a040ad000 tx=0x7f2a040a6000 comp rx=0 tx=0).stop 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a1815bdb0 0x7f2a181643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a18159230 0x7f2a1812b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 --2- 172.21.15.46:0/1726438698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cda80 0x7f2a18166150 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.530+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 >> 172.21.15.46:0/1726438698 conn(0x7f2a180b12a0 msgr2=0x7f2a180b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.534+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 shutdown_connections 2024-04-25T08:45:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.534+0000 7f2a1ca00640 1 -- 172.21.15.46:0/1726438698 wait complete. 2024-04-25T08:45:00.535 INFO:tasks.ceph.mds.a.smithi046.stderr: -5764> 2024-04-25T08:44:19.798+0000 7fb8b6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:45:00.540 INFO:tasks.ceph.mds.a.smithi046.stderr: -3166> 2024-04-25T08:44:23.126+0000 7fb8b1c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5752=0-4194304@1} caps={5752=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5752 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x560e37de1700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:45:00.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:45:00.588 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:45:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1233c00640 1 Processor -- start 2024-04-25T08:45:00.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1233c00640 1 -- start start 2024-04-25T08:45:00.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1233c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f122c15dae0 0x7f122c072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1233c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122c0b3850 con 0x7f122c06ea60 2024-04-25T08:45:00.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1233c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122c0b3af0 con 0x7f122c06e620 2024-04-25T08:45:00.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1233c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122c06e190 con 0x7f122c15dae0 2024-04-25T08:45:00.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1231c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f122c06ea60 0x7f122c0acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48534/0 (socket says 172.21.15.46:48534) 2024-04-25T08:45:00.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1231c00640 1 -- 172.21.15.46:0/630508013 learned_addr learned my addr 172.21.15.46:0/630508013 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1231200640 1 --2- 172.21.15.46:0/630508013 >> v2:172.21.15.82:3300/0 conn(0x7f122c15dae0 0x7f122c072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f122b400640 1 -- 172.21.15.46:0/630508013 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118390820 0 0) 0x7f122c0b3850 con 0x7f122c06ea60 2024-04-25T08:45:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f122b400640 1 -- 172.21.15.46:0/630508013 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1214003530 con 0x7f122c06ea60 2024-04-25T08:45:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1231200640 1 -- 172.21.15.46:0/630508013 >> v1:172.21.15.82:6789/0 conn(0x7f122c06e620 legacy=0x7f122c06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f122b400640 1 -- 172.21.15.46:0/630508013 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58259696 0 0) 0x7f122c0b3af0 con 0x7f122c06e620 2024-04-25T08:45:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1231200640 1 -- 172.21.15.46:0/630508013 >> v1:172.21.15.46:6789/0 conn(0x7f122c06ea60 legacy=0x7f122c0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.702+0000 7f1231200640 1 -- 172.21.15.46:0/630508013 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c0b3dd0 con 0x7f122c15dae0 2024-04-25T08:45:00.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f1231200640 1 --2- 172.21.15.46:0/630508013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f122c15dae0 0x7f122c072600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f12200097c0 tx=0x7f122001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76976457401d89af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f122b400640 1 -- 172.21.15.46:0/630508013 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12200a7070 con 0x7f122c15dae0 2024-04-25T08:45:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f122b400640 1 -- 172.21.15.46:0/630508013 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12200b0070 con 0x7f122c15dae0 2024-04-25T08:45:00.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f122b400640 1 -- 172.21.15.46:0/630508013 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12200ab890 con 0x7f122c15dae0 2024-04-25T08:45:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f1233c00640 1 -- 172.21.15.46:0/630508013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f122c15dae0 msgr2=0x7f122c072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f1233c00640 1 --2- 172.21.15.46:0/630508013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f122c15dae0 0x7f122c072600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f12200097c0 tx=0x7f122001c470 comp rx=0 tx=0).stop 2024-04-25T08:45:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f1233c00640 1 -- 172.21.15.46:0/630508013 shutdown_connections 2024-04-25T08:45:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f1233c00640 1 --2- 172.21.15.46:0/630508013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f122c15dae0 0x7f122c072600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f1233c00640 1 -- 172.21.15.46:0/630508013 >> 172.21.15.46:0/630508013 conn(0x7f122c071ae0 msgr2=0x7f122c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f1233c00640 1 -- 172.21.15.46:0/630508013 shutdown_connections 2024-04-25T08:45:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.706+0000 7f1233c00640 1 -- 172.21.15.46:0/630508013 wait complete. 2024-04-25T08:45:00.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 Processor -- start 2024-04-25T08:45:00.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 -- start start 2024-04-25T08:45:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f122c06e620 0x7f122c14c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06ea60 0x7f122c158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f122c15dae0 0x7f122c159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06f6d0 con 0x7f122c06ea60 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c15dec0 con 0x7f122c06e620 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122c06f850 con 0x7f122c15dae0 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f122be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06ea60 0x7f122c158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1231200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f122c06e620 0x7f122c14c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f122be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06ea60 0x7f122c158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56708/0 (socket says 172.21.15.46:56708) 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f122be00640 1 -- 172.21.15.46:0/982132192 learned_addr learned my addr 172.21.15.46:0/982132192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1231c00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f122c15dae0 0x7f122c159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f122be00640 1 -- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f122c15dae0 msgr2=0x7f122c159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f122be00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f122c15dae0 0x7f122c159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f122be00640 1 -- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f122c06e620 msgr2=0x7f122c14c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f122be00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f122c06e620 0x7f122c14c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f122be00640 1 -- 172.21.15.46:0/982132192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c14cf10 con 0x7f122c06ea60 2024-04-25T08:45:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f122be00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06ea60 0x7f122c158e30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f121800c4d0 tx=0x7f121800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1228c00640 1 -- 172.21.15.46:0/982132192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218017070 con 0x7f122c06ea60 2024-04-25T08:45:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1228c00640 1 -- 172.21.15.46:0/982132192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218005480 con 0x7f122c06ea60 2024-04-25T08:45:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1228c00640 1 -- 172.21.15.46:0/982132192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218003840 con 0x7f122c06ea60 2024-04-25T08:45:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c150270 con 0x7f122c06ea60 2024-04-25T08:45:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.710+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c150830 con 0x7f122c06ea60 2024-04-25T08:45:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.714+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c069010 con 0x7f122c06ea60 2024-04-25T08:45:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.714+0000 7f1228c00640 1 -- 172.21.15.46:0/982132192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f121801a070 con 0x7f122c06ea60 2024-04-25T08:45:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.714+0000 7f1228c00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f120c066f60 0x7f120c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.714+0000 7f1228c00640 1 -- 172.21.15.46:0/982132192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f12180844b0 con 0x7f122c06ea60 2024-04-25T08:45:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.714+0000 7f1231200640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f120c066f60 0x7f120c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:00.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.714+0000 7f1231200640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f120c066f60 0x7f120c069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f12200af000 tx=0x7f12200a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:00.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.718+0000 7f1228c00640 1 -- 172.21.15.46:0/982132192 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1218054de0 con 0x7f122c06ea60 2024-04-25T08:45:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:00.914+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f122c151fd0 con 0x7f122c06ea60 2024-04-25T08:45:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.530+0000 7f1228c00640 1 -- 172.21.15.46:0/982132192 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1218058e80 con 0x7f122c06ea60 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f120c066f60 msgr2=0x7f120c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f120c066f60 0x7f120c069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f12200af000 tx=0x7f12200a7000 comp rx=0 tx=0).stop 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06ea60 msgr2=0x7f122c158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06ea60 0x7f122c158e30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f121800c4d0 tx=0x7f121800c990 comp rx=0 tx=0).stop 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 shutdown_connections 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f120c066f60 0x7f120c069420 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f122c15dae0 0x7f122c159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f122c06ea60 0x7f122c158e30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 --2- 172.21.15.46:0/982132192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f122c06e620 0x7f122c14c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 >> 172.21.15.46:0/982132192 conn(0x7f122c071ae0 msgr2=0x7f122c1600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 shutdown_connections 2024-04-25T08:45:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.534+0000 7f1233c00640 1 -- 172.21.15.46:0/982132192 wait complete. 2024-04-25T08:45:01.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:45:01.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.750+0000 7efce7200640 1 Processor -- start 2024-04-25T08:45:01.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.750+0000 7efce7200640 1 -- start start 2024-04-25T08:45:01.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.750+0000 7efce7200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efce013e6b0 0x7efce00711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:01.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015c680 0x7efce006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:01.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce006d330 con 0x7efce0071720 2024-04-25T08:45:01.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce7200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce015cce0 con 0x7efce013e6b0 2024-04-25T08:45:01.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce006d610 con 0x7efce015c680 2024-04-25T08:45:01.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efcdfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efce013e6b0 0x7efce00711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce5200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015c680 0x7efce006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce5200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015c680 0x7efce006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:34930/0 (socket says 172.21.15.46:34930) 2024-04-25T08:45:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efcdfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efce013e6b0 0x7efce00711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:58064/0 (socket says 172.21.15.46:58064) 2024-04-25T08:45:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce5200640 1 -- 172.21.15.46:0/665341902 learned_addr learned my addr 172.21.15.46:0/665341902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce5200640 1 -- 172.21.15.46:0/665341902 >> v2:172.21.15.82:3300/0 conn(0x7efce013e6b0 msgr2=0x7efce00711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce5200640 1 --2- 172.21.15.46:0/665341902 >> v2:172.21.15.82:3300/0 conn(0x7efce013e6b0 0x7efce00711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce5200640 1 -- 172.21.15.46:0/665341902 >> v1:172.21.15.82:6789/0 conn(0x7efce0071720 legacy=0x7efce015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:01.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.754+0000 7efce5200640 1 -- 172.21.15.46:0/665341902 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce006d8f0 con 0x7efce015c680 2024-04-25T08:45:01.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce5200640 1 --2- 172.21.15.46:0/665341902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015c680 0x7efce006e0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009e50 tx=0x7efcd801c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3717e14ba531bfc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:01.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efcdf400640 1 -- 172.21.15.46:0/665341902 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80a55d0 con 0x7efce015c680 2024-04-25T08:45:01.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efcdf400640 1 -- 172.21.15.46:0/665341902 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd80a5770 con 0x7efce015c680 2024-04-25T08:45:01.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efcdf400640 1 -- 172.21.15.46:0/665341902 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80a5b40 con 0x7efce015c680 2024-04-25T08:45:01.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 -- 172.21.15.46:0/665341902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015c680 msgr2=0x7efce006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:01.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 --2- 172.21.15.46:0/665341902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015c680 0x7efce006e0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009e50 tx=0x7efcd801c910 comp rx=0 tx=0).stop 2024-04-25T08:45:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 -- 172.21.15.46:0/665341902 shutdown_connections 2024-04-25T08:45:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 --2- 172.21.15.46:0/665341902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce015c680 0x7efce006e0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 --2- 172.21.15.46:0/665341902 >> v2:172.21.15.82:3300/0 conn(0x7efce013e6b0 0x7efce00711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 -- 172.21.15.46:0/665341902 >> 172.21.15.46:0/665341902 conn(0x7efce0067050 msgr2=0x7efce006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 -- 172.21.15.46:0/665341902 shutdown_connections 2024-04-25T08:45:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 -- 172.21.15.46:0/665341902 wait complete. 2024-04-25T08:45:01.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 Processor -- start 2024-04-25T08:45:01.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 -- start start 2024-04-25T08:45:01.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071720 0x7efce01f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:01.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce013e6b0 0x7efce01f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:01.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce015c680 0x7efce013a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:01.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce0071b30 con 0x7efce015c680 2024-04-25T08:45:01.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce0168590 con 0x7efce013e6b0 2024-04-25T08:45:01.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.758+0000 7efce7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce0168710 con 0x7efce0071720 2024-04-25T08:45:01.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce015c680 0x7efce013a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:01.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce015c680 0x7efce013a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56722/0 (socket says 172.21.15.46:56722) 2024-04-25T08:45:01.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce5200640 1 -- 172.21.15.46:0/716674502 learned_addr learned my addr 172.21.15.46:0/716674502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:01.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efcd7e00640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce013e6b0 0x7efce01f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:01.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efcdfe00640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071720 0x7efce01f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:01.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce5200640 1 -- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071720 msgr2=0x7efce01f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:01.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce5200640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071720 0x7efce01f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:01.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce5200640 1 -- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce013e6b0 msgr2=0x7efce01f9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:01.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce5200640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce013e6b0 0x7efce01f9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:01.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce5200640 1 -- 172.21.15.46:0/716674502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce01fa070 con 0x7efce015c680 2024-04-25T08:45:01.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce5200640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce015c680 0x7efce013a160 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efcd80af040 tx=0x7efcd80196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:01.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efcdcc00640 1 -- 172.21.15.46:0/716674502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80bc020 con 0x7efce015c680 2024-04-25T08:45:01.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efcdcc00640 1 -- 172.21.15.46:0/716674502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd80b8670 con 0x7efce015c680 2024-04-25T08:45:01.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efcdcc00640 1 -- 172.21.15.46:0/716674502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8009510 con 0x7efce015c680 2024-04-25T08:45:01.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce0138bf0 con 0x7efce015c680 2024-04-25T08:45:01.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce0139170 con 0x7efce015c680 2024-04-25T08:45:01.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce0069080 con 0x7efce015c680 2024-04-25T08:45:01.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.762+0000 7efcdcc00640 1 -- 172.21.15.46:0/716674502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcd80bf020 con 0x7efce015c680 2024-04-25T08:45:01.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.766+0000 7efcdcc00640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efcbc066f60 0x7efcbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:01.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.766+0000 7efcdcc00640 1 -- 172.21.15.46:0/716674502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6245+0+0 (secure 0 0 0) 0x7efcd8120150 con 0x7efce015c680 2024-04-25T08:45:01.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.766+0000 7efcdfe00640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efcbc066f60 0x7efcbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:01.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.766+0000 7efcdfe00640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efcbc066f60 0x7efcbc069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009b60 tx=0x7efcd0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:01.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.770+0000 7efcdcc00640 1 -- 172.21.15.46:0/716674502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcd80c6030 con 0x7efce015c680 2024-04-25T08:45:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:01.934+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efce00cf240 con 0x7efce015c680 2024-04-25T08:45:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.538+0000 7efcdcc00640 1 -- 172.21.15.46:0/716674502 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7efce01fa070 con 0x7efce015c680 2024-04-25T08:45:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:45:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efcbc066f60 msgr2=0x7efcbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efcbc066f60 0x7efcbc069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009b60 tx=0x7efcd0009190 comp rx=0 tx=0).stop 2024-04-25T08:45:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce015c680 msgr2=0x7efce013a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce015c680 0x7efce013a160 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efcd80af040 tx=0x7efcd80196b0 comp rx=0 tx=0).stop 2024-04-25T08:45:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 shutdown_connections 2024-04-25T08:45:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efcbc066f60 0x7efcbc069420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce015c680 0x7efce013a160 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efce013e6b0 0x7efce01f9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 --2- 172.21.15.46:0/716674502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efce0071720 0x7efce01f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 >> 172.21.15.46:0/716674502 conn(0x7efce0067050 msgr2=0x7efce015a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 shutdown_connections 2024-04-25T08:45:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.542+0000 7efce7200640 1 -- 172.21.15.46:0/716674502 wait complete. 2024-04-25T08:45:02.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:45:02.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcc2c00640 1 Processor -- start 2024-04-25T08:45:02.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcc2c00640 1 -- start start 2024-04-25T08:45:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcc2c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdcbc06b320 0x7fdcbc0cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcc2c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcbc160300 con 0x7fdcbc15c290 2024-04-25T08:45:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcc2c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcbc15f780 con 0x7fdcbc15beb0 2024-04-25T08:45:02.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcc2c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcbc0cc6c0 con 0x7fdcbc06b320 2024-04-25T08:45:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcc0c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdcbc15c290 0x7fdcbc15c700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48538/0 (socket says 172.21.15.46:48538) 2024-04-25T08:45:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcc0c00640 1 -- 172.21.15.46:0/1532606538 learned_addr learned my addr 172.21.15.46:0/1532606538 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcb3400640 1 --2- 172.21.15.46:0/1532606538 >> v2:172.21.15.82:3300/0 conn(0x7fdcbc06b320 0x7fdcbc0cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcbb400640 1 -- 172.21.15.46:0/1532606538 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347907956 0 0) 0x7fdcbc160300 con 0x7fdcbc15c290 2024-04-25T08:45:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcbb400640 1 -- 172.21.15.46:0/1532606538 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca0003530 con 0x7fdcbc15c290 2024-04-25T08:45:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcb3400640 1 -- 172.21.15.46:0/1532606538 >> v1:172.21.15.82:6789/0 conn(0x7fdcbc15beb0 legacy=0x7fdcbc06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcb3400640 1 -- 172.21.15.46:0/1532606538 >> v1:172.21.15.46:6789/0 conn(0x7fdcbc15c290 legacy=0x7fdcbc15c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:02.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.722+0000 7fdcb3400640 1 -- 172.21.15.46:0/1532606538 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc15fa60 con 0x7fdcbc06b320 2024-04-25T08:45:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcb3400640 1 --2- 172.21.15.46:0/1532606538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcbc06b320 0x7fdcbc0cc180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdca40097c0 tx=0x7fdca401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=643a3b7c454754e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcbb400640 1 -- 172.21.15.46:0/1532606538 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca40b0070 con 0x7fdcbc06b320 2024-04-25T08:45:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcbb400640 1 -- 172.21.15.46:0/1532606538 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca40ab600 con 0x7fdcbc06b320 2024-04-25T08:45:02.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcbb400640 1 -- 172.21.15.46:0/1532606538 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca40ab9b0 con 0x7fdcbc06b320 2024-04-25T08:45:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1532606538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcbc06b320 msgr2=0x7fdcbc0cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcc2c00640 1 --2- 172.21.15.46:0/1532606538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcbc06b320 0x7fdcbc0cc180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdca40097c0 tx=0x7fdca401c410 comp rx=0 tx=0).stop 2024-04-25T08:45:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1532606538 shutdown_connections 2024-04-25T08:45:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcc2c00640 1 --2- 172.21.15.46:0/1532606538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcbc06b320 0x7fdcbc0cc180 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1532606538 >> 172.21.15.46:0/1532606538 conn(0x7fdcbc099a00 msgr2=0x7fdcbc0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1532606538 shutdown_connections 2024-04-25T08:45:02.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.726+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1532606538 wait complete. 2024-04-25T08:45:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 Processor -- start 2024-04-25T08:45:02.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 -- start start 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdcbc06b320 0x7fdcbc14f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcbc15beb0 0x7fdcbc14fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15c290 0x7fdcbc1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0b2fb0 con 0x7fdcbc15c290 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0b2e30 con 0x7fdcbc15beb0 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc152930 con 0x7fdcbc06b320 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15c290 0x7fdcbc1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15c290 0x7fdcbc1575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56724/0 (socket says 172.21.15.46:56724) 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc0c00640 1 -- 172.21.15.46:0/1357308218 learned_addr learned my addr 172.21.15.46:0/1357308218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcb3400640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcbc15beb0 0x7fdcbc14fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:02.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcbbe00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdcbc06b320 0x7fdcbc14f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc0c00640 1 -- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdcbc06b320 msgr2=0x7fdcbc14f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc0c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdcbc06b320 0x7fdcbc14f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc0c00640 1 -- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcbc15beb0 msgr2=0x7fdcbc14fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:02.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc0c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcbc15beb0 0x7fdcbc14fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc0c00640 1 -- 172.21.15.46:0/1357308218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc150770 con 0x7fdcbc15c290 2024-04-25T08:45:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc0c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15c290 0x7fdcbc1575a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400cb40 tx=0x7fdcb400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcb8c00640 1 -- 172.21.15.46:0/1357308218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4017070 con 0x7fdcbc15c290 2024-04-25T08:45:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcb8c00640 1 -- 172.21.15.46:0/1357308218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb400aca0 con 0x7fdcbc15c290 2024-04-25T08:45:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcb8c00640 1 -- 172.21.15.46:0/1357308218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4012620 con 0x7fdcbc15c290 2024-04-25T08:45:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc150a50 con 0x7fdcbc15c290 2024-04-25T08:45:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.730+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcbc151010 con 0x7fdcbc15c290 2024-04-25T08:45:02.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.734+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcbc069080 con 0x7fdcbc15c290 2024-04-25T08:45:02.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.734+0000 7fdcb8c00640 1 -- 172.21.15.46:0/1357308218 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcb401a070 con 0x7fdcbc15c290 2024-04-25T08:45:02.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.734+0000 7fdcb8c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc98066f60 0x7fdc98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:02.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.734+0000 7fdcb8c00640 1 -- 172.21.15.46:0/1357308218 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5807+0+0 (secure 0 0 0) 0x7fdcb4084e90 con 0x7fdcbc15c290 2024-04-25T08:45:02.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.734+0000 7fdcbbe00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc98066f60 0x7fdc98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:02.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.734+0000 7fdcbbe00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc98066f60 0x7fdc98069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdcac001810 tx=0x7fdcac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.738+0000 7fdcb8c00640 1 -- 172.21.15.46:0/1357308218 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdcb4055750 con 0x7fdcbc15c290 2024-04-25T08:45:02.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:02.906+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdcbc072510 con 0x7fdcbc15c290 2024-04-25T08:45:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.538+0000 7fdcb8c00640 1 -- 172.21.15.46:0/1357308218 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdcb40860d0 con 0x7fdcbc15c290 2024-04-25T08:45:03.533 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-25T08:45:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc98066f60 msgr2=0x7fdc98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc98066f60 0x7fdc98069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdcac001810 tx=0x7fdcac009040 comp rx=0 tx=0).stop 2024-04-25T08:45:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15c290 msgr2=0x7fdcbc1575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15c290 0x7fdcbc1575a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400cb40 tx=0x7fdcb400a060 comp rx=0 tx=0).stop 2024-04-25T08:45:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 shutdown_connections 2024-04-25T08:45:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc98066f60 0x7fdc98069420 secure :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdcac001810 tx=0x7fdcac009040 comp rx=0 tx=0).stop 2024-04-25T08:45:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15c290 0x7fdcbc1575a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcbc15beb0 0x7fdcbc14fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 --2- 172.21.15.46:0/1357308218 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdcbc06b320 0x7fdcbc14f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 >> 172.21.15.46:0/1357308218 conn(0x7fdcbc099a00 msgr2=0x7fdcbc0ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 shutdown_connections 2024-04-25T08:45:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.542+0000 7fdcc2c00640 1 -- 172.21.15.46:0/1357308218 wait complete. 2024-04-25T08:45:03.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:45:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa8400640 1 Processor -- start 2024-04-25T08:45:03.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa8400640 1 -- start start 2024-04-25T08:45:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa8400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7fa006dc70 0x7f7fa00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa8400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa00cd930 con 0x7f7fa0145310 2024-04-25T08:45:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa8400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa00cdbd0 con 0x7f7fa00cf1b0 2024-04-25T08:45:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa8400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa00cdeb0 con 0x7f7fa006dc70 2024-04-25T08:45:03.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa5a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7fa0145310 0x7f7fa006e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48550/0 (socket says 172.21.15.46:48550) 2024-04-25T08:45:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa5a00640 1 -- 172.21.15.46:0/1292963458 learned_addr learned my addr 172.21.15.46:0/1292963458 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa5000640 1 --2- 172.21.15.46:0/1292963458 >> v2:172.21.15.46:3300/0 conn(0x7f7fa006dc70 0x7f7fa00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa5000640 1 -- 172.21.15.46:0/1292963458 >> v1:172.21.15.82:6789/0 conn(0x7f7fa00cf1b0 legacy=0x7f7fa015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa5000640 1 -- 172.21.15.46:0/1292963458 >> v1:172.21.15.46:6789/0 conn(0x7f7fa0145310 legacy=0x7f7fa006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7f9fe00640 1 -- 172.21.15.46:0/1292963458 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57168039 0 0) 0x7f7fa00cd930 con 0x7f7fa0145310 2024-04-25T08:45:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.718+0000 7f7fa5000640 1 -- 172.21.15.46:0/1292963458 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa00ce190 con 0x7f7fa006dc70 2024-04-25T08:45:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa5000640 1 --2- 172.21.15.46:0/1292963458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa006dc70 0x7f7fa00cec70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0097c0 tx=0x7f7f8c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb43af81246381bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7f9fe00640 1 -- 172.21.15.46:0/1292963458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0a7070 con 0x7f7fa006dc70 2024-04-25T08:45:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7f9fe00640 1 -- 172.21.15.46:0/1292963458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c0b0070 con 0x7f7fa006dc70 2024-04-25T08:45:03.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7f9fe00640 1 -- 172.21.15.46:0/1292963458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c0ab890 con 0x7f7fa006dc70 2024-04-25T08:45:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa8400640 1 -- 172.21.15.46:0/1292963458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa006dc70 msgr2=0x7f7fa00cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa8400640 1 --2- 172.21.15.46:0/1292963458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa006dc70 0x7f7fa00cec70 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0097c0 tx=0x7f7f8c01c470 comp rx=0 tx=0).stop 2024-04-25T08:45:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa8400640 1 -- 172.21.15.46:0/1292963458 shutdown_connections 2024-04-25T08:45:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa8400640 1 --2- 172.21.15.46:0/1292963458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa006dc70 0x7f7fa00cec70 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa8400640 1 -- 172.21.15.46:0/1292963458 >> 172.21.15.46:0/1292963458 conn(0x7f7fa006ac10 msgr2=0x7f7fa006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa8400640 1 -- 172.21.15.46:0/1292963458 shutdown_connections 2024-04-25T08:45:03.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa8400640 1 -- 172.21.15.46:0/1292963458 wait complete. 2024-04-25T08:45:03.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa8400640 1 Processor -- start 2024-04-25T08:45:03.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.722+0000 7f7fa8400640 1 -- start start 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa8400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa00cf1b0 0x7f7fa0152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa8400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa0145310 0x7f7fa0140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa8400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa00acc90 0x7f7fa00ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa8400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01696a0 con 0x7f7fa00acc90 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa8400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0169520 con 0x7f7fa0145310 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa8400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0169820 con 0x7f7fa00cf1b0 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa00acc90 0x7f7fa00ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa00acc90 0x7f7fa00ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56740/0 (socket says 172.21.15.46:56740) 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa6400640 1 -- 172.21.15.46:0/1101420720 learned_addr learned my addr 172.21.15.46:0/1101420720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa5a00640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa00cf1b0 0x7f7fa0152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa5000640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa0145310 0x7f7fa0140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa6400640 1 -- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa00cf1b0 msgr2=0x7f7fa0152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa6400640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa00cf1b0 0x7f7fa0152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa6400640 1 -- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa0145310 msgr2=0x7f7fa0140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa6400640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa0145310 0x7f7fa0140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa6400640 1 -- 172.21.15.46:0/1101420720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0159e90 con 0x7f7fa00acc90 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa5a00640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa00cf1b0 0x7f7fa0152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa5000640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa0145310 0x7f7fa0140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa6400640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa00acc90 0x7f7fa00ad120 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800c7a0 tx=0x7f7f9800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7f9d600640 1 -- 172.21.15.46:0/1101420720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98003b80 con 0x7f7fa00acc90 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7f9d600640 1 -- 172.21.15.46:0/1101420720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f98003d20 con 0x7f7fa00acc90 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa015a170 con 0x7f7fa00acc90 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7f9d600640 1 -- 172.21.15.46:0/1101420720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800aa90 con 0x7f7fa00acc90 2024-04-25T08:45:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa015a730 con 0x7f7fa00acc90 2024-04-25T08:45:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.726+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa0069010 con 0x7f7fa00acc90 2024-04-25T08:45:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.730+0000 7f7f9d600640 1 -- 172.21.15.46:0/1101420720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f9800ac30 con 0x7f7fa00acc90 2024-04-25T08:45:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.730+0000 7f7f9d600640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:03.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.730+0000 7f7f9d600640 1 -- 172.21.15.46:0/1101420720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7f98084e80 con 0x7f7fa00acc90 2024-04-25T08:45:03.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.730+0000 7f7fa5a00640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:03.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.730+0000 7f7fa5a00640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f94000f70 tx=0x7f7f94009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.734+0000 7f7f9d600640 1 -- 172.21.15.46:0/1101420720 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f98055f60 con 0x7f7fa00acc90 2024-04-25T08:45:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.894+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7fa006c250 con 0x7f7fa00acc90 2024-04-25T08:45:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7f9d600640 1 -- 172.21.15.46:0/1101420720 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7f98055860 con 0x7f7fa00acc90 2024-04-25T08:45:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 msgr2=0x7f7f80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:03.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7f94000f70 tx=0x7f7f94009040 comp rx=0 tx=0).stop 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa00acc90 msgr2=0x7f7fa00ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa00acc90 0x7f7fa00ad120 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800c7a0 tx=0x7f7f9800cc60 comp rx=0 tx=0).stop 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 shutdown_connections 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa00acc90 0x7f7fa00ad120 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa0145310 0x7f7fa0140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 --2- 172.21.15.46:0/1101420720 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa00cf1b0 0x7f7fa0152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 >> 172.21.15.46:0/1101420720 conn(0x7f7fa006ac10 msgr2=0x7f7fa0143b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 shutdown_connections 2024-04-25T08:45:03.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:03.898+0000 7f7fa8400640 1 -- 172.21.15.46:0/1101420720 wait complete. 2024-04-25T08:45:03.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T08:45:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bf600640 1 Processor -- start 2024-04-25T08:45:04.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bf600640 1 -- start start 2024-04-25T08:45:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bf600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa8b8159230 0x7fa8b815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bf600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8b80cdc50 0x7fa8b815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bf600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b80cc700 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bf600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b80cc9e0 con 0x7fa8b8159230 2024-04-25T08:45:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bf600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b80cccc0 con 0x7fa8b80cdc50 2024-04-25T08:45:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bcc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa8b8159230 0x7fa8b815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bcc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa8b8159230 0x7fa8b815bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56746/0 (socket says 172.21.15.46:56746) 2024-04-25T08:45:04.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bcc00640 1 -- 172.21.15.46:0/3753260213 learned_addr learned my addr 172.21.15.46:0/3753260213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bd600640 1 --2- 172.21.15.46:0/3753260213 >> v2:172.21.15.82:3300/0 conn(0x7fa8b80cdc50 0x7fa8b815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bcc00640 1 -- 172.21.15.46:0/3753260213 >> v2:172.21.15.82:3300/0 conn(0x7fa8b80cdc50 msgr2=0x7fa8b815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bcc00640 1 --2- 172.21.15.46:0/3753260213 >> v2:172.21.15.82:3300/0 conn(0x7fa8b80cdc50 0x7fa8b815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bcc00640 1 -- 172.21.15.46:0/3753260213 >> v1:172.21.15.82:6789/0 conn(0x7fa8b80cbeb0 legacy=0x7fa8b80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:04.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bcc00640 1 -- 172.21.15.46:0/3753260213 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b80ccfa0 con 0x7fa8b8159230 2024-04-25T08:45:04.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.062+0000 7fa8bd600640 1 --2- 172.21.15.46:0/3753260213 >> v2:172.21.15.82:3300/0 conn(0x7fa8b80cdc50 0x7fa8b815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8bcc00640 1 --2- 172.21.15.46:0/3753260213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b8159230 0x7fa8b815bdb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80b2340 tx=0x7fa8a801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2a4c18bd2ef33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8b7e00640 1 -- 172.21.15.46:0/3753260213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80ab640 con 0x7fa8b8159230 2024-04-25T08:45:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8b7e00640 1 -- 172.21.15.46:0/3753260213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a80b0070 con 0x7fa8b8159230 2024-04-25T08:45:04.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8b7e00640 1 -- 172.21.15.46:0/3753260213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80aba00 con 0x7fa8b8159230 2024-04-25T08:45:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8bf600640 1 -- 172.21.15.46:0/3753260213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b8159230 msgr2=0x7fa8b815bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8bf600640 1 --2- 172.21.15.46:0/3753260213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b8159230 0x7fa8b815bdb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa8b80b2340 tx=0x7fa8a801c420 comp rx=0 tx=0).stop 2024-04-25T08:45:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8bf600640 1 -- 172.21.15.46:0/3753260213 shutdown_connections 2024-04-25T08:45:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8bf600640 1 --2- 172.21.15.46:0/3753260213 >> v2:172.21.15.82:3300/0 conn(0x7fa8b80cdc50 0x7fa8b815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8bf600640 1 --2- 172.21.15.46:0/3753260213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b8159230 0x7fa8b815bdb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8bf600640 1 -- 172.21.15.46:0/3753260213 >> 172.21.15.46:0/3753260213 conn(0x7fa8b80b12a0 msgr2=0x7fa8b80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8bf600640 1 -- 172.21.15.46:0/3753260213 shutdown_connections 2024-04-25T08:45:04.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.066+0000 7fa8bf600640 1 -- 172.21.15.46:0/3753260213 wait complete. 2024-04-25T08:45:04.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 Processor -- start 2024-04-25T08:45:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 -- start start 2024-04-25T08:45:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b80cbeb0 0x7fa8b813f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b80cdc50 0x7fa8b8165a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8b8159230 0x7fa8b813f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b815e0d0 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b815df50 con 0x7fa8b80cdc50 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b815e250 con 0x7fa8b8159230 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b80cbeb0 0x7fa8b813f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b80cbeb0 0x7fa8b813f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56758/0 (socket says 172.21.15.46:56758) 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bcc00640 1 -- 172.21.15.46:0/3131890534 learned_addr learned my addr 172.21.15.46:0/3131890534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8afe00640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b80cdc50 0x7fa8b8165a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bd600640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8b8159230 0x7fa8b813f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bcc00640 1 -- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8b8159230 msgr2=0x7fa8b813f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bcc00640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8b8159230 0x7fa8b813f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bcc00640 1 -- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b80cdc50 msgr2=0x7fa8b8165a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bcc00640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b80cdc50 0x7fa8b8165a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bcc00640 1 -- 172.21.15.46:0/3131890534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b813f590 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bcc00640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b80cbeb0 0x7fa8b813f050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80ad040 tx=0x7fa8a801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8b5600640 1 -- 172.21.15.46:0/3131890534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80be5f0 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8b5600640 1 -- 172.21.15.46:0/3131890534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a80abe50 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b81649a0 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8b5600640 1 -- 172.21.15.46:0/3131890534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a80be9f0 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b8164f60 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.070+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b8069080 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.074+0000 7fa8b5600640 1 -- 172.21.15.46:0/3131890534 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8a80b43f0 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.074+0000 7fa8b5600640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa894066f90 0x7fa894069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.074+0000 7fa8b5600640 1 -- 172.21.15.46:0/3131890534 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa8a811f4c0 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.074+0000 7fa8afe00640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa894066f90 0x7fa894069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.074+0000 7fa8afe00640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa894066f90 0x7fa894069450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8141ea0 tx=0x7fa8a0007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.078+0000 7fa8b5600640 1 -- 172.21.15.46:0/3131890534 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8a80c4060 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.310+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fa8b80723d0 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.546+0000 7fa8b5600640 1 -- 172.21.15.46:0/3131890534 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v163) v1 ==== 149+0+0 (secure 0 0 0) 0x7fa8a80160d0 con 0x7fa8b80cbeb0 2024-04-25T08:45:04.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa894066f90 msgr2=0x7fa894069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa894066f90 0x7fa894069450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8141ea0 tx=0x7fa8a0007e40 comp rx=0 tx=0).stop 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b80cbeb0 msgr2=0x7fa8b813f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b80cbeb0 0x7fa8b813f050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80ad040 tx=0x7fa8a801ac60 comp rx=0 tx=0).stop 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 shutdown_connections 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa894066f90 0x7fa894069450 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa8b8159230 0x7fa8b813f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b80cdc50 0x7fa8b8165a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 --2- 172.21.15.46:0/3131890534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8b80cbeb0 0x7fa8b813f050 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 >> 172.21.15.46:0/3131890534 conn(0x7fa8b80b12a0 msgr2=0x7fa8b81408a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 shutdown_connections 2024-04-25T08:45:04.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.550+0000 7fa8bf600640 1 -- 172.21.15.46:0/3131890534 wait complete. 2024-04-25T08:45:04.564 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-25T08:45:04.564 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T08:45:04.564 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:45:04.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:45:04.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f112be00640 1 Processor -- start 2024-04-25T08:45:04.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f112be00640 1 -- start start 2024-04-25T08:45:04.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f112be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f112415ef20 0x7f11240cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f112be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1124099a00 0x7f1124099e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f112be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112415f300 con 0x7f11240cd6d0 2024-04-25T08:45:04.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f112be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11240cddb0 con 0x7f1124099a00 2024-04-25T08:45:04.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f112be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11240cc3d0 con 0x7f112415ef20 2024-04-25T08:45:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f1129e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1124099a00 0x7f1124099e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f1129e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1124099a00 0x7f1124099e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56770/0 (socket says 172.21.15.46:56770) 2024-04-25T08:45:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f1129e00640 1 -- 172.21.15.46:0/4028809774 learned_addr learned my addr 172.21.15.46:0/4028809774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f1129400640 1 --2- 172.21.15.46:0/4028809774 >> v2:172.21.15.82:3300/0 conn(0x7f112415ef20 0x7f11240cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f1129e00640 1 -- 172.21.15.46:0/4028809774 >> v2:172.21.15.82:3300/0 conn(0x7f112415ef20 msgr2=0x7f11240cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f1129e00640 1 --2- 172.21.15.46:0/4028809774 >> v2:172.21.15.82:3300/0 conn(0x7f112415ef20 0x7f11240cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f1129e00640 1 -- 172.21.15.46:0/4028809774 >> v1:172.21.15.82:6789/0 conn(0x7f11240cd6d0 legacy=0x7f112409a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f1129e00640 1 -- 172.21.15.46:0/4028809774 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11240cc6b0 con 0x7f1124099a00 2024-04-25T08:45:04.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.714+0000 7f1129400640 1 --2- 172.21.15.46:0/4028809774 >> v2:172.21.15.82:3300/0 conn(0x7f112415ef20 0x7f11240cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:04.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f1129e00640 1 --2- 172.21.15.46:0/4028809774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124099a00 0x7f1124099e90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f112409a5e0 tx=0x7f111c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7034d527785a8f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:04.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f1123e00640 1 -- 172.21.15.46:0/4028809774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c0a57a0 con 0x7f1124099a00 2024-04-25T08:45:04.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f1123e00640 1 -- 172.21.15.46:0/4028809774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c0a5940 con 0x7f1124099a00 2024-04-25T08:45:04.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f1123e00640 1 -- 172.21.15.46:0/4028809774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c0a5d10 con 0x7f1124099a00 2024-04-25T08:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f112be00640 1 -- 172.21.15.46:0/4028809774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124099a00 msgr2=0x7f1124099e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f112be00640 1 --2- 172.21.15.46:0/4028809774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124099a00 0x7f1124099e90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f112409a5e0 tx=0x7f111c01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f112be00640 1 -- 172.21.15.46:0/4028809774 shutdown_connections 2024-04-25T08:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f112be00640 1 --2- 172.21.15.46:0/4028809774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1124099a00 0x7f1124099e90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f112be00640 1 --2- 172.21.15.46:0/4028809774 >> v2:172.21.15.82:3300/0 conn(0x7f112415ef20 0x7f11240cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f112be00640 1 -- 172.21.15.46:0/4028809774 >> 172.21.15.46:0/4028809774 conn(0x7f11240b3120 msgr2=0x7f11240b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f112be00640 1 -- 172.21.15.46:0/4028809774 shutdown_connections 2024-04-25T08:45:04.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.718+0000 7f112be00640 1 -- 172.21.15.46:0/4028809774 wait complete. 2024-04-25T08:45:04.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 Processor -- start 2024-04-25T08:45:04.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 -- start start 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1124099a00 0x7f1124158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112415ef20 0x7f11240b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f112409b4c0 0x7f112409b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112409aea0 con 0x7f112415ef20 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112409ad20 con 0x7f112409b4c0 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1124168880 con 0x7f1124099a00 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1128a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112415ef20 0x7f11240b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1128a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112415ef20 0x7f11240b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56780/0 (socket says 172.21.15.46:56780) 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1128a00640 1 -- 172.21.15.46:0/2342878735 learned_addr learned my addr 172.21.15.46:0/2342878735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1129e00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f112409b4c0 0x7f112409b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1129400640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1124099a00 0x7f1124158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1128a00640 1 -- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1124099a00 msgr2=0x7f1124158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1128a00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1124099a00 0x7f1124158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1128a00640 1 -- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f112409b4c0 msgr2=0x7f112409b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1128a00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f112409b4c0 0x7f112409b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1128a00640 1 -- 172.21.15.46:0/2342878735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112415b4a0 con 0x7f112415ef20 2024-04-25T08:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1128a00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112415ef20 0x7f11240b2410 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f111000c370 tx=0x7f111000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1121600640 1 -- 172.21.15.46:0/2342878735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100052d0 con 0x7f112415ef20 2024-04-25T08:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1121600640 1 -- 172.21.15.46:0/2342878735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1110005470 con 0x7f112415ef20 2024-04-25T08:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f1121600640 1 -- 172.21.15.46:0/2342878735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110007260 con 0x7f112415ef20 2024-04-25T08:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1124159170 con 0x7f112415ef20 2024-04-25T08:45:04.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.722+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11240ca4d0 con 0x7f112415ef20 2024-04-25T08:45:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.726+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1124069080 con 0x7f112415ef20 2024-04-25T08:45:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.726+0000 7f1121600640 1 -- 172.21.15.46:0/2342878735 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1110007400 con 0x7f112415ef20 2024-04-25T08:45:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.726+0000 7f1121600640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1104066f90 0x7f1104069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.726+0000 7f1121600640 1 -- 172.21.15.46:0/2342878735 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1110084390 con 0x7f112415ef20 2024-04-25T08:45:04.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.726+0000 7f1129400640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1104066f90 0x7f1104069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:04.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.726+0000 7f1129400640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1104066f90 0x7f1104069450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1118009950 tx=0x7f1118008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:04.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.730+0000 7f1121600640 1 -- 172.21.15.46:0/2342878735 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1110054da0 con 0x7f112415ef20 2024-04-25T08:45:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.894+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11240d0700 con 0x7f112415ef20 2024-04-25T08:45:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.894+0000 7f1121600640 1 -- 172.21.15.46:0/2342878735 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13245 (secure 0 0 0) 0x7f1110058e40 con 0x7f112415ef20 2024-04-25T08:45:04.889 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:04.890 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":57,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:45:03.540857+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/3843568539":"2024-04-26T08:45:00.530258+0000","172.21.15.46:6832/3843568539":"2024-04-26T08:45:00.530258+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:45:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1104066f90 msgr2=0x7f1104069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1104066f90 0x7f1104069450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1118009950 tx=0x7f1118008040 comp rx=0 tx=0).stop 2024-04-25T08:45:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112415ef20 msgr2=0x7f11240b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112415ef20 0x7f11240b2410 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f111000c370 tx=0x7f111000c830 comp rx=0 tx=0).stop 2024-04-25T08:45:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 shutdown_connections 2024-04-25T08:45:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1104066f90 0x7f1104069450 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f112409b4c0 0x7f112409b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f112415ef20 0x7f11240b2410 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 --2- 172.21.15.46:0/2342878735 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1124099a00 0x7f1124158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 >> 172.21.15.46:0/2342878735 conn(0x7f11240b3120 msgr2=0x7f1124159360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 shutdown_connections 2024-04-25T08:45:04.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:04.898+0000 7f112be00640 1 -- 172.21.15.46:0/2342878735 wait complete. 2024-04-25T08:45:04.912 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:45:04.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:45:05.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.062+0000 7f3905c00640 1 Processor -- start 2024-04-25T08:45:05.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.062+0000 7f3905c00640 1 -- start start 2024-04-25T08:45:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f3905c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3900143990 0x7f39000cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f3905c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39001440d0 0x7f390006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f3905c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390015e440 con 0x7f390006ef80 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f3905c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390015ed30 con 0x7f39001440d0 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f3905c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39000b3730 con 0x7f3900143990 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38f6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f39001440d0 0x7f390006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38ff400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f390006ef80 0x7f39000cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48558/0 (socket says 172.21.15.46:48558) 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38ff400640 1 -- 172.21.15.46:0/821748168 learned_addr learned my addr 172.21.15.46:0/821748168 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38fea00640 1 --2- 172.21.15.46:0/821748168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3900143990 0x7f39000cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38f6000640 1 -- 172.21.15.46:0/821748168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3900143990 msgr2=0x7f39000cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38f6000640 1 --2- 172.21.15.46:0/821748168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3900143990 0x7f39000cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38f6000640 1 -- 172.21.15.46:0/821748168 >> v1:172.21.15.46:6789/0 conn(0x7f390006ef80 legacy=0x7f39000cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38f6000640 1 -- 172.21.15.46:0/821748168 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390015e6e0 con 0x7f39001440d0 2024-04-25T08:45:05.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38fe000640 1 -- 172.21.15.46:0/821748168 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757346301 0 0) 0x7f390015e440 con 0x7f390006ef80 2024-04-25T08:45:05.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38fea00640 1 --2- 172.21.15.46:0/821748168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3900143990 0x7f39000cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:05.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.066+0000 7f38f6000640 1 --2- 172.21.15.46:0/821748168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39001440d0 0x7f390006ea40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f38e80097c0 tx=0x7f38e801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec9e100548212b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:05.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f38fe000640 1 -- 172.21.15.46:0/821748168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80b0070 con 0x7f39001440d0 2024-04-25T08:45:05.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f38fe000640 1 -- 172.21.15.46:0/821748168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80ab600 con 0x7f39001440d0 2024-04-25T08:45:05.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f38fe000640 1 -- 172.21.15.46:0/821748168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80ab9d0 con 0x7f39001440d0 2024-04-25T08:45:05.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 -- 172.21.15.46:0/821748168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39001440d0 msgr2=0x7f390006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 --2- 172.21.15.46:0/821748168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39001440d0 0x7f390006ea40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f38e80097c0 tx=0x7f38e801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 -- 172.21.15.46:0/821748168 shutdown_connections 2024-04-25T08:45:05.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 --2- 172.21.15.46:0/821748168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39001440d0 0x7f390006ea40 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 --2- 172.21.15.46:0/821748168 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3900143990 0x7f39000cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 -- 172.21.15.46:0/821748168 >> 172.21.15.46:0/821748168 conn(0x7f390006ac10 msgr2=0x7f390006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:05.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 -- 172.21.15.46:0/821748168 shutdown_connections 2024-04-25T08:45:05.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 -- 172.21.15.46:0/821748168 wait complete. 2024-04-25T08:45:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 Processor -- start 2024-04-25T08:45:05.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 -- start start 2024-04-25T08:45:05.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390006ef80 0x7f39000b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3900143990 0x7f3900141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39001440d0 0x7f390013b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900143d70 con 0x7f3900143990 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390015f1a0 con 0x7f390006ef80 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f3905c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3900143ef0 con 0x7f39001440d0 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f38f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3900143990 0x7f3900141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f38f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3900143990 0x7f3900141f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56796/0 (socket says 172.21.15.46:56796) 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.070+0000 7f38f6000640 1 -- 172.21.15.46:0/1137935368 learned_addr learned my addr 172.21.15.46:0/1137935368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f6000640 1 -- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39001440d0 msgr2=0x7f390013b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f6000640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39001440d0 0x7f390013b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f6000640 1 -- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390006ef80 msgr2=0x7f39000b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f6000640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390006ef80 0x7f39000b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f6000640 1 -- 172.21.15.46:0/1137935368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39000af370 con 0x7f3900143990 2024-04-25T08:45:05.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f6000640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3900143990 0x7f3900141f80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f38e80097c0 tx=0x7f38e801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:05.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f7400640 1 -- 172.21.15.46:0/1137935368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80bc070 con 0x7f3900143990 2024-04-25T08:45:05.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f7400640 1 -- 172.21.15.46:0/1137935368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80b1040 con 0x7f3900143990 2024-04-25T08:45:05.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f7400640 1 -- 172.21.15.46:0/1137935368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80b0050 con 0x7f3900143990 2024-04-25T08:45:05.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39000af650 con 0x7f3900143990 2024-04-25T08:45:05.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39000afc10 con 0x7f3900143990 2024-04-25T08:45:05.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3900069010 con 0x7f3900143990 2024-04-25T08:45:05.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f7400640 1 -- 172.21.15.46:0/1137935368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38e80bf070 con 0x7f3900143990 2024-04-25T08:45:05.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f7400640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38dc066f60 0x7f38dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38f7400640 1 -- 172.21.15.46:0/1137935368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f38e811e930 con 0x7f3900143990 2024-04-25T08:45:05.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.074+0000 7f38fea00640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38dc066f60 0x7f38dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:05.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.078+0000 7f38fea00640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38dc066f60 0x7f38dc069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f38f00094c0 tx=0x7f38f0009430 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.078+0000 7f38f7400640 1 -- 172.21.15.46:0/1137935368 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38e8016090 con 0x7f3900143990 2024-04-25T08:45:05.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3900143150 con 0x7f3900143990 2024-04-25T08:45:05.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f38f7400640 1 -- 172.21.15.46:0/1137935368 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38e80efa10 con 0x7f3900143990 2024-04-25T08:45:05.236 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:45:05.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38dc066f60 msgr2=0x7f38dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38dc066f60 0x7f38dc069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f38f00094c0 tx=0x7f38f0009430 comp rx=0 tx=0).stop 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3900143990 msgr2=0x7f3900141f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3900143990 0x7f3900141f80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f38e80097c0 tx=0x7f38e801ace0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 shutdown_connections 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38dc066f60 0x7f38dc069420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f39001440d0 0x7f390013b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3900143990 0x7f3900141f80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 --2- 172.21.15.46:0/1137935368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390006ef80 0x7f39000b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.242+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 >> 172.21.15.46:0/1137935368 conn(0x7f390006ac10 msgr2=0x7f390015c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.246+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 shutdown_connections 2024-04-25T08:45:05.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.246+0000 7f3905c00640 1 -- 172.21.15.46:0/1137935368 wait complete. 2024-04-25T08:45:05.257 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T08:45:05.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.402+0000 7f9f74c00640 1 Processor -- start 2024-04-25T08:45:05.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f74c00640 1 -- start start 2024-04-25T08:45:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f74c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9f7015a750 0x7f9f7006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f74c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f700cc9e0 0x7f9f7006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f74c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7006d5e0 con 0x7f9f700aed10 2024-04-25T08:45:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f74c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7015ab30 con 0x7f9f700cc9e0 2024-04-25T08:45:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f74c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f700ce4f0 con 0x7f9f7015a750 2024-04-25T08:45:05.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f67e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9f700aed10 0x7f9f700b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48564/0 (socket says 172.21.15.46:48564) 2024-04-25T08:45:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f6e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f700cc9e0 0x7f9f7006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f67e00640 1 -- 172.21.15.46:0/3470013534 learned_addr learned my addr 172.21.15.46:0/3470013534 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:05.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f6de00640 1 --2- 172.21.15.46:0/3470013534 >> v2:172.21.15.82:3300/0 conn(0x7f9f7015a750 0x7f9f7006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f6d400640 1 -- 172.21.15.46:0/3470013534 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233846649 0 0) 0x7f9f7006d5e0 con 0x7f9f700aed10 2024-04-25T08:45:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f6d400640 1 -- 172.21.15.46:0/3470013534 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f54005740 con 0x7f9f700aed10 2024-04-25T08:45:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f6e800640 1 -- 172.21.15.46:0/3470013534 >> v2:172.21.15.82:3300/0 conn(0x7f9f7015a750 msgr2=0x7f9f7006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f6e800640 1 --2- 172.21.15.46:0/3470013534 >> v2:172.21.15.82:3300/0 conn(0x7f9f7015a750 0x7f9f7006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f6e800640 1 -- 172.21.15.46:0/3470013534 >> v1:172.21.15.46:6789/0 conn(0x7f9f700aed10 legacy=0x7f9f700b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.406+0000 7f9f6e800640 1 -- 172.21.15.46:0/3470013534 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f700ce720 con 0x7f9f700cc9e0 2024-04-25T08:45:05.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f6e800640 1 --2- 172.21.15.46:0/3470013534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 0x7f9f7006f150 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9f7004ffd0 tx=0x7f9f6801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3496c3c65c09d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f6d400640 1 -- 172.21.15.46:0/3470013534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680b2070 con 0x7f9f700cc9e0 2024-04-25T08:45:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f6d400640 1 -- 172.21.15.46:0/3470013534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f680a58e0 con 0x7f9f700cc9e0 2024-04-25T08:45:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f6d400640 1 -- 172.21.15.46:0/3470013534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f680a5cb0 con 0x7f9f700cc9e0 2024-04-25T08:45:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 -- 172.21.15.46:0/3470013534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 msgr2=0x7f9f7006f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 --2- 172.21.15.46:0/3470013534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 0x7f9f7006f150 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f9f7004ffd0 tx=0x7f9f6801c930 comp rx=0 tx=0).stop 2024-04-25T08:45:05.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 -- 172.21.15.46:0/3470013534 shutdown_connections 2024-04-25T08:45:05.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 --2- 172.21.15.46:0/3470013534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 0x7f9f7006f150 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 --2- 172.21.15.46:0/3470013534 >> v2:172.21.15.82:3300/0 conn(0x7f9f7015a750 0x7f9f7006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 -- 172.21.15.46:0/3470013534 >> 172.21.15.46:0/3470013534 conn(0x7f9f700b12a0 msgr2=0x7f9f700b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:05.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 -- 172.21.15.46:0/3470013534 shutdown_connections 2024-04-25T08:45:05.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 -- 172.21.15.46:0/3470013534 wait complete. 2024-04-25T08:45:05.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 Processor -- start 2024-04-25T08:45:05.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.410+0000 7f9f74c00640 1 -- start start 2024-04-25T08:45:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f74c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f700aed10 0x7f9f70140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f74c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 0x7f9f7015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f74c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f7015a750 0x7f9f70150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f74c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7006db00 con 0x7f9f700cc9e0 2024-04-25T08:45:05.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f74c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7006d270 con 0x7f9f700aed10 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f74c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7006dc80 con 0x7f9f7015a750 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 0x7f9f7015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 0x7f9f7015bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56820/0 (socket says 172.21.15.46:56820) 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f67e00640 1 -- 172.21.15.46:0/1719536066 learned_addr learned my addr 172.21.15.46:0/1719536066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f6de00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f700aed10 0x7f9f70140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f67e00640 1 -- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f7015a750 msgr2=0x7f9f70150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f67e00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f7015a750 0x7f9f70150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f67e00640 1 -- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f700aed10 msgr2=0x7f9f70140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f67e00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f700aed10 0x7f9f70140760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f67e00640 1 -- 172.21.15.46:0/1719536066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7015c7c0 con 0x7f9f700cc9e0 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f6de00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f700aed10 0x7f9f70140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:05.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f67e00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 0x7f9f7015bcc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800c420 tx=0x7f9f5800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f66000640 1 -- 172.21.15.46:0/1719536066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58017070 con 0x7f9f700cc9e0 2024-04-25T08:45:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f66000640 1 -- 172.21.15.46:0/1719536066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f58005d10 con 0x7f9f700cc9e0 2024-04-25T08:45:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f66000640 1 -- 172.21.15.46:0/1719536066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58007310 con 0x7f9f700cc9e0 2024-04-25T08:45:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7015caa0 con 0x7f9f700cc9e0 2024-04-25T08:45:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f70144400 con 0x7f9f700cc9e0 2024-04-25T08:45:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f70069080 con 0x7f9f700cc9e0 2024-04-25T08:45:05.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.414+0000 7f9f66000640 1 -- 172.21.15.46:0/1719536066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f5801a070 con 0x7f9f700cc9e0 2024-04-25T08:45:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.418+0000 7f9f66000640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f4c066f90 0x7f9f4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.418+0000 7f9f66000640 1 -- 172.21.15.46:0/1719536066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9f580843c0 con 0x7f9f700cc9e0 2024-04-25T08:45:05.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.418+0000 7f9f6de00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f4c066f90 0x7f9f4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:05.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.418+0000 7f9f6de00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f4c066f90 0x7f9f4c069450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009a40 tx=0x7f9f60008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:05.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.422+0000 7f9f66000640 1 -- 172.21.15.46:0/1719536066 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f580554d0 con 0x7f9f700cc9e0 2024-04-25T08:45:05.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:05.642+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f9f7006e630 con 0x7f9f700cc9e0 2024-04-25T08:45:06.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.554+0000 7f9f66000640 1 -- 172.21.15.46:0/1719536066 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v165) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9f58054dd0 con 0x7f9f700cc9e0 2024-04-25T08:45:06.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f4c066f90 msgr2=0x7f9f4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:06.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f4c066f90 0x7f9f4c069450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009a40 tx=0x7f9f60008040 comp rx=0 tx=0).stop 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 msgr2=0x7f9f7015bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 0x7f9f7015bcc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800c420 tx=0x7f9f5800c8e0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 shutdown_connections 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f4c066f90 0x7f9f4c069450 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f7015a750 0x7f9f70150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f700cc9e0 0x7f9f7015bcc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 --2- 172.21.15.46:0/1719536066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f700aed10 0x7f9f70140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 >> 172.21.15.46:0/1719536066 conn(0x7f9f700b12a0 msgr2=0x7f9f7014cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 shutdown_connections 2024-04-25T08:45:06.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.558+0000 7f9f74c00640 1 -- 172.21.15.46:0/1719536066 wait complete. 2024-04-25T08:45:06.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:45:06.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.730+0000 7f5b7a200640 1 Processor -- start 2024-04-25T08:45:06.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.730+0000 7f5b7a200640 1 -- start start 2024-04-25T08:45:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.730+0000 7f5b7a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5b7415dae0 0x7f5b74072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.730+0000 7f5b7a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7406e620 0x7f5b7406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.730+0000 7f5b7a200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b740b3850 con 0x7f5b7406ea60 2024-04-25T08:45:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.730+0000 7f5b7a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7406e190 con 0x7f5b7415dae0 2024-04-25T08:45:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.730+0000 7f5b7a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b740b3af0 con 0x7f5b7406e620 2024-04-25T08:45:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b73400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5b7415dae0 0x7f5b74072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b73400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5b7415dae0 0x7f5b74072600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56828/0 (socket says 172.21.15.46:56828) 2024-04-25T08:45:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b73400640 1 -- 172.21.15.46:0/637831078 learned_addr learned my addr 172.21.15.46:0/637831078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:06.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b6be00640 1 --2- 172.21.15.46:0/637831078 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7406e620 0x7f5b7406bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:06.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b73400640 1 -- 172.21.15.46:0/637831078 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7406e620 msgr2=0x7f5b7406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:06.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b73400640 1 --2- 172.21.15.46:0/637831078 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7406e620 0x7f5b7406bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b73400640 1 -- 172.21.15.46:0/637831078 >> v1:172.21.15.82:6789/0 conn(0x7f5b7406ea60 legacy=0x7f5b740acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:06.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b73400640 1 -- 172.21.15.46:0/637831078 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b740b3dd0 con 0x7f5b7415dae0 2024-04-25T08:45:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b73400640 1 --2- 172.21.15.46:0/637831078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7415dae0 0x7f5b74072600 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5b7406e310 tx=0x7f5b6401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7cee6cd6217efd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b72a00640 1 -- 172.21.15.46:0/637831078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640b1070 con 0x7f5b7415dae0 2024-04-25T08:45:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b72a00640 1 -- 172.21.15.46:0/637831078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b640ac600 con 0x7f5b7415dae0 2024-04-25T08:45:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.734+0000 7f5b72a00640 1 -- 172.21.15.46:0/637831078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640ac9d0 con 0x7f5b7415dae0 2024-04-25T08:45:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 -- 172.21.15.46:0/637831078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7415dae0 msgr2=0x7f5b74072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 --2- 172.21.15.46:0/637831078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7415dae0 0x7f5b74072600 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5b7406e310 tx=0x7f5b6401c4d0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 -- 172.21.15.46:0/637831078 shutdown_connections 2024-04-25T08:45:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 --2- 172.21.15.46:0/637831078 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7406e620 0x7f5b7406bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 --2- 172.21.15.46:0/637831078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7415dae0 0x7f5b74072600 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 -- 172.21.15.46:0/637831078 >> 172.21.15.46:0/637831078 conn(0x7f5b74071ae0 msgr2=0x7f5b74071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 -- 172.21.15.46:0/637831078 shutdown_connections 2024-04-25T08:45:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 -- 172.21.15.46:0/637831078 wait complete. 2024-04-25T08:45:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 Processor -- start 2024-04-25T08:45:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 -- start start 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7406e620 0x7f5b7414c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ea60 0x7f5b7414af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7415dae0 0x7f5b74151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7415dec0 con 0x7f5b7406e620 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7406c570 con 0x7f5b7406ea60 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b7a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7406f6d0 con 0x7f5b7415dae0 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7406e620 0x7f5b7414c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7406e620 0x7f5b7414c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56830/0 (socket says 172.21.15.46:56830) 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b73400640 1 -- 172.21.15.46:0/3001639848 learned_addr learned my addr 172.21.15.46:0/3001639848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b6be00640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ea60 0x7f5b7414af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b73e00640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7415dae0 0x7f5b74151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b73400640 1 -- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7415dae0 msgr2=0x7f5b74151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b73400640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7415dae0 0x7f5b74151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b73400640 1 -- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ea60 msgr2=0x7f5b7414af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b73400640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ea60 0x7f5b7414af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.738+0000 7f5b73400640 1 -- 172.21.15.46:0/3001639848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7414d4e0 con 0x7f5b7406e620 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b73400640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7406e620 0x7f5b7414c150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5b6401ab80 tx=0x7f5b6401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b6b400640 1 -- 172.21.15.46:0/3001639848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640b1040 con 0x7f5b7406e620 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b6b400640 1 -- 172.21.15.46:0/3001639848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b640b7420 con 0x7f5b7406e620 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b6b400640 1 -- 172.21.15.46:0/3001639848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640b77d0 con 0x7f5b7406e620 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7414c990 con 0x7f5b7406e620 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7414da60 con 0x7f5b7406e620 2024-04-25T08:45:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b74069010 con 0x7f5b7406e620 2024-04-25T08:45:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b6b400640 1 -- 172.21.15.46:0/3001639848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b640ac4c0 con 0x7f5b7406e620 2024-04-25T08:45:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b6b400640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5b50066f60 0x7f5b50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b6b400640 1 -- 172.21.15.46:0/3001639848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5b6411e9b0 con 0x7f5b7406e620 2024-04-25T08:45:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.742+0000 7f5b6be00640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5b50066f60 0x7f5b50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:06.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.746+0000 7f5b6be00640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5b50066f60 0x7f5b50069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b740ad3c0 tx=0x7f5b5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.746+0000 7f5b6b400640 1 -- 172.21.15.46:0/3001639848 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b640160f0 con 0x7f5b7406e620 2024-04-25T08:45:06.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.910+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5b74153850 con 0x7f5b7406e620 2024-04-25T08:45:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b6b400640 1 -- 172.21.15.46:0/3001639848 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5b640efb30 con 0x7f5b7406e620 2024-04-25T08:45:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5b50066f60 msgr2=0x7f5b50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5b50066f60 0x7f5b50069420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b740ad3c0 tx=0x7f5b5c008040 comp rx=0 tx=0).stop 2024-04-25T08:45:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7406e620 msgr2=0x7f5b7414c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7406e620 0x7f5b7414c150 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5b6401ab80 tx=0x7f5b6401ad10 comp rx=0 tx=0).stop 2024-04-25T08:45:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 shutdown_connections 2024-04-25T08:45:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5b50066f60 0x7f5b50069420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5b7415dae0 0x7f5b74151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ea60 0x7f5b7414af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 --2- 172.21.15.46:0/3001639848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b7406e620 0x7f5b7414c150 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 >> 172.21.15.46:0/3001639848 conn(0x7f5b74071ae0 msgr2=0x7f5b7406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 shutdown_connections 2024-04-25T08:45:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:06.914+0000 7f5b7a200640 1 -- 172.21.15.46:0/3001639848 wait complete. 2024-04-25T08:45:06.928 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:45:06.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:45:07.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:45:07.096 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:45:07.096 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:45:07.096 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:45:07.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:45:07.105 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:45:07.105 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:45:07.114 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:45:07.115 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:45:07.115 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:45:07.157 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:45:07.157 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:45:07.158 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:45:07.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:45:07.167 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:45:07.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:45:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.346+0000 7f5993c00640 1 Processor -- start 2024-04-25T08:45:07.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.346+0000 7f5993c00640 1 -- start start 2024-04-25T08:45:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5993c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c15a750 0x7f598c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5993c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f598c0cc9e0 0x7f598c06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5993c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598c06d5e0 con 0x7f598c0aed10 2024-04-25T08:45:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5993c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c15ab30 con 0x7f598c0cc9e0 2024-04-25T08:45:07.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5993c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c0ce4f0 con 0x7f598c15a750 2024-04-25T08:45:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f598c0cc9e0 0x7f598c06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c15a750 0x7f598c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f598c0cc9e0 0x7f598c06f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:58162/0 (socket says 172.21.15.46:58162) 2024-04-25T08:45:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c15a750 0x7f598c06ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35016/0 (socket says 172.21.15.46:35016) 2024-04-25T08:45:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5983e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f598c0aed10 0x7f598c0b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:54644/0 (socket says 172.21.15.46:54644) 2024-04-25T08:45:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991200640 1 -- 172.21.15.46:0/1715368836 learned_addr learned my addr 172.21.15.46:0/1715368836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991200640 1 -- 172.21.15.46:0/1715368836 >> v2:172.21.15.82:3300/0 conn(0x7f598c0cc9e0 msgr2=0x7f598c06f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:07.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991200640 1 --2- 172.21.15.46:0/1715368836 >> v2:172.21.15.82:3300/0 conn(0x7f598c0cc9e0 0x7f598c06f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991200640 1 -- 172.21.15.46:0/1715368836 >> v1:172.21.15.82:6789/0 conn(0x7f598c0aed10 legacy=0x7f598c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991200640 1 -- 172.21.15.46:0/1715368836 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0ce720 con 0x7f598c15a750 2024-04-25T08:45:07.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.350+0000 7f5991c00640 1 --2- 172.21.15.46:0/1715368836 >> v2:172.21.15.82:3300/0 conn(0x7f598c0cc9e0 0x7f598c06f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f5991200640 1 --2- 172.21.15.46:0/1715368836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c15a750 0x7f598c06ec10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f597c009870 tx=0x7f597c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d6e0907b06fcf48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:07.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f598be00640 1 -- 172.21.15.46:0/1715368836 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c0ab650 con 0x7f598c15a750 2024-04-25T08:45:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f598be00640 1 -- 172.21.15.46:0/1715368836 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c0b0070 con 0x7f598c15a750 2024-04-25T08:45:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f598be00640 1 -- 172.21.15.46:0/1715368836 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c0aba10 con 0x7f598c15a750 2024-04-25T08:45:07.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f5993c00640 1 -- 172.21.15.46:0/1715368836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c15a750 msgr2=0x7f598c06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f5993c00640 1 --2- 172.21.15.46:0/1715368836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c15a750 0x7f598c06ec10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f597c009870 tx=0x7f597c01c560 comp rx=0 tx=0).stop 2024-04-25T08:45:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f5993c00640 1 -- 172.21.15.46:0/1715368836 shutdown_connections 2024-04-25T08:45:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f5993c00640 1 --2- 172.21.15.46:0/1715368836 >> v2:172.21.15.82:3300/0 conn(0x7f598c0cc9e0 0x7f598c06f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f5993c00640 1 --2- 172.21.15.46:0/1715368836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c15a750 0x7f598c06ec10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f5993c00640 1 -- 172.21.15.46:0/1715368836 >> 172.21.15.46:0/1715368836 conn(0x7f598c0b12a0 msgr2=0x7f598c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f5993c00640 1 -- 172.21.15.46:0/1715368836 shutdown_connections 2024-04-25T08:45:07.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.354+0000 7f5993c00640 1 -- 172.21.15.46:0/1715368836 wait complete. 2024-04-25T08:45:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 Processor -- start 2024-04-25T08:45:07.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 -- start start 2024-04-25T08:45:07.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c0aed10 0x7f598c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f598c0cc9e0 0x7f598c15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15a750 0x7f598c15d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c06db00 con 0x7f598c15a750 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c06d270 con 0x7f598c0cc9e0 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c06dc80 con 0x7f598c0aed10 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15a750 0x7f598c15d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15a750 0x7f598c15d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56836/0 (socket says 172.21.15.46:56836) 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991c00640 1 -- 172.21.15.46:0/3693375226 learned_addr learned my addr 172.21.15.46:0/3693375226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5983e00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f598c0cc9e0 0x7f598c15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991200640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c0aed10 0x7f598c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991c00640 1 -- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c0aed10 msgr2=0x7f598c153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991c00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c0aed10 0x7f598c153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991c00640 1 -- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f598c0cc9e0 msgr2=0x7f598c15cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991c00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f598c0cc9e0 0x7f598c15cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991c00640 1 -- 172.21.15.46:0/3693375226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0ad5a0 con 0x7f598c15a750 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5983e00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f598c0cc9e0 0x7f598c15cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:07.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5991c00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15a750 0x7f598c15d420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f598400e0f0 tx=0x7f598400e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5989600640 1 -- 172.21.15.46:0/3693375226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598400b2d0 con 0x7f598c15a750 2024-04-25T08:45:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5989600640 1 -- 172.21.15.46:0/3693375226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59840039a0 con 0x7f598c15a750 2024-04-25T08:45:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598c0ad880 con 0x7f598c15a750 2024-04-25T08:45:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5989600640 1 -- 172.21.15.46:0/3693375226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984003d50 con 0x7f598c15a750 2024-04-25T08:45:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.358+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c0ade40 con 0x7f598c15a750 2024-04-25T08:45:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.362+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c069080 con 0x7f598c15a750 2024-04-25T08:45:07.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.362+0000 7f5989600640 1 -- 172.21.15.46:0/3693375226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5984009540 con 0x7f598c15a750 2024-04-25T08:45:07.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.362+0000 7f5989600640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5968066f60 0x7f5968069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.362+0000 7f5989600640 1 -- 172.21.15.46:0/3693375226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5984085480 con 0x7f598c15a750 2024-04-25T08:45:07.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.362+0000 7f5991200640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5968066f60 0x7f5968069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.362+0000 7f5991200640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5968066f60 0x7f5968069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f597c0af000 tx=0x7f597c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:07.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.366+0000 7f5989600640 1 -- 172.21.15.46:0/3693375226 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5984055f10 con 0x7f598c15a750 2024-04-25T08:45:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.562+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f598c06e360 con 0x7f598c15a750 2024-04-25T08:45:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.562+0000 7f5989600640 1 -- 172.21.15.46:0/3693375226 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f5984059fb0 con 0x7f598c15a750 2024-04-25T08:45:07.556 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:07.556 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5273,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/850914154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":850914154},{"type":"v1","addr":"172.21.15.46:6835","nonce":850914154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/1115060038","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1115060038},{"type":"v1","addr":"172.21.15.82:6836","nonce":1115060038}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5620,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1383788710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1383788710},{"type":"v1","addr":"172.21.15.82:6837","nonce":1383788710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6120,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/475603116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":475603116},{"type":"v1","addr":"172.21.15.46:6834","nonce":475603116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2024-04-25T08:45:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-04-25T08:45:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5968066f60 msgr2=0x7f5968069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5968066f60 0x7f5968069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f597c0af000 tx=0x7f597c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:45:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15a750 msgr2=0x7f598c15d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15a750 0x7f598c15d420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f598400e0f0 tx=0x7f598400e5b0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 shutdown_connections 2024-04-25T08:45:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5968066f60 0x7f5968069420 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15a750 0x7f598c15d420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f598c0cc9e0 0x7f598c15cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 --2- 172.21.15.46:0/3693375226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f598c0aed10 0x7f598c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 >> 172.21.15.46:0/3693375226 conn(0x7f598c0b12a0 msgr2=0x7f598c071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 shutdown_connections 2024-04-25T08:45:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.566+0000 7f5993c00640 1 -- 172.21.15.46:0/3693375226 wait complete. 2024-04-25T08:45:07.579 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:45:07.579 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:45:07.579 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:45:07.579 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:45:07.579 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:45:07.580 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:45:07.580 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:45:07.580 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:45:07.580 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:45:07.580 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:45:07.580 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:45:07.580 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:45:07.581 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:45:07.581 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:45:07.581 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:45:07.581 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:45:07.591 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:45:07.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:45:07.592 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:45:07.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:45:07.594 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:45:07.594 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:45:07.596 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:45:07.596 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:45:07.639 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:45:07.639 DEBUG:teuthology.parallel:result is None 2024-04-25T08:45:07.640 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:45:07.640 DEBUG:teuthology.parallel:result is None 2024-04-25T08:45:07.641 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:45:07.641 DEBUG:teuthology.parallel:result is None 2024-04-25T08:45:07.642 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:45:07.642 DEBUG:teuthology.parallel:result is None 2024-04-25T08:45:07.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:45:07.727 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:45:07.728 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-04-25T08:45:07.728 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-04-25T08:45:07.730 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:45:07.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.794+0000 7fd81f600640 1 Processor -- start 2024-04-25T08:45:07.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.794+0000 7fd81f600640 1 -- start start 2024-04-25T08:45:07.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81f600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd818159230 0x7fd81815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81f600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 0x7fd8180cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81f600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8180cda80 0x7fd81815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81f600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8180cc700 con 0x7fd8180cda80 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81f600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8180cc9e0 con 0x7fd818159230 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81f600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180cccc0 con 0x7fd8180cbeb0 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8180cda80 0x7fd81815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81cc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd818159230 0x7fd81815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8180cda80 0x7fd81815f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56884/0 (socket says 172.21.15.46:56884) 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81d600640 1 -- 172.21.15.46:0/231257547 learned_addr learned my addr 172.21.15.46:0/231257547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd80fe00640 1 --2- 172.21.15.46:0/231257547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 0x7fd8180cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81d600640 1 -- 172.21.15.46:0/231257547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 msgr2=0x7fd8180cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81d600640 1 --2- 172.21.15.46:0/231257547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 0x7fd8180cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81d600640 1 -- 172.21.15.46:0/231257547 >> v2:172.21.15.82:3300/0 conn(0x7fd818159230 msgr2=0x7fd81815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81d600640 1 --2- 172.21.15.46:0/231257547 >> v2:172.21.15.82:3300/0 conn(0x7fd818159230 0x7fd81815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81d600640 1 -- 172.21.15.46:0/231257547 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8180ccfa0 con 0x7fd8180cda80 2024-04-25T08:45:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81cc00640 1 --2- 172.21.15.46:0/231257547 >> v2:172.21.15.82:3300/0 conn(0x7fd818159230 0x7fd81815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.798+0000 7fd81d600640 1 --2- 172.21.15.46:0/231257547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 0x7fd81815f3e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd818072a90 tx=0x7fd81001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f03f56a9dbc2a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd817e00640 1 -- 172.21.15.46:0/231257547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100ac020 con 0x7fd8180cda80 2024-04-25T08:45:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd817e00640 1 -- 172.21.15.46:0/231257547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8100a6440 con 0x7fd8180cda80 2024-04-25T08:45:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd817e00640 1 -- 172.21.15.46:0/231257547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100a6810 con 0x7fd8180cda80 2024-04-25T08:45:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 -- 172.21.15.46:0/231257547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 msgr2=0x7fd81815f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 --2- 172.21.15.46:0/231257547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 0x7fd81815f3e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd818072a90 tx=0x7fd81001c6a0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 -- 172.21.15.46:0/231257547 shutdown_connections 2024-04-25T08:45:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 --2- 172.21.15.46:0/231257547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 0x7fd81815f3e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 --2- 172.21.15.46:0/231257547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 0x7fd8180cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 --2- 172.21.15.46:0/231257547 >> v2:172.21.15.82:3300/0 conn(0x7fd818159230 0x7fd81815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 -- 172.21.15.46:0/231257547 >> 172.21.15.46:0/231257547 conn(0x7fd8180b12a0 msgr2=0x7fd8180b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 -- 172.21.15.46:0/231257547 shutdown_connections 2024-04-25T08:45:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 -- 172.21.15.46:0/231257547 wait complete. 2024-04-25T08:45:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 Processor -- start 2024-04-25T08:45:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 -- start start 2024-04-25T08:45:07.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 0x7fd818140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 0x7fd8180a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 0x7fd8180a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd80fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 0x7fd8180a8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56886/0 (socket says 172.21.15.46:56886) 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 0x7fd818140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 0x7fd818140ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35072/0 (socket says 172.21.15.46:35072) 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd818159230 0x7fd8180a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd80fe00640 1 -- 172.21.15.46:0/3243817306 learned_addr learned my addr 172.21.15.46:0/3243817306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.802+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81815d200 con 0x7fd8180cda80 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180cc4a0 con 0x7fd818159230 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81815d380 con 0x7fd8180cbeb0 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd81d600640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd818159230 0x7fd8180a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd80fe00640 1 -- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 msgr2=0x7fd818140ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd80fe00640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 0x7fd818140ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd80fe00640 1 -- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd818159230 msgr2=0x7fd8180a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd80fe00640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd818159230 0x7fd8180a8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd80fe00640 1 -- 172.21.15.46:0/3243817306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81815b0c0 con 0x7fd8180cda80 2024-04-25T08:45:07.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd80fe00640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 0x7fd8180a8600 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd80000e8b0 tx=0x7fd80000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:07.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd815600640 1 -- 172.21.15.46:0/3243817306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800017070 con 0x7fd8180cda80 2024-04-25T08:45:07.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd815600640 1 -- 172.21.15.46:0/3243817306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80000c940 con 0x7fd8180cda80 2024-04-25T08:45:07.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8181414b0 con 0x7fd8180cda80 2024-04-25T08:45:07.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd815600640 1 -- 172.21.15.46:0/3243817306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80000cd10 con 0x7fd8180cda80 2024-04-25T08:45:07.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd818140ff0 con 0x7fd8180cda80 2024-04-25T08:45:07.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd818069080 con 0x7fd8180cda80 2024-04-25T08:45:07.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.806+0000 7fd815600640 1 -- 172.21.15.46:0/3243817306 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd80001a070 con 0x7fd8180cda80 2024-04-25T08:45:07.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.810+0000 7fd815600640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd7f4066f60 0x7fd7f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.810+0000 7fd815600640 1 -- 172.21.15.46:0/3243817306 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd8000851d0 con 0x7fd8180cda80 2024-04-25T08:45:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.810+0000 7fd81cc00640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd7f4066f60 0x7fd7f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:07.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.810+0000 7fd81cc00640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd7f4066f60 0x7fd7f4069420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd808009580 tx=0x7fd808009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:07.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.814+0000 7fd815600640 1 -- 172.21.15.46:0/3243817306 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd800055cf0 con 0x7fd8180cda80 2024-04-25T08:45:07.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:07.986+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd8180723d0 con 0x7fd8180cda80 2024-04-25T08:45:08.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.570+0000 7fd815600640 1 -- 172.21.15.46:0/3243817306 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd800059d90 con 0x7fd8180cda80 2024-04-25T08:45:08.564 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-25T08:45:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd7f4066f60 msgr2=0x7fd7f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd7f4066f60 0x7fd7f4069420 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd808009580 tx=0x7fd808009190 comp rx=0 tx=0).stop 2024-04-25T08:45:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 msgr2=0x7fd8180a8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 0x7fd8180a8600 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd80000e8b0 tx=0x7fd80000ed70 comp rx=0 tx=0).stop 2024-04-25T08:45:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 shutdown_connections 2024-04-25T08:45:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd7f4066f60 0x7fd7f4069420 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd818159230 0x7fd8180a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8180cda80 0x7fd8180a8600 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 --2- 172.21.15.46:0/3243817306 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8180cbeb0 0x7fd818140ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 >> 172.21.15.46:0/3243817306 conn(0x7fd8180b12a0 msgr2=0x7fd8180b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 shutdown_connections 2024-04-25T08:45:08.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.574+0000 7fd81f600640 1 -- 172.21.15.46:0/3243817306 wait complete. 2024-04-25T08:45:08.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:45:08.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.742+0000 7fed71c00640 1 Processor -- start 2024-04-25T08:45:08.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.742+0000 7fed71c00640 1 -- start start 2024-04-25T08:45:08.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed71c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c15b9a0 0x7fed6c0af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed71c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed6c0ad4c0 0x7fed6c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed71c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed6c15fed0 con 0x7fed6c06f570 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed71c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c1601b0 con 0x7fed6c0ad4c0 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed71c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c15c0f0 con 0x7fed6c15b9a0 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6a000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fed6c06f570 0x7fed6c0acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50788/0 (socket says 172.21.15.46:50788) 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6a000640 1 -- 172.21.15.46:0/2315112011 learned_addr learned my addr 172.21.15.46:0/2315112011 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6b400640 1 --2- 172.21.15.46:0/2315112011 >> v2:172.21.15.82:3300/0 conn(0x7fed6c0ad4c0 0x7fed6c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6aa00640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c15b9a0 0x7fed6c0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452929546 0 0) 0x7fed6c15fed0 con 0x7fed6c06f570 2024-04-25T08:45:08.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed50003530 con 0x7fed6c06f570 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3475454403 0 0) 0x7fed50003530 con 0x7fed6c06f570 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c15b9a0 msgr2=0x7fed6c0af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c15b9a0 0x7fed6c0af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 >> v2:172.21.15.82:3300/0 conn(0x7fed6c0ad4c0 msgr2=0x7fed6c15c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 --2- 172.21.15.46:0/2315112011 >> v2:172.21.15.82:3300/0 conn(0x7fed6c0ad4c0 0x7fed6c15c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c15c3d0 con 0x7fed6c06f570 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fed58001680 con 0x7fed6c06f570 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 >> v1:172.21.15.46:6789/0 conn(0x7fed6c06f570 legacy=0x7fed6c0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed500039c0 0x7fed50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c15b9a0 0x7fed50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:08.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed50004d50 0x7fed500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c1601b0 con 0x7fed50004d50 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c15c0f0 con 0x7fed500039c0 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5000b9d0 con 0x7fed6c15b9a0 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6b400640 1 --2- 172.21.15.46:0/2315112011 >> v2:172.21.15.82:3300/0 conn(0x7fed6c0ad4c0 0x7fed6c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6a000640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c15b9a0 0x7fed50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6b400640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed50004d50 0x7fed500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6aa00640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed500039c0 0x7fed50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6b400640 1 -- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c15b9a0 msgr2=0x7fed50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6b400640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c15b9a0 0x7fed50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6b400640 1 -- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed500039c0 msgr2=0x7fed50003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6b400640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed500039c0 0x7fed50003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.746+0000 7fed6b400640 1 -- 172.21.15.46:0/2315112011 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50003530 con 0x7fed50004d50 2024-04-25T08:45:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed6b400640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed50004d50 0x7fed500051d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0afa80 tx=0x7fed6401ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a383d803bdf809 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed640b2070 con 0x7fed50004d50 2024-04-25T08:45:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed640a5e30 con 0x7fed50004d50 2024-04-25T08:45:08.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed69600640 1 -- 172.21.15.46:0/2315112011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64009de0 con 0x7fed50004d50 2024-04-25T08:45:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed71c00640 1 -- 172.21.15.46:0/2315112011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed50004d50 msgr2=0x7fed500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed71c00640 1 --2- 172.21.15.46:0/2315112011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed50004d50 0x7fed500051d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0afa80 tx=0x7fed6401ce20 comp rx=0 tx=0).stop 2024-04-25T08:45:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed6b400640 1 -- 172.21.15.46:0/2315112011 reap_dead start 2024-04-25T08:45:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed71c00640 1 -- 172.21.15.46:0/2315112011 shutdown_connections 2024-04-25T08:45:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed71c00640 1 -- 172.21.15.46:0/2315112011 >> 172.21.15.46:0/2315112011 conn(0x7fed6c0cfd90 msgr2=0x7fed6c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed71c00640 1 -- 172.21.15.46:0/2315112011 shutdown_connections 2024-04-25T08:45:08.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed71c00640 1 -- 172.21.15.46:0/2315112011 wait complete. 2024-04-25T08:45:08.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.750+0000 7fed71c00640 1 Processor -- start 2024-04-25T08:45:08.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 -- start start 2024-04-25T08:45:08.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed50004d50 0x7fed6c1f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c150650 0x7fed6c0ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c15f6b0 0x7fed6c155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0adaf0 con 0x7fed6c15f6b0 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0ad970 con 0x7fed50004d50 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0af030 con 0x7fed6c150650 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c15f6b0 0x7fed6c155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c15f6b0 0x7fed6c155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33444/0 (socket says 172.21.15.46:33444) 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6b400640 1 -- 172.21.15.46:0/3125908071 learned_addr learned my addr 172.21.15.46:0/3125908071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6a000640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c150650 0x7fed6c0ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6aa00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed50004d50 0x7fed6c1f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6b400640 1 -- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c150650 msgr2=0x7fed6c0ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:08.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6b400640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c150650 0x7fed6c0ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6b400640 1 -- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed50004d50 msgr2=0x7fed6c1f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6b400640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed50004d50 0x7fed6c1f58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6b400640 1 -- 172.21.15.46:0/3125908071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c15fb30 con 0x7fed6c15f6b0 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6aa00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed50004d50 0x7fed6c1f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed6b400640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c15f6b0 0x7fed6c155bf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed64098950 tx=0x7fed64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed62a00640 1 -- 172.21.15.46:0/3125908071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed640b2040 con 0x7fed6c15f6b0 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed62a00640 1 -- 172.21.15.46:0/3125908071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed640a5c30 con 0x7fed6c15f6b0 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed62a00640 1 -- 172.21.15.46:0/3125908071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64009de0 con 0x7fed6c15f6b0 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c0cefc0 con 0x7fed6c15f6b0 2024-04-25T08:45:08.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c1f4de0 con 0x7fed6c15f6b0 2024-04-25T08:45:08.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.754+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6c069080 con 0x7fed6c15f6b0 2024-04-25T08:45:08.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.758+0000 7fed62a00640 1 -- 172.21.15.46:0/3125908071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed640be020 con 0x7fed6c15f6b0 2024-04-25T08:45:08.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.758+0000 7fed62a00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed4c066f90 0x7fed4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:08.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.758+0000 7fed62a00640 1 -- 172.21.15.46:0/3125908071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fed6411ee20 con 0x7fed6c15f6b0 2024-04-25T08:45:08.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.758+0000 7fed6aa00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed4c066f90 0x7fed4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:08.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.758+0000 7fed6aa00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed4c066f90 0x7fed4c069450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009a60 tx=0x7fed5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:08.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.762+0000 7fed62a00640 1 -- 172.21.15.46:0/3125908071 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed64016b30 con 0x7fed6c15f6b0 2024-04-25T08:45:08.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.978+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fed6c0cc600 con 0x7fed6c15f6b0 2024-04-25T08:45:08.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.978+0000 7fed62a00640 1 -- 172.21.15.46:0/3125908071 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fed640ef990 con 0x7fed6c15f6b0 2024-04-25T08:45:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:08.973 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:45:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-25T08:45:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-25T08:45:08.973 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmq+gDGRAAtB5ObM/zelMmCZ4cJPghdQ==", 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmyIaAJRAA9zodtgftP7faNth4UPyCTA==", 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:45:08.974 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmFahWIBAAORHmzDLXPjuANF+/W2wKpQ==", 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-25T08:45:08.975 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmRhrPLBAATXOHl1jGJYmIlnYk/JPcOw==", 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA0FypmaHiqEhAAYzasCl8t+pls5IHe9tCP4g==", 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:45:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA3FypmOGTuGhAA37F+qRI/+pj5dkLc3TBIUQ==", 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-25T08:45:08.977 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA6Fypm/YbNJRAAP1QyYDLY/xFvRz63R+H/Pw==", 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA9FypmiT8ALhAAX/2CJJeJ+2SGNhwah8NTBQ==", 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.978 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBDFypmyG88JBAAq1065gzLIIR/6fYaw0bBTg==", 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.979 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBGFypmNir3KxAAszCXJJ1KFtCusDZhBHAZSQ==", 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBJFypm4r7LNxAA42A07ntI6OOgblk+4i1Yiw==", 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.982 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBNFypmG1fOBhAAOKRgw04jIo8H3u2hWIZe2g==", 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.983 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q==", 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmxPyHOhAADBuigZlq3GHURsJkB0nnyA==", 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:45:08.984 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypm2PP0NhAAixV/kkY7eFPXUW/nJDOCOA==", 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAzFypmaVDtBBAAKnWk9CxZzg2r6T/IycuFEg==", 2024-04-25T08:45:08.985 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAxFypm2W2MOBAAN7TDQiZXFV6+pWMTar95sg==", 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.986 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmzhptFBAAlFQZQn1GSseEguy5zfw2CA==", 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmAY6CEBAA2qHNALNTRKmCJRZatDqAvw==", 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:45:08.987 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:45:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:45:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:08.988 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-25T08:45:08.988 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed4c066f90 msgr2=0x7fed4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed4c066f90 0x7fed4c069450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009a60 tx=0x7fed5c008040 comp rx=0 tx=0).stop 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c15f6b0 msgr2=0x7fed6c155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c15f6b0 0x7fed6c155bf0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fed64098950 tx=0x7fed64009750 comp rx=0 tx=0).stop 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 shutdown_connections 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed4c066f90 0x7fed4c069450 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed6c15f6b0 0x7fed6c155bf0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed6c150650 0x7fed6c0ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 --2- 172.21.15.46:0/3125908071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed50004d50 0x7fed6c1f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 >> 172.21.15.46:0/3125908071 conn(0x7fed6c0cfd90 msgr2=0x7fed6c071ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 shutdown_connections 2024-04-25T08:45:08.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:08.982+0000 7fed71c00640 1 -- 172.21.15.46:0/3125908071 wait complete. 2024-04-25T08:45:08.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:45:09.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.142+0000 7f0780e00640 1 Processor -- start 2024-04-25T08:45:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.142+0000 7f0780e00640 1 -- start start 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f0780e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c0af3f0 0x7f077c0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f0780e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f077c06f630 0x7f077c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f0780e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c15e180 con 0x7f077c06db40 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f0780e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c0b3ca0 con 0x7f077c06f630 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f0780e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c15e460 con 0x7f077c0af3f0 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f077a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f077c06f630 0x7f077c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f077a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f077c06f630 0x7f077c0b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33456/0 (socket says 172.21.15.46:33456) 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f077a800640 1 -- 172.21.15.46:0/2583639601 learned_addr learned my addr 172.21.15.46:0/2583639601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f0771400640 1 --1- 172.21.15.46:0/2583639601 >> v1:172.21.15.82:6789/0 conn(0x7f077c06db40 0x7f077c06e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:32772/0 (socket says 172.21.15.46:32772) 2024-04-25T08:45:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f0779e00640 1 --2- 172.21.15.46:0/2583639601 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c0af3f0 0x7f077c0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f077a800640 1 -- 172.21.15.46:0/2583639601 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c0af3f0 msgr2=0x7f077c0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f077a800640 1 --2- 172.21.15.46:0/2583639601 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c0af3f0 0x7f077c0b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f077a800640 1 -- 172.21.15.46:0/2583639601 >> v1:172.21.15.82:6789/0 conn(0x7f077c06db40 legacy=0x7f077c06e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f077a800640 1 -- 172.21.15.46:0/2583639601 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c15e740 con 0x7f077c06f630 2024-04-25T08:45:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.146+0000 7f077a800640 1 --2- 172.21.15.46:0/2583639601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 0x7f077c0b3380 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0774009e50 tx=0x7f077401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1531c55a988a3e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0779400640 1 -- 172.21.15.46:0/2583639601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740a5610 con 0x7f077c06f630 2024-04-25T08:45:09.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0779400640 1 -- 172.21.15.46:0/2583639601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07740a57b0 con 0x7f077c06f630 2024-04-25T08:45:09.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0779400640 1 -- 172.21.15.46:0/2583639601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740a5b80 con 0x7f077c06f630 2024-04-25T08:45:09.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 -- 172.21.15.46:0/2583639601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 msgr2=0x7f077c0b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 --2- 172.21.15.46:0/2583639601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 0x7f077c0b3380 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0774009e50 tx=0x7f077401ca30 comp rx=0 tx=0).stop 2024-04-25T08:45:09.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 -- 172.21.15.46:0/2583639601 shutdown_connections 2024-04-25T08:45:09.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 --2- 172.21.15.46:0/2583639601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 0x7f077c0b3380 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 --2- 172.21.15.46:0/2583639601 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c0af3f0 0x7f077c0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 -- 172.21.15.46:0/2583639601 >> 172.21.15.46:0/2583639601 conn(0x7f077c0cfd90 msgr2=0x7f077c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:09.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 -- 172.21.15.46:0/2583639601 shutdown_connections 2024-04-25T08:45:09.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 -- 172.21.15.46:0/2583639601 wait complete. 2024-04-25T08:45:09.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 Processor -- start 2024-04-25T08:45:09.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 -- start start 2024-04-25T08:45:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c06db40 0x7f077c13f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 0x7f077c15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0af3f0 0x7f077c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c06bda0 con 0x7f077c06f630 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c06bc20 con 0x7f077c0af3f0 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0780e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c16a6f0 con 0x7f077c06db40 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0771400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 0x7f077c15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0771400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 0x7f077c15c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33472/0 (socket says 172.21.15.46:33472) 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0771400640 1 -- 172.21.15.46:0/1799147953 learned_addr learned my addr 172.21.15.46:0/1799147953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f077a800640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0af3f0 0x7f077c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0779e00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c06db40 0x7f077c13f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0771400640 1 -- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c06db40 msgr2=0x7f077c13f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0771400640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c06db40 0x7f077c13f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0771400640 1 -- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0af3f0 msgr2=0x7f077c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0771400640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0af3f0 0x7f077c153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.150+0000 7f0771400640 1 -- 172.21.15.46:0/1799147953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c15ced0 con 0x7f077c06f630 2024-04-25T08:45:09.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0771400640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 0x7f077c15c4a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f076400c420 tx=0x7f076400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0772a00640 1 -- 172.21.15.46:0/1799147953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07640074e0 con 0x7f077c06f630 2024-04-25T08:45:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0772a00640 1 -- 172.21.15.46:0/1799147953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0764007680 con 0x7f077c06f630 2024-04-25T08:45:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0772a00640 1 -- 172.21.15.46:0/1799147953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764004a30 con 0x7f077c06f630 2024-04-25T08:45:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c0afc10 con 0x7f077c06f630 2024-04-25T08:45:09.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c0b01d0 con 0x7f077c06f630 2024-04-25T08:45:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c069080 con 0x7f077c06f630 2024-04-25T08:45:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0772a00640 1 -- 172.21.15.46:0/1799147953 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f076401a020 con 0x7f077c06f630 2024-04-25T08:45:09.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0772a00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0758066f90 0x7f0758069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0772a00640 1 -- 172.21.15.46:0/1799147953 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f07640845c0 con 0x7f077c06f630 2024-04-25T08:45:09.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.154+0000 7f0779e00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0758066f90 0x7f0758069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.158+0000 7f0779e00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0758066f90 0x7f0758069450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f076c009ac0 tx=0x7f076c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.158+0000 7f0772a00640 1 -- 172.21.15.46:0/1799147953 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07640550a0 con 0x7f077c06f630 2024-04-25T08:45:09.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f077c0b03a0 con 0x7f077c06f630 2024-04-25T08:45:09.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0772a00640 1 -- 172.21.15.46:0/1799147953 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13116 (secure 0 0 0) 0x7f0764059140 con 0x7f077c06f630 2024-04-25T08:45:09.315 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:09.315 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":61,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:45:08.585676+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:45:09.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0758066f90 msgr2=0x7f0758069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0758066f90 0x7f0758069450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f076c009ac0 tx=0x7f076c009190 comp rx=0 tx=0).stop 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 msgr2=0x7f077c15c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 0x7f077c15c4a0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f076400c420 tx=0x7f076400c8e0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 shutdown_connections 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0758066f90 0x7f0758069450 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f077c0af3f0 0x7f077c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f077c06f630 0x7f077c15c4a0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 --2- 172.21.15.46:0/1799147953 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f077c06db40 0x7f077c13f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.322+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 >> 172.21.15.46:0/1799147953 conn(0x7f077c0cfd90 msgr2=0x7f077c0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.326+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 shutdown_connections 2024-04-25T08:45:09.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.326+0000 7f0780e00640 1 -- 172.21.15.46:0/1799147953 wait complete. 2024-04-25T08:45:09.337 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:45:09.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:45:09.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.486+0000 7fe149800640 1 Processor -- start 2024-04-25T08:45:09.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe149800640 1 -- start start 2024-04-25T08:45:09.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe149800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe14406ea60 0x7fe1440acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe149800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1440b3850 con 0x7fe14415dae0 2024-04-25T08:45:09.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe149800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1440b3af0 con 0x7fe14406e620 2024-04-25T08:45:09.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe149800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14406e190 con 0x7fe14406ea60 2024-04-25T08:45:09.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe143400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe14406ea60 0x7fe1440acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe142a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe14415dae0 0x7fe144072600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50802/0 (socket says 172.21.15.46:50802) 2024-04-25T08:45:09.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe142a00640 1 -- 172.21.15.46:0/3729340821 learned_addr learned my addr 172.21.15.46:0/3729340821 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:09.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe143400640 1 -- 172.21.15.46:0/3729340821 >> v1:172.21.15.82:6789/0 conn(0x7fe14406e620 legacy=0x7fe14406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:09.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe143400640 1 -- 172.21.15.46:0/3729340821 >> v1:172.21.15.46:6789/0 conn(0x7fe14415dae0 legacy=0x7fe144072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe143400640 1 -- 172.21.15.46:0/3729340821 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440b3dd0 con 0x7fe14406ea60 2024-04-25T08:45:09.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.490+0000 7fe142000640 1 -- 172.21.15.46:0/3729340821 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698771757 0 0) 0x7fe1440b3850 con 0x7fe14415dae0 2024-04-25T08:45:09.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe143400640 1 --2- 172.21.15.46:0/3729340821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14406ea60 0x7fe1440acdb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009d90 tx=0x7fe13c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fc1c28694c578d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:09.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe142000640 1 -- 172.21.15.46:0/3729340821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0b2070 con 0x7fe14406ea60 2024-04-25T08:45:09.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe142000640 1 -- 172.21.15.46:0/3729340821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c0a59c0 con 0x7fe14406ea60 2024-04-25T08:45:09.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe142000640 1 -- 172.21.15.46:0/3729340821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0a5d90 con 0x7fe14406ea60 2024-04-25T08:45:09.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe149800640 1 -- 172.21.15.46:0/3729340821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14406ea60 msgr2=0x7fe1440acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe149800640 1 --2- 172.21.15.46:0/3729340821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14406ea60 0x7fe1440acdb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009d90 tx=0x7fe13c01c900 comp rx=0 tx=0).stop 2024-04-25T08:45:09.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe149800640 1 -- 172.21.15.46:0/3729340821 shutdown_connections 2024-04-25T08:45:09.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe149800640 1 --2- 172.21.15.46:0/3729340821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14406ea60 0x7fe1440acdb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe149800640 1 -- 172.21.15.46:0/3729340821 >> 172.21.15.46:0/3729340821 conn(0x7fe144071ae0 msgr2=0x7fe144071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:09.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe149800640 1 -- 172.21.15.46:0/3729340821 shutdown_connections 2024-04-25T08:45:09.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe149800640 1 -- 172.21.15.46:0/3729340821 wait complete. 2024-04-25T08:45:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe149800640 1 Processor -- start 2024-04-25T08:45:09.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.494+0000 7fe149800640 1 -- start start 2024-04-25T08:45:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe149800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe14406e620 0x7fe144160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe149800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe14406ea60 0x7fe1441f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe149800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14415dae0 0x7fe14415e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe149800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14406f850 con 0x7fe14415dae0 2024-04-25T08:45:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe149800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14406f6d0 con 0x7fe14406e620 2024-04-25T08:45:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe149800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14406c330 con 0x7fe14406ea60 2024-04-25T08:45:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe143400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14415dae0 0x7fe14415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe143400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14415dae0 0x7fe14415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33486/0 (socket says 172.21.15.46:33486) 2024-04-25T08:45:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe143400640 1 -- 172.21.15.46:0/2251250559 learned_addr learned my addr 172.21.15.46:0/2251250559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe13be00640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe14406ea60 0x7fe1441f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe142a00640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe14406e620 0x7fe144160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe143400640 1 -- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe14406ea60 msgr2=0x7fe1441f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe143400640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe14406ea60 0x7fe1441f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe143400640 1 -- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe14406e620 msgr2=0x7fe144160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe143400640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe14406e620 0x7fe144160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe143400640 1 -- 172.21.15.46:0/2251250559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1441f64c0 con 0x7fe14415dae0 2024-04-25T08:45:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe142a00640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe14406e620 0x7fe144160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe143400640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14415dae0 0x7fe14415e720 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe13c01a530 tx=0x7fe13c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:09.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe13aa00640 1 -- 172.21.15.46:0/2251250559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0b2050 con 0x7fe14415dae0 2024-04-25T08:45:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe13aa00640 1 -- 172.21.15.46:0/2251250559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c0be070 con 0x7fe14415dae0 2024-04-25T08:45:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe13aa00640 1 -- 172.21.15.46:0/2251250559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0adbd0 con 0x7fe14415dae0 2024-04-25T08:45:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1441357a0 con 0x7fe14415dae0 2024-04-25T08:45:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1441f6a50 con 0x7fe14415dae0 2024-04-25T08:45:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.498+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe144135970 con 0x7fe14415dae0 2024-04-25T08:45:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.502+0000 7fe13aa00640 1 -- 172.21.15.46:0/2251250559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe13c0c3460 con 0x7fe14415dae0 2024-04-25T08:45:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.502+0000 7fe13aa00640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe120066f60 0x7fe120069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.502+0000 7fe13aa00640 1 -- 172.21.15.46:0/2251250559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe13c120330 con 0x7fe14415dae0 2024-04-25T08:45:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.502+0000 7fe142a00640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe120066f60 0x7fe120069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.502+0000 7fe142a00640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe120066f60 0x7fe120069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe134000f70 tx=0x7fe134007cb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.506+0000 7fe13aa00640 1 -- 172.21.15.46:0/2251250559 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe13c016520 con 0x7fe14415dae0 2024-04-25T08:45:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.670+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe144069010 con 0x7fe14415dae0 2024-04-25T08:45:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe13aa00640 1 -- 172.21.15.46:0/2251250559 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe13c0f14f0 con 0x7fe14415dae0 2024-04-25T08:45:09.666 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe120066f60 msgr2=0x7fe120069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe120066f60 0x7fe120069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe134000f70 tx=0x7fe134007cb0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14415dae0 msgr2=0x7fe14415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14415dae0 0x7fe14415e720 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe13c01a530 tx=0x7fe13c0196a0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 shutdown_connections 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe120066f60 0x7fe120069420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe14415dae0 0x7fe14415e720 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe14406ea60 0x7fe1441f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 --2- 172.21.15.46:0/2251250559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe14406e620 0x7fe144160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 >> 172.21.15.46:0/2251250559 conn(0x7fe144071ae0 msgr2=0x7fe144151490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 shutdown_connections 2024-04-25T08:45:09.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.674+0000 7fe149800640 1 -- 172.21.15.46:0/2251250559 wait complete. 2024-04-25T08:45:09.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:45:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.834+0000 7f646aa00640 1 Processor -- start 2024-04-25T08:45:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.834+0000 7f646aa00640 1 -- start start 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f646aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 0x7f64640b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f646aa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646406bdc0 con 0x7f64640ccde0 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f646aa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646406c0a0 con 0x7f6464099a00 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f646aa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646406db90 con 0x7f64640b2240 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f645be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f64640ccde0 0x7f64640cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50818/0 (socket says 172.21.15.46:50818) 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f645be00640 1 -- 172.21.15.46:0/1566797790 learned_addr learned my addr 172.21.15.46:0/1566797790 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f6468a00640 1 --2- 172.21.15.46:0/1566797790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 0x7f64640b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f6463400640 1 -- 172.21.15.46:0/1566797790 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992254258 0 0) 0x7f646406bdc0 con 0x7f64640ccde0 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f6463400640 1 -- 172.21.15.46:0/1566797790 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6448003530 con 0x7f64640ccde0 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f6463400640 1 -- 172.21.15.46:0/1566797790 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813528182 0 0) 0x7f646406c0a0 con 0x7f6464099a00 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f6463400640 1 -- 172.21.15.46:0/1566797790 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f646406bdc0 con 0x7f6464099a00 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f6468a00640 1 -- 172.21.15.46:0/1566797790 >> v1:172.21.15.82:6789/0 conn(0x7f6464099a00 legacy=0x7f6464099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f6468a00640 1 -- 172.21.15.46:0/1566797790 >> v1:172.21.15.46:6789/0 conn(0x7f64640ccde0 legacy=0x7f64640cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f6463400640 1 -- 172.21.15.46:0/1566797790 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3139444842 0 0) 0x7f6448003530 con 0x7f64640ccde0 2024-04-25T08:45:09.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.838+0000 7f6468a00640 1 -- 172.21.15.46:0/1566797790 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646406c380 con 0x7f64640b2240 2024-04-25T08:45:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f6468a00640 1 --2- 172.21.15.46:0/1566797790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 0x7f64640b26d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f645c009df0 tx=0x7f645c01c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55690ca7f1d8fec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f6463400640 1 -- 172.21.15.46:0/1566797790 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0b2070 con 0x7f64640b2240 2024-04-25T08:45:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f6463400640 1 -- 172.21.15.46:0/1566797790 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c0a5520 con 0x7f64640b2240 2024-04-25T08:45:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f6463400640 1 -- 172.21.15.46:0/1566797790 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c001680 con 0x7f64640b2240 2024-04-25T08:45:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 -- 172.21.15.46:0/1566797790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 msgr2=0x7f64640b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 --2- 172.21.15.46:0/1566797790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 0x7f64640b26d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f645c009df0 tx=0x7f645c01c820 comp rx=0 tx=0).stop 2024-04-25T08:45:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 -- 172.21.15.46:0/1566797790 shutdown_connections 2024-04-25T08:45:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 --2- 172.21.15.46:0/1566797790 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 0x7f64640b26d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 -- 172.21.15.46:0/1566797790 >> 172.21.15.46:0/1566797790 conn(0x7f64640b3120 msgr2=0x7f64640b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 -- 172.21.15.46:0/1566797790 shutdown_connections 2024-04-25T08:45:09.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 -- 172.21.15.46:0/1566797790 wait complete. 2024-04-25T08:45:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 Processor -- start 2024-04-25T08:45:09.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 -- start start 2024-04-25T08:45:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6464099a00 0x7f646415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 0x7f646414e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64640ccde0 0x7f64641602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646415de90 con 0x7f64640ccde0 2024-04-25T08:45:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646415dd10 con 0x7f6464099a00 2024-04-25T08:45:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f646aa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646416be10 con 0x7f64640b2240 2024-04-25T08:45:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.842+0000 7f6468a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64640ccde0 0x7f64641602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f645be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 0x7f646414e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6463e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6464099a00 0x7f646415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6468a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64640ccde0 0x7f64641602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33502/0 (socket says 172.21.15.46:33502) 2024-04-25T08:45:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6468a00640 1 -- 172.21.15.46:0/3498304161 learned_addr learned my addr 172.21.15.46:0/3498304161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6468a00640 1 -- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 msgr2=0x7f646414e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6468a00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 0x7f646414e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6468a00640 1 -- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6464099a00 msgr2=0x7f646415c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6468a00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6464099a00 0x7f646415c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6468a00640 1 -- 172.21.15.46:0/3498304161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646414f100 con 0x7f64640ccde0 2024-04-25T08:45:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6468a00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64640ccde0 0x7f64641602c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f645c0b0040 tx=0x7f645c0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6460c00640 1 -- 172.21.15.46:0/3498304161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0bd020 con 0x7f64640ccde0 2024-04-25T08:45:09.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6460c00640 1 -- 172.21.15.46:0/3498304161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c0b3040 con 0x7f64640ccde0 2024-04-25T08:45:09.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6460c00640 1 -- 172.21.15.46:0/3498304161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0b2050 con 0x7f64640ccde0 2024-04-25T08:45:09.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646415e140 con 0x7f64640ccde0 2024-04-25T08:45:09.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646415e700 con 0x7f64640ccde0 2024-04-25T08:45:09.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646415edc0 con 0x7f64640ccde0 2024-04-25T08:45:09.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.846+0000 7f6460c00640 1 -- 172.21.15.46:0/3498304161 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f645c0c0070 con 0x7f64640ccde0 2024-04-25T08:45:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.850+0000 7f6460c00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6440066f60 0x7f6440069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.850+0000 7f6460c00640 1 -- 172.21.15.46:0/3498304161 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f645c11eff0 con 0x7f64640ccde0 2024-04-25T08:45:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.850+0000 7f6463e00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6440066f60 0x7f6440069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:09.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.850+0000 7f6463e00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6440066f60 0x7f6440069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f64540011f0 tx=0x7f6454009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:09.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:09.854+0000 7f6460c00640 1 -- 172.21.15.46:0/3498304161 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f646415edc0 con 0x7f64640ccde0 2024-04-25T08:45:10.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6464069080 con 0x7f64640ccde0 2024-04-25T08:45:10.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f6460c00640 1 -- 172.21.15.46:0/3498304161 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+65 (secure 0 0 0) 0x7f645c0164e0 con 0x7f64640ccde0 2024-04-25T08:45:10.011 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:10.011 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:45:10.011 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:10.011 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-25T08:45:10.011 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-25T08:45:10.011 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:10.011 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6440066f60 msgr2=0x7f6440069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6440066f60 0x7f6440069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f64540011f0 tx=0x7f6454009040 comp rx=0 tx=0).stop 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64640ccde0 msgr2=0x7f64641602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64640ccde0 0x7f64641602c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f645c0b0040 tx=0x7f645c0195c0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 shutdown_connections 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6440066f60 0x7f6440069420 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64640ccde0 0x7f64641602c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64640b2240 0x7f646414e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 --2- 172.21.15.46:0/3498304161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6464099a00 0x7f646415c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 >> 172.21.15.46:0/3498304161 conn(0x7f64640b3120 msgr2=0x7f646415cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 shutdown_connections 2024-04-25T08:45:10.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.018+0000 7f646aa00640 1 -- 172.21.15.46:0/3498304161 wait complete. 2024-04-25T08:45:10.032 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:45:10.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:45:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42fe000640 1 Processor -- start 2024-04-25T08:45:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42fe000640 1 -- start start 2024-04-25T08:45:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42fe000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42f806ea20 0x7f42f80ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42fe000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f8141580 0x7f42f80cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42fe000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f80b2e60 con 0x7f42f8140ee0 2024-04-25T08:45:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42fe000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f81412c0 con 0x7f42f806ea20 2024-04-25T08:45:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42fe000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80cef50 con 0x7f42f8141580 2024-04-25T08:45:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42f6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42f806ea20 0x7f42f80ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42f6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42f806ea20 0x7f42f80ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33514/0 (socket says 172.21.15.46:33514) 2024-04-25T08:45:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42f6a00640 1 -- 172.21.15.46:0/3388908978 learned_addr learned my addr 172.21.15.46:0/3388908978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42f7e00640 1 --2- 172.21.15.46:0/3388908978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f8141580 0x7f42f80cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42f6a00640 1 -- 172.21.15.46:0/3388908978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f8141580 msgr2=0x7f42f80cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42f6a00640 1 --2- 172.21.15.46:0/3388908978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f8141580 0x7f42f80cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42f6a00640 1 -- 172.21.15.46:0/3388908978 >> v1:172.21.15.46:6789/0 conn(0x7f42f8140ee0 legacy=0x7f42f806e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42f6a00640 1 -- 172.21.15.46:0/3388908978 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80b30d0 con 0x7f42f806ea20 2024-04-25T08:45:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.186+0000 7f42f7e00640 1 --2- 172.21.15.46:0/3388908978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f8141580 0x7f42f80cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42f6a00640 1 --2- 172.21.15.46:0/3388908978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f806ea20 0x7f42f80ce4d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f42e40098a0 tx=0x7f42e401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c712a70a17083ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42f6000640 1 -- 172.21.15.46:0/3388908978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e40a9400 con 0x7f42f806ea20 2024-04-25T08:45:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42f6000640 1 -- 172.21.15.46:0/3388908978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e40a95a0 con 0x7f42f806ea20 2024-04-25T08:45:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42f6000640 1 -- 172.21.15.46:0/3388908978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e40a9970 con 0x7f42f806ea20 2024-04-25T08:45:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42fe000640 1 -- 172.21.15.46:0/3388908978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f806ea20 msgr2=0x7f42f80ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42fe000640 1 --2- 172.21.15.46:0/3388908978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f806ea20 0x7f42f80ce4d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f42e40098a0 tx=0x7f42e401c540 comp rx=0 tx=0).stop 2024-04-25T08:45:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42fe000640 1 -- 172.21.15.46:0/3388908978 shutdown_connections 2024-04-25T08:45:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42fe000640 1 --2- 172.21.15.46:0/3388908978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f8141580 0x7f42f80cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42fe000640 1 --2- 172.21.15.46:0/3388908978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f806ea20 0x7f42f80ce4d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42fe000640 1 -- 172.21.15.46:0/3388908978 >> 172.21.15.46:0/3388908978 conn(0x7f42f8067050 msgr2=0x7f42f806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42fe000640 1 -- 172.21.15.46:0/3388908978 shutdown_connections 2024-04-25T08:45:10.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42fe000640 1 -- 172.21.15.46:0/3388908978 wait complete. 2024-04-25T08:45:10.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.190+0000 7f42fe000640 1 Processor -- start 2024-04-25T08:45:10.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 -- start start 2024-04-25T08:45:10.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f806ea20 0x7f42f8158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f8140ee0 0x7f42f80af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8141580 0x7f42f813dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b4540 con 0x7f42f8141580 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b43c0 con 0x7f42f8140ee0 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f80b46c0 con 0x7f42f806ea20 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8141580 0x7f42f813dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8141580 0x7f42f813dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33528/0 (socket says 172.21.15.46:33528) 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7e00640 1 -- 172.21.15.46:0/126377689 learned_addr learned my addr 172.21.15.46:0/126377689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7e00640 1 -- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f806ea20 msgr2=0x7f42f8158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7400640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f806ea20 0x7f42f8158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7e00640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f806ea20 0x7f42f8158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7e00640 1 -- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f8140ee0 msgr2=0x7f42f80af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f6a00640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f8140ee0 0x7f42f80af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7e00640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f8140ee0 0x7f42f80af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7e00640 1 -- 172.21.15.46:0/126377689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80ae3f0 con 0x7f42f8141580 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42f7e00640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8141580 0x7f42f813dd70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f42f000cd80 tx=0x7f42f000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42ef400640 1 -- 172.21.15.46:0/126377689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f000f680 con 0x7f42f8141580 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42ef400640 1 -- 172.21.15.46:0/126377689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f000f840 con 0x7f42f8141580 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42ef400640 1 -- 172.21.15.46:0/126377689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f000fc30 con 0x7f42f8141580 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f80aea20 con 0x7f42f8141580 2024-04-25T08:45:10.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f80ae110 con 0x7f42f8141580 2024-04-25T08:45:10.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.194+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f8069080 con 0x7f42f8141580 2024-04-25T08:45:10.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.198+0000 7f42ef400640 1 -- 172.21.15.46:0/126377689 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42f000e3e0 con 0x7f42f8141580 2024-04-25T08:45:10.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.198+0000 7f42ef400640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42d8066f60 0x7f42d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.198+0000 7f42ef400640 1 -- 172.21.15.46:0/126377689 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f42f0085e70 con 0x7f42f8141580 2024-04-25T08:45:10.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.198+0000 7f42f7400640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42d8066f60 0x7f42d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.198+0000 7f42f7400640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42d8066f60 0x7f42d8069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42e8007fd0 tx=0x7f42e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:10.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.202+0000 7f42ef400640 1 -- 172.21.15.46:0/126377689 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42f0014030 con 0x7f42f8141580 2024-04-25T08:45:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.362+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f42f8071280 con 0x7f42f8141580 2024-04-25T08:45:10.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.578+0000 7f42ef400640 1 -- 172.21.15.46:0/126377689 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 138+0+0 (secure 0 0 0) 0x7f42f0057020 con 0x7f42f8141580 2024-04-25T08:45:10.574 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42d8066f60 msgr2=0x7f42d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42d8066f60 0x7f42d8069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f42e8007fd0 tx=0x7f42e8009040 comp rx=0 tx=0).stop 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8141580 msgr2=0x7f42f813dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8141580 0x7f42f813dd70 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f42f000cd80 tx=0x7f42f000a650 comp rx=0 tx=0).stop 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 shutdown_connections 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42d8066f60 0x7f42d8069420 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8141580 0x7f42f813dd70 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f8140ee0 0x7f42f80af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 --2- 172.21.15.46:0/126377689 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42f806ea20 0x7f42f8158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 >> 172.21.15.46:0/126377689 conn(0x7f42f8067050 msgr2=0x7f42f80ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:10.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 shutdown_connections 2024-04-25T08:45:10.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.582+0000 7f42fe000640 1 -- 172.21.15.46:0/126377689 wait complete. 2024-04-25T08:45:10.601 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:45:10.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.754+0000 7ff2dca00640 1 Processor -- start 2024-04-25T08:45:10.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.754+0000 7ff2dca00640 1 -- start start 2024-04-25T08:45:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2dca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d80b7100 0x7ff2d80cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2dca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff2d809a110 0x7ff2d80b5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2dca00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d80cd720 con 0x7ff2d80cd1a0 2024-04-25T08:45:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2dca00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d80b74e0 con 0x7ff2d809a110 2024-04-25T08:45:10.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2dca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d80b4da0 con 0x7ff2d80b7100 2024-04-25T08:45:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d7400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d80b7100 0x7ff2d80cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff2d809a110 0x7ff2d80b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d7400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d80b7100 0x7ff2d80cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35774/0 (socket says 172.21.15.46:35774) 2024-04-25T08:45:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d6a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff2d80cd1a0 0x7ff2d8099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:32796/0 (socket says 172.21.15.46:32796) 2024-04-25T08:45:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d7400640 1 -- 172.21.15.46:0/2466468791 learned_addr learned my addr 172.21.15.46:0/2466468791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d7e00640 1 -- 172.21.15.46:0/2466468791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d80b7100 msgr2=0x7ff2d80cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d7e00640 1 --2- 172.21.15.46:0/2466468791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d80b7100 0x7ff2d80cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d7e00640 1 -- 172.21.15.46:0/2466468791 >> v1:172.21.15.82:6789/0 conn(0x7ff2d80cd1a0 legacy=0x7ff2d8099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d7e00640 1 -- 172.21.15.46:0/2466468791 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80b44b0 con 0x7ff2d809a110 2024-04-25T08:45:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.758+0000 7ff2d6000640 1 -- 172.21.15.46:0/2466468791 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259653698 0 0) 0x7ff2d80cd720 con 0x7ff2d80cd1a0 2024-04-25T08:45:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2d7e00640 1 --2- 172.21.15.46:0/2466468791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d809a110 0x7ff2d80b5230 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009d90 tx=0x7ff2d001c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e831ba2d10a89851 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2d6000640 1 -- 172.21.15.46:0/2466468791 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00b2070 con 0x7ff2d809a110 2024-04-25T08:45:10.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2d6000640 1 -- 172.21.15.46:0/2466468791 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d00a5800 con 0x7ff2d809a110 2024-04-25T08:45:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2d6000640 1 -- 172.21.15.46:0/2466468791 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00a5bd0 con 0x7ff2d809a110 2024-04-25T08:45:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 -- 172.21.15.46:0/2466468791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d809a110 msgr2=0x7ff2d80b5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 --2- 172.21.15.46:0/2466468791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d809a110 0x7ff2d80b5230 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009d90 tx=0x7ff2d001c850 comp rx=0 tx=0).stop 2024-04-25T08:45:10.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 -- 172.21.15.46:0/2466468791 shutdown_connections 2024-04-25T08:45:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 --2- 172.21.15.46:0/2466468791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d809a110 0x7ff2d80b5230 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 --2- 172.21.15.46:0/2466468791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d80b7100 0x7ff2d80cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 -- 172.21.15.46:0/2466468791 >> 172.21.15.46:0/2466468791 conn(0x7ff2d80d14f0 msgr2=0x7ff2d80d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 -- 172.21.15.46:0/2466468791 shutdown_connections 2024-04-25T08:45:10.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 -- 172.21.15.46:0/2466468791 wait complete. 2024-04-25T08:45:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 Processor -- start 2024-04-25T08:45:10.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.762+0000 7ff2dca00640 1 -- start start 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2dca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d809a110 0x7ff2d81589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2dca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d80b7100 0x7ff2d80c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2dca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d80cd1a0 0x7ff2d815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2dca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d80b6100 con 0x7ff2d80b7100 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2dca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d80b5f80 con 0x7ff2d80cd1a0 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2dca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d80b6280 con 0x7ff2d809a110 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d80b7100 0x7ff2d80c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d80b7100 0x7ff2d80c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33532/0 (socket says 172.21.15.46:33532) 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d6a00640 1 -- 172.21.15.46:0/2046325201 learned_addr learned my addr 172.21.15.46:0/2046325201 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d7400640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d809a110 0x7ff2d81589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d7e00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d80cd1a0 0x7ff2d815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d6a00640 1 -- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d809a110 msgr2=0x7ff2d81589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d6a00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d809a110 0x7ff2d81589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d6a00640 1 -- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d80cd1a0 msgr2=0x7ff2d815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d6a00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d80cd1a0 0x7ff2d815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:10.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d6a00640 1 -- 172.21.15.46:0/2046325201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d80ca190 con 0x7ff2d80b7100 2024-04-25T08:45:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2d6a00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d80b7100 0x7ff2d80c8f80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400c450 tx=0x7ff2c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2cf400640 1 -- 172.21.15.46:0/2046325201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4018070 con 0x7ff2d80b7100 2024-04-25T08:45:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2cf400640 1 -- 172.21.15.46:0/2046325201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c40074e0 con 0x7ff2d80b7100 2024-04-25T08:45:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d80c98f0 con 0x7ff2d80b7100 2024-04-25T08:45:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2cf400640 1 -- 172.21.15.46:0/2046325201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40049f0 con 0x7ff2d80b7100 2024-04-25T08:45:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d80c9eb0 con 0x7ff2d80b7100 2024-04-25T08:45:10.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.766+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d8069010 con 0x7ff2d80b7100 2024-04-25T08:45:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.770+0000 7ff2cf400640 1 -- 172.21.15.46:0/2046325201 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2c401b070 con 0x7ff2d80b7100 2024-04-25T08:45:10.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.770+0000 7ff2cf400640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff2b8066f60 0x7ff2b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.770+0000 7ff2cf400640 1 -- 172.21.15.46:0/2046325201 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5619+0+0 (secure 0 0 0) 0x7ff2c40842b0 con 0x7ff2d80b7100 2024-04-25T08:45:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.770+0000 7ff2d7400640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff2b8066f60 0x7ff2b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:10.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.770+0000 7ff2d7400640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff2b8066f60 0x7ff2b8069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80cb7a0 tx=0x7ff2c8009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:10.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.774+0000 7ff2cf400640 1 -- 172.21.15.46:0/2046325201 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2c4054cb0 con 0x7ff2d80b7100 2024-04-25T08:45:10.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:10.938+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff2d815d0e0 con 0x7ff2d80b7100 2024-04-25T08:45:11.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.582+0000 7ff2cf400640 1 -- 172.21.15.46:0/2046325201 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff2c4058d50 con 0x7ff2d80b7100 2024-04-25T08:45:11.577 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-25T08:45:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff2b8066f60 msgr2=0x7ff2b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff2b8066f60 0x7ff2b8069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80cb7a0 tx=0x7ff2c8009240 comp rx=0 tx=0).stop 2024-04-25T08:45:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d80b7100 msgr2=0x7ff2d80c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d80b7100 0x7ff2d80c8f80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400c450 tx=0x7ff2c400c910 comp rx=0 tx=0).stop 2024-04-25T08:45:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 shutdown_connections 2024-04-25T08:45:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff2b8066f60 0x7ff2b8069420 secure :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80cb7a0 tx=0x7ff2c8009240 comp rx=0 tx=0).stop 2024-04-25T08:45:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d80cd1a0 0x7ff2d815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d80b7100 0x7ff2d80c8f80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 --2- 172.21.15.46:0/2046325201 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2d809a110 0x7ff2d81589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 >> 172.21.15.46:0/2046325201 conn(0x7ff2d80d14f0 msgr2=0x7ff2d80cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 shutdown_connections 2024-04-25T08:45:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.586+0000 7ff2dca00640 1 -- 172.21.15.46:0/2046325201 wait complete. 2024-04-25T08:45:11.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:45:11.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.770+0000 7ff1c7200640 1 Processor -- start 2024-04-25T08:45:11.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.770+0000 7ff1c7200640 1 -- start start 2024-04-25T08:45:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.770+0000 7ff1c7200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff1c01440d0 0x7ff1c00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.770+0000 7ff1c7200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c015e440 con 0x7ff1c0143990 2024-04-25T08:45:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.770+0000 7ff1c7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1c015e6e0 con 0x7ff1c006f150 2024-04-25T08:45:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1c7200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1c015ed30 con 0x7ff1c01440d0 2024-04-25T08:45:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff1c0143990 0x7ff1c006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50834/0 (socket says 172.21.15.46:50834) 2024-04-25T08:45:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bfe00640 1 -- 172.21.15.46:0/1109281728 learned_addr learned my addr 172.21.15.46:0/1109281728 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/1109281728 >> v2:172.21.15.82:3300/0 conn(0x7ff1c01440d0 0x7ff1c00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096306684 0 0) 0x7ff1c015e440 con 0x7ff1c0143990 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a4003530 con 0x7ff1c0143990 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3623315150 0 0) 0x7ff1a4003530 con 0x7ff1c0143990 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 >> v2:172.21.15.82:3300/0 conn(0x7ff1c01440d0 msgr2=0x7ff1c00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 --2- 172.21.15.46:0/1109281728 >> v2:172.21.15.82:3300/0 conn(0x7ff1c01440d0 0x7ff1c00cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 >> v1:172.21.15.82:6789/0 conn(0x7ff1c006f150 legacy=0x7ff1c00b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c015e9c0 con 0x7ff1c0143990 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7ff1b0001680 con 0x7ff1c0143990 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 >> v1:172.21.15.46:6789/0 conn(0x7ff1c0143990 legacy=0x7ff1c006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a4003a30 0x7ff1a4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4004810 0x7ff1a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a40051d0 0x7ff1a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c015ed30 con 0x7ff1a4003a30 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a400c070 con 0x7ff1a4004810 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a400c270 con 0x7ff1a40051d0 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bfe00640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a4003a30 0x7ff1a4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4004810 0x7ff1a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1c5200640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a40051d0 0x7ff1a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bfe00640 1 -- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a40051d0 msgr2=0x7ff1a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bfe00640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a40051d0 0x7ff1a400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bfe00640 1 -- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4004810 msgr2=0x7ff1a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bfe00640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4004810 0x7ff1a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1bfe00640 1 -- 172.21.15.46:0/1109281728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a4003530 con 0x7ff1a4003a30 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1c5200640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1a40051d0 0x7ff1a400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4004810 0x7ff1a4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.774+0000 7ff1c5200640 1 -- 172.21.15.46:0/1109281728 reap_dead start 2024-04-25T08:45:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1bfe00640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a4003a30 0x7ff1a4003e30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000c210 tx=0x7ff1b001cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33187db44d8ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0007cf0 con 0x7ff1a4003a30 2024-04-25T08:45:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b00a9070 con 0x7ff1a4003a30 2024-04-25T08:45:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1bf400640 1 -- 172.21.15.46:0/1109281728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0005d80 con 0x7ff1a4003a30 2024-04-25T08:45:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1c7200640 1 -- 172.21.15.46:0/1109281728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a4003a30 msgr2=0x7ff1a4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1c7200640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a4003a30 0x7ff1a4003e30 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000c210 tx=0x7ff1b001cbb0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1c7200640 1 -- 172.21.15.46:0/1109281728 shutdown_connections 2024-04-25T08:45:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1c7200640 1 --2- 172.21.15.46:0/1109281728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1a4003a30 0x7ff1a4003e30 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1c7200640 1 -- 172.21.15.46:0/1109281728 >> 172.21.15.46:0/1109281728 conn(0x7ff1c006ac10 msgr2=0x7ff1c006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1c7200640 1 -- 172.21.15.46:0/1109281728 shutdown_connections 2024-04-25T08:45:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1c7200640 1 -- 172.21.15.46:0/1109281728 wait complete. 2024-04-25T08:45:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1c7200640 1 Processor -- start 2024-04-25T08:45:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.778+0000 7ff1c7200640 1 -- start start 2024-04-25T08:45:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4003a30 0x7ff1c015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1c00b3550 0x7ff1c00b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c01e7d20 0x7ff1c01e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0143d70 con 0x7ff1c01e7d20 2024-04-25T08:45:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c015f1a0 con 0x7ff1a4003a30 2024-04-25T08:45:11.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c0143ef0 con 0x7ff1c00b3550 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c01e7d20 0x7ff1c01e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c01e7d20 0x7ff1c01e81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33546/0 (socket says 172.21.15.46:33546) 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c5200640 1 -- 172.21.15.46:0/1530031855 learned_addr learned my addr 172.21.15.46:0/1530031855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1c00b3550 0x7ff1c00b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1bfe00640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4003a30 0x7ff1c015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c5200640 1 -- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1c00b3550 msgr2=0x7ff1c00b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c5200640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1c00b3550 0x7ff1c00b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c5200640 1 -- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4003a30 msgr2=0x7ff1c015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c5200640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4003a30 0x7ff1c015a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c5200640 1 -- 172.21.15.46:0/1530031855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c01e9af0 con 0x7ff1c01e7d20 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c5200640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c01e7d20 0x7ff1c01e81a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00ccfe0 tx=0x7ff1b800ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1bcc00640 1 -- 172.21.15.46:0/1530031855 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8018070 con 0x7ff1c01e7d20 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1bcc00640 1 -- 172.21.15.46:0/1530031855 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b8003b90 con 0x7ff1c01e7d20 2024-04-25T08:45:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1bcc00640 1 -- 172.21.15.46:0/1530031855 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80096b0 con 0x7ff1c01e7d20 2024-04-25T08:45:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c01e8a50 con 0x7ff1c01e7d20 2024-04-25T08:45:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c01e9010 con 0x7ff1c01e7d20 2024-04-25T08:45:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.782+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c00b3ef0 con 0x7ff1c01e7d20 2024-04-25T08:45:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.786+0000 7ff1bcc00640 1 -- 172.21.15.46:0/1530031855 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1b801b070 con 0x7ff1c01e7d20 2024-04-25T08:45:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.786+0000 7ff1bcc00640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff19c066f90 0x7ff19c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:11.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.786+0000 7ff1bcc00640 1 -- 172.21.15.46:0/1530031855 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6001+0+0 (secure 0 0 0) 0x7ff1b8084150 con 0x7ff1c01e7d20 2024-04-25T08:45:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.786+0000 7ff1bfe00640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff19c066f90 0x7ff19c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.786+0000 7ff1bfe00640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff19c066f90 0x7ff19c069450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00ab000 tx=0x7ff1b00b0040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.790+0000 7ff1bcc00640 1 -- 172.21.15.46:0/1530031855 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1b8054f00 con 0x7ff1c01e7d20 2024-04-25T08:45:11.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:11.986+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff1c00ccab0 con 0x7ff1c01e7d20 2024-04-25T08:45:12.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.606+0000 7ff1bcc00640 1 -- 172.21.15.46:0/1530031855 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v30) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff1b8054800 con 0x7ff1c01e7d20 2024-04-25T08:45:12.603 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T08:45:12.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.614+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff19c066f90 msgr2=0x7ff19c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:12.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.614+0000 7ff1c7200640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff19c066f90 0x7ff19c069450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00ab000 tx=0x7ff1b00b0040 comp rx=0 tx=0).stop 2024-04-25T08:45:12.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.614+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c01e7d20 msgr2=0x7ff1c01e81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:12.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.614+0000 7ff1c7200640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c01e7d20 0x7ff1c01e81a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff1c00ccfe0 tx=0x7ff1b800ea60 comp rx=0 tx=0).stop 2024-04-25T08:45:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.618+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 shutdown_connections 2024-04-25T08:45:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.618+0000 7ff1c7200640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff19c066f90 0x7ff19c069450 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.618+0000 7ff1c7200640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1c01e7d20 0x7ff1c01e81a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.618+0000 7ff1c7200640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1c00b3550 0x7ff1c00b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.618+0000 7ff1c7200640 1 --2- 172.21.15.46:0/1530031855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1a4003a30 0x7ff1c015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.618+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 >> 172.21.15.46:0/1530031855 conn(0x7ff1c006ac10 msgr2=0x7ff1c015cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.618+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 shutdown_connections 2024-04-25T08:45:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.618+0000 7ff1c7200640 1 -- 172.21.15.46:0/1530031855 wait complete. 2024-04-25T08:45:12.646 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:45:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.794+0000 7f5e71600640 1 Processor -- start 2024-04-25T08:45:12.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.794+0000 7f5e71600640 1 -- start start 2024-04-25T08:45:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.794+0000 7f5e71600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e6c06db40 0x7f5e6c06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.794+0000 7f5e71600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c06b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.794+0000 7f5e71600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6c15e180 con 0x7f5e6c0af3f0 2024-04-25T08:45:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e71600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e6c0b3ca0 con 0x7f5e6c06db40 2024-04-25T08:45:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e71600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c15e460 con 0x7f5e6c0b3550 2024-04-25T08:45:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e63e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e6c06db40 0x7f5e6c06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e63e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e6c06db40 0x7f5e6c06ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33552/0 (socket says 172.21.15.46:33552) 2024-04-25T08:45:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e63e00640 1 -- 172.21.15.46:0/1079313744 learned_addr learned my addr 172.21.15.46:0/1079313744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:12.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e6b400640 1 --2- 172.21.15.46:0/1079313744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c06b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:12.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e63e00640 1 -- 172.21.15.46:0/1079313744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 msgr2=0x7f5e6c06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:12.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e63e00640 1 --2- 172.21.15.46:0/1079313744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c06b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e63e00640 1 -- 172.21.15.46:0/1079313744 >> v1:172.21.15.82:6789/0 conn(0x7f5e6c0af3f0 legacy=0x7f5e6c0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:12.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e63e00640 1 -- 172.21.15.46:0/1079313744 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c15e740 con 0x7f5e6c06db40 2024-04-25T08:45:12.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e6b400640 1 --2- 172.21.15.46:0/1079313744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c06b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e63e00640 1 --2- 172.21.15.46:0/1079313744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 0x7f5e6c06ac10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009870 tx=0x7f5e5401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0da06e9b093452f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e6a000640 1 -- 172.21.15.46:0/1079313744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e540b2070 con 0x7f5e6c06db40 2024-04-25T08:45:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.798+0000 7f5e6a000640 1 -- 172.21.15.46:0/1079313744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e540a9070 con 0x7f5e6c06db40 2024-04-25T08:45:12.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6a000640 1 -- 172.21.15.46:0/1079313744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e540ad880 con 0x7f5e6c06db40 2024-04-25T08:45:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 -- 172.21.15.46:0/1079313744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 msgr2=0x7f5e6c06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 --2- 172.21.15.46:0/1079313744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 0x7f5e6c06ac10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009870 tx=0x7f5e5401c560 comp rx=0 tx=0).stop 2024-04-25T08:45:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 -- 172.21.15.46:0/1079313744 shutdown_connections 2024-04-25T08:45:12.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 --2- 172.21.15.46:0/1079313744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c06b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 --2- 172.21.15.46:0/1079313744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 0x7f5e6c06ac10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 -- 172.21.15.46:0/1079313744 >> 172.21.15.46:0/1079313744 conn(0x7f5e6c0cfd90 msgr2=0x7f5e6c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 -- 172.21.15.46:0/1079313744 shutdown_connections 2024-04-25T08:45:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 -- 172.21.15.46:0/1079313744 wait complete. 2024-04-25T08:45:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 Processor -- start 2024-04-25T08:45:12.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 -- start start 2024-04-25T08:45:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 0x7f5e6c157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6c0af3f0 0x7f5e6c0ad470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c16a870 con 0x7f5e6c06db40 2024-04-25T08:45:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c16a6f0 con 0x7f5e6c0af3f0 2024-04-25T08:45:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e71600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6c16a9f0 con 0x7f5e6c0b3550 2024-04-25T08:45:12.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 0x7f5e6c157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 0x7f5e6c157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33560/0 (socket says 172.21.15.46:33560) 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6aa00640 1 -- 172.21.15.46:0/3795486002 learned_addr learned my addr 172.21.15.46:0/3795486002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e63e00640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6c0af3f0 0x7f5e6c0ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6b400640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6aa00640 1 -- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 msgr2=0x7f5e6c0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6aa00640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c0ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6aa00640 1 -- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6c0af3f0 msgr2=0x7f5e6c0ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6aa00640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6c0af3f0 0x7f5e6c0ad470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6aa00640 1 -- 172.21.15.46:0/3795486002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c158250 con 0x7f5e6c06db40 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.802+0000 7f5e6b400640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:12.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e6aa00640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 0x7f5e6c157970 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c420 tx=0x7f5e5c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e62a00640 1 -- 172.21.15.46:0/3795486002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c0052d0 con 0x7f5e6c06db40 2024-04-25T08:45:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e62a00640 1 -- 172.21.15.46:0/3795486002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5c005470 con 0x7f5e6c06db40 2024-04-25T08:45:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6c158530 con 0x7f5e6c06db40 2024-04-25T08:45:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e62a00640 1 -- 172.21.15.46:0/3795486002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c007260 con 0x7f5e6c06db40 2024-04-25T08:45:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e6c158af0 con 0x7f5e6c06db40 2024-04-25T08:45:12.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e6c069080 con 0x7f5e6c06db40 2024-04-25T08:45:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e62a00640 1 -- 172.21.15.46:0/3795486002 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e5c007400 con 0x7f5e6c06db40 2024-04-25T08:45:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e62a00640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e48066f60 0x7f5e48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:12.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e62a00640 1 -- 172.21.15.46:0/3795486002 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e5c0846b0 con 0x7f5e6c06db40 2024-04-25T08:45:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.806+0000 7f5e63e00640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e48066f60 0x7f5e48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.810+0000 7f5e63e00640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e48066f60 0x7f5e48069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5e540af000 tx=0x7f5e54008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.810+0000 7f5e62a00640 1 -- 172.21.15.46:0/3795486002 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e5c054e40 con 0x7f5e6c06db40 2024-04-25T08:45:12.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.974+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e6c15b610 con 0x7f5e6c06db40 2024-04-25T08:45:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.974+0000 7f5e62a00640 1 -- 172.21.15.46:0/3795486002 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16849 (secure 0 0 0) 0x7f5e5c058ee0 con 0x7f5e6c06db40 2024-04-25T08:45:12.968 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:12.968 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":64,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:45:11.989524+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:45:10.412423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:45:10.987443+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:45:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e48066f60 msgr2=0x7f5e48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e48066f60 0x7f5e48069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5e540af000 tx=0x7f5e54008040 comp rx=0 tx=0).stop 2024-04-25T08:45:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 msgr2=0x7f5e6c157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 0x7f5e6c157970 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00c420 tx=0x7f5e5c00c8e0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 shutdown_connections 2024-04-25T08:45:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e48066f60 0x7f5e48069420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e6c0b3550 0x7f5e6c0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e6c0af3f0 0x7f5e6c0ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 --2- 172.21.15.46:0/3795486002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e6c06db40 0x7f5e6c157970 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 >> 172.21.15.46:0/3795486002 conn(0x7f5e6c0cfd90 msgr2=0x7f5e6c06e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 shutdown_connections 2024-04-25T08:45:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:12.978+0000 7f5e71600640 1 -- 172.21.15.46:0/3795486002 wait complete. 2024-04-25T08:45:12.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:45:13.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09ce000640 1 Processor -- start 2024-04-25T08:45:13.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09ce000640 1 -- start start 2024-04-25T08:45:13.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09ce000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ccde0 0x7f09c80cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09ce000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c806bdc0 con 0x7f09c8099a00 2024-04-25T08:45:13.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09ce000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c806c0a0 con 0x7f09c80b2240 2024-04-25T08:45:13.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09ce000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c806db90 con 0x7f09c80ccde0 2024-04-25T08:45:13.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f09c8099a00 0x7f09c8099de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50848/0 (socket says 172.21.15.46:50848) 2024-04-25T08:45:13.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c7400640 1 -- 172.21.15.46:0/3523129951 learned_addr learned my addr 172.21.15.46:0/3523129951 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:13.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09bea00640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ccde0 0x7f09c80cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049762118 0 0) 0x7f09c806bdc0 con 0x7f09c8099a00 2024-04-25T08:45:13.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09ac003530 con 0x7f09c8099a00 2024-04-25T08:45:13.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401901796 0 0) 0x7f09c806c0a0 con 0x7f09c80b2240 2024-04-25T08:45:13.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09c806bdc0 con 0x7f09c80b2240 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2429244437 0 0) 0x7f09ac003530 con 0x7f09c8099a00 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ccde0 msgr2=0x7f09c80cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ccde0 0x7f09c80cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 >> v1:172.21.15.82:6789/0 conn(0x7f09c80b2240 legacy=0x7f09c80b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c806c380 con 0x7f09c8099a00 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f09b8001a00 con 0x7f09c8099a00 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09bea00640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ccde0 0x7f09c80cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 >> v1:172.21.15.46:6789/0 conn(0x7f09c8099a00 legacy=0x7f09c8099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09ac003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ccde0 0x7f09ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09ac004d50 0x7f09ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c806db90 con 0x7f09ac003b80 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac00bae0 con 0x7f09ac004d50 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac00bce0 con 0x7f09c80ccde0 2024-04-25T08:45:13.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.138+0000 7f09c7400640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09ac003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09bea00640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ccde0 0x7f09ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c7e00640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09ac004d50 0x7f09ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c7400640 1 -- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ccde0 msgr2=0x7f09ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:13.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c7400640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ccde0 0x7f09ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c7400640 1 -- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09ac004d50 msgr2=0x7f09ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:13.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c7400640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09ac004d50 0x7f09ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c7400640 1 -- 172.21.15.46:0/3523129951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac003530 con 0x7f09ac003b80 2024-04-25T08:45:13.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c7400640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09ac003f80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f09b800a820 tx=0x7f09b801d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb725dac350126d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b800a450 con 0x7f09ac003b80 2024-04-25T08:45:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b80072d0 con 0x7f09ac003b80 2024-04-25T08:45:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c6a00640 1 -- 172.21.15.46:0/3523129951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8007700 con 0x7f09ac003b80 2024-04-25T08:45:13.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09ce000640 1 -- 172.21.15.46:0/3523129951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 msgr2=0x7f09ac003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09ce000640 1 --2- 172.21.15.46:0/3523129951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09ac003f80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f09b800a820 tx=0x7f09b801d2f0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.142+0000 7f09c7e00640 1 -- 172.21.15.46:0/3523129951 reap_dead start 2024-04-25T08:45:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 -- 172.21.15.46:0/3523129951 shutdown_connections 2024-04-25T08:45:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 -- 172.21.15.46:0/3523129951 >> 172.21.15.46:0/3523129951 conn(0x7f09c80b3120 msgr2=0x7f09c80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 -- 172.21.15.46:0/3523129951 shutdown_connections 2024-04-25T08:45:13.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 -- 172.21.15.46:0/3523129951 wait complete. 2024-04-25T08:45:13.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 Processor -- start 2024-04-25T08:45:13.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 -- start start 2024-04-25T08:45:13.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09c80c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09c81567e0 0x7f09c80c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ca4b0 0x7f09c8157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c80b2f60 con 0x7f09ac003b80 2024-04-25T08:45:13.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c80b2de0 con 0x7f09c81567e0 2024-04-25T08:45:13.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09ce000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c815dd10 con 0x7f09c80ca4b0 2024-04-25T08:45:13.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09c80c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09c80c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33574/0 (socket says 172.21.15.46:33574) 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7400640 1 -- 172.21.15.46:0/2586157456 learned_addr learned my addr 172.21.15.46:0/2586157456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09bea00640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09c81567e0 0x7f09c80c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7e00640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ca4b0 0x7f09c8157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7400640 1 -- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ca4b0 msgr2=0x7f09c8157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7400640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ca4b0 0x7f09c8157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7400640 1 -- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09c81567e0 msgr2=0x7f09c80c9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7400640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09c81567e0 0x7f09c80c9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7400640 1 -- 172.21.15.46:0/2586157456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c80c8a30 con 0x7f09ac003b80 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7e00640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ca4b0 0x7f09c8157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:13.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.146+0000 7f09c7400640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09c80c5240 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f09b800a820 tx=0x7f09ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:13.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09bfe00640 1 -- 172.21.15.46:0/2586157456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b80b2040 con 0x7f09ac003b80 2024-04-25T08:45:13.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09bfe00640 1 -- 172.21.15.46:0/2586157456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b80013b0 con 0x7f09ac003b80 2024-04-25T08:45:13.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c80c8d10 con 0x7f09ac003b80 2024-04-25T08:45:13.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09bfe00640 1 -- 172.21.15.46:0/2586157456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8001800 con 0x7f09ac003b80 2024-04-25T08:45:13.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c80c92d0 con 0x7f09ac003b80 2024-04-25T08:45:13.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c8069080 con 0x7f09ac003b80 2024-04-25T08:45:13.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09bfe00640 1 -- 172.21.15.46:0/2586157456 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09b80be020 con 0x7f09ac003b80 2024-04-25T08:45:13.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09bfe00640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f09a4066f60 0x7f09a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09bfe00640 1 -- 172.21.15.46:0/2586157456 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09b811f820 con 0x7f09ac003b80 2024-04-25T08:45:13.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.150+0000 7f09bea00640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f09a4066f60 0x7f09a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.154+0000 7f09bea00640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f09a4066f60 0x7f09a4069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f09c80cb770 tx=0x7f09b0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:13.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.154+0000 7f09bfe00640 1 -- 172.21.15.46:0/2586157456 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09b8017030 con 0x7f09ac003b80 2024-04-25T08:45:13.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.318+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09c809b090 con 0x7f09ac003b80 2024-04-25T08:45:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.318+0000 7f09bfe00640 1 -- 172.21.15.46:0/2586157456 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16849 (secure 0 0 0) 0x7f09b80f0600 con 0x7f09ac003b80 2024-04-25T08:45:13.312 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:13.312 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":64,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:45:11.989524+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:45:10.412423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:45:10.987443+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:45:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f09a4066f60 msgr2=0x7f09a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f09a4066f60 0x7f09a4069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f09c80cb770 tx=0x7f09b0009190 comp rx=0 tx=0).stop 2024-04-25T08:45:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 msgr2=0x7f09c80c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09c80c5240 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f09b800a820 tx=0x7f09ac005840 comp rx=0 tx=0).stop 2024-04-25T08:45:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 shutdown_connections 2024-04-25T08:45:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f09a4066f60 0x7f09a4069420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f09c80ca4b0 0x7f09c8157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09c81567e0 0x7f09c80c9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 --2- 172.21.15.46:0/2586157456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09ac003b80 0x7f09c80c5240 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 >> 172.21.15.46:0/2586157456 conn(0x7f09c80b3120 msgr2=0x7f09c815d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 shutdown_connections 2024-04-25T08:45:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.322+0000 7f09ce000640 1 -- 172.21.15.46:0/2586157456 wait complete. 2024-04-25T08:45:13.333 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:45:13.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.482+0000 7ff607600640 1 Processor -- start 2024-04-25T08:45:13.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.482+0000 7ff607600640 1 -- start start 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff607600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff600143990 0x7ff6000cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff607600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff60006f150 0x7ff6000b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff607600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60015e440 con 0x7ff6001440d0 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff607600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60015ed30 con 0x7ff60006f150 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff607600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60015e6e0 con 0x7ff600143990 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff5ffe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff6001440d0 0x7ff60006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50854/0 (socket says 172.21.15.46:50854) 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff5ffe00640 1 -- 172.21.15.46:0/2025322881 learned_addr learned my addr 172.21.15.46:0/2025322881 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff605600640 1 --2- 172.21.15.46:0/2025322881 >> v2:172.21.15.46:3300/0 conn(0x7ff60006f150 0x7ff6000b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff604c00640 1 --2- 172.21.15.46:0/2025322881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff600143990 0x7ff6000cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff5ff400640 1 -- 172.21.15.46:0/2025322881 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505014171 0 0) 0x7ff60015e440 con 0x7ff6001440d0 2024-04-25T08:45:13.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff605600640 1 -- 172.21.15.46:0/2025322881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff600143990 msgr2=0x7ff6000cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:13.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff605600640 1 --2- 172.21.15.46:0/2025322881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff600143990 0x7ff6000cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff605600640 1 -- 172.21.15.46:0/2025322881 >> v1:172.21.15.46:6789/0 conn(0x7ff6001440d0 legacy=0x7ff60006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.486+0000 7ff605600640 1 -- 172.21.15.46:0/2025322881 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60015e9c0 con 0x7ff60006f150 2024-04-25T08:45:13.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff605600640 1 --2- 172.21.15.46:0/2025322881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60006f150 0x7ff6000b2720 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009d90 tx=0x7ff5f801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e78580123d2f850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:13.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff5ff400640 1 -- 172.21.15.46:0/2025322881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80b2070 con 0x7ff60006f150 2024-04-25T08:45:13.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff5ff400640 1 -- 172.21.15.46:0/2025322881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f80a58b0 con 0x7ff60006f150 2024-04-25T08:45:13.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff5ff400640 1 -- 172.21.15.46:0/2025322881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80a5c80 con 0x7ff60006f150 2024-04-25T08:45:13.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 -- 172.21.15.46:0/2025322881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60006f150 msgr2=0x7ff6000b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 --2- 172.21.15.46:0/2025322881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60006f150 0x7ff6000b2720 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009d90 tx=0x7ff5f801c8e0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 -- 172.21.15.46:0/2025322881 shutdown_connections 2024-04-25T08:45:13.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 --2- 172.21.15.46:0/2025322881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff60006f150 0x7ff6000b2720 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 --2- 172.21.15.46:0/2025322881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff600143990 0x7ff6000cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 -- 172.21.15.46:0/2025322881 >> 172.21.15.46:0/2025322881 conn(0x7ff60006ac10 msgr2=0x7ff60006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:13.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 -- 172.21.15.46:0/2025322881 shutdown_connections 2024-04-25T08:45:13.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 -- 172.21.15.46:0/2025322881 wait complete. 2024-04-25T08:45:13.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 Processor -- start 2024-04-25T08:45:13.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 -- start start 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff60006f150 0x7ff600159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff600143990 0x7ff6000b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6001440d0 0x7ff6000b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000b35f0 con 0x7ff600143990 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.490+0000 7ff607600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000b3470 con 0x7ff6001440d0 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff607600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000b3770 con 0x7ff60006f150 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff600143990 0x7ff6000b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff600143990 0x7ff6000b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33584/0 (socket says 172.21.15.46:33584) 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5ffe00640 1 -- 172.21.15.46:0/3153587106 learned_addr learned my addr 172.21.15.46:0/3153587106 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff605600640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6001440d0 0x7ff6000b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff604c00640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff60006f150 0x7ff600159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5ffe00640 1 -- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff60006f150 msgr2=0x7ff600159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5ffe00640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff60006f150 0x7ff600159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5ffe00640 1 -- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6001440d0 msgr2=0x7ff6000b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5ffe00640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6001440d0 0x7ff6000b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5ffe00640 1 -- 172.21.15.46:0/3153587106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60015ba50 con 0x7ff600143990 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff605600640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6001440d0 0x7ff6000b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5ffe00640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff600143990 0x7ff6000b18d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00c4b0 tx=0x7ff5ec00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3153587106 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec018070 con 0x7ff600143990 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3153587106 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec007500 con 0x7ff600143990 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff600140c80 con 0x7ff600143990 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3153587106 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec004a30 con 0x7ff600143990 2024-04-25T08:45:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff600141240 con 0x7ff600143990 2024-04-25T08:45:13.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff600069010 con 0x7ff600143990 2024-04-25T08:45:13.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.494+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3153587106 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5ec01b070 con 0x7ff600143990 2024-04-25T08:45:13.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.498+0000 7ff5fcc00640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5e0066f90 0x7ff5e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.498+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3153587106 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5ec084420 con 0x7ff600143990 2024-04-25T08:45:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.498+0000 7ff604c00640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5e0066f90 0x7ff5e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.498+0000 7ff604c00640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5e0066f90 0x7ff5e0069450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009b60 tx=0x7ff5f4009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.502+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3153587106 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5ec054bb0 con 0x7ff600143990 2024-04-25T08:45:13.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:13.702+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff60013a3f0 con 0x7ff600143990 2024-04-25T08:45:14.569 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:45:14.574+0000 7f8f01400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:45:14.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.614+0000 7ff5fcc00640 1 -- 172.21.15.46:0/3153587106 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff5ec058c50 con 0x7ff600143990 2024-04-25T08:45:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5e0066f90 msgr2=0x7ff5e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5e0066f90 0x7ff5e0069450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009b60 tx=0x7ff5f4009190 comp rx=0 tx=0).stop 2024-04-25T08:45:14.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff600143990 msgr2=0x7ff6000b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff600143990 0x7ff6000b18d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00c4b0 tx=0x7ff5ec00c970 comp rx=0 tx=0).stop 2024-04-25T08:45:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 shutdown_connections 2024-04-25T08:45:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5e0066f90 0x7ff5e0069450 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6001440d0 0x7ff6000b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff600143990 0x7ff6000b18d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 --2- 172.21.15.46:0/3153587106 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff60006f150 0x7ff600159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 >> 172.21.15.46:0/3153587106 conn(0x7ff60006ac10 msgr2=0x7ff60013d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 shutdown_connections 2024-04-25T08:45:14.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.618+0000 7ff607600640 1 -- 172.21.15.46:0/3153587106 wait complete. 2024-04-25T08:45:14.636 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:45:14.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:45:14.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.790+0000 7fbc18800640 1 Processor -- start 2024-04-25T08:45:14.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.790+0000 7fbc18800640 1 -- start start 2024-04-25T08:45:14.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc18800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc10145310 0x7fbc1006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc18800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc1006dc70 0x7fbc100cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc18800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc100cd930 con 0x7fbc100cf1b0 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc18800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc100cdbd0 con 0x7fbc1006dc70 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc18800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc100cdeb0 con 0x7fbc10145310 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc1006dc70 0x7fbc100cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc1006dc70 0x7fbc100cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33586/0 (socket says 172.21.15.46:33586) 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15400640 1 -- 172.21.15.46:0/1608464112 learned_addr learned my addr 172.21.15.46:0/1608464112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15e00640 1 --2- 172.21.15.46:0/1608464112 >> v2:172.21.15.82:3300/0 conn(0x7fbc10145310 0x7fbc1006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15400640 1 -- 172.21.15.46:0/1608464112 >> v2:172.21.15.82:3300/0 conn(0x7fbc10145310 msgr2=0x7fbc1006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15400640 1 --2- 172.21.15.46:0/1608464112 >> v2:172.21.15.82:3300/0 conn(0x7fbc10145310 0x7fbc1006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15400640 1 -- 172.21.15.46:0/1608464112 >> v1:172.21.15.82:6789/0 conn(0x7fbc100cf1b0 legacy=0x7fbc1015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15400640 1 -- 172.21.15.46:0/1608464112 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc100ce190 con 0x7fbc1006dc70 2024-04-25T08:45:14.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15e00640 1 --2- 172.21.15.46:0/1608464112 >> v2:172.21.15.82:3300/0 conn(0x7fbc10145310 0x7fbc1006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.794+0000 7fbc15400640 1 --2- 172.21.15.46:0/1608464112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1006dc70 0x7fbc100cec70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbc10072530 tx=0x7fbc0001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744b494424437e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:14.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc14a00640 1 -- 172.21.15.46:0/1608464112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000aa610 con 0x7fbc1006dc70 2024-04-25T08:45:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc14a00640 1 -- 172.21.15.46:0/1608464112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc000af070 con 0x7fbc1006dc70 2024-04-25T08:45:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc14a00640 1 -- 172.21.15.46:0/1608464112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc000aa9d0 con 0x7fbc1006dc70 2024-04-25T08:45:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 -- 172.21.15.46:0/1608464112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1006dc70 msgr2=0x7fbc100cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 --2- 172.21.15.46:0/1608464112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1006dc70 0x7fbc100cec70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbc10072530 tx=0x7fbc0001c4b0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 -- 172.21.15.46:0/1608464112 shutdown_connections 2024-04-25T08:45:14.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 --2- 172.21.15.46:0/1608464112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc1006dc70 0x7fbc100cec70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 --2- 172.21.15.46:0/1608464112 >> v2:172.21.15.82:3300/0 conn(0x7fbc10145310 0x7fbc1006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 -- 172.21.15.46:0/1608464112 >> 172.21.15.46:0/1608464112 conn(0x7fbc1006ac10 msgr2=0x7fbc1006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 -- 172.21.15.46:0/1608464112 shutdown_connections 2024-04-25T08:45:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 -- 172.21.15.46:0/1608464112 wait complete. 2024-04-25T08:45:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 Processor -- start 2024-04-25T08:45:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 -- start start 2024-04-25T08:45:14.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc100cf1b0 0x7fbc10151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc10145310 0x7fbc100b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc101377c0 0x7fbc100b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1015d920 con 0x7fbc100cf1b0 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1015d7a0 con 0x7fbc101377c0 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc18800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10168510 con 0x7fbc10145310 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc100cf1b0 0x7fbc10151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc100cf1b0 0x7fbc10151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33596/0 (socket says 172.21.15.46:33596) 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc15e00640 1 -- 172.21.15.46:0/2165594456 learned_addr learned my addr 172.21.15.46:0/2165594456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc15400640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc10145310 0x7fbc100b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.798+0000 7fbc16800640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc101377c0 0x7fbc100b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc15e00640 1 -- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc10145310 msgr2=0x7fbc100b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc15e00640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc10145310 0x7fbc100b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc15e00640 1 -- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc101377c0 msgr2=0x7fbc100b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc15e00640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc101377c0 0x7fbc100b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc15e00640 1 -- 172.21.15.46:0/2165594456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc101406a0 con 0x7fbc100cf1b0 2024-04-25T08:45:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc15e00640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc100cf1b0 0x7fbc10151200 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400c7f0 tx=0x7fbc0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc0a000640 1 -- 172.21.15.46:0/2165594456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04015070 con 0x7fbc100cf1b0 2024-04-25T08:45:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc0a000640 1 -- 172.21.15.46:0/2165594456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc04007820 con 0x7fbc100cf1b0 2024-04-25T08:45:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc0a000640 1 -- 172.21.15.46:0/2165594456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc04007c10 con 0x7fbc100cf1b0 2024-04-25T08:45:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc18800640 1 -- 172.21.15.46:0/2165594456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc10138ac0 con 0x7fbc100cf1b0 2024-04-25T08:45:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc18800640 1 -- 172.21.15.46:0/2165594456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc10140b70 con 0x7fbc100cf1b0 2024-04-25T08:45:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbbff400640 1 -- 172.21.15.46:0/2165594456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc10069010 con 0x7fbc100cf1b0 2024-04-25T08:45:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc0a000640 1 -- 172.21.15.46:0/2165594456 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc04018070 con 0x7fbc100cf1b0 2024-04-25T08:45:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc0a000640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbbf4066f60 0x7fbbf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.802+0000 7fbc0a000640 1 -- 172.21.15.46:0/2165594456 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc040852c0 con 0x7fbc100cf1b0 2024-04-25T08:45:14.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.806+0000 7fbc15400640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbbf4066f60 0x7fbbf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.806+0000 7fbc15400640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbbf4066f60 0x7fbbf4069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc000ae000 tx=0x7fbc000a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:14.806+0000 7fbc0a000640 1 -- 172.21.15.46:0/2165594456 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc04055a50 con 0x7fbc100cf1b0 2024-04-25T08:45:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.006+0000 7fbbff400640 1 -- 172.21.15.46:0/2165594456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbc100b16c0 con 0x7fbc100cf1b0 2024-04-25T08:45:15.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.618+0000 7fbc0a000640 1 -- 172.21.15.46:0/2165594456 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbc04059af0 con 0x7fbc100cf1b0 2024-04-25T08:45:15.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 -- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbbf4066f60 msgr2=0x7fbbf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbbf4066f60 0x7fbbf4069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbc000ae000 tx=0x7fbc000a7000 comp rx=0 tx=0).stop 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 -- 172.21.15.46:0/2165594456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc100cf1b0 msgr2=0x7fbc10151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc100cf1b0 0x7fbc10151200 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400c7f0 tx=0x7fbc0400ccb0 comp rx=0 tx=0).stop 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 -- 172.21.15.46:0/2165594456 shutdown_connections 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbbf4066f60 0x7fbbf4069420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc101377c0 0x7fbc100b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc10145310 0x7fbc100b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 --2- 172.21.15.46:0/2165594456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc100cf1b0 0x7fbc10151200 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 -- 172.21.15.46:0/2165594456 >> 172.21.15.46:0/2165594456 conn(0x7fbc1006ac10 msgr2=0x7fbc100b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 -- 172.21.15.46:0/2165594456 shutdown_connections 2024-04-25T08:45:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.622+0000 7fbbff400640 1 -- 172.21.15.46:0/2165594456 wait complete. 2024-04-25T08:45:15.639 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:45:15.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:45:15.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.790+0000 7ff74dc00640 1 Processor -- start 2024-04-25T08:45:15.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.790+0000 7ff74dc00640 1 -- start start 2024-04-25T08:45:15.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff74dc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff74806f570 0x7ff7480af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff74dc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74815fed0 con 0x7ff7480ad4c0 2024-04-25T08:45:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff74dc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7481601b0 con 0x7ff74815b9a0 2024-04-25T08:45:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff74dc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff74815c0f0 con 0x7ff74806f570 2024-04-25T08:45:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff747400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff74806f570 0x7ff7480af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff73e000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff7480ad4c0 0x7ff74806ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50870/0 (socket says 172.21.15.46:50870) 2024-04-25T08:45:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff73e000640 1 -- 172.21.15.46:0/3507136714 learned_addr learned my addr 172.21.15.46:0/3507136714 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff747400640 1 -- 172.21.15.46:0/3507136714 >> v1:172.21.15.82:6789/0 conn(0x7ff74815b9a0 legacy=0x7ff7480acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff747400640 1 -- 172.21.15.46:0/3507136714 >> v1:172.21.15.46:6789/0 conn(0x7ff7480ad4c0 legacy=0x7ff74806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff746000640 1 -- 172.21.15.46:0/3507136714 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904187975 0 0) 0x7ff74815fed0 con 0x7ff7480ad4c0 2024-04-25T08:45:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.794+0000 7ff747400640 1 -- 172.21.15.46:0/3507136714 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74815c3d0 con 0x7ff74806f570 2024-04-25T08:45:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff747400640 1 --2- 172.21.15.46:0/3507136714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74806f570 0x7ff7480af030 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff740009d90 tx=0x7ff74001c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d985caa9d76502c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff746000640 1 -- 172.21.15.46:0/3507136714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400b2070 con 0x7ff74806f570 2024-04-25T08:45:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff746000640 1 -- 172.21.15.46:0/3507136714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7400a5900 con 0x7ff74806f570 2024-04-25T08:45:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff746000640 1 -- 172.21.15.46:0/3507136714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7400a5cd0 con 0x7ff74806f570 2024-04-25T08:45:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff74dc00640 1 -- 172.21.15.46:0/3507136714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74806f570 msgr2=0x7ff7480af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff74dc00640 1 --2- 172.21.15.46:0/3507136714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74806f570 0x7ff7480af030 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff740009d90 tx=0x7ff74001c820 comp rx=0 tx=0).stop 2024-04-25T08:45:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff74dc00640 1 -- 172.21.15.46:0/3507136714 shutdown_connections 2024-04-25T08:45:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff74dc00640 1 --2- 172.21.15.46:0/3507136714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff74806f570 0x7ff7480af030 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff74dc00640 1 -- 172.21.15.46:0/3507136714 >> 172.21.15.46:0/3507136714 conn(0x7ff7480cfd90 msgr2=0x7ff7480d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff74dc00640 1 -- 172.21.15.46:0/3507136714 shutdown_connections 2024-04-25T08:45:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff74dc00640 1 -- 172.21.15.46:0/3507136714 wait complete. 2024-04-25T08:45:15.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff74dc00640 1 Processor -- start 2024-04-25T08:45:15.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.798+0000 7ff74dc00640 1 -- start start 2024-04-25T08:45:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff74dc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff74806f570 0x7ff7480ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff74dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7480ad4c0 0x7ff7480cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff74dc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff74815b9a0 0x7ff748152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff74dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74806b8b0 con 0x7ff7480ad4c0 2024-04-25T08:45:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff74dc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74806b730 con 0x7ff74815b9a0 2024-04-25T08:45:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff74dc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff748169f50 con 0x7ff74806f570 2024-04-25T08:45:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7480ad4c0 0x7ff7480cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7480ad4c0 0x7ff7480cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33612/0 (socket says 172.21.15.46:33612) 2024-04-25T08:45:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73e000640 1 -- 172.21.15.46:0/1496229672 learned_addr learned my addr 172.21.15.46:0/1496229672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff747400640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff74815b9a0 0x7ff748152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff746a00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff74806f570 0x7ff7480ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73e000640 1 -- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff74806f570 msgr2=0x7ff7480ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73e000640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff74806f570 0x7ff7480ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73e000640 1 -- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff74815b9a0 msgr2=0x7ff748152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73e000640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff74815b9a0 0x7ff748152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73e000640 1 -- 172.21.15.46:0/1496229672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480cf120 con 0x7ff7480ad4c0 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff746a00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff74806f570 0x7ff7480ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73e000640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7480ad4c0 0x7ff7480cd230 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c440 tx=0x7ff73800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73f400640 1 -- 172.21.15.46:0/1496229672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738018070 con 0x7ff7480ad4c0 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73f400640 1 -- 172.21.15.46:0/1496229672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7380074e0 con 0x7ff7480ad4c0 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73f400640 1 -- 172.21.15.46:0/1496229672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738004a10 con 0x7ff7480ad4c0 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7480cf400 con 0x7ff7480ad4c0 2024-04-25T08:45:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7480cf9c0 con 0x7ff7480ad4c0 2024-04-25T08:45:15.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff748069080 con 0x7ff7480ad4c0 2024-04-25T08:45:15.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.802+0000 7ff73f400640 1 -- 172.21.15.46:0/1496229672 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff73801b070 con 0x7ff7480ad4c0 2024-04-25T08:45:15.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.806+0000 7ff73f400640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff724066f90 0x7ff724069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:15.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.806+0000 7ff73f400640 1 -- 172.21.15.46:0/1496229672 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff738084360 con 0x7ff7480ad4c0 2024-04-25T08:45:15.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.806+0000 7ff746a00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff724066f90 0x7ff724069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:15.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.806+0000 7ff746a00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff724066f90 0x7ff724069450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff7481fa330 tx=0x7ff730009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:15.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:15.810+0000 7ff73f400640 1 -- 172.21.15.46:0/1496229672 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff738055270 con 0x7ff7480ad4c0 2024-04-25T08:45:15.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.002+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff748066c40 con 0x7ff7480ad4c0 2024-04-25T08:45:15.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff73f400640 1 -- 172.21.15.46:0/1496229672 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4939 (secure 0 0 0) 0x7ff738054b70 con 0x7ff7480ad4c0 2024-04-25T08:45:15.998 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:15.998 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:45:11.989847+0000","modified":"2024-04-25T08:45:15.010275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6207},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6207":{"gid":6207,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1670658295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670658295},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670658295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:45:15.998 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-04-25T08:45:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff724066f90 msgr2=0x7ff724069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff724066f90 0x7ff724069450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff7481fa330 tx=0x7ff730009040 comp rx=0 tx=0).stop 2024-04-25T08:45:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7480ad4c0 msgr2=0x7ff7480cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7480ad4c0 0x7ff7480cd230 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c440 tx=0x7ff73800c900 comp rx=0 tx=0).stop 2024-04-25T08:45:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 shutdown_connections 2024-04-25T08:45:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff724066f90 0x7ff724069450 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff74815b9a0 0x7ff748152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7480ad4c0 0x7ff7480cd230 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 --2- 172.21.15.46:0/1496229672 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff74806f570 0x7ff7480ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 >> 172.21.15.46:0/1496229672 conn(0x7ff7480cfd90 msgr2=0x7ff748157210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:16.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 shutdown_connections 2024-04-25T08:45:16.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.006+0000 7ff74dc00640 1 -- 172.21.15.46:0/1496229672 wait complete. 2024-04-25T08:45:16.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:45:16.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.166+0000 7effa2800640 1 Processor -- start 2024-04-25T08:45:16.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.166+0000 7effa2800640 1 -- start start 2024-04-25T08:45:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.166+0000 7effa2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7eff9c0af8a0 0x7eff9c15e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0c6fe0 0x7eff9c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa2800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9c06b0d0 con 0x7eff9c0cffc0 2024-04-25T08:45:16.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c06b690 con 0x7eff9c0af8a0 2024-04-25T08:45:16.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c06b3b0 con 0x7eff9c0c6fe0 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa0a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7eff9c0cffc0 0x7eff9c0c68d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50872/0 (socket says 172.21.15.46:50872) 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa0a00640 1 -- 172.21.15.46:0/2180146100 learned_addr learned my addr 172.21.15.46:0/2180146100 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa1400640 1 --2- 172.21.15.46:0/2180146100 >> v2:172.21.15.82:3300/0 conn(0x7eff9c0af8a0 0x7eff9c15e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa1e00640 1 --2- 172.21.15.46:0/2180146100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0c6fe0 0x7eff9c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7eff9be00640 1 -- 172.21.15.46:0/2180146100 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498820454 0 0) 0x7eff9c06b0d0 con 0x7eff9c0cffc0 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7eff9be00640 1 -- 172.21.15.46:0/2180146100 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff84003530 con 0x7eff9c0cffc0 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa1400640 1 -- 172.21.15.46:0/2180146100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0c6fe0 msgr2=0x7eff9c0ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa1400640 1 --2- 172.21.15.46:0/2180146100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0c6fe0 0x7eff9c0ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa1400640 1 -- 172.21.15.46:0/2180146100 >> v1:172.21.15.46:6789/0 conn(0x7eff9c0cffc0 legacy=0x7eff9c0c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa1400640 1 -- 172.21.15.46:0/2180146100 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c15e030 con 0x7eff9c0af8a0 2024-04-25T08:45:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa1e00640 1 --2- 172.21.15.46:0/2180146100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0c6fe0 0x7eff9c0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.170+0000 7effa1400640 1 --2- 172.21.15.46:0/2180146100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0af8a0 0x7eff9c15e830 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7eff900097c0 tx=0x7eff9001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a2925529dfe9f7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7eff9be00640 1 -- 172.21.15.46:0/2180146100 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900b0070 con 0x7eff9c0af8a0 2024-04-25T08:45:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7eff9be00640 1 -- 172.21.15.46:0/2180146100 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff900ab600 con 0x7eff9c0af8a0 2024-04-25T08:45:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7eff9be00640 1 -- 172.21.15.46:0/2180146100 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900ab9d0 con 0x7eff9c0af8a0 2024-04-25T08:45:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 -- 172.21.15.46:0/2180146100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0af8a0 msgr2=0x7eff9c15e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 --2- 172.21.15.46:0/2180146100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0af8a0 0x7eff9c15e830 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7eff900097c0 tx=0x7eff9001c380 comp rx=0 tx=0).stop 2024-04-25T08:45:16.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 -- 172.21.15.46:0/2180146100 shutdown_connections 2024-04-25T08:45:16.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 --2- 172.21.15.46:0/2180146100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0c6fe0 0x7eff9c0ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 --2- 172.21.15.46:0/2180146100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0af8a0 0x7eff9c15e830 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 -- 172.21.15.46:0/2180146100 >> 172.21.15.46:0/2180146100 conn(0x7eff9c099a00 msgr2=0x7eff9c0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:16.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 -- 172.21.15.46:0/2180146100 shutdown_connections 2024-04-25T08:45:16.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 -- 172.21.15.46:0/2180146100 wait complete. 2024-04-25T08:45:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 Processor -- start 2024-04-25T08:45:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 -- start start 2024-04-25T08:45:16.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff9c0af8a0 0x7eff9c145150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0c6fe0 0x7eff9c0aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0cffc0 0x7eff9c15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0b2700 con 0x7eff9c0af8a0 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c06ae20 con 0x7eff9c0c6fe0 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.174+0000 7effa2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0b2880 con 0x7eff9c0cffc0 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff9c0af8a0 0x7eff9c145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff9c0af8a0 0x7eff9c145150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33626/0 (socket says 172.21.15.46:33626) 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1400640 1 -- 172.21.15.46:0/3995192031 learned_addr learned my addr 172.21.15.46:0/3995192031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa0a00640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0c6fe0 0x7eff9c0aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1e00640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0cffc0 0x7eff9c15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1400640 1 -- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0cffc0 msgr2=0x7eff9c15a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1400640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0cffc0 0x7eff9c15a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1400640 1 -- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0c6fe0 msgr2=0x7eff9c0aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1400640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0c6fe0 0x7eff9c0aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1400640 1 -- 172.21.15.46:0/3995192031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c0b08d0 con 0x7eff9c0af8a0 2024-04-25T08:45:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa1400640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff9c0af8a0 0x7eff9c145150 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7eff900097c0 tx=0x7eff90019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7eff99600640 1 -- 172.21.15.46:0/3995192031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900b0040 con 0x7eff9c0af8a0 2024-04-25T08:45:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7eff99600640 1 -- 172.21.15.46:0/3995192031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff900b9070 con 0x7eff9c0af8a0 2024-04-25T08:45:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7eff99600640 1 -- 172.21.15.46:0/3995192031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900b4650 con 0x7eff9c0af8a0 2024-04-25T08:45:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c0b0bb0 con 0x7eff9c0af8a0 2024-04-25T08:45:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c0b31d0 con 0x7eff9c0af8a0 2024-04-25T08:45:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c0b0ef0 con 0x7eff9c0af8a0 2024-04-25T08:45:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7eff99600640 1 -- 172.21.15.46:0/3995192031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff900b47f0 con 0x7eff9c0af8a0 2024-04-25T08:45:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7eff99600640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff7c066f60 0x7eff7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.178+0000 7eff99600640 1 -- 172.21.15.46:0/3995192031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7eff9011ef30 con 0x7eff9c0af8a0 2024-04-25T08:45:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.182+0000 7effa0a00640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff7c066f60 0x7eff7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.182+0000 7effa0a00640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff7c066f60 0x7eff7c069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0b1220 tx=0x7eff88009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:16.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.182+0000 7eff99600640 1 -- 172.21.15.46:0/3995192031 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff90016090 con 0x7eff9c0af8a0 2024-04-25T08:45:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.358+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff9c069010 con 0x7eff9c0af8a0 2024-04-25T08:45:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.358+0000 7eff99600640 1 -- 172.21.15.46:0/3995192031 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16836 (secure 0 0 0) 0x7eff900efd10 con 0x7eff9c0af8a0 2024-04-25T08:45:16.352 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:16.353 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:45:13.608590+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:45:10.412423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:45:10.987443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff7c066f60 msgr2=0x7eff7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff7c066f60 0x7eff7c069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0b1220 tx=0x7eff88009040 comp rx=0 tx=0).stop 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff9c0af8a0 msgr2=0x7eff9c145150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff9c0af8a0 0x7eff9c145150 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7eff900097c0 tx=0x7eff90019120 comp rx=0 tx=0).stop 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 shutdown_connections 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7eff7c066f60 0x7eff7c069420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7eff9c0cffc0 0x7eff9c15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff9c0c6fe0 0x7eff9c0aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 --2- 172.21.15.46:0/3995192031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff9c0af8a0 0x7eff9c145150 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 >> 172.21.15.46:0/3995192031 conn(0x7eff9c099a00 msgr2=0x7eff9c160630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 shutdown_connections 2024-04-25T08:45:16.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.362+0000 7effa2800640 1 -- 172.21.15.46:0/3995192031 wait complete. 2024-04-25T08:45:16.374 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:45:16.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:45:16.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff223e00640 1 Processor -- start 2024-04-25T08:45:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff223e00640 1 -- start start 2024-04-25T08:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff223e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c15bdb0 0x7ff21c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff223e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff21c0cda80 0x7ff21c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff223e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff21c0cc700 con 0x7ff21c159230 2024-04-25T08:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff223e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21c0cc9e0 con 0x7ff21c0cda80 2024-04-25T08:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff223e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0cccc0 con 0x7ff21c15bdb0 2024-04-25T08:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff221e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff21c0cda80 0x7ff21c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff221400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff21c159230 0x7ff21c0cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50874/0 (socket says 172.21.15.46:50874) 2024-04-25T08:45:16.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.522+0000 7ff221400640 1 -- 172.21.15.46:0/2373414483 learned_addr learned my addr 172.21.15.46:0/2373414483 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff220a00640 1 --2- 172.21.15.46:0/2373414483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c15bdb0 0x7ff21c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff221e00640 1 -- 172.21.15.46:0/2373414483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c15bdb0 msgr2=0x7ff21c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff221e00640 1 --2- 172.21.15.46:0/2373414483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c15bdb0 0x7ff21c0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff221e00640 1 -- 172.21.15.46:0/2373414483 >> v1:172.21.15.46:6789/0 conn(0x7ff21c159230 legacy=0x7ff21c0cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff221e00640 1 -- 172.21.15.46:0/2373414483 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0ccfa0 con 0x7ff21c0cda80 2024-04-25T08:45:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff221e00640 1 --2- 172.21.15.46:0/2373414483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c0cda80 0x7ff21c15d200 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff214009d90 tx=0x7ff21401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa17758ecbf3fb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff21be00640 1 -- 172.21.15.46:0/2373414483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140b2070 con 0x7ff21c0cda80 2024-04-25T08:45:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff21be00640 1 -- 172.21.15.46:0/2373414483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2140a57a0 con 0x7ff21c0cda80 2024-04-25T08:45:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff21be00640 1 -- 172.21.15.46:0/2373414483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140a5b70 con 0x7ff21c0cda80 2024-04-25T08:45:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff223e00640 1 -- 172.21.15.46:0/2373414483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c0cda80 msgr2=0x7ff21c15d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff223e00640 1 --2- 172.21.15.46:0/2373414483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c0cda80 0x7ff21c15d200 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff214009d90 tx=0x7ff21401c8e0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff223e00640 1 -- 172.21.15.46:0/2373414483 shutdown_connections 2024-04-25T08:45:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff223e00640 1 --2- 172.21.15.46:0/2373414483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c0cda80 0x7ff21c15d200 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff223e00640 1 --2- 172.21.15.46:0/2373414483 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c15bdb0 0x7ff21c0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.526+0000 7ff223e00640 1 -- 172.21.15.46:0/2373414483 >> 172.21.15.46:0/2373414483 conn(0x7ff21c0b12a0 msgr2=0x7ff21c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 -- 172.21.15.46:0/2373414483 shutdown_connections 2024-04-25T08:45:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 -- 172.21.15.46:0/2373414483 wait complete. 2024-04-25T08:45:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 Processor -- start 2024-04-25T08:45:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 -- start start 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cda80 0x7ff21c0ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c159230 0x7ff21c0ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c15bdb0 0x7ff21c0ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c15e2e0 con 0x7ff21c15bdb0 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c15e160 con 0x7ff21c0cda80 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff223e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c15e460 con 0x7ff21c159230 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c15bdb0 0x7ff21c0ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c15bdb0 0x7ff21c0ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33640/0 (socket says 172.21.15.46:33640) 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221e00640 1 -- 172.21.15.46:0/230107009 learned_addr learned my addr 172.21.15.46:0/230107009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff220a00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c159230 0x7ff21c0ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221400640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cda80 0x7ff21c0ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221e00640 1 -- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c159230 msgr2=0x7ff21c0ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221e00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c159230 0x7ff21c0ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221e00640 1 -- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cda80 msgr2=0x7ff21c0ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221e00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cda80 0x7ff21c0ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221e00640 1 -- 172.21.15.46:0/230107009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c15a930 con 0x7ff21c15bdb0 2024-04-25T08:45:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff221e00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c15bdb0 0x7ff21c0ae7b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff214009d90 tx=0x7ff214019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff219600640 1 -- 172.21.15.46:0/230107009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140b2050 con 0x7ff21c15bdb0 2024-04-25T08:45:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff219600640 1 -- 172.21.15.46:0/230107009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2140be070 con 0x7ff21c15bdb0 2024-04-25T08:45:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.530+0000 7ff219600640 1 -- 172.21.15.46:0/230107009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140ad870 con 0x7ff21c15bdb0 2024-04-25T08:45:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.534+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21c15ac10 con 0x7ff21c15bdb0 2024-04-25T08:45:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.534+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c15b1d0 con 0x7ff21c15bdb0 2024-04-25T08:45:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.534+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21c069080 con 0x7ff21c15bdb0 2024-04-25T08:45:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.534+0000 7ff219600640 1 -- 172.21.15.46:0/230107009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2140ada10 con 0x7ff21c15bdb0 2024-04-25T08:45:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.534+0000 7ff219600640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1fc066f60 0x7ff1fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.534+0000 7ff219600640 1 -- 172.21.15.46:0/230107009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff214120300 con 0x7ff21c15bdb0 2024-04-25T08:45:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.534+0000 7ff221400640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1fc066f60 0x7ff1fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.534+0000 7ff221400640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1fc066f60 0x7ff1fc069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff210000f70 tx=0x7ff210009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:16.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.538+0000 7ff219600640 1 -- 172.21.15.46:0/230107009 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff214016560 con 0x7ff21c15bdb0 2024-04-25T08:45:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.698+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff21c068520 con 0x7ff1fc066f60 2024-04-25T08:45:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.706+0000 7ff219600640 1 -- 172.21.15.46:0/230107009 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246711 (secure 0 0 0) 0x7ff2100058a0 con 0x7ff1fc066f60 2024-04-25T08:45:16.702 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1fc066f60 msgr2=0x7ff1fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1fc066f60 0x7ff1fc069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff210000f70 tx=0x7ff210009040 comp rx=0 tx=0).stop 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c15bdb0 msgr2=0x7ff21c0ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c15bdb0 0x7ff21c0ae7b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff214009d90 tx=0x7ff214019680 comp rx=0 tx=0).stop 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 shutdown_connections 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff1fc066f60 0x7ff1fc069420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff21c15bdb0 0x7ff21c0ae7b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff21c159230 0x7ff21c0ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 --2- 172.21.15.46:0/230107009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cda80 0x7ff21c0ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 >> 172.21.15.46:0/230107009 conn(0x7ff21c0b12a0 msgr2=0x7ff21c0b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 shutdown_connections 2024-04-25T08:45:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.710+0000 7ff223e00640 1 -- 172.21.15.46:0/230107009 wait complete. 2024-04-25T08:45:16.722 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-25T08:45:16.297142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244080,"kb_used_data":7088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730640,"statfs":{"total":773094113280,"available":772844175360,"internally_reserved":0,"allocated":7258112,"data_stored":3901378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.713434"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596202+0000","last_change":"2024-04-25T08:45:11.596202+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596202+0000","last_undegraded":"2024-04-25T08:45:11.596202+0000","last_fullsized":"2024-04-25T08:45:11.596202+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595066+0000","last_change":"2024-04-25T08:45:11.595066+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595066+0000","last_undegraded":"2024-04-25T08:45:11.595066+0000","last_fullsized":"2024-04-25T08:45:11.595066+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592923+0000","last_change":"2024-04-25T08:45:11.592923+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592923+0000","last_undegraded":"2024-04-25T08:45:11.592923+0000","last_fullsized":"2024-04-25T08:45:11.592923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599420+0000","last_change":"2024-04-25T08:45:11.599420+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599420+0000","last_undegraded":"2024-04-25T08:45:11.599420+0000","last_fullsized":"2024-04-25T08:45:11.599420+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594152+0000","last_change":"2024-04-25T08:45:11.594152+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594152+0000","last_undegraded":"2024-04-25T08:45:11.594152+0000","last_fullsized":"2024-04-25T08:45:11.594152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600272+0000","last_change":"2024-04-25T08:45:11.600272+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600272+0000","last_undegraded":"2024-04-25T08:45:11.600272+0000","last_fullsized":"2024-04-25T08:45:11.600272+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.605419+0000","last_change":"2024-04-25T08:45:11.605419+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.605419+0000","last_undegraded":"2024-04-25T08:45:11.605419+0000","last_fullsized":"2024-04-25T08:45:11.605419+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602566+0000","last_change":"2024-04-25T08:45:11.602566+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602566+0000","last_undegraded":"2024-04-25T08:45:11.602566+0000","last_fullsized":"2024-04-25T08:45:11.602566+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593504+0000","last_change":"2024-04-25T08:45:11.593504+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593504+0000","last_undegraded":"2024-04-25T08:45:11.593504+0000","last_fullsized":"2024-04-25T08:45:11.593504+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592774+0000","last_change":"2024-04-25T08:45:11.592774+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592774+0000","last_undegraded":"2024-04-25T08:45:11.592774+0000","last_fullsized":"2024-04-25T08:45:11.592774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600340+0000","last_change":"2024-04-25T08:45:11.600340+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600340+0000","last_undegraded":"2024-04-25T08:45:11.600340+0000","last_fullsized":"2024-04-25T08:45:11.600340+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.608903+0000","last_change":"2024-04-25T08:45:11.608903+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.608903+0000","last_undegraded":"2024-04-25T08:45:11.608903+0000","last_fullsized":"2024-04-25T08:45:11.608903+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604995+0000","last_change":"2024-04-25T08:45:11.604995+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604995+0000","last_undegraded":"2024-04-25T08:45:11.604995+0000","last_fullsized":"2024-04-25T08:45:11.604995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593267+0000","last_change":"2024-04-25T08:45:11.593267+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593267+0000","last_undegraded":"2024-04-25T08:45:11.593267+0000","last_fullsized":"2024-04-25T08:45:11.593267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594640+0000","last_change":"2024-04-25T08:45:11.594640+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594640+0000","last_undegraded":"2024-04-25T08:45:11.594640+0000","last_fullsized":"2024-04-25T08:45:11.594640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.606433+0000","last_change":"2024-04-25T08:45:11.606433+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.606433+0000","last_undegraded":"2024-04-25T08:45:11.606433+0000","last_fullsized":"2024-04-25T08:45:11.606433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595245+0000","last_change":"2024-04-25T08:45:11.595245+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595245+0000","last_undegraded":"2024-04-25T08:45:11.595245+0000","last_fullsized":"2024-04-25T08:45:11.595245+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601808+0000","last_change":"2024-04-25T08:45:11.601808+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601808+0000","last_undegraded":"2024-04-25T08:45:11.601808+0000","last_fullsized":"2024-04-25T08:45:11.601808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594375+0000","last_change":"2024-04-25T08:45:11.594375+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594375+0000","last_undegraded":"2024-04-25T08:45:11.594375+0000","last_fullsized":"2024-04-25T08:45:11.594375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593868+0000","last_change":"2024-04-25T08:45:11.593868+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593868+0000","last_undegraded":"2024-04-25T08:45:11.593868+0000","last_fullsized":"2024-04-25T08:45:11.593868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604594+0000","last_change":"2024-04-25T08:45:11.604594+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604594+0000","last_undegraded":"2024-04-25T08:45:11.604594+0000","last_fullsized":"2024-04-25T08:45:11.604594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593331+0000","last_change":"2024-04-25T08:45:11.593331+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593331+0000","last_undegraded":"2024-04-25T08:45:11.593331+0000","last_fullsized":"2024-04-25T08:45:11.593331+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593128+0000","last_change":"2024-04-25T08:45:11.593128+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593128+0000","last_undegraded":"2024-04-25T08:45:11.593128+0000","last_fullsized":"2024-04-25T08:45:11.593128+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592664+0000","last_change":"2024-04-25T08:45:11.592664+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592664+0000","last_undegraded":"2024-04-25T08:45:11.592664+0000","last_fullsized":"2024-04-25T08:45:11.592664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604961+0000","last_change":"2024-04-25T08:45:11.604961+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604961+0000","last_undegraded":"2024-04-25T08:45:11.604961+0000","last_fullsized":"2024-04-25T08:45:11.604961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.608708+0000","last_change":"2024-04-25T08:45:11.608708+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.608708+0000","last_undegraded":"2024-04-25T08:45:11.608708+0000","last_fullsized":"2024-04-25T08:45:11.608708+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599852+0000","last_change":"2024-04-25T08:45:11.599852+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599852+0000","last_undegraded":"2024-04-25T08:45:11.599852+0000","last_fullsized":"2024-04-25T08:45:11.599852+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595481+0000","last_change":"2024-04-25T08:45:11.595481+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595481+0000","last_undegraded":"2024-04-25T08:45:11.595481+0000","last_fullsized":"2024-04-25T08:45:11.595481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592683+0000","last_change":"2024-04-25T08:45:11.592683+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592683+0000","last_undegraded":"2024-04-25T08:45:11.592683+0000","last_fullsized":"2024-04-25T08:45:11.592683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604336+0000","last_change":"2024-04-25T08:45:11.604336+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604336+0000","last_undegraded":"2024-04-25T08:45:11.604336+0000","last_fullsized":"2024-04-25T08:45:11.604336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.598275+0000","last_change":"2024-04-25T08:45:11.598275+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.598275+0000","last_undegraded":"2024-04-25T08:45:11.598275+0000","last_fullsized":"2024-04-25T08:45:11.598275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602360+0000","last_change":"2024-04-25T08:45:11.602360+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602360+0000","last_undegraded":"2024-04-25T08:45:11.602360+0000","last_fullsized":"2024-04-25T08:45:11.602360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593787+0000","last_change":"2024-04-25T08:45:11.593787+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593787+0000","last_undegraded":"2024-04-25T08:45:11.593787+0000","last_fullsized":"2024-04-25T08:45:11.593787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618446+0000","last_change":"2024-04-25T08:45:11.616357+0000","last_active":"2024-04-25T08:45:11.618446+0000","last_peered":"2024-04-25T08:45:11.618446+0000","last_clean":"2024-04-25T08:45:11.618446+0000","last_became_active":"2024-04-25T08:45:11.612904+0000","last_became_peered":"2024-04-25T08:45:11.612904+0000","last_unstale":"2024-04-25T08:45:11.618446+0000","last_undegraded":"2024-04-25T08:45:11.618446+0000","last_fullsized":"2024-04-25T08:45:11.618446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599374+0000","last_change":"2024-04-25T08:45:11.599374+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599374+0000","last_undegraded":"2024-04-25T08:45:11.599374+0000","last_fullsized":"2024-04-25T08:45:11.599374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614412+0000","last_change":"2024-04-25T08:45:11.609407+0000","last_active":"2024-04-25T08:45:11.614412+0000","last_peered":"2024-04-25T08:45:11.614412+0000","last_clean":"2024-04-25T08:45:11.614412+0000","last_became_active":"2024-04-25T08:45:11.609129+0000","last_became_peered":"2024-04-25T08:45:11.609129+0000","last_unstale":"2024-04-25T08:45:11.614412+0000","last_undegraded":"2024-04-25T08:45:11.614412+0000","last_fullsized":"2024-04-25T08:45:11.614412+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.303293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603765+0000","last_change":"2024-04-25T08:45:11.603765+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603765+0000","last_undegraded":"2024-04-25T08:45:11.603765+0000","last_fullsized":"2024-04-25T08:45:11.603765+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617069+0000","last_change":"2024-04-25T08:45:11.615084+0000","last_active":"2024-04-25T08:45:11.617069+0000","last_peered":"2024-04-25T08:45:11.617069+0000","last_clean":"2024-04-25T08:45:11.617069+0000","last_became_active":"2024-04-25T08:45:11.614747+0000","last_became_peered":"2024-04-25T08:45:11.614747+0000","last_unstale":"2024-04-25T08:45:11.617069+0000","last_undegraded":"2024-04-25T08:45:11.617069+0000","last_fullsized":"2024-04-25T08:45:11.617069+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.879227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594756+0000","last_change":"2024-04-25T08:45:11.594756+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594756+0000","last_undegraded":"2024-04-25T08:45:11.594756+0000","last_fullsized":"2024-04-25T08:45:11.594756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614851+0000","last_change":"2024-04-25T08:45:11.610526+0000","last_active":"2024-04-25T08:45:11.614851+0000","last_peered":"2024-04-25T08:45:11.614851+0000","last_clean":"2024-04-25T08:45:11.614851+0000","last_became_active":"2024-04-25T08:45:11.610262+0000","last_became_peered":"2024-04-25T08:45:11.610262+0000","last_unstale":"2024-04-25T08:45:11.614851+0000","last_undegraded":"2024-04-25T08:45:11.614851+0000","last_fullsized":"2024-04-25T08:45:11.614851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:35.392449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603978+0000","last_change":"2024-04-25T08:45:11.603978+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603978+0000","last_undegraded":"2024-04-25T08:45:11.603978+0000","last_fullsized":"2024-04-25T08:45:11.603978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614866+0000","last_change":"2024-04-25T08:45:11.608750+0000","last_active":"2024-04-25T08:45:11.614866+0000","last_peered":"2024-04-25T08:45:11.614866+0000","last_clean":"2024-04-25T08:45:11.614866+0000","last_became_active":"2024-04-25T08:45:11.608428+0000","last_became_peered":"2024-04-25T08:45:11.608428+0000","last_unstale":"2024-04-25T08:45:11.614866+0000","last_undegraded":"2024-04-25T08:45:11.614866+0000","last_fullsized":"2024-04-25T08:45:11.614866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:35.847974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594081+0000","last_change":"2024-04-25T08:45:11.594081+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594081+0000","last_undegraded":"2024-04-25T08:45:11.594081+0000","last_fullsized":"2024-04-25T08:45:11.594081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614545+0000","last_change":"2024-04-25T08:45:11.611352+0000","last_active":"2024-04-25T08:45:11.614545+0000","last_peered":"2024-04-25T08:45:11.614545+0000","last_clean":"2024-04-25T08:45:11.614545+0000","last_became_active":"2024-04-25T08:45:11.611094+0000","last_became_peered":"2024-04-25T08:45:11.611094+0000","last_unstale":"2024-04-25T08:45:11.614545+0000","last_undegraded":"2024-04-25T08:45:11.614545+0000","last_fullsized":"2024-04-25T08:45:11.614545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:36.596184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.597842+0000","last_change":"2024-04-25T08:45:11.597842+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.597842+0000","last_undegraded":"2024-04-25T08:45:11.597842+0000","last_fullsized":"2024-04-25T08:45:11.597842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.609031+0000","last_change":"2024-04-25T08:45:11.604893+0000","last_active":"2024-04-25T08:45:11.609031+0000","last_peered":"2024-04-25T08:45:11.609031+0000","last_clean":"2024-04-25T08:45:11.609031+0000","last_became_active":"2024-04-25T08:45:11.603555+0000","last_became_peered":"2024-04-25T08:45:11.603555+0000","last_unstale":"2024-04-25T08:45:11.609031+0000","last_undegraded":"2024-04-25T08:45:11.609031+0000","last_fullsized":"2024-04-25T08:45:11.609031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:20.076400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593075+0000","last_change":"2024-04-25T08:45:11.593075+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593075+0000","last_undegraded":"2024-04-25T08:45:11.593075+0000","last_fullsized":"2024-04-25T08:45:11.593075+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.616463+0000","last_change":"2024-04-25T08:45:11.613393+0000","last_active":"2024-04-25T08:45:11.616463+0000","last_peered":"2024-04-25T08:45:11.616463+0000","last_clean":"2024-04-25T08:45:11.616463+0000","last_became_active":"2024-04-25T08:45:11.611939+0000","last_became_peered":"2024-04-25T08:45:11.611939+0000","last_unstale":"2024-04-25T08:45:11.616463+0000","last_undegraded":"2024-04-25T08:45:11.616463+0000","last_fullsized":"2024-04-25T08:45:11.616463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:33.153694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594910+0000","last_change":"2024-04-25T08:45:11.594910+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594910+0000","last_undegraded":"2024-04-25T08:45:11.594910+0000","last_fullsized":"2024-04-25T08:45:11.594910+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615906+0000","last_change":"2024-04-25T08:45:11.611641+0000","last_active":"2024-04-25T08:45:11.615906+0000","last_peered":"2024-04-25T08:45:11.615906+0000","last_clean":"2024-04-25T08:45:11.615906+0000","last_became_active":"2024-04-25T08:45:11.611380+0000","last_became_peered":"2024-04-25T08:45:11.611380+0000","last_unstale":"2024-04-25T08:45:11.615906+0000","last_undegraded":"2024-04-25T08:45:11.615906+0000","last_fullsized":"2024-04-25T08:45:11.615906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:57.910041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593106+0000","last_change":"2024-04-25T08:45:11.593106+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593106+0000","last_undegraded":"2024-04-25T08:45:11.593106+0000","last_fullsized":"2024-04-25T08:45:11.593106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615489+0000","last_change":"2024-04-25T08:45:11.612291+0000","last_active":"2024-04-25T08:45:11.615489+0000","last_peered":"2024-04-25T08:45:11.615489+0000","last_clean":"2024-04-25T08:45:11.615489+0000","last_became_active":"2024-04-25T08:45:11.612016+0000","last_became_peered":"2024-04-25T08:45:11.612016+0000","last_unstale":"2024-04-25T08:45:11.615489+0000","last_undegraded":"2024-04-25T08:45:11.615489+0000","last_fullsized":"2024-04-25T08:45:11.615489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:28.466919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594590+0000","last_change":"2024-04-25T08:45:11.594590+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594590+0000","last_undegraded":"2024-04-25T08:45:11.594590+0000","last_fullsized":"2024-04-25T08:45:11.594590+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615757+0000","last_change":"2024-04-25T08:45:11.612056+0000","last_active":"2024-04-25T08:45:11.615757+0000","last_peered":"2024-04-25T08:45:11.615757+0000","last_clean":"2024-04-25T08:45:11.615757+0000","last_became_active":"2024-04-25T08:45:11.611677+0000","last_became_peered":"2024-04-25T08:45:11.611677+0000","last_unstale":"2024-04-25T08:45:11.615757+0000","last_undegraded":"2024-04-25T08:45:11.615757+0000","last_fullsized":"2024-04-25T08:45:11.615757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:29.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602391+0000","last_change":"2024-04-25T08:45:11.602391+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602391+0000","last_undegraded":"2024-04-25T08:45:11.602391+0000","last_fullsized":"2024-04-25T08:45:11.602391+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617146+0000","last_change":"2024-04-25T08:45:11.615380+0000","last_active":"2024-04-25T08:45:11.617146+0000","last_peered":"2024-04-25T08:45:11.617146+0000","last_clean":"2024-04-25T08:45:11.617146+0000","last_became_active":"2024-04-25T08:45:11.612863+0000","last_became_peered":"2024-04-25T08:45:11.612863+0000","last_unstale":"2024-04-25T08:45:11.617146+0000","last_undegraded":"2024-04-25T08:45:11.617146+0000","last_fullsized":"2024-04-25T08:45:11.617146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599723+0000","last_change":"2024-04-25T08:45:11.599723+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599723+0000","last_undegraded":"2024-04-25T08:45:11.599723+0000","last_fullsized":"2024-04-25T08:45:11.599723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614344+0000","last_change":"2024-04-25T08:45:11.609636+0000","last_active":"2024-04-25T08:45:11.614344+0000","last_peered":"2024-04-25T08:45:11.614344+0000","last_clean":"2024-04-25T08:45:11.614344+0000","last_became_active":"2024-04-25T08:45:11.609296+0000","last_became_peered":"2024-04-25T08:45:11.609296+0000","last_unstale":"2024-04-25T08:45:11.614344+0000","last_undegraded":"2024-04-25T08:45:11.614344+0000","last_fullsized":"2024-04-25T08:45:11.614344+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603834+0000","last_change":"2024-04-25T08:45:11.603834+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603834+0000","last_undegraded":"2024-04-25T08:45:11.603834+0000","last_fullsized":"2024-04-25T08:45:11.603834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.619721+0000","last_change":"2024-04-25T08:45:11.618604+0000","last_active":"2024-04-25T08:45:11.619721+0000","last_peered":"2024-04-25T08:45:11.619721+0000","last_clean":"2024-04-25T08:45:11.619721+0000","last_became_active":"2024-04-25T08:45:11.618224+0000","last_became_peered":"2024-04-25T08:45:11.618224+0000","last_unstale":"2024-04-25T08:45:11.619721+0000","last_undegraded":"2024-04-25T08:45:11.619721+0000","last_fullsized":"2024-04-25T08:45:11.619721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:57.910041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596991+0000","last_change":"2024-04-25T08:45:11.596991+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596991+0000","last_undegraded":"2024-04-25T08:45:11.596991+0000","last_fullsized":"2024-04-25T08:45:11.596991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615681+0000","last_change":"2024-04-25T08:45:11.611078+0000","last_active":"2024-04-25T08:45:11.615681+0000","last_peered":"2024-04-25T08:45:11.615681+0000","last_clean":"2024-04-25T08:45:11.615681+0000","last_became_active":"2024-04-25T08:45:11.610749+0000","last_became_peered":"2024-04-25T08:45:11.610749+0000","last_unstale":"2024-04-25T08:45:11.615681+0000","last_undegraded":"2024-04-25T08:45:11.615681+0000","last_fullsized":"2024-04-25T08:45:11.615681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.517500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594513+0000","last_change":"2024-04-25T08:45:11.594513+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594513+0000","last_undegraded":"2024-04-25T08:45:11.594513+0000","last_fullsized":"2024-04-25T08:45:11.594513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615892+0000","last_change":"2024-04-25T08:45:11.613128+0000","last_active":"2024-04-25T08:45:11.615892+0000","last_peered":"2024-04-25T08:45:11.615892+0000","last_clean":"2024-04-25T08:45:11.615892+0000","last_became_active":"2024-04-25T08:45:11.611609+0000","last_became_peered":"2024-04-25T08:45:11.611609+0000","last_unstale":"2024-04-25T08:45:11.615892+0000","last_undegraded":"2024-04-25T08:45:11.615892+0000","last_fullsized":"2024-04-25T08:45:11.615892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:51.214222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594748+0000","last_change":"2024-04-25T08:45:11.594748+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594748+0000","last_undegraded":"2024-04-25T08:45:11.594748+0000","last_fullsized":"2024-04-25T08:45:11.594748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.619710+0000","last_change":"2024-04-25T08:45:11.618549+0000","last_active":"2024-04-25T08:45:11.619710+0000","last_peered":"2024-04-25T08:45:11.619710+0000","last_clean":"2024-04-25T08:45:11.619710+0000","last_became_active":"2024-04-25T08:45:11.618187+0000","last_became_peered":"2024-04-25T08:45:11.618187+0000","last_unstale":"2024-04-25T08:45:11.619710+0000","last_undegraded":"2024-04-25T08:45:11.619710+0000","last_fullsized":"2024-04-25T08:45:11.619710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.517500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.598619+0000","last_change":"2024-04-25T08:45:11.598619+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.598619+0000","last_undegraded":"2024-04-25T08:45:11.598619+0000","last_fullsized":"2024-04-25T08:45:11.598619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614216+0000","last_change":"2024-04-25T08:45:11.609242+0000","last_active":"2024-04-25T08:45:11.614216+0000","last_peered":"2024-04-25T08:45:11.614216+0000","last_clean":"2024-04-25T08:45:11.614216+0000","last_became_active":"2024-04-25T08:45:11.608930+0000","last_became_peered":"2024-04-25T08:45:11.608930+0000","last_unstale":"2024-04-25T08:45:11.614216+0000","last_undegraded":"2024-04-25T08:45:11.614216+0000","last_fullsized":"2024-04-25T08:45:11.614216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601349+0000","last_change":"2024-04-25T08:45:11.601349+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601349+0000","last_undegraded":"2024-04-25T08:45:11.601349+0000","last_fullsized":"2024-04-25T08:45:11.601349+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.613852+0000","last_change":"2024-04-25T08:45:11.608806+0000","last_active":"2024-04-25T08:45:11.613852+0000","last_peered":"2024-04-25T08:45:11.613852+0000","last_clean":"2024-04-25T08:45:11.613852+0000","last_became_active":"2024-04-25T08:45:11.607645+0000","last_became_peered":"2024-04-25T08:45:11.607645+0000","last_unstale":"2024-04-25T08:45:11.613852+0000","last_undegraded":"2024-04-25T08:45:11.613852+0000","last_fullsized":"2024-04-25T08:45:11.613852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:35.633342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596595+0000","last_change":"2024-04-25T08:45:11.596595+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596595+0000","last_undegraded":"2024-04-25T08:45:11.596595+0000","last_fullsized":"2024-04-25T08:45:11.596595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.607835+0000","last_change":"2024-04-25T08:45:11.605238+0000","last_active":"2024-04-25T08:45:11.607835+0000","last_peered":"2024-04-25T08:45:11.607835+0000","last_clean":"2024-04-25T08:45:11.607835+0000","last_became_active":"2024-04-25T08:45:11.604975+0000","last_became_peered":"2024-04-25T08:45:11.604975+0000","last_unstale":"2024-04-25T08:45:11.607835+0000","last_undegraded":"2024-04-25T08:45:11.607835+0000","last_fullsized":"2024-04-25T08:45:11.607835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.879227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601918+0000","last_change":"2024-04-25T08:45:11.601918+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601918+0000","last_undegraded":"2024-04-25T08:45:11.601918+0000","last_fullsized":"2024-04-25T08:45:11.601918+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618499+0000","last_change":"2024-04-25T08:45:11.616493+0000","last_active":"2024-04-25T08:45:11.618499+0000","last_peered":"2024-04-25T08:45:11.618499+0000","last_clean":"2024-04-25T08:45:11.618499+0000","last_became_active":"2024-04-25T08:45:11.616241+0000","last_became_peered":"2024-04-25T08:45:11.616241+0000","last_unstale":"2024-04-25T08:45:11.618499+0000","last_undegraded":"2024-04-25T08:45:11.618499+0000","last_fullsized":"2024-04-25T08:45:11.618499+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:35.847974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603447+0000","last_change":"2024-04-25T08:45:11.603447+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603447+0000","last_undegraded":"2024-04-25T08:45:11.603447+0000","last_fullsized":"2024-04-25T08:45:11.603447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617745+0000","last_change":"2024-04-25T08:45:11.615924+0000","last_active":"2024-04-25T08:45:11.617745+0000","last_peered":"2024-04-25T08:45:11.617745+0000","last_clean":"2024-04-25T08:45:11.617745+0000","last_became_active":"2024-04-25T08:45:11.613543+0000","last_became_peered":"2024-04-25T08:45:11.613543+0000","last_unstale":"2024-04-25T08:45:11.617745+0000","last_undegraded":"2024-04-25T08:45:11.617745+0000","last_fullsized":"2024-04-25T08:45:11.617745+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593904+0000","last_change":"2024-04-25T08:45:11.593904+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593904+0000","last_undegraded":"2024-04-25T08:45:11.593904+0000","last_fullsized":"2024-04-25T08:45:11.593904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617738+0000","last_change":"2024-04-25T08:45:11.613996+0000","last_active":"2024-04-25T08:45:11.617738+0000","last_peered":"2024-04-25T08:45:11.617738+0000","last_clean":"2024-04-25T08:45:11.617738+0000","last_became_active":"2024-04-25T08:45:11.613688+0000","last_became_peered":"2024-04-25T08:45:11.613688+0000","last_unstale":"2024-04-25T08:45:11.617738+0000","last_undegraded":"2024-04-25T08:45:11.617738+0000","last_fullsized":"2024-04-25T08:45:11.617738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:20.076400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600734+0000","last_change":"2024-04-25T08:45:11.600734+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600734+0000","last_undegraded":"2024-04-25T08:45:11.600734+0000","last_fullsized":"2024-04-25T08:45:11.600734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618392+0000","last_change":"2024-04-25T08:45:11.616861+0000","last_active":"2024-04-25T08:45:11.618392+0000","last_peered":"2024-04-25T08:45:11.618392+0000","last_clean":"2024-04-25T08:45:11.618392+0000","last_became_active":"2024-04-25T08:45:11.616539+0000","last_became_peered":"2024-04-25T08:45:11.616539+0000","last_unstale":"2024-04-25T08:45:11.618392+0000","last_undegraded":"2024-04-25T08:45:11.618392+0000","last_fullsized":"2024-04-25T08:45:11.618392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.606983+0000","last_change":"2024-04-25T08:45:11.606983+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.606983+0000","last_undegraded":"2024-04-25T08:45:11.606983+0000","last_fullsized":"2024-04-25T08:45:11.606983+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617742+0000","last_change":"2024-04-25T08:45:11.616486+0000","last_active":"2024-04-25T08:45:11.617742+0000","last_peered":"2024-04-25T08:45:11.617742+0000","last_clean":"2024-04-25T08:45:11.617742+0000","last_became_active":"2024-04-25T08:45:11.614742+0000","last_became_peered":"2024-04-25T08:45:11.614742+0000","last_unstale":"2024-04-25T08:45:11.617742+0000","last_undegraded":"2024-04-25T08:45:11.617742+0000","last_fullsized":"2024-04-25T08:45:11.617742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.605924+0000","last_change":"2024-04-25T08:45:11.605924+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.605924+0000","last_undegraded":"2024-04-25T08:45:11.605924+0000","last_fullsized":"2024-04-25T08:45:11.605924+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617089+0000","last_change":"2024-04-25T08:45:11.615530+0000","last_active":"2024-04-25T08:45:11.617089+0000","last_peered":"2024-04-25T08:45:11.617089+0000","last_clean":"2024-04-25T08:45:11.617089+0000","last_became_active":"2024-04-25T08:45:11.615180+0000","last_became_peered":"2024-04-25T08:45:11.615180+0000","last_unstale":"2024-04-25T08:45:11.617089+0000","last_undegraded":"2024-04-25T08:45:11.617089+0000","last_fullsized":"2024-04-25T08:45:11.617089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:46.666427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596149+0000","last_change":"2024-04-25T08:45:11.596149+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596149+0000","last_undegraded":"2024-04-25T08:45:11.596149+0000","last_fullsized":"2024-04-25T08:45:11.596149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617053+0000","last_change":"2024-04-25T08:45:11.615371+0000","last_active":"2024-04-25T08:45:11.617053+0000","last_peered":"2024-04-25T08:45:11.617053+0000","last_clean":"2024-04-25T08:45:11.617053+0000","last_became_active":"2024-04-25T08:45:11.611714+0000","last_became_peered":"2024-04-25T08:45:11.611714+0000","last_unstale":"2024-04-25T08:45:11.617053+0000","last_undegraded":"2024-04-25T08:45:11.617053+0000","last_fullsized":"2024-04-25T08:45:11.617053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595213+0000","last_change":"2024-04-25T08:45:11.595213+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595213+0000","last_undegraded":"2024-04-25T08:45:11.595213+0000","last_fullsized":"2024-04-25T08:45:11.595213+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614365+0000","last_change":"2024-04-25T08:45:11.610635+0000","last_active":"2024-04-25T08:45:11.614365+0000","last_peered":"2024-04-25T08:45:11.614365+0000","last_clean":"2024-04-25T08:45:11.614365+0000","last_became_active":"2024-04-25T08:45:11.609768+0000","last_became_peered":"2024-04-25T08:45:11.609768+0000","last_unstale":"2024-04-25T08:45:11.614365+0000","last_undegraded":"2024-04-25T08:45:11.614365+0000","last_fullsized":"2024-04-25T08:45:11.614365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.597735+0000","last_change":"2024-04-25T08:45:11.597735+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.597735+0000","last_undegraded":"2024-04-25T08:45:11.597735+0000","last_fullsized":"2024-04-25T08:45:11.597735+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.603873+0000","last_change":"2024-04-25T08:45:11.599255+0000","last_active":"2024-04-25T08:45:11.603873+0000","last_peered":"2024-04-25T08:45:11.603873+0000","last_clean":"2024-04-25T08:45:11.603873+0000","last_became_active":"2024-04-25T08:45:11.598904+0000","last_became_peered":"2024-04-25T08:45:11.598904+0000","last_unstale":"2024-04-25T08:45:11.603873+0000","last_undegraded":"2024-04-25T08:45:11.603873+0000","last_fullsized":"2024-04-25T08:45:11.603873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.303293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595394+0000","last_change":"2024-04-25T08:45:11.595394+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595394+0000","last_undegraded":"2024-04-25T08:45:11.595394+0000","last_fullsized":"2024-04-25T08:45:11.595394+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.609055+0000","last_change":"2024-04-25T08:45:11.603866+0000","last_active":"2024-04-25T08:45:11.609055+0000","last_peered":"2024-04-25T08:45:11.609055+0000","last_clean":"2024-04-25T08:45:11.609055+0000","last_became_active":"2024-04-25T08:45:11.603106+0000","last_became_peered":"2024-04-25T08:45:11.603106+0000","last_unstale":"2024-04-25T08:45:11.609055+0000","last_undegraded":"2024-04-25T08:45:11.609055+0000","last_fullsized":"2024-04-25T08:45:11.609055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:35.392449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.607579+0000","last_change":"2024-04-25T08:45:11.607579+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.607579+0000","last_undegraded":"2024-04-25T08:45:11.607579+0000","last_fullsized":"2024-04-25T08:45:11.607579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614345+0000","last_change":"2024-04-25T08:45:11.610404+0000","last_active":"2024-04-25T08:45:11.614345+0000","last_peered":"2024-04-25T08:45:11.614345+0000","last_clean":"2024-04-25T08:45:11.614345+0000","last_became_active":"2024-04-25T08:45:11.610147+0000","last_became_peered":"2024-04-25T08:45:11.610147+0000","last_unstale":"2024-04-25T08:45:11.614345+0000","last_undegraded":"2024-04-25T08:45:11.614345+0000","last_fullsized":"2024-04-25T08:45:11.614345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:35.633342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593591+0000","last_change":"2024-04-25T08:45:11.593591+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593591+0000","last_undegraded":"2024-04-25T08:45:11.593591+0000","last_fullsized":"2024-04-25T08:45:11.593591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.612199+0000","last_change":"2024-04-25T08:45:11.606470+0000","last_active":"2024-04-25T08:45:11.612199+0000","last_peered":"2024-04-25T08:45:11.612199+0000","last_clean":"2024-04-25T08:45:11.612199+0000","last_became_active":"2024-04-25T08:45:11.606206+0000","last_became_peered":"2024-04-25T08:45:11.606206+0000","last_unstale":"2024-04-25T08:45:11.612199+0000","last_undegraded":"2024-04-25T08:45:11.612199+0000","last_fullsized":"2024-04-25T08:45:11.612199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:46.666427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.595703+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:45:11.595703+0000","last_peered":"2024-04-25T08:45:11.595703+0000","last_clean":"2024-04-25T08:45:11.595703+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:45:11.595703+0000","last_undegraded":"2024-04-25T08:45:11.595703+0000","last_fullsized":"2024-04-25T08:45:11.595703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":932113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":335086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":749568,"data_stored":335086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":932113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:45:16.726 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:45:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f2824e00640 1 Processor -- start 2024-04-25T08:45:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f2824e00640 1 -- start start 2024-04-25T08:45:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f2824e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f2820099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f2824e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f28200ccde0 0x7f28200cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f2824e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282009a320 con 0x7f28200b2240 2024-04-25T08:45:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f2824e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282006db90 con 0x7f28200ccde0 2024-04-25T08:45:16.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f2824e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282006bdc0 con 0x7f2820099a00 2024-04-25T08:45:16.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f281de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f2820099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f2817e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f28200ccde0 0x7f28200cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f281de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f2820099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35910/0 (socket says 172.21.15.46:35910) 2024-04-25T08:45:16.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f281de00640 1 -- 172.21.15.46:0/4237705079 learned_addr learned my addr 172.21.15.46:0/4237705079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:16.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f281de00640 1 -- 172.21.15.46:0/4237705079 >> v2:172.21.15.82:3300/0 conn(0x7f28200ccde0 msgr2=0x7f28200cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:16.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f281de00640 1 --2- 172.21.15.46:0/4237705079 >> v2:172.21.15.82:3300/0 conn(0x7f28200ccde0 0x7f28200cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f281de00640 1 -- 172.21.15.46:0/4237705079 >> v1:172.21.15.82:6789/0 conn(0x7f28200b2240 legacy=0x7f28200b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.914+0000 7f281de00640 1 -- 172.21.15.46:0/4237705079 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282006c0a0 con 0x7f2820099a00 2024-04-25T08:45:16.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f281de00640 1 --2- 172.21.15.46:0/4237705079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f2820099de0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2810009870 tx=0x7f281001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=341defa7624ab5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:16.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f281d400640 1 -- 172.21.15.46:0/4237705079 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28100aa600 con 0x7f2820099a00 2024-04-25T08:45:16.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f281d400640 1 -- 172.21.15.46:0/4237705079 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28100af070 con 0x7f2820099a00 2024-04-25T08:45:16.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f281d400640 1 -- 172.21.15.46:0/4237705079 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28100aa9c0 con 0x7f2820099a00 2024-04-25T08:45:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f2824e00640 1 -- 172.21.15.46:0/4237705079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 msgr2=0x7f2820099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f2824e00640 1 --2- 172.21.15.46:0/4237705079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f2820099de0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2810009870 tx=0x7f281001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f2824e00640 1 -- 172.21.15.46:0/4237705079 shutdown_connections 2024-04-25T08:45:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f2824e00640 1 --2- 172.21.15.46:0/4237705079 >> v2:172.21.15.82:3300/0 conn(0x7f28200ccde0 0x7f28200cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f2824e00640 1 --2- 172.21.15.46:0/4237705079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f2820099de0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f2824e00640 1 -- 172.21.15.46:0/4237705079 >> 172.21.15.46:0/4237705079 conn(0x7f28200b3120 msgr2=0x7f28200b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:16.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f2824e00640 1 -- 172.21.15.46:0/4237705079 shutdown_connections 2024-04-25T08:45:16.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.918+0000 7f2824e00640 1 -- 172.21.15.46:0/4237705079 wait complete. 2024-04-25T08:45:16.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 Processor -- start 2024-04-25T08:45:16.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 -- start start 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f28200b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28200b2240 0x7f2820152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28200ccde0 0x7f28200c5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28200cd7a0 con 0x7f28200ccde0 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28200cdb80 con 0x7f28200b2240 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28200cd920 con 0x7f2820099a00 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f28200b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28200ccde0 0x7f28200c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f28200b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:35918/0 (socket says 172.21.15.46:35918) 2024-04-25T08:45:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281de00640 1 -- 172.21.15.46:0/3356307452 learned_addr learned my addr 172.21.15.46:0/3356307452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2817e00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28200b2240 0x7f2820152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281de00640 1 -- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28200b2240 msgr2=0x7f2820152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281de00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28200b2240 0x7f2820152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281de00640 1 -- 172.21.15.46:0/3356307452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28200ccde0 msgr2=0x7f28200c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281de00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28200ccde0 0x7f28200c5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281de00640 1 -- 172.21.15.46:0/3356307452 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28200c58d0 con 0x7f2820099a00 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281e800640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28200ccde0 0x7f28200c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f281de00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f28200b6a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f28100ac040 tx=0x7f281001ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2816000640 1 -- 172.21.15.46:0/3356307452 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28100bd5f0 con 0x7f2820099a00 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2816000640 1 -- 172.21.15.46:0/3356307452 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28100bd7b0 con 0x7f2820099a00 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2816000640 1 -- 172.21.15.46:0/3356307452 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28100bdba0 con 0x7f2820099a00 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28200c5bb0 con 0x7f2820099a00 2024-04-25T08:45:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28200c6170 con 0x7f2820099a00 2024-04-25T08:45:16.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.922+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2820069080 con 0x7f2820099a00 2024-04-25T08:45:16.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.926+0000 7f2816000640 1 -- 172.21.15.46:0/3356307452 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28100a9070 con 0x7f2820099a00 2024-04-25T08:45:16.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.926+0000 7f2816000640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27fc066f60 0x7f27fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.926+0000 7f2816000640 1 -- 172.21.15.46:0/3356307452 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f281011f780 con 0x7f2820099a00 2024-04-25T08:45:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.926+0000 7f2817e00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27fc066f60 0x7f27fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.926+0000 7f2817e00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27fc066f60 0x7f27fc069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f282009a6c0 tx=0x7f2808009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:16.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:16.930+0000 7f2816000640 1 -- 172.21.15.46:0/3356307452 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2810016150 con 0x7f2820099a00 2024-04-25T08:45:17.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.090+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f282015d540 con 0x7f27fc066f60 2024-04-25T08:45:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.094+0000 7f2816000640 1 -- 172.21.15.46:0/3356307452 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246711 (secure 0 0 0) 0x7f282015d540 con 0x7f27fc066f60 2024-04-25T08:45:17.091 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:45:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27fc066f60 msgr2=0x7f27fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27fc066f60 0x7f27fc069420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f282009a6c0 tx=0x7f2808009190 comp rx=0 tx=0).stop 2024-04-25T08:45:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 msgr2=0x7f28200b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f28200b6a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f28100ac040 tx=0x7f281001ace0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 shutdown_connections 2024-04-25T08:45:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f27fc066f60 0x7f27fc069420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28200ccde0 0x7f28200c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28200b2240 0x7f2820152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 --2- 172.21.15.46:0/3356307452 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2820099a00 0x7f28200b6a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 >> 172.21.15.46:0/3356307452 conn(0x7f28200b3120 msgr2=0x7f28200705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 shutdown_connections 2024-04-25T08:45:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.098+0000 7f2824e00640 1 -- 172.21.15.46:0/3356307452 wait complete. 2024-04-25T08:45:17.112 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-25T08:45:16.297142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244080,"kb_used_data":7088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730640,"statfs":{"total":773094113280,"available":772844175360,"internally_reserved":0,"allocated":7258112,"data_stored":3901378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.713434"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596202+0000","last_change":"2024-04-25T08:45:11.596202+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596202+0000","last_undegraded":"2024-04-25T08:45:11.596202+0000","last_fullsized":"2024-04-25T08:45:11.596202+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595066+0000","last_change":"2024-04-25T08:45:11.595066+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595066+0000","last_undegraded":"2024-04-25T08:45:11.595066+0000","last_fullsized":"2024-04-25T08:45:11.595066+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592923+0000","last_change":"2024-04-25T08:45:11.592923+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592923+0000","last_undegraded":"2024-04-25T08:45:11.592923+0000","last_fullsized":"2024-04-25T08:45:11.592923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599420+0000","last_change":"2024-04-25T08:45:11.599420+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599420+0000","last_undegraded":"2024-04-25T08:45:11.599420+0000","last_fullsized":"2024-04-25T08:45:11.599420+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594152+0000","last_change":"2024-04-25T08:45:11.594152+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594152+0000","last_undegraded":"2024-04-25T08:45:11.594152+0000","last_fullsized":"2024-04-25T08:45:11.594152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600272+0000","last_change":"2024-04-25T08:45:11.600272+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600272+0000","last_undegraded":"2024-04-25T08:45:11.600272+0000","last_fullsized":"2024-04-25T08:45:11.600272+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.605419+0000","last_change":"2024-04-25T08:45:11.605419+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.605419+0000","last_undegraded":"2024-04-25T08:45:11.605419+0000","last_fullsized":"2024-04-25T08:45:11.605419+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602566+0000","last_change":"2024-04-25T08:45:11.602566+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602566+0000","last_undegraded":"2024-04-25T08:45:11.602566+0000","last_fullsized":"2024-04-25T08:45:11.602566+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593504+0000","last_change":"2024-04-25T08:45:11.593504+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593504+0000","last_undegraded":"2024-04-25T08:45:11.593504+0000","last_fullsized":"2024-04-25T08:45:11.593504+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592774+0000","last_change":"2024-04-25T08:45:11.592774+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592774+0000","last_undegraded":"2024-04-25T08:45:11.592774+0000","last_fullsized":"2024-04-25T08:45:11.592774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600340+0000","last_change":"2024-04-25T08:45:11.600340+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600340+0000","last_undegraded":"2024-04-25T08:45:11.600340+0000","last_fullsized":"2024-04-25T08:45:11.600340+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.608903+0000","last_change":"2024-04-25T08:45:11.608903+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.608903+0000","last_undegraded":"2024-04-25T08:45:11.608903+0000","last_fullsized":"2024-04-25T08:45:11.608903+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604995+0000","last_change":"2024-04-25T08:45:11.604995+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604995+0000","last_undegraded":"2024-04-25T08:45:11.604995+0000","last_fullsized":"2024-04-25T08:45:11.604995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593267+0000","last_change":"2024-04-25T08:45:11.593267+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593267+0000","last_undegraded":"2024-04-25T08:45:11.593267+0000","last_fullsized":"2024-04-25T08:45:11.593267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594640+0000","last_change":"2024-04-25T08:45:11.594640+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594640+0000","last_undegraded":"2024-04-25T08:45:11.594640+0000","last_fullsized":"2024-04-25T08:45:11.594640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.606433+0000","last_change":"2024-04-25T08:45:11.606433+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.606433+0000","last_undegraded":"2024-04-25T08:45:11.606433+0000","last_fullsized":"2024-04-25T08:45:11.606433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595245+0000","last_change":"2024-04-25T08:45:11.595245+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595245+0000","last_undegraded":"2024-04-25T08:45:11.595245+0000","last_fullsized":"2024-04-25T08:45:11.595245+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601808+0000","last_change":"2024-04-25T08:45:11.601808+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601808+0000","last_undegraded":"2024-04-25T08:45:11.601808+0000","last_fullsized":"2024-04-25T08:45:11.601808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594375+0000","last_change":"2024-04-25T08:45:11.594375+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594375+0000","last_undegraded":"2024-04-25T08:45:11.594375+0000","last_fullsized":"2024-04-25T08:45:11.594375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593868+0000","last_change":"2024-04-25T08:45:11.593868+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593868+0000","last_undegraded":"2024-04-25T08:45:11.593868+0000","last_fullsized":"2024-04-25T08:45:11.593868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604594+0000","last_change":"2024-04-25T08:45:11.604594+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604594+0000","last_undegraded":"2024-04-25T08:45:11.604594+0000","last_fullsized":"2024-04-25T08:45:11.604594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593331+0000","last_change":"2024-04-25T08:45:11.593331+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593331+0000","last_undegraded":"2024-04-25T08:45:11.593331+0000","last_fullsized":"2024-04-25T08:45:11.593331+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593128+0000","last_change":"2024-04-25T08:45:11.593128+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593128+0000","last_undegraded":"2024-04-25T08:45:11.593128+0000","last_fullsized":"2024-04-25T08:45:11.593128+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592664+0000","last_change":"2024-04-25T08:45:11.592664+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592664+0000","last_undegraded":"2024-04-25T08:45:11.592664+0000","last_fullsized":"2024-04-25T08:45:11.592664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604961+0000","last_change":"2024-04-25T08:45:11.604961+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604961+0000","last_undegraded":"2024-04-25T08:45:11.604961+0000","last_fullsized":"2024-04-25T08:45:11.604961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.608708+0000","last_change":"2024-04-25T08:45:11.608708+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.608708+0000","last_undegraded":"2024-04-25T08:45:11.608708+0000","last_fullsized":"2024-04-25T08:45:11.608708+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599852+0000","last_change":"2024-04-25T08:45:11.599852+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599852+0000","last_undegraded":"2024-04-25T08:45:11.599852+0000","last_fullsized":"2024-04-25T08:45:11.599852+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595481+0000","last_change":"2024-04-25T08:45:11.595481+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595481+0000","last_undegraded":"2024-04-25T08:45:11.595481+0000","last_fullsized":"2024-04-25T08:45:11.595481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592683+0000","last_change":"2024-04-25T08:45:11.592683+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592683+0000","last_undegraded":"2024-04-25T08:45:11.592683+0000","last_fullsized":"2024-04-25T08:45:11.592683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604336+0000","last_change":"2024-04-25T08:45:11.604336+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604336+0000","last_undegraded":"2024-04-25T08:45:11.604336+0000","last_fullsized":"2024-04-25T08:45:11.604336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.598275+0000","last_change":"2024-04-25T08:45:11.598275+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.598275+0000","last_undegraded":"2024-04-25T08:45:11.598275+0000","last_fullsized":"2024-04-25T08:45:11.598275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602360+0000","last_change":"2024-04-25T08:45:11.602360+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602360+0000","last_undegraded":"2024-04-25T08:45:11.602360+0000","last_fullsized":"2024-04-25T08:45:11.602360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593787+0000","last_change":"2024-04-25T08:45:11.593787+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593787+0000","last_undegraded":"2024-04-25T08:45:11.593787+0000","last_fullsized":"2024-04-25T08:45:11.593787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618446+0000","last_change":"2024-04-25T08:45:11.616357+0000","last_active":"2024-04-25T08:45:11.618446+0000","last_peered":"2024-04-25T08:45:11.618446+0000","last_clean":"2024-04-25T08:45:11.618446+0000","last_became_active":"2024-04-25T08:45:11.612904+0000","last_became_peered":"2024-04-25T08:45:11.612904+0000","last_unstale":"2024-04-25T08:45:11.618446+0000","last_undegraded":"2024-04-25T08:45:11.618446+0000","last_fullsized":"2024-04-25T08:45:11.618446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599374+0000","last_change":"2024-04-25T08:45:11.599374+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599374+0000","last_undegraded":"2024-04-25T08:45:11.599374+0000","last_fullsized":"2024-04-25T08:45:11.599374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614412+0000","last_change":"2024-04-25T08:45:11.609407+0000","last_active":"2024-04-25T08:45:11.614412+0000","last_peered":"2024-04-25T08:45:11.614412+0000","last_clean":"2024-04-25T08:45:11.614412+0000","last_became_active":"2024-04-25T08:45:11.609129+0000","last_became_peered":"2024-04-25T08:45:11.609129+0000","last_unstale":"2024-04-25T08:45:11.614412+0000","last_undegraded":"2024-04-25T08:45:11.614412+0000","last_fullsized":"2024-04-25T08:45:11.614412+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.303293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603765+0000","last_change":"2024-04-25T08:45:11.603765+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603765+0000","last_undegraded":"2024-04-25T08:45:11.603765+0000","last_fullsized":"2024-04-25T08:45:11.603765+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617069+0000","last_change":"2024-04-25T08:45:11.615084+0000","last_active":"2024-04-25T08:45:11.617069+0000","last_peered":"2024-04-25T08:45:11.617069+0000","last_clean":"2024-04-25T08:45:11.617069+0000","last_became_active":"2024-04-25T08:45:11.614747+0000","last_became_peered":"2024-04-25T08:45:11.614747+0000","last_unstale":"2024-04-25T08:45:11.617069+0000","last_undegraded":"2024-04-25T08:45:11.617069+0000","last_fullsized":"2024-04-25T08:45:11.617069+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.879227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594756+0000","last_change":"2024-04-25T08:45:11.594756+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594756+0000","last_undegraded":"2024-04-25T08:45:11.594756+0000","last_fullsized":"2024-04-25T08:45:11.594756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614851+0000","last_change":"2024-04-25T08:45:11.610526+0000","last_active":"2024-04-25T08:45:11.614851+0000","last_peered":"2024-04-25T08:45:11.614851+0000","last_clean":"2024-04-25T08:45:11.614851+0000","last_became_active":"2024-04-25T08:45:11.610262+0000","last_became_peered":"2024-04-25T08:45:11.610262+0000","last_unstale":"2024-04-25T08:45:11.614851+0000","last_undegraded":"2024-04-25T08:45:11.614851+0000","last_fullsized":"2024-04-25T08:45:11.614851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:35.392449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603978+0000","last_change":"2024-04-25T08:45:11.603978+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603978+0000","last_undegraded":"2024-04-25T08:45:11.603978+0000","last_fullsized":"2024-04-25T08:45:11.603978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614866+0000","last_change":"2024-04-25T08:45:11.608750+0000","last_active":"2024-04-25T08:45:11.614866+0000","last_peered":"2024-04-25T08:45:11.614866+0000","last_clean":"2024-04-25T08:45:11.614866+0000","last_became_active":"2024-04-25T08:45:11.608428+0000","last_became_peered":"2024-04-25T08:45:11.608428+0000","last_unstale":"2024-04-25T08:45:11.614866+0000","last_undegraded":"2024-04-25T08:45:11.614866+0000","last_fullsized":"2024-04-25T08:45:11.614866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:35.847974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594081+0000","last_change":"2024-04-25T08:45:11.594081+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594081+0000","last_undegraded":"2024-04-25T08:45:11.594081+0000","last_fullsized":"2024-04-25T08:45:11.594081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614545+0000","last_change":"2024-04-25T08:45:11.611352+0000","last_active":"2024-04-25T08:45:11.614545+0000","last_peered":"2024-04-25T08:45:11.614545+0000","last_clean":"2024-04-25T08:45:11.614545+0000","last_became_active":"2024-04-25T08:45:11.611094+0000","last_became_peered":"2024-04-25T08:45:11.611094+0000","last_unstale":"2024-04-25T08:45:11.614545+0000","last_undegraded":"2024-04-25T08:45:11.614545+0000","last_fullsized":"2024-04-25T08:45:11.614545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:36.596184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.597842+0000","last_change":"2024-04-25T08:45:11.597842+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.597842+0000","last_undegraded":"2024-04-25T08:45:11.597842+0000","last_fullsized":"2024-04-25T08:45:11.597842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.609031+0000","last_change":"2024-04-25T08:45:11.604893+0000","last_active":"2024-04-25T08:45:11.609031+0000","last_peered":"2024-04-25T08:45:11.609031+0000","last_clean":"2024-04-25T08:45:11.609031+0000","last_became_active":"2024-04-25T08:45:11.603555+0000","last_became_peered":"2024-04-25T08:45:11.603555+0000","last_unstale":"2024-04-25T08:45:11.609031+0000","last_undegraded":"2024-04-25T08:45:11.609031+0000","last_fullsized":"2024-04-25T08:45:11.609031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:20.076400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593075+0000","last_change":"2024-04-25T08:45:11.593075+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593075+0000","last_undegraded":"2024-04-25T08:45:11.593075+0000","last_fullsized":"2024-04-25T08:45:11.593075+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.616463+0000","last_change":"2024-04-25T08:45:11.613393+0000","last_active":"2024-04-25T08:45:11.616463+0000","last_peered":"2024-04-25T08:45:11.616463+0000","last_clean":"2024-04-25T08:45:11.616463+0000","last_became_active":"2024-04-25T08:45:11.611939+0000","last_became_peered":"2024-04-25T08:45:11.611939+0000","last_unstale":"2024-04-25T08:45:11.616463+0000","last_undegraded":"2024-04-25T08:45:11.616463+0000","last_fullsized":"2024-04-25T08:45:11.616463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:33.153694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594910+0000","last_change":"2024-04-25T08:45:11.594910+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594910+0000","last_undegraded":"2024-04-25T08:45:11.594910+0000","last_fullsized":"2024-04-25T08:45:11.594910+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615906+0000","last_change":"2024-04-25T08:45:11.611641+0000","last_active":"2024-04-25T08:45:11.615906+0000","last_peered":"2024-04-25T08:45:11.615906+0000","last_clean":"2024-04-25T08:45:11.615906+0000","last_became_active":"2024-04-25T08:45:11.611380+0000","last_became_peered":"2024-04-25T08:45:11.611380+0000","last_unstale":"2024-04-25T08:45:11.615906+0000","last_undegraded":"2024-04-25T08:45:11.615906+0000","last_fullsized":"2024-04-25T08:45:11.615906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:57.910041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593106+0000","last_change":"2024-04-25T08:45:11.593106+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593106+0000","last_undegraded":"2024-04-25T08:45:11.593106+0000","last_fullsized":"2024-04-25T08:45:11.593106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615489+0000","last_change":"2024-04-25T08:45:11.612291+0000","last_active":"2024-04-25T08:45:11.615489+0000","last_peered":"2024-04-25T08:45:11.615489+0000","last_clean":"2024-04-25T08:45:11.615489+0000","last_became_active":"2024-04-25T08:45:11.612016+0000","last_became_peered":"2024-04-25T08:45:11.612016+0000","last_unstale":"2024-04-25T08:45:11.615489+0000","last_undegraded":"2024-04-25T08:45:11.615489+0000","last_fullsized":"2024-04-25T08:45:11.615489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:28.466919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594590+0000","last_change":"2024-04-25T08:45:11.594590+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594590+0000","last_undegraded":"2024-04-25T08:45:11.594590+0000","last_fullsized":"2024-04-25T08:45:11.594590+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615757+0000","last_change":"2024-04-25T08:45:11.612056+0000","last_active":"2024-04-25T08:45:11.615757+0000","last_peered":"2024-04-25T08:45:11.615757+0000","last_clean":"2024-04-25T08:45:11.615757+0000","last_became_active":"2024-04-25T08:45:11.611677+0000","last_became_peered":"2024-04-25T08:45:11.611677+0000","last_unstale":"2024-04-25T08:45:11.615757+0000","last_undegraded":"2024-04-25T08:45:11.615757+0000","last_fullsized":"2024-04-25T08:45:11.615757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:29.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602391+0000","last_change":"2024-04-25T08:45:11.602391+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602391+0000","last_undegraded":"2024-04-25T08:45:11.602391+0000","last_fullsized":"2024-04-25T08:45:11.602391+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617146+0000","last_change":"2024-04-25T08:45:11.615380+0000","last_active":"2024-04-25T08:45:11.617146+0000","last_peered":"2024-04-25T08:45:11.617146+0000","last_clean":"2024-04-25T08:45:11.617146+0000","last_became_active":"2024-04-25T08:45:11.612863+0000","last_became_peered":"2024-04-25T08:45:11.612863+0000","last_unstale":"2024-04-25T08:45:11.617146+0000","last_undegraded":"2024-04-25T08:45:11.617146+0000","last_fullsized":"2024-04-25T08:45:11.617146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599723+0000","last_change":"2024-04-25T08:45:11.599723+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599723+0000","last_undegraded":"2024-04-25T08:45:11.599723+0000","last_fullsized":"2024-04-25T08:45:11.599723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614344+0000","last_change":"2024-04-25T08:45:11.609636+0000","last_active":"2024-04-25T08:45:11.614344+0000","last_peered":"2024-04-25T08:45:11.614344+0000","last_clean":"2024-04-25T08:45:11.614344+0000","last_became_active":"2024-04-25T08:45:11.609296+0000","last_became_peered":"2024-04-25T08:45:11.609296+0000","last_unstale":"2024-04-25T08:45:11.614344+0000","last_undegraded":"2024-04-25T08:45:11.614344+0000","last_fullsized":"2024-04-25T08:45:11.614344+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603834+0000","last_change":"2024-04-25T08:45:11.603834+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603834+0000","last_undegraded":"2024-04-25T08:45:11.603834+0000","last_fullsized":"2024-04-25T08:45:11.603834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.619721+0000","last_change":"2024-04-25T08:45:11.618604+0000","last_active":"2024-04-25T08:45:11.619721+0000","last_peered":"2024-04-25T08:45:11.619721+0000","last_clean":"2024-04-25T08:45:11.619721+0000","last_became_active":"2024-04-25T08:45:11.618224+0000","last_became_peered":"2024-04-25T08:45:11.618224+0000","last_unstale":"2024-04-25T08:45:11.619721+0000","last_undegraded":"2024-04-25T08:45:11.619721+0000","last_fullsized":"2024-04-25T08:45:11.619721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:57.910041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596991+0000","last_change":"2024-04-25T08:45:11.596991+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596991+0000","last_undegraded":"2024-04-25T08:45:11.596991+0000","last_fullsized":"2024-04-25T08:45:11.596991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615681+0000","last_change":"2024-04-25T08:45:11.611078+0000","last_active":"2024-04-25T08:45:11.615681+0000","last_peered":"2024-04-25T08:45:11.615681+0000","last_clean":"2024-04-25T08:45:11.615681+0000","last_became_active":"2024-04-25T08:45:11.610749+0000","last_became_peered":"2024-04-25T08:45:11.610749+0000","last_unstale":"2024-04-25T08:45:11.615681+0000","last_undegraded":"2024-04-25T08:45:11.615681+0000","last_fullsized":"2024-04-25T08:45:11.615681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.517500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594513+0000","last_change":"2024-04-25T08:45:11.594513+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594513+0000","last_undegraded":"2024-04-25T08:45:11.594513+0000","last_fullsized":"2024-04-25T08:45:11.594513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615892+0000","last_change":"2024-04-25T08:45:11.613128+0000","last_active":"2024-04-25T08:45:11.615892+0000","last_peered":"2024-04-25T08:45:11.615892+0000","last_clean":"2024-04-25T08:45:11.615892+0000","last_became_active":"2024-04-25T08:45:11.611609+0000","last_became_peered":"2024-04-25T08:45:11.611609+0000","last_unstale":"2024-04-25T08:45:11.615892+0000","last_undegraded":"2024-04-25T08:45:11.615892+0000","last_fullsized":"2024-04-25T08:45:11.615892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:51.214222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594748+0000","last_change":"2024-04-25T08:45:11.594748+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594748+0000","last_undegraded":"2024-04-25T08:45:11.594748+0000","last_fullsized":"2024-04-25T08:45:11.594748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.619710+0000","last_change":"2024-04-25T08:45:11.618549+0000","last_active":"2024-04-25T08:45:11.619710+0000","last_peered":"2024-04-25T08:45:11.619710+0000","last_clean":"2024-04-25T08:45:11.619710+0000","last_became_active":"2024-04-25T08:45:11.618187+0000","last_became_peered":"2024-04-25T08:45:11.618187+0000","last_unstale":"2024-04-25T08:45:11.619710+0000","last_undegraded":"2024-04-25T08:45:11.619710+0000","last_fullsized":"2024-04-25T08:45:11.619710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.517500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.598619+0000","last_change":"2024-04-25T08:45:11.598619+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.598619+0000","last_undegraded":"2024-04-25T08:45:11.598619+0000","last_fullsized":"2024-04-25T08:45:11.598619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614216+0000","last_change":"2024-04-25T08:45:11.609242+0000","last_active":"2024-04-25T08:45:11.614216+0000","last_peered":"2024-04-25T08:45:11.614216+0000","last_clean":"2024-04-25T08:45:11.614216+0000","last_became_active":"2024-04-25T08:45:11.608930+0000","last_became_peered":"2024-04-25T08:45:11.608930+0000","last_unstale":"2024-04-25T08:45:11.614216+0000","last_undegraded":"2024-04-25T08:45:11.614216+0000","last_fullsized":"2024-04-25T08:45:11.614216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601349+0000","last_change":"2024-04-25T08:45:11.601349+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601349+0000","last_undegraded":"2024-04-25T08:45:11.601349+0000","last_fullsized":"2024-04-25T08:45:11.601349+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.613852+0000","last_change":"2024-04-25T08:45:11.608806+0000","last_active":"2024-04-25T08:45:11.613852+0000","last_peered":"2024-04-25T08:45:11.613852+0000","last_clean":"2024-04-25T08:45:11.613852+0000","last_became_active":"2024-04-25T08:45:11.607645+0000","last_became_peered":"2024-04-25T08:45:11.607645+0000","last_unstale":"2024-04-25T08:45:11.613852+0000","last_undegraded":"2024-04-25T08:45:11.613852+0000","last_fullsized":"2024-04-25T08:45:11.613852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:35.633342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596595+0000","last_change":"2024-04-25T08:45:11.596595+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596595+0000","last_undegraded":"2024-04-25T08:45:11.596595+0000","last_fullsized":"2024-04-25T08:45:11.596595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.607835+0000","last_change":"2024-04-25T08:45:11.605238+0000","last_active":"2024-04-25T08:45:11.607835+0000","last_peered":"2024-04-25T08:45:11.607835+0000","last_clean":"2024-04-25T08:45:11.607835+0000","last_became_active":"2024-04-25T08:45:11.604975+0000","last_became_peered":"2024-04-25T08:45:11.604975+0000","last_unstale":"2024-04-25T08:45:11.607835+0000","last_undegraded":"2024-04-25T08:45:11.607835+0000","last_fullsized":"2024-04-25T08:45:11.607835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.879227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601918+0000","last_change":"2024-04-25T08:45:11.601918+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601918+0000","last_undegraded":"2024-04-25T08:45:11.601918+0000","last_fullsized":"2024-04-25T08:45:11.601918+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618499+0000","last_change":"2024-04-25T08:45:11.616493+0000","last_active":"2024-04-25T08:45:11.618499+0000","last_peered":"2024-04-25T08:45:11.618499+0000","last_clean":"2024-04-25T08:45:11.618499+0000","last_became_active":"2024-04-25T08:45:11.616241+0000","last_became_peered":"2024-04-25T08:45:11.616241+0000","last_unstale":"2024-04-25T08:45:11.618499+0000","last_undegraded":"2024-04-25T08:45:11.618499+0000","last_fullsized":"2024-04-25T08:45:11.618499+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:35.847974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603447+0000","last_change":"2024-04-25T08:45:11.603447+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603447+0000","last_undegraded":"2024-04-25T08:45:11.603447+0000","last_fullsized":"2024-04-25T08:45:11.603447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617745+0000","last_change":"2024-04-25T08:45:11.615924+0000","last_active":"2024-04-25T08:45:11.617745+0000","last_peered":"2024-04-25T08:45:11.617745+0000","last_clean":"2024-04-25T08:45:11.617745+0000","last_became_active":"2024-04-25T08:45:11.613543+0000","last_became_peered":"2024-04-25T08:45:11.613543+0000","last_unstale":"2024-04-25T08:45:11.617745+0000","last_undegraded":"2024-04-25T08:45:11.617745+0000","last_fullsized":"2024-04-25T08:45:11.617745+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593904+0000","last_change":"2024-04-25T08:45:11.593904+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593904+0000","last_undegraded":"2024-04-25T08:45:11.593904+0000","last_fullsized":"2024-04-25T08:45:11.593904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617738+0000","last_change":"2024-04-25T08:45:11.613996+0000","last_active":"2024-04-25T08:45:11.617738+0000","last_peered":"2024-04-25T08:45:11.617738+0000","last_clean":"2024-04-25T08:45:11.617738+0000","last_became_active":"2024-04-25T08:45:11.613688+0000","last_became_peered":"2024-04-25T08:45:11.613688+0000","last_unstale":"2024-04-25T08:45:11.617738+0000","last_undegraded":"2024-04-25T08:45:11.617738+0000","last_fullsized":"2024-04-25T08:45:11.617738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:20.076400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600734+0000","last_change":"2024-04-25T08:45:11.600734+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600734+0000","last_undegraded":"2024-04-25T08:45:11.600734+0000","last_fullsized":"2024-04-25T08:45:11.600734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618392+0000","last_change":"2024-04-25T08:45:11.616861+0000","last_active":"2024-04-25T08:45:11.618392+0000","last_peered":"2024-04-25T08:45:11.618392+0000","last_clean":"2024-04-25T08:45:11.618392+0000","last_became_active":"2024-04-25T08:45:11.616539+0000","last_became_peered":"2024-04-25T08:45:11.616539+0000","last_unstale":"2024-04-25T08:45:11.618392+0000","last_undegraded":"2024-04-25T08:45:11.618392+0000","last_fullsized":"2024-04-25T08:45:11.618392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.606983+0000","last_change":"2024-04-25T08:45:11.606983+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.606983+0000","last_undegraded":"2024-04-25T08:45:11.606983+0000","last_fullsized":"2024-04-25T08:45:11.606983+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617742+0000","last_change":"2024-04-25T08:45:11.616486+0000","last_active":"2024-04-25T08:45:11.617742+0000","last_peered":"2024-04-25T08:45:11.617742+0000","last_clean":"2024-04-25T08:45:11.617742+0000","last_became_active":"2024-04-25T08:45:11.614742+0000","last_became_peered":"2024-04-25T08:45:11.614742+0000","last_unstale":"2024-04-25T08:45:11.617742+0000","last_undegraded":"2024-04-25T08:45:11.617742+0000","last_fullsized":"2024-04-25T08:45:11.617742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.605924+0000","last_change":"2024-04-25T08:45:11.605924+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.605924+0000","last_undegraded":"2024-04-25T08:45:11.605924+0000","last_fullsized":"2024-04-25T08:45:11.605924+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617089+0000","last_change":"2024-04-25T08:45:11.615530+0000","last_active":"2024-04-25T08:45:11.617089+0000","last_peered":"2024-04-25T08:45:11.617089+0000","last_clean":"2024-04-25T08:45:11.617089+0000","last_became_active":"2024-04-25T08:45:11.615180+0000","last_became_peered":"2024-04-25T08:45:11.615180+0000","last_unstale":"2024-04-25T08:45:11.617089+0000","last_undegraded":"2024-04-25T08:45:11.617089+0000","last_fullsized":"2024-04-25T08:45:11.617089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:46.666427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596149+0000","last_change":"2024-04-25T08:45:11.596149+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596149+0000","last_undegraded":"2024-04-25T08:45:11.596149+0000","last_fullsized":"2024-04-25T08:45:11.596149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617053+0000","last_change":"2024-04-25T08:45:11.615371+0000","last_active":"2024-04-25T08:45:11.617053+0000","last_peered":"2024-04-25T08:45:11.617053+0000","last_clean":"2024-04-25T08:45:11.617053+0000","last_became_active":"2024-04-25T08:45:11.611714+0000","last_became_peered":"2024-04-25T08:45:11.611714+0000","last_unstale":"2024-04-25T08:45:11.617053+0000","last_undegraded":"2024-04-25T08:45:11.617053+0000","last_fullsized":"2024-04-25T08:45:11.617053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595213+0000","last_change":"2024-04-25T08:45:11.595213+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595213+0000","last_undegraded":"2024-04-25T08:45:11.595213+0000","last_fullsized":"2024-04-25T08:45:11.595213+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614365+0000","last_change":"2024-04-25T08:45:11.610635+0000","last_active":"2024-04-25T08:45:11.614365+0000","last_peered":"2024-04-25T08:45:11.614365+0000","last_clean":"2024-04-25T08:45:11.614365+0000","last_became_active":"2024-04-25T08:45:11.609768+0000","last_became_peered":"2024-04-25T08:45:11.609768+0000","last_unstale":"2024-04-25T08:45:11.614365+0000","last_undegraded":"2024-04-25T08:45:11.614365+0000","last_fullsized":"2024-04-25T08:45:11.614365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.597735+0000","last_change":"2024-04-25T08:45:11.597735+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.597735+0000","last_undegraded":"2024-04-25T08:45:11.597735+0000","last_fullsized":"2024-04-25T08:45:11.597735+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.603873+0000","last_change":"2024-04-25T08:45:11.599255+0000","last_active":"2024-04-25T08:45:11.603873+0000","last_peered":"2024-04-25T08:45:11.603873+0000","last_clean":"2024-04-25T08:45:11.603873+0000","last_became_active":"2024-04-25T08:45:11.598904+0000","last_became_peered":"2024-04-25T08:45:11.598904+0000","last_unstale":"2024-04-25T08:45:11.603873+0000","last_undegraded":"2024-04-25T08:45:11.603873+0000","last_fullsized":"2024-04-25T08:45:11.603873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.303293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595394+0000","last_change":"2024-04-25T08:45:11.595394+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595394+0000","last_undegraded":"2024-04-25T08:45:11.595394+0000","last_fullsized":"2024-04-25T08:45:11.595394+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.609055+0000","last_change":"2024-04-25T08:45:11.603866+0000","last_active":"2024-04-25T08:45:11.609055+0000","last_peered":"2024-04-25T08:45:11.609055+0000","last_clean":"2024-04-25T08:45:11.609055+0000","last_became_active":"2024-04-25T08:45:11.603106+0000","last_became_peered":"2024-04-25T08:45:11.603106+0000","last_unstale":"2024-04-25T08:45:11.609055+0000","last_undegraded":"2024-04-25T08:45:11.609055+0000","last_fullsized":"2024-04-25T08:45:11.609055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:35.392449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.607579+0000","last_change":"2024-04-25T08:45:11.607579+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.607579+0000","last_undegraded":"2024-04-25T08:45:11.607579+0000","last_fullsized":"2024-04-25T08:45:11.607579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614345+0000","last_change":"2024-04-25T08:45:11.610404+0000","last_active":"2024-04-25T08:45:11.614345+0000","last_peered":"2024-04-25T08:45:11.614345+0000","last_clean":"2024-04-25T08:45:11.614345+0000","last_became_active":"2024-04-25T08:45:11.610147+0000","last_became_peered":"2024-04-25T08:45:11.610147+0000","last_unstale":"2024-04-25T08:45:11.614345+0000","last_undegraded":"2024-04-25T08:45:11.614345+0000","last_fullsized":"2024-04-25T08:45:11.614345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:35.633342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593591+0000","last_change":"2024-04-25T08:45:11.593591+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593591+0000","last_undegraded":"2024-04-25T08:45:11.593591+0000","last_fullsized":"2024-04-25T08:45:11.593591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.612199+0000","last_change":"2024-04-25T08:45:11.606470+0000","last_active":"2024-04-25T08:45:11.612199+0000","last_peered":"2024-04-25T08:45:11.612199+0000","last_clean":"2024-04-25T08:45:11.612199+0000","last_became_active":"2024-04-25T08:45:11.606206+0000","last_became_peered":"2024-04-25T08:45:11.606206+0000","last_unstale":"2024-04-25T08:45:11.612199+0000","last_undegraded":"2024-04-25T08:45:11.612199+0000","last_fullsized":"2024-04-25T08:45:11.612199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:46.666427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.595703+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:45:11.595703+0000","last_peered":"2024-04-25T08:45:11.595703+0000","last_clean":"2024-04-25T08:45:11.595703+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:45:11.595703+0000","last_undegraded":"2024-04-25T08:45:11.595703+0000","last_fullsized":"2024-04-25T08:45:11.595703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":932113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":335086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":749568,"data_stored":335086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":932113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:45:17.115 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:45:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5a00640 1 Processor -- start 2024-04-25T08:45:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5a00640 1 -- start start 2024-04-25T08:45:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcba015d9f0 0x7fcba00c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcba006b320 0x7fcba00ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba00ce9f0 con 0x7fcba00b2aa0 2024-04-25T08:45:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba00c53b0 con 0x7fcba006b320 2024-04-25T08:45:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba00cf2b0 con 0x7fcba015d9f0 2024-04-25T08:45:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcba006b320 0x7fcba00ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcb9f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcba00b2aa0 0x7fcba006ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50882/0 (socket says 172.21.15.46:50882) 2024-04-25T08:45:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcb9f400640 1 -- 172.21.15.46:0/926722294 learned_addr learned my addr 172.21.15.46:0/926722294 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcb9fe00640 1 --2- 172.21.15.46:0/926722294 >> v2:172.21.15.82:3300/0 conn(0x7fcba015d9f0 0x7fcba00c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5000640 1 -- 172.21.15.46:0/926722294 >> v2:172.21.15.82:3300/0 conn(0x7fcba015d9f0 msgr2=0x7fcba00c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5000640 1 --2- 172.21.15.46:0/926722294 >> v2:172.21.15.82:3300/0 conn(0x7fcba015d9f0 0x7fcba00c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5000640 1 -- 172.21.15.46:0/926722294 >> v1:172.21.15.46:6789/0 conn(0x7fcba00b2aa0 legacy=0x7fcba006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.302+0000 7fcba5000640 1 -- 172.21.15.46:0/926722294 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00cec60 con 0x7fcba006b320 2024-04-25T08:45:17.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5000640 1 --2- 172.21.15.46:0/926722294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 0x7fcba00ca9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cde00 tx=0x7fcb9801f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e175f96a8e5fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcb9ea00640 1 -- 172.21.15.46:0/926722294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb980b4070 con 0x7fcba006b320 2024-04-25T08:45:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcb9ea00640 1 -- 172.21.15.46:0/926722294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb980af600 con 0x7fcba006b320 2024-04-25T08:45:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcb9ea00640 1 -- 172.21.15.46:0/926722294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb980af9d0 con 0x7fcba006b320 2024-04-25T08:45:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 -- 172.21.15.46:0/926722294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 msgr2=0x7fcba00ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 --2- 172.21.15.46:0/926722294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 0x7fcba00ca9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cde00 tx=0x7fcb9801f110 comp rx=0 tx=0).stop 2024-04-25T08:45:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 -- 172.21.15.46:0/926722294 shutdown_connections 2024-04-25T08:45:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 --2- 172.21.15.46:0/926722294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 0x7fcba00ca9c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 --2- 172.21.15.46:0/926722294 >> v2:172.21.15.82:3300/0 conn(0x7fcba015d9f0 0x7fcba00c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 -- 172.21.15.46:0/926722294 >> 172.21.15.46:0/926722294 conn(0x7fcba00b1f80 msgr2=0x7fcba00b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 -- 172.21.15.46:0/926722294 shutdown_connections 2024-04-25T08:45:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 -- 172.21.15.46:0/926722294 wait complete. 2024-04-25T08:45:17.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 Processor -- start 2024-04-25T08:45:17.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.306+0000 7fcba5a00640 1 -- start start 2024-04-25T08:45:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 0x7fcba0155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcba00b2aa0 0x7fcba00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcba015d9f0 0x7fcba00cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c7170 con 0x7fcba006b320 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00cf740 con 0x7fcba00b2aa0 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba00c6df0 con 0x7fcba015d9f0 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 0x7fcba0155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 0x7fcba0155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33676/0 (socket says 172.21.15.46:33676) 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9fe00640 1 -- 172.21.15.46:0/1911004571 learned_addr learned my addr 172.21.15.46:0/1911004571 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9f400640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcba00b2aa0 0x7fcba00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5000640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcba015d9f0 0x7fcba00cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9fe00640 1 -- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcba015d9f0 msgr2=0x7fcba00cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9fe00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcba015d9f0 0x7fcba00cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9fe00640 1 -- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcba00b2aa0 msgr2=0x7fcba00c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9fe00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcba00b2aa0 0x7fcba00c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9fe00640 1 -- 172.21.15.46:0/1911004571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00c8e80 con 0x7fcba006b320 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9f400640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcba00b2aa0 0x7fcba00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb9fe00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 0x7fcba0155ca0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000d620 tx=0x7fcb9000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb97e00640 1 -- 172.21.15.46:0/1911004571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000b620 con 0x7fcba006b320 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb97e00640 1 -- 172.21.15.46:0/1911004571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90019070 con 0x7fcba006b320 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb97e00640 1 -- 172.21.15.46:0/1911004571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000b9e0 con 0x7fcba006b320 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba00b33f0 con 0x7fcba006b320 2024-04-25T08:45:17.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba00b39b0 con 0x7fcba006b320 2024-04-25T08:45:17.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba0069010 con 0x7fcba006b320 2024-04-25T08:45:17.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.310+0000 7fcb97e00640 1 -- 172.21.15.46:0/1911004571 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb9000bb80 con 0x7fcba006b320 2024-04-25T08:45:17.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.314+0000 7fcb97e00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb80066f90 0x7fcb80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.314+0000 7fcb97e00640 1 -- 172.21.15.46:0/1911004571 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb900855e0 con 0x7fcba006b320 2024-04-25T08:45:17.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.314+0000 7fcb9f400640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb80066f90 0x7fcb80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.314+0000 7fcb9f400640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb80066f90 0x7fcb80069450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcba015c2d0 tx=0x7fcb8c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.318+0000 7fcb97e00640 1 -- 172.21.15.46:0/1911004571 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb90056450 con 0x7fcba006b320 2024-04-25T08:45:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.474+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcba00b2490 con 0x7fcb80066f90 2024-04-25T08:45:17.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.482+0000 7fcb97e00640 1 -- 172.21.15.46:0/1911004571 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246711 (secure 0 0 0) 0x7fcba00b2490 con 0x7fcb80066f90 2024-04-25T08:45:17.475 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:17.478 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:45:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb80066f90 msgr2=0x7fcb80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb80066f90 0x7fcb80069450 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcba015c2d0 tx=0x7fcb8c009040 comp rx=0 tx=0).stop 2024-04-25T08:45:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 msgr2=0x7fcba0155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 0x7fcba0155ca0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000d620 tx=0x7fcb9000dae0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 shutdown_connections 2024-04-25T08:45:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcb80066f90 0x7fcb80069450 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcba015d9f0 0x7fcba00cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcba00b2aa0 0x7fcba00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 --2- 172.21.15.46:0/1911004571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcba006b320 0x7fcba0155ca0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 >> 172.21.15.46:0/1911004571 conn(0x7fcba00b1f80 msgr2=0x7fcba015aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 shutdown_connections 2024-04-25T08:45:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.486+0000 7fcba5a00640 1 -- 172.21.15.46:0/1911004571 wait complete. 2024-04-25T08:45:17.497 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-25T08:45:16.297142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244080,"kb_used_data":7088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730640,"statfs":{"total":773094113280,"available":772844175360,"internally_reserved":0,"allocated":7258112,"data_stored":3901378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.713434"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596202+0000","last_change":"2024-04-25T08:45:11.596202+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596202+0000","last_undegraded":"2024-04-25T08:45:11.596202+0000","last_fullsized":"2024-04-25T08:45:11.596202+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595066+0000","last_change":"2024-04-25T08:45:11.595066+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595066+0000","last_undegraded":"2024-04-25T08:45:11.595066+0000","last_fullsized":"2024-04-25T08:45:11.595066+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592923+0000","last_change":"2024-04-25T08:45:11.592923+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592923+0000","last_undegraded":"2024-04-25T08:45:11.592923+0000","last_fullsized":"2024-04-25T08:45:11.592923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599420+0000","last_change":"2024-04-25T08:45:11.599420+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599420+0000","last_undegraded":"2024-04-25T08:45:11.599420+0000","last_fullsized":"2024-04-25T08:45:11.599420+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594152+0000","last_change":"2024-04-25T08:45:11.594152+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594152+0000","last_undegraded":"2024-04-25T08:45:11.594152+0000","last_fullsized":"2024-04-25T08:45:11.594152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600272+0000","last_change":"2024-04-25T08:45:11.600272+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600272+0000","last_undegraded":"2024-04-25T08:45:11.600272+0000","last_fullsized":"2024-04-25T08:45:11.600272+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.605419+0000","last_change":"2024-04-25T08:45:11.605419+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.605419+0000","last_undegraded":"2024-04-25T08:45:11.605419+0000","last_fullsized":"2024-04-25T08:45:11.605419+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602566+0000","last_change":"2024-04-25T08:45:11.602566+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602566+0000","last_undegraded":"2024-04-25T08:45:11.602566+0000","last_fullsized":"2024-04-25T08:45:11.602566+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593504+0000","last_change":"2024-04-25T08:45:11.593504+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593504+0000","last_undegraded":"2024-04-25T08:45:11.593504+0000","last_fullsized":"2024-04-25T08:45:11.593504+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592774+0000","last_change":"2024-04-25T08:45:11.592774+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592774+0000","last_undegraded":"2024-04-25T08:45:11.592774+0000","last_fullsized":"2024-04-25T08:45:11.592774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600340+0000","last_change":"2024-04-25T08:45:11.600340+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600340+0000","last_undegraded":"2024-04-25T08:45:11.600340+0000","last_fullsized":"2024-04-25T08:45:11.600340+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.608903+0000","last_change":"2024-04-25T08:45:11.608903+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.608903+0000","last_undegraded":"2024-04-25T08:45:11.608903+0000","last_fullsized":"2024-04-25T08:45:11.608903+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604995+0000","last_change":"2024-04-25T08:45:11.604995+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604995+0000","last_undegraded":"2024-04-25T08:45:11.604995+0000","last_fullsized":"2024-04-25T08:45:11.604995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593267+0000","last_change":"2024-04-25T08:45:11.593267+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593267+0000","last_undegraded":"2024-04-25T08:45:11.593267+0000","last_fullsized":"2024-04-25T08:45:11.593267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594640+0000","last_change":"2024-04-25T08:45:11.594640+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594640+0000","last_undegraded":"2024-04-25T08:45:11.594640+0000","last_fullsized":"2024-04-25T08:45:11.594640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.606433+0000","last_change":"2024-04-25T08:45:11.606433+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.606433+0000","last_undegraded":"2024-04-25T08:45:11.606433+0000","last_fullsized":"2024-04-25T08:45:11.606433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595245+0000","last_change":"2024-04-25T08:45:11.595245+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595245+0000","last_undegraded":"2024-04-25T08:45:11.595245+0000","last_fullsized":"2024-04-25T08:45:11.595245+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601808+0000","last_change":"2024-04-25T08:45:11.601808+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601808+0000","last_undegraded":"2024-04-25T08:45:11.601808+0000","last_fullsized":"2024-04-25T08:45:11.601808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594375+0000","last_change":"2024-04-25T08:45:11.594375+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594375+0000","last_undegraded":"2024-04-25T08:45:11.594375+0000","last_fullsized":"2024-04-25T08:45:11.594375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593868+0000","last_change":"2024-04-25T08:45:11.593868+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593868+0000","last_undegraded":"2024-04-25T08:45:11.593868+0000","last_fullsized":"2024-04-25T08:45:11.593868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604594+0000","last_change":"2024-04-25T08:45:11.604594+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604594+0000","last_undegraded":"2024-04-25T08:45:11.604594+0000","last_fullsized":"2024-04-25T08:45:11.604594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593331+0000","last_change":"2024-04-25T08:45:11.593331+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593331+0000","last_undegraded":"2024-04-25T08:45:11.593331+0000","last_fullsized":"2024-04-25T08:45:11.593331+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593128+0000","last_change":"2024-04-25T08:45:11.593128+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593128+0000","last_undegraded":"2024-04-25T08:45:11.593128+0000","last_fullsized":"2024-04-25T08:45:11.593128+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592664+0000","last_change":"2024-04-25T08:45:11.592664+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592664+0000","last_undegraded":"2024-04-25T08:45:11.592664+0000","last_fullsized":"2024-04-25T08:45:11.592664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604961+0000","last_change":"2024-04-25T08:45:11.604961+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604961+0000","last_undegraded":"2024-04-25T08:45:11.604961+0000","last_fullsized":"2024-04-25T08:45:11.604961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.608708+0000","last_change":"2024-04-25T08:45:11.608708+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.608708+0000","last_undegraded":"2024-04-25T08:45:11.608708+0000","last_fullsized":"2024-04-25T08:45:11.608708+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599852+0000","last_change":"2024-04-25T08:45:11.599852+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599852+0000","last_undegraded":"2024-04-25T08:45:11.599852+0000","last_fullsized":"2024-04-25T08:45:11.599852+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595481+0000","last_change":"2024-04-25T08:45:11.595481+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595481+0000","last_undegraded":"2024-04-25T08:45:11.595481+0000","last_fullsized":"2024-04-25T08:45:11.595481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592683+0000","last_change":"2024-04-25T08:45:11.592683+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592683+0000","last_undegraded":"2024-04-25T08:45:11.592683+0000","last_fullsized":"2024-04-25T08:45:11.592683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604336+0000","last_change":"2024-04-25T08:45:11.604336+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604336+0000","last_undegraded":"2024-04-25T08:45:11.604336+0000","last_fullsized":"2024-04-25T08:45:11.604336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.598275+0000","last_change":"2024-04-25T08:45:11.598275+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.598275+0000","last_undegraded":"2024-04-25T08:45:11.598275+0000","last_fullsized":"2024-04-25T08:45:11.598275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602360+0000","last_change":"2024-04-25T08:45:11.602360+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602360+0000","last_undegraded":"2024-04-25T08:45:11.602360+0000","last_fullsized":"2024-04-25T08:45:11.602360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593787+0000","last_change":"2024-04-25T08:45:11.593787+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593787+0000","last_undegraded":"2024-04-25T08:45:11.593787+0000","last_fullsized":"2024-04-25T08:45:11.593787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618446+0000","last_change":"2024-04-25T08:45:11.616357+0000","last_active":"2024-04-25T08:45:11.618446+0000","last_peered":"2024-04-25T08:45:11.618446+0000","last_clean":"2024-04-25T08:45:11.618446+0000","last_became_active":"2024-04-25T08:45:11.612904+0000","last_became_peered":"2024-04-25T08:45:11.612904+0000","last_unstale":"2024-04-25T08:45:11.618446+0000","last_undegraded":"2024-04-25T08:45:11.618446+0000","last_fullsized":"2024-04-25T08:45:11.618446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599374+0000","last_change":"2024-04-25T08:45:11.599374+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599374+0000","last_undegraded":"2024-04-25T08:45:11.599374+0000","last_fullsized":"2024-04-25T08:45:11.599374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614412+0000","last_change":"2024-04-25T08:45:11.609407+0000","last_active":"2024-04-25T08:45:11.614412+0000","last_peered":"2024-04-25T08:45:11.614412+0000","last_clean":"2024-04-25T08:45:11.614412+0000","last_became_active":"2024-04-25T08:45:11.609129+0000","last_became_peered":"2024-04-25T08:45:11.609129+0000","last_unstale":"2024-04-25T08:45:11.614412+0000","last_undegraded":"2024-04-25T08:45:11.614412+0000","last_fullsized":"2024-04-25T08:45:11.614412+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.303293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603765+0000","last_change":"2024-04-25T08:45:11.603765+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603765+0000","last_undegraded":"2024-04-25T08:45:11.603765+0000","last_fullsized":"2024-04-25T08:45:11.603765+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617069+0000","last_change":"2024-04-25T08:45:11.615084+0000","last_active":"2024-04-25T08:45:11.617069+0000","last_peered":"2024-04-25T08:45:11.617069+0000","last_clean":"2024-04-25T08:45:11.617069+0000","last_became_active":"2024-04-25T08:45:11.614747+0000","last_became_peered":"2024-04-25T08:45:11.614747+0000","last_unstale":"2024-04-25T08:45:11.617069+0000","last_undegraded":"2024-04-25T08:45:11.617069+0000","last_fullsized":"2024-04-25T08:45:11.617069+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.879227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594756+0000","last_change":"2024-04-25T08:45:11.594756+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594756+0000","last_undegraded":"2024-04-25T08:45:11.594756+0000","last_fullsized":"2024-04-25T08:45:11.594756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614851+0000","last_change":"2024-04-25T08:45:11.610526+0000","last_active":"2024-04-25T08:45:11.614851+0000","last_peered":"2024-04-25T08:45:11.614851+0000","last_clean":"2024-04-25T08:45:11.614851+0000","last_became_active":"2024-04-25T08:45:11.610262+0000","last_became_peered":"2024-04-25T08:45:11.610262+0000","last_unstale":"2024-04-25T08:45:11.614851+0000","last_undegraded":"2024-04-25T08:45:11.614851+0000","last_fullsized":"2024-04-25T08:45:11.614851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:35.392449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603978+0000","last_change":"2024-04-25T08:45:11.603978+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603978+0000","last_undegraded":"2024-04-25T08:45:11.603978+0000","last_fullsized":"2024-04-25T08:45:11.603978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614866+0000","last_change":"2024-04-25T08:45:11.608750+0000","last_active":"2024-04-25T08:45:11.614866+0000","last_peered":"2024-04-25T08:45:11.614866+0000","last_clean":"2024-04-25T08:45:11.614866+0000","last_became_active":"2024-04-25T08:45:11.608428+0000","last_became_peered":"2024-04-25T08:45:11.608428+0000","last_unstale":"2024-04-25T08:45:11.614866+0000","last_undegraded":"2024-04-25T08:45:11.614866+0000","last_fullsized":"2024-04-25T08:45:11.614866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:35.847974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594081+0000","last_change":"2024-04-25T08:45:11.594081+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594081+0000","last_undegraded":"2024-04-25T08:45:11.594081+0000","last_fullsized":"2024-04-25T08:45:11.594081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614545+0000","last_change":"2024-04-25T08:45:11.611352+0000","last_active":"2024-04-25T08:45:11.614545+0000","last_peered":"2024-04-25T08:45:11.614545+0000","last_clean":"2024-04-25T08:45:11.614545+0000","last_became_active":"2024-04-25T08:45:11.611094+0000","last_became_peered":"2024-04-25T08:45:11.611094+0000","last_unstale":"2024-04-25T08:45:11.614545+0000","last_undegraded":"2024-04-25T08:45:11.614545+0000","last_fullsized":"2024-04-25T08:45:11.614545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:36.596184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.597842+0000","last_change":"2024-04-25T08:45:11.597842+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.597842+0000","last_undegraded":"2024-04-25T08:45:11.597842+0000","last_fullsized":"2024-04-25T08:45:11.597842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.609031+0000","last_change":"2024-04-25T08:45:11.604893+0000","last_active":"2024-04-25T08:45:11.609031+0000","last_peered":"2024-04-25T08:45:11.609031+0000","last_clean":"2024-04-25T08:45:11.609031+0000","last_became_active":"2024-04-25T08:45:11.603555+0000","last_became_peered":"2024-04-25T08:45:11.603555+0000","last_unstale":"2024-04-25T08:45:11.609031+0000","last_undegraded":"2024-04-25T08:45:11.609031+0000","last_fullsized":"2024-04-25T08:45:11.609031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:20.076400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593075+0000","last_change":"2024-04-25T08:45:11.593075+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593075+0000","last_undegraded":"2024-04-25T08:45:11.593075+0000","last_fullsized":"2024-04-25T08:45:11.593075+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.616463+0000","last_change":"2024-04-25T08:45:11.613393+0000","last_active":"2024-04-25T08:45:11.616463+0000","last_peered":"2024-04-25T08:45:11.616463+0000","last_clean":"2024-04-25T08:45:11.616463+0000","last_became_active":"2024-04-25T08:45:11.611939+0000","last_became_peered":"2024-04-25T08:45:11.611939+0000","last_unstale":"2024-04-25T08:45:11.616463+0000","last_undegraded":"2024-04-25T08:45:11.616463+0000","last_fullsized":"2024-04-25T08:45:11.616463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:33.153694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594910+0000","last_change":"2024-04-25T08:45:11.594910+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594910+0000","last_undegraded":"2024-04-25T08:45:11.594910+0000","last_fullsized":"2024-04-25T08:45:11.594910+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615906+0000","last_change":"2024-04-25T08:45:11.611641+0000","last_active":"2024-04-25T08:45:11.615906+0000","last_peered":"2024-04-25T08:45:11.615906+0000","last_clean":"2024-04-25T08:45:11.615906+0000","last_became_active":"2024-04-25T08:45:11.611380+0000","last_became_peered":"2024-04-25T08:45:11.611380+0000","last_unstale":"2024-04-25T08:45:11.615906+0000","last_undegraded":"2024-04-25T08:45:11.615906+0000","last_fullsized":"2024-04-25T08:45:11.615906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:57.910041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593106+0000","last_change":"2024-04-25T08:45:11.593106+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593106+0000","last_undegraded":"2024-04-25T08:45:11.593106+0000","last_fullsized":"2024-04-25T08:45:11.593106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615489+0000","last_change":"2024-04-25T08:45:11.612291+0000","last_active":"2024-04-25T08:45:11.615489+0000","last_peered":"2024-04-25T08:45:11.615489+0000","last_clean":"2024-04-25T08:45:11.615489+0000","last_became_active":"2024-04-25T08:45:11.612016+0000","last_became_peered":"2024-04-25T08:45:11.612016+0000","last_unstale":"2024-04-25T08:45:11.615489+0000","last_undegraded":"2024-04-25T08:45:11.615489+0000","last_fullsized":"2024-04-25T08:45:11.615489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:28.466919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594590+0000","last_change":"2024-04-25T08:45:11.594590+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594590+0000","last_undegraded":"2024-04-25T08:45:11.594590+0000","last_fullsized":"2024-04-25T08:45:11.594590+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615757+0000","last_change":"2024-04-25T08:45:11.612056+0000","last_active":"2024-04-25T08:45:11.615757+0000","last_peered":"2024-04-25T08:45:11.615757+0000","last_clean":"2024-04-25T08:45:11.615757+0000","last_became_active":"2024-04-25T08:45:11.611677+0000","last_became_peered":"2024-04-25T08:45:11.611677+0000","last_unstale":"2024-04-25T08:45:11.615757+0000","last_undegraded":"2024-04-25T08:45:11.615757+0000","last_fullsized":"2024-04-25T08:45:11.615757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:29.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602391+0000","last_change":"2024-04-25T08:45:11.602391+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602391+0000","last_undegraded":"2024-04-25T08:45:11.602391+0000","last_fullsized":"2024-04-25T08:45:11.602391+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617146+0000","last_change":"2024-04-25T08:45:11.615380+0000","last_active":"2024-04-25T08:45:11.617146+0000","last_peered":"2024-04-25T08:45:11.617146+0000","last_clean":"2024-04-25T08:45:11.617146+0000","last_became_active":"2024-04-25T08:45:11.612863+0000","last_became_peered":"2024-04-25T08:45:11.612863+0000","last_unstale":"2024-04-25T08:45:11.617146+0000","last_undegraded":"2024-04-25T08:45:11.617146+0000","last_fullsized":"2024-04-25T08:45:11.617146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599723+0000","last_change":"2024-04-25T08:45:11.599723+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599723+0000","last_undegraded":"2024-04-25T08:45:11.599723+0000","last_fullsized":"2024-04-25T08:45:11.599723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614344+0000","last_change":"2024-04-25T08:45:11.609636+0000","last_active":"2024-04-25T08:45:11.614344+0000","last_peered":"2024-04-25T08:45:11.614344+0000","last_clean":"2024-04-25T08:45:11.614344+0000","last_became_active":"2024-04-25T08:45:11.609296+0000","last_became_peered":"2024-04-25T08:45:11.609296+0000","last_unstale":"2024-04-25T08:45:11.614344+0000","last_undegraded":"2024-04-25T08:45:11.614344+0000","last_fullsized":"2024-04-25T08:45:11.614344+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603834+0000","last_change":"2024-04-25T08:45:11.603834+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603834+0000","last_undegraded":"2024-04-25T08:45:11.603834+0000","last_fullsized":"2024-04-25T08:45:11.603834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.619721+0000","last_change":"2024-04-25T08:45:11.618604+0000","last_active":"2024-04-25T08:45:11.619721+0000","last_peered":"2024-04-25T08:45:11.619721+0000","last_clean":"2024-04-25T08:45:11.619721+0000","last_became_active":"2024-04-25T08:45:11.618224+0000","last_became_peered":"2024-04-25T08:45:11.618224+0000","last_unstale":"2024-04-25T08:45:11.619721+0000","last_undegraded":"2024-04-25T08:45:11.619721+0000","last_fullsized":"2024-04-25T08:45:11.619721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:57.910041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596991+0000","last_change":"2024-04-25T08:45:11.596991+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596991+0000","last_undegraded":"2024-04-25T08:45:11.596991+0000","last_fullsized":"2024-04-25T08:45:11.596991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615681+0000","last_change":"2024-04-25T08:45:11.611078+0000","last_active":"2024-04-25T08:45:11.615681+0000","last_peered":"2024-04-25T08:45:11.615681+0000","last_clean":"2024-04-25T08:45:11.615681+0000","last_became_active":"2024-04-25T08:45:11.610749+0000","last_became_peered":"2024-04-25T08:45:11.610749+0000","last_unstale":"2024-04-25T08:45:11.615681+0000","last_undegraded":"2024-04-25T08:45:11.615681+0000","last_fullsized":"2024-04-25T08:45:11.615681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.517500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594513+0000","last_change":"2024-04-25T08:45:11.594513+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594513+0000","last_undegraded":"2024-04-25T08:45:11.594513+0000","last_fullsized":"2024-04-25T08:45:11.594513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615892+0000","last_change":"2024-04-25T08:45:11.613128+0000","last_active":"2024-04-25T08:45:11.615892+0000","last_peered":"2024-04-25T08:45:11.615892+0000","last_clean":"2024-04-25T08:45:11.615892+0000","last_became_active":"2024-04-25T08:45:11.611609+0000","last_became_peered":"2024-04-25T08:45:11.611609+0000","last_unstale":"2024-04-25T08:45:11.615892+0000","last_undegraded":"2024-04-25T08:45:11.615892+0000","last_fullsized":"2024-04-25T08:45:11.615892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:51.214222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594748+0000","last_change":"2024-04-25T08:45:11.594748+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594748+0000","last_undegraded":"2024-04-25T08:45:11.594748+0000","last_fullsized":"2024-04-25T08:45:11.594748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.619710+0000","last_change":"2024-04-25T08:45:11.618549+0000","last_active":"2024-04-25T08:45:11.619710+0000","last_peered":"2024-04-25T08:45:11.619710+0000","last_clean":"2024-04-25T08:45:11.619710+0000","last_became_active":"2024-04-25T08:45:11.618187+0000","last_became_peered":"2024-04-25T08:45:11.618187+0000","last_unstale":"2024-04-25T08:45:11.619710+0000","last_undegraded":"2024-04-25T08:45:11.619710+0000","last_fullsized":"2024-04-25T08:45:11.619710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.517500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.598619+0000","last_change":"2024-04-25T08:45:11.598619+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.598619+0000","last_undegraded":"2024-04-25T08:45:11.598619+0000","last_fullsized":"2024-04-25T08:45:11.598619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614216+0000","last_change":"2024-04-25T08:45:11.609242+0000","last_active":"2024-04-25T08:45:11.614216+0000","last_peered":"2024-04-25T08:45:11.614216+0000","last_clean":"2024-04-25T08:45:11.614216+0000","last_became_active":"2024-04-25T08:45:11.608930+0000","last_became_peered":"2024-04-25T08:45:11.608930+0000","last_unstale":"2024-04-25T08:45:11.614216+0000","last_undegraded":"2024-04-25T08:45:11.614216+0000","last_fullsized":"2024-04-25T08:45:11.614216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601349+0000","last_change":"2024-04-25T08:45:11.601349+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601349+0000","last_undegraded":"2024-04-25T08:45:11.601349+0000","last_fullsized":"2024-04-25T08:45:11.601349+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.613852+0000","last_change":"2024-04-25T08:45:11.608806+0000","last_active":"2024-04-25T08:45:11.613852+0000","last_peered":"2024-04-25T08:45:11.613852+0000","last_clean":"2024-04-25T08:45:11.613852+0000","last_became_active":"2024-04-25T08:45:11.607645+0000","last_became_peered":"2024-04-25T08:45:11.607645+0000","last_unstale":"2024-04-25T08:45:11.613852+0000","last_undegraded":"2024-04-25T08:45:11.613852+0000","last_fullsized":"2024-04-25T08:45:11.613852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:35.633342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596595+0000","last_change":"2024-04-25T08:45:11.596595+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596595+0000","last_undegraded":"2024-04-25T08:45:11.596595+0000","last_fullsized":"2024-04-25T08:45:11.596595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.607835+0000","last_change":"2024-04-25T08:45:11.605238+0000","last_active":"2024-04-25T08:45:11.607835+0000","last_peered":"2024-04-25T08:45:11.607835+0000","last_clean":"2024-04-25T08:45:11.607835+0000","last_became_active":"2024-04-25T08:45:11.604975+0000","last_became_peered":"2024-04-25T08:45:11.604975+0000","last_unstale":"2024-04-25T08:45:11.607835+0000","last_undegraded":"2024-04-25T08:45:11.607835+0000","last_fullsized":"2024-04-25T08:45:11.607835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.879227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601918+0000","last_change":"2024-04-25T08:45:11.601918+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601918+0000","last_undegraded":"2024-04-25T08:45:11.601918+0000","last_fullsized":"2024-04-25T08:45:11.601918+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618499+0000","last_change":"2024-04-25T08:45:11.616493+0000","last_active":"2024-04-25T08:45:11.618499+0000","last_peered":"2024-04-25T08:45:11.618499+0000","last_clean":"2024-04-25T08:45:11.618499+0000","last_became_active":"2024-04-25T08:45:11.616241+0000","last_became_peered":"2024-04-25T08:45:11.616241+0000","last_unstale":"2024-04-25T08:45:11.618499+0000","last_undegraded":"2024-04-25T08:45:11.618499+0000","last_fullsized":"2024-04-25T08:45:11.618499+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:35.847974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603447+0000","last_change":"2024-04-25T08:45:11.603447+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603447+0000","last_undegraded":"2024-04-25T08:45:11.603447+0000","last_fullsized":"2024-04-25T08:45:11.603447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617745+0000","last_change":"2024-04-25T08:45:11.615924+0000","last_active":"2024-04-25T08:45:11.617745+0000","last_peered":"2024-04-25T08:45:11.617745+0000","last_clean":"2024-04-25T08:45:11.617745+0000","last_became_active":"2024-04-25T08:45:11.613543+0000","last_became_peered":"2024-04-25T08:45:11.613543+0000","last_unstale":"2024-04-25T08:45:11.617745+0000","last_undegraded":"2024-04-25T08:45:11.617745+0000","last_fullsized":"2024-04-25T08:45:11.617745+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593904+0000","last_change":"2024-04-25T08:45:11.593904+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593904+0000","last_undegraded":"2024-04-25T08:45:11.593904+0000","last_fullsized":"2024-04-25T08:45:11.593904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617738+0000","last_change":"2024-04-25T08:45:11.613996+0000","last_active":"2024-04-25T08:45:11.617738+0000","last_peered":"2024-04-25T08:45:11.617738+0000","last_clean":"2024-04-25T08:45:11.617738+0000","last_became_active":"2024-04-25T08:45:11.613688+0000","last_became_peered":"2024-04-25T08:45:11.613688+0000","last_unstale":"2024-04-25T08:45:11.617738+0000","last_undegraded":"2024-04-25T08:45:11.617738+0000","last_fullsized":"2024-04-25T08:45:11.617738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:20.076400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600734+0000","last_change":"2024-04-25T08:45:11.600734+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600734+0000","last_undegraded":"2024-04-25T08:45:11.600734+0000","last_fullsized":"2024-04-25T08:45:11.600734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618392+0000","last_change":"2024-04-25T08:45:11.616861+0000","last_active":"2024-04-25T08:45:11.618392+0000","last_peered":"2024-04-25T08:45:11.618392+0000","last_clean":"2024-04-25T08:45:11.618392+0000","last_became_active":"2024-04-25T08:45:11.616539+0000","last_became_peered":"2024-04-25T08:45:11.616539+0000","last_unstale":"2024-04-25T08:45:11.618392+0000","last_undegraded":"2024-04-25T08:45:11.618392+0000","last_fullsized":"2024-04-25T08:45:11.618392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.606983+0000","last_change":"2024-04-25T08:45:11.606983+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.606983+0000","last_undegraded":"2024-04-25T08:45:11.606983+0000","last_fullsized":"2024-04-25T08:45:11.606983+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617742+0000","last_change":"2024-04-25T08:45:11.616486+0000","last_active":"2024-04-25T08:45:11.617742+0000","last_peered":"2024-04-25T08:45:11.617742+0000","last_clean":"2024-04-25T08:45:11.617742+0000","last_became_active":"2024-04-25T08:45:11.614742+0000","last_became_peered":"2024-04-25T08:45:11.614742+0000","last_unstale":"2024-04-25T08:45:11.617742+0000","last_undegraded":"2024-04-25T08:45:11.617742+0000","last_fullsized":"2024-04-25T08:45:11.617742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.605924+0000","last_change":"2024-04-25T08:45:11.605924+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.605924+0000","last_undegraded":"2024-04-25T08:45:11.605924+0000","last_fullsized":"2024-04-25T08:45:11.605924+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617089+0000","last_change":"2024-04-25T08:45:11.615530+0000","last_active":"2024-04-25T08:45:11.617089+0000","last_peered":"2024-04-25T08:45:11.617089+0000","last_clean":"2024-04-25T08:45:11.617089+0000","last_became_active":"2024-04-25T08:45:11.615180+0000","last_became_peered":"2024-04-25T08:45:11.615180+0000","last_unstale":"2024-04-25T08:45:11.617089+0000","last_undegraded":"2024-04-25T08:45:11.617089+0000","last_fullsized":"2024-04-25T08:45:11.617089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:46.666427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596149+0000","last_change":"2024-04-25T08:45:11.596149+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596149+0000","last_undegraded":"2024-04-25T08:45:11.596149+0000","last_fullsized":"2024-04-25T08:45:11.596149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617053+0000","last_change":"2024-04-25T08:45:11.615371+0000","last_active":"2024-04-25T08:45:11.617053+0000","last_peered":"2024-04-25T08:45:11.617053+0000","last_clean":"2024-04-25T08:45:11.617053+0000","last_became_active":"2024-04-25T08:45:11.611714+0000","last_became_peered":"2024-04-25T08:45:11.611714+0000","last_unstale":"2024-04-25T08:45:11.617053+0000","last_undegraded":"2024-04-25T08:45:11.617053+0000","last_fullsized":"2024-04-25T08:45:11.617053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595213+0000","last_change":"2024-04-25T08:45:11.595213+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595213+0000","last_undegraded":"2024-04-25T08:45:11.595213+0000","last_fullsized":"2024-04-25T08:45:11.595213+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614365+0000","last_change":"2024-04-25T08:45:11.610635+0000","last_active":"2024-04-25T08:45:11.614365+0000","last_peered":"2024-04-25T08:45:11.614365+0000","last_clean":"2024-04-25T08:45:11.614365+0000","last_became_active":"2024-04-25T08:45:11.609768+0000","last_became_peered":"2024-04-25T08:45:11.609768+0000","last_unstale":"2024-04-25T08:45:11.614365+0000","last_undegraded":"2024-04-25T08:45:11.614365+0000","last_fullsized":"2024-04-25T08:45:11.614365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.597735+0000","last_change":"2024-04-25T08:45:11.597735+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.597735+0000","last_undegraded":"2024-04-25T08:45:11.597735+0000","last_fullsized":"2024-04-25T08:45:11.597735+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.603873+0000","last_change":"2024-04-25T08:45:11.599255+0000","last_active":"2024-04-25T08:45:11.603873+0000","last_peered":"2024-04-25T08:45:11.603873+0000","last_clean":"2024-04-25T08:45:11.603873+0000","last_became_active":"2024-04-25T08:45:11.598904+0000","last_became_peered":"2024-04-25T08:45:11.598904+0000","last_unstale":"2024-04-25T08:45:11.603873+0000","last_undegraded":"2024-04-25T08:45:11.603873+0000","last_fullsized":"2024-04-25T08:45:11.603873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.303293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595394+0000","last_change":"2024-04-25T08:45:11.595394+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595394+0000","last_undegraded":"2024-04-25T08:45:11.595394+0000","last_fullsized":"2024-04-25T08:45:11.595394+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.609055+0000","last_change":"2024-04-25T08:45:11.603866+0000","last_active":"2024-04-25T08:45:11.609055+0000","last_peered":"2024-04-25T08:45:11.609055+0000","last_clean":"2024-04-25T08:45:11.609055+0000","last_became_active":"2024-04-25T08:45:11.603106+0000","last_became_peered":"2024-04-25T08:45:11.603106+0000","last_unstale":"2024-04-25T08:45:11.609055+0000","last_undegraded":"2024-04-25T08:45:11.609055+0000","last_fullsized":"2024-04-25T08:45:11.609055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:35.392449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.607579+0000","last_change":"2024-04-25T08:45:11.607579+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.607579+0000","last_undegraded":"2024-04-25T08:45:11.607579+0000","last_fullsized":"2024-04-25T08:45:11.607579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614345+0000","last_change":"2024-04-25T08:45:11.610404+0000","last_active":"2024-04-25T08:45:11.614345+0000","last_peered":"2024-04-25T08:45:11.614345+0000","last_clean":"2024-04-25T08:45:11.614345+0000","last_became_active":"2024-04-25T08:45:11.610147+0000","last_became_peered":"2024-04-25T08:45:11.610147+0000","last_unstale":"2024-04-25T08:45:11.614345+0000","last_undegraded":"2024-04-25T08:45:11.614345+0000","last_fullsized":"2024-04-25T08:45:11.614345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:35.633342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593591+0000","last_change":"2024-04-25T08:45:11.593591+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593591+0000","last_undegraded":"2024-04-25T08:45:11.593591+0000","last_fullsized":"2024-04-25T08:45:11.593591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.612199+0000","last_change":"2024-04-25T08:45:11.606470+0000","last_active":"2024-04-25T08:45:11.612199+0000","last_peered":"2024-04-25T08:45:11.612199+0000","last_clean":"2024-04-25T08:45:11.612199+0000","last_became_active":"2024-04-25T08:45:11.606206+0000","last_became_peered":"2024-04-25T08:45:11.606206+0000","last_unstale":"2024-04-25T08:45:11.612199+0000","last_undegraded":"2024-04-25T08:45:11.612199+0000","last_fullsized":"2024-04-25T08:45:11.612199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:46.666427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.595703+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:45:11.595703+0000","last_peered":"2024-04-25T08:45:11.595703+0000","last_clean":"2024-04-25T08:45:11.595703+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:45:11.595703+0000","last_undegraded":"2024-04-25T08:45:11.595703+0000","last_fullsized":"2024-04-25T08:45:11.595703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":932113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":335086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":749568,"data_stored":335086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":932113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:45:17.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3f is not active+clean 2024-04-25T08:45:17.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.596202+0000', 'last_change': '2024-04-25T08:45:11.596202+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.596202+0000', 'last_undegraded': '2024-04-25T08:45:11.596202+0000', 'last_fullsized': '2024-04-25T08:45:11.596202+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:45:17.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3e is not active+clean 2024-04-25T08:45:17.501 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.595066+0000', 'last_change': '2024-04-25T08:45:11.595066+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.595066+0000', 'last_undegraded': '2024-04-25T08:45:11.595066+0000', 'last_fullsized': '2024-04-25T08:45:11.595066+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:45:17.501 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3d is not active+clean 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.592923+0000', 'last_change': '2024-04-25T08:45:11.592923+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.592923+0000', 'last_undegraded': '2024-04-25T08:45:11.592923+0000', 'last_fullsized': '2024-04-25T08:45:11.592923+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3c is not active+clean 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.599420+0000', 'last_change': '2024-04-25T08:45:11.599420+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.599420+0000', 'last_undegraded': '2024-04-25T08:45:11.599420+0000', 'last_fullsized': '2024-04-25T08:45:11.599420+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3b is not active+clean 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.594152+0000', 'last_change': '2024-04-25T08:45:11.594152+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.594152+0000', 'last_undegraded': '2024-04-25T08:45:11.594152+0000', 'last_fullsized': '2024-04-25T08:45:11.594152+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3a is not active+clean 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.600272+0000', 'last_change': '2024-04-25T08:45:11.600272+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.600272+0000', 'last_undegraded': '2024-04-25T08:45:11.600272+0000', 'last_fullsized': '2024-04-25T08:45:11.600272+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.39 is not active+clean 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.605419+0000', 'last_change': '2024-04-25T08:45:11.605419+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.605419+0000', 'last_undegraded': '2024-04-25T08:45:11.605419+0000', 'last_fullsized': '2024-04-25T08:45:11.605419+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:45:17.502 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.38 is not active+clean 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.602566+0000', 'last_change': '2024-04-25T08:45:11.602566+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.602566+0000', 'last_undegraded': '2024-04-25T08:45:11.602566+0000', 'last_fullsized': '2024-04-25T08:45:11.602566+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.37 is not active+clean 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593504+0000', 'last_change': '2024-04-25T08:45:11.593504+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593504+0000', 'last_undegraded': '2024-04-25T08:45:11.593504+0000', 'last_fullsized': '2024-04-25T08:45:11.593504+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.36 is not active+clean 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.592774+0000', 'last_change': '2024-04-25T08:45:11.592774+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.592774+0000', 'last_undegraded': '2024-04-25T08:45:11.592774+0000', 'last_fullsized': '2024-04-25T08:45:11.592774+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.35 is not active+clean 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.600340+0000', 'last_change': '2024-04-25T08:45:11.600340+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.600340+0000', 'last_undegraded': '2024-04-25T08:45:11.600340+0000', 'last_fullsized': '2024-04-25T08:45:11.600340+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.34 is not active+clean 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.608903+0000', 'last_change': '2024-04-25T08:45:11.608903+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.608903+0000', 'last_undegraded': '2024-04-25T08:45:11.608903+0000', 'last_fullsized': '2024-04-25T08:45:11.608903+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.33 is not active+clean 2024-04-25T08:45:17.503 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.604995+0000', 'last_change': '2024-04-25T08:45:11.604995+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.604995+0000', 'last_undegraded': '2024-04-25T08:45:11.604995+0000', 'last_fullsized': '2024-04-25T08:45:11.604995+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.32 is not active+clean 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593267+0000', 'last_change': '2024-04-25T08:45:11.593267+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593267+0000', 'last_undegraded': '2024-04-25T08:45:11.593267+0000', 'last_fullsized': '2024-04-25T08:45:11.593267+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.31 is not active+clean 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.594640+0000', 'last_change': '2024-04-25T08:45:11.594640+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.594640+0000', 'last_undegraded': '2024-04-25T08:45:11.594640+0000', 'last_fullsized': '2024-04-25T08:45:11.594640+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.30 is not active+clean 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.606433+0000', 'last_change': '2024-04-25T08:45:11.606433+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.606433+0000', 'last_undegraded': '2024-04-25T08:45:11.606433+0000', 'last_fullsized': '2024-04-25T08:45:11.606433+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2f is not active+clean 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.595245+0000', 'last_change': '2024-04-25T08:45:11.595245+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.595245+0000', 'last_undegraded': '2024-04-25T08:45:11.595245+0000', 'last_fullsized': '2024-04-25T08:45:11.595245+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2e is not active+clean 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.601808+0000', 'last_change': '2024-04-25T08:45:11.601808+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.601808+0000', 'last_undegraded': '2024-04-25T08:45:11.601808+0000', 'last_fullsized': '2024-04-25T08:45:11.601808+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2d is not active+clean 2024-04-25T08:45:17.504 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.594375+0000', 'last_change': '2024-04-25T08:45:11.594375+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.594375+0000', 'last_undegraded': '2024-04-25T08:45:11.594375+0000', 'last_fullsized': '2024-04-25T08:45:11.594375+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2c is not active+clean 2024-04-25T08:45:17.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593868+0000', 'last_change': '2024-04-25T08:45:11.593868+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593868+0000', 'last_undegraded': '2024-04-25T08:45:11.593868+0000', 'last_fullsized': '2024-04-25T08:45:11.593868+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:45:17.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2b is not active+clean 2024-04-25T08:45:17.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.604594+0000', 'last_change': '2024-04-25T08:45:11.604594+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.604594+0000', 'last_undegraded': '2024-04-25T08:45:11.604594+0000', 'last_fullsized': '2024-04-25T08:45:11.604594+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2a is not active+clean 2024-04-25T08:45:17.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593331+0000', 'last_change': '2024-04-25T08:45:11.593331+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593331+0000', 'last_undegraded': '2024-04-25T08:45:11.593331+0000', 'last_fullsized': '2024-04-25T08:45:11.593331+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:45:17.505 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.29 is not active+clean 2024-04-25T08:45:17.505 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593128+0000', 'last_change': '2024-04-25T08:45:11.593128+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593128+0000', 'last_undegraded': '2024-04-25T08:45:11.593128+0000', 'last_fullsized': '2024-04-25T08:45:11.593128+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.28 is not active+clean 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.592664+0000', 'last_change': '2024-04-25T08:45:11.592664+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.592664+0000', 'last_undegraded': '2024-04-25T08:45:11.592664+0000', 'last_fullsized': '2024-04-25T08:45:11.592664+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.27 is not active+clean 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.604961+0000', 'last_change': '2024-04-25T08:45:11.604961+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.604961+0000', 'last_undegraded': '2024-04-25T08:45:11.604961+0000', 'last_fullsized': '2024-04-25T08:45:11.604961+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.26 is not active+clean 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.608708+0000', 'last_change': '2024-04-25T08:45:11.608708+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.608708+0000', 'last_undegraded': '2024-04-25T08:45:11.608708+0000', 'last_fullsized': '2024-04-25T08:45:11.608708+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.25 is not active+clean 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.599852+0000', 'last_change': '2024-04-25T08:45:11.599852+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.599852+0000', 'last_undegraded': '2024-04-25T08:45:11.599852+0000', 'last_fullsized': '2024-04-25T08:45:11.599852+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.24 is not active+clean 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.595481+0000', 'last_change': '2024-04-25T08:45:11.595481+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.595481+0000', 'last_undegraded': '2024-04-25T08:45:11.595481+0000', 'last_fullsized': '2024-04-25T08:45:11.595481+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:45:17.506 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.23 is not active+clean 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.592683+0000', 'last_change': '2024-04-25T08:45:11.592683+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.592683+0000', 'last_undegraded': '2024-04-25T08:45:11.592683+0000', 'last_fullsized': '2024-04-25T08:45:11.592683+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.22 is not active+clean 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.604336+0000', 'last_change': '2024-04-25T08:45:11.604336+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.604336+0000', 'last_undegraded': '2024-04-25T08:45:11.604336+0000', 'last_fullsized': '2024-04-25T08:45:11.604336+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.21 is not active+clean 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.598275+0000', 'last_change': '2024-04-25T08:45:11.598275+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.598275+0000', 'last_undegraded': '2024-04-25T08:45:11.598275+0000', 'last_fullsized': '2024-04-25T08:45:11.598275+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.20 is not active+clean 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.602360+0000', 'last_change': '2024-04-25T08:45:11.602360+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.602360+0000', 'last_undegraded': '2024-04-25T08:45:11.602360+0000', 'last_fullsized': '2024-04-25T08:45:11.602360+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593787+0000', 'last_change': '2024-04-25T08:45:11.593787+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593787+0000', 'last_undegraded': '2024-04-25T08:45:11.593787+0000', 'last_fullsized': '2024-04-25T08:45:11.593787+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2024-04-25T08:45:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.599374+0000', 'last_change': '2024-04-25T08:45:11.599374+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.599374+0000', 'last_undegraded': '2024-04-25T08:45:11.599374+0000', 'last_fullsized': '2024-04-25T08:45:11.599374+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.603765+0000', 'last_change': '2024-04-25T08:45:11.603765+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.603765+0000', 'last_undegraded': '2024-04-25T08:45:11.603765+0000', 'last_fullsized': '2024-04-25T08:45:11.603765+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.594756+0000', 'last_change': '2024-04-25T08:45:11.594756+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.594756+0000', 'last_undegraded': '2024-04-25T08:45:11.594756+0000', 'last_fullsized': '2024-04-25T08:45:11.594756+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.603978+0000', 'last_change': '2024-04-25T08:45:11.603978+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.603978+0000', 'last_undegraded': '2024-04-25T08:45:11.603978+0000', 'last_fullsized': '2024-04-25T08:45:11.603978+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.594081+0000', 'last_change': '2024-04-25T08:45:11.594081+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.594081+0000', 'last_undegraded': '2024-04-25T08:45:11.594081+0000', 'last_fullsized': '2024-04-25T08:45:11.594081+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.597842+0000', 'last_change': '2024-04-25T08:45:11.597842+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.597842+0000', 'last_undegraded': '2024-04-25T08:45:11.597842+0000', 'last_fullsized': '2024-04-25T08:45:11.597842+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.508 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593075+0000', 'last_change': '2024-04-25T08:45:11.593075+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593075+0000', 'last_undegraded': '2024-04-25T08:45:11.593075+0000', 'last_fullsized': '2024-04-25T08:45:11.593075+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.594910+0000', 'last_change': '2024-04-25T08:45:11.594910+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.594910+0000', 'last_undegraded': '2024-04-25T08:45:11.594910+0000', 'last_fullsized': '2024-04-25T08:45:11.594910+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593106+0000', 'last_change': '2024-04-25T08:45:11.593106+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593106+0000', 'last_undegraded': '2024-04-25T08:45:11.593106+0000', 'last_fullsized': '2024-04-25T08:45:11.593106+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.594590+0000', 'last_change': '2024-04-25T08:45:11.594590+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.594590+0000', 'last_undegraded': '2024-04-25T08:45:11.594590+0000', 'last_fullsized': '2024-04-25T08:45:11.594590+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.602391+0000', 'last_change': '2024-04-25T08:45:11.602391+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.602391+0000', 'last_undegraded': '2024-04-25T08:45:11.602391+0000', 'last_fullsized': '2024-04-25T08:45:11.602391+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2024-04-25T08:45:17.509 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.599723+0000', 'last_change': '2024-04-25T08:45:11.599723+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.599723+0000', 'last_undegraded': '2024-04-25T08:45:11.599723+0000', 'last_fullsized': '2024-04-25T08:45:11.599723+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.603834+0000', 'last_change': '2024-04-25T08:45:11.603834+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.603834+0000', 'last_undegraded': '2024-04-25T08:45:11.603834+0000', 'last_fullsized': '2024-04-25T08:45:11.603834+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.596991+0000', 'last_change': '2024-04-25T08:45:11.596991+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.596991+0000', 'last_undegraded': '2024-04-25T08:45:11.596991+0000', 'last_fullsized': '2024-04-25T08:45:11.596991+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.594513+0000', 'last_change': '2024-04-25T08:45:11.594513+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.594513+0000', 'last_undegraded': '2024-04-25T08:45:11.594513+0000', 'last_fullsized': '2024-04-25T08:45:11.594513+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.594748+0000', 'last_change': '2024-04-25T08:45:11.594748+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.594748+0000', 'last_undegraded': '2024-04-25T08:45:11.594748+0000', 'last_fullsized': '2024-04-25T08:45:11.594748+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.598619+0000', 'last_change': '2024-04-25T08:45:11.598619+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.598619+0000', 'last_undegraded': '2024-04-25T08:45:11.598619+0000', 'last_fullsized': '2024-04-25T08:45:11.598619+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.510 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.601349+0000', 'last_change': '2024-04-25T08:45:11.601349+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.601349+0000', 'last_undegraded': '2024-04-25T08:45:11.601349+0000', 'last_fullsized': '2024-04-25T08:45:11.601349+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.596595+0000', 'last_change': '2024-04-25T08:45:11.596595+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.596595+0000', 'last_undegraded': '2024-04-25T08:45:11.596595+0000', 'last_fullsized': '2024-04-25T08:45:11.596595+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.601918+0000', 'last_change': '2024-04-25T08:45:11.601918+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.601918+0000', 'last_undegraded': '2024-04-25T08:45:11.601918+0000', 'last_fullsized': '2024-04-25T08:45:11.601918+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.603447+0000', 'last_change': '2024-04-25T08:45:11.603447+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.603447+0000', 'last_undegraded': '2024-04-25T08:45:11.603447+0000', 'last_fullsized': '2024-04-25T08:45:11.603447+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593904+0000', 'last_change': '2024-04-25T08:45:11.593904+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593904+0000', 'last_undegraded': '2024-04-25T08:45:11.593904+0000', 'last_fullsized': '2024-04-25T08:45:11.593904+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2024-04-25T08:45:17.511 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.600734+0000', 'last_change': '2024-04-25T08:45:11.600734+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.600734+0000', 'last_undegraded': '2024-04-25T08:45:11.600734+0000', 'last_fullsized': '2024-04-25T08:45:11.600734+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.606983+0000', 'last_change': '2024-04-25T08:45:11.606983+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.606983+0000', 'last_undegraded': '2024-04-25T08:45:11.606983+0000', 'last_fullsized': '2024-04-25T08:45:11.606983+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.605924+0000', 'last_change': '2024-04-25T08:45:11.605924+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.605924+0000', 'last_undegraded': '2024-04-25T08:45:11.605924+0000', 'last_fullsized': '2024-04-25T08:45:11.605924+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.596149+0000', 'last_change': '2024-04-25T08:45:11.596149+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.596149+0000', 'last_undegraded': '2024-04-25T08:45:11.596149+0000', 'last_fullsized': '2024-04-25T08:45:11.596149+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.595213+0000', 'last_change': '2024-04-25T08:45:11.595213+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.595213+0000', 'last_undegraded': '2024-04-25T08:45:11.595213+0000', 'last_fullsized': '2024-04-25T08:45:11.595213+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.597735+0000', 'last_change': '2024-04-25T08:45:11.597735+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.597735+0000', 'last_undegraded': '2024-04-25T08:45:11.597735+0000', 'last_fullsized': '2024-04-25T08:45:11.597735+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2024-04-25T08:45:17.512 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.595394+0000', 'last_change': '2024-04-25T08:45:11.595394+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.595394+0000', 'last_undegraded': '2024-04-25T08:45:11.595394+0000', 'last_fullsized': '2024-04-25T08:45:11.595394+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:45:17.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2024-04-25T08:45:17.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.607579+0000', 'last_change': '2024-04-25T08:45:11.607579+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.607579+0000', 'last_undegraded': '2024-04-25T08:45:11.607579+0000', 'last_fullsized': '2024-04-25T08:45:11.607579+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:45:17.513 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2024-04-25T08:45:17.513 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:45:11.593591+0000', 'last_change': '2024-04-25T08:45:11.593591+0000', 'last_active': '2024-04-25T08:45:11.583601+0000', 'last_peered': '2024-04-25T08:45:11.583601+0000', 'last_clean': '2024-04-25T08:45:11.583601+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:45:11.593591+0000', 'last_undegraded': '2024-04-25T08:45:11.593591+0000', 'last_fullsized': '2024-04-25T08:45:11.593591+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'last_clean_scrub_stamp': '2024-04-25T08:45:11.583601+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:45:17.513 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:45:17.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.686+0000 7f64f5600640 1 Processor -- start 2024-04-25T08:45:17.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.686+0000 7f64f5600640 1 -- start start 2024-04-25T08:45:17.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64f5600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 0x7f64f006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64f5600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64f00b3ed0 0x7f64f00cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64f5600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64f00cc820 0x7f64f0071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64f5600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f006ecc0 con 0x7f64f00b3ed0 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64f5600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f00b3110 con 0x7f64f00cc820 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64f5600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f00b33f0 con 0x7f64f006e3a0 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64e6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64f00b3ed0 0x7f64f00cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64e6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64f00b3ed0 0x7f64f00cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33686/0 (socket says 172.21.15.46:33686) 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64e6000640 1 -- 172.21.15.46:0/3389214661 learned_addr learned my addr 172.21.15.46:0/3389214661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64ef400640 1 --2- 172.21.15.46:0/3389214661 >> v2:172.21.15.82:3300/0 conn(0x7f64f00cc820 0x7f64f0071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64eea00640 1 --2- 172.21.15.46:0/3389214661 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 0x7f64f006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64e6000640 1 -- 172.21.15.46:0/3389214661 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 msgr2=0x7f64f006e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64e6000640 1 --2- 172.21.15.46:0/3389214661 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 0x7f64f006e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64e6000640 1 -- 172.21.15.46:0/3389214661 >> v2:172.21.15.82:3300/0 conn(0x7f64f00cc820 msgr2=0x7f64f0071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64e6000640 1 --2- 172.21.15.46:0/3389214661 >> v2:172.21.15.82:3300/0 conn(0x7f64f00cc820 0x7f64f0071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64e6000640 1 -- 172.21.15.46:0/3389214661 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f00b36d0 con 0x7f64f00b3ed0 2024-04-25T08:45:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64ef400640 1 --2- 172.21.15.46:0/3389214661 >> v2:172.21.15.82:3300/0 conn(0x7f64f00cc820 0x7f64f0071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64e6000640 1 --2- 172.21.15.46:0/3389214661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00b3ed0 0x7f64f00cc2e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f64f006f080 tx=0x7f64d801e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d18a38564d9335 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.690+0000 7f64ee000640 1 -- 172.21.15.46:0/3389214661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80b4070 con 0x7f64f00b3ed0 2024-04-25T08:45:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64ee000640 1 -- 172.21.15.46:0/3389214661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d80a78d0 con 0x7f64f00b3ed0 2024-04-25T08:45:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64ee000640 1 -- 172.21.15.46:0/3389214661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d80a7ca0 con 0x7f64f00b3ed0 2024-04-25T08:45:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 -- 172.21.15.46:0/3389214661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00b3ed0 msgr2=0x7f64f00cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 --2- 172.21.15.46:0/3389214661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00b3ed0 0x7f64f00cc2e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f64f006f080 tx=0x7f64d801e930 comp rx=0 tx=0).stop 2024-04-25T08:45:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 -- 172.21.15.46:0/3389214661 shutdown_connections 2024-04-25T08:45:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 --2- 172.21.15.46:0/3389214661 >> v2:172.21.15.82:3300/0 conn(0x7f64f00cc820 0x7f64f0071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 --2- 172.21.15.46:0/3389214661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00b3ed0 0x7f64f00cc2e0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 --2- 172.21.15.46:0/3389214661 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 0x7f64f006e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 -- 172.21.15.46:0/3389214661 >> 172.21.15.46:0/3389214661 conn(0x7f64f0067000 msgr2=0x7f64f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 -- 172.21.15.46:0/3389214661 shutdown_connections 2024-04-25T08:45:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 -- 172.21.15.46:0/3389214661 wait complete. 2024-04-25T08:45:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 Processor -- start 2024-04-25T08:45:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 -- start start 2024-04-25T08:45:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 0x7f64f00ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00b3ed0 0x7f64f006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00cc820 0x7f64f006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f015ab50 con 0x7f64f00cc820 2024-04-25T08:45:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f015acd0 con 0x7f64f00b3ed0 2024-04-25T08:45:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64f5600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f015ae50 con 0x7f64f006e3a0 2024-04-25T08:45:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00cc820 0x7f64f006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00cc820 0x7f64f006d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33690/0 (socket says 172.21.15.46:33690) 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64ef400640 1 -- 172.21.15.46:0/1912579072 learned_addr learned my addr 172.21.15.46:0/1912579072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64eea00640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 0x7f64f00ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.694+0000 7f64e6000640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00b3ed0 0x7f64f006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64ef400640 1 -- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 msgr2=0x7f64f00ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64ef400640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 0x7f64f00ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64ef400640 1 -- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00b3ed0 msgr2=0x7f64f006d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64ef400640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00b3ed0 0x7f64f006d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64ef400640 1 -- 172.21.15.46:0/1912579072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f013e5f0 con 0x7f64f00cc820 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64ef400640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00cc820 0x7f64f006d720 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f64e8007e70 tx=0x7f64e800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64e7400640 1 -- 172.21.15.46:0/1912579072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e801a070 con 0x7f64f00cc820 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64e7400640 1 -- 172.21.15.46:0/1912579072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e8003bc0 con 0x7f64f00cc820 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64e7400640 1 -- 172.21.15.46:0/1912579072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8004030 con 0x7f64f00cc820 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f00ab210 con 0x7f64f00cc820 2024-04-25T08:45:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f00ab7d0 con 0x7f64f00cc820 2024-04-25T08:45:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f0074c60 con 0x7f64f00cc820 2024-04-25T08:45:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64e7400640 1 -- 172.21.15.46:0/1912579072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64e801d070 con 0x7f64f00cc820 2024-04-25T08:45:17.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64e7400640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:17.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64e7400640 1 -- 172.21.15.46:0/1912579072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f64e80853d0 con 0x7f64f00cc820 2024-04-25T08:45:17.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.698+0000 7f64eea00640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:17.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.702+0000 7f64eea00640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f64e00045f0 tx=0x7f64e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.702+0000 7f64e7400640 1 -- 172.21.15.46:0/1912579072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64e8010070 con 0x7f64f00cc820 2024-04-25T08:45:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.922+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f64f015a8c0 con 0x7f64f00cc820 2024-04-25T08:45:17.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.922+0000 7f64e7400640 1 -- 172.21.15.46:0/1912579072 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1453 (secure 0 0 0) 0x7f64e8055bf0 con 0x7f64f00cc820 2024-04-25T08:45:17.916 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:17.916 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":202,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":8,"num_up_osds":8,"osd_up_since":1714034527,"num_in_osds":8,"osd_in_since":1714034517,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":249937920,"bytes_avail":772844175360,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":34,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6207}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-25T08:44:02.278428+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:45:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 msgr2=0x7f64cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f64e00045f0 tx=0x7f64e0008040 comp rx=0 tx=0).stop 2024-04-25T08:45:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00cc820 msgr2=0x7f64f006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00cc820 0x7f64f006d720 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f64e8007e70 tx=0x7f64e800d4a0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 shutdown_connections 2024-04-25T08:45:17.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f00cc820 0x7f64f006d720 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00b3ed0 0x7f64f006d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 --2- 172.21.15.46:0/1912579072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f006e3a0 0x7f64f00ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:17.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 >> 172.21.15.46:0/1912579072 conn(0x7f64f0067000 msgr2=0x7f64f0142080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:17.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 shutdown_connections 2024-04-25T08:45:17.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:17.926+0000 7f64f5600640 1 -- 172.21.15.46:0/1912579072 wait complete. 2024-04-25T08:45:17.938 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:45:17.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:45:18.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f674b200640 1 Processor -- start 2024-04-25T08:45:18.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f674b200640 1 -- start start 2024-04-25T08:45:18.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f674b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f674c0b29c0 0x7f674c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:18.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f674b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c06c2b0 0x7f674c161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:18.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f674b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674c0b2da0 con 0x7f674c099a00 2024-04-25T08:45:18.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f674b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f674c1623a0 con 0x7f674c0b29c0 2024-04-25T08:45:18.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f674b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c161de0 con 0x7f674c06c2b0 2024-04-25T08:45:18.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f6749e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f674c0b29c0 0x7f674c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:18.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f6749e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f674c0b29c0 0x7f674c15f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49598/0 (socket says 172.21.15.46:49598) 2024-04-25T08:45:18.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f6749e00640 1 -- 172.21.15.46:0/1391687715 learned_addr learned my addr 172.21.15.46:0/1391687715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:18.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f6741400640 1 --2- 172.21.15.46:0/1391687715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c06c2b0 0x7f674c161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:18.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f6749e00640 1 -- 172.21.15.46:0/1391687715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c06c2b0 msgr2=0x7f674c161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:18.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f6749e00640 1 --2- 172.21.15.46:0/1391687715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c06c2b0 0x7f674c161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f6749e00640 1 -- 172.21.15.46:0/1391687715 >> v1:172.21.15.82:6789/0 conn(0x7f674c099a00 legacy=0x7f674c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:18.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.086+0000 7f6749e00640 1 -- 172.21.15.46:0/1391687715 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c1620c0 con 0x7f674c0b29c0 2024-04-25T08:45:18.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f6749e00640 1 --2- 172.21.15.46:0/1391687715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c0b29c0 0x7f674c15f250 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f673c009e50 tx=0x7f673c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1de77b1a1c2a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:18.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f6749400640 1 -- 172.21.15.46:0/1391687715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0a58d0 con 0x7f674c0b29c0 2024-04-25T08:45:18.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f6749400640 1 -- 172.21.15.46:0/1391687715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c0a5a70 con 0x7f674c0b29c0 2024-04-25T08:45:18.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f6749400640 1 -- 172.21.15.46:0/1391687715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0a5e40 con 0x7f674c0b29c0 2024-04-25T08:45:18.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f674b200640 1 -- 172.21.15.46:0/1391687715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c0b29c0 msgr2=0x7f674c15f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:18.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f674b200640 1 --2- 172.21.15.46:0/1391687715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c0b29c0 0x7f674c15f250 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f673c009e50 tx=0x7f673c01ca00 comp rx=0 tx=0).stop 2024-04-25T08:45:18.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f674b200640 1 -- 172.21.15.46:0/1391687715 shutdown_connections 2024-04-25T08:45:18.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f674b200640 1 --2- 172.21.15.46:0/1391687715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c06c2b0 0x7f674c161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f674b200640 1 --2- 172.21.15.46:0/1391687715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c0b29c0 0x7f674c15f250 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f674b200640 1 -- 172.21.15.46:0/1391687715 >> 172.21.15.46:0/1391687715 conn(0x7f674c0b2f90 msgr2=0x7f674c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:18.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f674b200640 1 -- 172.21.15.46:0/1391687715 shutdown_connections 2024-04-25T08:45:18.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.090+0000 7f674b200640 1 -- 172.21.15.46:0/1391687715 wait complete. 2024-04-25T08:45:18.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 Processor -- start 2024-04-25T08:45:18.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 -- start start 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674c06c2b0 0x7f674c0c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c099a00 0x7f674c0b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c0b29c0 0x7f674c14b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0d0e50 con 0x7f674c099a00 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674c0d0cd0 con 0x7f674c06c2b0 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674c1619b0 con 0x7f674c0b29c0 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6741400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c099a00 0x7f674c0b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6741400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c099a00 0x7f674c0b6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49606/0 (socket says 172.21.15.46:49606) 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6741400640 1 -- 172.21.15.46:0/1142116114 learned_addr learned my addr 172.21.15.46:0/1142116114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6749e00640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674c06c2b0 0x7f674c0c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:18.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674a800640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c0b29c0 0x7f674c14b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6741400640 1 -- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c0b29c0 msgr2=0x7f674c14b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6741400640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c0b29c0 0x7f674c14b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6741400640 1 -- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674c06c2b0 msgr2=0x7f674c0c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6741400640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674c06c2b0 0x7f674c0c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6741400640 1 -- 172.21.15.46:0/1142116114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c14bce0 con 0x7f674c099a00 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6741400640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c099a00 0x7f674c0b6900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f673400d800 tx=0x7f673400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6742a00640 1 -- 172.21.15.46:0/1142116114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400b6c0 con 0x7f674c099a00 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6742a00640 1 -- 172.21.15.46:0/1142116114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6734019070 con 0x7f674c099a00 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f6742a00640 1 -- 172.21.15.46:0/1142116114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400ba60 con 0x7f674c099a00 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674c0c5600 con 0x7f674c099a00 2024-04-25T08:45:18.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.094+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c0c5bc0 con 0x7f674c099a00 2024-04-25T08:45:18.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.098+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674c0c62e0 con 0x7f674c099a00 2024-04-25T08:45:18.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.098+0000 7f6742a00640 1 -- 172.21.15.46:0/1142116114 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f673400bc00 con 0x7f674c099a00 2024-04-25T08:45:18.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.098+0000 7f6742a00640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6728066f60 0x7f6728069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:18.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.098+0000 7f6742a00640 1 -- 172.21.15.46:0/1142116114 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6734085530 con 0x7f674c099a00 2024-04-25T08:45:18.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.098+0000 7f6749e00640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6728066f60 0x7f6728069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:18.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.098+0000 7f6749e00640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6728066f60 0x7f6728069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f673c0ae000 tx=0x7f673c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:18.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.102+0000 7f6742a00640 1 -- 172.21.15.46:0/1142116114 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f674c14bce0 con 0x7f674c099a00 2024-04-25T08:45:18.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.262+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f674c069010 con 0x7f6728066f60 2024-04-25T08:45:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.266+0000 7f6742a00640 1 -- 172.21.15.46:0/1142116114 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246711 (secure 0 0 0) 0x7f674c069010 con 0x7f6728066f60 2024-04-25T08:45:18.262 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:18.264 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6728066f60 msgr2=0x7f6728069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6728066f60 0x7f6728069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f673c0ae000 tx=0x7f673c0a7070 comp rx=0 tx=0).stop 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c099a00 msgr2=0x7f674c0b6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c099a00 0x7f674c0b6900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f673400d800 tx=0x7f673400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 shutdown_connections 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6728066f60 0x7f6728069420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f674c0b29c0 0x7f674c14b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674c099a00 0x7f674c0b6900 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 --2- 172.21.15.46:0/1142116114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674c06c2b0 0x7f674c0c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:18.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 >> 172.21.15.46:0/1142116114 conn(0x7f674c0b2f90 msgr2=0x7f674c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:18.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 shutdown_connections 2024-04-25T08:45:18.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:18.270+0000 7f674b200640 1 -- 172.21.15.46:0/1142116114 wait complete. 2024-04-25T08:45:18.283 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-04-25T08:45:16.297142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244080,"kb_used_data":7088,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730640,"statfs":{"total":773094113280,"available":772844175360,"internally_reserved":0,"allocated":7258112,"data_stored":3901378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.713434"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596202+0000","last_change":"2024-04-25T08:45:11.596202+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596202+0000","last_undegraded":"2024-04-25T08:45:11.596202+0000","last_fullsized":"2024-04-25T08:45:11.596202+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595066+0000","last_change":"2024-04-25T08:45:11.595066+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595066+0000","last_undegraded":"2024-04-25T08:45:11.595066+0000","last_fullsized":"2024-04-25T08:45:11.595066+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592923+0000","last_change":"2024-04-25T08:45:11.592923+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592923+0000","last_undegraded":"2024-04-25T08:45:11.592923+0000","last_fullsized":"2024-04-25T08:45:11.592923+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599420+0000","last_change":"2024-04-25T08:45:11.599420+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599420+0000","last_undegraded":"2024-04-25T08:45:11.599420+0000","last_fullsized":"2024-04-25T08:45:11.599420+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594152+0000","last_change":"2024-04-25T08:45:11.594152+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594152+0000","last_undegraded":"2024-04-25T08:45:11.594152+0000","last_fullsized":"2024-04-25T08:45:11.594152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600272+0000","last_change":"2024-04-25T08:45:11.600272+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600272+0000","last_undegraded":"2024-04-25T08:45:11.600272+0000","last_fullsized":"2024-04-25T08:45:11.600272+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.605419+0000","last_change":"2024-04-25T08:45:11.605419+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.605419+0000","last_undegraded":"2024-04-25T08:45:11.605419+0000","last_fullsized":"2024-04-25T08:45:11.605419+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602566+0000","last_change":"2024-04-25T08:45:11.602566+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602566+0000","last_undegraded":"2024-04-25T08:45:11.602566+0000","last_fullsized":"2024-04-25T08:45:11.602566+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593504+0000","last_change":"2024-04-25T08:45:11.593504+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593504+0000","last_undegraded":"2024-04-25T08:45:11.593504+0000","last_fullsized":"2024-04-25T08:45:11.593504+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592774+0000","last_change":"2024-04-25T08:45:11.592774+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592774+0000","last_undegraded":"2024-04-25T08:45:11.592774+0000","last_fullsized":"2024-04-25T08:45:11.592774+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600340+0000","last_change":"2024-04-25T08:45:11.600340+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600340+0000","last_undegraded":"2024-04-25T08:45:11.600340+0000","last_fullsized":"2024-04-25T08:45:11.600340+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.608903+0000","last_change":"2024-04-25T08:45:11.608903+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.608903+0000","last_undegraded":"2024-04-25T08:45:11.608903+0000","last_fullsized":"2024-04-25T08:45:11.608903+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604995+0000","last_change":"2024-04-25T08:45:11.604995+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604995+0000","last_undegraded":"2024-04-25T08:45:11.604995+0000","last_fullsized":"2024-04-25T08:45:11.604995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593267+0000","last_change":"2024-04-25T08:45:11.593267+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593267+0000","last_undegraded":"2024-04-25T08:45:11.593267+0000","last_fullsized":"2024-04-25T08:45:11.593267+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594640+0000","last_change":"2024-04-25T08:45:11.594640+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594640+0000","last_undegraded":"2024-04-25T08:45:11.594640+0000","last_fullsized":"2024-04-25T08:45:11.594640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.606433+0000","last_change":"2024-04-25T08:45:11.606433+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.606433+0000","last_undegraded":"2024-04-25T08:45:11.606433+0000","last_fullsized":"2024-04-25T08:45:11.606433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595245+0000","last_change":"2024-04-25T08:45:11.595245+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595245+0000","last_undegraded":"2024-04-25T08:45:11.595245+0000","last_fullsized":"2024-04-25T08:45:11.595245+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601808+0000","last_change":"2024-04-25T08:45:11.601808+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601808+0000","last_undegraded":"2024-04-25T08:45:11.601808+0000","last_fullsized":"2024-04-25T08:45:11.601808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594375+0000","last_change":"2024-04-25T08:45:11.594375+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594375+0000","last_undegraded":"2024-04-25T08:45:11.594375+0000","last_fullsized":"2024-04-25T08:45:11.594375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593868+0000","last_change":"2024-04-25T08:45:11.593868+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593868+0000","last_undegraded":"2024-04-25T08:45:11.593868+0000","last_fullsized":"2024-04-25T08:45:11.593868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604594+0000","last_change":"2024-04-25T08:45:11.604594+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604594+0000","last_undegraded":"2024-04-25T08:45:11.604594+0000","last_fullsized":"2024-04-25T08:45:11.604594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593331+0000","last_change":"2024-04-25T08:45:11.593331+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593331+0000","last_undegraded":"2024-04-25T08:45:11.593331+0000","last_fullsized":"2024-04-25T08:45:11.593331+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593128+0000","last_change":"2024-04-25T08:45:11.593128+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593128+0000","last_undegraded":"2024-04-25T08:45:11.593128+0000","last_fullsized":"2024-04-25T08:45:11.593128+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592664+0000","last_change":"2024-04-25T08:45:11.592664+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592664+0000","last_undegraded":"2024-04-25T08:45:11.592664+0000","last_fullsized":"2024-04-25T08:45:11.592664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604961+0000","last_change":"2024-04-25T08:45:11.604961+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604961+0000","last_undegraded":"2024-04-25T08:45:11.604961+0000","last_fullsized":"2024-04-25T08:45:11.604961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.608708+0000","last_change":"2024-04-25T08:45:11.608708+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.608708+0000","last_undegraded":"2024-04-25T08:45:11.608708+0000","last_fullsized":"2024-04-25T08:45:11.608708+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599852+0000","last_change":"2024-04-25T08:45:11.599852+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599852+0000","last_undegraded":"2024-04-25T08:45:11.599852+0000","last_fullsized":"2024-04-25T08:45:11.599852+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595481+0000","last_change":"2024-04-25T08:45:11.595481+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595481+0000","last_undegraded":"2024-04-25T08:45:11.595481+0000","last_fullsized":"2024-04-25T08:45:11.595481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.592683+0000","last_change":"2024-04-25T08:45:11.592683+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.592683+0000","last_undegraded":"2024-04-25T08:45:11.592683+0000","last_fullsized":"2024-04-25T08:45:11.592683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.604336+0000","last_change":"2024-04-25T08:45:11.604336+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.604336+0000","last_undegraded":"2024-04-25T08:45:11.604336+0000","last_fullsized":"2024-04-25T08:45:11.604336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.598275+0000","last_change":"2024-04-25T08:45:11.598275+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.598275+0000","last_undegraded":"2024-04-25T08:45:11.598275+0000","last_fullsized":"2024-04-25T08:45:11.598275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602360+0000","last_change":"2024-04-25T08:45:11.602360+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602360+0000","last_undegraded":"2024-04-25T08:45:11.602360+0000","last_fullsized":"2024-04-25T08:45:11.602360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593787+0000","last_change":"2024-04-25T08:45:11.593787+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593787+0000","last_undegraded":"2024-04-25T08:45:11.593787+0000","last_fullsized":"2024-04-25T08:45:11.593787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618446+0000","last_change":"2024-04-25T08:45:11.616357+0000","last_active":"2024-04-25T08:45:11.618446+0000","last_peered":"2024-04-25T08:45:11.618446+0000","last_clean":"2024-04-25T08:45:11.618446+0000","last_became_active":"2024-04-25T08:45:11.612904+0000","last_became_peered":"2024-04-25T08:45:11.612904+0000","last_unstale":"2024-04-25T08:45:11.618446+0000","last_undegraded":"2024-04-25T08:45:11.618446+0000","last_fullsized":"2024-04-25T08:45:11.618446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599374+0000","last_change":"2024-04-25T08:45:11.599374+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599374+0000","last_undegraded":"2024-04-25T08:45:11.599374+0000","last_fullsized":"2024-04-25T08:45:11.599374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614412+0000","last_change":"2024-04-25T08:45:11.609407+0000","last_active":"2024-04-25T08:45:11.614412+0000","last_peered":"2024-04-25T08:45:11.614412+0000","last_clean":"2024-04-25T08:45:11.614412+0000","last_became_active":"2024-04-25T08:45:11.609129+0000","last_became_peered":"2024-04-25T08:45:11.609129+0000","last_unstale":"2024-04-25T08:45:11.614412+0000","last_undegraded":"2024-04-25T08:45:11.614412+0000","last_fullsized":"2024-04-25T08:45:11.614412+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.303293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603765+0000","last_change":"2024-04-25T08:45:11.603765+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603765+0000","last_undegraded":"2024-04-25T08:45:11.603765+0000","last_fullsized":"2024-04-25T08:45:11.603765+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617069+0000","last_change":"2024-04-25T08:45:11.615084+0000","last_active":"2024-04-25T08:45:11.617069+0000","last_peered":"2024-04-25T08:45:11.617069+0000","last_clean":"2024-04-25T08:45:11.617069+0000","last_became_active":"2024-04-25T08:45:11.614747+0000","last_became_peered":"2024-04-25T08:45:11.614747+0000","last_unstale":"2024-04-25T08:45:11.617069+0000","last_undegraded":"2024-04-25T08:45:11.617069+0000","last_fullsized":"2024-04-25T08:45:11.617069+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.879227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594756+0000","last_change":"2024-04-25T08:45:11.594756+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594756+0000","last_undegraded":"2024-04-25T08:45:11.594756+0000","last_fullsized":"2024-04-25T08:45:11.594756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614851+0000","last_change":"2024-04-25T08:45:11.610526+0000","last_active":"2024-04-25T08:45:11.614851+0000","last_peered":"2024-04-25T08:45:11.614851+0000","last_clean":"2024-04-25T08:45:11.614851+0000","last_became_active":"2024-04-25T08:45:11.610262+0000","last_became_peered":"2024-04-25T08:45:11.610262+0000","last_unstale":"2024-04-25T08:45:11.614851+0000","last_undegraded":"2024-04-25T08:45:11.614851+0000","last_fullsized":"2024-04-25T08:45:11.614851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:35.392449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603978+0000","last_change":"2024-04-25T08:45:11.603978+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603978+0000","last_undegraded":"2024-04-25T08:45:11.603978+0000","last_fullsized":"2024-04-25T08:45:11.603978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614866+0000","last_change":"2024-04-25T08:45:11.608750+0000","last_active":"2024-04-25T08:45:11.614866+0000","last_peered":"2024-04-25T08:45:11.614866+0000","last_clean":"2024-04-25T08:45:11.614866+0000","last_became_active":"2024-04-25T08:45:11.608428+0000","last_became_peered":"2024-04-25T08:45:11.608428+0000","last_unstale":"2024-04-25T08:45:11.614866+0000","last_undegraded":"2024-04-25T08:45:11.614866+0000","last_fullsized":"2024-04-25T08:45:11.614866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:35.847974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594081+0000","last_change":"2024-04-25T08:45:11.594081+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594081+0000","last_undegraded":"2024-04-25T08:45:11.594081+0000","last_fullsized":"2024-04-25T08:45:11.594081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614545+0000","last_change":"2024-04-25T08:45:11.611352+0000","last_active":"2024-04-25T08:45:11.614545+0000","last_peered":"2024-04-25T08:45:11.614545+0000","last_clean":"2024-04-25T08:45:11.614545+0000","last_became_active":"2024-04-25T08:45:11.611094+0000","last_became_peered":"2024-04-25T08:45:11.611094+0000","last_unstale":"2024-04-25T08:45:11.614545+0000","last_undegraded":"2024-04-25T08:45:11.614545+0000","last_fullsized":"2024-04-25T08:45:11.614545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:36.596184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.597842+0000","last_change":"2024-04-25T08:45:11.597842+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.597842+0000","last_undegraded":"2024-04-25T08:45:11.597842+0000","last_fullsized":"2024-04-25T08:45:11.597842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.609031+0000","last_change":"2024-04-25T08:45:11.604893+0000","last_active":"2024-04-25T08:45:11.609031+0000","last_peered":"2024-04-25T08:45:11.609031+0000","last_clean":"2024-04-25T08:45:11.609031+0000","last_became_active":"2024-04-25T08:45:11.603555+0000","last_became_peered":"2024-04-25T08:45:11.603555+0000","last_unstale":"2024-04-25T08:45:11.609031+0000","last_undegraded":"2024-04-25T08:45:11.609031+0000","last_fullsized":"2024-04-25T08:45:11.609031+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:20.076400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593075+0000","last_change":"2024-04-25T08:45:11.593075+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593075+0000","last_undegraded":"2024-04-25T08:45:11.593075+0000","last_fullsized":"2024-04-25T08:45:11.593075+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.616463+0000","last_change":"2024-04-25T08:45:11.613393+0000","last_active":"2024-04-25T08:45:11.616463+0000","last_peered":"2024-04-25T08:45:11.616463+0000","last_clean":"2024-04-25T08:45:11.616463+0000","last_became_active":"2024-04-25T08:45:11.611939+0000","last_became_peered":"2024-04-25T08:45:11.611939+0000","last_unstale":"2024-04-25T08:45:11.616463+0000","last_undegraded":"2024-04-25T08:45:11.616463+0000","last_fullsized":"2024-04-25T08:45:11.616463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:33.153694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594910+0000","last_change":"2024-04-25T08:45:11.594910+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594910+0000","last_undegraded":"2024-04-25T08:45:11.594910+0000","last_fullsized":"2024-04-25T08:45:11.594910+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615906+0000","last_change":"2024-04-25T08:45:11.611641+0000","last_active":"2024-04-25T08:45:11.615906+0000","last_peered":"2024-04-25T08:45:11.615906+0000","last_clean":"2024-04-25T08:45:11.615906+0000","last_became_active":"2024-04-25T08:45:11.611380+0000","last_became_peered":"2024-04-25T08:45:11.611380+0000","last_unstale":"2024-04-25T08:45:11.615906+0000","last_undegraded":"2024-04-25T08:45:11.615906+0000","last_fullsized":"2024-04-25T08:45:11.615906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:57.910041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593106+0000","last_change":"2024-04-25T08:45:11.593106+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593106+0000","last_undegraded":"2024-04-25T08:45:11.593106+0000","last_fullsized":"2024-04-25T08:45:11.593106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615489+0000","last_change":"2024-04-25T08:45:11.612291+0000","last_active":"2024-04-25T08:45:11.615489+0000","last_peered":"2024-04-25T08:45:11.615489+0000","last_clean":"2024-04-25T08:45:11.615489+0000","last_became_active":"2024-04-25T08:45:11.612016+0000","last_became_peered":"2024-04-25T08:45:11.612016+0000","last_unstale":"2024-04-25T08:45:11.615489+0000","last_undegraded":"2024-04-25T08:45:11.615489+0000","last_fullsized":"2024-04-25T08:45:11.615489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:28.466919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594590+0000","last_change":"2024-04-25T08:45:11.594590+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594590+0000","last_undegraded":"2024-04-25T08:45:11.594590+0000","last_fullsized":"2024-04-25T08:45:11.594590+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615757+0000","last_change":"2024-04-25T08:45:11.612056+0000","last_active":"2024-04-25T08:45:11.615757+0000","last_peered":"2024-04-25T08:45:11.615757+0000","last_clean":"2024-04-25T08:45:11.615757+0000","last_became_active":"2024-04-25T08:45:11.611677+0000","last_became_peered":"2024-04-25T08:45:11.611677+0000","last_unstale":"2024-04-25T08:45:11.615757+0000","last_undegraded":"2024-04-25T08:45:11.615757+0000","last_fullsized":"2024-04-25T08:45:11.615757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:29.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.602391+0000","last_change":"2024-04-25T08:45:11.602391+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.602391+0000","last_undegraded":"2024-04-25T08:45:11.602391+0000","last_fullsized":"2024-04-25T08:45:11.602391+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617146+0000","last_change":"2024-04-25T08:45:11.615380+0000","last_active":"2024-04-25T08:45:11.617146+0000","last_peered":"2024-04-25T08:45:11.617146+0000","last_clean":"2024-04-25T08:45:11.617146+0000","last_became_active":"2024-04-25T08:45:11.612863+0000","last_became_peered":"2024-04-25T08:45:11.612863+0000","last_unstale":"2024-04-25T08:45:11.617146+0000","last_undegraded":"2024-04-25T08:45:11.617146+0000","last_fullsized":"2024-04-25T08:45:11.617146+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.599723+0000","last_change":"2024-04-25T08:45:11.599723+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.599723+0000","last_undegraded":"2024-04-25T08:45:11.599723+0000","last_fullsized":"2024-04-25T08:45:11.599723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614344+0000","last_change":"2024-04-25T08:45:11.609636+0000","last_active":"2024-04-25T08:45:11.614344+0000","last_peered":"2024-04-25T08:45:11.614344+0000","last_clean":"2024-04-25T08:45:11.614344+0000","last_became_active":"2024-04-25T08:45:11.609296+0000","last_became_peered":"2024-04-25T08:45:11.609296+0000","last_unstale":"2024-04-25T08:45:11.614344+0000","last_undegraded":"2024-04-25T08:45:11.614344+0000","last_fullsized":"2024-04-25T08:45:11.614344+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603834+0000","last_change":"2024-04-25T08:45:11.603834+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603834+0000","last_undegraded":"2024-04-25T08:45:11.603834+0000","last_fullsized":"2024-04-25T08:45:11.603834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.619721+0000","last_change":"2024-04-25T08:45:11.618604+0000","last_active":"2024-04-25T08:45:11.619721+0000","last_peered":"2024-04-25T08:45:11.619721+0000","last_clean":"2024-04-25T08:45:11.619721+0000","last_became_active":"2024-04-25T08:45:11.618224+0000","last_became_peered":"2024-04-25T08:45:11.618224+0000","last_unstale":"2024-04-25T08:45:11.619721+0000","last_undegraded":"2024-04-25T08:45:11.619721+0000","last_fullsized":"2024-04-25T08:45:11.619721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:57.910041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596991+0000","last_change":"2024-04-25T08:45:11.596991+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596991+0000","last_undegraded":"2024-04-25T08:45:11.596991+0000","last_fullsized":"2024-04-25T08:45:11.596991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615681+0000","last_change":"2024-04-25T08:45:11.611078+0000","last_active":"2024-04-25T08:45:11.615681+0000","last_peered":"2024-04-25T08:45:11.615681+0000","last_clean":"2024-04-25T08:45:11.615681+0000","last_became_active":"2024-04-25T08:45:11.610749+0000","last_became_peered":"2024-04-25T08:45:11.610749+0000","last_unstale":"2024-04-25T08:45:11.615681+0000","last_undegraded":"2024-04-25T08:45:11.615681+0000","last_fullsized":"2024-04-25T08:45:11.615681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.517500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594513+0000","last_change":"2024-04-25T08:45:11.594513+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594513+0000","last_undegraded":"2024-04-25T08:45:11.594513+0000","last_fullsized":"2024-04-25T08:45:11.594513+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.615892+0000","last_change":"2024-04-25T08:45:11.613128+0000","last_active":"2024-04-25T08:45:11.615892+0000","last_peered":"2024-04-25T08:45:11.615892+0000","last_clean":"2024-04-25T08:45:11.615892+0000","last_became_active":"2024-04-25T08:45:11.611609+0000","last_became_peered":"2024-04-25T08:45:11.611609+0000","last_unstale":"2024-04-25T08:45:11.615892+0000","last_undegraded":"2024-04-25T08:45:11.615892+0000","last_fullsized":"2024-04-25T08:45:11.615892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:51.214222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.594748+0000","last_change":"2024-04-25T08:45:11.594748+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.594748+0000","last_undegraded":"2024-04-25T08:45:11.594748+0000","last_fullsized":"2024-04-25T08:45:11.594748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.619710+0000","last_change":"2024-04-25T08:45:11.618549+0000","last_active":"2024-04-25T08:45:11.619710+0000","last_peered":"2024-04-25T08:45:11.619710+0000","last_clean":"2024-04-25T08:45:11.619710+0000","last_became_active":"2024-04-25T08:45:11.618187+0000","last_became_peered":"2024-04-25T08:45:11.618187+0000","last_unstale":"2024-04-25T08:45:11.619710+0000","last_undegraded":"2024-04-25T08:45:11.619710+0000","last_fullsized":"2024-04-25T08:45:11.619710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.517500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.598619+0000","last_change":"2024-04-25T08:45:11.598619+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.598619+0000","last_undegraded":"2024-04-25T08:45:11.598619+0000","last_fullsized":"2024-04-25T08:45:11.598619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614216+0000","last_change":"2024-04-25T08:45:11.609242+0000","last_active":"2024-04-25T08:45:11.614216+0000","last_peered":"2024-04-25T08:45:11.614216+0000","last_clean":"2024-04-25T08:45:11.614216+0000","last_became_active":"2024-04-25T08:45:11.608930+0000","last_became_peered":"2024-04-25T08:45:11.608930+0000","last_unstale":"2024-04-25T08:45:11.614216+0000","last_undegraded":"2024-04-25T08:45:11.614216+0000","last_fullsized":"2024-04-25T08:45:11.614216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601349+0000","last_change":"2024-04-25T08:45:11.601349+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601349+0000","last_undegraded":"2024-04-25T08:45:11.601349+0000","last_fullsized":"2024-04-25T08:45:11.601349+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.613852+0000","last_change":"2024-04-25T08:45:11.608806+0000","last_active":"2024-04-25T08:45:11.613852+0000","last_peered":"2024-04-25T08:45:11.613852+0000","last_clean":"2024-04-25T08:45:11.613852+0000","last_became_active":"2024-04-25T08:45:11.607645+0000","last_became_peered":"2024-04-25T08:45:11.607645+0000","last_unstale":"2024-04-25T08:45:11.613852+0000","last_undegraded":"2024-04-25T08:45:11.613852+0000","last_fullsized":"2024-04-25T08:45:11.613852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:35.633342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596595+0000","last_change":"2024-04-25T08:45:11.596595+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596595+0000","last_undegraded":"2024-04-25T08:45:11.596595+0000","last_fullsized":"2024-04-25T08:45:11.596595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.607835+0000","last_change":"2024-04-25T08:45:11.605238+0000","last_active":"2024-04-25T08:45:11.607835+0000","last_peered":"2024-04-25T08:45:11.607835+0000","last_clean":"2024-04-25T08:45:11.607835+0000","last_became_active":"2024-04-25T08:45:11.604975+0000","last_became_peered":"2024-04-25T08:45:11.604975+0000","last_unstale":"2024-04-25T08:45:11.607835+0000","last_undegraded":"2024-04-25T08:45:11.607835+0000","last_fullsized":"2024-04-25T08:45:11.607835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:05.879227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.601918+0000","last_change":"2024-04-25T08:45:11.601918+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.601918+0000","last_undegraded":"2024-04-25T08:45:11.601918+0000","last_fullsized":"2024-04-25T08:45:11.601918+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618499+0000","last_change":"2024-04-25T08:45:11.616493+0000","last_active":"2024-04-25T08:45:11.618499+0000","last_peered":"2024-04-25T08:45:11.618499+0000","last_clean":"2024-04-25T08:45:11.618499+0000","last_became_active":"2024-04-25T08:45:11.616241+0000","last_became_peered":"2024-04-25T08:45:11.616241+0000","last_unstale":"2024-04-25T08:45:11.618499+0000","last_undegraded":"2024-04-25T08:45:11.618499+0000","last_fullsized":"2024-04-25T08:45:11.618499+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:35.847974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.603447+0000","last_change":"2024-04-25T08:45:11.603447+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.603447+0000","last_undegraded":"2024-04-25T08:45:11.603447+0000","last_fullsized":"2024-04-25T08:45:11.603447+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617745+0000","last_change":"2024-04-25T08:45:11.615924+0000","last_active":"2024-04-25T08:45:11.617745+0000","last_peered":"2024-04-25T08:45:11.617745+0000","last_clean":"2024-04-25T08:45:11.617745+0000","last_became_active":"2024-04-25T08:45:11.613543+0000","last_became_peered":"2024-04-25T08:45:11.613543+0000","last_unstale":"2024-04-25T08:45:11.617745+0000","last_undegraded":"2024-04-25T08:45:11.617745+0000","last_fullsized":"2024-04-25T08:45:11.617745+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593904+0000","last_change":"2024-04-25T08:45:11.593904+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593904+0000","last_undegraded":"2024-04-25T08:45:11.593904+0000","last_fullsized":"2024-04-25T08:45:11.593904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617738+0000","last_change":"2024-04-25T08:45:11.613996+0000","last_active":"2024-04-25T08:45:11.617738+0000","last_peered":"2024-04-25T08:45:11.617738+0000","last_clean":"2024-04-25T08:45:11.617738+0000","last_became_active":"2024-04-25T08:45:11.613688+0000","last_became_peered":"2024-04-25T08:45:11.613688+0000","last_unstale":"2024-04-25T08:45:11.617738+0000","last_undegraded":"2024-04-25T08:45:11.617738+0000","last_fullsized":"2024-04-25T08:45:11.617738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:20.076400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.600734+0000","last_change":"2024-04-25T08:45:11.600734+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.600734+0000","last_undegraded":"2024-04-25T08:45:11.600734+0000","last_fullsized":"2024-04-25T08:45:11.600734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.618392+0000","last_change":"2024-04-25T08:45:11.616861+0000","last_active":"2024-04-25T08:45:11.618392+0000","last_peered":"2024-04-25T08:45:11.618392+0000","last_clean":"2024-04-25T08:45:11.618392+0000","last_became_active":"2024-04-25T08:45:11.616539+0000","last_became_peered":"2024-04-25T08:45:11.616539+0000","last_unstale":"2024-04-25T08:45:11.618392+0000","last_undegraded":"2024-04-25T08:45:11.618392+0000","last_fullsized":"2024-04-25T08:45:11.618392+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.606983+0000","last_change":"2024-04-25T08:45:11.606983+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.606983+0000","last_undegraded":"2024-04-25T08:45:11.606983+0000","last_fullsized":"2024-04-25T08:45:11.606983+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617742+0000","last_change":"2024-04-25T08:45:11.616486+0000","last_active":"2024-04-25T08:45:11.617742+0000","last_peered":"2024-04-25T08:45:11.617742+0000","last_clean":"2024-04-25T08:45:11.617742+0000","last_became_active":"2024-04-25T08:45:11.614742+0000","last_became_peered":"2024-04-25T08:45:11.614742+0000","last_unstale":"2024-04-25T08:45:11.617742+0000","last_undegraded":"2024-04-25T08:45:11.617742+0000","last_fullsized":"2024-04-25T08:45:11.617742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:41.457600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.605924+0000","last_change":"2024-04-25T08:45:11.605924+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.605924+0000","last_undegraded":"2024-04-25T08:45:11.605924+0000","last_fullsized":"2024-04-25T08:45:11.605924+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617089+0000","last_change":"2024-04-25T08:45:11.615530+0000","last_active":"2024-04-25T08:45:11.617089+0000","last_peered":"2024-04-25T08:45:11.617089+0000","last_clean":"2024-04-25T08:45:11.617089+0000","last_became_active":"2024-04-25T08:45:11.615180+0000","last_became_peered":"2024-04-25T08:45:11.615180+0000","last_unstale":"2024-04-25T08:45:11.617089+0000","last_undegraded":"2024-04-25T08:45:11.617089+0000","last_fullsized":"2024-04-25T08:45:11.617089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:46.666427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.596149+0000","last_change":"2024-04-25T08:45:11.596149+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.596149+0000","last_undegraded":"2024-04-25T08:45:11.596149+0000","last_fullsized":"2024-04-25T08:45:11.596149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.617053+0000","last_change":"2024-04-25T08:45:11.615371+0000","last_active":"2024-04-25T08:45:11.617053+0000","last_peered":"2024-04-25T08:45:11.617053+0000","last_clean":"2024-04-25T08:45:11.617053+0000","last_became_active":"2024-04-25T08:45:11.611714+0000","last_became_peered":"2024-04-25T08:45:11.611714+0000","last_unstale":"2024-04-25T08:45:11.617053+0000","last_undegraded":"2024-04-25T08:45:11.617053+0000","last_fullsized":"2024-04-25T08:45:11.617053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:59.050987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595213+0000","last_change":"2024-04-25T08:45:11.595213+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595213+0000","last_undegraded":"2024-04-25T08:45:11.595213+0000","last_fullsized":"2024-04-25T08:45:11.595213+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614365+0000","last_change":"2024-04-25T08:45:11.610635+0000","last_active":"2024-04-25T08:45:11.614365+0000","last_peered":"2024-04-25T08:45:11.614365+0000","last_clean":"2024-04-25T08:45:11.614365+0000","last_became_active":"2024-04-25T08:45:11.609768+0000","last_became_peered":"2024-04-25T08:45:11.609768+0000","last_unstale":"2024-04-25T08:45:11.614365+0000","last_undegraded":"2024-04-25T08:45:11.614365+0000","last_fullsized":"2024-04-25T08:45:11.614365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:45.463433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.597735+0000","last_change":"2024-04-25T08:45:11.597735+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.597735+0000","last_undegraded":"2024-04-25T08:45:11.597735+0000","last_fullsized":"2024-04-25T08:45:11.597735+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.603873+0000","last_change":"2024-04-25T08:45:11.599255+0000","last_active":"2024-04-25T08:45:11.603873+0000","last_peered":"2024-04-25T08:45:11.603873+0000","last_clean":"2024-04-25T08:45:11.603873+0000","last_became_active":"2024-04-25T08:45:11.598904+0000","last_became_peered":"2024-04-25T08:45:11.598904+0000","last_unstale":"2024-04-25T08:45:11.603873+0000","last_undegraded":"2024-04-25T08:45:11.603873+0000","last_fullsized":"2024-04-25T08:45:11.603873+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:43.303293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.595394+0000","last_change":"2024-04-25T08:45:11.595394+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.595394+0000","last_undegraded":"2024-04-25T08:45:11.595394+0000","last_fullsized":"2024-04-25T08:45:11.595394+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.609055+0000","last_change":"2024-04-25T08:45:11.603866+0000","last_active":"2024-04-25T08:45:11.609055+0000","last_peered":"2024-04-25T08:45:11.609055+0000","last_clean":"2024-04-25T08:45:11.609055+0000","last_became_active":"2024-04-25T08:45:11.603106+0000","last_became_peered":"2024-04-25T08:45:11.603106+0000","last_unstale":"2024-04-25T08:45:11.609055+0000","last_undegraded":"2024-04-25T08:45:11.609055+0000","last_fullsized":"2024-04-25T08:45:11.609055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:35.392449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.607579+0000","last_change":"2024-04-25T08:45:11.607579+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.607579+0000","last_undegraded":"2024-04-25T08:45:11.607579+0000","last_fullsized":"2024-04-25T08:45:11.607579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.614345+0000","last_change":"2024-04-25T08:45:11.610404+0000","last_active":"2024-04-25T08:45:11.614345+0000","last_peered":"2024-04-25T08:45:11.614345+0000","last_clean":"2024-04-25T08:45:11.614345+0000","last_became_active":"2024-04-25T08:45:11.610147+0000","last_became_peered":"2024-04-25T08:45:11.610147+0000","last_unstale":"2024-04-25T08:45:11.614345+0000","last_undegraded":"2024-04-25T08:45:11.614345+0000","last_fullsized":"2024-04-25T08:45:11.614345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:35.633342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-04-25T08:45:11.593591+0000","last_change":"2024-04-25T08:45:11.593591+0000","last_active":"2024-04-25T08:45:11.583601+0000","last_peered":"2024-04-25T08:45:11.583601+0000","last_clean":"2024-04-25T08:45:11.583601+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:45:11.593591+0000","last_undegraded":"2024-04-25T08:45:11.593591+0000","last_fullsized":"2024-04-25T08:45:11.593591+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.612199+0000","last_change":"2024-04-25T08:45:11.606470+0000","last_active":"2024-04-25T08:45:11.612199+0000","last_peered":"2024-04-25T08:45:11.612199+0000","last_clean":"2024-04-25T08:45:11.612199+0000","last_became_active":"2024-04-25T08:45:11.606206+0000","last_became_peered":"2024-04-25T08:45:11.606206+0000","last_unstale":"2024-04-25T08:45:11.612199+0000","last_undegraded":"2024-04-25T08:45:11.612199+0000","last_fullsized":"2024-04-25T08:45:11.612199+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:46.666427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T08:45:11.595703+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:45:11.595703+0000","last_peered":"2024-04-25T08:45:11.595703+0000","last_clean":"2024-04-25T08:45:11.595703+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:45:11.595703+0000","last_undegraded":"2024-04-25T08:45:11.595703+0000","last_fullsized":"2024-04-25T08:45:11.595703+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":932113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":335086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":749568,"data_stored":335086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":761856,"data_stored":341745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":932113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:45:20.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:45:20.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:45:21.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:45:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7488c00640 1 Processor -- start 2024-04-25T08:45:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7488c00640 1 -- start start 2024-04-25T08:45:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7488c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7484159230 0x7f748415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7488c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f74840cdc50 0x7f748415d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7488c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74840cc700 con 0x7f74840cbeb0 2024-04-25T08:45:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7488c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840cc9e0 con 0x7f74840cdc50 2024-04-25T08:45:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7488c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74840cccc0 con 0x7f7484159230 2024-04-25T08:45:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f747be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f74840cbeb0 0x7f74840cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49086/0 (socket says 172.21.15.46:49086) 2024-04-25T08:45:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f747be00640 1 -- 172.21.15.46:0/4134766099 learned_addr learned my addr 172.21.15.46:0/4134766099 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7481e00640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7484159230 0x7f748415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7482800640 1 --2- 172.21.15.46:0/4134766099 >> v2:172.21.15.82:3300/0 conn(0x7f74840cdc50 0x7f748415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200327754 0 0) 0x7f74840cc700 con 0x7f74840cbeb0 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.446+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7468003530 con 0x7f74840cbeb0 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4033174042 0 0) 0x7f7468003530 con 0x7f74840cbeb0 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7484159230 msgr2=0x7f748415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7484159230 0x7f748415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 >> v2:172.21.15.82:3300/0 conn(0x7f74840cdc50 msgr2=0x7f748415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 --2- 172.21.15.46:0/4134766099 >> v2:172.21.15.82:3300/0 conn(0x7f74840cdc50 0x7f748415d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840ccfa0 con 0x7f74840cbeb0 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f746c001680 con 0x7f74840cbeb0 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 >> v1:172.21.15.46:6789/0 conn(0x7f74840cbeb0 legacy=0x7f74840cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680039c0 0x7f7468003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468004810 0x7f7468004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7484159230 0x7f74680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74840cc9e0 con 0x7f7468004810 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74840cccc0 con 0x7f74680039c0 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746800b9d0 con 0x7f7484159230 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f747be00640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468004810 0x7f7468004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7481e00640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680039c0 0x7f7468003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f7482800640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7484159230 0x7f74680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f747be00640 1 -- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7484159230 msgr2=0x7f74680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f747be00640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7484159230 0x7f74680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f747be00640 1 -- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680039c0 msgr2=0x7f7468003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f747be00640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680039c0 0x7f7468003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f747be00640 1 -- 172.21.15.46:0/4134766099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7468003530 con 0x7f7468004810 2024-04-25T08:45:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.450+0000 7f747be00640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468004810 0x7f7468004c90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c270 tx=0x7f746c01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cda625b779e979 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c007d40 con 0x7f7468004810 2024-04-25T08:45:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c0a9020 con 0x7f7468004810 2024-04-25T08:45:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7481400640 1 -- 172.21.15.46:0/4134766099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c005ae0 con 0x7f7468004810 2024-04-25T08:45:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 -- 172.21.15.46:0/4134766099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468004810 msgr2=0x7f7468004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 --2- 172.21.15.46:0/4134766099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7468004810 0x7f7468004c90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c270 tx=0x7f746c01cc70 comp rx=0 tx=0).stop 2024-04-25T08:45:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7482800640 1 -- 172.21.15.46:0/4134766099 reap_dead start 2024-04-25T08:45:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 -- 172.21.15.46:0/4134766099 shutdown_connections 2024-04-25T08:45:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 -- 172.21.15.46:0/4134766099 >> 172.21.15.46:0/4134766099 conn(0x7f74840b12a0 msgr2=0x7f74840b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 -- 172.21.15.46:0/4134766099 shutdown_connections 2024-04-25T08:45:21.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 -- 172.21.15.46:0/4134766099 wait complete. 2024-04-25T08:45:21.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 Processor -- start 2024-04-25T08:45:21.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 -- start start 2024-04-25T08:45:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7468004810 0x7f74841583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74840b3500 0x7f748406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74840b3980 0x7f748406f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748415df50 con 0x7f74840b3980 2024-04-25T08:45:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74840cc4e0 con 0x7f74840b3500 2024-04-25T08:45:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7488c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748415e0d0 con 0x7f7468004810 2024-04-25T08:45:21.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7482800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74840b3980 0x7f748406f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f747be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74840b3500 0x7f748406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7481e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7468004810 0x7f74841583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7482800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74840b3980 0x7f748406f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49622/0 (socket says 172.21.15.46:49622) 2024-04-25T08:45:21.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.454+0000 7f7482800640 1 -- 172.21.15.46:0/3035627839 learned_addr learned my addr 172.21.15.46:0/3035627839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f7482800640 1 -- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7468004810 msgr2=0x7f74841583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f7482800640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7468004810 0x7f74841583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f7482800640 1 -- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74840b3500 msgr2=0x7f748406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f7482800640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74840b3500 0x7f748406ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f7482800640 1 -- 172.21.15.46:0/3035627839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748406d5c0 con 0x7f74840b3980 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f7482800640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74840b3980 0x7f748406f180 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f748406cee0 tx=0x7f747c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f747a000640 1 -- 172.21.15.46:0/3035627839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c011e60 con 0x7f74840b3980 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f747a000640 1 -- 172.21.15.46:0/3035627839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c01d070 con 0x7f74840b3980 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f747a000640 1 -- 172.21.15.46:0/3035627839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c015650 con 0x7f74840b3980 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748406d8a0 con 0x7f74840b3980 2024-04-25T08:45:21.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748406de60 con 0x7f74840b3980 2024-04-25T08:45:21.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7484069080 con 0x7f74840b3980 2024-04-25T08:45:21.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f747a000640 1 -- 172.21.15.46:0/3035627839 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f747c0157f0 con 0x7f74840b3980 2024-04-25T08:45:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.458+0000 7f747a000640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7460066f90 0x7f7460069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.462+0000 7f747a000640 1 -- 172.21.15.46:0/3035627839 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f747c085a30 con 0x7f74840b3980 2024-04-25T08:45:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.462+0000 7f7481e00640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7460066f90 0x7f7460069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.462+0000 7f7481e00640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7460066f90 0x7f7460069450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7474009a80 tx=0x7f7474009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:21.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.466+0000 7f747a000640 1 -- 172.21.15.46:0/3035627839 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f747c010040 con 0x7f74840b3980 2024-04-25T08:45:21.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.622+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7484068520 con 0x7f7460066f90 2024-04-25T08:45:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.630+0000 7f747a000640 1 -- 172.21.15.46:0/3035627839 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252020 (secure 0 0 0) 0x7f7474014af0 con 0x7f7460066f90 2024-04-25T08:45:21.626 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:45:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7460066f90 msgr2=0x7f7460069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7460066f90 0x7f7460069450 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7474009a80 tx=0x7f7474009190 comp rx=0 tx=0).stop 2024-04-25T08:45:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74840b3980 msgr2=0x7f748406f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74840b3980 0x7f748406f180 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f748406cee0 tx=0x7f747c00f580 comp rx=0 tx=0).stop 2024-04-25T08:45:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 shutdown_connections 2024-04-25T08:45:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7460066f90 0x7f7460069450 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f74840b3980 0x7f748406f180 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74840b3500 0x7f748406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 --2- 172.21.15.46:0/3035627839 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7468004810 0x7f74841583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 >> 172.21.15.46:0/3035627839 conn(0x7f74840b12a0 msgr2=0x7f74840ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 shutdown_connections 2024-04-25T08:45:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.634+0000 7f7488c00640 1 -- 172.21.15.46:0/3035627839 wait complete. 2024-04-25T08:45:21.648 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-25T08:45:20.298158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244356,"kb_used_data":7368,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730364,"statfs":{"total":773094113280,"available":772843892736,"internally_reserved":0,"allocated":7544832,"data_stored":4030388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002019"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617906+0000","last_change":"2024-04-25T08:45:12.628721+0000","last_active":"2024-04-25T08:45:13.617906+0000","last_peered":"2024-04-25T08:45:13.617906+0000","last_clean":"2024-04-25T08:45:13.617906+0000","last_became_active":"2024-04-25T08:45:12.623121+0000","last_became_peered":"2024-04-25T08:45:12.623121+0000","last_unstale":"2024-04-25T08:45:13.617906+0000","last_undegraded":"2024-04-25T08:45:13.617906+0000","last_fullsized":"2024-04-25T08:45:13.617906+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:44.309134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617772+0000","last_change":"2024-04-25T08:45:12.624047+0000","last_active":"2024-04-25T08:45:13.617772+0000","last_peered":"2024-04-25T08:45:13.617772+0000","last_clean":"2024-04-25T08:45:13.617772+0000","last_became_active":"2024-04-25T08:45:12.623778+0000","last_became_peered":"2024-04-25T08:45:12.623778+0000","last_unstale":"2024-04-25T08:45:13.617772+0000","last_undegraded":"2024-04-25T08:45:13.617772+0000","last_fullsized":"2024-04-25T08:45:13.617772+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:40.445779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055126+0000","last_change":"2024-04-25T08:45:12.626882+0000","last_active":"2024-04-25T08:45:14.055126+0000","last_peered":"2024-04-25T08:45:14.055126+0000","last_clean":"2024-04-25T08:45:14.055126+0000","last_became_active":"2024-04-25T08:45:12.626530+0000","last_became_peered":"2024-04-25T08:45:12.626530+0000","last_unstale":"2024-04-25T08:45:14.055126+0000","last_undegraded":"2024-04-25T08:45:14.055126+0000","last_fullsized":"2024-04-25T08:45:14.055126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:47.672268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616723+0000","last_change":"2024-04-25T08:45:12.610941+0000","last_active":"2024-04-25T08:45:13.616723+0000","last_peered":"2024-04-25T08:45:13.616723+0000","last_clean":"2024-04-25T08:45:13.616723+0000","last_became_active":"2024-04-25T08:45:12.610508+0000","last_became_peered":"2024-04-25T08:45:12.610508+0000","last_unstale":"2024-04-25T08:45:13.616723+0000","last_undegraded":"2024-04-25T08:45:13.616723+0000","last_fullsized":"2024-04-25T08:45:13.616723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:27.918263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054466+0000","last_change":"2024-04-25T08:45:12.631237+0000","last_active":"2024-04-25T08:45:14.054466+0000","last_peered":"2024-04-25T08:45:14.054466+0000","last_clean":"2024-04-25T08:45:14.054466+0000","last_became_active":"2024-04-25T08:45:12.630931+0000","last_became_peered":"2024-04-25T08:45:12.630931+0000","last_unstale":"2024-04-25T08:45:14.054466+0000","last_undegraded":"2024-04-25T08:45:14.054466+0000","last_fullsized":"2024-04-25T08:45:14.054466+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:28.626411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054541+0000","last_change":"2024-04-25T08:45:12.631087+0000","last_active":"2024-04-25T08:45:14.054541+0000","last_peered":"2024-04-25T08:45:14.054541+0000","last_clean":"2024-04-25T08:45:14.054541+0000","last_became_active":"2024-04-25T08:45:12.630731+0000","last_became_peered":"2024-04-25T08:45:12.630731+0000","last_unstale":"2024-04-25T08:45:14.054541+0000","last_undegraded":"2024-04-25T08:45:14.054541+0000","last_fullsized":"2024-04-25T08:45:14.054541+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:07.010545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616425+0000","last_change":"2024-04-25T08:45:12.629553+0000","last_active":"2024-04-25T08:45:13.616425+0000","last_peered":"2024-04-25T08:45:13.616425+0000","last_clean":"2024-04-25T08:45:13.616425+0000","last_became_active":"2024-04-25T08:45:12.629200+0000","last_became_peered":"2024-04-25T08:45:12.629200+0000","last_unstale":"2024-04-25T08:45:13.616425+0000","last_undegraded":"2024-04-25T08:45:13.616425+0000","last_fullsized":"2024-04-25T08:45:13.616425+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:04.968439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.625606+0000","last_change":"2024-04-25T08:45:12.624072+0000","last_active":"2024-04-25T08:45:13.625606+0000","last_peered":"2024-04-25T08:45:13.625606+0000","last_clean":"2024-04-25T08:45:13.625606+0000","last_became_active":"2024-04-25T08:45:12.623758+0000","last_became_peered":"2024-04-25T08:45:12.623758+0000","last_unstale":"2024-04-25T08:45:13.625606+0000","last_undegraded":"2024-04-25T08:45:13.625606+0000","last_fullsized":"2024-04-25T08:45:13.625606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:36.398290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055458+0000","last_change":"2024-04-25T08:45:12.628675+0000","last_active":"2024-04-25T08:45:14.055458+0000","last_peered":"2024-04-25T08:45:14.055458+0000","last_clean":"2024-04-25T08:45:14.055458+0000","last_became_active":"2024-04-25T08:45:12.628399+0000","last_became_peered":"2024-04-25T08:45:12.628399+0000","last_unstale":"2024-04-25T08:45:14.055458+0000","last_undegraded":"2024-04-25T08:45:14.055458+0000","last_fullsized":"2024-04-25T08:45:14.055458+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:01.635558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055142+0000","last_change":"2024-04-25T08:45:12.639848+0000","last_active":"2024-04-25T08:45:14.055142+0000","last_peered":"2024-04-25T08:45:14.055142+0000","last_clean":"2024-04-25T08:45:14.055142+0000","last_became_active":"2024-04-25T08:45:12.639599+0000","last_became_peered":"2024-04-25T08:45:12.639599+0000","last_unstale":"2024-04-25T08:45:14.055142+0000","last_undegraded":"2024-04-25T08:45:14.055142+0000","last_fullsized":"2024-04-25T08:45:14.055142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:06.885068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616680+0000","last_change":"2024-04-25T08:45:12.636222+0000","last_active":"2024-04-25T08:45:13.616680+0000","last_peered":"2024-04-25T08:45:13.616680+0000","last_clean":"2024-04-25T08:45:13.616680+0000","last_became_active":"2024-04-25T08:45:12.635918+0000","last_became_peered":"2024-04-25T08:45:12.635918+0000","last_unstale":"2024-04-25T08:45:13.616680+0000","last_undegraded":"2024-04-25T08:45:13.616680+0000","last_fullsized":"2024-04-25T08:45:13.616680+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:18.482282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.625749+0000","last_change":"2024-04-25T08:45:12.633991+0000","last_active":"2024-04-25T08:45:13.625749+0000","last_peered":"2024-04-25T08:45:13.625749+0000","last_clean":"2024-04-25T08:45:13.625749+0000","last_became_active":"2024-04-25T08:45:12.633667+0000","last_became_peered":"2024-04-25T08:45:12.633667+0000","last_unstale":"2024-04-25T08:45:13.625749+0000","last_undegraded":"2024-04-25T08:45:13.625749+0000","last_fullsized":"2024-04-25T08:45:13.625749+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:47.672268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.625812+0000","last_change":"2024-04-25T08:45:12.628084+0000","last_active":"2024-04-25T08:45:13.625812+0000","last_peered":"2024-04-25T08:45:13.625812+0000","last_clean":"2024-04-25T08:45:13.625812+0000","last_became_active":"2024-04-25T08:45:12.627816+0000","last_became_peered":"2024-04-25T08:45:12.627816+0000","last_unstale":"2024-04-25T08:45:13.625812+0000","last_undegraded":"2024-04-25T08:45:13.625812+0000","last_fullsized":"2024-04-25T08:45:13.625812+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:36.853815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055690+0000","last_change":"2024-04-25T08:45:12.626905+0000","last_active":"2024-04-25T08:45:14.055690+0000","last_peered":"2024-04-25T08:45:14.055690+0000","last_clean":"2024-04-25T08:45:14.055690+0000","last_became_active":"2024-04-25T08:45:12.626597+0000","last_became_peered":"2024-04-25T08:45:12.626597+0000","last_unstale":"2024-04-25T08:45:14.055690+0000","last_undegraded":"2024-04-25T08:45:14.055690+0000","last_fullsized":"2024-04-25T08:45:14.055690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:02.657607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055281+0000","last_change":"2024-04-25T08:45:12.640017+0000","last_active":"2024-04-25T08:45:14.055281+0000","last_peered":"2024-04-25T08:45:14.055281+0000","last_clean":"2024-04-25T08:45:14.055281+0000","last_became_active":"2024-04-25T08:45:12.639721+0000","last_became_peered":"2024-04-25T08:45:12.639721+0000","last_unstale":"2024-04-25T08:45:14.055281+0000","last_undegraded":"2024-04-25T08:45:14.055281+0000","last_fullsized":"2024-04-25T08:45:14.055281+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:37.602025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616469+0000","last_change":"2024-04-25T08:45:12.630828+0000","last_active":"2024-04-25T08:45:13.616469+0000","last_peered":"2024-04-25T08:45:13.616469+0000","last_clean":"2024-04-25T08:45:13.616469+0000","last_became_active":"2024-04-25T08:45:12.630571+0000","last_became_peered":"2024-04-25T08:45:12.630571+0000","last_unstale":"2024-04-25T08:45:13.616469+0000","last_undegraded":"2024-04-25T08:45:13.616469+0000","last_fullsized":"2024-04-25T08:45:13.616469+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:46.718713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617952+0000","last_change":"2024-04-25T08:45:12.629842+0000","last_active":"2024-04-25T08:45:13.617952+0000","last_peered":"2024-04-25T08:45:13.617952+0000","last_clean":"2024-04-25T08:45:13.617952+0000","last_became_active":"2024-04-25T08:45:12.628809+0000","last_became_peered":"2024-04-25T08:45:12.628809+0000","last_unstale":"2024-04-25T08:45:13.617952+0000","last_undegraded":"2024-04-25T08:45:13.617952+0000","last_fullsized":"2024-04-25T08:45:13.617952+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:36.398290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617417+0000","last_change":"2024-04-25T08:45:12.612624+0000","last_active":"2024-04-25T08:45:13.617417+0000","last_peered":"2024-04-25T08:45:13.617417+0000","last_clean":"2024-04-25T08:45:13.617417+0000","last_became_active":"2024-04-25T08:45:12.612300+0000","last_became_peered":"2024-04-25T08:45:12.612300+0000","last_unstale":"2024-04-25T08:45:13.617417+0000","last_undegraded":"2024-04-25T08:45:13.617417+0000","last_fullsized":"2024-04-25T08:45:13.617417+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:55.017974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055097+0000","last_change":"2024-04-25T08:45:12.617846+0000","last_active":"2024-04-25T08:45:14.055097+0000","last_peered":"2024-04-25T08:45:14.055097+0000","last_clean":"2024-04-25T08:45:14.055097+0000","last_became_active":"2024-04-25T08:45:12.617566+0000","last_became_peered":"2024-04-25T08:45:12.617566+0000","last_unstale":"2024-04-25T08:45:14.055097+0000","last_undegraded":"2024-04-25T08:45:14.055097+0000","last_fullsized":"2024-04-25T08:45:14.055097+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.935904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054991+0000","last_change":"2024-04-25T08:45:12.633609+0000","last_active":"2024-04-25T08:45:14.054991+0000","last_peered":"2024-04-25T08:45:14.054991+0000","last_clean":"2024-04-25T08:45:14.054991+0000","last_became_active":"2024-04-25T08:45:12.629969+0000","last_became_peered":"2024-04-25T08:45:12.629969+0000","last_unstale":"2024-04-25T08:45:14.054991+0000","last_undegraded":"2024-04-25T08:45:14.054991+0000","last_fullsized":"2024-04-25T08:45:14.054991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:40.452865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.625855+0000","last_change":"2024-04-25T08:45:12.642421+0000","last_active":"2024-04-25T08:45:13.625855+0000","last_peered":"2024-04-25T08:45:13.625855+0000","last_clean":"2024-04-25T08:45:13.625855+0000","last_became_active":"2024-04-25T08:45:12.641818+0000","last_became_peered":"2024-04-25T08:45:12.641818+0000","last_unstale":"2024-04-25T08:45:13.625855+0000","last_undegraded":"2024-04-25T08:45:13.625855+0000","last_fullsized":"2024-04-25T08:45:13.625855+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:37.602025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055077+0000","last_change":"2024-04-25T08:45:12.630268+0000","last_active":"2024-04-25T08:45:14.055077+0000","last_peered":"2024-04-25T08:45:14.055077+0000","last_clean":"2024-04-25T08:45:14.055077+0000","last_became_active":"2024-04-25T08:45:12.629848+0000","last_became_peered":"2024-04-25T08:45:12.629848+0000","last_unstale":"2024-04-25T08:45:14.055077+0000","last_undegraded":"2024-04-25T08:45:14.055077+0000","last_fullsized":"2024-04-25T08:45:14.055077+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:25.317071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054729+0000","last_change":"2024-04-25T08:45:12.630100+0000","last_active":"2024-04-25T08:45:14.054729+0000","last_peered":"2024-04-25T08:45:14.054729+0000","last_clean":"2024-04-25T08:45:14.054729+0000","last_became_active":"2024-04-25T08:45:12.629815+0000","last_became_peered":"2024-04-25T08:45:12.629815+0000","last_unstale":"2024-04-25T08:45:14.054729+0000","last_undegraded":"2024-04-25T08:45:14.054729+0000","last_fullsized":"2024-04-25T08:45:14.054729+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:07.010545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054757+0000","last_change":"2024-04-25T08:45:12.627286+0000","last_active":"2024-04-25T08:45:14.054757+0000","last_peered":"2024-04-25T08:45:14.054757+0000","last_clean":"2024-04-25T08:45:14.054757+0000","last_became_active":"2024-04-25T08:45:12.627036+0000","last_became_peered":"2024-04-25T08:45:12.627036+0000","last_unstale":"2024-04-25T08:45:14.054757+0000","last_undegraded":"2024-04-25T08:45:14.054757+0000","last_fullsized":"2024-04-25T08:45:14.054757+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:36.398290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.626138+0000","last_change":"2024-04-25T08:45:12.619788+0000","last_active":"2024-04-25T08:45:13.626138+0000","last_peered":"2024-04-25T08:45:13.626138+0000","last_clean":"2024-04-25T08:45:13.626138+0000","last_became_active":"2024-04-25T08:45:12.619434+0000","last_became_peered":"2024-04-25T08:45:12.619434+0000","last_unstale":"2024-04-25T08:45:13.626138+0000","last_undegraded":"2024-04-25T08:45:13.626138+0000","last_fullsized":"2024-04-25T08:45:13.626138+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:02.657607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.626027+0000","last_change":"2024-04-25T08:45:12.618580+0000","last_active":"2024-04-25T08:45:13.626027+0000","last_peered":"2024-04-25T08:45:13.626027+0000","last_clean":"2024-04-25T08:45:13.626027+0000","last_became_active":"2024-04-25T08:45:12.616232+0000","last_became_peered":"2024-04-25T08:45:12.616232+0000","last_unstale":"2024-04-25T08:45:13.626027+0000","last_undegraded":"2024-04-25T08:45:13.626027+0000","last_fullsized":"2024-04-25T08:45:13.626027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:11.202296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617117+0000","last_change":"2024-04-25T08:45:12.617745+0000","last_active":"2024-04-25T08:45:13.617117+0000","last_peered":"2024-04-25T08:45:13.617117+0000","last_clean":"2024-04-25T08:45:13.617117+0000","last_became_active":"2024-04-25T08:45:12.617529+0000","last_became_peered":"2024-04-25T08:45:12.617529+0000","last_unstale":"2024-04-25T08:45:13.617117+0000","last_undegraded":"2024-04-25T08:45:13.617117+0000","last_fullsized":"2024-04-25T08:45:13.617117+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:06.287405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617263+0000","last_change":"2024-04-25T08:45:12.625995+0000","last_active":"2024-04-25T08:45:13.617263+0000","last_peered":"2024-04-25T08:45:13.617263+0000","last_clean":"2024-04-25T08:45:13.617263+0000","last_became_active":"2024-04-25T08:45:12.625488+0000","last_became_peered":"2024-04-25T08:45:12.625488+0000","last_unstale":"2024-04-25T08:45:13.617263+0000","last_undegraded":"2024-04-25T08:45:13.617263+0000","last_fullsized":"2024-04-25T08:45:13.617263+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:32.406528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055579+0000","last_change":"2024-04-25T08:45:12.626190+0000","last_active":"2024-04-25T08:45:14.055579+0000","last_peered":"2024-04-25T08:45:14.055579+0000","last_clean":"2024-04-25T08:45:14.055579+0000","last_became_active":"2024-04-25T08:45:12.625903+0000","last_became_peered":"2024-04-25T08:45:12.625903+0000","last_unstale":"2024-04-25T08:45:14.055579+0000","last_undegraded":"2024-04-25T08:45:14.055579+0000","last_fullsized":"2024-04-25T08:45:14.055579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:11.202296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.625616+0000","last_change":"2024-04-25T08:45:12.634361+0000","last_active":"2024-04-25T08:45:13.625616+0000","last_peered":"2024-04-25T08:45:13.625616+0000","last_clean":"2024-04-25T08:45:13.625616+0000","last_became_active":"2024-04-25T08:45:12.634120+0000","last_became_peered":"2024-04-25T08:45:12.634120+0000","last_unstale":"2024-04-25T08:45:13.625616+0000","last_undegraded":"2024-04-25T08:45:13.625616+0000","last_fullsized":"2024-04-25T08:45:13.625616+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:06.885068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616647+0000","last_change":"2024-04-25T08:45:12.613708+0000","last_active":"2024-04-25T08:45:13.616647+0000","last_peered":"2024-04-25T08:45:13.616647+0000","last_clean":"2024-04-25T08:45:13.616647+0000","last_became_active":"2024-04-25T08:45:12.613467+0000","last_became_peered":"2024-04-25T08:45:12.613467+0000","last_unstale":"2024-04-25T08:45:13.616647+0000","last_undegraded":"2024-04-25T08:45:13.616647+0000","last_fullsized":"2024-04-25T08:45:13.616647+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:23.057014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054431+0000","last_change":"2024-04-25T08:45:12.616392+0000","last_active":"2024-04-25T08:45:14.054431+0000","last_peered":"2024-04-25T08:45:14.054431+0000","last_clean":"2024-04-25T08:45:14.054431+0000","last_became_active":"2024-04-25T08:45:12.615845+0000","last_became_peered":"2024-04-25T08:45:12.615845+0000","last_unstale":"2024-04-25T08:45:14.054431+0000","last_undegraded":"2024-04-25T08:45:14.054431+0000","last_fullsized":"2024-04-25T08:45:14.054431+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:45.749081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054638+0000","last_change":"2024-04-25T08:45:12.638352+0000","last_active":"2024-04-25T08:45:14.054638+0000","last_peered":"2024-04-25T08:45:14.054638+0000","last_clean":"2024-04-25T08:45:14.054638+0000","last_became_active":"2024-04-25T08:45:12.638028+0000","last_became_peered":"2024-04-25T08:45:12.638028+0000","last_unstale":"2024-04-25T08:45:14.054638+0000","last_undegraded":"2024-04-25T08:45:14.054638+0000","last_fullsized":"2024-04-25T08:45:14.054638+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:55.017974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.618005+0000","last_change":"2024-04-25T08:45:11.616357+0000","last_active":"2024-04-25T08:45:13.618005+0000","last_peered":"2024-04-25T08:45:13.618005+0000","last_clean":"2024-04-25T08:45:13.618005+0000","last_became_active":"2024-04-25T08:45:11.612904+0000","last_became_peered":"2024-04-25T08:45:11.612904+0000","last_unstale":"2024-04-25T08:45:13.618005+0000","last_undegraded":"2024-04-25T08:45:13.618005+0000","last_fullsized":"2024-04-25T08:45:13.618005+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:17.476441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617159+0000","last_change":"2024-04-25T08:45:12.626830+0000","last_active":"2024-04-25T08:45:13.617159+0000","last_peered":"2024-04-25T08:45:13.617159+0000","last_clean":"2024-04-25T08:45:13.617159+0000","last_became_active":"2024-04-25T08:45:12.626586+0000","last_became_peered":"2024-04-25T08:45:12.626586+0000","last_unstale":"2024-04-25T08:45:13.617159+0000","last_undegraded":"2024-04-25T08:45:13.617159+0000","last_fullsized":"2024-04-25T08:45:13.617159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:28.626411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055026+0000","last_change":"2024-04-25T08:45:11.609407+0000","last_active":"2024-04-25T08:45:14.055026+0000","last_peered":"2024-04-25T08:45:14.055026+0000","last_clean":"2024-04-25T08:45:14.055026+0000","last_became_active":"2024-04-25T08:45:11.609129+0000","last_became_peered":"2024-04-25T08:45:11.609129+0000","last_unstale":"2024-04-25T08:45:14.055026+0000","last_undegraded":"2024-04-25T08:45:14.055026+0000","last_fullsized":"2024-04-25T08:45:14.055026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:41.076296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054760+0000","last_change":"2024-04-25T08:45:12.616329+0000","last_active":"2024-04-25T08:45:14.054760+0000","last_peered":"2024-04-25T08:45:14.054760+0000","last_clean":"2024-04-25T08:45:14.054760+0000","last_became_active":"2024-04-25T08:45:12.615800+0000","last_became_peered":"2024-04-25T08:45:12.615800+0000","last_unstale":"2024-04-25T08:45:14.054760+0000","last_undegraded":"2024-04-25T08:45:14.054760+0000","last_fullsized":"2024-04-25T08:45:14.054760+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:51:40.452865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054480+0000","last_change":"2024-04-25T08:45:11.615084+0000","last_active":"2024-04-25T08:45:14.054480+0000","last_peered":"2024-04-25T08:45:14.054480+0000","last_clean":"2024-04-25T08:45:14.054480+0000","last_became_active":"2024-04-25T08:45:11.614747+0000","last_became_peered":"2024-04-25T08:45:11.614747+0000","last_unstale":"2024-04-25T08:45:14.054480+0000","last_undegraded":"2024-04-25T08:45:14.054480+0000","last_fullsized":"2024-04-25T08:45:14.054480+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:10.196456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055240+0000","last_change":"2024-04-25T08:45:12.643820+0000","last_active":"2024-04-25T08:45:14.055240+0000","last_peered":"2024-04-25T08:45:14.055240+0000","last_clean":"2024-04-25T08:45:14.055240+0000","last_became_active":"2024-04-25T08:45:12.643553+0000","last_became_peered":"2024-04-25T08:45:12.643553+0000","last_unstale":"2024-04-25T08:45:14.055240+0000","last_undegraded":"2024-04-25T08:45:14.055240+0000","last_fullsized":"2024-04-25T08:45:14.055240+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:36.853815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054497+0000","last_change":"2024-04-25T08:45:11.610526+0000","last_active":"2024-04-25T08:45:14.054497+0000","last_peered":"2024-04-25T08:45:14.054497+0000","last_clean":"2024-04-25T08:45:14.054497+0000","last_became_active":"2024-04-25T08:45:11.610262+0000","last_became_peered":"2024-04-25T08:45:11.610262+0000","last_unstale":"2024-04-25T08:45:14.054497+0000","last_undegraded":"2024-04-25T08:45:14.054497+0000","last_fullsized":"2024-04-25T08:45:14.054497+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:49.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616590+0000","last_change":"2024-04-25T08:45:12.638814+0000","last_active":"2024-04-25T08:45:13.616590+0000","last_peered":"2024-04-25T08:45:13.616590+0000","last_clean":"2024-04-25T08:45:13.616590+0000","last_became_active":"2024-04-25T08:45:12.638535+0000","last_became_peered":"2024-04-25T08:45:12.638535+0000","last_unstale":"2024-04-25T08:45:13.616590+0000","last_undegraded":"2024-04-25T08:45:13.616590+0000","last_fullsized":"2024-04-25T08:45:13.616590+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:14.272893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054950+0000","last_change":"2024-04-25T08:45:11.608750+0000","last_active":"2024-04-25T08:45:14.054950+0000","last_peered":"2024-04-25T08:45:14.054950+0000","last_clean":"2024-04-25T08:45:14.054950+0000","last_became_active":"2024-04-25T08:45:11.608428+0000","last_became_peered":"2024-04-25T08:45:11.608428+0000","last_unstale":"2024-04-25T08:45:14.054950+0000","last_undegraded":"2024-04-25T08:45:14.054950+0000","last_fullsized":"2024-04-25T08:45:14.054950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:01.651766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616759+0000","last_change":"2024-04-25T08:45:12.623318+0000","last_active":"2024-04-25T08:45:13.616759+0000","last_peered":"2024-04-25T08:45:13.616759+0000","last_clean":"2024-04-25T08:45:13.616759+0000","last_became_active":"2024-04-25T08:45:12.623040+0000","last_became_peered":"2024-04-25T08:45:12.623040+0000","last_unstale":"2024-04-25T08:45:13.616759+0000","last_undegraded":"2024-04-25T08:45:13.616759+0000","last_fullsized":"2024-04-25T08:45:13.616759+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:58.473699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055191+0000","last_change":"2024-04-25T08:45:11.611352+0000","last_active":"2024-04-25T08:45:14.055191+0000","last_peered":"2024-04-25T08:45:14.055191+0000","last_clean":"2024-04-25T08:45:14.055191+0000","last_became_active":"2024-04-25T08:45:11.611094+0000","last_became_peered":"2024-04-25T08:45:11.611094+0000","last_unstale":"2024-04-25T08:45:14.055191+0000","last_undegraded":"2024-04-25T08:45:14.055191+0000","last_fullsized":"2024-04-25T08:45:14.055191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:10.196456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054197+0000","last_change":"2024-04-25T08:45:12.617410+0000","last_active":"2024-04-25T08:45:14.054197+0000","last_peered":"2024-04-25T08:45:14.054197+0000","last_clean":"2024-04-25T08:45:14.054197+0000","last_became_active":"2024-04-25T08:45:12.617114+0000","last_became_peered":"2024-04-25T08:45:12.617114+0000","last_unstale":"2024-04-25T08:45:14.054197+0000","last_undegraded":"2024-04-25T08:45:14.054197+0000","last_fullsized":"2024-04-25T08:45:14.054197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:23.057014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054794+0000","last_change":"2024-04-25T08:45:11.604893+0000","last_active":"2024-04-25T08:45:14.054794+0000","last_peered":"2024-04-25T08:45:14.054794+0000","last_clean":"2024-04-25T08:45:14.054794+0000","last_became_active":"2024-04-25T08:45:11.603555+0000","last_became_peered":"2024-04-25T08:45:11.603555+0000","last_unstale":"2024-04-25T08:45:14.054794+0000","last_undegraded":"2024-04-25T08:45:14.054794+0000","last_fullsized":"2024-04-25T08:45:14.054794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:41.076296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055025+0000","last_change":"2024-04-25T08:45:12.617378+0000","last_active":"2024-04-25T08:45:14.055025+0000","last_peered":"2024-04-25T08:45:14.055025+0000","last_clean":"2024-04-25T08:45:14.055025+0000","last_became_active":"2024-04-25T08:45:12.617019+0000","last_became_peered":"2024-04-25T08:45:12.617019+0000","last_unstale":"2024-04-25T08:45:14.055025+0000","last_undegraded":"2024-04-25T08:45:14.055025+0000","last_fullsized":"2024-04-25T08:45:14.055025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:21.762963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054217+0000","last_change":"2024-04-25T08:45:11.613393+0000","last_active":"2024-04-25T08:45:14.054217+0000","last_peered":"2024-04-25T08:45:14.054217+0000","last_clean":"2024-04-25T08:45:14.054217+0000","last_became_active":"2024-04-25T08:45:11.611939+0000","last_became_peered":"2024-04-25T08:45:11.611939+0000","last_unstale":"2024-04-25T08:45:14.054217+0000","last_undegraded":"2024-04-25T08:45:14.054217+0000","last_fullsized":"2024-04-25T08:45:14.054217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:26.349334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617459+0000","last_change":"2024-04-25T08:45:12.625753+0000","last_active":"2024-04-25T08:45:13.617459+0000","last_peered":"2024-04-25T08:45:13.617459+0000","last_clean":"2024-04-25T08:45:13.617459+0000","last_became_active":"2024-04-25T08:45:12.625145+0000","last_became_peered":"2024-04-25T08:45:12.625145+0000","last_unstale":"2024-04-25T08:45:13.617459+0000","last_undegraded":"2024-04-25T08:45:13.617459+0000","last_fullsized":"2024-04-25T08:45:13.617459+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:18.509131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055368+0000","last_change":"2024-04-25T08:45:11.611641+0000","last_active":"2024-04-25T08:45:14.055368+0000","last_peered":"2024-04-25T08:45:14.055368+0000","last_clean":"2024-04-25T08:45:14.055368+0000","last_became_active":"2024-04-25T08:45:11.611380+0000","last_became_peered":"2024-04-25T08:45:11.611380+0000","last_unstale":"2024-04-25T08:45:14.055368+0000","last_undegraded":"2024-04-25T08:45:14.055368+0000","last_fullsized":"2024-04-25T08:45:14.055368+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:05.281565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055155+0000","last_change":"2024-04-25T08:45:12.617333+0000","last_active":"2024-04-25T08:45:14.055155+0000","last_peered":"2024-04-25T08:45:14.055155+0000","last_clean":"2024-04-25T08:45:14.055155+0000","last_became_active":"2024-04-25T08:45:12.616957+0000","last_became_peered":"2024-04-25T08:45:12.616957+0000","last_unstale":"2024-04-25T08:45:14.055155+0000","last_undegraded":"2024-04-25T08:45:14.055155+0000","last_fullsized":"2024-04-25T08:45:14.055155+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:23.887239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617514+0000","last_change":"2024-04-25T08:45:11.612291+0000","last_active":"2024-04-25T08:45:13.617514+0000","last_peered":"2024-04-25T08:45:13.617514+0000","last_clean":"2024-04-25T08:45:13.617514+0000","last_became_active":"2024-04-25T08:45:11.612016+0000","last_became_peered":"2024-04-25T08:45:11.612016+0000","last_unstale":"2024-04-25T08:45:13.617514+0000","last_undegraded":"2024-04-25T08:45:13.617514+0000","last_fullsized":"2024-04-25T08:45:13.617514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:21.851757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054736+0000","last_change":"2024-04-25T08:45:12.624785+0000","last_active":"2024-04-25T08:45:14.054736+0000","last_peered":"2024-04-25T08:45:14.054736+0000","last_clean":"2024-04-25T08:45:14.054736+0000","last_became_active":"2024-04-25T08:45:12.624436+0000","last_became_peered":"2024-04-25T08:45:12.624436+0000","last_unstale":"2024-04-25T08:45:14.054736+0000","last_undegraded":"2024-04-25T08:45:14.054736+0000","last_fullsized":"2024-04-25T08:45:14.054736+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:58.473699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055438+0000","last_change":"2024-04-25T08:45:11.612056+0000","last_active":"2024-04-25T08:45:14.055438+0000","last_peered":"2024-04-25T08:45:14.055438+0000","last_clean":"2024-04-25T08:45:14.055438+0000","last_became_active":"2024-04-25T08:45:11.611677+0000","last_became_peered":"2024-04-25T08:45:11.611677+0000","last_unstale":"2024-04-25T08:45:14.055438+0000","last_undegraded":"2024-04-25T08:45:14.055438+0000","last_fullsized":"2024-04-25T08:45:14.055438+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:26.912423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617416+0000","last_change":"2024-04-25T08:45:12.612376+0000","last_active":"2024-04-25T08:45:13.617416+0000","last_peered":"2024-04-25T08:45:13.617416+0000","last_clean":"2024-04-25T08:45:13.617416+0000","last_became_active":"2024-04-25T08:45:12.610881+0000","last_became_peered":"2024-04-25T08:45:12.610881+0000","last_unstale":"2024-04-25T08:45:13.617416+0000","last_undegraded":"2024-04-25T08:45:13.617416+0000","last_fullsized":"2024-04-25T08:45:13.617416+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:26.336019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":20,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617431+0000","last_change":"2024-04-25T08:45:11.615380+0000","last_active":"2024-04-25T08:45:13.617431+0000","last_peered":"2024-04-25T08:45:13.617431+0000","last_clean":"2024-04-25T08:45:13.617431+0000","last_became_active":"2024-04-25T08:45:11.612863+0000","last_became_peered":"2024-04-25T08:45:11.612863+0000","last_unstale":"2024-04-25T08:45:13.617431+0000","last_undegraded":"2024-04-25T08:45:13.617431+0000","last_fullsized":"2024-04-25T08:45:13.617431+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:57.910041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616448+0000","last_change":"2024-04-25T08:45:12.624348+0000","last_active":"2024-04-25T08:45:13.616448+0000","last_peered":"2024-04-25T08:45:13.616448+0000","last_clean":"2024-04-25T08:45:13.616448+0000","last_became_active":"2024-04-25T08:45:12.624035+0000","last_became_peered":"2024-04-25T08:45:12.624035+0000","last_unstale":"2024-04-25T08:45:13.616448+0000","last_undegraded":"2024-04-25T08:45:13.616448+0000","last_fullsized":"2024-04-25T08:45:13.616448+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:30.841208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055234+0000","last_change":"2024-04-25T08:45:11.609636+0000","last_active":"2024-04-25T08:45:14.055234+0000","last_peered":"2024-04-25T08:45:14.055234+0000","last_clean":"2024-04-25T08:45:14.055234+0000","last_became_active":"2024-04-25T08:45:11.609296+0000","last_became_peered":"2024-04-25T08:45:11.609296+0000","last_unstale":"2024-04-25T08:45:14.055234+0000","last_undegraded":"2024-04-25T08:45:14.055234+0000","last_fullsized":"2024-04-25T08:45:14.055234+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:22.051173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.625600+0000","last_change":"2024-04-25T08:45:12.628835+0000","last_active":"2024-04-25T08:45:13.625600+0000","last_peered":"2024-04-25T08:45:13.625600+0000","last_clean":"2024-04-25T08:45:13.625600+0000","last_became_active":"2024-04-25T08:45:12.628471+0000","last_became_peered":"2024-04-25T08:45:12.628471+0000","last_unstale":"2024-04-25T08:45:13.625600+0000","last_undegraded":"2024-04-25T08:45:13.625600+0000","last_fullsized":"2024-04-25T08:45:13.625600+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:21.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.625529+0000","last_change":"2024-04-25T08:45:11.618604+0000","last_active":"2024-04-25T08:45:13.625529+0000","last_peered":"2024-04-25T08:45:13.625529+0000","last_clean":"2024-04-25T08:45:13.625529+0000","last_became_active":"2024-04-25T08:45:11.618224+0000","last_became_peered":"2024-04-25T08:45:11.618224+0000","last_unstale":"2024-04-25T08:45:13.625529+0000","last_undegraded":"2024-04-25T08:45:13.625529+0000","last_fullsized":"2024-04-25T08:45:13.625529+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:03.641098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054352+0000","last_change":"2024-04-25T08:45:12.631732+0000","last_active":"2024-04-25T08:45:14.054352+0000","last_peered":"2024-04-25T08:45:14.054352+0000","last_clean":"2024-04-25T08:45:14.054352+0000","last_became_active":"2024-04-25T08:45:12.631412+0000","last_became_peered":"2024-04-25T08:45:12.631412+0000","last_unstale":"2024-04-25T08:45:14.054352+0000","last_undegraded":"2024-04-25T08:45:14.054352+0000","last_fullsized":"2024-04-25T08:45:14.054352+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:06.287405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055499+0000","last_change":"2024-04-25T08:45:11.611078+0000","last_active":"2024-04-25T08:45:14.055499+0000","last_peered":"2024-04-25T08:45:14.055499+0000","last_clean":"2024-04-25T08:45:14.055499+0000","last_became_active":"2024-04-25T08:45:11.610749+0000","last_became_peered":"2024-04-25T08:45:11.610749+0000","last_unstale":"2024-04-25T08:45:14.055499+0000","last_undegraded":"2024-04-25T08:45:14.055499+0000","last_fullsized":"2024-04-25T08:45:14.055499+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:27.620570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617259+0000","last_change":"2024-04-25T08:45:12.632646+0000","last_active":"2024-04-25T08:45:13.617259+0000","last_peered":"2024-04-25T08:45:13.617259+0000","last_clean":"2024-04-25T08:45:13.617259+0000","last_became_active":"2024-04-25T08:45:12.632362+0000","last_became_peered":"2024-04-25T08:45:12.632362+0000","last_unstale":"2024-04-25T08:45:13.617259+0000","last_undegraded":"2024-04-25T08:45:13.617259+0000","last_fullsized":"2024-04-25T08:45:13.617259+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:27.935904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616403+0000","last_change":"2024-04-25T08:45:11.613128+0000","last_active":"2024-04-25T08:45:13.616403+0000","last_peered":"2024-04-25T08:45:13.616403+0000","last_clean":"2024-04-25T08:45:13.616403+0000","last_became_active":"2024-04-25T08:45:11.611609+0000","last_became_peered":"2024-04-25T08:45:11.611609+0000","last_unstale":"2024-04-25T08:45:13.616403+0000","last_undegraded":"2024-04-25T08:45:13.616403+0000","last_fullsized":"2024-04-25T08:45:13.616403+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:33.153694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055038+0000","last_change":"2024-04-25T08:45:12.628755+0000","last_active":"2024-04-25T08:45:14.055038+0000","last_peered":"2024-04-25T08:45:14.055038+0000","last_clean":"2024-04-25T08:45:14.055038+0000","last_became_active":"2024-04-25T08:45:12.628415+0000","last_became_peered":"2024-04-25T08:45:12.628415+0000","last_unstale":"2024-04-25T08:45:14.055038+0000","last_undegraded":"2024-04-25T08:45:14.055038+0000","last_fullsized":"2024-04-25T08:45:14.055038+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:21.082241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.626084+0000","last_change":"2024-04-25T08:45:11.618549+0000","last_active":"2024-04-25T08:45:13.626084+0000","last_peered":"2024-04-25T08:45:13.626084+0000","last_clean":"2024-04-25T08:45:13.626084+0000","last_became_active":"2024-04-25T08:45:11.618187+0000","last_became_peered":"2024-04-25T08:45:11.618187+0000","last_unstale":"2024-04-25T08:45:13.626084+0000","last_undegraded":"2024-04-25T08:45:13.626084+0000","last_fullsized":"2024-04-25T08:45:13.626084+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:29.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055002+0000","last_change":"2024-04-25T08:45:12.631779+0000","last_active":"2024-04-25T08:45:14.055002+0000","last_peered":"2024-04-25T08:45:14.055002+0000","last_clean":"2024-04-25T08:45:14.055002+0000","last_became_active":"2024-04-25T08:45:12.631434+0000","last_became_peered":"2024-04-25T08:45:12.631434+0000","last_unstale":"2024-04-25T08:45:14.055002+0000","last_undegraded":"2024-04-25T08:45:14.055002+0000","last_fullsized":"2024-04-25T08:45:14.055002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:13.232088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055104+0000","last_change":"2024-04-25T08:45:11.609242+0000","last_active":"2024-04-25T08:45:14.055104+0000","last_peered":"2024-04-25T08:45:14.055104+0000","last_clean":"2024-04-25T08:45:14.055104+0000","last_became_active":"2024-04-25T08:45:11.608930+0000","last_became_peered":"2024-04-25T08:45:11.608930+0000","last_unstale":"2024-04-25T08:45:14.055104+0000","last_undegraded":"2024-04-25T08:45:14.055104+0000","last_fullsized":"2024-04-25T08:45:14.055104+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:17.476441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617563+0000","last_change":"2024-04-25T08:45:12.622990+0000","last_active":"2024-04-25T08:45:13.617563+0000","last_peered":"2024-04-25T08:45:13.617563+0000","last_clean":"2024-04-25T08:45:13.617563+0000","last_became_active":"2024-04-25T08:45:12.622719+0000","last_became_peered":"2024-04-25T08:45:12.622719+0000","last_unstale":"2024-04-25T08:45:13.617563+0000","last_undegraded":"2024-04-25T08:45:13.617563+0000","last_fullsized":"2024-04-25T08:45:13.617563+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:49.303156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055307+0000","last_change":"2024-04-25T08:45:11.608806+0000","last_active":"2024-04-25T08:45:14.055307+0000","last_peered":"2024-04-25T08:45:14.055307+0000","last_clean":"2024-04-25T08:45:14.055307+0000","last_became_active":"2024-04-25T08:45:11.607645+0000","last_became_peered":"2024-04-25T08:45:11.607645+0000","last_unstale":"2024-04-25T08:45:14.055307+0000","last_undegraded":"2024-04-25T08:45:14.055307+0000","last_fullsized":"2024-04-25T08:45:14.055307+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:25.330178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055313+0000","last_change":"2024-04-25T08:45:12.625848+0000","last_active":"2024-04-25T08:45:14.055313+0000","last_peered":"2024-04-25T08:45:14.055313+0000","last_clean":"2024-04-25T08:45:14.055313+0000","last_became_active":"2024-04-25T08:45:12.625565+0000","last_became_peered":"2024-04-25T08:45:12.625565+0000","last_unstale":"2024-04-25T08:45:14.055313+0000","last_undegraded":"2024-04-25T08:45:14.055313+0000","last_fullsized":"2024-04-25T08:45:14.055313+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:18.509131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055056+0000","last_change":"2024-04-25T08:45:11.605238+0000","last_active":"2024-04-25T08:45:14.055056+0000","last_peered":"2024-04-25T08:45:14.055056+0000","last_clean":"2024-04-25T08:45:14.055056+0000","last_became_active":"2024-04-25T08:45:11.604975+0000","last_became_peered":"2024-04-25T08:45:11.604975+0000","last_unstale":"2024-04-25T08:45:14.055056+0000","last_undegraded":"2024-04-25T08:45:14.055056+0000","last_fullsized":"2024-04-25T08:45:14.055056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:06.042685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054368+0000","last_change":"2024-04-25T08:45:12.616153+0000","last_active":"2024-04-25T08:45:14.054368+0000","last_peered":"2024-04-25T08:45:14.054368+0000","last_clean":"2024-04-25T08:45:14.054368+0000","last_became_active":"2024-04-25T08:45:12.615393+0000","last_became_peered":"2024-04-25T08:45:12.615393+0000","last_unstale":"2024-04-25T08:45:14.054368+0000","last_undegraded":"2024-04-25T08:45:14.054368+0000","last_fullsized":"2024-04-25T08:45:14.054368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:08.274280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054358+0000","last_change":"2024-04-25T08:45:11.616493+0000","last_active":"2024-04-25T08:45:14.054358+0000","last_peered":"2024-04-25T08:45:14.054358+0000","last_clean":"2024-04-25T08:45:14.054358+0000","last_became_active":"2024-04-25T08:45:11.616241+0000","last_became_peered":"2024-04-25T08:45:11.616241+0000","last_unstale":"2024-04-25T08:45:14.054358+0000","last_undegraded":"2024-04-25T08:45:14.054358+0000","last_fullsized":"2024-04-25T08:45:14.054358+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:00.629717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.625610+0000","last_change":"2024-04-25T08:45:12.634083+0000","last_active":"2024-04-25T08:45:13.625610+0000","last_peered":"2024-04-25T08:45:13.625610+0000","last_clean":"2024-04-25T08:45:13.625610+0000","last_became_active":"2024-04-25T08:45:12.633734+0000","last_became_peered":"2024-04-25T08:45:12.633734+0000","last_unstale":"2024-04-25T08:45:13.625610+0000","last_undegraded":"2024-04-25T08:45:13.625610+0000","last_fullsized":"2024-04-25T08:45:13.625610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:44.309134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.625677+0000","last_change":"2024-04-25T08:45:11.615924+0000","last_active":"2024-04-25T08:45:13.625677+0000","last_peered":"2024-04-25T08:45:13.625677+0000","last_clean":"2024-04-25T08:45:13.625677+0000","last_became_active":"2024-04-25T08:45:11.613543+0000","last_became_peered":"2024-04-25T08:45:11.613543+0000","last_unstale":"2024-04-25T08:45:13.625677+0000","last_undegraded":"2024-04-25T08:45:13.625677+0000","last_fullsized":"2024-04-25T08:45:13.625677+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:22.051173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054547+0000","last_change":"2024-04-25T08:45:12.617304+0000","last_active":"2024-04-25T08:45:14.054547+0000","last_peered":"2024-04-25T08:45:14.054547+0000","last_clean":"2024-04-25T08:45:14.054547+0000","last_became_active":"2024-04-25T08:45:12.617002+0000","last_became_peered":"2024-04-25T08:45:12.617002+0000","last_unstale":"2024-04-25T08:45:14.054547+0000","last_undegraded":"2024-04-25T08:45:14.054547+0000","last_fullsized":"2024-04-25T08:45:14.054547+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:26.336019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054222+0000","last_change":"2024-04-25T08:45:11.613996+0000","last_active":"2024-04-25T08:45:14.054222+0000","last_peered":"2024-04-25T08:45:14.054222+0000","last_clean":"2024-04-25T08:45:14.054222+0000","last_became_active":"2024-04-25T08:45:11.613688+0000","last_became_peered":"2024-04-25T08:45:11.613688+0000","last_unstale":"2024-04-25T08:45:14.054222+0000","last_undegraded":"2024-04-25T08:45:14.054222+0000","last_fullsized":"2024-04-25T08:45:14.054222+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:36.596184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054765+0000","last_change":"2024-04-25T08:45:12.614583+0000","last_active":"2024-04-25T08:45:14.054765+0000","last_peered":"2024-04-25T08:45:14.054765+0000","last_clean":"2024-04-25T08:45:14.054765+0000","last_became_active":"2024-04-25T08:45:12.613687+0000","last_became_peered":"2024-04-25T08:45:12.613687+0000","last_unstale":"2024-04-25T08:45:14.054765+0000","last_undegraded":"2024-04-25T08:45:14.054765+0000","last_fullsized":"2024-04-25T08:45:14.054765+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:25.317071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.626245+0000","last_change":"2024-04-25T08:45:11.616861+0000","last_active":"2024-04-25T08:45:13.626245+0000","last_peered":"2024-04-25T08:45:13.626245+0000","last_clean":"2024-04-25T08:45:13.626245+0000","last_became_active":"2024-04-25T08:45:11.616539+0000","last_became_peered":"2024-04-25T08:45:11.616539+0000","last_unstale":"2024-04-25T08:45:13.626245+0000","last_undegraded":"2024-04-25T08:45:13.626245+0000","last_fullsized":"2024-04-25T08:45:13.626245+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:17.476441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616739+0000","last_change":"2024-04-25T08:45:12.618473+0000","last_active":"2024-04-25T08:45:13.616739+0000","last_peered":"2024-04-25T08:45:13.616739+0000","last_clean":"2024-04-25T08:45:13.616739+0000","last_became_active":"2024-04-25T08:45:12.618097+0000","last_became_peered":"2024-04-25T08:45:12.618097+0000","last_unstale":"2024-04-25T08:45:13.616739+0000","last_undegraded":"2024-04-25T08:45:13.616739+0000","last_fullsized":"2024-04-25T08:45:13.616739+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:30.160528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"65'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.626165+0000","last_change":"2024-04-25T08:45:11.616486+0000","last_active":"2024-04-25T08:45:13.626165+0000","last_peered":"2024-04-25T08:45:13.626165+0000","last_clean":"2024-04-25T08:45:13.626165+0000","last_became_active":"2024-04-25T08:45:11.614742+0000","last_became_peered":"2024-04-25T08:45:11.614742+0000","last_unstale":"2024-04-25T08:45:13.626165+0000","last_undegraded":"2024-04-25T08:45:13.626165+0000","last_fullsized":"2024-04-25T08:45:13.626165+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:26.912423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.626027+0000","last_change":"2024-04-25T08:45:12.620457+0000","last_active":"2024-04-25T08:45:13.626027+0000","last_peered":"2024-04-25T08:45:13.626027+0000","last_clean":"2024-04-25T08:45:13.626027+0000","last_became_active":"2024-04-25T08:45:12.620202+0000","last_became_peered":"2024-04-25T08:45:12.620202+0000","last_unstale":"2024-04-25T08:45:13.626027+0000","last_undegraded":"2024-04-25T08:45:13.626027+0000","last_fullsized":"2024-04-25T08:45:13.626027+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:01.635558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054550+0000","last_change":"2024-04-25T08:45:11.615530+0000","last_active":"2024-04-25T08:45:14.054550+0000","last_peered":"2024-04-25T08:45:14.054550+0000","last_clean":"2024-04-25T08:45:14.054550+0000","last_became_active":"2024-04-25T08:45:11.615180+0000","last_became_peered":"2024-04-25T08:45:11.615180+0000","last_unstale":"2024-04-25T08:45:14.054550+0000","last_undegraded":"2024-04-25T08:45:14.054550+0000","last_fullsized":"2024-04-25T08:45:14.054550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:01.651766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055311+0000","last_change":"2024-04-25T08:45:12.624252+0000","last_active":"2024-04-25T08:45:14.055311+0000","last_peered":"2024-04-25T08:45:14.055311+0000","last_clean":"2024-04-25T08:45:14.055311+0000","last_became_active":"2024-04-25T08:45:12.622248+0000","last_became_peered":"2024-04-25T08:45:12.622248+0000","last_unstale":"2024-04-25T08:45:14.055311+0000","last_undegraded":"2024-04-25T08:45:14.055311+0000","last_fullsized":"2024-04-25T08:45:14.055311+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:32.406528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617574+0000","last_change":"2024-04-25T08:45:11.615371+0000","last_active":"2024-04-25T08:45:13.617574+0000","last_peered":"2024-04-25T08:45:13.617574+0000","last_clean":"2024-04-25T08:45:13.617574+0000","last_became_active":"2024-04-25T08:45:11.611714+0000","last_became_peered":"2024-04-25T08:45:11.611714+0000","last_unstale":"2024-04-25T08:45:13.617574+0000","last_undegraded":"2024-04-25T08:45:13.617574+0000","last_fullsized":"2024-04-25T08:45:13.617574+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:29.835367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054230+0000","last_change":"2024-04-25T08:45:12.637306+0000","last_active":"2024-04-25T08:45:14.054230+0000","last_peered":"2024-04-25T08:45:14.054230+0000","last_clean":"2024-04-25T08:45:14.054230+0000","last_became_active":"2024-04-25T08:45:12.637053+0000","last_became_peered":"2024-04-25T08:45:12.637053+0000","last_unstale":"2024-04-25T08:45:14.054230+0000","last_undegraded":"2024-04-25T08:45:14.054230+0000","last_fullsized":"2024-04-25T08:45:14.054230+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:04.646939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055351+0000","last_change":"2024-04-25T08:45:11.610635+0000","last_active":"2024-04-25T08:45:14.055351+0000","last_peered":"2024-04-25T08:45:14.055351+0000","last_clean":"2024-04-25T08:45:14.055351+0000","last_became_active":"2024-04-25T08:45:11.609768+0000","last_became_peered":"2024-04-25T08:45:11.609768+0000","last_unstale":"2024-04-25T08:45:14.055351+0000","last_undegraded":"2024-04-25T08:45:14.055351+0000","last_fullsized":"2024-04-25T08:45:14.055351+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:03.641098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616770+0000","last_change":"2024-04-25T08:45:12.631433+0000","last_active":"2024-04-25T08:45:13.616770+0000","last_peered":"2024-04-25T08:45:13.616770+0000","last_clean":"2024-04-25T08:45:13.616770+0000","last_became_active":"2024-04-25T08:45:12.631165+0000","last_became_peered":"2024-04-25T08:45:12.631165+0000","last_unstale":"2024-04-25T08:45:13.616770+0000","last_undegraded":"2024-04-25T08:45:13.616770+0000","last_fullsized":"2024-04-25T08:45:13.616770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:04.646939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.055298+0000","last_change":"2024-04-25T08:45:11.599255+0000","last_active":"2024-04-25T08:45:14.055298+0000","last_peered":"2024-04-25T08:45:14.055298+0000","last_clean":"2024-04-25T08:45:14.055298+0000","last_became_active":"2024-04-25T08:45:11.598904+0000","last_became_peered":"2024-04-25T08:45:11.598904+0000","last_unstale":"2024-04-25T08:45:14.055298+0000","last_undegraded":"2024-04-25T08:45:14.055298+0000","last_fullsized":"2024-04-25T08:45:14.055298+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:49.102965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.617235+0000","last_change":"2024-04-25T08:45:12.616155+0000","last_active":"2024-04-25T08:45:13.617235+0000","last_peered":"2024-04-25T08:45:13.617235+0000","last_clean":"2024-04-25T08:45:13.617235+0000","last_became_active":"2024-04-25T08:45:12.615868+0000","last_became_peered":"2024-04-25T08:45:12.615868+0000","last_unstale":"2024-04-25T08:45:13.617235+0000","last_undegraded":"2024-04-25T08:45:13.617235+0000","last_fullsized":"2024-04-25T08:45:13.617235+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:13.232088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054845+0000","last_change":"2024-04-25T08:45:11.603866+0000","last_active":"2024-04-25T08:45:14.054845+0000","last_peered":"2024-04-25T08:45:14.054845+0000","last_clean":"2024-04-25T08:45:14.054845+0000","last_became_active":"2024-04-25T08:45:11.603106+0000","last_became_peered":"2024-04-25T08:45:11.603106+0000","last_unstale":"2024-04-25T08:45:14.054845+0000","last_undegraded":"2024-04-25T08:45:14.054845+0000","last_fullsized":"2024-04-25T08:45:14.054845+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:36.537439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616530+0000","last_change":"2024-04-25T08:45:12.628445+0000","last_active":"2024-04-25T08:45:13.616530+0000","last_peered":"2024-04-25T08:45:13.616530+0000","last_clean":"2024-04-25T08:45:13.616530+0000","last_became_active":"2024-04-25T08:45:12.627669+0000","last_became_peered":"2024-04-25T08:45:12.627669+0000","last_unstale":"2024-04-25T08:45:13.616530+0000","last_undegraded":"2024-04-25T08:45:13.616530+0000","last_fullsized":"2024-04-25T08:45:13.616530+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:19.724624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:13.616459+0000","last_change":"2024-04-25T08:45:11.610404+0000","last_active":"2024-04-25T08:45:13.616459+0000","last_peered":"2024-04-25T08:45:13.616459+0000","last_clean":"2024-04-25T08:45:13.616459+0000","last_became_active":"2024-04-25T08:45:11.610147+0000","last_became_peered":"2024-04-25T08:45:11.610147+0000","last_unstale":"2024-04-25T08:45:13.616459+0000","last_undegraded":"2024-04-25T08:45:13.616459+0000","last_fullsized":"2024-04-25T08:45:13.616459+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:27.517500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054718+0000","last_change":"2024-04-25T08:45:12.637230+0000","last_active":"2024-04-25T08:45:14.054718+0000","last_peered":"2024-04-25T08:45:14.054718+0000","last_clean":"2024-04-25T08:45:14.054718+0000","last_became_active":"2024-04-25T08:45:12.636865+0000","last_became_peered":"2024-04-25T08:45:12.636865+0000","last_unstale":"2024-04-25T08:45:14.054718+0000","last_undegraded":"2024-04-25T08:45:14.054718+0000","last_fullsized":"2024-04-25T08:45:14.054718+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:11.583601+0000","last_clean_scrub_stamp":"2024-04-25T08:45:11.583601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:49.303156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054632+0000","last_change":"2024-04-25T08:45:11.606470+0000","last_active":"2024-04-25T08:45:14.054632+0000","last_peered":"2024-04-25T08:45:14.054632+0000","last_clean":"2024-04-25T08:45:14.054632+0000","last_became_active":"2024-04-25T08:45:11.606206+0000","last_became_peered":"2024-04-25T08:45:11.606206+0000","last_unstale":"2024-04-25T08:45:14.054632+0000","last_undegraded":"2024-04-25T08:45:14.054632+0000","last_fullsized":"2024-04-25T08:45:14.054632+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:45:10.577760+0000","last_clean_scrub_stamp":"2024-04-25T08:45:10.577760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:00.629717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T08:45:14.054839+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:45:14.054839+0000","last_peered":"2024-04-25T08:45:14.054839+0000","last_clean":"2024-04-25T08:45:14.054839+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:45:14.054839+0000","last_undegraded":"2024-04-25T08:45:14.054839+0000","last_fullsized":"2024-04-25T08:45:14.054839+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1396736,"data_stored":946748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":355638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":355626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":357216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31948,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339892,"statfs":{"total":96636764160,"available":96604049408,"internally_reserved":0,"allocated":786432,"data_stored":355604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":786432,"data_stored":355604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":815104,"data_stored":357980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1380352,"data_stored":945972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:45:21.651 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:45:21.651 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:45:21.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.842+0000 7f374ee00640 1 Processor -- start 2024-04-25T08:45:21.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.842+0000 7f374ee00640 1 -- start start 2024-04-25T08:45:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ee00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f374815dd50 0x7f37480cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37480d0450 0x7f37480cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374809a780 con 0x7f374806f4d0 2024-04-25T08:45:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ee00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37480708c0 con 0x7f374815dd50 2024-04-25T08:45:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374809aa60 con 0x7f37480d0450 2024-04-25T08:45:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f3747e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f374806f4d0 0x7f37480cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49102/0 (socket says 172.21.15.46:49102) 2024-04-25T08:45:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f3747e00640 1 -- 172.21.15.46:0/2891935791 learned_addr learned my addr 172.21.15.46:0/2891935791 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f373f400640 1 --2- 172.21.15.46:0/2891935791 >> v2:172.21.15.82:3300/0 conn(0x7f374815dd50 0x7f37480cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ce00640 1 --2- 172.21.15.46:0/2891935791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37480d0450 0x7f37480cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f3747400640 1 -- 172.21.15.46:0/2891935791 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466889872 0 0) 0x7f374809a780 con 0x7f374806f4d0 2024-04-25T08:45:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f3747400640 1 -- 172.21.15.46:0/2891935791 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f372c003530 con 0x7f374806f4d0 2024-04-25T08:45:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ce00640 1 -- 172.21.15.46:0/2891935791 >> v2:172.21.15.82:3300/0 conn(0x7f374815dd50 msgr2=0x7f37480cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ce00640 1 --2- 172.21.15.46:0/2891935791 >> v2:172.21.15.82:3300/0 conn(0x7f374815dd50 0x7f37480cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ce00640 1 -- 172.21.15.46:0/2891935791 >> v1:172.21.15.46:6789/0 conn(0x7f374806f4d0 legacy=0x7f37480cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.846+0000 7f374ce00640 1 -- 172.21.15.46:0/2891935791 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374809ad40 con 0x7f37480d0450 2024-04-25T08:45:21.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ce00640 1 --2- 172.21.15.46:0/2891935791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37480d0450 0x7f37480cc4c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3740009d90 tx=0x7f374001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ecb3be9e66a7e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f3747400640 1 -- 172.21.15.46:0/2891935791 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37400b2070 con 0x7f37480d0450 2024-04-25T08:45:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f3747400640 1 -- 172.21.15.46:0/2891935791 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37400a5a00 con 0x7f37480d0450 2024-04-25T08:45:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f3747400640 1 -- 172.21.15.46:0/2891935791 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37400a5dd0 con 0x7f37480d0450 2024-04-25T08:45:21.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 -- 172.21.15.46:0/2891935791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37480d0450 msgr2=0x7f37480cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 --2- 172.21.15.46:0/2891935791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37480d0450 0x7f37480cc4c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3740009d90 tx=0x7f374001c940 comp rx=0 tx=0).stop 2024-04-25T08:45:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 -- 172.21.15.46:0/2891935791 shutdown_connections 2024-04-25T08:45:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 --2- 172.21.15.46:0/2891935791 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f37480d0450 0x7f37480cc4c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 --2- 172.21.15.46:0/2891935791 >> v2:172.21.15.82:3300/0 conn(0x7f374815dd50 0x7f37480cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 -- 172.21.15.46:0/2891935791 >> 172.21.15.46:0/2891935791 conn(0x7f37480b3160 msgr2=0x7f37480b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 -- 172.21.15.46:0/2891935791 shutdown_connections 2024-04-25T08:45:21.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 -- 172.21.15.46:0/2891935791 wait complete. 2024-04-25T08:45:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 Processor -- start 2024-04-25T08:45:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 -- start start 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f374806f4d0 0x7f37480b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37480d0450 0x7f37481600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f374815dd50 0x7f37480b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37480cd390 con 0x7f374806f4d0 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37480cd210 con 0x7f37480d0450 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.850+0000 7f374ee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37480cd510 con 0x7f374815dd50 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3747e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f374806f4d0 0x7f37480b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3747e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f374806f4d0 0x7f37480b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49624/0 (socket says 172.21.15.46:49624) 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3747e00640 1 -- 172.21.15.46:0/609899909 learned_addr learned my addr 172.21.15.46:0/609899909 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f374ce00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f374815dd50 0x7f37480b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f373f400640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37480d0450 0x7f37481600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3747e00640 1 -- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f374815dd50 msgr2=0x7f37480b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3747e00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f374815dd50 0x7f37480b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3747e00640 1 -- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37480d0450 msgr2=0x7f37481600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3747e00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37480d0450 0x7f37481600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3747e00640 1 -- 172.21.15.46:0/609899909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374815aee0 con 0x7f374806f4d0 2024-04-25T08:45:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3747e00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f374806f4d0 0x7f37480b6c90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f373800c450 tx=0x7f373800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3744c00640 1 -- 172.21.15.46:0/609899909 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738017070 con 0x7f374806f4d0 2024-04-25T08:45:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3744c00640 1 -- 172.21.15.46:0/609899909 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3738005cf0 con 0x7f374806f4d0 2024-04-25T08:45:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3744c00640 1 -- 172.21.15.46:0/609899909 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738007290 con 0x7f374806f4d0 2024-04-25T08:45:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374815ec00 con 0x7f374806f4d0 2024-04-25T08:45:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374815b1c0 con 0x7f374806f4d0 2024-04-25T08:45:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3748069010 con 0x7f374806f4d0 2024-04-25T08:45:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3744c00640 1 -- 172.21.15.46:0/609899909 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f373801a070 con 0x7f374806f4d0 2024-04-25T08:45:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3744c00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3724066f60 0x7f3724069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.854+0000 7f3744c00640 1 -- 172.21.15.46:0/609899909 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3738084570 con 0x7f374806f4d0 2024-04-25T08:45:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.858+0000 7f373f400640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3724066f60 0x7f3724069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.858+0000 7f373f400640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3724066f60 0x7f3724069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3748161460 tx=0x7f3730009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:21.858+0000 7f3744c00640 1 -- 172.21.15.46:0/609899909 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3738054d00 con 0x7f374806f4d0 2024-04-25T08:45:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.082+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f37480cc610 con 0x7f374806f4d0 2024-04-25T08:45:22.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.082+0000 7f3744c00640 1 -- 172.21.15.46:0/609899909 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3738058da0 con 0x7f374806f4d0 2024-04-25T08:45:22.077 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-25T08:45:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q== 2024-04-25T08:45:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-25T08:45:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-25T08:45:22.077 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:45:22.078 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-25T08:45:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3724066f60 msgr2=0x7f3724069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3724066f60 0x7f3724069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3748161460 tx=0x7f3730009190 comp rx=0 tx=0).stop 2024-04-25T08:45:22.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f374806f4d0 msgr2=0x7f37480b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f374806f4d0 0x7f37480b6c90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f373800c450 tx=0x7f373800c910 comp rx=0 tx=0).stop 2024-04-25T08:45:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 shutdown_connections 2024-04-25T08:45:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3724066f60 0x7f3724069420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f374815dd50 0x7f37480b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37480d0450 0x7f37481600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 --2- 172.21.15.46:0/609899909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f374806f4d0 0x7f37480b6c90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 >> 172.21.15.46:0/609899909 conn(0x7f37480b3160 msgr2=0x7f37480af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 shutdown_connections 2024-04-25T08:45:22.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.086+0000 7f374ee00640 1 -- 172.21.15.46:0/609899909 wait complete. 2024-04-25T08:45:22.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:45:22.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.246+0000 7f675fa00640 1 Processor -- start 2024-04-25T08:45:22.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.246+0000 7f675fa00640 1 -- start start 2024-04-25T08:45:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.246+0000 7f675fa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f67580b3340 0x7f6758164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.246+0000 7f675fa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67580b3790 con 0x7f67580cc150 2024-04-25T08:45:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.246+0000 7f675fa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67580b25a0 con 0x7f6758157b00 2024-04-25T08:45:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.246+0000 7f675fa00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67580b2eb0 con 0x7f67580b3340 2024-04-25T08:45:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.246+0000 7f6757e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f67580cc150 0x7f67580ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49112/0 (socket says 172.21.15.46:49112) 2024-04-25T08:45:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.246+0000 7f6757e00640 1 -- 172.21.15.46:0/1580646911 learned_addr learned my addr 172.21.15.46:0/1580646911 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:22.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.246+0000 7f675da00640 1 --2- 172.21.15.46:0/1580646911 >> v2:172.21.15.82:3300/0 conn(0x7f67580b3340 0x7f6758164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.250+0000 7f6757400640 1 -- 172.21.15.46:0/1580646911 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223980370 0 0) 0x7f67580b3790 con 0x7f67580cc150 2024-04-25T08:45:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.250+0000 7f6757400640 1 -- 172.21.15.46:0/1580646911 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6740003530 con 0x7f67580cc150 2024-04-25T08:45:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.250+0000 7f675da00640 1 -- 172.21.15.46:0/1580646911 >> v1:172.21.15.82:6789/0 conn(0x7f6758157b00 legacy=0x7f67580cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.250+0000 7f675da00640 1 -- 172.21.15.46:0/1580646911 >> v1:172.21.15.46:6789/0 conn(0x7f67580cc150 legacy=0x7f67580ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:22.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.250+0000 7f675da00640 1 -- 172.21.15.46:0/1580646911 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67580cc6f0 con 0x7f67580b3340 2024-04-25T08:45:22.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.250+0000 7f675da00640 1 --2- 172.21.15.46:0/1580646911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 0x7f6758164dd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f675000aea0 tx=0x7f675001d950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da9fcfe7f99941ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.250+0000 7f6757400640 1 -- 172.21.15.46:0/1580646911 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500a6680 con 0x7f67580b3340 2024-04-25T08:45:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.250+0000 7f6757400640 1 -- 172.21.15.46:0/1580646911 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67500a6820 con 0x7f67580b3340 2024-04-25T08:45:22.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.250+0000 7f6757400640 1 -- 172.21.15.46:0/1580646911 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500a6bf0 con 0x7f67580b3340 2024-04-25T08:45:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 -- 172.21.15.46:0/1580646911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 msgr2=0x7f6758164dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 --2- 172.21.15.46:0/1580646911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 0x7f6758164dd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f675000aea0 tx=0x7f675001d950 comp rx=0 tx=0).stop 2024-04-25T08:45:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 -- 172.21.15.46:0/1580646911 shutdown_connections 2024-04-25T08:45:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 --2- 172.21.15.46:0/1580646911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 0x7f6758164dd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 -- 172.21.15.46:0/1580646911 >> 172.21.15.46:0/1580646911 conn(0x7f67580b12a0 msgr2=0x7f67580b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 -- 172.21.15.46:0/1580646911 shutdown_connections 2024-04-25T08:45:22.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 -- 172.21.15.46:0/1580646911 wait complete. 2024-04-25T08:45:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 Processor -- start 2024-04-25T08:45:22.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 -- start start 2024-04-25T08:45:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 0x7f6758157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67580cc150 0x7f6758141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6758157b00 0x7f67581527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67581674a0 con 0x7f67580cc150 2024-04-25T08:45:22.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6758167320 con 0x7f67580b3340 2024-04-25T08:45:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675fa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6758167620 con 0x7f6758157b00 2024-04-25T08:45:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6757e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67580cc150 0x7f6758141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6757e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67580cc150 0x7f6758141e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49634/0 (socket says 172.21.15.46:49634) 2024-04-25T08:45:22.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6757e00640 1 -- 172.21.15.46:0/4094576558 learned_addr learned my addr 172.21.15.46:0/4094576558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675d000640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 0x7f6758157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675da00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6758157b00 0x7f67581527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6757e00640 1 -- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6758157b00 msgr2=0x7f67581527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6757e00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6758157b00 0x7f67581527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6757e00640 1 -- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 msgr2=0x7f6758157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6757e00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 0x7f6758157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6757e00640 1 -- 172.21.15.46:0/4094576558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675815fca0 con 0x7f67580cc150 2024-04-25T08:45:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f675d000640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 0x7f6758157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:22.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6757e00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67580cc150 0x7f6758141e60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f674400c420 tx=0x7f674400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.254+0000 7f6754c00640 1 -- 172.21.15.46:0/4094576558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744017070 con 0x7f67580cc150 2024-04-25T08:45:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.258+0000 7f6754c00640 1 -- 172.21.15.46:0/4094576558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6744005cf0 con 0x7f67580cc150 2024-04-25T08:45:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.258+0000 7f6754c00640 1 -- 172.21.15.46:0/4094576558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67440072d0 con 0x7f67580cc150 2024-04-25T08:45:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.258+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675815ff80 con 0x7f67580cc150 2024-04-25T08:45:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.258+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6758160540 con 0x7f67580cc150 2024-04-25T08:45:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.258+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6758069080 con 0x7f67580cc150 2024-04-25T08:45:22.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.258+0000 7f6754c00640 1 -- 172.21.15.46:0/4094576558 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f674401a070 con 0x7f67580cc150 2024-04-25T08:45:22.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.258+0000 7f6754c00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6738066f60 0x7f6738069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:22.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.258+0000 7f6754c00640 1 -- 172.21.15.46:0/4094576558 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6744084510 con 0x7f67580cc150 2024-04-25T08:45:22.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.258+0000 7f675d000640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6738066f60 0x7f6738069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:22.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.262+0000 7f675d000640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6738066f60 0x7f6738069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f674c000f70 tx=0x7f674c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:22.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.262+0000 7f6754c00640 1 -- 172.21.15.46:0/4094576558 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6744054d40 con 0x7f67580cc150 2024-04-25T08:45:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.458+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67580724a0 con 0x7f67580cc150 2024-04-25T08:45:22.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.458+0000 7f6754c00640 1 -- 172.21.15.46:0/4094576558 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f6744058de0 con 0x7f67580cc150 2024-04-25T08:45:22.453 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:22.453 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:45:11.989847+0000","modified":"2024-04-25T08:45:15.010275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6207},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6207":{"gid":6207,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1670658295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670658295},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670658295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:45:22.453 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-04-25T08:45:22.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6738066f60 msgr2=0x7f6738069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:22.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6738066f60 0x7f6738069420 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f674c000f70 tx=0x7f674c009040 comp rx=0 tx=0).stop 2024-04-25T08:45:22.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67580cc150 msgr2=0x7f6758141e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:22.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67580cc150 0x7f6758141e60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f674400c420 tx=0x7f674400c8e0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 shutdown_connections 2024-04-25T08:45:22.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6738066f60 0x7f6738069420 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6758157b00 0x7f67581527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67580cc150 0x7f6758141e60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 --2- 172.21.15.46:0/4094576558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67580b3340 0x7f6758157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 >> 172.21.15.46:0/4094576558 conn(0x7f67580b12a0 msgr2=0x7f67580cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 shutdown_connections 2024-04-25T08:45:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:22.462+0000 7f675fa00640 1 -- 172.21.15.46:0/4094576558 wait complete. 2024-04-25T08:45:22.877 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:45:11.989847+0000', 'modified': '2024-04-25T08:45:15.010275+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6207}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6207': {'gid': 6207, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/1670658295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1670658295}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1670658295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:45:22.906 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:45:22.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:45:23.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9cb2000640 1 Processor -- start 2024-04-25T08:45:23.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9cb2000640 1 -- start start 2024-04-25T08:45:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9cb2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06f320 0x7f9cac162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9cb2000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9cac0aed10 0x7f9cac164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9cb2000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac0b35b0 con 0x7f9cac15a750 2024-04-25T08:45:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9cb2000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cac15ab30 con 0x7f9cac0aed10 2024-04-25T08:45:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9cb2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0ce4f0 con 0x7f9cac06f320 2024-04-25T08:45:23.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06f320 0x7f9cac162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9cabe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9cac0aed10 0x7f9cac164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.058+0000 7f9cab400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9cac15a750 0x7f9cac06ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:59716/0 (socket says 172.21.15.46:59716) 2024-04-25T08:45:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.062+0000 7f9cab400640 1 -- 172.21.15.46:0/2942478771 learned_addr learned my addr 172.21.15.46:0/2942478771 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.062+0000 7f9ca3e00640 1 -- 172.21.15.46:0/2942478771 >> v2:172.21.15.82:3300/0 conn(0x7f9cac0aed10 msgr2=0x7f9cac164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.062+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/2942478771 >> v2:172.21.15.82:3300/0 conn(0x7f9cac0aed10 0x7f9cac164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.062+0000 7f9ca3e00640 1 -- 172.21.15.46:0/2942478771 >> v1:172.21.15.82:6789/0 conn(0x7f9cac15a750 legacy=0x7f9cac06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.062+0000 7f9ca3e00640 1 -- 172.21.15.46:0/2942478771 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac06d5e0 con 0x7f9cac06f320 2024-04-25T08:45:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.062+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/2942478771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06f320 0x7f9cac162200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c940097c0 tx=0x7f9c9401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5778b2b00c8f8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.062+0000 7f9caaa00640 1 -- 172.21.15.46:0/2942478771 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940b2070 con 0x7f9cac06f320 2024-04-25T08:45:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.062+0000 7f9caaa00640 1 -- 172.21.15.46:0/2942478771 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940ad640 con 0x7f9cac06f320 2024-04-25T08:45:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.062+0000 7f9caaa00640 1 -- 172.21.15.46:0/2942478771 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940ada10 con 0x7f9cac06f320 2024-04-25T08:45:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 -- 172.21.15.46:0/2942478771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06f320 msgr2=0x7f9cac162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 --2- 172.21.15.46:0/2942478771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06f320 0x7f9cac162200 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c940097c0 tx=0x7f9c9401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 -- 172.21.15.46:0/2942478771 shutdown_connections 2024-04-25T08:45:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 --2- 172.21.15.46:0/2942478771 >> v2:172.21.15.82:3300/0 conn(0x7f9cac0aed10 0x7f9cac164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 --2- 172.21.15.46:0/2942478771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac06f320 0x7f9cac162200 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 -- 172.21.15.46:0/2942478771 >> 172.21.15.46:0/2942478771 conn(0x7f9cac0b12a0 msgr2=0x7f9cac0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 -- 172.21.15.46:0/2942478771 shutdown_connections 2024-04-25T08:45:23.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 -- 172.21.15.46:0/2942478771 wait complete. 2024-04-25T08:45:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 Processor -- start 2024-04-25T08:45:23.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 -- start start 2024-04-25T08:45:23.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac06f320 0x7f9cac153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cac0aed10 0x7f9cac14d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac15a750 0x7f9cac152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac06d270 con 0x7f9cac0aed10 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0cf200 con 0x7f9cac06f320 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cb2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac06db00 con 0x7f9cac15a750 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cac0aed10 0x7f9cac14d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cac0aed10 0x7f9cac14d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49636/0 (socket says 172.21.15.46:49636) 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9ca3e00640 1 -- 172.21.15.46:0/956379945 learned_addr learned my addr 172.21.15.46:0/956379945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cabe00640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac15a750 0x7f9cac152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9cab400640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac06f320 0x7f9cac153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9ca3e00640 1 -- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac15a750 msgr2=0x7f9cac152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac15a750 0x7f9cac152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9ca3e00640 1 -- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac06f320 msgr2=0x7f9cac153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac06f320 0x7f9cac153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.066+0000 7f9ca3e00640 1 -- 172.21.15.46:0/956379945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac152be0 con 0x7f9cac0aed10 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cac0aed10 0x7f9cac14d360 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9c9401a0d0 tx=0x7f9c94019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9ca3400640 1 -- 172.21.15.46:0/956379945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940b2040 con 0x7f9cac0aed10 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9ca3400640 1 -- 172.21.15.46:0/956379945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940bb070 con 0x7f9cac0aed10 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9cb2000640 1 -- 172.21.15.46:0/956379945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac152ec0 con 0x7f9cac0aed10 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9ca3400640 1 -- 172.21.15.46:0/956379945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940b6650 con 0x7f9cac0aed10 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9cb2000640 1 -- 172.21.15.46:0/956379945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac153480 con 0x7f9cac0aed10 2024-04-25T08:45:23.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9ca3400640 1 -- 172.21.15.46:0/956379945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c940b67f0 con 0x7f9cac0aed10 2024-04-25T08:45:23.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9ca3400640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9c88066f90 0x7f9c88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9ca3400640 1 -- 172.21.15.46:0/956379945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c9411eff0 con 0x7f9cac0aed10 2024-04-25T08:45:23.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9cab400640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9c88066f90 0x7f9c88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.070+0000 7f9cb54ba000 1 Processor -- start 2024-04-25T08:45:23.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- start start 2024-04-25T08:45:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cab400640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9c88066f90 0x7f9c88069450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c000f70 tx=0x7f9c9c007e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512a920 0x56473512ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x5647350e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5647350e0f30 0x5647350e1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564734fc85f0 con 0x56473512b240 2024-04-25T08:45:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564734f2fda0 con 0x56473512a920 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564734eb8150 con 0x5647350e0f30 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x5647350e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9ca3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x5647350e09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49652/0 (socket says 172.21.15.46:49652) 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9ca3e00640 1 -- 172.21.15.46:0/2306098115 learned_addr learned my addr 172.21.15.46:0/2306098115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cab400640 1 --2- 172.21.15.46:0/2306098115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512a920 0x56473512ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cabe00640 1 --2- 172.21.15.46:0/2306098115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5647350e0f30 0x5647350e1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9ca3e00640 1 -- 172.21.15.46:0/2306098115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5647350e0f30 msgr2=0x5647350e1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/2306098115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5647350e0f30 0x5647350e1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9ca3e00640 1 -- 172.21.15.46:0/2306098115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512a920 msgr2=0x56473512ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/2306098115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512a920 0x56473512ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9ca3e00640 1 -- 172.21.15.46:0/2306098115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564734ed1ca0 con 0x56473512b240 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/2306098115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x5647350e09f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9c940ef850 tx=0x7f9c940ef220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04ab628f0d308ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9c9be00640 1 -- 172.21.15.46:0/2306098115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940f7f20 con 0x56473512b240 2024-04-25T08:45:23.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9c9be00640 1 -- 172.21.15.46:0/2306098115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940edbc0 con 0x56473512b240 2024-04-25T08:45:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9c9be00640 1 -- 172.21.15.46:0/2306098115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940f7a90 con 0x56473512b240 2024-04-25T08:45:23.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- 172.21.15.46:0/2306098115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 msgr2=0x5647350e09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/2306098115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x5647350e09f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9c940ef850 tx=0x7f9c940ef220 comp rx=0 tx=0).stop 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- 172.21.15.46:0/2306098115 shutdown_connections 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/2306098115 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5647350e0f30 0x5647350e1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/2306098115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x5647350e09f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/2306098115 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512a920 0x56473512ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- 172.21.15.46:0/2306098115 >> 172.21.15.46:0/2306098115 conn(0x5647350e72a0 msgr2=0x5647350df2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- 172.21.15.46:0/2306098115 shutdown_connections 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- 172.21.15.46:0/2306098115 wait complete. 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 -- start start 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.074+0000 7f9cb54ba000 1 Processor -- start 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735131d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x5647351322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 0x5647350e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564734eea990 con 0x5647350e0f30 2024-04-25T08:45:23.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564734f2be40 con 0x56473512b240 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564734ebb900 con 0x56473512a920 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 10 client.0 _ll_register_callbacks cb 0x564734fe31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735131d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735131d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49668/0 (socket says 172.21.15.46:49668) 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cab400640 1 -- 172.21.15.46:0/4162794751 learned_addr learned my addr 172.21.15.46:0/4162794751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x5647351322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cabe00640 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 0x5647350e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cab400640 1 -- 172.21.15.46:0/4162794751 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 msgr2=0x5647351322a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cab400640 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x5647351322a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cab400640 1 -- 172.21.15.46:0/4162794751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 msgr2=0x5647350e4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cab400640 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 0x5647350e4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cab400640 1 -- 172.21.15.46:0/4162794751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564734ee8630 con 0x5647350e0f30 2024-04-25T08:45:23.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x5647351322a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cab400640 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735131d60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c001ea0 tx=0x7f9c9c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab6cca2a38fa1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9c98c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9c98c00640 1 -- 172.21.15.46:0/4162794751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c027060 con 0x5647350e0f30 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9c98c00640 1 -- 172.21.15.46:0/4162794751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c9c013e00 con 0x5647350e0f30 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 -- 172.21.15.46:0/4162794751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c940edbc0 con 0x5647350e0f30 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9c98c00640 1 -- 172.21.15.46:0/4162794751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c9c012560 con 0x5647350e0f30 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 10 client.6360 fetch_fsmap learned FSMap version 34 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 -- 172.21.15.46:0/4162794751 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x564734eb83a0 con 0x5647350e0f30 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9c98c00640 1 -- 172.21.15.46:0/4162794751 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940edbc0 con 0x5647350e0f30 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9c98c00640 1 -- 172.21.15.46:0/4162794751 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9c9c012700 con 0x5647350e0f30 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 10 client.6360 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 10 client.6360 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 20 client.6360 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5647350e6160 0x56473513bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 4 client.6360 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:45:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9cb54ba000 1 -- 172.21.15.46:0/4162794751 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564734ecba50 con 0x5647350e6160 2024-04-25T08:45:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5647350e6160 0x56473513bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5647350e6160 0x56473513bdc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831547f447fafa88 server_cookie=71fc35170213dad0 in_seq=0 out_seq=0 2024-04-25T08:45:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.078+0000 7f9c98c00640 10 client.6360 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9c98c00640 1 -- 172.21.15.46:0/4162794751 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564734ecba50 con 0x5647350e6160 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9c98c00640 10 client.6360 handle_command_reply: tid=0 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 client.6360 shutdown 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 -- 172.21.15.46:0/4162794751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 msgr2=0x564735131d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735131d60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c001ea0 tx=0x7f9c9c000bd0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 -- 172.21.15.46:0/4162794751 shutdown_connections 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5647350e6160 0x56473513bdc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 0x5647350e4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x5647351322a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9c98c00640 0 client.6360 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/4162794751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735131d60 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 -- 172.21.15.46:0/4162794751 >> 172.21.15.46:0/4162794751 conn(0x5647350e72a0 msgr2=0x5647350df9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 -- 172.21.15.46:0/4162794751 shutdown_connections 2024-04-25T08:45:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 1 -- 172.21.15.46:0/4162794751 wait complete. 2024-04-25T08:45:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.082+0000 7f9cb54ba000 20 client.6360 trim_cache size 0 max 16384 2024-04-25T08:45:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 Processor -- start 2024-04-25T08:45:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 -- start start 2024-04-25T08:45:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647350e0f30 0x5647350e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x5647350e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x564735133990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac202780 con 0x56473512b240 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac14f1a0 con 0x5647350e0f30 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac202350 con 0x56473512a920 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x564735133990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x564735133990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49684/0 (socket says 172.21.15.46:49684) 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cabe00640 1 -- 172.21.15.46:0/3708850911 learned_addr learned my addr 172.21.15.46:0/3708850911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cab400640 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647350e0f30 0x5647350e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x5647350e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cabe00640 1 -- 172.21.15.46:0/3708850911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 msgr2=0x5647350e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cabe00640 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x5647350e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cabe00640 1 -- 172.21.15.46:0/3708850911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647350e0f30 msgr2=0x5647350e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cabe00640 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647350e0f30 0x5647350e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cabe00640 1 -- 172.21.15.46:0/3708850911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56473512b070 con 0x56473512b240 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cab400640 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647350e0f30 0x5647350e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cabe00640 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x564735133990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4003eb0 tx=0x7f9ca4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76920bd3024625dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9c9b400640 1 -- 172.21.15.46:0/3708850911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4004870 con 0x56473512b240 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9c9b400640 1 -- 172.21.15.46:0/3708850911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca401b070 con 0x56473512b240 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9c9b400640 1 -- 172.21.15.46:0/3708850911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4020870 con 0x56473512b240 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 -- 172.21.15.46:0/3708850911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 msgr2=0x564735133990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x564735133990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4003eb0 tx=0x7f9ca4009750 comp rx=0 tx=0).stop 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 -- 172.21.15.46:0/3708850911 shutdown_connections 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56473512b240 0x564735133990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x5647350e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/3708850911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5647350e0f30 0x5647350e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 -- 172.21.15.46:0/3708850911 >> 172.21.15.46:0/3708850911 conn(0x5647350e72a0 msgr2=0x5647350e3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 -- 172.21.15.46:0/3708850911 shutdown_connections 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.098+0000 7f9cb54ba000 1 -- 172.21.15.46:0/3708850911 wait complete. 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 -- start start 2024-04-25T08:45:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 Processor -- start 2024-04-25T08:45:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735136f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x564735171950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 0x564735171e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564734f343d0 con 0x5647350e0f30 2024-04-25T08:45:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564734febd50 con 0x56473512b240 2024-04-25T08:45:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564734eca980 con 0x56473512a920 2024-04-25T08:45:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 10 client.0 _ll_register_callbacks cb 0x7f9c94009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735136f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735136f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49696/0 (socket says 172.21.15.46:49696) 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cab400640 1 -- 172.21.15.46:0/527030450 learned_addr learned my addr 172.21.15.46:0/527030450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x564735171950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cabe00640 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 0x564735171e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cab400640 1 -- 172.21.15.46:0/527030450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 msgr2=0x564735171950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cab400640 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x564735171950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cab400640 1 -- 172.21.15.46:0/527030450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 msgr2=0x564735171e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cab400640 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 0x564735171e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cab400640 1 -- 172.21.15.46:0/527030450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564734fb1600 con 0x5647350e0f30 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cab400640 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735136f50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c004ee0 tx=0x7f9c9c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535006a8888b8243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9c98c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9c98c00640 1 -- 172.21.15.46:0/527030450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c004af0 con 0x5647350e0f30 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9c98c00640 1 -- 172.21.15.46:0/527030450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c9c010050 con 0x5647350e0f30 2024-04-25T08:45:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 -- 172.21.15.46:0/527030450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ca401b070 con 0x5647350e0f30 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9c98c00640 1 -- 172.21.15.46:0/527030450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c9c044050 con 0x5647350e0f30 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 10 client.6366 fetch_fsmap learned FSMap version 34 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 -- 172.21.15.46:0/527030450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x564734ec49b0 con 0x5647350e0f30 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9c98c00640 1 -- 172.21.15.46:0/527030450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca401b070 con 0x5647350e0f30 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9c98c00640 1 -- 172.21.15.46:0/527030450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9c9c047020 con 0x5647350e0f30 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 10 client.6366 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 10 client.6366 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 20 client.6366 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x564735137500 0x564735145840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 4 client.6366 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9cb54ba000 1 -- 172.21.15.46:0/527030450 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564734f3d2e0 con 0x564735137500 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.102+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x564735137500 0x564735145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9c98c00640 10 client.6366 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9ca3e00640 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x564735137500 0x564735145840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c21d68352c1260 server_cookie=18a3f39f050f23e3 in_seq=0 out_seq=0 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9c98c00640 1 -- 172.21.15.46:0/527030450 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x564734f3d2e0 con 0x564735137500 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9c98c00640 10 client.6366 handle_command_reply: tid=0 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 client.6366 shutdown 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 -- 172.21.15.46:0/527030450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 msgr2=0x564735136f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735136f50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c004ee0 tx=0x7f9c9c014000 comp rx=0 tx=0).stop 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 -- 172.21.15.46:0/527030450 shutdown_connections 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x564735137500 0x564735145840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56473512b240 0x564735171e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56473512a920 0x564735171950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9c98c00640 0 client.6366 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 --2- 172.21.15.46:0/527030450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5647350e0f30 0x564735136f50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 -- 172.21.15.46:0/527030450 >> 172.21.15.46:0/527030450 conn(0x5647350e72a0 msgr2=0x5647350e42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 -- 172.21.15.46:0/527030450 shutdown_connections 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 1 -- 172.21.15.46:0/527030450 wait complete. 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb54ba000 20 client.6366 trim_cache size 0 max 16384 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6207, 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.496836095999999, 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 34, 2024-04-25T08:45:23.104 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 65, 2024-04-25T08:45:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 65, 2024-04-25T08:45:23.105 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.363497422 2024-04-25T08:45:23.105 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:45:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 -- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9c88066f90 msgr2=0x7f9c88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9c88066f90 0x7f9c88069450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c000f70 tx=0x7f9c9c007e90 comp rx=0 tx=0).stop 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 -- 172.21.15.46:0/956379945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cac0aed10 msgr2=0x7f9cac14d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cac0aed10 0x7f9cac14d360 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9c9401a0d0 tx=0x7f9c94019240 comp rx=0 tx=0).stop 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 -- 172.21.15.46:0/956379945 shutdown_connections 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9c88066f90 0x7f9c88069450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9cac15a750 0x7f9cac152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cac0aed10 0x7f9cac14d360 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 --2- 172.21.15.46:0/956379945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cac06f320 0x7f9cac153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 -- 172.21.15.46:0/956379945 >> 172.21.15.46:0/956379945 conn(0x7f9cac0b12a0 msgr2=0x7f9cac0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 -- 172.21.15.46:0/956379945 shutdown_connections 2024-04-25T08:45:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.106+0000 7f9cb2000640 1 -- 172.21.15.46:0/956379945 wait complete. 2024-04-25T08:45:23.119 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:45:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.266+0000 7f4d47e00640 1 Processor -- start 2024-04-25T08:45:23.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.266+0000 7f4d47e00640 1 -- start start 2024-04-25T08:45:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.266+0000 7f4d47e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4d400b3550 0x7f4d4006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.266+0000 7f4d47e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4015e180 con 0x7f4d400af3f0 2024-04-25T08:45:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.266+0000 7f4d47e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d4015e460 con 0x7f4d4006db40 2024-04-25T08:45:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.266+0000 7f4d47e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d400b3ca0 con 0x7f4d400b3550 2024-04-25T08:45:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.266+0000 7f4d45400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4d400af3f0 0x7f4d400b2e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49120/0 (socket says 172.21.15.46:49120) 2024-04-25T08:45:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.266+0000 7f4d45400640 1 -- 172.21.15.46:0/3177891172 learned_addr learned my addr 172.21.15.46:0/3177891172 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.266+0000 7f4d45e00640 1 --2- 172.21.15.46:0/3177891172 >> v2:172.21.15.46:3300/0 conn(0x7f4d400b3550 0x7f4d4006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d3fe00640 1 -- 172.21.15.46:0/3177891172 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395625995 0 0) 0x7f4d4015e180 con 0x7f4d400af3f0 2024-04-25T08:45:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d3fe00640 1 -- 172.21.15.46:0/3177891172 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d28003530 con 0x7f4d400af3f0 2024-04-25T08:45:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d45e00640 1 -- 172.21.15.46:0/3177891172 >> v1:172.21.15.82:6789/0 conn(0x7f4d4006db40 legacy=0x7f4d4006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d45e00640 1 -- 172.21.15.46:0/3177891172 >> v1:172.21.15.46:6789/0 conn(0x7f4d400af3f0 legacy=0x7f4d400b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d45e00640 1 -- 172.21.15.46:0/3177891172 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4015e740 con 0x7f4d400b3550 2024-04-25T08:45:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d45e00640 1 --2- 172.21.15.46:0/3177891172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 0x7f4d4006b320 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009d90 tx=0x7f4d3801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed5a7f87eba8edd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d3fe00640 1 -- 172.21.15.46:0/3177891172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380b2070 con 0x7f4d400b3550 2024-04-25T08:45:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d3fe00640 1 -- 172.21.15.46:0/3177891172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380a57a0 con 0x7f4d400b3550 2024-04-25T08:45:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d3fe00640 1 -- 172.21.15.46:0/3177891172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380a5b70 con 0x7f4d400b3550 2024-04-25T08:45:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d47e00640 1 -- 172.21.15.46:0/3177891172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 msgr2=0x7f4d4006b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d47e00640 1 --2- 172.21.15.46:0/3177891172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 0x7f4d4006b320 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009d90 tx=0x7f4d3801c8e0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d47e00640 1 -- 172.21.15.46:0/3177891172 shutdown_connections 2024-04-25T08:45:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d47e00640 1 --2- 172.21.15.46:0/3177891172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 0x7f4d4006b320 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d47e00640 1 -- 172.21.15.46:0/3177891172 >> 172.21.15.46:0/3177891172 conn(0x7f4d400cfd90 msgr2=0x7f4d400d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d47e00640 1 -- 172.21.15.46:0/3177891172 shutdown_connections 2024-04-25T08:45:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.270+0000 7f4d47e00640 1 -- 172.21.15.46:0/3177891172 wait complete. 2024-04-25T08:45:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d47e00640 1 Processor -- start 2024-04-25T08:45:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d47e00640 1 -- start start 2024-04-25T08:45:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4006db40 0x7f4d4014d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d400af3f0 0x7f4d400ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 0x7f4d400cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d47e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4006bdf0 con 0x7f4d400b3550 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d47e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4006bc70 con 0x7f4d4006db40 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d47e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4016bf10 con 0x7f4d400af3f0 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 0x7f4d400cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 0x7f4d400cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49716/0 (socket says 172.21.15.46:49716) 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45e00640 1 -- 172.21.15.46:0/3453482725 learned_addr learned my addr 172.21.15.46:0/3453482725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d44a00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d400af3f0 0x7f4d400ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45400640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4006db40 0x7f4d4014d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45e00640 1 -- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d400af3f0 msgr2=0x7f4d400ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45e00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d400af3f0 0x7f4d400ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45e00640 1 -- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4006db40 msgr2=0x7f4d4014d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45e00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4006db40 0x7f4d4014d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45e00640 1 -- 172.21.15.46:0/3453482725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d400684b0 con 0x7f4d400b3550 2024-04-25T08:45:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d45e00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 0x7f4d400cf090 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4d380b0040 tx=0x7f4d38019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d3d600640 1 -- 172.21.15.46:0/3453482725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380b2050 con 0x7f4d400b3550 2024-04-25T08:45:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d3d600640 1 -- 172.21.15.46:0/3453482725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380be070 con 0x7f4d400b3550 2024-04-25T08:45:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d400cdaa0 con 0x7f4d400b3550 2024-04-25T08:45:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.274+0000 7f4d3d600640 1 -- 172.21.15.46:0/3453482725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380ad890 con 0x7f4d400b3550 2024-04-25T08:45:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.278+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d400ce060 con 0x7f4d400b3550 2024-04-25T08:45:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.278+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d40069080 con 0x7f4d400b3550 2024-04-25T08:45:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.278+0000 7f4d3d600640 1 -- 172.21.15.46:0/3453482725 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d380ada30 con 0x7f4d400b3550 2024-04-25T08:45:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.278+0000 7f4d3d600640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d20066f90 0x7f4d20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.278+0000 7f4d3d600640 1 -- 172.21.15.46:0/3453482725 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d3811fff0 con 0x7f4d400b3550 2024-04-25T08:45:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.278+0000 7f4d45400640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d20066f90 0x7f4d20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.278+0000 7f4d45400640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d20066f90 0x7f4d20069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d340011f0 tx=0x7f4d34007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.282+0000 7f4d3d600640 1 -- 172.21.15.46:0/3453482725 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d38016560 con 0x7f4d400b3550 2024-04-25T08:45:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d40154290 con 0x7f4d400b3550 2024-04-25T08:45:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d3d600640 1 -- 172.21.15.46:0/3453482725 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d380f0930 con 0x7f4d400b3550 2024-04-25T08:45:23.476 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:45:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d20066f90 msgr2=0x7f4d20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d20066f90 0x7f4d20069450 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4d340011f0 tx=0x7f4d34007d40 comp rx=0 tx=0).stop 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 msgr2=0x7f4d400cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 0x7f4d400cf090 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4d380b0040 tx=0x7f4d38019680 comp rx=0 tx=0).stop 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 shutdown_connections 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4d20066f90 0x7f4d20069450 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d400b3550 0x7f4d400cf090 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4d400af3f0 0x7f4d400ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 --2- 172.21.15.46:0/3453482725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4006db40 0x7f4d4014d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.482+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 >> 172.21.15.46:0/3453482725 conn(0x7f4d400cfd90 msgr2=0x7f4d4006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.486+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 shutdown_connections 2024-04-25T08:45:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:23.486+0000 7f4d47e00640 1 -- 172.21.15.46:0/3453482725 wait complete. 2024-04-25T08:45:23.497 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:45:23.497 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:45:23.497 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:45:23.497 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:45:23.497 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:45:23.497 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:45:23.497 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:45:23.497 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:45:23.497 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:45:23.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:23.498 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:45:23.510 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:45:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:45:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:45:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:45:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:45:23.510 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42279sec preferred_lft 42279sec 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:45:23.511 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:45:23.512 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:45:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:45:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:45:23.513 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:45:23.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:23.514 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:45:23.574 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:45:23.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:23.577 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:45:23.626 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:45:23.626 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:45:23.626 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:45:23.626 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:45:23.626 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:45:23.626 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:45:23.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:23.626 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:45:23.626 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:45:23.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:45:23.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:45:23.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:45:23.627 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:45:23.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:23.718 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:45:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:45:23.726 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:45:23.727 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:45:23.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:23.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:45:23.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:45:23.793 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:45:23.849 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:45:23.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:45:23.849 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:45:23.900 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:45:23.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:23.900 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:45:23.990 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:45:23.990 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:45:23.990 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:45:24.006 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:45:24.006 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:45:24.006 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:45:24.006 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:45:24.006 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:45:24.015 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:45:24.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:45:24.015 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:45:24.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:45:24.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:45:24.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:45:24.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:45:24.015 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:45:24.019 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:45:24.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:24.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:45:24.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:24.039 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:45:24.101 INFO:teuthology.orchestra.run.smithi046.stdout:client6169 2024-04-25T08:45:24.101 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:45:24.101 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:45:24.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:24.153 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-25T08:45:24.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:24.216 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-25T08:45:24.268 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:45:24.268 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:45:24.268 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:45:24.268 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:45:24.268 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-25T08:45:24.268 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:45:24.375 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-04-25T08:45:24.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:45:24.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:45:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.570+0000 7fb80ca00640 1 Processor -- start 2024-04-25T08:45:24.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.570+0000 7fb80ca00640 1 -- start start 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb80ca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb80815a750 0x7fb80806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb80ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8080aed10 0x7fb80806f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb80ca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80806d5e0 con 0x7fb8080cc9e0 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb80ca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80815ab30 con 0x7fb8080aed10 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb80ca00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8080ce4f0 con 0x7fb80815a750 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb7fd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8080aed10 0x7fb80806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb7fd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8080aed10 0x7fb80806f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49718/0 (socket says 172.21.15.46:49718) 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb7fd400640 1 -- 172.21.15.46:0/1181686342 learned_addr learned my addr 172.21.15.46:0/1181686342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb806800640 1 --1- 172.21.15.46:0/1181686342 >> v1:172.21.15.46:6789/0 conn(0x7fb8080cc9e0 0x7fb8080b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49134/0 (socket says 172.21.15.46:49134) 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb805e00640 1 --2- 172.21.15.46:0/1181686342 >> v2:172.21.15.82:3300/0 conn(0x7fb80815a750 0x7fb80806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb7fd400640 1 -- 172.21.15.46:0/1181686342 >> v2:172.21.15.82:3300/0 conn(0x7fb80815a750 msgr2=0x7fb80806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb7fd400640 1 --2- 172.21.15.46:0/1181686342 >> v2:172.21.15.82:3300/0 conn(0x7fb80815a750 0x7fb80806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb7fd400640 1 -- 172.21.15.46:0/1181686342 >> v1:172.21.15.46:6789/0 conn(0x7fb8080cc9e0 legacy=0x7fb8080b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb7fd400640 1 -- 172.21.15.46:0/1181686342 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080ce720 con 0x7fb8080aed10 2024-04-25T08:45:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.574+0000 7fb805e00640 1 --2- 172.21.15.46:0/1181686342 >> v2:172.21.15.82:3300/0 conn(0x7fb80815a750 0x7fb80806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb7fd400640 1 --2- 172.21.15.46:0/1181686342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 0x7fb80806f150 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb80804ffd0 tx=0x7fb7f001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5381db9f96ed76dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb805400640 1 -- 172.21.15.46:0/1181686342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00b4070 con 0x7fb8080aed10 2024-04-25T08:45:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb805400640 1 -- 172.21.15.46:0/1181686342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f00a7bb0 con 0x7fb8080aed10 2024-04-25T08:45:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb805400640 1 -- 172.21.15.46:0/1181686342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00af440 con 0x7fb8080aed10 2024-04-25T08:45:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 -- 172.21.15.46:0/1181686342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 msgr2=0x7fb80806f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 --2- 172.21.15.46:0/1181686342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 0x7fb80806f150 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb80804ffd0 tx=0x7fb7f001ebf0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 -- 172.21.15.46:0/1181686342 shutdown_connections 2024-04-25T08:45:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 --2- 172.21.15.46:0/1181686342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 0x7fb80806f150 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 --2- 172.21.15.46:0/1181686342 >> v2:172.21.15.82:3300/0 conn(0x7fb80815a750 0x7fb80806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 -- 172.21.15.46:0/1181686342 >> 172.21.15.46:0/1181686342 conn(0x7fb8080b12a0 msgr2=0x7fb8080b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 -- 172.21.15.46:0/1181686342 shutdown_connections 2024-04-25T08:45:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 -- 172.21.15.46:0/1181686342 wait complete. 2024-04-25T08:45:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 Processor -- start 2024-04-25T08:45:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 -- start start 2024-04-25T08:45:24.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 0x7fb808155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8080cc9e0 0x7fb8080acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb80815a750 0x7fb80813ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806db00 con 0x7fb8080aed10 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806d270 con 0x7fb80815a750 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.578+0000 7fb80ca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806dc80 con 0x7fb8080cc9e0 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb805e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 0x7fb808155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb805e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 0x7fb808155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49728/0 (socket says 172.21.15.46:49728) 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb805e00640 1 -- 172.21.15.46:0/4061198616 learned_addr learned my addr 172.21.15.46:0/4061198616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb806800640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb80815a750 0x7fb80813ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb7fd400640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8080cc9e0 0x7fb8080acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb805e00640 1 -- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8080cc9e0 msgr2=0x7fb8080acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb805e00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8080cc9e0 0x7fb8080acf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb805e00640 1 -- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb80815a750 msgr2=0x7fb80813ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb805e00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb80815a750 0x7fb80813ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb805e00640 1 -- 172.21.15.46:0/4061198616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8081fa540 con 0x7fb8080aed10 2024-04-25T08:45:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb805e00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 0x7fb808155f40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800d830 tx=0x7fb7f800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb7fea00640 1 -- 172.21.15.46:0/4061198616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8018020 con 0x7fb8080aed10 2024-04-25T08:45:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb7fea00640 1 -- 172.21.15.46:0/4061198616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f8009660 con 0x7fb8080aed10 2024-04-25T08:45:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb7fea00640 1 -- 172.21.15.46:0/4061198616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8009a50 con 0x7fb8080aed10 2024-04-25T08:45:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8081fa820 con 0x7fb8080aed10 2024-04-25T08:45:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8081fade0 con 0x7fb8080aed10 2024-04-25T08:45:24.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb808069080 con 0x7fb8080aed10 2024-04-25T08:45:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.582+0000 7fb7fea00640 1 -- 172.21.15.46:0/4061198616 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7f801b020 con 0x7fb8080aed10 2024-04-25T08:45:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.586+0000 7fb7fea00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e4066f60 0x7fb7e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.586+0000 7fb7fea00640 1 -- 172.21.15.46:0/4061198616 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb7f80859a0 con 0x7fb8080aed10 2024-04-25T08:45:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.586+0000 7fb7fd400640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e4066f60 0x7fb7e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.586+0000 7fb7fd400640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e4066f60 0x7fb7e4069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00b1000 tx=0x7fb7f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.590+0000 7fb7fea00640 1 -- 172.21.15.46:0/4061198616 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7f8022020 con 0x7fb8080aed10 2024-04-25T08:45:24.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb808156810 con 0x7fb8080aed10 2024-04-25T08:45:24.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb7fea00640 1 -- 172.21.15.46:0/4061198616 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fb7f8011340 con 0x7fb8080aed10 2024-04-25T08:45:24.779 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:24.779 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:45:11.989847+0000","modified":"2024-04-25T08:45:15.010275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6207},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6207":{"gid":6207,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1670658295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670658295},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670658295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:45:24.780 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-04-25T08:45:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e4066f60 msgr2=0x7fb7e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e4066f60 0x7fb7e4069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00b1000 tx=0x7fb7f0008040 comp rx=0 tx=0).stop 2024-04-25T08:45:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 msgr2=0x7fb808155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 0x7fb808155f40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb7f800d830 tx=0x7fb7f800dcf0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 shutdown_connections 2024-04-25T08:45:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7e4066f60 0x7fb7e4069420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb80815a750 0x7fb80813ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb8080cc9e0 0x7fb8080acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 --2- 172.21.15.46:0/4061198616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8080aed10 0x7fb808155f40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 >> 172.21.15.46:0/4061198616 conn(0x7fb8080b12a0 msgr2=0x7fb8080af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 shutdown_connections 2024-04-25T08:45:24.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:24.786+0000 7fb80ca00640 1 -- 172.21.15.46:0/4061198616 wait complete. 2024-04-25T08:45:24.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:45:24.971 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-25T08:45:24.971 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:45:24.972 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T08:45:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 Processor -- start 2024-04-25T08:45:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 -- start start 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 0x5640935ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 0x5640935c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640935c6270 con 0x564093655730 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640935249f0 con 0x564093580a00 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640935c5fd0 con 0x564093655350 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa769000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa769000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49738/0 (socket says 172.21.15.46:49738) 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa769000640 1 -- 172.21.15.46:0/181064651 learned_addr learned my addr 172.21.15.46:0/181064651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa768600640 1 --2- 172.21.15.46:0/181064651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 0x5640935ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa767c00640 1 --2- 172.21.15.46:0/181064651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 0x5640935c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa769000640 1 -- 172.21.15.46:0/181064651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 msgr2=0x5640935c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa769000640 1 --2- 172.21.15.46:0/181064651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 0x5640935c2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa769000640 1 -- 172.21.15.46:0/181064651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 msgr2=0x5640935ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa769000640 1 --2- 172.21.15.46:0/181064651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 0x5640935ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa769000640 1 -- 172.21.15.46:0/181064651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564093657a80 con 0x564093655730 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa769000640 1 --2- 172.21.15.46:0/181064651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c2f40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa76001b580 tx=0x7fa7600033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169d57ea8d78c222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa766800640 1 -- 172.21.15.46:0/181064651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7600a8a70 con 0x564093655730 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa766800640 1 -- 172.21.15.46:0/181064651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7600a8c10 con 0x564093655730 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa766800640 1 -- 172.21.15.46:0/181064651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7600a9440 con 0x564093655730 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 -- 172.21.15.46:0/181064651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 msgr2=0x5640935c2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 --2- 172.21.15.46:0/181064651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c2f40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa76001b580 tx=0x7fa7600033b0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 -- 172.21.15.46:0/181064651 shutdown_connections 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 --2- 172.21.15.46:0/181064651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c2f40 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 --2- 172.21.15.46:0/181064651 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 0x5640935c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 --2- 172.21.15.46:0/181064651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 0x5640935ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 -- 172.21.15.46:0/181064651 >> 172.21.15.46:0/181064651 conn(0x56409357e230 msgr2=0x5640935c5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 -- 172.21.15.46:0/181064651 shutdown_connections 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.026+0000 7fa76b2e2080 1 -- 172.21.15.46:0/181064651 wait complete. 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 Processor -- start 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 -- start start 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 0x564093656b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 0x5640935a7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564093657d20 con 0x564093655730 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564093657fc0 con 0x564093580a00 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564093652a80 con 0x564093655350 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa769000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa769000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49752/0 (socket says 172.21.15.46:49752) 2024-04-25T08:45:25.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa769000640 1 -- 172.21.15.46:0/812847771 learned_addr learned my addr 172.21.15.46:0/812847771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa767c00640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 0x5640935a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa768600640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 0x564093656b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa769000640 1 -- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 msgr2=0x5640935a7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa769000640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 0x5640935a7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa769000640 1 -- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 msgr2=0x564093656b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa769000640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 0x564093656b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa769000640 1 -- 172.21.15.46:0/812847771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564093657220 con 0x564093655730 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa767c00640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564093655350 0x5640935a7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa768600640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564093580a00 0x564093656b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa769000640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c5390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa76001d990 tx=0x7fa76001a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa765400640 1 -- 172.21.15.46:0/812847771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7600a8a70 con 0x564093655730 2024-04-25T08:45:25.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa765400640 1 -- 172.21.15.46:0/812847771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7600a9a80 con 0x564093655730 2024-04-25T08:45:25.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa765400640 1 -- 172.21.15.46:0/812847771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7600a9c30 con 0x564093655730 2024-04-25T08:45:25.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5640936574c0 con 0x564093655730 2024-04-25T08:45:25.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.030+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564093651110 con 0x564093655730 2024-04-25T08:45:25.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa765400640 1 -- 172.21.15.46:0/812847771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7600bc410 con 0x564093655730 2024-04-25T08:45:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa765400640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa748065940 0x7fa748067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa765400640 1 -- 172.21.15.46:0/812847771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa7601229e0 con 0x564093655730 2024-04-25T08:45:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5640935c58d0 0x5640936515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65) v9 -- 0x564093651af0 con 0x5640935c58d0 2024-04-25T08:45:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa768600640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa748065940 0x7fa748067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa767c00640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5640935c58d0 0x5640936515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa768600640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa748065940 0x7fa748067e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa758005fd0 tx=0x7fa75800d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa767c00640 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5640935c58d0 0x5640936515b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa767c00640 1 -- 172.21.15.46:0/812847771 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7fa75400b740 con 0x5640935c58d0 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5640935c58d0 msgr2=0x5640936515b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5640935c58d0 0x5640936515b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa748065940 msgr2=0x7fa748067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa748065940 0x7fa748067e00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa758005fd0 tx=0x7fa75800d040 comp rx=0 tx=0).stop 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 msgr2=0x5640935c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 --2- 172.21.15.46:0/812847771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564093655730 0x5640935c5390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa76001d990 tx=0x7fa76001a6f0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa769000640 1 -- 172.21.15.46:0/812847771 reap_dead start 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 shutdown_connections 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 >> 172.21.15.46:0/812847771 conn(0x56409357e230 msgr2=0x5640935c29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 shutdown_connections 2024-04-25T08:45:25.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.034+0000 7fa76b2e2080 1 -- 172.21.15.46:0/812847771 wait complete. 2024-04-25T08:45:25.031 INFO:teuthology.orchestra.run.smithi046.stdout:Xfiledir_test 2024-04-25T08:45:25.033 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:25.034 DEBUG:teuthology.orchestra.run.smithi046:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T08:45:25.081 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:45:25.081 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: "ancestors": [ 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1099511627776, 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "file", 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 2 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1, 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "dir_test", 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 4 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 10, 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout: "old_pools": [] 2024-04-25T08:45:25.082 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:45:25.083 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:25.086 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-25T08:45:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.182+0000 7f33f4a4a080 1 Processor -- start 2024-04-25T08:45:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.182+0000 7f33f4a4a080 1 -- start start 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314becf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a314b83f20 0x55a314c10a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a314bf2560 0x55a314c10fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a314bf3530 con 0x55a314b3c4c0 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a314ae09f0 con 0x55a314b83f20 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a314bf37d0 con 0x55a314bf2560 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314becf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314becf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49766/0 (socket says 172.21.15.46:49766) 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1000640 1 -- 172.21.15.46:0/1611510634 learned_addr learned my addr 172.21.15.46:0/1611510634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1a00640 1 --2- 172.21.15.46:0/1611510634 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a314bf2560 0x55a314c10fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33ebe00640 1 --2- 172.21.15.46:0/1611510634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a314b83f20 0x55a314c10a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1000640 1 -- 172.21.15.46:0/1611510634 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a314bf2560 msgr2=0x55a314c10fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1000640 1 --2- 172.21.15.46:0/1611510634 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a314bf2560 0x55a314c10fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1000640 1 -- 172.21.15.46:0/1611510634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a314b83f20 msgr2=0x55a314c10a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1000640 1 --2- 172.21.15.46:0/1611510634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a314b83f20 0x55a314c10a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1000640 1 -- 172.21.15.46:0/1611510634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a314b3c080 con 0x55a314b3c4c0 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f1000640 1 --2- 172.21.15.46:0/1611510634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314becf50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f33e400b220 tx=0x7f33e400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbac2c5dda7cc176 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33eaa00640 1 -- 172.21.15.46:0/1611510634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e400a040 con 0x55a314b3c4c0 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33eaa00640 1 -- 172.21.15.46:0/1611510634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e400be50 con 0x55a314b3c4c0 2024-04-25T08:45:25.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33eaa00640 1 -- 172.21.15.46:0/1611510634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e400f860 con 0x55a314b3c4c0 2024-04-25T08:45:25.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 -- 172.21.15.46:0/1611510634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 msgr2=0x55a314becf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 --2- 172.21.15.46:0/1611510634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314becf50 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f33e400b220 tx=0x7f33e400b6e0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 -- 172.21.15.46:0/1611510634 shutdown_connections 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 --2- 172.21.15.46:0/1611510634 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a314bf2560 0x55a314c10fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 --2- 172.21.15.46:0/1611510634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a314b83f20 0x55a314c10a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 --2- 172.21.15.46:0/1611510634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314becf50 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 -- 172.21.15.46:0/1611510634 >> 172.21.15.46:0/1611510634 conn(0x55a314b3a230 msgr2=0x55a314b83290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 -- 172.21.15.46:0/1611510634 shutdown_connections 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 -- 172.21.15.46:0/1611510634 wait complete. 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 Processor -- start 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.186+0000 7f33f4a4a080 1 -- start start 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f4a4a080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314b835c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f4a4a080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a314b83f20 0x55a314bef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f4a4a080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a314bf2560 0x55a314c097a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f4a4a080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a314c0ef00 con 0x55a314b3c4c0 2024-04-25T08:45:25.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f4a4a080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a314c0ec60 con 0x55a314b83f20 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f4a4a080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a314c0f1a0 con 0x55a314bf2560 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314b835c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314b835c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49776/0 (socket says 172.21.15.46:49776) 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f1000640 1 -- 172.21.15.46:0/2819821328 learned_addr learned my addr 172.21.15.46:0/2819821328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f1000640 1 -- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a314bf2560 msgr2=0x55a314c097a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f1000640 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a314bf2560 0x55a314c097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f1000640 1 -- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a314b83f20 msgr2=0x55a314bef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f1000640 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a314b83f20 0x55a314bef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f1000640 1 -- 172.21.15.46:0/2819821328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a314befad0 con 0x55a314b3c4c0 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f1000640 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314b835c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f33e400ff10 tx=0x7f33e4000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33e9600640 1 -- 172.21.15.46:0/2819821328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e401c650 con 0x55a314b3c4c0 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33e9600640 1 -- 172.21.15.46:0/2819821328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e401c810 con 0x55a314b3c4c0 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33e9600640 1 -- 172.21.15.46:0/2819821328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e401cc00 con 0x55a314b3c4c0 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a314befd70 con 0x55a314b3c4c0 2024-04-25T08:45:25.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a314bf02b0 con 0x55a314b3c4c0 2024-04-25T08:45:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33e9600640 1 -- 172.21.15.46:0/2819821328 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33e4017040 con 0x55a314b3c4c0 2024-04-25T08:45:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.190+0000 7f33e9600640 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33d0065970 0x7f33d0067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33e9600640 1 -- 172.21.15.46:0/2819821328 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f33e4085d70 con 0x55a314b3c4c0 2024-04-25T08:45:25.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55a314c09ce0 0x55a314cb6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65) v9 -- 0x55a314cb6ec0 con 0x55a314c09ce0 2024-04-25T08:45:25.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33ebe00640 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33d0065970 0x7f33d0067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f1a00640 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55a314c09ce0 0x55a314cb6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33ebe00640 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33d0065970 0x7f33d0067e30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009850 tx=0x7f33dc00e040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f1a00640 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55a314c09ce0 0x55a314cb6980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f1a00640 1 -- 172.21.15.46:0/2819821328 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v65'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f33ec0aa960 con 0x55a314c09ce0 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55a314c09ce0 msgr2=0x55a314cb6980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55a314c09ce0 0x55a314cb6980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33d0065970 msgr2=0x7f33d0067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33d0065970 0x7f33d0067e30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009850 tx=0x7f33dc00e040 comp rx=0 tx=0).stop 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 msgr2=0x55a314b835c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 --2- 172.21.15.46:0/2819821328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a314b3c4c0 0x55a314b835c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f33e400ff10 tx=0x7f33e4000bd0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f1a00640 1 -- 172.21.15.46:0/2819821328 reap_dead start 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 shutdown_connections 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 >> 172.21.15.46:0/2819821328 conn(0x55a314b3a230 msgr2=0x55a314becf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 shutdown_connections 2024-04-25T08:45:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.194+0000 7f33f4a4a080 1 -- 172.21.15.46:0/2819821328 wait complete. 2024-04-25T08:45:25.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:25.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-25T08:45:25.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.390+0000 7f8398e00640 1 Processor -- start 2024-04-25T08:45:25.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.390+0000 7f8398e00640 1 -- start start 2024-04-25T08:45:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8398e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83940d11e0 0x7f839406c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8398e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8394162540 con 0x7f8394099e80 2024-04-25T08:45:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8398e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839415e910 con 0x7f8394099a00 2024-04-25T08:45:25.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8398e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83940712b0 con 0x7f83940d11e0 2024-04-25T08:45:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8392800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8394099e80 0x7f8394161e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49148/0 (socket says 172.21.15.46:49148) 2024-04-25T08:45:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8392800640 1 -- 172.21.15.46:0/3264158612 learned_addr learned my addr 172.21.15.46:0/3264158612 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8391e00640 1 --2- 172.21.15.46:0/3264158612 >> v2:172.21.15.82:3300/0 conn(0x7f83940d11e0 0x7f839406c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699064787 0 0) 0x7f8394162540 con 0x7f8394099e80 2024-04-25T08:45:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8378003530 con 0x7f8394099e80 2024-04-25T08:45:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3365846209 0 0) 0x7f8378003530 con 0x7f8394099e80 2024-04-25T08:45:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 >> v2:172.21.15.82:3300/0 conn(0x7f83940d11e0 msgr2=0x7f839406c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 --2- 172.21.15.46:0/3264158612 >> v2:172.21.15.82:3300/0 conn(0x7f83940d11e0 0x7f839406c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 >> v1:172.21.15.82:6789/0 conn(0x7f8394099a00 legacy=0x7f8394161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839415ebf0 con 0x7f8394099e80 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f838c001a00 con 0x7f8394099e80 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 >> v1:172.21.15.46:6789/0 conn(0x7f8394099e80 legacy=0x7f8394161e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8378003a10 0x7f8378003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8378004810 0x7f8378004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83780051d0 0x7f837800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837800c0a0 con 0x7f8378004810 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837800c250 con 0x7f83780051d0 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837800c450 con 0x7f8378003a10 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8391400640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8378004810 0x7f8378004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8391e00640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8378003a10 0x7f8378003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8392800640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83780051d0 0x7f837800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8391400640 1 -- 172.21.15.46:0/3264158612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8378003a10 msgr2=0x7f8378003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8391400640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8378003a10 0x7f8378003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8391400640 1 -- 172.21.15.46:0/3264158612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83780051d0 msgr2=0x7f837800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8391400640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83780051d0 0x7f837800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8391400640 1 -- 172.21.15.46:0/3264158612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8378003530 con 0x7f8378004810 2024-04-25T08:45:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8392800640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83780051d0 0x7f837800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.394+0000 7f8392800640 1 -- 172.21.15.46:0/3264158612 reap_dead start 2024-04-25T08:45:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8391400640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8378004810 0x7f8378004c90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8380001780 tx=0x7f838001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffde0787940c9de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83800013a0 con 0x7f8378004810 2024-04-25T08:45:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83800052f0 con 0x7f8378004810 2024-04-25T08:45:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8390a00640 1 -- 172.21.15.46:0/3264158612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380003840 con 0x7f8378004810 2024-04-25T08:45:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8398e00640 1 -- 172.21.15.46:0/3264158612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8378004810 msgr2=0x7f8378004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8398e00640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8378004810 0x7f8378004c90 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8380001780 tx=0x7f838001c400 comp rx=0 tx=0).stop 2024-04-25T08:45:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8398e00640 1 -- 172.21.15.46:0/3264158612 shutdown_connections 2024-04-25T08:45:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8398e00640 1 --2- 172.21.15.46:0/3264158612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8378004810 0x7f8378004c90 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8398e00640 1 -- 172.21.15.46:0/3264158612 >> 172.21.15.46:0/3264158612 conn(0x7f83940d16c0 msgr2=0x7f83940d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8398e00640 1 -- 172.21.15.46:0/3264158612 shutdown_connections 2024-04-25T08:45:25.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8398e00640 1 -- 172.21.15.46:0/3264158612 wait complete. 2024-04-25T08:45:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8398e00640 1 Processor -- start 2024-04-25T08:45:25.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.398+0000 7f8398e00640 1 -- start start 2024-04-25T08:45:25.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8398e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8378004810 0x7f839415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8398e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83940c51f0 0x7f83940c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8398e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83940c5bb0 0x7f83941e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8398e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839409a2f0 con 0x7f83940c5bb0 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8398e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839406fe10 con 0x7f8378004810 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8398e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839406fa50 con 0x7f83940c51f0 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8392800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83940c5bb0 0x7f83941e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8391e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8378004810 0x7f839415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8392800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83940c5bb0 0x7f83941e8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49790/0 (socket says 172.21.15.46:49790) 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8392800640 1 -- 172.21.15.46:0/427074798 learned_addr learned my addr 172.21.15.46:0/427074798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8391400640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83940c51f0 0x7f83940c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8392800640 1 -- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83940c51f0 msgr2=0x7f83940c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8392800640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83940c51f0 0x7f83940c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8392800640 1 -- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8378004810 msgr2=0x7f839415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8392800640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8378004810 0x7f839415dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8392800640 1 -- 172.21.15.46:0/427074798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839415e000 con 0x7f83940c5bb0 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8392800640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83940c5bb0 0x7f83941e8b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f839406dce0 tx=0x7f838c0118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f838a000640 1 -- 172.21.15.46:0/427074798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c01c070 con 0x7f83940c5bb0 2024-04-25T08:45:25.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f838a000640 1 -- 172.21.15.46:0/427074798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c0137a0 con 0x7f83940c5bb0 2024-04-25T08:45:25.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8398e00640 1 -- 172.21.15.46:0/427074798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83941e7ab0 con 0x7f83940c5bb0 2024-04-25T08:45:25.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f838a000640 1 -- 172.21.15.46:0/427074798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c010040 con 0x7f83940c5bb0 2024-04-25T08:45:25.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.402+0000 7f8398e00640 1 -- 172.21.15.46:0/427074798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83941e8070 con 0x7f83940c5bb0 2024-04-25T08:45:25.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f838a000640 1 -- 172.21.15.46:0/427074798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f838c01f070 con 0x7f83940c5bb0 2024-04-25T08:45:25.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f838a000640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8374066f60 0x7f8374069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f839c398000 1 Processor -- start 2024-04-25T08:45:25.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f838a000640 1 -- 172.21.15.46:0/427074798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f838c087ad0 con 0x7f83940c5bb0 2024-04-25T08:45:25.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f839c398000 1 -- start start 2024-04-25T08:45:25.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391e00640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8374066f60 0x7f8374069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f839c398000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfe1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f839c398000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f839c398000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7bfc62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f839c398000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7bde4fd0 con 0x556c7bfe1ad0 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f839c398000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7bd6d170 con 0x556c7bfc5f10 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f839c398000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c7bde1070 con 0x556c7bfe11b0 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391e00640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8374066f60 0x7f8374069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8384009c60 tx=0x7f8384009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfe1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49792/0 (socket says 172.21.15.46:49792) 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391400640 1 -- 172.21.15.46:0/3688061059 learned_addr learned my addr 172.21.15.46:0/3688061059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfe1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:44306/0 (socket says 172.21.15.46:44306) 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8392800640 1 --2- 172.21.15.46:0/3688061059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7bfc62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391400640 1 -- 172.21.15.46:0/3688061059 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 msgr2=0x556c7bfe1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391400640 1 --2- 172.21.15.46:0/3688061059 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfe1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391400640 1 -- 172.21.15.46:0/3688061059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 msgr2=0x556c7bfc62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391400640 1 --2- 172.21.15.46:0/3688061059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7bfc62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.406+0000 7f8391400640 1 -- 172.21.15.46:0/3688061059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c7be2bd20 con 0x556c7bfe1ad0 2024-04-25T08:45:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8391400640 1 --2- 172.21.15.46:0/3688061059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc59d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8380001780 tx=0x7f8380001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f97f01b22088f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f837ea00640 1 -- 172.21.15.46:0/3688061059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380005490 con 0x556c7bfe1ad0 2024-04-25T08:45:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f837ea00640 1 -- 172.21.15.46:0/3688061059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8380003600 con 0x556c7bfe1ad0 2024-04-25T08:45:25.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f837ea00640 1 -- 172.21.15.46:0/3688061059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83800039b0 con 0x556c7bfe1ad0 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 -- 172.21.15.46:0/3688061059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 msgr2=0x556c7bfc59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 --2- 172.21.15.46:0/3688061059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc59d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8380001780 tx=0x7f8380001130 comp rx=0 tx=0).stop 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 -- 172.21.15.46:0/3688061059 shutdown_connections 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 --2- 172.21.15.46:0/3688061059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7bfc62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 --2- 172.21.15.46:0/3688061059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc59d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 --2- 172.21.15.46:0/3688061059 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfe1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 -- 172.21.15.46:0/3688061059 >> 172.21.15.46:0/3688061059 conn(0x556c7bf9d710 msgr2=0x556c7bf9daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 -- 172.21.15.46:0/3688061059 shutdown_connections 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 -- 172.21.15.46:0/3688061059 wait complete. 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 -- start start 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 Processor -- start 2024-04-25T08:45:25.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfc5f10 0x556c7bfe8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfe11b0 0x556c7bfe8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7bea2c10 con 0x556c7bfe1ad0 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7bd9f680 con 0x556c7bfe11b0 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c7bde9600 con 0x556c7bfc5f10 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f839c398000 10 client.0 _ll_register_callbacks cb 0x556c7be9a080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8392800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8391e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfc5f10 0x556c7bfe8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8391400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfe11b0 0x556c7bfe8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8391e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfc5f10 0x556c7bfe8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:44310/0 (socket says 172.21.15.46:44310) 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8391e00640 1 -- 172.21.15.46:0/1489036359 learned_addr learned my addr 172.21.15.46:0/1489036359 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8392800640 1 -- 172.21.15.46:0/1489036359 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfc5f10 msgr2=0x556c7bfe8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8392800640 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfc5f10 0x556c7bfe8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8392800640 1 -- 172.21.15.46:0/1489036359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfe11b0 msgr2=0x556c7bfe8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8392800640 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfe11b0 0x556c7bfe8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8392800640 1 -- 172.21.15.46:0/1489036359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c7bdbcd30 con 0x556c7bfe1ad0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8391e00640 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfc5f10 0x556c7bfe8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8391400640 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfe11b0 0x556c7bfe8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8392800640 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc62f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f838c085e60 tx=0x7f838c05c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e38b1e41f73de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.410+0000 7f8373400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8373400640 1 -- 172.21.15.46:0/1489036359 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c084060 con 0x556c7bfe1ad0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8373400640 1 -- 172.21.15.46:0/1489036359 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f838c057940 con 0x556c7bfe1ad0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 -- 172.21.15.46:0/1489036359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8380003600 con 0x556c7bfe1ad0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8373400640 1 -- 172.21.15.46:0/1489036359 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f838c074050 con 0x556c7bfe1ad0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 10 client.6414 fetch_fsmap learned FSMap version 34 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 -- 172.21.15.46:0/1489036359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x556c7bd6d3c0 con 0x556c7bfe1ad0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8373400640 1 -- 172.21.15.46:0/1489036359 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8380003600 con 0x556c7bfe1ad0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8373400640 1 -- 172.21.15.46:0/1489036359 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f838c05e320 con 0x556c7bfe1ad0 2024-04-25T08:45:25.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 10 client.6414 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:45:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 10 client.6414 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 20 client.6414 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x556c7bfc7060 0x556c7bfc7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 4 client.6414 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:45:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 -- 172.21.15.46:0/1489036359 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c7bd80e00 con 0x556c7bfc7060 2024-04-25T08:45:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8391e00640 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x556c7bfc7060 0x556c7bfc7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8391e00640 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x556c7bfc7060 0x556c7bfc7440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e97f14498647b2b server_cookie=b04e19ecedda878b in_seq=0 out_seq=0 2024-04-25T08:45:25.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8373400640 10 client.6414 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8373400640 1 -- 172.21.15.46:0/1489036359 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c7bd80e00 con 0x556c7bfc7060 2024-04-25T08:45:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8373400640 10 client.6414 handle_command_reply: tid=0 2024-04-25T08:45:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 client.6414 shutdown 2024-04-25T08:45:25.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 -- 172.21.15.46:0/1489036359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 msgr2=0x556c7bfc62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc62f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f838c085e60 tx=0x7f838c05c450 comp rx=0 tx=0).stop 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 -- 172.21.15.46:0/1489036359 shutdown_connections 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x556c7bfc7060 0x556c7bfc7440 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfc62f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfe11b0 0x556c7bfe8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 --2- 172.21.15.46:0/1489036359 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfc5f10 0x556c7bfe8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f8373400640 0 client.6414 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 -- 172.21.15.46:0/1489036359 >> 172.21.15.46:0/1489036359 conn(0x556c7bf9d710 msgr2=0x556c7bf95950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 -- 172.21.15.46:0/1489036359 shutdown_connections 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 1 -- 172.21.15.46:0/1489036359 wait complete. 2024-04-25T08:45:25.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.414+0000 7f839c398000 20 client.6414 trim_cache size 0 max 16384 2024-04-25T08:45:25.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f839c398000 1 Processor -- start 2024-04-25T08:45:25.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f839c398000 1 -- start start 2024-04-25T08:45:25.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f839c398000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7bf9cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f839c398000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfdf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f839c398000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfdfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f839c398000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83942008b0 con 0x556c7bfe1ad0 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f839c398000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83941ffa60 con 0x556c7bfc5f10 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f839c398000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83940b63d0 con 0x556c7bfe11b0 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8392800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfdfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8392800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfdfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49812/0 (socket says 172.21.15.46:49812) 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8392800640 1 -- 172.21.15.46:0/2103342076 learned_addr learned my addr 172.21.15.46:0/2103342076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8391400640 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfdf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8391e00640 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7bf9cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8392800640 1 -- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 msgr2=0x556c7bfdf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8392800640 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfdf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8392800640 1 -- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 msgr2=0x556c7bf9cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8392800640 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7bf9cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8392800640 1 -- 172.21.15.46:0/2103342076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c7bf9a460 con 0x556c7bfe1ad0 2024-04-25T08:45:25.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8391400640 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfdf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:25.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8391e00640 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7bf9cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:25.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.430+0000 7f8392800640 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfdfa40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f838c076500 tx=0x7f838c05c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5eb0d2fbdb3d59c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f837e000640 1 -- 172.21.15.46:0/2103342076 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c057740 con 0x556c7bfe1ad0 2024-04-25T08:45:25.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f837e000640 1 -- 172.21.15.46:0/2103342076 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c0578e0 con 0x556c7bfe1ad0 2024-04-25T08:45:25.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f837e000640 1 -- 172.21.15.46:0/2103342076 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c013d70 con 0x556c7bfe1ad0 2024-04-25T08:45:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- 172.21.15.46:0/2103342076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 msgr2=0x556c7bfdfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfdfa40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f838c076500 tx=0x7f838c05c050 comp rx=0 tx=0).stop 2024-04-25T08:45:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- 172.21.15.46:0/2103342076 shutdown_connections 2024-04-25T08:45:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bfdfa40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7bfdf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 --2- 172.21.15.46:0/2103342076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7bf9cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- 172.21.15.46:0/2103342076 >> 172.21.15.46:0/2103342076 conn(0x556c7bf9d710 msgr2=0x556c7bfc55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- 172.21.15.46:0/2103342076 shutdown_connections 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- 172.21.15.46:0/2103342076 wait complete. 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- start start 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 Processor -- start 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7c041a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7c041f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bf96a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7bd9df30 con 0x556c7bfe1ad0 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83941ff490 con 0x556c7bfc5f10 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c7bde42e0 con 0x556c7bfe11b0 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 10 client.0 _ll_register_callbacks cb 0x556c7bd82b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8392800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bf96a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8392800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bf96a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49826/0 (socket says 172.21.15.46:49826) 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8391e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7c041a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8392800640 1 -- 172.21.15.46:0/1574347772 learned_addr learned my addr 172.21.15.46:0/1574347772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8391400640 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7c041f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8392800640 1 -- 172.21.15.46:0/1574347772 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 msgr2=0x556c7c041f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8392800640 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7c041f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8392800640 1 -- 172.21.15.46:0/1574347772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 msgr2=0x556c7c041a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8392800640 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7c041a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8392800640 1 -- 172.21.15.46:0/1574347772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c7be670f0 con 0x556c7bfe1ad0 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8392800640 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bf96a80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f838c054750 tx=0x7f838c019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6978b8c344f7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8373400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8373400640 1 -- 172.21.15.46:0/1574347772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c065060 con 0x556c7bfe1ad0 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8373400640 1 -- 172.21.15.46:0/1574347772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f838c060da0 con 0x556c7bfe1ad0 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- 172.21.15.46:0/1574347772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f838c0578e0 con 0x556c7bfe1ad0 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f8373400640 1 -- 172.21.15.46:0/1574347772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f838c06b050 con 0x556c7bfe1ad0 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 10 client.6420 fetch_fsmap learned FSMap version 34 2024-04-25T08:45:25.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.434+0000 7f839c398000 1 -- 172.21.15.46:0/1574347772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x556c7bd79d20 con 0x556c7bfe1ad0 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8373400640 1 -- 172.21.15.46:0/1574347772 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c0578e0 con 0x556c7bfe1ad0 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8373400640 1 -- 172.21.15.46:0/1574347772 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f838c054040 con 0x556c7bfe1ad0 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 10 client.6420 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 10 client.6420 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 20 client.6420 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x556c7bf973b0 0x556c7bf977c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 4 client.6420 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 -- 172.21.15.46:0/1574347772 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x556c7bdf2510 con 0x556c7bf973b0 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8391e00640 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x556c7bf973b0 0x556c7bf977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8391e00640 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x556c7bf973b0 0x556c7bf977c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c298f8978770ba64 server_cookie=1c207a7ac010b70f in_seq=0 out_seq=0 2024-04-25T08:45:25.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8373400640 10 client.6420 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8373400640 1 -- 172.21.15.46:0/1574347772 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556c7bdf2510 con 0x556c7bf973b0 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8373400640 10 client.6420 handle_command_reply: tid=0 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 client.6420 shutdown 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 -- 172.21.15.46:0/1574347772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 msgr2=0x556c7bf96a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bf96a80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f838c054750 tx=0x7f838c019040 comp rx=0 tx=0).stop 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 -- 172.21.15.46:0/1574347772 shutdown_connections 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x556c7bf973b0 0x556c7bf977c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7bfe1ad0 0x556c7bf96a80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556c7bfe11b0 0x556c7c041f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 --2- 172.21.15.46:0/1574347772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556c7bfc5f10 0x556c7c041a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.435 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8efc400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6169=0-4194304@1} caps={6169=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6169 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55bc26ca1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:45:25.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 -- 172.21.15.46:0/1574347772 >> 172.21.15.46:0/1574347772 conn(0x556c7bf9d710 msgr2=0x556c7bf9bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8373400640 0 client.6420 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:25.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 -- 172.21.15.46:0/1574347772 shutdown_connections 2024-04-25T08:45:25.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 1 -- 172.21.15.46:0/1574347772 wait complete. 2024-04-25T08:45:25.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f839c398000 20 client.6420 trim_cache size 0 max 16384 2024-04-25T08:45:25.436 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:45:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:45:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "d360e85f-15e0-4a7d-b3c9-b7cf24058308", 2024-04-25T08:45:25.436 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:45:25.436 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:45:25.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 -- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8374066f60 msgr2=0x7f8374069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8374066f60 0x7f8374069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8384009c60 tx=0x7f8384009190 comp rx=0 tx=0).stop 2024-04-25T08:45:25.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 -- 172.21.15.46:0/427074798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83940c5bb0 msgr2=0x7f83941e8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83940c5bb0 0x7f83941e8b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f839406dce0 tx=0x7f838c0118c0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 -- 172.21.15.46:0/427074798 shutdown_connections 2024-04-25T08:45:25.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8374066f60 0x7f8374069420 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83940c5bb0 0x7f83941e8b50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83940c51f0 0x7f83940c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 --2- 172.21.15.46:0/427074798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8378004810 0x7f839415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 -- 172.21.15.46:0/427074798 >> 172.21.15.46:0/427074798 conn(0x7f83940d16c0 msgr2=0x7f83940b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 -- 172.21.15.46:0/427074798 shutdown_connections 2024-04-25T08:45:25.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.438+0000 7f8398e00640 1 -- 172.21.15.46:0/427074798 wait complete. 2024-04-25T08:45:25.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:25.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T08:45:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f97000640 1 Processor -- start 2024-04-25T08:45:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f97000640 1 -- start start 2024-04-25T08:45:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f97000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0f900d1dc0 0x7f0f900d3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f97000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f900b4c50 0x7f0f9015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f97000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f900ce860 con 0x7f0f900ce2d0 2024-04-25T08:45:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f97000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f900d17a0 con 0x7f0f900b4c50 2024-04-25T08:45:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f97000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f900ceb40 con 0x7f0f900d1dc0 2024-04-25T08:45:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f95200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f900b4c50 0x7f0f9015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f95200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f900b4c50 0x7f0f9015d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49840/0 (socket says 172.21.15.46:49840) 2024-04-25T08:45:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f95200640 1 -- 172.21.15.46:0/1148263805 learned_addr learned my addr 172.21.15.46:0/1148263805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f95c00640 1 --2- 172.21.15.46:0/1148263805 >> v2:172.21.15.82:3300/0 conn(0x7f0f900d1dc0 0x7f0f900d3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f95200640 1 -- 172.21.15.46:0/1148263805 >> v2:172.21.15.82:3300/0 conn(0x7f0f900d1dc0 msgr2=0x7f0f900d3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f95200640 1 --2- 172.21.15.46:0/1148263805 >> v2:172.21.15.82:3300/0 conn(0x7f0f900d1dc0 0x7f0f900d3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f95200640 1 -- 172.21.15.46:0/1148263805 >> v1:172.21.15.82:6789/0 conn(0x7f0f900ce2d0 legacy=0x7f0f90161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.602+0000 7f0f95200640 1 -- 172.21.15.46:0/1148263805 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f900cee20 con 0x7f0f900b4c50 2024-04-25T08:45:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f95200640 1 --2- 172.21.15.46:0/1148263805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900b4c50 0x7f0f9015d590 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0f9008ba40 tx=0x7f0f7c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10660f4cf7794f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f8fe00640 1 -- 172.21.15.46:0/1148263805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0ab430 con 0x7f0f900b4c50 2024-04-25T08:45:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f8fe00640 1 -- 172.21.15.46:0/1148263805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c0b0070 con 0x7f0f900b4c50 2024-04-25T08:45:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f8fe00640 1 -- 172.21.15.46:0/1148263805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0ab7f0 con 0x7f0f900b4c50 2024-04-25T08:45:25.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f97000640 1 -- 172.21.15.46:0/1148263805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900b4c50 msgr2=0x7f0f9015d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f97000640 1 --2- 172.21.15.46:0/1148263805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900b4c50 0x7f0f9015d590 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0f9008ba40 tx=0x7f0f7c01c520 comp rx=0 tx=0).stop 2024-04-25T08:45:25.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f97000640 1 -- 172.21.15.46:0/1148263805 shutdown_connections 2024-04-25T08:45:25.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f97000640 1 --2- 172.21.15.46:0/1148263805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900b4c50 0x7f0f9015d590 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f97000640 1 --2- 172.21.15.46:0/1148263805 >> v2:172.21.15.82:3300/0 conn(0x7f0f900d1dc0 0x7f0f900d3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f97000640 1 -- 172.21.15.46:0/1148263805 >> 172.21.15.46:0/1148263805 conn(0x7f0f9008af90 msgr2=0x7f0f900d3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f97000640 1 -- 172.21.15.46:0/1148263805 shutdown_connections 2024-04-25T08:45:25.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.606+0000 7f0f97000640 1 -- 172.21.15.46:0/1148263805 wait complete. 2024-04-25T08:45:25.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 Processor -- start 2024-04-25T08:45:25.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 -- start start 2024-04-25T08:45:25.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f900b4c50 0x7f0f901519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f900ce2d0 0x7f0f90157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900d1dc0 0x7f0f90158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9016ab10 con 0x7f0f900d1dc0 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9016a990 con 0x7f0f900b4c50 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9016ac90 con 0x7f0f900ce2d0 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f96600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900d1dc0 0x7f0f90158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f96600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900d1dc0 0x7f0f90158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49850/0 (socket says 172.21.15.46:49850) 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f96600640 1 -- 172.21.15.46:0/1783934345 learned_addr learned my addr 172.21.15.46:0/1783934345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f95c00640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f900b4c50 0x7f0f901519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f95200640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f900ce2d0 0x7f0f90157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f96600640 1 -- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f900ce2d0 msgr2=0x7f0f90157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f96600640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f900ce2d0 0x7f0f90157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f96600640 1 -- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f900b4c50 msgr2=0x7f0f901519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f96600640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f900b4c50 0x7f0f901519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f96600640 1 -- 172.21.15.46:0/1783934345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f90156b60 con 0x7f0f900d1dc0 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f96600640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900d1dc0 0x7f0f90158150 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c6c0 tx=0x7f0f8800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f8d600640 1 -- 172.21.15.46:0/1783934345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88003b80 con 0x7f0f900d1dc0 2024-04-25T08:45:25.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f8d600640 1 -- 172.21.15.46:0/1783934345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88003d20 con 0x7f0f900d1dc0 2024-04-25T08:45:25.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f8d600640 1 -- 172.21.15.46:0/1783934345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880052f0 con 0x7f0f900d1dc0 2024-04-25T08:45:25.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 -- 172.21.15.46:0/1783934345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f90156e40 con 0x7f0f900d1dc0 2024-04-25T08:45:25.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.610+0000 7f0f97000640 1 -- 172.21.15.46:0/1783934345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f90157400 con 0x7f0f900d1dc0 2024-04-25T08:45:25.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.614+0000 7f0f8d600640 1 -- 172.21.15.46:0/1783934345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f88005490 con 0x7f0f900d1dc0 2024-04-25T08:45:25.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.614+0000 7f0f8d600640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f70066f60 0x7f0f70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.614+0000 7f0f8d600640 1 -- 172.21.15.46:0/1783934345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f88084e50 con 0x7f0f900d1dc0 2024-04-25T08:45:25.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.614+0000 7f0f9ba14000 1 Processor -- start 2024-04-25T08:45:25.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.614+0000 7f0f9ba14000 1 -- start start 2024-04-25T08:45:25.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.614+0000 7f0f95c00640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f70066f60 0x7f0f70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f73d59d0 0x55a6f73d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d62f0 0x55a6f738bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f738c0a0 0x55a6f738c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6f71d8ff0 con 0x55a6f73d59d0 2024-04-25T08:45:25.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6f7161150 con 0x55a6f738c0a0 2024-04-25T08:45:25.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6f71d5090 con 0x55a6f73d62f0 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f70066f60 0x7f0f70069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0f840079c0 tx=0x7f0f84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f73d59d0 0x55a6f73d5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f73d59d0 0x55a6f73d5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49866/0 (socket says 172.21.15.46:49866) 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d62f0 0x55a6f738bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 -- 172.21.15.46:0/130423019 learned_addr learned my addr 172.21.15.46:0/130423019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f96600640 1 --2- 172.21.15.46:0/130423019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f738c0a0 0x55a6f738c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 -- 172.21.15.46:0/130423019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d62f0 msgr2=0x55a6f738bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 --2- 172.21.15.46:0/130423019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d62f0 0x55a6f738bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 -- 172.21.15.46:0/130423019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f738c0a0 msgr2=0x55a6f738c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 --2- 172.21.15.46:0/130423019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f738c0a0 0x55a6f738c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 -- 172.21.15.46:0/130423019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6f736f0d0 con 0x55a6f73d59d0 2024-04-25T08:45:25.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f95c00640 1 --2- 172.21.15.46:0/130423019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f73d59d0 0x55a6f73d5db0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0f84004660 tx=0x7f0f84013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f422e13e6b7fcaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f82000640 1 -- 172.21.15.46:0/130423019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8401c050 con 0x55a6f73d59d0 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f82000640 1 -- 172.21.15.46:0/130423019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84026070 con 0x55a6f73d59d0 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f82000640 1 -- 172.21.15.46:0/130423019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84021440 con 0x55a6f73d59d0 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 -- 172.21.15.46:0/130423019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f73d59d0 msgr2=0x55a6f73d5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/130423019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f73d59d0 0x55a6f73d5db0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0f84004660 tx=0x7f0f84013000 comp rx=0 tx=0).stop 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 -- 172.21.15.46:0/130423019 shutdown_connections 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/130423019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f738c0a0 0x55a6f738c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/130423019 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d62f0 0x55a6f738bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/130423019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f73d59d0 0x55a6f73d5db0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 -- 172.21.15.46:0/130423019 >> 172.21.15.46:0/130423019 conn(0x55a6f7388020 msgr2=0x55a6f738a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 -- 172.21.15.46:0/130423019 shutdown_connections 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 -- 172.21.15.46:0/130423019 wait complete. 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 -- start start 2024-04-25T08:45:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.618+0000 7f0f9ba14000 1 Processor -- start 2024-04-25T08:45:25.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f73bad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f73bb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6f719d250 con 0x55a6f738c0a0 2024-04-25T08:45:25.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6f7193690 con 0x55a6f73d62f0 2024-04-25T08:45:25.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6f7161a40 con 0x55a6f73d59d0 2024-04-25T08:45:25.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 10 client.0 _ll_register_callbacks cb 0x55a6f7376e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:25.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73ba7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49870/0 (socket says 172.21.15.46:49870) 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95c00640 1 -- 172.21.15.46:0/1058056705 learned_addr learned my addr 172.21.15.46:0/1058056705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95200640 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f73bad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f96600640 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f73bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95c00640 1 -- 172.21.15.46:0/1058056705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 msgr2=0x55a6f73bad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95c00640 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f73bad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95c00640 1 -- 172.21.15.46:0/1058056705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 msgr2=0x55a6f73bb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95c00640 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f73bb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95c00640 1 -- 172.21.15.46:0/1058056705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6f72861c0 con 0x55a6f738c0a0 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95c00640 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73ba7f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x55a6f71953d0 tx=0x7f0f84004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506006be5f36c79c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 1 -- 172.21.15.46:0/1058056705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84016d90 con 0x55a6f738c0a0 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 1 -- 172.21.15.46:0/1058056705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f84036c70 con 0x55a6f738c0a0 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 -- 172.21.15.46:0/1058056705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f84026070 con 0x55a6f738c0a0 2024-04-25T08:45:25.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 1 -- 172.21.15.46:0/1058056705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f8401c020 con 0x55a6f738c0a0 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 10 client.6432 fetch_fsmap learned FSMap version 34 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 -- 172.21.15.46:0/1058056705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55a6f7282170 con 0x55a6f738c0a0 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 1 -- 172.21.15.46:0/1058056705 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84026070 con 0x55a6f738c0a0 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 1 -- 172.21.15.46:0/1058056705 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0f84021ba0 con 0x55a6f738c0a0 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 10 client.6432 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 10 client.6432 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 20 client.6432 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6f73e5c80 0x55a6f73e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 4 client.6432 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 -- 172.21.15.46:0/1058056705 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6f7174c60 con 0x55a6f73e5c80 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95200640 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6f73e5c80 0x55a6f73e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f95200640 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6f73e5c80 0x55a6f73e8070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6ac6e780a6325d9 server_cookie=eac2909d6c44b98 in_seq=0 out_seq=0 2024-04-25T08:45:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 10 client.6432 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 1 -- 172.21.15.46:0/1058056705 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a6f7174c60 con 0x55a6f73e5c80 2024-04-25T08:45:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 10 client.6432 handle_command_reply: tid=0 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 client.6432 shutdown 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 -- 172.21.15.46:0/1058056705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 msgr2=0x55a6f73ba7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73ba7f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x55a6f71953d0 tx=0x7f0f84004740 comp rx=0 tx=0).stop 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 -- 172.21.15.46:0/1058056705 shutdown_connections 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6f73e5c80 0x55a6f73e8070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f73bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f73bad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f76a00640 0 client.6432 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/1058056705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73ba7f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.622+0000 7f0f9ba14000 1 -- 172.21.15.46:0/1058056705 >> 172.21.15.46:0/1058056705 conn(0x55a6f7388020 msgr2=0x55a6f738aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.626+0000 7f0f9ba14000 1 -- 172.21.15.46:0/1058056705 shutdown_connections 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.626+0000 7f0f9ba14000 1 -- 172.21.15.46:0/1058056705 wait complete. 2024-04-25T08:45:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.626+0000 7f0f9ba14000 20 client.6432 trim_cache size 0 max 16384 2024-04-25T08:45:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.638+0000 7f0f9ba14000 1 Processor -- start 2024-04-25T08:45:25.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.638+0000 7f0f9ba14000 1 -- start start 2024-04-25T08:45:25.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f74320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f7432630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f7432b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f902038b0 con 0x55a6f738c0a0 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f90202a60 con 0x55a6f73d62f0 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900cb6b0 con 0x55a6f73d59d0 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f74320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f74320f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49872/0 (socket says 172.21.15.46:49872) 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 -- 172.21.15.46:0/3772621818 learned_addr learned my addr 172.21.15.46:0/3772621818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95200640 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f7432630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f96600640 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f7432b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 -- 172.21.15.46:0/3772621818 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 msgr2=0x55a6f7432630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f7432630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 -- 172.21.15.46:0/3772621818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 msgr2=0x55a6f7432b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f7432b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 -- 172.21.15.46:0/3772621818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6f721fde0 con 0x55a6f738c0a0 2024-04-25T08:45:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f96600640 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f7432b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f74320f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0f84010a00 tx=0x7f0f8402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e182bd1717e366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f81600640 1 -- 172.21.15.46:0/3772621818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84021a20 con 0x55a6f738c0a0 2024-04-25T08:45:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f81600640 1 -- 172.21.15.46:0/3772621818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84026070 con 0x55a6f738c0a0 2024-04-25T08:45:25.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f81600640 1 -- 172.21.15.46:0/3772621818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84004040 con 0x55a6f738c0a0 2024-04-25T08:45:25.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- 172.21.15.46:0/3772621818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 msgr2=0x55a6f74320f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f74320f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0f84010a00 tx=0x7f0f8402e070 comp rx=0 tx=0).stop 2024-04-25T08:45:25.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- 172.21.15.46:0/3772621818 shutdown_connections 2024-04-25T08:45:25.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f7432b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f7432630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/3772621818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f74320f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- 172.21.15.46:0/3772621818 >> 172.21.15.46:0/3772621818 conn(0x55a6f7388020 msgr2=0x55a6f73e2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- 172.21.15.46:0/3772621818 shutdown_connections 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- 172.21.15.46:0/3772621818 wait complete. 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- start start 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 Processor -- start 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73e8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f73e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f73e8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6f7173e00 con 0x55a6f738c0a0 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900cbcb0 con 0x55a6f73d62f0 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6f71d8300 con 0x55a6f73d59d0 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f9ba14000 10 client.0 _ll_register_callbacks cb 0x55a6f7357040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73e8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49880/0 (socket says 172.21.15.46:49880) 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.642+0000 7f0f95c00640 1 -- 172.21.15.46:0/2764843792 learned_addr learned my addr 172.21.15.46:0/2764843792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f95200640 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f73e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f96600640 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f73e8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f95c00640 1 -- 172.21.15.46:0/2764843792 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 msgr2=0x55a6f73e88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f95c00640 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f73e88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f95c00640 1 -- 172.21.15.46:0/2764843792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 msgr2=0x55a6f73e8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f95c00640 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f73e8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f95c00640 1 -- 172.21.15.46:0/2764843792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6f717a8e0 con 0x55a6f738c0a0 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f96600640 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f73e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f95c00640 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73e8380 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0f84010620 tx=0x7f0f84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174e24ebb97d956 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 1 -- 172.21.15.46:0/2764843792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8404a020 con 0x55a6f738c0a0 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 1 -- 172.21.15.46:0/2764843792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f84035630 con 0x55a6f738c0a0 2024-04-25T08:45:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 -- 172.21.15.46:0/2764843792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f84026070 con 0x55a6f738c0a0 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 1 -- 172.21.15.46:0/2764843792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f84054050 con 0x55a6f738c0a0 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 10 client.6438 fetch_fsmap learned FSMap version 34 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 -- 172.21.15.46:0/2764843792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55a6f72328f0 con 0x55a6f738c0a0 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 1 -- 172.21.15.46:0/2764843792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84026070 con 0x55a6f738c0a0 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 1 -- 172.21.15.46:0/2764843792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0f84057020 con 0x55a6f738c0a0 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 10 client.6438 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 10 client.6438 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 20 client.6438 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6f73e9730 0x55a6f743df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 4 client.6438 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 -- 172.21.15.46:0/2764843792 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a6f71e6530 con 0x55a6f73e9730 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f95200640 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6f73e9730 0x55a6f743df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f95200640 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6f73e9730 0x55a6f743df60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87fc4db36ae0464 server_cookie=2d1984a0183ee7cf in_seq=0 out_seq=0 2024-04-25T08:45:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 10 client.6438 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 1 -- 172.21.15.46:0/2764843792 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x55a6f71e6530 con 0x55a6f73e9730 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 10 client.6438 handle_command_reply: tid=0 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 client.6438 shutdown 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 -- 172.21.15.46:0/2764843792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 msgr2=0x55a6f73e8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73e8380 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0f84010620 tx=0x7f0f84016000 comp rx=0 tx=0).stop 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 -- 172.21.15.46:0/2764843792 shutdown_connections 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6f73e9730 0x55a6f743df60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6f73d62f0 0x55a6f73e8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6f73d59d0 0x55a6f73e88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 --2- 172.21.15.46:0/2764843792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6f738c0a0 0x55a6f73e8380 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 -- 172.21.15.46:0/2764843792 >> 172.21.15.46:0/2764843792 conn(0x55a6f7388020 msgr2=0x55a6f73d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f76a00640 0 client.6438 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 -- 172.21.15.46:0/2764843792 shutdown_connections 2024-04-25T08:45:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 1 -- 172.21.15.46:0/2764843792 wait complete. 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f9ba14000 20 client.6438 trim_cache size 0 max 16384 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout: "d360e85f-15e0-4a7d-b3c9-b7cf24058308": { 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "d360e85f-15e0-4a7d-b3c9-b7cf24058308", 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:25.645 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:45:25.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 -- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f70066f60 msgr2=0x7f0f70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f70066f60 0x7f0f70069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0f840079c0 tx=0x7f0f84008040 comp rx=0 tx=0).stop 2024-04-25T08:45:25.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 -- 172.21.15.46:0/1783934345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900d1dc0 msgr2=0x7f0f90158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:25.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900d1dc0 0x7f0f90158150 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800c6c0 tx=0x7f0f8800cb80 comp rx=0 tx=0).stop 2024-04-25T08:45:25.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 -- 172.21.15.46:0/1783934345 shutdown_connections 2024-04-25T08:45:25.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f70066f60 0x7f0f70069420 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f900d1dc0 0x7f0f90158150 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f900ce2d0 0x7f0f90157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 --2- 172.21.15.46:0/1783934345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f900b4c50 0x7f0f901519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:25.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 -- 172.21.15.46:0/1783934345 >> 172.21.15.46:0/1783934345 conn(0x7f0f9008af90 msgr2=0x7f0f900cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:25.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 -- 172.21.15.46:0/1783934345 shutdown_connections 2024-04-25T08:45:25.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:25.646+0000 7f0f97000640 1 -- 172.21.15.46:0/1783934345 wait complete. 2024-04-25T08:45:25.660 INFO:tasks.cephfs.filesystem:scrub status for tag:d360e85f-15e0-4a7d-b3c9-b7cf24058308 - {'path': '/', 'tag': 'd360e85f-15e0-4a7d-b3c9-b7cf24058308', 'options': 'recursive'} 2024-04-25T08:45:29.092 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:45:29.098+0000 7fccb4a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:45:50.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:45:50.414 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:45:55.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:55.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T08:45:55.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.822+0000 7ff858000640 1 Processor -- start 2024-04-25T08:45:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.822+0000 7ff858000640 1 -- start start 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.822+0000 7ff858000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff850143990 0x7ff8500cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.822+0000 7ff858000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff85006f150 0x7ff8500b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.822+0000 7ff858000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85015e440 con 0x7ff8501440d0 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.822+0000 7ff858000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85015ed30 con 0x7ff850143990 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.822+0000 7ff858000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85015e6e0 con 0x7ff85006f150 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff855600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff850143990 0x7ff8500cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff854c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff8501440d0 0x7ff85006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49512/0 (socket says 172.21.15.46:49512) 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff854c00640 1 -- 172.21.15.46:0/237655358 learned_addr learned my addr 172.21.15.46:0/237655358 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff856000640 1 --2- 172.21.15.46:0/237655358 >> v2:172.21.15.82:3300/0 conn(0x7ff85006f150 0x7ff8500b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff855600640 1 -- 172.21.15.46:0/237655358 >> v2:172.21.15.82:3300/0 conn(0x7ff85006f150 msgr2=0x7ff8500b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff855600640 1 --2- 172.21.15.46:0/237655358 >> v2:172.21.15.82:3300/0 conn(0x7ff85006f150 0x7ff8500b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff855600640 1 -- 172.21.15.46:0/237655358 >> v1:172.21.15.46:6789/0 conn(0x7ff8501440d0 legacy=0x7ff85006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff855600640 1 -- 172.21.15.46:0/237655358 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85015e9c0 con 0x7ff850143990 2024-04-25T08:45:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff855600640 1 --2- 172.21.15.46:0/237655358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 0x7ff8500cecc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff8500af320 tx=0x7ff84401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6cba4566c11e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:55.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff84fe00640 1 -- 172.21.15.46:0/237655358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440b0070 con 0x7ff850143990 2024-04-25T08:45:55.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff84fe00640 1 -- 172.21.15.46:0/237655358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8440ab600 con 0x7ff850143990 2024-04-25T08:45:55.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.826+0000 7ff84fe00640 1 -- 172.21.15.46:0/237655358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440ab9d0 con 0x7ff850143990 2024-04-25T08:45:55.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 -- 172.21.15.46:0/237655358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 msgr2=0x7ff8500cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 --2- 172.21.15.46:0/237655358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 0x7ff8500cecc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff8500af320 tx=0x7ff84401c480 comp rx=0 tx=0).stop 2024-04-25T08:45:55.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 -- 172.21.15.46:0/237655358 shutdown_connections 2024-04-25T08:45:55.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 --2- 172.21.15.46:0/237655358 >> v2:172.21.15.82:3300/0 conn(0x7ff85006f150 0x7ff8500b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 --2- 172.21.15.46:0/237655358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 0x7ff8500cecc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 -- 172.21.15.46:0/237655358 >> 172.21.15.46:0/237655358 conn(0x7ff85006ac10 msgr2=0x7ff85006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:55.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 -- 172.21.15.46:0/237655358 shutdown_connections 2024-04-25T08:45:55.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 -- 172.21.15.46:0/237655358 wait complete. 2024-04-25T08:45:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 Processor -- start 2024-04-25T08:45:55.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 -- start start 2024-04-25T08:45:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff85006f150 0x7ff850159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 0x7ff85013b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8501440d0 0x7ff85015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b35f0 con 0x7ff850143990 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b3470 con 0x7ff8501440d0 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff858000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b3770 con 0x7ff85006f150 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff854c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 0x7ff85013b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff854c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 0x7ff85013b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43914/0 (socket says 172.21.15.46:43914) 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff854c00640 1 -- 172.21.15.46:0/3955139450 learned_addr learned my addr 172.21.15.46:0/3955139450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff855600640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff85006f150 0x7ff850159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff856000640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8501440d0 0x7ff85015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff854c00640 1 -- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff85006f150 msgr2=0x7ff850159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff854c00640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff85006f150 0x7ff850159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff854c00640 1 -- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8501440d0 msgr2=0x7ff85015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff854c00640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8501440d0 0x7ff85015d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff854c00640 1 -- 172.21.15.46:0/3955139450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8501338f0 con 0x7ff850143990 2024-04-25T08:45:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.830+0000 7ff855600640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff85006f150 0x7ff850159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.834+0000 7ff854c00640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 0x7ff85013b910 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c370 tx=0x7ff83c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.834+0000 7ff84d600640 1 -- 172.21.15.46:0/3955139450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c017070 con 0x7ff850143990 2024-04-25T08:45:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.834+0000 7ff84d600640 1 -- 172.21.15.46:0/3955139450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c005cf0 con 0x7ff850143990 2024-04-25T08:45:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.834+0000 7ff84d600640 1 -- 172.21.15.46:0/3955139450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c0072f0 con 0x7ff850143990 2024-04-25T08:45:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.834+0000 7ff858000640 1 -- 172.21.15.46:0/3955139450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff850132850 con 0x7ff850143990 2024-04-25T08:45:55.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.834+0000 7ff858000640 1 -- 172.21.15.46:0/3955139450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff850132e10 con 0x7ff850143990 2024-04-25T08:45:55.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.834+0000 7ff84d600640 1 -- 172.21.15.46:0/3955139450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff83c01a070 con 0x7ff850143990 2024-04-25T08:45:55.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.834+0000 7ff84d600640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff830066f60 0x7ff830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.834+0000 7ff84d600640 1 -- 172.21.15.46:0/3955139450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff83c084600 con 0x7ff850143990 2024-04-25T08:45:55.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 Processor -- start 2024-04-25T08:45:55.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff830066f60 0x7ff830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 -- start start 2024-04-25T08:45:55.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff830066f60 0x7ff830069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff8440ad000 tx=0x7ff844008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f429210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 0x56236f3def70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 0x56236f3df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f227ff0 con 0x56236f428e30 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f1b0150 con 0x56236f3df4b0 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236f224090 con 0x56236f429750 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f429210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f429210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43922/0 (socket says 172.21.15.46:43922) 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 -- 172.21.15.46:0/4239588961 learned_addr learned my addr 172.21.15.46:0/4239588961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff856000640 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 0x56236f3df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff854c00640 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 0x56236f3def70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 -- 172.21.15.46:0/4239588961 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 msgr2=0x56236f3def70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 0x56236f3def70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 -- 172.21.15.46:0/4239588961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 msgr2=0x56236f3df890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 0x56236f3df890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 -- 172.21.15.46:0/4239588961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236f3be0d0 con 0x56236f428e30 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff854c00640 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 0x56236f3def70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff855600640 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f429210 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff84401a0b0 tx=0x7ff844004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109053ffa4b458ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff842000640 1 -- 172.21.15.46:0/4239588961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440c7070 con 0x56236f428e30 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff842000640 1 -- 172.21.15.46:0/4239588961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8440c2470 con 0x56236f428e30 2024-04-25T08:45:55.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff842000640 1 -- 172.21.15.46:0/4239588961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440b0040 con 0x56236f428e30 2024-04-25T08:45:55.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 -- 172.21.15.46:0/4239588961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 msgr2=0x56236f429210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f429210 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff84401a0b0 tx=0x7ff844004740 comp rx=0 tx=0).stop 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 -- 172.21.15.46:0/4239588961 shutdown_connections 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 0x56236f3df890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 0x56236f3def70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 --2- 172.21.15.46:0/4239588961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f429210 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 -- 172.21.15.46:0/4239588961 >> 172.21.15.46:0/4239588961 conn(0x56236f3d7020 msgr2=0x56236f3d9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 -- 172.21.15.46:0/4239588961 shutdown_connections 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.838+0000 7ff85b469000 1 -- 172.21.15.46:0/4239588961 wait complete. 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 -- start start 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 Processor -- start 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f3df4b0 0x56236f4287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f428e30 0x56236f3e5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f429750 0x56236f3e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f22c620 con 0x56236f429750 2024-04-25T08:45:55.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f1e2690 con 0x56236f428e30 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236f1b0a40 con 0x56236f3df4b0 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 10 client.0 _ll_register_callbacks cb 0x56236f3c5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff856000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f429750 0x56236f3e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff856000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f429750 0x56236f3e58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43930/0 (socket says 172.21.15.46:43930) 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff856000640 1 -- 172.21.15.46:0/3401818529 learned_addr learned my addr 172.21.15.46:0/3401818529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff854c00640 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f428e30 0x56236f3e5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff855600640 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f3df4b0 0x56236f4287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff856000640 1 -- 172.21.15.46:0/3401818529 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f3df4b0 msgr2=0x56236f4287b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff856000640 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f3df4b0 0x56236f4287b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff856000640 1 -- 172.21.15.46:0/3401818529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f428e30 msgr2=0x56236f3e5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:55.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff856000640 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f428e30 0x56236f3e5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff856000640 1 -- 172.21.15.46:0/3401818529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56236f2d51c0 con 0x56236f429750 2024-04-25T08:45:55.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff856000640 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f429750 0x56236f3e58a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff848007e10 tx=0x7ff848012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfe594fd33d3553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:55.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff836a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff836a00640 1 -- 172.21.15.46:0/3401818529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84801c020 con 0x56236f429750 2024-04-25T08:45:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff836a00640 1 -- 172.21.15.46:0/3401818529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff848018670 con 0x56236f429750 2024-04-25T08:45:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 -- 172.21.15.46:0/3401818529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8440c2470 con 0x56236f429750 2024-04-25T08:45:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff836a00640 1 -- 172.21.15.46:0/3401818529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff848026050 con 0x56236f429750 2024-04-25T08:45:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 10 client.6453 fetch_fsmap learned FSMap version 35 2024-04-25T08:45:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 -- 172.21.15.46:0/3401818529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x56236f2d1170 con 0x56236f429750 2024-04-25T08:45:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff836a00640 1 -- 172.21.15.46:0/3401818529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8440c2470 con 0x56236f429750 2024-04-25T08:45:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff836a00640 1 -- 172.21.15.46:0/3401818529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff848029020 con 0x56236f429750 2024-04-25T08:45:55.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 10 client.6453 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 10 client.6453 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 20 client.6453 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56236f3e6610 0x56236f43b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 4 client.6453 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff85b469000 1 -- 172.21.15.46:0/3401818529 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56236f1c3c60 con 0x56236f3e6610 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.842+0000 7ff855600640 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56236f3e6610 0x56236f43b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff855600640 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56236f3e6610 0x56236f43b0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ea156d0aa4625e server_cookie=f0e8734d16358481 in_seq=0 out_seq=0 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff836a00640 10 client.6453 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff836a00640 1 -- 172.21.15.46:0/3401818529 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56236f1c3c60 con 0x56236f3e6610 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff836a00640 10 client.6453 handle_command_reply: tid=0 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 client.6453 shutdown 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 -- 172.21.15.46:0/3401818529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f429750 msgr2=0x56236f3e58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f429750 0x56236f3e58a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff848007e10 tx=0x7ff848012610 comp rx=0 tx=0).stop 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 -- 172.21.15.46:0/3401818529 shutdown_connections 2024-04-25T08:45:55.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56236f3e6610 0x56236f43b0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f429750 0x56236f3e58a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f428e30 0x56236f3e5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 --2- 172.21.15.46:0/3401818529 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f3df4b0 0x56236f4287b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff836a00640 0 client.6453 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 -- 172.21.15.46:0/3401818529 >> 172.21.15.46:0/3401818529 conn(0x56236f3d7020 msgr2=0x56236f3e0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 -- 172.21.15.46:0/3401818529 shutdown_connections 2024-04-25T08:45:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 1 -- 172.21.15.46:0/3401818529 wait complete. 2024-04-25T08:45:55.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.846+0000 7ff85b469000 20 client.6453 trim_cache size 0 max 16384 2024-04-25T08:45:55.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 Processor -- start 2024-04-25T08:45:55.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- start start 2024-04-25T08:45:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f3df4b0 0x56236f43d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f428e30 0x56236f407b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f429750 0x56236f408040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b13a0 con 0x56236f3df4b0 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8500b0580 con 0x56236f429750 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff850131c30 con 0x56236f428e30 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff855600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f3df4b0 0x56236f43d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff855600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f3df4b0 0x56236f43d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43946/0 (socket says 172.21.15.46:43946) 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff855600640 1 -- 172.21.15.46:0/1308325221 learned_addr learned my addr 172.21.15.46:0/1308325221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff856000640 1 --2- 172.21.15.46:0/1308325221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f429750 0x56236f408040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff854c00640 1 --2- 172.21.15.46:0/1308325221 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f428e30 0x56236f407b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff855600640 1 -- 172.21.15.46:0/1308325221 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f428e30 msgr2=0x56236f407b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff855600640 1 --2- 172.21.15.46:0/1308325221 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f428e30 0x56236f407b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff855600640 1 -- 172.21.15.46:0/1308325221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f429750 msgr2=0x56236f408040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff855600640 1 --2- 172.21.15.46:0/1308325221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f429750 0x56236f408040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff855600640 1 -- 172.21.15.46:0/1308325221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236f26ede0 con 0x56236f3df4b0 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff855600640 1 --2- 172.21.15.46:0/1308325221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f3df4b0 0x56236f43d320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff84401cd20 tx=0x7ff844009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3662ef5c5975e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff841600640 1 -- 172.21.15.46:0/1308325221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440c7040 con 0x56236f3df4b0 2024-04-25T08:45:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff841600640 1 -- 172.21.15.46:0/1308325221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8440c27a0 con 0x56236f3df4b0 2024-04-25T08:45:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff841600640 1 -- 172.21.15.46:0/1308325221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8440b0040 con 0x56236f3df4b0 2024-04-25T08:45:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- 172.21.15.46:0/1308325221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f3df4b0 msgr2=0x56236f43d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 --2- 172.21.15.46:0/1308325221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f3df4b0 0x56236f43d320 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff84401cd20 tx=0x7ff844009190 comp rx=0 tx=0).stop 2024-04-25T08:45:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- 172.21.15.46:0/1308325221 shutdown_connections 2024-04-25T08:45:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 --2- 172.21.15.46:0/1308325221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f429750 0x56236f408040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 --2- 172.21.15.46:0/1308325221 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f428e30 0x56236f407b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 --2- 172.21.15.46:0/1308325221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f3df4b0 0x56236f43d320 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- 172.21.15.46:0/1308325221 >> 172.21.15.46:0/1308325221 conn(0x56236f3d7020 msgr2=0x56236f435b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- 172.21.15.46:0/1308325221 shutdown_connections 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- 172.21.15.46:0/1308325221 wait complete. 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 -- start start 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.862+0000 7ff85b469000 1 Processor -- start 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 0x56236f43b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f43b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 0x56236f43bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236f1c2e00 con 0x56236f428e30 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff850135ee0 con 0x56236f3df4b0 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236f227300 con 0x56236f429750 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 10 client.0 _ll_register_callbacks cb 0x56236f3a6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff854c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f43b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff854c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f43b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43958/0 (socket says 172.21.15.46:43958) 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff854c00640 1 -- 172.21.15.46:0/2993565860 learned_addr learned my addr 172.21.15.46:0/2993565860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff856000640 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 0x56236f43bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff855600640 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 0x56236f43b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff854c00640 1 -- 172.21.15.46:0/2993565860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 msgr2=0x56236f43bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff854c00640 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 0x56236f43bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff854c00640 1 -- 172.21.15.46:0/2993565860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 msgr2=0x56236f43b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff854c00640 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 0x56236f43b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff854c00640 1 -- 172.21.15.46:0/2993565860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56236f1c98e0 con 0x56236f428e30 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff854c00640 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f43b630 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00cd30 tx=0x7ff83c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c490bc0ce22f085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 1 -- 172.21.15.46:0/2993565860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c007490 con 0x56236f428e30 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 1 -- 172.21.15.46:0/2993565860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff83c007630 con 0x56236f428e30 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- 172.21.15.46:0/2993565860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8440c27a0 con 0x56236f428e30 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 1 -- 172.21.15.46:0/2993565860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff83c054130 con 0x56236f428e30 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 10 client.6459 fetch_fsmap learned FSMap version 35 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- 172.21.15.46:0/2993565860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x56236f2818f0 con 0x56236f428e30 2024-04-25T08:45:55.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 1 -- 172.21.15.46:0/2993565860 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c075070 con 0x56236f428e30 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 1 -- 172.21.15.46:0/2993565860 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff83c0542d0 con 0x56236f428e30 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 10 client.6459 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 10 client.6459 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 20 client.6459 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56236f48efd0 0x56236f4913c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 4 client.6459 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- 172.21.15.46:0/2993565860 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56236f235530 con 0x56236f48efd0 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff855600640 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56236f48efd0 0x56236f4913c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff855600640 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56236f48efd0 0x56236f4913c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0a388f53e67b88 server_cookie=b886b70dc69fde84 in_seq=0 out_seq=0 2024-04-25T08:45:55.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 10 client.6459 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 1 -- 172.21.15.46:0/2993565860 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56236f235530 con 0x56236f48efd0 2024-04-25T08:45:55.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 10 client.6459 handle_command_reply: tid=0 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 client.6459 shutdown 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- 172.21.15.46:0/2993565860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 msgr2=0x56236f43b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f43b630 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00cd30 tx=0x7ff83c000ee0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- 172.21.15.46:0/2993565860 shutdown_connections 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56236f48efd0 0x56236f4913c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56236f429750 0x56236f43bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56236f428e30 0x56236f43b630 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff836a00640 0 client.6459 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 --2- 172.21.15.46:0/2993565860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56236f3df4b0 0x56236f43b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- 172.21.15.46:0/2993565860 >> 172.21.15.46:0/2993565860 conn(0x56236f3d7020 msgr2=0x56236f436420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.866+0000 7ff85b469000 1 -- 172.21.15.46:0/2993565860 shutdown_connections 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff85b469000 1 -- 172.21.15.46:0/2993565860 wait complete. 2024-04-25T08:45:55.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff85b469000 20 client.6459 trim_cache size 0 max 16384 2024-04-25T08:45:55.866 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:45:55.866 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:45:55.866 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:45:55.866 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 -- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff830066f60 msgr2=0x7ff830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff830066f60 0x7ff830069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff8440ad000 tx=0x7ff844008040 comp rx=0 tx=0).stop 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 -- 172.21.15.46:0/3955139450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 msgr2=0x7ff85013b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 0x7ff85013b910 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00c370 tx=0x7ff83c00c830 comp rx=0 tx=0).stop 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 -- 172.21.15.46:0/3955139450 shutdown_connections 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff830066f60 0x7ff830069420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8501440d0 0x7ff85015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff850143990 0x7ff85013b910 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 --2- 172.21.15.46:0/3955139450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff85006f150 0x7ff850159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 -- 172.21.15.46:0/3955139450 >> 172.21.15.46:0/3955139450 conn(0x7ff85006ac10 msgr2=0x7ff85013dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 -- 172.21.15.46:0/3955139450 shutdown_connections 2024-04-25T08:45:55.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:55.870+0000 7ff858000640 1 -- 172.21.15.46:0/3955139450 wait complete. 2024-04-25T08:45:55.881 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:45:55.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:45:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.026+0000 7ff5eec00640 1 Processor -- start 2024-04-25T08:45:56.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.026+0000 7ff5eec00640 1 -- start start 2024-04-25T08:45:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5eec00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5e80af3f0 0x7ff5e80b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5eec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e806ee30 con 0x7ff5e806f630 2024-04-25T08:45:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5eec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e815e180 con 0x7ff5e806db40 2024-04-25T08:45:56.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5eec00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e80b3ca0 con 0x7ff5e80af3f0 2024-04-25T08:45:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5ecc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff5e806f630 0x7ff5e806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49516/0 (socket says 172.21.15.46:49516) 2024-04-25T08:45:56.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5ecc00640 1 -- 172.21.15.46:0/2170554164 learned_addr learned my addr 172.21.15.46:0/2170554164 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:56.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/2170554164 >> v2:172.21.15.82:3300/0 conn(0x7ff5e80af3f0 0x7ff5e80b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564800725 0 0) 0x7ff5e806ee30 con 0x7ff5e806f630 2024-04-25T08:45:56.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc003530 con 0x7ff5e806f630 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2523374655 0 0) 0x7ff5cc003530 con 0x7ff5e806f630 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 >> v2:172.21.15.82:3300/0 conn(0x7ff5e80af3f0 msgr2=0x7ff5e80b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 --2- 172.21.15.46:0/2170554164 >> v2:172.21.15.82:3300/0 conn(0x7ff5e80af3f0 0x7ff5e80b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 >> v1:172.21.15.82:6789/0 conn(0x7ff5e806db40 legacy=0x7ff5e806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e815e460 con 0x7ff5e806f630 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7ff5e0001680 con 0x7ff5e806f630 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 >> v1:172.21.15.46:6789/0 conn(0x7ff5e806f630 legacy=0x7ff5e806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5cc0039c0 0x7ff5cc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0043c0 0x7ff5cc004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc005090 0x7ff5cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80b3ca0 con 0x7ff5cc005090 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc00bff0 con 0x7ff5cc0043c0 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc00c1f0 con 0x7ff5cc0039c0 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5ecc00640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc005090 0x7ff5cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5cc0039c0 0x7ff5cc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5df400640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0043c0 0x7ff5cc004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5ecc00640 1 -- 172.21.15.46:0/2170554164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5cc0039c0 msgr2=0x7ff5cc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5ecc00640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5cc0039c0 0x7ff5cc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5ecc00640 1 -- 172.21.15.46:0/2170554164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0043c0 msgr2=0x7ff5cc004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5ecc00640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5cc0043c0 0x7ff5cc004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.030+0000 7ff5ecc00640 1 -- 172.21.15.46:0/2170554164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc003530 con 0x7ff5cc005090 2024-04-25T08:45:56.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5ecc00640 1 -- 172.21.15.46:0/2170554164 reap_dead start 2024-04-25T08:45:56.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5ecc00640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc005090 0x7ff5cc00b9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c240 tx=0x7ff5e001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277897d51e8e8e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0007d40 con 0x7ff5cc005090 2024-04-25T08:45:56.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0005b00 con 0x7ff5cc005090 2024-04-25T08:45:56.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5e7400640 1 -- 172.21.15.46:0/2170554164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0007070 con 0x7ff5cc005090 2024-04-25T08:45:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5eec00640 1 -- 172.21.15.46:0/2170554164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc005090 msgr2=0x7ff5cc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5eec00640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc005090 0x7ff5cc00b9d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c240 tx=0x7ff5e001f3f0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5eec00640 1 -- 172.21.15.46:0/2170554164 shutdown_connections 2024-04-25T08:45:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5eec00640 1 --2- 172.21.15.46:0/2170554164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5cc005090 0x7ff5cc00b9d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5eec00640 1 -- 172.21.15.46:0/2170554164 >> 172.21.15.46:0/2170554164 conn(0x7ff5e80cfd90 msgr2=0x7ff5e80d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5eec00640 1 -- 172.21.15.46:0/2170554164 shutdown_connections 2024-04-25T08:45:56.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.034+0000 7ff5eec00640 1 -- 172.21.15.46:0/2170554164 wait complete. 2024-04-25T08:45:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 Processor -- start 2024-04-25T08:45:56.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 -- start start 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5cc005090 0x7ff5e81f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e814a680 0x7ff5e81f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e80b2e20 0x7ff5e80b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80af7d0 con 0x7ff5e814a680 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815ea20 con 0x7ff5e80b2e20 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8161560 con 0x7ff5cc005090 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e814a680 0x7ff5e81f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e814a680 0x7ff5e81f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43980/0 (socket says 172.21.15.46:43980) 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5df400640 1 -- 172.21.15.46:0/304413843 learned_addr learned my addr 172.21.15.46:0/304413843 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5cc005090 0x7ff5e81f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5ecc00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e80b2e20 0x7ff5e80b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5df400640 1 -- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5cc005090 msgr2=0x7ff5e81f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5df400640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5cc005090 0x7ff5e81f2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5df400640 1 -- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e80b2e20 msgr2=0x7ff5e80b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5df400640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e80b2e20 0x7ff5e80b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5df400640 1 -- 172.21.15.46:0/304413843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e81f77a0 con 0x7ff5e814a680 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5ecc00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e80b2e20 0x7ff5e80b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5df400640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e814a680 0x7ff5e81f7260 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000f7f0 tx=0x7ff5d000fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5e4c00640 1 -- 172.21.15.46:0/304413843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000fea0 con 0x7ff5e814a680 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5e4c00640 1 -- 172.21.15.46:0/304413843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d0018070 con 0x7ff5e814a680 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5e4c00640 1 -- 172.21.15.46:0/304413843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d000d6d0 con 0x7ff5e814a680 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e80cd550 con 0x7ff5e814a680 2024-04-25T08:45:56.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.038+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e8158f50 con 0x7ff5e814a680 2024-04-25T08:45:56.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.042+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e8069080 con 0x7ff5e814a680 2024-04-25T08:45:56.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.042+0000 7ff5e4c00640 1 -- 172.21.15.46:0/304413843 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5d000d870 con 0x7ff5e814a680 2024-04-25T08:45:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.042+0000 7ff5e4c00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5c4066f90 0x7ff5c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.042+0000 7ff5e4c00640 1 -- 172.21.15.46:0/304413843 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5d0084f00 con 0x7ff5e814a680 2024-04-25T08:45:56.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.042+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5c4066f90 0x7ff5c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.042+0000 7ff5e7e00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5c4066f90 0x7ff5c4069450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8014450 tx=0x7ff5d800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.046+0000 7ff5e4c00640 1 -- 172.21.15.46:0/304413843 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5e81f77a0 con 0x7ff5e814a680 2024-04-25T08:45:56.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.242+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5e80cda70 con 0x7ff5e814a680 2024-04-25T08:45:56.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.242+0000 7ff5e4c00640 1 -- 172.21.15.46:0/304413843 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7ff5d0055d70 con 0x7ff5e814a680 2024-04-25T08:45:56.236 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:56.237 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:45:11.989847+0000","modified":"2024-04-25T08:45:28.649888+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6207},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6207":{"gid":6207,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":7,"addr":"172.21.15.46:6835/1670658295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670658295},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670658295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:45:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 35 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5c4066f90 msgr2=0x7ff5c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5c4066f90 0x7ff5c4069450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8014450 tx=0x7ff5d800b040 comp rx=0 tx=0).stop 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e814a680 msgr2=0x7ff5e81f7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e814a680 0x7ff5e81f7260 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000f7f0 tx=0x7ff5d000fcb0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 shutdown_connections 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff5c4066f90 0x7ff5c4069450 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5e80b2e20 0x7ff5e80b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5e814a680 0x7ff5e81f7260 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 --2- 172.21.15.46:0/304413843 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff5cc005090 0x7ff5e81f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 >> 172.21.15.46:0/304413843 conn(0x7ff5e80cfd90 msgr2=0x7ff5e815f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 shutdown_connections 2024-04-25T08:45:56.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.246+0000 7ff5eec00640 1 -- 172.21.15.46:0/304413843 wait complete. 2024-04-25T08:45:56.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:45:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.406+0000 7f13fa400640 1 Processor -- start 2024-04-25T08:45:56.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.406+0000 7f13fa400640 1 -- start start 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13fa400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f13f415d900 0x7f13f40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13fa400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f406b470 0x7f13f40d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13fa400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f406fca0 con 0x7f13f40afc20 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13fa400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f40b0070 con 0x7f13f415d900 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13fa400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f406ff80 con 0x7f13f406b470 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13f3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f13f415d900 0x7f13f40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13f3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f13f415d900 0x7f13f40af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43990/0 (socket says 172.21.15.46:43990) 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13f3400640 1 -- 172.21.15.46:0/2954359429 learned_addr learned my addr 172.21.15.46:0/2954359429 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13eaa00640 1 --1- 172.21.15.46:0/2954359429 >> v1:172.21.15.46:6789/0 conn(0x7f13f40afc20 0x7f13f406ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49518/0 (socket says 172.21.15.46:49518) 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13f3e00640 1 --2- 172.21.15.46:0/2954359429 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f406b470 0x7f13f40d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13f3400640 1 -- 172.21.15.46:0/2954359429 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f406b470 msgr2=0x7f13f40d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13f3400640 1 --2- 172.21.15.46:0/2954359429 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f406b470 0x7f13f40d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13f3400640 1 -- 172.21.15.46:0/2954359429 >> v1:172.21.15.46:6789/0 conn(0x7f13f40afc20 legacy=0x7f13f406ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13f3400640 1 -- 172.21.15.46:0/2954359429 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f4070260 con 0x7f13f415d900 2024-04-25T08:45:56.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.410+0000 7f13f3e00640 1 --2- 172.21.15.46:0/2954359429 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f406b470 0x7f13f40d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13f3400640 1 --2- 172.21.15.46:0/2954359429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f415d900 0x7f13f40af6e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009e50 tx=0x7f13e401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d224744b660fa85e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13f2a00640 1 -- 172.21.15.46:0/2954359429 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40a5750 con 0x7f13f415d900 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13f2a00640 1 -- 172.21.15.46:0/2954359429 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e40a58f0 con 0x7f13f415d900 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13f2a00640 1 -- 172.21.15.46:0/2954359429 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40a5cc0 con 0x7f13f415d900 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 -- 172.21.15.46:0/2954359429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f415d900 msgr2=0x7f13f40af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 --2- 172.21.15.46:0/2954359429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f415d900 0x7f13f40af6e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009e50 tx=0x7f13e401c9a0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 -- 172.21.15.46:0/2954359429 shutdown_connections 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 --2- 172.21.15.46:0/2954359429 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f406b470 0x7f13f40d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 --2- 172.21.15.46:0/2954359429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f415d900 0x7f13f40af6e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 -- 172.21.15.46:0/2954359429 >> 172.21.15.46:0/2954359429 conn(0x7f13f406e3e0 msgr2=0x7f13f406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 -- 172.21.15.46:0/2954359429 shutdown_connections 2024-04-25T08:45:56.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 -- 172.21.15.46:0/2954359429 wait complete. 2024-04-25T08:45:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 Processor -- start 2024-04-25T08:45:56.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 -- start start 2024-04-25T08:45:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f406b470 0x7f13f4142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f40afc20 0x7f13f40c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f415d900 0x7f13f415fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40d10e0 con 0x7f13f406b470 2024-04-25T08:45:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40d0f60 con 0x7f13f40afc20 2024-04-25T08:45:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.414+0000 7f13fa400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40d1260 con 0x7f13f415d900 2024-04-25T08:45:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f406b470 0x7f13f4142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f406b470 0x7f13f4142090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44002/0 (socket says 172.21.15.46:44002) 2024-04-25T08:45:56.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3400640 1 -- 172.21.15.46:0/3689954039 learned_addr learned my addr 172.21.15.46:0/3689954039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13eaa00640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f40afc20 0x7f13f40c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3e00640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f415d900 0x7f13f415fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3400640 1 -- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f415d900 msgr2=0x7f13f415fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3400640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f415d900 0x7f13f415fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3400640 1 -- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f40afc20 msgr2=0x7f13f40c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3400640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f40afc20 0x7f13f40c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3400640 1 -- 172.21.15.46:0/3689954039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40ca390 con 0x7f13f406b470 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13eaa00640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f40afc20 0x7f13f40c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13f3400640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f406b470 0x7f13f4142090 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f13e40af040 tx=0x7f13e4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13ebe00640 1 -- 172.21.15.46:0/3689954039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40bc020 con 0x7f13f406b470 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13ebe00640 1 -- 172.21.15.46:0/3689954039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e40b88c0 con 0x7f13f406b470 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13ebe00640 1 -- 172.21.15.46:0/3689954039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40b8cb0 con 0x7f13f406b470 2024-04-25T08:45:56.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13fa400640 1 -- 172.21.15.46:0/3689954039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f40ca670 con 0x7f13f406b470 2024-04-25T08:45:56.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13fa400640 1 -- 172.21.15.46:0/3689954039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f40cac30 con 0x7f13f406b470 2024-04-25T08:45:56.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13ebe00640 1 -- 172.21.15.46:0/3689954039 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13e40bf020 con 0x7f13f406b470 2024-04-25T08:45:56.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.418+0000 7f13ebe00640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f13d0066f90 0x7f13d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13ebe00640 1 -- 172.21.15.46:0/3689954039 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13e41203d0 con 0x7f13f406b470 2024-04-25T08:45:56.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13eaa00640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f13d0066f90 0x7f13d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 Processor -- start 2024-04-25T08:45:56.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- start start 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13eaa00640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f13d0066f90 0x7f13d0069450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f13f41f9f40 tx=0x7f13dc007d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e413a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e40f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e3fd7ba0 con 0x5646e413a2e0 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e3f3d090 con 0x5646e413ac00 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646e3ec5160 con 0x5646e40f08e0 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e413a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e413a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44006/0 (socket says 172.21.15.46:44006) 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13f3400640 1 -- 172.21.15.46:0/350900900 learned_addr learned my addr 172.21.15.46:0/350900900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13eaa00640 1 --2- 172.21.15.46:0/350900900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13f3400640 1 -- 172.21.15.46:0/350900900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 msgr2=0x5646e40f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13f3400640 1 --2- 172.21.15.46:0/350900900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e40f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13f3400640 1 -- 172.21.15.46:0/350900900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 msgr2=0x5646e40f03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13f3400640 1 --2- 172.21.15.46:0/350900900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13f3400640 1 -- 172.21.15.46:0/350900900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646e3fe6c20 con 0x5646e413a2e0 2024-04-25T08:45:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13eaa00640 1 --2- 172.21.15.46:0/350900900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13f3400640 1 --2- 172.21.15.46:0/350900900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e413a6c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f13e40f0b00 tx=0x7f13e4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fac24719e7c6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13e3e00640 1 -- 172.21.15.46:0/350900900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40f1270 con 0x5646e413a2e0 2024-04-25T08:45:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13e3e00640 1 -- 172.21.15.46:0/350900900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e40f0b70 con 0x5646e413a2e0 2024-04-25T08:45:56.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13e3e00640 1 -- 172.21.15.46:0/350900900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40f4cc0 con 0x5646e413a2e0 2024-04-25T08:45:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- 172.21.15.46:0/350900900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 msgr2=0x5646e413a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 --2- 172.21.15.46:0/350900900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e413a6c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f13e40f0b00 tx=0x7f13e4009810 comp rx=0 tx=0).stop 2024-04-25T08:45:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- 172.21.15.46:0/350900900 shutdown_connections 2024-04-25T08:45:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 --2- 172.21.15.46:0/350900900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e40f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 --2- 172.21.15.46:0/350900900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 --2- 172.21.15.46:0/350900900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e413a6c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- 172.21.15.46:0/350900900 >> 172.21.15.46:0/350900900 conn(0x5646e40ec890 msgr2=0x5646e40eec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- 172.21.15.46:0/350900900 shutdown_connections 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- 172.21.15.46:0/350900900 wait complete. 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.422+0000 7f13fd94d000 1 -- start start 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 Processor -- start 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e40fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413a2e0 0x5646e40fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413ac00 0x5646e40fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e3f39130 con 0x5646e413ac00 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e3f01130 con 0x5646e413a2e0 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646e3ffb300 con 0x5646e40f08e0 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 10 client.0 _ll_register_callbacks cb 0x5646e40dbbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413ac00 0x5646e40fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413ac00 0x5646e40fbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44014/0 (socket says 172.21.15.46:44014) 2024-04-25T08:45:56.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3e00640 1 -- 172.21.15.46:0/1771675145 learned_addr learned my addr 172.21.15.46:0/1771675145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13eaa00640 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413a2e0 0x5646e40fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3400640 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e40fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3e00640 1 -- 172.21.15.46:0/1771675145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 msgr2=0x5646e40fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3e00640 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e40fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3e00640 1 -- 172.21.15.46:0/1771675145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413a2e0 msgr2=0x5646e40fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3e00640 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413a2e0 0x5646e40fb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3e00640 1 -- 172.21.15.46:0/1771675145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646e3f1a060 con 0x5646e413ac00 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3e00640 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413ac00 0x5646e40fbf00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00e280 tx=0x7f13ec00e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3af0ffce5cc902b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13e0c00640 1 -- 172.21.15.46:0/1771675145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec018020 con 0x5646e413ac00 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13e0c00640 1 -- 172.21.15.46:0/1771675145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13ec013660 con 0x5646e413ac00 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 -- 172.21.15.46:0/1771675145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13e40f0b70 con 0x5646e413ac00 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13e0c00640 1 -- 172.21.15.46:0/1771675145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13ec022050 con 0x5646e413ac00 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 10 client.6480 fetch_fsmap learned FSMap version 35 2024-04-25T08:45:56.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 -- 172.21.15.46:0/1771675145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5646e3ede7c0 con 0x5646e413ac00 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13e0c00640 1 -- 172.21.15.46:0/1771675145 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e40f0b70 con 0x5646e413ac00 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13e0c00640 1 -- 172.21.15.46:0/1771675145 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f13ec025020 con 0x5646e413ac00 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 10 client.6480 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 10 client.6480 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 20 client.6480 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5646e40fc7d0 0x5646e414b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 4 client.6480 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13fd94d000 1 -- 172.21.15.46:0/1771675145 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646e3ed8cd0 con 0x5646e40fc7d0 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.426+0000 7f13f3400640 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5646e40fc7d0 0x5646e414b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13e0c00640 10 client.6480 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:56.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13f3400640 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5646e40fc7d0 0x5646e414b850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c70b15a3354a06c6 server_cookie=f608127cbfcf1da3 in_seq=0 out_seq=0 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13e0c00640 1 -- 172.21.15.46:0/1771675145 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646e3ed8cd0 con 0x5646e40fc7d0 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13e0c00640 10 client.6480 handle_command_reply: tid=0 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 client.6480 shutdown 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 -- 172.21.15.46:0/1771675145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413ac00 msgr2=0x5646e40fbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413ac00 0x5646e40fbf00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00e280 tx=0x7f13ec00e740 comp rx=0 tx=0).stop 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 -- 172.21.15.46:0/1771675145 shutdown_connections 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5646e40fc7d0 0x5646e414b850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413ac00 0x5646e40fbf00 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413a2e0 0x5646e40fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 --2- 172.21.15.46:0/1771675145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e40fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13e0c00640 0 client.6480 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 -- 172.21.15.46:0/1771675145 >> 172.21.15.46:0/1771675145 conn(0x5646e40ec890 msgr2=0x5646e40ef510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 -- 172.21.15.46:0/1771675145 shutdown_connections 2024-04-25T08:45:56.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 1 -- 172.21.15.46:0/1771675145 wait complete. 2024-04-25T08:45:56.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.430+0000 7f13fd94d000 20 client.6480 trim_cache size 0 max 16384 2024-04-25T08:45:56.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 Processor -- start 2024-04-25T08:45:56.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 -- start start 2024-04-25T08:45:56.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e40f08e0 0x5646e41398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e413a2e0 0x5646e40efc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f42023c0 con 0x5646e40f08e0 2024-04-25T08:45:56.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40cbee0 con 0x5646e413ac00 2024-04-25T08:45:56.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40cc4e0 con 0x5646e413a2e0 2024-04-25T08:45:56.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e40f08e0 0x5646e41398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e40f08e0 0x5646e41398c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44016/0 (socket says 172.21.15.46:44016) 2024-04-25T08:45:56.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3400640 1 -- 172.21.15.46:0/3115995920 learned_addr learned my addr 172.21.15.46:0/3115995920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3e00640 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13eaa00640 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e413a2e0 0x5646e40efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3400640 1 -- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e413a2e0 msgr2=0x5646e40efc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3400640 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e413a2e0 0x5646e40efc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3400640 1 -- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 msgr2=0x5646e40f01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3400640 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3400640 1 -- 172.21.15.46:0/3115995920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646e3ef8fc0 con 0x5646e40f08e0 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3e00640 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13eaa00640 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e413a2e0 0x5646e40efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13f3400640 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e40f08e0 0x5646e41398c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f13e4098470 tx=0x7f13e40f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9f10b07f88fbd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13e3400640 1 -- 172.21.15.46:0/3115995920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40f4cc0 con 0x5646e40f08e0 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13e3400640 1 -- 172.21.15.46:0/3115995920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e40eec10 con 0x5646e40f08e0 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13e3400640 1 -- 172.21.15.46:0/3115995920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e411e310 con 0x5646e40f08e0 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 -- 172.21.15.46:0/3115995920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e40f08e0 msgr2=0x5646e41398c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e40f08e0 0x5646e41398c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f13e4098470 tx=0x7f13e40f2000 comp rx=0 tx=0).stop 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 -- 172.21.15.46:0/3115995920 shutdown_connections 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e40f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e413a2e0 0x5646e40efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3115995920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e40f08e0 0x5646e41398c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 -- 172.21.15.46:0/3115995920 >> 172.21.15.46:0/3115995920 conn(0x5646e40ec890 msgr2=0x5646e4138d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 -- 172.21.15.46:0/3115995920 shutdown_connections 2024-04-25T08:45:56.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.442+0000 7f13fd94d000 1 -- 172.21.15.46:0/3115995920 wait complete. 2024-04-25T08:45:56.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 -- start start 2024-04-25T08:45:56.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 Processor -- start 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e419a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e41552d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e3ed7e70 con 0x5646e413a2e0 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e3ec5a50 con 0x5646e413ac00 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646e3f3c3a0 con 0x5646e40f08e0 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 10 client.0 _ll_register_callbacks cb 0x5646e3ee9240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44020/0 (socket says 172.21.15.46:44020) 2024-04-25T08:45:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13eaa00640 1 -- 172.21.15.46:0/3818978487 learned_addr learned my addr 172.21.15.46:0/3818978487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13f3400640 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13f3e00640 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e41552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13eaa00640 1 -- 172.21.15.46:0/3818978487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 msgr2=0x5646e419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13eaa00640 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13eaa00640 1 -- 172.21.15.46:0/3818978487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 msgr2=0x5646e41552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13eaa00640 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e41552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13eaa00640 1 -- 172.21.15.46:0/3818978487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646e3f969e0 con 0x5646e413a2e0 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13f3400640 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13eaa00640 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e419a5b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f13dc001db0 tx=0x7f13dc000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22e9758f8eda266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13e0c00640 1 -- 172.21.15.46:0/3818978487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc005550 con 0x5646e413a2e0 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13e0c00640 1 -- 172.21.15.46:0/3818978487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13dc010040 con 0x5646e413a2e0 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 -- 172.21.15.46:0/3818978487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13e40eec10 con 0x5646e413a2e0 2024-04-25T08:45:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13e0c00640 1 -- 172.21.15.46:0/3818978487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13dc02d050 con 0x5646e413a2e0 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 10 client.6486 fetch_fsmap learned FSMap version 35 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 -- 172.21.15.46:0/3818978487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5646e3edc630 con 0x5646e413a2e0 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13e0c00640 1 -- 172.21.15.46:0/3818978487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e40eec10 con 0x5646e413a2e0 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13e0c00640 1 -- 172.21.15.46:0/3818978487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f13dc030020 con 0x5646e413a2e0 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 10 client.6486 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 10 client.6486 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 20 client.6486 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5646e41a0ed0 0x5646e41a32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 4 client.6486 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13fd94d000 1 -- 172.21.15.46:0/3818978487 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5646e3f4a5d0 con 0x5646e41a0ed0 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.446+0000 7f13f3400640 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5646e41a0ed0 0x5646e41a32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13f3400640 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5646e41a0ed0 0x5646e41a32c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb104a3d48b9421 server_cookie=6d6309684c72ea28 in_seq=0 out_seq=0 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13e0c00640 10 client.6486 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13e0c00640 1 -- 172.21.15.46:0/3818978487 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x5646e3f4a5d0 con 0x5646e41a0ed0 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13e0c00640 10 client.6486 handle_command_reply: tid=0 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 client.6486 shutdown 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 -- 172.21.15.46:0/3818978487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 msgr2=0x5646e419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e419a5b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f13dc001db0 tx=0x7f13dc000bd0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 -- 172.21.15.46:0/3818978487 shutdown_connections 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5646e41a0ed0 0x5646e41a32c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646e413ac00 0x5646e41552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5646e413a2e0 0x5646e419a5b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 --2- 172.21.15.46:0/3818978487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5646e40f08e0 0x5646e419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13e0c00640 0 client.6486 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 -- 172.21.15.46:0/3818978487 >> 172.21.15.46:0/3818978487 conn(0x5646e40ec890 msgr2=0x5646e40f5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 -- 172.21.15.46:0/3818978487 shutdown_connections 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 1 -- 172.21.15.46:0/3818978487 wait complete. 2024-04-25T08:45:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fd94d000 20 client.6486 trim_cache size 0 max 16384 2024-04-25T08:45:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:45:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:45:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-04-25T08:45:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 2011830694, 2024-04-25T08:45:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-04-25T08:45:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir_test/file" 2024-04-25T08:45:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:45:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 -- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f13d0066f90 msgr2=0x7f13d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f13d0066f90 0x7f13d0069450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f13f41f9f40 tx=0x7f13dc007d70 comp rx=0 tx=0).stop 2024-04-25T08:45:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 -- 172.21.15.46:0/3689954039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f406b470 msgr2=0x7f13f4142090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f406b470 0x7f13f4142090 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f13e40af040 tx=0x7f13e4019740 comp rx=0 tx=0).stop 2024-04-25T08:45:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 -- 172.21.15.46:0/3689954039 shutdown_connections 2024-04-25T08:45:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f13d0066f90 0x7f13d0069450 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f13f415d900 0x7f13f415fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13f40afc20 0x7f13f40c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 --2- 172.21.15.46:0/3689954039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13f406b470 0x7f13f4142090 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 -- 172.21.15.46:0/3689954039 >> 172.21.15.46:0/3689954039 conn(0x7f13f406e3e0 msgr2=0x7f13f40b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 -- 172.21.15.46:0/3689954039 shutdown_connections 2024-04-25T08:45:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.450+0000 7f13fa400640 1 -- 172.21.15.46:0/3689954039 wait complete. 2024-04-25T08:45:56.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:45:56.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3b1a00640 1 Processor -- start 2024-04-25T08:45:56.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3b1a00640 1 -- start start 2024-04-25T08:45:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3b1a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb3ac0cdd60 0x7fb3ac0cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3b1a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac09a7b0 con 0x7fb3ac0d0410 2024-04-25T08:45:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3b1a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ac09aa90 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3b1a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ac0cf2b0 con 0x7fb3ac0cdd60 2024-04-25T08:45:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3ab400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb3ac0d0410 0x7fb3ac15f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49534/0 (socket says 172.21.15.46:49534) 2024-04-25T08:45:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3ab400640 1 -- 172.21.15.46:0/152007189 learned_addr learned my addr 172.21.15.46:0/152007189 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3a2000640 1 --2- 172.21.15.46:0/152007189 >> v2:172.21.15.82:3300/0 conn(0x7fb3ac0cdd60 0x7fb3ac0cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3aa000640 1 -- 172.21.15.46:0/152007189 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212903729 0 0) 0x7fb3ac09a7b0 con 0x7fb3ac0d0410 2024-04-25T08:45:56.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3aa000640 1 -- 172.21.15.46:0/152007189 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb390003530 con 0x7fb3ac0d0410 2024-04-25T08:45:56.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3a2000640 1 -- 172.21.15.46:0/152007189 >> v1:172.21.15.82:6789/0 conn(0x7fb3ac15f2b0 legacy=0x7fb3ac0cfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3a2000640 1 -- 172.21.15.46:0/152007189 >> v1:172.21.15.46:6789/0 conn(0x7fb3ac0d0410 legacy=0x7fb3ac15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.609+0000 7fb3a2000640 1 -- 172.21.15.46:0/152007189 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0cf500 con 0x7fb3ac0cdd60 2024-04-25T08:45:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3a2000640 1 --2- 172.21.15.46:0/152007189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3ac0cdd60 0x7fb3ac0cc180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb3940097c0 tx=0x7fb39401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2bef839043c43ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3aa000640 1 -- 172.21.15.46:0/152007189 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940b0070 con 0x7fb3ac0cdd60 2024-04-25T08:45:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3aa000640 1 -- 172.21.15.46:0/152007189 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3940ab600 con 0x7fb3ac0cdd60 2024-04-25T08:45:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3aa000640 1 -- 172.21.15.46:0/152007189 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940ab9b0 con 0x7fb3ac0cdd60 2024-04-25T08:45:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3b1a00640 1 -- 172.21.15.46:0/152007189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3ac0cdd60 msgr2=0x7fb3ac0cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3b1a00640 1 --2- 172.21.15.46:0/152007189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3ac0cdd60 0x7fb3ac0cc180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb3940097c0 tx=0x7fb39401c3e0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3b1a00640 1 -- 172.21.15.46:0/152007189 shutdown_connections 2024-04-25T08:45:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3b1a00640 1 --2- 172.21.15.46:0/152007189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3ac0cdd60 0x7fb3ac0cc180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3b1a00640 1 -- 172.21.15.46:0/152007189 >> 172.21.15.46:0/152007189 conn(0x7fb3ac0b31d0 msgr2=0x7fb3ac0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3b1a00640 1 -- 172.21.15.46:0/152007189 shutdown_connections 2024-04-25T08:45:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.613+0000 7fb3b1a00640 1 -- 172.21.15.46:0/152007189 wait complete. 2024-04-25T08:45:56.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 Processor -- start 2024-04-25T08:45:56.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 -- start start 2024-04-25T08:45:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3ac0cdd60 0x7fb3ac150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3ac0d0410 0x7fb3ac152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3ac15f2b0 0x7fb3ac155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac09a590 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac09a410 con 0x7fb3ac0cdd60 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac169e50 con 0x7fb3ac0d0410 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3ac15f2b0 0x7fb3ac155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3aaa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3ac0cdd60 0x7fb3ac150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3a2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3ac0d0410 0x7fb3ac152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3ab400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3ac15f2b0 0x7fb3ac155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44030/0 (socket says 172.21.15.46:44030) 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3ab400640 1 -- 172.21.15.46:0/481778069 learned_addr learned my addr 172.21.15.46:0/481778069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3ab400640 1 -- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3ac0d0410 msgr2=0x7fb3ac152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3ab400640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3ac0d0410 0x7fb3ac152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3ab400640 1 -- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3ac0cdd60 msgr2=0x7fb3ac150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3ab400640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3ac0cdd60 0x7fb3ac150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3ab400640 1 -- 172.21.15.46:0/481778069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac0c7810 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3ab400640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3ac15f2b0 0x7fb3ac155a10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c480 tx=0x7fb3a400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3a3400640 1 -- 172.21.15.46:0/481778069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4017070 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3a3400640 1 -- 172.21.15.46:0/481778069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a4005cf0 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3a3400640 1 -- 172.21.15.46:0/481778069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40072b0 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ac0c7af0 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ac0c73a0 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.617+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ac074c60 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.621+0000 7fb3a3400640 1 -- 172.21.15.46:0/481778069 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3a401a070 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.621+0000 7fb3a3400640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb388066f60 0x7fb388069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.621+0000 7fb3a3400640 1 -- 172.21.15.46:0/481778069 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3a4084460 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.621+0000 7fb3aaa00640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb388066f60 0x7fb388069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.621+0000 7fb3aaa00640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb388066f60 0x7fb388069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb39c001e00 tx=0x7fb39c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.625+0000 7fb3a3400640 1 -- 172.21.15.46:0/481778069 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3a4054c80 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb3ac0cd850 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3a3400640 1 -- 172.21.15.46:0/481778069 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fb3a4058d20 con 0x7fb3ac15f2b0 2024-04-25T08:45:56.814 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:45:56.814 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T08:45:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb388066f60 msgr2=0x7fb388069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb388066f60 0x7fb388069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb39c001e00 tx=0x7fb39c009040 comp rx=0 tx=0).stop 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3ac15f2b0 msgr2=0x7fb3ac155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3ac15f2b0 0x7fb3ac155a10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400c480 tx=0x7fb3a400c940 comp rx=0 tx=0).stop 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 shutdown_connections 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb388066f60 0x7fb388069420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3ac15f2b0 0x7fb3ac155a10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb3ac0d0410 0x7fb3ac152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 --2- 172.21.15.46:0/481778069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3ac0cdd60 0x7fb3ac150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 >> 172.21.15.46:0/481778069 conn(0x7fb3ac0b31d0 msgr2=0x7fb3ac1541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 shutdown_connections 2024-04-25T08:45:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.821+0000 7fb3b1a00640 1 -- 172.21.15.46:0/481778069 wait complete. 2024-04-25T08:45:56.836 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:45:56.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:56.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-25T08:45:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.985+0000 7f68eaa00640 1 Processor -- start 2024-04-25T08:45:56.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.985+0000 7f68eaa00640 1 -- start start 2024-04-25T08:45:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68eaa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f68e4140ee0 0x7f68e406e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68eaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e40cebe0 0x7f68e406e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68eaa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e40b2e60 con 0x7f68e4141580 2024-04-25T08:45:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68eaa00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e41412c0 con 0x7f68e4140ee0 2024-04-25T08:45:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68eaa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40b30d0 con 0x7f68e40cebe0 2024-04-25T08:45:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68e8a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e40cebe0 0x7f68e406e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68e3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f68e4140ee0 0x7f68e406e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68dbe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f68e4141580 0x7f68e40ce4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:38200/0 (socket says 172.21.15.46:38200) 2024-04-25T08:45:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68e8a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e40cebe0 0x7f68e406e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:34346/0 (socket says 172.21.15.46:34346) 2024-04-25T08:45:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68dbe00640 1 -- 172.21.15.46:0/1429286773 learned_addr learned my addr 172.21.15.46:0/1429286773 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:45:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68e8a00640 1 -- 172.21.15.46:0/1429286773 >> v2:172.21.15.82:3300/0 conn(0x7f68e4140ee0 msgr2=0x7f68e406e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68e8a00640 1 --2- 172.21.15.46:0/1429286773 >> v2:172.21.15.82:3300/0 conn(0x7f68e4140ee0 0x7f68e406e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68e8a00640 1 -- 172.21.15.46:0/1429286773 >> v1:172.21.15.82:6789/0 conn(0x7f68e4141580 legacy=0x7f68e40ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68e8a00640 1 -- 172.21.15.46:0/1429286773 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e40b33b0 con 0x7f68e40cebe0 2024-04-25T08:45:56.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.989+0000 7f68e3400640 1 -- 172.21.15.46:0/1429286773 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951434099 0 0) 0x7f68e40b2e60 con 0x7f68e4141580 2024-04-25T08:45:56.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68e8a00640 1 --2- 172.21.15.46:0/1429286773 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e40cebe0 0x7f68e406e850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00bfa0 tx=0x7f68dc01eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3373ce43a206e216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68e3400640 1 -- 172.21.15.46:0/1429286773 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00b2d0 con 0x7f68e40cebe0 2024-04-25T08:45:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68e3400640 1 -- 172.21.15.46:0/1429286773 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc0a7a60 con 0x7f68e40cebe0 2024-04-25T08:45:56.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68e3400640 1 -- 172.21.15.46:0/1429286773 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc0a7e30 con 0x7f68e40cebe0 2024-04-25T08:45:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68eaa00640 1 -- 172.21.15.46:0/1429286773 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e40cebe0 msgr2=0x7f68e406e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68eaa00640 1 --2- 172.21.15.46:0/1429286773 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e40cebe0 0x7f68e406e850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00bfa0 tx=0x7f68dc01eb70 comp rx=0 tx=0).stop 2024-04-25T08:45:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68eaa00640 1 -- 172.21.15.46:0/1429286773 shutdown_connections 2024-04-25T08:45:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68eaa00640 1 --2- 172.21.15.46:0/1429286773 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e40cebe0 0x7f68e406e850 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68eaa00640 1 --2- 172.21.15.46:0/1429286773 >> v2:172.21.15.82:3300/0 conn(0x7f68e4140ee0 0x7f68e406e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68eaa00640 1 -- 172.21.15.46:0/1429286773 >> 172.21.15.46:0/1429286773 conn(0x7f68e4067050 msgr2=0x7f68e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68eaa00640 1 -- 172.21.15.46:0/1429286773 shutdown_connections 2024-04-25T08:45:56.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.993+0000 7f68eaa00640 1 -- 172.21.15.46:0/1429286773 wait complete. 2024-04-25T08:45:56.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 Processor -- start 2024-04-25T08:45:56.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 -- start start 2024-04-25T08:45:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e40cebe0 0x7f68e413a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4140ee0 0x7f68e4159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e4141580 0x7f68e415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40b4750 con 0x7f68e40cebe0 2024-04-25T08:45:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40b45d0 con 0x7f68e4140ee0 2024-04-25T08:45:56.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40b48d0 con 0x7f68e4141580 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e40cebe0 0x7f68e413a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e40cebe0 0x7f68e413a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44042/0 (socket says 172.21.15.46:44042) 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e3e00640 1 -- 172.21.15.46:0/2384895863 learned_addr learned my addr 172.21.15.46:0/2384895863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e8a00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e4141580 0x7f68e415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68dbe00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4140ee0 0x7f68e4159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e3e00640 1 -- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e4141580 msgr2=0x7f68e415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e3e00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e4141580 0x7f68e415d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e3e00640 1 -- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4140ee0 msgr2=0x7f68e4159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e3e00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4140ee0 0x7f68e4159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e3e00640 1 -- 172.21.15.46:0/2384895863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e41596d0 con 0x7f68e40cebe0 2024-04-25T08:45:56.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e3e00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e40cebe0 0x7f68e413a710 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f68d400e930 tx=0x7f68d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e0c00640 1 -- 172.21.15.46:0/2384895863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4017070 con 0x7f68e40cebe0 2024-04-25T08:45:56.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e0c00640 1 -- 172.21.15.46:0/2384895863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d400ca00 con 0x7f68e40cebe0 2024-04-25T08:45:56.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68e0c00640 1 -- 172.21.15.46:0/2384895863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d400cdb0 con 0x7f68e40cebe0 2024-04-25T08:45:56.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 -- 172.21.15.46:0/2384895863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e415cc90 con 0x7f68e40cebe0 2024-04-25T08:45:56.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:56.997+0000 7f68eaa00640 1 -- 172.21.15.46:0/2384895863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e415d250 con 0x7f68e40cebe0 2024-04-25T08:45:56.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e0c00640 1 -- 172.21.15.46:0/2384895863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68d401a070 con 0x7f68e40cebe0 2024-04-25T08:45:56.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e0c00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f68c0066f60 0x7f68c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e0c00640 1 -- 172.21.15.46:0/2384895863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68d4085290 con 0x7f68e40cebe0 2024-04-25T08:45:56.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68edf80000 1 Processor -- start 2024-04-25T08:45:56.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68dbe00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f68c0066f60 0x7f68c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68edf80000 1 -- start start 2024-04-25T08:45:56.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68dbe00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f68c0066f60 0x7f68c0069420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f68e415e0c0 tx=0x7f68cc007ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68edf80000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4c520 0x5631e0f4c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68edf80000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4ce40 0x5631e0f03430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68edf80000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f03970 0x5631e0f03d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68edf80000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0d50d00 con 0x5631e0f4c520 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68edf80000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0cd8df0 con 0x5631e0f03970 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68edf80000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631e0d4cda0 con 0x5631e0f4ce40 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4c520 0x5631e0f4c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4c520 0x5631e0f4c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44046/0 (socket says 172.21.15.46:44046) 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e3e00640 1 -- 172.21.15.46:0/1400800139 learned_addr learned my addr 172.21.15.46:0/1400800139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68dbe00640 1 --2- 172.21.15.46:0/1400800139 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4ce40 0x5631e0f03430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e8a00640 1 --2- 172.21.15.46:0/1400800139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f03970 0x5631e0f03d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:56.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e3e00640 1 -- 172.21.15.46:0/1400800139 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4ce40 msgr2=0x5631e0f03430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e3e00640 1 --2- 172.21.15.46:0/1400800139 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4ce40 0x5631e0f03430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e3e00640 1 -- 172.21.15.46:0/1400800139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f03970 msgr2=0x5631e0f03d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:56.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e3e00640 1 --2- 172.21.15.46:0/1400800139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f03970 0x5631e0f03d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:56.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.001+0000 7f68e3e00640 1 -- 172.21.15.46:0/1400800139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e0d0b580 con 0x5631e0f4c520 2024-04-25T08:45:56.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e3e00640 1 --2- 172.21.15.46:0/1400800139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4c520 0x5631e0f4c900 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f68d40559c0 tx=0x7f68d4085110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50428a5f4047f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:56.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68d8c00640 1 -- 172.21.15.46:0/1400800139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4021040 con 0x5631e0f4c520 2024-04-25T08:45:56.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68d8c00640 1 -- 172.21.15.46:0/1400800139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4081070 con 0x5631e0f4c520 2024-04-25T08:45:56.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68d8c00640 1 -- 172.21.15.46:0/1400800139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d405e140 con 0x5631e0f4c520 2024-04-25T08:45:57.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 -- 172.21.15.46:0/1400800139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4c520 msgr2=0x5631e0f4c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 --2- 172.21.15.46:0/1400800139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4c520 0x5631e0f4c900 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f68d40559c0 tx=0x7f68d4085110 comp rx=0 tx=0).stop 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 -- 172.21.15.46:0/1400800139 shutdown_connections 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 --2- 172.21.15.46:0/1400800139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f03970 0x5631e0f03d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 --2- 172.21.15.46:0/1400800139 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4ce40 0x5631e0f03430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 --2- 172.21.15.46:0/1400800139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4c520 0x5631e0f4c900 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 -- 172.21.15.46:0/1400800139 >> 172.21.15.46:0/1400800139 conn(0x5631e0deac10 msgr2=0x5631e0f01d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 -- 172.21.15.46:0/1400800139 shutdown_connections 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 -- 172.21.15.46:0/1400800139 wait complete. 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 -- start start 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 Processor -- start 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 0x5631e0f52ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 0x5631e0f53000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f07ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0ce96d0 con 0x5631e0f4ce40 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0d0b950 con 0x5631e0f4c520 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631e0d50010 con 0x5631e0f03970 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68edf80000 10 client.0 _ll_register_callbacks cb 0x5631e0eef030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f07ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f07ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44048/0 (socket says 172.21.15.46:44048) 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e8a00640 1 -- 172.21.15.46:0/199987355 learned_addr learned my addr 172.21.15.46:0/199987355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e3e00640 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 0x5631e0f52ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e8a00640 1 -- 172.21.15.46:0/199987355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 msgr2=0x5631e0f52ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e8a00640 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 0x5631e0f52ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e8a00640 1 -- 172.21.15.46:0/199987355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 msgr2=0x5631e0f53000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e8a00640 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 0x5631e0f53000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e8a00640 1 -- 172.21.15.46:0/199987355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631e0cedd40 con 0x5631e0f4ce40 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e3e00640 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 0x5631e0f52ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68e8a00640 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f07ef0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0b1040 tx=0x7f68dc01d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b269edbbc74ed82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.005+0000 7f68d1600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68d1600640 1 -- 172.21.15.46:0/199987355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc0c2640 con 0x5631e0f4ce40 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68d1600640 1 -- 172.21.15.46:0/199987355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68dc0b83f0 con 0x5631e0f4ce40 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 -- 172.21.15.46:0/199987355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68d4081070 con 0x5631e0f4ce40 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68d1600640 1 -- 172.21.15.46:0/199987355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68dc0c27e0 con 0x5631e0f4ce40 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 10 client.6501 fetch_fsmap learned FSMap version 35 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 -- 172.21.15.46:0/199987355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5631e0d4bcf0 con 0x5631e0f4ce40 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68d1600640 1 -- 172.21.15.46:0/199987355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4081070 con 0x5631e0f4ce40 2024-04-25T08:45:57.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68d1600640 1 -- 172.21.15.46:0/199987355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f68dc0c2980 con 0x5631e0f4ce40 2024-04-25T08:45:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 10 client.6501 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:45:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 10 client.6501 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 20 client.6501 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5631e0f08c60 0x5631e0f09040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 4 client.6501 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:45:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 -- 172.21.15.46:0/199987355 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631e0cec9d0 con 0x5631e0f08c60 2024-04-25T08:45:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68e3e00640 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5631e0f08c60 0x5631e0f09040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68d1600640 10 client.6501 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68e3e00640 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5631e0f08c60 0x5631e0f09040 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d794e8aa041dacc3 server_cookie=316b1e109959d842 in_seq=0 out_seq=0 2024-04-25T08:45:57.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68d1600640 1 -- 172.21.15.46:0/199987355 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631e0cec9d0 con 0x5631e0f08c60 2024-04-25T08:45:57.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68d1600640 10 client.6501 handle_command_reply: tid=0 2024-04-25T08:45:57.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 client.6501 shutdown 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 -- 172.21.15.46:0/199987355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 msgr2=0x5631e0f07ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f07ef0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0b1040 tx=0x7f68dc01d3b0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 -- 172.21.15.46:0/199987355 shutdown_connections 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5631e0f08c60 0x5631e0f09040 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f07ef0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 0x5631e0f53000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 --2- 172.21.15.46:0/199987355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 0x5631e0f52ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 -- 172.21.15.46:0/199987355 >> 172.21.15.46:0/199987355 conn(0x5631e0deac10 msgr2=0x5631e0f02420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68d1600640 0 client.6501 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 -- 172.21.15.46:0/199987355 shutdown_connections 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 1 -- 172.21.15.46:0/199987355 wait complete. 2024-04-25T08:45:57.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.009+0000 7f68edf80000 20 client.6501 trim_cache size 0 max 16384 2024-04-25T08:45:57.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68edf80000 1 Processor -- start 2024-04-25T08:45:57.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68edf80000 1 -- start start 2024-04-25T08:45:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68edf80000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f03970 0x5631e0fa7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68edf80000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4c520 0x5631e0fa8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68edf80000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4ce40 0x5631e0fa8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68edf80000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4202ef0 con 0x5631e0f03970 2024-04-25T08:45:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68edf80000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e42020d0 con 0x5631e0f4ce40 2024-04-25T08:45:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68edf80000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4158600 con 0x5631e0f4c520 2024-04-25T08:45:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f03970 0x5631e0fa7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f03970 0x5631e0fa7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44060/0 (socket says 172.21.15.46:44060) 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e3e00640 1 -- 172.21.15.46:0/2119219128 learned_addr learned my addr 172.21.15.46:0/2119219128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68dbe00640 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4c520 0x5631e0fa8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e8a00640 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4ce40 0x5631e0fa8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e3e00640 1 -- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4c520 msgr2=0x5631e0fa8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e3e00640 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4c520 0x5631e0fa8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e3e00640 1 -- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4ce40 msgr2=0x5631e0fa8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e3e00640 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4ce40 0x5631e0fa8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e3e00640 1 -- 172.21.15.46:0/2119219128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631e0f4cc70 con 0x5631e0f03970 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68dbe00640 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4c520 0x5631e0fa8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e8a00640 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4ce40 0x5631e0fa8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.025+0000 7f68e3e00640 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f03970 0x5631e0fa7c10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f68d4000c10 tx=0x7f68d401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d1d22e6f380617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68d3e00640 1 -- 172.21.15.46:0/2119219128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4021040 con 0x5631e0f03970 2024-04-25T08:45:57.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68d3e00640 1 -- 172.21.15.46:0/2119219128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4081040 con 0x5631e0f03970 2024-04-25T08:45:57.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68d3e00640 1 -- 172.21.15.46:0/2119219128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4054e80 con 0x5631e0f03970 2024-04-25T08:45:57.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- 172.21.15.46:0/2119219128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f03970 msgr2=0x5631e0fa7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f03970 0x5631e0fa7c10 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f68d4000c10 tx=0x7f68d401e000 comp rx=0 tx=0).stop 2024-04-25T08:45:57.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- 172.21.15.46:0/2119219128 shutdown_connections 2024-04-25T08:45:57.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4ce40 0x5631e0fa8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f4c520 0x5631e0fa8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 --2- 172.21.15.46:0/2119219128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f03970 0x5631e0fa7c10 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- 172.21.15.46:0/2119219128 >> 172.21.15.46:0/2119219128 conn(0x5631e0deac10 msgr2=0x5631e0f4b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- 172.21.15.46:0/2119219128 shutdown_connections 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- 172.21.15.46:0/2119219128 wait complete. 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- start start 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 Processor -- start 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 0x5631e0f93820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 0x5631e0f941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f94720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631e0d55330 con 0x5631e0f4ce40 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4201d10 con 0x5631e0f4c520 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631e0cd97e0 con 0x5631e0f03970 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 10 client.0 _ll_register_callbacks cb 0x7f68d4051cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f94720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f94720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44076/0 (socket says 172.21.15.46:44076) 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e8a00640 1 -- 172.21.15.46:0/1460798982 learned_addr learned my addr 172.21.15.46:0/1460798982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e3e00640 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 0x5631e0f93820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68dbe00640 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 0x5631e0f941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e8a00640 1 -- 172.21.15.46:0/1460798982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 msgr2=0x5631e0f93820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e8a00640 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 0x5631e0f93820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e8a00640 1 -- 172.21.15.46:0/1460798982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 msgr2=0x5631e0f941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e8a00640 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 0x5631e0f941e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e8a00640 1 -- 172.21.15.46:0/1460798982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631e0dd3d30 con 0x5631e0f4ce40 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68dbe00640 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 0x5631e0f941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68e8a00640 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f94720 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f68dc01b780 tx=0x7f68dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5544b6f2d48ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68d1600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68d1600640 1 -- 172.21.15.46:0/1460798982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00b660 con 0x5631e0f4ce40 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68d1600640 1 -- 172.21.15.46:0/1460798982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68dc003bb0 con 0x5631e0f4ce40 2024-04-25T08:45:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.029+0000 7f68edf80000 1 -- 172.21.15.46:0/1460798982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68d4081040 con 0x5631e0f4ce40 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68d1600640 1 -- 172.21.15.46:0/1460798982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68dc0a7a60 con 0x5631e0f4ce40 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 10 client.6507 fetch_fsmap learned FSMap version 35 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 -- 172.21.15.46:0/1460798982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5631e0d1d5a0 con 0x5631e0f4ce40 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68d1600640 1 -- 172.21.15.46:0/1460798982 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d4081040 con 0x5631e0f4ce40 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68d1600640 1 -- 172.21.15.46:0/1460798982 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f68dc004030 con 0x5631e0f4ce40 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 10 client.6507 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 10 client.6507 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 20 client.6507 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5631e0f94c60 0x5631e0f95070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 4 client.6507 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 -- 172.21.15.46:0/1460798982 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x5631e0d2d610 con 0x5631e0f94c60 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68e3e00640 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5631e0f94c60 0x5631e0f95070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68e3e00640 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5631e0f94c60 0x5631e0f95070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61cb1eba3571bc52 server_cookie=fd9ff4322a36447 in_seq=0 out_seq=0 2024-04-25T08:45:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68d1600640 10 client.6507 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:57.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68d1600640 1 -- 172.21.15.46:0/1460798982 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5631e0d2d610 con 0x5631e0f94c60 2024-04-25T08:45:57.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68d1600640 10 client.6507 handle_command_reply: tid=0 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 client.6507 shutdown 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 -- 172.21.15.46:0/1460798982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 msgr2=0x5631e0f94720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f94720 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f68dc01b780 tx=0x7f68dc00b960 comp rx=0 tx=0).stop 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 -- 172.21.15.46:0/1460798982 shutdown_connections 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5631e0f94c60 0x5631e0f95070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5631e0f4ce40 0x5631e0f94720 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5631e0f4c520 0x5631e0f941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 --2- 172.21.15.46:0/1460798982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5631e0f03970 0x5631e0f93820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 -- 172.21.15.46:0/1460798982 >> 172.21.15.46:0/1460798982 conn(0x5631e0deac10 msgr2=0x5631e0f4d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68d1600640 0 client.6507 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:57.032 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f8efc400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6169=0-4194304@1} caps={6169=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6169 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55bc26ca1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:45:57.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 -- 172.21.15.46:0/1460798982 shutdown_connections 2024-04-25T08:45:57.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 1 -- 172.21.15.46:0/1460798982 wait complete. 2024-04-25T08:45:57.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68edf80000 20 client.6507 trim_cache size 0 max 16384 2024-04-25T08:45:57.033 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:45:57.033 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:45:57.033 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "f026cd5d-b800-4727-8e26-5c5561316672", 2024-04-25T08:45:57.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:45:57.033 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:45:57.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 -- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f68c0066f60 msgr2=0x7f68c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f68c0066f60 0x7f68c0069420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f68e415e0c0 tx=0x7f68cc007ba0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 -- 172.21.15.46:0/2384895863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e40cebe0 msgr2=0x7f68e413a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e40cebe0 0x7f68e413a710 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f68d400e930 tx=0x7f68d400edf0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 -- 172.21.15.46:0/2384895863 shutdown_connections 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f68c0066f60 0x7f68c0069420 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68e4141580 0x7f68e415d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4140ee0 0x7f68e4159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 --2- 172.21.15.46:0/2384895863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e40cebe0 0x7f68e413a710 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 -- 172.21.15.46:0/2384895863 >> 172.21.15.46:0/2384895863 conn(0x7f68e4067050 msgr2=0x7f68e413b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 -- 172.21.15.46:0/2384895863 shutdown_connections 2024-04-25T08:45:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.033+0000 7f68eaa00640 1 -- 172.21.15.46:0/2384895863 wait complete. 2024-04-25T08:45:57.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:45:57.048 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T08:45:57.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.197+0000 7f4043a00640 1 Processor -- start 2024-04-25T08:45:57.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.197+0000 7f4043a00640 1 -- start start 2024-04-25T08:45:57.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.197+0000 7f4043a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c06ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.197+0000 7f4043a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f403c06b2a0 0x7f403c0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.197+0000 7f4043a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c06b6c0 con 0x7f403c15d900 2024-04-25T08:45:57.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.197+0000 7f4043a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c06fca0 con 0x7f403c06b2a0 2024-04-25T08:45:57.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.197+0000 7f4043a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c06ff80 con 0x7f403c0afdf0 2024-04-25T08:45:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c06ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c06ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:34396/0 (socket says 172.21.15.46:34396) 2024-04-25T08:45:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403be00640 1 -- 172.21.15.46:0/722557107 learned_addr learned my addr 172.21.15.46:0/722557107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f4041a00640 1 --2- 172.21.15.46:0/722557107 >> v2:172.21.15.82:3300/0 conn(0x7f403c06b2a0 0x7f403c0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f4041000640 1 --1- 172.21.15.46:0/722557107 >> v1:172.21.15.82:6789/0 conn(0x7f403c15d900 0x7f403c0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:38210/0 (socket says 172.21.15.46:38210) 2024-04-25T08:45:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403be00640 1 -- 172.21.15.46:0/722557107 >> v2:172.21.15.82:3300/0 conn(0x7f403c06b2a0 msgr2=0x7f403c0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403be00640 1 --2- 172.21.15.46:0/722557107 >> v2:172.21.15.82:3300/0 conn(0x7f403c06b2a0 0x7f403c0d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403b400640 1 -- 172.21.15.46:0/722557107 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105516609 0 0) 0x7f403c06b6c0 con 0x7f403c15d900 2024-04-25T08:45:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403be00640 1 -- 172.21.15.46:0/722557107 >> v1:172.21.15.82:6789/0 conn(0x7f403c15d900 legacy=0x7f403c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403be00640 1 -- 172.21.15.46:0/722557107 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c070260 con 0x7f403c0afdf0 2024-04-25T08:45:57.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403be00640 1 --2- 172.21.15.46:0/722557107 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c06ad60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4028009870 tx=0x7f402801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccb2d47a2e6f3bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403b400640 1 -- 172.21.15.46:0/722557107 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280b1070 con 0x7f403c0afdf0 2024-04-25T08:45:57.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403b400640 1 -- 172.21.15.46:0/722557107 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40280ac620 con 0x7f403c0afdf0 2024-04-25T08:45:57.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.201+0000 7f403b400640 1 -- 172.21.15.46:0/722557107 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280ac9f0 con 0x7f403c0afdf0 2024-04-25T08:45:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 -- 172.21.15.46:0/722557107 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 msgr2=0x7f403c06ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 --2- 172.21.15.46:0/722557107 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c06ad60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4028009870 tx=0x7f402801c4d0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 -- 172.21.15.46:0/722557107 shutdown_connections 2024-04-25T08:45:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 --2- 172.21.15.46:0/722557107 >> v2:172.21.15.82:3300/0 conn(0x7f403c06b2a0 0x7f403c0d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 --2- 172.21.15.46:0/722557107 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c06ad60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 -- 172.21.15.46:0/722557107 >> 172.21.15.46:0/722557107 conn(0x7f403c06e3e0 msgr2=0x7f403c06e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 -- 172.21.15.46:0/722557107 shutdown_connections 2024-04-25T08:45:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 -- 172.21.15.46:0/722557107 wait complete. 2024-04-25T08:45:57.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 Processor -- start 2024-04-25T08:45:57.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 -- start start 2024-04-25T08:45:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c06b2a0 0x7f403c14aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f403c15d900 0x7f403c147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0d10e0 con 0x7f403c06b2a0 2024-04-25T08:45:57.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0d0f60 con 0x7f403c15d900 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4043a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0d1260 con 0x7f403c0afdf0 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4041000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c06b2a0 0x7f403c14aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4041000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c06b2a0 0x7f403c14aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44084/0 (socket says 172.21.15.46:44084) 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4041000640 1 -- 172.21.15.46:0/4239162842 learned_addr learned my addr 172.21.15.46:0/4239162842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f403be00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4041a00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f403c15d900 0x7f403c147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4041000640 1 -- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 msgr2=0x7f403c148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4041000640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4041000640 1 -- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f403c15d900 msgr2=0x7f403c147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4041000640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f403c15d900 0x7f403c147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.205+0000 7f4041000640 1 -- 172.21.15.46:0/4239162842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c14b070 con 0x7f403c06b2a0 2024-04-25T08:45:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.209+0000 7f4041000640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c06b2a0 0x7f403c14aa90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f403000c480 tx=0x7f403000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.209+0000 7f4038c00640 1 -- 172.21.15.46:0/4239162842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030018070 con 0x7f403c06b2a0 2024-04-25T08:45:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.209+0000 7f4038c00640 1 -- 172.21.15.46:0/4239162842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4030007500 con 0x7f403c06b2a0 2024-04-25T08:45:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.209+0000 7f4038c00640 1 -- 172.21.15.46:0/4239162842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030004a30 con 0x7f403c06b2a0 2024-04-25T08:45:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.209+0000 7f4043a00640 1 -- 172.21.15.46:0/4239162842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c14b350 con 0x7f403c06b2a0 2024-04-25T08:45:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.209+0000 7f4043a00640 1 -- 172.21.15.46:0/4239162842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c14b910 con 0x7f403c06b2a0 2024-04-25T08:45:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.209+0000 7f4038c00640 1 -- 172.21.15.46:0/4239162842 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f403001b070 con 0x7f403c06b2a0 2024-04-25T08:45:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.209+0000 7f4038c00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f401c066f60 0x7f401c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.209+0000 7f4038c00640 1 -- 172.21.15.46:0/4239162842 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f40300844f0 con 0x7f403c06b2a0 2024-04-25T08:45:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f401c066f60 0x7f401c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 Processor -- start 2024-04-25T08:45:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 -- start start 2024-04-25T08:45:57.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f401c066f60 0x7f401c069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f40280ae000 tx=0x7f4028008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210389d0 0x561721038db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x561720feeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561720fef0a0 0x561720fef480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561720e3bff0 con 0x5617210392f0 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561720dc4150 con 0x5617210389d0 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561720e38090 con 0x561720fef0a0 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x561720feeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x561720feeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44090/0 (socket says 172.21.15.46:44090) 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 -- 172.21.15.46:0/840794844 learned_addr learned my addr 172.21.15.46:0/840794844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4041a00640 1 --2- 172.21.15.46:0/840794844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561720fef0a0 0x561720fef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4041000640 1 --2- 172.21.15.46:0/840794844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210389d0 0x561721038db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 -- 172.21.15.46:0/840794844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561720fef0a0 msgr2=0x561720fef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 --2- 172.21.15.46:0/840794844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561720fef0a0 0x561720fef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 -- 172.21.15.46:0/840794844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210389d0 msgr2=0x561721038db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 --2- 172.21.15.46:0/840794844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210389d0 0x561721038db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 -- 172.21.15.46:0/840794844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561720fd20d0 con 0x5617210392f0 2024-04-25T08:45:57.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f403be00640 1 --2- 172.21.15.46:0/840794844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x561720feeb60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4028009870 tx=0x7f40280a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16afc10c7ecf737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f402d600640 1 -- 172.21.15.46:0/840794844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280c8070 con 0x5617210392f0 2024-04-25T08:45:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f402d600640 1 -- 172.21.15.46:0/840794844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40280b7670 con 0x5617210392f0 2024-04-25T08:45:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f402d600640 1 -- 172.21.15.46:0/840794844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280b1040 con 0x5617210392f0 2024-04-25T08:45:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 -- 172.21.15.46:0/840794844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 msgr2=0x561720feeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 --2- 172.21.15.46:0/840794844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x561720feeb60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4028009870 tx=0x7f40280a7000 comp rx=0 tx=0).stop 2024-04-25T08:45:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 -- 172.21.15.46:0/840794844 shutdown_connections 2024-04-25T08:45:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 --2- 172.21.15.46:0/840794844 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561720fef0a0 0x561720fef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 --2- 172.21.15.46:0/840794844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x561720feeb60 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 --2- 172.21.15.46:0/840794844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210389d0 0x561721038db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 -- 172.21.15.46:0/840794844 >> 172.21.15.46:0/840794844 conn(0x561720feb020 msgr2=0x561720fed440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 -- 172.21.15.46:0/840794844 shutdown_connections 2024-04-25T08:45:57.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.213+0000 7f4046e6e000 1 -- 172.21.15.46:0/840794844 wait complete. 2024-04-25T08:45:57.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 -- start start 2024-04-25T08:45:57.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 Processor -- start 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 0x56172101d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 0x56172101dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x56172101e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561720e00250 con 0x5617210392f0 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561720df6690 con 0x561720fef0a0 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561720dc4a40 con 0x5617210389d0 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 10 client.0 _ll_register_callbacks cb 0x561720fd9e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x56172101e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x56172101e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44104/0 (socket says 172.21.15.46:44104) 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041a00640 1 -- 172.21.15.46:0/477849671 learned_addr learned my addr 172.21.15.46:0/477849671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041000640 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 0x56172101d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f403be00640 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 0x56172101dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041a00640 1 -- 172.21.15.46:0/477849671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 msgr2=0x56172101dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041a00640 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 0x56172101dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041a00640 1 -- 172.21.15.46:0/477849671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 msgr2=0x56172101d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041a00640 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 0x56172101d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041a00640 1 -- 172.21.15.46:0/477849671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561720ee91c0 con 0x5617210392f0 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041a00640 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x56172101e270 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4034014d00 tx=0x7f4034012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201604a48abe2943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4022000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4022000640 1 -- 172.21.15.46:0/477849671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034003ba0 con 0x5617210392f0 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4022000640 1 -- 172.21.15.46:0/477849671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4034003d40 con 0x5617210392f0 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 -- 172.21.15.46:0/477849671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40280b7670 con 0x5617210392f0 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4022000640 1 -- 172.21.15.46:0/477849671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4034004030 con 0x5617210392f0 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 10 client.6516 fetch_fsmap learned FSMap version 35 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 -- 172.21.15.46:0/477849671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x561720ee5170 con 0x5617210392f0 2024-04-25T08:45:57.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4022000640 1 -- 172.21.15.46:0/477849671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40280b7670 con 0x5617210392f0 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4022000640 1 -- 172.21.15.46:0/477849671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f40340041d0 con 0x5617210392f0 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 10 client.6516 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 10 client.6516 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 20 client.6516 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x561721048c80 0x56172104b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 4 client.6516 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4046e6e000 1 -- 172.21.15.46:0/477849671 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561720dd7c60 con 0x561721048c80 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.217+0000 7f4041000640 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x561721048c80 0x56172104b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4041000640 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x561721048c80 0x56172104b070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2077cf19b19c8e79 server_cookie=fc8578b54b5d9a20 in_seq=0 out_seq=0 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4022000640 10 client.6516 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4022000640 1 -- 172.21.15.46:0/477849671 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561720dd7c60 con 0x561721048c80 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4022000640 10 client.6516 handle_command_reply: tid=0 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 client.6516 shutdown 2024-04-25T08:45:57.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 -- 172.21.15.46:0/477849671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 msgr2=0x56172101e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x56172101e270 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4034014d00 tx=0x7f4034012670 comp rx=0 tx=0).stop 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 -- 172.21.15.46:0/477849671 shutdown_connections 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x561721048c80 0x56172104b070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210392f0 0x56172101e270 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 0x56172101dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 --2- 172.21.15.46:0/477849671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 0x56172101d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4022000640 0 client.6516 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 -- 172.21.15.46:0/477849671 >> 172.21.15.46:0/477849671 conn(0x561720feb020 msgr2=0x561720fedca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 -- 172.21.15.46:0/477849671 shutdown_connections 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 1 -- 172.21.15.46:0/477849671 wait complete. 2024-04-25T08:45:57.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.221+0000 7f4046e6e000 20 client.6516 trim_cache size 0 max 16384 2024-04-25T08:45:57.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 Processor -- start 2024-04-25T08:45:57.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- start start 2024-04-25T08:45:57.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 0x561721037de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210389d0 0x561720fee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210392f0 0x561720fee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c202f40 con 0x5617210389d0 2024-04-25T08:45:57.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c202230 con 0x561720fef0a0 2024-04-25T08:45:57.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c155a50 con 0x5617210392f0 2024-04-25T08:45:57.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f403be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210389d0 0x561720fee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f403be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210389d0 0x561720fee420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44110/0 (socket says 172.21.15.46:44110) 2024-04-25T08:45:57.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f403be00640 1 -- 172.21.15.46:0/1752986020 learned_addr learned my addr 172.21.15.46:0/1752986020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4041000640 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 0x561721037de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4041a00640 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210392f0 0x561720fee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f403be00640 1 -- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210392f0 msgr2=0x561720fee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f403be00640 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210392f0 0x561720fee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f403be00640 1 -- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 msgr2=0x561721037de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f403be00640 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 0x561721037de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f403be00640 1 -- 172.21.15.46:0/1752986020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561720e82de0 con 0x5617210389d0 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4041a00640 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210392f0 0x561720fee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4041000640 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 0x561721037de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f403be00640 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210389d0 0x561720fee420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f402801c9b0 tx=0x7f4028009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d66a9976877e8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f402cc00640 1 -- 172.21.15.46:0/1752986020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280c8040 con 0x5617210389d0 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f402cc00640 1 -- 172.21.15.46:0/1752986020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40280d2070 con 0x5617210389d0 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f402cc00640 1 -- 172.21.15.46:0/1752986020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40280b1040 con 0x5617210389d0 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- 172.21.15.46:0/1752986020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210389d0 msgr2=0x561720fee420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210389d0 0x561720fee420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f402801c9b0 tx=0x7f4028009240 comp rx=0 tx=0).stop 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- 172.21.15.46:0/1752986020 shutdown_connections 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210392f0 0x561720fee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617210389d0 0x561720fee420 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 --2- 172.21.15.46:0/1752986020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561720fef0a0 0x561721037de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- 172.21.15.46:0/1752986020 >> 172.21.15.46:0/1752986020 conn(0x561720feb020 msgr2=0x5617210370a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- 172.21.15.46:0/1752986020 shutdown_connections 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- 172.21.15.46:0/1752986020 wait complete. 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 -- start start 2024-04-25T08:45:57.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.237+0000 7f4046e6e000 1 Processor -- start 2024-04-25T08:45:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561720fef0a0 0x561721099520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 0x561721099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210392f0 0x56172104b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561720dd6e00 con 0x561720fef0a0 2024-04-25T08:45:57.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0ca3a0 con 0x5617210392f0 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561720e3b300 con 0x5617210389d0 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 10 client.0 _ll_register_callbacks cb 0x561720fba040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561720fef0a0 0x561721099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561720fef0a0 0x561721099520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44122/0 (socket says 172.21.15.46:44122) 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041000640 1 -- 172.21.15.46:0/3219618973 learned_addr learned my addr 172.21.15.46:0/3219618973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f403be00640 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 0x561721099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041a00640 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210392f0 0x56172104b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041000640 1 -- 172.21.15.46:0/3219618973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 msgr2=0x561721099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041000640 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 0x561721099a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041000640 1 -- 172.21.15.46:0/3219618973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210392f0 msgr2=0x56172104b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041000640 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210392f0 0x56172104b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041000640 1 -- 172.21.15.46:0/3219618973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561720ddd8e0 con 0x561720fef0a0 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f403be00640 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 0x561721099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4041000640 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561720fef0a0 0x561721099520 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4030051170 tx=0x7f4030082750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb15018dd40af88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:45:57.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 1 -- 172.21.15.46:0/3219618973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4030051380 con 0x561720fef0a0 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 1 -- 172.21.15.46:0/3219618973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4030054f70 con 0x561720fef0a0 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 -- 172.21.15.46:0/3219618973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40280d2070 con 0x561720fef0a0 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 1 -- 172.21.15.46:0/3219618973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4030071050 con 0x561720fef0a0 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 10 client.6522 fetch_fsmap learned FSMap version 35 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 -- 172.21.15.46:0/3219618973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x561720e958f0 con 0x561720fef0a0 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 1 -- 172.21.15.46:0/3219618973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40280d2070 con 0x561720fef0a0 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 1 -- 172.21.15.46:0/3219618973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f403005cee0 con 0x561720fef0a0 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 10 client.6522 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 10 client.6522 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 20 client.6522 populate_metadata read hostname 'smithi046' 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56172104bc50 0x56172104c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 4 client.6522 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 -- 172.21.15.46:0/3219618973 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561720e49530 con 0x56172104bc50 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f403be00640 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56172104bc50 0x56172104c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:45:57.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f403be00640 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56172104bc50 0x56172104c030 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9ecf491dfdfc97 server_cookie=b368f93493e0abe5 in_seq=0 out_seq=0 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 10 client.6522 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 1 -- 172.21.15.46:0/3219618973 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x561720e49530 con 0x56172104bc50 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 10 client.6522 handle_command_reply: tid=0 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 client.6522 shutdown 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 -- 172.21.15.46:0/3219618973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561720fef0a0 msgr2=0x561721099520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561720fef0a0 0x561721099520 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4030051170 tx=0x7f4030082750 comp rx=0 tx=0).stop 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 -- 172.21.15.46:0/3219618973 shutdown_connections 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x56172104bc50 0x56172104c030 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617210392f0 0x56172104b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5617210389d0 0x561721099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 --2- 172.21.15.46:0/3219618973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561720fef0a0 0x561721099520 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4046e6e000 1 -- 172.21.15.46:0/3219618973 >> 172.21.15.46:0/3219618973 conn(0x561720feb020 msgr2=0x561721037930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.241+0000 7f4022000640 0 client.6522 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4046e6e000 1 -- 172.21.15.46:0/3219618973 shutdown_connections 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4046e6e000 1 -- 172.21.15.46:0/3219618973 wait complete. 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4046e6e000 20 client.6522 trim_cache size 0 max 16384 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stdout: "f026cd5d-b800-4727-8e26-5c5561316672": { 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "f026cd5d-b800-4727-8e26-5c5561316672", 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair,force" 2024-04-25T08:45:57.242 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:57.243 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:45:57.243 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:45:57.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 -- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f401c066f60 msgr2=0x7f401c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f401c066f60 0x7f401c069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f40280ae000 tx=0x7f4028008040 comp rx=0 tx=0).stop 2024-04-25T08:45:57.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 -- 172.21.15.46:0/4239162842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c06b2a0 msgr2=0x7f403c14aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:45:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c06b2a0 0x7f403c14aa90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f403000c480 tx=0x7f403000c940 comp rx=0 tx=0).stop 2024-04-25T08:45:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 -- 172.21.15.46:0/4239162842 shutdown_connections 2024-04-25T08:45:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f401c066f60 0x7f401c069420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f403c15d900 0x7f403c147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f403c0afdf0 0x7f403c148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 --2- 172.21.15.46:0/4239162842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c06b2a0 0x7f403c14aa90 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:45:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 -- 172.21.15.46:0/4239162842 >> 172.21.15.46:0/4239162842 conn(0x7f403c06e3e0 msgr2=0x7f403c0cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:45:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 -- 172.21.15.46:0/4239162842 shutdown_connections 2024-04-25T08:45:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:45:57.245+0000 7f4043a00640 1 -- 172.21.15.46:0/4239162842 wait complete. 2024-04-25T08:45:57.257 INFO:tasks.cephfs.filesystem:scrub status for tag:f026cd5d-b800-4727-8e26-5c5561316672 - {'path': '/', 'tag': 'f026cd5d-b800-4727-8e26-5c5561316672', 'options': 'recursive,repair,force'} 2024-04-25T08:46:20.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:46:20.518 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:46:27.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:27.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T08:46:27.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.417+0000 7f2543a00640 1 Processor -- start 2024-04-25T08:46:27.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.417+0000 7f2543a00640 1 -- start start 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2543a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f253c145310 0x7f253c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2543a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c15d090 0x7f253c15d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2543a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c0cd930 con 0x7f253c06dc70 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2543a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c0cdbd0 con 0x7f253c145310 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2543a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0cdeb0 con 0x7f253c15d090 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2541000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f253c145310 0x7f253c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2541000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f253c145310 0x7f253c06e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55866/0 (socket says 172.21.15.46:55866) 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2541000640 1 -- 172.21.15.46:0/2976083123 learned_addr learned my addr 172.21.15.46:0/2976083123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2541a00640 1 --2- 172.21.15.46:0/2976083123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c15d090 0x7f253c15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2541000640 1 -- 172.21.15.46:0/2976083123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c15d090 msgr2=0x7f253c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2541000640 1 --2- 172.21.15.46:0/2976083123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c15d090 0x7f253c15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2541000640 1 -- 172.21.15.46:0/2976083123 >> v1:172.21.15.82:6789/0 conn(0x7f253c06dc70 legacy=0x7f253c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.421+0000 7f2541000640 1 -- 172.21.15.46:0/2976083123 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0ce190 con 0x7f253c145310 2024-04-25T08:46:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2541000640 1 --2- 172.21.15.46:0/2976083123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c145310 0x7f253c06e0d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2530009870 tx=0x7f253001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ebbca063aafdfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f253b400640 1 -- 172.21.15.46:0/2976083123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300ab650 con 0x7f253c145310 2024-04-25T08:46:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f253b400640 1 -- 172.21.15.46:0/2976083123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300b0070 con 0x7f253c145310 2024-04-25T08:46:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f253b400640 1 -- 172.21.15.46:0/2976083123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300aba10 con 0x7f253c145310 2024-04-25T08:46:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 -- 172.21.15.46:0/2976083123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c145310 msgr2=0x7f253c06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 --2- 172.21.15.46:0/2976083123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c145310 0x7f253c06e0d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2530009870 tx=0x7f253001c560 comp rx=0 tx=0).stop 2024-04-25T08:46:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 -- 172.21.15.46:0/2976083123 shutdown_connections 2024-04-25T08:46:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 --2- 172.21.15.46:0/2976083123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c15d090 0x7f253c15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 --2- 172.21.15.46:0/2976083123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c145310 0x7f253c06e0d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 -- 172.21.15.46:0/2976083123 >> 172.21.15.46:0/2976083123 conn(0x7f253c06ac10 msgr2=0x7f253c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 -- 172.21.15.46:0/2976083123 shutdown_connections 2024-04-25T08:46:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 -- 172.21.15.46:0/2976083123 wait complete. 2024-04-25T08:46:27.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 Processor -- start 2024-04-25T08:46:27.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.425+0000 7f2543a00640 1 -- start start 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2543a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c145310 0x7f253c141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2543a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15d090 0x7f253c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2543a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c0b38c0 0x7f253c0a87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2543a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c168510 con 0x7f253c15d090 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2543a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0cf380 con 0x7f253c0b38c0 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2543a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c168690 con 0x7f253c145310 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f253be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15d090 0x7f253c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f253be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15d090 0x7f253c0b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55874/0 (socket says 172.21.15.46:55874) 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f253be00640 1 -- 172.21.15.46:0/2191121270 learned_addr learned my addr 172.21.15.46:0/2191121270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2541000640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c145310 0x7f253c141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2541a00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c0b38c0 0x7f253c0a87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f253be00640 1 -- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c145310 msgr2=0x7f253c141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f253be00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c145310 0x7f253c141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f253be00640 1 -- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c0b38c0 msgr2=0x7f253c0a87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f253be00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c0b38c0 0x7f253c0a87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f253be00640 1 -- 172.21.15.46:0/2191121270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0a9b10 con 0x7f253c15d090 2024-04-25T08:46:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f253be00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15d090 0x7f253c0b3380 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f252800c450 tx=0x7f252800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2538c00640 1 -- 172.21.15.46:0/2191121270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528016020 con 0x7f253c15d090 2024-04-25T08:46:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2538c00640 1 -- 172.21.15.46:0/2191121270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25280052d0 con 0x7f253c15d090 2024-04-25T08:46:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2538c00640 1 -- 172.21.15.46:0/2191121270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528007260 con 0x7f253c15d090 2024-04-25T08:46:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2543a00640 1 -- 172.21.15.46:0/2191121270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c0a9df0 con 0x7f253c15d090 2024-04-25T08:46:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2543a00640 1 -- 172.21.15.46:0/2191121270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c0aa3b0 con 0x7f253c15d090 2024-04-25T08:46:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.429+0000 7f2538c00640 1 -- 172.21.15.46:0/2191121270 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2528007400 con 0x7f253c15d090 2024-04-25T08:46:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2538c00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2538c00640 1 -- 172.21.15.46:0/2191121270 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2528084a80 con 0x7f253c15d090 2024-04-25T08:46:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2546e9f000 1 Processor -- start 2024-04-25T08:46:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2546e9f000 1 -- start start 2024-04-25T08:46:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f25300af000 tx=0x7f25300a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665890db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665846b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 0x55a665847480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2546e9f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a665693ff0 con 0x55a6658909d0 2024-04-25T08:46:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2546e9f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66561c150 con 0x55a6658912f0 2024-04-25T08:46:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2546e9f000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a665690090 con 0x55a6658470a0 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665890db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665890db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55884/0 (socket says 172.21.15.46:55884) 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 -- 172.21.15.46:0/311056917 learned_addr learned my addr 172.21.15.46:0/311056917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f253be00640 1 --2- 172.21.15.46:0/311056917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665846b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541a00640 1 --2- 172.21.15.46:0/311056917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 0x55a665847480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 -- 172.21.15.46:0/311056917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 msgr2=0x55a665847480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 --2- 172.21.15.46:0/311056917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 0x55a665847480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 -- 172.21.15.46:0/311056917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 msgr2=0x55a665846b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 --2- 172.21.15.46:0/311056917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665846b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 -- 172.21.15.46:0/311056917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a66582a0d0 con 0x55a6658909d0 2024-04-25T08:46:27.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f2541000640 1 --2- 172.21.15.46:0/311056917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665890db0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f25300ad040 tx=0x7f2530004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeaf28bb207d3e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.433+0000 7f252d600640 1 -- 172.21.15.46:0/311056917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300c9630 con 0x55a6658909d0 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f252d600640 1 -- 172.21.15.46:0/311056917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300b0040 con 0x55a6658909d0 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f252d600640 1 -- 172.21.15.46:0/311056917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300c99d0 con 0x55a6658909d0 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- 172.21.15.46:0/311056917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 msgr2=0x55a665890db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 --2- 172.21.15.46:0/311056917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665890db0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f25300ad040 tx=0x7f2530004740 comp rx=0 tx=0).stop 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- 172.21.15.46:0/311056917 shutdown_connections 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 --2- 172.21.15.46:0/311056917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 0x55a665847480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 --2- 172.21.15.46:0/311056917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665846b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 --2- 172.21.15.46:0/311056917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665890db0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- 172.21.15.46:0/311056917 >> 172.21.15.46:0/311056917 conn(0x55a665843020 msgr2=0x55a665845440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- 172.21.15.46:0/311056917 shutdown_connections 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- 172.21.15.46:0/311056917 wait complete. 2024-04-25T08:46:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- start start 2024-04-25T08:46:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 Processor -- start 2024-04-25T08:46:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 0x55a6658757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665875d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665876270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a665658250 con 0x55a6658909d0 2024-04-25T08:46:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66564e690 con 0x55a6658912f0 2024-04-25T08:46:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a66561ca40 con 0x55a6658470a0 2024-04-25T08:46:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 10 client.0 _ll_register_callbacks cb 0x55a665831e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:46:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f253be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665875d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f253be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665875d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55890/0 (socket says 172.21.15.46:55890) 2024-04-25T08:46:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f253be00640 1 -- 172.21.15.46:0/3129198878 learned_addr learned my addr 172.21.15.46:0/3129198878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2541a00640 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665876270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2541000640 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 0x55a6658757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f253be00640 1 -- 172.21.15.46:0/3129198878 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 msgr2=0x55a6658757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f253be00640 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 0x55a6658757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f253be00640 1 -- 172.21.15.46:0/3129198878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 msgr2=0x55a665876270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f253be00640 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665876270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f253be00640 1 -- 172.21.15.46:0/3129198878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6657411c0 con 0x55a6658909d0 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f253be00640 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665875d30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f252800c420 tx=0x7f2528055390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22102118199d165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2522000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2522000640 1 -- 172.21.15.46:0/3129198878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528053370 con 0x55a6658909d0 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2522000640 1 -- 172.21.15.46:0/3129198878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f25280535d0 con 0x55a6658909d0 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- 172.21.15.46:0/3129198878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25300b0040 con 0x55a6658909d0 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2522000640 1 -- 172.21.15.46:0/3129198878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2528073290 con 0x55a6658909d0 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 10 client.6534 fetch_fsmap learned FSMap version 36 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.437+0000 7f2546e9f000 1 -- 172.21.15.46:0/3129198878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55a66573d170 con 0x55a6658909d0 2024-04-25T08:46:27.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2522000640 1 -- 172.21.15.46:0/3129198878 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300b0040 con 0x55a6658909d0 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2522000640 1 -- 172.21.15.46:0/3129198878 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2528051940 con 0x55a6658909d0 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 10 client.6534 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 10 client.6534 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 20 client.6534 populate_metadata read hostname 'smithi046' 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6658a0c80 0x55a6658a3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 4 client.6534 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 -- 172.21.15.46:0/3129198878 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a66562fc60 con 0x55a6658a0c80 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2541000640 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6658a0c80 0x55a6658a3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2541000640 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6658a0c80 0x55a6658a3070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5851dc78eb7d072b server_cookie=ba60e423f4c32ac6 in_seq=0 out_seq=0 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2522000640 10 client.6534 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2522000640 1 -- 172.21.15.46:0/3129198878 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a66562fc60 con 0x55a6658a0c80 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2522000640 10 client.6534 handle_command_reply: tid=0 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 client.6534 shutdown 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 -- 172.21.15.46:0/3129198878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 msgr2=0x55a665875d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665875d30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f252800c420 tx=0x7f2528055390 comp rx=0 tx=0).stop 2024-04-25T08:46:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 -- 172.21.15.46:0/3129198878 shutdown_connections 2024-04-25T08:46:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6658a0c80 0x55a6658a3070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665876270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658909d0 0x55a665875d30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 --2- 172.21.15.46:0/3129198878 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658470a0 0x55a6658757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 -- 172.21.15.46:0/3129198878 >> 172.21.15.46:0/3129198878 conn(0x55a665843020 msgr2=0x55a665845ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2522000640 0 client.6534 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:46:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 -- 172.21.15.46:0/3129198878 shutdown_connections 2024-04-25T08:46:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 1 -- 172.21.15.46:0/3129198878 wait complete. 2024-04-25T08:46:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.441+0000 7f2546e9f000 20 client.6534 trim_cache size 0 max 16384 2024-04-25T08:46:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 Processor -- start 2024-04-25T08:46:27.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 -- start start 2024-04-25T08:46:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658470a0 0x55a66588fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 0x55a665846420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665846960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c2036c0 con 0x55a6658470a0 2024-04-25T08:46:27.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c2028a0 con 0x55a6658912f0 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c0ad0e0 con 0x55a6658909d0 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658470a0 0x55a66588fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658470a0 0x55a66588fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55892/0 (socket says 172.21.15.46:55892) 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541000640 1 -- 172.21.15.46:0/2435867546 learned_addr learned my addr 172.21.15.46:0/2435867546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f253be00640 1 --2- 172.21.15.46:0/2435867546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 0x55a665846420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541a00640 1 --2- 172.21.15.46:0/2435867546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665846960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541000640 1 -- 172.21.15.46:0/2435867546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 msgr2=0x55a665846420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541000640 1 --2- 172.21.15.46:0/2435867546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 0x55a665846420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541000640 1 -- 172.21.15.46:0/2435867546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 msgr2=0x55a665846960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541000640 1 --2- 172.21.15.46:0/2435867546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665846960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541000640 1 -- 172.21.15.46:0/2435867546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6656dade0 con 0x55a6658470a0 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2541000640 1 --2- 172.21.15.46:0/2435867546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658470a0 0x55a66588fe10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f253001ca40 tx=0x7f2530008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24ef76df3749a7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f252cc00640 1 -- 172.21.15.46:0/2435867546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300091c0 con 0x55a6658470a0 2024-04-25T08:46:27.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f252cc00640 1 -- 172.21.15.46:0/2435867546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300b0040 con 0x55a6658470a0 2024-04-25T08:46:27.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f252cc00640 1 -- 172.21.15.46:0/2435867546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300d7320 con 0x55a6658470a0 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 -- 172.21.15.46:0/2435867546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658470a0 msgr2=0x55a66588fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 --2- 172.21.15.46:0/2435867546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658470a0 0x55a66588fe10 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f253001ca40 tx=0x7f2530008040 comp rx=0 tx=0).stop 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 -- 172.21.15.46:0/2435867546 shutdown_connections 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 --2- 172.21.15.46:0/2435867546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658912f0 0x55a665846960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 --2- 172.21.15.46:0/2435867546 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 0x55a665846420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 --2- 172.21.15.46:0/2435867546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658470a0 0x55a66588fe10 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 -- 172.21.15.46:0/2435867546 >> 172.21.15.46:0/2435867546 conn(0x55a665843020 msgr2=0x55a66588f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 -- 172.21.15.46:0/2435867546 shutdown_connections 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.457+0000 7f2546e9f000 1 -- 172.21.15.46:0/2435867546 wait complete. 2024-04-25T08:46:27.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 -- start start 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 Processor -- start 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658470a0 0x55a6658f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 0x55a6658f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658912f0 0x55a6658a3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a66562ee00 con 0x55a6658912f0 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c2022a0 con 0x55a6658470a0 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a665693300 con 0x55a6658909d0 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 10 client.0 _ll_register_callbacks cb 0x55a665812040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658912f0 0x55a6658a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658912f0 0x55a6658a3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55900/0 (socket says 172.21.15.46:55900) 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541a00640 1 -- 172.21.15.46:0/1402175732 learned_addr learned my addr 172.21.15.46:0/1402175732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541000640 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658470a0 0x55a6658f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f253be00640 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 0x55a6658f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541a00640 1 -- 172.21.15.46:0/1402175732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 msgr2=0x55a6658f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:27.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541a00640 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 0x55a6658f1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541a00640 1 -- 172.21.15.46:0/1402175732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658470a0 msgr2=0x55a6658f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541a00640 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658470a0 0x55a6658f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541a00640 1 -- 172.21.15.46:0/1402175732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6656358e0 con 0x55a6658912f0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541000640 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658470a0 0x55a6658f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541a00640 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658912f0 0x55a6658a3380 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f25340042d0 tx=0x7f2534004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8a7e7db4c41cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2522000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2522000640 1 -- 172.21.15.46:0/1402175732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253401d070 con 0x55a6658912f0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2522000640 1 -- 172.21.15.46:0/1402175732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2534017ac0 con 0x55a6658912f0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 -- 172.21.15.46:0/1402175732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25300b0040 con 0x55a6658912f0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2522000640 1 -- 172.21.15.46:0/1402175732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f253402a050 con 0x55a6658912f0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 10 client.6540 fetch_fsmap learned FSMap version 36 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 -- 172.21.15.46:0/1402175732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x55a6656ed8f0 con 0x55a6658912f0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2522000640 1 -- 172.21.15.46:0/1402175732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300b0040 con 0x55a6658912f0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2522000640 1 -- 172.21.15.46:0/1402175732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f253402d020 con 0x55a6658912f0 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 10 client.6540 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T08:46:27.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 10 client.6540 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:46:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 20 client.6540 populate_metadata read hostname 'smithi046' 2024-04-25T08:46:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6658a40d0 0x55a6658a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 4 client.6540 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:46:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2546e9f000 1 -- 172.21.15.46:0/1402175732 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a6656a1530 con 0x55a6658a40d0 2024-04-25T08:46:27.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.461+0000 7f2541000640 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6658a40d0 0x55a6658a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2541000640 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6658a40d0 0x55a6658a44e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd8c8930f50e930 server_cookie=574051cea275b149 in_seq=0 out_seq=0 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2522000640 10 client.6540 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2522000640 1 -- 172.21.15.46:0/1402175732 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a6656a1530 con 0x55a6658a40d0 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2522000640 10 client.6540 handle_command_reply: tid=0 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 client.6540 shutdown 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 -- 172.21.15.46:0/1402175732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658912f0 msgr2=0x55a6658a3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658912f0 0x55a6658a3380 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f25340042d0 tx=0x7f2534004790 comp rx=0 tx=0).stop 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 -- 172.21.15.46:0/1402175732 shutdown_connections 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x55a6658a40d0 0x55a6658a44e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a6658912f0 0x55a6658a3380 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a6658909d0 0x55a6658f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 --2- 172.21.15.46:0/1402175732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a6658470a0 0x55a6658f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2522000640 0 client.6540 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 -- 172.21.15.46:0/1402175732 >> 172.21.15.46:0/1402175732 conn(0x55a665843020 msgr2=0x55a66588f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 -- 172.21.15.46:0/1402175732 shutdown_connections 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 1 -- 172.21.15.46:0/1402175732 wait complete. 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2546e9f000 20 client.6540 trim_cache size 0 max 16384 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:46:27.462 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:46:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 -- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 msgr2=0x7f251c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f25300af000 tx=0x7f25300a7000 comp rx=0 tx=0).stop 2024-04-25T08:46:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 -- 172.21.15.46:0/2191121270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15d090 msgr2=0x7f253c0b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15d090 0x7f253c0b3380 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f252800c450 tx=0x7f252800c910 comp rx=0 tx=0).stop 2024-04-25T08:46:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 -- 172.21.15.46:0/2191121270 shutdown_connections 2024-04-25T08:46:27.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c0b38c0 0x7f253c0a87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15d090 0x7f253c0b3380 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 --2- 172.21.15.46:0/2191121270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c145310 0x7f253c141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 -- 172.21.15.46:0/2191121270 >> 172.21.15.46:0/2191121270 conn(0x7f253c06ac10 msgr2=0x7f253c15ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 -- 172.21.15.46:0/2191121270 shutdown_connections 2024-04-25T08:46:27.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.465+0000 7f2543a00640 1 -- 172.21.15.46:0/2191121270 wait complete. 2024-04-25T08:46:27.478 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:46:27.478 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2439600640 1 Processor -- start 2024-04-25T08:46:27.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2439600640 1 -- start start 2024-04-25T08:46:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2439600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 0x7f243415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2439600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2434161330 con 0x7f2434160f50 2024-04-25T08:46:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2439600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24340ae810 con 0x7f24340ae360 2024-04-25T08:46:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2439600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243406d680 con 0x7f243406f110 2024-04-25T08:46:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2432a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2434160f50 0x7f24340adc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46630/0 (socket says 172.21.15.46:46630) 2024-04-25T08:46:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2432a00640 1 -- 172.21.15.46:0/2778395635 learned_addr learned my addr 172.21.15.46:0/2778395635 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2433400640 1 --2- 172.21.15.46:0/2778395635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 0x7f243415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2432000640 1 -- 172.21.15.46:0/2778395635 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505789680 0 0) 0x7f2434161330 con 0x7f2434160f50 2024-04-25T08:46:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2432000640 1 -- 172.21.15.46:0/2778395635 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2418003530 con 0x7f2434160f50 2024-04-25T08:46:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2432000640 1 -- 172.21.15.46:0/2778395635 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907932162 0 0) 0x7f24340ae810 con 0x7f24340ae360 2024-04-25T08:46:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.625+0000 7f2432000640 1 -- 172.21.15.46:0/2778395635 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2434161330 con 0x7f24340ae360 2024-04-25T08:46:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2433400640 1 -- 172.21.15.46:0/2778395635 >> v1:172.21.15.82:6789/0 conn(0x7f24340ae360 legacy=0x7f243406ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2433400640 1 -- 172.21.15.46:0/2778395635 >> v1:172.21.15.46:6789/0 conn(0x7f2434160f50 legacy=0x7f24340adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2433400640 1 -- 172.21.15.46:0/2778395635 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24340aeaf0 con 0x7f243406f110 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2433400640 1 --2- 172.21.15.46:0/2778395635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 0x7f243415f1a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f242c009dc0 tx=0x7f242c01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcf9b8c069bdbec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2432000640 1 -- 172.21.15.46:0/2778395635 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0b2070 con 0x7f243406f110 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2432000640 1 -- 172.21.15.46:0/2778395635 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c0a5a40 con 0x7f243406f110 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2432000640 1 -- 172.21.15.46:0/2778395635 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0a5e10 con 0x7f243406f110 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2439600640 1 -- 172.21.15.46:0/2778395635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 msgr2=0x7f243415f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2439600640 1 --2- 172.21.15.46:0/2778395635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 0x7f243415f1a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f242c009dc0 tx=0x7f242c01c960 comp rx=0 tx=0).stop 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2439600640 1 -- 172.21.15.46:0/2778395635 shutdown_connections 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2439600640 1 --2- 172.21.15.46:0/2778395635 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 0x7f243415f1a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2439600640 1 -- 172.21.15.46:0/2778395635 >> 172.21.15.46:0/2778395635 conn(0x7f24340b12a0 msgr2=0x7f24340b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2439600640 1 -- 172.21.15.46:0/2778395635 shutdown_connections 2024-04-25T08:46:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.629+0000 7f2439600640 1 -- 172.21.15.46:0/2778395635 wait complete. 2024-04-25T08:46:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2439600640 1 Processor -- start 2024-04-25T08:46:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2439600640 1 -- start start 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2439600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 0x7f24340cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2439600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340ae360 0x7f24341328d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2439600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2434160f50 0x7f2434132e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2439600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243406e480 con 0x7f2434160f50 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2439600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243406e300 con 0x7f24340ae360 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2439600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243406e600 con 0x7f243406f110 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2433400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2434160f50 0x7f2434132e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2433400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2434160f50 0x7f2434132e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55902/0 (socket says 172.21.15.46:55902) 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2433400640 1 -- 172.21.15.46:0/1412594119 learned_addr learned my addr 172.21.15.46:0/1412594119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2432a00640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 0x7f24340cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f242a000640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340ae360 0x7f24341328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2433400640 1 -- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 msgr2=0x7f24340cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2433400640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 0x7f24340cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2433400640 1 -- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340ae360 msgr2=0x7f24341328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2433400640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340ae360 0x7f24341328d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2433400640 1 -- 172.21.15.46:0/1412594119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243414eb30 con 0x7f2434160f50 2024-04-25T08:46:27.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f2433400640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2434160f50 0x7f2434132e10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f242c019570 tx=0x7f242c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.633+0000 7f242b400640 1 -- 172.21.15.46:0/1412594119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0b2070 con 0x7f2434160f50 2024-04-25T08:46:27.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f242b400640 1 -- 172.21.15.46:0/1412594119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c0bf070 con 0x7f2434160f50 2024-04-25T08:46:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243414ee10 con 0x7f2434160f50 2024-04-25T08:46:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f242b400640 1 -- 172.21.15.46:0/1412594119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0b9480 con 0x7f2434160f50 2024-04-25T08:46:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243414f3d0 con 0x7f2434160f50 2024-04-25T08:46:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2434069080 con 0x7f2434160f50 2024-04-25T08:46:27.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f242b400640 1 -- 172.21.15.46:0/1412594119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f242c0b9620 con 0x7f2434160f50 2024-04-25T08:46:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f242b400640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2410066f60 0x7f2410069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f242b400640 1 -- 172.21.15.46:0/1412594119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f242c1204e0 con 0x7f2434160f50 2024-04-25T08:46:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f2432a00640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2410066f60 0x7f2410069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.637+0000 7f2432a00640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2410066f60 0x7f2410069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2424000f70 tx=0x7f2424009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:27.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.641+0000 7f242b400640 1 -- 172.21.15.46:0/1412594119 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f242c016580 con 0x7f2434160f50 2024-04-25T08:46:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.837+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24340b2430 con 0x7f2434160f50 2024-04-25T08:46:27.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f242b400640 1 -- 172.21.15.46:0/1412594119 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f242c0f0c90 con 0x7f2434160f50 2024-04-25T08:46:27.834 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:27.834 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:45:11.989847+0000","modified":"2024-04-25T08:46:00.651586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6207},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6207":{"gid":6207,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/1670658295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670658295},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670658295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:46:27.834 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2410066f60 msgr2=0x7f2410069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2410066f60 0x7f2410069420 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2424000f70 tx=0x7f2424009040 comp rx=0 tx=0).stop 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2434160f50 msgr2=0x7f2434132e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2434160f50 0x7f2434132e10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f242c019570 tx=0x7f242c019700 comp rx=0 tx=0).stop 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 shutdown_connections 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2410066f60 0x7f2410069420 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2434160f50 0x7f2434132e10 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340ae360 0x7f24341328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 --2- 172.21.15.46:0/1412594119 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f243406f110 0x7f24340cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 >> 172.21.15.46:0/1412594119 conn(0x7f24340b12a0 msgr2=0x7f24340738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 shutdown_connections 2024-04-25T08:46:27.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:27.841+0000 7f2439600640 1 -- 172.21.15.46:0/1412594119 wait complete. 2024-04-25T08:46:27.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:46:27.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff808e00640 1 Processor -- start 2024-04-25T08:46:27.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff808e00640 1 -- start start 2024-04-25T08:46:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff808e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff80406f150 0x7ff8040cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff808e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80415e440 con 0x7ff8041440d0 2024-04-25T08:46:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff808e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80415e6e0 con 0x7ff804143990 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff808e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80415ed30 con 0x7ff80406f150 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff7fbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff8041440d0 0x7ff8040b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46646/0 (socket says 172.21.15.46:46646) 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff7fbe00640 1 -- 172.21.15.46:0/2562400840 learned_addr learned my addr 172.21.15.46:0/2562400840 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff802800640 1 --2- 172.21.15.46:0/2562400840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff80406f150 0x7ff8040cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff801400640 1 -- 172.21.15.46:0/2562400840 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352470168 0 0) 0x7ff80415e440 con 0x7ff8041440d0 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff801400640 1 -- 172.21.15.46:0/2562400840 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e8003530 con 0x7ff8041440d0 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff801400640 1 -- 172.21.15.46:0/2562400840 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061588584 0 0) 0x7ff80415e6e0 con 0x7ff804143990 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff801400640 1 -- 172.21.15.46:0/2562400840 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff80415e440 con 0x7ff804143990 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff802800640 1 -- 172.21.15.46:0/2562400840 >> v1:172.21.15.82:6789/0 conn(0x7ff804143990 legacy=0x7ff80406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff802800640 1 -- 172.21.15.46:0/2562400840 >> v1:172.21.15.46:6789/0 conn(0x7ff8041440d0 legacy=0x7ff8040b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.005+0000 7ff802800640 1 -- 172.21.15.46:0/2562400840 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80415e9c0 con 0x7ff80406f150 2024-04-25T08:46:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff802800640 1 --2- 172.21.15.46:0/2562400840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff80406f150 0x7ff8040cecc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff80415eeb0 tx=0x7ff7fc01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc3a9ef9086658a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff801400640 1 -- 172.21.15.46:0/2562400840 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0b2070 con 0x7ff80406f150 2024-04-25T08:46:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff801400640 1 -- 172.21.15.46:0/2562400840 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7fc0a5850 con 0x7ff80406f150 2024-04-25T08:46:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff801400640 1 -- 172.21.15.46:0/2562400840 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc0a5c20 con 0x7ff80406f150 2024-04-25T08:46:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff808e00640 1 -- 172.21.15.46:0/2562400840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff80406f150 msgr2=0x7ff8040cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff808e00640 1 --2- 172.21.15.46:0/2562400840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff80406f150 0x7ff8040cecc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff80415eeb0 tx=0x7ff7fc01c990 comp rx=0 tx=0).stop 2024-04-25T08:46:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff808e00640 1 -- 172.21.15.46:0/2562400840 shutdown_connections 2024-04-25T08:46:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff808e00640 1 --2- 172.21.15.46:0/2562400840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff80406f150 0x7ff8040cecc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff808e00640 1 -- 172.21.15.46:0/2562400840 >> 172.21.15.46:0/2562400840 conn(0x7ff80406ac10 msgr2=0x7ff80406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff808e00640 1 -- 172.21.15.46:0/2562400840 shutdown_connections 2024-04-25T08:46:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.009+0000 7ff808e00640 1 -- 172.21.15.46:0/2562400840 wait complete. 2024-04-25T08:46:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 Processor -- start 2024-04-25T08:46:28.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 -- start start 2024-04-25T08:46:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff80406f150 0x7ff804138b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff804143990 0x7ff804141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff8041440d0 0x7ff8040b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8040713f0 con 0x7ff80406f150 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804071270 con 0x7ff804143990 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff804071570 con 0x7ff8041440d0 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff801e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff80406f150 0x7ff804138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff801e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff80406f150 0x7ff804138b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55918/0 (socket says 172.21.15.46:55918) 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff801e00640 1 -- 172.21.15.46:0/25311382 learned_addr learned my addr 172.21.15.46:0/25311382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff804143990 0x7ff804141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff802800640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff8041440d0 0x7ff8040b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff801e00640 1 -- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff8041440d0 msgr2=0x7ff8040b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff801e00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff8041440d0 0x7ff8040b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff801e00640 1 -- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff804143990 msgr2=0x7ff804141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff801e00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff804143990 0x7ff804141520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff801e00640 1 -- 172.21.15.46:0/25311382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804138910 con 0x7ff80406f150 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff801e00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff80406f150 0x7ff804138b40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c530 tx=0x7ff7f400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff7fa000640 1 -- 172.21.15.46:0/25311382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40052d0 con 0x7ff80406f150 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff7fa000640 1 -- 172.21.15.46:0/25311382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f4005490 con 0x7ff80406f150 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff7fa000640 1 -- 172.21.15.46:0/25311382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4012850 con 0x7ff80406f150 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 -- 172.21.15.46:0/25311382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8040aff20 con 0x7ff80406f150 2024-04-25T08:46:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.013+0000 7ff808e00640 1 -- 172.21.15.46:0/25311382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8040b04a0 con 0x7ff80406f150 2024-04-25T08:46:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff7fa000640 1 -- 172.21.15.46:0/25311382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7f40129f0 con 0x7ff80406f150 2024-04-25T08:46:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff7fa000640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff7e0066f90 0x7ff7e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff7fa000640 1 -- 172.21.15.46:0/25311382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff7f4084dc0 con 0x7ff80406f150 2024-04-25T08:46:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff80c250000 1 Processor -- start 2024-04-25T08:46:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff7e0066f90 0x7ff7e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff80c250000 1 -- start start 2024-04-25T08:46:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff7e0066f90 0x7ff7e0069450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff804142cd0 tx=0x7ff7ec007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff80c250000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 0x5637727c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff80c250000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c1c00 0x5637727773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff80c250000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727778e0 0x563772777cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff80c250000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56377265eba0 con 0x5637727c1c00 2024-04-25T08:46:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff80c250000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637725c4090 con 0x5637727778e0 2024-04-25T08:46:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff80c250000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x56377254c160 con 0x5637727c12e0 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c1c00 0x5637727773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c1c00 0x5637727773a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55928/0 (socket says 172.21.15.46:55928) 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff7fbe00640 1 -- 172.21.15.46:0/3030872647 learned_addr learned my addr 172.21.15.46:0/3030872647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff802800640 1 --2- 172.21.15.46:0/3030872647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727778e0 0x563772777cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.017+0000 7ff801e00640 1 --2- 172.21.15.46:0/3030872647 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 0x5637727c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 -- 172.21.15.46:0/3030872647 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 msgr2=0x5637727c16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/3030872647 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 0x5637727c16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 -- 172.21.15.46:0/3030872647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727778e0 msgr2=0x563772777cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/3030872647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727778e0 0x563772777cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 -- 172.21.15.46:0/3030872647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56377266dc20 con 0x5637727c1c00 2024-04-25T08:46:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/3030872647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c1c00 0x5637727773a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec007fd0 tx=0x7ff7ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a25c686039f1a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7f2a00640 1 -- 172.21.15.46:0/3030872647 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec026070 con 0x5637727c1c00 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7f2a00640 1 -- 172.21.15.46:0/3030872647 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec01b050 con 0x5637727c1c00 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7f2a00640 1 -- 172.21.15.46:0/3030872647 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec014bd0 con 0x5637727c1c00 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 -- 172.21.15.46:0/3030872647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c1c00 msgr2=0x5637727773a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 --2- 172.21.15.46:0/3030872647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c1c00 0x5637727773a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec007fd0 tx=0x7ff7ec012000 comp rx=0 tx=0).stop 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 -- 172.21.15.46:0/3030872647 shutdown_connections 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 --2- 172.21.15.46:0/3030872647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727778e0 0x563772777cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 --2- 172.21.15.46:0/3030872647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c1c00 0x5637727773a0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 --2- 172.21.15.46:0/3030872647 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 0x5637727c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 -- 172.21.15.46:0/3030872647 >> 172.21.15.46:0/3030872647 conn(0x563772773890 msgr2=0x563772775c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 -- 172.21.15.46:0/3030872647 shutdown_connections 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 -- 172.21.15.46:0/3030872647 wait complete. 2024-04-25T08:46:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 -- start start 2024-04-25T08:46:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 Processor -- start 2024-04-25T08:46:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 0x5637727c8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x5637727c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x5637727af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637725c0130 con 0x5637727c12e0 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563772588130 con 0x5637727c1c00 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563772682300 con 0x5637727778e0 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff80c250000 10 client.0 _ll_register_callbacks cb 0x563772762bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x5637727c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x5637727c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55938/0 (socket says 172.21.15.46:55938) 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 -- 172.21.15.46:0/1410659825 learned_addr learned my addr 172.21.15.46:0/1410659825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff801e00640 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 0x5637727c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff802800640 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x5637727af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 -- 172.21.15.46:0/1410659825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 msgr2=0x5637727c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 0x5637727c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 -- 172.21.15.46:0/1410659825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 msgr2=0x5637727af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x5637727af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff7fbe00640 1 -- 172.21.15.46:0/1410659825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637725a1060 con 0x5637727c12e0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.021+0000 7ff802800640 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x5637727af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x5637727c8b80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x5637725c7150 tx=0x7ff7ec000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b66e1b7f15480b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 1 -- 172.21.15.46:0/1410659825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec026070 con 0x5637727c12e0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 1 -- 172.21.15.46:0/1410659825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff7ec02dc70 con 0x5637727c12e0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 -- 172.21.15.46:0/1410659825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7ec01b050 con 0x5637727c12e0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 1 -- 172.21.15.46:0/1410659825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ec005e20 con 0x5637727c12e0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 10 client.6558 fetch_fsmap learned FSMap version 36 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 -- 172.21.15.46:0/1410659825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x5637725657c0 con 0x5637727c12e0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 1 -- 172.21.15.46:0/1410659825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec01b050 con 0x5637727c12e0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 1 -- 172.21.15.46:0/1410659825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff7ec02d410 con 0x5637727c12e0 2024-04-25T08:46:28.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 10 client.6558 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 10 client.6558 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 20 client.6558 populate_metadata read hostname 'smithi046' 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5637727afe00 0x5637727b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 4 client.6558 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 -- 172.21.15.46:0/1410659825 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56377255fcd0 con 0x5637727afe00 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff801e00640 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5637727afe00 0x5637727b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff801e00640 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5637727afe00 0x5637727b01e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178e9407c4f73c21 server_cookie=da7660d9b9cafd98 in_seq=0 out_seq=0 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 10 client.6558 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 1 -- 172.21.15.46:0/1410659825 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56377255fcd0 con 0x5637727afe00 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 10 client.6558 handle_command_reply: tid=0 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 client.6558 shutdown 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 -- 172.21.15.46:0/1410659825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 msgr2=0x5637727c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x5637727c8b80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x5637725c7150 tx=0x7ff7ec000ee0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 -- 172.21.15.46:0/1410659825 shutdown_connections 2024-04-25T08:46:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5637727afe00 0x5637727b01e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x5637727af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x5637727c8b80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff7e7400640 0 client.6558 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:46:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 --2- 172.21.15.46:0/1410659825 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 0x5637727c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 -- 172.21.15.46:0/1410659825 >> 172.21.15.46:0/1410659825 conn(0x563772773890 msgr2=0x563772776510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 -- 172.21.15.46:0/1410659825 shutdown_connections 2024-04-25T08:46:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 1 -- 172.21.15.46:0/1410659825 wait complete. 2024-04-25T08:46:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.025+0000 7ff80c250000 20 client.6558 trim_cache size 0 max 16384 2024-04-25T08:46:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.037+0000 7ff80c250000 1 Processor -- start 2024-04-25T08:46:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.037+0000 7ff80c250000 1 -- start start 2024-04-25T08:46:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727778e0 0x56377281d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 0x56377281da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x56377281df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804202760 con 0x5637727778e0 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804136970 con 0x5637727c1c00 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff804202160 con 0x5637727c12e0 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff801e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727778e0 0x56377281d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff801e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727778e0 0x56377281d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55952/0 (socket says 172.21.15.46:55952) 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff801e00640 1 -- 172.21.15.46:0/2668536967 learned_addr learned my addr 172.21.15.46:0/2668536967 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 0x56377281da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff802800640 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x56377281df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff801e00640 1 -- 172.21.15.46:0/2668536967 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 msgr2=0x56377281da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff801e00640 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 0x56377281da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff801e00640 1 -- 172.21.15.46:0/2668536967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 msgr2=0x56377281df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff801e00640 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x56377281df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff801e00640 1 -- 172.21.15.46:0/2668536967 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56377257ffc0 con 0x5637727778e0 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff802800640 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x56377281df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:46:28.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff801e00640 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727778e0 0x56377281d500 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40551f0 tx=0x7ff7f4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cce56f6aa264a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff7f2000640 1 -- 172.21.15.46:0/2668536967 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40537f0 con 0x5637727778e0 2024-04-25T08:46:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff7f2000640 1 -- 172.21.15.46:0/2668536967 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f4073040 con 0x5637727778e0 2024-04-25T08:46:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff7f2000640 1 -- 172.21.15.46:0/2668536967 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4012ce0 con 0x5637727778e0 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- 172.21.15.46:0/2668536967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727778e0 msgr2=0x56377281d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727778e0 0x56377281d500 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff7f40551f0 tx=0x7ff7f4000ee0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- 172.21.15.46:0/2668536967 shutdown_connections 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x56377281df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727c12e0 0x56377281da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- 172.21.15.46:0/2668536967 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727778e0 0x56377281d500 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- 172.21.15.46:0/2668536967 >> 172.21.15.46:0/2668536967 conn(0x563772773890 msgr2=0x5637727c0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- 172.21.15.46:0/2668536967 shutdown_connections 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- 172.21.15.46:0/2668536967 wait complete. 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- start start 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 Processor -- start 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 0x563772821510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x563772821a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x5637727c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56377255ee70 con 0x5637727c12e0 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56377254ca50 con 0x5637727c1c00 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5637725c33a0 con 0x5637727778e0 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff80c250000 10 client.0 _ll_register_callbacks cb 0x563772570240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.041+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x563772821a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x563772821a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55958/0 (socket says 172.21.15.46:55958) 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7fbe00640 1 -- 172.21.15.46:0/1102016910 learned_addr learned my addr 172.21.15.46:0/1102016910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff802800640 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x5637727c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff801e00640 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 0x563772821510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7fbe00640 1 -- 172.21.15.46:0/1102016910 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 msgr2=0x563772821510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 0x563772821510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7fbe00640 1 -- 172.21.15.46:0/1102016910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 msgr2=0x5637727c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x5637727c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7fbe00640 1 -- 172.21.15.46:0/1102016910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56377261d9e0 con 0x5637727c12e0 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7fbe00640 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x563772821a50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec008010 tx=0x7ff7ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a68ce4ea3df211d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 1 -- 172.21.15.46:0/1102016910 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec03c070 con 0x5637727c12e0 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 1 -- 172.21.15.46:0/1102016910 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff7ec041cc0 con 0x5637727c12e0 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 -- 172.21.15.46:0/1102016910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7f4073040 con 0x5637727c12e0 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 1 -- 172.21.15.46:0/1102016910 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ec026070 con 0x5637727c12e0 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 10 client.6564 fetch_fsmap learned FSMap version 36 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 -- 172.21.15.46:0/1102016910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x563772563630 con 0x5637727c12e0 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 1 -- 172.21.15.46:0/1102016910 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f4073040 con 0x5637727c12e0 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 1 -- 172.21.15.46:0/1102016910 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 36) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff7ec0047f0 con 0x5637727c12e0 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 10 client.6564 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 10 client.6564 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 20 client.6564 populate_metadata read hostname 'smithi046' 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5637727c6310 0x5637727c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 4 client.6564 mds_command: new command op to 6207 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 -- 172.21.15.46:0/1102016910 --> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5637725d15d0 con 0x5637727c6310 2024-04-25T08:46:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff801e00640 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5637727c6310 0x5637727c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff801e00640 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5637727c6310 0x5637727c66f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae87264b9bb1d73 server_cookie=3be935eda803218d in_seq=0 out_seq=0 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 10 client.6564 ms_handle_connect on v2:172.21.15.46:6834/1670658295 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 1 -- 172.21.15.46:0/1102016910 <== mds.0 v2:172.21.15.46:6834/1670658295 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637725d15d0 con 0x5637727c6310 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 10 client.6564 handle_command_reply: tid=0 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 client.6564 shutdown 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 -- 172.21.15.46:0/1102016910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 msgr2=0x563772821a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x563772821a50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec008010 tx=0x7ff7ec016000 comp rx=0 tx=0).stop 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 -- 172.21.15.46:0/1102016910 shutdown_connections 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.46:6834/1670658295,v1:172.21.15.46:6835/1670658295] conn(0x5637727c6310 0x5637727c66f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5637727c1c00 0x5637727c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637727c12e0 0x563772821a50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff7e7400640 0 client.6564 ms_handle_reset on v2:172.21.15.46:6834/1670658295 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 --2- 172.21.15.46:0/1102016910 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5637727778e0 0x563772821510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 -- 172.21.15.46:0/1102016910 >> 172.21.15.46:0/1102016910 conn(0x563772773890 msgr2=0x5637727c0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:28.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 -- 172.21.15.46:0/1102016910 shutdown_connections 2024-04-25T08:46:28.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 1 -- 172.21.15.46:0/1102016910 wait complete. 2024-04-25T08:46:28.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff80c250000 20 client.6564 trim_cache size 0 max 16384 2024-04-25T08:46:28.044 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:28.044 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-25T08:46:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 -- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff7e0066f90 msgr2=0x7ff7e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff7e0066f90 0x7ff7e0069450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff804142cd0 tx=0x7ff7ec007d40 comp rx=0 tx=0).stop 2024-04-25T08:46:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 -- 172.21.15.46:0/25311382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff80406f150 msgr2=0x7ff804138b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff80406f150 0x7ff804138b40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400c530 tx=0x7ff7f400c9f0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 -- 172.21.15.46:0/25311382 shutdown_connections 2024-04-25T08:46:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff7e0066f90 0x7ff7e0069450 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff8041440d0 0x7ff8040b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff804143990 0x7ff804141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 --2- 172.21.15.46:0/25311382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff80406f150 0x7ff804138b40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 -- 172.21.15.46:0/25311382 >> 172.21.15.46:0/25311382 conn(0x7ff80406ac10 msgr2=0x7ff8040ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 -- 172.21.15.46:0/25311382 shutdown_connections 2024-04-25T08:46:28.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.045+0000 7ff808e00640 1 -- 172.21.15.46:0/25311382 wait complete. 2024-04-25T08:46:28.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:46:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.205+0000 7f00eac00640 1 Processor -- start 2024-04-25T08:46:28.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.205+0000 7f00eac00640 1 -- start start 2024-04-25T08:46:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00eac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f00e4161080 0x7f00e40cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00eac00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00e40cd2e0 0x7f00e40cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00eac00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e40cc520 con 0x7f00e40d0200 2024-04-25T08:46:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00eac00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e40cc2f0 con 0x7f00e40cd2e0 2024-04-25T08:46:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00eac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e40cce50 con 0x7f00e4161080 2024-04-25T08:46:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00dbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f00e40d0200 0x7f00e406bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34836/0 (socket says 172.21.15.46:34836) 2024-04-25T08:46:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00dbe00640 1 -- 172.21.15.46:0/2762241861 learned_addr learned my addr 172.21.15.46:0/2762241861 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:28.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00e8c00640 1 --2- 172.21.15.46:0/2762241861 >> v2:172.21.15.46:3300/0 conn(0x7f00e40cd2e0 0x7f00e40cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00e3e00640 1 --2- 172.21.15.46:0/2762241861 >> v2:172.21.15.82:3300/0 conn(0x7f00e4161080 0x7f00e40cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00e3400640 1 -- 172.21.15.46:0/2762241861 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865317695 0 0) 0x7f00e40cc520 con 0x7f00e40d0200 2024-04-25T08:46:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00e3400640 1 -- 172.21.15.46:0/2762241861 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00c8003530 con 0x7f00e40d0200 2024-04-25T08:46:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00e8c00640 1 -- 172.21.15.46:0/2762241861 >> v2:172.21.15.82:3300/0 conn(0x7f00e4161080 msgr2=0x7f00e40cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00e8c00640 1 --2- 172.21.15.46:0/2762241861 >> v2:172.21.15.82:3300/0 conn(0x7f00e4161080 0x7f00e40cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00e8c00640 1 -- 172.21.15.46:0/2762241861 >> v1:172.21.15.46:6789/0 conn(0x7f00e40d0200 legacy=0x7f00e406bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00e8c00640 1 -- 172.21.15.46:0/2762241861 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e40cc800 con 0x7f00e40cd2e0 2024-04-25T08:46:28.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.209+0000 7f00e8c00640 1 --2- 172.21.15.46:0/2762241861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e40cd2e0 0x7f00e40cd730 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f00e4072ab0 tx=0x7f00dc01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307e3081974cfef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00e3400640 1 -- 172.21.15.46:0/2762241861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0b2070 con 0x7f00e40cd2e0 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00e3400640 1 -- 172.21.15.46:0/2762241861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc0a59e0 con 0x7f00e40cd2e0 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00e3400640 1 -- 172.21.15.46:0/2762241861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0a5db0 con 0x7f00e40cd2e0 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 -- 172.21.15.46:0/2762241861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e40cd2e0 msgr2=0x7f00e40cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 --2- 172.21.15.46:0/2762241861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e40cd2e0 0x7f00e40cd730 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f00e4072ab0 tx=0x7f00dc01c900 comp rx=0 tx=0).stop 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 -- 172.21.15.46:0/2762241861 shutdown_connections 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 --2- 172.21.15.46:0/2762241861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e40cd2e0 0x7f00e40cd730 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 --2- 172.21.15.46:0/2762241861 >> v2:172.21.15.82:3300/0 conn(0x7f00e4161080 0x7f00e40cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 -- 172.21.15.46:0/2762241861 >> 172.21.15.46:0/2762241861 conn(0x7f00e40b31d0 msgr2=0x7f00e40b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 -- 172.21.15.46:0/2762241861 shutdown_connections 2024-04-25T08:46:28.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 -- 172.21.15.46:0/2762241861 wait complete. 2024-04-25T08:46:28.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 Processor -- start 2024-04-25T08:46:28.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 -- start start 2024-04-25T08:46:28.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cd2e0 0x7f00e415f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40d0200 0x7f00e40b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e4161080 0x7f00e4153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e41615e0 con 0x7f00e4161080 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e4161460 con 0x7f00e40d0200 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00eac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e40b2830 con 0x7f00e40cd2e0 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00e8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e4161080 0x7f00e4153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00e8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e4161080 0x7f00e4153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49086/0 (socket says 172.21.15.46:49086) 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00e8c00640 1 -- 172.21.15.46:0/58912683 learned_addr learned my addr 172.21.15.46:0/58912683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.213+0000 7f00dbe00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40d0200 0x7f00e40b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e3e00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cd2e0 0x7f00e415f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e8c00640 1 -- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cd2e0 msgr2=0x7f00e415f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e8c00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cd2e0 0x7f00e415f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e8c00640 1 -- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40d0200 msgr2=0x7f00e40b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e8c00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40d0200 0x7f00e40b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e8c00640 1 -- 172.21.15.46:0/58912683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e4156860 con 0x7f00e4161080 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e3e00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cd2e0 0x7f00e415f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:28.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00dbe00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40d0200 0x7f00e40b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e8c00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e4161080 0x7f00e4153c50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f00dc019510 tx=0x7f00dc0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e0c00640 1 -- 172.21.15.46:0/58912683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0b2050 con 0x7f00e4161080 2024-04-25T08:46:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e0c00640 1 -- 172.21.15.46:0/58912683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc0be070 con 0x7f00e4161080 2024-04-25T08:46:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e4156b40 con 0x7f00e4161080 2024-04-25T08:46:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e0c00640 1 -- 172.21.15.46:0/58912683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0ada90 con 0x7f00e4161080 2024-04-25T08:46:28.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e415fb00 con 0x7f00e4161080 2024-04-25T08:46:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e4074c60 con 0x7f00e4161080 2024-04-25T08:46:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e0c00640 1 -- 172.21.15.46:0/58912683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00dc0adc30 con 0x7f00e4161080 2024-04-25T08:46:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e0c00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f90 0x7f00c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.217+0000 7f00e0c00640 1 -- 172.21.15.46:0/58912683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f00dc11ff20 con 0x7f00e4161080 2024-04-25T08:46:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.221+0000 7f00e3e00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f90 0x7f00c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.221+0000 7f00e3e00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f90 0x7f00c0069450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009b10 tx=0x7f00d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.221+0000 7f00e0c00640 1 -- 172.21.15.46:0/58912683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00dc0c7020 con 0x7f00e4161080 2024-04-25T08:46:28.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.417+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f00e40b7660 con 0x7f00e4161080 2024-04-25T08:46:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.417+0000 7f00e0c00640 1 -- 172.21.15.46:0/58912683 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f00dc016520 con 0x7f00e4161080 2024-04-25T08:46:28.412 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:28.412 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:46:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f90 msgr2=0x7f00c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f90 0x7f00c0069450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00d4009b10 tx=0x7f00d4008040 comp rx=0 tx=0).stop 2024-04-25T08:46:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e4161080 msgr2=0x7f00e4153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e4161080 0x7f00e4153c50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f00dc019510 tx=0x7f00dc0196a0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 shutdown_connections 2024-04-25T08:46:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00c0066f90 0x7f00c0069450 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00e4161080 0x7f00e4153c50 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00e40d0200 0x7f00e40b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 --2- 172.21.15.46:0/58912683 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00e40cd2e0 0x7f00e415f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 >> 172.21.15.46:0/58912683 conn(0x7f00e40b31d0 msgr2=0x7f00e40706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 shutdown_connections 2024-04-25T08:46:28.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.421+0000 7f00eac00640 1 -- 172.21.15.46:0/58912683 wait complete. 2024-04-25T08:46:28.433 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:46:28.433 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:46:28.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:46:28.543 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:46:28.647 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:46:28.647 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:46:28.647 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:28.657 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:46:28.657 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:28.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.897+0000 7f4ba0600640 1 Processor -- start 2024-04-25T08:46:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.897+0000 7f4ba0600640 1 -- start start 2024-04-25T08:46:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4ba0600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b9806f320 0x7f4b98162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4ba0600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b980b35b0 con 0x7f4b9815a750 2024-04-25T08:46:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4ba0600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9806d5e0 con 0x7f4b980aed10 2024-04-25T08:46:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4ba0600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9815ab30 con 0x7f4b9806f320 2024-04-25T08:46:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9dc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4b9815a750 0x7f4b9806ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34842/0 (socket says 172.21.15.46:34842) 2024-04-25T08:46:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9dc00640 1 -- 172.21.15.46:0/3350771656 learned_addr learned my addr 172.21.15.46:0/3350771656 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9d200640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b9806f320 0x7f4b98162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775059618 0 0) 0x7f4b980b35b0 con 0x7f4b9815a750 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b80003530 con 0x7f4b9815a750 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 813847264 0 0) 0x7f4b80003530 con 0x7f4b9815a750 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b9806f320 msgr2=0x7f4b98162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b9806f320 0x7f4b98162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 >> v1:172.21.15.82:6789/0 conn(0x7f4b980aed10 legacy=0x7f4b98164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980ce720 con 0x7f4b9815a750 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f4b8c001680 con 0x7f4b9815a750 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 >> v1:172.21.15.46:6789/0 conn(0x7f4b9815a750 legacy=0x7f4b9806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800039f0 0x7f4b80003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b9806f320 0x7f4b80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b80004d50 0x7f4b800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8000dc90 con 0x7f4b800039f0 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8000de10 con 0x7f4b80004d50 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8000e010 con 0x7f4b9806f320 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9dc00640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800039f0 0x7f4b80003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9d200640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b9806f320 0x7f4b80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9e600640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b80004d50 0x7f4b800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9dc00640 1 -- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b9806f320 msgr2=0x7f4b80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9dc00640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b9806f320 0x7f4b80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9dc00640 1 -- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b80004d50 msgr2=0x7f4b800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9dc00640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b80004d50 0x7f4b800051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9dc00640 1 -- 172.21.15.46:0/3350771656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80003530 con 0x7f4b800039f0 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9e600640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b80004d50 0x7f4b800051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.901+0000 7f4b9d200640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b9806f320 0x7f4b80004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4b9dc00640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800039f0 0x7f4b80003e10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00c210 tx=0x7f4b8c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6595564fc85751ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c007d20 con 0x7f4b800039f0 2024-04-25T08:46:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8c0a9070 con 0x7f4b800039f0 2024-04-25T08:46:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4b97e00640 1 -- 172.21.15.46:0/3350771656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c005d40 con 0x7f4b800039f0 2024-04-25T08:46:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4ba0600640 1 -- 172.21.15.46:0/3350771656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800039f0 msgr2=0x7f4b80003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4ba0600640 1 --2- 172.21.15.46:0/3350771656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b800039f0 0x7f4b80003e10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00c210 tx=0x7f4b8c01cc60 comp rx=0 tx=0).stop 2024-04-25T08:46:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4b9e600640 1 -- 172.21.15.46:0/3350771656 reap_dead start 2024-04-25T08:46:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4ba0600640 1 -- 172.21.15.46:0/3350771656 shutdown_connections 2024-04-25T08:46:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4ba0600640 1 -- 172.21.15.46:0/3350771656 >> 172.21.15.46:0/3350771656 conn(0x7f4b980b12a0 msgr2=0x7f4b980b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4ba0600640 1 -- 172.21.15.46:0/3350771656 shutdown_connections 2024-04-25T08:46:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.905+0000 7f4ba0600640 1 -- 172.21.15.46:0/3350771656 wait complete. 2024-04-25T08:46:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 Processor -- start 2024-04-25T08:46:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 -- start start 2024-04-25T08:46:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b800039f0 0x7f4b981fa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b9815c410 0x7f4b980ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b9815b3d0 0x7f4b98156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9806e310 con 0x7f4b9815b3d0 2024-04-25T08:46:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9806d270 con 0x7f4b9815c410 2024-04-25T08:46:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9806e490 con 0x7f4b800039f0 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b9815b3d0 0x7f4b98156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b9815b3d0 0x7f4b98156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49096/0 (socket says 172.21.15.46:49096) 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9e600640 1 -- 172.21.15.46:0/1002980265 learned_addr learned my addr 172.21.15.46:0/1002980265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9d200640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b9815c410 0x7f4b980ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9dc00640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b800039f0 0x7f4b981fa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9e600640 1 -- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b800039f0 msgr2=0x7f4b981fa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9e600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b800039f0 0x7f4b981fa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9e600640 1 -- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b9815c410 msgr2=0x7f4b980ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9e600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b9815c410 0x7f4b980ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9e600640 1 -- 172.21.15.46:0/1002980265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b981fae40 con 0x7f4b9815b3d0 2024-04-25T08:46:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b9e600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b9815b3d0 0x7f4b98156bb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4b90000f90 tx=0x7f4b9000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b95600640 1 -- 172.21.15.46:0/1002980265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90006b60 con 0x7f4b9815b3d0 2024-04-25T08:46:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b95600640 1 -- 172.21.15.46:0/1002980265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90006d20 con 0x7f4b9815b3d0 2024-04-25T08:46:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4b95600640 1 -- 172.21.15.46:0/1002980265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90003980 con 0x7f4b9815b3d0 2024-04-25T08:46:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b981fb120 con 0x7f4b9815b3d0 2024-04-25T08:46:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.909+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9815abd0 con 0x7f4b9815b3d0 2024-04-25T08:46:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.913+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b98069080 con 0x7f4b9815b3d0 2024-04-25T08:46:28.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.913+0000 7f4b95600640 1 -- 172.21.15.46:0/1002980265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b9001a020 con 0x7f4b9815b3d0 2024-04-25T08:46:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.913+0000 7f4b95600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b78066f90 0x7f4b78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.913+0000 7f4b95600640 1 -- 172.21.15.46:0/1002980265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f4b900848d0 con 0x7f4b9815b3d0 2024-04-25T08:46:28.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.913+0000 7f4b9dc00640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b78066f90 0x7f4b78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.913+0000 7f4b9dc00640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b78066f90 0x7f4b78069450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0ab000 tx=0x7f4b8c0b1040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:28.917+0000 7f4b95600640 1 -- 172.21.15.46:0/1002980265 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b90055640 con 0x7f4b9815b3d0 2024-04-25T08:46:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.081+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b981565d0 con 0x7f4b9815b3d0 2024-04-25T08:46:29.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.081+0000 7f4b95600640 1 -- 172.21.15.46:0/1002980265 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17028 (secure 0 0 0) 0x7f4b90054f40 con 0x7f4b9815b3d0 2024-04-25T08:46:29.077 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:29.077 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:03.180374+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:45:10.412423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:45:10.987443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b78066f90 msgr2=0x7f4b78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b78066f90 0x7f4b78069450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0ab000 tx=0x7f4b8c0b1040 comp rx=0 tx=0).stop 2024-04-25T08:46:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b9815b3d0 msgr2=0x7f4b98156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b9815b3d0 0x7f4b98156bb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f4b90000f90 tx=0x7f4b9000eb90 comp rx=0 tx=0).stop 2024-04-25T08:46:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 shutdown_connections 2024-04-25T08:46:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b78066f90 0x7f4b78069450 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b9815b3d0 0x7f4b98156bb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b9815c410 0x7f4b980ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 --2- 172.21.15.46:0/1002980265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b800039f0 0x7f4b981fa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 >> 172.21.15.46:0/1002980265 conn(0x7f4b980b12a0 msgr2=0x7f4b98154b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 shutdown_connections 2024-04-25T08:46:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.085+0000 7f4ba0600640 1 -- 172.21.15.46:0/1002980265 wait complete. 2024-04-25T08:46:29.099 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:45:10.412423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:45:10.987443+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:46:29.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:46:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6742800640 1 Processor -- start 2024-04-25T08:46:29.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6742800640 1 -- start start 2024-04-25T08:46:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6742800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c13d0b0 0x7f673c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6742800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f673c071270 0x7f673c0716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6742800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f673c15db50 0x7f673c15dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6742800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c071c10 con 0x7f673c071270 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6742800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c13d490 con 0x7f673c15db50 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6742800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c15eab0 con 0x7f673c13d0b0 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6732a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f673c071270 0x7f673c0716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6732a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f673c071270 0x7f673c0716d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49102/0 (socket says 172.21.15.46:49102) 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.249+0000 7f6732a00640 1 -- 172.21.15.46:0/3657226 learned_addr learned my addr 172.21.15.46:0/3657226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f673be00640 1 --2- 172.21.15.46:0/3657226 >> v2:172.21.15.82:3300/0 conn(0x7f673c15db50 0x7f673c15dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f673b400640 1 --2- 172.21.15.46:0/3657226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c13d0b0 0x7f673c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6732a00640 1 -- 172.21.15.46:0/3657226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c13d0b0 msgr2=0x7f673c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6732a00640 1 --2- 172.21.15.46:0/3657226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c13d0b0 0x7f673c15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6732a00640 1 -- 172.21.15.46:0/3657226 >> v2:172.21.15.82:3300/0 conn(0x7f673c15db50 msgr2=0x7f673c15dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6732a00640 1 --2- 172.21.15.46:0/3657226 >> v2:172.21.15.82:3300/0 conn(0x7f673c15db50 0x7f673c15dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6732a00640 1 -- 172.21.15.46:0/3657226 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c15e520 con 0x7f673c071270 2024-04-25T08:46:29.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f673be00640 1 --2- 172.21.15.46:0/3657226 >> v2:172.21.15.82:3300/0 conn(0x7f673c15db50 0x7f673c15dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6732a00640 1 --2- 172.21.15.46:0/3657226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c071270 0x7f673c0716d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f673c0ccba0 tx=0x7f672401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e223d05e31fe0f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f673aa00640 1 -- 172.21.15.46:0/3657226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67240b0070 con 0x7f673c071270 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f673aa00640 1 -- 172.21.15.46:0/3657226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6724004230 con 0x7f673c071270 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f673aa00640 1 -- 172.21.15.46:0/3657226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724003620 con 0x7f673c071270 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6742800640 1 -- 172.21.15.46:0/3657226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c071270 msgr2=0x7f673c0716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6742800640 1 --2- 172.21.15.46:0/3657226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c071270 0x7f673c0716d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f673c0ccba0 tx=0x7f672401c520 comp rx=0 tx=0).stop 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6742800640 1 -- 172.21.15.46:0/3657226 shutdown_connections 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6742800640 1 --2- 172.21.15.46:0/3657226 >> v2:172.21.15.82:3300/0 conn(0x7f673c15db50 0x7f673c15dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6742800640 1 --2- 172.21.15.46:0/3657226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c071270 0x7f673c0716d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6742800640 1 --2- 172.21.15.46:0/3657226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c13d0b0 0x7f673c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6742800640 1 -- 172.21.15.46:0/3657226 >> 172.21.15.46:0/3657226 conn(0x7f673c06ac10 msgr2=0x7f673c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.253+0000 7f6742800640 1 -- 172.21.15.46:0/3657226 shutdown_connections 2024-04-25T08:46:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 -- 172.21.15.46:0/3657226 wait complete. 2024-04-25T08:46:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 Processor -- start 2024-04-25T08:46:29.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 -- start start 2024-04-25T08:46:29.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673c071270 0x7f673c153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c13d0b0 0x7f673c13b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c15db50 0x7f673c13ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0b2720 con 0x7f673c13d0b0 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c15aca0 con 0x7f673c071270 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0b28a0 con 0x7f673c15db50 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6732a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c13d0b0 0x7f673c13b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6732a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c13d0b0 0x7f673c13b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49110/0 (socket says 172.21.15.46:49110) 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6732a00640 1 -- 172.21.15.46:0/1135537335 learned_addr learned my addr 172.21.15.46:0/1135537335 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f673b400640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673c071270 0x7f673c153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f673be00640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c15db50 0x7f673c13ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6732a00640 1 -- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c15db50 msgr2=0x7f673c13ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6732a00640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c15db50 0x7f673c13ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6732a00640 1 -- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673c071270 msgr2=0x7f673c153e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6732a00640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673c071270 0x7f673c153e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6732a00640 1 -- 172.21.15.46:0/1135537335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c154850 con 0x7f673c13d0b0 2024-04-25T08:46:29.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f673b400640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673c071270 0x7f673c153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:29.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6732a00640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c13d0b0 0x7f673c13b540 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f67240ae040 tx=0x7f67240192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:29.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6733e00640 1 -- 172.21.15.46:0/1135537335 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67240b0040 con 0x7f673c13d0b0 2024-04-25T08:46:29.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6733e00640 1 -- 172.21.15.46:0/1135537335 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67240bb070 con 0x7f673c13d0b0 2024-04-25T08:46:29.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6733e00640 1 -- 172.21.15.46:0/1135537335 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67240b6440 con 0x7f673c13d0b0 2024-04-25T08:46:29.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c154b30 con 0x7f673c13d0b0 2024-04-25T08:46:29.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.257+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c1550f0 con 0x7f673c13d0b0 2024-04-25T08:46:29.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.261+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c069010 con 0x7f673c13d0b0 2024-04-25T08:46:29.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.261+0000 7f6733e00640 1 -- 172.21.15.46:0/1135537335 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67240b65e0 con 0x7f673c13d0b0 2024-04-25T08:46:29.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.261+0000 7f6733e00640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6718066f60 0x7f6718069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.261+0000 7f6733e00640 1 -- 172.21.15.46:0/1135537335 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f672411f6b0 con 0x7f673c13d0b0 2024-04-25T08:46:29.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.261+0000 7f673b400640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6718066f60 0x7f6718069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.261+0000 7f673b400640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6718066f60 0x7f6718069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f672c004660 tx=0x7f672c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:29.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.265+0000 7f6733e00640 1 -- 172.21.15.46:0/1135537335 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6724016110 con 0x7f673c13d0b0 2024-04-25T08:46:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.429+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f673c0cce90 con 0x7f673c13d0b0 2024-04-25T08:46:29.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.429+0000 7f6733e00640 1 -- 172.21.15.46:0/1135537335 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67240f05d0 con 0x7f673c13d0b0 2024-04-25T08:46:29.454 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:46:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6718066f60 msgr2=0x7f6718069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6718066f60 0x7f6718069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f672c004660 tx=0x7f672c008040 comp rx=0 tx=0).stop 2024-04-25T08:46:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c13d0b0 msgr2=0x7f673c13b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c13d0b0 0x7f673c13b540 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f67240ae040 tx=0x7f67240192c0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 shutdown_connections 2024-04-25T08:46:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6718066f60 0x7f6718069420 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f673c15db50 0x7f673c13ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f673c13d0b0 0x7f673c13b540 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 --2- 172.21.15.46:0/1135537335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f673c071270 0x7f673c153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 >> 172.21.15.46:0/1135537335 conn(0x7f673c06ac10 msgr2=0x7f673c158700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:29.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 shutdown_connections 2024-04-25T08:46:29.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.433+0000 7f6742800640 1 -- 172.21.15.46:0/1135537335 wait complete. 2024-04-25T08:46:29.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:46:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.605+0000 7ffa08c00640 1 Processor -- start 2024-04-25T08:46:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.605+0000 7ffa08c00640 1 -- start start 2024-04-25T08:46:29.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.605+0000 7ffa08c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa040cd4d0 0x7ffa04168690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.605+0000 7ffa08c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0409a780 con 0x7ffa0406f4d0 2024-04-25T08:46:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.605+0000 7ffa08c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0409aa60 con 0x7ffa0415dd50 2024-04-25T08:46:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.605+0000 7ffa08c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa040708c0 con 0x7ffa040cd4d0 2024-04-25T08:46:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.605+0000 7ffa02800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa040cd4d0 0x7ffa04168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.605+0000 7ffa01e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa0406f4d0 0x7ffa040cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34844/0 (socket says 172.21.15.46:34844) 2024-04-25T08:46:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.605+0000 7ffa01e00640 1 -- 172.21.15.46:0/1612629944 learned_addr learned my addr 172.21.15.46:0/1612629944 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa01400640 1 -- 172.21.15.46:0/1612629944 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787671276 0 0) 0x7ffa0409a780 con 0x7ffa0406f4d0 2024-04-25T08:46:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa02800640 1 -- 172.21.15.46:0/1612629944 >> v1:172.21.15.82:6789/0 conn(0x7ffa0415dd50 legacy=0x7ffa040cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa02800640 1 -- 172.21.15.46:0/1612629944 >> v1:172.21.15.46:6789/0 conn(0x7ffa0406f4d0 legacy=0x7ffa040cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa02800640 1 -- 172.21.15.46:0/1612629944 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0409ad40 con 0x7ffa040cd4d0 2024-04-25T08:46:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa02800640 1 --2- 172.21.15.46:0/1612629944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa040cd4d0 0x7ffa04168690 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009d90 tx=0x7ff9fc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bfb24ab5a9baf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa01400640 1 -- 172.21.15.46:0/1612629944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0b2070 con 0x7ffa040cd4d0 2024-04-25T08:46:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa01400640 1 -- 172.21.15.46:0/1612629944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc0a5a00 con 0x7ffa040cd4d0 2024-04-25T08:46:29.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa01400640 1 -- 172.21.15.46:0/1612629944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0a5dd0 con 0x7ffa040cd4d0 2024-04-25T08:46:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa08c00640 1 -- 172.21.15.46:0/1612629944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa040cd4d0 msgr2=0x7ffa04168690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa08c00640 1 --2- 172.21.15.46:0/1612629944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa040cd4d0 0x7ffa04168690 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009d90 tx=0x7ff9fc01c940 comp rx=0 tx=0).stop 2024-04-25T08:46:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa08c00640 1 -- 172.21.15.46:0/1612629944 shutdown_connections 2024-04-25T08:46:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa08c00640 1 --2- 172.21.15.46:0/1612629944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa040cd4d0 0x7ffa04168690 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa08c00640 1 -- 172.21.15.46:0/1612629944 >> 172.21.15.46:0/1612629944 conn(0x7ffa040b3160 msgr2=0x7ffa040b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa08c00640 1 -- 172.21.15.46:0/1612629944 shutdown_connections 2024-04-25T08:46:29.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.609+0000 7ffa08c00640 1 -- 172.21.15.46:0/1612629944 wait complete. 2024-04-25T08:46:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 Processor -- start 2024-04-25T08:46:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 -- start start 2024-04-25T08:46:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0406f4d0 0x7ffa04148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa040cd4d0 0x7ffa041f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0415dd50 0x7ffa041fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0416b800 con 0x7ffa0415dd50 2024-04-25T08:46:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa040d0620 con 0x7ffa0406f4d0 2024-04-25T08:46:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0416b980 con 0x7ffa040cd4d0 2024-04-25T08:46:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa02800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0415dd50 0x7ffa041fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa02800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0415dd50 0x7ffa041fa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49140/0 (socket says 172.21.15.46:49140) 2024-04-25T08:46:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa02800640 1 -- 172.21.15.46:0/1801747113 learned_addr learned my addr 172.21.15.46:0/1801747113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ff9f9400640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa040cd4d0 0x7ffa041f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa01e00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0406f4d0 0x7ffa04148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa02800640 1 -- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa040cd4d0 msgr2=0x7ffa041f9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa02800640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa040cd4d0 0x7ffa041f9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa02800640 1 -- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0406f4d0 msgr2=0x7ffa04148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa02800640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0406f4d0 0x7ffa04148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa02800640 1 -- 172.21.15.46:0/1801747113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04149260 con 0x7ffa0415dd50 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa02800640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0415dd50 0x7ffa041fa2b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0b0040 tx=0x7ff9fc0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ff9faa00640 1 -- 172.21.15.46:0/1801747113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0b2040 con 0x7ffa0415dd50 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ff9faa00640 1 -- 172.21.15.46:0/1801747113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc0bf070 con 0x7ffa0415dd50 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ff9faa00640 1 -- 172.21.15.46:0/1801747113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0ada70 con 0x7ffa0415dd50 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0406bdc0 con 0x7ffa0415dd50 2024-04-25T08:46:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.613+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0406c380 con 0x7ffa0415dd50 2024-04-25T08:46:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.617+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa04069010 con 0x7ffa0415dd50 2024-04-25T08:46:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.617+0000 7ff9faa00640 1 -- 172.21.15.46:0/1801747113 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9fc0adc10 con 0x7ffa0415dd50 2024-04-25T08:46:29.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.617+0000 7ff9faa00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9e0066f60 0x7ff9e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.617+0000 7ff9faa00640 1 -- 172.21.15.46:0/1801747113 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff9fc120600 con 0x7ffa0415dd50 2024-04-25T08:46:29.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.617+0000 7ffa01e00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9e0066f60 0x7ff9e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.617+0000 7ffa01e00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9e0066f60 0x7ff9e0069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007fd0 tx=0x7ff9f4007ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:29.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.621+0000 7ff9faa00640 1 -- 172.21.15.46:0/1801747113 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9fc016560 con 0x7ffa0415dd50 2024-04-25T08:46:29.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.781+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa040c4ec0 con 0x7ffa0415dd50 2024-04-25T08:46:29.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ff9faa00640 1 -- 172.21.15.46:0/1801747113 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9fc0ab340 con 0x7ffa0415dd50 2024-04-25T08:46:29.777 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9e0066f60 msgr2=0x7ff9e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9e0066f60 0x7ff9e0069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007fd0 tx=0x7ff9f4007ba0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0415dd50 msgr2=0x7ffa041fa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0415dd50 0x7ffa041fa2b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0b0040 tx=0x7ff9fc0196e0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 shutdown_connections 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff9e0066f60 0x7ff9e0069420 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0415dd50 0x7ffa041fa2b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffa040cd4d0 0x7ffa041f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 --2- 172.21.15.46:0/1801747113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0406f4d0 0x7ffa04148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 >> 172.21.15.46:0/1801747113 conn(0x7ffa040b3160 msgr2=0x7ffa0415c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 shutdown_connections 2024-04-25T08:46:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.785+0000 7ffa08c00640 1 -- 172.21.15.46:0/1801747113 wait complete. 2024-04-25T08:46:29.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:46:29.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd85800640 1 Processor -- start 2024-04-25T08:46:29.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd85800640 1 -- start start 2024-04-25T08:46:29.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd85800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cf890 0x7fbd800b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd85800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd800cc180 0x7fbd800cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd85800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd800b2d50 con 0x7fbd800cf340 2024-04-25T08:46:29.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd85800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd800d01b0 con 0x7fbd800cc180 2024-04-25T08:46:29.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd85800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015f2f0 con 0x7fbd800cf890 2024-04-25T08:46:29.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd77e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd800cc180 0x7fbd800cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd77e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd800cc180 0x7fbd800cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49154/0 (socket says 172.21.15.46:49154) 2024-04-25T08:46:29.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd77e00640 1 -- 172.21.15.46:0/2930647958 learned_addr learned my addr 172.21.15.46:0/2930647958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:29.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.949+0000 7fbd7ea00640 1 --2- 172.21.15.46:0/2930647958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cf890 0x7fbd800b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd77e00640 1 -- 172.21.15.46:0/2930647958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cf890 msgr2=0x7fbd800b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:29.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd77e00640 1 --2- 172.21.15.46:0/2930647958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cf890 0x7fbd800b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd77e00640 1 -- 172.21.15.46:0/2930647958 >> v1:172.21.15.82:6789/0 conn(0x7fbd800cf340 legacy=0x7fbd800ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd77e00640 1 -- 172.21.15.46:0/2930647958 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8015f5d0 con 0x7fbd800cc180 2024-04-25T08:46:29.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd77e00640 1 --2- 172.21.15.46:0/2930647958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cc180 0x7fbd800cfc70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009870 tx=0x7fbd6801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3e41288f5792f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:29.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd7e000640 1 -- 172.21.15.46:0/2930647958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd680ab680 con 0x7fbd800cc180 2024-04-25T08:46:29.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd7e000640 1 -- 172.21.15.46:0/2930647958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd680b0070 con 0x7fbd800cc180 2024-04-25T08:46:29.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd7e000640 1 -- 172.21.15.46:0/2930647958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd680aba40 con 0x7fbd800cc180 2024-04-25T08:46:29.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd85800640 1 -- 172.21.15.46:0/2930647958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cc180 msgr2=0x7fbd800cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.953+0000 7fbd85800640 1 --2- 172.21.15.46:0/2930647958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cc180 0x7fbd800cfc70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009870 tx=0x7fbd6801c560 comp rx=0 tx=0).stop 2024-04-25T08:46:29.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 -- 172.21.15.46:0/2930647958 shutdown_connections 2024-04-25T08:46:29.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 --2- 172.21.15.46:0/2930647958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cc180 0x7fbd800cfc70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 --2- 172.21.15.46:0/2930647958 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cf890 0x7fbd800b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 -- 172.21.15.46:0/2930647958 >> 172.21.15.46:0/2930647958 conn(0x7fbd80099a00 msgr2=0x7fbd800b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:29.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 -- 172.21.15.46:0/2930647958 shutdown_connections 2024-04-25T08:46:29.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 -- 172.21.15.46:0/2930647958 wait complete. 2024-04-25T08:46:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 Processor -- start 2024-04-25T08:46:29.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 -- start start 2024-04-25T08:46:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cc180 0x7fbd8015db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd800cf340 0x7fbd801469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cf890 0x7fbd800ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015cb80 con 0x7fbd800cf890 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd80168690 con 0x7fbd800cf340 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd85800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd80146ef0 con 0x7fbd800cc180 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cf890 0x7fbd800ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cf890 0x7fbd800ae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49168/0 (socket says 172.21.15.46:49168) 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7f400640 1 -- 172.21.15.46:0/4270814512 learned_addr learned my addr 172.21.15.46:0/4270814512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd77e00640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd800cf340 0x7fbd801469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7ea00640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cc180 0x7fbd8015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7f400640 1 -- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cc180 msgr2=0x7fbd8015db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7f400640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cc180 0x7fbd8015db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7f400640 1 -- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd800cf340 msgr2=0x7fbd801469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7f400640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd800cf340 0x7fbd801469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7f400640 1 -- 172.21.15.46:0/4270814512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8015e180 con 0x7fbd800cf890 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7ea00640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cc180 0x7fbd8015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.957+0000 7fbd7f400640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cf890 0x7fbd800ae1b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800c7a0 tx=0x7fbd7800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.961+0000 7fbd76a00640 1 -- 172.21.15.46:0/4270814512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78003b80 con 0x7fbd800cf890 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.961+0000 7fbd76a00640 1 -- 172.21.15.46:0/4270814512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd78003d20 con 0x7fbd800cf890 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.961+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8015e460 con 0x7fbd800cf890 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.961+0000 7fbd76a00640 1 -- 172.21.15.46:0/4270814512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7800aa90 con 0x7fbd800cf890 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.961+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8015ea20 con 0x7fbd800cf890 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.961+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd80069010 con 0x7fbd800cf890 2024-04-25T08:46:29.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.961+0000 7fbd76a00640 1 -- 172.21.15.46:0/4270814512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd7800ac30 con 0x7fbd800cf890 2024-04-25T08:46:29.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.965+0000 7fbd76a00640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c066f60 0x7fbd5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.965+0000 7fbd76a00640 1 -- 172.21.15.46:0/4270814512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fbd78085ad0 con 0x7fbd800cf890 2024-04-25T08:46:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.965+0000 7fbd7ea00640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c066f60 0x7fbd5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:29.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.965+0000 7fbd7ea00640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c066f60 0x7fbd5c069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009b10 tx=0x7fbd70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:29.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:29.969+0000 7fbd76a00640 1 -- 172.21.15.46:0/4270814512 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd78085e20 con 0x7fbd800cf890 2024-04-25T08:46:30.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.129+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd800b2fd0 con 0x7fbd800cf890 2024-04-25T08:46:30.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.129+0000 7fbd76a00640 1 -- 172.21.15.46:0/4270814512 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd800b2d50 con 0x7fbd800cf890 2024-04-25T08:46:30.124 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c066f60 msgr2=0x7fbd5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c066f60 0x7fbd5c069420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009b10 tx=0x7fbd70008040 comp rx=0 tx=0).stop 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cf890 msgr2=0x7fbd800ae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cf890 0x7fbd800ae1b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800c7a0 tx=0x7fbd7800cc60 comp rx=0 tx=0).stop 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 shutdown_connections 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c066f60 0x7fbd5c069420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd800cf890 0x7fbd800ae1b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd800cf340 0x7fbd801469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 --2- 172.21.15.46:0/4270814512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd800cc180 0x7fbd8015db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 >> 172.21.15.46:0/4270814512 conn(0x7fbd80099a00 msgr2=0x7fbd80165260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 shutdown_connections 2024-04-25T08:46:30.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.133+0000 7fbd85800640 1 -- 172.21.15.46:0/4270814512 wait complete. 2024-04-25T08:46:30.146 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:46:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.293+0000 7f94cb400640 1 Processor -- start 2024-04-25T08:46:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.293+0000 7f94cb400640 1 -- start start 2024-04-25T08:46:30.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.293+0000 7f94cb400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94c40cdc50 0x7f94c4166770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.293+0000 7f94cb400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c40cc700 con 0x7f94c415bdb0 2024-04-25T08:46:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.293+0000 7f94cb400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c40cc9e0 con 0x7f94c4159230 2024-04-25T08:46:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.293+0000 7f94cb400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c40cccc0 con 0x7f94c40cdc50 2024-04-25T08:46:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.293+0000 7f94bbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f94c415bdb0 0x7f94c40cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34852/0 (socket says 172.21.15.46:34852) 2024-04-25T08:46:30.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.293+0000 7f94bbe00640 1 -- 172.21.15.46:0/2075365611 learned_addr learned my addr 172.21.15.46:0/2075365611 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c9400640 1 --2- 172.21.15.46:0/2075365611 >> v2:172.21.15.82:3300/0 conn(0x7f94c40cdc50 0x7f94c4166770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c3e00640 1 -- 172.21.15.46:0/2075365611 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652552123 0 0) 0x7f94c40cc700 con 0x7f94c415bdb0 2024-04-25T08:46:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c3e00640 1 -- 172.21.15.46:0/2075365611 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a8003530 con 0x7f94c415bdb0 2024-04-25T08:46:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c3e00640 1 -- 172.21.15.46:0/2075365611 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384293423 0 0) 0x7f94c40cc9e0 con 0x7f94c4159230 2024-04-25T08:46:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c3e00640 1 -- 172.21.15.46:0/2075365611 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94c40cc700 con 0x7f94c4159230 2024-04-25T08:46:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c3e00640 1 -- 172.21.15.46:0/2075365611 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642128145 0 0) 0x7f94a8003530 con 0x7f94c415bdb0 2024-04-25T08:46:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c9400640 1 -- 172.21.15.46:0/2075365611 >> v1:172.21.15.82:6789/0 conn(0x7f94c4159230 legacy=0x7f94c40cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c9400640 1 -- 172.21.15.46:0/2075365611 >> v1:172.21.15.46:6789/0 conn(0x7f94c415bdb0 legacy=0x7f94c40cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c9400640 1 -- 172.21.15.46:0/2075365611 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c40ccfa0 con 0x7f94c40cdc50 2024-04-25T08:46:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c9400640 1 --2- 172.21.15.46:0/2075365611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c40cdc50 0x7f94c4166770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009d90 tx=0x7f94bc01c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c8093f528f3869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c3e00640 1 -- 172.21.15.46:0/2075365611 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0b2070 con 0x7f94c40cdc50 2024-04-25T08:46:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c3e00640 1 -- 172.21.15.46:0/2075365611 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94bc0a5550 con 0x7f94c40cdc50 2024-04-25T08:46:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.297+0000 7f94c3e00640 1 -- 172.21.15.46:0/2075365611 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac001680 con 0x7f94c40cdc50 2024-04-25T08:46:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 -- 172.21.15.46:0/2075365611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c40cdc50 msgr2=0x7f94c4166770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 --2- 172.21.15.46:0/2075365611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c40cdc50 0x7f94c4166770 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009d90 tx=0x7f94bc01c800 comp rx=0 tx=0).stop 2024-04-25T08:46:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 -- 172.21.15.46:0/2075365611 shutdown_connections 2024-04-25T08:46:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 --2- 172.21.15.46:0/2075365611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c40cdc50 0x7f94c4166770 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 -- 172.21.15.46:0/2075365611 >> 172.21.15.46:0/2075365611 conn(0x7f94c40b12a0 msgr2=0x7f94c40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 -- 172.21.15.46:0/2075365611 shutdown_connections 2024-04-25T08:46:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 -- 172.21.15.46:0/2075365611 wait complete. 2024-04-25T08:46:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 Processor -- start 2024-04-25T08:46:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 -- start start 2024-04-25T08:46:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c40cdc50 0x7f94c406ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4159230 0x7f94c40ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c415bdb0 0x7f94c415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4169860 con 0x7f94c40cdc50 2024-04-25T08:46:30.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41696e0 con 0x7f94c4159230 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94cb400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41699e0 con 0x7f94c415bdb0 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c40cdc50 0x7f94c406ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c40cdc50 0x7f94c406ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49172/0 (socket says 172.21.15.46:49172) 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c8a00640 1 -- 172.21.15.46:0/1274901965 learned_addr learned my addr 172.21.15.46:0/1274901965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94bbe00640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4159230 0x7f94c40ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c9400640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c415bdb0 0x7f94c415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c8a00640 1 -- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c415bdb0 msgr2=0x7f94c415ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c8a00640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c415bdb0 0x7f94c415ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c8a00640 1 -- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4159230 msgr2=0x7f94c40ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c8a00640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4159230 0x7f94c40ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c8a00640 1 -- 172.21.15.46:0/1274901965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c40ae270 con 0x7f94c40cdc50 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94bbe00640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4159230 0x7f94c40ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.301+0000 7f94c9400640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c415bdb0 0x7f94c415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94c8a00640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c40cdc50 0x7f94c406ee50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c500 tx=0x7f94b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:30.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94c1600640 1 -- 172.21.15.46:0/1274901965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b40052d0 con 0x7f94c40cdc50 2024-04-25T08:46:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94c1600640 1 -- 172.21.15.46:0/1274901965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4005490 con 0x7f94c40cdc50 2024-04-25T08:46:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c406f510 con 0x7f94c40cdc50 2024-04-25T08:46:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94c1600640 1 -- 172.21.15.46:0/1274901965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4012850 con 0x7f94c40cdc50 2024-04-25T08:46:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c40adf90 con 0x7f94c40cdc50 2024-04-25T08:46:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c4069080 con 0x7f94c40cdc50 2024-04-25T08:46:30.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94c1600640 1 -- 172.21.15.46:0/1274901965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94b40129f0 con 0x7f94c40cdc50 2024-04-25T08:46:30.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94c1600640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a0066f60 0x7f94a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94c1600640 1 -- 172.21.15.46:0/1274901965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f94b4085070 con 0x7f94c40cdc50 2024-04-25T08:46:30.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.305+0000 7f94bbe00640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a0066f60 0x7f94a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.309+0000 7f94bbe00640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a0066f60 0x7f94a0069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0017f0 tx=0x7f94ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:30.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.309+0000 7f94c1600640 1 -- 172.21.15.46:0/1274901965 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94b4055df0 con 0x7f94c40cdc50 2024-04-25T08:46:30.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.485+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f94c4072160 con 0x7f94c40cdc50 2024-04-25T08:46:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.485+0000 7f94c1600640 1 -- 172.21.15.46:0/1274901965 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) v1 ==== 80+0+0 (secure 0 0 0) 0x7f94b40556f0 con 0x7f94c40cdc50 2024-04-25T08:46:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:46:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a0066f60 msgr2=0x7f94a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a0066f60 0x7f94a0069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0017f0 tx=0x7f94ac009040 comp rx=0 tx=0).stop 2024-04-25T08:46:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c40cdc50 msgr2=0x7f94c406ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c40cdc50 0x7f94c406ee50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c500 tx=0x7f94b400c9c0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 shutdown_connections 2024-04-25T08:46:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f94a0066f60 0x7f94a0069420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f94c415bdb0 0x7f94c415ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94c4159230 0x7f94c40ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 --2- 172.21.15.46:0/1274901965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94c40cdc50 0x7f94c406ee50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 >> 172.21.15.46:0/1274901965 conn(0x7f94c40b12a0 msgr2=0x7f94c40b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 shutdown_connections 2024-04-25T08:46:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.489+0000 7f94cb400640 1 -- 172.21.15.46:0/1274901965 wait complete. 2024-04-25T08:46:30.500 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:46:30.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:30.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:46:30.545 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:46:30.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:30.545 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:46:30.597 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:46:30.597 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:30.606 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:46:30.607 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:46:30.607 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:30.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:46:30.649 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:46:30.649 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:30.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:46:30.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.837+0000 7fc34b000640 1 Processor -- start 2024-04-25T08:46:30.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.837+0000 7fc34b000640 1 -- start start 2024-04-25T08:46:30.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc34b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc34415a750 0x7fc34406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc34b000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc3440cc9e0 0x7fc34406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc34b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34406d5e0 con 0x7fc3440aed10 2024-04-25T08:46:30.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc34b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc34415ab30 con 0x7fc34415a750 2024-04-25T08:46:30.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc34b000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3440ce4f0 con 0x7fc3440cc9e0 2024-04-25T08:46:30.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc343e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc34415a750 0x7fc34406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc33be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc3440aed10 0x7fc3440b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34866/0 (socket says 172.21.15.46:34866) 2024-04-25T08:46:30.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc343e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc34415a750 0x7fc34406ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49182/0 (socket says 172.21.15.46:49182) 2024-04-25T08:46:30.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc33be00640 1 -- 172.21.15.46:0/2842568077 learned_addr learned my addr 172.21.15.46:0/2842568077 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:30.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc343e00640 1 -- 172.21.15.46:0/2842568077 >> v2:172.21.15.82:3300/0 conn(0x7fc3440cc9e0 msgr2=0x7fc34406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:30.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc349000640 1 --2- 172.21.15.46:0/2842568077 >> v2:172.21.15.82:3300/0 conn(0x7fc3440cc9e0 0x7fc34406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc343e00640 1 --2- 172.21.15.46:0/2842568077 >> v2:172.21.15.82:3300/0 conn(0x7fc3440cc9e0 0x7fc34406f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc343e00640 1 -- 172.21.15.46:0/2842568077 >> v1:172.21.15.46:6789/0 conn(0x7fc3440aed10 legacy=0x7fc3440b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc343e00640 1 -- 172.21.15.46:0/2842568077 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3440ce720 con 0x7fc34415a750 2024-04-25T08:46:30.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.841+0000 7fc343400640 1 -- 172.21.15.46:0/2842568077 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518178925 0 0) 0x7fc34406d5e0 con 0x7fc3440aed10 2024-04-25T08:46:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc343e00640 1 --2- 172.21.15.46:0/2842568077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34415a750 0x7fc34406ec10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc344050030 tx=0x7fc33401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136ec853ce8939de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc343400640 1 -- 172.21.15.46:0/2842568077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3340b0070 con 0x7fc34415a750 2024-04-25T08:46:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc343400640 1 -- 172.21.15.46:0/2842568077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3340ab600 con 0x7fc34415a750 2024-04-25T08:46:30.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc343400640 1 -- 172.21.15.46:0/2842568077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3340ab9d0 con 0x7fc34415a750 2024-04-25T08:46:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 -- 172.21.15.46:0/2842568077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34415a750 msgr2=0x7fc34406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 --2- 172.21.15.46:0/2842568077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34415a750 0x7fc34406ec10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc344050030 tx=0x7fc33401c480 comp rx=0 tx=0).stop 2024-04-25T08:46:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 -- 172.21.15.46:0/2842568077 shutdown_connections 2024-04-25T08:46:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 --2- 172.21.15.46:0/2842568077 >> v2:172.21.15.82:3300/0 conn(0x7fc3440cc9e0 0x7fc34406f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 --2- 172.21.15.46:0/2842568077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc34415a750 0x7fc34406ec10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 -- 172.21.15.46:0/2842568077 >> 172.21.15.46:0/2842568077 conn(0x7fc3440b12a0 msgr2=0x7fc3440b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:30.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 -- 172.21.15.46:0/2842568077 shutdown_connections 2024-04-25T08:46:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 -- 172.21.15.46:0/2842568077 wait complete. 2024-04-25T08:46:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 Processor -- start 2024-04-25T08:46:30.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 -- start start 2024-04-25T08:46:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3440aed10 0x7fc344146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3440cc9e0 0x7fc34414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34415a750 0x7fc344136cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344167230 con 0x7fc3440cc9e0 2024-04-25T08:46:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34406d270 con 0x7fc34415a750 2024-04-25T08:46:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc34b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3441673b0 con 0x7fc3440aed10 2024-04-25T08:46:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc33be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3440cc9e0 0x7fc34414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc33be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3440cc9e0 0x7fc34414e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49186/0 (socket says 172.21.15.46:49186) 2024-04-25T08:46:30.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.845+0000 7fc33be00640 1 -- 172.21.15.46:0/2516185935 learned_addr learned my addr 172.21.15.46:0/2516185935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc343e00640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3440aed10 0x7fc344146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc349000640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34415a750 0x7fc344136cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc33be00640 1 -- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3440aed10 msgr2=0x7fc344146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc33be00640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3440aed10 0x7fc344146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc33be00640 1 -- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34415a750 msgr2=0x7fc344136cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc33be00640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34415a750 0x7fc344136cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc33be00640 1 -- 172.21.15.46:0/2516185935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc344147390 con 0x7fc3440cc9e0 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc33be00640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3440cc9e0 0x7fc34414e300 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00c4e0 tx=0x7fc32c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc340c00640 1 -- 172.21.15.46:0/2516185935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c018070 con 0x7fc3440cc9e0 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc340c00640 1 -- 172.21.15.46:0/2516185935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c007500 con 0x7fc3440cc9e0 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc340c00640 1 -- 172.21.15.46:0/2516185935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c004a30 con 0x7fc3440cc9e0 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc344147670 con 0x7fc3440cc9e0 2024-04-25T08:46:30.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc344147c30 con 0x7fc3440cc9e0 2024-04-25T08:46:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc344069080 con 0x7fc3440cc9e0 2024-04-25T08:46:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc340c00640 1 -- 172.21.15.46:0/2516185935 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc32c01b070 con 0x7fc3440cc9e0 2024-04-25T08:46:30.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc340c00640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc320066f60 0x7fc320069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:30.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.849+0000 7fc340c00640 1 -- 172.21.15.46:0/2516185935 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc32c084550 con 0x7fc3440cc9e0 2024-04-25T08:46:30.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.853+0000 7fc343e00640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc320066f60 0x7fc320069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:30.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.853+0000 7fc343e00640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc320066f60 0x7fc320069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc3340ad000 tx=0x7fc334008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:30.853+0000 7fc340c00640 1 -- 172.21.15.46:0/2516185935 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc32c054d30 con 0x7fc3440cc9e0 2024-04-25T08:46:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc34406f470 con 0x7fc3440cc9e0 2024-04-25T08:46:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc340c00640 1 -- 172.21.15.46:0/2516185935 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fc32c05b080 con 0x7fc3440cc9e0 2024-04-25T08:46:31.043 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:31.043 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:45:11.989847+0000","modified":"2024-04-25T08:46:00.651586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6207},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6207":{"gid":6207,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/1670658295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670658295},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670658295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:46:31.043 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-04-25T08:46:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc320066f60 msgr2=0x7fc320069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc320066f60 0x7fc320069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc3340ad000 tx=0x7fc334008040 comp rx=0 tx=0).stop 2024-04-25T08:46:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3440cc9e0 msgr2=0x7fc34414e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3440cc9e0 0x7fc34414e300 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00c4e0 tx=0x7fc32c00c9a0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 shutdown_connections 2024-04-25T08:46:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc320066f60 0x7fc320069420 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc34415a750 0x7fc344136cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3440cc9e0 0x7fc34414e300 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 --2- 172.21.15.46:0/2516185935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3440aed10 0x7fc344146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 >> 172.21.15.46:0/2516185935 conn(0x7fc3440b12a0 msgr2=0x7fc34415acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 shutdown_connections 2024-04-25T08:46:31.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.049+0000 7fc34b000640 1 -- 172.21.15.46:0/2516185935 wait complete. 2024-04-25T08:46:31.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb55c800640 1 Processor -- start 2024-04-25T08:46:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb55c800640 1 -- start start 2024-04-25T08:46:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb55c800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb554099a00 0x7fb554099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb55c800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 0x7fb5540cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb55c800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55409a320 con 0x7fb5540b2240 2024-04-25T08:46:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb55c800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55406db90 con 0x7fb554099a00 2024-04-25T08:46:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb55c800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55406bdc0 con 0x7fb5540ccde0 2024-04-25T08:46:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb559e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb554099a00 0x7fb554099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb55a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb5540b2240 0x7fb5540b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34876/0 (socket says 172.21.15.46:34876) 2024-04-25T08:46:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb55a800640 1 -- 172.21.15.46:0/2979697170 learned_addr learned my addr 172.21.15.46:0/2979697170 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb559400640 1 --2- 172.21.15.46:0/2979697170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 0x7fb5540cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb559e00640 1 -- 172.21.15.46:0/2979697170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 msgr2=0x7fb5540cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb559e00640 1 --2- 172.21.15.46:0/2979697170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 0x7fb5540cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb559e00640 1 -- 172.21.15.46:0/2979697170 >> v1:172.21.15.46:6789/0 conn(0x7fb5540b2240 legacy=0x7fb5540b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb559e00640 1 -- 172.21.15.46:0/2979697170 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55406c0a0 con 0x7fb554099a00 2024-04-25T08:46:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.213+0000 7fb558a00640 1 -- 172.21.15.46:0/2979697170 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330698393 0 0) 0x7fb55409a320 con 0x7fb5540b2240 2024-04-25T08:46:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb559e00640 1 --2- 172.21.15.46:0/2979697170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554099a00 0x7fb554099de0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb5480097c0 tx=0x7fb54801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c80b29e389c3336 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb558a00640 1 -- 172.21.15.46:0/2979697170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5480b1070 con 0x7fb554099a00 2024-04-25T08:46:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb558a00640 1 -- 172.21.15.46:0/2979697170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5480a8050 con 0x7fb554099a00 2024-04-25T08:46:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb558a00640 1 -- 172.21.15.46:0/2979697170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5480ac860 con 0x7fb554099a00 2024-04-25T08:46:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb55c800640 1 -- 172.21.15.46:0/2979697170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554099a00 msgr2=0x7fb554099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb55c800640 1 --2- 172.21.15.46:0/2979697170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554099a00 0x7fb554099de0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb5480097c0 tx=0x7fb54801c3e0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb55c800640 1 -- 172.21.15.46:0/2979697170 shutdown_connections 2024-04-25T08:46:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb55c800640 1 --2- 172.21.15.46:0/2979697170 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 0x7fb5540cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb55c800640 1 --2- 172.21.15.46:0/2979697170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb554099a00 0x7fb554099de0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb55c800640 1 -- 172.21.15.46:0/2979697170 >> 172.21.15.46:0/2979697170 conn(0x7fb5540b3120 msgr2=0x7fb5540b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb55c800640 1 -- 172.21.15.46:0/2979697170 shutdown_connections 2024-04-25T08:46:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb55c800640 1 -- 172.21.15.46:0/2979697170 wait complete. 2024-04-25T08:46:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.217+0000 7fb55c800640 1 Processor -- start 2024-04-25T08:46:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 -- start start 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb554099a00 0x7fb554154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5540b2240 0x7fb5540b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 0x7fb5540c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540cd7a0 con 0x7fb5540b2240 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540cdb80 con 0x7fb554099a00 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540cd920 con 0x7fb5540ccde0 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5540b2240 0x7fb5540b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5540b2240 0x7fb5540b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49206/0 (socket says 172.21.15.46:49206) 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559400640 1 -- 172.21.15.46:0/1819775205 learned_addr learned my addr 172.21.15.46:0/1819775205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559e00640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb554099a00 0x7fb554154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55a800640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 0x7fb5540c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559400640 1 -- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 msgr2=0x7fb5540c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559400640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 0x7fb5540c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559400640 1 -- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb554099a00 msgr2=0x7fb554154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559400640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb554099a00 0x7fb554154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559400640 1 -- 172.21.15.46:0/1819775205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554067e60 con 0x7fb5540b2240 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb559400640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5540b2240 0x7fb5540b62a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb54400e8a0 tx=0x7fb54400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb54e000640 1 -- 172.21.15.46:0/1819775205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544017070 con 0x7fb5540b2240 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb54e000640 1 -- 172.21.15.46:0/1819775205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54400c940 con 0x7fb5540b2240 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb54e000640 1 -- 172.21.15.46:0/1819775205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54400ccf0 con 0x7fb5540b2240 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5540b5200 con 0x7fb5540b2240 2024-04-25T08:46:31.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5540b57c0 con 0x7fb5540b2240 2024-04-25T08:46:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.221+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554069080 con 0x7fb5540b2240 2024-04-25T08:46:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.225+0000 7fb54e000640 1 -- 172.21.15.46:0/1819775205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb54401a070 con 0x7fb5540b2240 2024-04-25T08:46:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.225+0000 7fb54e000640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb538066f60 0x7fb538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.225+0000 7fb54e000640 1 -- 172.21.15.46:0/1819775205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb544085090 con 0x7fb5540b2240 2024-04-25T08:46:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.225+0000 7fb559e00640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb538066f60 0x7fb538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.225+0000 7fb559e00640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb538066f60 0x7fb538069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb5480ae000 tx=0x7fb5480a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:31.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.229+0000 7fb54e000640 1 -- 172.21.15.46:0/1819775205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb544055840 con 0x7fb5540b2240 2024-04-25T08:46:31.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.389+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb55409a870 con 0x7fb5540b2240 2024-04-25T08:46:31.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb54e000640 1 -- 172.21.15.46:0/1819775205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17028 (secure 0 0 0) 0x7fb5440598e0 con 0x7fb5540b2240 2024-04-25T08:46:31.386 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:31.386 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:03.180374+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:45:10.412423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:45:10.987443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb538066f60 msgr2=0x7fb538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb538066f60 0x7fb538069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb5480ae000 tx=0x7fb5480a7000 comp rx=0 tx=0).stop 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5540b2240 msgr2=0x7fb5540b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5540b2240 0x7fb5540b62a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb54400e8a0 tx=0x7fb54400eca0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 shutdown_connections 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb538066f60 0x7fb538069420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb5540ccde0 0x7fb5540c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5540b2240 0x7fb5540b62a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 --2- 172.21.15.46:0/1819775205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb554099a00 0x7fb554154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 >> 172.21.15.46:0/1819775205 conn(0x7fb5540b3120 msgr2=0x7fb5540ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 shutdown_connections 2024-04-25T08:46:31.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.393+0000 7fb55c800640 1 -- 172.21.15.46:0/1819775205 wait complete. 2024-04-25T08:46:31.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:45:10.412423+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:45:10.987443+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:46:31.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:46:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00afc00640 1 Processor -- start 2024-04-25T08:46:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00afc00640 1 -- start start 2024-04-25T08:46:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00afc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00a80cf210 0x7f00a8160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00afc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a80cc530 con 0x7f00a809a040 2024-04-25T08:46:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00afc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a80cc810 con 0x7f00a8099c60 2024-04-25T08:46:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00afc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a80cc300 con 0x7f00a80cf210 2024-04-25T08:46:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00a7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00a80cf210 0x7f00a8160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00adc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f00a809a040 0x7f00a815eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34888/0 (socket says 172.21.15.46:34888) 2024-04-25T08:46:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00a7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00a80cf210 0x7f00a8160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49212/0 (socket says 172.21.15.46:49212) 2024-04-25T08:46:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.557+0000 7f00adc00640 1 -- 172.21.15.46:0/798828939 learned_addr learned my addr 172.21.15.46:0/798828939 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00a7e00640 1 -- 172.21.15.46:0/798828939 >> v1:172.21.15.82:6789/0 conn(0x7f00a8099c60 legacy=0x7f00a80cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:31.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00a7e00640 1 -- 172.21.15.46:0/798828939 >> v1:172.21.15.46:6789/0 conn(0x7f00a809a040 legacy=0x7f00a815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00a7e00640 1 -- 172.21.15.46:0/798828939 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a80ccaf0 con 0x7f00a80cf210 2024-04-25T08:46:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00a7400640 1 -- 172.21.15.46:0/798828939 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029306581 0 0) 0x7f00a80cc530 con 0x7f00a809a040 2024-04-25T08:46:31.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00a7e00640 1 --2- 172.21.15.46:0/798828939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a80cf210 0x7f00a8160140 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f00940097c0 tx=0x7f009401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72faadd1836a313e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00a7400640 1 -- 172.21.15.46:0/798828939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00940b1070 con 0x7f00a80cf210 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00a7400640 1 -- 172.21.15.46:0/798828939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00940ac5f0 con 0x7f00a80cf210 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00a7400640 1 -- 172.21.15.46:0/798828939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00940ac9c0 con 0x7f00a80cf210 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00afc00640 1 -- 172.21.15.46:0/798828939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a80cf210 msgr2=0x7f00a8160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00afc00640 1 --2- 172.21.15.46:0/798828939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a80cf210 0x7f00a8160140 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f00940097c0 tx=0x7f009401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00afc00640 1 -- 172.21.15.46:0/798828939 shutdown_connections 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00afc00640 1 --2- 172.21.15.46:0/798828939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a80cf210 0x7f00a8160140 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00afc00640 1 -- 172.21.15.46:0/798828939 >> 172.21.15.46:0/798828939 conn(0x7f00a80b3130 msgr2=0x7f00a80b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00afc00640 1 -- 172.21.15.46:0/798828939 shutdown_connections 2024-04-25T08:46:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.561+0000 7f00afc00640 1 -- 172.21.15.46:0/798828939 wait complete. 2024-04-25T08:46:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 Processor -- start 2024-04-25T08:46:31.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 -- start start 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a8099c60 0x7f00a8166b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00a809a040 0x7f00a80b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a80cf210 0x7f00a815b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80b1b80 con 0x7f00a8099c60 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80b1a00 con 0x7f00a80cf210 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a8169a80 con 0x7f00a809a040 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00ad200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a8099c60 0x7f00a8166b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00ad200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a8099c60 0x7f00a8166b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49224/0 (socket says 172.21.15.46:49224) 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00ad200640 1 -- 172.21.15.46:0/2739670137 learned_addr learned my addr 172.21.15.46:0/2739670137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00a7e00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00a809a040 0x7f00a80b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00adc00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a80cf210 0x7f00a815b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00ad200640 1 -- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00a809a040 msgr2=0x7f00a80b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00ad200640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00a809a040 0x7f00a80b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00ad200640 1 -- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a80cf210 msgr2=0x7f00a815b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00ad200640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a80cf210 0x7f00a815b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00ad200640 1 -- 172.21.15.46:0/2739670137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a815bc90 con 0x7f00a8099c60 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00ad200640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a8099c60 0x7f00a8166b00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f009c00c500 tx=0x7f009c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00a4c00640 1 -- 172.21.15.46:0/2739670137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c016020 con 0x7f00a8099c60 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00a4c00640 1 -- 172.21.15.46:0/2739670137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c005220 con 0x7f00a8099c60 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00a4c00640 1 -- 172.21.15.46:0/2739670137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c007030 con 0x7f00a8099c60 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a80b4d30 con 0x7f00a8099c60 2024-04-25T08:46:31.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.565+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a80b52f0 con 0x7f00a8099c60 2024-04-25T08:46:31.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.569+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a8069010 con 0x7f00a8099c60 2024-04-25T08:46:31.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.569+0000 7f00a4c00640 1 -- 172.21.15.46:0/2739670137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f009c0071f0 con 0x7f00a8099c60 2024-04-25T08:46:31.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.569+0000 7f00a4c00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0088066f60 0x7f0088069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.569+0000 7f00a4c00640 1 -- 172.21.15.46:0/2739670137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f009c084c90 con 0x7f00a8099c60 2024-04-25T08:46:31.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.569+0000 7f00a7e00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0088066f60 0x7f0088069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.569+0000 7f00a7e00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0088066f60 0x7f0088069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f00940ae000 tx=0x7f0094008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:31.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.573+0000 7f00a4c00640 1 -- 172.21.15.46:0/2739670137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f009c020020 con 0x7f00a8099c60 2024-04-25T08:46:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00a80c82a0 con 0x7f00a8099c60 2024-04-25T08:46:31.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00a4c00640 1 -- 172.21.15.46:0/2739670137 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f009c055a10 con 0x7f00a8099c60 2024-04-25T08:46:31.731 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:46:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0088066f60 msgr2=0x7f0088069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0088066f60 0x7f0088069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f00940ae000 tx=0x7f0094008040 comp rx=0 tx=0).stop 2024-04-25T08:46:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a8099c60 msgr2=0x7f00a8166b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a8099c60 0x7f00a8166b00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f009c00c500 tx=0x7f009c00c9c0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 shutdown_connections 2024-04-25T08:46:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0088066f60 0x7f0088069420 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00a80cf210 0x7f00a815b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00a809a040 0x7f00a80b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 --2- 172.21.15.46:0/2739670137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00a8099c60 0x7f00a8166b00 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 >> 172.21.15.46:0/2739670137 conn(0x7f00a80b3130 msgr2=0x7f00a80ce050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 shutdown_connections 2024-04-25T08:46:31.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.737+0000 7f00afc00640 1 -- 172.21.15.46:0/2739670137 wait complete. 2024-04-25T08:46:31.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:46:31.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f3e00640 1 Processor -- start 2024-04-25T08:46:31.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f3e00640 1 -- start start 2024-04-25T08:46:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cf890 0x7f03ec0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03ec0cc180 0x7f03ec0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ec0b2d50 con 0x7f03ec0cf340 2024-04-25T08:46:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec0d01b0 con 0x7f03ec0cc180 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec15f2f0 con 0x7f03ec0cf890 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03ec0cc180 0x7f03ec0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f03ec0cc180 0x7f03ec0cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49234/0 (socket says 172.21.15.46:49234) 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f0a00640 1 -- 172.21.15.46:0/3563068902 learned_addr learned my addr 172.21.15.46:0/3563068902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f1400640 1 --2- 172.21.15.46:0/3563068902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cf890 0x7f03ec0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f0a00640 1 -- 172.21.15.46:0/3563068902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cf890 msgr2=0x7f03ec0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f0a00640 1 --2- 172.21.15.46:0/3563068902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cf890 0x7f03ec0b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f0a00640 1 -- 172.21.15.46:0/3563068902 >> v1:172.21.15.82:6789/0 conn(0x7f03ec0cf340 legacy=0x7f03ec0ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f0a00640 1 -- 172.21.15.46:0/3563068902 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec15f5d0 con 0x7f03ec0cc180 2024-04-25T08:46:31.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.901+0000 7f03f1400640 1 --2- 172.21.15.46:0/3563068902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cf890 0x7f03ec0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:31.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03f0a00640 1 --2- 172.21.15.46:0/3563068902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cc180 0x7f03ec0cfc70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009870 tx=0x7f03d801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a71045819e5fedd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:31.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03ebe00640 1 -- 172.21.15.46:0/3563068902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d80ab680 con 0x7f03ec0cc180 2024-04-25T08:46:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03ebe00640 1 -- 172.21.15.46:0/3563068902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d80b0070 con 0x7f03ec0cc180 2024-04-25T08:46:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03ebe00640 1 -- 172.21.15.46:0/3563068902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d80aba40 con 0x7f03ec0cc180 2024-04-25T08:46:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03f3e00640 1 -- 172.21.15.46:0/3563068902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cc180 msgr2=0x7f03ec0cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03f3e00640 1 --2- 172.21.15.46:0/3563068902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cc180 0x7f03ec0cfc70 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009870 tx=0x7f03d801c560 comp rx=0 tx=0).stop 2024-04-25T08:46:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03f3e00640 1 -- 172.21.15.46:0/3563068902 shutdown_connections 2024-04-25T08:46:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03f3e00640 1 --2- 172.21.15.46:0/3563068902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cc180 0x7f03ec0cfc70 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03f3e00640 1 --2- 172.21.15.46:0/3563068902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cf890 0x7f03ec0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03f3e00640 1 -- 172.21.15.46:0/3563068902 >> 172.21.15.46:0/3563068902 conn(0x7f03ec099a00 msgr2=0x7f03ec0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03f3e00640 1 -- 172.21.15.46:0/3563068902 shutdown_connections 2024-04-25T08:46:31.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.905+0000 7f03f3e00640 1 -- 172.21.15.46:0/3563068902 wait complete. 2024-04-25T08:46:31.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 Processor -- start 2024-04-25T08:46:31.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 -- start start 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cc180 0x7f03ec151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cf340 0x7f03ec15d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03ec0cf890 0x7f03ec15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec15cb80 con 0x7f03ec0cf340 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec168690 con 0x7f03ec0cf890 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec166080 con 0x7f03ec0cc180 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cf340 0x7f03ec15d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cf340 0x7f03ec15d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49238/0 (socket says 172.21.15.46:49238) 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f0a00640 1 -- 172.21.15.46:0/879557810 learned_addr learned my addr 172.21.15.46:0/879557810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f1e00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03ec0cf890 0x7f03ec15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f1400640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cc180 0x7f03ec151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f0a00640 1 -- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cc180 msgr2=0x7f03ec151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f0a00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cc180 0x7f03ec151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f0a00640 1 -- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03ec0cf890 msgr2=0x7f03ec15dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:31.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f0a00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03ec0cf890 0x7f03ec15dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f0a00640 1 -- 172.21.15.46:0/879557810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec151c70 con 0x7f03ec0cf340 2024-04-25T08:46:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f1400640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cc180 0x7f03ec151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:31.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f0a00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cf340 0x7f03ec15d7c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f03d80ad040 tx=0x7f03d801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03e9600640 1 -- 172.21.15.46:0/879557810 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d80be5f0 con 0x7f03ec0cf340 2024-04-25T08:46:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03e9600640 1 -- 172.21.15.46:0/879557810 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d80a7070 con 0x7f03ec0cf340 2024-04-25T08:46:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03e9600640 1 -- 172.21.15.46:0/879557810 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d80be9d0 con 0x7f03ec0cf340 2024-04-25T08:46:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec165470 con 0x7f03ec0cf340 2024-04-25T08:46:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.909+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec165a30 con 0x7f03ec0cf340 2024-04-25T08:46:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.913+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec069010 con 0x7f03ec0cf340 2024-04-25T08:46:31.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.913+0000 7f03e9600640 1 -- 172.21.15.46:0/879557810 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03d80c8420 con 0x7f03ec0cf340 2024-04-25T08:46:31.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.913+0000 7f03e9600640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f03cc066f60 0x7f03cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:31.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.913+0000 7f03e9600640 1 -- 172.21.15.46:0/879557810 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f03d811fa10 con 0x7f03ec0cf340 2024-04-25T08:46:31.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.913+0000 7f03f1400640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f03cc066f60 0x7f03cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:31.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.913+0000 7f03f1400640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f03cc066f60 0x7f03cc069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f03e000b990 tx=0x7f03e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:31.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:31.917+0000 7f03e9600640 1 -- 172.21.15.46:0/879557810 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03d80c4060 con 0x7f03ec0cf340 2024-04-25T08:46:32.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.077+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03ec15eea0 con 0x7f03ec0cf340 2024-04-25T08:46:32.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.077+0000 7f03e9600640 1 -- 172.21.15.46:0/879557810 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03d8016150 con 0x7f03ec0cf340 2024-04-25T08:46:32.072 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f03cc066f60 msgr2=0x7f03cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f03cc066f60 0x7f03cc069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f03e000b990 tx=0x7f03e0008040 comp rx=0 tx=0).stop 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cf340 msgr2=0x7f03ec15d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cf340 0x7f03ec15d7c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f03d80ad040 tx=0x7f03d801ada0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 shutdown_connections 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f03cc066f60 0x7f03cc069420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03ec0cf890 0x7f03ec15dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ec0cf340 0x7f03ec15d7c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 --2- 172.21.15.46:0/879557810 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03ec0cc180 0x7f03ec151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 >> 172.21.15.46:0/879557810 conn(0x7f03ec099a00 msgr2=0x7f03ec1550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 shutdown_connections 2024-04-25T08:46:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.081+0000 7f03f3e00640 1 -- 172.21.15.46:0/879557810 wait complete. 2024-04-25T08:46:32.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:46:32.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8f000640 1 Processor -- start 2024-04-25T08:46:32.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8f000640 1 -- start start 2024-04-25T08:46:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8f000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 0x7f3f8806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8f000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f880cdb70 0x7f3f880cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8f000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f880cc0e0 0x7f3f880cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8f000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880ce5f0 con 0x7f3f880cc0e0 2024-04-25T08:46:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8f000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880ce8d0 con 0x7f3f880cdb70 2024-04-25T08:46:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8f000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880cebb0 con 0x7f3f8815b870 2024-04-25T08:46:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f880cc0e0 0x7f3f880cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f87400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f880cdb70 0x7f3f880cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f87e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 0x7f3f8806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f87400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f880cdb70 0x7f3f880cdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:56578/0 (socket says 172.21.15.46:56578) 2024-04-25T08:46:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f87400640 1 -- 172.21.15.46:0/66185438 learned_addr learned my addr 172.21.15.46:0/66185438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8d000640 1 -- 172.21.15.46:0/66185438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 msgr2=0x7f3f8806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8d000640 1 --2- 172.21.15.46:0/66185438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 0x7f3f8806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8d000640 1 -- 172.21.15.46:0/66185438 >> v2:172.21.15.82:3300/0 conn(0x7f3f880cdb70 msgr2=0x7f3f880cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8d000640 1 --2- 172.21.15.46:0/66185438 >> v2:172.21.15.82:3300/0 conn(0x7f3f880cdb70 0x7f3f880cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f8d000640 1 -- 172.21.15.46:0/66185438 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f880b2900 con 0x7f3f880cc0e0 2024-04-25T08:46:32.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.245+0000 7f3f87400640 1 --2- 172.21.15.46:0/66185438 >> v2:172.21.15.82:3300/0 conn(0x7f3f880cdb70 0x7f3f880cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:46:32.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8d000640 1 --2- 172.21.15.46:0/66185438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 0x7f3f880cc590 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806d170 tx=0x7f3f8001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4850f84e590d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:32.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f86a00640 1 -- 172.21.15.46:0/66185438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f800b2070 con 0x7f3f880cc0e0 2024-04-25T08:46:32.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f86a00640 1 -- 172.21.15.46:0/66185438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f800a5720 con 0x7f3f880cc0e0 2024-04-25T08:46:32.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f86a00640 1 -- 172.21.15.46:0/66185438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f800a5af0 con 0x7f3f880cc0e0 2024-04-25T08:46:32.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 -- 172.21.15.46:0/66185438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 msgr2=0x7f3f880cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 --2- 172.21.15.46:0/66185438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 0x7f3f880cc590 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806d170 tx=0x7f3f8001c8a0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 -- 172.21.15.46:0/66185438 shutdown_connections 2024-04-25T08:46:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 --2- 172.21.15.46:0/66185438 >> v2:172.21.15.82:3300/0 conn(0x7f3f880cdb70 0x7f3f880cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 --2- 172.21.15.46:0/66185438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 0x7f3f880cc590 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 --2- 172.21.15.46:0/66185438 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 0x7f3f8806d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 -- 172.21.15.46:0/66185438 >> 172.21.15.46:0/66185438 conn(0x7f3f8806ac10 msgr2=0x7f3f8806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 -- 172.21.15.46:0/66185438 shutdown_connections 2024-04-25T08:46:32.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 -- 172.21.15.46:0/66185438 wait complete. 2024-04-25T08:46:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.249+0000 7f3f8f000640 1 Processor -- start 2024-04-25T08:46:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 -- start start 2024-04-25T08:46:32.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 0x7f3f88158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f880cdb70 0x7f3f88155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 0x7f3f880af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88071e10 con 0x7f3f880cc0e0 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88071c90 con 0x7f3f880cdb70 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88071f90 con 0x7f3f8815b870 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 0x7f3f88158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 0x7f3f88158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49250/0 (socket says 172.21.15.46:49250) 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87e00640 1 -- 172.21.15.46:0/916079569 learned_addr learned my addr 172.21.15.46:0/916079569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8d000640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 0x7f3f880af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87400640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f880cdb70 0x7f3f88155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87e00640 1 -- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 msgr2=0x7f3f880af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87e00640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 0x7f3f880af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87e00640 1 -- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f880cdb70 msgr2=0x7f3f88155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87e00640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f880cdb70 0x7f3f88155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87e00640 1 -- 172.21.15.46:0/916079569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f88156000 con 0x7f3f880cc0e0 2024-04-25T08:46:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f87e00640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 0x7f3f88158800 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800e930 tx=0x7f3f7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f7fe00640 1 -- 172.21.15.46:0/916079569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78017070 con 0x7f3f880cc0e0 2024-04-25T08:46:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f7fe00640 1 -- 172.21.15.46:0/916079569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7800ca00 con 0x7f3f880cc0e0 2024-04-25T08:46:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f7fe00640 1 -- 172.21.15.46:0/916079569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800cdb0 con 0x7f3f880cc0e0 2024-04-25T08:46:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f881562e0 con 0x7f3f880cc0e0 2024-04-25T08:46:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f880af8d0 con 0x7f3f880cc0e0 2024-04-25T08:46:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.253+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f88069010 con 0x7f3f880cc0e0 2024-04-25T08:46:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.257+0000 7f3f7fe00640 1 -- 172.21.15.46:0/916079569 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f7801a070 con 0x7f3f880cc0e0 2024-04-25T08:46:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.257+0000 7f3f7fe00640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f68066f60 0x7f3f68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.257+0000 7f3f7fe00640 1 -- 172.21.15.46:0/916079569 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f3f78085580 con 0x7f3f880cc0e0 2024-04-25T08:46:32.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.257+0000 7f3f87400640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f68066f60 0x7f3f68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.257+0000 7f3f87400640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f68066f60 0x7f3f68069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3f88158040 tx=0x7f3f74009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:32.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.261+0000 7f3f7fe00640 1 -- 172.21.15.46:0/916079569 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f78055d60 con 0x7f3f880cc0e0 2024-04-25T08:46:32.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.421+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f8815aa50 con 0x7f3f880cc0e0 2024-04-25T08:46:32.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.421+0000 7f3f7fe00640 1 -- 172.21.15.46:0/916079569 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f78059e00 con 0x7f3f880cc0e0 2024-04-25T08:46:32.416 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:46:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f68066f60 msgr2=0x7f3f68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f68066f60 0x7f3f68069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3f88158040 tx=0x7f3f74009240 comp rx=0 tx=0).stop 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 msgr2=0x7f3f88158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 0x7f3f88158800 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800e930 tx=0x7f3f7800edf0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 shutdown_connections 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3f68066f60 0x7f3f68069420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f8815b870 0x7f3f880af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f880cdb70 0x7f3f88155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 --2- 172.21.15.46:0/916079569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880cc0e0 0x7f3f88158800 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 >> 172.21.15.46:0/916079569 conn(0x7f3f8806ac10 msgr2=0x7f3f88159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 shutdown_connections 2024-04-25T08:46:32.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.425+0000 7f3f8f000640 1 -- 172.21.15.46:0/916079569 wait complete. 2024-04-25T08:46:32.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.585+0000 7f66a7200640 1 Processor -- start 2024-04-25T08:46:32.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.585+0000 7f66a7200640 1 -- start start 2024-04-25T08:46:32.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f66a7200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f66a015a750 0x7f66a006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f66a7200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a006d5e0 con 0x7f66a00aed10 2024-04-25T08:46:32.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f66a7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a00ce720 con 0x7f66a00cc9e0 2024-04-25T08:46:32.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f66a7200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a015ab30 con 0x7f66a015a750 2024-04-25T08:46:32.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f6697400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f66a00aed10 0x7f66a00b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34904/0 (socket says 172.21.15.46:34904) 2024-04-25T08:46:32.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f6697400640 1 -- 172.21.15.46:0/417672861 learned_addr learned my addr 172.21.15.46:0/417672861 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f669fe00640 1 --2- 172.21.15.46:0/417672861 >> v2:172.21.15.82:3300/0 conn(0x7f66a015a750 0x7f66a006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f669f400640 1 -- 172.21.15.46:0/417672861 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132212625 0 0) 0x7f66a006d5e0 con 0x7f66a00aed10 2024-04-25T08:46:32.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f669f400640 1 -- 172.21.15.46:0/417672861 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6684003530 con 0x7f66a00aed10 2024-04-25T08:46:32.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f669fe00640 1 -- 172.21.15.46:0/417672861 >> v1:172.21.15.82:6789/0 conn(0x7f66a00cc9e0 legacy=0x7f66a00b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f669f400640 1 -- 172.21.15.46:0/417672861 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 378741041 0 0) 0x7f6684003530 con 0x7f66a00aed10 2024-04-25T08:46:32.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f669fe00640 1 -- 172.21.15.46:0/417672861 >> v1:172.21.15.46:6789/0 conn(0x7f66a00aed10 legacy=0x7f66a00b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.589+0000 7f669fe00640 1 -- 172.21.15.46:0/417672861 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a00cea00 con 0x7f66a015a750 2024-04-25T08:46:32.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f669fe00640 1 --2- 172.21.15.46:0/417672861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a015a750 0x7f66a006ec10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f66900097c0 tx=0x7f669001c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7183886d1ed6f330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f669f400640 1 -- 172.21.15.46:0/417672861 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900b0070 con 0x7f66a015a750 2024-04-25T08:46:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f669f400640 1 -- 172.21.15.46:0/417672861 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66900ab400 con 0x7f66a015a750 2024-04-25T08:46:32.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f669f400640 1 -- 172.21.15.46:0/417672861 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688001a00 con 0x7f66a015a750 2024-04-25T08:46:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f66a7200640 1 -- 172.21.15.46:0/417672861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a015a750 msgr2=0x7f66a006ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f66a7200640 1 --2- 172.21.15.46:0/417672861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a015a750 0x7f66a006ec10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f66900097c0 tx=0x7f669001c270 comp rx=0 tx=0).stop 2024-04-25T08:46:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f66a7200640 1 -- 172.21.15.46:0/417672861 shutdown_connections 2024-04-25T08:46:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f66a7200640 1 --2- 172.21.15.46:0/417672861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a015a750 0x7f66a006ec10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f66a7200640 1 -- 172.21.15.46:0/417672861 >> 172.21.15.46:0/417672861 conn(0x7f66a00b12a0 msgr2=0x7f66a00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f66a7200640 1 -- 172.21.15.46:0/417672861 shutdown_connections 2024-04-25T08:46:32.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f66a7200640 1 -- 172.21.15.46:0/417672861 wait complete. 2024-04-25T08:46:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f66a7200640 1 Processor -- start 2024-04-25T08:46:32.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.593+0000 7f66a7200640 1 -- start start 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a00aed10 0x7f66a0152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a00cc9e0 0x7f66a014b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a7200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f66a015a750 0x7f66a014a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a006d8c0 con 0x7f66a00cc9e0 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a00cf200 con 0x7f66a00aed10 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a7200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a006d270 con 0x7f66a015a750 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f6697400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a00cc9e0 0x7f66a014b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f6697400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a00cc9e0 0x7f66a014b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49264/0 (socket says 172.21.15.46:49264) 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f6697400640 1 -- 172.21.15.46:0/1949009840 learned_addr learned my addr 172.21.15.46:0/1949009840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f669fe00640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a00aed10 0x7f66a0152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a5200640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f66a015a750 0x7f66a014a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f6697400640 1 -- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f66a015a750 msgr2=0x7f66a014a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f6697400640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f66a015a750 0x7f66a014a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f6697400640 1 -- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a00aed10 msgr2=0x7f66a0152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f6697400640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a00aed10 0x7f66a0152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f6697400640 1 -- 172.21.15.46:0/1949009840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a01526a0 con 0x7f66a00cc9e0 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f669fe00640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a00aed10 0x7f66a0152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f6697400640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a00cc9e0 0x7f66a014b200 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f668800cc00 tx=0x7f668800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f669cc00640 1 -- 172.21.15.46:0/1949009840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688017070 con 0x7f66a00cc9e0 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f669cc00640 1 -- 172.21.15.46:0/1949009840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668800ad40 con 0x7f66a00cc9e0 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f669cc00640 1 -- 172.21.15.46:0/1949009840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688012620 con 0x7f66a00cc9e0 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a0149a80 con 0x7f66a00cc9e0 2024-04-25T08:46:32.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a014a040 con 0x7f66a00cc9e0 2024-04-25T08:46:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a014a240 con 0x7f66a00cc9e0 2024-04-25T08:46:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.597+0000 7f669cc00640 1 -- 172.21.15.46:0/1949009840 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f668801a070 con 0x7f66a00cc9e0 2024-04-25T08:46:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.601+0000 7f669cc00640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f667c066fb0 0x7f667c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.601+0000 7f669cc00640 1 -- 172.21.15.46:0/1949009840 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6688084f80 con 0x7f66a00cc9e0 2024-04-25T08:46:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.601+0000 7f669fe00640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f667c066fb0 0x7f667c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.601+0000 7f669fe00640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f667c066fb0 0x7f667c069470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66900ad000 tx=0x7f66900a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:32.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.605+0000 7f669cc00640 1 -- 172.21.15.46:0/1949009840 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66a014a240 con 0x7f66a00cc9e0 2024-04-25T08:46:32.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.801+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66a0069080 con 0x7f66a00cc9e0 2024-04-25T08:46:32.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.801+0000 7f669cc00640 1 -- 172.21.15.46:0/1949009840 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f6688055720 con 0x7f66a00cc9e0 2024-04-25T08:46:32.796 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:32.796 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:45:11.989847+0000","modified":"2024-04-25T08:46:00.651586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6207},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6207":{"gid":6207,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/1670658295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670658295},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670658295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:46:32.796 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-04-25T08:46:32.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f667c066fb0 msgr2=0x7f667c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f667c066fb0 0x7f667c069470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f66900ad000 tx=0x7f66900a6000 comp rx=0 tx=0).stop 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a00cc9e0 msgr2=0x7f66a014b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a00cc9e0 0x7f66a014b200 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f668800cc00 tx=0x7f668800a430 comp rx=0 tx=0).stop 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 shutdown_connections 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f667c066fb0 0x7f667c069470 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f66a015a750 0x7f66a014a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a00cc9e0 0x7f66a014b200 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 --2- 172.21.15.46:0/1949009840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66a00aed10 0x7f66a0152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 >> 172.21.15.46:0/1949009840 conn(0x7f66a00b12a0 msgr2=0x7f66a00b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 shutdown_connections 2024-04-25T08:46:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.805+0000 7f66a7200640 1 -- 172.21.15.46:0/1949009840 wait complete. 2024-04-25T08:46:32.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:32.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bf800640 1 Processor -- start 2024-04-25T08:46:32.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bf800640 1 -- start start 2024-04-25T08:46:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bf800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 0x7f22b8099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bf800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22b80ccde0 0x7f22b80cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bf800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f22b80b2240 0x7f22b80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bf800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b809a320 con 0x7f22b80ccde0 2024-04-25T08:46:32.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bf800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b806bdc0 con 0x7f22b80b2240 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bf800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b806c0a0 con 0x7f22b8099a00 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22b80ccde0 0x7f22b80cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22b80ccde0 0x7f22b80cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49266/0 (socket says 172.21.15.46:49266) 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22afe00640 1 -- 172.21.15.46:0/2239841845 learned_addr learned my addr 172.21.15.46:0/2239841845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bd800640 1 --2- 172.21.15.46:0/2239841845 >> v2:172.21.15.82:3300/0 conn(0x7f22b80b2240 0x7f22b80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bce00640 1 --2- 172.21.15.46:0/2239841845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 0x7f22b8099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22afe00640 1 -- 172.21.15.46:0/2239841845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 msgr2=0x7f22b8099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22afe00640 1 --2- 172.21.15.46:0/2239841845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 0x7f22b8099de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22afe00640 1 -- 172.21.15.46:0/2239841845 >> v2:172.21.15.82:3300/0 conn(0x7f22b80b2240 msgr2=0x7f22b80b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22afe00640 1 --2- 172.21.15.46:0/2239841845 >> v2:172.21.15.82:3300/0 conn(0x7f22b80b2240 0x7f22b80b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22afe00640 1 -- 172.21.15.46:0/2239841845 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b806c380 con 0x7f22b80ccde0 2024-04-25T08:46:32.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.965+0000 7f22bd800640 1 --2- 172.21.15.46:0/2239841845 >> v2:172.21.15.82:3300/0 conn(0x7f22b80b2240 0x7f22b80b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22afe00640 1 --2- 172.21.15.46:0/2239841845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 0x7f22b80cd260 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f22b80b3cd0 tx=0x7f22a001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed85b5301462fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:32.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22b7e00640 1 -- 172.21.15.46:0/2239841845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00b1070 con 0x7f22b80ccde0 2024-04-25T08:46:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22b7e00640 1 -- 172.21.15.46:0/2239841845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a00ac410 con 0x7f22b80ccde0 2024-04-25T08:46:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22b7e00640 1 -- 172.21.15.46:0/2239841845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00ac7e0 con 0x7f22b80ccde0 2024-04-25T08:46:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22bf800640 1 -- 172.21.15.46:0/2239841845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 msgr2=0x7f22b80cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22bf800640 1 --2- 172.21.15.46:0/2239841845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 0x7f22b80cd260 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f22b80b3cd0 tx=0x7f22a001c460 comp rx=0 tx=0).stop 2024-04-25T08:46:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22bf800640 1 -- 172.21.15.46:0/2239841845 shutdown_connections 2024-04-25T08:46:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22bf800640 1 --2- 172.21.15.46:0/2239841845 >> v2:172.21.15.82:3300/0 conn(0x7f22b80b2240 0x7f22b80b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22bf800640 1 --2- 172.21.15.46:0/2239841845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 0x7f22b80cd260 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22bf800640 1 --2- 172.21.15.46:0/2239841845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 0x7f22b8099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22bf800640 1 -- 172.21.15.46:0/2239841845 >> 172.21.15.46:0/2239841845 conn(0x7f22b80b3120 msgr2=0x7f22b80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22bf800640 1 -- 172.21.15.46:0/2239841845 shutdown_connections 2024-04-25T08:46:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.969+0000 7f22bf800640 1 -- 172.21.15.46:0/2239841845 wait complete. 2024-04-25T08:46:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 Processor -- start 2024-04-25T08:46:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 -- start start 2024-04-25T08:46:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 0x7f22b8157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b80b2240 0x7f22b8154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 0x7f22b8154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80cd920 con 0x7f22b80ccde0 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80cd7a0 con 0x7f22b80b2240 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b815dd10 con 0x7f22b8099a00 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 0x7f22b8154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 0x7f22b8154a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49276/0 (socket says 172.21.15.46:49276) 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 0x7f22b8157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bd800640 1 -- 172.21.15.46:0/1386166623 learned_addr learned my addr 172.21.15.46:0/1386166623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22afe00640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b80b2240 0x7f22b8154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bd800640 1 -- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 msgr2=0x7f22b8157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bd800640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 0x7f22b8157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bd800640 1 -- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b80b2240 msgr2=0x7f22b8154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bd800640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b80b2240 0x7f22b8154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:32.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bd800640 1 -- 172.21.15.46:0/1386166623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b8153750 con 0x7f22b80ccde0 2024-04-25T08:46:32.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bd800640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 0x7f22b8154a50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f22b0007ef0 tx=0x7f22b000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:32.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22b5600640 1 -- 172.21.15.46:0/1386166623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b001a070 con 0x7f22b80ccde0 2024-04-25T08:46:32.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22b5600640 1 -- 172.21.15.46:0/1386166623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b0003ba0 con 0x7f22b80ccde0 2024-04-25T08:46:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b8153a30 con 0x7f22b80ccde0 2024-04-25T08:46:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22b5600640 1 -- 172.21.15.46:0/1386166623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0004030 con 0x7f22b80ccde0 2024-04-25T08:46:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.973+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b8153ff0 con 0x7f22b80ccde0 2024-04-25T08:46:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.977+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b8069080 con 0x7f22b80ccde0 2024-04-25T08:46:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.977+0000 7f22b5600640 1 -- 172.21.15.46:0/1386166623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22b001d070 con 0x7f22b80ccde0 2024-04-25T08:46:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.977+0000 7f22b5600640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2294066f60 0x7f2294069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.977+0000 7f22b5600640 1 -- 172.21.15.46:0/1386166623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f22b0085460 con 0x7f22b80ccde0 2024-04-25T08:46:32.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.977+0000 7f22bce00640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2294066f60 0x7f2294069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:32.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.977+0000 7f22bce00640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2294066f60 0x7f2294069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f22a80045f0 tx=0x7f22a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:32.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:32.981+0000 7f22b5600640 1 -- 172.21.15.46:0/1386166623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22b0010070 con 0x7f22b80ccde0 2024-04-25T08:46:33.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.145+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22b8067cb0 con 0x7f22b80ccde0 2024-04-25T08:46:33.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.145+0000 7f22b5600640 1 -- 172.21.15.46:0/1386166623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17028 (secure 0 0 0) 0x7f22b0055c20 con 0x7f22b80ccde0 2024-04-25T08:46:33.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:33.141 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:03.180374+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:45:10.412423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:45:10.987443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2294066f60 msgr2=0x7f2294069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2294066f60 0x7f2294069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f22a80045f0 tx=0x7f22a8008040 comp rx=0 tx=0).stop 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 msgr2=0x7f22b8154a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 0x7f22b8154a50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f22b0007ef0 tx=0x7f22b000d4a0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 shutdown_connections 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2294066f60 0x7f2294069420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22b80ccde0 0x7f22b8154a50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b80b2240 0x7f22b8154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 --2- 172.21.15.46:0/1386166623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22b8099a00 0x7f22b8157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 >> 172.21.15.46:0/1386166623 conn(0x7f22b80b3120 msgr2=0x7f22b80ce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:33.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 shutdown_connections 2024-04-25T08:46:33.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.149+0000 7f22bf800640 1 -- 172.21.15.46:0/1386166623 wait complete. 2024-04-25T08:46:33.163 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:46:33.163 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:33.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.309+0000 7f3c91200640 1 Processor -- start 2024-04-25T08:46:33.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.309+0000 7f3c91200640 1 -- start start 2024-04-25T08:46:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.309+0000 7f3c91200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c8c15dbd0 0x7f3c8c0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.309+0000 7f3c91200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 0x7f3c8c070c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.309+0000 7f3c91200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8c0b5250 con 0x7f3c8c0711d0 2024-04-25T08:46:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.309+0000 7f3c91200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8c0b5af0 con 0x7f3c8c15dbd0 2024-04-25T08:46:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.309+0000 7f3c91200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c0b5530 con 0x7f3c8c0b0c30 2024-04-25T08:46:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c8c15dbd0 0x7f3c8c0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:33.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c8c15dbd0 0x7f3c8c0b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49288/0 (socket says 172.21.15.46:49288) 2024-04-25T08:46:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89e00640 1 -- 172.21.15.46:0/2948509300 learned_addr learned my addr 172.21.15.46:0/2948509300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c83e00640 1 --2- 172.21.15.46:0/2948509300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 0x7f3c8c070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89e00640 1 -- 172.21.15.46:0/2948509300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 msgr2=0x7f3c8c070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89e00640 1 --2- 172.21.15.46:0/2948509300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 0x7f3c8c070c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89e00640 1 -- 172.21.15.46:0/2948509300 >> v1:172.21.15.46:6789/0 conn(0x7f3c8c0711d0 legacy=0x7f3c8c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89e00640 1 -- 172.21.15.46:0/2948509300 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c0b5810 con 0x7f3c8c15dbd0 2024-04-25T08:46:33.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89400640 1 -- 172.21.15.46:0/2948509300 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057291275 0 0) 0x7f3c8c0b5250 con 0x7f3c8c0711d0 2024-04-25T08:46:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89e00640 1 --2- 172.21.15.46:0/2948509300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8c15dbd0 0x7f3c8c0b06f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009870 tx=0x7f3c7c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c374de434d87e158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89400640 1 -- 172.21.15.46:0/2948509300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0b2070 con 0x7f3c8c15dbd0 2024-04-25T08:46:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89400640 1 -- 172.21.15.46:0/2948509300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c0a9070 con 0x7f3c8c15dbd0 2024-04-25T08:46:33.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.313+0000 7f3c89400640 1 -- 172.21.15.46:0/2948509300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0ad880 con 0x7f3c8c15dbd0 2024-04-25T08:46:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 -- 172.21.15.46:0/2948509300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8c15dbd0 msgr2=0x7f3c8c0b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 --2- 172.21.15.46:0/2948509300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8c15dbd0 0x7f3c8c0b06f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009870 tx=0x7f3c7c01c560 comp rx=0 tx=0).stop 2024-04-25T08:46:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 -- 172.21.15.46:0/2948509300 shutdown_connections 2024-04-25T08:46:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 --2- 172.21.15.46:0/2948509300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 0x7f3c8c070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 --2- 172.21.15.46:0/2948509300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8c15dbd0 0x7f3c8c0b06f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 -- 172.21.15.46:0/2948509300 >> 172.21.15.46:0/2948509300 conn(0x7f3c8c06f4f0 msgr2=0x7f3c8c06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 -- 172.21.15.46:0/2948509300 shutdown_connections 2024-04-25T08:46:33.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 -- 172.21.15.46:0/2948509300 wait complete. 2024-04-25T08:46:33.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 Processor -- start 2024-04-25T08:46:33.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 -- start start 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c8c0711d0 0x7f3c8c0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 0x7f3c8c13ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8c15dbd0 0x7f3c8c13f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c071620 con 0x7f3c8c15dbd0 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c0b1010 con 0x7f3c8c0711d0 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c91200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c16a850 con 0x7f3c8c0b0c30 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c89e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c8c0711d0 0x7f3c8c0cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c8a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8c15dbd0 0x7f3c8c13f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c89e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c8c0711d0 0x7f3c8c0cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:56630/0 (socket says 172.21.15.46:56630) 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c89e00640 1 -- 172.21.15.46:0/3254980102 learned_addr learned my addr 172.21.15.46:0/3254980102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c83e00640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 0x7f3c8c13ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c89e00640 1 -- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 msgr2=0x7f3c8c13ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c89e00640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 0x7f3c8c13ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c89e00640 1 -- 172.21.15.46:0/3254980102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8c15dbd0 msgr2=0x7f3c8c13f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c89e00640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8c15dbd0 0x7f3c8c13f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.317+0000 7f3c89e00640 1 -- 172.21.15.46:0/3254980102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c0cff20 con 0x7f3c8c0711d0 2024-04-25T08:46:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.321+0000 7f3c89e00640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c8c0711d0 0x7f3c8c0cac80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0b0040 tx=0x7f3c7c01ada0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.321+0000 7f3c82000640 1 -- 172.21.15.46:0/3254980102 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0bc020 con 0x7f3c8c0711d0 2024-04-25T08:46:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.321+0000 7f3c82000640 1 -- 172.21.15.46:0/3254980102 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c0b8620 con 0x7f3c8c0711d0 2024-04-25T08:46:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.321+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8c0cd500 con 0x7f3c8c0711d0 2024-04-25T08:46:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.321+0000 7f3c82000640 1 -- 172.21.15.46:0/3254980102 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0b2040 con 0x7f3c8c0711d0 2024-04-25T08:46:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.321+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8c155460 con 0x7f3c8c0711d0 2024-04-25T08:46:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.321+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8c069080 con 0x7f3c8c0711d0 2024-04-25T08:46:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.321+0000 7f3c82000640 1 -- 172.21.15.46:0/3254980102 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c7c0bf020 con 0x7f3c8c0711d0 2024-04-25T08:46:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.321+0000 7f3c82000640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c68066f60 0x7f3c68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.325+0000 7f3c82000640 1 -- 172.21.15.46:0/3254980102 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f3c7c11f930 con 0x7f3c8c0711d0 2024-04-25T08:46:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.325+0000 7f3c83e00640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c68066f60 0x7f3c68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:33.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.325+0000 7f3c83e00640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c68066f60 0x7f3c68069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c1519e0 tx=0x7f3c74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.329+0000 7f3c82000640 1 -- 172.21.15.46:0/3254980102 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c7c0c6030 con 0x7f3c8c0711d0 2024-04-25T08:46:33.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c8c1603a0 con 0x7f3c8c0711d0 2024-04-25T08:46:33.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c82000640 1 -- 172.21.15.46:0/3254980102 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f3c7c016150 con 0x7f3c8c0711d0 2024-04-25T08:46:33.519 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:33.519 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:45:11.989847+0000","modified":"2024-04-25T08:46:00.651586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6207},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6207":{"gid":6207,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/1670658295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670658295},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670658295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:46:33.519 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c68066f60 msgr2=0x7f3c68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c68066f60 0x7f3c68069420 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c1519e0 tx=0x7f3c74008040 comp rx=0 tx=0).stop 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c8c0711d0 msgr2=0x7f3c8c0cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c8c0711d0 0x7f3c8c0cac80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0b0040 tx=0x7f3c7c01ada0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 shutdown_connections 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c68066f60 0x7f3c68069420 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c8c15dbd0 0x7f3c8c13f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c8c0b0c30 0x7f3c8c13ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 --2- 172.21.15.46:0/3254980102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c8c0711d0 0x7f3c8c0cac80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 >> 172.21.15.46:0/3254980102 conn(0x7f3c8c06f4f0 msgr2=0x7f3c8c1608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 shutdown_connections 2024-04-25T08:46:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.525+0000 7f3c91200640 1 -- 172.21.15.46:0/3254980102 wait complete. 2024-04-25T08:46:33.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:33.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.685+0000 7f9c03c00640 1 Processor -- start 2024-04-25T08:46:33.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.685+0000 7f9c03c00640 1 -- start start 2024-04-25T08:46:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9c03c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9bfc15a750 0x7f9bfc06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9c03c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc06d5e0 con 0x7f9bfc0aed10 2024-04-25T08:46:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9c03c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0ce720 con 0x7f9bfc0cc9e0 2024-04-25T08:46:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9c03c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc15ab30 con 0x7f9bfc15a750 2024-04-25T08:46:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9bf3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9bfc0aed10 0x7f9bfc0b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34916/0 (socket says 172.21.15.46:34916) 2024-04-25T08:46:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9bf3e00640 1 -- 172.21.15.46:0/1063040212 learned_addr learned my addr 172.21.15.46:0/1063040212 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9c01200640 1 --2- 172.21.15.46:0/1063040212 >> v2:172.21.15.82:3300/0 conn(0x7f9bfc15a750 0x7f9bfc06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9bfbe00640 1 -- 172.21.15.46:0/1063040212 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797797456 0 0) 0x7f9bfc06d5e0 con 0x7f9bfc0aed10 2024-04-25T08:46:33.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9bfbe00640 1 -- 172.21.15.46:0/1063040212 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be0003530 con 0x7f9bfc0aed10 2024-04-25T08:46:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9bfbe00640 1 -- 172.21.15.46:0/1063040212 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987682085 0 0) 0x7f9bfc0ce720 con 0x7f9bfc0cc9e0 2024-04-25T08:46:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9c01200640 1 -- 172.21.15.46:0/1063040212 >> v1:172.21.15.82:6789/0 conn(0x7f9bfc0cc9e0 legacy=0x7f9bfc0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9c01200640 1 -- 172.21.15.46:0/1063040212 >> v1:172.21.15.46:6789/0 conn(0x7f9bfc0aed10 legacy=0x7f9bfc0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.689+0000 7f9c01200640 1 -- 172.21.15.46:0/1063040212 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0cea00 con 0x7f9bfc15a750 2024-04-25T08:46:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9c01200640 1 --2- 172.21.15.46:0/1063040212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bfc15a750 0x7f9bfc06ec10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0097c0 tx=0x7f9bec01c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=207a78a78f253b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9bfbe00640 1 -- 172.21.15.46:0/1063040212 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0b0070 con 0x7f9bfc15a750 2024-04-25T08:46:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9bfbe00640 1 -- 172.21.15.46:0/1063040212 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec0ab400 con 0x7f9bfc15a750 2024-04-25T08:46:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9bfbe00640 1 -- 172.21.15.46:0/1063040212 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be4001680 con 0x7f9bfc15a750 2024-04-25T08:46:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9c03c00640 1 -- 172.21.15.46:0/1063040212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bfc15a750 msgr2=0x7f9bfc06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9c03c00640 1 --2- 172.21.15.46:0/1063040212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bfc15a750 0x7f9bfc06ec10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0097c0 tx=0x7f9bec01c2e0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9c03c00640 1 -- 172.21.15.46:0/1063040212 shutdown_connections 2024-04-25T08:46:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9c03c00640 1 --2- 172.21.15.46:0/1063040212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bfc15a750 0x7f9bfc06ec10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9c03c00640 1 -- 172.21.15.46:0/1063040212 >> 172.21.15.46:0/1063040212 conn(0x7f9bfc0b12a0 msgr2=0x7f9bfc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9c03c00640 1 -- 172.21.15.46:0/1063040212 shutdown_connections 2024-04-25T08:46:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9c03c00640 1 -- 172.21.15.46:0/1063040212 wait complete. 2024-04-25T08:46:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.693+0000 7f9c03c00640 1 Processor -- start 2024-04-25T08:46:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 -- start start 2024-04-25T08:46:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0aed10 0x7f9bfc14b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bfc0cc9e0 0x7f9bfc147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bfc15a750 0x7f9bfc136d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc168a50 con 0x7f9bfc0aed10 2024-04-25T08:46:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc06f560 con 0x7f9bfc0cc9e0 2024-04-25T08:46:33.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc168bd0 con 0x7f9bfc15a750 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c01200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0aed10 0x7f9bfc14b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c01200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0aed10 0x7f9bfc14b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49298/0 (socket says 172.21.15.46:49298) 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c01200640 1 -- 172.21.15.46:0/678737799 learned_addr learned my addr 172.21.15.46:0/678737799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c01200640 1 -- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bfc15a750 msgr2=0x7f9bfc136d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c01200640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bfc15a750 0x7f9bfc136d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c01200640 1 -- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bfc0cc9e0 msgr2=0x7f9bfc147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c01200640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bfc0cc9e0 0x7f9bfc147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c01200640 1 -- 172.21.15.46:0/678737799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc14a5a0 con 0x7f9bfc0aed10 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c01200640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0aed10 0x7f9bfc14b640 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0ae040 tx=0x7f9bec01ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9bf9600640 1 -- 172.21.15.46:0/678737799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0b0050 con 0x7f9bfc0aed10 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9bf9600640 1 -- 172.21.15.46:0/678737799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec0bc070 con 0x7f9bfc0aed10 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9bf9600640 1 -- 172.21.15.46:0/678737799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0b7890 con 0x7f9bfc0aed10 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc14a880 con 0x7f9bfc0aed10 2024-04-25T08:46:33.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc14ae40 con 0x7f9bfc0aed10 2024-04-25T08:46:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.697+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc069080 con 0x7f9bfc0aed10 2024-04-25T08:46:33.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.701+0000 7f9bf9600640 1 -- 172.21.15.46:0/678737799 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bec0b7a30 con 0x7f9bfc0aed10 2024-04-25T08:46:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.701+0000 7f9bf9600640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bd8066f60 0x7f9bd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.701+0000 7f9bf9600640 1 -- 172.21.15.46:0/678737799 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9bec11fb20 con 0x7f9bfc0aed10 2024-04-25T08:46:33.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.701+0000 7f9bf3e00640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bd8066f60 0x7f9bd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:33.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.701+0000 7f9bf3e00640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bd8066f60 0x7f9bd8069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9be40017f0 tx=0x7f9be4001500 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:33.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.705+0000 7f9bf9600640 1 -- 172.21.15.46:0/678737799 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bec015f90 con 0x7f9bfc0aed10 2024-04-25T08:46:33.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.897+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9bfc06e8f0 con 0x7f9bfc0aed10 2024-04-25T08:46:33.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9bf9600640 1 -- 172.21.15.46:0/678737799 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f9bec0f09b0 con 0x7f9bfc0aed10 2024-04-25T08:46:33.893 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:33.893 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:45:11.989847+0000","modified":"2024-04-25T08:46:00.651586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6207},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6207":{"gid":6207,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/1670658295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1670658295},{"type":"v1","addr":"172.21.15.46:6835","nonce":1670658295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:46:33.894 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-04-25T08:46:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bd8066f60 msgr2=0x7f9bd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bd8066f60 0x7f9bd8069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9be40017f0 tx=0x7f9be4001500 comp rx=0 tx=0).stop 2024-04-25T08:46:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0aed10 msgr2=0x7f9bfc14b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:33.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0aed10 0x7f9bfc14b640 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0ae040 tx=0x7f9bec01ab20 comp rx=0 tx=0).stop 2024-04-25T08:46:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 shutdown_connections 2024-04-25T08:46:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bd8066f60 0x7f9bd8069420 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bfc15a750 0x7f9bfc136d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bfc0cc9e0 0x7f9bfc147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 --2- 172.21.15.46:0/678737799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bfc0aed10 0x7f9bfc14b640 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 >> 172.21.15.46:0/678737799 conn(0x7f9bfc0b12a0 msgr2=0x7f9bfc0ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 shutdown_connections 2024-04-25T08:46:33.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:33.901+0000 7f9c03c00640 1 -- 172.21.15.46:0/678737799 wait complete. 2024-04-25T08:46:33.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:34.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.061+0000 7f557e400640 1 Processor -- start 2024-04-25T08:46:34.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.061+0000 7f557e400640 1 -- start start 2024-04-25T08:46:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f557e400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f557815d710 0x7f55780b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f557e400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55780b9530 0x7f55780ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f557e400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55780d1ff0 con 0x7f557815daf0 2024-04-25T08:46:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f557e400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55780b72b0 con 0x7f55780b9530 2024-04-25T08:46:34.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f557e400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55780b7590 con 0x7f557815d710 2024-04-25T08:46:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f5577e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55780b9530 0x7f55780ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f5577e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55780b9530 0x7f55780ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49306/0 (socket says 172.21.15.46:49306) 2024-04-25T08:46:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f5577e00640 1 -- 172.21.15.46:0/393878396 learned_addr learned my addr 172.21.15.46:0/393878396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f5577400640 1 --2- 172.21.15.46:0/393878396 >> v2:172.21.15.82:3300/0 conn(0x7f557815d710 0x7f55780b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f5577e00640 1 -- 172.21.15.46:0/393878396 >> v2:172.21.15.82:3300/0 conn(0x7f557815d710 msgr2=0x7f55780b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f5577e00640 1 --2- 172.21.15.46:0/393878396 >> v2:172.21.15.82:3300/0 conn(0x7f557815d710 0x7f55780b6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f5577e00640 1 -- 172.21.15.46:0/393878396 >> v1:172.21.15.82:6789/0 conn(0x7f557815daf0 legacy=0x7f55780d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f5577e00640 1 -- 172.21.15.46:0/393878396 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55780b7870 con 0x7f55780b9530 2024-04-25T08:46:34.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.065+0000 7f5577400640 1 --2- 172.21.15.46:0/393878396 >> v2:172.21.15.82:3300/0 conn(0x7f557815d710 0x7f55780b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:46:34.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f5577e00640 1 --2- 172.21.15.46:0/393878396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55780b9530 0x7f55780ce550 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5570009e50 tx=0x7f557001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e739d3f8ee33704b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:34.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f5576a00640 1 -- 172.21.15.46:0/393878396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700a5790 con 0x7f55780b9530 2024-04-25T08:46:34.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f5576a00640 1 -- 172.21.15.46:0/393878396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55700a5930 con 0x7f55780b9530 2024-04-25T08:46:34.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f5576a00640 1 -- 172.21.15.46:0/393878396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700a5d00 con 0x7f55780b9530 2024-04-25T08:46:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 -- 172.21.15.46:0/393878396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55780b9530 msgr2=0x7f55780ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 --2- 172.21.15.46:0/393878396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55780b9530 0x7f55780ce550 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5570009e50 tx=0x7f557001c9c0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 -- 172.21.15.46:0/393878396 shutdown_connections 2024-04-25T08:46:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 --2- 172.21.15.46:0/393878396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55780b9530 0x7f55780ce550 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 --2- 172.21.15.46:0/393878396 >> v2:172.21.15.82:3300/0 conn(0x7f557815d710 0x7f55780b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 -- 172.21.15.46:0/393878396 >> 172.21.15.46:0/393878396 conn(0x7f5578072770 msgr2=0x7f5578072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 -- 172.21.15.46:0/393878396 shutdown_connections 2024-04-25T08:46:34.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 -- 172.21.15.46:0/393878396 wait complete. 2024-04-25T08:46:34.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 Processor -- start 2024-04-25T08:46:34.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.069+0000 7f557e400640 1 -- start start 2024-04-25T08:46:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f557e400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55780b9530 0x7f5578152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f557e400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f557815d710 0x7f55780d3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f557e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f557815daf0 0x7f55780d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f557e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557816a620 con 0x7f557815daf0 2024-04-25T08:46:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f557e400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55780b7060 con 0x7f557815d710 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f557e400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557816a7a0 con 0x7f55780b9530 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f557815daf0 0x7f55780d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f557815daf0 0x7f55780d39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49314/0 (socket says 172.21.15.46:49314) 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577e00640 1 -- 172.21.15.46:0/3252997467 learned_addr learned my addr 172.21.15.46:0/3252997467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55780b9530 0x7f5578152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f556fe00640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f557815d710 0x7f55780d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577e00640 1 -- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55780b9530 msgr2=0x7f5578152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577e00640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55780b9530 0x7f5578152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577e00640 1 -- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f557815d710 msgr2=0x7f55780d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577e00640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f557815d710 0x7f55780d3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577e00640 1 -- 172.21.15.46:0/3252997467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578153230 con 0x7f557815daf0 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f5577e00640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f557815daf0 0x7f55780d39a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f55700195d0 tx=0x7f5570019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f556f400640 1 -- 172.21.15.46:0/3252997467 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700b8470 con 0x7f557815daf0 2024-04-25T08:46:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f556f400640 1 -- 172.21.15.46:0/3252997467 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55700b8630 con 0x7f557815daf0 2024-04-25T08:46:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f556f400640 1 -- 172.21.15.46:0/3252997467 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700b8a20 con 0x7f557815daf0 2024-04-25T08:46:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5578153510 con 0x7f557815daf0 2024-04-25T08:46:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5578153ad0 con 0x7f557815daf0 2024-04-25T08:46:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5578069080 con 0x7f557815daf0 2024-04-25T08:46:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.073+0000 7f556f400640 1 -- 172.21.15.46:0/3252997467 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55700c0020 con 0x7f557815daf0 2024-04-25T08:46:34.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.077+0000 7f556f400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5554066f60 0x7f5554069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.077+0000 7f556f400640 1 -- 172.21.15.46:0/3252997467 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f557011fb10 con 0x7f557815daf0 2024-04-25T08:46:34.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.077+0000 7f5577400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5554066f60 0x7f5554069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.077+0000 7f5577400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5554066f60 0x7f5554069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5568009580 tx=0x7f5568008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:34.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.077+0000 7f556f400640 1 -- 172.21.15.46:0/3252997467 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55700c7020 con 0x7f557815daf0 2024-04-25T08:46:34.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.241+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5578150f60 con 0x7f557815daf0 2024-04-25T08:46:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f556f400640 1 -- 172.21.15.46:0/3252997467 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17028 (secure 0 0 0) 0x7f55700165e0 con 0x7f557815daf0 2024-04-25T08:46:34.238 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:34.239 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:03.180374+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:45:10.412423+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:45:10.987443+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.15","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.21","mappings":[{"from":4,"to":3}]},{"pgid":"10.28","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:34.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5554066f60 msgr2=0x7f5554069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5554066f60 0x7f5554069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5568009580 tx=0x7f5568008040 comp rx=0 tx=0).stop 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f557815daf0 msgr2=0x7f55780d39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f557815daf0 0x7f55780d39a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f55700195d0 tx=0x7f5570019760 comp rx=0 tx=0).stop 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 shutdown_connections 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5554066f60 0x7f5554069420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f557815daf0 0x7f55780d39a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f557815d710 0x7f55780d3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 --2- 172.21.15.46:0/3252997467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55780b9530 0x7f5578152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 >> 172.21.15.46:0/3252997467 conn(0x7f5578072770 msgr2=0x7f5578157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 shutdown_connections 2024-04-25T08:46:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.245+0000 7f557e400640 1 -- 172.21.15.46:0/3252997467 wait complete. 2024-04-25T08:46:34.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:46:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f61000640 1 Processor -- start 2024-04-25T08:46:34.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f61000640 1 -- start start 2024-04-25T08:46:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f61000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c06f570 0x7f7f5c0af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f61000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5c15fed0 con 0x7f7f5c0ad4c0 2024-04-25T08:46:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f61000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5c1601b0 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f61000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c15c0f0 con 0x7f7f5c06f570 2024-04-25T08:46:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f51400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7f5c0ad4c0 0x7f7f5c06ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34924/0 (socket says 172.21.15.46:34924) 2024-04-25T08:46:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f51400640 1 -- 172.21.15.46:0/1209402164 learned_addr learned my addr 172.21.15.46:0/1209402164 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:34.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f5a800640 1 --2- 172.21.15.46:0/1209402164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c06f570 0x7f7f5c0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f59400640 1 -- 172.21.15.46:0/1209402164 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303268717 0 0) 0x7f7f5c15fed0 con 0x7f7f5c0ad4c0 2024-04-25T08:46:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f59400640 1 -- 172.21.15.46:0/1209402164 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f40003530 con 0x7f7f5c0ad4c0 2024-04-25T08:46:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f59400640 1 -- 172.21.15.46:0/1209402164 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1994412152 0 0) 0x7f7f5c1601b0 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f59400640 1 -- 172.21.15.46:0/1209402164 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f5c15fed0 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f5a800640 1 -- 172.21.15.46:0/1209402164 >> v1:172.21.15.82:6789/0 conn(0x7f7f5c15b9a0 legacy=0x7f7f5c0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f5a800640 1 -- 172.21.15.46:0/1209402164 >> v1:172.21.15.46:6789/0 conn(0x7f7f5c0ad4c0 legacy=0x7f7f5c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.409+0000 7f7f5a800640 1 -- 172.21.15.46:0/1209402164 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c15c3d0 con 0x7f7f5c06f570 2024-04-25T08:46:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f5a800640 1 --2- 172.21.15.46:0/1209402164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c06f570 0x7f7f5c0af030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009dc0 tx=0x7f7f5401c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71991354ab2590b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f59400640 1 -- 172.21.15.46:0/1209402164 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540b2070 con 0x7f7f5c06f570 2024-04-25T08:46:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f59400640 1 -- 172.21.15.46:0/1209402164 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f540a5920 con 0x7f7f5c06f570 2024-04-25T08:46:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f59400640 1 -- 172.21.15.46:0/1209402164 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540a5cf0 con 0x7f7f5c06f570 2024-04-25T08:46:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f61000640 1 -- 172.21.15.46:0/1209402164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c06f570 msgr2=0x7f7f5c0af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f61000640 1 --2- 172.21.15.46:0/1209402164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c06f570 0x7f7f5c0af030 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009dc0 tx=0x7f7f5401c840 comp rx=0 tx=0).stop 2024-04-25T08:46:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f61000640 1 -- 172.21.15.46:0/1209402164 shutdown_connections 2024-04-25T08:46:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f61000640 1 --2- 172.21.15.46:0/1209402164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c06f570 0x7f7f5c0af030 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f61000640 1 -- 172.21.15.46:0/1209402164 >> 172.21.15.46:0/1209402164 conn(0x7f7f5c0cfd90 msgr2=0x7f7f5c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f61000640 1 -- 172.21.15.46:0/1209402164 shutdown_connections 2024-04-25T08:46:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.413+0000 7f7f61000640 1 -- 172.21.15.46:0/1209402164 wait complete. 2024-04-25T08:46:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 Processor -- start 2024-04-25T08:46:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 -- start start 2024-04-25T08:46:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f5c06f570 0x7f7f5c15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c0ad4c0 0x7f7f5c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5c15b9a0 0x7f7f5c0b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c06b8b0 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c06b730 con 0x7f7f5c06f570 2024-04-25T08:46:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c169f50 con 0x7f7f5c0ad4c0 2024-04-25T08:46:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f5a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5c15b9a0 0x7f7f5c0b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f5a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5c15b9a0 0x7f7f5c0b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49320/0 (socket says 172.21.15.46:49320) 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f5a800640 1 -- 172.21.15.46:0/1032793329 learned_addr learned my addr 172.21.15.46:0/1032793329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f59e00640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f5c06f570 0x7f7f5c15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f51400640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c0ad4c0 0x7f7f5c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f5a800640 1 -- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c0ad4c0 msgr2=0x7f7f5c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f5a800640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c0ad4c0 0x7f7f5c15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f5a800640 1 -- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f5c06f570 msgr2=0x7f7f5c15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f5a800640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f5c06f570 0x7f7f5c15ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f5a800640 1 -- 172.21.15.46:0/1032793329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c164a50 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f51400640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c0ad4c0 0x7f7f5c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:34.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f5a800640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5c15b9a0 0x7f7f5c0b0c90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7f5401cd20 tx=0x7f7f540195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:34.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f52a00640 1 -- 172.21.15.46:0/1032793329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540b2050 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f52a00640 1 -- 172.21.15.46:0/1032793329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f540be070 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f52a00640 1 -- 172.21.15.46:0/1032793329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540ad890 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c164d30 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.417+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c1652f0 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.421+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c069080 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.421+0000 7f7f52a00640 1 -- 172.21.15.46:0/1032793329 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f540ada30 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.421+0000 7f7f52a00640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f38066f60 0x7f7f38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.421+0000 7f7f52a00640 1 -- 172.21.15.46:0/1032793329 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f7f54120260 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.421+0000 7f7f59e00640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f38066f60 0x7f7f38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.421+0000 7f7f59e00640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f38066f60 0x7f7f38069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c001720 tx=0x7f7f4c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:34.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.425+0000 7f7f52a00640 1 -- 172.21.15.46:0/1032793329 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f54016580 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.621+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7f5c153be0 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.616 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:46:34.621+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:46:34.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.625+0000 7f7f52a00640 1 -- 172.21.15.46:0/1032793329 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7f540f0980 con 0x7f7f5c15b9a0 2024-04-25T08:46:34.618 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:46:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.625+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f38066f60 msgr2=0x7f7f38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.625+0000 7f7f61000640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f38066f60 0x7f7f38069420 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c001720 tx=0x7f7f4c009040 comp rx=0 tx=0).stop 2024-04-25T08:46:34.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.625+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5c15b9a0 msgr2=0x7f7f5c0b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.625+0000 7f7f61000640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5c15b9a0 0x7f7f5c0b0c90 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7f5401cd20 tx=0x7f7f540195e0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.629+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 shutdown_connections 2024-04-25T08:46:34.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.629+0000 7f7f61000640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f38066f60 0x7f7f38069420 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.629+0000 7f7f61000640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f5c15b9a0 0x7f7f5c0b0c90 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.629+0000 7f7f61000640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7f5c0ad4c0 0x7f7f5c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.629+0000 7f7f61000640 1 --2- 172.21.15.46:0/1032793329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f5c06f570 0x7f7f5c15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.629+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 >> 172.21.15.46:0/1032793329 conn(0x7f7f5c0cfd90 msgr2=0x7f7f5c15d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:34.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.629+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 shutdown_connections 2024-04-25T08:46:34.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.629+0000 7f7f61000640 1 -- 172.21.15.46:0/1032793329 wait complete. 2024-04-25T08:46:34.633 INFO:tasks.ceph.mds.c.smithi046.stderr: -7732> 2024-04-25T08:45:14.574+0000 7f8f01400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:46:34.636 INFO:tasks.ceph.mds.c.smithi046.stderr: -4886> 2024-04-25T08:45:25.438+0000 7f8efc400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6169=0-4194304@1} caps={6169=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6169 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55bc26ca1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:46:34.638 INFO:tasks.ceph.mds.c.smithi046.stderr: -3109> 2024-04-25T08:45:57.033+0000 7f8efc400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6169=0-4194304@1} caps={6169=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6169 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55bc26ca1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:46:34.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:46:34.684 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:46:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddfaa00640 1 Processor -- start 2024-04-25T08:46:34.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddfaa00640 1 -- start start 2024-04-25T08:46:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddfaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf406f150 0x7fddf40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddfaa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf415e440 con 0x7fddf41440d0 2024-04-25T08:46:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddfaa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf415e6e0 con 0x7fddf4143990 2024-04-25T08:46:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddfaa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf415ed30 con 0x7fddf406f150 2024-04-25T08:46:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddebe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fddf41440d0 0x7fddf40b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34928/0 (socket says 172.21.15.46:34928) 2024-04-25T08:46:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddebe00640 1 -- 172.21.15.46:0/891592311 learned_addr learned my addr 172.21.15.46:0/891592311 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddf8a00640 1 --2- 172.21.15.46:0/891592311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf406f150 0x7fddf40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddf3400640 1 -- 172.21.15.46:0/891592311 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969383479 0 0) 0x7fddf415e440 con 0x7fddf41440d0 2024-04-25T08:46:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddf3400640 1 -- 172.21.15.46:0/891592311 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddd8003530 con 0x7fddf41440d0 2024-04-25T08:46:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddf3400640 1 -- 172.21.15.46:0/891592311 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293775173 0 0) 0x7fddf415e6e0 con 0x7fddf4143990 2024-04-25T08:46:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddf8a00640 1 -- 172.21.15.46:0/891592311 >> v1:172.21.15.82:6789/0 conn(0x7fddf4143990 legacy=0x7fddf406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddf8a00640 1 -- 172.21.15.46:0/891592311 >> v1:172.21.15.46:6789/0 conn(0x7fddf41440d0 legacy=0x7fddf40b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.797+0000 7fddf8a00640 1 -- 172.21.15.46:0/891592311 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf415e9c0 con 0x7fddf406f150 2024-04-25T08:46:34.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddf8a00640 1 --2- 172.21.15.46:0/891592311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf406f150 0x7fddf40cecc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fddf415eeb0 tx=0x7fddec01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f9fbea142f58f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddf3400640 1 -- 172.21.15.46:0/891592311 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec0b2070 con 0x7fddf406f150 2024-04-25T08:46:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddf3400640 1 -- 172.21.15.46:0/891592311 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddec0a5850 con 0x7fddf406f150 2024-04-25T08:46:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddf3400640 1 -- 172.21.15.46:0/891592311 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddec0a5c20 con 0x7fddf406f150 2024-04-25T08:46:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddfaa00640 1 -- 172.21.15.46:0/891592311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf406f150 msgr2=0x7fddf40cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddfaa00640 1 --2- 172.21.15.46:0/891592311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf406f150 0x7fddf40cecc0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fddf415eeb0 tx=0x7fddec01c990 comp rx=0 tx=0).stop 2024-04-25T08:46:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddfaa00640 1 -- 172.21.15.46:0/891592311 shutdown_connections 2024-04-25T08:46:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddfaa00640 1 --2- 172.21.15.46:0/891592311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf406f150 0x7fddf40cecc0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddfaa00640 1 -- 172.21.15.46:0/891592311 >> 172.21.15.46:0/891592311 conn(0x7fddf406ac10 msgr2=0x7fddf406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddfaa00640 1 -- 172.21.15.46:0/891592311 shutdown_connections 2024-04-25T08:46:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.801+0000 7fddfaa00640 1 -- 172.21.15.46:0/891592311 wait complete. 2024-04-25T08:46:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 Processor -- start 2024-04-25T08:46:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 -- start start 2024-04-25T08:46:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf406f150 0x7fddf415b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf4143990 0x7fddf406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf41440d0 0x7fddf4159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40713f0 con 0x7fddf4143990 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4071270 con 0x7fddf406f150 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4071570 con 0x7fddf41440d0 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf4143990 0x7fddf406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf4143990 0x7fddf406d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49344/0 (socket says 172.21.15.46:49344) 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddebe00640 1 -- 172.21.15.46:0/4002299746 learned_addr learned my addr 172.21.15.46:0/4002299746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddf8a00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf41440d0 0x7fddf4159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddf3e00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf406f150 0x7fddf415b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddebe00640 1 -- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf41440d0 msgr2=0x7fddf4159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddebe00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf41440d0 0x7fddf4159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddebe00640 1 -- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf406f150 msgr2=0x7fddf415b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddebe00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf406f150 0x7fddf415b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddebe00640 1 -- 172.21.15.46:0/4002299746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf415af70 con 0x7fddf4143990 2024-04-25T08:46:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddebe00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf4143990 0x7fddf406d7d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00c3c0 tx=0x7fdddc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddf0c00640 1 -- 172.21.15.46:0/4002299746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc017070 con 0x7fddf4143990 2024-04-25T08:46:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddf0c00640 1 -- 172.21.15.46:0/4002299746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdddc005cf0 con 0x7fddf4143990 2024-04-25T08:46:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddf0c00640 1 -- 172.21.15.46:0/4002299746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdddc0072f0 con 0x7fddf4143990 2024-04-25T08:46:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf415c0f0 con 0x7fddf4143990 2024-04-25T08:46:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.805+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf413b590 con 0x7fddf4143990 2024-04-25T08:46:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.809+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf4069010 con 0x7fddf4143990 2024-04-25T08:46:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.809+0000 7fddf0c00640 1 -- 172.21.15.46:0/4002299746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdddc01a070 con 0x7fddf4143990 2024-04-25T08:46:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.809+0000 7fddf0c00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddd0066f60 0x7fddd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.809+0000 7fddf0c00640 1 -- 172.21.15.46:0/4002299746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fdddc0845f0 con 0x7fddf4143990 2024-04-25T08:46:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.809+0000 7fddf3e00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddd0066f60 0x7fddd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.809+0000 7fddf3e00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddd0066f60 0x7fddd0069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdde4000f70 tx=0x7fdde4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:34.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:34.813+0000 7fddf0c00640 1 -- 172.21.15.46:0/4002299746 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdddc055440 con 0x7fddf4143990 2024-04-25T08:46:35.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.009+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fddf413fa60 con 0x7fddf4143990 2024-04-25T08:46:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.625+0000 7fddf0c00640 1 -- 172.21.15.46:0/4002299746 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdddc054d40 con 0x7fddf4143990 2024-04-25T08:46:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddd0066f60 msgr2=0x7fddd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddd0066f60 0x7fddd0069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdde4000f70 tx=0x7fdde4009040 comp rx=0 tx=0).stop 2024-04-25T08:46:35.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf4143990 msgr2=0x7fddf406d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf4143990 0x7fddf406d7d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00c3c0 tx=0x7fdddc00c880 comp rx=0 tx=0).stop 2024-04-25T08:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 shutdown_connections 2024-04-25T08:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddd0066f60 0x7fddd0069420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddf41440d0 0x7fddf4159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddf4143990 0x7fddf406d7d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 --2- 172.21.15.46:0/4002299746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddf406f150 0x7fddf415b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 >> 172.21.15.46:0/4002299746 conn(0x7fddf406ac10 msgr2=0x7fddf406ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 shutdown_connections 2024-04-25T08:46:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.629+0000 7fddfaa00640 1 -- 172.21.15.46:0/4002299746 wait complete. 2024-04-25T08:46:35.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:46:35.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd3200640 1 Processor -- start 2024-04-25T08:46:35.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd3200640 1 -- start start 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd3200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7effcc06bc20 0x7effcc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc15dce0 0x7effcc06db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7effcc072600 0x7effcc06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc06c540 con 0x7effcc072600 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd3200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc06ee30 con 0x7effcc06bc20 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc06f110 con 0x7effcc15dce0 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7effcc072600 0x7effcc06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7effcc072600 0x7effcc06e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49348/0 (socket says 172.21.15.46:49348) 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd1200640 1 -- 172.21.15.46:0/2152419913 learned_addr learned my addr 172.21.15.46:0/2152419913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effc3e00640 1 --2- 172.21.15.46:0/2152419913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc15dce0 0x7effcc06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:35.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effcbe00640 1 --2- 172.21.15.46:0/2152419913 >> v2:172.21.15.82:3300/0 conn(0x7effcc06bc20 0x7effcc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:35.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd1200640 1 -- 172.21.15.46:0/2152419913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc15dce0 msgr2=0x7effcc06db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:35.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd1200640 1 --2- 172.21.15.46:0/2152419913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc15dce0 0x7effcc06db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd1200640 1 -- 172.21.15.46:0/2152419913 >> v2:172.21.15.82:3300/0 conn(0x7effcc06bc20 msgr2=0x7effcc06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:35.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd1200640 1 --2- 172.21.15.46:0/2152419913 >> v2:172.21.15.82:3300/0 conn(0x7effcc06bc20 0x7effcc06c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effd1200640 1 -- 172.21.15.46:0/2152419913 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc06f3f0 con 0x7effcc072600 2024-04-25T08:46:35.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.841+0000 7effcbe00640 1 --2- 172.21.15.46:0/2152419913 >> v2:172.21.15.82:3300/0 conn(0x7effcc06bc20 0x7effcc06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd1200640 1 --2- 172.21.15.46:0/2152419913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc072600 0x7effcc06e080 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7effcc0679f0 tx=0x7effc401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7386ea7ceeff7de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effcb400640 1 -- 172.21.15.46:0/2152419913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40a5800 con 0x7effcc072600 2024-04-25T08:46:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effcb400640 1 -- 172.21.15.46:0/2152419913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc40a59a0 con 0x7effcc072600 2024-04-25T08:46:35.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effcb400640 1 -- 172.21.15.46:0/2152419913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40a5d70 con 0x7effcc072600 2024-04-25T08:46:35.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 -- 172.21.15.46:0/2152419913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc072600 msgr2=0x7effcc06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:35.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 --2- 172.21.15.46:0/2152419913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc072600 0x7effcc06e080 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7effcc0679f0 tx=0x7effc401c9c0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 -- 172.21.15.46:0/2152419913 shutdown_connections 2024-04-25T08:46:35.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 --2- 172.21.15.46:0/2152419913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc072600 0x7effcc06e080 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 --2- 172.21.15.46:0/2152419913 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc15dce0 0x7effcc06db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 --2- 172.21.15.46:0/2152419913 >> v2:172.21.15.82:3300/0 conn(0x7effcc06bc20 0x7effcc06c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 -- 172.21.15.46:0/2152419913 >> 172.21.15.46:0/2152419913 conn(0x7effcc071ae0 msgr2=0x7effcc071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 -- 172.21.15.46:0/2152419913 shutdown_connections 2024-04-25T08:46:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 -- 172.21.15.46:0/2152419913 wait complete. 2024-04-25T08:46:35.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 Processor -- start 2024-04-25T08:46:35.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.845+0000 7effd3200640 1 -- start start 2024-04-25T08:46:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effcc06bc20 0x7effcc0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc072600 0x7effcc1382d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc15dce0 0x7effcc15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0b3b10 con 0x7effcc15dce0 2024-04-25T08:46:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc06ebe0 con 0x7effcc06bc20 2024-04-25T08:46:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0b3c90 con 0x7effcc072600 2024-04-25T08:46:35.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc15dce0 0x7effcc15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc15dce0 0x7effcc15b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49354/0 (socket says 172.21.15.46:49354) 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd1200640 1 -- 172.21.15.46:0/433039890 learned_addr learned my addr 172.21.15.46:0/433039890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effc3e00640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc072600 0x7effcc1382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effcbe00640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effcc06bc20 0x7effcc0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd1200640 1 -- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc072600 msgr2=0x7effcc1382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd1200640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc072600 0x7effcc1382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd1200640 1 -- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effcc06bc20 msgr2=0x7effcc0b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd1200640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effcc06bc20 0x7effcc0b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd1200640 1 -- 172.21.15.46:0/433039890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc0b0ce0 con 0x7effcc15dce0 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd1200640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc15dce0 0x7effcc15b570 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7effc40af040 tx=0x7effc4019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effc8c00640 1 -- 172.21.15.46:0/433039890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40b8670 con 0x7effcc15dce0 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effc8c00640 1 -- 172.21.15.46:0/433039890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc40be070 con 0x7effcc15dce0 2024-04-25T08:46:35.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effc8c00640 1 -- 172.21.15.46:0/433039890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40b8a50 con 0x7effcc15dce0 2024-04-25T08:46:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc161190 con 0x7effcc15dce0 2024-04-25T08:46:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc0affa0 con 0x7effcc15dce0 2024-04-25T08:46:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.849+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effcc067820 con 0x7effcc15dce0 2024-04-25T08:46:35.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.853+0000 7effc8c00640 1 -- 172.21.15.46:0/433039890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effc40b8bf0 con 0x7effcc15dce0 2024-04-25T08:46:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.853+0000 7effc8c00640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7effa8066f60 0x7effa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.853+0000 7effc8c00640 1 -- 172.21.15.46:0/433039890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6303+0+0 (secure 0 0 0) 0x7effc4120390 con 0x7effcc15dce0 2024-04-25T08:46:35.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.853+0000 7effcbe00640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7effa8066f60 0x7effa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:35.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.853+0000 7effcbe00640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7effa8066f60 0x7effa8069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7effbc0045f0 tx=0x7effbc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:35.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:35.857+0000 7effc8c00640 1 -- 172.21.15.46:0/433039890 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effc40165e0 con 0x7effcc15dce0 2024-04-25T08:46:36.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.017+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7effcc15ce40 con 0x7effcc15dce0 2024-04-25T08:46:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.629+0000 7effc8c00640 1 -- 172.21.15.46:0/433039890 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) v1 ==== 181+0+0 (secure 0 0 0) 0x7effc40f0b20 con 0x7effcc15dce0 2024-04-25T08:46:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:46:36.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.629+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7effa8066f60 msgr2=0x7effa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:36.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.629+0000 7effd3200640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7effa8066f60 0x7effa8069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7effbc0045f0 tx=0x7effbc008040 comp rx=0 tx=0).stop 2024-04-25T08:46:36.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc15dce0 msgr2=0x7effcc15b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc15dce0 0x7effcc15b570 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7effc40af040 tx=0x7effc4019760 comp rx=0 tx=0).stop 2024-04-25T08:46:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 shutdown_connections 2024-04-25T08:46:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7effa8066f60 0x7effa8069420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effcc15dce0 0x7effcc15b570 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7effcc072600 0x7effcc1382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 --2- 172.21.15.46:0/433039890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effcc06bc20 0x7effcc0b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 >> 172.21.15.46:0/433039890 conn(0x7effcc071ae0 msgr2=0x7effcc0aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 shutdown_connections 2024-04-25T08:46:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.633+0000 7effd3200640 1 -- 172.21.15.46:0/433039890 wait complete. 2024-04-25T08:46:36.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:46:36.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8562400640 1 Processor -- start 2024-04-25T08:46:36.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8562400640 1 -- start start 2024-04-25T08:46:36.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8562400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f855c15b7a0 0x7f855c0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8562400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f855c0cfb10 0x7f855c0cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8562400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855c0cc250 con 0x7f855c0b2260 2024-04-25T08:46:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8562400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855c15bb80 con 0x7f855c0cfb10 2024-04-25T08:46:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8562400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855c0b1e60 con 0x7f855c15b7a0 2024-04-25T08:46:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f855be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f855c0cfb10 0x7f855c0cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8561a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f855c0b2260 0x7f855c0b2690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34934/0 (socket says 172.21.15.46:34934) 2024-04-25T08:46:36.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8561a00640 1 -- 172.21.15.46:0/2604289275 learned_addr learned my addr 172.21.15.46:0/2604289275 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f8561000640 1 --2- 172.21.15.46:0/2604289275 >> v2:172.21.15.82:3300/0 conn(0x7f855c15b7a0 0x7f855c0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f855be00640 1 -- 172.21.15.46:0/2604289275 >> v2:172.21.15.82:3300/0 conn(0x7f855c15b7a0 msgr2=0x7f855c0cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f855be00640 1 --2- 172.21.15.46:0/2604289275 >> v2:172.21.15.82:3300/0 conn(0x7f855c15b7a0 0x7f855c0cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f855be00640 1 -- 172.21.15.46:0/2604289275 >> v1:172.21.15.46:6789/0 conn(0x7f855c0b2260 legacy=0x7f855c0b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f855b400640 1 -- 172.21.15.46:0/2604289275 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330617954 0 0) 0x7f855c0cc250 con 0x7f855c0b2260 2024-04-25T08:46:36.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.801+0000 7f855be00640 1 -- 172.21.15.46:0/2604289275 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c0cc530 con 0x7f855c0cfb10 2024-04-25T08:46:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f855be00640 1 --2- 172.21.15.46:0/2604289275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c0cfb10 0x7f855c0cffa0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f855c072590 tx=0x7f854801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df93352474c27fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f855b400640 1 -- 172.21.15.46:0/2604289275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480b0070 con 0x7f855c0cfb10 2024-04-25T08:46:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f855b400640 1 -- 172.21.15.46:0/2604289275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85480ab600 con 0x7f855c0cfb10 2024-04-25T08:46:36.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f855b400640 1 -- 172.21.15.46:0/2604289275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480ab9d0 con 0x7f855c0cfb10 2024-04-25T08:46:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 -- 172.21.15.46:0/2604289275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c0cfb10 msgr2=0x7f855c0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 --2- 172.21.15.46:0/2604289275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c0cfb10 0x7f855c0cffa0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f855c072590 tx=0x7f854801c480 comp rx=0 tx=0).stop 2024-04-25T08:46:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 -- 172.21.15.46:0/2604289275 shutdown_connections 2024-04-25T08:46:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 --2- 172.21.15.46:0/2604289275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c0cfb10 0x7f855c0cffa0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 --2- 172.21.15.46:0/2604289275 >> v2:172.21.15.82:3300/0 conn(0x7f855c15b7a0 0x7f855c0cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:36.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 -- 172.21.15.46:0/2604289275 >> 172.21.15.46:0/2604289275 conn(0x7f855c0b3020 msgr2=0x7f855c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 -- 172.21.15.46:0/2604289275 shutdown_connections 2024-04-25T08:46:36.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 -- 172.21.15.46:0/2604289275 wait complete. 2024-04-25T08:46:36.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 Processor -- start 2024-04-25T08:46:36.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.805+0000 7f8562400640 1 -- start start 2024-04-25T08:46:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8562400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855c0b2260 0x7f855c15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8562400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f855c0cfb10 0x7f855c0af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8562400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c15b7a0 0x7f855c15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8562400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0cd2f0 con 0x7f855c15b7a0 2024-04-25T08:46:36.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8562400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0b2da0 con 0x7f855c0b2260 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8562400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0cd470 con 0x7f855c0cfb10 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c15b7a0 0x7f855c15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c15b7a0 0x7f855c15a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49362/0 (socket says 172.21.15.46:49362) 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561a00640 1 -- 172.21.15.46:0/1577370536 learned_addr learned my addr 172.21.15.46:0/1577370536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561000640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855c0b2260 0x7f855c15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f855be00640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f855c0cfb10 0x7f855c0af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561a00640 1 -- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f855c0cfb10 msgr2=0x7f855c0af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561a00640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f855c0cfb10 0x7f855c0af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561a00640 1 -- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855c0b2260 msgr2=0x7f855c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561a00640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855c0b2260 0x7f855c15e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561a00640 1 -- 172.21.15.46:0/1577370536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c0afdb0 con 0x7f855c15b7a0 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8561a00640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c15b7a0 0x7f855c15a300 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f855400c730 tx=0x7f855400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8558c00640 1 -- 172.21.15.46:0/1577370536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554018070 con 0x7f855c15b7a0 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8558c00640 1 -- 172.21.15.46:0/1577370536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855400ce40 con 0x7f855c15b7a0 2024-04-25T08:46:36.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8558c00640 1 -- 172.21.15.46:0/1577370536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400a860 con 0x7f855c15b7a0 2024-04-25T08:46:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c1561c0 con 0x7f855c15b7a0 2024-04-25T08:46:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c156780 con 0x7f855c15b7a0 2024-04-25T08:46:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.809+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c069010 con 0x7f855c15b7a0 2024-04-25T08:46:36.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.813+0000 7f8558c00640 1 -- 172.21.15.46:0/1577370536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f855401b070 con 0x7f855c15b7a0 2024-04-25T08:46:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.813+0000 7f8558c00640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f853c066f90 0x7f853c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.813+0000 7f8558c00640 1 -- 172.21.15.46:0/1577370536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5836+0+0 (secure 0 0 0) 0x7f85540846b0 con 0x7f855c15b7a0 2024-04-25T08:46:36.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.813+0000 7f8561000640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f853c066f90 0x7f853c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:36.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.813+0000 7f8561000640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f853c066f90 0x7f853c069450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f855000b9d0 tx=0x7f8550008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:36.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.817+0000 7f8558c00640 1 -- 172.21.15.46:0/1577370536 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85540555a0 con 0x7f855c15b7a0 2024-04-25T08:46:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:36.977+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f855c0b3c10 con 0x7f855c15b7a0 2024-04-25T08:46:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.637+0000 7f8558c00640 1 -- 172.21.15.46:0/1577370536 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8554054ea0 con 0x7f855c15b7a0 2024-04-25T08:46:37.630 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-25T08:46:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f853c066f90 msgr2=0x7f853c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f853c066f90 0x7f853c069450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f855000b9d0 tx=0x7f8550008040 comp rx=0 tx=0).stop 2024-04-25T08:46:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c15b7a0 msgr2=0x7f855c15a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:37.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c15b7a0 0x7f855c15a300 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f855400c730 tx=0x7f855400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 shutdown_connections 2024-04-25T08:46:37.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f853c066f90 0x7f853c069450 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f855c15b7a0 0x7f855c15a300 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f855c0cfb10 0x7f855c0af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 --2- 172.21.15.46:0/1577370536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855c0b2260 0x7f855c15e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.641+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 >> 172.21.15.46:0/1577370536 conn(0x7f855c0b3020 msgr2=0x7f855c0b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:37.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.645+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 shutdown_connections 2024-04-25T08:46:37.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.645+0000 7f8562400640 1 -- 172.21.15.46:0/1577370536 wait complete. 2024-04-25T08:46:37.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:46:37.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.817+0000 7fd2ce800640 1 Processor -- start 2024-04-25T08:46:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.817+0000 7fd2ce800640 1 -- start start 2024-04-25T08:46:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2ce800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2c80b4760 0x7fd2c8072440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2ce800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2c815e960 0x7fd2c806bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2ce800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c80b4b40 con 0x7fd2c806c160 2024-04-25T08:46:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2ce800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c806c5b0 con 0x7fd2c815e960 2024-04-25T08:46:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2ce800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c80ad170 con 0x7fd2c80b4760 2024-04-25T08:46:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2cca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd2c815e960 0x7fd2c806bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2cde00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd2c806c160 0x7fd2c8153df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34940/0 (socket says 172.21.15.46:34940) 2024-04-25T08:46:37.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2cde00640 1 -- 172.21.15.46:0/2591553118 learned_addr learned my addr 172.21.15.46:0/2591553118 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2cd400640 1 --2- 172.21.15.46:0/2591553118 >> v2:172.21.15.82:3300/0 conn(0x7fd2c80b4760 0x7fd2c8072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2cca00640 1 -- 172.21.15.46:0/2591553118 >> v2:172.21.15.82:3300/0 conn(0x7fd2c80b4760 msgr2=0x7fd2c8072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2cca00640 1 --2- 172.21.15.46:0/2591553118 >> v2:172.21.15.82:3300/0 conn(0x7fd2c80b4760 0x7fd2c8072440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2cca00640 1 -- 172.21.15.46:0/2591553118 >> v1:172.21.15.46:6789/0 conn(0x7fd2c806c160 legacy=0x7fd2c8153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2cca00640 1 -- 172.21.15.46:0/2591553118 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80ad3c0 con 0x7fd2c815e960 2024-04-25T08:46:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2cd400640 1 --2- 172.21.15.46:0/2591553118 >> v2:172.21.15.82:3300/0 conn(0x7fd2c80b4760 0x7fd2c8072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:46:37.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.821+0000 7fd2c7e00640 1 -- 172.21.15.46:0/2591553118 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477582602 0 0) 0x7fd2c80b4b40 con 0x7fd2c806c160 2024-04-25T08:46:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2cca00640 1 --2- 172.21.15.46:0/2591553118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c815e960 0x7fd2c806bc20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8066ec0 tx=0x7fd2b401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347ae27d8d6c69e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2c7e00640 1 -- 172.21.15.46:0/2591553118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40b0070 con 0x7fd2c815e960 2024-04-25T08:46:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2c7e00640 1 -- 172.21.15.46:0/2591553118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b40ab600 con 0x7fd2c815e960 2024-04-25T08:46:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2c7e00640 1 -- 172.21.15.46:0/2591553118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40ab9d0 con 0x7fd2c815e960 2024-04-25T08:46:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 -- 172.21.15.46:0/2591553118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c815e960 msgr2=0x7fd2c806bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 --2- 172.21.15.46:0/2591553118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c815e960 0x7fd2c806bc20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8066ec0 tx=0x7fd2b401c270 comp rx=0 tx=0).stop 2024-04-25T08:46:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 -- 172.21.15.46:0/2591553118 shutdown_connections 2024-04-25T08:46:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 --2- 172.21.15.46:0/2591553118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c815e960 0x7fd2c806bc20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 --2- 172.21.15.46:0/2591553118 >> v2:172.21.15.82:3300/0 conn(0x7fd2c80b4760 0x7fd2c8072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 -- 172.21.15.46:0/2591553118 >> 172.21.15.46:0/2591553118 conn(0x7fd2c8071920 msgr2=0x7fd2c8071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 -- 172.21.15.46:0/2591553118 shutdown_connections 2024-04-25T08:46:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 -- 172.21.15.46:0/2591553118 wait complete. 2024-04-25T08:46:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 Processor -- start 2024-04-25T08:46:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 -- start start 2024-04-25T08:46:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c806c160 0x7fd2c81499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c80b4760 0x7fd2c80cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.825+0000 7fd2ce800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2c815e960 0x7fd2c80cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2ce800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80b2bf0 con 0x7fd2c806c160 2024-04-25T08:46:37.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2ce800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c8072980 con 0x7fd2c80b4760 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2ce800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c80b2d70 con 0x7fd2c815e960 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c806c160 0x7fd2c81499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c806c160 0x7fd2c81499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49378/0 (socket says 172.21.15.46:49378) 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cd400640 1 -- 172.21.15.46:0/2434083500 learned_addr learned my addr 172.21.15.46:0/2434083500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cca00640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c80b4760 0x7fd2c80cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cde00640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2c815e960 0x7fd2c80cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cd400640 1 -- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2c815e960 msgr2=0x7fd2c80cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cd400640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2c815e960 0x7fd2c80cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cd400640 1 -- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c80b4760 msgr2=0x7fd2c80cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cd400640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c80b4760 0x7fd2c80cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cd400640 1 -- 172.21.15.46:0/2434083500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c80cd9a0 con 0x7fd2c806c160 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2cd400640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c806c160 0x7fd2c81499b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c7f0 tx=0x7fd2bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2c5600640 1 -- 172.21.15.46:0/2434083500 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc007a30 con 0x7fd2c806c160 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2c5600640 1 -- 172.21.15.46:0/2434083500 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc007bd0 con 0x7fd2c806c160 2024-04-25T08:46:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2c5600640 1 -- 172.21.15.46:0/2434083500 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0113f0 con 0x7fd2c806c160 2024-04-25T08:46:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c8155f80 con 0x7fd2c806c160 2024-04-25T08:46:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c8156540 con 0x7fd2c806c160 2024-04-25T08:46:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c8069010 con 0x7fd2c806c160 2024-04-25T08:46:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.829+0000 7fd2c5600640 1 -- 172.21.15.46:0/2434083500 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2bc011590 con 0x7fd2c806c160 2024-04-25T08:46:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.833+0000 7fd2c5600640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd2a8066f90 0x7fd2a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.833+0000 7fd2c5600640 1 -- 172.21.15.46:0/2434083500 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd2bc0851b0 con 0x7fd2c806c160 2024-04-25T08:46:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.833+0000 7fd2cca00640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd2a8066f90 0x7fd2a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.833+0000 7fd2cca00640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd2a8066f90 0x7fd2a8069450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40ad000 tx=0x7fd2b40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:37.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.837+0000 7fd2c5600640 1 -- 172.21.15.46:0/2434083500 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2c80cd9a0 con 0x7fd2c806c160 2024-04-25T08:46:37.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:37.997+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd2c8066ef0 con 0x7fd2c806c160 2024-04-25T08:46:37.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.001+0000 7fd2c5600640 1 -- 172.21.15.46:0/2434083500 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd2bc015020 con 0x7fd2c806c160 2024-04-25T08:46:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd2a8066f90 msgr2=0x7fd2a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd2a8066f90 0x7fd2a8069450 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40ad000 tx=0x7fd2b40a6000 comp rx=0 tx=0).stop 2024-04-25T08:46:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c806c160 msgr2=0x7fd2c81499b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c806c160 0x7fd2c81499b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00c7f0 tx=0x7fd2bc00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 shutdown_connections 2024-04-25T08:46:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd2a8066f90 0x7fd2a8069450 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2c815e960 0x7fd2c80cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2c80b4760 0x7fd2c80cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 --2- 172.21.15.46:0/2434083500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2c806c160 0x7fd2c81499b0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 >> 172.21.15.46:0/2434083500 conn(0x7fd2c8071920 msgr2=0x7fd2c8071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 shutdown_connections 2024-04-25T08:46:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.005+0000 7fd2ce800640 1 -- 172.21.15.46:0/2434083500 wait complete. 2024-04-25T08:46:38.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T08:46:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f827ec00640 1 Processor -- start 2024-04-25T08:46:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f827ec00640 1 -- start start 2024-04-25T08:46:38.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f827ec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f827806f320 0x7f8278162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f827ec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82780b35b0 con 0x7f82780aed10 2024-04-25T08:46:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f827ec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827806d5e0 con 0x7f827815a750 2024-04-25T08:46:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f827ec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827815ab30 con 0x7f827806f320 2024-04-25T08:46:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f826fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f827806f320 0x7f8278162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f826fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f827806f320 0x7f8278162200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54128/0 (socket says 172.21.15.46:54128) 2024-04-25T08:46:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f826fe00640 1 -- 172.21.15.46:0/2797910595 learned_addr learned my addr 172.21.15.46:0/2797910595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f827cc00640 1 --1- 172.21.15.46:0/2797910595 >> v1:172.21.15.46:6789/0 conn(0x7f82780aed10 0x7f8278164750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59314/0 (socket says 172.21.15.46:59314) 2024-04-25T08:46:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f826fe00640 1 -- 172.21.15.46:0/2797910595 >> v1:172.21.15.82:6789/0 conn(0x7f827815a750 legacy=0x7f827806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f826fe00640 1 -- 172.21.15.46:0/2797910595 >> v1:172.21.15.46:6789/0 conn(0x7f82780aed10 legacy=0x7f8278164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.165+0000 7f826fe00640 1 -- 172.21.15.46:0/2797910595 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82780ce720 con 0x7f827806f320 2024-04-25T08:46:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f826fe00640 1 --2- 172.21.15.46:0/2797910595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827806f320 0x7f8278162200 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8260009870 tx=0x7f826001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33a0bb962cac9b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f8277400640 1 -- 172.21.15.46:0/2797910595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82600b0070 con 0x7f827806f320 2024-04-25T08:46:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f8277400640 1 -- 172.21.15.46:0/2797910595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82600ab600 con 0x7f827806f320 2024-04-25T08:46:38.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f8277400640 1 -- 172.21.15.46:0/2797910595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82600ab9d0 con 0x7f827806f320 2024-04-25T08:46:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f827ec00640 1 -- 172.21.15.46:0/2797910595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827806f320 msgr2=0x7f8278162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f827ec00640 1 --2- 172.21.15.46:0/2797910595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827806f320 0x7f8278162200 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8260009870 tx=0x7f826001c560 comp rx=0 tx=0).stop 2024-04-25T08:46:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f827ec00640 1 -- 172.21.15.46:0/2797910595 shutdown_connections 2024-04-25T08:46:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f827ec00640 1 --2- 172.21.15.46:0/2797910595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827806f320 0x7f8278162200 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f827ec00640 1 -- 172.21.15.46:0/2797910595 >> 172.21.15.46:0/2797910595 conn(0x7f82780b12a0 msgr2=0x7f82780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f827ec00640 1 -- 172.21.15.46:0/2797910595 shutdown_connections 2024-04-25T08:46:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.169+0000 7f827ec00640 1 -- 172.21.15.46:0/2797910595 wait complete. 2024-04-25T08:46:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 Processor -- start 2024-04-25T08:46:38.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 -- start start 2024-04-25T08:46:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827806f320 0x7f82781488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82780aed10 0x7f82781f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a750 0x7f82781f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827806e310 con 0x7f827815a750 2024-04-25T08:46:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827806d270 con 0x7f827806f320 2024-04-25T08:46:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827806e490 con 0x7f82780aed10 2024-04-25T08:46:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a750 0x7f82781f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a750 0x7f82781f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54142/0 (socket says 172.21.15.46:54142) 2024-04-25T08:46:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827cc00640 1 -- 172.21.15.46:0/2306913246 learned_addr learned my addr 172.21.15.46:0/2306913246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f8277e00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827806f320 0x7f82781488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f826fe00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82780aed10 0x7f82781f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827cc00640 1 -- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82780aed10 msgr2=0x7f82781f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827cc00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82780aed10 0x7f82781f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827cc00640 1 -- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827806f320 msgr2=0x7f82781488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827cc00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827806f320 0x7f82781488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827cc00640 1 -- 172.21.15.46:0/2306913246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82781f7820 con 0x7f827815a750 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f8277e00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827806f320 0x7f82781488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827cc00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a750 0x7f82781f60b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f827000cad0 tx=0x7f827000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f8274c00640 1 -- 172.21.15.46:0/2306913246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8270017070 con 0x7f827815a750 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f8274c00640 1 -- 172.21.15.46:0/2306913246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827000a760 con 0x7f827815a750 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f8274c00640 1 -- 172.21.15.46:0/2306913246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827000ab10 con 0x7f827815a750 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8278148f80 con 0x7f827815a750 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.173+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8278149540 con 0x7f827815a750 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.177+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8278069080 con 0x7f827815a750 2024-04-25T08:46:38.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.177+0000 7f8274c00640 1 -- 172.21.15.46:0/2306913246 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f827001a070 con 0x7f827815a750 2024-04-25T08:46:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.177+0000 7f8274c00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8254066f60 0x7f8254069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.177+0000 7f8274c00640 1 -- 172.21.15.46:0/2306913246 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f82700846e0 con 0x7f827815a750 2024-04-25T08:46:38.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.177+0000 7f8277e00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8254066f60 0x7f8254069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.181+0000 7f8277e00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8254066f60 0x7f8254069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8268000f70 tx=0x7f8268009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:38.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.181+0000 7f8274c00640 1 -- 172.21.15.46:0/2306913246 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f82781f7820 con 0x7f827815a750 2024-04-25T08:46:38.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.397+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f827806efe0 con 0x7f827815a750 2024-04-25T08:46:38.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.637+0000 7f8274c00640 1 -- 172.21.15.46:0/2306913246 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v229) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8270055170 con 0x7f827815a750 2024-04-25T08:46:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8254066f60 msgr2=0x7f8254069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8254066f60 0x7f8254069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8268000f70 tx=0x7f8268009040 comp rx=0 tx=0).stop 2024-04-25T08:46:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a750 msgr2=0x7f82781f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a750 0x7f82781f60b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f827000cad0 tx=0x7f827000ced0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 shutdown_connections 2024-04-25T08:46:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8254066f60 0x7f8254069420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a750 0x7f82781f60b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f82780aed10 0x7f82781f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 --2- 172.21.15.46:0/2306913246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f827806f320 0x7f82781488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 >> 172.21.15.46:0/2306913246 conn(0x7f82780b12a0 msgr2=0x7f8278153f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 shutdown_connections 2024-04-25T08:46:38.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.641+0000 7f827ec00640 1 -- 172.21.15.46:0/2306913246 wait complete. 2024-04-25T08:46:38.654 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T08:46:38.654 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T08:46:38.654 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:46:38.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.805+0000 7f1c86a00640 1 Processor -- start 2024-04-25T08:46:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.805+0000 7f1c86a00640 1 -- start start 2024-04-25T08:46:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c86a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c8015dca0 0x7f1c800d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c86a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 0x7f1c800d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c86a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c8006ed70 con 0x7f1c800cf140 2024-04-25T08:46:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c86a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c800afb40 con 0x7f1c8015dca0 2024-04-25T08:46:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c86a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c80162530 con 0x7f1c800afd40 2024-04-25T08:46:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c77400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c8015dca0 0x7f1c800d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c84a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 0x7f1c800d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c7fe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1c800cf140 0x7f1c8015d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:36234/0 (socket says 172.21.15.46:36234) 2024-04-25T08:46:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c7fe00640 1 -- 172.21.15.46:0/772934799 learned_addr learned my addr 172.21.15.46:0/772934799 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c77400640 1 -- 172.21.15.46:0/772934799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 msgr2=0x7f1c800d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c77400640 1 --2- 172.21.15.46:0/772934799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 0x7f1c800d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c77400640 1 -- 172.21.15.46:0/772934799 >> v1:172.21.15.82:6789/0 conn(0x7f1c800cf140 legacy=0x7f1c8015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c77400640 1 -- 172.21.15.46:0/772934799 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c80161770 con 0x7f1c8015dca0 2024-04-25T08:46:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.809+0000 7f1c84a00640 1 --2- 172.21.15.46:0/772934799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 0x7f1c800d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:46:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c77400640 1 --2- 172.21.15.46:0/772934799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8015dca0 0x7f1c800d03e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1c800afcc0 tx=0x7f1c6801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57425cfc2eed362 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c7f400640 1 -- 172.21.15.46:0/772934799 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680ab610 con 0x7f1c8015dca0 2024-04-25T08:46:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c7f400640 1 -- 172.21.15.46:0/772934799 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c680b0070 con 0x7f1c8015dca0 2024-04-25T08:46:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c7f400640 1 -- 172.21.15.46:0/772934799 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680ab9d0 con 0x7f1c8015dca0 2024-04-25T08:46:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 -- 172.21.15.46:0/772934799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8015dca0 msgr2=0x7f1c800d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 --2- 172.21.15.46:0/772934799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8015dca0 0x7f1c800d03e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1c800afcc0 tx=0x7f1c6801c410 comp rx=0 tx=0).stop 2024-04-25T08:46:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 -- 172.21.15.46:0/772934799 shutdown_connections 2024-04-25T08:46:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 --2- 172.21.15.46:0/772934799 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 0x7f1c800d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 --2- 172.21.15.46:0/772934799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8015dca0 0x7f1c800d03e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 -- 172.21.15.46:0/772934799 >> 172.21.15.46:0/772934799 conn(0x7f1c800728c0 msgr2=0x7f1c80099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 -- 172.21.15.46:0/772934799 shutdown_connections 2024-04-25T08:46:38.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 -- 172.21.15.46:0/772934799 wait complete. 2024-04-25T08:46:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 Processor -- start 2024-04-25T08:46:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.813+0000 7f1c86a00640 1 -- start start 2024-04-25T08:46:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c86a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 0x7f1c80155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c86a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c800cf140 0x7f1c80157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c86a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8015dca0 0x7f1c80151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c86a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800d1270 con 0x7f1c800cf140 2024-04-25T08:46:38.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c86a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800af7f0 con 0x7f1c8015dca0 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c86a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800d13f0 con 0x7f1c800afd40 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c800cf140 0x7f1c80157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c800cf140 0x7f1c80157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54146/0 (socket says 172.21.15.46:54146) 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c77400640 1 -- 172.21.15.46:0/2840986870 learned_addr learned my addr 172.21.15.46:0/2840986870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c7fe00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 0x7f1c80155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c84a00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8015dca0 0x7f1c80151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c77400640 1 -- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 msgr2=0x7f1c80155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c77400640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 0x7f1c80155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c77400640 1 -- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8015dca0 msgr2=0x7f1c80151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c77400640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8015dca0 0x7f1c80151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c77400640 1 -- 172.21.15.46:0/2840986870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8014ff20 con 0x7f1c800cf140 2024-04-25T08:46:38.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c77400640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c800cf140 0x7f1c80157f00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1c6801a040 tx=0x7f1c680191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c7cc00640 1 -- 172.21.15.46:0/2840986870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680abcc0 con 0x7f1c800cf140 2024-04-25T08:46:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c7cc00640 1 -- 172.21.15.46:0/2840986870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c680bf450 con 0x7f1c800cf140 2024-04-25T08:46:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c7cc00640 1 -- 172.21.15.46:0/2840986870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680abe60 con 0x7f1c800cf140 2024-04-25T08:46:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c801556b0 con 0x7f1c800cf140 2024-04-25T08:46:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c80155c70 con 0x7f1c800cf140 2024-04-25T08:46:38.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8006ccb0 con 0x7f1c800cf140 2024-04-25T08:46:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.817+0000 7f1c7cc00640 1 -- 172.21.15.46:0/2840986870 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c680c83f0 con 0x7f1c800cf140 2024-04-25T08:46:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.821+0000 7f1c7cc00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c5c066f60 0x7f1c5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:38.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.821+0000 7f1c7cc00640 1 -- 172.21.15.46:0/2840986870 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1c6811f530 con 0x7f1c800cf140 2024-04-25T08:46:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.821+0000 7f1c7fe00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c5c066f60 0x7f1c5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:38.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.821+0000 7f1c7fe00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c5c066f60 0x7f1c5c069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000f70 tx=0x7f1c70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:38.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.825+0000 7f1c7cc00640 1 -- 172.21.15.46:0/2840986870 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c680a8050 con 0x7f1c800cf140 2024-04-25T08:46:38.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c80069080 con 0x7f1c800cf140 2024-04-25T08:46:38.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c7cc00640 1 -- 172.21.15.46:0/2840986870 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+13246 (secure 0 0 0) 0x7f1c68016090 con 0x7f1c800cf140 2024-04-25T08:46:38.983 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:38.983 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":70,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:37.638613+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1670658295":"2024-04-26T08:46:34.626488+0000","172.21.15.46:6834/1670658295":"2024-04-26T08:46:34.626488+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:38.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c5c066f60 msgr2=0x7f1c5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c5c066f60 0x7f1c5c069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1c70000f70 tx=0x7f1c70009040 comp rx=0 tx=0).stop 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c800cf140 msgr2=0x7f1c80157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c800cf140 0x7f1c80157f00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f1c6801a040 tx=0x7f1c680191b0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 shutdown_connections 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c5c066f60 0x7f1c5c069420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c8015dca0 0x7f1c80151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c800cf140 0x7f1c80157f00 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 --2- 172.21.15.46:0/2840986870 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c800afd40 0x7f1c80155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 >> 172.21.15.46:0/2840986870 conn(0x7f1c800728c0 msgr2=0x7f1c8015af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 shutdown_connections 2024-04-25T08:46:38.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:38.989+0000 7f1c86a00640 1 -- 172.21.15.46:0/2840986870 wait complete. 2024-04-25T08:46:39.005 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:46:39.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:46:39.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc59e000640 1 Processor -- start 2024-04-25T08:46:39.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc59e000640 1 -- start start 2024-04-25T08:46:39.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc59e000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 0x7fc5980cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc59e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc59815dd50 0x7fc5980cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc59e000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5980d06f0 con 0x7fc5980d0280 2024-04-25T08:46:39.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc59e000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59809a780 con 0x7fc59815dd50 2024-04-25T08:46:39.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc59e000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59809aa60 con 0x7fc59806f4d0 2024-04-25T08:46:39.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc58ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc59815dd50 0x7fc5980cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc597e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc5980d0280 0x7fc59806f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59324/0 (socket says 172.21.15.46:59324) 2024-04-25T08:46:39.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc597e00640 1 -- 172.21.15.46:0/2280965966 learned_addr learned my addr 172.21.15.46:0/2280965966 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:39.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc597400640 1 --2- 172.21.15.46:0/2280965966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 0x7fc5980cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc596a00640 1 -- 172.21.15.46:0/2280965966 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948803710 0 0) 0x7fc5980d06f0 con 0x7fc5980d0280 2024-04-25T08:46:39.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc596a00640 1 -- 172.21.15.46:0/2280965966 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc57c003530 con 0x7fc5980d0280 2024-04-25T08:46:39.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc58ea00640 1 -- 172.21.15.46:0/2280965966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 msgr2=0x7fc5980cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:39.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc58ea00640 1 --2- 172.21.15.46:0/2280965966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 0x7fc5980cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc58ea00640 1 -- 172.21.15.46:0/2280965966 >> v1:172.21.15.46:6789/0 conn(0x7fc5980d0280 legacy=0x7fc59806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.157+0000 7fc58ea00640 1 -- 172.21.15.46:0/2280965966 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59809ad40 con 0x7fc59815dd50 2024-04-25T08:46:39.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc58ea00640 1 --2- 172.21.15.46:0/2280965966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 0x7fc5980cfd40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc5800097c0 tx=0x7fc58001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6272680101ad5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:39.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc596a00640 1 -- 172.21.15.46:0/2280965966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5800a7070 con 0x7fc59815dd50 2024-04-25T08:46:39.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc596a00640 1 -- 172.21.15.46:0/2280965966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5800b0070 con 0x7fc59815dd50 2024-04-25T08:46:39.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc596a00640 1 -- 172.21.15.46:0/2280965966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5800ab890 con 0x7fc59815dd50 2024-04-25T08:46:39.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc59e000640 1 -- 172.21.15.46:0/2280965966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 msgr2=0x7fc5980cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc59e000640 1 --2- 172.21.15.46:0/2280965966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 0x7fc5980cfd40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc5800097c0 tx=0x7fc58001c470 comp rx=0 tx=0).stop 2024-04-25T08:46:39.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc59e000640 1 -- 172.21.15.46:0/2280965966 shutdown_connections 2024-04-25T08:46:39.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc59e000640 1 --2- 172.21.15.46:0/2280965966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 0x7fc5980cfd40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc59e000640 1 --2- 172.21.15.46:0/2280965966 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 0x7fc5980cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc59e000640 1 -- 172.21.15.46:0/2280965966 >> 172.21.15.46:0/2280965966 conn(0x7fc5980b3160 msgr2=0x7fc5980b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:39.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc59e000640 1 -- 172.21.15.46:0/2280965966 shutdown_connections 2024-04-25T08:46:39.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc59e000640 1 -- 172.21.15.46:0/2280965966 wait complete. 2024-04-25T08:46:39.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.161+0000 7fc59e000640 1 Processor -- start 2024-04-25T08:46:39.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 -- start start 2024-04-25T08:46:39.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 0x7fc598154650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5980d0280 0x7fc5980cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 0x7fc598153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59816a820 con 0x7fc59815dd50 2024-04-25T08:46:39.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59816a6a0 con 0x7fc5980d0280 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59816a9a0 con 0x7fc59806f4d0 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 0x7fc598153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 0x7fc598153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54160/0 (socket says 172.21.15.46:54160) 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597e00640 1 -- 172.21.15.46:0/3224547450 learned_addr learned my addr 172.21.15.46:0/3224547450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc58ea00640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5980d0280 0x7fc5980cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597400640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 0x7fc598154650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597e00640 1 -- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 msgr2=0x7fc598154650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597e00640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 0x7fc598154650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597e00640 1 -- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5980d0280 msgr2=0x7fc5980cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597e00640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5980d0280 0x7fc5980cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597e00640 1 -- 172.21.15.46:0/3224547450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5980cb520 con 0x7fc59815dd50 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc597e00640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 0x7fc598153dd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc59000c450 tx=0x7fc59000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc58fe00640 1 -- 172.21.15.46:0/3224547450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590017070 con 0x7fc59815dd50 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc58fe00640 1 -- 172.21.15.46:0/3224547450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc590005cf0 con 0x7fc59815dd50 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc58fe00640 1 -- 172.21.15.46:0/3224547450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590007310 con 0x7fc59815dd50 2024-04-25T08:46:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc598154ce0 con 0x7fc59815dd50 2024-04-25T08:46:39.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5981552a0 con 0x7fc59815dd50 2024-04-25T08:46:39.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.165+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc598069010 con 0x7fc59815dd50 2024-04-25T08:46:39.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.169+0000 7fc58fe00640 1 -- 172.21.15.46:0/3224547450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5900074b0 con 0x7fc59815dd50 2024-04-25T08:46:39.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.169+0000 7fc58fe00640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc574066f60 0x7fc574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.169+0000 7fc58fe00640 1 -- 172.21.15.46:0/3224547450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc590084270 con 0x7fc59815dd50 2024-04-25T08:46:39.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.169+0000 7fc597400640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc574066f60 0x7fc574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.169+0000 7fc597400640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc574066f60 0x7fc574069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc588003fd0 tx=0x7fc5880039a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:39.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.173+0000 7fc58fe00640 1 -- 172.21.15.46:0/3224547450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5900553e0 con 0x7fc59815dd50 2024-04-25T08:46:39.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc59806fb00 con 0x7fc59815dd50 2024-04-25T08:46:39.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc58fe00640 1 -- 172.21.15.46:0/3224547450 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc590054ce0 con 0x7fc59815dd50 2024-04-25T08:46:39.330 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc574066f60 msgr2=0x7fc574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc574066f60 0x7fc574069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc588003fd0 tx=0x7fc5880039a0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 msgr2=0x7fc598153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 0x7fc598153dd0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc59000c450 tx=0x7fc59000c910 comp rx=0 tx=0).stop 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 shutdown_connections 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc574066f60 0x7fc574069420 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59815dd50 0x7fc598153dd0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5980d0280 0x7fc5980cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 --2- 172.21.15.46:0/3224547450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc59806f4d0 0x7fc598154650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 >> 172.21.15.46:0/3224547450 conn(0x7fc5980b3160 msgr2=0x7fc598099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:39.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 shutdown_connections 2024-04-25T08:46:39.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.337+0000 7fc59e000640 1 -- 172.21.15.46:0/3224547450 wait complete. 2024-04-25T08:46:39.351 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T08:46:39.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.497+0000 7fd48b400640 1 Processor -- start 2024-04-25T08:46:39.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.497+0000 7fd48b400640 1 -- start start 2024-04-25T08:46:39.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd48b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd48415a750 0x7fd48406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd48b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd4840aed10 0x7fd48406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd48b400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48406d5e0 con 0x7fd4840cc9e0 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd48b400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd48415ab30 con 0x7fd4840aed10 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd48b400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4840ce4f0 con 0x7fd48415a750 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd483e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd4840aed10 0x7fd48406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd489400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd4840cc9e0 0x7fd4840b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59338/0 (socket says 172.21.15.46:59338) 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd489400640 1 -- 172.21.15.46:0/1168159085 learned_addr learned my addr 172.21.15.46:0/1168159085 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd483e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd4840aed10 0x7fd48406f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54164/0 (socket says 172.21.15.46:54164) 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd488a00640 1 --2- 172.21.15.46:0/1168159085 >> v2:172.21.15.82:3300/0 conn(0x7fd48415a750 0x7fd48406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd483e00640 1 -- 172.21.15.46:0/1168159085 >> v2:172.21.15.82:3300/0 conn(0x7fd48415a750 msgr2=0x7fd48406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd483e00640 1 --2- 172.21.15.46:0/1168159085 >> v2:172.21.15.82:3300/0 conn(0x7fd48415a750 0x7fd48406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd483e00640 1 -- 172.21.15.46:0/1168159085 >> v1:172.21.15.46:6789/0 conn(0x7fd4840cc9e0 legacy=0x7fd4840b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd483e00640 1 -- 172.21.15.46:0/1168159085 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4840ce720 con 0x7fd4840aed10 2024-04-25T08:46:39.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.501+0000 7fd483400640 1 -- 172.21.15.46:0/1168159085 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415500180 0 0) 0x7fd48406d5e0 con 0x7fd4840cc9e0 2024-04-25T08:46:39.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd483e00640 1 --2- 172.21.15.46:0/1168159085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 0x7fd48406f150 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd48404ffd0 tx=0x7fd47001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d95b2f68811927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:39.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd483400640 1 -- 172.21.15.46:0/1168159085 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4700b0070 con 0x7fd4840aed10 2024-04-25T08:46:39.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd483400640 1 -- 172.21.15.46:0/1168159085 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4700ab600 con 0x7fd4840aed10 2024-04-25T08:46:39.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd483400640 1 -- 172.21.15.46:0/1168159085 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4700ab9d0 con 0x7fd4840aed10 2024-04-25T08:46:39.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 -- 172.21.15.46:0/1168159085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 msgr2=0x7fd48406f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 --2- 172.21.15.46:0/1168159085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 0x7fd48406f150 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd48404ffd0 tx=0x7fd47001c480 comp rx=0 tx=0).stop 2024-04-25T08:46:39.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 -- 172.21.15.46:0/1168159085 shutdown_connections 2024-04-25T08:46:39.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 --2- 172.21.15.46:0/1168159085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 0x7fd48406f150 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 --2- 172.21.15.46:0/1168159085 >> v2:172.21.15.82:3300/0 conn(0x7fd48415a750 0x7fd48406ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 -- 172.21.15.46:0/1168159085 >> 172.21.15.46:0/1168159085 conn(0x7fd4840b12a0 msgr2=0x7fd4840b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:39.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 -- 172.21.15.46:0/1168159085 shutdown_connections 2024-04-25T08:46:39.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 -- 172.21.15.46:0/1168159085 wait complete. 2024-04-25T08:46:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 Processor -- start 2024-04-25T08:46:39.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 -- start start 2024-04-25T08:46:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.505+0000 7fd48b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 0x7fd484156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd48b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4840cc9e0 0x7fd484157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd48b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd48415a750 0x7fd484148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd48b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd484167230 con 0x7fd4840aed10 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd48b400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48406d270 con 0x7fd4840cc9e0 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd48b400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4841673b0 con 0x7fd48415a750 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd488a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 0x7fd484156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd488a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 0x7fd484156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54174/0 (socket says 172.21.15.46:54174) 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd488a00640 1 -- 172.21.15.46:0/87778154 learned_addr learned my addr 172.21.15.46:0/87778154 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd483e00640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4840cc9e0 0x7fd484157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd489400640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd48415a750 0x7fd484148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd488a00640 1 -- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd48415a750 msgr2=0x7fd484148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd488a00640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd48415a750 0x7fd484148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd488a00640 1 -- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4840cc9e0 msgr2=0x7fd484157d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd488a00640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4840cc9e0 0x7fd484157d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd488a00640 1 -- 172.21.15.46:0/87778154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484158290 con 0x7fd4840aed10 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd483e00640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4840cc9e0 0x7fd484157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:39.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd488a00640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 0x7fd484156570 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e8a0 tx=0x7fd47800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd480c00640 1 -- 172.21.15.46:0/87778154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478017070 con 0x7fd4840aed10 2024-04-25T08:46:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd480c00640 1 -- 172.21.15.46:0/87778154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47800c940 con 0x7fd4840aed10 2024-04-25T08:46:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd480c00640 1 -- 172.21.15.46:0/87778154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47800ccf0 con 0x7fd4840aed10 2024-04-25T08:46:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4841595d0 con 0x7fd4840aed10 2024-04-25T08:46:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd484158810 con 0x7fd4840aed10 2024-04-25T08:46:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd484069080 con 0x7fd4840aed10 2024-04-25T08:46:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.509+0000 7fd480c00640 1 -- 172.21.15.46:0/87778154 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd47801a070 con 0x7fd4840aed10 2024-04-25T08:46:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.513+0000 7fd480c00640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd464066f60 0x7fd464069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.513+0000 7fd480c00640 1 -- 172.21.15.46:0/87778154 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd478084e00 con 0x7fd4840aed10 2024-04-25T08:46:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.513+0000 7fd483e00640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd464066f60 0x7fd464069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.513+0000 7fd483e00640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd464066f60 0x7fd464069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd4700ad000 tx=0x7fd4700a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.517+0000 7fd480c00640 1 -- 172.21.15.46:0/87778154 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd478055f70 con 0x7fd4840aed10 2024-04-25T08:46:39.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:39.737+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7fd484157440 con 0x7fd4840aed10 2024-04-25T08:46:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.645+0000 7fd480c00640 1 -- 172.21.15.46:0/87778154 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v231) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd478055870 con 0x7fd4840aed10 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd464066f60 msgr2=0x7fd464069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd464066f60 0x7fd464069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd4700ad000 tx=0x7fd4700a6000 comp rx=0 tx=0).stop 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 msgr2=0x7fd484156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 0x7fd484156570 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd47800e8a0 tx=0x7fd47800eca0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 shutdown_connections 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd464066f60 0x7fd464069420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd48415a750 0x7fd484148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd4840cc9e0 0x7fd484157d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 --2- 172.21.15.46:0/87778154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4840aed10 0x7fd484156570 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 >> 172.21.15.46:0/87778154 conn(0x7fd4840b12a0 msgr2=0x7fd4840af690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 shutdown_connections 2024-04-25T08:46:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.649+0000 7fd48b400640 1 -- 172.21.15.46:0/87778154 wait complete. 2024-04-25T08:46:40.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:46:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb31da00640 1 Processor -- start 2024-04-25T08:46:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb31da00640 1 -- start start 2024-04-25T08:46:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb31da00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb318145310 0x7fb31806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb31da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb31806dc70 0x7fb3180cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb31da00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3180cd930 con 0x7fb3180cf1b0 2024-04-25T08:46:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb31da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180cdbd0 con 0x7fb31806dc70 2024-04-25T08:46:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb31da00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180cdeb0 con 0x7fb318145310 2024-04-25T08:46:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb30fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb31806dc70 0x7fb3180cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb30fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb31806dc70 0x7fb3180cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54178/0 (socket says 172.21.15.46:54178) 2024-04-25T08:46:40.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.821+0000 7fb30fe00640 1 -- 172.21.15.46:0/2745100390 learned_addr learned my addr 172.21.15.46:0/2745100390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb316a00640 1 --2- 172.21.15.46:0/2745100390 >> v2:172.21.15.82:3300/0 conn(0x7fb318145310 0x7fb31806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb30fe00640 1 -- 172.21.15.46:0/2745100390 >> v2:172.21.15.82:3300/0 conn(0x7fb318145310 msgr2=0x7fb31806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb30fe00640 1 --2- 172.21.15.46:0/2745100390 >> v2:172.21.15.82:3300/0 conn(0x7fb318145310 0x7fb31806e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb30fe00640 1 -- 172.21.15.46:0/2745100390 >> v1:172.21.15.82:6789/0 conn(0x7fb3180cf1b0 legacy=0x7fb31815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb30fe00640 1 -- 172.21.15.46:0/2745100390 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180ce190 con 0x7fb31806dc70 2024-04-25T08:46:40.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb316a00640 1 --2- 172.21.15.46:0/2745100390 >> v2:172.21.15.82:3300/0 conn(0x7fb318145310 0x7fb31806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb30fe00640 1 --2- 172.21.15.46:0/2745100390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31806dc70 0x7fb3180cec70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb318072530 tx=0x7fb30001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784e0cc9371c9265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb316000640 1 -- 172.21.15.46:0/2745100390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3000aa650 con 0x7fb31806dc70 2024-04-25T08:46:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb316000640 1 -- 172.21.15.46:0/2745100390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3000af070 con 0x7fb31806dc70 2024-04-25T08:46:40.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb316000640 1 -- 172.21.15.46:0/2745100390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3000aaa10 con 0x7fb31806dc70 2024-04-25T08:46:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb31da00640 1 -- 172.21.15.46:0/2745100390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31806dc70 msgr2=0x7fb3180cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb31da00640 1 --2- 172.21.15.46:0/2745100390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31806dc70 0x7fb3180cec70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fb318072530 tx=0x7fb30001c4f0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.825+0000 7fb31da00640 1 -- 172.21.15.46:0/2745100390 shutdown_connections 2024-04-25T08:46:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 --2- 172.21.15.46:0/2745100390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb31806dc70 0x7fb3180cec70 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 --2- 172.21.15.46:0/2745100390 >> v2:172.21.15.82:3300/0 conn(0x7fb318145310 0x7fb31806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 -- 172.21.15.46:0/2745100390 >> 172.21.15.46:0/2745100390 conn(0x7fb31806ac10 msgr2=0x7fb31806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 -- 172.21.15.46:0/2745100390 shutdown_connections 2024-04-25T08:46:40.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 -- 172.21.15.46:0/2745100390 wait complete. 2024-04-25T08:46:40.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 Processor -- start 2024-04-25T08:46:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 -- start start 2024-04-25T08:46:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180cf1b0 0x7fb318157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318145310 0x7fb3181576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3181584a0 0x7fb318158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31815d920 con 0x7fb3181584a0 2024-04-25T08:46:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31815d7a0 con 0x7fb3180cf1b0 2024-04-25T08:46:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb31da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb318168510 con 0x7fb318145310 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb317400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3181584a0 0x7fb318158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb316a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180cf1b0 0x7fb318157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb316a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180cf1b0 0x7fb318157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:52346/0 (socket says 172.21.15.46:52346) 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb317400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3181584a0 0x7fb318158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54192/0 (socket says 172.21.15.46:54192) 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb316a00640 1 -- 172.21.15.46:0/664821299 learned_addr learned my addr 172.21.15.46:0/664821299 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb30fe00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318145310 0x7fb3181576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb317400640 1 -- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318145310 msgr2=0x7fb3181576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb317400640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318145310 0x7fb3181576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb317400640 1 -- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180cf1b0 msgr2=0x7fb318157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb317400640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180cf1b0 0x7fb318157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:40.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb317400640 1 -- 172.21.15.46:0/664821299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180afe10 con 0x7fb3181584a0 2024-04-25T08:46:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.829+0000 7fb317400640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3181584a0 0x7fb318158a90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb31000c700 tx=0x7fb31000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb30ea00640 1 -- 172.21.15.46:0/664821299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310003b80 con 0x7fb3181584a0 2024-04-25T08:46:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb30ea00640 1 -- 172.21.15.46:0/664821299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb310003d20 con 0x7fb3181584a0 2024-04-25T08:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb30ea00640 1 -- 172.21.15.46:0/664821299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31000aa30 con 0x7fb3181584a0 2024-04-25T08:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3180b14a0 con 0x7fb3181584a0 2024-04-25T08:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3180b1a60 con 0x7fb3181584a0 2024-04-25T08:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb318069010 con 0x7fb3181584a0 2024-04-25T08:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb30ea00640 1 -- 172.21.15.46:0/664821299 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb31000abd0 con 0x7fb3181584a0 2024-04-25T08:46:40.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb30ea00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2f4066f90 0x7fb2f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:40.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb30ea00640 1 -- 172.21.15.46:0/664821299 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb310084c90 con 0x7fb3181584a0 2024-04-25T08:46:40.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.833+0000 7fb316a00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2f4066f90 0x7fb2f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:40.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.837+0000 7fb316a00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2f4066f90 0x7fb2f4069450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb3180b3b50 tx=0x7fb308008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:40.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:40.837+0000 7fb30ea00640 1 -- 172.21.15.46:0/664821299 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb310055720 con 0x7fb3181584a0 2024-04-25T08:46:40.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.005+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb3180acd30 con 0x7fb3181584a0 2024-04-25T08:46:41.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb30ea00640 1 -- 172.21.15.46:0/664821299 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb3100597c0 con 0x7fb3181584a0 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2f4066f90 msgr2=0x7fb2f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2f4066f90 0x7fb2f4069450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb3180b3b50 tx=0x7fb308008040 comp rx=0 tx=0).stop 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3181584a0 msgr2=0x7fb318158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3181584a0 0x7fb318158a90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb31000c700 tx=0x7fb31000cbc0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 shutdown_connections 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2f4066f90 0x7fb2f4069450 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3181584a0 0x7fb318158a90 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb318145310 0x7fb3181576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 --2- 172.21.15.46:0/664821299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3180cf1b0 0x7fb318157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 >> 172.21.15.46:0/664821299 conn(0x7fb31806ac10 msgr2=0x7fb3181444e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:41.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 shutdown_connections 2024-04-25T08:46:41.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.009+0000 7fb31da00640 1 -- 172.21.15.46:0/664821299 wait complete. 2024-04-25T08:46:41.023 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:46:41.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:46:41.123 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:46:41.214 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:46:41.214 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:46:41.214 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:41.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:46:41.223 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:46:41.223 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:41.232 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:46:41.233 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:46:41.233 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:41.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:46:41.272 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:46:41.273 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:41.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:46:41.284 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:46:41.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:41.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.465+0000 7f685fa00640 1 Processor -- start 2024-04-25T08:46:41.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.465+0000 7f685fa00640 1 -- start start 2024-04-25T08:46:41.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f685fa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580cc150 0x7f68580b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f685fa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68580b25a0 con 0x7f68580ce420 2024-04-25T08:46:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f685fa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68580cc6f0 con 0x7f6858157b00 2024-04-25T08:46:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f685fa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68580b2eb0 con 0x7f68580cc150 2024-04-25T08:46:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f685da00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f68580ce420 0x7f685815da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59340/0 (socket says 172.21.15.46:59340) 2024-04-25T08:46:41.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f685da00640 1 -- 172.21.15.46:0/1549715492 learned_addr learned my addr 172.21.15.46:0/1549715492 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f684fe00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580cc150 0x7f68580b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274619892 0 0) 0x7f68580b25a0 con 0x7f68580ce420 2024-04-25T08:46:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f683c003450 con 0x7f68580ce420 2024-04-25T08:46:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160985267 0 0) 0x7f68580cc6f0 con 0x7f6858157b00 2024-04-25T08:46:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68580b25a0 con 0x7f6858157b00 2024-04-25T08:46:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1793551691 0 0) 0x7f683c003450 con 0x7f68580ce420 2024-04-25T08:46:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580cc150 msgr2=0x7f68580b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:41.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580cc150 0x7f68580b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 >> v1:172.21.15.82:6789/0 conn(0x7f6858157b00 legacy=0x7f68580cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68580b2860 con 0x7f68580ce420 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f6850001a00 con 0x7f68580ce420 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 >> v1:172.21.15.46:6789/0 conn(0x7f68580ce420 legacy=0x7f685815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f683c003c70 0x7f683c008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c004100 0x7f683c00add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c00b440 0x7f683c00d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68580b2eb0 con 0x7f683c004100 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c00e1b0 con 0x7f683c00b440 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c00e3b0 con 0x7f683c003c70 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f684fe00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580cc150 0x7f68580b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f684fe00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c004100 0x7f683c00add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f685d000640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f683c003c70 0x7f683c008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f685da00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c00b440 0x7f683c00d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f684fe00640 1 -- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f683c003c70 msgr2=0x7f683c008860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:41.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f684fe00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f683c003c70 0x7f683c008860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f684fe00640 1 -- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c00b440 msgr2=0x7f683c00d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:41.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f684fe00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c00b440 0x7f683c00d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.469+0000 7f684fe00640 1 -- 172.21.15.46:0/1549715492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c003450 con 0x7f683c004100 2024-04-25T08:46:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f684fe00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c004100 0x7f683c00add0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6858070620 tx=0x7f684001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5453250aeeb7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400097f0 con 0x7f683c004100 2024-04-25T08:46:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840004030 con 0x7f683c004100 2024-04-25T08:46:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f6857e00640 1 -- 172.21.15.46:0/1549715492 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840004400 con 0x7f683c004100 2024-04-25T08:46:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f685fa00640 1 -- 172.21.15.46:0/1549715492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c004100 msgr2=0x7f683c00add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f685fa00640 1 --2- 172.21.15.46:0/1549715492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f683c004100 0x7f683c00add0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6858070620 tx=0x7f684001cab0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f685da00640 1 -- 172.21.15.46:0/1549715492 reap_dead start 2024-04-25T08:46:41.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f685fa00640 1 -- 172.21.15.46:0/1549715492 shutdown_connections 2024-04-25T08:46:41.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f685fa00640 1 -- 172.21.15.46:0/1549715492 >> 172.21.15.46:0/1549715492 conn(0x7f68580b12a0 msgr2=0x7f68580b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:41.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f685fa00640 1 -- 172.21.15.46:0/1549715492 shutdown_connections 2024-04-25T08:46:41.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.473+0000 7f685fa00640 1 -- 172.21.15.46:0/1549715492 wait complete. 2024-04-25T08:46:41.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 Processor -- start 2024-04-25T08:46:41.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 -- start start 2024-04-25T08:46:41.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c004100 0x7f685814ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68580af5e0 0x7f68580afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580ade70 0x7f68580ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68580ce8b0 con 0x7f68580af5e0 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68580b4300 con 0x7f683c004100 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68580cea30 con 0x7f68580ade70 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f684fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68580af5e0 0x7f68580afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f684fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68580af5e0 0x7f68580afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54214/0 (socket says 172.21.15.46:54214) 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f684fe00640 1 -- 172.21.15.46:0/3971511000 learned_addr learned my addr 172.21.15.46:0/3971511000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685da00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580ade70 0x7f68580ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685d000640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c004100 0x7f685814ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f684fe00640 1 -- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580ade70 msgr2=0x7f68580ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f684fe00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580ade70 0x7f68580ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f684fe00640 1 -- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c004100 msgr2=0x7f685814ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f684fe00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c004100 0x7f685814ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f684fe00640 1 -- 172.21.15.46:0/3971511000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68581f43f0 con 0x7f68580af5e0 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f684fe00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68580af5e0 0x7f68580afa40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f684001c8e0 tx=0x7f683c003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:41.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f6855600640 1 -- 172.21.15.46:0/3971511000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400bb070 con 0x7f68580af5e0 2024-04-25T08:46:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f6855600640 1 -- 172.21.15.46:0/3971511000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68400a5930 con 0x7f68580af5e0 2024-04-25T08:46:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68581f3630 con 0x7f68580af5e0 2024-04-25T08:46:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f6855600640 1 -- 172.21.15.46:0/3971511000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400a5ce0 con 0x7f68580af5e0 2024-04-25T08:46:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.477+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68581f3bf0 con 0x7f68580af5e0 2024-04-25T08:46:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.481+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6858069080 con 0x7f68580af5e0 2024-04-25T08:46:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.481+0000 7f6855600640 1 -- 172.21.15.46:0/3971511000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68400be070 con 0x7f68580af5e0 2024-04-25T08:46:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.481+0000 7f6855600640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6834066f60 0x7f6834069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.481+0000 7f6855600640 1 -- 172.21.15.46:0/3971511000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f684011eae0 con 0x7f68580af5e0 2024-04-25T08:46:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.481+0000 7f685d000640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6834066f60 0x7f6834069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.481+0000 7f685d000640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6834066f60 0x7f6834069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6848005630 tx=0x7f6848007250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:41.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.485+0000 7f6855600640 1 -- 172.21.15.46:0/3971511000 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68400167c0 con 0x7f68580af5e0 2024-04-25T08:46:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68580b1db0 con 0x7f68580af5e0 2024-04-25T08:46:41.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f6855600640 1 -- 172.21.15.46:0/3971511000 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f68400ef600 con 0x7f68580af5e0 2024-04-25T08:46:41.677 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:41.678 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2218033825","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2218033825},{"type":"v1","addr":"172.21.15.82:6836","nonce":2218033825}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6016,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3469118160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3469118160},{"type":"v1","addr":"172.21.15.82:6837","nonce":3469118160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6198,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3623135912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3623135912},{"type":"v1","addr":"172.21.15.46:6833","nonce":3623135912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6690,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2785834291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2785834291},{"type":"v1","addr":"172.21.15.46:6835","nonce":2785834291}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[]} 2024-04-25T08:46:41.678 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 38 2024-04-25T08:46:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6834066f60 msgr2=0x7f6834069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6834066f60 0x7f6834069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6848005630 tx=0x7f6848007250 comp rx=0 tx=0).stop 2024-04-25T08:46:41.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68580af5e0 msgr2=0x7f68580afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68580af5e0 0x7f68580afa40 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f684001c8e0 tx=0x7f683c003740 comp rx=0 tx=0).stop 2024-04-25T08:46:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 shutdown_connections 2024-04-25T08:46:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6834066f60 0x7f6834069420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f68580ade70 0x7f68580ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68580af5e0 0x7f68580afa40 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 --2- 172.21.15.46:0/3971511000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f683c004100 0x7f685814ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 >> 172.21.15.46:0/3971511000 conn(0x7f68580b12a0 msgr2=0x7f68580cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 shutdown_connections 2024-04-25T08:46:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.685+0000 7f685fa00640 1 -- 172.21.15.46:0/3971511000 wait complete. 2024-04-25T08:46:41.699 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:46:41.699 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:46:41.699 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:46:41.699 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:46:41.699 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:46:41.700 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:46:41.700 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:46:41.700 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:46:41.700 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:46:41.700 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:46:41.700 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:46:41.700 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:46:41.700 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:46:41.701 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:46:41.701 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:46:41.701 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:46:41.714 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:46:41.714 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:46:41.715 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:46:41.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:46:41.717 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:46:41.717 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:46:41.717 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:46:41.718 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:46:41.759 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:46:41.759 DEBUG:teuthology.parallel:result is None 2024-04-25T08:46:41.761 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:46:41.761 DEBUG:teuthology.parallel:result is None 2024-04-25T08:46:41.766 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:46:41.766 DEBUG:teuthology.parallel:result is None 2024-04-25T08:46:41.768 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:46:41.768 DEBUG:teuthology.parallel:result is None 2024-04-25T08:46:41.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:46:41.846 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:46:41.847 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:46:41.859 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-04-25T08:46:41.861 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-04-25T08:46:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed11400640 1 Processor -- start 2024-04-25T08:46:41.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed11400640 1 -- start start 2024-04-25T08:46:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed11400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed0c070c90 0x7fed0c071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed11400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed0c0b5250 con 0x7fed0c15dbd0 2024-04-25T08:46:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed11400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed0c0b5530 con 0x7fed0c06c380 2024-04-25T08:46:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed11400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed0c0b5af0 con 0x7fed0c070c90 2024-04-25T08:46:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed09e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed0c070c90 0x7fed0c071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed0a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fed0c15dbd0 0x7fed0c0b06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59350/0 (socket says 172.21.15.46:59350) 2024-04-25T08:46:41.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed0a800640 1 -- 172.21.15.46:0/3074335817 learned_addr learned my addr 172.21.15.46:0/3074335817 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed09e00640 1 -- 172.21.15.46:0/3074335817 >> v1:172.21.15.82:6789/0 conn(0x7fed0c06c380 legacy=0x7fed0c06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed09e00640 1 -- 172.21.15.46:0/3074335817 >> v1:172.21.15.46:6789/0 conn(0x7fed0c15dbd0 legacy=0x7fed0c0b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed09e00640 1 -- 172.21.15.46:0/3074335817 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c0b5810 con 0x7fed0c070c90 2024-04-25T08:46:41.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.925+0000 7fed09400640 1 -- 172.21.15.46:0/3074335817 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614454386 0 0) 0x7fed0c0b5250 con 0x7fed0c15dbd0 2024-04-25T08:46:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed09e00640 1 --2- 172.21.15.46:0/3074335817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c070c90 0x7fed0c071110 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fecf80097c0 tx=0x7fecf801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb834515008a96c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed09400640 1 -- 172.21.15.46:0/3074335817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80b0070 con 0x7fed0c070c90 2024-04-25T08:46:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed09400640 1 -- 172.21.15.46:0/3074335817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf80ab600 con 0x7fed0c070c90 2024-04-25T08:46:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed09400640 1 -- 172.21.15.46:0/3074335817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80ab9d0 con 0x7fed0c070c90 2024-04-25T08:46:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed11400640 1 -- 172.21.15.46:0/3074335817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c070c90 msgr2=0x7fed0c071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed11400640 1 --2- 172.21.15.46:0/3074335817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c070c90 0x7fed0c071110 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fecf80097c0 tx=0x7fecf801c460 comp rx=0 tx=0).stop 2024-04-25T08:46:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed11400640 1 -- 172.21.15.46:0/3074335817 shutdown_connections 2024-04-25T08:46:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed11400640 1 --2- 172.21.15.46:0/3074335817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c070c90 0x7fed0c071110 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed11400640 1 -- 172.21.15.46:0/3074335817 >> 172.21.15.46:0/3074335817 conn(0x7fed0c06f4f0 msgr2=0x7fed0c06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed11400640 1 -- 172.21.15.46:0/3074335817 shutdown_connections 2024-04-25T08:46:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed11400640 1 -- 172.21.15.46:0/3074335817 wait complete. 2024-04-25T08:46:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed11400640 1 Processor -- start 2024-04-25T08:46:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.929+0000 7fed11400640 1 -- start start 2024-04-25T08:46:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed11400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c06c380 0x7fed0c150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed11400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed0c070c90 0x7fed0c0b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed11400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed0c15dbd0 0x7fed0c0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed11400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c16b860 con 0x7fed0c06c380 2024-04-25T08:46:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed11400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c0b1010 con 0x7fed0c15dbd0 2024-04-25T08:46:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed11400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c16b9e0 con 0x7fed0c070c90 2024-04-25T08:46:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c06c380 0x7fed0c150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c06c380 0x7fed0c150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54264/0 (socket says 172.21.15.46:54264) 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0a800640 1 -- 172.21.15.46:0/1015226854 learned_addr learned my addr 172.21.15.46:0/1015226854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed09e00640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed0c070c90 0x7fed0c0b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0b200640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed0c15dbd0 0x7fed0c0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0a800640 1 -- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed0c070c90 msgr2=0x7fed0c0b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0a800640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed0c070c90 0x7fed0c0b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0a800640 1 -- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed0c15dbd0 msgr2=0x7fed0c0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0a800640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed0c15dbd0 0x7fed0c0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0a800640 1 -- 172.21.15.46:0/1015226854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c1515d0 con 0x7fed0c06c380 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed0a800640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c06c380 0x7fed0c150b00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c4b0 tx=0x7fecfc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed02a00640 1 -- 172.21.15.46:0/1015226854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc018070 con 0x7fed0c06c380 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed02a00640 1 -- 172.21.15.46:0/1015226854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc007500 con 0x7fed0c06c380 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed02a00640 1 -- 172.21.15.46:0/1015226854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc004a30 con 0x7fed0c06c380 2024-04-25T08:46:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0c0cf500 con 0x7fed0c06c380 2024-04-25T08:46:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0c0cea20 con 0x7fed0c06c380 2024-04-25T08:46:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0c069080 con 0x7fed0c06c380 2024-04-25T08:46:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.933+0000 7fed02a00640 1 -- 172.21.15.46:0/1015226854 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fecfc004bd0 con 0x7fed0c06c380 2024-04-25T08:46:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.937+0000 7fed02a00640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecec066f90 0x7fecec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.937+0000 7fed02a00640 1 -- 172.21.15.46:0/1015226854 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fecfc083fe0 con 0x7fed0c06c380 2024-04-25T08:46:41.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.937+0000 7fed09e00640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecec066f90 0x7fecec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.937+0000 7fed09e00640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecec066f90 0x7fecec069450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fecf80ad000 tx=0x7fecf80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:41.941+0000 7fed02a00640 1 -- 172.21.15.46:0/1015226854 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed0c1515d0 con 0x7fed0c06c380 2024-04-25T08:46:42.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.113+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fed0c157ef0 con 0x7fed0c06c380 2024-04-25T08:46:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.657+0000 7fed02a00640 1 -- 172.21.15.46:0/1015226854 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7fecfc0551d0 con 0x7fed0c06c380 2024-04-25T08:46:42.652 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-25T08:46:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecec066f90 msgr2=0x7fecec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecec066f90 0x7fecec069450 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fecf80ad000 tx=0x7fecf80a6000 comp rx=0 tx=0).stop 2024-04-25T08:46:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c06c380 msgr2=0x7fed0c150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c06c380 0x7fed0c150b00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c4b0 tx=0x7fecfc00c970 comp rx=0 tx=0).stop 2024-04-25T08:46:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 shutdown_connections 2024-04-25T08:46:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecec066f90 0x7fecec069450 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed0c15dbd0 0x7fed0c0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed0c070c90 0x7fed0c0b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 --2- 172.21.15.46:0/1015226854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0c06c380 0x7fed0c150b00 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 >> 172.21.15.46:0/1015226854 conn(0x7fed0c06f4f0 msgr2=0x7fed0c0b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 shutdown_connections 2024-04-25T08:46:42.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.661+0000 7fed11400640 1 -- 172.21.15.46:0/1015226854 wait complete. 2024-04-25T08:46:42.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:46:42.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.829+0000 7f4f68600640 1 Processor -- start 2024-04-25T08:46:42.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.829+0000 7f4f68600640 1 -- start start 2024-04-25T08:46:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.829+0000 7f4f68600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f6015bdb0 0x7f4f600cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.829+0000 7f4f68600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600cc700 con 0x7f4f600cda80 2024-04-25T08:46:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f68600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f600cc9e0 con 0x7f4f60159230 2024-04-25T08:46:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f68600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f600cded0 con 0x7f4f6015bdb0 2024-04-25T08:46:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f65200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f6015bdb0 0x7f4f600cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f66600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4f600cda80 0x7f4f6015d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59354/0 (socket says 172.21.15.46:59354) 2024-04-25T08:46:42.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f66600640 1 -- 172.21.15.46:0/3435753270 learned_addr learned my addr 172.21.15.46:0/3435753270 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:42.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f65200640 1 -- 172.21.15.46:0/3435753270 >> v1:172.21.15.82:6789/0 conn(0x7f4f60159230 legacy=0x7f4f600cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:42.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f65200640 1 -- 172.21.15.46:0/3435753270 >> v1:172.21.15.46:6789/0 conn(0x7f4f600cda80 legacy=0x7f4f6015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:42.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f65200640 1 -- 172.21.15.46:0/3435753270 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600cccc0 con 0x7f4f6015bdb0 2024-04-25T08:46:42.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f5fe00640 1 -- 172.21.15.46:0/3435753270 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102992628 0 0) 0x7f4f600cc700 con 0x7f4f600cda80 2024-04-25T08:46:42.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f65200640 1 --2- 172.21.15.46:0/3435753270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f6015bdb0 0x7f4f600cd540 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0097c0 tx=0x7f4f4c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27eaa6299cdbf0bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:42.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f5fe00640 1 -- 172.21.15.46:0/3435753270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0b0070 con 0x7f4f6015bdb0 2024-04-25T08:46:42.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f5fe00640 1 -- 172.21.15.46:0/3435753270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c0ab600 con 0x7f4f6015bdb0 2024-04-25T08:46:42.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.833+0000 7f4f5fe00640 1 -- 172.21.15.46:0/3435753270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0ab9d0 con 0x7f4f6015bdb0 2024-04-25T08:46:42.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 -- 172.21.15.46:0/3435753270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f6015bdb0 msgr2=0x7f4f600cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:42.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 --2- 172.21.15.46:0/3435753270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f6015bdb0 0x7f4f600cd540 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0097c0 tx=0x7f4f4c01c410 comp rx=0 tx=0).stop 2024-04-25T08:46:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 -- 172.21.15.46:0/3435753270 shutdown_connections 2024-04-25T08:46:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 --2- 172.21.15.46:0/3435753270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f6015bdb0 0x7f4f600cd540 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 -- 172.21.15.46:0/3435753270 >> 172.21.15.46:0/3435753270 conn(0x7f4f600b12a0 msgr2=0x7f4f600b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 -- 172.21.15.46:0/3435753270 shutdown_connections 2024-04-25T08:46:42.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 -- 172.21.15.46:0/3435753270 wait complete. 2024-04-25T08:46:42.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 Processor -- start 2024-04-25T08:46:42.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 -- start start 2024-04-25T08:46:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f600cda80 0x7f4f6013d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60159230 0x7f4f60159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f6015bdb0 0x7f4f600b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:42.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015e2a0 con 0x7f4f60159230 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015e120 con 0x7f4f6015bdb0 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f68600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6015e420 con 0x7f4f600cda80 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f65200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60159230 0x7f4f60159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f65200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60159230 0x7f4f60159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54286/0 (socket says 172.21.15.46:54286) 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f65200640 1 -- 172.21.15.46:0/3255899637 learned_addr learned my addr 172.21.15.46:0/3255899637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f66600640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f6015bdb0 0x7f4f600b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f65c00640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f600cda80 0x7f4f6013d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f65200640 1 -- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f600cda80 msgr2=0x7f4f6013d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f65200640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f600cda80 0x7f4f6013d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f65200640 1 -- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f6015bdb0 msgr2=0x7f4f600b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f65200640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f6015bdb0 0x7f4f600b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:42.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.837+0000 7f4f65200640 1 -- 172.21.15.46:0/3255899637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6015a840 con 0x7f4f60159230 2024-04-25T08:46:42.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f65200640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60159230 0x7f4f60159e40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c019020 tx=0x7f4f4c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:42.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f5d600640 1 -- 172.21.15.46:0/3255899637 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0ba070 con 0x7f4f60159230 2024-04-25T08:46:42.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f5d600640 1 -- 172.21.15.46:0/3255899637 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c0b5680 con 0x7f4f60159230 2024-04-25T08:46:42.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f5d600640 1 -- 172.21.15.46:0/3255899637 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0b0040 con 0x7f4f60159230 2024-04-25T08:46:42.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6015ab20 con 0x7f4f60159230 2024-04-25T08:46:42.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6015b0e0 con 0x7f4f60159230 2024-04-25T08:46:42.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f60069080 con 0x7f4f60159230 2024-04-25T08:46:42.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f5d600640 1 -- 172.21.15.46:0/3255899637 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f4c0bd070 con 0x7f4f60159230 2024-04-25T08:46:42.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f5d600640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f40066f90 0x7f4f40069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:42.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f5d600640 1 -- 172.21.15.46:0/3255899637 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4f4c11f3e0 con 0x7f4f60159230 2024-04-25T08:46:42.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.841+0000 7f4f65c00640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f40066f90 0x7f4f40069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:42.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.845+0000 7f4f65c00640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f40066f90 0x7f4f40069450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4f6015ce40 tx=0x7f4f54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:42.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:42.845+0000 7f4f5d600640 1 -- 172.21.15.46:0/3255899637 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f4c016090 con 0x7f4f60159230 2024-04-25T08:46:43.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.065+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4f60073db0 con 0x7f4f60159230 2024-04-25T08:46:43.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f5d600640 1 -- 172.21.15.46:0/3255899637 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4f4c0f05a0 con 0x7f4f60159230 2024-04-25T08:46:43.061 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:43.061 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmq+gDGRAAtB5ObM/zelMmCZ4cJPghdQ==", 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.062 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmyIaAJRAA9zodtgftP7faNth4UPyCTA==", 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmFahWIBAAORHmzDLXPjuANF+/W2wKpQ==", 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:46:43.063 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmRhrPLBAATXOHl1jGJYmIlnYk/JPcOw==", 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.064 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA0FypmaHiqEhAAYzasCl8t+pls5IHe9tCP4g==", 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA3FypmOGTuGhAA37F+qRI/+pj5dkLc3TBIUQ==", 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:46:43.065 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA6Fypm/YbNJRAAP1QyYDLY/xFvRz63R+H/Pw==", 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-25T08:46:43.066 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA9FypmiT8ALhAAX/2CJJeJ+2SGNhwah8NTBQ==", 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBDFypmyG88JBAAq1065gzLIIR/6fYaw0bBTg==", 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:46:43.067 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBGFypmNir3KxAAszCXJJ1KFtCusDZhBHAZSQ==", 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-25T08:46:43.068 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBJFypm4r7LNxAA42A07ntI6OOgblk+4i1Yiw==", 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBNFypmG1fOBhAAOKRgw04jIo8H3u2hWIZe2g==", 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q==", 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmxPyHOhAADBuigZlq3GHURsJkB0nnyA==", 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.070 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypm2PP0NhAAixV/kkY7eFPXUW/nJDOCOA==", 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:46:43.071 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAzFypmaVDtBBAAKnWk9CxZzg2r6T/IycuFEg==", 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.072 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAxFypm2W2MOBAAN7TDQiZXFV6+pWMTar95sg==", 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmzhptFBAAlFQZQn1GSseEguy5zfw2CA==", 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:46:43.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmAY6CEBAA2qHNALNTRKmCJRZatDqAvw==", 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-25T08:46:43.074 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:46:43.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f40066f90 msgr2=0x7f4f40069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f40066f90 0x7f4f40069450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4f6015ce40 tx=0x7f4f54009040 comp rx=0 tx=0).stop 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60159230 msgr2=0x7f4f60159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60159230 0x7f4f60159e40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c019020 tx=0x7f4f4c0191b0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 shutdown_connections 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f40066f90 0x7f4f40069450 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f6015bdb0 0x7f4f600b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60159230 0x7f4f60159e40 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 --2- 172.21.15.46:0/3255899637 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f600cda80 0x7f4f6013d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 >> 172.21.15.46:0/3255899637 conn(0x7f4f600b12a0 msgr2=0x7f4f601435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 shutdown_connections 2024-04-25T08:46:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.069+0000 7f4f68600640 1 -- 172.21.15.46:0/3255899637 wait complete. 2024-04-25T08:46:43.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:43.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.229+0000 7f23da600640 1 Processor -- start 2024-04-25T08:46:43.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.229+0000 7f23da600640 1 -- start start 2024-04-25T08:46:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23da600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f23d415dca0 0x7f23d40d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23da600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 0x7f23d40d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23da600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d406ed70 con 0x7f23d40cf140 2024-04-25T08:46:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23da600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d40afb40 con 0x7f23d415dca0 2024-04-25T08:46:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23da600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4162530 con 0x7f23d40afd40 2024-04-25T08:46:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23cbe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f23d415dca0 0x7f23d40d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23d3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 0x7f23d40d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23d3400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f23d40cf140 0x7f23d415d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:36268/0 (socket says 172.21.15.46:36268) 2024-04-25T08:46:43.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23cbe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f23d415dca0 0x7f23d40d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:52406/0 (socket says 172.21.15.46:52406) 2024-04-25T08:46:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23d3400640 1 -- 172.21.15.46:0/759209505 learned_addr learned my addr 172.21.15.46:0/759209505 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23cbe00640 1 -- 172.21.15.46:0/759209505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 msgr2=0x7f23d40d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23cbe00640 1 --2- 172.21.15.46:0/759209505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 0x7f23d40d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23cbe00640 1 -- 172.21.15.46:0/759209505 >> v1:172.21.15.82:6789/0 conn(0x7f23d40cf140 legacy=0x7f23d415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23cbe00640 1 -- 172.21.15.46:0/759209505 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d4161770 con 0x7f23d415dca0 2024-04-25T08:46:43.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.233+0000 7f23d2a00640 1 -- 172.21.15.46:0/759209505 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239502138 0 0) 0x7f23c4001120 con 0x7f23d40cf140 2024-04-25T08:46:43.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23cbe00640 1 --2- 172.21.15.46:0/759209505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d415dca0 0x7f23d40d03e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23d40afcc0 tx=0x7f23bc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177f2b9dde14cf6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:43.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23d2a00640 1 -- 172.21.15.46:0/759209505 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0ab610 con 0x7f23d415dca0 2024-04-25T08:46:43.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23d2a00640 1 -- 172.21.15.46:0/759209505 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc0b0070 con 0x7f23d415dca0 2024-04-25T08:46:43.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23d2a00640 1 -- 172.21.15.46:0/759209505 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0ab9d0 con 0x7f23d415dca0 2024-04-25T08:46:43.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 -- 172.21.15.46:0/759209505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d415dca0 msgr2=0x7f23d40d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 --2- 172.21.15.46:0/759209505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d415dca0 0x7f23d40d03e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f23d40afcc0 tx=0x7f23bc01c410 comp rx=0 tx=0).stop 2024-04-25T08:46:43.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 -- 172.21.15.46:0/759209505 shutdown_connections 2024-04-25T08:46:43.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 --2- 172.21.15.46:0/759209505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 0x7f23d40d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 --2- 172.21.15.46:0/759209505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d415dca0 0x7f23d40d03e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 -- 172.21.15.46:0/759209505 >> 172.21.15.46:0/759209505 conn(0x7f23d40728c0 msgr2=0x7f23d4099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:43.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 -- 172.21.15.46:0/759209505 shutdown_connections 2024-04-25T08:46:43.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 -- 172.21.15.46:0/759209505 wait complete. 2024-04-25T08:46:43.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 Processor -- start 2024-04-25T08:46:43.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 -- start start 2024-04-25T08:46:43.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 0x7f23d40b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d40cf140 0x7f23d4158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d415dca0 0x7f23d4152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40d1270 con 0x7f23d415dca0 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40af7f0 con 0x7f23d40cf140 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.237+0000 7f23da600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40d13f0 con 0x7f23d40afd40 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d415dca0 0x7f23d4152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d415dca0 0x7f23d4152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54294/0 (socket says 172.21.15.46:54294) 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3e00640 1 -- 172.21.15.46:0/751231521 learned_addr learned my addr 172.21.15.46:0/751231521 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23cbe00640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d40cf140 0x7f23d4158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3400640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 0x7f23d40b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3e00640 1 -- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 msgr2=0x7f23d40b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3e00640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 0x7f23d40b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3e00640 1 -- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d40cf140 msgr2=0x7f23d4158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3e00640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d40cf140 0x7f23d4158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3e00640 1 -- 172.21.15.46:0/751231521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d415e500 con 0x7f23d415dca0 2024-04-25T08:46:43.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23d3e00640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d415dca0 0x7f23d4152e20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009dd0 tx=0x7f23cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:43.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23cb400640 1 -- 172.21.15.46:0/751231521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc018020 con 0x7f23d415dca0 2024-04-25T08:46:43.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23cb400640 1 -- 172.21.15.46:0/751231521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc014480 con 0x7f23d415dca0 2024-04-25T08:46:43.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23cb400640 1 -- 172.21.15.46:0/751231521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc014870 con 0x7f23d415dca0 2024-04-25T08:46:43.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d4158a80 con 0x7f23d415dca0 2024-04-25T08:46:43.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d40b5440 con 0x7f23d415dca0 2024-04-25T08:46:43.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d415e6d0 con 0x7f23d415dca0 2024-04-25T08:46:43.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.241+0000 7f23cb400640 1 -- 172.21.15.46:0/751231521 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23cc01b020 con 0x7f23d415dca0 2024-04-25T08:46:43.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.245+0000 7f23cb400640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f23b0066f60 0x7f23b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.245+0000 7f23cb400640 1 -- 172.21.15.46:0/751231521 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f23cc085870 con 0x7f23d415dca0 2024-04-25T08:46:43.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.245+0000 7f23d3400640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f23b0066f60 0x7f23b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.245+0000 7f23d3400640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f23b0066f60 0x7f23b0069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f23c4000f70 tx=0x7f23c4007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:43.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.249+0000 7f23cb400640 1 -- 172.21.15.46:0/751231521 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23cc022070 con 0x7f23d415dca0 2024-04-25T08:46:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.409+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23d406ccb0 con 0x7f23d415dca0 2024-04-25T08:46:43.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23cb400640 1 -- 172.21.15.46:0/751231521 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13117 (secure 0 0 0) 0x7f23cc010040 con 0x7f23d415dca0 2024-04-25T08:46:43.406 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:43.406 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:42.669600+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f23b0066f60 msgr2=0x7f23b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f23b0066f60 0x7f23b0069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f23c4000f70 tx=0x7f23c4007b60 comp rx=0 tx=0).stop 2024-04-25T08:46:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d415dca0 msgr2=0x7f23d4152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d415dca0 0x7f23d4152e20 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009dd0 tx=0x7f23cc00c490 comp rx=0 tx=0).stop 2024-04-25T08:46:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 shutdown_connections 2024-04-25T08:46:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f23b0066f60 0x7f23b0069420 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23d415dca0 0x7f23d4152e20 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23d40cf140 0x7f23d4158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 --2- 172.21.15.46:0/751231521 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23d40afd40 0x7f23d40b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 >> 172.21.15.46:0/751231521 conn(0x7f23d40728c0 msgr2=0x7f23d415adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:43.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 shutdown_connections 2024-04-25T08:46:43.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.413+0000 7f23da600640 1 -- 172.21.15.46:0/751231521 wait complete. 2024-04-25T08:46:43.427 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:46:43.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:46:43.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.577+0000 7f8f38c00640 1 Processor -- start 2024-04-25T08:46:43.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.577+0000 7f8f38c00640 1 -- start start 2024-04-25T08:46:43.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f38c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3409a110 0x7f8f3406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f38c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f340cfc40 con 0x7f8f3415db10 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f38c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f340cff20 con 0x7f8f340cd3c0 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f38c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340d0200 con 0x7f8f3409a110 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8f3415db10 0x7f8f34099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59364/0 (socket says 172.21.15.46:59364) 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33400640 1 -- 172.21.15.46:0/1906863641 learned_addr learned my addr 172.21.15.46:0/1906863641 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f2be00640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3409a110 0x7f8f3406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103463487 0 0) 0x7f8f340cfc40 con 0x7f8f3415db10 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f18003530 con 0x7f8f3415db10 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1484870616 0 0) 0x7f8f18003530 con 0x7f8f3415db10 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3409a110 msgr2=0x7f8f3406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3409a110 0x7f8f3406c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 >> v1:172.21.15.82:6789/0 conn(0x7f8f340cd3c0 legacy=0x7f8f3406c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340cbbb0 con 0x7f8f3415db10 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f8f24001680 con 0x7f8f3415db10 2024-04-25T08:46:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 >> v1:172.21.15.46:6789/0 conn(0x7f8f3415db10 legacy=0x7f8f34099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f180039f0 0x7f8f18003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3409a110 0x7f8f18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f18004d50 0x7f8f180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f340d0200 con 0x7f8f180039f0 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1800bb10 con 0x7f8f18004d50 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1800bd10 con 0x7f8f3409a110 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33400640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f180039f0 0x7f8f18003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f2be00640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3409a110 0x7f8f18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33e00640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f18004d50 0x7f8f180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33400640 1 -- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3409a110 msgr2=0x7f8f18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33400640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f3409a110 0x7f8f18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33400640 1 -- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f18004d50 msgr2=0x7f8f180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33400640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f18004d50 0x7f8f180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33400640 1 -- 172.21.15.46:0/1906863641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f18003530 con 0x7f8f180039f0 2024-04-25T08:46:43.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.581+0000 7f8f33e00640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f18004d50 0x7f8f180051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f33400640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f180039f0 0x7f8f18003e10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8f24001ec0 tx=0x7f8f2401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb76534f9d03123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24007d40 con 0x7f8f180039f0 2024-04-25T08:46:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f24005b00 con 0x7f8f180039f0 2024-04-25T08:46:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f32a00640 1 -- 172.21.15.46:0/1906863641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24007070 con 0x7f8f180039f0 2024-04-25T08:46:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f38c00640 1 -- 172.21.15.46:0/1906863641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f180039f0 msgr2=0x7f8f18003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f38c00640 1 --2- 172.21.15.46:0/1906863641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f180039f0 0x7f8f18003e10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f8f24001ec0 tx=0x7f8f2401f3d0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f33e00640 1 -- 172.21.15.46:0/1906863641 reap_dead start 2024-04-25T08:46:43.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f38c00640 1 -- 172.21.15.46:0/1906863641 shutdown_connections 2024-04-25T08:46:43.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f38c00640 1 -- 172.21.15.46:0/1906863641 >> 172.21.15.46:0/1906863641 conn(0x7f8f340b2fb0 msgr2=0x7f8f340b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:43.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f38c00640 1 -- 172.21.15.46:0/1906863641 shutdown_connections 2024-04-25T08:46:43.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f38c00640 1 -- 172.21.15.46:0/1906863641 wait complete. 2024-04-25T08:46:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f38c00640 1 Processor -- start 2024-04-25T08:46:43.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.585+0000 7f8f38c00640 1 -- start start 2024-04-25T08:46:43.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f38c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f180039f0 0x7f8f341f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f38c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3415c380 0x7f8f340b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f38c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f340b3f40 0x7f8f3415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f38c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3415e070 con 0x7f8f340b3f40 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f38c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3415def0 con 0x7f8f3415c380 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f38c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3415e1f0 con 0x7f8f180039f0 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f340b3f40 0x7f8f3415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f340b3f40 0x7f8f3415e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54320/0 (socket says 172.21.15.46:54320) 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33e00640 1 -- 172.21.15.46:0/705638491 learned_addr learned my addr 172.21.15.46:0/705638491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33400640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f180039f0 0x7f8f341f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f2be00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3415c380 0x7f8f340b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33e00640 1 -- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f180039f0 msgr2=0x7f8f341f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33e00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f180039f0 0x7f8f341f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33e00640 1 -- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3415c380 msgr2=0x7f8f340b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33e00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3415c380 0x7f8f340b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33e00640 1 -- 172.21.15.46:0/705638491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3415c800 con 0x7f8f340b3f40 2024-04-25T08:46:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f33e00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f340b3f40 0x7f8f3415e370 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c000f90 tx=0x7f8f2c00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f2b400640 1 -- 172.21.15.46:0/705638491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c003b90 con 0x7f8f340b3f40 2024-04-25T08:46:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f2b400640 1 -- 172.21.15.46:0/705638491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c003d50 con 0x7f8f340b3f40 2024-04-25T08:46:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f2b400640 1 -- 172.21.15.46:0/705638491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c006b60 con 0x7f8f340b3f40 2024-04-25T08:46:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f340b5ec0 con 0x7f8f340b3f40 2024-04-25T08:46:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f340b3460 con 0x7f8f340b3f40 2024-04-25T08:46:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.589+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f34069010 con 0x7f8f340b3f40 2024-04-25T08:46:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.593+0000 7f8f2b400640 1 -- 172.21.15.46:0/705638491 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f2c01a030 con 0x7f8f340b3f40 2024-04-25T08:46:43.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.593+0000 7f8f2b400640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f10066f90 0x7f8f10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.593+0000 7f8f2b400640 1 -- 172.21.15.46:0/705638491 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8f2c083fe0 con 0x7f8f340b3f40 2024-04-25T08:46:43.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.593+0000 7f8f33400640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f10066f90 0x7f8f10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.593+0000 7f8f33400640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f10066f90 0x7f8f10069450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8f240af000 tx=0x7f8f240aa000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.597+0000 7f8f2b400640 1 -- 172.21.15.46:0/705638491 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f2c021060 con 0x7f8f340b3f40 2024-04-25T08:46:43.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.765+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f3409a6d0 con 0x7f8f340b3f40 2024-04-25T08:46:43.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.765+0000 7f8f2b400640 1 -- 172.21.15.46:0/705638491 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f2c055200 con 0x7f8f340b3f40 2024-04-25T08:46:43.760 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:46:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f10066f90 msgr2=0x7f8f10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f10066f90 0x7f8f10069450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8f240af000 tx=0x7f8f240aa000 comp rx=0 tx=0).stop 2024-04-25T08:46:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f340b3f40 msgr2=0x7f8f3415e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f340b3f40 0x7f8f3415e370 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c000f90 tx=0x7f8f2c00ead0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 shutdown_connections 2024-04-25T08:46:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f10066f90 0x7f8f10069450 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f340b3f40 0x7f8f3415e370 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3415c380 0x7f8f340b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 --2- 172.21.15.46:0/705638491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f180039f0 0x7f8f341f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 >> 172.21.15.46:0/705638491 conn(0x7f8f340b2fb0 msgr2=0x7f8f3406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:43.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 shutdown_connections 2024-04-25T08:46:43.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.769+0000 7f8f38c00640 1 -- 172.21.15.46:0/705638491 wait complete. 2024-04-25T08:46:43.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:46:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3ae6200640 1 Processor -- start 2024-04-25T08:46:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3ae6200640 1 -- start start 2024-04-25T08:46:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3ae6200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ae015dca0 0x7f3ae00d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3ae6200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ae00afd40 0x7f3ae00d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3ae6200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae006ed70 con 0x7f3ae00cf140 2024-04-25T08:46:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3ae6200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae00afb40 con 0x7f3ae00afd40 2024-04-25T08:46:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3ae6200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae0162530 con 0x7f3ae015dca0 2024-04-25T08:46:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3adfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ae00afd40 0x7f3ae00d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3adfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3ae00afd40 0x7f3ae00d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54328/0 (socket says 172.21.15.46:54328) 2024-04-25T08:46:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3adfe00640 1 -- 172.21.15.46:0/2227187987 learned_addr learned my addr 172.21.15.46:0/2227187987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3adfe00640 1 -- 172.21.15.46:0/2227187987 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ae015dca0 msgr2=0x7f3ae00d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3adfe00640 1 --2- 172.21.15.46:0/2227187987 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ae015dca0 0x7f3ae00d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3adfe00640 1 -- 172.21.15.46:0/2227187987 >> v1:172.21.15.82:6789/0 conn(0x7f3ae00cf140 legacy=0x7f3ae015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.929+0000 7f3adfe00640 1 -- 172.21.15.46:0/2227187987 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0161770 con 0x7f3ae00afd40 2024-04-25T08:46:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3adfe00640 1 --2- 172.21.15.46:0/2227187987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae00afd40 0x7f3ae00d0920 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009e50 tx=0x7f3ad801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80ac38cf5311bda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3adea00640 1 -- 172.21.15.46:0/2227187987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80a5600 con 0x7f3ae00afd40 2024-04-25T08:46:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3adea00640 1 -- 172.21.15.46:0/2227187987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad80a57a0 con 0x7f3ae00afd40 2024-04-25T08:46:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3adea00640 1 -- 172.21.15.46:0/2227187987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80a5b70 con 0x7f3ae00afd40 2024-04-25T08:46:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3ae6200640 1 -- 172.21.15.46:0/2227187987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae00afd40 msgr2=0x7f3ae00d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3ae6200640 1 --2- 172.21.15.46:0/2227187987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae00afd40 0x7f3ae00d0920 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009e50 tx=0x7f3ad801c780 comp rx=0 tx=0).stop 2024-04-25T08:46:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3ae6200640 1 -- 172.21.15.46:0/2227187987 shutdown_connections 2024-04-25T08:46:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3ae6200640 1 --2- 172.21.15.46:0/2227187987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae00afd40 0x7f3ae00d0920 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3ae6200640 1 --2- 172.21.15.46:0/2227187987 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ae015dca0 0x7f3ae00d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3ae6200640 1 -- 172.21.15.46:0/2227187987 >> 172.21.15.46:0/2227187987 conn(0x7f3ae00728c0 msgr2=0x7f3ae0099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3ae6200640 1 -- 172.21.15.46:0/2227187987 shutdown_connections 2024-04-25T08:46:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.933+0000 7f3ae6200640 1 -- 172.21.15.46:0/2227187987 wait complete. 2024-04-25T08:46:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 Processor -- start 2024-04-25T08:46:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 -- start start 2024-04-25T08:46:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae00afd40 0x7f3ae006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ae00cf140 0x7f3ae014bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae015dca0 0x7f3ae014c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00d1270 con 0x7f3ae015dca0 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00af7f0 con 0x7f3ae00afd40 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00d13f0 con 0x7f3ae00cf140 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae015dca0 0x7f3ae014c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae015dca0 0x7f3ae014c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54338/0 (socket says 172.21.15.46:54338) 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adfe00640 1 -- 172.21.15.46:0/2661218668 learned_addr learned my addr 172.21.15.46:0/2661218668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adf400640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae00afd40 0x7f3ae006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ad7e00640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ae00cf140 0x7f3ae014bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adfe00640 1 -- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ae00cf140 msgr2=0x7f3ae014bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adfe00640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ae00cf140 0x7f3ae014bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adfe00640 1 -- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae00afd40 msgr2=0x7f3ae006f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adfe00640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae00afd40 0x7f3ae006f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adfe00640 1 -- 172.21.15.46:0/2661218668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae014d8b0 con 0x7f3ae015dca0 2024-04-25T08:46:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3adfe00640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae015dca0 0x7f3ae014c4e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009e50 tx=0x7f3ad8019520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ad7400640 1 -- 172.21.15.46:0/2661218668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80bc020 con 0x7f3ae015dca0 2024-04-25T08:46:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ad7400640 1 -- 172.21.15.46:0/2661218668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad80b8670 con 0x7f3ae015dca0 2024-04-25T08:46:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ad7400640 1 -- 172.21.15.46:0/2661218668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80b8a60 con 0x7f3ae015dca0 2024-04-25T08:46:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae014db90 con 0x7f3ae015dca0 2024-04-25T08:46:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.937+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae014de70 con 0x7f3ae015dca0 2024-04-25T08:46:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.941+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae006ccb0 con 0x7f3ae015dca0 2024-04-25T08:46:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.941+0000 7f3ad7400640 1 -- 172.21.15.46:0/2661218668 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ad80bf020 con 0x7f3ae015dca0 2024-04-25T08:46:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.941+0000 7f3ad7400640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3abc066f60 0x7f3abc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.941+0000 7f3ad7400640 1 -- 172.21.15.46:0/2661218668 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3ad811fbd0 con 0x7f3ae015dca0 2024-04-25T08:46:43.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.941+0000 7f3adf400640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3abc066f60 0x7f3abc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:43.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.945+0000 7f3adf400640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3abc066f60 0x7f3abc069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0007fd0 tx=0x7f3ad0007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:43.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:43.945+0000 7f3ad7400640 1 -- 172.21.15.46:0/2661218668 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ad80c6030 con 0x7f3ae015dca0 2024-04-25T08:46:44.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3ae0069080 con 0x7f3ae015dca0 2024-04-25T08:46:44.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ad7400640 1 -- 172.21.15.46:0/2661218668 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v74) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3ae006ed70 con 0x7f3ae015dca0 2024-04-25T08:46:44.102 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:46:44.102 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:46:44.102 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-25T08:46:44.102 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-25T08:46:44.103 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:46:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:46:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3abc066f60 msgr2=0x7f3abc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3abc066f60 0x7f3abc069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0007fd0 tx=0x7f3ad0007e40 comp rx=0 tx=0).stop 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae015dca0 msgr2=0x7f3ae014c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae015dca0 0x7f3ae014c4e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009e50 tx=0x7f3ad8019520 comp rx=0 tx=0).stop 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 shutdown_connections 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3abc066f60 0x7f3abc069420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ae015dca0 0x7f3ae014c4e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3ae00cf140 0x7f3ae014bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 --2- 172.21.15.46:0/2661218668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ae00afd40 0x7f3ae006f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 >> 172.21.15.46:0/2661218668 conn(0x7f3ae00728c0 msgr2=0x7f3ae015b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 shutdown_connections 2024-04-25T08:46:44.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.109+0000 7f3ae6200640 1 -- 172.21.15.46:0/2661218668 wait complete. 2024-04-25T08:46:44.125 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:46:44.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:46:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5488400640 1 Processor -- start 2024-04-25T08:46:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5488400640 1 -- start start 2024-04-25T08:46:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5488400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54800d11e0 0x7f548006c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5488400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5480099a00 0x7f5480099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5488400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548015e910 con 0x7f5480161720 2024-04-25T08:46:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5488400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54800712b0 con 0x7f5480099a00 2024-04-25T08:46:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5488400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548006fe10 con 0x7f54800d11e0 2024-04-25T08:46:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5485000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5480099a00 0x7f5480099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5486400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5480161720 0x7f5480161b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59376/0 (socket says 172.21.15.46:59376) 2024-04-25T08:46:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5486400640 1 -- 172.21.15.46:0/2425533633 learned_addr learned my addr 172.21.15.46:0/2425533633 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:44.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5485a00640 1 --2- 172.21.15.46:0/2425533633 >> v2:172.21.15.82:3300/0 conn(0x7f54800d11e0 0x7f548006c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5485000640 1 -- 172.21.15.46:0/2425533633 >> v2:172.21.15.82:3300/0 conn(0x7f54800d11e0 msgr2=0x7f548006c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:44.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.277+0000 7f5485000640 1 --2- 172.21.15.46:0/2425533633 >> v2:172.21.15.82:3300/0 conn(0x7f54800d11e0 0x7f548006c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5485000640 1 -- 172.21.15.46:0/2425533633 >> v1:172.21.15.46:6789/0 conn(0x7f5480161720 legacy=0x7f5480161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5485000640 1 -- 172.21.15.46:0/2425533633 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548015ebf0 con 0x7f5480099a00 2024-04-25T08:46:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5485000640 1 --2- 172.21.15.46:0/2425533633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5480099a00 0x7f5480099e80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5480072ab0 tx=0x7f546c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db90bf8a89ffc3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f547fe00640 1 -- 172.21.15.46:0/2425533633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0b1070 con 0x7f5480099a00 2024-04-25T08:46:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f547fe00640 1 -- 172.21.15.46:0/2425533633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c0ac640 con 0x7f5480099a00 2024-04-25T08:46:44.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f547fe00640 1 -- 172.21.15.46:0/2425533633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0aca10 con 0x7f5480099a00 2024-04-25T08:46:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5488400640 1 -- 172.21.15.46:0/2425533633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5480099a00 msgr2=0x7f5480099e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5488400640 1 --2- 172.21.15.46:0/2425533633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5480099a00 0x7f5480099e80 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5480072ab0 tx=0x7f546c01c480 comp rx=0 tx=0).stop 2024-04-25T08:46:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5488400640 1 -- 172.21.15.46:0/2425533633 shutdown_connections 2024-04-25T08:46:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5488400640 1 --2- 172.21.15.46:0/2425533633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5480099a00 0x7f5480099e80 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5488400640 1 --2- 172.21.15.46:0/2425533633 >> v2:172.21.15.82:3300/0 conn(0x7f54800d11e0 0x7f548006c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5488400640 1 -- 172.21.15.46:0/2425533633 >> 172.21.15.46:0/2425533633 conn(0x7f54800d16c0 msgr2=0x7f54800d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5488400640 1 -- 172.21.15.46:0/2425533633 shutdown_connections 2024-04-25T08:46:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.281+0000 7f5488400640 1 -- 172.21.15.46:0/2425533633 wait complete. 2024-04-25T08:46:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 Processor -- start 2024-04-25T08:46:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 -- start start 2024-04-25T08:46:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5480099a00 0x7f5480160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54800d11e0 0x7f5480143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480161720 0x7f5480155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548015dcc0 con 0x7f54800d11e0 2024-04-25T08:46:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548015db40 con 0x7f5480161720 2024-04-25T08:46:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548015de40 con 0x7f5480099a00 2024-04-25T08:46:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54800d11e0 0x7f5480143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54800d11e0 0x7f5480143c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54362/0 (socket says 172.21.15.46:54362) 2024-04-25T08:46:44.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485000640 1 -- 172.21.15.46:0/1458957372 learned_addr learned my addr 172.21.15.46:0/1458957372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485a00640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5480099a00 0x7f5480160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5486400640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480161720 0x7f5480155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485000640 1 -- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5480099a00 msgr2=0x7f5480160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485000640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5480099a00 0x7f5480160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485000640 1 -- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480161720 msgr2=0x7f5480155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485000640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480161720 0x7f5480155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485000640 1 -- 172.21.15.46:0/1458957372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480156b10 con 0x7f54800d11e0 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5486400640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480161720 0x7f5480155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5485000640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54800d11e0 0x7f5480143c10 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f546c01a0b0 tx=0x7f546c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f547d600640 1 -- 172.21.15.46:0/1458957372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0b1040 con 0x7f54800d11e0 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f547d600640 1 -- 172.21.15.46:0/1458957372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c0ba070 con 0x7f54800d11e0 2024-04-25T08:46:44.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f547d600640 1 -- 172.21.15.46:0/1458957372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0b5650 con 0x7f54800d11e0 2024-04-25T08:46:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5480144150 con 0x7f54800d11e0 2024-04-25T08:46:44.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.285+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548015f8b0 con 0x7f54800d11e0 2024-04-25T08:46:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.289+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5480069080 con 0x7f54800d11e0 2024-04-25T08:46:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.289+0000 7f547d600640 1 -- 172.21.15.46:0/1458957372 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f546c0b57f0 con 0x7f54800d11e0 2024-04-25T08:46:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.289+0000 7f547d600640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5460066f90 0x7f5460069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.289+0000 7f547d600640 1 -- 172.21.15.46:0/1458957372 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f546c11ec60 con 0x7f54800d11e0 2024-04-25T08:46:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.289+0000 7f5485a00640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5460066f90 0x7f5460069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.289+0000 7f5485a00640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5460066f90 0x7f5460069450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5474009b60 tx=0x7f5474009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:44.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.293+0000 7f547d600640 1 -- 172.21.15.46:0/1458957372 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5480144150 con 0x7f54800d11e0 2024-04-25T08:46:44.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.457+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f54800d24c0 con 0x7f54800d11e0 2024-04-25T08:46:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f547d600640 1 -- 172.21.15.46:0/1458957372 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v75) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5480156b10 con 0x7f54800d11e0 2024-04-25T08:46:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5460066f90 msgr2=0x7f5460069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5460066f90 0x7f5460069450 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5474009b60 tx=0x7f5474009190 comp rx=0 tx=0).stop 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54800d11e0 msgr2=0x7f5480143c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54800d11e0 0x7f5480143c10 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f546c01a0b0 tx=0x7f546c019220 comp rx=0 tx=0).stop 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 shutdown_connections 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5460066f90 0x7f5460069450 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480161720 0x7f5480155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54800d11e0 0x7f5480143c10 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 --2- 172.21.15.46:0/1458957372 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5480099a00 0x7f5480160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 >> 172.21.15.46:0/1458957372 conn(0x7f54800d16c0 msgr2=0x7f54800b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 shutdown_connections 2024-04-25T08:46:44.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.669+0000 7f5488400640 1 -- 172.21.15.46:0/1458957372 wait complete. 2024-04-25T08:46:44.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:46:44.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.841+0000 7f6749600640 1 Processor -- start 2024-04-25T08:46:44.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.841+0000 7f6749600640 1 -- start start 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.841+0000 7f6749600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 0x7f674406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.841+0000 7f6749600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f674415d090 0x7f674415d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.841+0000 7f6749600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67440cd930 con 0x7f674406dc70 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.841+0000 7f6749600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67440cdbd0 con 0x7f674415d090 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.841+0000 7f6749600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67440cdeb0 con 0x7f6744145310 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.841+0000 7f6743400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f674415d090 0x7f674415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6743400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f674415d090 0x7f674415d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54378/0 (socket says 172.21.15.46:54378) 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6743400640 1 -- 172.21.15.46:0/146654349 learned_addr learned my addr 172.21.15.46:0/146654349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.841+0000 7f673a000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f674406dc70 0x7f67440cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59380/0 (socket says 172.21.15.46:59380) 2024-04-25T08:46:44.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6742a00640 1 --2- 172.21.15.46:0/146654349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 0x7f674406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6742000640 1 -- 172.21.15.46:0/146654349 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491247942 0 0) 0x7f67440cd930 con 0x7f674406dc70 2024-04-25T08:46:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6742000640 1 -- 172.21.15.46:0/146654349 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6728003530 con 0x7f674406dc70 2024-04-25T08:46:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6743400640 1 -- 172.21.15.46:0/146654349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 msgr2=0x7f674406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6743400640 1 --2- 172.21.15.46:0/146654349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 0x7f674406e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6743400640 1 -- 172.21.15.46:0/146654349 >> v1:172.21.15.46:6789/0 conn(0x7f674406dc70 legacy=0x7f67440cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6743400640 1 -- 172.21.15.46:0/146654349 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440ce190 con 0x7f674415d090 2024-04-25T08:46:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6743400640 1 --2- 172.21.15.46:0/146654349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 0x7f674415d540 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f673c009870 tx=0x7f673c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd6b3f8922be712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6742000640 1 -- 172.21.15.46:0/146654349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0ab650 con 0x7f674415d090 2024-04-25T08:46:44.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6742000640 1 -- 172.21.15.46:0/146654349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c0b0070 con 0x7f674415d090 2024-04-25T08:46:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.845+0000 7f6742000640 1 -- 172.21.15.46:0/146654349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0aba10 con 0x7f674415d090 2024-04-25T08:46:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 -- 172.21.15.46:0/146654349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 msgr2=0x7f674415d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 --2- 172.21.15.46:0/146654349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 0x7f674415d540 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f673c009870 tx=0x7f673c01c560 comp rx=0 tx=0).stop 2024-04-25T08:46:44.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 -- 172.21.15.46:0/146654349 shutdown_connections 2024-04-25T08:46:44.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 --2- 172.21.15.46:0/146654349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 0x7f674415d540 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 --2- 172.21.15.46:0/146654349 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 0x7f674406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 -- 172.21.15.46:0/146654349 >> 172.21.15.46:0/146654349 conn(0x7f674406ac10 msgr2=0x7f674406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:44.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 -- 172.21.15.46:0/146654349 shutdown_connections 2024-04-25T08:46:44.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 -- 172.21.15.46:0/146654349 wait complete. 2024-04-25T08:46:44.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 Processor -- start 2024-04-25T08:46:44.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 -- start start 2024-04-25T08:46:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 0x7f6744159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 0x7f674413f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674413f620 0x7f674413ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744168510 con 0x7f674415d090 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67440cf380 con 0x7f674413f620 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6749600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6744168690 con 0x7f6744145310 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f673a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 0x7f674413f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f673a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 0x7f674413f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54384/0 (socket says 172.21.15.46:54384) 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f673a000640 1 -- 172.21.15.46:0/2348951806 learned_addr learned my addr 172.21.15.46:0/2348951806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6743400640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674413f620 0x7f674413ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6742a00640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 0x7f6744159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f673a000640 1 -- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 msgr2=0x7f6744159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f673a000640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 0x7f6744159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f673a000640 1 -- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674413f620 msgr2=0x7f674413ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f673a000640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674413f620 0x7f674413ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f673a000640 1 -- 172.21.15.46:0/2348951806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440ae880 con 0x7f674415d090 2024-04-25T08:46:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f6742a00640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 0x7f6744159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:44.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.849+0000 7f673a000640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 0x7f674413f0e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c460 tx=0x7f672c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.853+0000 7f673b400640 1 -- 172.21.15.46:0/2348951806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c018070 con 0x7f674415d090 2024-04-25T08:46:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.853+0000 7f673b400640 1 -- 172.21.15.46:0/2348951806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c0074a0 con 0x7f674415d090 2024-04-25T08:46:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.853+0000 7f673b400640 1 -- 172.21.15.46:0/2348951806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c004a30 con 0x7f674415d090 2024-04-25T08:46:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.853+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674413da30 con 0x7f674415d090 2024-04-25T08:46:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.853+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674413dff0 con 0x7f674415d090 2024-04-25T08:46:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.853+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674413b3c0 con 0x7f674415d090 2024-04-25T08:46:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.853+0000 7f673b400640 1 -- 172.21.15.46:0/2348951806 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f672c01b070 con 0x7f674415d090 2024-04-25T08:46:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.853+0000 7f673b400640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6720066f90 0x7f6720069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.853+0000 7f673b400640 1 -- 172.21.15.46:0/2348951806 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f672c084200 con 0x7f674415d090 2024-04-25T08:46:44.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.857+0000 7f6742a00640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6720066f90 0x7f6720069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:44.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.857+0000 7f6742a00640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6720066f90 0x7f6720069450 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f67340074c0 tx=0x7f673400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:44.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:44.861+0000 7f673b400640 1 -- 172.21.15.46:0/2348951806 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f672c054b70 con 0x7f674415d090 2024-04-25T08:46:45.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.021+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f674415d6f0 con 0x7f674415d090 2024-04-25T08:46:45.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.669+0000 7f673b400640 1 -- 172.21.15.46:0/2348951806 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) v1 ==== 170+0+0 (secure 0 0 0) 0x7f672c058c10 con 0x7f674415d090 2024-04-25T08:46:45.665 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-25T08:46:45.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6720066f90 msgr2=0x7f6720069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6720066f90 0x7f6720069450 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f67340074c0 tx=0x7f673400b040 comp rx=0 tx=0).stop 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 msgr2=0x7f674413f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 0x7f674413f0e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c460 tx=0x7f672c00c920 comp rx=0 tx=0).stop 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 shutdown_connections 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6720066f90 0x7f6720069450 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f674413f620 0x7f674413ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f674415d090 0x7f674413f0e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 --2- 172.21.15.46:0/2348951806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6744145310 0x7f6744159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 >> 172.21.15.46:0/2348951806 conn(0x7f674406ac10 msgr2=0x7f67440b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 shutdown_connections 2024-04-25T08:46:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.673+0000 7f6749600640 1 -- 172.21.15.46:0/2348951806 wait complete. 2024-04-25T08:46:45.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:46:45.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b9a600640 1 Processor -- start 2024-04-25T08:46:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b9a600640 1 -- start start 2024-04-25T08:46:45.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b9a600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b9415d900 0x7f1b940af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b9a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b940afc20 0x7f1b9406ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b9a600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b940b0040 con 0x7f1b9406b2a0 2024-04-25T08:46:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b9a600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9406b6c0 con 0x7f1b9415d900 2024-04-25T08:46:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b9a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9406fca0 con 0x7f1b940afc20 2024-04-25T08:46:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b93400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b9415d900 0x7f1b940af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b92a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b940afc20 0x7f1b9406ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b93400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b9415d900 0x7f1b940af6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:52470/0 (socket says 172.21.15.46:52470) 2024-04-25T08:46:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b92a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b940afc20 0x7f1b9406ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:44468/0 (socket says 172.21.15.46:44468) 2024-04-25T08:46:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b93400640 1 -- 172.21.15.46:0/2485820492 learned_addr learned my addr 172.21.15.46:0/2485820492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b93400640 1 -- 172.21.15.46:0/2485820492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b940afc20 msgr2=0x7f1b9406ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b93400640 1 --2- 172.21.15.46:0/2485820492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b940afc20 0x7f1b9406ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b93400640 1 -- 172.21.15.46:0/2485820492 >> v1:172.21.15.82:6789/0 conn(0x7f1b9406b2a0 legacy=0x7f1b9415dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.861+0000 7f1b93400640 1 -- 172.21.15.46:0/2485820492 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9406ff80 con 0x7f1b9415d900 2024-04-25T08:46:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b93400640 1 --2- 172.21.15.46:0/2485820492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b9415d900 0x7f1b940af6e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009870 tx=0x7f1b8401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a901066e60bdcdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b92000640 1 -- 172.21.15.46:0/2485820492 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840b1070 con 0x7f1b9415d900 2024-04-25T08:46:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b92000640 1 -- 172.21.15.46:0/2485820492 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b840ac620 con 0x7f1b9415d900 2024-04-25T08:46:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b92000640 1 -- 172.21.15.46:0/2485820492 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b840ac9f0 con 0x7f1b9415d900 2024-04-25T08:46:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b9a600640 1 -- 172.21.15.46:0/2485820492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b9415d900 msgr2=0x7f1b940af6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b9a600640 1 --2- 172.21.15.46:0/2485820492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b9415d900 0x7f1b940af6e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009870 tx=0x7f1b8401c4d0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b9a600640 1 -- 172.21.15.46:0/2485820492 shutdown_connections 2024-04-25T08:46:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b9a600640 1 --2- 172.21.15.46:0/2485820492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b940afc20 0x7f1b9406ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b9a600640 1 --2- 172.21.15.46:0/2485820492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b9415d900 0x7f1b940af6e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b9a600640 1 -- 172.21.15.46:0/2485820492 >> 172.21.15.46:0/2485820492 conn(0x7f1b9406e3e0 msgr2=0x7f1b9406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b9a600640 1 -- 172.21.15.46:0/2485820492 shutdown_connections 2024-04-25T08:46:45.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.865+0000 7f1b9a600640 1 -- 172.21.15.46:0/2485820492 wait complete. 2024-04-25T08:46:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 Processor -- start 2024-04-25T08:46:45.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 -- start start 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b9406b2a0 0x7f1b9413eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940afc20 0x7f1b94150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b9415d900 0x7f1b940b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940d13a0 con 0x7f1b9415d900 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940d1220 con 0x7f1b940afc20 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9416a900 con 0x7f1b9406b2a0 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b9415d900 0x7f1b940b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b9415d900 0x7f1b940b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54392/0 (socket says 172.21.15.46:54392) 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93e00640 1 -- 172.21.15.46:0/343307105 learned_addr learned my addr 172.21.15.46:0/343307105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b92a00640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940afc20 0x7f1b94150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93400640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b9406b2a0 0x7f1b9413eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93e00640 1 -- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b9406b2a0 msgr2=0x7f1b9413eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93e00640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b9406b2a0 0x7f1b9413eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93e00640 1 -- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940afc20 msgr2=0x7f1b94150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93e00640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940afc20 0x7f1b94150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93e00640 1 -- 172.21.15.46:0/343307105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b94151010 con 0x7f1b9415d900 2024-04-25T08:46:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b92a00640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940afc20 0x7f1b94150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b93e00640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b9415d900 0x7f1b940b23e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c730 tx=0x7f1b8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b8b400640 1 -- 172.21.15.46:0/343307105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c003b80 con 0x7f1b9415d900 2024-04-25T08:46:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b8b400640 1 -- 172.21.15.46:0/343307105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c003d20 con 0x7f1b9415d900 2024-04-25T08:46:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b941512f0 con 0x7f1b9415d900 2024-04-25T08:46:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b8b400640 1 -- 172.21.15.46:0/343307105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c00aa30 con 0x7f1b9415d900 2024-04-25T08:46:45.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.869+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b941628b0 con 0x7f1b9415d900 2024-04-25T08:46:45.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.873+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b94069010 con 0x7f1b9415d900 2024-04-25T08:46:45.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.873+0000 7f1b8b400640 1 -- 172.21.15.46:0/343307105 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b8c00abd0 con 0x7f1b9415d900 2024-04-25T08:46:45.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.873+0000 7f1b8b400640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b74066f60 0x7f1b74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:45.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.873+0000 7f1b8b400640 1 -- 172.21.15.46:0/343307105 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f1b8c085ad0 con 0x7f1b9415d900 2024-04-25T08:46:45.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.873+0000 7f1b93400640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b74066f60 0x7f1b74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.877+0000 7f1b93400640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b74066f60 0x7f1b74069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1b840ae000 tx=0x7f1b84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:45.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:45.877+0000 7f1b8b400640 1 -- 172.21.15.46:0/343307105 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b8c056a20 con 0x7f1b9415d900 2024-04-25T08:46:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.073+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1b94072890 con 0x7f1b9415d900 2024-04-25T08:46:46.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.701+0000 7f1b8b400640 1 -- 172.21.15.46:0/343307105 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v41) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1b8c056320 con 0x7f1b9415d900 2024-04-25T08:46:46.699 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T08:46:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.717+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b74066f60 msgr2=0x7f1b74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.717+0000 7f1b9a600640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b74066f60 0x7f1b74069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1b840ae000 tx=0x7f1b84008040 comp rx=0 tx=0).stop 2024-04-25T08:46:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.717+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b9415d900 msgr2=0x7f1b940b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:46.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.717+0000 7f1b9a600640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b9415d900 0x7f1b940b23e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c730 tx=0x7f1b8c00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.721+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 shutdown_connections 2024-04-25T08:46:46.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.721+0000 7f1b9a600640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1b74066f60 0x7f1b74069420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.721+0000 7f1b9a600640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b9415d900 0x7f1b940b23e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.721+0000 7f1b9a600640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940afc20 0x7f1b94150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.721+0000 7f1b9a600640 1 --2- 172.21.15.46:0/343307105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1b9406b2a0 0x7f1b9413eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.721+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 >> 172.21.15.46:0/343307105 conn(0x7f1b9406e3e0 msgr2=0x7f1b941609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.721+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 shutdown_connections 2024-04-25T08:46:46.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.721+0000 7f1b9a600640 1 -- 172.21.15.46:0/343307105 wait complete. 2024-04-25T08:46:46.741 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:46.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.889+0000 7f7895800640 1 Processor -- start 2024-04-25T08:46:46.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.889+0000 7f7895800640 1 -- start start 2024-04-25T08:46:46.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f7895800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7890159230 0x7f789015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:46.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f7895800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78900cbeb0 0x7f78900cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:46.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f7895800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78900cc700 con 0x7f78900cda80 2024-04-25T08:46:46.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f7895800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78900cc9e0 con 0x7f78900cbeb0 2024-04-25T08:46:46.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f7895800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78900cccc0 con 0x7f7890159230 2024-04-25T08:46:46.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f788f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f78900cda80 0x7f789015f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59388/0 (socket says 172.21.15.46:59388) 2024-04-25T08:46:46.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f788f400640 1 -- 172.21.15.46:0/3133550144 learned_addr learned my addr 172.21.15.46:0/3133550144 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:46.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f788ea00640 1 --2- 172.21.15.46:0/3133550144 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7890159230 0x7f789015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:46.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f7886000640 1 --2- 172.21.15.46:0/3133550144 >> v2:172.21.15.82:3300/0 conn(0x7f78900cbeb0 0x7f78900cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:46.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f788e000640 1 -- 172.21.15.46:0/3133550144 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271782016 0 0) 0x7f78900cc700 con 0x7f78900cda80 2024-04-25T08:46:46.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f788e000640 1 -- 172.21.15.46:0/3133550144 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7874003530 con 0x7f78900cda80 2024-04-25T08:46:46.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f788ea00640 1 -- 172.21.15.46:0/3133550144 >> v2:172.21.15.82:3300/0 conn(0x7f78900cbeb0 msgr2=0x7f78900cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:46.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f788ea00640 1 --2- 172.21.15.46:0/3133550144 >> v2:172.21.15.82:3300/0 conn(0x7f78900cbeb0 0x7f78900cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f788ea00640 1 -- 172.21.15.46:0/3133550144 >> v1:172.21.15.46:6789/0 conn(0x7f78900cda80 legacy=0x7f789015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:46.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f788ea00640 1 -- 172.21.15.46:0/3133550144 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78900ccfa0 con 0x7f7890159230 2024-04-25T08:46:46.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.893+0000 7f7886000640 1 --2- 172.21.15.46:0/3133550144 >> v2:172.21.15.82:3300/0 conn(0x7f78900cbeb0 0x7f78900cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:46:46.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f788ea00640 1 --2- 172.21.15.46:0/3133550144 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7890159230 0x7f789015bdb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78800097c0 tx=0x7f788001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f406f3fe19046fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:46.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f788e000640 1 -- 172.21.15.46:0/3133550144 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800b0070 con 0x7f7890159230 2024-04-25T08:46:46.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f788e000640 1 -- 172.21.15.46:0/3133550144 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78800ab600 con 0x7f7890159230 2024-04-25T08:46:46.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f788e000640 1 -- 172.21.15.46:0/3133550144 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800ab9d0 con 0x7f7890159230 2024-04-25T08:46:46.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 -- 172.21.15.46:0/3133550144 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7890159230 msgr2=0x7f789015bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:46.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 --2- 172.21.15.46:0/3133550144 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7890159230 0x7f789015bdb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f78800097c0 tx=0x7f788001c410 comp rx=0 tx=0).stop 2024-04-25T08:46:46.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 -- 172.21.15.46:0/3133550144 shutdown_connections 2024-04-25T08:46:46.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 --2- 172.21.15.46:0/3133550144 >> v2:172.21.15.82:3300/0 conn(0x7f78900cbeb0 0x7f78900cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 --2- 172.21.15.46:0/3133550144 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7890159230 0x7f789015bdb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 -- 172.21.15.46:0/3133550144 >> 172.21.15.46:0/3133550144 conn(0x7f78900b12a0 msgr2=0x7f78900b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:46.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 -- 172.21.15.46:0/3133550144 shutdown_connections 2024-04-25T08:46:46.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 -- 172.21.15.46:0/3133550144 wait complete. 2024-04-25T08:46:46.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 Processor -- start 2024-04-25T08:46:46.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 -- start start 2024-04-25T08:46:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78900cbeb0 0x7f789006ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78900cda80 0x7f789013feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890159230 0x7f789015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7890168780 con 0x7f78900cda80 2024-04-25T08:46:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78900cc4a0 con 0x7f7890159230 2024-04-25T08:46:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7895800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7890168900 con 0x7f78900cbeb0 2024-04-25T08:46:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7886000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78900cda80 0x7f789013feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7886000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78900cda80 0x7f789013feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54398/0 (socket says 172.21.15.46:54398) 2024-04-25T08:46:46.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.897+0000 7f7886000640 1 -- 172.21.15.46:0/4090090491 learned_addr learned my addr 172.21.15.46:0/4090090491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f788f400640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890159230 0x7f789015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f788ea00640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78900cbeb0 0x7f789006ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7886000640 1 -- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78900cbeb0 msgr2=0x7f789006ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7886000640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78900cbeb0 0x7f789006ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7886000640 1 -- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890159230 msgr2=0x7f789015a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7886000640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890159230 0x7f789015a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7886000640 1 -- 172.21.15.46:0/4090090491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78901403f0 con 0x7f78900cda80 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f788f400640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890159230 0x7f789015a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7886000640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78900cda80 0x7f789013feb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f787800e9b0 tx=0x7f787800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7887400640 1 -- 172.21.15.46:0/4090090491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878017070 con 0x7f78900cda80 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7887400640 1 -- 172.21.15.46:0/4090090491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787800ca00 con 0x7f78900cda80 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7887400640 1 -- 172.21.15.46:0/4090090491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787800cdd0 con 0x7f78900cda80 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789006d9e0 con 0x7f78900cda80 2024-04-25T08:46:46.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789006dfa0 con 0x7f78900cda80 2024-04-25T08:46:46.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7890069080 con 0x7f78900cda80 2024-04-25T08:46:46.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7887400640 1 -- 172.21.15.46:0/4090090491 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f787801a070 con 0x7f78900cda80 2024-04-25T08:46:46.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7887400640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f786c066f90 0x7f786c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:46.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.901+0000 7f7887400640 1 -- 172.21.15.46:0/4090090491 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7878085110 con 0x7f78900cda80 2024-04-25T08:46:46.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.905+0000 7f788ea00640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f786c066f90 0x7f786c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:46.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.905+0000 7f788ea00640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f786c066f90 0x7f786c069450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f78800ad000 tx=0x7f78800a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:46.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:46.905+0000 7f7887400640 1 -- 172.21.15.46:0/4090090491 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7878055ef0 con 0x7f78900cda80 2024-04-25T08:46:47.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.069+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78900723d0 con 0x7f78900cda80 2024-04-25T08:46:47.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.069+0000 7f7887400640 1 -- 172.21.15.46:0/4090090491 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16910 (secure 0 0 0) 0x7f78780557f0 con 0x7f78900cda80 2024-04-25T08:46:47.064 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:47.064 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:46.077371+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:46:44.505856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:46:45.072095+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f786c066f90 msgr2=0x7f786c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f786c066f90 0x7f786c069450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f78800ad000 tx=0x7f78800a6000 comp rx=0 tx=0).stop 2024-04-25T08:46:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78900cda80 msgr2=0x7f789013feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78900cda80 0x7f789013feb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f787800e9b0 tx=0x7f787800ee70 comp rx=0 tx=0).stop 2024-04-25T08:46:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 shutdown_connections 2024-04-25T08:46:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f786c066f90 0x7f786c069450 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890159230 0x7f789015a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78900cda80 0x7f789013feb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 --2- 172.21.15.46:0/4090090491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f78900cbeb0 0x7f789006ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 >> 172.21.15.46:0/4090090491 conn(0x7f78900b12a0 msgr2=0x7f78900af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 shutdown_connections 2024-04-25T08:46:47.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.073+0000 7f7895800640 1 -- 172.21.15.46:0/4090090491 wait complete. 2024-04-25T08:46:47.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:47.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f871ac00640 1 Processor -- start 2024-04-25T08:46:47.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f871ac00640 1 -- start start 2024-04-25T08:46:47.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f871ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 0x7f871406c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f871ac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f87140cdd00 0x7f87140ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f871ac00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87140cf6d0 con 0x7f8714099a00 2024-04-25T08:46:47.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f871ac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87140cf4a0 con 0x7f87140cdd00 2024-04-25T08:46:47.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f871ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87140b2100 con 0x7f871415d7a0 2024-04-25T08:46:47.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f8718e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f87140cdd00 0x7f87140ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f871a200640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8714099a00 0x7f8714099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:36308/0 (socket says 172.21.15.46:36308) 2024-04-25T08:46:47.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f871a200640 1 -- 172.21.15.46:0/2562242951 learned_addr learned my addr 172.21.15.46:0/2562242951 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:47.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f8719800640 1 --2- 172.21.15.46:0/2562242951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 0x7f871406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f8713e00640 1 -- 172.21.15.46:0/2562242951 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777953562 0 0) 0x7f87140cf6d0 con 0x7f8714099a00 2024-04-25T08:46:47.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f8713e00640 1 -- 172.21.15.46:0/2562242951 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc003530 con 0x7f8714099a00 2024-04-25T08:46:47.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f8719800640 1 -- 172.21.15.46:0/2562242951 >> v2:172.21.15.82:3300/0 conn(0x7f87140cdd00 msgr2=0x7f87140ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:47.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f8719800640 1 --2- 172.21.15.46:0/2562242951 >> v2:172.21.15.82:3300/0 conn(0x7f87140cdd00 0x7f87140ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.233+0000 7f8719800640 1 -- 172.21.15.46:0/2562242951 >> v1:172.21.15.82:6789/0 conn(0x7f8714099a00 legacy=0x7f8714099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f8719800640 1 -- 172.21.15.46:0/2562242951 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87140cf9b0 con 0x7f871415d7a0 2024-04-25T08:46:47.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f8719800640 1 --2- 172.21.15.46:0/2562242951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 0x7f871406c730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f87080097c0 tx=0x7f870801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2975d0fcc6004c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:47.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f8713e00640 1 -- 172.21.15.46:0/2562242951 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87080b2070 con 0x7f871415d7a0 2024-04-25T08:46:47.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f8713e00640 1 -- 172.21.15.46:0/2562242951 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87080ad640 con 0x7f871415d7a0 2024-04-25T08:46:47.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f8713e00640 1 -- 172.21.15.46:0/2562242951 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87080ada10 con 0x7f871415d7a0 2024-04-25T08:46:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f871ac00640 1 -- 172.21.15.46:0/2562242951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 msgr2=0x7f871406c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f871ac00640 1 --2- 172.21.15.46:0/2562242951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 0x7f871406c730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f87080097c0 tx=0x7f870801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f871ac00640 1 -- 172.21.15.46:0/2562242951 shutdown_connections 2024-04-25T08:46:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f871ac00640 1 --2- 172.21.15.46:0/2562242951 >> v2:172.21.15.82:3300/0 conn(0x7f87140cdd00 0x7f87140ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f871ac00640 1 --2- 172.21.15.46:0/2562242951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 0x7f871406c730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f871ac00640 1 -- 172.21.15.46:0/2562242951 >> 172.21.15.46:0/2562242951 conn(0x7f87140d1480 msgr2=0x7f87140d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f871ac00640 1 -- 172.21.15.46:0/2562242951 shutdown_connections 2024-04-25T08:46:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.237+0000 7f871ac00640 1 -- 172.21.15.46:0/2562242951 wait complete. 2024-04-25T08:46:47.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 Processor -- start 2024-04-25T08:46:47.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 -- start start 2024-04-25T08:46:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8714099a00 0x7f8714152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87140cdd00 0x7f87141505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 0x7f8714157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87140d0770 con 0x7f8714099a00 2024-04-25T08:46:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871415db80 con 0x7f87140cdd00 2024-04-25T08:46:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8714070f50 con 0x7f871415d7a0 2024-04-25T08:46:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8719800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8714099a00 0x7f8714152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8719800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8714099a00 0x7f8714152b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54406/0 (socket says 172.21.15.46:54406) 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8719800640 1 -- 172.21.15.46:0/548470973 learned_addr learned my addr 172.21.15.46:0/548470973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8718e00640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87140cdd00 0x7f87141505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871a200640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 0x7f8714157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8719800640 1 -- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 msgr2=0x7f8714157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8719800640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 0x7f8714157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8719800640 1 -- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87140cdd00 msgr2=0x7f87141505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8719800640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87140cdd00 0x7f87141505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8719800640 1 -- 172.21.15.46:0/548470973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87140703d0 con 0x7f8714099a00 2024-04-25T08:46:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8719800640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8714099a00 0x7f8714152b90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f870801ab50 tx=0x7f8708019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8711600640 1 -- 172.21.15.46:0/548470973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87080bc070 con 0x7f8714099a00 2024-04-25T08:46:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87140706b0 con 0x7f8714099a00 2024-04-25T08:46:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8711600640 1 -- 172.21.15.46:0/548470973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87080b7460 con 0x7f8714099a00 2024-04-25T08:46:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f8711600640 1 -- 172.21.15.46:0/548470973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87080b2040 con 0x7f8714099a00 2024-04-25T08:46:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.241+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8714070c70 con 0x7f8714099a00 2024-04-25T08:46:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.245+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8714069010 con 0x7f8714099a00 2024-04-25T08:46:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.245+0000 7f8711600640 1 -- 172.21.15.46:0/548470973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87080bf070 con 0x7f8714099a00 2024-04-25T08:46:47.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.245+0000 7f8711600640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86f4066f60 0x7f86f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.245+0000 7f8711600640 1 -- 172.21.15.46:0/548470973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f870811f7e0 con 0x7f8714099a00 2024-04-25T08:46:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.245+0000 7f8718e00640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86f4066f60 0x7f86f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.249+0000 7f8718e00640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86f4066f60 0x7f86f4069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8714159280 tx=0x7f8700009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.249+0000 7f8711600640 1 -- 172.21.15.46:0/548470973 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8708016090 con 0x7f8714099a00 2024-04-25T08:46:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.417+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87140d1260 con 0x7f8714099a00 2024-04-25T08:46:47.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.417+0000 7f8711600640 1 -- 172.21.15.46:0/548470973 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16910 (secure 0 0 0) 0x7f87080f0140 con 0x7f8714099a00 2024-04-25T08:46:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:47.411 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:46.077371+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:46:44.505856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:46:45.072095+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86f4066f60 msgr2=0x7f86f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86f4066f60 0x7f86f4069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8714159280 tx=0x7f8700009190 comp rx=0 tx=0).stop 2024-04-25T08:46:47.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8714099a00 msgr2=0x7f8714152b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8714099a00 0x7f8714152b90 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f870801ab50 tx=0x7f8708019240 comp rx=0 tx=0).stop 2024-04-25T08:46:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 shutdown_connections 2024-04-25T08:46:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86f4066f60 0x7f86f4069420 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f871415d7a0 0x7f8714157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87140cdd00 0x7f87141505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 --2- 172.21.15.46:0/548470973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8714099a00 0x7f8714152b90 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 >> 172.21.15.46:0/548470973 conn(0x7f87140d1480 msgr2=0x7f871406c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 shutdown_connections 2024-04-25T08:46:47.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.421+0000 7f871ac00640 1 -- 172.21.15.46:0/548470973 wait complete. 2024-04-25T08:46:47.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:46:47.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.581+0000 7f9223800640 1 Processor -- start 2024-04-25T08:46:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.581+0000 7f9223800640 1 -- start start 2024-04-25T08:46:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9223800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f921c143990 0x7f921c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9223800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921c15e440 con 0x7f921c06f150 2024-04-25T08:46:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9223800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921c15e6e0 con 0x7f921c1440d0 2024-04-25T08:46:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9223800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921c15ed30 con 0x7f921c143990 2024-04-25T08:46:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9221800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f921c06f150 0x7f921c0b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59394/0 (socket says 172.21.15.46:59394) 2024-04-25T08:46:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9221800640 1 -- 172.21.15.46:0/3953125841 learned_addr learned my addr 172.21.15.46:0/3953125841 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9220e00640 1 --2- 172.21.15.46:0/3953125841 >> v2:172.21.15.82:3300/0 conn(0x7f921c143990 0x7f921c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f921be00640 1 -- 172.21.15.46:0/3953125841 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307016973 0 0) 0x7f921c15e440 con 0x7f921c06f150 2024-04-25T08:46:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f921be00640 1 -- 172.21.15.46:0/3953125841 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9200003530 con 0x7f921c06f150 2024-04-25T08:46:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f921be00640 1 -- 172.21.15.46:0/3953125841 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445260110 0 0) 0x7f921c15e6e0 con 0x7f921c1440d0 2024-04-25T08:46:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f921be00640 1 -- 172.21.15.46:0/3953125841 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f921c15e440 con 0x7f921c1440d0 2024-04-25T08:46:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9220e00640 1 -- 172.21.15.46:0/3953125841 >> v1:172.21.15.82:6789/0 conn(0x7f921c1440d0 legacy=0x7f921c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9220e00640 1 -- 172.21.15.46:0/3953125841 >> v1:172.21.15.46:6789/0 conn(0x7f921c06f150 legacy=0x7f921c0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.585+0000 7f9220e00640 1 -- 172.21.15.46:0/3953125841 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c15e9c0 con 0x7f921c143990 2024-04-25T08:46:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9220e00640 1 --2- 172.21.15.46:0/3953125841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f921c143990 0x7f921c0cecc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f921c15eeb0 tx=0x7f920c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43309c3f4c388cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f921be00640 1 -- 172.21.15.46:0/3953125841 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b0070 con 0x7f921c143990 2024-04-25T08:46:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f921be00640 1 -- 172.21.15.46:0/3953125841 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c0ab600 con 0x7f921c143990 2024-04-25T08:46:47.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f921be00640 1 -- 172.21.15.46:0/3953125841 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0ab9d0 con 0x7f921c143990 2024-04-25T08:46:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9223800640 1 -- 172.21.15.46:0/3953125841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f921c143990 msgr2=0x7f921c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9223800640 1 --2- 172.21.15.46:0/3953125841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f921c143990 0x7f921c0cecc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f921c15eeb0 tx=0x7f920c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9223800640 1 -- 172.21.15.46:0/3953125841 shutdown_connections 2024-04-25T08:46:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9223800640 1 --2- 172.21.15.46:0/3953125841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f921c143990 0x7f921c0cecc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9223800640 1 -- 172.21.15.46:0/3953125841 >> 172.21.15.46:0/3953125841 conn(0x7f921c06ac10 msgr2=0x7f921c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9223800640 1 -- 172.21.15.46:0/3953125841 shutdown_connections 2024-04-25T08:46:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9223800640 1 -- 172.21.15.46:0/3953125841 wait complete. 2024-04-25T08:46:47.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9223800640 1 Processor -- start 2024-04-25T08:46:47.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.589+0000 7f9223800640 1 -- start start 2024-04-25T08:46:47.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9223800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f921c06f150 0x7f921c15a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9223800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921c143990 0x7f921c141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9223800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921c1440d0 0x7f921c0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9223800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c0713f0 con 0x7f921c143990 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9223800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c071270 con 0x7f921c06f150 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9223800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c071570 con 0x7f921c1440d0 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9213e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921c143990 0x7f921c141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9213e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921c143990 0x7f921c141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54418/0 (socket says 172.21.15.46:54418) 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9213e00640 1 -- 172.21.15.46:0/3600538823 learned_addr learned my addr 172.21.15.46:0/3600538823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9220e00640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f921c06f150 0x7f921c15a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9221800640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921c1440d0 0x7f921c0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9213e00640 1 -- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921c1440d0 msgr2=0x7f921c0aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9213e00640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921c1440d0 0x7f921c0aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9213e00640 1 -- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f921c06f150 msgr2=0x7f921c15a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9213e00640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f921c06f150 0x7f921c15a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:47.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9213e00640 1 -- 172.21.15.46:0/3600538823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c1425e0 con 0x7f921c143990 2024-04-25T08:46:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9213e00640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921c143990 0x7f921c141ab0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f920400c430 tx=0x7f920400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9219600640 1 -- 172.21.15.46:0/3600538823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204017070 con 0x7f921c143990 2024-04-25T08:46:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9219600640 1 -- 172.21.15.46:0/3600538823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92040052f0 con 0x7f921c143990 2024-04-25T08:46:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9219600640 1 -- 172.21.15.46:0/3600538823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204003840 con 0x7f921c143990 2024-04-25T08:46:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c0af4a0 con 0x7f921c143990 2024-04-25T08:46:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c0af740 con 0x7f921c143990 2024-04-25T08:46:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.593+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921c069010 con 0x7f921c143990 2024-04-25T08:46:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.597+0000 7f9219600640 1 -- 172.21.15.46:0/3600538823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f920401a070 con 0x7f921c143990 2024-04-25T08:46:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.597+0000 7f9219600640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91f8066f90 0x7f91f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.597+0000 7f9219600640 1 -- 172.21.15.46:0/3600538823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9204084510 con 0x7f921c143990 2024-04-25T08:46:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.597+0000 7f9220e00640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91f8066f90 0x7f91f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.597+0000 7f9220e00640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91f8066f90 0x7f91f8069450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ad000 tx=0x7f920c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.601+0000 7f9219600640 1 -- 172.21.15.46:0/3600538823 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9204054d30 con 0x7f921c143990 2024-04-25T08:46:47.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:47.801+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f921c13b4d0 con 0x7f921c143990 2024-04-25T08:46:48.663 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:46:48.669+0000 7f590c600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:46:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.709+0000 7f9219600640 1 -- 172.21.15.46:0/3600538823 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) v1 ==== 118+0+0 (secure 0 0 0) 0x7f920405b080 con 0x7f921c143990 2024-04-25T08:46:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91f8066f90 msgr2=0x7f91f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91f8066f90 0x7f91f8069450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ad000 tx=0x7f920c0a6000 comp rx=0 tx=0).stop 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921c143990 msgr2=0x7f921c141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921c143990 0x7f921c141ab0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f920400c430 tx=0x7f920400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 shutdown_connections 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f91f8066f90 0x7f91f8069450 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f921c1440d0 0x7f921c0aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f921c143990 0x7f921c141ab0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 --2- 172.21.15.46:0/3600538823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f921c06f150 0x7f921c15a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 >> 172.21.15.46:0/3600538823 conn(0x7f921c06ac10 msgr2=0x7f921c0ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 shutdown_connections 2024-04-25T08:46:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.713+0000 7f9223800640 1 -- 172.21.15.46:0/3600538823 wait complete. 2024-04-25T08:46:48.731 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:46:48.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:46:48.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe32a00640 1 Processor -- start 2024-04-25T08:46:48.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe32a00640 1 -- start start 2024-04-25T08:46:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe32a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbe2c06f4d0 0x7fbe2c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe32a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2c09a780 con 0x7fbe2c15dd50 2024-04-25T08:46:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe32a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2c09aa60 con 0x7fbe2c0d0450 2024-04-25T08:46:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe32a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe2c0708c0 con 0x7fbe2c06f4d0 2024-04-25T08:46:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe2be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbe2c06f4d0 0x7fbe2c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe23400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbe2c15dd50 0x7fbe2c0cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34890/0 (socket says 172.21.15.46:34890) 2024-04-25T08:46:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe23400640 1 -- 172.21.15.46:0/2766603238 learned_addr learned my addr 172.21.15.46:0/2766603238 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe2be00640 1 -- 172.21.15.46:0/2766603238 >> v1:172.21.15.82:6789/0 conn(0x7fbe2c0d0450 legacy=0x7fbe2c0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe2be00640 1 -- 172.21.15.46:0/2766603238 >> v1:172.21.15.46:6789/0 conn(0x7fbe2c15dd50 legacy=0x7fbe2c0cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe2b400640 1 -- 172.21.15.46:0/2766603238 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382314322 0 0) 0x7fbe2c09a780 con 0x7fbe2c15dd50 2024-04-25T08:46:48.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.885+0000 7fbe2be00640 1 -- 172.21.15.46:0/2766603238 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c09ad40 con 0x7fbe2c06f4d0 2024-04-25T08:46:48.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe2be00640 1 --2- 172.21.15.46:0/2766603238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c06f4d0 0x7fbe2c0cf800 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0097c0 tx=0x7fbe1c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34675573869986f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:48.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe2b400640 1 -- 172.21.15.46:0/2766603238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c0ab430 con 0x7fbe2c06f4d0 2024-04-25T08:46:48.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe2b400640 1 -- 172.21.15.46:0/2766603238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1c0b0070 con 0x7fbe2c06f4d0 2024-04-25T08:46:48.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe2b400640 1 -- 172.21.15.46:0/2766603238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1c0ab7f0 con 0x7fbe2c06f4d0 2024-04-25T08:46:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe32a00640 1 -- 172.21.15.46:0/2766603238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c06f4d0 msgr2=0x7fbe2c0cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe32a00640 1 --2- 172.21.15.46:0/2766603238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c06f4d0 0x7fbe2c0cf800 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0097c0 tx=0x7fbe1c01c430 comp rx=0 tx=0).stop 2024-04-25T08:46:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe32a00640 1 -- 172.21.15.46:0/2766603238 shutdown_connections 2024-04-25T08:46:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe32a00640 1 --2- 172.21.15.46:0/2766603238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c06f4d0 0x7fbe2c0cf800 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe32a00640 1 -- 172.21.15.46:0/2766603238 >> 172.21.15.46:0/2766603238 conn(0x7fbe2c0b3160 msgr2=0x7fbe2c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe32a00640 1 -- 172.21.15.46:0/2766603238 shutdown_connections 2024-04-25T08:46:48.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.889+0000 7fbe32a00640 1 -- 172.21.15.46:0/2766603238 wait complete. 2024-04-25T08:46:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 Processor -- start 2024-04-25T08:46:48.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 -- start start 2024-04-25T08:46:48.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe2c06f4d0 0x7fbe2c15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:48.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c0d0450 0x7fbe2c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:48.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15dd50 0x7fbe2c0ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:48.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c0cd560 con 0x7fbe2c0d0450 2024-04-25T08:46:48.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c0cd3e0 con 0x7fbe2c15dd50 2024-04-25T08:46:48.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c0cd6e0 con 0x7fbe2c06f4d0 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe23400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c0d0450 0x7fbe2c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe23400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c0d0450 0x7fbe2c159b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39680/0 (socket says 172.21.15.46:39680) 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe23400640 1 -- 172.21.15.46:0/1649259346 learned_addr learned my addr 172.21.15.46:0/1649259346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe2be00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe2c06f4d0 0x7fbe2c15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe30a00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15dd50 0x7fbe2c0ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe23400640 1 -- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe2c06f4d0 msgr2=0x7fbe2c15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe23400640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe2c06f4d0 0x7fbe2c15c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe23400640 1 -- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15dd50 msgr2=0x7fbe2c0ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe23400640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15dd50 0x7fbe2c0ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:48.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe23400640 1 -- 172.21.15.46:0/1649259346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c15b750 con 0x7fbe2c0d0450 2024-04-25T08:46:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe23400640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c0d0450 0x7fbe2c159b10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c480 tx=0x7fbe1400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe28c00640 1 -- 172.21.15.46:0/1649259346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14018070 con 0x7fbe2c0d0450 2024-04-25T08:46:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe28c00640 1 -- 172.21.15.46:0/1649259346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe14007500 con 0x7fbe2c0d0450 2024-04-25T08:46:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe28c00640 1 -- 172.21.15.46:0/1649259346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14004a30 con 0x7fbe2c0d0450 2024-04-25T08:46:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2c15ba30 con 0x7fbe2c0d0450 2024-04-25T08:46:48.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c15bff0 con 0x7fbe2c0d0450 2024-04-25T08:46:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.893+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2c069010 con 0x7fbe2c0d0450 2024-04-25T08:46:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.897+0000 7fbe28c00640 1 -- 172.21.15.46:0/1649259346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe14004bd0 con 0x7fbe2c0d0450 2024-04-25T08:46:48.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.897+0000 7fbe28c00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbe08066f60 0x7fbe08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.897+0000 7fbe28c00640 1 -- 172.21.15.46:0/1649259346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe14084400 con 0x7fbe2c0d0450 2024-04-25T08:46:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.897+0000 7fbe2be00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbe08066f60 0x7fbe08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:48.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.897+0000 7fbe2be00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbe08066f60 0x7fbe08069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0af000 tx=0x7fbe1c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:48.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:48.901+0000 7fbe28c00640 1 -- 172.21.15.46:0/1649259346 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe14054c20 con 0x7fbe2c0d0450 2024-04-25T08:46:49.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.097+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbe2c0737e0 con 0x7fbe2c0d0450 2024-04-25T08:46:49.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.713+0000 7fbe28c00640 1 -- 172.21.15.46:0/1649259346 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbe14058cc0 con 0x7fbe2c0d0450 2024-04-25T08:46:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbe08066f60 msgr2=0x7fbe08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbe08066f60 0x7fbe08069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0af000 tx=0x7fbe1c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:46:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c0d0450 msgr2=0x7fbe2c159b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c0d0450 0x7fbe2c159b10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbe1400c480 tx=0x7fbe1400c940 comp rx=0 tx=0).stop 2024-04-25T08:46:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 shutdown_connections 2024-04-25T08:46:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbe08066f60 0x7fbe08069420 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe2c15dd50 0x7fbe2c0ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:49.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbe2c0d0450 0x7fbe2c159b10 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 --2- 172.21.15.46:0/1649259346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbe2c06f4d0 0x7fbe2c15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 >> 172.21.15.46:0/1649259346 conn(0x7fbe2c0b3160 msgr2=0x7fbe2c099ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 shutdown_connections 2024-04-25T08:46:49.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.717+0000 7fbe32a00640 1 -- 172.21.15.46:0/1649259346 wait complete. 2024-04-25T08:46:49.733 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:46:49.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:49.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e69400640 1 Processor -- start 2024-04-25T08:46:49.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e69400640 1 -- start start 2024-04-25T08:46:49.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e69400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e6415d7a0 0x7f3e6406c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:49.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e69400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640ce1b0 0x7f3e6409a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:49.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e69400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e640ce600 con 0x7f3e640cdd00 2024-04-25T08:46:49.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e69400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e640cf4a0 con 0x7f3e6415d7a0 2024-04-25T08:46:49.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e69400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e640b2100 con 0x7f3e640ce1b0 2024-04-25T08:46:49.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e63e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e6415d7a0 0x7f3e6406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:49.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e63e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e6415d7a0 0x7f3e6406c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39686/0 (socket says 172.21.15.46:39686) 2024-04-25T08:46:49.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e63e00640 1 -- 172.21.15.46:0/904084902 learned_addr learned my addr 172.21.15.46:0/904084902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:49.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e68a00640 1 --2- 172.21.15.46:0/904084902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640ce1b0 0x7f3e6409a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:49.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e63e00640 1 -- 172.21.15.46:0/904084902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640ce1b0 msgr2=0x7f3e6409a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:49.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e63e00640 1 --2- 172.21.15.46:0/904084902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640ce1b0 0x7f3e6409a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:49.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e63e00640 1 -- 172.21.15.46:0/904084902 >> v1:172.21.15.82:6789/0 conn(0x7f3e640cdd00 legacy=0x7f3e64099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:49.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e63e00640 1 -- 172.21.15.46:0/904084902 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e640cf6d0 con 0x7f3e6415d7a0 2024-04-25T08:46:49.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.885+0000 7f3e68a00640 1 --2- 172.21.15.46:0/904084902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640ce1b0 0x7f3e6409a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:49.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e63e00640 1 --2- 172.21.15.46:0/904084902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6415d7a0 0x7f3e6406c730 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009870 tx=0x7f3e5401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f717c25895b8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e63400640 1 -- 172.21.15.46:0/904084902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540a95f0 con 0x7f3e6415d7a0 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e63400640 1 -- 172.21.15.46:0/904084902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540a9790 con 0x7f3e6415d7a0 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e63400640 1 -- 172.21.15.46:0/904084902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e540a9b60 con 0x7f3e6415d7a0 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 -- 172.21.15.46:0/904084902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6415d7a0 msgr2=0x7f3e6406c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 --2- 172.21.15.46:0/904084902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6415d7a0 0x7f3e6406c730 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009870 tx=0x7f3e5401c530 comp rx=0 tx=0).stop 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 -- 172.21.15.46:0/904084902 shutdown_connections 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 --2- 172.21.15.46:0/904084902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640ce1b0 0x7f3e6409a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 --2- 172.21.15.46:0/904084902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6415d7a0 0x7f3e6406c730 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 -- 172.21.15.46:0/904084902 >> 172.21.15.46:0/904084902 conn(0x7f3e640d1480 msgr2=0x7f3e640d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 -- 172.21.15.46:0/904084902 shutdown_connections 2024-04-25T08:46:49.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 -- 172.21.15.46:0/904084902 wait complete. 2024-04-25T08:46:49.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 Processor -- start 2024-04-25T08:46:49.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.889+0000 7f3e69400640 1 -- start start 2024-04-25T08:46:49.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e69400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640cdd00 0x7f3e6414d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e69400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e640ce1b0 0x7f3e6414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e69400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6415d7a0 0x7f3e640cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e69400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6415db80 con 0x7f3e640ce1b0 2024-04-25T08:46:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e69400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e640d0250 con 0x7f3e6415d7a0 2024-04-25T08:46:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e69400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e640d0770 con 0x7f3e640cdd00 2024-04-25T08:46:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e640ce1b0 0x7f3e6414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e640ce1b0 0x7f3e6414afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39692/0 (socket says 172.21.15.46:39692) 2024-04-25T08:46:49.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e5be00640 1 -- 172.21.15.46:0/812044714 learned_addr learned my addr 172.21.15.46:0/812044714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e63e00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640cdd00 0x7f3e6414d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e5be00640 1 -- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640cdd00 msgr2=0x7f3e6414d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e5be00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640cdd00 0x7f3e6414d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e5be00640 1 -- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6415d7a0 msgr2=0x7f3e640cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e68a00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6415d7a0 0x7f3e640cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e5be00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6415d7a0 0x7f3e640cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e5be00640 1 -- 172.21.15.46:0/812044714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6414df50 con 0x7f3e640ce1b0 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e63e00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640cdd00 0x7f3e6414d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e68a00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6415d7a0 0x7f3e640cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e5be00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e640ce1b0 0x7f3e6414afd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c450 tx=0x7f3e4c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:49.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e60c00640 1 -- 172.21.15.46:0/812044714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c016020 con 0x7f3e640ce1b0 2024-04-25T08:46:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e60c00640 1 -- 172.21.15.46:0/812044714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c0052d0 con 0x7f3e640ce1b0 2024-04-25T08:46:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e60c00640 1 -- 172.21.15.46:0/812044714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c007260 con 0x7f3e640ce1b0 2024-04-25T08:46:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6414e230 con 0x7f3e640ce1b0 2024-04-25T08:46:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6414c640 con 0x7f3e640ce1b0 2024-04-25T08:46:49.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e64069010 con 0x7f3e640ce1b0 2024-04-25T08:46:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.893+0000 7f3e60c00640 1 -- 172.21.15.46:0/812044714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e4c007400 con 0x7f3e640ce1b0 2024-04-25T08:46:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.897+0000 7f3e60c00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e40066f60 0x7f3e40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.897+0000 7f3e60c00640 1 -- 172.21.15.46:0/812044714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3e4c084ac0 con 0x7f3e640ce1b0 2024-04-25T08:46:49.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.897+0000 7f3e63e00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e40066f60 0x7f3e40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:49.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.897+0000 7f3e63e00640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e40066f60 0x7f3e40069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3e540ae000 tx=0x7f3e540a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:49.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:49.901+0000 7f3e60c00640 1 -- 172.21.15.46:0/812044714 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e4c020020 con 0x7f3e640ce1b0 2024-04-25T08:46:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e6415cfd0 con 0x7f3e640ce1b0 2024-04-25T08:46:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e60c00640 1 -- 172.21.15.46:0/812044714 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f3e4c005480 con 0x7f3e640ce1b0 2024-04-25T08:46:50.095 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:50.095 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:46:49.102068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6774},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6774":{"gid":6774,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/960745191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":960745191},{"type":"v1","addr":"172.21.15.46:6835","nonce":960745191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:46:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 45 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e40066f60 msgr2=0x7f3e40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e40066f60 0x7f3e40069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3e540ae000 tx=0x7f3e540a7000 comp rx=0 tx=0).stop 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e640ce1b0 msgr2=0x7f3e6414afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e640ce1b0 0x7f3e6414afd0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c450 tx=0x7f3e4c00c910 comp rx=0 tx=0).stop 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 shutdown_connections 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e40066f60 0x7f3e40069420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e6415d7a0 0x7f3e640cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e640ce1b0 0x7f3e6414afd0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 --2- 172.21.15.46:0/812044714 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e640cdd00 0x7f3e6414d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 >> 172.21.15.46:0/812044714 conn(0x7f3e640d1480 msgr2=0x7f3e640b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 shutdown_connections 2024-04-25T08:46:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.101+0000 7f3e69400640 1 -- 172.21.15.46:0/812044714 wait complete. 2024-04-25T08:46:50.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:50.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35aca00640 1 Processor -- start 2024-04-25T08:46:50.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35aca00640 1 -- start start 2024-04-25T08:46:50.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35aca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35a809a610 0x7f35a809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35aca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a815f300 con 0x7f35a815ef20 2024-04-25T08:46:50.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35aca00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a80cd8a0 con 0x7f35a8099a00 2024-04-25T08:46:50.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35aca00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a80cddb0 con 0x7f35a809a610 2024-04-25T08:46:50.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35a5e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f35a815ef20 0x7f35a80cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34892/0 (socket says 172.21.15.46:34892) 2024-04-25T08:46:50.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35a5e00640 1 -- 172.21.15.46:0/4090184083 learned_addr learned my addr 172.21.15.46:0/4090184083 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:50.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f359fe00640 1 --2- 172.21.15.46:0/4090184083 >> v2:172.21.15.82:3300/0 conn(0x7f35a809a610 0x7f35a809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35a5400640 1 -- 172.21.15.46:0/4090184083 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874057653 0 0) 0x7f35a815f300 con 0x7f35a815ef20 2024-04-25T08:46:50.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f35a5400640 1 -- 172.21.15.46:0/4090184083 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f358c003530 con 0x7f35a815ef20 2024-04-25T08:46:50.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f359fe00640 1 -- 172.21.15.46:0/4090184083 >> v1:172.21.15.82:6789/0 conn(0x7f35a8099a00 legacy=0x7f35a80c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f359fe00640 1 -- 172.21.15.46:0/4090184083 >> v1:172.21.15.46:6789/0 conn(0x7f35a815ef20 legacy=0x7f35a80cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.265+0000 7f359fe00640 1 -- 172.21.15.46:0/4090184083 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80cc3d0 con 0x7f35a809a610 2024-04-25T08:46:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f359fe00640 1 --2- 172.21.15.46:0/4090184083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35a809a610 0x7f35a809aa10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f35900097c0 tx=0x7f359001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83a7f5d4015adddf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35a5400640 1 -- 172.21.15.46:0/4090184083 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35900b1070 con 0x7f35a809a610 2024-04-25T08:46:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35a5400640 1 -- 172.21.15.46:0/4090184083 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35900ac5f0 con 0x7f35a809a610 2024-04-25T08:46:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35a5400640 1 -- 172.21.15.46:0/4090184083 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35900ac9c0 con 0x7f35a809a610 2024-04-25T08:46:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35aca00640 1 -- 172.21.15.46:0/4090184083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35a809a610 msgr2=0x7f35a809aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35aca00640 1 --2- 172.21.15.46:0/4090184083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35a809a610 0x7f35a809aa10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f35900097c0 tx=0x7f359001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35aca00640 1 -- 172.21.15.46:0/4090184083 shutdown_connections 2024-04-25T08:46:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35aca00640 1 --2- 172.21.15.46:0/4090184083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35a809a610 0x7f35a809aa10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35aca00640 1 -- 172.21.15.46:0/4090184083 >> 172.21.15.46:0/4090184083 conn(0x7f35a80b3120 msgr2=0x7f35a80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35aca00640 1 -- 172.21.15.46:0/4090184083 shutdown_connections 2024-04-25T08:46:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.269+0000 7f35aca00640 1 -- 172.21.15.46:0/4090184083 wait complete. 2024-04-25T08:46:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 Processor -- start 2024-04-25T08:46:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 -- start start 2024-04-25T08:46:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8099a00 0x7f35a80b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35a809a610 0x7f35a80b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35a815ef20 0x7f35a8152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a809a220 con 0x7f35a8099a00 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80cc010 con 0x7f35a809a610 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a80b4fd0 con 0x7f35a815ef20 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8099a00 0x7f35a80b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8099a00 0x7f35a80b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39698/0 (socket says 172.21.15.46:39698) 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a5e00640 1 -- 172.21.15.46:0/1776761300 learned_addr learned my addr 172.21.15.46:0/1776761300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f359fe00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35a809a610 0x7f35a80b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a6800640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35a815ef20 0x7f35a8152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a5e00640 1 -- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35a815ef20 msgr2=0x7f35a8152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a5e00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35a815ef20 0x7f35a8152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a5e00640 1 -- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35a809a610 msgr2=0x7f35a80b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a5e00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35a809a610 0x7f35a80b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a5e00640 1 -- 172.21.15.46:0/1776761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a80b5200 con 0x7f35a8099a00 2024-04-25T08:46:50.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35a5e00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8099a00 0x7f35a80b6a20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f359800c500 tx=0x7f359800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:50.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f359e000640 1 -- 172.21.15.46:0/1776761300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598017070 con 0x7f35a8099a00 2024-04-25T08:46:50.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f359e000640 1 -- 172.21.15.46:0/1776761300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598005cf0 con 0x7f35a8099a00 2024-04-25T08:46:50.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a80b54e0 con 0x7f35a8099a00 2024-04-25T08:46:50.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f359e000640 1 -- 172.21.15.46:0/1776761300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980072f0 con 0x7f35a8099a00 2024-04-25T08:46:50.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.273+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a8153130 con 0x7f35a8099a00 2024-04-25T08:46:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.277+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a8069080 con 0x7f35a8099a00 2024-04-25T08:46:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.277+0000 7f359e000640 1 -- 172.21.15.46:0/1776761300 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f359801a070 con 0x7f35a8099a00 2024-04-25T08:46:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.277+0000 7f359e000640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3584066f90 0x7f3584069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.277+0000 7f359e000640 1 -- 172.21.15.46:0/1776761300 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3598084540 con 0x7f35a8099a00 2024-04-25T08:46:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.277+0000 7f359fe00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3584066f90 0x7f3584069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.277+0000 7f359fe00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3584066f90 0x7f3584069450 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f35900ae000 tx=0x7f3590008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:50.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.281+0000 7f359e000640 1 -- 172.21.15.46:0/1776761300 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3598054d60 con 0x7f35a8099a00 2024-04-25T08:46:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.441+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35a80d0700 con 0x7f35a8099a00 2024-04-25T08:46:50.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f359e000640 1 -- 172.21.15.46:0/1776761300 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f3598058e00 con 0x7f35a8099a00 2024-04-25T08:46:50.438 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:50.439 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:47.688186+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:46:44.505856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:46:45.072095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3584066f90 msgr2=0x7f3584069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3584066f90 0x7f3584069450 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f35900ae000 tx=0x7f3590008040 comp rx=0 tx=0).stop 2024-04-25T08:46:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8099a00 msgr2=0x7f35a80b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8099a00 0x7f35a80b6a20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f359800c500 tx=0x7f359800c9c0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 shutdown_connections 2024-04-25T08:46:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3584066f90 0x7f3584069450 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35a815ef20 0x7f35a8152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35a809a610 0x7f35a80b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 --2- 172.21.15.46:0/1776761300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35a8099a00 0x7f35a80b6a20 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 >> 172.21.15.46:0/1776761300 conn(0x7f35a80b3120 msgr2=0x7f35a80c71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 shutdown_connections 2024-04-25T08:46:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.445+0000 7f35aca00640 1 -- 172.21.15.46:0/1776761300 wait complete. 2024-04-25T08:46:50.460 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:46:50.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:46:50.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:46:50.570 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:46:50.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.605+0000 7f8805800640 1 Processor -- start 2024-04-25T08:46:50.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8805800640 1 -- start start 2024-04-25T08:46:50.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8805800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 0x7f88000b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8805800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88000ce740 0x7f88000ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8805800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8800144230 con 0x7f8800143e50 2024-04-25T08:46:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8805800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88000cf0d0 con 0x7f88000ce740 2024-04-25T08:46:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8805800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88000b4640 con 0x7f88000b3af0 2024-04-25T08:46:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f87ffe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8800143e50 0x7f880015bd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34902/0 (socket says 172.21.15.46:34902) 2024-04-25T08:46:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f87ffe00640 1 -- 172.21.15.46:0/1017309223 learned_addr learned my addr 172.21.15.46:0/1017309223 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:50.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8804e00640 1 --2- 172.21.15.46:0/1017309223 >> v2:172.21.15.46:3300/0 conn(0x7f88000ce740 0x7f88000ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f87ff400640 1 --2- 172.21.15.46:0/1017309223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 0x7f88000b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f87fea00640 1 -- 172.21.15.46:0/1017309223 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428830965 0 0) 0x7f8800144230 con 0x7f8800143e50 2024-04-25T08:46:50.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f87fea00640 1 -- 172.21.15.46:0/1017309223 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87e4003530 con 0x7f8800143e50 2024-04-25T08:46:50.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8804e00640 1 -- 172.21.15.46:0/1017309223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 msgr2=0x7f88000b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:50.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8804e00640 1 --2- 172.21.15.46:0/1017309223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 0x7f88000b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8804e00640 1 -- 172.21.15.46:0/1017309223 >> v1:172.21.15.46:6789/0 conn(0x7f8800143e50 legacy=0x7f880015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f8804e00640 1 -- 172.21.15.46:0/1017309223 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880006d240 con 0x7f88000ce740 2024-04-25T08:46:50.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.609+0000 7f87ff400640 1 --2- 172.21.15.46:0/1017309223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 0x7f88000b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:46:50.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8804e00640 1 --2- 172.21.15.46:0/1017309223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 0x7f88000ceb90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009d90 tx=0x7f87f801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db11ddf177d15c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:50.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f87fea00640 1 -- 172.21.15.46:0/1017309223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80b2070 con 0x7f88000ce740 2024-04-25T08:46:50.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f87fea00640 1 -- 172.21.15.46:0/1017309223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f80a57a0 con 0x7f88000ce740 2024-04-25T08:46:50.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f87fea00640 1 -- 172.21.15.46:0/1017309223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80a5b70 con 0x7f88000ce740 2024-04-25T08:46:50.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 -- 172.21.15.46:0/1017309223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 msgr2=0x7f88000ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 --2- 172.21.15.46:0/1017309223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 0x7f88000ceb90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009d90 tx=0x7f87f801c8e0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 -- 172.21.15.46:0/1017309223 shutdown_connections 2024-04-25T08:46:50.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 --2- 172.21.15.46:0/1017309223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 0x7f88000ceb90 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 --2- 172.21.15.46:0/1017309223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 0x7f88000b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 -- 172.21.15.46:0/1017309223 >> 172.21.15.46:0/1017309223 conn(0x7f8800066fa0 msgr2=0x7f880006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:50.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 -- 172.21.15.46:0/1017309223 shutdown_connections 2024-04-25T08:46:50.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 -- 172.21.15.46:0/1017309223 wait complete. 2024-04-25T08:46:50.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 Processor -- start 2024-04-25T08:46:50.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.613+0000 7f8805800640 1 -- start start 2024-04-25T08:46:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8805800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 0x7f8800164240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8805800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 0x7f8800159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8805800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8800143e50 0x7f88000a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8805800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880006e740 con 0x7f88000ce740 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8805800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88001671b0 con 0x7f8800143e50 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8805800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88000aa4a0 con 0x7f88000b3af0 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 0x7f8800159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 0x7f8800159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39722/0 (socket says 172.21.15.46:39722) 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ff400640 1 -- 172.21.15.46:0/1992127491 learned_addr learned my addr 172.21.15.46:0/1992127491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ffe00640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 0x7f8800164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8804e00640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8800143e50 0x7f88000a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ff400640 1 -- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 msgr2=0x7f8800164240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ff400640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 0x7f8800164240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ff400640 1 -- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8800143e50 msgr2=0x7f88000a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ff400640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8800143e50 0x7f88000a8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ff400640 1 -- 172.21.15.46:0/1992127491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88000a92b0 con 0x7f88000ce740 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87ff400640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 0x7f8800159f50 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00c7f0 tx=0x7f87ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87f7e00640 1 -- 172.21.15.46:0/1992127491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec007a30 con 0x7f88000ce740 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87f7e00640 1 -- 172.21.15.46:0/1992127491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec007bd0 con 0x7f88000ce740 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87f7e00640 1 -- 172.21.15.46:0/1992127491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0113f0 con 0x7f88000ce740 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88000a9f20 con 0x7f88000ce740 2024-04-25T08:46:50.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88000a8570 con 0x7f88000ce740 2024-04-25T08:46:50.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8800069010 con 0x7f88000ce740 2024-04-25T08:46:50.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.617+0000 7f87f7e00640 1 -- 172.21.15.46:0/1992127491 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87ec011590 con 0x7f88000ce740 2024-04-25T08:46:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.621+0000 7f87f7e00640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87e0066f60 0x7f87e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.621+0000 7f87f7e00640 1 -- 172.21.15.46:0/1992127491 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87ec085460 con 0x7f88000ce740 2024-04-25T08:46:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.621+0000 7f87ffe00640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87e0066f60 0x7f87e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.621+0000 7f87ffe00640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87e0066f60 0x7f87e0069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f87f0005890 tx=0x7f87f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:50.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.625+0000 7f87f7e00640 1 -- 172.21.15.46:0/1992127491 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87ec015020 con 0x7f88000ce740 2024-04-25T08:46:50.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.785+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88000b0710 con 0x7f87e0066f60 2024-04-25T08:46:50.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.793+0000 7f87f7e00640 1 -- 172.21.15.46:0/1992127491 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252006 (secure 0 0 0) 0x7f88000b0710 con 0x7f87e0066f60 2024-04-25T08:46:50.789 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:50.791 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:46:50.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87e0066f60 msgr2=0x7f87e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87e0066f60 0x7f87e0069420 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f87f0005890 tx=0x7f87f0009040 comp rx=0 tx=0).stop 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 msgr2=0x7f8800159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 0x7f8800159f50 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00c7f0 tx=0x7f87ec00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 shutdown_connections 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87e0066f60 0x7f87e0069420 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8800143e50 0x7f88000a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88000ce740 0x7f8800159f50 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 --2- 172.21.15.46:0/1992127491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f88000b3af0 0x7f8800164240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 >> 172.21.15.46:0/1992127491 conn(0x7f8800066fa0 msgr2=0x7f8800142460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 shutdown_connections 2024-04-25T08:46:50.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:50.797+0000 7f8805800640 1 -- 172.21.15.46:0/1992127491 wait complete. 2024-04-25T08:46:50.809 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-04-25T08:46:50.321980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245528,"kb_used_data":8540,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729192,"statfs":{"total":773094113280,"available":772842692608,"internally_reserved":0,"allocated":8744960,"data_stored":4606786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.648898"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716805+0000","last_change":"2024-04-25T08:46:46.711267+0000","last_active":"2024-04-25T08:46:46.716805+0000","last_peered":"2024-04-25T08:46:46.716805+0000","last_clean":"2024-04-25T08:46:46.716805+0000","last_became_active":"2024-04-25T08:46:46.711006+0000","last_became_peered":"2024-04-25T08:46:46.711006+0000","last_unstale":"2024-04-25T08:46:46.716805+0000","last_undegraded":"2024-04-25T08:46:46.716805+0000","last_fullsized":"2024-04-25T08:46:46.716805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:39.465212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.722045+0000","last_change":"2024-04-25T08:46:46.720962+0000","last_active":"2024-04-25T08:46:46.722045+0000","last_peered":"2024-04-25T08:46:46.722045+0000","last_clean":"2024-04-25T08:46:46.722045+0000","last_became_active":"2024-04-25T08:46:46.720631+0000","last_became_peered":"2024-04-25T08:46:46.720631+0000","last_unstale":"2024-04-25T08:46:46.722045+0000","last_undegraded":"2024-04-25T08:46:46.722045+0000","last_fullsized":"2024-04-25T08:46:46.722045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:32.376940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.723795+0000","last_change":"2024-04-25T08:46:46.711904+0000","last_active":"2024-04-25T08:46:46.723795+0000","last_peered":"2024-04-25T08:46:46.723795+0000","last_clean":"2024-04-25T08:46:46.723795+0000","last_became_active":"2024-04-25T08:46:46.711541+0000","last_became_peered":"2024-04-25T08:46:46.711541+0000","last_unstale":"2024-04-25T08:46:46.723795+0000","last_undegraded":"2024-04-25T08:46:46.723795+0000","last_fullsized":"2024-04-25T08:46:46.723795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:57.505207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717431+0000","last_change":"2024-04-25T08:46:46.711372+0000","last_active":"2024-04-25T08:46:46.717431+0000","last_peered":"2024-04-25T08:46:46.717431+0000","last_clean":"2024-04-25T08:46:46.717431+0000","last_became_active":"2024-04-25T08:46:46.711086+0000","last_became_peered":"2024-04-25T08:46:46.711086+0000","last_unstale":"2024-04-25T08:46:46.717431+0000","last_undegraded":"2024-04-25T08:46:46.717431+0000","last_fullsized":"2024-04-25T08:46:46.717431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:33.858904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.728234+0000","last_change":"2024-04-25T08:46:46.716059+0000","last_active":"2024-04-25T08:46:46.728234+0000","last_peered":"2024-04-25T08:46:46.728234+0000","last_clean":"2024-04-25T08:46:46.728234+0000","last_became_active":"2024-04-25T08:46:46.715574+0000","last_became_peered":"2024-04-25T08:46:46.715574+0000","last_unstale":"2024-04-25T08:46:46.728234+0000","last_undegraded":"2024-04-25T08:46:46.728234+0000","last_fullsized":"2024-04-25T08:46:46.728234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:02.024539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.712270+0000","last_change":"2024-04-25T08:46:46.709523+0000","last_active":"2024-04-25T08:46:46.712270+0000","last_peered":"2024-04-25T08:46:46.712270+0000","last_clean":"2024-04-25T08:46:46.712270+0000","last_became_active":"2024-04-25T08:46:46.709225+0000","last_became_peered":"2024-04-25T08:46:46.709225+0000","last_unstale":"2024-04-25T08:46:46.712270+0000","last_undegraded":"2024-04-25T08:46:46.712270+0000","last_fullsized":"2024-04-25T08:46:46.712270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:32.376940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717577+0000","last_change":"2024-04-25T08:46:46.711107+0000","last_active":"2024-04-25T08:46:46.717577+0000","last_peered":"2024-04-25T08:46:46.717577+0000","last_clean":"2024-04-25T08:46:46.717577+0000","last_became_active":"2024-04-25T08:46:46.709053+0000","last_became_peered":"2024-04-25T08:46:46.709053+0000","last_unstale":"2024-04-25T08:46:46.717577+0000","last_undegraded":"2024-04-25T08:46:46.717577+0000","last_fullsized":"2024-04-25T08:46:46.717577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:36.030392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.724356+0000","last_change":"2024-04-25T08:46:46.711798+0000","last_active":"2024-04-25T08:46:46.724356+0000","last_peered":"2024-04-25T08:46:46.724356+0000","last_clean":"2024-04-25T08:46:46.724356+0000","last_became_active":"2024-04-25T08:46:46.711519+0000","last_became_peered":"2024-04-25T08:46:46.711519+0000","last_unstale":"2024-04-25T08:46:46.724356+0000","last_undegraded":"2024-04-25T08:46:46.724356+0000","last_fullsized":"2024-04-25T08:46:46.724356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:29.106610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716109+0000","last_change":"2024-04-25T08:46:46.711387+0000","last_active":"2024-04-25T08:46:46.716109+0000","last_peered":"2024-04-25T08:46:46.716109+0000","last_clean":"2024-04-25T08:46:46.716109+0000","last_became_active":"2024-04-25T08:46:46.710510+0000","last_became_peered":"2024-04-25T08:46:46.710510+0000","last_unstale":"2024-04-25T08:46:46.716109+0000","last_undegraded":"2024-04-25T08:46:46.716109+0000","last_fullsized":"2024-04-25T08:46:46.716109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.838342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718615+0000","last_change":"2024-04-25T08:46:46.715926+0000","last_active":"2024-04-25T08:46:46.718615+0000","last_peered":"2024-04-25T08:46:46.718615+0000","last_clean":"2024-04-25T08:46:46.718615+0000","last_became_active":"2024-04-25T08:46:46.715646+0000","last_became_peered":"2024-04-25T08:46:46.715646+0000","last_unstale":"2024-04-25T08:46:46.718615+0000","last_undegraded":"2024-04-25T08:46:46.718615+0000","last_fullsized":"2024-04-25T08:46:46.718615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:29.285849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.729653+0000","last_change":"2024-04-25T08:46:46.724934+0000","last_active":"2024-04-25T08:46:46.729653+0000","last_peered":"2024-04-25T08:46:46.729653+0000","last_clean":"2024-04-25T08:46:46.729653+0000","last_became_active":"2024-04-25T08:46:46.724677+0000","last_became_peered":"2024-04-25T08:46:46.724677+0000","last_unstale":"2024-04-25T08:46:46.729653+0000","last_undegraded":"2024-04-25T08:46:46.729653+0000","last_fullsized":"2024-04-25T08:46:46.729653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:43.813241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717884+0000","last_change":"2024-04-25T08:46:46.712588+0000","last_active":"2024-04-25T08:46:46.717884+0000","last_peered":"2024-04-25T08:46:46.717884+0000","last_clean":"2024-04-25T08:46:46.717884+0000","last_became_active":"2024-04-25T08:46:46.712335+0000","last_became_peered":"2024-04-25T08:46:46.712335+0000","last_unstale":"2024-04-25T08:46:46.717884+0000","last_undegraded":"2024-04-25T08:46:46.717884+0000","last_fullsized":"2024-04-25T08:46:46.717884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:19.222006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.728214+0000","last_change":"2024-04-25T08:46:46.715840+0000","last_active":"2024-04-25T08:46:46.728214+0000","last_peered":"2024-04-25T08:46:46.728214+0000","last_clean":"2024-04-25T08:46:46.728214+0000","last_became_active":"2024-04-25T08:46:46.715242+0000","last_became_peered":"2024-04-25T08:46:46.715242+0000","last_unstale":"2024-04-25T08:46:46.728214+0000","last_undegraded":"2024-04-25T08:46:46.728214+0000","last_fullsized":"2024-04-25T08:46:46.728214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:47.320724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.706167+0000","last_change":"2024-04-25T08:46:46.703079+0000","last_active":"2024-04-25T08:46:46.706167+0000","last_peered":"2024-04-25T08:46:46.706167+0000","last_clean":"2024-04-25T08:46:46.706167+0000","last_became_active":"2024-04-25T08:46:46.702729+0000","last_became_peered":"2024-04-25T08:46:46.702729+0000","last_unstale":"2024-04-25T08:46:46.706167+0000","last_undegraded":"2024-04-25T08:46:46.706167+0000","last_fullsized":"2024-04-25T08:46:46.706167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:20.916999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.722897+0000","last_change":"2024-04-25T08:46:46.721800+0000","last_active":"2024-04-25T08:46:46.722897+0000","last_peered":"2024-04-25T08:46:46.722897+0000","last_clean":"2024-04-25T08:46:46.722897+0000","last_became_active":"2024-04-25T08:46:46.721529+0000","last_became_peered":"2024-04-25T08:46:46.721529+0000","last_unstale":"2024-04-25T08:46:46.722897+0000","last_undegraded":"2024-04-25T08:46:46.722897+0000","last_fullsized":"2024-04-25T08:46:46.722897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:30.964215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.714778+0000","last_change":"2024-04-25T08:46:46.709729+0000","last_active":"2024-04-25T08:46:46.714778+0000","last_peered":"2024-04-25T08:46:46.714778+0000","last_clean":"2024-04-25T08:46:46.714778+0000","last_became_active":"2024-04-25T08:46:46.709347+0000","last_became_peered":"2024-04-25T08:46:46.709347+0000","last_unstale":"2024-04-25T08:46:46.714778+0000","last_undegraded":"2024-04-25T08:46:46.714778+0000","last_fullsized":"2024-04-25T08:46:46.714778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:10.285093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.731528+0000","last_change":"2024-04-25T08:46:46.719370+0000","last_active":"2024-04-25T08:46:46.731528+0000","last_peered":"2024-04-25T08:46:46.731528+0000","last_clean":"2024-04-25T08:46:46.731528+0000","last_became_active":"2024-04-25T08:46:46.719092+0000","last_became_peered":"2024-04-25T08:46:46.719092+0000","last_unstale":"2024-04-25T08:46:46.731528+0000","last_undegraded":"2024-04-25T08:46:46.731528+0000","last_fullsized":"2024-04-25T08:46:46.731528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:52.597766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716318+0000","last_change":"2024-04-25T08:46:46.703403+0000","last_active":"2024-04-25T08:46:46.716318+0000","last_peered":"2024-04-25T08:46:46.716318+0000","last_clean":"2024-04-25T08:46:46.716318+0000","last_became_active":"2024-04-25T08:46:46.703124+0000","last_became_peered":"2024-04-25T08:46:46.703124+0000","last_unstale":"2024-04-25T08:46:46.716318+0000","last_undegraded":"2024-04-25T08:46:46.716318+0000","last_fullsized":"2024-04-25T08:46:46.716318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.453249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.723140+0000","last_change":"2024-04-25T08:46:46.710178+0000","last_active":"2024-04-25T08:46:46.723140+0000","last_peered":"2024-04-25T08:46:46.723140+0000","last_clean":"2024-04-25T08:46:46.723140+0000","last_became_active":"2024-04-25T08:46:46.709918+0000","last_became_peered":"2024-04-25T08:46:46.709918+0000","last_unstale":"2024-04-25T08:46:46.723140+0000","last_undegraded":"2024-04-25T08:46:46.723140+0000","last_fullsized":"2024-04-25T08:46:46.723140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:30.964215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.727739+0000","last_change":"2024-04-25T08:46:46.723206+0000","last_active":"2024-04-25T08:46:46.727739+0000","last_peered":"2024-04-25T08:46:46.727739+0000","last_clean":"2024-04-25T08:46:46.727739+0000","last_became_active":"2024-04-25T08:46:46.722947+0000","last_became_peered":"2024-04-25T08:46:46.722947+0000","last_unstale":"2024-04-25T08:46:46.727739+0000","last_undegraded":"2024-04-25T08:46:46.727739+0000","last_fullsized":"2024-04-25T08:46:46.727739+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:14.534414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715990+0000","last_change":"2024-04-25T08:46:46.711710+0000","last_active":"2024-04-25T08:46:46.715990+0000","last_peered":"2024-04-25T08:46:46.715990+0000","last_clean":"2024-04-25T08:46:46.715990+0000","last_became_active":"2024-04-25T08:46:46.709735+0000","last_became_peered":"2024-04-25T08:46:46.709735+0000","last_unstale":"2024-04-25T08:46:46.715990+0000","last_undegraded":"2024-04-25T08:46:46.715990+0000","last_fullsized":"2024-04-25T08:46:46.715990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:31.591586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.724433+0000","last_change":"2024-04-25T08:46:46.712951+0000","last_active":"2024-04-25T08:46:46.724433+0000","last_peered":"2024-04-25T08:46:46.724433+0000","last_clean":"2024-04-25T08:46:46.724433+0000","last_became_active":"2024-04-25T08:46:46.712575+0000","last_became_peered":"2024-04-25T08:46:46.712575+0000","last_unstale":"2024-04-25T08:46:46.724433+0000","last_undegraded":"2024-04-25T08:46:46.724433+0000","last_fullsized":"2024-04-25T08:46:46.724433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:32.562335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.709855+0000","last_change":"2024-04-25T08:46:46.706281+0000","last_active":"2024-04-25T08:46:46.709855+0000","last_peered":"2024-04-25T08:46:46.709855+0000","last_clean":"2024-04-25T08:46:46.709855+0000","last_became_active":"2024-04-25T08:46:46.705982+0000","last_became_peered":"2024-04-25T08:46:46.705982+0000","last_unstale":"2024-04-25T08:46:46.709855+0000","last_undegraded":"2024-04-25T08:46:46.709855+0000","last_fullsized":"2024-04-25T08:46:46.709855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:22.339722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719232+0000","last_change":"2024-04-25T08:46:46.715308+0000","last_active":"2024-04-25T08:46:46.719232+0000","last_peered":"2024-04-25T08:46:46.719232+0000","last_clean":"2024-04-25T08:46:46.719232+0000","last_became_active":"2024-04-25T08:46:46.715016+0000","last_became_peered":"2024-04-25T08:46:46.715016+0000","last_unstale":"2024-04-25T08:46:46.719232+0000","last_undegraded":"2024-04-25T08:46:46.719232+0000","last_fullsized":"2024-04-25T08:46:46.719232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:54.427003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.720942+0000","last_change":"2024-04-25T08:46:46.719982+0000","last_active":"2024-04-25T08:46:46.720942+0000","last_peered":"2024-04-25T08:46:46.720942+0000","last_clean":"2024-04-25T08:46:46.720942+0000","last_became_active":"2024-04-25T08:46:46.719626+0000","last_became_peered":"2024-04-25T08:46:46.719626+0000","last_unstale":"2024-04-25T08:46:46.720942+0000","last_undegraded":"2024-04-25T08:46:46.720942+0000","last_fullsized":"2024-04-25T08:46:46.720942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:20.916999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717279+0000","last_change":"2024-04-25T08:46:46.709590+0000","last_active":"2024-04-25T08:46:46.717279+0000","last_peered":"2024-04-25T08:46:46.717279+0000","last_clean":"2024-04-25T08:46:46.717279+0000","last_became_active":"2024-04-25T08:46:46.709309+0000","last_became_peered":"2024-04-25T08:46:46.709309+0000","last_unstale":"2024-04-25T08:46:46.717279+0000","last_undegraded":"2024-04-25T08:46:46.717279+0000","last_fullsized":"2024-04-25T08:46:46.717279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:59.860963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718247+0000","last_change":"2024-04-25T08:46:46.712249+0000","last_active":"2024-04-25T08:46:46.718247+0000","last_peered":"2024-04-25T08:46:46.718247+0000","last_clean":"2024-04-25T08:46:46.718247+0000","last_became_active":"2024-04-25T08:46:46.711926+0000","last_became_peered":"2024-04-25T08:46:46.711926+0000","last_unstale":"2024-04-25T08:46:46.718247+0000","last_undegraded":"2024-04-25T08:46:46.718247+0000","last_fullsized":"2024-04-25T08:46:46.718247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:36.534633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.723888+0000","last_change":"2024-04-25T08:46:46.707846+0000","last_active":"2024-04-25T08:46:46.723888+0000","last_peered":"2024-04-25T08:46:46.723888+0000","last_clean":"2024-04-25T08:46:46.723888+0000","last_became_active":"2024-04-25T08:46:46.707441+0000","last_became_peered":"2024-04-25T08:46:46.707441+0000","last_unstale":"2024-04-25T08:46:46.723888+0000","last_undegraded":"2024-04-25T08:46:46.723888+0000","last_fullsized":"2024-04-25T08:46:46.723888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.424654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719767+0000","last_change":"2024-04-25T08:46:46.715739+0000","last_active":"2024-04-25T08:46:46.719767+0000","last_peered":"2024-04-25T08:46:46.719767+0000","last_clean":"2024-04-25T08:46:46.719767+0000","last_became_active":"2024-04-25T08:46:46.715424+0000","last_became_peered":"2024-04-25T08:46:46.715424+0000","last_unstale":"2024-04-25T08:46:46.719767+0000","last_undegraded":"2024-04-25T08:46:46.719767+0000","last_fullsized":"2024-04-25T08:46:46.719767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:35.224653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718224+0000","last_change":"2024-04-25T08:46:46.712243+0000","last_active":"2024-04-25T08:46:46.718224+0000","last_peered":"2024-04-25T08:46:46.718224+0000","last_clean":"2024-04-25T08:46:46.718224+0000","last_became_active":"2024-04-25T08:46:46.711890+0000","last_became_peered":"2024-04-25T08:46:46.711890+0000","last_unstale":"2024-04-25T08:46:46.718224+0000","last_undegraded":"2024-04-25T08:46:46.718224+0000","last_fullsized":"2024-04-25T08:46:46.718224+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:00.394910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.708805+0000","last_change":"2024-04-25T08:46:46.697110+0000","last_active":"2024-04-25T08:46:46.708805+0000","last_peered":"2024-04-25T08:46:46.708805+0000","last_clean":"2024-04-25T08:46:46.708805+0000","last_became_active":"2024-04-25T08:46:46.696803+0000","last_became_peered":"2024-04-25T08:46:46.696803+0000","last_unstale":"2024-04-25T08:46:46.708805+0000","last_undegraded":"2024-04-25T08:46:46.708805+0000","last_fullsized":"2024-04-25T08:46:46.708805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.838342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718233+0000","last_change":"2024-04-25T08:46:46.713492+0000","last_active":"2024-04-25T08:46:46.718233+0000","last_peered":"2024-04-25T08:46:46.718233+0000","last_clean":"2024-04-25T08:46:46.718233+0000","last_became_active":"2024-04-25T08:46:46.713135+0000","last_became_peered":"2024-04-25T08:46:46.713135+0000","last_unstale":"2024-04-25T08:46:46.718233+0000","last_undegraded":"2024-04-25T08:46:46.718233+0000","last_fullsized":"2024-04-25T08:46:46.718233+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:01.738098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.708158+0000","last_change":"2024-04-25T08:46:46.699051+0000","last_active":"2024-04-25T08:46:46.708158+0000","last_peered":"2024-04-25T08:46:46.708158+0000","last_clean":"2024-04-25T08:46:46.708158+0000","last_became_active":"2024-04-25T08:46:46.698756+0000","last_became_peered":"2024-04-25T08:46:46.698756+0000","last_unstale":"2024-04-25T08:46:46.708158+0000","last_undegraded":"2024-04-25T08:46:46.708158+0000","last_fullsized":"2024-04-25T08:46:46.708158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:31.591586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.737123+0000","last_change":"2024-04-25T08:46:45.704820+0000","last_active":"2024-04-25T08:46:46.737123+0000","last_peered":"2024-04-25T08:46:46.737123+0000","last_clean":"2024-04-25T08:46:46.737123+0000","last_became_active":"2024-04-25T08:46:45.704544+0000","last_became_peered":"2024-04-25T08:46:45.704544+0000","last_unstale":"2024-04-25T08:46:46.737123+0000","last_undegraded":"2024-04-25T08:46:46.737123+0000","last_fullsized":"2024-04-25T08:46:46.737123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:01.000996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.713865+0000","last_change":"2024-04-25T08:46:46.710904+0000","last_active":"2024-04-25T08:46:46.713865+0000","last_peered":"2024-04-25T08:46:46.713865+0000","last_clean":"2024-04-25T08:46:46.713865+0000","last_became_active":"2024-04-25T08:46:46.710616+0000","last_became_peered":"2024-04-25T08:46:46.710616+0000","last_unstale":"2024-04-25T08:46:46.713865+0000","last_undegraded":"2024-04-25T08:46:46.713865+0000","last_fullsized":"2024-04-25T08:46:46.713865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:01.738098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.731025+0000","last_change":"2024-04-25T08:46:45.702752+0000","last_active":"2024-04-25T08:46:46.731025+0000","last_peered":"2024-04-25T08:46:46.731025+0000","last_clean":"2024-04-25T08:46:46.731025+0000","last_became_active":"2024-04-25T08:46:45.702438+0000","last_became_peered":"2024-04-25T08:46:45.702438+0000","last_unstale":"2024-04-25T08:46:46.731025+0000","last_undegraded":"2024-04-25T08:46:46.731025+0000","last_fullsized":"2024-04-25T08:46:46.731025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:32.853001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.709315+0000","last_change":"2024-04-25T08:46:46.704059+0000","last_active":"2024-04-25T08:46:46.709315+0000","last_peered":"2024-04-25T08:46:46.709315+0000","last_clean":"2024-04-25T08:46:46.709315+0000","last_became_active":"2024-04-25T08:46:46.703781+0000","last_became_peered":"2024-04-25T08:46:46.703781+0000","last_unstale":"2024-04-25T08:46:46.709315+0000","last_undegraded":"2024-04-25T08:46:46.709315+0000","last_fullsized":"2024-04-25T08:46:46.709315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:06.549872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733047+0000","last_change":"2024-04-25T08:46:45.703488+0000","last_active":"2024-04-25T08:46:46.733047+0000","last_peered":"2024-04-25T08:46:46.733047+0000","last_clean":"2024-04-25T08:46:46.733047+0000","last_became_active":"2024-04-25T08:46:45.703159+0000","last_became_peered":"2024-04-25T08:46:45.703159+0000","last_unstale":"2024-04-25T08:46:46.733047+0000","last_undegraded":"2024-04-25T08:46:46.733047+0000","last_fullsized":"2024-04-25T08:46:46.733047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:55.545720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715163+0000","last_change":"2024-04-25T08:46:46.709942+0000","last_active":"2024-04-25T08:46:46.715163+0000","last_peered":"2024-04-25T08:46:46.715163+0000","last_clean":"2024-04-25T08:46:46.715163+0000","last_became_active":"2024-04-25T08:46:46.709636+0000","last_became_peered":"2024-04-25T08:46:46.709636+0000","last_unstale":"2024-04-25T08:46:46.715163+0000","last_undegraded":"2024-04-25T08:46:46.715163+0000","last_fullsized":"2024-04-25T08:46:46.715163+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:36.290118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733408+0000","last_change":"2024-04-25T08:46:45.699880+0000","last_active":"2024-04-25T08:46:46.733408+0000","last_peered":"2024-04-25T08:46:46.733408+0000","last_clean":"2024-04-25T08:46:46.733408+0000","last_became_active":"2024-04-25T08:46:45.698006+0000","last_became_peered":"2024-04-25T08:46:45.698006+0000","last_unstale":"2024-04-25T08:46:46.733408+0000","last_undegraded":"2024-04-25T08:46:46.733408+0000","last_fullsized":"2024-04-25T08:46:46.733408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:35.284216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715082+0000","last_change":"2024-04-25T08:46:46.711119+0000","last_active":"2024-04-25T08:46:46.715082+0000","last_peered":"2024-04-25T08:46:46.715082+0000","last_clean":"2024-04-25T08:46:46.715082+0000","last_became_active":"2024-04-25T08:46:46.710851+0000","last_became_peered":"2024-04-25T08:46:46.710851+0000","last_unstale":"2024-04-25T08:46:46.715082+0000","last_undegraded":"2024-04-25T08:46:46.715082+0000","last_fullsized":"2024-04-25T08:46:46.715082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:36.534633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733080+0000","last_change":"2024-04-25T08:46:45.700271+0000","last_active":"2024-04-25T08:46:46.733080+0000","last_peered":"2024-04-25T08:46:46.733080+0000","last_clean":"2024-04-25T08:46:46.733080+0000","last_became_active":"2024-04-25T08:46:45.699926+0000","last_became_peered":"2024-04-25T08:46:45.699926+0000","last_unstale":"2024-04-25T08:46:46.733080+0000","last_undegraded":"2024-04-25T08:46:46.733080+0000","last_fullsized":"2024-04-25T08:46:46.733080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:41.735775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.726649+0000","last_change":"2024-04-25T08:46:46.724686+0000","last_active":"2024-04-25T08:46:46.726649+0000","last_peered":"2024-04-25T08:46:46.726649+0000","last_clean":"2024-04-25T08:46:46.726649+0000","last_became_active":"2024-04-25T08:46:46.724401+0000","last_became_peered":"2024-04-25T08:46:46.724401+0000","last_unstale":"2024-04-25T08:46:46.726649+0000","last_undegraded":"2024-04-25T08:46:46.726649+0000","last_fullsized":"2024-04-25T08:46:46.726649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:42.697992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.730449+0000","last_change":"2024-04-25T08:46:45.700937+0000","last_active":"2024-04-25T08:46:46.730449+0000","last_peered":"2024-04-25T08:46:46.730449+0000","last_clean":"2024-04-25T08:46:46.730449+0000","last_became_active":"2024-04-25T08:46:45.700635+0000","last_became_peered":"2024-04-25T08:46:45.700635+0000","last_unstale":"2024-04-25T08:46:46.730449+0000","last_undegraded":"2024-04-25T08:46:46.730449+0000","last_fullsized":"2024-04-25T08:46:46.730449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:26.219846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715929+0000","last_change":"2024-04-25T08:46:46.710953+0000","last_active":"2024-04-25T08:46:46.715929+0000","last_peered":"2024-04-25T08:46:46.715929+0000","last_clean":"2024-04-25T08:46:46.715929+0000","last_became_active":"2024-04-25T08:46:46.710553+0000","last_became_peered":"2024-04-25T08:46:46.710553+0000","last_unstale":"2024-04-25T08:46:46.715929+0000","last_undegraded":"2024-04-25T08:46:46.715929+0000","last_fullsized":"2024-04-25T08:46:46.715929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:31.364562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.703115+0000","last_change":"2024-04-25T08:46:45.701649+0000","last_active":"2024-04-25T08:46:46.703115+0000","last_peered":"2024-04-25T08:46:46.703115+0000","last_clean":"2024-04-25T08:46:46.703115+0000","last_became_active":"2024-04-25T08:46:45.701383+0000","last_became_peered":"2024-04-25T08:46:45.701383+0000","last_unstale":"2024-04-25T08:46:46.703115+0000","last_undegraded":"2024-04-25T08:46:46.703115+0000","last_fullsized":"2024-04-25T08:46:46.703115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:38.459309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717089+0000","last_change":"2024-04-25T08:46:46.708138+0000","last_active":"2024-04-25T08:46:46.717089+0000","last_peered":"2024-04-25T08:46:46.717089+0000","last_clean":"2024-04-25T08:46:46.717089+0000","last_became_active":"2024-04-25T08:46:46.707846+0000","last_became_peered":"2024-04-25T08:46:46.707846+0000","last_unstale":"2024-04-25T08:46:46.717089+0000","last_undegraded":"2024-04-25T08:46:46.717089+0000","last_fullsized":"2024-04-25T08:46:46.717089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:56.727233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.698034+0000","last_change":"2024-04-25T08:46:45.705976+0000","last_active":"2024-04-25T08:46:46.698034+0000","last_peered":"2024-04-25T08:46:46.698034+0000","last_clean":"2024-04-25T08:46:46.698034+0000","last_became_active":"2024-04-25T08:46:45.705667+0000","last_became_peered":"2024-04-25T08:46:45.705667+0000","last_unstale":"2024-04-25T08:46:46.698034+0000","last_undegraded":"2024-04-25T08:46:46.698034+0000","last_fullsized":"2024-04-25T08:46:46.698034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:51.565014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719257+0000","last_change":"2024-04-25T08:46:46.718095+0000","last_active":"2024-04-25T08:46:46.719257+0000","last_peered":"2024-04-25T08:46:46.719257+0000","last_clean":"2024-04-25T08:46:46.719257+0000","last_became_active":"2024-04-25T08:46:46.717803+0000","last_became_peered":"2024-04-25T08:46:46.717803+0000","last_unstale":"2024-04-25T08:46:46.719257+0000","last_undegraded":"2024-04-25T08:46:46.719257+0000","last_fullsized":"2024-04-25T08:46:46.719257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.701065+0000","last_change":"2024-04-25T08:46:45.702418+0000","last_active":"2024-04-25T08:46:46.701065+0000","last_peered":"2024-04-25T08:46:46.701065+0000","last_clean":"2024-04-25T08:46:46.701065+0000","last_became_active":"2024-04-25T08:46:45.701693+0000","last_became_peered":"2024-04-25T08:46:45.701693+0000","last_unstale":"2024-04-25T08:46:46.701065+0000","last_undegraded":"2024-04-25T08:46:46.701065+0000","last_fullsized":"2024-04-25T08:46:46.701065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:18.216103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715703+0000","last_change":"2024-04-25T08:46:46.707644+0000","last_active":"2024-04-25T08:46:46.715703+0000","last_peered":"2024-04-25T08:46:46.715703+0000","last_clean":"2024-04-25T08:46:46.715703+0000","last_became_active":"2024-04-25T08:46:46.707234+0000","last_became_peered":"2024-04-25T08:46:46.707234+0000","last_unstale":"2024-04-25T08:46:46.715703+0000","last_undegraded":"2024-04-25T08:46:46.715703+0000","last_fullsized":"2024-04-25T08:46:46.715703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:45.056546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.701821+0000","last_change":"2024-04-25T08:46:45.700606+0000","last_active":"2024-04-25T08:46:46.701821+0000","last_peered":"2024-04-25T08:46:46.701821+0000","last_clean":"2024-04-25T08:46:46.701821+0000","last_became_active":"2024-04-25T08:46:45.700314+0000","last_became_peered":"2024-04-25T08:46:45.700314+0000","last_unstale":"2024-04-25T08:46:46.701821+0000","last_undegraded":"2024-04-25T08:46:46.701821+0000","last_fullsized":"2024-04-25T08:46:46.701821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:30.489415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.706145+0000","last_change":"2024-04-25T08:46:46.701191+0000","last_active":"2024-04-25T08:46:46.706145+0000","last_peered":"2024-04-25T08:46:46.706145+0000","last_clean":"2024-04-25T08:46:46.706145+0000","last_became_active":"2024-04-25T08:46:46.696650+0000","last_became_peered":"2024-04-25T08:46:46.696650+0000","last_unstale":"2024-04-25T08:46:46.706145+0000","last_undegraded":"2024-04-25T08:46:46.706145+0000","last_fullsized":"2024-04-25T08:46:46.706145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733991+0000","last_change":"2024-04-25T08:46:45.700414+0000","last_active":"2024-04-25T08:46:46.733991+0000","last_peered":"2024-04-25T08:46:46.733991+0000","last_clean":"2024-04-25T08:46:46.733991+0000","last_became_active":"2024-04-25T08:46:45.699873+0000","last_became_peered":"2024-04-25T08:46:45.699873+0000","last_unstale":"2024-04-25T08:46:46.733991+0000","last_undegraded":"2024-04-25T08:46:46.733991+0000","last_fullsized":"2024-04-25T08:46:46.733991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:01.709143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719627+0000","last_change":"2024-04-25T08:46:46.707969+0000","last_active":"2024-04-25T08:46:46.719627+0000","last_peered":"2024-04-25T08:46:46.719627+0000","last_clean":"2024-04-25T08:46:46.719627+0000","last_became_active":"2024-04-25T08:46:46.707622+0000","last_became_peered":"2024-04-25T08:46:46.707622+0000","last_unstale":"2024-04-25T08:46:46.719627+0000","last_undegraded":"2024-04-25T08:46:46.719627+0000","last_fullsized":"2024-04-25T08:46:46.719627+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:57.610010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"77'6","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.741536+0000","last_change":"2024-04-25T08:46:45.691426+0000","last_active":"2024-04-25T08:46:46.741536+0000","last_peered":"2024-04-25T08:46:46.741536+0000","last_clean":"2024-04-25T08:46:46.741536+0000","last_became_active":"2024-04-25T08:46:45.691141+0000","last_became_peered":"2024-04-25T08:46:45.691141+0000","last_unstale":"2024-04-25T08:46:46.741536+0000","last_undegraded":"2024-04-25T08:46:46.741536+0000","last_fullsized":"2024-04-25T08:46:46.741536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:30.358659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717909+0000","last_change":"2024-04-25T08:46:46.714019+0000","last_active":"2024-04-25T08:46:46.717909+0000","last_peered":"2024-04-25T08:46:46.717909+0000","last_clean":"2024-04-25T08:46:46.717909+0000","last_became_active":"2024-04-25T08:46:46.713459+0000","last_became_peered":"2024-04-25T08:46:46.713459+0000","last_unstale":"2024-04-25T08:46:46.717909+0000","last_undegraded":"2024-04-25T08:46:46.717909+0000","last_fullsized":"2024-04-25T08:46:46.717909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:56.551623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733722+0000","last_change":"2024-04-25T08:46:45.697503+0000","last_active":"2024-04-25T08:46:46.733722+0000","last_peered":"2024-04-25T08:46:46.733722+0000","last_clean":"2024-04-25T08:46:46.733722+0000","last_became_active":"2024-04-25T08:46:45.696801+0000","last_became_peered":"2024-04-25T08:46:45.696801+0000","last_unstale":"2024-04-25T08:46:46.733722+0000","last_undegraded":"2024-04-25T08:46:46.733722+0000","last_fullsized":"2024-04-25T08:46:46.733722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:35.024489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.702833+0000","last_change":"2024-04-25T08:46:46.697506+0000","last_active":"2024-04-25T08:46:46.702833+0000","last_peered":"2024-04-25T08:46:46.702833+0000","last_clean":"2024-04-25T08:46:46.702833+0000","last_became_active":"2024-04-25T08:46:46.697229+0000","last_became_peered":"2024-04-25T08:46:46.697229+0000","last_unstale":"2024-04-25T08:46:46.702833+0000","last_undegraded":"2024-04-25T08:46:46.702833+0000","last_fullsized":"2024-04-25T08:46:46.702833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.838342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.698524+0000","last_change":"2024-04-25T08:46:45.706187+0000","last_active":"2024-04-25T08:46:46.698524+0000","last_peered":"2024-04-25T08:46:46.698524+0000","last_clean":"2024-04-25T08:46:46.698524+0000","last_became_active":"2024-04-25T08:46:45.705916+0000","last_became_peered":"2024-04-25T08:46:45.705916+0000","last_unstale":"2024-04-25T08:46:46.698524+0000","last_undegraded":"2024-04-25T08:46:46.698524+0000","last_fullsized":"2024-04-25T08:46:46.698524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:58.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717603+0000","last_change":"2024-04-25T08:46:46.708549+0000","last_active":"2024-04-25T08:46:46.717603+0000","last_peered":"2024-04-25T08:46:46.717603+0000","last_clean":"2024-04-25T08:46:46.717603+0000","last_became_active":"2024-04-25T08:46:46.708188+0000","last_became_peered":"2024-04-25T08:46:46.708188+0000","last_unstale":"2024-04-25T08:46:46.717603+0000","last_undegraded":"2024-04-25T08:46:46.717603+0000","last_fullsized":"2024-04-25T08:46:46.717603+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:48.684016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.701263+0000","last_change":"2024-04-25T08:46:45.704805+0000","last_active":"2024-04-25T08:46:46.701263+0000","last_peered":"2024-04-25T08:46:46.701263+0000","last_clean":"2024-04-25T08:46:46.701263+0000","last_became_active":"2024-04-25T08:46:45.704489+0000","last_became_peered":"2024-04-25T08:46:45.704489+0000","last_unstale":"2024-04-25T08:46:46.701263+0000","last_undegraded":"2024-04-25T08:46:46.701263+0000","last_fullsized":"2024-04-25T08:46:46.701263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:37.729671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.714735+0000","last_change":"2024-04-25T08:46:46.705829+0000","last_active":"2024-04-25T08:46:46.714735+0000","last_peered":"2024-04-25T08:46:46.714735+0000","last_clean":"2024-04-25T08:46:46.714735+0000","last_became_active":"2024-04-25T08:46:46.705535+0000","last_became_peered":"2024-04-25T08:46:46.705535+0000","last_unstale":"2024-04-25T08:46:46.714735+0000","last_undegraded":"2024-04-25T08:46:46.714735+0000","last_fullsized":"2024-04-25T08:46:46.714735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:57.108537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.739675+0000","last_change":"2024-04-25T08:46:45.702125+0000","last_active":"2024-04-25T08:46:46.739675+0000","last_peered":"2024-04-25T08:46:46.739675+0000","last_clean":"2024-04-25T08:46:46.739675+0000","last_became_active":"2024-04-25T08:46:45.701183+0000","last_became_peered":"2024-04-25T08:46:45.701183+0000","last_unstale":"2024-04-25T08:46:46.739675+0000","last_undegraded":"2024-04-25T08:46:46.739675+0000","last_fullsized":"2024-04-25T08:46:46.739675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:59.389007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718973+0000","last_change":"2024-04-25T08:46:46.709579+0000","last_active":"2024-04-25T08:46:46.718973+0000","last_peered":"2024-04-25T08:46:46.718973+0000","last_clean":"2024-04-25T08:46:46.718973+0000","last_became_active":"2024-04-25T08:46:46.708668+0000","last_became_peered":"2024-04-25T08:46:46.708668+0000","last_unstale":"2024-04-25T08:46:46.718973+0000","last_undegraded":"2024-04-25T08:46:46.718973+0000","last_fullsized":"2024-04-25T08:46:46.718973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:43.813241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.732918+0000","last_change":"2024-04-25T08:46:45.702463+0000","last_active":"2024-04-25T08:46:46.732918+0000","last_peered":"2024-04-25T08:46:46.732918+0000","last_clean":"2024-04-25T08:46:46.732918+0000","last_became_active":"2024-04-25T08:46:45.702083+0000","last_became_peered":"2024-04-25T08:46:45.702083+0000","last_unstale":"2024-04-25T08:46:46.732918+0000","last_undegraded":"2024-04-25T08:46:46.732918+0000","last_fullsized":"2024-04-25T08:46:46.732918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.085368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.728173+0000","last_change":"2024-04-25T08:46:46.718318+0000","last_active":"2024-04-25T08:46:46.728173+0000","last_peered":"2024-04-25T08:46:46.728173+0000","last_clean":"2024-04-25T08:46:46.728173+0000","last_became_active":"2024-04-25T08:46:46.718059+0000","last_became_peered":"2024-04-25T08:46:46.718059+0000","last_unstale":"2024-04-25T08:46:46.728173+0000","last_undegraded":"2024-04-25T08:46:46.728173+0000","last_fullsized":"2024-04-25T08:46:46.728173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:06.495163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.731330+0000","last_change":"2024-04-25T08:46:45.686160+0000","last_active":"2024-04-25T08:46:46.731330+0000","last_peered":"2024-04-25T08:46:46.731330+0000","last_clean":"2024-04-25T08:46:46.731330+0000","last_became_active":"2024-04-25T08:46:45.685831+0000","last_became_peered":"2024-04-25T08:46:45.685831+0000","last_unstale":"2024-04-25T08:46:46.731330+0000","last_undegraded":"2024-04-25T08:46:46.731330+0000","last_fullsized":"2024-04-25T08:46:46.731330+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:58.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.724412+0000","last_change":"2024-04-25T08:46:46.712349+0000","last_active":"2024-04-25T08:46:46.724412+0000","last_peered":"2024-04-25T08:46:46.724412+0000","last_clean":"2024-04-25T08:46:46.724412+0000","last_became_active":"2024-04-25T08:46:46.712086+0000","last_became_peered":"2024-04-25T08:46:46.712086+0000","last_unstale":"2024-04-25T08:46:46.724412+0000","last_undegraded":"2024-04-25T08:46:46.724412+0000","last_fullsized":"2024-04-25T08:46:46.724412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:23.391791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.736318+0000","last_change":"2024-04-25T08:46:45.706507+0000","last_active":"2024-04-25T08:46:46.736318+0000","last_peered":"2024-04-25T08:46:46.736318+0000","last_clean":"2024-04-25T08:46:46.736318+0000","last_became_active":"2024-04-25T08:46:45.706179+0000","last_became_peered":"2024-04-25T08:46:45.706179+0000","last_unstale":"2024-04-25T08:46:46.736318+0000","last_undegraded":"2024-04-25T08:46:46.736318+0000","last_fullsized":"2024-04-25T08:46:46.736318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:56.139746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719661+0000","last_change":"2024-04-25T08:46:46.712042+0000","last_active":"2024-04-25T08:46:46.719661+0000","last_peered":"2024-04-25T08:46:46.719661+0000","last_clean":"2024-04-25T08:46:46.719661+0000","last_became_active":"2024-04-25T08:46:46.711738+0000","last_became_peered":"2024-04-25T08:46:46.711738+0000","last_unstale":"2024-04-25T08:46:46.719661+0000","last_undegraded":"2024-04-25T08:46:46.719661+0000","last_fullsized":"2024-04-25T08:46:46.719661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:42.697992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.690357+0000","last_change":"2024-04-25T08:46:45.698601+0000","last_active":"2024-04-25T08:46:46.690357+0000","last_peered":"2024-04-25T08:46:46.690357+0000","last_clean":"2024-04-25T08:46:46.690357+0000","last_became_active":"2024-04-25T08:46:45.698298+0000","last_became_peered":"2024-04-25T08:46:45.698298+0000","last_unstale":"2024-04-25T08:46:46.690357+0000","last_undegraded":"2024-04-25T08:46:46.690357+0000","last_fullsized":"2024-04-25T08:46:46.690357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:30.358659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717937+0000","last_change":"2024-04-25T08:46:46.716551+0000","last_active":"2024-04-25T08:46:46.717937+0000","last_peered":"2024-04-25T08:46:46.717937+0000","last_clean":"2024-04-25T08:46:46.717937+0000","last_became_active":"2024-04-25T08:46:46.716288+0000","last_became_peered":"2024-04-25T08:46:46.716288+0000","last_unstale":"2024-04-25T08:46:46.717937+0000","last_undegraded":"2024-04-25T08:46:46.717937+0000","last_fullsized":"2024-04-25T08:46:46.717937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:46.210853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.736501+0000","last_change":"2024-04-25T08:46:45.693253+0000","last_active":"2024-04-25T08:46:46.736501+0000","last_peered":"2024-04-25T08:46:46.736501+0000","last_clean":"2024-04-25T08:46:46.736501+0000","last_became_active":"2024-04-25T08:46:45.692001+0000","last_became_peered":"2024-04-25T08:46:45.692001+0000","last_unstale":"2024-04-25T08:46:46.736501+0000","last_undegraded":"2024-04-25T08:46:46.736501+0000","last_fullsized":"2024-04-25T08:46:46.736501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:53.421100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716699+0000","last_change":"2024-04-25T08:46:46.712113+0000","last_active":"2024-04-25T08:46:46.716699+0000","last_peered":"2024-04-25T08:46:46.716699+0000","last_clean":"2024-04-25T08:46:46.716699+0000","last_became_active":"2024-04-25T08:46:46.711825+0000","last_became_peered":"2024-04-25T08:46:46.711825+0000","last_unstale":"2024-04-25T08:46:46.716699+0000","last_undegraded":"2024-04-25T08:46:46.716699+0000","last_fullsized":"2024-04-25T08:46:46.716699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:14.347016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.696494+0000","last_change":"2024-04-25T08:46:45.695903+0000","last_active":"2024-04-25T08:46:46.696494+0000","last_peered":"2024-04-25T08:46:46.696494+0000","last_clean":"2024-04-25T08:46:46.696494+0000","last_became_active":"2024-04-25T08:46:45.695603+0000","last_became_peered":"2024-04-25T08:46:45.695603+0000","last_unstale":"2024-04-25T08:46:46.696494+0000","last_undegraded":"2024-04-25T08:46:46.696494+0000","last_fullsized":"2024-04-25T08:46:46.696494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:34.218750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.714761+0000","last_change":"2024-04-25T08:46:46.706428+0000","last_active":"2024-04-25T08:46:46.714761+0000","last_peered":"2024-04-25T08:46:46.714761+0000","last_clean":"2024-04-25T08:46:46.714761+0000","last_became_active":"2024-04-25T08:46:46.706135+0000","last_became_peered":"2024-04-25T08:46:46.706135+0000","last_unstale":"2024-04-25T08:46:46.714761+0000","last_undegraded":"2024-04-25T08:46:46.714761+0000","last_fullsized":"2024-04-25T08:46:46.714761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:57.610010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.696170+0000","last_change":"2024-04-25T08:46:45.699319+0000","last_active":"2024-04-25T08:46:46.696170+0000","last_peered":"2024-04-25T08:46:46.696170+0000","last_clean":"2024-04-25T08:46:46.696170+0000","last_became_active":"2024-04-25T08:46:45.699066+0000","last_became_peered":"2024-04-25T08:46:45.699066+0000","last_unstale":"2024-04-25T08:46:46.696170+0000","last_undegraded":"2024-04-25T08:46:46.696170+0000","last_fullsized":"2024-04-25T08:46:46.696170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:08.537484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.706625+0000","last_change":"2024-04-25T08:46:46.700693+0000","last_active":"2024-04-25T08:46:46.706625+0000","last_peered":"2024-04-25T08:46:46.706625+0000","last_clean":"2024-04-25T08:46:46.706625+0000","last_became_active":"2024-04-25T08:46:46.700411+0000","last_became_peered":"2024-04-25T08:46:46.700411+0000","last_unstale":"2024-04-25T08:46:46.706625+0000","last_undegraded":"2024-04-25T08:46:46.706625+0000","last_fullsized":"2024-04-25T08:46:46.706625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:53.477544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.731335+0000","last_change":"2024-04-25T08:46:45.694939+0000","last_active":"2024-04-25T08:46:46.731335+0000","last_peered":"2024-04-25T08:46:46.731335+0000","last_clean":"2024-04-25T08:46:46.731335+0000","last_became_active":"2024-04-25T08:46:45.692914+0000","last_became_peered":"2024-04-25T08:46:45.692914+0000","last_unstale":"2024-04-25T08:46:46.731335+0000","last_undegraded":"2024-04-25T08:46:46.731335+0000","last_fullsized":"2024-04-25T08:46:46.731335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:13.637793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.714850+0000","last_change":"2024-04-25T08:46:46.703345+0000","last_active":"2024-04-25T08:46:46.714850+0000","last_peered":"2024-04-25T08:46:46.714850+0000","last_clean":"2024-04-25T08:46:46.714850+0000","last_became_active":"2024-04-25T08:46:46.703073+0000","last_became_peered":"2024-04-25T08:46:46.703073+0000","last_unstale":"2024-04-25T08:46:46.714850+0000","last_undegraded":"2024-04-25T08:46:46.714850+0000","last_fullsized":"2024-04-25T08:46:46.714850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.453249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.702175+0000","last_change":"2024-04-25T08:46:45.707691+0000","last_active":"2024-04-25T08:46:46.702175+0000","last_peered":"2024-04-25T08:46:46.702175+0000","last_clean":"2024-04-25T08:46:46.702175+0000","last_became_active":"2024-04-25T08:46:45.706884+0000","last_became_peered":"2024-04-25T08:46:45.706884+0000","last_unstale":"2024-04-25T08:46:46.702175+0000","last_undegraded":"2024-04-25T08:46:46.702175+0000","last_fullsized":"2024-04-25T08:46:46.702175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.279190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.709185+0000","last_change":"2024-04-25T08:46:46.703979+0000","last_active":"2024-04-25T08:46:46.709185+0000","last_peered":"2024-04-25T08:46:46.709185+0000","last_clean":"2024-04-25T08:46:46.709185+0000","last_became_active":"2024-04-25T08:46:46.703654+0000","last_became_peered":"2024-04-25T08:46:46.703654+0000","last_unstale":"2024-04-25T08:46:46.709185+0000","last_undegraded":"2024-04-25T08:46:46.709185+0000","last_fullsized":"2024-04-25T08:46:46.709185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:03.564585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.702509+0000","last_change":"2024-04-25T08:46:45.700854+0000","last_active":"2024-04-25T08:46:46.702509+0000","last_peered":"2024-04-25T08:46:46.702509+0000","last_clean":"2024-04-25T08:46:46.702509+0000","last_became_active":"2024-04-25T08:46:45.700537+0000","last_became_peered":"2024-04-25T08:46:45.700537+0000","last_unstale":"2024-04-25T08:46:46.702509+0000","last_undegraded":"2024-04-25T08:46:46.702509+0000","last_fullsized":"2024-04-25T08:46:46.702509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:28.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716933+0000","last_change":"2024-04-25T08:46:46.713274+0000","last_active":"2024-04-25T08:46:46.716933+0000","last_peered":"2024-04-25T08:46:46.716933+0000","last_clean":"2024-04-25T08:46:46.716933+0000","last_became_active":"2024-04-25T08:46:46.712971+0000","last_became_peered":"2024-04-25T08:46:46.712971+0000","last_unstale":"2024-04-25T08:46:46.716933+0000","last_undegraded":"2024-04-25T08:46:46.716933+0000","last_fullsized":"2024-04-25T08:46:46.716933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.543387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.697978+0000","last_change":"2024-04-25T08:46:45.701102+0000","last_active":"2024-04-25T08:46:46.697978+0000","last_peered":"2024-04-25T08:46:46.697978+0000","last_clean":"2024-04-25T08:46:46.697978+0000","last_became_active":"2024-04-25T08:46:45.700781+0000","last_became_peered":"2024-04-25T08:46:45.700781+0000","last_unstale":"2024-04-25T08:46:46.697978+0000","last_undegraded":"2024-04-25T08:46:46.697978+0000","last_fullsized":"2024-04-25T08:46:46.697978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:44.050643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.708393+0000","last_change":"2024-04-25T08:46:46.699835+0000","last_active":"2024-04-25T08:46:46.708393+0000","last_peered":"2024-04-25T08:46:46.708393+0000","last_clean":"2024-04-25T08:46:46.708393+0000","last_became_active":"2024-04-25T08:46:46.698199+0000","last_became_peered":"2024-04-25T08:46:46.698199+0000","last_unstale":"2024-04-25T08:46:46.708393+0000","last_undegraded":"2024-04-25T08:46:46.708393+0000","last_fullsized":"2024-04-25T08:46:46.708393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:31.591586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.695742+0000","last_change":"2024-04-25T08:46:45.698193+0000","last_active":"2024-04-25T08:46:46.695742+0000","last_peered":"2024-04-25T08:46:46.695742+0000","last_clean":"2024-04-25T08:46:46.695742+0000","last_became_active":"2024-04-25T08:46:45.697913+0000","last_became_peered":"2024-04-25T08:46:45.697913+0000","last_unstale":"2024-04-25T08:46:46.695742+0000","last_undegraded":"2024-04-25T08:46:46.695742+0000","last_fullsized":"2024-04-25T08:46:46.695742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:21.333819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716962+0000","last_change":"2024-04-25T08:46:46.708017+0000","last_active":"2024-04-25T08:46:46.716962+0000","last_peered":"2024-04-25T08:46:46.716962+0000","last_clean":"2024-04-25T08:46:46.716962+0000","last_became_active":"2024-04-25T08:46:46.707706+0000","last_became_peered":"2024-04-25T08:46:46.707706+0000","last_unstale":"2024-04-25T08:46:46.716962+0000","last_undegraded":"2024-04-25T08:46:46.716962+0000","last_fullsized":"2024-04-25T08:46:46.716962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:57.505207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.702236+0000","last_change":"2024-04-25T08:46:45.705442+0000","last_active":"2024-04-25T08:46:46.702236+0000","last_peered":"2024-04-25T08:46:46.702236+0000","last_clean":"2024-04-25T08:46:46.702236+0000","last_became_active":"2024-04-25T08:46:45.705139+0000","last_became_peered":"2024-04-25T08:46:45.705139+0000","last_unstale":"2024-04-25T08:46:46.702236+0000","last_undegraded":"2024-04-25T08:46:46.702236+0000","last_fullsized":"2024-04-25T08:46:46.702236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:39.370138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.723836+0000","last_change":"2024-04-25T08:46:46.712747+0000","last_active":"2024-04-25T08:46:46.723836+0000","last_peered":"2024-04-25T08:46:46.723836+0000","last_clean":"2024-04-25T08:46:46.723836+0000","last_became_active":"2024-04-25T08:46:46.712450+0000","last_became_peered":"2024-04-25T08:46:46.712450+0000","last_unstale":"2024-04-25T08:46:46.723836+0000","last_undegraded":"2024-04-25T08:46:46.723836+0000","last_fullsized":"2024-04-25T08:46:46.723836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:56.727233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.695700+0000","last_change":"2024-04-25T08:46:45.696519+0000","last_active":"2024-04-25T08:46:46.695700+0000","last_peered":"2024-04-25T08:46:46.695700+0000","last_clean":"2024-04-25T08:46:46.695700+0000","last_became_active":"2024-04-25T08:46:45.695784+0000","last_became_peered":"2024-04-25T08:46:45.695784+0000","last_unstale":"2024-04-25T08:46:46.695700+0000","last_undegraded":"2024-04-25T08:46:46.695700+0000","last_fullsized":"2024-04-25T08:46:46.695700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:05.543970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.711596+0000","last_change":"2024-04-25T08:46:46.704676+0000","last_active":"2024-04-25T08:46:46.711596+0000","last_peered":"2024-04-25T08:46:46.711596+0000","last_clean":"2024-04-25T08:46:46.711596+0000","last_became_active":"2024-04-25T08:46:46.704400+0000","last_became_peered":"2024-04-25T08:46:46.704400+0000","last_unstale":"2024-04-25T08:46:46.711596+0000","last_undegraded":"2024-04-25T08:46:46.711596+0000","last_fullsized":"2024-04-25T08:46:46.711596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:57.108537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.690240+0000","last_change":"2024-04-25T08:46:45.694419+0000","last_active":"2024-04-25T08:46:46.690240+0000","last_peered":"2024-04-25T08:46:46.690240+0000","last_clean":"2024-04-25T08:46:46.690240+0000","last_became_active":"2024-04-25T08:46:45.694105+0000","last_became_peered":"2024-04-25T08:46:45.694105+0000","last_unstale":"2024-04-25T08:46:46.690240+0000","last_undegraded":"2024-04-25T08:46:46.690240+0000","last_fullsized":"2024-04-25T08:46:46.690240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:35.284216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718280+0000","last_change":"2024-04-25T08:46:46.714203+0000","last_active":"2024-04-25T08:46:46.718280+0000","last_peered":"2024-04-25T08:46:46.718280+0000","last_clean":"2024-04-25T08:46:46.718280+0000","last_became_active":"2024-04-25T08:46:46.713949+0000","last_became_peered":"2024-04-25T08:46:46.713949+0000","last_unstale":"2024-04-25T08:46:46.718280+0000","last_undegraded":"2024-04-25T08:46:46.718280+0000","last_fullsized":"2024-04-25T08:46:46.718280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:14.347016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.696552+0000","last_change":"2024-04-25T08:46:45.700280+0000","last_active":"2024-04-25T08:46:46.696552+0000","last_peered":"2024-04-25T08:46:46.696552+0000","last_clean":"2024-04-25T08:46:46.696552+0000","last_became_active":"2024-04-25T08:46:45.699959+0000","last_became_peered":"2024-04-25T08:46:45.699959+0000","last_unstale":"2024-04-25T08:46:46.696552+0000","last_undegraded":"2024-04-25T08:46:46.696552+0000","last_fullsized":"2024-04-25T08:46:46.696552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:45.204950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":275,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.698521+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:46:46.698521+0000","last_peered":"2024-04-25T08:46:46.698521+0000","last_clean":"2024-04-25T08:46:46.698521+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:46:46.698521+0000","last_undegraded":"2024-04-25T08:46:46.698521+0000","last_fullsized":"2024-04-25T08:46:46.698521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":1020051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":422494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":429695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":962560,"data_stored":431377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":430243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":954368,"data_stored":430403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":422494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1536000,"data_stored":1020029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:46:50.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:46:50.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc03b200640 1 Processor -- start 2024-04-25T08:46:50.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc03b200640 1 -- start start 2024-04-25T08:46:50.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc03b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc03c1450b0 0x7fc03c15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc03b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc03c06b320 0x7fc03c0aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:50.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc03b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03c0af600 con 0x7fc03c0aeb20 2024-04-25T08:46:50.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc03b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c0b0210 con 0x7fc03c1450b0 2024-04-25T08:46:50.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc03b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c0af8e0 con 0x7fc03c06b320 2024-04-25T08:46:50.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc039e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc03c1450b0 0x7fc03c15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:50.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc039e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc03c1450b0 0x7fc03c15bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39730/0 (socket says 172.21.15.46:39730) 2024-04-25T08:46:50.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc039e00640 1 -- 172.21.15.46:0/3076190610 learned_addr learned my addr 172.21.15.46:0/3076190610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:51.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc03a800640 1 --2- 172.21.15.46:0/3076190610 >> v2:172.21.15.82:3300/0 conn(0x7fc03c06b320 0x7fc03c0aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc039e00640 1 -- 172.21.15.46:0/3076190610 >> v2:172.21.15.82:3300/0 conn(0x7fc03c06b320 msgr2=0x7fc03c0aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:51.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc039e00640 1 --2- 172.21.15.46:0/3076190610 >> v2:172.21.15.82:3300/0 conn(0x7fc03c06b320 0x7fc03c0aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc039e00640 1 -- 172.21.15.46:0/3076190610 >> v1:172.21.15.82:6789/0 conn(0x7fc03c0aeb20 legacy=0x7fc03c06ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:51.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.005+0000 7fc039e00640 1 -- 172.21.15.46:0/3076190610 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c0afbc0 con 0x7fc03c1450b0 2024-04-25T08:46:51.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc039e00640 1 --2- 172.21.15.46:0/3076190610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c1450b0 0x7fc03c15bb70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0d1390 tx=0x7fc02c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2883de753146ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:51.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc038a00640 1 -- 172.21.15.46:0/3076190610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0ab5f0 con 0x7fc03c1450b0 2024-04-25T08:46:51.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc038a00640 1 -- 172.21.15.46:0/3076190610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c0b0070 con 0x7fc03c1450b0 2024-04-25T08:46:51.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc038a00640 1 -- 172.21.15.46:0/3076190610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0ab9b0 con 0x7fc03c1450b0 2024-04-25T08:46:51.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc03b200640 1 -- 172.21.15.46:0/3076190610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c1450b0 msgr2=0x7fc03c15bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc03b200640 1 --2- 172.21.15.46:0/3076190610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c1450b0 0x7fc03c15bb70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0d1390 tx=0x7fc02c01c4e0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc03b200640 1 -- 172.21.15.46:0/3076190610 shutdown_connections 2024-04-25T08:46:51.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc03b200640 1 --2- 172.21.15.46:0/3076190610 >> v2:172.21.15.82:3300/0 conn(0x7fc03c06b320 0x7fc03c0aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc03b200640 1 --2- 172.21.15.46:0/3076190610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c1450b0 0x7fc03c15bb70 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc03b200640 1 -- 172.21.15.46:0/3076190610 >> 172.21.15.46:0/3076190610 conn(0x7fc03c099a00 msgr2=0x7fc03c0b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:51.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc03b200640 1 -- 172.21.15.46:0/3076190610 shutdown_connections 2024-04-25T08:46:51.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc03b200640 1 -- 172.21.15.46:0/3076190610 wait complete. 2024-04-25T08:46:51.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.009+0000 7fc03b200640 1 Processor -- start 2024-04-25T08:46:51.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 -- start start 2024-04-25T08:46:51.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc03c06b320 0x7fc03c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c0aeb20 0x7fc03c143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc03c1450b0 0x7fc03c1442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0b1280 con 0x7fc03c0aeb20 2024-04-25T08:46:51.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0b1100 con 0x7fc03c1450b0 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0b1400 con 0x7fc03c06b320 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c0aeb20 0x7fc03c143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c0aeb20 0x7fc03c143d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39744/0 (socket says 172.21.15.46:39744) 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039400640 1 -- 172.21.15.46:0/958491927 learned_addr learned my addr 172.21.15.46:0/958491927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03a800640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc03c1450b0 0x7fc03c1442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039e00640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc03c06b320 0x7fc03c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039400640 1 -- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc03c06b320 msgr2=0x7fc03c15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039400640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc03c06b320 0x7fc03c15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039400640 1 -- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc03c1450b0 msgr2=0x7fc03c1442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039400640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc03c1450b0 0x7fc03c1442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039400640 1 -- 172.21.15.46:0/958491927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c164810 con 0x7fc03c0aeb20 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc039400640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c0aeb20 0x7fc03c143d80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c450 tx=0x7fc02800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc032000640 1 -- 172.21.15.46:0/958491927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028012830 con 0x7fc03c0aeb20 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc032000640 1 -- 172.21.15.46:0/958491927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc028017070 con 0x7fc03c0aeb20 2024-04-25T08:46:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc032000640 1 -- 172.21.15.46:0/958491927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028012c30 con 0x7fc03c0aeb20 2024-04-25T08:46:51.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c164af0 con 0x7fc03c0aeb20 2024-04-25T08:46:51.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c1650b0 con 0x7fc03c0aeb20 2024-04-25T08:46:51.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.013+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c069010 con 0x7fc03c0aeb20 2024-04-25T08:46:51.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.017+0000 7fc032000640 1 -- 172.21.15.46:0/958491927 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc02801c460 con 0x7fc03c0aeb20 2024-04-25T08:46:51.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.017+0000 7fc032000640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc01c066f60 0x7fc01c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.017+0000 7fc032000640 1 -- 172.21.15.46:0/958491927 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc028084f80 con 0x7fc03c0aeb20 2024-04-25T08:46:51.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.017+0000 7fc039e00640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc01c066f60 0x7fc01c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.017+0000 7fc039e00640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc01c066f60 0x7fc01c069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0af000 tx=0x7fc02c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:51.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.021+0000 7fc032000640 1 -- 172.21.15.46:0/958491927 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc028055df0 con 0x7fc03c0aeb20 2024-04-25T08:46:51.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.181+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc03c160290 con 0x7fc01c066f60 2024-04-25T08:46:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.189+0000 7fc032000640 1 -- 172.21.15.46:0/958491927 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252006 (secure 0 0 0) 0x7fc03c160290 con 0x7fc01c066f60 2024-04-25T08:46:51.185 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc01c066f60 msgr2=0x7fc01c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc01c066f60 0x7fc01c069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0af000 tx=0x7fc02c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c0aeb20 msgr2=0x7fc03c143d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c0aeb20 0x7fc03c143d80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc02800c450 tx=0x7fc02800c910 comp rx=0 tx=0).stop 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 shutdown_connections 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc01c066f60 0x7fc01c069420 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc03c1450b0 0x7fc03c1442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc03c0aeb20 0x7fc03c143d80 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 --2- 172.21.15.46:0/958491927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc03c06b320 0x7fc03c15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 >> 172.21.15.46:0/958491927 conn(0x7fc03c099a00 msgr2=0x7fc03c06f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 shutdown_connections 2024-04-25T08:46:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.193+0000 7fc03b200640 1 -- 172.21.15.46:0/958491927 wait complete. 2024-04-25T08:46:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-04-25T08:46:50.321980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245528,"kb_used_data":8540,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729192,"statfs":{"total":773094113280,"available":772842692608,"internally_reserved":0,"allocated":8744960,"data_stored":4606786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.648898"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716805+0000","last_change":"2024-04-25T08:46:46.711267+0000","last_active":"2024-04-25T08:46:46.716805+0000","last_peered":"2024-04-25T08:46:46.716805+0000","last_clean":"2024-04-25T08:46:46.716805+0000","last_became_active":"2024-04-25T08:46:46.711006+0000","last_became_peered":"2024-04-25T08:46:46.711006+0000","last_unstale":"2024-04-25T08:46:46.716805+0000","last_undegraded":"2024-04-25T08:46:46.716805+0000","last_fullsized":"2024-04-25T08:46:46.716805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:39.465212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.722045+0000","last_change":"2024-04-25T08:46:46.720962+0000","last_active":"2024-04-25T08:46:46.722045+0000","last_peered":"2024-04-25T08:46:46.722045+0000","last_clean":"2024-04-25T08:46:46.722045+0000","last_became_active":"2024-04-25T08:46:46.720631+0000","last_became_peered":"2024-04-25T08:46:46.720631+0000","last_unstale":"2024-04-25T08:46:46.722045+0000","last_undegraded":"2024-04-25T08:46:46.722045+0000","last_fullsized":"2024-04-25T08:46:46.722045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:32.376940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.723795+0000","last_change":"2024-04-25T08:46:46.711904+0000","last_active":"2024-04-25T08:46:46.723795+0000","last_peered":"2024-04-25T08:46:46.723795+0000","last_clean":"2024-04-25T08:46:46.723795+0000","last_became_active":"2024-04-25T08:46:46.711541+0000","last_became_peered":"2024-04-25T08:46:46.711541+0000","last_unstale":"2024-04-25T08:46:46.723795+0000","last_undegraded":"2024-04-25T08:46:46.723795+0000","last_fullsized":"2024-04-25T08:46:46.723795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:57.505207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717431+0000","last_change":"2024-04-25T08:46:46.711372+0000","last_active":"2024-04-25T08:46:46.717431+0000","last_peered":"2024-04-25T08:46:46.717431+0000","last_clean":"2024-04-25T08:46:46.717431+0000","last_became_active":"2024-04-25T08:46:46.711086+0000","last_became_peered":"2024-04-25T08:46:46.711086+0000","last_unstale":"2024-04-25T08:46:46.717431+0000","last_undegraded":"2024-04-25T08:46:46.717431+0000","last_fullsized":"2024-04-25T08:46:46.717431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:33.858904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.728234+0000","last_change":"2024-04-25T08:46:46.716059+0000","last_active":"2024-04-25T08:46:46.728234+0000","last_peered":"2024-04-25T08:46:46.728234+0000","last_clean":"2024-04-25T08:46:46.728234+0000","last_became_active":"2024-04-25T08:46:46.715574+0000","last_became_peered":"2024-04-25T08:46:46.715574+0000","last_unstale":"2024-04-25T08:46:46.728234+0000","last_undegraded":"2024-04-25T08:46:46.728234+0000","last_fullsized":"2024-04-25T08:46:46.728234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:02.024539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.712270+0000","last_change":"2024-04-25T08:46:46.709523+0000","last_active":"2024-04-25T08:46:46.712270+0000","last_peered":"2024-04-25T08:46:46.712270+0000","last_clean":"2024-04-25T08:46:46.712270+0000","last_became_active":"2024-04-25T08:46:46.709225+0000","last_became_peered":"2024-04-25T08:46:46.709225+0000","last_unstale":"2024-04-25T08:46:46.712270+0000","last_undegraded":"2024-04-25T08:46:46.712270+0000","last_fullsized":"2024-04-25T08:46:46.712270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:32.376940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717577+0000","last_change":"2024-04-25T08:46:46.711107+0000","last_active":"2024-04-25T08:46:46.717577+0000","last_peered":"2024-04-25T08:46:46.717577+0000","last_clean":"2024-04-25T08:46:46.717577+0000","last_became_active":"2024-04-25T08:46:46.709053+0000","last_became_peered":"2024-04-25T08:46:46.709053+0000","last_unstale":"2024-04-25T08:46:46.717577+0000","last_undegraded":"2024-04-25T08:46:46.717577+0000","last_fullsized":"2024-04-25T08:46:46.717577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:36.030392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.724356+0000","last_change":"2024-04-25T08:46:46.711798+0000","last_active":"2024-04-25T08:46:46.724356+0000","last_peered":"2024-04-25T08:46:46.724356+0000","last_clean":"2024-04-25T08:46:46.724356+0000","last_became_active":"2024-04-25T08:46:46.711519+0000","last_became_peered":"2024-04-25T08:46:46.711519+0000","last_unstale":"2024-04-25T08:46:46.724356+0000","last_undegraded":"2024-04-25T08:46:46.724356+0000","last_fullsized":"2024-04-25T08:46:46.724356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:29.106610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716109+0000","last_change":"2024-04-25T08:46:46.711387+0000","last_active":"2024-04-25T08:46:46.716109+0000","last_peered":"2024-04-25T08:46:46.716109+0000","last_clean":"2024-04-25T08:46:46.716109+0000","last_became_active":"2024-04-25T08:46:46.710510+0000","last_became_peered":"2024-04-25T08:46:46.710510+0000","last_unstale":"2024-04-25T08:46:46.716109+0000","last_undegraded":"2024-04-25T08:46:46.716109+0000","last_fullsized":"2024-04-25T08:46:46.716109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.838342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718615+0000","last_change":"2024-04-25T08:46:46.715926+0000","last_active":"2024-04-25T08:46:46.718615+0000","last_peered":"2024-04-25T08:46:46.718615+0000","last_clean":"2024-04-25T08:46:46.718615+0000","last_became_active":"2024-04-25T08:46:46.715646+0000","last_became_peered":"2024-04-25T08:46:46.715646+0000","last_unstale":"2024-04-25T08:46:46.718615+0000","last_undegraded":"2024-04-25T08:46:46.718615+0000","last_fullsized":"2024-04-25T08:46:46.718615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:29.285849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.729653+0000","last_change":"2024-04-25T08:46:46.724934+0000","last_active":"2024-04-25T08:46:46.729653+0000","last_peered":"2024-04-25T08:46:46.729653+0000","last_clean":"2024-04-25T08:46:46.729653+0000","last_became_active":"2024-04-25T08:46:46.724677+0000","last_became_peered":"2024-04-25T08:46:46.724677+0000","last_unstale":"2024-04-25T08:46:46.729653+0000","last_undegraded":"2024-04-25T08:46:46.729653+0000","last_fullsized":"2024-04-25T08:46:46.729653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:43.813241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717884+0000","last_change":"2024-04-25T08:46:46.712588+0000","last_active":"2024-04-25T08:46:46.717884+0000","last_peered":"2024-04-25T08:46:46.717884+0000","last_clean":"2024-04-25T08:46:46.717884+0000","last_became_active":"2024-04-25T08:46:46.712335+0000","last_became_peered":"2024-04-25T08:46:46.712335+0000","last_unstale":"2024-04-25T08:46:46.717884+0000","last_undegraded":"2024-04-25T08:46:46.717884+0000","last_fullsized":"2024-04-25T08:46:46.717884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:19.222006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.728214+0000","last_change":"2024-04-25T08:46:46.715840+0000","last_active":"2024-04-25T08:46:46.728214+0000","last_peered":"2024-04-25T08:46:46.728214+0000","last_clean":"2024-04-25T08:46:46.728214+0000","last_became_active":"2024-04-25T08:46:46.715242+0000","last_became_peered":"2024-04-25T08:46:46.715242+0000","last_unstale":"2024-04-25T08:46:46.728214+0000","last_undegraded":"2024-04-25T08:46:46.728214+0000","last_fullsized":"2024-04-25T08:46:46.728214+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:47.320724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.706167+0000","last_change":"2024-04-25T08:46:46.703079+0000","last_active":"2024-04-25T08:46:46.706167+0000","last_peered":"2024-04-25T08:46:46.706167+0000","last_clean":"2024-04-25T08:46:46.706167+0000","last_became_active":"2024-04-25T08:46:46.702729+0000","last_became_peered":"2024-04-25T08:46:46.702729+0000","last_unstale":"2024-04-25T08:46:46.706167+0000","last_undegraded":"2024-04-25T08:46:46.706167+0000","last_fullsized":"2024-04-25T08:46:46.706167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:20.916999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.722897+0000","last_change":"2024-04-25T08:46:46.721800+0000","last_active":"2024-04-25T08:46:46.722897+0000","last_peered":"2024-04-25T08:46:46.722897+0000","last_clean":"2024-04-25T08:46:46.722897+0000","last_became_active":"2024-04-25T08:46:46.721529+0000","last_became_peered":"2024-04-25T08:46:46.721529+0000","last_unstale":"2024-04-25T08:46:46.722897+0000","last_undegraded":"2024-04-25T08:46:46.722897+0000","last_fullsized":"2024-04-25T08:46:46.722897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:30.964215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.714778+0000","last_change":"2024-04-25T08:46:46.709729+0000","last_active":"2024-04-25T08:46:46.714778+0000","last_peered":"2024-04-25T08:46:46.714778+0000","last_clean":"2024-04-25T08:46:46.714778+0000","last_became_active":"2024-04-25T08:46:46.709347+0000","last_became_peered":"2024-04-25T08:46:46.709347+0000","last_unstale":"2024-04-25T08:46:46.714778+0000","last_undegraded":"2024-04-25T08:46:46.714778+0000","last_fullsized":"2024-04-25T08:46:46.714778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:10.285093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.731528+0000","last_change":"2024-04-25T08:46:46.719370+0000","last_active":"2024-04-25T08:46:46.731528+0000","last_peered":"2024-04-25T08:46:46.731528+0000","last_clean":"2024-04-25T08:46:46.731528+0000","last_became_active":"2024-04-25T08:46:46.719092+0000","last_became_peered":"2024-04-25T08:46:46.719092+0000","last_unstale":"2024-04-25T08:46:46.731528+0000","last_undegraded":"2024-04-25T08:46:46.731528+0000","last_fullsized":"2024-04-25T08:46:46.731528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:52.597766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716318+0000","last_change":"2024-04-25T08:46:46.703403+0000","last_active":"2024-04-25T08:46:46.716318+0000","last_peered":"2024-04-25T08:46:46.716318+0000","last_clean":"2024-04-25T08:46:46.716318+0000","last_became_active":"2024-04-25T08:46:46.703124+0000","last_became_peered":"2024-04-25T08:46:46.703124+0000","last_unstale":"2024-04-25T08:46:46.716318+0000","last_undegraded":"2024-04-25T08:46:46.716318+0000","last_fullsized":"2024-04-25T08:46:46.716318+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.453249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.723140+0000","last_change":"2024-04-25T08:46:46.710178+0000","last_active":"2024-04-25T08:46:46.723140+0000","last_peered":"2024-04-25T08:46:46.723140+0000","last_clean":"2024-04-25T08:46:46.723140+0000","last_became_active":"2024-04-25T08:46:46.709918+0000","last_became_peered":"2024-04-25T08:46:46.709918+0000","last_unstale":"2024-04-25T08:46:46.723140+0000","last_undegraded":"2024-04-25T08:46:46.723140+0000","last_fullsized":"2024-04-25T08:46:46.723140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:30.964215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.727739+0000","last_change":"2024-04-25T08:46:46.723206+0000","last_active":"2024-04-25T08:46:46.727739+0000","last_peered":"2024-04-25T08:46:46.727739+0000","last_clean":"2024-04-25T08:46:46.727739+0000","last_became_active":"2024-04-25T08:46:46.722947+0000","last_became_peered":"2024-04-25T08:46:46.722947+0000","last_unstale":"2024-04-25T08:46:46.727739+0000","last_undegraded":"2024-04-25T08:46:46.727739+0000","last_fullsized":"2024-04-25T08:46:46.727739+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:14.534414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715990+0000","last_change":"2024-04-25T08:46:46.711710+0000","last_active":"2024-04-25T08:46:46.715990+0000","last_peered":"2024-04-25T08:46:46.715990+0000","last_clean":"2024-04-25T08:46:46.715990+0000","last_became_active":"2024-04-25T08:46:46.709735+0000","last_became_peered":"2024-04-25T08:46:46.709735+0000","last_unstale":"2024-04-25T08:46:46.715990+0000","last_undegraded":"2024-04-25T08:46:46.715990+0000","last_fullsized":"2024-04-25T08:46:46.715990+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:31.591586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.724433+0000","last_change":"2024-04-25T08:46:46.712951+0000","last_active":"2024-04-25T08:46:46.724433+0000","last_peered":"2024-04-25T08:46:46.724433+0000","last_clean":"2024-04-25T08:46:46.724433+0000","last_became_active":"2024-04-25T08:46:46.712575+0000","last_became_peered":"2024-04-25T08:46:46.712575+0000","last_unstale":"2024-04-25T08:46:46.724433+0000","last_undegraded":"2024-04-25T08:46:46.724433+0000","last_fullsized":"2024-04-25T08:46:46.724433+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:32.562335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.709855+0000","last_change":"2024-04-25T08:46:46.706281+0000","last_active":"2024-04-25T08:46:46.709855+0000","last_peered":"2024-04-25T08:46:46.709855+0000","last_clean":"2024-04-25T08:46:46.709855+0000","last_became_active":"2024-04-25T08:46:46.705982+0000","last_became_peered":"2024-04-25T08:46:46.705982+0000","last_unstale":"2024-04-25T08:46:46.709855+0000","last_undegraded":"2024-04-25T08:46:46.709855+0000","last_fullsized":"2024-04-25T08:46:46.709855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:22.339722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719232+0000","last_change":"2024-04-25T08:46:46.715308+0000","last_active":"2024-04-25T08:46:46.719232+0000","last_peered":"2024-04-25T08:46:46.719232+0000","last_clean":"2024-04-25T08:46:46.719232+0000","last_became_active":"2024-04-25T08:46:46.715016+0000","last_became_peered":"2024-04-25T08:46:46.715016+0000","last_unstale":"2024-04-25T08:46:46.719232+0000","last_undegraded":"2024-04-25T08:46:46.719232+0000","last_fullsized":"2024-04-25T08:46:46.719232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:54.427003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.720942+0000","last_change":"2024-04-25T08:46:46.719982+0000","last_active":"2024-04-25T08:46:46.720942+0000","last_peered":"2024-04-25T08:46:46.720942+0000","last_clean":"2024-04-25T08:46:46.720942+0000","last_became_active":"2024-04-25T08:46:46.719626+0000","last_became_peered":"2024-04-25T08:46:46.719626+0000","last_unstale":"2024-04-25T08:46:46.720942+0000","last_undegraded":"2024-04-25T08:46:46.720942+0000","last_fullsized":"2024-04-25T08:46:46.720942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:20.916999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717279+0000","last_change":"2024-04-25T08:46:46.709590+0000","last_active":"2024-04-25T08:46:46.717279+0000","last_peered":"2024-04-25T08:46:46.717279+0000","last_clean":"2024-04-25T08:46:46.717279+0000","last_became_active":"2024-04-25T08:46:46.709309+0000","last_became_peered":"2024-04-25T08:46:46.709309+0000","last_unstale":"2024-04-25T08:46:46.717279+0000","last_undegraded":"2024-04-25T08:46:46.717279+0000","last_fullsized":"2024-04-25T08:46:46.717279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:59.860963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718247+0000","last_change":"2024-04-25T08:46:46.712249+0000","last_active":"2024-04-25T08:46:46.718247+0000","last_peered":"2024-04-25T08:46:46.718247+0000","last_clean":"2024-04-25T08:46:46.718247+0000","last_became_active":"2024-04-25T08:46:46.711926+0000","last_became_peered":"2024-04-25T08:46:46.711926+0000","last_unstale":"2024-04-25T08:46:46.718247+0000","last_undegraded":"2024-04-25T08:46:46.718247+0000","last_fullsized":"2024-04-25T08:46:46.718247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:36.534633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.723888+0000","last_change":"2024-04-25T08:46:46.707846+0000","last_active":"2024-04-25T08:46:46.723888+0000","last_peered":"2024-04-25T08:46:46.723888+0000","last_clean":"2024-04-25T08:46:46.723888+0000","last_became_active":"2024-04-25T08:46:46.707441+0000","last_became_peered":"2024-04-25T08:46:46.707441+0000","last_unstale":"2024-04-25T08:46:46.723888+0000","last_undegraded":"2024-04-25T08:46:46.723888+0000","last_fullsized":"2024-04-25T08:46:46.723888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:00.424654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719767+0000","last_change":"2024-04-25T08:46:46.715739+0000","last_active":"2024-04-25T08:46:46.719767+0000","last_peered":"2024-04-25T08:46:46.719767+0000","last_clean":"2024-04-25T08:46:46.719767+0000","last_became_active":"2024-04-25T08:46:46.715424+0000","last_became_peered":"2024-04-25T08:46:46.715424+0000","last_unstale":"2024-04-25T08:46:46.719767+0000","last_undegraded":"2024-04-25T08:46:46.719767+0000","last_fullsized":"2024-04-25T08:46:46.719767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:35.224653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718224+0000","last_change":"2024-04-25T08:46:46.712243+0000","last_active":"2024-04-25T08:46:46.718224+0000","last_peered":"2024-04-25T08:46:46.718224+0000","last_clean":"2024-04-25T08:46:46.718224+0000","last_became_active":"2024-04-25T08:46:46.711890+0000","last_became_peered":"2024-04-25T08:46:46.711890+0000","last_unstale":"2024-04-25T08:46:46.718224+0000","last_undegraded":"2024-04-25T08:46:46.718224+0000","last_fullsized":"2024-04-25T08:46:46.718224+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:00.394910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.708805+0000","last_change":"2024-04-25T08:46:46.697110+0000","last_active":"2024-04-25T08:46:46.708805+0000","last_peered":"2024-04-25T08:46:46.708805+0000","last_clean":"2024-04-25T08:46:46.708805+0000","last_became_active":"2024-04-25T08:46:46.696803+0000","last_became_peered":"2024-04-25T08:46:46.696803+0000","last_unstale":"2024-04-25T08:46:46.708805+0000","last_undegraded":"2024-04-25T08:46:46.708805+0000","last_fullsized":"2024-04-25T08:46:46.708805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.838342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718233+0000","last_change":"2024-04-25T08:46:46.713492+0000","last_active":"2024-04-25T08:46:46.718233+0000","last_peered":"2024-04-25T08:46:46.718233+0000","last_clean":"2024-04-25T08:46:46.718233+0000","last_became_active":"2024-04-25T08:46:46.713135+0000","last_became_peered":"2024-04-25T08:46:46.713135+0000","last_unstale":"2024-04-25T08:46:46.718233+0000","last_undegraded":"2024-04-25T08:46:46.718233+0000","last_fullsized":"2024-04-25T08:46:46.718233+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:01.738098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.708158+0000","last_change":"2024-04-25T08:46:46.699051+0000","last_active":"2024-04-25T08:46:46.708158+0000","last_peered":"2024-04-25T08:46:46.708158+0000","last_clean":"2024-04-25T08:46:46.708158+0000","last_became_active":"2024-04-25T08:46:46.698756+0000","last_became_peered":"2024-04-25T08:46:46.698756+0000","last_unstale":"2024-04-25T08:46:46.708158+0000","last_undegraded":"2024-04-25T08:46:46.708158+0000","last_fullsized":"2024-04-25T08:46:46.708158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:31.591586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"77'4","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.737123+0000","last_change":"2024-04-25T08:46:45.704820+0000","last_active":"2024-04-25T08:46:46.737123+0000","last_peered":"2024-04-25T08:46:46.737123+0000","last_clean":"2024-04-25T08:46:46.737123+0000","last_became_active":"2024-04-25T08:46:45.704544+0000","last_became_peered":"2024-04-25T08:46:45.704544+0000","last_unstale":"2024-04-25T08:46:46.737123+0000","last_undegraded":"2024-04-25T08:46:46.737123+0000","last_fullsized":"2024-04-25T08:46:46.737123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:01.000996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.713865+0000","last_change":"2024-04-25T08:46:46.710904+0000","last_active":"2024-04-25T08:46:46.713865+0000","last_peered":"2024-04-25T08:46:46.713865+0000","last_clean":"2024-04-25T08:46:46.713865+0000","last_became_active":"2024-04-25T08:46:46.710616+0000","last_became_peered":"2024-04-25T08:46:46.710616+0000","last_unstale":"2024-04-25T08:46:46.713865+0000","last_undegraded":"2024-04-25T08:46:46.713865+0000","last_fullsized":"2024-04-25T08:46:46.713865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:01.738098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.731025+0000","last_change":"2024-04-25T08:46:45.702752+0000","last_active":"2024-04-25T08:46:46.731025+0000","last_peered":"2024-04-25T08:46:46.731025+0000","last_clean":"2024-04-25T08:46:46.731025+0000","last_became_active":"2024-04-25T08:46:45.702438+0000","last_became_peered":"2024-04-25T08:46:45.702438+0000","last_unstale":"2024-04-25T08:46:46.731025+0000","last_undegraded":"2024-04-25T08:46:46.731025+0000","last_fullsized":"2024-04-25T08:46:46.731025+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:32.853001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.709315+0000","last_change":"2024-04-25T08:46:46.704059+0000","last_active":"2024-04-25T08:46:46.709315+0000","last_peered":"2024-04-25T08:46:46.709315+0000","last_clean":"2024-04-25T08:46:46.709315+0000","last_became_active":"2024-04-25T08:46:46.703781+0000","last_became_peered":"2024-04-25T08:46:46.703781+0000","last_unstale":"2024-04-25T08:46:46.709315+0000","last_undegraded":"2024-04-25T08:46:46.709315+0000","last_fullsized":"2024-04-25T08:46:46.709315+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:06.549872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733047+0000","last_change":"2024-04-25T08:46:45.703488+0000","last_active":"2024-04-25T08:46:46.733047+0000","last_peered":"2024-04-25T08:46:46.733047+0000","last_clean":"2024-04-25T08:46:46.733047+0000","last_became_active":"2024-04-25T08:46:45.703159+0000","last_became_peered":"2024-04-25T08:46:45.703159+0000","last_unstale":"2024-04-25T08:46:46.733047+0000","last_undegraded":"2024-04-25T08:46:46.733047+0000","last_fullsized":"2024-04-25T08:46:46.733047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:55.545720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715163+0000","last_change":"2024-04-25T08:46:46.709942+0000","last_active":"2024-04-25T08:46:46.715163+0000","last_peered":"2024-04-25T08:46:46.715163+0000","last_clean":"2024-04-25T08:46:46.715163+0000","last_became_active":"2024-04-25T08:46:46.709636+0000","last_became_peered":"2024-04-25T08:46:46.709636+0000","last_unstale":"2024-04-25T08:46:46.715163+0000","last_undegraded":"2024-04-25T08:46:46.715163+0000","last_fullsized":"2024-04-25T08:46:46.715163+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:36.290118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733408+0000","last_change":"2024-04-25T08:46:45.699880+0000","last_active":"2024-04-25T08:46:46.733408+0000","last_peered":"2024-04-25T08:46:46.733408+0000","last_clean":"2024-04-25T08:46:46.733408+0000","last_became_active":"2024-04-25T08:46:45.698006+0000","last_became_peered":"2024-04-25T08:46:45.698006+0000","last_unstale":"2024-04-25T08:46:46.733408+0000","last_undegraded":"2024-04-25T08:46:46.733408+0000","last_fullsized":"2024-04-25T08:46:46.733408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:35.284216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715082+0000","last_change":"2024-04-25T08:46:46.711119+0000","last_active":"2024-04-25T08:46:46.715082+0000","last_peered":"2024-04-25T08:46:46.715082+0000","last_clean":"2024-04-25T08:46:46.715082+0000","last_became_active":"2024-04-25T08:46:46.710851+0000","last_became_peered":"2024-04-25T08:46:46.710851+0000","last_unstale":"2024-04-25T08:46:46.715082+0000","last_undegraded":"2024-04-25T08:46:46.715082+0000","last_fullsized":"2024-04-25T08:46:46.715082+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:36.534633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733080+0000","last_change":"2024-04-25T08:46:45.700271+0000","last_active":"2024-04-25T08:46:46.733080+0000","last_peered":"2024-04-25T08:46:46.733080+0000","last_clean":"2024-04-25T08:46:46.733080+0000","last_became_active":"2024-04-25T08:46:45.699926+0000","last_became_peered":"2024-04-25T08:46:45.699926+0000","last_unstale":"2024-04-25T08:46:46.733080+0000","last_undegraded":"2024-04-25T08:46:46.733080+0000","last_fullsized":"2024-04-25T08:46:46.733080+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:41.735775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.726649+0000","last_change":"2024-04-25T08:46:46.724686+0000","last_active":"2024-04-25T08:46:46.726649+0000","last_peered":"2024-04-25T08:46:46.726649+0000","last_clean":"2024-04-25T08:46:46.726649+0000","last_became_active":"2024-04-25T08:46:46.724401+0000","last_became_peered":"2024-04-25T08:46:46.724401+0000","last_unstale":"2024-04-25T08:46:46.726649+0000","last_undegraded":"2024-04-25T08:46:46.726649+0000","last_fullsized":"2024-04-25T08:46:46.726649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:42.697992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.730449+0000","last_change":"2024-04-25T08:46:45.700937+0000","last_active":"2024-04-25T08:46:46.730449+0000","last_peered":"2024-04-25T08:46:46.730449+0000","last_clean":"2024-04-25T08:46:46.730449+0000","last_became_active":"2024-04-25T08:46:45.700635+0000","last_became_peered":"2024-04-25T08:46:45.700635+0000","last_unstale":"2024-04-25T08:46:46.730449+0000","last_undegraded":"2024-04-25T08:46:46.730449+0000","last_fullsized":"2024-04-25T08:46:46.730449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:26.219846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715929+0000","last_change":"2024-04-25T08:46:46.710953+0000","last_active":"2024-04-25T08:46:46.715929+0000","last_peered":"2024-04-25T08:46:46.715929+0000","last_clean":"2024-04-25T08:46:46.715929+0000","last_became_active":"2024-04-25T08:46:46.710553+0000","last_became_peered":"2024-04-25T08:46:46.710553+0000","last_unstale":"2024-04-25T08:46:46.715929+0000","last_undegraded":"2024-04-25T08:46:46.715929+0000","last_fullsized":"2024-04-25T08:46:46.715929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:31.364562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.703115+0000","last_change":"2024-04-25T08:46:45.701649+0000","last_active":"2024-04-25T08:46:46.703115+0000","last_peered":"2024-04-25T08:46:46.703115+0000","last_clean":"2024-04-25T08:46:46.703115+0000","last_became_active":"2024-04-25T08:46:45.701383+0000","last_became_peered":"2024-04-25T08:46:45.701383+0000","last_unstale":"2024-04-25T08:46:46.703115+0000","last_undegraded":"2024-04-25T08:46:46.703115+0000","last_fullsized":"2024-04-25T08:46:46.703115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:38.459309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717089+0000","last_change":"2024-04-25T08:46:46.708138+0000","last_active":"2024-04-25T08:46:46.717089+0000","last_peered":"2024-04-25T08:46:46.717089+0000","last_clean":"2024-04-25T08:46:46.717089+0000","last_became_active":"2024-04-25T08:46:46.707846+0000","last_became_peered":"2024-04-25T08:46:46.707846+0000","last_unstale":"2024-04-25T08:46:46.717089+0000","last_undegraded":"2024-04-25T08:46:46.717089+0000","last_fullsized":"2024-04-25T08:46:46.717089+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:56.727233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.698034+0000","last_change":"2024-04-25T08:46:45.705976+0000","last_active":"2024-04-25T08:46:46.698034+0000","last_peered":"2024-04-25T08:46:46.698034+0000","last_clean":"2024-04-25T08:46:46.698034+0000","last_became_active":"2024-04-25T08:46:45.705667+0000","last_became_peered":"2024-04-25T08:46:45.705667+0000","last_unstale":"2024-04-25T08:46:46.698034+0000","last_undegraded":"2024-04-25T08:46:46.698034+0000","last_fullsized":"2024-04-25T08:46:46.698034+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:51.565014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719257+0000","last_change":"2024-04-25T08:46:46.718095+0000","last_active":"2024-04-25T08:46:46.719257+0000","last_peered":"2024-04-25T08:46:46.719257+0000","last_clean":"2024-04-25T08:46:46.719257+0000","last_became_active":"2024-04-25T08:46:46.717803+0000","last_became_peered":"2024-04-25T08:46:46.717803+0000","last_unstale":"2024-04-25T08:46:46.719257+0000","last_undegraded":"2024-04-25T08:46:46.719257+0000","last_fullsized":"2024-04-25T08:46:46.719257+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.701065+0000","last_change":"2024-04-25T08:46:45.702418+0000","last_active":"2024-04-25T08:46:46.701065+0000","last_peered":"2024-04-25T08:46:46.701065+0000","last_clean":"2024-04-25T08:46:46.701065+0000","last_became_active":"2024-04-25T08:46:45.701693+0000","last_became_peered":"2024-04-25T08:46:45.701693+0000","last_unstale":"2024-04-25T08:46:46.701065+0000","last_undegraded":"2024-04-25T08:46:46.701065+0000","last_fullsized":"2024-04-25T08:46:46.701065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:18.216103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.715703+0000","last_change":"2024-04-25T08:46:46.707644+0000","last_active":"2024-04-25T08:46:46.715703+0000","last_peered":"2024-04-25T08:46:46.715703+0000","last_clean":"2024-04-25T08:46:46.715703+0000","last_became_active":"2024-04-25T08:46:46.707234+0000","last_became_peered":"2024-04-25T08:46:46.707234+0000","last_unstale":"2024-04-25T08:46:46.715703+0000","last_undegraded":"2024-04-25T08:46:46.715703+0000","last_fullsized":"2024-04-25T08:46:46.715703+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:45.056546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.701821+0000","last_change":"2024-04-25T08:46:45.700606+0000","last_active":"2024-04-25T08:46:46.701821+0000","last_peered":"2024-04-25T08:46:46.701821+0000","last_clean":"2024-04-25T08:46:46.701821+0000","last_became_active":"2024-04-25T08:46:45.700314+0000","last_became_peered":"2024-04-25T08:46:45.700314+0000","last_unstale":"2024-04-25T08:46:46.701821+0000","last_undegraded":"2024-04-25T08:46:46.701821+0000","last_fullsized":"2024-04-25T08:46:46.701821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:30.489415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.706145+0000","last_change":"2024-04-25T08:46:46.701191+0000","last_active":"2024-04-25T08:46:46.706145+0000","last_peered":"2024-04-25T08:46:46.706145+0000","last_clean":"2024-04-25T08:46:46.706145+0000","last_became_active":"2024-04-25T08:46:46.696650+0000","last_became_peered":"2024-04-25T08:46:46.696650+0000","last_unstale":"2024-04-25T08:46:46.706145+0000","last_undegraded":"2024-04-25T08:46:46.706145+0000","last_fullsized":"2024-04-25T08:46:46.706145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:45.044885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"77'3","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733991+0000","last_change":"2024-04-25T08:46:45.700414+0000","last_active":"2024-04-25T08:46:46.733991+0000","last_peered":"2024-04-25T08:46:46.733991+0000","last_clean":"2024-04-25T08:46:46.733991+0000","last_became_active":"2024-04-25T08:46:45.699873+0000","last_became_peered":"2024-04-25T08:46:45.699873+0000","last_unstale":"2024-04-25T08:46:46.733991+0000","last_undegraded":"2024-04-25T08:46:46.733991+0000","last_fullsized":"2024-04-25T08:46:46.733991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:01.709143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719627+0000","last_change":"2024-04-25T08:46:46.707969+0000","last_active":"2024-04-25T08:46:46.719627+0000","last_peered":"2024-04-25T08:46:46.719627+0000","last_clean":"2024-04-25T08:46:46.719627+0000","last_became_active":"2024-04-25T08:46:46.707622+0000","last_became_peered":"2024-04-25T08:46:46.707622+0000","last_unstale":"2024-04-25T08:46:46.719627+0000","last_undegraded":"2024-04-25T08:46:46.719627+0000","last_fullsized":"2024-04-25T08:46:46.719627+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:57.610010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"77'6","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.741536+0000","last_change":"2024-04-25T08:46:45.691426+0000","last_active":"2024-04-25T08:46:46.741536+0000","last_peered":"2024-04-25T08:46:46.741536+0000","last_clean":"2024-04-25T08:46:46.741536+0000","last_became_active":"2024-04-25T08:46:45.691141+0000","last_became_peered":"2024-04-25T08:46:45.691141+0000","last_unstale":"2024-04-25T08:46:46.741536+0000","last_undegraded":"2024-04-25T08:46:46.741536+0000","last_fullsized":"2024-04-25T08:46:46.741536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:30.358659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717909+0000","last_change":"2024-04-25T08:46:46.714019+0000","last_active":"2024-04-25T08:46:46.717909+0000","last_peered":"2024-04-25T08:46:46.717909+0000","last_clean":"2024-04-25T08:46:46.717909+0000","last_became_active":"2024-04-25T08:46:46.713459+0000","last_became_peered":"2024-04-25T08:46:46.713459+0000","last_unstale":"2024-04-25T08:46:46.717909+0000","last_undegraded":"2024-04-25T08:46:46.717909+0000","last_fullsized":"2024-04-25T08:46:46.717909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:56.551623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.733722+0000","last_change":"2024-04-25T08:46:45.697503+0000","last_active":"2024-04-25T08:46:46.733722+0000","last_peered":"2024-04-25T08:46:46.733722+0000","last_clean":"2024-04-25T08:46:46.733722+0000","last_became_active":"2024-04-25T08:46:45.696801+0000","last_became_peered":"2024-04-25T08:46:45.696801+0000","last_unstale":"2024-04-25T08:46:46.733722+0000","last_undegraded":"2024-04-25T08:46:46.733722+0000","last_fullsized":"2024-04-25T08:46:46.733722+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:35.024489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.702833+0000","last_change":"2024-04-25T08:46:46.697506+0000","last_active":"2024-04-25T08:46:46.702833+0000","last_peered":"2024-04-25T08:46:46.702833+0000","last_clean":"2024-04-25T08:46:46.702833+0000","last_became_active":"2024-04-25T08:46:46.697229+0000","last_became_peered":"2024-04-25T08:46:46.697229+0000","last_unstale":"2024-04-25T08:46:46.702833+0000","last_undegraded":"2024-04-25T08:46:46.702833+0000","last_fullsized":"2024-04-25T08:46:46.702833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.838342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.698524+0000","last_change":"2024-04-25T08:46:45.706187+0000","last_active":"2024-04-25T08:46:46.698524+0000","last_peered":"2024-04-25T08:46:46.698524+0000","last_clean":"2024-04-25T08:46:46.698524+0000","last_became_active":"2024-04-25T08:46:45.705916+0000","last_became_peered":"2024-04-25T08:46:45.705916+0000","last_unstale":"2024-04-25T08:46:46.698524+0000","last_undegraded":"2024-04-25T08:46:46.698524+0000","last_fullsized":"2024-04-25T08:46:46.698524+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:58.855060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717603+0000","last_change":"2024-04-25T08:46:46.708549+0000","last_active":"2024-04-25T08:46:46.717603+0000","last_peered":"2024-04-25T08:46:46.717603+0000","last_clean":"2024-04-25T08:46:46.717603+0000","last_became_active":"2024-04-25T08:46:46.708188+0000","last_became_peered":"2024-04-25T08:46:46.708188+0000","last_unstale":"2024-04-25T08:46:46.717603+0000","last_undegraded":"2024-04-25T08:46:46.717603+0000","last_fullsized":"2024-04-25T08:46:46.717603+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:48.684016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.701263+0000","last_change":"2024-04-25T08:46:45.704805+0000","last_active":"2024-04-25T08:46:46.701263+0000","last_peered":"2024-04-25T08:46:46.701263+0000","last_clean":"2024-04-25T08:46:46.701263+0000","last_became_active":"2024-04-25T08:46:45.704489+0000","last_became_peered":"2024-04-25T08:46:45.704489+0000","last_unstale":"2024-04-25T08:46:46.701263+0000","last_undegraded":"2024-04-25T08:46:46.701263+0000","last_fullsized":"2024-04-25T08:46:46.701263+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:46:37.729671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.714735+0000","last_change":"2024-04-25T08:46:46.705829+0000","last_active":"2024-04-25T08:46:46.714735+0000","last_peered":"2024-04-25T08:46:46.714735+0000","last_clean":"2024-04-25T08:46:46.714735+0000","last_became_active":"2024-04-25T08:46:46.705535+0000","last_became_peered":"2024-04-25T08:46:46.705535+0000","last_unstale":"2024-04-25T08:46:46.714735+0000","last_undegraded":"2024-04-25T08:46:46.714735+0000","last_fullsized":"2024-04-25T08:46:46.714735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:57.108537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.739675+0000","last_change":"2024-04-25T08:46:45.702125+0000","last_active":"2024-04-25T08:46:46.739675+0000","last_peered":"2024-04-25T08:46:46.739675+0000","last_clean":"2024-04-25T08:46:46.739675+0000","last_became_active":"2024-04-25T08:46:45.701183+0000","last_became_peered":"2024-04-25T08:46:45.701183+0000","last_unstale":"2024-04-25T08:46:46.739675+0000","last_undegraded":"2024-04-25T08:46:46.739675+0000","last_fullsized":"2024-04-25T08:46:46.739675+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:59.389007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718973+0000","last_change":"2024-04-25T08:46:46.709579+0000","last_active":"2024-04-25T08:46:46.718973+0000","last_peered":"2024-04-25T08:46:46.718973+0000","last_clean":"2024-04-25T08:46:46.718973+0000","last_became_active":"2024-04-25T08:46:46.708668+0000","last_became_peered":"2024-04-25T08:46:46.708668+0000","last_unstale":"2024-04-25T08:46:46.718973+0000","last_undegraded":"2024-04-25T08:46:46.718973+0000","last_fullsized":"2024-04-25T08:46:46.718973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:43.813241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.732918+0000","last_change":"2024-04-25T08:46:45.702463+0000","last_active":"2024-04-25T08:46:46.732918+0000","last_peered":"2024-04-25T08:46:46.732918+0000","last_clean":"2024-04-25T08:46:46.732918+0000","last_became_active":"2024-04-25T08:46:45.702083+0000","last_became_peered":"2024-04-25T08:46:45.702083+0000","last_unstale":"2024-04-25T08:46:46.732918+0000","last_undegraded":"2024-04-25T08:46:46.732918+0000","last_fullsized":"2024-04-25T08:46:46.732918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:47.085368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.728173+0000","last_change":"2024-04-25T08:46:46.718318+0000","last_active":"2024-04-25T08:46:46.728173+0000","last_peered":"2024-04-25T08:46:46.728173+0000","last_clean":"2024-04-25T08:46:46.728173+0000","last_became_active":"2024-04-25T08:46:46.718059+0000","last_became_peered":"2024-04-25T08:46:46.718059+0000","last_unstale":"2024-04-25T08:46:46.728173+0000","last_undegraded":"2024-04-25T08:46:46.728173+0000","last_fullsized":"2024-04-25T08:46:46.728173+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:06.495163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.731330+0000","last_change":"2024-04-25T08:46:45.686160+0000","last_active":"2024-04-25T08:46:46.731330+0000","last_peered":"2024-04-25T08:46:46.731330+0000","last_clean":"2024-04-25T08:46:46.731330+0000","last_became_active":"2024-04-25T08:46:45.685831+0000","last_became_peered":"2024-04-25T08:46:45.685831+0000","last_unstale":"2024-04-25T08:46:46.731330+0000","last_undegraded":"2024-04-25T08:46:46.731330+0000","last_fullsized":"2024-04-25T08:46:46.731330+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:58.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.724412+0000","last_change":"2024-04-25T08:46:46.712349+0000","last_active":"2024-04-25T08:46:46.724412+0000","last_peered":"2024-04-25T08:46:46.724412+0000","last_clean":"2024-04-25T08:46:46.724412+0000","last_became_active":"2024-04-25T08:46:46.712086+0000","last_became_peered":"2024-04-25T08:46:46.712086+0000","last_unstale":"2024-04-25T08:46:46.724412+0000","last_undegraded":"2024-04-25T08:46:46.724412+0000","last_fullsized":"2024-04-25T08:46:46.724412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:23.391791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"77'1","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.736318+0000","last_change":"2024-04-25T08:46:45.706507+0000","last_active":"2024-04-25T08:46:46.736318+0000","last_peered":"2024-04-25T08:46:46.736318+0000","last_clean":"2024-04-25T08:46:46.736318+0000","last_became_active":"2024-04-25T08:46:45.706179+0000","last_became_peered":"2024-04-25T08:46:45.706179+0000","last_unstale":"2024-04-25T08:46:46.736318+0000","last_undegraded":"2024-04-25T08:46:46.736318+0000","last_fullsized":"2024-04-25T08:46:46.736318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:56.139746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.719661+0000","last_change":"2024-04-25T08:46:46.712042+0000","last_active":"2024-04-25T08:46:46.719661+0000","last_peered":"2024-04-25T08:46:46.719661+0000","last_clean":"2024-04-25T08:46:46.719661+0000","last_became_active":"2024-04-25T08:46:46.711738+0000","last_became_peered":"2024-04-25T08:46:46.711738+0000","last_unstale":"2024-04-25T08:46:46.719661+0000","last_undegraded":"2024-04-25T08:46:46.719661+0000","last_fullsized":"2024-04-25T08:46:46.719661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:42.697992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.690357+0000","last_change":"2024-04-25T08:46:45.698601+0000","last_active":"2024-04-25T08:46:46.690357+0000","last_peered":"2024-04-25T08:46:46.690357+0000","last_clean":"2024-04-25T08:46:46.690357+0000","last_became_active":"2024-04-25T08:46:45.698298+0000","last_became_peered":"2024-04-25T08:46:45.698298+0000","last_unstale":"2024-04-25T08:46:46.690357+0000","last_undegraded":"2024-04-25T08:46:46.690357+0000","last_fullsized":"2024-04-25T08:46:46.690357+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:30.358659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.717937+0000","last_change":"2024-04-25T08:46:46.716551+0000","last_active":"2024-04-25T08:46:46.717937+0000","last_peered":"2024-04-25T08:46:46.717937+0000","last_clean":"2024-04-25T08:46:46.717937+0000","last_became_active":"2024-04-25T08:46:46.716288+0000","last_became_peered":"2024-04-25T08:46:46.716288+0000","last_unstale":"2024-04-25T08:46:46.717937+0000","last_undegraded":"2024-04-25T08:46:46.717937+0000","last_fullsized":"2024-04-25T08:46:46.717937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:46.210853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.736501+0000","last_change":"2024-04-25T08:46:45.693253+0000","last_active":"2024-04-25T08:46:46.736501+0000","last_peered":"2024-04-25T08:46:46.736501+0000","last_clean":"2024-04-25T08:46:46.736501+0000","last_became_active":"2024-04-25T08:46:45.692001+0000","last_became_peered":"2024-04-25T08:46:45.692001+0000","last_unstale":"2024-04-25T08:46:46.736501+0000","last_undegraded":"2024-04-25T08:46:46.736501+0000","last_fullsized":"2024-04-25T08:46:46.736501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:53.421100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716699+0000","last_change":"2024-04-25T08:46:46.712113+0000","last_active":"2024-04-25T08:46:46.716699+0000","last_peered":"2024-04-25T08:46:46.716699+0000","last_clean":"2024-04-25T08:46:46.716699+0000","last_became_active":"2024-04-25T08:46:46.711825+0000","last_became_peered":"2024-04-25T08:46:46.711825+0000","last_unstale":"2024-04-25T08:46:46.716699+0000","last_undegraded":"2024-04-25T08:46:46.716699+0000","last_fullsized":"2024-04-25T08:46:46.716699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:14.347016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.696494+0000","last_change":"2024-04-25T08:46:45.695903+0000","last_active":"2024-04-25T08:46:46.696494+0000","last_peered":"2024-04-25T08:46:46.696494+0000","last_clean":"2024-04-25T08:46:46.696494+0000","last_became_active":"2024-04-25T08:46:45.695603+0000","last_became_peered":"2024-04-25T08:46:45.695603+0000","last_unstale":"2024-04-25T08:46:46.696494+0000","last_undegraded":"2024-04-25T08:46:46.696494+0000","last_fullsized":"2024-04-25T08:46:46.696494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:34.218750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.714761+0000","last_change":"2024-04-25T08:46:46.706428+0000","last_active":"2024-04-25T08:46:46.714761+0000","last_peered":"2024-04-25T08:46:46.714761+0000","last_clean":"2024-04-25T08:46:46.714761+0000","last_became_active":"2024-04-25T08:46:46.706135+0000","last_became_peered":"2024-04-25T08:46:46.706135+0000","last_unstale":"2024-04-25T08:46:46.714761+0000","last_undegraded":"2024-04-25T08:46:46.714761+0000","last_fullsized":"2024-04-25T08:46:46.714761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:57.610010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.696170+0000","last_change":"2024-04-25T08:46:45.699319+0000","last_active":"2024-04-25T08:46:46.696170+0000","last_peered":"2024-04-25T08:46:46.696170+0000","last_clean":"2024-04-25T08:46:46.696170+0000","last_became_active":"2024-04-25T08:46:45.699066+0000","last_became_peered":"2024-04-25T08:46:45.699066+0000","last_unstale":"2024-04-25T08:46:46.696170+0000","last_undegraded":"2024-04-25T08:46:46.696170+0000","last_fullsized":"2024-04-25T08:46:46.696170+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:08.537484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.706625+0000","last_change":"2024-04-25T08:46:46.700693+0000","last_active":"2024-04-25T08:46:46.706625+0000","last_peered":"2024-04-25T08:46:46.706625+0000","last_clean":"2024-04-25T08:46:46.706625+0000","last_became_active":"2024-04-25T08:46:46.700411+0000","last_became_peered":"2024-04-25T08:46:46.700411+0000","last_unstale":"2024-04-25T08:46:46.706625+0000","last_undegraded":"2024-04-25T08:46:46.706625+0000","last_fullsized":"2024-04-25T08:46:46.706625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:53.477544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"77'2","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.731335+0000","last_change":"2024-04-25T08:46:45.694939+0000","last_active":"2024-04-25T08:46:46.731335+0000","last_peered":"2024-04-25T08:46:46.731335+0000","last_clean":"2024-04-25T08:46:46.731335+0000","last_became_active":"2024-04-25T08:46:45.692914+0000","last_became_peered":"2024-04-25T08:46:45.692914+0000","last_unstale":"2024-04-25T08:46:46.731335+0000","last_undegraded":"2024-04-25T08:46:46.731335+0000","last_fullsized":"2024-04-25T08:46:46.731335+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:13.637793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.714850+0000","last_change":"2024-04-25T08:46:46.703345+0000","last_active":"2024-04-25T08:46:46.714850+0000","last_peered":"2024-04-25T08:46:46.714850+0000","last_clean":"2024-04-25T08:46:46.714850+0000","last_became_active":"2024-04-25T08:46:46.703073+0000","last_became_peered":"2024-04-25T08:46:46.703073+0000","last_unstale":"2024-04-25T08:46:46.714850+0000","last_undegraded":"2024-04-25T08:46:46.714850+0000","last_fullsized":"2024-04-25T08:46:46.714850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.453249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.702175+0000","last_change":"2024-04-25T08:46:45.707691+0000","last_active":"2024-04-25T08:46:46.702175+0000","last_peered":"2024-04-25T08:46:46.702175+0000","last_clean":"2024-04-25T08:46:46.702175+0000","last_became_active":"2024-04-25T08:46:45.706884+0000","last_became_peered":"2024-04-25T08:46:45.706884+0000","last_unstale":"2024-04-25T08:46:46.702175+0000","last_undegraded":"2024-04-25T08:46:46.702175+0000","last_fullsized":"2024-04-25T08:46:46.702175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.279190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.709185+0000","last_change":"2024-04-25T08:46:46.703979+0000","last_active":"2024-04-25T08:46:46.709185+0000","last_peered":"2024-04-25T08:46:46.709185+0000","last_clean":"2024-04-25T08:46:46.709185+0000","last_became_active":"2024-04-25T08:46:46.703654+0000","last_became_peered":"2024-04-25T08:46:46.703654+0000","last_unstale":"2024-04-25T08:46:46.709185+0000","last_undegraded":"2024-04-25T08:46:46.709185+0000","last_fullsized":"2024-04-25T08:46:46.709185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:03.564585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.702509+0000","last_change":"2024-04-25T08:46:45.700854+0000","last_active":"2024-04-25T08:46:46.702509+0000","last_peered":"2024-04-25T08:46:46.702509+0000","last_clean":"2024-04-25T08:46:46.702509+0000","last_became_active":"2024-04-25T08:46:45.700537+0000","last_became_peered":"2024-04-25T08:46:45.700537+0000","last_unstale":"2024-04-25T08:46:46.702509+0000","last_undegraded":"2024-04-25T08:46:46.702509+0000","last_fullsized":"2024-04-25T08:46:46.702509+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:28.279946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716933+0000","last_change":"2024-04-25T08:46:46.713274+0000","last_active":"2024-04-25T08:46:46.716933+0000","last_peered":"2024-04-25T08:46:46.716933+0000","last_clean":"2024-04-25T08:46:46.716933+0000","last_became_active":"2024-04-25T08:46:46.712971+0000","last_became_peered":"2024-04-25T08:46:46.712971+0000","last_unstale":"2024-04-25T08:46:46.716933+0000","last_undegraded":"2024-04-25T08:46:46.716933+0000","last_fullsized":"2024-04-25T08:46:46.716933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:09.543387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.697978+0000","last_change":"2024-04-25T08:46:45.701102+0000","last_active":"2024-04-25T08:46:46.697978+0000","last_peered":"2024-04-25T08:46:46.697978+0000","last_clean":"2024-04-25T08:46:46.697978+0000","last_became_active":"2024-04-25T08:46:45.700781+0000","last_became_peered":"2024-04-25T08:46:45.700781+0000","last_unstale":"2024-04-25T08:46:46.697978+0000","last_undegraded":"2024-04-25T08:46:46.697978+0000","last_fullsized":"2024-04-25T08:46:46.697978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:44.050643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.708393+0000","last_change":"2024-04-25T08:46:46.699835+0000","last_active":"2024-04-25T08:46:46.708393+0000","last_peered":"2024-04-25T08:46:46.708393+0000","last_clean":"2024-04-25T08:46:46.708393+0000","last_became_active":"2024-04-25T08:46:46.698199+0000","last_became_peered":"2024-04-25T08:46:46.698199+0000","last_unstale":"2024-04-25T08:46:46.708393+0000","last_undegraded":"2024-04-25T08:46:46.708393+0000","last_fullsized":"2024-04-25T08:46:46.708393+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:31.591586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.695742+0000","last_change":"2024-04-25T08:46:45.698193+0000","last_active":"2024-04-25T08:46:46.695742+0000","last_peered":"2024-04-25T08:46:46.695742+0000","last_clean":"2024-04-25T08:46:46.695742+0000","last_became_active":"2024-04-25T08:46:45.697913+0000","last_became_peered":"2024-04-25T08:46:45.697913+0000","last_unstale":"2024-04-25T08:46:46.695742+0000","last_undegraded":"2024-04-25T08:46:46.695742+0000","last_fullsized":"2024-04-25T08:46:46.695742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:21.333819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.716962+0000","last_change":"2024-04-25T08:46:46.708017+0000","last_active":"2024-04-25T08:46:46.716962+0000","last_peered":"2024-04-25T08:46:46.716962+0000","last_clean":"2024-04-25T08:46:46.716962+0000","last_became_active":"2024-04-25T08:46:46.707706+0000","last_became_peered":"2024-04-25T08:46:46.707706+0000","last_unstale":"2024-04-25T08:46:46.716962+0000","last_undegraded":"2024-04-25T08:46:46.716962+0000","last_fullsized":"2024-04-25T08:46:46.716962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:57.505207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.702236+0000","last_change":"2024-04-25T08:46:45.705442+0000","last_active":"2024-04-25T08:46:46.702236+0000","last_peered":"2024-04-25T08:46:46.702236+0000","last_clean":"2024-04-25T08:46:46.702236+0000","last_became_active":"2024-04-25T08:46:45.705139+0000","last_became_peered":"2024-04-25T08:46:45.705139+0000","last_unstale":"2024-04-25T08:46:46.702236+0000","last_undegraded":"2024-04-25T08:46:46.702236+0000","last_fullsized":"2024-04-25T08:46:46.702236+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:39.370138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.723836+0000","last_change":"2024-04-25T08:46:46.712747+0000","last_active":"2024-04-25T08:46:46.723836+0000","last_peered":"2024-04-25T08:46:46.723836+0000","last_clean":"2024-04-25T08:46:46.723836+0000","last_became_active":"2024-04-25T08:46:46.712450+0000","last_became_peered":"2024-04-25T08:46:46.712450+0000","last_unstale":"2024-04-25T08:46:46.723836+0000","last_undegraded":"2024-04-25T08:46:46.723836+0000","last_fullsized":"2024-04-25T08:46:46.723836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:56.727233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.695700+0000","last_change":"2024-04-25T08:46:45.696519+0000","last_active":"2024-04-25T08:46:46.695700+0000","last_peered":"2024-04-25T08:46:46.695700+0000","last_clean":"2024-04-25T08:46:46.695700+0000","last_became_active":"2024-04-25T08:46:45.695784+0000","last_became_peered":"2024-04-25T08:46:45.695784+0000","last_unstale":"2024-04-25T08:46:46.695700+0000","last_undegraded":"2024-04-25T08:46:46.695700+0000","last_fullsized":"2024-04-25T08:46:46.695700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:05.543970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.711596+0000","last_change":"2024-04-25T08:46:46.704676+0000","last_active":"2024-04-25T08:46:46.711596+0000","last_peered":"2024-04-25T08:46:46.711596+0000","last_clean":"2024-04-25T08:46:46.711596+0000","last_became_active":"2024-04-25T08:46:46.704400+0000","last_became_peered":"2024-04-25T08:46:46.704400+0000","last_unstale":"2024-04-25T08:46:46.711596+0000","last_undegraded":"2024-04-25T08:46:46.711596+0000","last_fullsized":"2024-04-25T08:46:46.711596+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:57.108537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.690240+0000","last_change":"2024-04-25T08:46:45.694419+0000","last_active":"2024-04-25T08:46:46.690240+0000","last_peered":"2024-04-25T08:46:46.690240+0000","last_clean":"2024-04-25T08:46:46.690240+0000","last_became_active":"2024-04-25T08:46:45.694105+0000","last_became_peered":"2024-04-25T08:46:45.694105+0000","last_unstale":"2024-04-25T08:46:46.690240+0000","last_undegraded":"2024-04-25T08:46:46.690240+0000","last_fullsized":"2024-04-25T08:46:46.690240+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:35.284216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.718280+0000","last_change":"2024-04-25T08:46:46.714203+0000","last_active":"2024-04-25T08:46:46.718280+0000","last_peered":"2024-04-25T08:46:46.718280+0000","last_clean":"2024-04-25T08:46:46.718280+0000","last_became_active":"2024-04-25T08:46:46.713949+0000","last_became_peered":"2024-04-25T08:46:46.713949+0000","last_unstale":"2024-04-25T08:46:46.718280+0000","last_undegraded":"2024-04-25T08:46:46.718280+0000","last_fullsized":"2024-04-25T08:46:46.718280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:45.672236+0000","last_clean_scrub_stamp":"2024-04-25T08:46:45.672236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:14.347016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.696552+0000","last_change":"2024-04-25T08:46:45.700280+0000","last_active":"2024-04-25T08:46:46.696552+0000","last_peered":"2024-04-25T08:46:46.696552+0000","last_clean":"2024-04-25T08:46:46.696552+0000","last_became_active":"2024-04-25T08:46:45.699959+0000","last_became_peered":"2024-04-25T08:46:45.699959+0000","last_unstale":"2024-04-25T08:46:46.696552+0000","last_undegraded":"2024-04-25T08:46:46.696552+0000","last_fullsized":"2024-04-25T08:46:46.696552+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:46:44.666333+0000","last_clean_scrub_stamp":"2024-04-25T08:46:44.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:45.204950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":275,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T08:46:46.698521+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:46:46.698521+0000","last_peered":"2024-04-25T08:46:46.698521+0000","last_clean":"2024-04-25T08:46:46.698521+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:46:46.698521+0000","last_undegraded":"2024-04-25T08:46:46.698521+0000","last_fullsized":"2024-04-25T08:46:46.698521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1540096,"data_stored":1020051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":929792,"data_stored":422494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":429695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":962560,"data_stored":431377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":430243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":954368,"data_stored":430403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":422494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32680,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339160,"statfs":{"total":96636764160,"available":96603299840,"internally_reserved":0,"allocated":1536000,"data_stored":1020029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:46:51.209 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:46:51.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:46:51.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.401+0000 7f9605800640 1 Processor -- start 2024-04-25T08:46:51.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.401+0000 7f9605800640 1 -- start start 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f9605800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f960015dae0 0x7f9600072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f9605800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 0x7f960006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f9605800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96000b3850 con 0x7f960006ea60 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f9605800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960006e190 con 0x7f960015dae0 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f9605800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96000b3af0 con 0x7f960006e620 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95fea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f960015dae0 0x7f9600072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95ff400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f960006ea60 0x7f96000acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34908/0 (socket says 172.21.15.46:34908) 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95fea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f960015dae0 0x7f9600072600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39752/0 (socket says 172.21.15.46:39752) 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95ff400640 1 -- 172.21.15.46:0/2573475690 learned_addr learned my addr 172.21.15.46:0/2573475690 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95f6000640 1 --2- 172.21.15.46:0/2573475690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 0x7f960006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95fea00640 1 -- 172.21.15.46:0/2573475690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 msgr2=0x7f960006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95fea00640 1 --2- 172.21.15.46:0/2573475690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 0x7f960006bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95fea00640 1 -- 172.21.15.46:0/2573475690 >> v1:172.21.15.46:6789/0 conn(0x7f960006ea60 legacy=0x7f96000acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95fea00640 1 -- 172.21.15.46:0/2573475690 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96000b3dd0 con 0x7f960015dae0 2024-04-25T08:46:51.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95fe000640 1 -- 172.21.15.46:0/2573475690 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758947828 0 0) 0x7f96000b3850 con 0x7f960006ea60 2024-04-25T08:46:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.405+0000 7f95fea00640 1 --2- 172.21.15.46:0/2573475690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 0x7f9600072600 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f960006e310 tx=0x7f95f001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2de01a72f08aa1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f95fe000640 1 -- 172.21.15.46:0/2573475690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00b2070 con 0x7f960015dae0 2024-04-25T08:46:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f95fe000640 1 -- 172.21.15.46:0/2573475690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f00a58f0 con 0x7f960015dae0 2024-04-25T08:46:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f95fe000640 1 -- 172.21.15.46:0/2573475690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f00a5cc0 con 0x7f960015dae0 2024-04-25T08:46:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 -- 172.21.15.46:0/2573475690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 msgr2=0x7f9600072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 --2- 172.21.15.46:0/2573475690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 0x7f9600072600 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f960006e310 tx=0x7f95f001c920 comp rx=0 tx=0).stop 2024-04-25T08:46:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 -- 172.21.15.46:0/2573475690 shutdown_connections 2024-04-25T08:46:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 --2- 172.21.15.46:0/2573475690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 0x7f960006bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 --2- 172.21.15.46:0/2573475690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 0x7f9600072600 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 -- 172.21.15.46:0/2573475690 >> 172.21.15.46:0/2573475690 conn(0x7f9600071ae0 msgr2=0x7f9600071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 -- 172.21.15.46:0/2573475690 shutdown_connections 2024-04-25T08:46:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 -- 172.21.15.46:0/2573475690 wait complete. 2024-04-25T08:46:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 Processor -- start 2024-04-25T08:46:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 -- start start 2024-04-25T08:46:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 0x7f960014a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960006ea60 0x7f9600157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 0x7f96001582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960015dec0 con 0x7f960015dae0 2024-04-25T08:46:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960006c570 con 0x7f960006ea60 2024-04-25T08:46:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f9605800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960006f6d0 con 0x7f960006e620 2024-04-25T08:46:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f95ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 0x7f96001582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f95ff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 0x7f96001582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39760/0 (socket says 172.21.15.46:39760) 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f95ff400640 1 -- 172.21.15.46:0/2845546707 learned_addr learned my addr 172.21.15.46:0/2845546707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f95f6000640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960006ea60 0x7f9600157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.409+0000 7f95fea00640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 0x7f960014a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95ff400640 1 -- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 msgr2=0x7f960014a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95ff400640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 0x7f960014a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95ff400640 1 -- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960006ea60 msgr2=0x7f9600157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95ff400640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960006ea60 0x7f9600157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95ff400640 1 -- 172.21.15.46:0/2845546707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96001547c0 con 0x7f960015dae0 2024-04-25T08:46:51.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95ff400640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 0x7f96001582a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c480 tx=0x7f95f800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95f7400640 1 -- 172.21.15.46:0/2845546707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8018070 con 0x7f960015dae0 2024-04-25T08:46:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95f7400640 1 -- 172.21.15.46:0/2845546707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f80074e0 con 0x7f960015dae0 2024-04-25T08:46:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95f7400640 1 -- 172.21.15.46:0/2845546707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8004a30 con 0x7f960015dae0 2024-04-25T08:46:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9600154200 con 0x7f960015dae0 2024-04-25T08:46:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9600155410 con 0x7f960015dae0 2024-04-25T08:46:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9600069010 con 0x7f960015dae0 2024-04-25T08:46:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95f7400640 1 -- 172.21.15.46:0/2845546707 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95f8004bd0 con 0x7f960015dae0 2024-04-25T08:46:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95f7400640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95dc066f60 0x7f95dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.413+0000 7f95f7400640 1 -- 172.21.15.46:0/2845546707 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95f80844c0 con 0x7f960015dae0 2024-04-25T08:46:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.417+0000 7f95fea00640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95dc066f60 0x7f95dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.417+0000 7f95fea00640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95dc066f60 0x7f95dc069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f95f00af000 tx=0x7f95f00a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:51.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.417+0000 7f95f7400640 1 -- 172.21.15.46:0/2845546707 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95f8054ce0 con 0x7f960015dae0 2024-04-25T08:46:51.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.637+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f960015cc80 con 0x7f960015dae0 2024-04-25T08:46:51.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.641+0000 7f95f7400640 1 -- 172.21.15.46:0/2845546707 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f95f8058d80 con 0x7f960015dae0 2024-04-25T08:46:51.635 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-25T08:46:51.635 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q== 2024-04-25T08:46:51.635 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-25T08:46:51.635 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-25T08:46:51.635 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:46:51.635 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-25T08:46:51.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.641+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95dc066f60 msgr2=0x7f95dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.641+0000 7f9605800640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95dc066f60 0x7f95dc069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f95f00af000 tx=0x7f95f00a7070 comp rx=0 tx=0).stop 2024-04-25T08:46:51.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.641+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 msgr2=0x7f96001582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.641+0000 7f9605800640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 0x7f96001582a0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f95f800c480 tx=0x7f95f800c940 comp rx=0 tx=0).stop 2024-04-25T08:46:51.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.645+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 shutdown_connections 2024-04-25T08:46:51.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.645+0000 7f9605800640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95dc066f60 0x7f95dc069420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.645+0000 7f9605800640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f960015dae0 0x7f96001582a0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.645+0000 7f9605800640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f960006ea60 0x7f9600157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.645+0000 7f9605800640 1 --2- 172.21.15.46:0/2845546707 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f960006e620 0x7f960014a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.645+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 >> 172.21.15.46:0/2845546707 conn(0x7f9600071ae0 msgr2=0x7f96000b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:51.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.645+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 shutdown_connections 2024-04-25T08:46:51.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.645+0000 7f9605800640 1 -- 172.21.15.46:0/2845546707 wait complete. 2024-04-25T08:46:51.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd24b800640 1 Processor -- start 2024-04-25T08:46:51.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd24b800640 1 -- start start 2024-04-25T08:46:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd24b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2440cdc50 0x7fd244166770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd24b800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2440cc700 con 0x7fd244159230 2024-04-25T08:46:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd24b800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2440cc9e0 con 0x7fd24415bdb0 2024-04-25T08:46:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd24b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440cccc0 con 0x7fd2440cdc50 2024-04-25T08:46:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd248e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd244159230 0x7fd2440cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34918/0 (socket says 172.21.15.46:34918) 2024-04-25T08:46:51.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd248e00640 1 -- 172.21.15.46:0/2742920304 learned_addr learned my addr 172.21.15.46:0/2742920304 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd249800640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2440cdc50 0x7fd244166770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678068529 0 0) 0x7fd2440cc700 con 0x7fd244159230 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c003530 con 0x7fd244159230 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033928257 0 0) 0x7fd2440cc9e0 con 0x7fd24415bdb0 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2440cc700 con 0x7fd24415bdb0 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 971525268 0 0) 0x7fd22c003530 con 0x7fd244159230 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2440cdc50 msgr2=0x7fd244166770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2440cdc50 0x7fd244166770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 >> v1:172.21.15.82:6789/0 conn(0x7fd24415bdb0 legacy=0x7fd2440cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440ccfa0 con 0x7fd244159230 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fd238001680 con 0x7fd244159230 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd249800640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2440cdc50 0x7fd244166770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:46:51.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 >> v1:172.21.15.46:6789/0 conn(0x7fd244159230 legacy=0x7fd2440cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd22c003b80 0x7fd22c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd22c004810 0x7fd22c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2440cdc50 0x7fd22c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440cccc0 con 0x7fd22c003b80 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c00bae0 con 0x7fd22c004810 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c00bce0 con 0x7fd2440cdc50 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd248e00640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd22c003b80 0x7fd22c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.805+0000 7fd243e00640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd22c004810 0x7fd22c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd249800640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2440cdc50 0x7fd22c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd248e00640 1 -- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2440cdc50 msgr2=0x7fd22c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd248e00640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd2440cdc50 0x7fd22c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd248e00640 1 -- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd22c004810 msgr2=0x7fd22c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd248e00640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd22c004810 0x7fd22c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd248e00640 1 -- 172.21.15.46:0/2742920304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c003530 con 0x7fd22c003b80 2024-04-25T08:46:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd248e00640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd22c003b80 0x7fd22c003f80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd238001ec0 tx=0x7fd23801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6734c603d6c3dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238007cb0 con 0x7fd22c003b80 2024-04-25T08:46:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2380a9020 con 0x7fd22c003b80 2024-04-25T08:46:51.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd243400640 1 -- 172.21.15.46:0/2742920304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238005b00 con 0x7fd22c003b80 2024-04-25T08:46:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd24b800640 1 -- 172.21.15.46:0/2742920304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd22c003b80 msgr2=0x7fd22c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd24b800640 1 --2- 172.21.15.46:0/2742920304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd22c003b80 0x7fd22c003f80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd238001ec0 tx=0x7fd23801cbc0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd249800640 1 -- 172.21.15.46:0/2742920304 reap_dead start 2024-04-25T08:46:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd24b800640 1 -- 172.21.15.46:0/2742920304 shutdown_connections 2024-04-25T08:46:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd24b800640 1 -- 172.21.15.46:0/2742920304 >> 172.21.15.46:0/2742920304 conn(0x7fd2440b12a0 msgr2=0x7fd2440b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd24b800640 1 -- 172.21.15.46:0/2742920304 shutdown_connections 2024-04-25T08:46:51.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.809+0000 7fd24b800640 1 -- 172.21.15.46:0/2742920304 wait complete. 2024-04-25T08:46:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd24b800640 1 Processor -- start 2024-04-25T08:46:51.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd24b800640 1 -- start start 2024-04-25T08:46:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd24b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd22c003b80 0x7fd24406ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd24b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd244142b20 0x7fd2441581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd24b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd244071250 0x7fd2440716d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd24b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244167310 con 0x7fd244071250 2024-04-25T08:46:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd24b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244167190 con 0x7fd244142b20 2024-04-25T08:46:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd24b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd244167490 con 0x7fd22c003b80 2024-04-25T08:46:51.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd249800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd244071250 0x7fd2440716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd249800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd244071250 0x7fd2440716d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39782/0 (socket says 172.21.15.46:39782) 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd249800640 1 -- 172.21.15.46:0/2899053514 learned_addr learned my addr 172.21.15.46:0/2899053514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd248e00640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd22c003b80 0x7fd24406ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd243e00640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd244142b20 0x7fd2441581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd249800640 1 -- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd22c003b80 msgr2=0x7fd24406ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd249800640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd22c003b80 0x7fd24406ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd249800640 1 -- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd244142b20 msgr2=0x7fd2441581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd249800640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd244142b20 0x7fd2441581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd249800640 1 -- 172.21.15.46:0/2899053514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244143290 con 0x7fd244071250 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd249800640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd244071250 0x7fd2440716d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b2360 tx=0x7fd23c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.813+0000 7fd240c00640 1 -- 172.21.15.46:0/2899053514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0039a0 con 0x7fd244071250 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd240c00640 1 -- 172.21.15.46:0/2899053514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c003b40 con 0x7fd244071250 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd240c00640 1 -- 172.21.15.46:0/2899053514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c004020 con 0x7fd244071250 2024-04-25T08:46:51.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd244143570 con 0x7fd244071250 2024-04-25T08:46:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd244143b30 con 0x7fd244071250 2024-04-25T08:46:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd244069080 con 0x7fd244071250 2024-04-25T08:46:51.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd240c00640 1 -- 172.21.15.46:0/2899053514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd23c0041c0 con 0x7fd244071250 2024-04-25T08:46:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd240c00640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd224066f90 0x7fd224069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd240c00640 1 -- 172.21.15.46:0/2899053514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd23c085e60 con 0x7fd244071250 2024-04-25T08:46:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd248e00640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd224066f90 0x7fd224069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:51.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.817+0000 7fd248e00640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd224066f90 0x7fd224069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2380aa000 tx=0x7fd2440cc8d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:51.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:51.821+0000 7fd240c00640 1 -- 172.21.15.46:0/2899053514 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd23c019030 con 0x7fd244071250 2024-04-25T08:46:52.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2440723d0 con 0x7fd244071250 2024-04-25T08:46:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd240c00640 1 -- 172.21.15.46:0/2899053514 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4937 (secure 0 0 0) 0x7fd23c010040 con 0x7fd244071250 2024-04-25T08:46:52.010 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:52.010 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:46:49.102068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6774},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6774":{"gid":6774,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/960745191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":960745191},{"type":"v1","addr":"172.21.15.46:6835","nonce":960745191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:46:52.010 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 45 2024-04-25T08:46:52.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd224066f90 msgr2=0x7fd224069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd224066f90 0x7fd224069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd2380aa000 tx=0x7fd2440cc8d0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd244071250 msgr2=0x7fd2440716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd244071250 0x7fd2440716d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b2360 tx=0x7fd23c009f90 comp rx=0 tx=0).stop 2024-04-25T08:46:52.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 shutdown_connections 2024-04-25T08:46:52.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd224066f90 0x7fd224069450 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd244071250 0x7fd2440716d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd244142b20 0x7fd2441581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 --2- 172.21.15.46:0/2899053514 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd22c003b80 0x7fd24406ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 >> 172.21.15.46:0/2899053514 conn(0x7fd2440b12a0 msgr2=0x7fd24415c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 shutdown_connections 2024-04-25T08:46:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.017+0000 7fd24b800640 1 -- 172.21.15.46:0/2899053514 wait complete. 2024-04-25T08:46:52.433 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:46:46.077710+0000', 'modified': '2024-04-25T08:46:49.102068+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6774}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6774': {'gid': 6774, 'name': 'c', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/960745191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 960745191}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 960745191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:46:52.433 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:46:52.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:46:52.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498f800640 1 Processor -- start 2024-04-25T08:46:52.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498f800640 1 -- start start 2024-04-25T08:46:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 0x7f49880cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498f800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49880cda80 0x7f498815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498f800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49880cc700 con 0x7f4988159230 2024-04-25T08:46:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498f800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49880cc9e0 con 0x7f49880cda80 2024-04-25T08:46:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49880cccc0 con 0x7f498815bdb0 2024-04-25T08:46:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49880cda80 0x7f498815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49880cda80 0x7f498815d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39798/0 (socket says 172.21.15.46:39798) 2024-04-25T08:46:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498d800640 1 -- 172.21.15.46:0/2765587907 learned_addr learned my addr 172.21.15.46:0/2765587907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f4987e00640 1 --2- 172.21.15.46:0/2765587907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 0x7f49880cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498d800640 1 -- 172.21.15.46:0/2765587907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 msgr2=0x7f49880cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498d800640 1 --2- 172.21.15.46:0/2765587907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 0x7f49880cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498d800640 1 -- 172.21.15.46:0/2765587907 >> v1:172.21.15.82:6789/0 conn(0x7f4988159230 legacy=0x7f49880cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.585+0000 7f498d800640 1 -- 172.21.15.46:0/2765587907 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880ccfa0 con 0x7f49880cda80 2024-04-25T08:46:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498d800640 1 --2- 172.21.15.46:0/2765587907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49880cda80 0x7f498815d200 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4980009e50 tx=0x7f498001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177e7fdb05fe600e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f4987400640 1 -- 172.21.15.46:0/2765587907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800a5660 con 0x7f49880cda80 2024-04-25T08:46:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f4987400640 1 -- 172.21.15.46:0/2765587907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49800a5800 con 0x7f49880cda80 2024-04-25T08:46:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f4987400640 1 -- 172.21.15.46:0/2765587907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800a5bd0 con 0x7f49880cda80 2024-04-25T08:46:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498f800640 1 -- 172.21.15.46:0/2765587907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49880cda80 msgr2=0x7f498815d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498f800640 1 --2- 172.21.15.46:0/2765587907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49880cda80 0x7f498815d200 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4980009e50 tx=0x7f498001c9a0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498f800640 1 -- 172.21.15.46:0/2765587907 shutdown_connections 2024-04-25T08:46:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498f800640 1 --2- 172.21.15.46:0/2765587907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49880cda80 0x7f498815d200 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498f800640 1 --2- 172.21.15.46:0/2765587907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 0x7f49880cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498f800640 1 -- 172.21.15.46:0/2765587907 >> 172.21.15.46:0/2765587907 conn(0x7f49880b12a0 msgr2=0x7f49880b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498f800640 1 -- 172.21.15.46:0/2765587907 shutdown_connections 2024-04-25T08:46:52.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498f800640 1 -- 172.21.15.46:0/2765587907 wait complete. 2024-04-25T08:46:52.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.589+0000 7f498f800640 1 Processor -- start 2024-04-25T08:46:52.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498f800640 1 -- start start 2024-04-25T08:46:52.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49880cda80 0x7f4988140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4988159230 0x7f49880a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 0x7f49880b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498815e2e0 con 0x7f4988159230 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498815e160 con 0x7f49880cda80 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498815e460 con 0x7f498815bdb0 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4987e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4988159230 0x7f49880a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4987e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4988159230 0x7f49880a8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39810/0 (socket says 172.21.15.46:39810) 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4987e00640 1 -- 172.21.15.46:0/4281632832 learned_addr learned my addr 172.21.15.46:0/4281632832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498d800640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 0x7f49880b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498ce00640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49880cda80 0x7f4988140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4987e00640 1 -- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 msgr2=0x7f49880b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4987e00640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 0x7f49880b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4987e00640 1 -- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49880cda80 msgr2=0x7f4988140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4987e00640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49880cda80 0x7f4988140cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4987e00640 1 -- 172.21.15.46:0/4281632832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880b4130 con 0x7f4988159230 2024-04-25T08:46:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4987e00640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4988159230 0x7f49880a8600 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f497400d5f0 tx=0x7f497400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4984c00640 1 -- 172.21.15.46:0/4281632832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740125f0 con 0x7f4988159230 2024-04-25T08:46:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4984c00640 1 -- 172.21.15.46:0/4281632832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4974019070 con 0x7f4988159230 2024-04-25T08:46:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f4984c00640 1 -- 172.21.15.46:0/4281632832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740129d0 con 0x7f4988159230 2024-04-25T08:46:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498f800640 1 -- 172.21.15.46:0/4281632832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49880b4410 con 0x7f4988159230 2024-04-25T08:46:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.593+0000 7f498f800640 1 -- 172.21.15.46:0/4281632832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49880b27a0 con 0x7f4988159230 2024-04-25T08:46:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4984c00640 1 -- 172.21.15.46:0/4281632832 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4974012b70 con 0x7f4988159230 2024-04-25T08:46:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4984c00640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4968066f60 0x7f4968069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4984c00640 1 -- 172.21.15.46:0/4281632832 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4974085660 con 0x7f4988159230 2024-04-25T08:46:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4992d3e000 1 Processor -- start 2024-04-25T08:46:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4968066f60 0x7f4968069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4992d3e000 1 -- start start 2024-04-25T08:46:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4968066f60 0x7f4968069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f49881f9c70 tx=0x7f497c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec30d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 0x55ab7ebe69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 0x55ab7ebe7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4992d3e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7eace5f0 con 0x55ab7ec30920 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4992d3e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7ea35da0 con 0x55ab7ebe6f30 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4992d3e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab7e9be150 con 0x55ab7ec31240 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec30d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec30d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39826/0 (socket says 172.21.15.46:39826) 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 -- 172.21.15.46:0/2508334568 learned_addr learned my addr 172.21.15.46:0/2508334568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498d800640 1 --2- 172.21.15.46:0/2508334568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 0x55ab7ebe7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f4987e00640 1 --2- 172.21.15.46:0/2508334568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 0x55ab7ebe69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 -- 172.21.15.46:0/2508334568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 msgr2=0x55ab7ebe69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 --2- 172.21.15.46:0/2508334568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 0x55ab7ebe69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 -- 172.21.15.46:0/2508334568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 msgr2=0x55ab7ebe7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 --2- 172.21.15.46:0/2508334568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 0x55ab7ebe7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 -- 172.21.15.46:0/2508334568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab7e9d7ca0 con 0x55ab7ec30920 2024-04-25T08:46:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.597+0000 7f498ce00640 1 --2- 172.21.15.46:0/2508334568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec30d00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f497c0052a0 tx=0x7f497c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523a60e8a47857a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4979600640 1 -- 172.21.15.46:0/2508334568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c013630 con 0x55ab7ec30920 2024-04-25T08:46:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4979600640 1 -- 172.21.15.46:0/2508334568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c0137d0 con 0x55ab7ec30920 2024-04-25T08:46:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4979600640 1 -- 172.21.15.46:0/2508334568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c013be0 con 0x55ab7ec30920 2024-04-25T08:46:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- 172.21.15.46:0/2508334568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 msgr2=0x55ab7ec30d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 --2- 172.21.15.46:0/2508334568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec30d00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f497c0052a0 tx=0x7f497c011000 comp rx=0 tx=0).stop 2024-04-25T08:46:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- 172.21.15.46:0/2508334568 shutdown_connections 2024-04-25T08:46:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 --2- 172.21.15.46:0/2508334568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 0x55ab7ebe7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 --2- 172.21.15.46:0/2508334568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 0x55ab7ebe69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 --2- 172.21.15.46:0/2508334568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec30d00 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- 172.21.15.46:0/2508334568 >> 172.21.15.46:0/2508334568 conn(0x55ab7ebed2a0 msgr2=0x55ab7ebe52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- 172.21.15.46:0/2508334568 shutdown_connections 2024-04-25T08:46:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- 172.21.15.46:0/2508334568 wait complete. 2024-04-25T08:46:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- start start 2024-04-25T08:46:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 Processor -- start 2024-04-25T08:46:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 0x55ab7ec37cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec38230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 0x55ab7ebeaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7e9f0990 con 0x55ab7ec30920 2024-04-25T08:46:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7ea31e40 con 0x55ab7ec31240 2024-04-25T08:46:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab7e9c1900 con 0x55ab7ebe6f30 2024-04-25T08:46:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 10 client.0 _ll_register_callbacks cb 0x55ab7eae91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:46:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4987e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec38230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4987e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec38230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39840/0 (socket says 172.21.15.46:39840) 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4987e00640 1 -- 172.21.15.46:0/4065073916 learned_addr learned my addr 172.21.15.46:0/4065073916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f498d800640 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 0x55ab7ebeaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f498ce00640 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 0x55ab7ec37cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4987e00640 1 -- 172.21.15.46:0/4065073916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 msgr2=0x55ab7ec37cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4987e00640 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 0x55ab7ec37cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4987e00640 1 -- 172.21.15.46:0/4065073916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 msgr2=0x55ab7ebeaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4987e00640 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 0x55ab7ebeaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4987e00640 1 -- 172.21.15.46:0/4065073916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab7e9ee630 con 0x55ab7ec30920 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4987e00640 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec38230 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4974055ad0 tx=0x7f4974055f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d775f4e3a482f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f496e000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f496e000640 1 -- 172.21.15.46:0/4065073916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497406d070 con 0x55ab7ec30920 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f496e000640 1 -- 172.21.15.46:0/4065073916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4974052440 con 0x55ab7ec30920 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.601+0000 7f4992d3e000 1 -- 172.21.15.46:0/4065073916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f497c0137d0 con 0x55ab7ec30920 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f496e000640 1 -- 172.21.15.46:0/4065073916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f497405e050 con 0x55ab7ec30920 2024-04-25T08:46:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 10 client.6918 fetch_fsmap learned FSMap version 45 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 -- 172.21.15.46:0/4065073916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x55ab7e9be3a0 con 0x55ab7ec30920 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f496e000640 1 -- 172.21.15.46:0/4065073916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c0137d0 con 0x55ab7ec30920 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f496e000640 1 -- 172.21.15.46:0/4065073916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 45) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4974013070 con 0x55ab7ec30920 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 10 client.6918 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 10 client.6918 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 20 client.6918 populate_metadata read hostname 'smithi046' 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] conn(0x55ab7ebec160 0x55ab7ec41d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 4 client.6918 mds_command: new command op to 6774 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 -- 172.21.15.46:0/4065073916 --> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab7e9d1a50 con 0x55ab7ebec160 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f498ce00640 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] conn(0x55ab7ebec160 0x55ab7ec41d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f498ce00640 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] conn(0x55ab7ebec160 0x55ab7ec41d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25cfbf203d05db0 server_cookie=1fbcce6eca36db4b in_seq=0 out_seq=0 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f496e000640 10 client.6918 ms_handle_connect on v2:172.21.15.46:6833/960745191 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f496e000640 1 -- 172.21.15.46:0/4065073916 <== mds.0 v2:172.21.15.46:6833/960745191 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ab7e9d1a50 con 0x55ab7ebec160 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f496e000640 10 client.6918 handle_command_reply: tid=0 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 client.6918 shutdown 2024-04-25T08:46:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 -- 172.21.15.46:0/4065073916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 msgr2=0x55ab7ec38230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec38230 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4974055ad0 tx=0x7f4974055f10 comp rx=0 tx=0).stop 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 -- 172.21.15.46:0/4065073916 shutdown_connections 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] conn(0x55ab7ebec160 0x55ab7ec41d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 0x55ab7ebeaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec38230 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 --2- 172.21.15.46:0/4065073916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 0x55ab7ec37cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f496e000640 0 client.6918 ms_handle_reset on v2:172.21.15.46:6833/960745191 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 -- 172.21.15.46:0/4065073916 >> 172.21.15.46:0/4065073916 conn(0x55ab7ebed2a0 msgr2=0x55ab7ebe59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 -- 172.21.15.46:0/4065073916 shutdown_connections 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 1 -- 172.21.15.46:0/4065073916 wait complete. 2024-04-25T08:46:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.605+0000 7f4992d3e000 20 client.6918 trim_cache size 0 max 16384 2024-04-25T08:46:52.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4992d3e000 1 Processor -- start 2024-04-25T08:46:52.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4992d3e000 1 -- start start 2024-04-25T08:46:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 0x55ab7ebe9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ebea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 0x55ab7ec3a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4992d3e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49882022f0 con 0x55ab7ec30920 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4992d3e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4988142370 con 0x55ab7ebe6f30 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4992d3e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4988201f60 con 0x55ab7ec31240 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4987e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ebea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4987e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ebea0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39842/0 (socket says 172.21.15.46:39842) 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f4987e00640 1 -- 172.21.15.46:0/1576568453 learned_addr learned my addr 172.21.15.46:0/1576568453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.621+0000 7f498ce00640 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 0x55ab7ebe9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f498d800640 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 0x55ab7ec3a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 -- 172.21.15.46:0/1576568453 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 msgr2=0x55ab7ec3a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 0x55ab7ec3a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 -- 172.21.15.46:0/1576568453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 msgr2=0x55ab7ebe9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 0x55ab7ebe9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 -- 172.21.15.46:0/1576568453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab7ec31070 con 0x55ab7ec30920 2024-04-25T08:46:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f498ce00640 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 0x55ab7ebe9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ebea0f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f49740521e0 tx=0x7f4974058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47fd6ff77fe2ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4978c00640 1 -- 172.21.15.46:0/1576568453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497406d040 con 0x55ab7ec30920 2024-04-25T08:46:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4978c00640 1 -- 172.21.15.46:0/1576568453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49740564d0 con 0x55ab7ec30920 2024-04-25T08:46:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4978c00640 1 -- 172.21.15.46:0/1576568453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497405c0b0 con 0x55ab7ec30920 2024-04-25T08:46:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 -- 172.21.15.46:0/1576568453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 msgr2=0x55ab7ebea0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ebea0f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f49740521e0 tx=0x7f4974058000 comp rx=0 tx=0).stop 2024-04-25T08:46:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 -- 172.21.15.46:0/1576568453 shutdown_connections 2024-04-25T08:46:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ec31240 0x55ab7ec3a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ebea0f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1576568453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ebe6f30 0x55ab7ebe9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 -- 172.21.15.46:0/1576568453 >> 172.21.15.46:0/1576568453 conn(0x55ab7ebed2a0 msgr2=0x55ab7ec2f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 -- 172.21.15.46:0/1576568453 shutdown_connections 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 -- 172.21.15.46:0/1576568453 wait complete. 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 -- start start 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 Processor -- start 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 0x55ab7ec750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 0x55ab7ec75ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7ea3a3d0 con 0x55ab7ec30920 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab7eaf1d50 con 0x55ab7ec31240 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ab7e9d0980 con 0x55ab7ebe6f30 2024-04-25T08:46:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4992d3e000 10 client.0 _ll_register_callbacks cb 0x7f497c001290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec759b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39848/0 (socket says 172.21.15.46:39848) 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 -- 172.21.15.46:0/1293542897 learned_addr learned my addr 172.21.15.46:0/1293542897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f498d800640 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 0x55ab7ec75ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f498ce00640 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 0x55ab7ec750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 -- 172.21.15.46:0/1293542897 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 msgr2=0x55ab7ec750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 0x55ab7ec750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 -- 172.21.15.46:0/1293542897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 msgr2=0x55ab7ec75ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 0x55ab7ec75ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f4987e00640 1 -- 172.21.15.46:0/1293542897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab7eab7600 con 0x55ab7ec30920 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.625+0000 7f498d800640 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 0x55ab7ec75ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4987e00640 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec759b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4974055ad0 tx=0x7f49740838c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d35017c51438b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 1 -- 172.21.15.46:0/1293542897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974064070 con 0x55ab7ec30920 2024-04-25T08:46:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 1 -- 172.21.15.46:0/1293542897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4974055220 con 0x55ab7ec30920 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 -- 172.21.15.46:0/1293542897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49740564d0 con 0x55ab7ec30920 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 1 -- 172.21.15.46:0/1293542897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4974047050 con 0x55ab7ec30920 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 10 client.6924 fetch_fsmap learned FSMap version 45 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 -- 172.21.15.46:0/1293542897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) v3 -- 0x55ab7e9ca9b0 con 0x55ab7ec30920 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 1 -- 172.21.15.46:0/1293542897 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49740564d0 con 0x55ab7ec30920 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 1 -- 172.21.15.46:0/1293542897 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 45) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f497404a020 con 0x55ab7ec30920 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 10 client.6924 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 10 client.6924 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 20 client.6924 populate_metadata read hostname 'smithi046' 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] conn(0x55ab7ec76430 0x55ab7ec76810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 4 client.6924 mds_command: new command op to 6774 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 -- 172.21.15.46:0/1293542897 --> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ab7ea432e0 con 0x55ab7ec76430 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498ce00640 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] conn(0x55ab7ec76430 0x55ab7ec76810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498ce00640 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] conn(0x55ab7ec76430 0x55ab7ec76810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3eb431f255dadd3 server_cookie=47b2aa8f26edca7a in_seq=0 out_seq=0 2024-04-25T08:46:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 10 client.6924 ms_handle_connect on v2:172.21.15.46:6833/960745191 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 1 -- 172.21.15.46:0/1293542897 <== mds.0 v2:172.21.15.46:6833/960745191 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55ab7ea432e0 con 0x55ab7ec76430 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 10 client.6924 handle_command_reply: tid=0 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 client.6924 shutdown 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 -- 172.21.15.46:0/1293542897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 msgr2=0x55ab7ec759b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec759b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4974055ad0 tx=0x7f49740838c0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 -- 172.21.15.46:0/1293542897 shutdown_connections 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.46:6833/960745191,v1:172.21.15.46:6835/960745191] conn(0x55ab7ec76430 0x55ab7ec76810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ab7ec31240 0x55ab7ec75ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ab7ec30920 0x55ab7ec759b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 --2- 172.21.15.46:0/1293542897 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ab7ebe6f30 0x55ab7ec750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f496e000640 0 client.6924 ms_handle_reset on v2:172.21.15.46:6833/960745191 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 -- 172.21.15.46:0/1293542897 >> 172.21.15.46:0/1293542897 conn(0x55ab7ebed2a0 msgr2=0x55ab7ec2ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 -- 172.21.15.46:0/1293542897 shutdown_connections 2024-04-25T08:46:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 1 -- 172.21.15.46:0/1293542897 wait complete. 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f4992d3e000 20 client.6924 trim_cache size 0 max 16384 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6774, 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.924838523, 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 45, 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 78, 2024-04-25T08:46:52.628 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 78, 2024-04-25T08:46:52.629 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 10.771850329999999 2024-04-25T08:46:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:46:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 -- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4968066f60 msgr2=0x7f4968069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4968066f60 0x7f4968069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f49881f9c70 tx=0x7f497c009040 comp rx=0 tx=0).stop 2024-04-25T08:46:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 -- 172.21.15.46:0/4281632832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4988159230 msgr2=0x7f49880a8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4988159230 0x7f49880a8600 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f497400d5f0 tx=0x7f497400dab0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 -- 172.21.15.46:0/4281632832 shutdown_connections 2024-04-25T08:46:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4968066f60 0x7f4968069420 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f498815bdb0 0x7f49880b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4988159230 0x7f49880a8600 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 --2- 172.21.15.46:0/4281632832 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49880cda80 0x7f4988140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 -- 172.21.15.46:0/4281632832 >> 172.21.15.46:0/4281632832 conn(0x7f49880b12a0 msgr2=0x7f49880b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 -- 172.21.15.46:0/4281632832 shutdown_connections 2024-04-25T08:46:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.629+0000 7f498f800640 1 -- 172.21.15.46:0/4281632832 wait complete. 2024-04-25T08:46:52.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:46:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.789+0000 7f9b86e00640 1 Processor -- start 2024-04-25T08:46:52.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b86e00640 1 -- start start 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b86e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b8015a750 0x7f9b8006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b86e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b800cc9e0 0x7f9b8006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b86e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8006d5e0 con 0x7f9b800aed10 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b86e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8015ab30 con 0x7f9b800cc9e0 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b86e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b800ce4f0 con 0x7f9b8015a750 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b84e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b800cc9e0 0x7f9b8006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b7f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9b800aed10 0x7f9b800b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34922/0 (socket says 172.21.15.46:34922) 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b84e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b800cc9e0 0x7f9b8006f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39864/0 (socket says 172.21.15.46:39864) 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b84e00640 1 -- 172.21.15.46:0/712993328 learned_addr learned my addr 172.21.15.46:0/712993328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b7fe00640 1 --2- 172.21.15.46:0/712993328 >> v2:172.21.15.82:3300/0 conn(0x7f9b8015a750 0x7f9b8006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b84e00640 1 -- 172.21.15.46:0/712993328 >> v2:172.21.15.82:3300/0 conn(0x7f9b8015a750 msgr2=0x7f9b8006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b84e00640 1 --2- 172.21.15.46:0/712993328 >> v2:172.21.15.82:3300/0 conn(0x7f9b8015a750 0x7f9b8006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b84e00640 1 -- 172.21.15.46:0/712993328 >> v1:172.21.15.46:6789/0 conn(0x7f9b800aed10 legacy=0x7f9b800b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.793+0000 7f9b84e00640 1 -- 172.21.15.46:0/712993328 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b800ce720 con 0x7f9b800cc9e0 2024-04-25T08:46:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b84e00640 1 --2- 172.21.15.46:0/712993328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 0x7f9b8006f150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9b8004ffd0 tx=0x7f9b7801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738a9acf534aad3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b7ea00640 1 -- 172.21.15.46:0/712993328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b780a57a0 con 0x7f9b800cc9e0 2024-04-25T08:46:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b7ea00640 1 -- 172.21.15.46:0/712993328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b780a5940 con 0x7f9b800cc9e0 2024-04-25T08:46:52.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b7ea00640 1 -- 172.21.15.46:0/712993328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b780a5d10 con 0x7f9b800cc9e0 2024-04-25T08:46:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 -- 172.21.15.46:0/712993328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 msgr2=0x7f9b8006f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 --2- 172.21.15.46:0/712993328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 0x7f9b8006f150 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9b8004ffd0 tx=0x7f9b7801c9f0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 -- 172.21.15.46:0/712993328 shutdown_connections 2024-04-25T08:46:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 --2- 172.21.15.46:0/712993328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 0x7f9b8006f150 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 --2- 172.21.15.46:0/712993328 >> v2:172.21.15.82:3300/0 conn(0x7f9b8015a750 0x7f9b8006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 -- 172.21.15.46:0/712993328 >> 172.21.15.46:0/712993328 conn(0x7f9b800b12a0 msgr2=0x7f9b800b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 -- 172.21.15.46:0/712993328 shutdown_connections 2024-04-25T08:46:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 -- 172.21.15.46:0/712993328 wait complete. 2024-04-25T08:46:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 Processor -- start 2024-04-25T08:46:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.797+0000 7f9b86e00640 1 -- start start 2024-04-25T08:46:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b86e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b800aed10 0x7f9b8015c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b86e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 0x7f9b8015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b86e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b8015a750 0x7f9b8015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b86e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80167230 con 0x7f9b800cc9e0 2024-04-25T08:46:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b86e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8006d270 con 0x7f9b800aed10 2024-04-25T08:46:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b86e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b801673b0 con 0x7f9b8015a750 2024-04-25T08:46:52.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 0x7f9b8015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 0x7f9b8015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39878/0 (socket says 172.21.15.46:39878) 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7f400640 1 -- 172.21.15.46:0/2562221417 learned_addr learned my addr 172.21.15.46:0/2562221417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7fe00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b800aed10 0x7f9b8015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b84e00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b8015a750 0x7f9b8015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7f400640 1 -- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b8015a750 msgr2=0x7f9b8015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7f400640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b8015a750 0x7f9b8015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7f400640 1 -- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b800aed10 msgr2=0x7f9b8015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7f400640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b800aed10 0x7f9b8015c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7f400640 1 -- 172.21.15.46:0/2562221417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8014bb50 con 0x7f9b800cc9e0 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b84e00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b8015a750 0x7f9b8015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7fe00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b800aed10 0x7f9b8015c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b7f400640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 0x7f9b8015ca90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c460 tx=0x7f9b6c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b77e00640 1 -- 172.21.15.46:0/2562221417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c018070 con 0x7f9b800cc9e0 2024-04-25T08:46:52.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b77e00640 1 -- 172.21.15.46:0/2562221417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c007480 con 0x7f9b800cc9e0 2024-04-25T08:46:52.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8015be50 con 0x7f9b800cc9e0 2024-04-25T08:46:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b77e00640 1 -- 172.21.15.46:0/2562221417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0047f0 con 0x7f9b800cc9e0 2024-04-25T08:46:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b800cbf80 con 0x7f9b800cc9e0 2024-04-25T08:46:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8015bc70 con 0x7f9b800cc9e0 2024-04-25T08:46:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.801+0000 7f9b77e00640 1 -- 172.21.15.46:0/2562221417 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b6c01b070 con 0x7f9b800cc9e0 2024-04-25T08:46:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.805+0000 7f9b77e00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b60066f60 0x7f9b60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.805+0000 7f9b77e00640 1 -- 172.21.15.46:0/2562221417 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b6c084510 con 0x7f9b800cc9e0 2024-04-25T08:46:52.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.805+0000 7f9b7fe00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b60066f60 0x7f9b60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.805+0000 7f9b7fe00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b60066f60 0x7f9b60069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9b8015c300 tx=0x7f9b70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:52.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:52.809+0000 7f9b77e00640 1 -- 172.21.15.46:0/2562221417 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b8015bc70 con 0x7f9b800cc9e0 2024-04-25T08:46:52.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.005+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9b80069080 con 0x7f9b800cc9e0 2024-04-25T08:46:53.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.005+0000 7f9b77e00640 1 -- 172.21.15.46:0/2562221417 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9b6c055380 con 0x7f9b800cc9e0 2024-04-25T08:46:53.000 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:46:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b60066f60 msgr2=0x7f9b60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b60066f60 0x7f9b60069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9b8015c300 tx=0x7f9b70008040 comp rx=0 tx=0).stop 2024-04-25T08:46:53.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 msgr2=0x7f9b8015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 0x7f9b8015ca90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c460 tx=0x7f9b6c00c920 comp rx=0 tx=0).stop 2024-04-25T08:46:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 shutdown_connections 2024-04-25T08:46:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b60066f60 0x7f9b60069420 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9b8015a750 0x7f9b8015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b800cc9e0 0x7f9b8015ca90 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 --2- 172.21.15.46:0/2562221417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b800aed10 0x7f9b8015c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 >> 172.21.15.46:0/2562221417 conn(0x7f9b800b12a0 msgr2=0x7f9b8006ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 shutdown_connections 2024-04-25T08:46:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53.009+0000 7f9b86e00640 1 -- 172.21.15.46:0/2562221417 wait complete. 2024-04-25T08:46:53.022 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:46:53.022 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:46:53.022 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:46:53.022 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:46:53.022 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:46:53.022 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:46:53.022 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:46:53.022 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:46:53.022 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:46:53.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.023 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:46:53.032 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:46:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:46:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:46:53.033 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:46:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:46:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:46:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:46:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42190sec preferred_lft 42190sec 2024-04-25T08:46:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:46:53.033 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:46:53.034 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:46:53.035 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:46:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:46:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:46:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:46:53.035 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:46:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:46:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:46:53.035 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:46:53.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.036 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:46:53.094 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:46:53.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.096 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:46:53.145 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:46:53.146 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:46:53.146 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:46:53.146 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:46:53.146 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:46:53.146 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:46:53.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.146 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:46:53.146 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:46:53.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:46:53.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:46:53.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:46:53.147 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:46:53.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.238 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:46:53.248 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:46:53.248 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:46:53.248 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:46:53.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:46:53.312 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:46:53.312 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:46:53.365 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:46:53.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:46:53.365 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:46:53.420 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:46:53.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:46:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:46:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:46:53.509 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:46:53.524 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:46:53.524 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:46:53.524 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:46:53.524 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:46:53.524 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:46:53.533 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:46:53.533 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:46:53.534 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:46:53.534 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:46:53.534 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:46:53.534 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:46:53.534 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:46:53.534 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:46:53.537 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:46:53.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:46:53.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.561 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:46:53.620 INFO:teuthology.orchestra.run.smithi046.stdout:client6077 2024-04-25T08:46:53.620 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:46:53.620 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:53.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.669 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-25T08:46:53.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.732 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-25T08:46:53.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:53.783 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-25T08:46:53.837 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:46:53.837 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:46:53.838 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:46:53.838 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:46:53.838 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-25T08:46:53.838 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:46:53.946 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-25T08:46:53.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:46:53.955 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:46:53.955 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:54.005 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:46:54.005 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:46:54.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:46:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.245+0000 7f67e9800640 1 Processor -- start 2024-04-25T08:46:54.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.245+0000 7f67e9800640 1 -- start start 2024-04-25T08:46:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67e9800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f67e406f320 0x7f67e4162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67e9800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e40b35b0 con 0x7f67e415a750 2024-04-25T08:46:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67e9800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e406d5e0 con 0x7f67e40aed10 2024-04-25T08:46:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67e9800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e415ab30 con 0x7f67e406f320 2024-04-25T08:46:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67e2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f67e415a750 0x7f67e406ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34938/0 (socket says 172.21.15.46:34938) 2024-04-25T08:46:54.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67e2a00640 1 -- 172.21.15.46:0/719197925 learned_addr learned my addr 172.21.15.46:0/719197925 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67dbe00640 1 --2- 172.21.15.46:0/719197925 >> v2:172.21.15.82:3300/0 conn(0x7f67e406f320 0x7f67e4162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67e2000640 1 -- 172.21.15.46:0/719197925 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574667581 0 0) 0x7f67e40b35b0 con 0x7f67e415a750 2024-04-25T08:46:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67e2000640 1 -- 172.21.15.46:0/719197925 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c8005740 con 0x7f67e415a750 2024-04-25T08:46:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67dbe00640 1 -- 172.21.15.46:0/719197925 >> v1:172.21.15.82:6789/0 conn(0x7f67e40aed10 legacy=0x7f67e4164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67dbe00640 1 -- 172.21.15.46:0/719197925 >> v1:172.21.15.46:6789/0 conn(0x7f67e415a750 legacy=0x7f67e406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.249+0000 7f67dbe00640 1 -- 172.21.15.46:0/719197925 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e40ce720 con 0x7f67e406f320 2024-04-25T08:46:54.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67dbe00640 1 --2- 172.21.15.46:0/719197925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e406f320 0x7f67e4162200 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0097c0 tx=0x7f67cc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5246b483f708794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e2000640 1 -- 172.21.15.46:0/719197925 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0b0070 con 0x7f67e406f320 2024-04-25T08:46:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e2000640 1 -- 172.21.15.46:0/719197925 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc0ab600 con 0x7f67e406f320 2024-04-25T08:46:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e2000640 1 -- 172.21.15.46:0/719197925 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0ab9d0 con 0x7f67e406f320 2024-04-25T08:46:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e9800640 1 -- 172.21.15.46:0/719197925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e406f320 msgr2=0x7f67e4162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e9800640 1 --2- 172.21.15.46:0/719197925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e406f320 0x7f67e4162200 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0097c0 tx=0x7f67cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e9800640 1 -- 172.21.15.46:0/719197925 shutdown_connections 2024-04-25T08:46:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e9800640 1 --2- 172.21.15.46:0/719197925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e406f320 0x7f67e4162200 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e9800640 1 -- 172.21.15.46:0/719197925 >> 172.21.15.46:0/719197925 conn(0x7f67e40b12a0 msgr2=0x7f67e40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e9800640 1 -- 172.21.15.46:0/719197925 shutdown_connections 2024-04-25T08:46:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e9800640 1 -- 172.21.15.46:0/719197925 wait complete. 2024-04-25T08:46:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e9800640 1 Processor -- start 2024-04-25T08:46:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.253+0000 7f67e9800640 1 -- start start 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67e406f320 0x7f67e415acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e40aed10 0x7f67e415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415a750 0x7f67e4149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e406e310 con 0x7f67e415a750 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e9800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e406d270 con 0x7f67e40aed10 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e406e490 con 0x7f67e406f320 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415a750 0x7f67e4149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415a750 0x7f67e4149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39882/0 (socket says 172.21.15.46:39882) 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e3400640 1 -- 172.21.15.46:0/2760742973 learned_addr learned my addr 172.21.15.46:0/2760742973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67dbe00640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e40aed10 0x7f67e415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e2a00640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67e406f320 0x7f67e415acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e3400640 1 -- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67e406f320 msgr2=0x7f67e415acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e3400640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67e406f320 0x7f67e415acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e3400640 1 -- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e40aed10 msgr2=0x7f67e415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e3400640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e40aed10 0x7f67e415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e3400640 1 -- 172.21.15.46:0/2760742973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e415b2a0 con 0x7f67e415a750 2024-04-25T08:46:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67dbe00640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e40aed10 0x7f67e415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e3400640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415a750 0x7f67e4149e70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00c7a0 tx=0x7f67dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67daa00640 1 -- 172.21.15.46:0/2760742973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc003b80 con 0x7f67e415a750 2024-04-25T08:46:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67daa00640 1 -- 172.21.15.46:0/2760742973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc003d20 con 0x7f67e415a750 2024-04-25T08:46:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67daa00640 1 -- 172.21.15.46:0/2760742973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00aa90 con 0x7f67e415a750 2024-04-25T08:46:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e415b580 con 0x7f67e415a750 2024-04-25T08:46:54.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e415bb40 con 0x7f67e415a750 2024-04-25T08:46:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e4069080 con 0x7f67e415a750 2024-04-25T08:46:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.257+0000 7f67daa00640 1 -- 172.21.15.46:0/2760742973 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67dc00ac30 con 0x7f67e415a750 2024-04-25T08:46:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.261+0000 7f67daa00640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67c0066f60 0x7f67c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.261+0000 7f67daa00640 1 -- 172.21.15.46:0/2760742973 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67dc0851b0 con 0x7f67e415a750 2024-04-25T08:46:54.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.261+0000 7f67e2a00640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67c0066f60 0x7f67c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.261+0000 7f67e2a00640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67c0066f60 0x7f67c0069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f67d40011f0 tx=0x7f67d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:54.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.265+0000 7f67daa00640 1 -- 172.21.15.46:0/2760742973 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67dc055f90 con 0x7f67e415a750 2024-04-25T08:46:54.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.425+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67e40b3c20 con 0x7f67e415a750 2024-04-25T08:46:54.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67daa00640 1 -- 172.21.15.46:0/2760742973 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f67dc055890 con 0x7f67e415a750 2024-04-25T08:46:54.421 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:54.421 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":78,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:47.688186+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:46:44.505856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:46:45.072095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67c0066f60 msgr2=0x7f67c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67c0066f60 0x7f67c0069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f67d40011f0 tx=0x7f67d4009040 comp rx=0 tx=0).stop 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415a750 msgr2=0x7f67e4149e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415a750 0x7f67e4149e70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00c7a0 tx=0x7f67dc00cc60 comp rx=0 tx=0).stop 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 shutdown_connections 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67c0066f60 0x7f67c0069420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67e415a750 0x7f67e4149e70 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67e40aed10 0x7f67e415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 --2- 172.21.15.46:0/2760742973 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67e406f320 0x7f67e415acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 >> 172.21.15.46:0/2760742973 conn(0x7f67e40b12a0 msgr2=0x7f67e40b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 shutdown_connections 2024-04-25T08:46:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.429+0000 7f67e9800640 1 -- 172.21.15.46:0/2760742973 wait complete. 2024-04-25T08:46:54.443 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:46:44.505856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:46:45.072095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:46:54.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:46:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb702800640 1 Processor -- start 2024-04-25T08:46:54.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb702800640 1 -- start start 2024-04-25T08:46:54.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb702800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6fc0afdf0 0x7fb6fc06ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb702800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6fc06b700 con 0x7fb6fc06b2a0 2024-04-25T08:46:54.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb702800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6fc06fca0 con 0x7fb6fc15d900 2024-04-25T08:46:54.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb702800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6fc06ff80 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb6f2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6fc0afdf0 0x7fb6fc06ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb6f2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6fc0afdf0 0x7fb6fc06ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39892/0 (socket says 172.21.15.46:39892) 2024-04-25T08:46:54.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb6f2a00640 1 -- 172.21.15.46:0/974057186 learned_addr learned my addr 172.21.15.46:0/974057186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:54.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb6fbe00640 1 --1- 172.21.15.46:0/974057186 >> v1:172.21.15.46:6789/0 conn(0x7fb6fc06b2a0 0x7fb6fc0d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34954/0 (socket says 172.21.15.46:34954) 2024-04-25T08:46:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb6f2a00640 1 -- 172.21.15.46:0/974057186 >> v1:172.21.15.82:6789/0 conn(0x7fb6fc15d900 legacy=0x7fb6fc0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb6f2a00640 1 -- 172.21.15.46:0/974057186 >> v1:172.21.15.46:6789/0 conn(0x7fb6fc06b2a0 legacy=0x7fb6fc0d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb6f2a00640 1 -- 172.21.15.46:0/974057186 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc070260 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.593+0000 7fb6faa00640 1 -- 172.21.15.46:0/974057186 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285051941 0 0) 0x7fb6fc06b700 con 0x7fb6fc06b2a0 2024-04-25T08:46:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb6f2a00640 1 --2- 172.21.15.46:0/974057186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 0x7fb6fc06ad60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb5f6b7fddcee1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb6faa00640 1 -- 172.21.15.46:0/974057186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40b1070 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb6faa00640 1 -- 172.21.15.46:0/974057186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e40ac620 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb6faa00640 1 -- 172.21.15.46:0/974057186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40ac9f0 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb702800640 1 -- 172.21.15.46:0/974057186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 msgr2=0x7fb6fc06ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb702800640 1 --2- 172.21.15.46:0/974057186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 0x7fb6fc06ad60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009870 tx=0x7fb6e401c4d0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb702800640 1 -- 172.21.15.46:0/974057186 shutdown_connections 2024-04-25T08:46:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb702800640 1 --2- 172.21.15.46:0/974057186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 0x7fb6fc06ad60 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb702800640 1 -- 172.21.15.46:0/974057186 >> 172.21.15.46:0/974057186 conn(0x7fb6fc06e3e0 msgr2=0x7fb6fc06e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb702800640 1 -- 172.21.15.46:0/974057186 shutdown_connections 2024-04-25T08:46:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.597+0000 7fb702800640 1 -- 172.21.15.46:0/974057186 wait complete. 2024-04-25T08:46:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 Processor -- start 2024-04-25T08:46:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 -- start start 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6fc06b2a0 0x7fb6fc0cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 0x7fb6fc0cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6fc15d900 0x7fb6fc0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc0d12b0 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc0d1130 con 0x7fb6fc06b2a0 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc0d1430 con 0x7fb6fc15d900 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 0x7fb6fc0cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 0x7fb6fc0cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39896/0 (socket says 172.21.15.46:39896) 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f2a00640 1 -- 172.21.15.46:0/661133311 learned_addr learned my addr 172.21.15.46:0/661133311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6fb400640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6fc06b2a0 0x7fb6fc0cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6fbe00640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6fc15d900 0x7fb6fc0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f2a00640 1 -- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6fc15d900 msgr2=0x7fb6fc0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f2a00640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6fc15d900 0x7fb6fc0cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f2a00640 1 -- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6fc06b2a0 msgr2=0x7fb6fc0cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f2a00640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6fc06b2a0 0x7fb6fc0cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f2a00640 1 -- 172.21.15.46:0/661133311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc0b0540 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6fbe00640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6fc15d900 0x7fb6fc0cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f2a00640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 0x7fb6fc0cd980 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40af040 tx=0x7fb6e4019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f3e00640 1 -- 172.21.15.46:0/661133311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40bc070 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f3e00640 1 -- 172.21.15.46:0/661133311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e40b7630 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6fc0b0820 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb6f3e00640 1 -- 172.21.15.46:0/661133311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40b1040 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6fc0b01d0 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.601+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6fc069010 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.605+0000 7fb6f3e00640 1 -- 172.21.15.46:0/661133311 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6e40bf070 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.605+0000 7fb6f3e00640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb6d8066f90 0x7fb6d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.605+0000 7fb6f3e00640 1 -- 172.21.15.46:0/661133311 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb6e411eb00 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.605+0000 7fb6fb400640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb6d8066f90 0x7fb6d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.605+0000 7fb6fb400640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb6d8066f90 0x7fb6d8069450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec007fd0 tx=0x7fb6ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.609+0000 7fb6f3e00640 1 -- 172.21.15.46:0/661133311 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6e4016150 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6fc072890 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb6f3e00640 1 -- 172.21.15.46:0/661133311 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6e40ef290 con 0x7fb6fc0afdf0 2024-04-25T08:46:54.771 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:46:54.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb6d8066f90 msgr2=0x7fb6d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb6d8066f90 0x7fb6d8069450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec007fd0 tx=0x7fb6ec009040 comp rx=0 tx=0).stop 2024-04-25T08:46:54.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 msgr2=0x7fb6fc0cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 0x7fb6fc0cd980 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb6e40af040 tx=0x7fb6e4019270 comp rx=0 tx=0).stop 2024-04-25T08:46:54.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 shutdown_connections 2024-04-25T08:46:54.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb6d8066f90 0x7fb6d8069450 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb6fc15d900 0x7fb6fc0cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6fc0afdf0 0x7fb6fc0cd980 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 --2- 172.21.15.46:0/661133311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6fc06b2a0 0x7fb6fc0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 >> 172.21.15.46:0/661133311 conn(0x7fb6fc06e3e0 msgr2=0x7fb6fc1601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:54.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 shutdown_connections 2024-04-25T08:46:54.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.777+0000 7fb702800640 1 -- 172.21.15.46:0/661133311 wait complete. 2024-04-25T08:46:54.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:46:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.941+0000 7f3414c00640 1 Processor -- start 2024-04-25T08:46:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.941+0000 7f3414c00640 1 -- start start 2024-04-25T08:46:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.941+0000 7f3414c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f341006dc70 0x7f34100cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.941+0000 7f3414c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34100cd930 con 0x7f34100cf1b0 2024-04-25T08:46:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.941+0000 7f3414c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34100cdbd0 con 0x7f3410145310 2024-04-25T08:46:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.941+0000 7f3414c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34100cdeb0 con 0x7f341006dc70 2024-04-25T08:46:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f3407e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f341006dc70 0x7f34100cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f340e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f34100cf1b0 0x7f341015d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34958/0 (socket says 172.21.15.46:34958) 2024-04-25T08:46:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f340e800640 1 -- 172.21.15.46:0/3028268918 learned_addr learned my addr 172.21.15.46:0/3028268918 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f3407e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f341006dc70 0x7f34100cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39908/0 (socket says 172.21.15.46:39908) 2024-04-25T08:46:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f3407e00640 1 -- 172.21.15.46:0/3028268918 >> v1:172.21.15.82:6789/0 conn(0x7f3410145310 legacy=0x7f341006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f3407e00640 1 -- 172.21.15.46:0/3028268918 >> v1:172.21.15.46:6789/0 conn(0x7f34100cf1b0 legacy=0x7f341015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f340d400640 1 -- 172.21.15.46:0/3028268918 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459065864 0 0) 0x7f34100cd930 con 0x7f34100cf1b0 2024-04-25T08:46:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f3407e00640 1 -- 172.21.15.46:0/3028268918 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34100ce190 con 0x7f341006dc70 2024-04-25T08:46:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f3407e00640 1 --2- 172.21.15.46:0/3028268918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f341006dc70 0x7f34100cec70 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f33f80097c0 tx=0x7f33f801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ea5e3111f6166b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f340d400640 1 -- 172.21.15.46:0/3028268918 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f80a7070 con 0x7f341006dc70 2024-04-25T08:46:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f340d400640 1 -- 172.21.15.46:0/3028268918 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f80b0070 con 0x7f341006dc70 2024-04-25T08:46:54.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.945+0000 7f340d400640 1 -- 172.21.15.46:0/3028268918 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f80ab890 con 0x7f341006dc70 2024-04-25T08:46:54.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 -- 172.21.15.46:0/3028268918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f341006dc70 msgr2=0x7f34100cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 --2- 172.21.15.46:0/3028268918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f341006dc70 0x7f34100cec70 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f33f80097c0 tx=0x7f33f801c470 comp rx=0 tx=0).stop 2024-04-25T08:46:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 -- 172.21.15.46:0/3028268918 shutdown_connections 2024-04-25T08:46:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 --2- 172.21.15.46:0/3028268918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f341006dc70 0x7f34100cec70 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 -- 172.21.15.46:0/3028268918 >> 172.21.15.46:0/3028268918 conn(0x7f341006ac10 msgr2=0x7f341006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 -- 172.21.15.46:0/3028268918 shutdown_connections 2024-04-25T08:46:54.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 -- 172.21.15.46:0/3028268918 wait complete. 2024-04-25T08:46:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 Processor -- start 2024-04-25T08:46:54.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 -- start start 2024-04-25T08:46:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34100cf1b0 0x7f341013f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3410145310 0x7f3410159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3410138d00 0x7f341015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34101696a0 con 0x7f3410145310 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410169520 con 0x7f3410138d00 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3414c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3410169820 con 0x7f34100cf1b0 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3407e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3410145310 0x7f3410159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3407e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3410145310 0x7f3410159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39924/0 (socket says 172.21.15.46:39924) 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3407e00640 1 -- 172.21.15.46:0/2554125574 learned_addr learned my addr 172.21.15.46:0/2554125574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f340e800640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3410138d00 0x7f341015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f340de00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34100cf1b0 0x7f341013f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3407e00640 1 -- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34100cf1b0 msgr2=0x7f341013f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3407e00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34100cf1b0 0x7f341013f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3407e00640 1 -- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3410138d00 msgr2=0x7f341015a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3407e00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3410138d00 0x7f341015a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.949+0000 7f3407e00640 1 -- 172.21.15.46:0/2554125574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341015a870 con 0x7f3410145310 2024-04-25T08:46:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3407e00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3410145310 0x7f3410159af0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f33f8019080 tx=0x7f33f801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3406000640 1 -- 172.21.15.46:0/2554125574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f80bf610 con 0x7f3410145310 2024-04-25T08:46:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3406000640 1 -- 172.21.15.46:0/2554125574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f80bf7d0 con 0x7f3410145310 2024-04-25T08:46:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3410155930 con 0x7f3410145310 2024-04-25T08:46:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3406000640 1 -- 172.21.15.46:0/2554125574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f80bfbc0 con 0x7f3410145310 2024-04-25T08:46:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341015a570 con 0x7f3410145310 2024-04-25T08:46:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341013fe30 con 0x7f3410145310 2024-04-25T08:46:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3406000640 1 -- 172.21.15.46:0/2554125574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33f80bd020 con 0x7f3410145310 2024-04-25T08:46:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3406000640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33ec066f60 0x7f33ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f3406000640 1 -- 172.21.15.46:0/2554125574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f33f811fd40 con 0x7f3410145310 2024-04-25T08:46:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.953+0000 7f340de00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33ec066f60 0x7f33ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.957+0000 7f340de00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33ec066f60 0x7f33ec069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3410155f40 tx=0x7f3400009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:54.957+0000 7f3406000640 1 -- 172.21.15.46:0/2554125574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33f8016090 con 0x7f3410145310 2024-04-25T08:46:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3410069010 con 0x7f3410145310 2024-04-25T08:46:55.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3406000640 1 -- 172.21.15.46:0/2554125574 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33f80f0cd0 con 0x7f3410145310 2024-04-25T08:46:55.114 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:46:55.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33ec066f60 msgr2=0x7f33ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33ec066f60 0x7f33ec069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3410155f40 tx=0x7f3400009040 comp rx=0 tx=0).stop 2024-04-25T08:46:55.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3410145310 msgr2=0x7f3410159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3410145310 0x7f3410159af0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f33f8019080 tx=0x7f33f801acb0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 shutdown_connections 2024-04-25T08:46:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f33ec066f60 0x7f33ec069420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3410138d00 0x7f341015a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3410145310 0x7f3410159af0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 --2- 172.21.15.46:0/2554125574 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34100cf1b0 0x7f341013f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 >> 172.21.15.46:0/2554125574 conn(0x7f341006ac10 msgr2=0x7f341006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 shutdown_connections 2024-04-25T08:46:55.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.121+0000 7f3414c00640 1 -- 172.21.15.46:0/2554125574 wait complete. 2024-04-25T08:46:55.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:46:55.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.285+0000 7f11ed400640 1 Processor -- start 2024-04-25T08:46:55.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.285+0000 7f11ed400640 1 -- start start 2024-04-25T08:46:55.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.285+0000 7f11ed400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f11e8143990 0x7f11e80cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.285+0000 7f11ed400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11e81440d0 0x7f11e806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.285+0000 7f11ed400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e806ef80 0x7f11e80cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.285+0000 7f11ed400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e815ed30 con 0x7f11e81440d0 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.285+0000 7f11ed400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e815e440 con 0x7f11e8143990 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.285+0000 7f11ed400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e815e6e0 con 0x7f11e806ef80 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11dde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11e81440d0 0x7f11e806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11dde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11e81440d0 0x7f11e806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39930/0 (socket says 172.21.15.46:39930) 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11dde00640 1 -- 172.21.15.46:0/2686782055 learned_addr learned my addr 172.21.15.46:0/2686782055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11e6800640 1 --2- 172.21.15.46:0/2686782055 >> v2:172.21.15.82:3300/0 conn(0x7f11e8143990 0x7f11e80cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11e7200640 1 --2- 172.21.15.46:0/2686782055 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e806ef80 0x7f11e80cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11dde00640 1 -- 172.21.15.46:0/2686782055 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e806ef80 msgr2=0x7f11e80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11dde00640 1 --2- 172.21.15.46:0/2686782055 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e806ef80 0x7f11e80cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11dde00640 1 -- 172.21.15.46:0/2686782055 >> v2:172.21.15.82:3300/0 conn(0x7f11e8143990 msgr2=0x7f11e80cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11dde00640 1 --2- 172.21.15.46:0/2686782055 >> v2:172.21.15.82:3300/0 conn(0x7f11e8143990 0x7f11e80cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11dde00640 1 -- 172.21.15.46:0/2686782055 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e815e9c0 con 0x7f11e81440d0 2024-04-25T08:46:55.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11e7200640 1 --2- 172.21.15.46:0/2686782055 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e806ef80 0x7f11e80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:55.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11dde00640 1 --2- 172.21.15.46:0/2686782055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e81440d0 0x7f11e806ea40 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f11e806b9d0 tx=0x7f11d001e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9700ac12fe78ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:55.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11e5e00640 1 -- 172.21.15.46:0/2686782055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d00b4070 con 0x7f11e81440d0 2024-04-25T08:46:55.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11e5e00640 1 -- 172.21.15.46:0/2686782055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d00a78d0 con 0x7f11e81440d0 2024-04-25T08:46:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.289+0000 7f11e5e00640 1 -- 172.21.15.46:0/2686782055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d00a7ca0 con 0x7f11e81440d0 2024-04-25T08:46:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 -- 172.21.15.46:0/2686782055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e81440d0 msgr2=0x7f11e806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 --2- 172.21.15.46:0/2686782055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e81440d0 0x7f11e806ea40 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f11e806b9d0 tx=0x7f11d001e930 comp rx=0 tx=0).stop 2024-04-25T08:46:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 -- 172.21.15.46:0/2686782055 shutdown_connections 2024-04-25T08:46:55.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 --2- 172.21.15.46:0/2686782055 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e806ef80 0x7f11e80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 --2- 172.21.15.46:0/2686782055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e81440d0 0x7f11e806ea40 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 --2- 172.21.15.46:0/2686782055 >> v2:172.21.15.82:3300/0 conn(0x7f11e8143990 0x7f11e80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 -- 172.21.15.46:0/2686782055 >> 172.21.15.46:0/2686782055 conn(0x7f11e806ac10 msgr2=0x7f11e806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 -- 172.21.15.46:0/2686782055 shutdown_connections 2024-04-25T08:46:55.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 -- 172.21.15.46:0/2686782055 wait complete. 2024-04-25T08:46:55.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 Processor -- start 2024-04-25T08:46:55.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 -- start start 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e806ef80 0x7f11e81f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e8143990 0x7f11e8135230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e81440d0 0x7f11e8140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e8143ef0 con 0x7f11e806ef80 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e8143d70 con 0x7f11e81440d0 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11ed400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e8071270 con 0x7f11e8143990 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e806ef80 0x7f11e81f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e806ef80 0x7f11e81f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39940/0 (socket says 172.21.15.46:39940) 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e6800640 1 -- 172.21.15.46:0/148804559 learned_addr learned my addr 172.21.15.46:0/148804559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e7200640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e81440d0 0x7f11e8140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11dde00640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e8143990 0x7f11e8135230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e6800640 1 -- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e8143990 msgr2=0x7f11e8135230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e6800640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e8143990 0x7f11e8135230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e6800640 1 -- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e81440d0 msgr2=0x7f11e8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e6800640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e81440d0 0x7f11e8140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e6800640 1 -- 172.21.15.46:0/148804559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e813c8c0 con 0x7f11e806ef80 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.293+0000 7f11e7200640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e81440d0 0x7f11e8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11e6800640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e806ef80 0x7f11e81f6570 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e930 tx=0x7f11d800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11df400640 1 -- 172.21.15.46:0/148804559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8017070 con 0x7f11e806ef80 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11df400640 1 -- 172.21.15.46:0/148804559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d800c9a0 con 0x7f11e806ef80 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e813cba0 con 0x7f11e806ef80 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11df400640 1 -- 172.21.15.46:0/148804559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800cd50 con 0x7f11e806ef80 2024-04-25T08:46:55.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e813d160 con 0x7f11e806ef80 2024-04-25T08:46:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e8069010 con 0x7f11e806ef80 2024-04-25T08:46:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11df400640 1 -- 172.21.15.46:0/148804559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11d801a070 con 0x7f11e806ef80 2024-04-25T08:46:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11df400640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c4066f90 0x7f11c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11df400640 1 -- 172.21.15.46:0/148804559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f11d80854c0 con 0x7f11e806ef80 2024-04-25T08:46:55.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.297+0000 7f11dde00640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c4066f90 0x7f11c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.301+0000 7f11dde00640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c4066f90 0x7f11c4069450 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f11d00b1000 tx=0x7f11d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:55.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.301+0000 7f11df400640 1 -- 172.21.15.46:0/148804559 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11d80563f0 con 0x7f11e806ef80 2024-04-25T08:46:55.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11e806b190 con 0x7f11e806ef80 2024-04-25T08:46:55.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11df400640 1 -- 172.21.15.46:0/148804559 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11d8055cf0 con 0x7f11e806ef80 2024-04-25T08:46:55.458 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:46:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c4066f90 msgr2=0x7f11c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c4066f90 0x7f11c4069450 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f11d00b1000 tx=0x7f11d0008040 comp rx=0 tx=0).stop 2024-04-25T08:46:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e806ef80 msgr2=0x7f11e81f6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e806ef80 0x7f11e81f6570 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e930 tx=0x7f11d800edf0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 shutdown_connections 2024-04-25T08:46:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c4066f90 0x7f11c4069450 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11e81440d0 0x7f11e8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11e8143990 0x7f11e8135230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 --2- 172.21.15.46:0/148804559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11e806ef80 0x7f11e81f6570 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 >> 172.21.15.46:0/148804559 conn(0x7f11e806ac10 msgr2=0x7f11e815b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 shutdown_connections 2024-04-25T08:46:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.465+0000 7f11ed400640 1 -- 172.21.15.46:0/148804559 wait complete. 2024-04-25T08:46:55.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:46:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78c600640 1 Processor -- start 2024-04-25T08:46:55.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78c600640 1 -- start start 2024-04-25T08:46:55.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78415a750 0x7fc78406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78c600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc7840aed10 0x7fc78406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78c600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc7840cc9e0 0x7fc7840b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78c600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78415ab30 con 0x7fc7840cc9e0 2024-04-25T08:46:55.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78c600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7840ce4f0 con 0x7fc7840aed10 2024-04-25T08:46:55.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78406d8c0 con 0x7fc78415a750 2024-04-25T08:46:55.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc7840cc9e0 0x7fc7840b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc7840cc9e0 0x7fc7840b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39950/0 (socket says 172.21.15.46:39950) 2024-04-25T08:46:55.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78a600640 1 -- 172.21.15.46:0/744532599 learned_addr learned my addr 172.21.15.46:0/744532599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc789200640 1 --2- 172.21.15.46:0/744532599 >> v2:172.21.15.82:3300/0 conn(0x7fc7840aed10 0x7fc78406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc789c00640 1 --2- 172.21.15.46:0/744532599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78415a750 0x7fc78406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78a600640 1 -- 172.21.15.46:0/744532599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78415a750 msgr2=0x7fc78406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78a600640 1 --2- 172.21.15.46:0/744532599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78415a750 0x7fc78406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78a600640 1 -- 172.21.15.46:0/744532599 >> v2:172.21.15.82:3300/0 conn(0x7fc7840aed10 msgr2=0x7fc78406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78a600640 1 --2- 172.21.15.46:0/744532599 >> v2:172.21.15.82:3300/0 conn(0x7fc7840aed10 0x7fc78406f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.629+0000 7fc78a600640 1 -- 172.21.15.46:0/744532599 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78406d5e0 con 0x7fc7840cc9e0 2024-04-25T08:46:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78a600640 1 --2- 172.21.15.46:0/744532599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7840cc9e0 0x7fc7840b25a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc784050000 tx=0x7fc77c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1be23918d655da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc783e00640 1 -- 172.21.15.46:0/744532599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0a5840 con 0x7fc7840cc9e0 2024-04-25T08:46:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc783e00640 1 -- 172.21.15.46:0/744532599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c0a59e0 con 0x7fc7840cc9e0 2024-04-25T08:46:55.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc783e00640 1 -- 172.21.15.46:0/744532599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0a5db0 con 0x7fc7840cc9e0 2024-04-25T08:46:55.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 -- 172.21.15.46:0/744532599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7840cc9e0 msgr2=0x7fc7840b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 --2- 172.21.15.46:0/744532599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7840cc9e0 0x7fc7840b25a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc784050000 tx=0x7fc77c01c9c0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 -- 172.21.15.46:0/744532599 shutdown_connections 2024-04-25T08:46:55.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 --2- 172.21.15.46:0/744532599 >> v2:172.21.15.82:3300/0 conn(0x7fc7840aed10 0x7fc78406f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 --2- 172.21.15.46:0/744532599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7840cc9e0 0x7fc7840b25a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 --2- 172.21.15.46:0/744532599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78415a750 0x7fc78406ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 -- 172.21.15.46:0/744532599 >> 172.21.15.46:0/744532599 conn(0x7fc7840b12a0 msgr2=0x7fc7840b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:55.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 -- 172.21.15.46:0/744532599 shutdown_connections 2024-04-25T08:46:55.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 -- 172.21.15.46:0/744532599 wait complete. 2024-04-25T08:46:55.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 Processor -- start 2024-04-25T08:46:55.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.633+0000 7fc78c600640 1 -- start start 2024-04-25T08:46:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7840aed10 0x7fc7840b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78c600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7840cc9e0 0x7fc78414e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc78415a750 0x7fc7841531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78406d270 con 0x7fc78415a750 2024-04-25T08:46:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78c600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840cf200 con 0x7fc7840cc9e0 2024-04-25T08:46:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7840b32f0 con 0x7fc7840aed10 2024-04-25T08:46:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc78415a750 0x7fc7841531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc78415a750 0x7fc7841531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39956/0 (socket says 172.21.15.46:39956) 2024-04-25T08:46:55.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78a600640 1 -- 172.21.15.46:0/3856982105 learned_addr learned my addr 172.21.15.46:0/3856982105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc789c00640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7840aed10 0x7fc7840b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc789200640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7840cc9e0 0x7fc78414e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78a600640 1 -- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7840aed10 msgr2=0x7fc7840b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78a600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7840aed10 0x7fc7840b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78a600640 1 -- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7840cc9e0 msgr2=0x7fc78414e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78a600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7840cc9e0 0x7fc78414e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78a600640 1 -- 172.21.15.46:0/3856982105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784159e50 con 0x7fc78415a750 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc789200640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7840cc9e0 0x7fc78414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78a600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc78415a750 0x7fc7841531b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc77c01b070 tx=0x7fc77c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc781600640 1 -- 172.21.15.46:0/3856982105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0bc020 con 0x7fc78415a750 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc781600640 1 -- 172.21.15.46:0/3856982105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c0b86b0 con 0x7fc78415a750 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc781600640 1 -- 172.21.15.46:0/3856982105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c0b8aa0 con 0x7fc78415a750 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc784160310 con 0x7fc78415a750 2024-04-25T08:46:55.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7841608d0 con 0x7fc78415a750 2024-04-25T08:46:55.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.637+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc784069080 con 0x7fc78415a750 2024-04-25T08:46:55.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.641+0000 7fc781600640 1 -- 172.21.15.46:0/3856982105 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc77c0bf020 con 0x7fc78415a750 2024-04-25T08:46:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.641+0000 7fc781600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc764066f60 0x7fc764069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.641+0000 7fc781600640 1 -- 172.21.15.46:0/3856982105 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc77c1200a0 con 0x7fc78415a750 2024-04-25T08:46:55.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.641+0000 7fc789c00640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc764066f60 0x7fc764069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:55.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.641+0000 7fc789c00640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc764066f60 0x7fc764069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc7841f95b0 tx=0x7fc778009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:55.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.645+0000 7fc781600640 1 -- 172.21.15.46:0/3856982105 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc77c0165e0 con 0x7fc78415a750 2024-04-25T08:46:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc78406c250 con 0x7fc78415a750 2024-04-25T08:46:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc781600640 1 -- 172.21.15.46:0/3856982105 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v78) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc77c0f0f10 con 0x7fc78415a750 2024-04-25T08:46:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:46:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc764066f60 msgr2=0x7fc764069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc764066f60 0x7fc764069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc7841f95b0 tx=0x7fc778009190 comp rx=0 tx=0).stop 2024-04-25T08:46:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc78415a750 msgr2=0x7fc7841531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc78415a750 0x7fc7841531b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc77c01b070 tx=0x7fc77c019760 comp rx=0 tx=0).stop 2024-04-25T08:46:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 shutdown_connections 2024-04-25T08:46:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc764066f60 0x7fc764069420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc78415a750 0x7fc7841531b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7840cc9e0 0x7fc78414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 --2- 172.21.15.46:0/3856982105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc7840aed10 0x7fc7840b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 >> 172.21.15.46:0/3856982105 conn(0x7fc7840b12a0 msgr2=0x7fc784071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 shutdown_connections 2024-04-25T08:46:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:55.821+0000 7fc78c600640 1 -- 172.21.15.46:0/3856982105 wait complete. 2024-04-25T08:46:55.835 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:46:55.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:55.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:46:55.900 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:46:55.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:46:55.901 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:46:55.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f8800640 1 Processor -- start 2024-04-25T08:46:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f8800640 1 -- start start 2024-04-25T08:46:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f8800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60f0099a00 0x7f60f0099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f8800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60f00ccde0 0x7f60f00cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f8800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f009a320 con 0x7f60f00b2240 2024-04-25T08:46:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f8800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f006db90 con 0x7f60f0099a00 2024-04-25T08:46:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f8800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f006bdc0 con 0x7f60f00ccde0 2024-04-25T08:46:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60f0099a00 0x7f60f0099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60f0099a00 0x7f60f0099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39960/0 (socket says 172.21.15.46:39960) 2024-04-25T08:46:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f5e00640 1 -- 172.21.15.46:0/2860555821 learned_addr learned my addr 172.21.15.46:0/2860555821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f5400640 1 --2- 172.21.15.46:0/2860555821 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60f00ccde0 0x7f60f00cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f5e00640 1 -- 172.21.15.46:0/2860555821 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60f00ccde0 msgr2=0x7f60f00cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f5e00640 1 --2- 172.21.15.46:0/2860555821 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60f00ccde0 0x7f60f00cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f5e00640 1 -- 172.21.15.46:0/2860555821 >> v1:172.21.15.46:6789/0 conn(0x7f60f00b2240 legacy=0x7f60f00b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f5e00640 1 -- 172.21.15.46:0/2860555821 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f006c0a0 con 0x7f60f0099a00 2024-04-25T08:46:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.109+0000 7f60f4a00640 1 -- 172.21.15.46:0/2860555821 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132855804 0 0) 0x7f60f009a320 con 0x7f60f00b2240 2024-04-25T08:46:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f5e00640 1 --2- 172.21.15.46:0/2860555821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 0x7f60f0099de0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009870 tx=0x7f60e401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a7cd7408a58490 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f4a00640 1 -- 172.21.15.46:0/2860555821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e40aa600 con 0x7f60f0099a00 2024-04-25T08:46:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f4a00640 1 -- 172.21.15.46:0/2860555821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e40af070 con 0x7f60f0099a00 2024-04-25T08:46:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f4a00640 1 -- 172.21.15.46:0/2860555821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e40aa9c0 con 0x7f60f0099a00 2024-04-25T08:46:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f8800640 1 -- 172.21.15.46:0/2860555821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 msgr2=0x7f60f0099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f8800640 1 --2- 172.21.15.46:0/2860555821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 0x7f60f0099de0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009870 tx=0x7f60e401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f8800640 1 -- 172.21.15.46:0/2860555821 shutdown_connections 2024-04-25T08:46:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f8800640 1 --2- 172.21.15.46:0/2860555821 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60f00ccde0 0x7f60f00cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f8800640 1 --2- 172.21.15.46:0/2860555821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 0x7f60f0099de0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f8800640 1 -- 172.21.15.46:0/2860555821 >> 172.21.15.46:0/2860555821 conn(0x7f60f00b3120 msgr2=0x7f60f00b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f8800640 1 -- 172.21.15.46:0/2860555821 shutdown_connections 2024-04-25T08:46:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f8800640 1 -- 172.21.15.46:0/2860555821 wait complete. 2024-04-25T08:46:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.113+0000 7f60f8800640 1 Processor -- start 2024-04-25T08:46:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 -- start start 2024-04-25T08:46:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 0x7f60f01607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60f00b2240 0x7f60f01544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60f00ccde0 0x7f60f00af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00cd7a0 con 0x7f60f0099a00 2024-04-25T08:46:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00cdb80 con 0x7f60f00ccde0 2024-04-25T08:46:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f00cd920 con 0x7f60f00b2240 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 0x7f60f01607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 0x7f60f01607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39976/0 (socket says 172.21.15.46:39976) 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f5e00640 1 -- 172.21.15.46:0/2587950348 learned_addr learned my addr 172.21.15.46:0/2587950348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f5e00640 1 -- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60f00b2240 msgr2=0x7f60f01544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f5e00640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60f00b2240 0x7f60f01544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f5e00640 1 -- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60f00ccde0 msgr2=0x7f60f00af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f5e00640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60f00ccde0 0x7f60f00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f5e00640 1 -- 172.21.15.46:0/2587950348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f015d0d0 con 0x7f60f0099a00 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f5e00640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 0x7f60f01607b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f60e401ab50 tx=0x7f60e401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60ea000640 1 -- 172.21.15.46:0/2587950348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e40bd5f0 con 0x7f60f0099a00 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60ea000640 1 -- 172.21.15.46:0/2587950348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e40bd7b0 con 0x7f60f0099a00 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60ea000640 1 -- 172.21.15.46:0/2587950348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e40bdba0 con 0x7f60f0099a00 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f015d3b0 con 0x7f60f0099a00 2024-04-25T08:46:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f00c5cd0 con 0x7f60f0099a00 2024-04-25T08:46:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.117+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f015d580 con 0x7f60f0099a00 2024-04-25T08:46:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.121+0000 7f60ea000640 1 -- 172.21.15.46:0/2587950348 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60e40a9070 con 0x7f60f0099a00 2024-04-25T08:46:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.121+0000 7f60ea000640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60d4066f60 0x7f60d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.121+0000 7f60ea000640 1 -- 172.21.15.46:0/2587950348 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60e411f9f0 con 0x7f60f0099a00 2024-04-25T08:46:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.121+0000 7f60f5400640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60d4066f60 0x7f60d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.121+0000 7f60f5400640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60d4066f60 0x7f60d4069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f60f00af3c0 tx=0x7f60e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.125+0000 7f60ea000640 1 -- 172.21.15.46:0/2587950348 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60f015d580 con 0x7f60f0099a00 2024-04-25T08:46:56.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60f0069080 con 0x7f60f0099a00 2024-04-25T08:46:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60ea000640 1 -- 172.21.15.46:0/2587950348 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f60e411f070 con 0x7f60f0099a00 2024-04-25T08:46:56.315 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:56.315 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:46:49.102068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6774},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6774":{"gid":6774,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/960745191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":960745191},{"type":"v1","addr":"172.21.15.46:6835","nonce":960745191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:46:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 45 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60d4066f60 msgr2=0x7f60d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60d4066f60 0x7f60d4069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f60f00af3c0 tx=0x7f60e0008040 comp rx=0 tx=0).stop 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 msgr2=0x7f60f01607b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 0x7f60f01607b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f60e401ab50 tx=0x7f60e401ace0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 shutdown_connections 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f60d4066f60 0x7f60d4069420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60f00ccde0 0x7f60f00af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f60f00b2240 0x7f60f01544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 --2- 172.21.15.46:0/2587950348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60f0099a00 0x7f60f01607b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 >> 172.21.15.46:0/2587950348 conn(0x7f60f00b3120 msgr2=0x7f60f0155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 shutdown_connections 2024-04-25T08:46:56.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.321+0000 7f60f8800640 1 -- 172.21.15.46:0/2587950348 wait complete. 2024-04-25T08:46:56.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:46:56.511 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-25T08:46:56.511 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:46:56.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:46:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd5ce00640 1 Processor -- start 2024-04-25T08:46:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd5ce00640 1 -- start start 2024-04-25T08:46:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd5ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5809a610 0x7fbd5809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd5ce00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd58099a00 0x7fbd58099e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd5ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5815f300 con 0x7fbd5815ef20 2024-04-25T08:46:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd5ce00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd580cd8a0 con 0x7fbd58099a00 2024-04-25T08:46:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd5ce00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580cddb0 con 0x7fbd5809a610 2024-04-25T08:46:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbd5815ef20 0x7fbd580cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34986/0 (socket says 172.21.15.46:34986) 2024-04-25T08:46:56.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55e00640 1 -- 172.21.15.46:0/2809332490 learned_addr learned my addr 172.21.15.46:0/2809332490 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd56800640 1 --2- 172.21.15.46:0/2809332490 >> v2:172.21.15.82:3300/0 conn(0x7fbd58099a00 0x7fbd58099e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd4fe00640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5809a610 0x7fbd5809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202003288 0 0) 0x7fbd5815f300 con 0x7fbd5815ef20 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd3c003530 con 0x7fbd5815ef20 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1003431141 0 0) 0x7fbd3c003530 con 0x7fbd5815ef20 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5809a610 msgr2=0x7fbd5809aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5809a610 0x7fbd5809aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 >> v2:172.21.15.82:3300/0 conn(0x7fbd58099a00 msgr2=0x7fbd58099e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 --2- 172.21.15.46:0/2809332490 >> v2:172.21.15.82:3300/0 conn(0x7fbd58099a00 0x7fbd58099e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd580cc3d0 con 0x7fbd5815ef20 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fbd48001680 con 0x7fbd5815ef20 2024-04-25T08:46:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd56800640 1 --2- 172.21.15.46:0/2809332490 >> v2:172.21.15.82:3300/0 conn(0x7fbd58099a00 0x7fbd58099e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 >> v1:172.21.15.46:6789/0 conn(0x7fbd5815ef20 legacy=0x7fbd580cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c0039c0 0x7fbd3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5809a610 0x7fbd3c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c004d50 0x7fbd3c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580cd8a0 con 0x7fbd3c0039c0 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580cddb0 con 0x7fbd3c004d50 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.661+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c00b9d0 con 0x7fbd5809a610 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55e00640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c0039c0 0x7fbd3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd4fe00640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5809a610 0x7fbd3c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd56800640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c004d50 0x7fbd3c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55e00640 1 -- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5809a610 msgr2=0x7fbd3c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55e00640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5809a610 0x7fbd3c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55e00640 1 -- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c004d50 msgr2=0x7fbd3c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:56.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55e00640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c004d50 0x7fbd3c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55e00640 1 -- 172.21.15.46:0/2809332490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c003530 con 0x7fbd3c0039c0 2024-04-25T08:46:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd56800640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c004d50 0x7fbd3c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55e00640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c0039c0 0x7fbd3c003de0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800c190 tx=0x7fbd4801cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7be488adc297d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48007c90 con 0x7fbd3c0039c0 2024-04-25T08:46:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd480a9020 con 0x7fbd3c0039c0 2024-04-25T08:46:56.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd55400640 1 -- 172.21.15.46:0/2809332490 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48005ae0 con 0x7fbd3c0039c0 2024-04-25T08:46:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd5ce00640 1 -- 172.21.15.46:0/2809332490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c0039c0 msgr2=0x7fbd3c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd5ce00640 1 --2- 172.21.15.46:0/2809332490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd3c0039c0 0x7fbd3c003de0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800c190 tx=0x7fbd4801cc00 comp rx=0 tx=0).stop 2024-04-25T08:46:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.665+0000 7fbd56800640 1 -- 172.21.15.46:0/2809332490 reap_dead start 2024-04-25T08:46:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 -- 172.21.15.46:0/2809332490 shutdown_connections 2024-04-25T08:46:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 -- 172.21.15.46:0/2809332490 >> 172.21.15.46:0/2809332490 conn(0x7fbd580b3120 msgr2=0x7fbd580b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 -- 172.21.15.46:0/2809332490 shutdown_connections 2024-04-25T08:46:56.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 -- 172.21.15.46:0/2809332490 wait complete. 2024-04-25T08:46:56.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 Processor -- start 2024-04-25T08:46:56.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 -- start start 2024-04-25T08:46:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c0039c0 0x7fbd5815e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5815a040 0x7fbd580cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd5815ca00 0x7fbd5815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580cc010 con 0x7fbd5815ca00 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5809ae00 con 0x7fbd3c0039c0 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd5ce00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580cc190 con 0x7fbd5815a040 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd5815ca00 0x7fbd5815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd5815ca00 0x7fbd5815ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39992/0 (socket says 172.21.15.46:39992) 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd56800640 1 -- 172.21.15.46:0/1558628667 learned_addr learned my addr 172.21.15.46:0/1558628667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd4fe00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5815a040 0x7fbd580cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd55e00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c0039c0 0x7fbd5815e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd56800640 1 -- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5815a040 msgr2=0x7fbd580cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd56800640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5815a040 0x7fbd580cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd56800640 1 -- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c0039c0 msgr2=0x7fbd5815e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd56800640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c0039c0 0x7fbd5815e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd56800640 1 -- 172.21.15.46:0/1558628667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd580c9110 con 0x7fbd5815ca00 2024-04-25T08:46:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.669+0000 7fbd56800640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd5815ca00 0x7fbd5815ceb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009f30 tx=0x7fbd500115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd4e000640 1 -- 172.21.15.46:0/1558628667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50011e40 con 0x7fbd5815ca00 2024-04-25T08:46:56.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd4e000640 1 -- 172.21.15.46:0/1558628667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5001b070 con 0x7fbd5815ca00 2024-04-25T08:46:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd4e000640 1 -- 172.21.15.46:0/1558628667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd500039a0 con 0x7fbd5815ca00 2024-04-25T08:46:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd580c93b0 con 0x7fbd5815ca00 2024-04-25T08:46:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd580c9970 con 0x7fbd5815ca00 2024-04-25T08:46:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd58069080 con 0x7fbd5815ca00 2024-04-25T08:46:56.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd4e000640 1 -- 172.21.15.46:0/1558628667 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd50003b40 con 0x7fbd5815ca00 2024-04-25T08:46:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd4e000640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd34066f60 0x7fbd34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd4e000640 1 -- 172.21.15.46:0/1558628667 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbd50085840 con 0x7fbd5815ca00 2024-04-25T08:46:56.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.673+0000 7fbd55e00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd34066f60 0x7fbd34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.677+0000 7fbd55e00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd34066f60 0x7fbd34069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbd480aa000 tx=0x7fbd480b1040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.677+0000 7fbd4e000640 1 -- 172.21.15.46:0/1558628667 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd500566b0 con 0x7fbd5815ca00 2024-04-25T08:46:56.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.877+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbd580cb9c0 con 0x7fbd5815ca00 2024-04-25T08:46:56.872 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:46:56.877+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:46:56.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.881+0000 7fbd4e000640 1 -- 172.21.15.46:0/1558628667 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbd50055fb0 con 0x7fbd5815ca00 2024-04-25T08:46:56.874 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.881+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd34066f60 msgr2=0x7fbd34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.881+0000 7fbd5ce00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd34066f60 0x7fbd34069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbd480aa000 tx=0x7fbd480b1040 comp rx=0 tx=0).stop 2024-04-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.881+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd5815ca00 msgr2=0x7fbd5815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.881+0000 7fbd5ce00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd5815ca00 0x7fbd5815ceb0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbd50009f30 tx=0x7fbd500115a0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.885+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 shutdown_connections 2024-04-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.885+0000 7fbd5ce00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd34066f60 0x7fbd34069420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.885+0000 7fbd5ce00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd5815ca00 0x7fbd5815ceb0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.885+0000 7fbd5ce00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd5815a040 0x7fbd580cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.885+0000 7fbd5ce00640 1 --2- 172.21.15.46:0/1558628667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd3c0039c0 0x7fbd5815e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.885+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 >> 172.21.15.46:0/1558628667 conn(0x7fbd580b3120 msgr2=0x7fbd580c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.885+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 shutdown_connections 2024-04-25T08:46:56.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.885+0000 7fbd5ce00640 1 -- 172.21.15.46:0/1558628667 wait complete. 2024-04-25T08:46:56.885 INFO:tasks.ceph.mds.c.smithi046.stderr: -4466> 2024-04-25T08:46:48.669+0000 7f590c600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:46:56.898 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T08:46:56.936 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:46:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f9890ac1080 1 Processor -- start 2024-04-25T08:46:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f9890ac1080 1 -- start start 2024-04-25T08:46:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f9890ac1080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b438c37d30 0x55b438c540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f9890ac1080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c3a6f0 0x55b438b814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f9890ac1080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 0x55b438c53210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f9890ac1080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b438c38240 con 0x55b438c3a6f0 2024-04-25T08:46:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f9890ac1080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b438b269f0 con 0x55b438c3aad0 2024-04-25T08:46:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f9890ac1080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b438c384e0 con 0x55b438c37d30 2024-04-25T08:46:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f988d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c3a6f0 0x55b438b814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f988d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c3a6f0 0x55b438b814e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40040/0 (socket says 172.21.15.46:40040) 2024-04-25T08:46:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f988d200640 1 -- 172.21.15.46:0/388529340 learned_addr learned my addr 172.21.15.46:0/388529340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f988dc00640 1 --2- 172.21.15.46:0/388529340 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b438c37d30 0x55b438c540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.953+0000 7f988e600640 1 --2- 172.21.15.46:0/388529340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 0x55b438c53210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988d200640 1 -- 172.21.15.46:0/388529340 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b438c37d30 msgr2=0x55b438c540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988d200640 1 --2- 172.21.15.46:0/388529340 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b438c37d30 0x55b438c540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988d200640 1 -- 172.21.15.46:0/388529340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 msgr2=0x55b438c53210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988d200640 1 --2- 172.21.15.46:0/388529340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 0x55b438c53210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988d200640 1 -- 172.21.15.46:0/388529340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b438bc64b0 con 0x55b438c3a6f0 2024-04-25T08:46:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988d200640 1 --2- 172.21.15.46:0/388529340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c3a6f0 0x55b438b814e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f987c01a410 tx=0x7f987c002fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c2c9548d56c3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9887400640 1 -- 172.21.15.46:0/388529340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0ae040 con 0x55b438c3a6f0 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9887400640 1 -- 172.21.15.46:0/388529340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c0085a0 con 0x55b438c3a6f0 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9887400640 1 -- 172.21.15.46:0/388529340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c008970 con 0x55b438c3a6f0 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 -- 172.21.15.46:0/388529340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c3a6f0 msgr2=0x55b438b814e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 --2- 172.21.15.46:0/388529340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c3a6f0 0x55b438b814e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f987c01a410 tx=0x7f987c002fc0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 -- 172.21.15.46:0/388529340 shutdown_connections 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 --2- 172.21.15.46:0/388529340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 0x55b438c53210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 --2- 172.21.15.46:0/388529340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c3a6f0 0x55b438b814e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 --2- 172.21.15.46:0/388529340 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b438c37d30 0x55b438c540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 -- 172.21.15.46:0/388529340 >> 172.21.15.46:0/388529340 conn(0x55b438b80230 msgr2=0x55b438babc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 -- 172.21.15.46:0/388529340 shutdown_connections 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 -- 172.21.15.46:0/388529340 wait complete. 2024-04-25T08:46:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 Processor -- start 2024-04-25T08:46:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 -- start start 2024-04-25T08:46:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c37d30 0x55b438bc1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b438c3a6f0 0x55b438c564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 0x55b438c372e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b438b80da0 con 0x55b438c37d30 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b438b81040 con 0x55b438c3aad0 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f9890ac1080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b438bc2490 con 0x55b438c3a6f0 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c37d30 0x55b438bc1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c37d30 0x55b438bc1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40050/0 (socket says 172.21.15.46:40050) 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988dc00640 1 -- 172.21.15.46:0/2857158878 learned_addr learned my addr 172.21.15.46:0/2857158878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988d200640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b438c3a6f0 0x55b438c564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988e600640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 0x55b438c372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988dc00640 1 -- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b438c3a6f0 msgr2=0x55b438c564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988dc00640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55b438c3a6f0 0x55b438c564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988dc00640 1 -- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 msgr2=0x55b438c372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988dc00640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 0x55b438c372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988dc00640 1 -- 172.21.15.46:0/2857158878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b438c569f0 con 0x55b438c37d30 2024-04-25T08:46:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.957+0000 7f988e600640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b438c3aad0 0x55b438c372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:46:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f988dc00640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c37d30 0x55b438bc1f50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f988000c450 tx=0x7f988000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9886000640 1 -- 172.21.15.46:0/2857158878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000b830 con 0x55b438c37d30 2024-04-25T08:46:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9886000640 1 -- 172.21.15.46:0/2857158878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988000b9d0 con 0x55b438c37d30 2024-04-25T08:46:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b438bc0d90 con 0x55b438c37d30 2024-04-25T08:46:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9886000640 1 -- 172.21.15.46:0/2857158878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000bdc0 con 0x55b438c37d30 2024-04-25T08:46:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b438bc12d0 con 0x55b438c37d30 2024-04-25T08:46:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9886000640 1 -- 172.21.15.46:0/2857158878 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9880020440 con 0x55b438c37d30 2024-04-25T08:46:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9886000640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9870065970 0x7f9870067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9886000640 1 -- 172.21.15.46:0/2857158878 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9880084920 con 0x55b438c37d30 2024-04-25T08:46:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9890ac1080 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55b438c37820 0x55b438bc3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f988d200640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9870065970 0x7f9870067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79) v9 -- 0x55b438bc3c90 con 0x55b438c37820 2024-04-25T08:46:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.961+0000 7f988e600640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55b438c37820 0x55b438bc3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.965+0000 7f988d200640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9870065970 0x7f9870067e30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f987c0bd000 tx=0x7f987c0ab040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.965+0000 7f988e600640 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55b438c37820 0x55b438bc3750 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.965+0000 7f988e600640 1 -- 172.21.15.46:0/2857158878 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v79'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f988800c9b0 con 0x55b438c37820 2024-04-25T08:46:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.965+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55b438c37820 msgr2=0x55b438bc3750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.965+0000 7f9890ac1080 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55b438c37820 0x55b438bc3750 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.969+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9870065970 msgr2=0x7f9870067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.969+0000 7f9890ac1080 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9870065970 0x7f9870067e30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f987c0bd000 tx=0x7f987c0ab040 comp rx=0 tx=0).stop 2024-04-25T08:46:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.969+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c37d30 msgr2=0x55b438bc1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.969+0000 7f9890ac1080 1 --2- 172.21.15.46:0/2857158878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b438c37d30 0x55b438bc1f50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f988000c450 tx=0x7f988000c910 comp rx=0 tx=0).stop 2024-04-25T08:46:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.969+0000 7f988e600640 1 -- 172.21.15.46:0/2857158878 reap_dead start 2024-04-25T08:46:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.969+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 shutdown_connections 2024-04-25T08:46:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.969+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 >> 172.21.15.46:0/2857158878 conn(0x55b438b80230 msgr2=0x55b438c527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.969+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 shutdown_connections 2024-04-25T08:46:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:56.969+0000 7f9890ac1080 1 -- 172.21.15.46:0/2857158878 wait complete. 2024-04-25T08:46:56.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T08:46:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.161+0000 7fe311c00640 1 Processor -- start 2024-04-25T08:46:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.161+0000 7fe311c00640 1 -- start start 2024-04-25T08:46:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe311c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe30c0cd6f0 0x7fe30c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe311c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe30c06f1d0 0x7fe30c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe311c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 0x7fe30c1616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe311c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c0ce010 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe311c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c06da90 con 0x7fe30c06f1d0 2024-04-25T08:46:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe311c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c06dd70 con 0x7fe30c1612c0 2024-04-25T08:46:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe30be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe30c0cd6f0 0x7fe30c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe30be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe30c0cd6f0 0x7fe30c15e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40064/0 (socket says 172.21.15.46:40064) 2024-04-25T08:46:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe30be00640 1 -- 172.21.15.46:0/2935552935 learned_addr learned my addr 172.21.15.46:0/2935552935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe311200640 1 --2- 172.21.15.46:0/2935552935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 0x7fe30c1616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe303e00640 1 --2- 172.21.15.46:0/2935552935 >> v2:172.21.15.82:3300/0 conn(0x7fe30c06f1d0 0x7fe30c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe30be00640 1 -- 172.21.15.46:0/2935552935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 msgr2=0x7fe30c1616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe30be00640 1 --2- 172.21.15.46:0/2935552935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 0x7fe30c1616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe30be00640 1 -- 172.21.15.46:0/2935552935 >> v2:172.21.15.82:3300/0 conn(0x7fe30c06f1d0 msgr2=0x7fe30c0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe30be00640 1 --2- 172.21.15.46:0/2935552935 >> v2:172.21.15.82:3300/0 conn(0x7fe30c06f1d0 0x7fe30c0cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe30be00640 1 -- 172.21.15.46:0/2935552935 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c06e050 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe303e00640 1 --2- 172.21.15.46:0/2935552935 >> v2:172.21.15.82:3300/0 conn(0x7fe30c06f1d0 0x7fe30c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:46:57.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.165+0000 7fe30be00640 1 --2- 172.21.15.46:0/2935552935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 0x7fe30c15e750 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0710f0 tx=0x7fe2fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48da160a2852ce37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe30b400640 1 -- 172.21.15.46:0/2935552935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0ab5f0 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe30b400640 1 -- 172.21.15.46:0/2935552935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc0b0070 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe30b400640 1 -- 172.21.15.46:0/2935552935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0ab9b0 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 -- 172.21.15.46:0/2935552935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 msgr2=0x7fe30c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 --2- 172.21.15.46:0/2935552935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 0x7fe30c15e750 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0710f0 tx=0x7fe2fc01c520 comp rx=0 tx=0).stop 2024-04-25T08:46:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 -- 172.21.15.46:0/2935552935 shutdown_connections 2024-04-25T08:46:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 --2- 172.21.15.46:0/2935552935 >> v2:172.21.15.82:3300/0 conn(0x7fe30c06f1d0 0x7fe30c0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 --2- 172.21.15.46:0/2935552935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 0x7fe30c1616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 --2- 172.21.15.46:0/2935552935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 0x7fe30c15e750 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 -- 172.21.15.46:0/2935552935 >> 172.21.15.46:0/2935552935 conn(0x7fe30c0cc5a0 msgr2=0x7fe30c0cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 -- 172.21.15.46:0/2935552935 shutdown_connections 2024-04-25T08:46:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 -- 172.21.15.46:0/2935552935 wait complete. 2024-04-25T08:46:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 Processor -- start 2024-04-25T08:46:57.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 -- start start 2024-04-25T08:46:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30c06f1d0 0x7fe30c149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 0x7fe30c1491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 0x7fe30c149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c15b630 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c15b4b0 con 0x7fe30c06f1d0 2024-04-25T08:46:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c06b520 con 0x7fe30c1612c0 2024-04-25T08:46:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe303e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 0x7fe30c1491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe303e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 0x7fe30c1491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40072/0 (socket says 172.21.15.46:40072) 2024-04-25T08:46:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe303e00640 1 -- 172.21.15.46:0/534356519 learned_addr learned my addr 172.21.15.46:0/534356519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe311200640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 0x7fe30c149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.169+0000 7fe30be00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30c06f1d0 0x7fe30c149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe303e00640 1 -- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 msgr2=0x7fe30c149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe303e00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 0x7fe30c149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe303e00640 1 -- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30c06f1d0 msgr2=0x7fe30c149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe303e00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30c06f1d0 0x7fe30c149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe303e00640 1 -- 172.21.15.46:0/534356519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c160900 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe303e00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 0x7fe30c1491c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400c7f0 tx=0x7fe2f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:57.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe308c00640 1 -- 172.21.15.46:0/534356519 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40079d0 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe308c00640 1 -- 172.21.15.46:0/534356519 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f4007b70 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c160be0 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe308c00640 1 -- 172.21.15.46:0/534356519 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40113f0 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c0aca30 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c069010 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe308c00640 1 -- 172.21.15.46:0/534356519 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2f4011590 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe308c00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e8066f60 0x7fe2e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe308c00640 1 -- 172.21.15.46:0/534356519 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe2f4085490 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.173+0000 7fe30be00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e8066f60 0x7fe2e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:57.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.177+0000 7fe30be00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e8066f60 0x7fe2e8069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0af000 tx=0x7fe2fc0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.177+0000 7fe308c00640 1 -- 172.21.15.46:0/534356519 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2f4015020 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.377+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fe30c0acc00 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.881+0000 7fe308c00640 1 -- 172.21.15.46:0/534356519 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v47) v1 ==== 162+0+0 (secure 0 0 0) 0x7fe2f4056220 con 0x7fe30c0cd6f0 2024-04-25T08:46:57.877 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T08:46:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e8066f60 msgr2=0x7fe2e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e8066f60 0x7fe2e8069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc0af000 tx=0x7fe2fc0a7000 comp rx=0 tx=0).stop 2024-04-25T08:46:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 msgr2=0x7fe30c1491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 0x7fe30c1491c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400c7f0 tx=0x7fe2f400ccb0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 shutdown_connections 2024-04-25T08:46:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e8066f60 0x7fe2e8069420 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe30c1612c0 0x7fe30c149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30c0cd6f0 0x7fe30c1491c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 --2- 172.21.15.46:0/534356519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30c06f1d0 0x7fe30c149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 >> 172.21.15.46:0/534356519 conn(0x7fe30c0cc5a0 msgr2=0x7fe30c073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 shutdown_connections 2024-04-25T08:46:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:57.885+0000 7fe311c00640 1 -- 172.21.15.46:0/534356519 wait complete. 2024-04-25T08:46:57.904 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:58.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6c0200640 1 Processor -- start 2024-04-25T08:46:58.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6c0200640 1 -- start start 2024-04-25T08:46:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6c0200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe6b80cc1e0 0x7fe6b815f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:58.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6c0200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 0x7fe6b815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6c0200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b806b7d0 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6c0200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b815be90 con 0x7fe6b80cc1e0 2024-04-25T08:46:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6c0200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b815c170 con 0x7fe6b80cf3a0 2024-04-25T08:46:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6bce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe6b80cc1e0 0x7fe6b815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6bce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe6b80cc1e0 0x7fe6b815f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40082/0 (socket says 172.21.15.46:40082) 2024-04-25T08:46:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6bce00640 1 -- 172.21.15.46:0/4156577610 learned_addr learned my addr 172.21.15.46:0/4156577610 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6be200640 1 --2- 172.21.15.46:0/4156577610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 0x7fe6b815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:58.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6bce00640 1 -- 172.21.15.46:0/4156577610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 msgr2=0x7fe6b815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6bce00640 1 --2- 172.21.15.46:0/4156577610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 0x7fe6b815f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6bce00640 1 -- 172.21.15.46:0/4156577610 >> v1:172.21.15.82:6789/0 conn(0x7fe6b80cf8a0 legacy=0x7fe6b80cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:58.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.053+0000 7fe6bce00640 1 -- 172.21.15.46:0/4156577610 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b815c450 con 0x7fe6b80cc1e0 2024-04-25T08:46:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6bce00640 1 --2- 172.21.15.46:0/4156577610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cc1e0 0x7fe6b815f370 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009870 tx=0x7fe6a401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a849c719663853af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6b7e00640 1 -- 172.21.15.46:0/4156577610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40b1070 con 0x7fe6b80cc1e0 2024-04-25T08:46:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6b7e00640 1 -- 172.21.15.46:0/4156577610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a40a8050 con 0x7fe6b80cc1e0 2024-04-25T08:46:58.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6b7e00640 1 -- 172.21.15.46:0/4156577610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40ac860 con 0x7fe6b80cc1e0 2024-04-25T08:46:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6c0200640 1 -- 172.21.15.46:0/4156577610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cc1e0 msgr2=0x7fe6b815f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6c0200640 1 --2- 172.21.15.46:0/4156577610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cc1e0 0x7fe6b815f370 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009870 tx=0x7fe6a401c4d0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6c0200640 1 -- 172.21.15.46:0/4156577610 shutdown_connections 2024-04-25T08:46:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6c0200640 1 --2- 172.21.15.46:0/4156577610 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 0x7fe6b815f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6c0200640 1 --2- 172.21.15.46:0/4156577610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cc1e0 0x7fe6b815f370 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6c0200640 1 -- 172.21.15.46:0/4156577610 >> 172.21.15.46:0/4156577610 conn(0x7fe6b8099a00 msgr2=0x7fe6b80b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6c0200640 1 -- 172.21.15.46:0/4156577610 shutdown_connections 2024-04-25T08:46:58.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.057+0000 7fe6c0200640 1 -- 172.21.15.46:0/4156577610 wait complete. 2024-04-25T08:46:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 Processor -- start 2024-04-25T08:46:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 -- start start 2024-04-25T08:46:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80cc1e0 0x7fe6b8155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 0x7fe6b80c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cf8a0 0x7fe6b80ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80b2ab0 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80b2930 con 0x7fe6b80cc1e0 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b8168f20 con 0x7fe6b80cf3a0 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6be200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cf8a0 0x7fe6b80ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6be200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cf8a0 0x7fe6b80ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47600/0 (socket says 172.21.15.46:47600) 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6be200640 1 -- 172.21.15.46:0/2550928382 learned_addr learned my addr 172.21.15.46:0/2550928382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6bce00640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 0x7fe6b80c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6bd800640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80cc1e0 0x7fe6b8155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6be200640 1 -- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 msgr2=0x7fe6b80c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6be200640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 0x7fe6b80c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6be200640 1 -- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80cc1e0 msgr2=0x7fe6b8155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6be200640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80cc1e0 0x7fe6b8155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6be200640 1 -- 172.21.15.46:0/2550928382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b81f6e30 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6bd800640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80cc1e0 0x7fe6b8155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:46:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6be200640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cf8a0 0x7fe6b80ca0f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000efc0 tx=0x7fe6b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6b5600640 1 -- 172.21.15.46:0/2550928382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0018020 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6b5600640 1 -- 172.21.15.46:0/2550928382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0014480 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6b5600640 1 -- 172.21.15.46:0/2550928382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0014870 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b81f7110 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b81f76d0 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.061+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b8069080 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.065+0000 7fe6b5600640 1 -- 172.21.15.46:0/2550928382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6b001b020 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.065+0000 7fe6b5600640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe698066f60 0x7fe698069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.065+0000 7fe6b5600640 1 -- 172.21.15.46:0/2550928382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe6b0085cc0 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.065+0000 7fe6bd800640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe698066f60 0x7fe698069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.065+0000 7fe6bd800640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe698066f60 0x7fe698069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe6b81665c0 tx=0x7fe6ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.069+0000 7fe6b5600640 1 -- 172.21.15.46:0/2550928382 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6b008c080 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.265+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6b8072e40 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.265+0000 7fe6b5600640 1 -- 172.21.15.46:0/2550928382 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fe6b0022070 con 0x7fe6b80cf8a0 2024-04-25T08:46:58.260 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:46:58.260 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:46:57.889354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"c","rank":0,"incarnation":48,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/4112568749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":4112568749},{"type":"v1","addr":"172.21.15.46:6835","nonce":4112568749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:46:58.260 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 48 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe698066f60 msgr2=0x7fe698069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe698066f60 0x7fe698069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe6b81665c0 tx=0x7fe6ac009040 comp rx=0 tx=0).stop 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cf8a0 msgr2=0x7fe6b80ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cf8a0 0x7fe6b80ca0f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000efc0 tx=0x7fe6b000c490 comp rx=0 tx=0).stop 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 shutdown_connections 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe698066f60 0x7fe698069420 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b80cf8a0 0x7fe6b80ca0f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe6b80cf3a0 0x7fe6b80c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 --2- 172.21.15.46:0/2550928382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6b80cc1e0 0x7fe6b8155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 >> 172.21.15.46:0/2550928382 conn(0x7fe6b8099a00 msgr2=0x7fe6b815d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 shutdown_connections 2024-04-25T08:46:58.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:58.269+0000 7fe6c0200640 1 -- 172.21.15.46:0/2550928382 wait complete. 2024-04-25T08:46:58.683 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:46:46.077710+0000', 'modified': '2024-04-25T08:46:57.889354+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 79, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6990': {'gid': 6990, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/4112568749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 4112568749}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 4112568749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:46:58.684 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6990:up:replay 2024-04-25T08:46:59.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:46:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.841+0000 7fad25a00640 1 Processor -- start 2024-04-25T08:46:59.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.841+0000 7fad25a00640 1 -- start start 2024-04-25T08:46:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad25a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 0x7fad2006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad25a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fad200cc9e0 0x7fad2006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad25a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2006d5e0 con 0x7fad200aed10 2024-04-25T08:46:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad25a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2015ab30 con 0x7fad200cc9e0 2024-04-25T08:46:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad25a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad200ce4f0 con 0x7fad2015a750 2024-04-25T08:46:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad16000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fad200aed10 0x7fad200b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48540/0 (socket says 172.21.15.46:48540) 2024-04-25T08:46:59.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad16000640 1 -- 172.21.15.46:0/3131792876 learned_addr learned my addr 172.21.15.46:0/3131792876 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:46:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad1f400640 1 --2- 172.21.15.46:0/3131792876 >> v2:172.21.15.82:3300/0 conn(0x7fad200cc9e0 0x7fad2006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:59.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad1ea00640 1 --2- 172.21.15.46:0/3131792876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 0x7fad2006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad1e000640 1 -- 172.21.15.46:0/3131792876 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427301719 0 0) 0x7fad2006d5e0 con 0x7fad200aed10 2024-04-25T08:46:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad1e000640 1 -- 172.21.15.46:0/3131792876 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad04005740 con 0x7fad200aed10 2024-04-25T08:46:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad1e000640 1 -- 172.21.15.46:0/3131792876 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 994851300 0 0) 0x7fad04005740 con 0x7fad200aed10 2024-04-25T08:46:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad1ea00640 1 -- 172.21.15.46:0/3131792876 >> v2:172.21.15.82:3300/0 conn(0x7fad200cc9e0 msgr2=0x7fad2006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:46:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad1ea00640 1 --2- 172.21.15.46:0/3131792876 >> v2:172.21.15.82:3300/0 conn(0x7fad200cc9e0 0x7fad2006f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad1ea00640 1 -- 172.21.15.46:0/3131792876 >> v1:172.21.15.46:6789/0 conn(0x7fad200aed10 legacy=0x7fad200b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:59.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.845+0000 7fad1ea00640 1 -- 172.21.15.46:0/3131792876 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200ce720 con 0x7fad2015a750 2024-04-25T08:46:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad1ea00640 1 --2- 172.21.15.46:0/3131792876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 0x7fad2006ec10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad100097c0 tx=0x7fad1001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b54573bef0c66d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad1e000640 1 -- 172.21.15.46:0/3131792876 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100b0070 con 0x7fad2015a750 2024-04-25T08:46:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad1e000640 1 -- 172.21.15.46:0/3131792876 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad100ab400 con 0x7fad2015a750 2024-04-25T08:46:59.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad1e000640 1 -- 172.21.15.46:0/3131792876 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08001680 con 0x7fad2015a750 2024-04-25T08:46:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad25a00640 1 -- 172.21.15.46:0/3131792876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 msgr2=0x7fad2006ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad25a00640 1 --2- 172.21.15.46:0/3131792876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 0x7fad2006ec10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fad100097c0 tx=0x7fad1001c3c0 comp rx=0 tx=0).stop 2024-04-25T08:46:59.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad25a00640 1 -- 172.21.15.46:0/3131792876 shutdown_connections 2024-04-25T08:46:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad25a00640 1 --2- 172.21.15.46:0/3131792876 >> v2:172.21.15.82:3300/0 conn(0x7fad200cc9e0 0x7fad2006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad25a00640 1 --2- 172.21.15.46:0/3131792876 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 0x7fad2006ec10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad25a00640 1 -- 172.21.15.46:0/3131792876 >> 172.21.15.46:0/3131792876 conn(0x7fad200b12a0 msgr2=0x7fad200b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:46:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad25a00640 1 -- 172.21.15.46:0/3131792876 shutdown_connections 2024-04-25T08:46:59.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.849+0000 7fad25a00640 1 -- 172.21.15.46:0/3131792876 wait complete. 2024-04-25T08:46:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 Processor -- start 2024-04-25T08:46:59.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 -- start start 2024-04-25T08:46:59.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad200aed10 0x7fad2015c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad200cc9e0 0x7fad20150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 0x7fad20152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2006db00 con 0x7fad200cc9e0 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2006d270 con 0x7fad200aed10 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2006dc80 con 0x7fad2015a750 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad16000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad200cc9e0 0x7fad20150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad16000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad200cc9e0 0x7fad20150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47610/0 (socket says 172.21.15.46:47610) 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad16000640 1 -- 172.21.15.46:0/3450746604 learned_addr learned my addr 172.21.15.46:0/3450746604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad1f400640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 0x7fad20152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad1ea00640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad200aed10 0x7fad2015c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad16000640 1 -- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 msgr2=0x7fad20152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad16000640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 0x7fad20152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad16000640 1 -- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad200aed10 msgr2=0x7fad2015c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:46:59.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad16000640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad200aed10 0x7fad2015c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:46:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad16000640 1 -- 172.21.15.46:0/3450746604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad20149810 con 0x7fad200cc9e0 2024-04-25T08:46:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad16000640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad200cc9e0 0x7fad20150710 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c730 tx=0x7fad0800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad17400640 1 -- 172.21.15.46:0/3450746604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08017070 con 0x7fad200cc9e0 2024-04-25T08:46:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad17400640 1 -- 172.21.15.46:0/3450746604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad08005ae0 con 0x7fad200cc9e0 2024-04-25T08:46:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad17400640 1 -- 172.21.15.46:0/3450746604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08007070 con 0x7fad200cc9e0 2024-04-25T08:46:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2014a070 con 0x7fad200cc9e0 2024-04-25T08:46:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad20149d90 con 0x7fad200cc9e0 2024-04-25T08:46:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.853+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad20069080 con 0x7fad200cc9e0 2024-04-25T08:46:59.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.857+0000 7fad17400640 1 -- 172.21.15.46:0/3450746604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad0801a070 con 0x7fad200cc9e0 2024-04-25T08:46:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.857+0000 7fad17400640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facfc066f60 0x7facfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:46:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.857+0000 7fad17400640 1 -- 172.21.15.46:0/3450746604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fad080847a0 con 0x7fad200cc9e0 2024-04-25T08:46:59.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.857+0000 7fad1ea00640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facfc066f60 0x7facfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:46:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.857+0000 7fad1ea00640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facfc066f60 0x7facfc069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fad100ad000 tx=0x7fad100a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:46:59.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:46:59.861+0000 7fad17400640 1 -- 172.21.15.46:0/3450746604 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad08054ee0 con 0x7fad200cc9e0 2024-04-25T08:47:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad20073b30 con 0x7fad200cc9e0 2024-04-25T08:47:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad17400640 1 -- 172.21.15.46:0/3450746604 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fad08058f80 con 0x7fad200cc9e0 2024-04-25T08:47:00.059 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:47:00.059 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:46:58.903608+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"c","rank":0,"incarnation":48,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/4112568749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":4112568749},{"type":"v1","addr":"172.21.15.46:6835","nonce":4112568749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:47:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 50 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facfc066f60 msgr2=0x7facfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facfc066f60 0x7facfc069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fad100ad000 tx=0x7fad100a6000 comp rx=0 tx=0).stop 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad200cc9e0 msgr2=0x7fad20150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad200cc9e0 0x7fad20150710 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c730 tx=0x7fad0800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 shutdown_connections 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facfc066f60 0x7facfc069420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad2015a750 0x7fad20152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad200cc9e0 0x7fad20150710 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 --2- 172.21.15.46:0/3450746604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad200aed10 0x7fad2015c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 >> 172.21.15.46:0/3450746604 conn(0x7fad200b12a0 msgr2=0x7fad200cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 shutdown_connections 2024-04-25T08:47:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:00.065+0000 7fad25a00640 1 -- 172.21.15.46:0/3450746604 wait complete. 2024-04-25T08:47:00.482 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:46:46.077710+0000', 'modified': '2024-04-25T08:46:58.903608+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 79, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6990': {'gid': 6990, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/4112568749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 4112568749}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 4112568749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:47:00.483 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6990:up:rejoin 2024-04-25T08:47:01.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:47:01.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.641+0000 7f2dbc000640 1 Processor -- start 2024-04-25T08:47:01.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.641+0000 7f2dbc000640 1 -- start start 2024-04-25T08:47:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2dbc000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2db415a750 0x7f2db406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2dbc000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40aed10 0x7f2db406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:01.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2dbc000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db406d5e0 con 0x7f2db40cc9e0 2024-04-25T08:47:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2dbc000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db415ab30 con 0x7f2db415a750 2024-04-25T08:47:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2dbc000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db40ce4f0 con 0x7f2db40aed10 2024-04-25T08:47:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2dba000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2db40cc9e0 0x7f2db40b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48556/0 (socket says 172.21.15.46:48556) 2024-04-25T08:47:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2dba000640 1 -- 172.21.15.46:0/239190402 learned_addr learned my addr 172.21.15.46:0/239190402 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:47:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2db9600640 1 --2- 172.21.15.46:0/239190402 >> v2:172.21.15.82:3300/0 conn(0x7f2db415a750 0x7f2db406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2db8c00640 1 --2- 172.21.15.46:0/239190402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40aed10 0x7f2db406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2db3e00640 1 -- 172.21.15.46:0/239190402 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4207881562 0 0) 0x7f2db406d5e0 con 0x7f2db40cc9e0 2024-04-25T08:47:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2db3e00640 1 -- 172.21.15.46:0/239190402 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d9c003530 con 0x7f2db40cc9e0 2024-04-25T08:47:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2db9600640 1 -- 172.21.15.46:0/239190402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40aed10 msgr2=0x7f2db406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2db9600640 1 --2- 172.21.15.46:0/239190402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40aed10 0x7f2db406f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2db9600640 1 -- 172.21.15.46:0/239190402 >> v1:172.21.15.46:6789/0 conn(0x7f2db40cc9e0 legacy=0x7f2db40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.645+0000 7f2db9600640 1 -- 172.21.15.46:0/239190402 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db40ce720 con 0x7f2db415a750 2024-04-25T08:47:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2db9600640 1 --2- 172.21.15.46:0/239190402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db415a750 0x7f2db406ec10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2da80097c0 tx=0x7f2da801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a14bdbf472d0a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2db3e00640 1 -- 172.21.15.46:0/239190402 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80b2070 con 0x7f2db415a750 2024-04-25T08:47:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2db3e00640 1 -- 172.21.15.46:0/239190402 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da80ad640 con 0x7f2db415a750 2024-04-25T08:47:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2db3e00640 1 -- 172.21.15.46:0/239190402 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80ada10 con 0x7f2db415a750 2024-04-25T08:47:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 -- 172.21.15.46:0/239190402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db415a750 msgr2=0x7f2db406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 --2- 172.21.15.46:0/239190402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db415a750 0x7f2db406ec10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2da80097c0 tx=0x7f2da801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 -- 172.21.15.46:0/239190402 shutdown_connections 2024-04-25T08:47:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 --2- 172.21.15.46:0/239190402 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40aed10 0x7f2db406f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 --2- 172.21.15.46:0/239190402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db415a750 0x7f2db406ec10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 -- 172.21.15.46:0/239190402 >> 172.21.15.46:0/239190402 conn(0x7f2db40b12a0 msgr2=0x7f2db40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 -- 172.21.15.46:0/239190402 shutdown_connections 2024-04-25T08:47:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 -- 172.21.15.46:0/239190402 wait complete. 2024-04-25T08:47:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 Processor -- start 2024-04-25T08:47:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.649+0000 7f2dbc000640 1 -- start start 2024-04-25T08:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dbc000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db40aed10 0x7f2db415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dbc000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40cc9e0 0x7f2db4156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dbc000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db415a750 0x7f2db4143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dbc000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4167230 con 0x7f2db415a750 2024-04-25T08:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dbc000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db406d270 con 0x7f2db40aed10 2024-04-25T08:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dbc000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db41673b0 con 0x7f2db40cc9e0 2024-04-25T08:47:01.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dba000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db415a750 0x7f2db4143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dba000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db415a750 0x7f2db4143ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47616/0 (socket says 172.21.15.46:47616) 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dba000640 1 -- 172.21.15.46:0/2605147419 learned_addr learned my addr 172.21.15.46:0/2605147419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2db9600640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db40aed10 0x7f2db415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2db8c00640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40cc9e0 0x7f2db4156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dba000640 1 -- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40cc9e0 msgr2=0x7f2db4156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dba000640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40cc9e0 0x7f2db4156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dba000640 1 -- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db40aed10 msgr2=0x7f2db415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dba000640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db40aed10 0x7f2db415c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dba000640 1 -- 172.21.15.46:0/2605147419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db415c8c0 con 0x7f2db415a750 2024-04-25T08:47:01.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dba000640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db415a750 0x7f2db4143ef0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00cb40 tx=0x7f2dac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2db1600640 1 -- 172.21.15.46:0/2605147419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac017070 con 0x7f2db415a750 2024-04-25T08:47:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2db1600640 1 -- 172.21.15.46:0/2605147419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac00aca0 con 0x7f2db415a750 2024-04-25T08:47:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db415cba0 con 0x7f2db415a750 2024-04-25T08:47:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2db1600640 1 -- 172.21.15.46:0/2605147419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac012620 con 0x7f2db415a750 2024-04-25T08:47:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db415d160 con 0x7f2db415a750 2024-04-25T08:47:01.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db4069080 con 0x7f2db415a750 2024-04-25T08:47:01.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.653+0000 7f2db1600640 1 -- 172.21.15.46:0/2605147419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2dac01a070 con 0x7f2db415a750 2024-04-25T08:47:01.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.657+0000 7f2db1600640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d94066f60 0x7f2d94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:01.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.657+0000 7f2db1600640 1 -- 172.21.15.46:0/2605147419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2dac084f80 con 0x7f2db415a750 2024-04-25T08:47:01.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.657+0000 7f2db9600640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d94066f60 0x7f2d94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:01.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.657+0000 7f2db9600640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d94066f60 0x7f2d94069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2da80af000 tx=0x7f2da8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:01.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.661+0000 7f2db1600640 1 -- 172.21.15.46:0/2605147419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2dac055750 con 0x7f2db415a750 2024-04-25T08:47:01.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2db40af980 con 0x7f2db415a750 2024-04-25T08:47:01.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2db1600640 1 -- 172.21.15.46:0/2605147419 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f2dac0597f0 con 0x7f2db415a750 2024-04-25T08:47:01.850 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:47:01.850 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:46:59.910049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/4112568749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":4112568749},{"type":"v1","addr":"172.21.15.46:6835","nonce":4112568749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:47:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 51 2024-04-25T08:47:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d94066f60 msgr2=0x7f2d94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d94066f60 0x7f2d94069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2da80af000 tx=0x7f2da8008040 comp rx=0 tx=0).stop 2024-04-25T08:47:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db415a750 msgr2=0x7f2db4143ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db415a750 0x7f2db4143ef0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2dac00cb40 tx=0x7f2dac00a060 comp rx=0 tx=0).stop 2024-04-25T08:47:01.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 shutdown_connections 2024-04-25T08:47:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d94066f60 0x7f2d94069420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2db415a750 0x7f2db4143ef0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2db40cc9e0 0x7f2db4156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 --2- 172.21.15.46:0/2605147419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2db40aed10 0x7f2db415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 >> 172.21.15.46:0/2605147419 conn(0x7f2db40b12a0 msgr2=0x7f2db4071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 shutdown_connections 2024-04-25T08:47:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:01.857+0000 7f2dbc000640 1 -- 172.21.15.46:0/2605147419 wait complete. 2024-04-25T08:47:02.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:46:46.077710+0000', 'modified': '2024-04-25T08:46:59.910049+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 79, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6990': {'gid': 6990, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/4112568749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 4112568749}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 4112568749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:47:02.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:47:02.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:47:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa309200640 1 Processor -- start 2024-04-25T08:47:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa309200640 1 -- start start 2024-04-25T08:47:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa309200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa30406db40 0x7fa30406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa309200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3040b3550 0x7fa30406b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa309200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30415e180 con 0x7fa3040af3f0 2024-04-25T08:47:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa309200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3040b3ca0 con 0x7fa3040b3550 2024-04-25T08:47:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa309200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30415e460 con 0x7fa30406db40 2024-04-25T08:47:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa302800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3040b3550 0x7fa30406b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa302800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3040b3550 0x7fa30406b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47630/0 (socket says 172.21.15.46:47630) 2024-04-25T08:47:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.425+0000 7fa302800640 1 -- 172.21.15.46:0/411002860 learned_addr learned my addr 172.21.15.46:0/411002860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.429+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/411002860 >> v2:172.21.15.82:3300/0 conn(0x7fa30406db40 0x7fa30406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.429+0000 7fa302800640 1 -- 172.21.15.46:0/411002860 >> v2:172.21.15.82:3300/0 conn(0x7fa30406db40 msgr2=0x7fa30406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.429+0000 7fa302800640 1 --2- 172.21.15.46:0/411002860 >> v2:172.21.15.82:3300/0 conn(0x7fa30406db40 0x7fa30406ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.429+0000 7fa302800640 1 -- 172.21.15.46:0/411002860 >> v1:172.21.15.82:6789/0 conn(0x7fa3040af3f0 legacy=0x7fa3040b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.429+0000 7fa302800640 1 -- 172.21.15.46:0/411002860 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30415e740 con 0x7fa3040b3550 2024-04-25T08:47:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.429+0000 7fa302800640 1 --2- 172.21.15.46:0/411002860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3040b3550 0x7fa30406b150 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa3041625a0 tx=0x7fa2fc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce51412237d49df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.429+0000 7fa301400640 1 -- 172.21.15.46:0/411002860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0a5620 con 0x7fa3040b3550 2024-04-25T08:47:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.429+0000 7fa301400640 1 -- 172.21.15.46:0/411002860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc0a57c0 con 0x7fa3040b3550 2024-04-25T08:47:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.429+0000 7fa301400640 1 -- 172.21.15.46:0/411002860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0a5b90 con 0x7fa3040b3550 2024-04-25T08:47:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 -- 172.21.15.46:0/411002860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3040b3550 msgr2=0x7fa30406b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 --2- 172.21.15.46:0/411002860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3040b3550 0x7fa30406b150 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa3041625a0 tx=0x7fa2fc01c960 comp rx=0 tx=0).stop 2024-04-25T08:47:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 -- 172.21.15.46:0/411002860 shutdown_connections 2024-04-25T08:47:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 --2- 172.21.15.46:0/411002860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3040b3550 0x7fa30406b150 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 --2- 172.21.15.46:0/411002860 >> v2:172.21.15.82:3300/0 conn(0x7fa30406db40 0x7fa30406ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 -- 172.21.15.46:0/411002860 >> 172.21.15.46:0/411002860 conn(0x7fa3040cfd90 msgr2=0x7fa3040d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:02.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 -- 172.21.15.46:0/411002860 shutdown_connections 2024-04-25T08:47:02.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 -- 172.21.15.46:0/411002860 wait complete. 2024-04-25T08:47:02.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 Processor -- start 2024-04-25T08:47:02.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 -- start start 2024-04-25T08:47:02.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30406db40 0x7fa30414d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3040af3f0 0x7fa3041fab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3040b3550 0x7fa3040b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30416a870 con 0x7fa30406db40 2024-04-25T08:47:02.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30416a6f0 con 0x7fa3040b3550 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa309200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30416a9f0 con 0x7fa3040af3f0 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa301e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30406db40 0x7fa30414d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa301e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30406db40 0x7fa30414d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47638/0 (socket says 172.21.15.46:47638) 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa301e00640 1 -- 172.21.15.46:0/1975144223 learned_addr learned my addr 172.21.15.46:0/1975144223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa302800640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3040b3550 0x7fa3040b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3040af3f0 0x7fa3041fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa301e00640 1 -- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3040af3f0 msgr2=0x7fa3041fab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa301e00640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3040af3f0 0x7fa3041fab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa301e00640 1 -- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3040b3550 msgr2=0x7fa3040b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa301e00640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3040b3550 0x7fa3040b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.433+0000 7fa301e00640 1 -- 172.21.15.46:0/1975144223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3041fb0b0 con 0x7fa30406db40 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa301e00640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30406db40 0x7fa30414d6e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c530 tx=0x7fa2f400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa2fa000640 1 -- 172.21.15.46:0/1975144223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4016020 con 0x7fa30406db40 2024-04-25T08:47:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa2fa000640 1 -- 172.21.15.46:0/1975144223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f4005220 con 0x7fa30406db40 2024-04-25T08:47:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa2fa000640 1 -- 172.21.15.46:0/1975144223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4007030 con 0x7fa30406db40 2024-04-25T08:47:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa309200640 1 -- 172.21.15.46:0/1975144223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30414dcc0 con 0x7fa30406db40 2024-04-25T08:47:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa309200640 1 -- 172.21.15.46:0/1975144223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30414e280 con 0x7fa30406db40 2024-04-25T08:47:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa2fa000640 1 -- 172.21.15.46:0/1975144223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2f40071d0 con 0x7fa30406db40 2024-04-25T08:47:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa2fa000640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2e0066f60 0x7fa2e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa2fa000640 1 -- 172.21.15.46:0/1975144223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa2f4084a90 con 0x7fa30406db40 2024-04-25T08:47:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa30c67d000 1 Processor -- start 2024-04-25T08:47:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2e0066f60 0x7fa2e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.437+0000 7fa30c67d000 1 -- start start 2024-04-25T08:47:02.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2e0066f60 0x7fa2e0069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa304067d30 tx=0x7fa2ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7000d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd6fb69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd6fb7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd6e9e5f0 con 0x562dd7000920 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd6e05da0 con 0x562dd6fb6f30 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd6d8e150 con 0x562dd7001240 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa301e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7000d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa301e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7000d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47646/0 (socket says 172.21.15.46:47646) 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa301e00640 1 -- 172.21.15.46:0/2103515595 learned_addr learned my addr 172.21.15.46:0/2103515595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/2103515595 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd6fb69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa302800640 1 --2- 172.21.15.46:0/2103515595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd6fb7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa301e00640 1 -- 172.21.15.46:0/2103515595 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 msgr2=0x562dd6fb69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa301e00640 1 --2- 172.21.15.46:0/2103515595 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd6fb69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa301e00640 1 -- 172.21.15.46:0/2103515595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 msgr2=0x562dd6fb7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa301e00640 1 --2- 172.21.15.46:0/2103515595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd6fb7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa301e00640 1 -- 172.21.15.46:0/2103515595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dd6da7ca0 con 0x562dd7000920 2024-04-25T08:47:02.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa301e00640 1 --2- 172.21.15.46:0/2103515595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7000d00 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40516c0 tx=0x7fa2f4055320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8514bdee9532467e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa2f2a00640 1 -- 172.21.15.46:0/2103515595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f406a070 con 0x562dd7000920 2024-04-25T08:47:02.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa2f2a00640 1 -- 172.21.15.46:0/2103515595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f40590b0 con 0x562dd7000920 2024-04-25T08:47:02.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa2f2a00640 1 -- 172.21.15.46:0/2103515595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40518d0 con 0x562dd7000920 2024-04-25T08:47:02.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 -- 172.21.15.46:0/2103515595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 msgr2=0x562dd7000d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 --2- 172.21.15.46:0/2103515595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7000d00 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40516c0 tx=0x7fa2f4055320 comp rx=0 tx=0).stop 2024-04-25T08:47:02.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 -- 172.21.15.46:0/2103515595 shutdown_connections 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 --2- 172.21.15.46:0/2103515595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd6fb7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 --2- 172.21.15.46:0/2103515595 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd6fb69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 --2- 172.21.15.46:0/2103515595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7000d00 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 -- 172.21.15.46:0/2103515595 >> 172.21.15.46:0/2103515595 conn(0x562dd6fbd2a0 msgr2=0x562dd6fb52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 -- 172.21.15.46:0/2103515595 shutdown_connections 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 -- 172.21.15.46:0/2103515595 wait complete. 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 -- start start 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.441+0000 7fa30c67d000 1 Processor -- start 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd7007cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7008230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd6fbaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd6dc0990 con 0x562dd7000920 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd6e01e40 con 0x562dd6fb6f30 2024-04-25T08:47:02.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd6d91900 con 0x562dd7001240 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 10 client.0 _ll_register_callbacks cb 0x562dd6eb91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7008230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7008230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47652/0 (socket says 172.21.15.46:47652) 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2fbe00640 1 -- 172.21.15.46:0/1316439196 learned_addr learned my addr 172.21.15.46:0/1316439196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa301e00640 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd7007cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa302800640 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd6fbaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2fbe00640 1 -- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 msgr2=0x562dd6fbaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd6fbaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2fbe00640 1 -- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 msgr2=0x562dd7007cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd7007cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2fbe00640 1 -- 172.21.15.46:0/1316439196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dd6dbe630 con 0x562dd7000920 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa301e00640 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd7007cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:47:02.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa302800640 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd6fbaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7008230 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec016800 tx=0x7fa2ec0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d59bd94a821f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2e7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2e7400640 1 -- 172.21.15.46:0/1316439196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec01c040 con 0x562dd7000920 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2e7400640 1 -- 172.21.15.46:0/1316439196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa2ec0149b0 con 0x562dd7000920 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 -- 172.21.15.46:0/1316439196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2f40590b0 con 0x562dd7000920 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2e7400640 1 -- 172.21.15.46:0/1316439196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2ec030050 con 0x562dd7000920 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 10 client.7038 fetch_fsmap learned FSMap version 51 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 -- 172.21.15.46:0/1316439196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x562dd6d8e3a0 con 0x562dd7000920 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2e7400640 1 -- 172.21.15.46:0/1316439196 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f40590b0 con 0x562dd7000920 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2e7400640 1 -- 172.21.15.46:0/1316439196 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa2ec033020 con 0x562dd7000920 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 10 client.7038 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 10 client.7038 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 20 client.7038 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x562dd6fbc160 0x562dd7011d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 4 client.7038 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:47:02.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa30c67d000 1 -- 172.21.15.46:0/1316439196 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562dd6da1a50 con 0x562dd6fbc160 2024-04-25T08:47:02.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa301e00640 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x562dd6fbc160 0x562dd7011d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa301e00640 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x562dd6fbc160 0x562dd7011d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d28928820b63a406 server_cookie=3bf0238a359ae651 in_seq=0 out_seq=0 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2e7400640 10 client.7038 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2e7400640 1 -- 172.21.15.46:0/1316439196 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562dd6da1a50 con 0x562dd6fbc160 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.445+0000 7fa2e7400640 10 client.7038 handle_command_reply: tid=0 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 client.7038 shutdown 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 -- 172.21.15.46:0/1316439196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 msgr2=0x562dd7008230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7008230 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec016800 tx=0x7fa2ec0046d0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 -- 172.21.15.46:0/1316439196 shutdown_connections 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x562dd6fbc160 0x562dd7011d50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd6fbaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd7008230 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa2e7400640 0 client.7038 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:02.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 --2- 172.21.15.46:0/1316439196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd7007cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 -- 172.21.15.46:0/1316439196 >> 172.21.15.46:0/1316439196 conn(0x562dd6fbd2a0 msgr2=0x562dd6fb59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:02.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 -- 172.21.15.46:0/1316439196 shutdown_connections 2024-04-25T08:47:02.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 1 -- 172.21.15.46:0/1316439196 wait complete. 2024-04-25T08:47:02.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.449+0000 7fa30c67d000 20 client.7038 trim_cache size 0 max 16384 2024-04-25T08:47:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.461+0000 7fa30c67d000 1 Processor -- start 2024-04-25T08:47:02.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- start start 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd6fba330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd6fba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd700a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3042029d0 con 0x562dd7000920 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30415b390 con 0x562dd6fb6f30 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa304202520 con 0x562dd7001240 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd6fba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd6fba870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47668/0 (socket says 172.21.15.46:47668) 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2fbe00640 1 -- 172.21.15.46:0/3386441130 learned_addr learned my addr 172.21.15.46:0/3386441130 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa301e00640 1 --2- 172.21.15.46:0/3386441130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd6fba330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa302800640 1 --2- 172.21.15.46:0/3386441130 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd700a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2fbe00640 1 -- 172.21.15.46:0/3386441130 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 msgr2=0x562dd700a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/3386441130 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd700a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2fbe00640 1 -- 172.21.15.46:0/3386441130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 msgr2=0x562dd6fba330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/3386441130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd6fba330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2fbe00640 1 -- 172.21.15.46:0/3386441130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562dd7001070 con 0x562dd7000920 2024-04-25T08:47:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/3386441130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd6fba870 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0167d0 tx=0x7fa2ec023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2dbf85c23c6659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2f2000640 1 -- 172.21.15.46:0/3386441130 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec038070 con 0x562dd7000920 2024-04-25T08:47:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2f2000640 1 -- 172.21.15.46:0/3386441130 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec01d040 con 0x562dd7000920 2024-04-25T08:47:02.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa2f2000640 1 -- 172.21.15.46:0/3386441130 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec01c070 con 0x562dd7000920 2024-04-25T08:47:02.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- 172.21.15.46:0/3386441130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 msgr2=0x562dd6fba870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3386441130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd6fba870 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0167d0 tx=0x7fa2ec023000 comp rx=0 tx=0).stop 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- 172.21.15.46:0/3386441130 shutdown_connections 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3386441130 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7001240 0x562dd700a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3386441130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd7000920 0x562dd6fba870 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3386441130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd6fb6f30 0x562dd6fba330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- 172.21.15.46:0/3386441130 >> 172.21.15.46:0/3386441130 conn(0x562dd6fbd2a0 msgr2=0x562dd6fed010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- 172.21.15.46:0/3386441130 shutdown_connections 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- 172.21.15.46:0/3386441130 wait complete. 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 -- start start 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.465+0000 7fa30c67d000 1 Processor -- start 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd6fb6f30 0x562dd700df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7000920 0x562dd7045090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd7001240 0x562dd70455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd6e0a3d0 con 0x562dd6fb6f30 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dd6ec1d50 con 0x562dd7001240 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dd6da0980 con 0x562dd7000920 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 10 client.0 _ll_register_callbacks cb 0x7fa2f4055030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa301e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd6fb6f30 0x562dd700df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa301e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd6fb6f30 0x562dd700df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47680/0 (socket says 172.21.15.46:47680) 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa301e00640 1 -- 172.21.15.46:0/3327929413 learned_addr learned my addr 172.21.15.46:0/3327929413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa302800640 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd7001240 0x562dd70455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7000920 0x562dd7045090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa301e00640 1 -- 172.21.15.46:0/3327929413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7000920 msgr2=0x562dd7045090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa301e00640 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7000920 0x562dd7045090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa301e00640 1 -- 172.21.15.46:0/3327929413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd7001240 msgr2=0x562dd70455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa301e00640 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd7001240 0x562dd70455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa301e00640 1 -- 172.21.15.46:0/3327929413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dd6e87600 con 0x562dd6fb6f30 2024-04-25T08:47:02.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa301e00640 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd6fb6f30 0x562dd700df20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40551a0 tx=0x7fa2f4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cede69d39c7b1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 1 -- 172.21.15.46:0/3327929413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40590b0 con 0x562dd6fb6f30 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 1 -- 172.21.15.46:0/3327929413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa2f4072050 con 0x562dd6fb6f30 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- 172.21.15.46:0/3327929413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2ec01d040 con 0x562dd6fb6f30 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 1 -- 172.21.15.46:0/3327929413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2f407c400 con 0x562dd6fb6f30 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 10 client.7044 fetch_fsmap learned FSMap version 51 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- 172.21.15.46:0/3327929413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x562dd6d9a9b0 con 0x562dd6fb6f30 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 1 -- 172.21.15.46:0/3327929413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec01d040 con 0x562dd6fb6f30 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 1 -- 172.21.15.46:0/3327929413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa2f40518d0 con 0x562dd6fb6f30 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 10 client.7044 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 10 client.7044 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 20 client.7044 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x562dd7045b10 0x562dd7060e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 4 client.7044 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:47:02.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- 172.21.15.46:0/3327929413 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562dd6e132e0 con 0x562dd7045b10 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x562dd7045b10 0x562dd7060e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 10 client.7044 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2fbe00640 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x562dd7045b10 0x562dd7060e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe275a951abe868d server_cookie=c72782b2c950bd9 in_seq=0 out_seq=0 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 1 -- 172.21.15.46:0/3327929413 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562dd6e132e0 con 0x562dd7045b10 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 10 client.7044 handle_command_reply: tid=0 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 client.7044 shutdown 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- 172.21.15.46:0/3327929413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd6fb6f30 msgr2=0x562dd700df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd6fb6f30 0x562dd700df20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa2f40551a0 tx=0x7fa2f4000ee0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- 172.21.15.46:0/3327929413 shutdown_connections 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x562dd7045b10 0x562dd7060e70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562dd7001240 0x562dd70455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562dd7000920 0x562dd7045090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 --2- 172.21.15.46:0/3327929413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562dd6fb6f30 0x562dd700df20 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa2e7400640 0 client.7044 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:02.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- 172.21.15.46:0/3327929413 >> 172.21.15.46:0/3327929413 conn(0x562dd6fbd2a0 msgr2=0x562dd6fb9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- 172.21.15.46:0/3327929413 shutdown_connections 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 1 -- 172.21.15.46:0/3327929413 wait complete. 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.469+0000 7fa30c67d000 20 client.7044 trim_cache size 0 max 16384 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6990, 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5795299480000002, 2024-04-25T08:47:02.469 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 51, 2024-04-25T08:47:02.470 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 79, 2024-04-25T08:47:02.470 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T08:47:02.470 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.5246832980000002 2024-04-25T08:47:02.470 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:47:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 -- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2e0066f60 msgr2=0x7fa2e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2e0066f60 0x7fa2e0069420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa304067d30 tx=0x7fa2ec008040 comp rx=0 tx=0).stop 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 -- 172.21.15.46:0/1975144223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30406db40 msgr2=0x7fa30414d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30406db40 0x7fa30414d6e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c530 tx=0x7fa2f400c9f0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 -- 172.21.15.46:0/1975144223 shutdown_connections 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2e0066f60 0x7fa2e0069420 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3040b3550 0x7fa3040b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa3040af3f0 0x7fa3041fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 --2- 172.21.15.46:0/1975144223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30406db40 0x7fa30414d6e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 -- 172.21.15.46:0/1975144223 >> 172.21.15.46:0/1975144223 conn(0x7fa3040cfd90 msgr2=0x7fa3040d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 -- 172.21.15.46:0/1975144223 shutdown_connections 2024-04-25T08:47:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.473+0000 7fa309200640 1 -- 172.21.15.46:0/1975144223 wait complete. 2024-04-25T08:47:02.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:47:02.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee9200640 1 Processor -- start 2024-04-25T08:47:02.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee9200640 1 -- start start 2024-04-25T08:47:02.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee9200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9ee415ef20 0x7f9ee40cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee9200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40cd6d0 0x7f9ee409a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee9200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee415f300 con 0x7f9ee409ab50 2024-04-25T08:47:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee9200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee40cddb0 con 0x7f9ee415ef20 2024-04-25T08:47:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee9200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee40cc3d0 con 0x7f9ee40cd6d0 2024-04-25T08:47:02.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee1e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9ee415ef20 0x7f9ee40cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee1400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40cd6d0 0x7f9ee409a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee1e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9ee415ef20 0x7f9ee40cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:49936/0 (socket says 172.21.15.46:49936) 2024-04-25T08:47:02.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee1e00640 1 -- 172.21.15.46:0/1032461690 learned_addr learned my addr 172.21.15.46:0/1032461690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:02.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee2800640 1 --1- 172.21.15.46:0/1032461690 >> v1:172.21.15.82:6789/0 conn(0x7f9ee409ab50 0x7f9ee4099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:38024/0 (socket says 172.21.15.46:38024) 2024-04-25T08:47:02.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee1e00640 1 -- 172.21.15.46:0/1032461690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40cd6d0 msgr2=0x7f9ee409a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:02.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee1e00640 1 --2- 172.21.15.46:0/1032461690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40cd6d0 0x7f9ee409a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee1e00640 1 -- 172.21.15.46:0/1032461690 >> v1:172.21.15.82:6789/0 conn(0x7f9ee409ab50 legacy=0x7f9ee4099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.633+0000 7f9ee1e00640 1 -- 172.21.15.46:0/1032461690 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee40cc6b0 con 0x7f9ee415ef20 2024-04-25T08:47:02.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee1e00640 1 --2- 172.21.15.46:0/1032461690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee415ef20 0x7f9ee40cd190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009870 tx=0x7f9ed401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcfdcdc381ac5aa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee0a00640 1 -- 172.21.15.46:0/1032461690 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40a95f0 con 0x7f9ee415ef20 2024-04-25T08:47:02.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee0a00640 1 -- 172.21.15.46:0/1032461690 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed40a9790 con 0x7f9ee415ef20 2024-04-25T08:47:02.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee0a00640 1 -- 172.21.15.46:0/1032461690 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40a9b60 con 0x7f9ee415ef20 2024-04-25T08:47:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee9200640 1 -- 172.21.15.46:0/1032461690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee415ef20 msgr2=0x7f9ee40cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee9200640 1 --2- 172.21.15.46:0/1032461690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee415ef20 0x7f9ee40cd190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009870 tx=0x7f9ed401c530 comp rx=0 tx=0).stop 2024-04-25T08:47:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee9200640 1 -- 172.21.15.46:0/1032461690 shutdown_connections 2024-04-25T08:47:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee9200640 1 --2- 172.21.15.46:0/1032461690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40cd6d0 0x7f9ee409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee9200640 1 --2- 172.21.15.46:0/1032461690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee415ef20 0x7f9ee40cd190 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee9200640 1 -- 172.21.15.46:0/1032461690 >> 172.21.15.46:0/1032461690 conn(0x7f9ee40b3120 msgr2=0x7f9ee40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee9200640 1 -- 172.21.15.46:0/1032461690 shutdown_connections 2024-04-25T08:47:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.637+0000 7f9ee9200640 1 -- 172.21.15.46:0/1032461690 wait complete. 2024-04-25T08:47:02.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 Processor -- start 2024-04-25T08:47:02.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 -- start start 2024-04-25T08:47:02.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee409ab50 0x7f9ee41566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee415ef20 0x7f9ee40b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40b6fc0 0x7f9ee40cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee40c5b10 con 0x7f9ee409ab50 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee40c5990 con 0x7f9ee415ef20 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee409a110 con 0x7f9ee40b6fc0 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee409ab50 0x7f9ee41566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee409ab50 0x7f9ee41566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47686/0 (socket says 172.21.15.46:47686) 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1e00640 1 -- 172.21.15.46:0/3419650145 learned_addr learned my addr 172.21.15.46:0/3419650145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1400640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee415ef20 0x7f9ee40b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee2800640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40b6fc0 0x7f9ee40cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1e00640 1 -- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40b6fc0 msgr2=0x7f9ee40cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1e00640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40b6fc0 0x7f9ee40cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1e00640 1 -- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee415ef20 msgr2=0x7f9ee40b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1e00640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee415ef20 0x7f9ee40b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1e00640 1 -- 172.21.15.46:0/3419650145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee409b810 con 0x7f9ee409ab50 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee1e00640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee409ab50 0x7f9ee41566c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9ed401ca10 tx=0x7f9ed40192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9eda000640 1 -- 172.21.15.46:0/3419650145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40b15f0 con 0x7f9ee409ab50 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9eda000640 1 -- 172.21.15.46:0/3419650145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed40b1790 con 0x7f9ee409ab50 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9eda000640 1 -- 172.21.15.46:0/3419650145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40b1b80 con 0x7f9ee409ab50 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee409baf0 con 0x7f9ee409ab50 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee409c0b0 con 0x7f9ee409ab50 2024-04-25T08:47:02.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.641+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee4069080 con 0x7f9ee409ab50 2024-04-25T08:47:02.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.645+0000 7f9eda000640 1 -- 172.21.15.46:0/3419650145 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ed40b8390 con 0x7f9ee409ab50 2024-04-25T08:47:02.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.645+0000 7f9eda000640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ec4066f90 0x7f9ec4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:02.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.645+0000 7f9eda000640 1 -- 172.21.15.46:0/3419650145 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9ed411f4f0 con 0x7f9ee409ab50 2024-04-25T08:47:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.645+0000 7f9ee1400640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ec4066f90 0x7f9ec4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:02.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.645+0000 7f9ee1400640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ec4066f90 0x7f9ec4069450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40cebe0 tx=0x7f9ed0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:02.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.649+0000 7f9eda000640 1 -- 172.21.15.46:0/3419650145 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ed40ad020 con 0x7f9ee409ab50 2024-04-25T08:47:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.849+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ee40d0700 con 0x7f9ee409ab50 2024-04-25T08:47:02.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.849+0000 7f9eda000640 1 -- 172.21.15.46:0/3419650145 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9ed4016150 con 0x7f9ee409ab50 2024-04-25T08:47:02.845 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:47:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ec4066f90 msgr2=0x7f9ec4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ec4066f90 0x7f9ec4069450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40cebe0 tx=0x7f9ed0009190 comp rx=0 tx=0).stop 2024-04-25T08:47:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee409ab50 msgr2=0x7f9ee41566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee409ab50 0x7f9ee41566c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9ed401ca10 tx=0x7f9ed40192d0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 shutdown_connections 2024-04-25T08:47:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ec4066f90 0x7f9ec4069450 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ee40b6fc0 0x7f9ee40cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ee415ef20 0x7f9ee40b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 --2- 172.21.15.46:0/3419650145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ee409ab50 0x7f9ee41566c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 >> 172.21.15.46:0/3419650145 conn(0x7f9ee40b3120 msgr2=0x7f9ee4070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 shutdown_connections 2024-04-25T08:47:02.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:02.853+0000 7f9ee9200640 1 -- 172.21.15.46:0/3419650145 wait complete. 2024-04-25T08:47:02.866 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:47:02.866 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:47:02.866 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:47:02.867 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:47:02.867 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:47:02.867 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:47:02.867 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:47:02.867 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:47:02.867 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:47:02.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:02.867 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:47:02.876 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:47:02.876 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:47:02.876 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42180sec preferred_lft 42180sec 2024-04-25T08:47:02.877 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:47:02.878 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:47:02.879 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:47:02.879 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:47:02.879 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:47:02.879 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:47:02.879 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:47:02.879 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:47:02.879 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:47:02.879 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:47:02.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:02.880 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:47:02.887 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:47:02.893+0000 7fb218a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:47:02.904 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:47:02.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:02.907 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:47:02.958 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:47:02.958 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:47:02.958 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:47:02.958 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:47:02.958 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:47:02.958 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:47:02.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:02.959 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:47:02.959 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:47:02.959 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:47:02.959 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:47:02.959 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:47:02.959 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:47:03.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:03.050 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:47:03.059 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:47:03.059 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:47:03.059 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:47:03.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:03.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:47:03.120 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:47:03.120 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:47:03.172 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:47:03.172 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:47:03.173 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:47:03.224 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:47:03.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:03.224 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:47:03.318 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:47:03.318 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:47:03.318 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:47:03.333 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:47:03.333 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:47:03.333 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:47:03.333 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:47:03.333 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:47:03.342 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:47:03.342 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:47:03.342 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:47:03.343 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:47:03.343 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:47:03.343 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:47:03.343 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:47:03.343 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:47:03.346 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:47:03.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:03.347 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:47:03.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:03.367 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:47:03.423 INFO:teuthology.orchestra.run.smithi046.stdout:client6164 2024-04-25T08:47:03.423 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:47:03.424 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:47:03.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:03.473 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-25T08:47:03.527 INFO:teuthology.orchestra.run.smithi046.stdout:file_undamaged 2024-04-25T08:47:03.529 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:47:03.529 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:47:03.529 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:47:03.529 DEBUG:teuthology.orchestra.run.smithi046:> import json 2024-04-25T08:47:03.529 DEBUG:teuthology.orchestra.run.smithi046:> import sys 2024-04-25T08:47:03.529 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:47:03.529 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-04-25T08:47:03.529 DEBUG:teuthology.orchestra.run.smithi046:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T08:47:03.530 DEBUG:teuthology.orchestra.run.smithi046:> except OSError as e: 2024-04-25T08:47:03.530 DEBUG:teuthology.orchestra.run.smithi046:> sys.exit(e.errno) 2024-04-25T08:47:03.530 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:47:03.530 DEBUG:teuthology.orchestra.run.smithi046:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T08:47:03.530 DEBUG:teuthology.orchestra.run.smithi046:> print(json.dumps( 2024-04-25T08:47:03.530 DEBUG:teuthology.orchestra.run.smithi046:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T08:47:03.530 DEBUG:teuthology.orchestra.run.smithi046:> indent=2)) 2024-04-25T08:47:03.530 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:47:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:03 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T08:47:03.652 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T08:47:03.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:03.653 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T08:47:03.706 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-04-25T08:47:03.706 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:47:03.706 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:47:03.756 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:47:03.757 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:47:03.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:47:03.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f0970600640 1 Processor -- start 2024-04-25T08:47:03.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f0970600640 1 -- start start 2024-04-25T08:47:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f0970600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f096815ef20 0x7f09680cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f0970600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09680cd6d0 0x7f096809a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f0970600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096815f300 con 0x7f096809ab50 2024-04-25T08:47:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f0970600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09680cddb0 con 0x7f09680cd6d0 2024-04-25T08:47:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f0970600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09680cc3d0 con 0x7f096815ef20 2024-04-25T08:47:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f096d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09680cd6d0 0x7f096809a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f096d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f09680cd6d0 0x7f096809a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47688/0 (socket says 172.21.15.46:47688) 2024-04-25T08:47:03.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f096d200640 1 -- 172.21.15.46:0/885515790 learned_addr learned my addr 172.21.15.46:0/885515790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f096e600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f096809ab50 0x7f0968099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48560/0 (socket says 172.21.15.46:48560) 2024-04-25T08:47:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f096dc00640 1 --2- 172.21.15.46:0/885515790 >> v2:172.21.15.82:3300/0 conn(0x7f096815ef20 0x7f09680cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f096d200640 1 -- 172.21.15.46:0/885515790 >> v2:172.21.15.82:3300/0 conn(0x7f096815ef20 msgr2=0x7f09680cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f096d200640 1 --2- 172.21.15.46:0/885515790 >> v2:172.21.15.82:3300/0 conn(0x7f096815ef20 0x7f09680cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f096d200640 1 -- 172.21.15.46:0/885515790 >> v1:172.21.15.46:6789/0 conn(0x7f096809ab50 legacy=0x7f0968099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f096d200640 1 -- 172.21.15.46:0/885515790 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09680cc6b0 con 0x7f09680cd6d0 2024-04-25T08:47:03.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.001+0000 7f0967e00640 1 -- 172.21.15.46:0/885515790 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334716120 0 0) 0x7f096815f300 con 0x7f096809ab50 2024-04-25T08:47:04.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f096d200640 1 --2- 172.21.15.46:0/885515790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09680cd6d0 0x7f096809a610 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f096809a5e0 tx=0x7f095401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11f2b5d863613fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:04.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0967e00640 1 -- 172.21.15.46:0/885515790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540aa650 con 0x7f09680cd6d0 2024-04-25T08:47:04.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0967e00640 1 -- 172.21.15.46:0/885515790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09540af070 con 0x7f09680cd6d0 2024-04-25T08:47:04.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0967e00640 1 -- 172.21.15.46:0/885515790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540aaa10 con 0x7f09680cd6d0 2024-04-25T08:47:04.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0970600640 1 -- 172.21.15.46:0/885515790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09680cd6d0 msgr2=0x7f096809a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0970600640 1 --2- 172.21.15.46:0/885515790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09680cd6d0 0x7f096809a610 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f096809a5e0 tx=0x7f095401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0970600640 1 -- 172.21.15.46:0/885515790 shutdown_connections 2024-04-25T08:47:04.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0970600640 1 --2- 172.21.15.46:0/885515790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f09680cd6d0 0x7f096809a610 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0970600640 1 --2- 172.21.15.46:0/885515790 >> v2:172.21.15.82:3300/0 conn(0x7f096815ef20 0x7f09680cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0970600640 1 -- 172.21.15.46:0/885515790 >> 172.21.15.46:0/885515790 conn(0x7f09680b3120 msgr2=0x7f09680b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:04.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0970600640 1 -- 172.21.15.46:0/885515790 shutdown_connections 2024-04-25T08:47:04.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.005+0000 7f0970600640 1 -- 172.21.15.46:0/885515790 wait complete. 2024-04-25T08:47:04.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 Processor -- start 2024-04-25T08:47:04.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 -- start start 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096809ab50 0x7f09681f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f096815ef20 0x7f0968164420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09680ce830 0x7f0968154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c5b10 con 0x7f096809ab50 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c5990 con 0x7f09680ce830 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096809a110 con 0x7f096815ef20 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096809ab50 0x7f09681f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096809ab50 0x7f09681f97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47702/0 (socket says 172.21.15.46:47702) 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096dc00640 1 -- 172.21.15.46:0/2123582576 learned_addr learned my addr 172.21.15.46:0/2123582576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096d200640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f096815ef20 0x7f0968164420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096e600640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09680ce830 0x7f0968154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096dc00640 1 -- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f096815ef20 msgr2=0x7f0968164420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096dc00640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f096815ef20 0x7f0968164420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096dc00640 1 -- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09680ce830 msgr2=0x7f0968154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096dc00640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09680ce830 0x7f0968154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096dc00640 1 -- 172.21.15.46:0/2123582576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09681fa6b0 con 0x7f096809ab50 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f096dc00640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096809ab50 0x7f09681f97a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f095c00e8a0 tx=0x7f095c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0965600640 1 -- 172.21.15.46:0/2123582576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c017070 con 0x7f096809ab50 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0965600640 1 -- 172.21.15.46:0/2123582576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c00c940 con 0x7f096809ab50 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0965600640 1 -- 172.21.15.46:0/2123582576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c00ccf0 con 0x7f096809ab50 2024-04-25T08:47:04.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09681577a0 con 0x7f096809ab50 2024-04-25T08:47:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.009+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09681fa0f0 con 0x7f096809ab50 2024-04-25T08:47:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.013+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0968069080 con 0x7f096809ab50 2024-04-25T08:47:04.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.013+0000 7f0965600640 1 -- 172.21.15.46:0/2123582576 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f095c01a070 con 0x7f096809ab50 2024-04-25T08:47:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.013+0000 7f0965600640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0948066f90 0x7f0948069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.013+0000 7f0965600640 1 -- 172.21.15.46:0/2123582576 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f095c084fd0 con 0x7f096809ab50 2024-04-25T08:47:04.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.013+0000 7f096d200640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0948066f90 0x7f0948069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.013+0000 7f096d200640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0948066f90 0x7f0948069450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f09540ae000 tx=0x7f09540a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:04.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.017+0000 7f0965600640 1 -- 172.21.15.46:0/2123582576 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f095c0557a0 con 0x7f096809ab50 2024-04-25T08:47:04.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.177+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09680d0700 con 0x7f096809ab50 2024-04-25T08:47:04.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0965600640 1 -- 172.21.15.46:0/2123582576 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17024 (secure 0 0 0) 0x7f095c059840 con 0x7f096809ab50 2024-04-25T08:47:04.174 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:47:04.174 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:56.883035+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:46:44.505856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:46:45.072095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/960745191":"2024-04-26T08:46:56.883011+0000","172.21.15.46:6833/960745191":"2024-04-26T08:46:56.883011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:47:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0948066f90 msgr2=0x7f0948069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0948066f90 0x7f0948069450 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f09540ae000 tx=0x7f09540a7000 comp rx=0 tx=0).stop 2024-04-25T08:47:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096809ab50 msgr2=0x7f09681f97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096809ab50 0x7f09681f97a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f095c00e8a0 tx=0x7f095c00eca0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 shutdown_connections 2024-04-25T08:47:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0948066f90 0x7f0948069450 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09680ce830 0x7f0968154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f096815ef20 0x7f0968164420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 --2- 172.21.15.46:0/2123582576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f096809ab50 0x7f09681f97a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 >> 172.21.15.46:0/2123582576 conn(0x7f09680b3120 msgr2=0x7f0968070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 shutdown_connections 2024-04-25T08:47:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.181+0000 7f0970600640 1 -- 172.21.15.46:0/2123582576 wait complete. 2024-04-25T08:47:04.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:46:44.505856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:46:45.072095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:47:04.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:47:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f671a000640 1 Processor -- start 2024-04-25T08:47:04.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f671a000640 1 -- start start 2024-04-25T08:47:04.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f671a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6714145310 0x7f671406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f671a000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f671415d090 0x7f671415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f671a000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67140cd930 con 0x7f671406dc70 2024-04-25T08:47:04.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f671a000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67140cdbd0 con 0x7f6714145310 2024-04-25T08:47:04.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f671a000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67140cdeb0 con 0x7f671415d090 2024-04-25T08:47:04.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f6713400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6714145310 0x7f671406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f670be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f671406dc70 0x7f67140cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48566/0 (socket says 172.21.15.46:48566) 2024-04-25T08:47:04.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f670be00640 1 -- 172.21.15.46:0/3845914127 learned_addr learned my addr 172.21.15.46:0/3845914127 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:47:04.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f6713e00640 1 --2- 172.21.15.46:0/3845914127 >> v2:172.21.15.82:3300/0 conn(0x7f671415d090 0x7f671415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f6712a00640 1 -- 172.21.15.46:0/3845914127 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101946462 0 0) 0x7f67140cd930 con 0x7f671406dc70 2024-04-25T08:47:04.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f6713400640 1 -- 172.21.15.46:0/3845914127 >> v2:172.21.15.82:3300/0 conn(0x7f671415d090 msgr2=0x7f671415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:04.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f6713400640 1 --2- 172.21.15.46:0/3845914127 >> v2:172.21.15.82:3300/0 conn(0x7f671415d090 0x7f671415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f6713400640 1 -- 172.21.15.46:0/3845914127 >> v1:172.21.15.46:6789/0 conn(0x7f671406dc70 legacy=0x7f67140cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.349+0000 7f6713400640 1 -- 172.21.15.46:0/3845914127 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140ce190 con 0x7f6714145310 2024-04-25T08:47:04.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f6713400640 1 --2- 172.21.15.46:0/3845914127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 0x7f671406e0d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f671406d060 tx=0x7f670401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b79e7c1d2d5a1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f6712a00640 1 -- 172.21.15.46:0/3845914127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040b0070 con 0x7f6714145310 2024-04-25T08:47:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f6712a00640 1 -- 172.21.15.46:0/3845914127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040ab600 con 0x7f6714145310 2024-04-25T08:47:04.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f6712a00640 1 -- 172.21.15.46:0/3845914127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040ab9d0 con 0x7f6714145310 2024-04-25T08:47:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f671a000640 1 -- 172.21.15.46:0/3845914127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 msgr2=0x7f671406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f671a000640 1 --2- 172.21.15.46:0/3845914127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 0x7f671406e0d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f671406d060 tx=0x7f670401c480 comp rx=0 tx=0).stop 2024-04-25T08:47:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f671a000640 1 -- 172.21.15.46:0/3845914127 shutdown_connections 2024-04-25T08:47:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f671a000640 1 --2- 172.21.15.46:0/3845914127 >> v2:172.21.15.82:3300/0 conn(0x7f671415d090 0x7f671415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f671a000640 1 --2- 172.21.15.46:0/3845914127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 0x7f671406e0d0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f671a000640 1 -- 172.21.15.46:0/3845914127 >> 172.21.15.46:0/3845914127 conn(0x7f671406ac10 msgr2=0x7f671406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f671a000640 1 -- 172.21.15.46:0/3845914127 shutdown_connections 2024-04-25T08:47:04.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f671a000640 1 -- 172.21.15.46:0/3845914127 wait complete. 2024-04-25T08:47:04.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.353+0000 7f671a000640 1 Processor -- start 2024-04-25T08:47:04.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 -- start start 2024-04-25T08:47:04.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 0x7f6714159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671415d090 0x7f6714158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6714158b50 0x7f671413ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6714168510 con 0x7f6714145310 2024-04-25T08:47:04.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67140cf380 con 0x7f671415d090 2024-04-25T08:47:04.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6714168690 con 0x7f6714158b50 2024-04-25T08:47:04.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f6713400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 0x7f6714159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f6713400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 0x7f6714159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47726/0 (socket says 172.21.15.46:47726) 2024-04-25T08:47:04.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f6713400640 1 -- 172.21.15.46:0/4244672767 learned_addr learned my addr 172.21.15.46:0/4244672767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:04.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f6713400640 1 -- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6714158b50 msgr2=0x7f671413ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f6713400640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6714158b50 0x7f671413ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f6713400640 1 -- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671415d090 msgr2=0x7f6714158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f6713400640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671415d090 0x7f6714158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f6713400640 1 -- 172.21.15.46:0/4244672767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140b3dd0 con 0x7f6714145310 2024-04-25T08:47:04.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f6713400640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 0x7f6714159080 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f67040ae040 tx=0x7f6704019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:04.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f670b400640 1 -- 172.21.15.46:0/4244672767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040b0040 con 0x7f6714145310 2024-04-25T08:47:04.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f670b400640 1 -- 172.21.15.46:0/4244672767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67040b9070 con 0x7f6714145310 2024-04-25T08:47:04.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67140b40b0 con 0x7f6714145310 2024-04-25T08:47:04.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f670b400640 1 -- 172.21.15.46:0/4244672767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67040b4650 con 0x7f6714145310 2024-04-25T08:47:04.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67140b4670 con 0x7f6714145310 2024-04-25T08:47:04.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.357+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6714069010 con 0x7f6714145310 2024-04-25T08:47:04.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.361+0000 7f670b400640 1 -- 172.21.15.46:0/4244672767 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67040b47f0 con 0x7f6714145310 2024-04-25T08:47:04.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.361+0000 7f670b400640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f66f0066f60 0x7f66f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.361+0000 7f670b400640 1 -- 172.21.15.46:0/4244672767 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f670411eef0 con 0x7f6714145310 2024-04-25T08:47:04.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.361+0000 7f670be00640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f66f0066f60 0x7f66f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.361+0000 7f670be00640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f66f0066f60 0x7f66f0069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0011f0 tx=0x7f66fc007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:04.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.365+0000 7f670b400640 1 -- 172.21.15.46:0/4244672767 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6704016070 con 0x7f6714145310 2024-04-25T08:47:04.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.529+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67140cd710 con 0x7f6714145310 2024-04-25T08:47:04.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.529+0000 7f670b400640 1 -- 172.21.15.46:0/4244672767 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f67040efc80 con 0x7f6714145310 2024-04-25T08:47:04.524 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:47:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f66f0066f60 msgr2=0x7f66f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f66f0066f60 0x7f66f0069420 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0011f0 tx=0x7f66fc007930 comp rx=0 tx=0).stop 2024-04-25T08:47:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 msgr2=0x7f6714159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 0x7f6714159080 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f67040ae040 tx=0x7f6704019220 comp rx=0 tx=0).stop 2024-04-25T08:47:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 shutdown_connections 2024-04-25T08:47:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f66f0066f60 0x7f66f0069420 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6714158b50 0x7f671413ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f671415d090 0x7f6714158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 --2- 172.21.15.46:0/4244672767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6714145310 0x7f6714159080 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 >> 172.21.15.46:0/4244672767 conn(0x7f671406ac10 msgr2=0x7f6714141cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 shutdown_connections 2024-04-25T08:47:04.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.533+0000 7f671a000640 1 -- 172.21.15.46:0/4244672767 wait complete. 2024-04-25T08:47:04.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:47:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.693+0000 7f854d800640 1 Processor -- start 2024-04-25T08:47:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.693+0000 7f854d800640 1 -- start start 2024-04-25T08:47:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f854d800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854815dae0 0x7f8548072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f854d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f854806ea60 0x7f85480acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f854d800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85480b3850 con 0x7f854806e620 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f854d800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854806e190 con 0x7f854806ea60 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f854d800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85480b3af0 con 0x7f854815dae0 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f8547400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f854806ea60 0x7f85480acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f853e000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f854806e620 0x7f854806bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48574/0 (socket says 172.21.15.46:48574) 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f8547400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f854806ea60 0x7f85480acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47738/0 (socket says 172.21.15.46:47738) 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f853e000640 1 -- 172.21.15.46:0/2642352955 learned_addr learned my addr 172.21.15.46:0/2642352955 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f8546a00640 1 --2- 172.21.15.46:0/2642352955 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854815dae0 0x7f8548072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f8547400640 1 -- 172.21.15.46:0/2642352955 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854815dae0 msgr2=0x7f8548072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f8547400640 1 --2- 172.21.15.46:0/2642352955 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854815dae0 0x7f8548072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f8547400640 1 -- 172.21.15.46:0/2642352955 >> v1:172.21.15.46:6789/0 conn(0x7f854806e620 legacy=0x7f854806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f8547400640 1 -- 172.21.15.46:0/2642352955 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85480b3dd0 con 0x7f854806ea60 2024-04-25T08:47:04.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.697+0000 7f8546000640 1 -- 172.21.15.46:0/2642352955 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677230620 0 0) 0x7f85480b3850 con 0x7f854806e620 2024-04-25T08:47:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f8547400640 1 --2- 172.21.15.46:0/2642352955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 0x7f85480acdb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f854806e310 tx=0x7f854001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb602f83c221ca15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f8546000640 1 -- 172.21.15.46:0/2642352955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85400b2070 con 0x7f854806ea60 2024-04-25T08:47:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f8546000640 1 -- 172.21.15.46:0/2642352955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85400a59e0 con 0x7f854806ea60 2024-04-25T08:47:04.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f8546000640 1 -- 172.21.15.46:0/2642352955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85400a5db0 con 0x7f854806ea60 2024-04-25T08:47:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 -- 172.21.15.46:0/2642352955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 msgr2=0x7f85480acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 --2- 172.21.15.46:0/2642352955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 0x7f85480acdb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f854806e310 tx=0x7f854001c920 comp rx=0 tx=0).stop 2024-04-25T08:47:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 -- 172.21.15.46:0/2642352955 shutdown_connections 2024-04-25T08:47:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 --2- 172.21.15.46:0/2642352955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 0x7f85480acdb0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 --2- 172.21.15.46:0/2642352955 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854815dae0 0x7f8548072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 -- 172.21.15.46:0/2642352955 >> 172.21.15.46:0/2642352955 conn(0x7f8548071ae0 msgr2=0x7f8548071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 -- 172.21.15.46:0/2642352955 shutdown_connections 2024-04-25T08:47:04.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 -- 172.21.15.46:0/2642352955 wait complete. 2024-04-25T08:47:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 Processor -- start 2024-04-25T08:47:04.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.701+0000 7f854d800640 1 -- start start 2024-04-25T08:47:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f854d800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854806e620 0x7f85481463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f854d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 0x7f8548146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f854d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854815dae0 0x7f85481602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f854d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854806f6d0 con 0x7f854806ea60 2024-04-25T08:47:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f854d800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854815dec0 con 0x7f854815dae0 2024-04-25T08:47:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f854d800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f854806f850 con 0x7f854806e620 2024-04-25T08:47:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 0x7f8548146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 0x7f8548146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47750/0 (socket says 172.21.15.46:47750) 2024-04-25T08:47:04.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853e000640 1 -- 172.21.15.46:0/1297326213 learned_addr learned my addr 172.21.15.46:0/1297326213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f8546a00640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854806e620 0x7f85481463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f8547400640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854815dae0 0x7f85481602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853e000640 1 -- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854806e620 msgr2=0x7f85481463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853e000640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854806e620 0x7f85481463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853e000640 1 -- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854815dae0 msgr2=0x7f85481602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853e000640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854815dae0 0x7f85481602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853e000640 1 -- 172.21.15.46:0/1297326213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548154df0 con 0x7f854806ea60 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f8547400640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854815dae0 0x7f85481602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853e000640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 0x7f8548146b70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f85300011f0 tx=0x7f853000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853f400640 1 -- 172.21.15.46:0/1297326213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530018070 con 0x7f854806ea60 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853f400640 1 -- 172.21.15.46:0/1297326213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85300074e0 con 0x7f854806ea60 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853f400640 1 -- 172.21.15.46:0/1297326213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530004a10 con 0x7f854806ea60 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85481550d0 con 0x7f854806ea60 2024-04-25T08:47:04.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8548155690 con 0x7f854806ea60 2024-04-25T08:47:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8548069010 con 0x7f854806ea60 2024-04-25T08:47:04.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.705+0000 7f853f400640 1 -- 172.21.15.46:0/1297326213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f853001b070 con 0x7f854806ea60 2024-04-25T08:47:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.709+0000 7f853f400640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8524066f90 0x7f8524069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.709+0000 7f853f400640 1 -- 172.21.15.46:0/1297326213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8530084620 con 0x7f854806ea60 2024-04-25T08:47:04.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.709+0000 7f8546a00640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8524066f90 0x7f8524069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:04.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.709+0000 7f8546a00640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8524066f90 0x7f8524069450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f854806e3f0 tx=0x7f8538008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:04.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.713+0000 7f853f400640 1 -- 172.21.15.46:0/1297326213 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85300553b0 con 0x7f854806ea60 2024-04-25T08:47:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.873+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8548152e70 con 0x7f854806ea60 2024-04-25T08:47:04.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.873+0000 7f853f400640 1 -- 172.21.15.46:0/1297326213 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8530054cb0 con 0x7f854806ea60 2024-04-25T08:47:04.869 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8524066f90 msgr2=0x7f8524069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8524066f90 0x7f8524069450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f854806e3f0 tx=0x7f8538008040 comp rx=0 tx=0).stop 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 msgr2=0x7f8548146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 0x7f8548146b70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f85300011f0 tx=0x7f853000c9b0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 shutdown_connections 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8524066f90 0x7f8524069450 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f854815dae0 0x7f85481602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806ea60 0x7f8548146b70 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 --2- 172.21.15.46:0/1297326213 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f854806e620 0x7f85481463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 >> 172.21.15.46:0/1297326213 conn(0x7f8548071ae0 msgr2=0x7f85480adba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 shutdown_connections 2024-04-25T08:47:04.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:04.877+0000 7f854d800640 1 -- 172.21.15.46:0/1297326213 wait complete. 2024-04-25T08:47:04.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:47:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc786200640 1 Processor -- start 2024-04-25T08:47:05.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc786200640 1 -- start start 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc786200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc780145310 0x7fc78006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc786200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc78015d090 0x7fc78015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc786200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7800cd930 con 0x7fc78006dc70 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc786200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7800cdbd0 con 0x7fc780145310 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc786200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7800cdeb0 con 0x7fc78015d090 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc77f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc780145310 0x7fc78006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc77f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc780145310 0x7fc78006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47754/0 (socket says 172.21.15.46:47754) 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc777e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc78006dc70 0x7fc7800cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48584/0 (socket says 172.21.15.46:48584) 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc77f400640 1 -- 172.21.15.46:0/1315493443 learned_addr learned my addr 172.21.15.46:0/1315493443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc77fe00640 1 --2- 172.21.15.46:0/1315493443 >> v2:172.21.15.82:3300/0 conn(0x7fc78015d090 0x7fc78015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc77f400640 1 -- 172.21.15.46:0/1315493443 >> v2:172.21.15.82:3300/0 conn(0x7fc78015d090 msgr2=0x7fc78015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc77f400640 1 --2- 172.21.15.46:0/1315493443 >> v2:172.21.15.82:3300/0 conn(0x7fc78015d090 0x7fc78015d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc77f400640 1 -- 172.21.15.46:0/1315493443 >> v1:172.21.15.46:6789/0 conn(0x7fc78006dc70 legacy=0x7fc7800cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.041+0000 7fc77f400640 1 -- 172.21.15.46:0/1315493443 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800ce190 con 0x7fc780145310 2024-04-25T08:47:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc77f400640 1 --2- 172.21.15.46:0/1315493443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 0x7fc78006e0d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc78006d060 tx=0x7fc77001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1910f3db6bb02b50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc77ea00640 1 -- 172.21.15.46:0/1315493443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700ab610 con 0x7fc780145310 2024-04-25T08:47:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc77ea00640 1 -- 172.21.15.46:0/1315493443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7700b0070 con 0x7fc780145310 2024-04-25T08:47:05.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc77ea00640 1 -- 172.21.15.46:0/1315493443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700ab9d0 con 0x7fc780145310 2024-04-25T08:47:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc786200640 1 -- 172.21.15.46:0/1315493443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 msgr2=0x7fc78006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc786200640 1 --2- 172.21.15.46:0/1315493443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 0x7fc78006e0d0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc78006d060 tx=0x7fc77001c520 comp rx=0 tx=0).stop 2024-04-25T08:47:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc786200640 1 -- 172.21.15.46:0/1315493443 shutdown_connections 2024-04-25T08:47:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc786200640 1 --2- 172.21.15.46:0/1315493443 >> v2:172.21.15.82:3300/0 conn(0x7fc78015d090 0x7fc78015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc786200640 1 --2- 172.21.15.46:0/1315493443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 0x7fc78006e0d0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc786200640 1 -- 172.21.15.46:0/1315493443 >> 172.21.15.46:0/1315493443 conn(0x7fc78006ac10 msgr2=0x7fc78006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc786200640 1 -- 172.21.15.46:0/1315493443 shutdown_connections 2024-04-25T08:47:05.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.045+0000 7fc786200640 1 -- 172.21.15.46:0/1315493443 wait complete. 2024-04-25T08:47:05.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc786200640 1 Processor -- start 2024-04-25T08:47:05.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc786200640 1 -- start start 2024-04-25T08:47:05.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc786200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 0x7fc7801f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc786200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78015d090 0x7fc7800c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc786200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78013b5e0 0x7fc78015acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc786200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc780168510 con 0x7fc780145310 2024-04-25T08:47:05.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc786200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800cf380 con 0x7fc78013b5e0 2024-04-25T08:47:05.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc786200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc780168690 con 0x7fc78015d090 2024-04-25T08:47:05.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 0x7fc7801f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 0x7fc7801f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47756/0 (socket says 172.21.15.46:47756) 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77f400640 1 -- 172.21.15.46:0/92386207 learned_addr learned my addr 172.21.15.46:0/92386207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc777e00640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78015d090 0x7fc7800c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77fe00640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78013b5e0 0x7fc78015acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77f400640 1 -- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78015d090 msgr2=0x7fc7800c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77f400640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78015d090 0x7fc7800c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77f400640 1 -- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78013b5e0 msgr2=0x7fc78015acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77f400640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78013b5e0 0x7fc78015acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77f400640 1 -- 172.21.15.46:0/92386207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7801f70d0 con 0x7fc780145310 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77fe00640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78013b5e0 0x7fc78015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc777e00640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78015d090 0x7fc7800c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:05.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc77f400640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 0x7fc7801f6970 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc770019130 tx=0x7fc77001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc777400640 1 -- 172.21.15.46:0/92386207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700be5f0 con 0x7fc780145310 2024-04-25T08:47:05.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc777400640 1 -- 172.21.15.46:0/92386207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7700be790 con 0x7fc780145310 2024-04-25T08:47:05.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7801f73b0 con 0x7fc780145310 2024-04-25T08:47:05.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.049+0000 7fc777400640 1 -- 172.21.15.46:0/92386207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7700beb60 con 0x7fc780145310 2024-04-25T08:47:05.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.053+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7801f7970 con 0x7fc780145310 2024-04-25T08:47:05.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.053+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc780069010 con 0x7fc780145310 2024-04-25T08:47:05.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.053+0000 7fc777400640 1 -- 172.21.15.46:0/92386207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7700b43f0 con 0x7fc780145310 2024-04-25T08:47:05.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.053+0000 7fc777400640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc75c066f90 0x7fc75c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.053+0000 7fc777400640 1 -- 172.21.15.46:0/92386207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc77011ff90 con 0x7fc780145310 2024-04-25T08:47:05.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.053+0000 7fc777e00640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc75c066f90 0x7fc75c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.053+0000 7fc777e00640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc75c066f90 0x7fc75c069450 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc780165ac0 tx=0x7fc768007d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.057+0000 7fc777400640 1 -- 172.21.15.46:0/92386207 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7700c4060 con 0x7fc780145310 2024-04-25T08:47:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc78015ca10 con 0x7fc780145310 2024-04-25T08:47:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc777400640 1 -- 172.21.15.46:0/92386207 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc7700a8050 con 0x7fc780145310 2024-04-25T08:47:05.215 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc75c066f90 msgr2=0x7fc75c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc75c066f90 0x7fc75c069450 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc780165ac0 tx=0x7fc768007d70 comp rx=0 tx=0).stop 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 msgr2=0x7fc7801f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 0x7fc7801f6970 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc770019130 tx=0x7fc77001ad60 comp rx=0 tx=0).stop 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 shutdown_connections 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc75c066f90 0x7fc75c069450 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc78013b5e0 0x7fc78015acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc78015d090 0x7fc7800c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 --2- 172.21.15.46:0/92386207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc780145310 0x7fc7801f6970 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 >> 172.21.15.46:0/92386207 conn(0x7fc78006ac10 msgr2=0x7fc7800ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 shutdown_connections 2024-04-25T08:47:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.221+0000 7fc786200640 1 -- 172.21.15.46:0/92386207 wait complete. 2024-04-25T08:47:05.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:47:05.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e6c200640 1 Processor -- start 2024-04-25T08:47:05.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e6c200640 1 -- start start 2024-04-25T08:47:05.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e6c200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e640d0980 0x7f9e6415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e6c200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 0x7f9e6406cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e6c200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e640d0f70 con 0x7f9e64099a00 2024-04-25T08:47:05.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e6c200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6415f360 con 0x7f9e640d0980 2024-04-25T08:47:05.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e6c200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6409a7c0 con 0x7f9e6406c7c0 2024-04-25T08:47:05.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e68e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e640d0980 0x7f9e6415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e68e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e640d0980 0x7f9e6415ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47764/0 (socket says 172.21.15.46:47764) 2024-04-25T08:47:05.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e68e00640 1 -- 172.21.15.46:0/3940030523 learned_addr learned my addr 172.21.15.46:0/3940030523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e6a200640 1 --2- 172.21.15.46:0/3940030523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 0x7f9e6406cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e68e00640 1 -- 172.21.15.46:0/3940030523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 msgr2=0x7f9e6406cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e68e00640 1 --2- 172.21.15.46:0/3940030523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 0x7f9e6406cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e68e00640 1 -- 172.21.15.46:0/3940030523 >> v1:172.21.15.82:6789/0 conn(0x7f9e64099a00 legacy=0x7f9e64099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e68e00640 1 -- 172.21.15.46:0/3940030523 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640d1250 con 0x7f9e640d0980 2024-04-25T08:47:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.385+0000 7f9e63e00640 1 -- 172.21.15.46:0/3940030523 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380436427 0 0) 0x7f9e640d0f70 con 0x7f9e64099a00 2024-04-25T08:47:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e68e00640 1 --2- 172.21.15.46:0/3940030523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 0x7f9e6415ee20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009870 tx=0x7f9e5001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b426dc55f71db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e63e00640 1 -- 172.21.15.46:0/3940030523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500b0070 con 0x7f9e640d0980 2024-04-25T08:47:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e63e00640 1 -- 172.21.15.46:0/3940030523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e500ab600 con 0x7f9e640d0980 2024-04-25T08:47:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e63e00640 1 -- 172.21.15.46:0/3940030523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500ab9d0 con 0x7f9e640d0980 2024-04-25T08:47:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e6c200640 1 -- 172.21.15.46:0/3940030523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 msgr2=0x7f9e6415ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e6c200640 1 --2- 172.21.15.46:0/3940030523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 0x7f9e6415ee20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009870 tx=0x7f9e5001c440 comp rx=0 tx=0).stop 2024-04-25T08:47:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e6c200640 1 -- 172.21.15.46:0/3940030523 shutdown_connections 2024-04-25T08:47:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e6c200640 1 --2- 172.21.15.46:0/3940030523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 0x7f9e6406cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e6c200640 1 --2- 172.21.15.46:0/3940030523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 0x7f9e6415ee20 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e6c200640 1 -- 172.21.15.46:0/3940030523 >> 172.21.15.46:0/3940030523 conn(0x7f9e640d16b0 msgr2=0x7f9e640d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e6c200640 1 -- 172.21.15.46:0/3940030523 shutdown_connections 2024-04-25T08:47:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.389+0000 7f9e6c200640 1 -- 172.21.15.46:0/3940030523 wait complete. 2024-04-25T08:47:05.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 Processor -- start 2024-04-25T08:47:05.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 -- start start 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 0x7f9e64147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e64099a00 0x7f9e640b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 0x7f9e640c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6416afe0 con 0x7f9e640d0980 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6416ae60 con 0x7f9e64099a00 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6416b160 con 0x7f9e6406c7c0 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 0x7f9e640c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 0x7f9e640c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47778/0 (socket says 172.21.15.46:47778) 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6a200640 1 -- 172.21.15.46:0/1026543859 learned_addr learned my addr 172.21.15.46:0/1026543859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e69800640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 0x7f9e64147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6a200640 1 -- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 msgr2=0x7f9e64147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e68e00640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e64099a00 0x7f9e640b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6a200640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 0x7f9e64147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6a200640 1 -- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e64099a00 msgr2=0x7f9e640b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6a200640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e64099a00 0x7f9e640b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6a200640 1 -- 172.21.15.46:0/1026543859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640b0820 con 0x7f9e640d0980 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e69800640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 0x7f9e64147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e68e00640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e64099a00 0x7f9e640b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6a200640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 0x7f9e640c5d20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00efc0 tx=0x7f9e5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e61600640 1 -- 172.21.15.46:0/1026543859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c018020 con 0x7f9e640d0980 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e61600640 1 -- 172.21.15.46:0/1026543859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c014480 con 0x7f9e640d0980 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e640af950 con 0x7f9e640d0980 2024-04-25T08:47:05.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e61600640 1 -- 172.21.15.46:0/1026543859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c014870 con 0x7f9e640d0980 2024-04-25T08:47:05.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.393+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e640c5240 con 0x7f9e640d0980 2024-04-25T08:47:05.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.397+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e640d23e0 con 0x7f9e640d0980 2024-04-25T08:47:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.397+0000 7f9e61600640 1 -- 172.21.15.46:0/1026543859 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e5c01b020 con 0x7f9e640d0980 2024-04-25T08:47:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.397+0000 7f9e61600640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e44066f60 0x7f9e44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.397+0000 7f9e61600640 1 -- 172.21.15.46:0/1026543859 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9e5c085c50 con 0x7f9e640d0980 2024-04-25T08:47:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.397+0000 7f9e69800640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e44066f60 0x7f9e44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.397+0000 7f9e69800640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e44066f60 0x7f9e44069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9e58000f70 tx=0x7f9e58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.401+0000 7f9e61600640 1 -- 172.21.15.46:0/1026543859 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e5c08b020 con 0x7f9e640d0980 2024-04-25T08:47:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.573+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9e64069060 con 0x7f9e640d0980 2024-04-25T08:47:05.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.573+0000 7f9e61600640 1 -- 172.21.15.46:0/1026543859 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v79) v1 ==== 80+0+120 (secure 0 0 0) 0x7f9e5c022070 con 0x7f9e640d0980 2024-04-25T08:47:05.569 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/960745191 2024-04-26T08:46:56.883011+0000 2024-04-25T08:47:05.569 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6833/960745191 2024-04-26T08:46:56.883011+0000 2024-04-25T08:47:05.569 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-04-25T08:47:05.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e44066f60 msgr2=0x7f9e44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e44066f60 0x7f9e44069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9e58000f70 tx=0x7f9e58009040 comp rx=0 tx=0).stop 2024-04-25T08:47:05.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 msgr2=0x7f9e640c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 0x7f9e640c5d20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c00efc0 tx=0x7f9e5c00c490 comp rx=0 tx=0).stop 2024-04-25T08:47:05.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 shutdown_connections 2024-04-25T08:47:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9e44066f60 0x7f9e44069420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e640d0980 0x7f9e640c5d20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e64099a00 0x7f9e640b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 --2- 172.21.15.46:0/1026543859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9e6406c7c0 0x7f9e64147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 >> 172.21.15.46:0/1026543859 conn(0x7f9e640d16b0 msgr2=0x7f9e64070250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 shutdown_connections 2024-04-25T08:47:05.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.577+0000 7f9e6c200640 1 -- 172.21.15.46:0/1026543859 wait complete. 2024-04-25T08:47:05.590 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:47:05.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:05.590 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:47:05.617 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:47:05.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:05.617 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:47:05.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:05.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-25T08:47:05.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.865+0000 7f2df8c00640 1 Processor -- start 2024-04-25T08:47:05.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.865+0000 7f2df8c00640 1 -- start start 2024-04-25T08:47:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df8c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2df406ea20 0x7f2df40ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df8c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df40b2e60 con 0x7f2df4140ee0 2024-04-25T08:47:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df8c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df40b30d0 con 0x7f2df4141580 2024-04-25T08:47:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df8c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df41412c0 con 0x7f2df406ea20 2024-04-25T08:47:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2df4140ee0 0x7f2df406e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48590/0 (socket says 172.21.15.46:48590) 2024-04-25T08:47:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df1e00640 1 -- 172.21.15.46:0/1526562959 learned_addr learned my addr 172.21.15.46:0/1526562959 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:47:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2de9400640 1 --2- 172.21.15.46:0/1526562959 >> v2:172.21.15.82:3300/0 conn(0x7f2df406ea20 0x7f2df40ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df1400640 1 -- 172.21.15.46:0/1526562959 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766576193 0 0) 0x7f2df40b2e60 con 0x7f2df4140ee0 2024-04-25T08:47:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df1400640 1 -- 172.21.15.46:0/1526562959 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dd8003530 con 0x7f2df4140ee0 2024-04-25T08:47:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df1400640 1 -- 172.21.15.46:0/1526562959 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614597876 0 0) 0x7f2df40b30d0 con 0x7f2df4141580 2024-04-25T08:47:05.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df1400640 1 -- 172.21.15.46:0/1526562959 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df40b2e60 con 0x7f2df4141580 2024-04-25T08:47:05.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2de9400640 1 -- 172.21.15.46:0/1526562959 >> v1:172.21.15.82:6789/0 conn(0x7f2df4141580 legacy=0x7f2df40b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2de9400640 1 -- 172.21.15.46:0/1526562959 >> v1:172.21.15.46:6789/0 conn(0x7f2df4140ee0 legacy=0x7f2df406e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2df1400640 1 -- 172.21.15.46:0/1526562959 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2943882070 0 0) 0x7f2dd8003530 con 0x7f2df4140ee0 2024-04-25T08:47:05.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.869+0000 7f2de9400640 1 -- 172.21.15.46:0/1526562959 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40b33b0 con 0x7f2df406ea20 2024-04-25T08:47:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2de9400640 1 --2- 172.21.15.46:0/1526562959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2df406ea20 0x7f2df40ce4d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0097c0 tx=0x7f2ddc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2593ac236e04e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df1400640 1 -- 172.21.15.46:0/1526562959 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc0aa620 con 0x7f2df406ea20 2024-04-25T08:47:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df1400640 1 -- 172.21.15.46:0/1526562959 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc0af070 con 0x7f2df406ea20 2024-04-25T08:47:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df1400640 1 -- 172.21.15.46:0/1526562959 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc0aa9c0 con 0x7f2df406ea20 2024-04-25T08:47:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df8c00640 1 -- 172.21.15.46:0/1526562959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2df406ea20 msgr2=0x7f2df40ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df8c00640 1 --2- 172.21.15.46:0/1526562959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2df406ea20 0x7f2df40ce4d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0097c0 tx=0x7f2ddc01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df8c00640 1 -- 172.21.15.46:0/1526562959 shutdown_connections 2024-04-25T08:47:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df8c00640 1 --2- 172.21.15.46:0/1526562959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2df406ea20 0x7f2df40ce4d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df8c00640 1 -- 172.21.15.46:0/1526562959 >> 172.21.15.46:0/1526562959 conn(0x7f2df4067050 msgr2=0x7f2df406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df8c00640 1 -- 172.21.15.46:0/1526562959 shutdown_connections 2024-04-25T08:47:05.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df8c00640 1 -- 172.21.15.46:0/1526562959 wait complete. 2024-04-25T08:47:05.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df8c00640 1 Processor -- start 2024-04-25T08:47:05.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.873+0000 7f2df8c00640 1 -- start start 2024-04-25T08:47:05.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df406ea20 0x7f2df4071a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df8c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2df4140ee0 0x7f2df40ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df8c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2df4141580 0x7f2df4139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4168540 con 0x7f2df406ea20 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df8c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df41683c0 con 0x7f2df4141580 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df8c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df41686c0 con 0x7f2df4140ee0 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df406ea20 0x7f2df4071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df406ea20 0x7f2df4071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47788/0 (socket says 172.21.15.46:47788) 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df1e00640 1 -- 172.21.15.46:0/438286405 learned_addr learned my addr 172.21.15.46:0/438286405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df1e00640 1 -- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2df4140ee0 msgr2=0x7f2df40ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df1e00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2df4140ee0 0x7f2df40ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df1e00640 1 -- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2df4141580 msgr2=0x7f2df4139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df1e00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2df4141580 0x7f2df4139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df1e00640 1 -- 172.21.15.46:0/438286405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40af290 con 0x7f2df406ea20 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df1e00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df406ea20 0x7f2df4071a00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2de400cb50 tx=0x7f2de400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2deaa00640 1 -- 172.21.15.46:0/438286405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4017070 con 0x7f2df406ea20 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2deaa00640 1 -- 172.21.15.46:0/438286405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de400ad40 con 0x7f2df406ea20 2024-04-25T08:47:05.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df8c00640 1 -- 172.21.15.46:0/438286405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df40af570 con 0x7f2df406ea20 2024-04-25T08:47:05.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2deaa00640 1 -- 172.21.15.46:0/438286405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4012640 con 0x7f2df406ea20 2024-04-25T08:47:05.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.877+0000 7f2df8c00640 1 -- 172.21.15.46:0/438286405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df40afb30 con 0x7f2df406ea20 2024-04-25T08:47:05.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2deaa00640 1 -- 172.21.15.46:0/438286405 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2de401a070 con 0x7f2df406ea20 2024-04-25T08:47:05.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2deaa00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dd0066f60 0x7f2dd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2deaa00640 1 -- 172.21.15.46:0/438286405 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2de40850f0 con 0x7f2df406ea20 2024-04-25T08:47:05.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2dfc05f000 1 Processor -- start 2024-04-25T08:47:05.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2dfc05f000 1 -- start start 2024-04-25T08:47:05.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2de9400640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dd0066f60 0x7f2dd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349a8c0 0x5627e349aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2de9400640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dd0066f60 0x7f2dd0069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0ae000 tx=0x7f2ddc004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 0x5627e3450110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e3450a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e329edd0 con 0x5627e349a8c0 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e3227010 con 0x5627e3450650 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627e329ae70 con 0x5627e349b1e0 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349a8c0 0x5627e349aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349a8c0 0x5627e349aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47802/0 (socket says 172.21.15.46:47802) 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df1e00640 1 -- 172.21.15.46:0/3029272529 learned_addr learned my addr 172.21.15.46:0/3029272529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2de9400640 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 0x5627e3450110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df2800640 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e3450a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df1e00640 1 -- 172.21.15.46:0/3029272529 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 msgr2=0x5627e3450110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df1e00640 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 0x5627e3450110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df1e00640 1 -- 172.21.15.46:0/3029272529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 msgr2=0x5627e3450a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df1e00640 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e3450a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df1e00640 1 -- 172.21.15.46:0/3029272529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e32579c0 con 0x5627e349a8c0 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df2800640 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e3450a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2df1e00640 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349a8c0 0x5627e349aca0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2de4055720 tx=0x7f2de4055850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75de872ef181e1bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2de2a00640 1 -- 172.21.15.46:0/3029272529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4053a90 con 0x5627e349a8c0 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2de2a00640 1 -- 172.21.15.46:0/3029272529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de4051e00 con 0x5627e349a8c0 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.881+0000 7f2de2a00640 1 -- 172.21.15.46:0/3029272529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de405dc10 con 0x5627e349a8c0 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- 172.21.15.46:0/3029272529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349a8c0 msgr2=0x5627e349aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349a8c0 0x5627e349aca0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2de4055720 tx=0x7f2de4055850 comp rx=0 tx=0).stop 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- 172.21.15.46:0/3029272529 shutdown_connections 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e3450a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 0x5627e3450110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/3029272529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349a8c0 0x5627e349aca0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- 172.21.15.46:0/3029272529 >> 172.21.15.46:0/3029272529 conn(0x5627e3338870 msgr2=0x5627e344e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- 172.21.15.46:0/3029272529 shutdown_connections 2024-04-25T08:47:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- 172.21.15.46:0/3029272529 wait complete. 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- start start 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 Processor -- start 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e3454f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349a8c0 0x5627e34554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 0x5627e34559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e3259420 con 0x5627e3450650 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e3301000 con 0x5627e349a8c0 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627e3459c20 con 0x5627e349b1e0 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 10 client.0 _ll_register_callbacks cb 0x5627e343bf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e3454f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e3454f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47812/0 (socket says 172.21.15.46:47812) 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df1e00640 1 -- 172.21.15.46:0/529925009 learned_addr learned my addr 172.21.15.46:0/529925009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df2800640 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 0x5627e34559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2de9400640 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349a8c0 0x5627e34554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df1e00640 1 -- 172.21.15.46:0/529925009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 msgr2=0x5627e34559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df1e00640 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 0x5627e34559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df1e00640 1 -- 172.21.15.46:0/529925009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349a8c0 msgr2=0x5627e34554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df1e00640 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349a8c0 0x5627e34554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df1e00640 1 -- 172.21.15.46:0/529925009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627e3227260 con 0x5627e3450650 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df2800640 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 0x5627e34559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2df1e00640 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e3454f60 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2de400cfd0 tx=0x7f2de400af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f1c31e5abe1a26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dd7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dd7400640 1 -- 172.21.15.46:0/529925009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4072060 con 0x5627e3450650 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dd7400640 1 -- 172.21.15.46:0/529925009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2de406d100 con 0x5627e3450650 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- 172.21.15.46:0/529925009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2de4051e00 con 0x5627e3450650 2024-04-25T08:47:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dd7400640 1 -- 172.21.15.46:0/529925009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de406d320 con 0x5627e3450650 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 10 client.7101 fetch_fsmap learned FSMap version 51 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dfc05f000 1 -- 172.21.15.46:0/529925009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x5627e3259b30 con 0x5627e3450650 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.885+0000 7f2dd7400640 1 -- 172.21.15.46:0/529925009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de4051e00 con 0x5627e3450650 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dd7400640 1 -- 172.21.15.46:0/529925009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2de4004b70 con 0x5627e3450650 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 10 client.7101 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 10 client.7101 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 20 client.7101 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5627e34563f0 0x5627e34ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 4 client.7101 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 -- 172.21.15.46:0/529925009 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5627e323ac80 con 0x5627e34563f0 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2de9400640 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5627e34563f0 0x5627e34ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2de9400640 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5627e34563f0 0x5627e34ab960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120297808846e0c5 server_cookie=76454d73a6ac3f72 in_seq=0 out_seq=0 2024-04-25T08:47:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dd7400640 10 client.7101 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dd7400640 1 -- 172.21.15.46:0/529925009 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5627e323ac80 con 0x5627e34563f0 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dd7400640 10 client.7101 handle_command_reply: tid=0 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 client.7101 shutdown 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 -- 172.21.15.46:0/529925009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 msgr2=0x5627e3454f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e3454f60 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2de400cfd0 tx=0x7f2de400af30 comp rx=0 tx=0).stop 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 -- 172.21.15.46:0/529925009 shutdown_connections 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5627e34563f0 0x5627e34ab960 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349b1e0 0x5627e34559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349a8c0 0x5627e34554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dd7400640 0 client.7101 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/529925009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e3454f60 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 -- 172.21.15.46:0/529925009 >> 172.21.15.46:0/529925009 conn(0x5627e3338870 msgr2=0x5627e344e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 -- 172.21.15.46:0/529925009 shutdown_connections 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 1 -- 172.21.15.46:0/529925009 wait complete. 2024-04-25T08:47:05.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.889+0000 7f2dfc05f000 20 client.7101 trim_cache size 0 max 16384 2024-04-25T08:47:05.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2dfc05f000 1 Processor -- start 2024-04-25T08:47:05.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2dfc05f000 1 -- start start 2024-04-25T08:47:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e34aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 0x5627e34aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349b1e0 0x5627e34ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4203590 con 0x5627e349b1e0 2024-04-25T08:47:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df4202840 con 0x5627e3450650 2024-04-25T08:47:05.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df413e880 con 0x5627e349a8c0 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349b1e0 0x5627e34ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349b1e0 0x5627e34ab3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47822/0 (socket says 172.21.15.46:47822) 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df2800640 1 -- 172.21.15.46:0/782119952 learned_addr learned my addr 172.21.15.46:0/782119952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df1e00640 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e34aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2de9400640 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 0x5627e34aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df2800640 1 -- 172.21.15.46:0/782119952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 msgr2=0x5627e34aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df2800640 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 0x5627e34aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df2800640 1 -- 172.21.15.46:0/782119952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 msgr2=0x5627e34aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df2800640 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e34aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df2800640 1 -- 172.21.15.46:0/782119952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627e33246d0 con 0x5627e349b1e0 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df1e00640 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e34aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.905+0000 7f2df2800640 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349b1e0 0x5627e34ab3e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2dec001490 tx=0x7f2dec001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5859bbb5f4ffc5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2de2000640 1 -- 172.21.15.46:0/782119952 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0052d0 con 0x5627e349b1e0 2024-04-25T08:47:05.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2de2000640 1 -- 172.21.15.46:0/782119952 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec005490 con 0x5627e349b1e0 2024-04-25T08:47:05.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2de2000640 1 -- 172.21.15.46:0/782119952 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec003820 con 0x5627e349b1e0 2024-04-25T08:47:05.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 -- 172.21.15.46:0/782119952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349b1e0 msgr2=0x5627e34ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349b1e0 0x5627e34ab3e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2dec001490 tx=0x7f2dec001950 comp rx=0 tx=0).stop 2024-04-25T08:47:05.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 -- 172.21.15.46:0/782119952 shutdown_connections 2024-04-25T08:47:05.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e349b1e0 0x5627e34ab3e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 0x5627e34aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/782119952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e3450650 0x5627e34aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 -- 172.21.15.46:0/782119952 >> 172.21.15.46:0/782119952 conn(0x5627e3338870 msgr2=0x5627e3499870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 -- 172.21.15.46:0/782119952 shutdown_connections 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 -- 172.21.15.46:0/782119952 wait complete. 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 -- start start 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 Processor -- start 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e34f7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 0x5627e34f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349b1e0 0x5627e34f8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627e329e0e0 con 0x5627e3450650 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df415c4f0 con 0x5627e349b1e0 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627e3245ae0 con 0x5627e349a8c0 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dfc05f000 10 client.0 _ll_register_callbacks cb 0x5627e323b200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:05.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e34f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e34f7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47838/0 (socket says 172.21.15.46:47838) 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df1e00640 1 -- 172.21.15.46:0/1750012953 learned_addr learned my addr 172.21.15.46:0/1750012953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2de9400640 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 0x5627e34f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df2800640 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349b1e0 0x5627e34f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df1e00640 1 -- 172.21.15.46:0/1750012953 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 msgr2=0x5627e34f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df1e00640 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 0x5627e34f7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df1e00640 1 -- 172.21.15.46:0/1750012953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349b1e0 msgr2=0x5627e34f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df1e00640 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349b1e0 0x5627e34f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df1e00640 1 -- 172.21.15.46:0/1750012953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627e3321900 con 0x5627e3450650 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2df1e00640 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e34f7800 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2de4055490 tx=0x7f2de4055610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c5a242972b4482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.909+0000 7f2dd7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:05.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dd7400640 1 -- 172.21.15.46:0/1750012953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4055e90 con 0x5627e3450650 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dd7400640 1 -- 172.21.15.46:0/1750012953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2de4054be0 con 0x5627e3450650 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 -- 172.21.15.46:0/1750012953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dec005490 con 0x5627e3450650 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dd7400640 1 -- 172.21.15.46:0/1750012953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de4072030 con 0x5627e3450650 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 10 client.7107 fetch_fsmap learned FSMap version 51 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 -- 172.21.15.46:0/1750012953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x5627e3245f20 con 0x5627e3450650 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dd7400640 1 -- 172.21.15.46:0/1750012953 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec005490 con 0x5627e3450650 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dd7400640 1 -- 172.21.15.46:0/1750012953 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2de40514e0 con 0x5627e3450650 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 10 client.7107 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 10 client.7107 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 20 client.7107 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5627e34f8bb0 0x5627e3503310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 4 client.7107 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 -- 172.21.15.46:0/1750012953 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x5627e32ac310 con 0x5627e34f8bb0 2024-04-25T08:47:05.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2de9400640 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5627e34f8bb0 0x5627e3503310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:05.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2de9400640 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5627e34f8bb0 0x5627e3503310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7401bed06d279902 server_cookie=84297125e8e33872 in_seq=0 out_seq=0 2024-04-25T08:47:05.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dd7400640 10 client.7107 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dd7400640 1 -- 172.21.15.46:0/1750012953 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5627e32ac310 con 0x5627e34f8bb0 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dd7400640 10 client.7107 handle_command_reply: tid=0 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 client.7107 shutdown 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 -- 172.21.15.46:0/1750012953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 msgr2=0x5627e34f7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e34f7800 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2de4055490 tx=0x7f2de4055610 comp rx=0 tx=0).stop 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 -- 172.21.15.46:0/1750012953 shutdown_connections 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5627e34f8bb0 0x5627e3503310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5627e349b1e0 0x5627e34f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5627e349a8c0 0x5627e34f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 --2- 172.21.15.46:0/1750012953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5627e3450650 0x5627e34f7800 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 -- 172.21.15.46:0/1750012953 >> 172.21.15.46:0/1750012953 conn(0x5627e3338870 msgr2=0x5627e3499f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dd7400640 0 client.7107 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:05.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 -- 172.21.15.46:0/1750012953 shutdown_connections 2024-04-25T08:47:05.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 1 -- 172.21.15.46:0/1750012953 wait complete. 2024-04-25T08:47:05.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2dfc05f000 20 client.7107 trim_cache size 0 max 16384 2024-04-25T08:47:05.912 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:47:05.912 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:47:05.912 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "d696032f-1064-4b71-bd03-d49dfbea588c", 2024-04-25T08:47:05.912 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:47:05.912 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 -- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dd0066f60 msgr2=0x7f2dd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dd0066f60 0x7f2dd0069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0ae000 tx=0x7f2ddc004740 comp rx=0 tx=0).stop 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 -- 172.21.15.46:0/438286405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df406ea20 msgr2=0x7f2df4071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df406ea20 0x7f2df4071a00 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2de400cb50 tx=0x7f2de400a430 comp rx=0 tx=0).stop 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 -- 172.21.15.46:0/438286405 shutdown_connections 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dd0066f60 0x7f2dd0069420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2df4141580 0x7f2df4139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2df4140ee0 0x7f2df40ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 --2- 172.21.15.46:0/438286405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2df406ea20 0x7f2df4071a00 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 -- 172.21.15.46:0/438286405 >> 172.21.15.46:0/438286405 conn(0x7f2df4067050 msgr2=0x7f2df413c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:05.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 -- 172.21.15.46:0/438286405 shutdown_connections 2024-04-25T08:47:05.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:05.913+0000 7f2df8c00640 1 -- 172.21.15.46:0/438286405 wait complete. 2024-04-25T08:47:05.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:05.928 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:47:06.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febf2400640 1 Processor -- start 2024-04-25T08:47:06.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febf2400640 1 -- start start 2024-04-25T08:47:06.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febf2400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febec0ce6f0 0x7febec0704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febf2400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec144b70 0x7febec0cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febf2400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febec1453e0 con 0x7febec0cddf0 2024-04-25T08:47:06.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febf2400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febec0cd960 con 0x7febec0ce6f0 2024-04-25T08:47:06.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febf2400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febec0cf010 con 0x7febec144b70 2024-04-25T08:47:06.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febeb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febec0ce6f0 0x7febec0704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febeb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febec0ce6f0 0x7febec0704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47846/0 (socket says 172.21.15.46:47846) 2024-04-25T08:47:06.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febeb400640 1 -- 172.21.15.46:0/1732575278 learned_addr learned my addr 172.21.15.46:0/1732575278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febe3e00640 1 --2- 172.21.15.46:0/1732575278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec144b70 0x7febec0cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febeb400640 1 -- 172.21.15.46:0/1732575278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec144b70 msgr2=0x7febec0cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febeb400640 1 --2- 172.21.15.46:0/1732575278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec144b70 0x7febec0cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febeb400640 1 -- 172.21.15.46:0/1732575278 >> v1:172.21.15.82:6789/0 conn(0x7febec0cddf0 legacy=0x7febec15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.077+0000 7febeb400640 1 -- 172.21.15.46:0/1732575278 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec070be0 con 0x7febec0ce6f0 2024-04-25T08:47:06.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febeb400640 1 --2- 172.21.15.46:0/1732575278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 0x7febec0704e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7febdc009870 tx=0x7febdc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51d747b0d21f73c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febeaa00640 1 -- 172.21.15.46:0/1732575278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0a9620 con 0x7febec0ce6f0 2024-04-25T08:47:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febeaa00640 1 -- 172.21.15.46:0/1732575278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc0a97c0 con 0x7febec0ce6f0 2024-04-25T08:47:06.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febeaa00640 1 -- 172.21.15.46:0/1732575278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0a9b90 con 0x7febec0ce6f0 2024-04-25T08:47:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febf2400640 1 -- 172.21.15.46:0/1732575278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 msgr2=0x7febec0704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febf2400640 1 --2- 172.21.15.46:0/1732575278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 0x7febec0704e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7febdc009870 tx=0x7febdc01c560 comp rx=0 tx=0).stop 2024-04-25T08:47:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febf2400640 1 -- 172.21.15.46:0/1732575278 shutdown_connections 2024-04-25T08:47:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febf2400640 1 --2- 172.21.15.46:0/1732575278 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec144b70 0x7febec0cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febf2400640 1 --2- 172.21.15.46:0/1732575278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 0x7febec0704e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febf2400640 1 -- 172.21.15.46:0/1732575278 >> 172.21.15.46:0/1732575278 conn(0x7febec0b23a0 msgr2=0x7febec0b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febf2400640 1 -- 172.21.15.46:0/1732575278 shutdown_connections 2024-04-25T08:47:06.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.081+0000 7febf2400640 1 -- 172.21.15.46:0/1732575278 wait complete. 2024-04-25T08:47:06.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 Processor -- start 2024-04-25T08:47:06.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 -- start start 2024-04-25T08:47:06.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec0cddf0 0x7febec15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 0x7febec13ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febec144b70 0x7febec15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec144870 con 0x7febec0ce6f0 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec0cd5f0 con 0x7febec144b70 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febec168c00 con 0x7febec0cddf0 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 0x7febec13ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 0x7febec13ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47856/0 (socket says 172.21.15.46:47856) 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3e00640 1 -- 172.21.15.46:0/1774378627 learned_addr learned my addr 172.21.15.46:0/1774378627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febeb400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec0cddf0 0x7febec15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febebe00640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febec144b70 0x7febec15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3e00640 1 -- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec0cddf0 msgr2=0x7febec15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3e00640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec0cddf0 0x7febec15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3e00640 1 -- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febec144b70 msgr2=0x7febec15a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3e00640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febec144b70 0x7febec15a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3e00640 1 -- 172.21.15.46:0/1774378627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec0cc440 con 0x7febec0ce6f0 2024-04-25T08:47:06.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febebe00640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febec144b70 0x7febec15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:06.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3e00640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 0x7febec13ed40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7febd400e8a0 tx=0x7febd400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:06.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3400640 1 -- 172.21.15.46:0/1774378627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd4017070 con 0x7febec0ce6f0 2024-04-25T08:47:06.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3400640 1 -- 172.21.15.46:0/1774378627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd400c940 con 0x7febec0ce6f0 2024-04-25T08:47:06.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febe3400640 1 -- 172.21.15.46:0/1774378627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd400ccf0 con 0x7febec0ce6f0 2024-04-25T08:47:06.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 -- 172.21.15.46:0/1774378627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec0cc160 con 0x7febec0ce6f0 2024-04-25T08:47:06.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.085+0000 7febf2400640 1 -- 172.21.15.46:0/1774378627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febec13a7f0 con 0x7febec0ce6f0 2024-04-25T08:47:06.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.089+0000 7febe3400640 1 -- 172.21.15.46:0/1774378627 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7febd401a070 con 0x7febec0ce6f0 2024-04-25T08:47:06.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.089+0000 7febe3400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7febc8066f60 0x7febc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.089+0000 7febe3400640 1 -- 172.21.15.46:0/1774378627 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7febd40853c0 con 0x7febec0ce6f0 2024-04-25T08:47:06.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.089+0000 7febeb400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7febc8066f60 0x7febc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.089+0000 7febf596a000 1 Processor -- start 2024-04-25T08:47:06.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.089+0000 7febf596a000 1 -- start start 2024-04-25T08:47:06.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.089+0000 7febeb400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7febc8066f60 0x7febc8069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7febdc0ae000 tx=0x7febdc0b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:06.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040959d0 0x55c504095db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040962f0 0x55c50404bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50404c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c503e98ff0 con 0x55c50404c0a0 2024-04-25T08:47:06.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c503e21150 con 0x55c5040962f0 2024-04-25T08:47:06.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c503e95090 con 0x55c5040959d0 2024-04-25T08:47:06.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50404c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50404c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47870/0 (socket says 172.21.15.46:47870) 2024-04-25T08:47:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febebe00640 1 -- 172.21.15.46:0/140537846 learned_addr learned my addr 172.21.15.46:0/140537846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febeb400640 1 --2- 172.21.15.46:0/140537846 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040959d0 0x55c504095db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febe3e00640 1 --2- 172.21.15.46:0/140537846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040962f0 0x55c50404bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febebe00640 1 -- 172.21.15.46:0/140537846 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040959d0 msgr2=0x55c504095db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febebe00640 1 --2- 172.21.15.46:0/140537846 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040959d0 0x55c504095db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febebe00640 1 -- 172.21.15.46:0/140537846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040962f0 msgr2=0x55c50404bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febebe00640 1 --2- 172.21.15.46:0/140537846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040962f0 0x55c50404bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febebe00640 1 -- 172.21.15.46:0/140537846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c50402f0d0 con 0x55c50404c0a0 2024-04-25T08:47:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febebe00640 1 --2- 172.21.15.46:0/140537846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50404c480 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7febe4001d20 tx=0x7febe40021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28980c890a5e678a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febdbe00640 1 -- 172.21.15.46:0/140537846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4019070 con 0x55c50404c0a0 2024-04-25T08:47:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febdbe00640 1 -- 172.21.15.46:0/140537846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe4003b90 con 0x55c50404c0a0 2024-04-25T08:47:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febdbe00640 1 -- 172.21.15.46:0/140537846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe400e920 con 0x55c50404c0a0 2024-04-25T08:47:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 -- 172.21.15.46:0/140537846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 msgr2=0x55c50404c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 --2- 172.21.15.46:0/140537846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50404c480 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7febe4001d20 tx=0x7febe40021e0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 -- 172.21.15.46:0/140537846 shutdown_connections 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 --2- 172.21.15.46:0/140537846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50404c480 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 --2- 172.21.15.46:0/140537846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040962f0 0x55c50404bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 --2- 172.21.15.46:0/140537846 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040959d0 0x55c504095db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 -- 172.21.15.46:0/140537846 >> 172.21.15.46:0/140537846 conn(0x55c504048020 msgr2=0x55c50404a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 -- 172.21.15.46:0/140537846 shutdown_connections 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 -- 172.21.15.46:0/140537846 wait complete. 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 -- start start 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.093+0000 7febf596a000 1 Processor -- start 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50407a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c50407ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 0x55c50407b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c503e5d250 con 0x55c50404c0a0 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c503e53690 con 0x55c5040959d0 2024-04-25T08:47:06.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c503e21a40 con 0x55c5040962f0 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 10 client.0 _ll_register_callbacks cb 0x55c504036e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50407a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50407a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47878/0 (socket says 172.21.15.46:47878) 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febeb400640 1 -- 172.21.15.46:0/1932182105 learned_addr learned my addr 172.21.15.46:0/1932182105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febebe00640 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 0x55c50407b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febe3e00640 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c50407ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febeb400640 1 -- 172.21.15.46:0/1932182105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 msgr2=0x55c50407b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febeb400640 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 0x55c50407b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febeb400640 1 -- 172.21.15.46:0/1932182105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 msgr2=0x55c50407ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febeb400640 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c50407ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febeb400640 1 -- 172.21.15.46:0/1932182105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c503f461c0 con 0x55c50404c0a0 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febeb400640 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50407a7f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7febdc0ab040 tx=0x7febdc01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21c40c2f1514541 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:06.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febd8c00640 1 -- 172.21.15.46:0/1932182105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0bcd80 con 0x55c50404c0a0 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febd8c00640 1 -- 172.21.15.46:0/1932182105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7febdc0c8440 con 0x55c50404c0a0 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 -- 172.21.15.46:0/1932182105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febe4003b90 con 0x55c50404c0a0 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febd8c00640 1 -- 172.21.15.46:0/1932182105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7febdc0d5050 con 0x55c50404c0a0 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 10 client.7119 fetch_fsmap learned FSMap version 51 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 -- 172.21.15.46:0/1932182105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55c503f42170 con 0x55c50404c0a0 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febd8c00640 1 -- 172.21.15.46:0/1932182105 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe4003b90 con 0x55c50404c0a0 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febd8c00640 1 -- 172.21.15.46:0/1932182105 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7febdc0d8020 con 0x55c50404c0a0 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 10 client.7119 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 10 client.7119 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 20 client.7119 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55c5040a5c80 0x55c5040a8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 4 client.7119 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:47:06.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 -- 172.21.15.46:0/1932182105 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c503e34c60 con 0x55c5040a5c80 2024-04-25T08:47:06.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febe3e00640 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55c5040a5c80 0x55c5040a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febe3e00640 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55c5040a5c80 0x55c5040a8070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188959cfe0a9c0f2 server_cookie=7543257b4199cf6 in_seq=0 out_seq=0 2024-04-25T08:47:06.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febd8c00640 10 client.7119 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febd8c00640 1 -- 172.21.15.46:0/1932182105 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c503e34c60 con 0x55c5040a5c80 2024-04-25T08:47:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febd8c00640 10 client.7119 handle_command_reply: tid=0 2024-04-25T08:47:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.097+0000 7febf596a000 1 client.7119 shutdown 2024-04-25T08:47:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 -- 172.21.15.46:0/1932182105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 msgr2=0x55c50407a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50407a7f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7febdc0ab040 tx=0x7febdc01ada0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 -- 172.21.15.46:0/1932182105 shutdown_connections 2024-04-25T08:47:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55c5040a5c80 0x55c5040a8070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 0x55c50407b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c50407ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 --2- 172.21.15.46:0/1932182105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c50407a7f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febd8c00640 0 client.7119 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 -- 172.21.15.46:0/1932182105 >> 172.21.15.46:0/1932182105 conn(0x55c504048020 msgr2=0x55c50404aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 -- 172.21.15.46:0/1932182105 shutdown_connections 2024-04-25T08:47:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 1 -- 172.21.15.46:0/1932182105 wait complete. 2024-04-25T08:47:06.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.101+0000 7febf596a000 20 client.7119 trim_cache size 0 max 16384 2024-04-25T08:47:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.113+0000 7febf596a000 1 Processor -- start 2024-04-25T08:47:06.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.113+0000 7febf596a000 1 -- start start 2024-04-25T08:47:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c5040f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 0x55c5040f2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec203950 con 0x55c50404c0a0 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec202af0 con 0x55c5040959d0 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febec13c040 con 0x55c5040962f0 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c5040f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c5040f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47892/0 (socket says 172.21.15.46:47892) 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febeb400640 1 -- 172.21.15.46:0/3599921285 learned_addr learned my addr 172.21.15.46:0/3599921285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febebe00640 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 0x55c5040f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febe3e00640 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febeb400640 1 -- 172.21.15.46:0/3599921285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 msgr2=0x55c5040f2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febeb400640 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 0x55c5040f2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febeb400640 1 -- 172.21.15.46:0/3599921285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 msgr2=0x55c5040f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febeb400640 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febeb400640 1 -- 172.21.15.46:0/3599921285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c503edfde0 con 0x55c50404c0a0 2024-04-25T08:47:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febe3e00640 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febeb400640 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c5040f1ee0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7febdc01ac10 tx=0x7febdc0be070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e5c964407b0c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febdb400640 1 -- 172.21.15.46:0/3599921285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0bcb80 con 0x55c50404c0a0 2024-04-25T08:47:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febdb400640 1 -- 172.21.15.46:0/3599921285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc0cd070 con 0x55c50404c0a0 2024-04-25T08:47:06.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febdb400640 1 -- 172.21.15.46:0/3599921285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0c8760 con 0x55c50404c0a0 2024-04-25T08:47:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 -- 172.21.15.46:0/3599921285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 msgr2=0x55c5040f1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c5040f1ee0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7febdc01ac10 tx=0x7febdc0be070 comp rx=0 tx=0).stop 2024-04-25T08:47:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 -- 172.21.15.46:0/3599921285 shutdown_connections 2024-04-25T08:47:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c5040962f0 0x55c5040f2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 --2- 172.21.15.46:0/3599921285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c50404c0a0 0x55c5040f1ee0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 -- 172.21.15.46:0/3599921285 >> 172.21.15.46:0/3599921285 conn(0x55c504048020 msgr2=0x55c5040a2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 -- 172.21.15.46:0/3599921285 shutdown_connections 2024-04-25T08:47:06.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 -- 172.21.15.46:0/3599921285 wait complete. 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 -- start start 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.117+0000 7febf596a000 1 Processor -- start 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c50404c0a0 0x55c5040a8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5040962f0 0x55c5040a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c503e33e00 con 0x55c5040962f0 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febec2024f0 con 0x55c5040959d0 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c503e98300 con 0x55c50404c0a0 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 10 client.0 _ll_register_callbacks cb 0x55c504017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5040962f0 0x55c5040a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5040962f0 0x55c5040a8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47902/0 (socket says 172.21.15.46:47902) 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febebe00640 1 -- 172.21.15.46:0/2448014937 learned_addr learned my addr 172.21.15.46:0/2448014937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febe3e00640 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febeb400640 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c50404c0a0 0x55c5040a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febebe00640 1 -- 172.21.15.46:0/2448014937 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c50404c0a0 msgr2=0x55c5040a8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febebe00640 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c50404c0a0 0x55c5040a8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febebe00640 1 -- 172.21.15.46:0/2448014937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 msgr2=0x55c5040a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febebe00640 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febebe00640 1 -- 172.21.15.46:0/2448014937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c503e3a8e0 con 0x55c5040962f0 2024-04-25T08:47:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febe3e00640 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febebe00640 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5040962f0 0x55c5040a8e00 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7febe4000f90 tx=0x7febe4001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95a4eb10a62c19a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:06.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:06.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 1 -- 172.21.15.46:0/2448014937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4019040 con 0x55c5040962f0 2024-04-25T08:47:06.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 1 -- 172.21.15.46:0/2448014937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7febe4006b60 con 0x55c5040962f0 2024-04-25T08:47:06.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- 172.21.15.46:0/2448014937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febdc0cd070 con 0x55c5040962f0 2024-04-25T08:47:06.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 1 -- 172.21.15.46:0/2448014937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe402f050 con 0x55c5040962f0 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 10 client.7125 fetch_fsmap learned FSMap version 51 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- 172.21.15.46:0/2448014937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55c503ef28f0 con 0x55c5040962f0 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 1 -- 172.21.15.46:0/2448014937 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc0cd070 con 0x55c5040962f0 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 1 -- 172.21.15.46:0/2448014937 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7febe4032020 con 0x55c5040962f0 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 10 client.7125 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 10 client.7125 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 20 client.7125 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55c5040a9a10 0x55c5040fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 4 client.7125 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- 172.21.15.46:0/2448014937 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c503ea6530 con 0x55c5040a9a10 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febeb400640 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55c5040a9a10 0x55c5040fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febeb400640 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55c5040a9a10 0x55c5040fe170 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e77c87c9a3a593a server_cookie=4aa3769d38f3f38a in_seq=0 out_seq=0 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 10 client.7125 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 1 -- 172.21.15.46:0/2448014937 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c503ea6530 con 0x55c5040a9a10 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 10 client.7125 handle_command_reply: tid=0 2024-04-25T08:47:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 client.7125 shutdown 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- 172.21.15.46:0/2448014937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5040962f0 msgr2=0x55c5040a8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5040962f0 0x55c5040a8e00 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7febe4000f90 tx=0x7febe4001260 comp rx=0 tx=0).stop 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- 172.21.15.46:0/2448014937 shutdown_connections 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55c5040a9a10 0x55c5040fe170 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5040962f0 0x55c5040a8e00 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5040959d0 0x55c5040a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 --2- 172.21.15.46:0/2448014937 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c50404c0a0 0x55c5040a8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febd8c00640 0 client.7125 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- 172.21.15.46:0/2448014937 >> 172.21.15.46:0/2448014937 conn(0x55c504048020 msgr2=0x55c5040940a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- 172.21.15.46:0/2448014937 shutdown_connections 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 1 -- 172.21.15.46:0/2448014937 wait complete. 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.121+0000 7febf596a000 20 client.7125 trim_cache size 0 max 16384 2024-04-25T08:47:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:47:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:47:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:47:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "d696032f-1064-4b71-bd03-d49dfbea588c": { 2024-04-25T08:47:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/subdir", 2024-04-25T08:47:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "d696032f-1064-4b71-bd03-d49dfbea588c", 2024-04-25T08:47:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-04-25T08:47:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:47:06.121 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:47:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:47:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 -- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7febc8066f60 msgr2=0x7febc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7febc8066f60 0x7febc8069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7febdc0ae000 tx=0x7febdc0b3040 comp rx=0 tx=0).stop 2024-04-25T08:47:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 -- 172.21.15.46:0/1774378627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 msgr2=0x7febec13ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 0x7febec13ed40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7febd400e8a0 tx=0x7febd400eca0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 -- 172.21.15.46:0/1774378627 shutdown_connections 2024-04-25T08:47:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7febc8066f60 0x7febc8069420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febec144b70 0x7febec15a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febec0ce6f0 0x7febec13ed40 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 --2- 172.21.15.46:0/1774378627 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7febec0cddf0 0x7febec15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 -- 172.21.15.46:0/1774378627 >> 172.21.15.46:0/1774378627 conn(0x7febec0b23a0 msgr2=0x7febec0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 -- 172.21.15.46:0/1774378627 shutdown_connections 2024-04-25T08:47:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:06.125+0000 7febf2400640 1 -- 172.21.15.46:0/1774378627 wait complete. 2024-04-25T08:47:06.136 INFO:tasks.cephfs.filesystem:scrub status for tag:d696032f-1064-4b71-bd03-d49dfbea588c - {'path': '/subdir', 'tag': 'd696032f-1064-4b71-bd03-d49dfbea588c', 'options': 'recursive'} 2024-04-25T08:47:07.930 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:47:07.937+0000 7fccb4a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:47:20.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:47:20.646 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:47:36.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:36.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:47:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c9000640 1 Processor -- start 2024-04-25T08:47:36.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c9000640 1 -- start start 2024-04-25T08:47:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c9000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc3c409a110 0x7fc3c40b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c40cce10 con 0x7fc3c409a740 2024-04-25T08:47:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c9000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c40cd0f0 con 0x7fc3c415efb0 2024-04-25T08:47:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c9000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c415f390 con 0x7fc3c409a110 2024-04-25T08:47:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc3c409a740 0x7fc3c40cf690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41348/0 (socket says 172.21.15.46:41348) 2024-04-25T08:47:36.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c2800640 1 -- 172.21.15.46:0/1371608445 learned_addr learned my addr 172.21.15.46:0/1371608445 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:47:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/1371608445 >> v2:172.21.15.82:3300/0 conn(0x7fc3c409a110 0x7fc3c40b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993940672 0 0) 0x7fc3c40cce10 con 0x7fc3c409a740 2024-04-25T08:47:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3a8003530 con 0x7fc3c409a740 2024-04-25T08:47:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2221637833 0 0) 0x7fc3a8003530 con 0x7fc3c409a740 2024-04-25T08:47:36.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 >> v2:172.21.15.82:3300/0 conn(0x7fc3c409a110 msgr2=0x7fc3c40b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 --2- 172.21.15.46:0/1371608445 >> v2:172.21.15.82:3300/0 conn(0x7fc3c409a110 0x7fc3c40b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 >> v1:172.21.15.82:6789/0 conn(0x7fc3c415efb0 legacy=0x7fc3c4099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40cd3d0 con 0x7fc3c409a740 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fc3bc001a00 con 0x7fc3c409a740 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 >> v1:172.21.15.46:6789/0 conn(0x7fc3c409a740 legacy=0x7fc3c40cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3a8003a80 0x7fc3a8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3a8004810 0x7fc3a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c415f390 con 0x7fc3a80052f0 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a800c1c0 con 0x7fc3a8004810 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a800c3c0 con 0x7fc3a8003a80 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c2800640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3a8004810 0x7fc3a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.301+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3a8003a80 0x7fc3a8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c2800640 1 -- 172.21.15.46:0/1371608445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3a8003a80 msgr2=0x7fc3a8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c2800640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3a8003a80 0x7fc3a8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c2800640 1 -- 172.21.15.46:0/1371608445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3a8004810 msgr2=0x7fc3a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c2800640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3a8004810 0x7fc3a8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c2800640 1 -- 172.21.15.46:0/1371608445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a8003530 con 0x7fc3a80052f0 2024-04-25T08:47:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c2800640 1 -- 172.21.15.46:0/1371608445 reap_dead start 2024-04-25T08:47:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c2800640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3a800b9d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00a7b0 tx=0x7fc3bc01fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3862a435468b951a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc00a3d0 con 0x7fc3a80052f0 2024-04-25T08:47:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3bc0aa470 con 0x7fc3a80052f0 2024-04-25T08:47:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c1400640 1 -- 172.21.15.46:0/1371608445 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0aa880 con 0x7fc3a80052f0 2024-04-25T08:47:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c9000640 1 -- 172.21.15.46:0/1371608445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 msgr2=0x7fc3a800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c9000640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3a800b9d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00a7b0 tx=0x7fc3bc01fa40 comp rx=0 tx=0).stop 2024-04-25T08:47:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c9000640 1 -- 172.21.15.46:0/1371608445 shutdown_connections 2024-04-25T08:47:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c9000640 1 --2- 172.21.15.46:0/1371608445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3a800b9d0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c9000640 1 -- 172.21.15.46:0/1371608445 >> 172.21.15.46:0/1371608445 conn(0x7fc3c40b3120 msgr2=0x7fc3c40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c9000640 1 -- 172.21.15.46:0/1371608445 shutdown_connections 2024-04-25T08:47:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.305+0000 7fc3c9000640 1 -- 172.21.15.46:0/1371608445 wait complete. 2024-04-25T08:47:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 Processor -- start 2024-04-25T08:47:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 -- start start 2024-04-25T08:47:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3c415dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3c406fec0 0x7fc3c41518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c4070340 0x7fc3c4151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40c5180 con 0x7fc3a80052f0 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40cf200 con 0x7fc3c4070340 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40c5300 con 0x7fc3c406fec0 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3c415dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3c415dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50240/0 (socket says 172.21.15.46:50240) 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c1e00640 1 -- 172.21.15.46:0/1450783441 learned_addr learned my addr 172.21.15.46:0/1450783441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c2800640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c4070340 0x7fc3c4151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3c406fec0 0x7fc3c41518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c1e00640 1 -- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3c406fec0 msgr2=0x7fc3c41518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3c406fec0 0x7fc3c41518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c1e00640 1 -- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c4070340 msgr2=0x7fc3c4151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c4070340 0x7fc3c4151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c1e00640 1 -- 172.21.15.46:0/1450783441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c40ca970 con 0x7fc3a80052f0 2024-04-25T08:47:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3c415dd40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400f7f0 tx=0x7fc3b400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3ba000640 1 -- 172.21.15.46:0/1450783441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40050d0 con 0x7fc3a80052f0 2024-04-25T08:47:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3ba000640 1 -- 172.21.15.46:0/1450783441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b4018070 con 0x7fc3a80052f0 2024-04-25T08:47:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3ba000640 1 -- 172.21.15.46:0/1450783441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4005490 con 0x7fc3a80052f0 2024-04-25T08:47:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 -- 172.21.15.46:0/1450783441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c40cac50 con 0x7fc3a80052f0 2024-04-25T08:47:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.309+0000 7fc3c9000640 1 -- 172.21.15.46:0/1450783441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c40cb210 con 0x7fc3a80052f0 2024-04-25T08:47:36.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.313+0000 7fc3ba000640 1 -- 172.21.15.46:0/1450783441 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3b400d730 con 0x7fc3a80052f0 2024-04-25T08:47:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.313+0000 7fc3ba000640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3a0067100 0x7fc3a00695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.313+0000 7fc3ba000640 1 -- 172.21.15.46:0/1450783441 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc3b4084ff0 con 0x7fc3a80052f0 2024-04-25T08:47:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.313+0000 7fc3cc553000 1 Processor -- start 2024-04-25T08:47:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.313+0000 7fc3cc553000 1 -- start start 2024-04-25T08:47:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.313+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3a0067100 0x7fc3a00695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.313+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec2f9d0 0x55632ec2fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec302f0 0x55632ebe5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ebe6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55632ea32ff0 con 0x55632ec302f0 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55632e9bb150 con 0x55632ebe60a0 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55632ea2f090 con 0x55632ec2f9d0 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3a0067100 0x7fc3a00695c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4159660 tx=0x7fc3ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec302f0 0x55632ebe5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec302f0 0x55632ebe5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50248/0 (socket says 172.21.15.46:50248) 2024-04-25T08:47:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 -- 172.21.15.46:0/2841882668 learned_addr learned my addr 172.21.15.46:0/2841882668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec2f9d0 0x55632ec2fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3c2800640 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ebe6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 -- 172.21.15.46:0/2841882668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec2f9d0 msgr2=0x55632ec2fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec2f9d0 0x55632ec2fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 -- 172.21.15.46:0/2841882668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 msgr2=0x55632ebe6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ebe6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 -- 172.21.15.46:0/2841882668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55632ebc90d0 con 0x55632ec302f0 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3c2800640 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ebe6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec302f0 0x55632ebe5b60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac014d20 tx=0x7fc3ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8a738fed2beedb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3b2a00640 1 -- 172.21.15.46:0/2841882668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac027070 con 0x55632ec302f0 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3b2a00640 1 -- 172.21.15.46:0/2841882668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac023450 con 0x55632ec302f0 2024-04-25T08:47:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3b2a00640 1 -- 172.21.15.46:0/2841882668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac023820 con 0x55632ec302f0 2024-04-25T08:47:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- 172.21.15.46:0/2841882668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec302f0 msgr2=0x55632ebe5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec302f0 0x55632ebe5b60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac014d20 tx=0x7fc3ac017000 comp rx=0 tx=0).stop 2024-04-25T08:47:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- 172.21.15.46:0/2841882668 shutdown_connections 2024-04-25T08:47:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ebe6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec302f0 0x55632ebe5b60 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2841882668 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec2f9d0 0x55632ec2fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- 172.21.15.46:0/2841882668 >> 172.21.15.46:0/2841882668 conn(0x55632ebe2020 msgr2=0x55632ebe4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- 172.21.15.46:0/2841882668 shutdown_connections 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- 172.21.15.46:0/2841882668 wait complete. 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- start start 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 Processor -- start 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec147f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec14d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec15270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55632e9f7250 con 0x55632ec2f9d0 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55632e9ed690 con 0x55632ebe60a0 2024-04-25T08:47:36.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55632e9bba40 con 0x55632ec302f0 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.317+0000 7fc3cc553000 10 client.0 _ll_register_callbacks cb 0x55632ebd0e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec14d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec147f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec14d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50260/0 (socket says 172.21.15.46:50260) 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3bbe00640 1 -- 172.21.15.46:0/989965721 learned_addr learned my addr 172.21.15.46:0/989965721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3c2800640 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec15270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3bbe00640 1 -- 172.21.15.46:0/989965721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 msgr2=0x55632ec15270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec15270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3bbe00640 1 -- 172.21.15.46:0/989965721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 msgr2=0x55632ec147f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec147f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3bbe00640 1 -- 172.21.15.46:0/989965721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55632eae01c0 con 0x55632ec2f9d0 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec14d30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac014720 tx=0x7fc3ac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d486a19854e183 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 1 -- 172.21.15.46:0/989965721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac027070 con 0x55632ec2f9d0 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 1 -- 172.21.15.46:0/989965721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc3ac036cd0 con 0x55632ec2f9d0 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 -- 172.21.15.46:0/989965721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3ac023450 con 0x55632ec2f9d0 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 1 -- 172.21.15.46:0/989965721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3ac023e50 con 0x55632ec2f9d0 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 10 client.7137 fetch_fsmap learned FSMap version 52 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 -- 172.21.15.46:0/989965721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55632eadc170 con 0x55632ec2f9d0 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 1 -- 172.21.15.46:0/989965721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac023450 con 0x55632ec2f9d0 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 1 -- 172.21.15.46:0/989965721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc3ac033020 con 0x55632ec2f9d0 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 10 client.7137 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 10 client.7137 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 20 client.7137 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55632ec3fc80 0x55632ec42070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 4 client.7137 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 -- 172.21.15.46:0/989965721 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55632e9cec60 con 0x55632ec3fc80 2024-04-25T08:47:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55632ec3fc80 0x55632ec42070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55632ec3fc80 0x55632ec42070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5530743feb7fb87b server_cookie=fc511434eccacb45 in_seq=0 out_seq=0 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 10 client.7137 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 1 -- 172.21.15.46:0/989965721 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55632e9cec60 con 0x55632ec3fc80 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 10 client.7137 handle_command_reply: tid=0 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 client.7137 shutdown 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 -- 172.21.15.46:0/989965721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 msgr2=0x55632ec14d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec14d30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac014720 tx=0x7fc3ac004740 comp rx=0 tx=0).stop 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 -- 172.21.15.46:0/989965721 shutdown_connections 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55632ec3fc80 0x55632ec42070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec15270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec14d30 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 --2- 172.21.15.46:0/989965721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec147f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 -- 172.21.15.46:0/989965721 >> 172.21.15.46:0/989965721 conn(0x55632ebe2020 msgr2=0x55632ebe4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3a7400640 0 client.7137 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:36.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 -- 172.21.15.46:0/989965721 shutdown_connections 2024-04-25T08:47:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 1 -- 172.21.15.46:0/989965721 wait complete. 2024-04-25T08:47:36.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.321+0000 7fc3cc553000 20 client.7137 trim_cache size 0 max 16384 2024-04-25T08:47:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3cc553000 1 Processor -- start 2024-04-25T08:47:36.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3cc553000 1 -- start start 2024-04-25T08:47:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ebe60a0 0x55632ec8c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ec2f9d0 0x55632ec8c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec8cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3cc553000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c42009f0 con 0x55632ebe60a0 2024-04-25T08:47:36.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3cc553000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c41ffbf0 con 0x55632ec2f9d0 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3cc553000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c41ff020 con 0x55632ec302f0 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ebe60a0 0x55632ec8c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ebe60a0 0x55632ec8c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50276/0 (socket says 172.21.15.46:50276) 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3c1e00640 1 -- 172.21.15.46:0/668785260 learned_addr learned my addr 172.21.15.46:0/668785260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/668785260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ec2f9d0 0x55632ec8c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.337+0000 7fc3c2800640 1 --2- 172.21.15.46:0/668785260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec8cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3c1e00640 1 -- 172.21.15.46:0/668785260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 msgr2=0x55632ec8cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/668785260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec8cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3c1e00640 1 -- 172.21.15.46:0/668785260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ec2f9d0 msgr2=0x55632ec8c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/668785260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ec2f9d0 0x55632ec8c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3c1e00640 1 -- 172.21.15.46:0/668785260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55632ea79de0 con 0x55632ebe60a0 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/668785260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ebe60a0 0x55632ec8c0f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40050a0 tx=0x7fc3b4051b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950e30dad91c94be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3b2000640 1 -- 172.21.15.46:0/668785260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4080070 con 0x55632ebe60a0 2024-04-25T08:47:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3b2000640 1 -- 172.21.15.46:0/668785260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b40598c0 con 0x55632ebe60a0 2024-04-25T08:47:36.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3b2000640 1 -- 172.21.15.46:0/668785260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b405de80 con 0x55632ebe60a0 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 -- 172.21.15.46:0/668785260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ebe60a0 msgr2=0x55632ec8c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 --2- 172.21.15.46:0/668785260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ebe60a0 0x55632ec8c0f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40050a0 tx=0x7fc3b4051b40 comp rx=0 tx=0).stop 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 -- 172.21.15.46:0/668785260 shutdown_connections 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 --2- 172.21.15.46:0/668785260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec8cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 --2- 172.21.15.46:0/668785260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ec2f9d0 0x55632ec8c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 --2- 172.21.15.46:0/668785260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ebe60a0 0x55632ec8c0f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 -- 172.21.15.46:0/668785260 >> 172.21.15.46:0/668785260 conn(0x55632ebe2020 msgr2=0x55632ec3cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 -- 172.21.15.46:0/668785260 shutdown_connections 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 -- 172.21.15.46:0/668785260 wait complete. 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 -- start start 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 Processor -- start 2024-04-25T08:47:36.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec42380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec42e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55632e9cde00 con 0x55632ec2f9d0 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c41ff5f0 con 0x55632ebe60a0 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55632ea32300 con 0x55632ec302f0 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3cc553000 10 client.0 _ll_register_callbacks cb 0x55632ebb1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50286/0 (socket says 172.21.15.46:50286) 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3bbe00640 1 -- 172.21.15.46:0/2375082021 learned_addr learned my addr 172.21.15.46:0/2375082021 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec42380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3c2800640 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec42e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3bbe00640 1 -- 172.21.15.46:0/2375082021 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 msgr2=0x55632ec42e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec42e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3bbe00640 1 -- 172.21.15.46:0/2375082021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 msgr2=0x55632ec42380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec42380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3bbe00640 1 -- 172.21.15.46:0/2375082021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55632e9d48e0 con 0x55632ec2f9d0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec42380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3bbe00640 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec428c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac006a50 tx=0x7fc3ac01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f9bc24e37ab056 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.341+0000 7fc3a7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3a7400640 1 -- 172.21.15.46:0/2375082021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac043070 con 0x55632ec2f9d0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3a7400640 1 -- 172.21.15.46:0/2375082021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc3ac035650 con 0x55632ec2f9d0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 -- 172.21.15.46:0/2375082021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3b40598c0 con 0x55632ec2f9d0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3a7400640 1 -- 172.21.15.46:0/2375082021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3ac050050 con 0x55632ec2f9d0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 10 client.7143 fetch_fsmap learned FSMap version 52 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 -- 172.21.15.46:0/2375082021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55632ea8c8f0 con 0x55632ec2f9d0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3a7400640 1 -- 172.21.15.46:0/2375082021 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b40598c0 con 0x55632ec2f9d0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3a7400640 1 -- 172.21.15.46:0/2375082021 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc3ac033040 con 0x55632ec2f9d0 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 10 client.7143 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 10 client.7143 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 20 client.7143 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55632ec437f0 0x55632ec97f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 4 client.7143 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:47:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 -- 172.21.15.46:0/2375082021 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55632ea40530 con 0x55632ec437f0 2024-04-25T08:47:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55632ec437f0 0x55632ec97f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c1e00640 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55632ec437f0 0x55632ec97f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48dbcef18a9a011 server_cookie=7ec9d363ed851134 in_seq=0 out_seq=0 2024-04-25T08:47:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3a7400640 10 client.7143 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3a7400640 1 -- 172.21.15.46:0/2375082021 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55632ea40530 con 0x55632ec437f0 2024-04-25T08:47:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3a7400640 10 client.7143 handle_command_reply: tid=0 2024-04-25T08:47:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 client.7143 shutdown 2024-04-25T08:47:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 -- 172.21.15.46:0/2375082021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 msgr2=0x55632ec428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec428c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac006a50 tx=0x7fc3ac01a000 comp rx=0 tx=0).stop 2024-04-25T08:47:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 -- 172.21.15.46:0/2375082021 shutdown_connections 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55632ec437f0 0x55632ec97f60 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55632ec302f0 0x55632ec42e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55632ec2f9d0 0x55632ec428c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 --2- 172.21.15.46:0/2375082021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55632ebe60a0 0x55632ec42380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3a7400640 0 client.7143 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 -- 172.21.15.46:0/2375082021 >> 172.21.15.46:0/2375082021 conn(0x55632ebe2020 msgr2=0x55632ec2e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 -- 172.21.15.46:0/2375082021 shutdown_connections 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 1 -- 172.21.15.46:0/2375082021 wait complete. 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3cc553000 20 client.7143 trim_cache size 0 max 16384 2024-04-25T08:47:36.343 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:47:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:47:36.344 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:47:36.344 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:47:36.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 -- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3a0067100 msgr2=0x7fc3a00695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3a0067100 0x7fc3a00695c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4159660 tx=0x7fc3ac008040 comp rx=0 tx=0).stop 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 -- 172.21.15.46:0/1450783441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 msgr2=0x7fc3c415dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3c415dd40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400f7f0 tx=0x7fc3b400fcb0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 -- 172.21.15.46:0/1450783441 shutdown_connections 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3a0067100 0x7fc3a00695c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3c4070340 0x7fc3c4151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3c406fec0 0x7fc3c41518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 --2- 172.21.15.46:0/1450783441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3a80052f0 0x7fc3c415dd40 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 -- 172.21.15.46:0/1450783441 >> 172.21.15.46:0/1450783441 conn(0x7fc3c40b3120 msgr2=0x7fc3c40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 -- 172.21.15.46:0/1450783441 shutdown_connections 2024-04-25T08:47:36.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.345+0000 7fc3c9000640 1 -- 172.21.15.46:0/1450783441 wait complete. 2024-04-25T08:47:36.359 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:47:36.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:47:36.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.505+0000 7f4b0da00640 1 Processor -- start 2024-04-25T08:47:36.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.505+0000 7f4b0da00640 1 -- start start 2024-04-25T08:47:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4b0da00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b080ad4c0 0x7f4b080af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4b0da00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0815fed0 con 0x7f4b0815b9a0 2024-04-25T08:47:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4b0da00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b081601b0 con 0x7f4b0806f570 2024-04-25T08:47:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4b0da00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0815c0f0 con 0x7f4b080ad4c0 2024-04-25T08:47:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4b06a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4b0815b9a0 0x7f4b080acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41354/0 (socket says 172.21.15.46:41354) 2024-04-25T08:47:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4b06a00640 1 -- 172.21.15.46:0/3543020328 learned_addr learned my addr 172.21.15.46:0/3543020328 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:47:36.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4afe000640 1 --2- 172.21.15.46:0/3543020328 >> v2:172.21.15.82:3300/0 conn(0x7f4b080ad4c0 0x7f4b080af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4b06000640 1 -- 172.21.15.46:0/3543020328 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380464960 0 0) 0x7f4b0815fed0 con 0x7f4b0815b9a0 2024-04-25T08:47:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4b06000640 1 -- 172.21.15.46:0/3543020328 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aec003530 con 0x7f4b0815b9a0 2024-04-25T08:47:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4afe000640 1 -- 172.21.15.46:0/3543020328 >> v1:172.21.15.82:6789/0 conn(0x7f4b0806f570 legacy=0x7f4b0806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4afe000640 1 -- 172.21.15.46:0/3543020328 >> v1:172.21.15.46:6789/0 conn(0x7f4b0815b9a0 legacy=0x7f4b080acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.509+0000 7f4afe000640 1 -- 172.21.15.46:0/3543020328 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0815c3d0 con 0x7f4b080ad4c0 2024-04-25T08:47:36.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4afe000640 1 --2- 172.21.15.46:0/3543020328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b080ad4c0 0x7f4b080af030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097c0 tx=0x7f4af001c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc442a21ed684c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b06000640 1 -- 172.21.15.46:0/3543020328 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af00b0070 con 0x7f4b080ad4c0 2024-04-25T08:47:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b06000640 1 -- 172.21.15.46:0/3543020328 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af00ab600 con 0x7f4b080ad4c0 2024-04-25T08:47:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b06000640 1 -- 172.21.15.46:0/3543020328 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af00ab9d0 con 0x7f4b080ad4c0 2024-04-25T08:47:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b0da00640 1 -- 172.21.15.46:0/3543020328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b080ad4c0 msgr2=0x7f4b080af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b0da00640 1 --2- 172.21.15.46:0/3543020328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b080ad4c0 0x7f4b080af030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4af00097c0 tx=0x7f4af001c350 comp rx=0 tx=0).stop 2024-04-25T08:47:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b0da00640 1 -- 172.21.15.46:0/3543020328 shutdown_connections 2024-04-25T08:47:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b0da00640 1 --2- 172.21.15.46:0/3543020328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b080ad4c0 0x7f4b080af030 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b0da00640 1 -- 172.21.15.46:0/3543020328 >> 172.21.15.46:0/3543020328 conn(0x7f4b080cfd90 msgr2=0x7f4b080d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b0da00640 1 -- 172.21.15.46:0/3543020328 shutdown_connections 2024-04-25T08:47:36.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b0da00640 1 -- 172.21.15.46:0/3543020328 wait complete. 2024-04-25T08:47:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b0da00640 1 Processor -- start 2024-04-25T08:47:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.513+0000 7f4b0da00640 1 -- start start 2024-04-25T08:47:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b0da00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b0806f570 0x7f4b081549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b0da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b080ad4c0 0x7f4b080b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b0da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b0815b9a0 0x7f4b080b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b0da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0806b8b0 con 0x7f4b0815b9a0 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b0da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0806b730 con 0x7f4b080ad4c0 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b0da00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08169f50 con 0x7f4b0806f570 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b0815b9a0 0x7f4b080b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b07400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b0815b9a0 0x7f4b080b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50300/0 (socket says 172.21.15.46:50300) 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b07400640 1 -- 172.21.15.46:0/2609571329 learned_addr learned my addr 172.21.15.46:0/2609571329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4afe000640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b080ad4c0 0x7f4b080b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b06a00640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b0806f570 0x7f4b081549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b07400640 1 -- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b0806f570 msgr2=0x7f4b081549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b07400640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b0806f570 0x7f4b081549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b07400640 1 -- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b080ad4c0 msgr2=0x7f4b080b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b07400640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b080ad4c0 0x7f4b080b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b07400640 1 -- 172.21.15.46:0/2609571329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b081566b0 con 0x7f4b0815b9a0 2024-04-25T08:47:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b07400640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b0815b9a0 0x7f4b080b4910 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000c3c0 tx=0x7f4b0000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4aff400640 1 -- 172.21.15.46:0/2609571329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00017070 con 0x7f4b0815b9a0 2024-04-25T08:47:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4aff400640 1 -- 172.21.15.46:0/2609571329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b000052d0 con 0x7f4b0815b9a0 2024-04-25T08:47:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4aff400640 1 -- 172.21.15.46:0/2609571329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00003840 con 0x7f4b0815b9a0 2024-04-25T08:47:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b08156990 con 0x7f4b0815b9a0 2024-04-25T08:47:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b08156f50 con 0x7f4b0815b9a0 2024-04-25T08:47:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08066c40 con 0x7f4b0815b9a0 2024-04-25T08:47:36.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.517+0000 7f4aff400640 1 -- 172.21.15.46:0/2609571329 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b0001a070 con 0x7f4b0815b9a0 2024-04-25T08:47:36.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.521+0000 7f4aff400640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4ae4066f60 0x7f4ae4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.521+0000 7f4aff400640 1 -- 172.21.15.46:0/2609571329 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4b00084350 con 0x7f4b0815b9a0 2024-04-25T08:47:36.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.521+0000 7f4b06a00640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4ae4066f60 0x7f4ae4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.521+0000 7f4b06a00640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4ae4066f60 0x7f4ae4069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4af8001570 tx=0x7f4af8001aa0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.525+0000 7f4aff400640 1 -- 172.21.15.46:0/2609571329 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b00005480 con 0x7f4b0815b9a0 2024-04-25T08:47:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.721+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b081f6f60 con 0x7f4b0815b9a0 2024-04-25T08:47:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.721+0000 7f4aff400640 1 -- 172.21.15.46:0/2609571329 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f4b00054b20 con 0x7f4b0815b9a0 2024-04-25T08:47:36.715 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:47:36.715 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:47:07.910312+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6835/4112568749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":4112568749},{"type":"v1","addr":"172.21.15.46:6835","nonce":4112568749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:47:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 52 2024-04-25T08:47:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4ae4066f60 msgr2=0x7f4ae4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4ae4066f60 0x7f4ae4069420 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4af8001570 tx=0x7f4af8001aa0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b0815b9a0 msgr2=0x7f4b080b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b0815b9a0 0x7f4b080b4910 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4b0000c3c0 tx=0x7f4b0000c880 comp rx=0 tx=0).stop 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 shutdown_connections 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4ae4066f60 0x7f4ae4069420 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b0815b9a0 0x7f4b080b4910 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b080ad4c0 0x7f4b080b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 --2- 172.21.15.46:0/2609571329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b0806f570 0x7f4b081549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 >> 172.21.15.46:0/2609571329 conn(0x7f4b080cfd90 msgr2=0x7f4b081592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 shutdown_connections 2024-04-25T08:47:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.725+0000 7f4b0da00640 1 -- 172.21.15.46:0/2609571329 wait complete. 2024-04-25T08:47:36.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:47:36.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c37e00640 1 Processor -- start 2024-04-25T08:47:36.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c37e00640 1 -- start start 2024-04-25T08:47:36.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c37e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2c38144e40 0x7f2c3815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c37e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380710b0 0x7f2c38071540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c37e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c380ccee0 con 0x7f2c380cd130 2024-04-25T08:47:36.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c37e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c380cc340 con 0x7f2c38144e40 2024-04-25T08:47:36.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c37e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380cde20 con 0x7f2c380710b0 2024-04-25T08:47:36.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c36a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2c38144e40 0x7f2c3815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c2fe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2c380cd130 0x7f2c380cc7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:50220/0 (socket says 172.21.15.46:50220) 2024-04-25T08:47:36.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c2fe00640 1 -- 172.21.15.46:0/1044025263 learned_addr learned my addr 172.21.15.46:0/1044025263 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:47:36.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.885+0000 7f2c37400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380710b0 0x7f2c38071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.889+0000 7f2c36a00640 1 -- 172.21.15.46:0/1044025263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380710b0 msgr2=0x7f2c38071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.889+0000 7f2c36a00640 1 --2- 172.21.15.46:0/1044025263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380710b0 0x7f2c38071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.889+0000 7f2c36a00640 1 -- 172.21.15.46:0/1044025263 >> v1:172.21.15.82:6789/0 conn(0x7f2c380cd130 legacy=0x7f2c380cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.889+0000 7f2c36a00640 1 -- 172.21.15.46:0/1044025263 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c380cd5c0 con 0x7f2c38144e40 2024-04-25T08:47:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.889+0000 7f2c36a00640 1 --2- 172.21.15.46:0/1044025263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c38144e40 0x7f2c3815fb50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2c280097c0 tx=0x7f2c2801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71a4f47bc2cbdca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.889+0000 7f2c36000640 1 -- 172.21.15.46:0/1044025263 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280b1070 con 0x7f2c38144e40 2024-04-25T08:47:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.889+0000 7f2c36000640 1 -- 172.21.15.46:0/1044025263 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c280ac410 con 0x7f2c38144e40 2024-04-25T08:47:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.889+0000 7f2c36000640 1 -- 172.21.15.46:0/1044025263 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280ac7e0 con 0x7f2c38144e40 2024-04-25T08:47:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 -- 172.21.15.46:0/1044025263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c38144e40 msgr2=0x7f2c3815fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 --2- 172.21.15.46:0/1044025263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c38144e40 0x7f2c3815fb50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2c280097c0 tx=0x7f2c2801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 -- 172.21.15.46:0/1044025263 shutdown_connections 2024-04-25T08:47:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 --2- 172.21.15.46:0/1044025263 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380710b0 0x7f2c38071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 --2- 172.21.15.46:0/1044025263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c38144e40 0x7f2c3815fb50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 -- 172.21.15.46:0/1044025263 >> 172.21.15.46:0/1044025263 conn(0x7f2c38066fa0 msgr2=0x7f2c3806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 -- 172.21.15.46:0/1044025263 shutdown_connections 2024-04-25T08:47:36.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 -- 172.21.15.46:0/1044025263 wait complete. 2024-04-25T08:47:36.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 Processor -- start 2024-04-25T08:47:36.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 -- start start 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c380710b0 0x7f2c3813fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380cd130 0x7f2c3815cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c38144e40 0x7f2c3814d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3815c320 con 0x7f2c380710b0 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380cbfd0 con 0x7f2c38144e40 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3815c4a0 con 0x7f2c380cd130 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c380710b0 0x7f2c3813fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c380710b0 0x7f2c3813fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50304/0 (socket says 172.21.15.46:50304) 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c36a00640 1 -- 172.21.15.46:0/1396712180 learned_addr learned my addr 172.21.15.46:0/1396712180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380cd130 0x7f2c3815cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.893+0000 7f2c37400640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c38144e40 0x7f2c3814d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c36a00640 1 -- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380cd130 msgr2=0x7f2c3815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c36a00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380cd130 0x7f2c3815cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c36a00640 1 -- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c38144e40 msgr2=0x7f2c3814d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c36a00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c38144e40 0x7f2c3814d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c36a00640 1 -- 172.21.15.46:0/1396712180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3815fa70 con 0x7f2c380710b0 2024-04-25T08:47:36.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380cd130 0x7f2c3815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:47:36.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c36a00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c380710b0 0x7f2c3813fb40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2c2801a0d0 tx=0x7f2c28019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c2ea00640 1 -- 172.21.15.46:0/1396712180 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280b1040 con 0x7f2c380710b0 2024-04-25T08:47:36.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c2ea00640 1 -- 172.21.15.46:0/1396712180 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c280bb070 con 0x7f2c380710b0 2024-04-25T08:47:36.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c37e00640 1 -- 172.21.15.46:0/1396712180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c38140420 con 0x7f2c380710b0 2024-04-25T08:47:36.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c2ea00640 1 -- 172.21.15.46:0/1396712180 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280b5410 con 0x7f2c380710b0 2024-04-25T08:47:36.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c37e00640 1 -- 172.21.15.46:0/1396712180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c380b26d0 con 0x7f2c380710b0 2024-04-25T08:47:36.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c2ea00640 1 -- 172.21.15.46:0/1396712180 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c280b55b0 con 0x7f2c380710b0 2024-04-25T08:47:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c2ea00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c14066f60 0x7f2c14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.897+0000 7f2c2ea00640 1 -- 172.21.15.46:0/1396712180 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2c2811ec90 con 0x7f2c380710b0 2024-04-25T08:47:36.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 Processor -- start 2024-04-25T08:47:36.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c14066f60 0x7f2c14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- start start 2024-04-25T08:47:36.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c14066f60 0x7f2c14069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c3813f740 tx=0x7f2c20009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aeb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aebc00 0x55a5a8aa13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 0x55a5a8aa1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8988ba0 con 0x55a5a8aebc00 2024-04-25T08:47:36.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a88ee090 con 0x55a5a8aeb2e0 2024-04-25T08:47:36.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8876160 con 0x55a5a8aa18e0 2024-04-25T08:47:36.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aebc00 0x55a5a8aa13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c36a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aeb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aebc00 0x55a5a8aa13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50314/0 (socket says 172.21.15.46:50314) 2024-04-25T08:47:36.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c2fe00640 1 -- 172.21.15.46:0/63718924 learned_addr learned my addr 172.21.15.46:0/63718924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c37400640 1 --2- 172.21.15.46:0/63718924 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 0x55a5a8aa1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c36a00640 1 -- 172.21.15.46:0/63718924 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 msgr2=0x55a5a8aa1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c36a00640 1 --2- 172.21.15.46:0/63718924 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 0x55a5a8aa1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c36a00640 1 -- 172.21.15.46:0/63718924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aebc00 msgr2=0x55a5a8aa13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c36a00640 1 --2- 172.21.15.46:0/63718924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aebc00 0x55a5a8aa13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c36a00640 1 -- 172.21.15.46:0/63718924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5a8997c20 con 0x55a5a8aeb2e0 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/63718924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aebc00 0x55a5a8aa13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c36a00640 1 --2- 172.21.15.46:0/63718924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aeb6c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000c10 tx=0x7f2c280eef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90107c85147e6cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c27400640 1 -- 172.21.15.46:0/63718924 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280f3470 con 0x55a5a8aeb2e0 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c27400640 1 -- 172.21.15.46:0/63718924 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c280f7a30 con 0x55a5a8aeb2e0 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c27400640 1 -- 172.21.15.46:0/63718924 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280f3120 con 0x55a5a8aeb2e0 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- 172.21.15.46:0/63718924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aeb2e0 msgr2=0x55a5a8aeb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 --2- 172.21.15.46:0/63718924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aeb6c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000c10 tx=0x7f2c280eef20 comp rx=0 tx=0).stop 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- 172.21.15.46:0/63718924 shutdown_connections 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 --2- 172.21.15.46:0/63718924 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 0x55a5a8aa1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 --2- 172.21.15.46:0/63718924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aebc00 0x55a5a8aa13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 --2- 172.21.15.46:0/63718924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aeb6c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- 172.21.15.46:0/63718924 >> 172.21.15.46:0/63718924 conn(0x55a5a8a9d890 msgr2=0x55a5a8a9fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- 172.21.15.46:0/63718924 shutdown_connections 2024-04-25T08:47:36.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- 172.21.15.46:0/63718924 wait complete. 2024-04-25T08:47:36.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 -- start start 2024-04-25T08:47:36.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.901+0000 7f2c4112e000 1 Processor -- start 2024-04-25T08:47:36.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aa18e0 0x55a5a8aac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 0x55a5a8aac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aebc00 0x55a5a8aacf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a88ea130 con 0x55a5a8aa18e0 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a88b2130 con 0x55a5a8aebc00 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5a89ac300 con 0x55a5a8aeb2e0 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 10 client.0 _ll_register_callbacks cb 0x55a5a8a8cbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aa18e0 0x55a5a8aac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aa18e0 0x55a5a8aac480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50318/0 (socket says 172.21.15.46:50318) 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c36a00640 1 -- 172.21.15.46:0/2740343000 learned_addr learned my addr 172.21.15.46:0/2740343000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c37400640 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aebc00 0x55a5a8aacf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 0x55a5a8aac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c36a00640 1 -- 172.21.15.46:0/2740343000 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 msgr2=0x55a5a8aac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c36a00640 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 0x55a5a8aac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c36a00640 1 -- 172.21.15.46:0/2740343000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aebc00 msgr2=0x55a5a8aacf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c36a00640 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aebc00 0x55a5a8aacf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c36a00640 1 -- 172.21.15.46:0/2740343000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5a88cb060 con 0x55a5a8aa18e0 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c36a00640 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aa18e0 0x55a5a8aac480 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2c2801ca10 tx=0x7f2c2801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fabd9e2782d3b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c1be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c1be00640 1 -- 172.21.15.46:0/2740343000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2810d060 con 0x55a5a8aa18e0 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c1be00640 1 -- 172.21.15.46:0/2740343000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2c280eb990 con 0x55a5a8aa18e0 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 -- 172.21.15.46:0/2740343000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c280f7a30 con 0x55a5a8aa18e0 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c1be00640 1 -- 172.21.15.46:0/2740343000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c28112050 con 0x55a5a8aa18e0 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 10 client.7158 fetch_fsmap learned FSMap version 52 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 -- 172.21.15.46:0/2740343000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55a5a888f7c0 con 0x55a5a8aa18e0 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c1be00640 1 -- 172.21.15.46:0/2740343000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c280f7a30 con 0x55a5a8aa18e0 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c1be00640 1 -- 172.21.15.46:0/2740343000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2c28115020 con 0x55a5a8aa18e0 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 10 client.7158 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 10 client.7158 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 20 client.7158 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55a5a8aad7d0 0x55a5a8afc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 4 client.7158 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:47:36.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c4112e000 1 -- 172.21.15.46:0/2740343000 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5a8889cd0 con 0x55a5a8aad7d0 2024-04-25T08:47:36.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55a5a8aad7d0 0x55a5a8afc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55a5a8aad7d0 0x55a5a8afc850 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47bb6ff026777b8d server_cookie=12fde32568a1352 in_seq=0 out_seq=0 2024-04-25T08:47:36.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c1be00640 10 client.7158 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c1be00640 1 -- 172.21.15.46:0/2740343000 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5a8889cd0 con 0x55a5a8aad7d0 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.905+0000 7f2c1be00640 10 client.7158 handle_command_reply: tid=0 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 client.7158 shutdown 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 -- 172.21.15.46:0/2740343000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aa18e0 msgr2=0x55a5a8aac480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aa18e0 0x55a5a8aac480 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2c2801ca10 tx=0x7f2c2801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 -- 172.21.15.46:0/2740343000 shutdown_connections 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55a5a8aad7d0 0x55a5a8afc850 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aebc00 0x55a5a8aacf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 0x55a5a8aac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c1be00640 0 client.7158 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:36.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 --2- 172.21.15.46:0/2740343000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aa18e0 0x55a5a8aac480 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 -- 172.21.15.46:0/2740343000 >> 172.21.15.46:0/2740343000 conn(0x55a5a8a9d890 msgr2=0x55a5a8aa0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 -- 172.21.15.46:0/2740343000 shutdown_connections 2024-04-25T08:47:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 1 -- 172.21.15.46:0/2740343000 wait complete. 2024-04-25T08:47:36.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.909+0000 7f2c4112e000 20 client.7158 trim_cache size 0 max 16384 2024-04-25T08:47:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 Processor -- start 2024-04-25T08:47:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 -- start start 2024-04-25T08:47:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8a9d890 0x55a5a8aeaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 0x55a5a8aa0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aa11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c38202250 con 0x55a5a8aeb2e0 2024-04-25T08:47:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380b37e0 con 0x55a5a8a9d890 2024-04-25T08:47:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c380b3de0 con 0x55a5a8aa18e0 2024-04-25T08:47:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aa11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aa11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50322/0 (socket says 172.21.15.46:50322) 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c36a00640 1 --2- 172.21.15.46:0/4093687970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8a9d890 0x55a5a8aeaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c37400640 1 -- 172.21.15.46:0/4093687970 learned_addr learned my addr 172.21.15.46:0/4093687970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/4093687970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 0x55a5a8aa0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c36a00640 1 -- 172.21.15.46:0/4093687970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 msgr2=0x55a5a8aa0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c36a00640 1 --2- 172.21.15.46:0/4093687970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 0x55a5a8aa0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c36a00640 1 -- 172.21.15.46:0/4093687970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aeb2e0 msgr2=0x55a5a8aa11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c36a00640 1 --2- 172.21.15.46:0/4093687970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aa11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c36a00640 1 -- 172.21.15.46:0/4093687970 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5a88a9fc0 con 0x55a5a8a9d890 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c36a00640 1 --2- 172.21.15.46:0/4093687970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8a9d890 0x55a5a8aeaad0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2c280eb780 tx=0x7f2c28118040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809bab6555b689a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c26a00640 1 -- 172.21.15.46:0/4093687970 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280f7a30 con 0x55a5a8a9d890 2024-04-25T08:47:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c26a00640 1 -- 172.21.15.46:0/4093687970 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c280eb990 con 0x55a5a8a9d890 2024-04-25T08:47:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c26a00640 1 -- 172.21.15.46:0/4093687970 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280ef780 con 0x55a5a8a9d890 2024-04-25T08:47:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 -- 172.21.15.46:0/4093687970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8a9d890 msgr2=0x55a5a8aeaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 --2- 172.21.15.46:0/4093687970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8a9d890 0x55a5a8aeaad0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2c280eb780 tx=0x7f2c28118040 comp rx=0 tx=0).stop 2024-04-25T08:47:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 -- 172.21.15.46:0/4093687970 shutdown_connections 2024-04-25T08:47:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 --2- 172.21.15.46:0/4093687970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8aeb2e0 0x55a5a8aa11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 --2- 172.21.15.46:0/4093687970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aa18e0 0x55a5a8aa0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 --2- 172.21.15.46:0/4093687970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8a9d890 0x55a5a8aeaad0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 -- 172.21.15.46:0/4093687970 >> 172.21.15.46:0/4093687970 conn(0x55a5a8aacec0 msgr2=0x55a5a8ae9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 -- 172.21.15.46:0/4093687970 shutdown_connections 2024-04-25T08:47:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.921+0000 7f2c4112e000 1 -- 172.21.15.46:0/4093687970 wait complete. 2024-04-25T08:47:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 -- start start 2024-04-25T08:47:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 Processor -- start 2024-04-25T08:47:36.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8a9d890 0x55a5a8b4b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aa18e0 0x55a5a8b4b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 0x55a5a8b062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8888e70 con 0x55a5a8a9d890 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5a8876a50 con 0x55a5a8aa18e0 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5a88ed3a0 con 0x55a5a8aeb2e0 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 10 client.0 _ll_register_callbacks cb 0x55a5a889a240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8a9d890 0x55a5a8b4b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8a9d890 0x55a5a8b4b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50328/0 (socket says 172.21.15.46:50328) 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c36a00640 1 -- 172.21.15.46:0/1597494744 learned_addr learned my addr 172.21.15.46:0/1597494744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aa18e0 0x55a5a8b4b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c37400640 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 0x55a5a8b062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c36a00640 1 -- 172.21.15.46:0/1597494744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 msgr2=0x55a5a8b062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c36a00640 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 0x55a5a8b062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c36a00640 1 -- 172.21.15.46:0/1597494744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aa18e0 msgr2=0x55a5a8b4b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c36a00640 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aa18e0 0x55a5a8b4b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c36a00640 1 -- 172.21.15.46:0/1597494744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5a89479e0 con 0x55a5a8a9d890 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c36a00640 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8a9d890 0x55a5a8b4b280 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2c280190b0 tx=0x7f2c280acf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b827b58b0d636544 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c1be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c1be00640 1 -- 172.21.15.46:0/1597494744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280f7a30 con 0x55a5a8a9d890 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c1be00640 1 -- 172.21.15.46:0/1597494744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2c281163f0 con 0x55a5a8a9d890 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 -- 172.21.15.46:0/1597494744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c280eb990 con 0x55a5a8a9d890 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c1be00640 1 -- 172.21.15.46:0/1597494744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c280f3470 con 0x55a5a8a9d890 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 10 client.7164 fetch_fsmap learned FSMap version 52 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 -- 172.21.15.46:0/1597494744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55a5a888d630 con 0x55a5a8a9d890 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c1be00640 1 -- 172.21.15.46:0/1597494744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c280eb990 con 0x55a5a8a9d890 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c1be00640 1 -- 172.21.15.46:0/1597494744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2c28115040 con 0x55a5a8a9d890 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 10 client.7164 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 10 client.7164 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:47:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 20 client.7164 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55a5a8aebc00 0x55a5a8b06c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 4 client.7164 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c4112e000 1 -- 172.21.15.46:0/1597494744 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55a5a88fb5d0 con 0x55a5a8aebc00 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55a5a8aebc00 0x55a5a8b06c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c1be00640 10 client.7164 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.925+0000 7f2c2fe00640 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55a5a8aebc00 0x55a5a8b06c30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e59da0d608f684 server_cookie=b1dc61792f97db9b in_seq=0 out_seq=0 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c1be00640 1 -- 172.21.15.46:0/1597494744 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+237 (crc 0 0 0) 0x55a5a88fb5d0 con 0x55a5a8aebc00 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c1be00640 10 client.7164 handle_command_reply: tid=0 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 client.7164 shutdown 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 -- 172.21.15.46:0/1597494744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8a9d890 msgr2=0x55a5a8b4b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8a9d890 0x55a5a8b4b280 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2c280190b0 tx=0x7f2c280acf90 comp rx=0 tx=0).stop 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 -- 172.21.15.46:0/1597494744 shutdown_connections 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55a5a8aebc00 0x55a5a8b06c30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55a5a8aeb2e0 0x55a5a8b062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a5a8aa18e0 0x55a5a8b4b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 --2- 172.21.15.46:0/1597494744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5a8a9d890 0x55a5a8b4b280 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 -- 172.21.15.46:0/1597494744 >> 172.21.15.46:0/1597494744 conn(0x55a5a8aacec0 msgr2=0x55a5a8aea620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c1be00640 0 client.7164 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 -- 172.21.15.46:0/1597494744 shutdown_connections 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 1 -- 172.21.15.46:0/1597494744 wait complete. 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c4112e000 20 client.7164 trim_cache size 0 max 16384 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "dentry", 2024-04-25T08:47:36.925 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4218330404, 2024-04-25T08:47:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627776, 2024-04-25T08:47:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "frag": "*", 2024-04-25T08:47:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "file_to_be_damaged", 2024-04-25T08:47:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_id": "head", 2024-04-25T08:47:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-25T08:47:36.926 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:47:36.926 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 -- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c14066f60 msgr2=0x7f2c14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c14066f60 0x7f2c14069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c3813f740 tx=0x7f2c20009040 comp rx=0 tx=0).stop 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 -- 172.21.15.46:0/1396712180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c380710b0 msgr2=0x7f2c3813fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c380710b0 0x7f2c3813fb40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2c2801a0d0 tx=0x7f2c28019240 comp rx=0 tx=0).stop 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 -- 172.21.15.46:0/1396712180 shutdown_connections 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c14066f60 0x7f2c14069420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c38144e40 0x7f2c3814d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c380cd130 0x7f2c3815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 --2- 172.21.15.46:0/1396712180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c380710b0 0x7f2c3813fb40 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 -- 172.21.15.46:0/1396712180 >> 172.21.15.46:0/1396712180 conn(0x7f2c38066fa0 msgr2=0x7f2c380afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 -- 172.21.15.46:0/1396712180 shutdown_connections 2024-04-25T08:47:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:36.929+0000 7f2c37e00640 1 -- 172.21.15.46:0/1396712180 wait complete. 2024-04-25T08:47:36.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:47:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5381000640 1 Processor -- start 2024-04-25T08:47:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5381000640 1 -- start start 2024-04-25T08:47:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5381000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c0cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5381000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0cc520 con 0x7f537c0d0200 2024-04-25T08:47:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5381000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0cc800 con 0x7f537c0cd2e0 2024-04-25T08:47:37.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5381000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0cc2f0 con 0x7f537c161080 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5371400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f537c0d0200 0x7f537c06bd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41356/0 (socket says 172.21.15.46:41356) 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5371400640 1 -- 172.21.15.46:0/562474596 learned_addr learned my addr 172.21.15.46:0/562474596 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5379e00640 1 --2- 172.21.15.46:0/562474596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c0cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5379400640 1 -- 172.21.15.46:0/562474596 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901464906 0 0) 0x7f537c0cc520 con 0x7f537c0d0200 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5379400640 1 -- 172.21.15.46:0/562474596 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5360003530 con 0x7f537c0d0200 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5379400640 1 -- 172.21.15.46:0/562474596 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114532893 0 0) 0x7f537c0cc800 con 0x7f537c0cd2e0 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5379400640 1 -- 172.21.15.46:0/562474596 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f537c0cc520 con 0x7f537c0cd2e0 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5379e00640 1 -- 172.21.15.46:0/562474596 >> v1:172.21.15.82:6789/0 conn(0x7f537c0cd2e0 legacy=0x7f537c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5379e00640 1 -- 172.21.15.46:0/562474596 >> v1:172.21.15.46:6789/0 conn(0x7f537c0d0200 legacy=0x7f537c06bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.089+0000 7f5379e00640 1 -- 172.21.15.46:0/562474596 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0ccae0 con 0x7f537c161080 2024-04-25T08:47:37.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5379e00640 1 --2- 172.21.15.46:0/562474596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c0cfcc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f536c0097f0 tx=0x7f536c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f13cfb39f0594c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5379400640 1 -- 172.21.15.46:0/562474596 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0b0070 con 0x7f537c161080 2024-04-25T08:47:37.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5379400640 1 -- 172.21.15.46:0/562474596 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c0ab600 con 0x7f537c161080 2024-04-25T08:47:37.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5379400640 1 -- 172.21.15.46:0/562474596 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0ab9d0 con 0x7f537c161080 2024-04-25T08:47:37.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5381000640 1 -- 172.21.15.46:0/562474596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 msgr2=0x7f537c0cfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5381000640 1 --2- 172.21.15.46:0/562474596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c0cfcc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f536c0097f0 tx=0x7f536c01c4e0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5381000640 1 -- 172.21.15.46:0/562474596 shutdown_connections 2024-04-25T08:47:37.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5381000640 1 --2- 172.21.15.46:0/562474596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c0cfcc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5381000640 1 -- 172.21.15.46:0/562474596 >> 172.21.15.46:0/562474596 conn(0x7f537c0b31d0 msgr2=0x7f537c0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5381000640 1 -- 172.21.15.46:0/562474596 shutdown_connections 2024-04-25T08:47:37.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.093+0000 7f5381000640 1 -- 172.21.15.46:0/562474596 wait complete. 2024-04-25T08:47:37.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 Processor -- start 2024-04-25T08:47:37.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 -- start start 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537c0cd2e0 0x7f537c0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c0d0200 0x7f537c0ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0b29b0 con 0x7f537c0cd2e0 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c0b2830 con 0x7f537c0d0200 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c09a110 con 0x7f537c161080 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5379e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537c0cd2e0 0x7f537c0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5379e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537c0cd2e0 0x7f537c0cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50332/0 (socket says 172.21.15.46:50332) 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5379e00640 1 -- 172.21.15.46:0/1502104430 learned_addr learned my addr 172.21.15.46:0/1502104430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5371400640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c0d0200 0x7f537c0ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5379e00640 1 -- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 msgr2=0x7f537c1518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f537a800640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5379e00640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c1518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5379e00640 1 -- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c0d0200 msgr2=0x7f537c0ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5379e00640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c0d0200 0x7f537c0ae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5379e00640 1 -- 172.21.15.46:0/1502104430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c0ae980 con 0x7f537c0cd2e0 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5371400640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c0d0200 0x7f537c0ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f537a800640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5379e00640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537c0cd2e0 0x7f537c0cb350 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f536c0190f0 tx=0x7f536c01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5372a00640 1 -- 172.21.15.46:0/1502104430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0b0050 con 0x7f537c0cd2e0 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5372a00640 1 -- 172.21.15.46:0/1502104430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c0bc070 con 0x7f537c0cd2e0 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5372a00640 1 -- 172.21.15.46:0/1502104430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c006e20 con 0x7f537c0cd2e0 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c0b1ac0 con 0x7f537c0cd2e0 2024-04-25T08:47:37.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.097+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c0b2080 con 0x7f537c0cd2e0 2024-04-25T08:47:37.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.101+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c074c60 con 0x7f537c0cd2e0 2024-04-25T08:47:37.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.101+0000 7f5372a00640 1 -- 172.21.15.46:0/1502104430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f536c0b5830 con 0x7f537c0cd2e0 2024-04-25T08:47:37.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.101+0000 7f5372a00640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5358066f60 0x7f5358069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.101+0000 7f5372a00640 1 -- 172.21.15.46:0/1502104430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f536c11f8b0 con 0x7f537c0cd2e0 2024-04-25T08:47:37.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.101+0000 7f5371400640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5358066f60 0x7f5358069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.101+0000 7f5371400640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5358066f60 0x7f5358069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f537c1609e0 tx=0x7f5364009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.105+0000 7f5372a00640 1 -- 172.21.15.46:0/1502104430 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f536c0160d0 con 0x7f537c0cd2e0 2024-04-25T08:47:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.301+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f537c06d640 con 0x7f537c0cd2e0 2024-04-25T08:47:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.301+0000 7f5372a00640 1 -- 172.21.15.46:0/1502104430 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f536c0f0640 con 0x7f537c0cd2e0 2024-04-25T08:47:37.297 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:47:37.297 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5358066f60 msgr2=0x7f5358069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5358066f60 0x7f5358069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f537c1609e0 tx=0x7f5364009040 comp rx=0 tx=0).stop 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537c0cd2e0 msgr2=0x7f537c0cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537c0cd2e0 0x7f537c0cb350 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f536c0190f0 tx=0x7f536c01ad20 comp rx=0 tx=0).stop 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 shutdown_connections 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5358066f60 0x7f5358069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f537c161080 0x7f537c1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f537c0d0200 0x7f537c0ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 --2- 172.21.15.46:0/1502104430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f537c0cd2e0 0x7f537c0cb350 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 >> 172.21.15.46:0/1502104430 conn(0x7f537c0b31d0 msgr2=0x7f537c06fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 shutdown_connections 2024-04-25T08:47:37.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.305+0000 7f5381000640 1 -- 172.21.15.46:0/1502104430 wait complete. 2024-04-25T08:47:37.318 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:47:37.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:37.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-25T08:47:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.469+0000 7f2166a00640 1 Processor -- start 2024-04-25T08:47:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.469+0000 7f2166a00640 1 -- start start 2024-04-25T08:47:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2166a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f216013e6b0 0x7f21600711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2166a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 0x7f216015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2166a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216006d330 con 0x7f216015c4b0 2024-04-25T08:47:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2166a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216015cce0 con 0x7f216013e6b0 2024-04-25T08:47:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2166a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216006d610 con 0x7f2160071720 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2164a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f216015c4b0 0x7f21600cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41372/0 (socket says 172.21.15.46:41372) 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2164a00640 1 -- 172.21.15.46:0/2392445673 learned_addr learned my addr 172.21.15.46:0/2392445673 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f215fe00640 1 --2- 172.21.15.46:0/2392445673 >> v2:172.21.15.82:3300/0 conn(0x7f216013e6b0 0x7f21600711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2157400640 1 --2- 172.21.15.46:0/2392445673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 0x7f216015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f215f400640 1 -- 172.21.15.46:0/2392445673 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927600458 0 0) 0x7f216006d330 con 0x7f216015c4b0 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f215f400640 1 -- 172.21.15.46:0/2392445673 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2144003530 con 0x7f216015c4b0 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2157400640 1 -- 172.21.15.46:0/2392445673 >> v2:172.21.15.82:3300/0 conn(0x7f216013e6b0 msgr2=0x7f21600711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2157400640 1 --2- 172.21.15.46:0/2392445673 >> v2:172.21.15.82:3300/0 conn(0x7f216013e6b0 0x7f21600711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2157400640 1 -- 172.21.15.46:0/2392445673 >> v1:172.21.15.46:6789/0 conn(0x7f216015c4b0 legacy=0x7f21600cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f2157400640 1 -- 172.21.15.46:0/2392445673 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216006d8f0 con 0x7f2160071720 2024-04-25T08:47:37.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.473+0000 7f215fe00640 1 --2- 172.21.15.46:0/2392445673 >> v2:172.21.15.82:3300/0 conn(0x7f216013e6b0 0x7f21600711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:47:37.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2157400640 1 --2- 172.21.15.46:0/2392445673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 0x7f216015bf70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f21480097c0 tx=0x7f214801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67a1a889ae4088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f215f400640 1 -- 172.21.15.46:0/2392445673 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21480b0070 con 0x7f2160071720 2024-04-25T08:47:37.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f215f400640 1 -- 172.21.15.46:0/2392445673 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21480ab600 con 0x7f2160071720 2024-04-25T08:47:37.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f215f400640 1 -- 172.21.15.46:0/2392445673 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21480ab9b0 con 0x7f2160071720 2024-04-25T08:47:37.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 -- 172.21.15.46:0/2392445673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 msgr2=0x7f216015bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 --2- 172.21.15.46:0/2392445673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 0x7f216015bf70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f21480097c0 tx=0x7f214801c380 comp rx=0 tx=0).stop 2024-04-25T08:47:37.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 -- 172.21.15.46:0/2392445673 shutdown_connections 2024-04-25T08:47:37.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 --2- 172.21.15.46:0/2392445673 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 0x7f216015bf70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 --2- 172.21.15.46:0/2392445673 >> v2:172.21.15.82:3300/0 conn(0x7f216013e6b0 0x7f21600711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 -- 172.21.15.46:0/2392445673 >> 172.21.15.46:0/2392445673 conn(0x7f2160067050 msgr2=0x7f216006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 -- 172.21.15.46:0/2392445673 shutdown_connections 2024-04-25T08:47:37.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 -- 172.21.15.46:0/2392445673 wait complete. 2024-04-25T08:47:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 Processor -- start 2024-04-25T08:47:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.477+0000 7f2166a00640 1 -- start start 2024-04-25T08:47:37.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2166a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 0x7f21601409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2166a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216013e6b0 0x7f2160156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2166a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f216015c4b0 0x7f2160154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2166a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2160168380 con 0x7f216013e6b0 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2166a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2160168500 con 0x7f216015c4b0 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2166a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2160168680 con 0x7f2160071720 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2157400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216013e6b0 0x7f2160156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2157400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216013e6b0 0x7f2160156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50342/0 (socket says 172.21.15.46:50342) 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2157400640 1 -- 172.21.15.46:0/579003606 learned_addr learned my addr 172.21.15.46:0/579003606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f215fe00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 0x7f21601409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2164a00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f216015c4b0 0x7f2160154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2157400640 1 -- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 msgr2=0x7f21601409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2157400640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 0x7f21601409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2157400640 1 -- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f216015c4b0 msgr2=0x7f2160154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2157400640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f216015c4b0 0x7f2160154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2157400640 1 -- 172.21.15.46:0/579003606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2160159900 con 0x7f216013e6b0 2024-04-25T08:47:37.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2164a00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f216015c4b0 0x7f2160154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:47:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2157400640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216013e6b0 0x7f2160156d60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2148000c10 tx=0x7f2148019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f215cc00640 1 -- 172.21.15.46:0/579003606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21480ba070 con 0x7f216013e6b0 2024-04-25T08:47:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f215cc00640 1 -- 172.21.15.46:0/579003606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21480ab9b0 con 0x7f216013e6b0 2024-04-25T08:47:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2166a00640 1 -- 172.21.15.46:0/579003606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2160159be0 con 0x7f216013e6b0 2024-04-25T08:47:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f215cc00640 1 -- 172.21.15.46:0/579003606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21480b0040 con 0x7f216013e6b0 2024-04-25T08:47:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f2166a00640 1 -- 172.21.15.46:0/579003606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216015a1a0 con 0x7f216013e6b0 2024-04-25T08:47:37.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.481+0000 7f215cc00640 1 -- 172.21.15.46:0/579003606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21480bd070 con 0x7f216013e6b0 2024-04-25T08:47:37.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f215cc00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f213c066f60 0x7f213c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f215cc00640 1 -- 172.21.15.46:0/579003606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f214811e480 con 0x7f216013e6b0 2024-04-25T08:47:37.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f215fe00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f213c066f60 0x7f213c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 Processor -- start 2024-04-25T08:47:37.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 -- start start 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f215fe00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f213c066f60 0x7f213c069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2150009bd0 tx=0x7f215000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa798120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa798660 0x559efa74e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74e5f0 0x559efa74e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efa59cc40 con 0x559efa74e5f0 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efa5251d0 con 0x559efa798660 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x559efa598ce0 con 0x559efa797d40 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2164a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74e5f0 0x559efa74e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2164a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74e5f0 0x559efa74e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50348/0 (socket says 172.21.15.46:50348) 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2164a00640 1 -- 172.21.15.46:0/2372263241 learned_addr learned my addr 172.21.15.46:0/2372263241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f215fe00640 1 --2- 172.21.15.46:0/2372263241 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa798120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2157400640 1 --2- 172.21.15.46:0/2372263241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa798660 0x559efa74e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2164a00640 1 -- 172.21.15.46:0/2372263241 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 msgr2=0x559efa798120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2164a00640 1 --2- 172.21.15.46:0/2372263241 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa798120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2164a00640 1 -- 172.21.15.46:0/2372263241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa798660 msgr2=0x559efa74e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2164a00640 1 --2- 172.21.15.46:0/2372263241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa798660 0x559efa74e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2164a00640 1 -- 172.21.15.46:0/2372263241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559efa5e22a0 con 0x559efa74e5f0 2024-04-25T08:47:37.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2164a00640 1 --2- 172.21.15.46:0/2372263241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74e5f0 0x559efa74e9d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2158001640 tx=0x7f2158001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218e531087a619fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2154c00640 1 -- 172.21.15.46:0/2372263241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158019070 con 0x559efa74e5f0 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2154c00640 1 -- 172.21.15.46:0/2372263241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2158005ad0 con 0x559efa74e5f0 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2154c00640 1 -- 172.21.15.46:0/2372263241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158001d50 con 0x559efa74e5f0 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 -- 172.21.15.46:0/2372263241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74e5f0 msgr2=0x559efa74e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 --2- 172.21.15.46:0/2372263241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74e5f0 0x559efa74e9d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2158001640 tx=0x7f2158001b00 comp rx=0 tx=0).stop 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 -- 172.21.15.46:0/2372263241 shutdown_connections 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 --2- 172.21.15.46:0/2372263241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74e5f0 0x559efa74e9d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 --2- 172.21.15.46:0/2372263241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa798660 0x559efa74e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 --2- 172.21.15.46:0/2372263241 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa798120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 -- 172.21.15.46:0/2372263241 >> 172.21.15.46:0/2372263241 conn(0x559efa74a5a0 msgr2=0x559efa74c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.485+0000 7f2169eef000 1 -- 172.21.15.46:0/2372263241 shutdown_connections 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 -- 172.21.15.46:0/2372263241 wait complete. 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 -- start start 2024-04-25T08:47:37.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 Processor -- start 2024-04-25T08:47:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa74e5f0 0x559efa7539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa797d40 0x559efa753f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa798660 0x559efa77cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efa557a90 con 0x559efa798660 2024-04-25T08:47:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efa59bc10 con 0x559efa797d40 2024-04-25T08:47:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x559efa56b3e0 con 0x559efa74e5f0 2024-04-25T08:47:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 10 client.0 _ll_register_callbacks cb 0x559efa739410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2164a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa798660 0x559efa77cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f215fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa74e5f0 0x559efa7539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2164a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa798660 0x559efa77cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50352/0 (socket says 172.21.15.46:50352) 2024-04-25T08:47:37.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2164a00640 1 -- 172.21.15.46:0/2128554739 learned_addr learned my addr 172.21.15.46:0/2128554739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2157400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa797d40 0x559efa753f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2164a00640 1 -- 172.21.15.46:0/2128554739 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa74e5f0 msgr2=0x559efa7539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2164a00640 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa74e5f0 0x559efa7539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2164a00640 1 -- 172.21.15.46:0/2128554739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa797d40 msgr2=0x559efa753f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2164a00640 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa797d40 0x559efa753f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2164a00640 1 -- 172.21.15.46:0/2128554739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559efa555730 con 0x559efa798660 2024-04-25T08:47:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2164a00640 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa798660 0x559efa77cd70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2158009010 tx=0x7f2158000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dafbad9e16bede5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f214d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f214d600640 1 -- 172.21.15.46:0/2128554739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158023060 con 0x559efa798660 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 -- 172.21.15.46:0/2128554739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2158005ad0 con 0x559efa798660 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f214d600640 1 -- 172.21.15.46:0/2128554739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f215800e740 con 0x559efa798660 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f214d600640 1 -- 172.21.15.46:0/2128554739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2158029050 con 0x559efa798660 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 10 client.7182 fetch_fsmap learned FSMap version 52 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 -- 172.21.15.46:0/2128554739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559efa525420 con 0x559efa798660 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f214d600640 1 -- 172.21.15.46:0/2128554739 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2158005ad0 con 0x559efa798660 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f214d600640 1 -- 172.21.15.46:0/2128554739 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f215802c020 con 0x559efa798660 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 10 client.7182 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 10 client.7182 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 20 client.7182 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x559efa77d780 0x559efa7aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 4 client.7182 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:47:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.489+0000 7f2169eef000 1 -- 172.21.15.46:0/2128554739 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559efa538b50 con 0x559efa77d780 2024-04-25T08:47:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f215fe00640 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x559efa77d780 0x559efa7aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f215fe00640 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x559efa77d780 0x559efa7aa000 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9009d4b32ab17408 server_cookie=2c0e215377240d20 in_seq=0 out_seq=0 2024-04-25T08:47:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f214d600640 10 client.7182 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f214d600640 1 -- 172.21.15.46:0/2128554739 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559efa538b50 con 0x559efa77d780 2024-04-25T08:47:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f214d600640 10 client.7182 handle_command_reply: tid=0 2024-04-25T08:47:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 client.7182 shutdown 2024-04-25T08:47:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 -- 172.21.15.46:0/2128554739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa798660 msgr2=0x559efa77cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa798660 0x559efa77cd70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2158009010 tx=0x7f2158000ee0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 -- 172.21.15.46:0/2128554739 shutdown_connections 2024-04-25T08:47:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x559efa77d780 0x559efa7aa000 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa798660 0x559efa77cd70 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa797d40 0x559efa753f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 --2- 172.21.15.46:0/2128554739 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa74e5f0 0x559efa7539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f214d600640 0 client.7182 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 -- 172.21.15.46:0/2128554739 >> 172.21.15.46:0/2128554739 conn(0x559efa74a5a0 msgr2=0x559efa74d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 -- 172.21.15.46:0/2128554739 shutdown_connections 2024-04-25T08:47:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 1 -- 172.21.15.46:0/2128554739 wait complete. 2024-04-25T08:47:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.493+0000 7f2169eef000 20 client.7182 trim_cache size 0 max 16384 2024-04-25T08:47:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 Processor -- start 2024-04-25T08:47:37.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 -- start start 2024-04-25T08:47:37.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa74d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 0x559efa74dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa7aa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2160200f50 con 0x559efa74a5a0 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2160200170 con 0x559efa74e5f0 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21601373e0 con 0x559efa797d40 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f215fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa74d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f215fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa74d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50368/0 (socket says 172.21.15.46:50368) 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f215fe00640 1 -- 172.21.15.46:0/3899415072 learned_addr learned my addr 172.21.15.46:0/3899415072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2157400640 1 --2- 172.21.15.46:0/3899415072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 0x559efa74dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2164a00640 1 --2- 172.21.15.46:0/3899415072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa7aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f215fe00640 1 -- 172.21.15.46:0/3899415072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 msgr2=0x559efa7aa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f215fe00640 1 --2- 172.21.15.46:0/3899415072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa7aa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f215fe00640 1 -- 172.21.15.46:0/3899415072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 msgr2=0x559efa74dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f215fe00640 1 --2- 172.21.15.46:0/3899415072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 0x559efa74dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f215fe00640 1 -- 172.21.15.46:0/3899415072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559efa7517c0 con 0x559efa74a5a0 2024-04-25T08:47:37.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f215fe00640 1 --2- 172.21.15.46:0/3899415072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa74d9a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2150003ca0 tx=0x7f2150014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a274bc80b60529 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f214fe00640 1 -- 172.21.15.46:0/3899415072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150030070 con 0x559efa74a5a0 2024-04-25T08:47:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f214fe00640 1 -- 172.21.15.46:0/3899415072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2150004640 con 0x559efa74a5a0 2024-04-25T08:47:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f214fe00640 1 -- 172.21.15.46:0/3899415072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215002a490 con 0x559efa74a5a0 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 -- 172.21.15.46:0/3899415072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 msgr2=0x559efa74d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 --2- 172.21.15.46:0/3899415072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa74d9a0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2150003ca0 tx=0x7f2150014000 comp rx=0 tx=0).stop 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 -- 172.21.15.46:0/3899415072 shutdown_connections 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 --2- 172.21.15.46:0/3899415072 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa7aa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 --2- 172.21.15.46:0/3899415072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 0x559efa74dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 --2- 172.21.15.46:0/3899415072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa74d9a0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 -- 172.21.15.46:0/3899415072 >> 172.21.15.46:0/3899415072 conn(0x559efa77ad60 msgr2=0x559efa7a5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 -- 172.21.15.46:0/3899415072 shutdown_connections 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.509+0000 7f2169eef000 1 -- 172.21.15.46:0/3899415072 wait complete. 2024-04-25T08:47:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 -- start start 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 Processor -- start 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa7fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 0x559efa7fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa7b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559efa557a90 con 0x559efa74a5a0 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21601379e0 con 0x559efa74e5f0 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x559efa568230 con 0x559efa797d40 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 10 client.0 _ll_register_callbacks cb 0x7f2158007e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f215fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa7fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f215fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa7fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50382/0 (socket says 172.21.15.46:50382) 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f215fe00640 1 -- 172.21.15.46:0/3256978629 learned_addr learned my addr 172.21.15.46:0/3256978629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2157400640 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 0x559efa7fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2164a00640 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa7b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f215fe00640 1 -- 172.21.15.46:0/3256978629 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 msgr2=0x559efa7b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f215fe00640 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa7b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f215fe00640 1 -- 172.21.15.46:0/3256978629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 msgr2=0x559efa7fe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f215fe00640 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 0x559efa7fe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f215fe00640 1 -- 172.21.15.46:0/3256978629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559efa621550 con 0x559efa74a5a0 2024-04-25T08:47:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f215fe00640 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa7fdb60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x559efa757180 tx=0x7f2150004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9eb6fbe7848af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f214d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f214d600640 1 -- 172.21.15.46:0/3256978629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2150030040 con 0x559efa74a5a0 2024-04-25T08:47:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 -- 172.21.15.46:0/3256978629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2150004640 con 0x559efa74a5a0 2024-04-25T08:47:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f214d600640 1 -- 172.21.15.46:0/3256978629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f215002a390 con 0x559efa74a5a0 2024-04-25T08:47:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f214d600640 1 -- 172.21.15.46:0/3256978629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2150041050 con 0x559efa74a5a0 2024-04-25T08:47:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 10 client.7188 fetch_fsmap learned FSMap version 52 2024-04-25T08:47:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 -- 172.21.15.46:0/3256978629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559efa54f490 con 0x559efa74a5a0 2024-04-25T08:47:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f214d600640 1 -- 172.21.15.46:0/3256978629 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2150004640 con 0x559efa74a5a0 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f214d600640 1 -- 172.21.15.46:0/3256978629 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2150044020 con 0x559efa74a5a0 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 10 client.7188 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 10 client.7188 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 20 client.7188 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x559efa798660 0x559efa7b9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 4 client.7188 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2169eef000 1 -- 172.21.15.46:0/3256978629 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x559efa5aa180 con 0x559efa798660 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.513+0000 7f2157400640 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x559efa798660 0x559efa7b9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2157400640 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x559efa798660 0x559efa7b9d20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10566e6e737ec9da server_cookie=7b10574def67ed06 in_seq=0 out_seq=0 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f214d600640 10 client.7188 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f214d600640 1 -- 172.21.15.46:0/3256978629 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559efa5aa180 con 0x559efa798660 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f214d600640 10 client.7188 handle_command_reply: tid=0 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 client.7188 shutdown 2024-04-25T08:47:37.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 -- 172.21.15.46:0/3256978629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 msgr2=0x559efa7fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa7fdb60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x559efa757180 tx=0x7f2150004a50 comp rx=0 tx=0).stop 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 -- 172.21.15.46:0/3256978629 shutdown_connections 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x559efa798660 0x559efa7b9d20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559efa797d40 0x559efa7b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559efa74e5f0 0x559efa7fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 --2- 172.21.15.46:0/3256978629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559efa74a5a0 0x559efa7fdb60 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f214d600640 0 client.7188 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 -- 172.21.15.46:0/3256978629 >> 172.21.15.46:0/3256978629 conn(0x559efa77ad60 msgr2=0x559efa74c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 -- 172.21.15.46:0/3256978629 shutdown_connections 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 1 -- 172.21.15.46:0/3256978629 wait complete. 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2169eef000 20 client.7188 trim_cache size 0 max 16384 2024-04-25T08:47:37.513 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:47:37.514 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:47:37.514 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "fd556ca6-8340-4998-9f37-55ef73046e5d", 2024-04-25T08:47:37.514 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:47:37.514 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:47:37.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 -- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f213c066f60 msgr2=0x7f213c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f213c066f60 0x7f213c069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2150009bd0 tx=0x7f215000b040 comp rx=0 tx=0).stop 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 -- 172.21.15.46:0/579003606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216013e6b0 msgr2=0x7f2160156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216013e6b0 0x7f2160156d60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2148000c10 tx=0x7f2148019120 comp rx=0 tx=0).stop 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 -- 172.21.15.46:0/579003606 shutdown_connections 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f213c066f60 0x7f213c069420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f216015c4b0 0x7f2160154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f216013e6b0 0x7f2160156d60 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 --2- 172.21.15.46:0/579003606 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2160071720 0x7f21601409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 -- 172.21.15.46:0/579003606 >> 172.21.15.46:0/579003606 conn(0x7f2160067050 msgr2=0x7f21600cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 -- 172.21.15.46:0/579003606 shutdown_connections 2024-04-25T08:47:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.517+0000 7f2166a00640 1 -- 172.21.15.46:0/579003606 wait complete. 2024-04-25T08:47:37.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:47:37.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:47:37.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797fe00640 1 Processor -- start 2024-04-25T08:47:37.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797fe00640 1 -- start start 2024-04-25T08:47:37.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f798015b6f0 0x7f79800cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79800cff30 0x7f798015bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f798015c010 0x7f798015f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79800ccdc0 con 0x7f798015c010 2024-04-25T08:47:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797fe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79800cbf50 con 0x7f798015b6f0 2024-04-25T08:47:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79800cc1b0 con 0x7f79800cff30 2024-04-25T08:47:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f798015c010 0x7f798015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f798015c010 0x7f798015f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50398/0 (socket says 172.21.15.46:50398) 2024-04-25T08:47:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.677+0000 7f797f400640 1 -- 172.21.15.46:0/1366569088 learned_addr learned my addr 172.21.15.46:0/1366569088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f7976000640 1 --2- 172.21.15.46:0/1366569088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79800cff30 0x7f798015bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797ea00640 1 --2- 172.21.15.46:0/1366569088 >> v2:172.21.15.82:3300/0 conn(0x7f798015b6f0 0x7f79800cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797f400640 1 -- 172.21.15.46:0/1366569088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79800cff30 msgr2=0x7f798015bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797f400640 1 --2- 172.21.15.46:0/1366569088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79800cff30 0x7f798015bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797f400640 1 -- 172.21.15.46:0/1366569088 >> v2:172.21.15.82:3300/0 conn(0x7f798015b6f0 msgr2=0x7f79800cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797f400640 1 --2- 172.21.15.46:0/1366569088 >> v2:172.21.15.82:3300/0 conn(0x7f798015b6f0 0x7f79800cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797f400640 1 -- 172.21.15.46:0/1366569088 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79800cc490 con 0x7f798015c010 2024-04-25T08:47:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797f400640 1 --2- 172.21.15.46:0/1366569088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f798015c010 0x7f798015f330 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f798009f460 tx=0x7f797801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb4264c51aef4ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797e000640 1 -- 172.21.15.46:0/1366569088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780a57b0 con 0x7f798015c010 2024-04-25T08:47:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797e000640 1 -- 172.21.15.46:0/1366569088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79780a5950 con 0x7f798015c010 2024-04-25T08:47:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797e000640 1 -- 172.21.15.46:0/1366569088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780a5d20 con 0x7f798015c010 2024-04-25T08:47:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797fe00640 1 -- 172.21.15.46:0/1366569088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f798015c010 msgr2=0x7f798015f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.681+0000 7f797fe00640 1 --2- 172.21.15.46:0/1366569088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f798015c010 0x7f798015f330 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f798009f460 tx=0x7f797801c930 comp rx=0 tx=0).stop 2024-04-25T08:47:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 -- 172.21.15.46:0/1366569088 shutdown_connections 2024-04-25T08:47:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 --2- 172.21.15.46:0/1366569088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f798015c010 0x7f798015f330 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 --2- 172.21.15.46:0/1366569088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79800cff30 0x7f798015bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 --2- 172.21.15.46:0/1366569088 >> v2:172.21.15.82:3300/0 conn(0x7f798015b6f0 0x7f79800cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 -- 172.21.15.46:0/1366569088 >> 172.21.15.46:0/1366569088 conn(0x7f79800b2f50 msgr2=0x7f79800b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 -- 172.21.15.46:0/1366569088 shutdown_connections 2024-04-25T08:47:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 -- 172.21.15.46:0/1366569088 wait complete. 2024-04-25T08:47:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 Processor -- start 2024-04-25T08:47:37.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 -- start start 2024-04-25T08:47:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800cff30 0x7f7980153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798015b6f0 0x7f7980155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f798015c010 0x7f798006c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800cd3b0 con 0x7f79800cff30 2024-04-25T08:47:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800cd230 con 0x7f798015b6f0 2024-04-25T08:47:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7980099a00 con 0x7f798015c010 2024-04-25T08:47:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800cff30 0x7f7980153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800cff30 0x7f7980153a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50412/0 (socket says 172.21.15.46:50412) 2024-04-25T08:47:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797ea00640 1 -- 172.21.15.46:0/3138629397 learned_addr learned my addr 172.21.15.46:0/3138629397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797f400640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f798015c010 0x7f798006c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f7976000640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798015b6f0 0x7f7980155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797ea00640 1 -- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f798015c010 msgr2=0x7f798006c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797ea00640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f798015c010 0x7f798006c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797ea00640 1 -- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798015b6f0 msgr2=0x7f7980155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797ea00640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798015b6f0 0x7f7980155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797ea00640 1 -- 172.21.15.46:0/3138629397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798006bea0 con 0x7f79800cff30 2024-04-25T08:47:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f797ea00640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800cff30 0x7f7980153a10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f797000d830 tx=0x7f797000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.685+0000 7f7977400640 1 -- 172.21.15.46:0/3138629397 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000b6c0 con 0x7f79800cff30 2024-04-25T08:47:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f7977400640 1 -- 172.21.15.46:0/3138629397 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970019070 con 0x7f79800cff30 2024-04-25T08:47:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f797fe00640 1 -- 172.21.15.46:0/3138629397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7980156220 con 0x7f79800cff30 2024-04-25T08:47:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f7977400640 1 -- 172.21.15.46:0/3138629397 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000ba60 con 0x7f79800cff30 2024-04-25T08:47:37.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f797fe00640 1 -- 172.21.15.46:0/3138629397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f798006d650 con 0x7f79800cff30 2024-04-25T08:47:37.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f7977400640 1 -- 172.21.15.46:0/3138629397 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f797000bc00 con 0x7f79800cff30 2024-04-25T08:47:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f7977400640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f7977400640 1 -- 172.21.15.46:0/3138629397 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7970085550 con 0x7f79800cff30 2024-04-25T08:47:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f7976000640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f7988be4000 1 Processor -- start 2024-04-25T08:47:37.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.689+0000 7f7988be4000 1 -- start start 2024-04-25T08:47:37.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7980165b40 tx=0x7f7968008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e745946db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e7458fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e7458fd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e745749ff0 con 0x55e7459472f0 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7456d2150 con 0x55e7458fd0a0 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e745746090 con 0x55e7459469d0 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e7458fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e7458fcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50416/0 (socket says 172.21.15.46:50416) 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f797ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e745946db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 -- 172.21.15.46:0/1222108113 learned_addr learned my addr 172.21.15.46:0/1222108113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f797f400640 1 --2- 172.21.15.46:0/1222108113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e7458fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 -- 172.21.15.46:0/1222108113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 msgr2=0x55e745946db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 --2- 172.21.15.46:0/1222108113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e745946db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 -- 172.21.15.46:0/1222108113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 msgr2=0x55e7458fd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 --2- 172.21.15.46:0/1222108113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e7458fd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 -- 172.21.15.46:0/1222108113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7458e00d0 con 0x55e7459472f0 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7976000640 1 --2- 172.21.15.46:0/1222108113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e7458fcb60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f796800b850 tx=0x7f7968013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c2a886128514e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f796f400640 1 -- 172.21.15.46:0/1222108113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796801c050 con 0x55e7459472f0 2024-04-25T08:47:37.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f796f400640 1 -- 172.21.15.46:0/1222108113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7968026070 con 0x55e7459472f0 2024-04-25T08:47:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f796f400640 1 -- 172.21.15.46:0/1222108113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968016c00 con 0x55e7459472f0 2024-04-25T08:47:37.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 -- 172.21.15.46:0/1222108113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 msgr2=0x55e7458fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 --2- 172.21.15.46:0/1222108113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e7458fcb60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f796800b850 tx=0x7f7968013000 comp rx=0 tx=0).stop 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 -- 172.21.15.46:0/1222108113 shutdown_connections 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 --2- 172.21.15.46:0/1222108113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e7458fd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 --2- 172.21.15.46:0/1222108113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e7458fcb60 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 --2- 172.21.15.46:0/1222108113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e745946db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 -- 172.21.15.46:0/1222108113 >> 172.21.15.46:0/1222108113 conn(0x55e7458f9020 msgr2=0x55e7458fb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 -- 172.21.15.46:0/1222108113 shutdown_connections 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 -- 172.21.15.46:0/1222108113 wait complete. 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 -- start start 2024-04-25T08:47:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.693+0000 7f7988be4000 1 Processor -- start 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e74592b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e74592bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e74592c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e74570e250 con 0x55e7459472f0 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e745704690 con 0x55e7458fd0a0 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7456d2a40 con 0x55e7459469d0 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 10 client.0 _ll_register_callbacks cb 0x55e7458e7e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e74592c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e74592c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50422/0 (socket says 172.21.15.46:50422) 2024-04-25T08:47:37.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797f400640 1 -- 172.21.15.46:0/362646709 learned_addr learned my addr 172.21.15.46:0/362646709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7976000640 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e74592bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797ea00640 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e74592b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797f400640 1 -- 172.21.15.46:0/362646709 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 msgr2=0x55e74592bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797f400640 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e74592bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797f400640 1 -- 172.21.15.46:0/362646709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 msgr2=0x55e74592b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797f400640 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e74592b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797f400640 1 -- 172.21.15.46:0/362646709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7457f71c0 con 0x55e7459472f0 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797f400640 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e74592c270 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f797801c970 tx=0x7f7978004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b658b4221c7526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7963e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7963e00640 1 -- 172.21.15.46:0/362646709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780a58b0 con 0x55e7459472f0 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7963e00640 1 -- 172.21.15.46:0/362646709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f79780c2cb0 con 0x55e7459472f0 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 -- 172.21.15.46:0/362646709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7968026070 con 0x55e7459472f0 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7963e00640 1 -- 172.21.15.46:0/362646709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79780a5a50 con 0x55e7459472f0 2024-04-25T08:47:37.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 10 client.7200 fetch_fsmap learned FSMap version 52 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 -- 172.21.15.46:0/362646709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55e7457f3170 con 0x55e7459472f0 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7963e00640 1 -- 172.21.15.46:0/362646709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7968026070 con 0x55e7459472f0 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7963e00640 1 -- 172.21.15.46:0/362646709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f79780c9050 con 0x55e7459472f0 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 10 client.7200 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 10 client.7200 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 20 client.7200 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e745956c80 0x55e745959070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 4 client.7200 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 -- 172.21.15.46:0/362646709 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7456e5c60 con 0x55e745956c80 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797ea00640 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e745956c80 0x55e745959070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f797ea00640 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e745956c80 0x55e745959070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d5069c3ff80cc9 server_cookie=830c7240c0125a1c in_seq=0 out_seq=0 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7963e00640 10 client.7200 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7963e00640 1 -- 172.21.15.46:0/362646709 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e7456e5c60 con 0x55e745956c80 2024-04-25T08:47:37.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7963e00640 10 client.7200 handle_command_reply: tid=0 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 client.7200 shutdown 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 -- 172.21.15.46:0/362646709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 msgr2=0x55e74592c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.697+0000 7f7988be4000 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e74592c270 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f797801c970 tx=0x7f7978004a50 comp rx=0 tx=0).stop 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7988be4000 1 -- 172.21.15.46:0/362646709 shutdown_connections 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7988be4000 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e745956c80 0x55e745959070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7988be4000 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e74592c270 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7988be4000 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e74592bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7988be4000 1 --2- 172.21.15.46:0/362646709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e74592b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7963e00640 0 client.7200 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7988be4000 1 -- 172.21.15.46:0/362646709 >> 172.21.15.46:0/362646709 conn(0x55e7458f9020 msgr2=0x55e7458fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7988be4000 1 -- 172.21.15.46:0/362646709 shutdown_connections 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7988be4000 1 -- 172.21.15.46:0/362646709 wait complete. 2024-04-25T08:47:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.701+0000 7f7988be4000 20 client.7200 trim_cache size 0 max 16384 2024-04-25T08:47:37.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.713+0000 7f7988be4000 1 Processor -- start 2024-04-25T08:47:37.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.713+0000 7f7988be4000 1 -- start start 2024-04-25T08:47:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e7458fc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e7459457e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745945d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79802035d0 con 0x55e7459472f0 2024-04-25T08:47:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7980202830 con 0x55e7458fd0a0 2024-04-25T08:47:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7980158550 con 0x55e7459469d0 2024-04-25T08:47:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745945d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745945d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50428/0 (socket says 172.21.15.46:50428) 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797f400640 1 -- 172.21.15.46:0/1329049389 learned_addr learned my addr 172.21.15.46:0/1329049389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7976000640 1 --2- 172.21.15.46:0/1329049389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e7459457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797ea00640 1 --2- 172.21.15.46:0/1329049389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e7458fc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797f400640 1 -- 172.21.15.46:0/1329049389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 msgr2=0x55e7459457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797f400640 1 --2- 172.21.15.46:0/1329049389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e7459457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797f400640 1 -- 172.21.15.46:0/1329049389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 msgr2=0x55e7458fc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797f400640 1 --2- 172.21.15.46:0/1329049389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e7458fc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797f400640 1 -- 172.21.15.46:0/1329049389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e745790de0 con 0x55e7459472f0 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f797f400640 1 --2- 172.21.15.46:0/1329049389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745945d20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f797801c760 tx=0x7f79780a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62aa7809c0e8867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f796ea00640 1 -- 172.21.15.46:0/1329049389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780cf020 con 0x55e7459472f0 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f796ea00640 1 -- 172.21.15.46:0/1329049389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978009af0 con 0x55e7459472f0 2024-04-25T08:47:37.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f796ea00640 1 -- 172.21.15.46:0/1329049389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978009510 con 0x55e7459472f0 2024-04-25T08:47:37.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- 172.21.15.46:0/1329049389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 msgr2=0x55e745945d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- 172.21.15.46:0/1329049389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745945d20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f797801c760 tx=0x7f79780a7000 comp rx=0 tx=0).stop 2024-04-25T08:47:37.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- 172.21.15.46:0/1329049389 shutdown_connections 2024-04-25T08:47:37.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- 172.21.15.46:0/1329049389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745945d20 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- 172.21.15.46:0/1329049389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7459469d0 0x55e7459457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- 172.21.15.46:0/1329049389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7458fd0a0 0x55e7458fc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- 172.21.15.46:0/1329049389 >> 172.21.15.46:0/1329049389 conn(0x55e7458f9020 msgr2=0x55e7459450d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- 172.21.15.46:0/1329049389 shutdown_connections 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- 172.21.15.46:0/1329049389 wait complete. 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- start start 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 Processor -- start 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7458fd0a0 0x55e7459a7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7459469d0 0x55e7459a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745959380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7456e4e00 con 0x55e7459472f0 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79800ae6f0 con 0x55e7459469d0 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e745749300 con 0x55e7458fd0a0 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.717+0000 7f7988be4000 10 client.0 _ll_register_callbacks cb 0x55e7458c8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7458fd0a0 0x55e7459a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7976000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7459469d0 0x55e7459a7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745959380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797ea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7458fd0a0 0x55e7459a7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:39694/0 (socket says 172.21.15.46:39694) 2024-04-25T08:47:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797ea00640 1 -- 172.21.15.46:0/73420469 learned_addr learned my addr 172.21.15.46:0/73420469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:47:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797ea00640 1 -- 172.21.15.46:0/73420469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7459469d0 msgr2=0x55e7459a7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797ea00640 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7459469d0 0x55e7459a7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797ea00640 1 -- 172.21.15.46:0/73420469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 msgr2=0x55e745959380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:47:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797ea00640 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745959380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797ea00640 1 -- 172.21.15.46:0/73420469 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7456eb8e0 con 0x55e7458fd0a0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797ea00640 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7458fd0a0 0x55e7459a7940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7970055a20 tx=0x7f7970085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e268a837039822 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3301/0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 1 -- 172.21.15.46:0/73420469 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970072060 con 0x55e7458fd0a0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 1 -- 172.21.15.46:0/73420469 <== mon.2 v2:172.21.15.82:3301/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f797005def0 con 0x55e7458fd0a0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 -- 172.21.15.46:0/73420469 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7978009af0 con 0x55e7458fd0a0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 1 -- 172.21.15.46:0/73420469 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f797005e0f0 con 0x55e7458fd0a0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 10 client.6281 fetch_fsmap learned FSMap version 52 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 -- 172.21.15.46:0/73420469 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55e7457a38f0 con 0x55e7458fd0a0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 1 -- 172.21.15.46:0/73420469 <== mon.2 v2:172.21.15.82:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978009af0 con 0x55e7458fd0a0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 1 -- 172.21.15.46:0/73420469 <== mon.2 v2:172.21.15.82:3301/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f797005ace0 con 0x55e7458fd0a0 2024-04-25T08:47:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 10 client.6281 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 10 client.6281 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 20 client.6281 populate_metadata read hostname 'smithi046' 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e745959f30 0x55e74595a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 4 client.6281 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 -- 172.21.15.46:0/73420469 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e745757530 con 0x55e745959f30 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7976000640 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e745959f30 0x55e74595a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7976000640 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e745959f30 0x55e74595a310 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f85e66281f355 server_cookie=3f3ee19775a2594 in_seq=0 out_seq=0 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 10 client.6281 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 1 -- 172.21.15.46:0/73420469 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55e745757530 con 0x55e745959f30 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 10 client.6281 handle_command_reply: tid=0 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 client.6281 shutdown 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 -- 172.21.15.46:0/73420469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7458fd0a0 msgr2=0x55e7459a7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7458fd0a0 0x55e7459a7940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7970055a20 tx=0x7f7970085040 comp rx=0 tx=0).stop 2024-04-25T08:47:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 -- 172.21.15.46:0/73420469 shutdown_connections 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e745959f30 0x55e74595a310 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7459472f0 0x55e745959380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7459469d0 0x55e7459a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 --2- 172.21.15.46:0/73420469 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7458fd0a0 0x55e7459a7940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 -- 172.21.15.46:0/73420469 >> 172.21.15.46:0/73420469 conn(0x55e7458f9020 msgr2=0x55e745945960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7963e00640 0 client.6281 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 -- 172.21.15.46:0/73420469 shutdown_connections 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 1 -- 172.21.15.46:0/73420469 wait complete. 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f7988be4000 20 client.6281 trim_cache size 0 max 16384 2024-04-25T08:47:37.720 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:47:37.721 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:47:37.721 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:47:37.721 INFO:teuthology.orchestra.run.smithi046.stdout: "fd556ca6-8340-4998-9f37-55ef73046e5d": { 2024-04-25T08:47:37.721 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/subdir", 2024-04-25T08:47:37.721 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "fd556ca6-8340-4998-9f37-55ef73046e5d", 2024-04-25T08:47:37.721 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-04-25T08:47:37.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:47:37.721 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:47:37.721 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:47:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 -- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 msgr2=0x7f795c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7980165b40 tx=0x7f7968008040 comp rx=0 tx=0).stop 2024-04-25T08:47:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 -- 172.21.15.46:0/3138629397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800cff30 msgr2=0x7f7980153a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:47:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800cff30 0x7f7980153a10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f797000d830 tx=0x7f797000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 -- 172.21.15.46:0/3138629397 shutdown_connections 2024-04-25T08:47:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f798015c010 0x7f798006c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798015b6f0 0x7f7980155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 --2- 172.21.15.46:0/3138629397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800cff30 0x7f7980153a10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:47:37.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.721+0000 7f797fe00640 1 -- 172.21.15.46:0/3138629397 >> 172.21.15.46:0/3138629397 conn(0x7f79800b2f50 msgr2=0x7f7980159a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:47:37.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.725+0000 7f797fe00640 1 -- 172.21.15.46:0/3138629397 shutdown_connections 2024-04-25T08:47:37.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:47:37.725+0000 7f797fe00640 1 -- 172.21.15.46:0/3138629397 wait complete. 2024-04-25T08:47:37.735 INFO:tasks.cephfs.filesystem:scrub status for tag:fd556ca6-8340-4998-9f37-55ef73046e5d - {'path': '/subdir', 'tag': 'fd556ca6-8340-4998-9f37-55ef73046e5d', 'options': 'recursive,repair'} 2024-04-25T08:47:50.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:47:50.706 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:48:07.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:07.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:48:07.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.896+0000 7f7307e00640 1 Processor -- start 2024-04-25T08:48:07.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.896+0000 7f7307e00640 1 -- start start 2024-04-25T08:48:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7307e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 0x7f730006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7307e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f730006dc70 0x7f73000cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7307e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73000cd930 con 0x7f73000cf1b0 2024-04-25T08:48:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7307e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73000cdbd0 con 0x7f730006dc70 2024-04-25T08:48:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7307e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73000cdeb0 con 0x7f7300145310 2024-04-25T08:48:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7305400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 0x7f730006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7304a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f730006dc70 0x7f73000cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7305400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 0x7f730006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:45760/0 (socket says 172.21.15.46:45760) 2024-04-25T08:48:07.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7305400640 1 -- 172.21.15.46:0/2004983218 learned_addr learned my addr 172.21.15.46:0/2004983218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7304a00640 1 -- 172.21.15.46:0/2004983218 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 msgr2=0x7f730006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7304a00640 1 --2- 172.21.15.46:0/2004983218 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 0x7f730006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7304a00640 1 -- 172.21.15.46:0/2004983218 >> v1:172.21.15.82:6789/0 conn(0x7f73000cf1b0 legacy=0x7f730015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.900+0000 7f7304a00640 1 -- 172.21.15.46:0/2004983218 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000ce190 con 0x7f730006dc70 2024-04-25T08:48:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7304a00640 1 --2- 172.21.15.46:0/2004983218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f730006dc70 0x7f73000cec70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0097c0 tx=0x7f72ec01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8acdb736643a332d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f72ffe00640 1 -- 172.21.15.46:0/2004983218 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0a7070 con 0x7f730006dc70 2024-04-25T08:48:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f72ffe00640 1 -- 172.21.15.46:0/2004983218 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec0b0070 con 0x7f730006dc70 2024-04-25T08:48:07.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f72ffe00640 1 -- 172.21.15.46:0/2004983218 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0ab890 con 0x7f730006dc70 2024-04-25T08:48:07.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 -- 172.21.15.46:0/2004983218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f730006dc70 msgr2=0x7f73000cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 --2- 172.21.15.46:0/2004983218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f730006dc70 0x7f73000cec70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0097c0 tx=0x7f72ec01c470 comp rx=0 tx=0).stop 2024-04-25T08:48:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 -- 172.21.15.46:0/2004983218 shutdown_connections 2024-04-25T08:48:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 --2- 172.21.15.46:0/2004983218 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f730006dc70 0x7f73000cec70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 --2- 172.21.15.46:0/2004983218 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 0x7f730006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 -- 172.21.15.46:0/2004983218 >> 172.21.15.46:0/2004983218 conn(0x7f730006ac10 msgr2=0x7f730006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 -- 172.21.15.46:0/2004983218 shutdown_connections 2024-04-25T08:48:07.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 -- 172.21.15.46:0/2004983218 wait complete. 2024-04-25T08:48:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 Processor -- start 2024-04-25T08:48:07.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.904+0000 7f7307e00640 1 -- start start 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7307e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73000cf1b0 0x7f73000b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7307e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 0x7f730013aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7307e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73000b2820 0x7f73000b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7307e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730015d920 con 0x7f73000cf1b0 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7307e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730015d7a0 con 0x7f73000b2820 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7307e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7300168510 con 0x7f7300145310 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73000cf1b0 0x7f73000b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73000cf1b0 0x7f73000b4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35498/0 (socket says 172.21.15.46:35498) 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305400640 1 -- 172.21.15.46:0/4077840835 learned_addr learned my addr 172.21.15.46:0/4077840835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305e00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73000b2820 0x7f73000b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7304a00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 0x7f730013aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305400640 1 -- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 msgr2=0x7f730013aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305400640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 0x7f730013aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305400640 1 -- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73000b2820 msgr2=0x7f73000b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305400640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73000b2820 0x7f73000b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305400640 1 -- 172.21.15.46:0/4077840835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000b32e0 con 0x7f73000cf1b0 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7305400640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73000cf1b0 0x7f73000b4380 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f72f400e820 tx=0x7f72f400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f72fd600640 1 -- 172.21.15.46:0/4077840835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4017070 con 0x7f73000cf1b0 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f72fd600640 1 -- 172.21.15.46:0/4077840835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f400c940 con 0x7f73000cf1b0 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f72fd600640 1 -- 172.21.15.46:0/4077840835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f400cd30 con 0x7f73000cf1b0 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7307e00640 1 -- 172.21.15.46:0/4077840835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73000b35c0 con 0x7f73000cf1b0 2024-04-25T08:48:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.908+0000 7f7307e00640 1 -- 172.21.15.46:0/4077840835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73000b3b80 con 0x7f73000cf1b0 2024-04-25T08:48:07.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f72fd600640 1 -- 172.21.15.46:0/4077840835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72f401a070 con 0x7f73000cf1b0 2024-04-25T08:48:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f72fd600640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e0066f60 0x7f72e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f72fd600640 1 -- 172.21.15.46:0/4077840835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f72f4085400 con 0x7f73000cf1b0 2024-04-25T08:48:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f730b309000 1 Processor -- start 2024-04-25T08:48:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f730b309000 1 -- start start 2024-04-25T08:48:07.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e0066f60 0x7f72e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f730b309000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b44fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f730b309000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b405b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f730b309000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 0x55e61b406480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f730b309000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61b252ff0 con 0x55e61b4502f0 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f730b309000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61b1db150 con 0x55e61b4060a0 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f730b309000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e61b24f090 con 0x55e61b44f9d0 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e0066f60 0x7f72e0069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0af000 tx=0x7f72ec0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b405b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b405b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35508/0 (socket says 172.21.15.46:35508) 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7305400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b44fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 -- 172.21.15.46:0/1256101624 learned_addr learned my addr 172.21.15.46:0/1256101624 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7305e00640 1 --2- 172.21.15.46:0/1256101624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 0x55e61b406480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 -- 172.21.15.46:0/1256101624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 msgr2=0x55e61b44fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 --2- 172.21.15.46:0/1256101624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b44fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 -- 172.21.15.46:0/1256101624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 msgr2=0x55e61b406480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 --2- 172.21.15.46:0/1256101624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 0x55e61b406480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 -- 172.21.15.46:0/1256101624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e61b3e90d0 con 0x55e61b4502f0 2024-04-25T08:48:07.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f7304a00640 1 --2- 172.21.15.46:0/1256101624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b405b60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f72ec01a0a0 tx=0x7f72ec004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d267b661e468547 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f72f2000640 1 -- 172.21.15.46:0/1256101624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0bc070 con 0x55e61b4502f0 2024-04-25T08:48:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f72f2000640 1 -- 172.21.15.46:0/1256101624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec0b0040 con 0x55e61b4502f0 2024-04-25T08:48:07.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.912+0000 7f72f2000640 1 -- 172.21.15.46:0/1256101624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0b6610 con 0x55e61b4502f0 2024-04-25T08:48:07.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- 172.21.15.46:0/1256101624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 msgr2=0x55e61b405b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 --2- 172.21.15.46:0/1256101624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b405b60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f72ec01a0a0 tx=0x7f72ec004740 comp rx=0 tx=0).stop 2024-04-25T08:48:07.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- 172.21.15.46:0/1256101624 shutdown_connections 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 --2- 172.21.15.46:0/1256101624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 0x55e61b406480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 --2- 172.21.15.46:0/1256101624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b405b60 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 --2- 172.21.15.46:0/1256101624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b44fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- 172.21.15.46:0/1256101624 >> 172.21.15.46:0/1256101624 conn(0x55e61b402020 msgr2=0x55e61b404440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- 172.21.15.46:0/1256101624 shutdown_connections 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- 172.21.15.46:0/1256101624 wait complete. 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- start start 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 Processor -- start 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4060a0 0x55e61b4347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b434d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4502f0 0x55e61b435270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61b217250 con 0x55e61b4060a0 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61b20d690 con 0x55e61b4502f0 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e61b1dba40 con 0x55e61b44f9d0 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 10 client.0 _ll_register_callbacks cb 0x55e61b3f0e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f7305400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4060a0 0x55e61b4347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f7305400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4060a0 0x55e61b4347f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35516/0 (socket says 172.21.15.46:35516) 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f7305400640 1 -- 172.21.15.46:0/1623865416 learned_addr learned my addr 172.21.15.46:0/1623865416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f7305400640 1 -- 172.21.15.46:0/1623865416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 msgr2=0x55e61b434d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f7305400640 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b434d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f7305400640 1 -- 172.21.15.46:0/1623865416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4502f0 msgr2=0x55e61b435270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f7305400640 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4502f0 0x55e61b435270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f7305400640 1 -- 172.21.15.46:0/1623865416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e61b3001c0 con 0x55e61b4060a0 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f7305400640 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4060a0 0x55e61b4347f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f72f4055840 tx=0x7f72f4055560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ad7a4277b76c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:07.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f72e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f72e6a00640 1 -- 172.21.15.46:0/1623865416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4021040 con 0x55e61b4060a0 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f72e6a00640 1 -- 172.21.15.46:0/1623865416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f72f4054f90 con 0x55e61b4060a0 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- 172.21.15.46:0/1623865416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72ec0b0040 con 0x55e61b4060a0 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f72e6a00640 1 -- 172.21.15.46:0/1623865416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72f405e1a0 con 0x55e61b4060a0 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 10 client.7215 fetch_fsmap learned FSMap version 53 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.916+0000 7f730b309000 1 -- 172.21.15.46:0/1623865416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55e61b2fc170 con 0x55e61b4060a0 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f72e6a00640 1 -- 172.21.15.46:0/1623865416 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec0b0040 con 0x55e61b4060a0 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f72e6a00640 1 -- 172.21.15.46:0/1623865416 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f72f405ab00 con 0x55e61b4060a0 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 10 client.7215 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 10 client.7215 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 20 client.7215 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e61b45fc80 0x55e61b462070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 4 client.7215 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:48:07.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 -- 172.21.15.46:0/1623865416 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e61b1eec60 con 0x55e61b45fc80 2024-04-25T08:48:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f7304a00640 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e61b45fc80 0x55e61b462070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f7304a00640 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e61b45fc80 0x55e61b462070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9ed8e9e2ee48f4 server_cookie=f550eaaad17d2433 in_seq=0 out_seq=0 2024-04-25T08:48:07.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f72e6a00640 10 client.7215 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:48:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f72e6a00640 1 -- 172.21.15.46:0/1623865416 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e61b1eec60 con 0x55e61b45fc80 2024-04-25T08:48:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f72e6a00640 10 client.7215 handle_command_reply: tid=0 2024-04-25T08:48:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 client.7215 shutdown 2024-04-25T08:48:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 -- 172.21.15.46:0/1623865416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4060a0 msgr2=0x55e61b4347f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4060a0 0x55e61b4347f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f72f4055840 tx=0x7f72f4055560 comp rx=0 tx=0).stop 2024-04-25T08:48:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 -- 172.21.15.46:0/1623865416 shutdown_connections 2024-04-25T08:48:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e61b45fc80 0x55e61b462070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4502f0 0x55e61b435270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b434d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 --2- 172.21.15.46:0/1623865416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4060a0 0x55e61b4347f0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f72e6a00640 0 client.7215 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:48:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 -- 172.21.15.46:0/1623865416 >> 172.21.15.46:0/1623865416 conn(0x55e61b402020 msgr2=0x55e61b404ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 -- 172.21.15.46:0/1623865416 shutdown_connections 2024-04-25T08:48:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 1 -- 172.21.15.46:0/1623865416 wait complete. 2024-04-25T08:48:07.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.920+0000 7f730b309000 20 client.7215 trim_cache size 0 max 16384 2024-04-25T08:48:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 Processor -- start 2024-04-25T08:48:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 -- start start 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b4060a0 0x55e61b4ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b44f9d0 0x55e61b4ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4502f0 0x55e61b4acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73002034b0 con 0x55e61b44f9d0 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7300202690 con 0x55e61b4502f0 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73000ad710 con 0x55e61b4060a0 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7304a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b44f9d0 0x55e61b4ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7304a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b44f9d0 0x55e61b4ac630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35520/0 (socket says 172.21.15.46:35520) 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7304a00640 1 -- 172.21.15.46:0/744835420 learned_addr learned my addr 172.21.15.46:0/744835420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7305400640 1 --2- 172.21.15.46:0/744835420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b4060a0 0x55e61b4ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7305e00640 1 --2- 172.21.15.46:0/744835420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4502f0 0x55e61b4acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7304a00640 1 -- 172.21.15.46:0/744835420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b4060a0 msgr2=0x55e61b4ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7304a00640 1 --2- 172.21.15.46:0/744835420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b4060a0 0x55e61b4ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7304a00640 1 -- 172.21.15.46:0/744835420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4502f0 msgr2=0x55e61b4acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7304a00640 1 --2- 172.21.15.46:0/744835420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4502f0 0x55e61b4acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7304a00640 1 -- 172.21.15.46:0/744835420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e61b299de0 con 0x55e61b44f9d0 2024-04-25T08:48:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f7304a00640 1 --2- 172.21.15.46:0/744835420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b44f9d0 0x55e61b4ac630 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0ad040 tx=0x7f72ec0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3032d035ef3c8659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f72f1600640 1 -- 172.21.15.46:0/744835420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0bc040 con 0x55e61b44f9d0 2024-04-25T08:48:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f72f1600640 1 -- 172.21.15.46:0/744835420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec0b0040 con 0x55e61b44f9d0 2024-04-25T08:48:07.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f72f1600640 1 -- 172.21.15.46:0/744835420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0d5880 con 0x55e61b44f9d0 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 -- 172.21.15.46:0/744835420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b44f9d0 msgr2=0x55e61b4ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 --2- 172.21.15.46:0/744835420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b44f9d0 0x55e61b4ac630 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0ad040 tx=0x7f72ec0046d0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 -- 172.21.15.46:0/744835420 shutdown_connections 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 --2- 172.21.15.46:0/744835420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4502f0 0x55e61b4acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 --2- 172.21.15.46:0/744835420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b44f9d0 0x55e61b4ac630 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.936+0000 7f730b309000 1 --2- 172.21.15.46:0/744835420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b4060a0 0x55e61b4ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- 172.21.15.46:0/744835420 >> 172.21.15.46:0/744835420 conn(0x55e61b402020 msgr2=0x55e61b45cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- 172.21.15.46:0/744835420 shutdown_connections 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- 172.21.15.46:0/744835420 wait complete. 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- start start 2024-04-25T08:48:07.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 Processor -- start 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 0x55e61b462380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b4628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b462e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e61b1ede00 con 0x55e61b4502f0 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7300202090 con 0x55e61b4060a0 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e61b252300 con 0x55e61b44f9d0 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 10 client.0 _ll_register_callbacks cb 0x55e61b3d1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b462e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b462e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35536/0 (socket says 172.21.15.46:35536) 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305e00640 1 -- 172.21.15.46:0/3770482163 learned_addr learned my addr 172.21.15.46:0/3770482163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305400640 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 0x55e61b462380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7304a00640 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b4628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305e00640 1 -- 172.21.15.46:0/3770482163 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 msgr2=0x55e61b4628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305e00640 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b4628c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305e00640 1 -- 172.21.15.46:0/3770482163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 msgr2=0x55e61b462380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305e00640 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 0x55e61b462380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305e00640 1 -- 172.21.15.46:0/3770482163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e61b1f48e0 con 0x55e61b4502f0 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305e00640 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b462e00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f72f80156a0 tx=0x7f72f8015b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc750780fcbadfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f72e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f72e6a00640 1 -- 172.21.15.46:0/3770482163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8003980 con 0x55e61b4502f0 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f72e6a00640 1 -- 172.21.15.46:0/3770482163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f72f8003b20 con 0x55e61b4502f0 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- 172.21.15.46:0/3770482163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72ec0b0040 con 0x55e61b4502f0 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f72e6a00640 1 -- 172.21.15.46:0/3770482163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72f8026050 con 0x55e61b4502f0 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 10 client.7221 fetch_fsmap learned FSMap version 53 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- 172.21.15.46:0/3770482163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55e61b2ac8f0 con 0x55e61b4502f0 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f72e6a00640 1 -- 172.21.15.46:0/3770482163 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec0b0040 con 0x55e61b4502f0 2024-04-25T08:48:07.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f72e6a00640 1 -- 172.21.15.46:0/3770482163 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f72f8029020 con 0x55e61b4502f0 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 10 client.7221 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 10 client.7221 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 20 client.7221 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e61b4b5f50 0x55e61b4b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 4 client.7221 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f730b309000 1 -- 172.21.15.46:0/3770482163 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e61b260530 con 0x55e61b4b5f50 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.940+0000 7f7305400640 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e61b4b5f50 0x55e61b4b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7305400640 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e61b4b5f50 0x55e61b4b8340 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286bfd4971d48082 server_cookie=bccdba90d9f2efc5 in_seq=0 out_seq=0 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f72e6a00640 10 client.7221 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f72e6a00640 1 -- 172.21.15.46:0/3770482163 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e61b260530 con 0x55e61b4b5f50 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f72e6a00640 10 client.7221 handle_command_reply: tid=0 2024-04-25T08:48:07.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 client.7221 shutdown 2024-04-25T08:48:07.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 -- 172.21.15.46:0/3770482163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 msgr2=0x55e61b462e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b462e00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f72f80156a0 tx=0x7f72f8015b60 comp rx=0 tx=0).stop 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 -- 172.21.15.46:0/3770482163 shutdown_connections 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x55e61b4b5f50 0x55e61b4b8340 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e61b4502f0 0x55e61b462e00 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e61b44f9d0 0x55e61b4628c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f72e6a00640 0 client.7221 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 --2- 172.21.15.46:0/3770482163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e61b4060a0 0x55e61b462380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 -- 172.21.15.46:0/3770482163 >> 172.21.15.46:0/3770482163 conn(0x55e61b402020 msgr2=0x55e61b44e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 -- 172.21.15.46:0/3770482163 shutdown_connections 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 1 -- 172.21.15.46:0/3770482163 wait complete. 2024-04-25T08:48:07.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f730b309000 20 client.7221 trim_cache size 0 max 16384 2024-04-25T08:48:07.943 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:48:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:48:07.943 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:48:07.943 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 -- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e0066f60 msgr2=0x7f72e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e0066f60 0x7f72e0069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0af000 tx=0x7f72ec0a7000 comp rx=0 tx=0).stop 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 -- 172.21.15.46:0/4077840835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73000cf1b0 msgr2=0x7f73000b4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73000cf1b0 0x7f73000b4380 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f72f400e820 tx=0x7f72f400ece0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 -- 172.21.15.46:0/4077840835 shutdown_connections 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e0066f60 0x7f72e0069420 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73000b2820 0x7f73000b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7300145310 0x7f730013aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 --2- 172.21.15.46:0/4077840835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73000cf1b0 0x7f73000b4380 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 -- 172.21.15.46:0/4077840835 >> 172.21.15.46:0/4077840835 conn(0x7f730006ac10 msgr2=0x7f7300159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:07.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 -- 172.21.15.46:0/4077840835 shutdown_connections 2024-04-25T08:48:07.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:07.944+0000 7f7307e00640 1 -- 172.21.15.46:0/4077840835 wait complete. 2024-04-25T08:48:07.957 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:48:07.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.104+0000 7f439b600640 1 Processor -- start 2024-04-25T08:48:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.104+0000 7f439b600640 1 -- start start 2024-04-25T08:48:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f439b600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 0x7f439415d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f439b600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f43940cf520 0x7f439415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f439b600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439406ed70 con 0x7f43940afd40 2024-04-25T08:48:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f439b600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43940afb40 con 0x7f43940cf520 2024-04-25T08:48:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f439b600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43940d13f0 con 0x7f43940cf140 2024-04-25T08:48:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4399600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f43940cf520 0x7f439415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4399600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f43940cf520 0x7f439415dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49488/0 (socket says 172.21.15.46:49488) 2024-04-25T08:48:08.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4399600640 1 -- 172.21.15.46:0/761890939 learned_addr learned my addr 172.21.15.46:0/761890939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:08.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4399600640 1 -- 172.21.15.46:0/761890939 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 msgr2=0x7f439415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4398c00640 1 --2- 172.21.15.46:0/761890939 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 0x7f439415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4399600640 1 --2- 172.21.15.46:0/761890939 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 0x7f439415d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4399600640 1 -- 172.21.15.46:0/761890939 >> v1:172.21.15.82:6789/0 conn(0x7f43940afd40 legacy=0x7f43940ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4399600640 1 -- 172.21.15.46:0/761890939 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4394162530 con 0x7f43940cf520 2024-04-25T08:48:08.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4398c00640 1 --2- 172.21.15.46:0/761890939 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 0x7f439415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:08.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.108+0000 7f4399600640 1 --2- 172.21.15.46:0/761890939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 0x7f439415dad0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f438c009e50 tx=0x7f438c01c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc2f620544e8885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f4393400640 1 -- 172.21.15.46:0/761890939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c0ac070 con 0x7f43940cf520 2024-04-25T08:48:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f4393400640 1 -- 172.21.15.46:0/761890939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c0a8410 con 0x7f43940cf520 2024-04-25T08:48:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f4393400640 1 -- 172.21.15.46:0/761890939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c0a87e0 con 0x7f43940cf520 2024-04-25T08:48:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 -- 172.21.15.46:0/761890939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 msgr2=0x7f439415dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 --2- 172.21.15.46:0/761890939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 0x7f439415dad0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f438c009e50 tx=0x7f438c01c6f0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 -- 172.21.15.46:0/761890939 shutdown_connections 2024-04-25T08:48:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 --2- 172.21.15.46:0/761890939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 0x7f439415dad0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 --2- 172.21.15.46:0/761890939 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 0x7f439415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 -- 172.21.15.46:0/761890939 >> 172.21.15.46:0/761890939 conn(0x7f43940728c0 msgr2=0x7f4394099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 -- 172.21.15.46:0/761890939 shutdown_connections 2024-04-25T08:48:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 -- 172.21.15.46:0/761890939 wait complete. 2024-04-25T08:48:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 Processor -- start 2024-04-25T08:48:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 -- start start 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43940afd40 0x7f43940b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 0x7f4394153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 0x7f4394151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43940af7f0 con 0x7f43940cf520 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4394162010 con 0x7f43940afd40 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f439b600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439406b600 con 0x7f43940cf140 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f4399600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 0x7f4394151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f4399600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 0x7f4394151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49500/0 (socket says 172.21.15.46:49500) 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f4399600640 1 -- 172.21.15.46:0/1947637695 learned_addr learned my addr 172.21.15.46:0/1947637695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f4398c00640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43940afd40 0x7f43940b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.112+0000 7f4393e00640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 0x7f4394153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4399600640 1 -- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 msgr2=0x7f4394153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4399600640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 0x7f4394153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4399600640 1 -- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43940afd40 msgr2=0x7f43940b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4399600640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43940afd40 0x7f43940b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4399600640 1 -- 172.21.15.46:0/1947637695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43941548b0 con 0x7f43940cf520 2024-04-25T08:48:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4399600640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 0x7f4394151b10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f438c009040 tx=0x7f438c019490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4390c00640 1 -- 172.21.15.46:0/1947637695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c0ac040 con 0x7f43940cf520 2024-04-25T08:48:08.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4390c00640 1 -- 172.21.15.46:0/1947637695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c0bd650 con 0x7f43940cf520 2024-04-25T08:48:08.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4390c00640 1 -- 172.21.15.46:0/1947637695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c009510 con 0x7f43940cf520 2024-04-25T08:48:08.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4394154b90 con 0x7f43940cf520 2024-04-25T08:48:08.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4394158870 con 0x7f43940cf520 2024-04-25T08:48:08.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439406ccb0 con 0x7f43940cf520 2024-04-25T08:48:08.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4390c00640 1 -- 172.21.15.46:0/1947637695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f438c0af070 con 0x7f43940cf520 2024-04-25T08:48:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4390c00640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4374066f60 0x7f4374069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4390c00640 1 -- 172.21.15.46:0/1947637695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f438c11fb40 con 0x7f43940cf520 2024-04-25T08:48:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.116+0000 7f4398c00640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4374066f60 0x7f4374069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.120+0000 7f4398c00640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4374066f60 0x7f4374069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f43940b4990 tx=0x7f4388008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.120+0000 7f4390c00640 1 -- 172.21.15.46:0/1947637695 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f439406ed70 con 0x7f43940cf520 2024-04-25T08:48:08.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.316+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43941550b0 con 0x7f43940cf520 2024-04-25T08:48:08.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.316+0000 7f4390c00640 1 -- 172.21.15.46:0/1947637695 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f43941550b0 con 0x7f43940cf520 2024-04-25T08:48:08.311 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:08.312 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:47:43.911465+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/4112568749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":4112568749},{"type":"v1","addr":"172.21.15.46:6835","nonce":4112568749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:48:08.312 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 53 2024-04-25T08:48:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4374066f60 msgr2=0x7f4374069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4374066f60 0x7f4374069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f43940b4990 tx=0x7f4388008040 comp rx=0 tx=0).stop 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 msgr2=0x7f4394151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 0x7f4394151b10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f438c009040 tx=0x7f438c019490 comp rx=0 tx=0).stop 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 shutdown_connections 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4374066f60 0x7f4374069420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43940cf520 0x7f4394151b10 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43940cf140 0x7f4394153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 --2- 172.21.15.46:0/1947637695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43940afd40 0x7f43940b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 >> 172.21.15.46:0/1947637695 conn(0x7f43940728c0 msgr2=0x7f43940ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 shutdown_connections 2024-04-25T08:48:08.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.320+0000 7f439b600640 1 -- 172.21.15.46:0/1947637695 wait complete. 2024-04-25T08:48:08.332 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:48:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.480+0000 7f8782a00640 1 Processor -- start 2024-04-25T08:48:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.480+0000 7f8782a00640 1 -- start start 2024-04-25T08:48:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8782a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f877c099c60 0x7f877c160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8782a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f877c0cf210 0x7f877c0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8782a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877c0cdb10 con 0x7f877c09a040 2024-04-25T08:48:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8782a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c0cc300 con 0x7f877c0cf210 2024-04-25T08:48:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8782a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c0cc530 con 0x7f877c099c60 2024-04-25T08:48:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8773400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f877c0cf210 0x7f877c0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8773400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f877c0cf210 0x7f877c0cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49510/0 (socket says 172.21.15.46:49510) 2024-04-25T08:48:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8773400640 1 -- 172.21.15.46:0/603571638 learned_addr learned my addr 172.21.15.46:0/603571638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8773400640 1 -- 172.21.15.46:0/603571638 >> v2:172.21.15.82:3300/0 conn(0x7f877c099c60 msgr2=0x7f877c160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f877be00640 1 --2- 172.21.15.46:0/603571638 >> v2:172.21.15.82:3300/0 conn(0x7f877c099c60 0x7f877c160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8773400640 1 --2- 172.21.15.46:0/603571638 >> v2:172.21.15.82:3300/0 conn(0x7f877c099c60 0x7f877c160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8773400640 1 -- 172.21.15.46:0/603571638 >> v1:172.21.15.82:6789/0 conn(0x7f877c09a040 legacy=0x7f877c15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8773400640 1 -- 172.21.15.46:0/603571638 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c0cc810 con 0x7f877c0cf210 2024-04-25T08:48:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f877be00640 1 --2- 172.21.15.46:0/603571638 >> v2:172.21.15.82:3300/0 conn(0x7f877c099c60 0x7f877c160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:08.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.484+0000 7f8773400640 1 --2- 172.21.15.46:0/603571638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c0cf210 0x7f877c0cd5d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f877c072a30 tx=0x7f876401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af75a971edbb121f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f877b400640 1 -- 172.21.15.46:0/603571638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640b1070 con 0x7f877c0cf210 2024-04-25T08:48:08.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f877b400640 1 -- 172.21.15.46:0/603571638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87640ac630 con 0x7f877c0cf210 2024-04-25T08:48:08.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f877b400640 1 -- 172.21.15.46:0/603571638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640a8050 con 0x7f877c0cf210 2024-04-25T08:48:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 -- 172.21.15.46:0/603571638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c0cf210 msgr2=0x7f877c0cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 --2- 172.21.15.46:0/603571638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c0cf210 0x7f877c0cd5d0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f877c072a30 tx=0x7f876401c4e0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 -- 172.21.15.46:0/603571638 shutdown_connections 2024-04-25T08:48:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 --2- 172.21.15.46:0/603571638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c0cf210 0x7f877c0cd5d0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 --2- 172.21.15.46:0/603571638 >> v2:172.21.15.82:3300/0 conn(0x7f877c099c60 0x7f877c160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 -- 172.21.15.46:0/603571638 >> 172.21.15.46:0/603571638 conn(0x7f877c0b3130 msgr2=0x7f877c0b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 -- 172.21.15.46:0/603571638 shutdown_connections 2024-04-25T08:48:08.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 -- 172.21.15.46:0/603571638 wait complete. 2024-04-25T08:48:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 Processor -- start 2024-04-25T08:48:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 -- start start 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c099c60 0x7f877c0c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f877c09a040 0x7f877c157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f877c0cf210 0x7f877c0c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0b1930 con 0x7f877c099c60 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0b17b0 con 0x7f877c09a040 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f8782a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0b1ab0 con 0x7f877c0cf210 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f877be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c099c60 0x7f877c0c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f877be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c099c60 0x7f877c0c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49520/0 (socket says 172.21.15.46:49520) 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.488+0000 7f877be00640 1 -- 172.21.15.46:0/1571867823 learned_addr learned my addr 172.21.15.46:0/1571867823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f877be00640 1 -- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f877c0cf210 msgr2=0x7f877c0c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f877be00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f877c0cf210 0x7f877c0c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f877be00640 1 -- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f877c09a040 msgr2=0x7f877c157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f8773400640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f877c09a040 0x7f877c157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f877be00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f877c09a040 0x7f877c157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f877be00640 1 -- 172.21.15.46:0/1571867823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c0b06d0 con 0x7f877c099c60 2024-04-25T08:48:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f877be00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c099c60 0x7f877c0c6030 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f876c00ce00 tx=0x7f876c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f8778c00640 1 -- 172.21.15.46:0/1571867823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c00f890 con 0x7f877c099c60 2024-04-25T08:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f8778c00640 1 -- 172.21.15.46:0/1571867823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c00fa30 con 0x7f877c099c60 2024-04-25T08:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f8778c00640 1 -- 172.21.15.46:0/1571867823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c00fe40 con 0x7f877c099c60 2024-04-25T08:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f8782a00640 1 -- 172.21.15.46:0/1571867823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c0c6b40 con 0x7f877c099c60 2024-04-25T08:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f8782a00640 1 -- 172.21.15.46:0/1571867823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c0c7100 con 0x7f877c099c60 2024-04-25T08:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.492+0000 7f8778c00640 1 -- 172.21.15.46:0/1571867823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f876c018400 con 0x7f877c099c60 2024-04-25T08:48:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8778c00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8758066f60 0x7f8758069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8778c00640 1 -- 172.21.15.46:0/1571867823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f876c0860b0 con 0x7f877c099c60 2024-04-25T08:48:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8758066f60 0x7f8758069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 Processor -- start 2024-04-25T08:48:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 -- start start 2024-04-25T08:48:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8758066f60 0x7f8758069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87640ae000 tx=0x7f87640a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633301082e0 0x5633301086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330108c00 0x5633300be3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 0x5633300becc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332ffa5ba0 con 0x563330108c00 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332ff0b090 con 0x5633301082e0 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x56332fe93160 con 0x5633300be8e0 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330108c00 0x5633300be3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f877be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633301082e0 0x5633301086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330108c00 0x5633300be3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49536/0 (socket says 172.21.15.46:49536) 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8780a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 0x5633300becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 -- 172.21.15.46:0/1989956332 learned_addr learned my addr 172.21.15.46:0/1989956332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 -- 172.21.15.46:0/1989956332 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 msgr2=0x5633300becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 --2- 172.21.15.46:0/1989956332 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 0x5633300becc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 -- 172.21.15.46:0/1989956332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633301082e0 msgr2=0x5633301086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 --2- 172.21.15.46:0/1989956332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633301082e0 0x5633301086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 -- 172.21.15.46:0/1989956332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56332ffb4c20 con 0x563330108c00 2024-04-25T08:48:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8773400640 1 --2- 172.21.15.46:0/1989956332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330108c00 0x5633300be3a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f876401c520 tx=0x7f8764004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5e4d1ccabba34d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8770c00640 1 -- 172.21.15.46:0/1989956332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640b1040 con 0x563330108c00 2024-04-25T08:48:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8770c00640 1 -- 172.21.15.46:0/1989956332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87640c6070 con 0x563330108c00 2024-04-25T08:48:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8770c00640 1 -- 172.21.15.46:0/1989956332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640a8040 con 0x563330108c00 2024-04-25T08:48:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 -- 172.21.15.46:0/1989956332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330108c00 msgr2=0x5633300be3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 --2- 172.21.15.46:0/1989956332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330108c00 0x5633300be3a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f876401c520 tx=0x7f8764004740 comp rx=0 tx=0).stop 2024-04-25T08:48:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 -- 172.21.15.46:0/1989956332 shutdown_connections 2024-04-25T08:48:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 --2- 172.21.15.46:0/1989956332 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 0x5633300becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 --2- 172.21.15.46:0/1989956332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563330108c00 0x5633300be3a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 --2- 172.21.15.46:0/1989956332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633301082e0 0x5633301086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 -- 172.21.15.46:0/1989956332 >> 172.21.15.46:0/1989956332 conn(0x5633300ba890 msgr2=0x5633300bcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 -- 172.21.15.46:0/1989956332 shutdown_connections 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.496+0000 7f8785e77000 1 -- 172.21.15.46:0/1989956332 wait complete. 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 -- start start 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 Processor -- start 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 0x56333010f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x56333010fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563330108c00 0x5633300f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332ff07130 con 0x5633301082e0 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332fecf130 con 0x563330108c00 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x56332ffc9300 con 0x5633300be8e0 2024-04-25T08:48:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 10 client.0 _ll_register_callbacks cb 0x5633300a9bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x56333010fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x56333010fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49538/0 (socket says 172.21.15.46:49538) 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8773400640 1 -- 172.21.15.46:0/1849573074 learned_addr learned my addr 172.21.15.46:0/1849573074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f877be00640 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 0x56333010f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8780a00640 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563330108c00 0x5633300f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8773400640 1 -- 172.21.15.46:0/1849573074 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 msgr2=0x56333010f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8773400640 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 0x56333010f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8773400640 1 -- 172.21.15.46:0/1849573074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563330108c00 msgr2=0x5633300f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8773400640 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563330108c00 0x5633300f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8773400640 1 -- 172.21.15.46:0/1849573074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56332fee8060 con 0x5633301082e0 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8773400640 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x56333010fb80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f876401c520 tx=0x7f876401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d858ffa6148f4664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8769600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8769600640 1 -- 172.21.15.46:0/1849573074 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640a8040 con 0x5633301082e0 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8769600640 1 -- 172.21.15.46:0/1849573074 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8764004480 con 0x5633301082e0 2024-04-25T08:48:08.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 -- 172.21.15.46:0/1849573074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87640c6070 con 0x5633301082e0 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8769600640 1 -- 172.21.15.46:0/1849573074 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87640d7050 con 0x5633301082e0 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 10 client.7239 fetch_fsmap learned FSMap version 53 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 -- 172.21.15.46:0/1849573074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x56332feac7c0 con 0x5633301082e0 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8769600640 1 -- 172.21.15.46:0/1849573074 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87640c6070 con 0x5633301082e0 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8769600640 1 -- 172.21.15.46:0/1849573074 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f87640da020 con 0x5633301082e0 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 10 client.7239 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 10 client.7239 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 20 client.7239 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5633300f6e30 0x5633300f7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 4 client.7239 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f8785e77000 1 -- 172.21.15.46:0/1849573074 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56332fea6cd0 con 0x5633300f6e30 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.500+0000 7f877be00640 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5633300f6e30 0x5633300f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f877be00640 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5633300f6e30 0x5633300f7210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dcb3d939d43ef97 server_cookie=9ff4e3340d0ffa4c in_seq=0 out_seq=0 2024-04-25T08:48:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8769600640 10 client.7239 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:48:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8769600640 1 -- 172.21.15.46:0/1849573074 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56332fea6cd0 con 0x5633300f6e30 2024-04-25T08:48:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8769600640 10 client.7239 handle_command_reply: tid=0 2024-04-25T08:48:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 client.7239 shutdown 2024-04-25T08:48:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 -- 172.21.15.46:0/1849573074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 msgr2=0x56333010fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x56333010fb80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f876401c520 tx=0x7f876401ad20 comp rx=0 tx=0).stop 2024-04-25T08:48:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 -- 172.21.15.46:0/1849573074 shutdown_connections 2024-04-25T08:48:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x5633300f6e30 0x5633300f7210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563330108c00 0x5633300f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x56333010fb80 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 --2- 172.21.15.46:0/1849573074 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5633300be8e0 0x56333010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8769600640 0 client.7239 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:48:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 -- 172.21.15.46:0/1849573074 >> 172.21.15.46:0/1849573074 conn(0x5633300ba890 msgr2=0x5633300bd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 -- 172.21.15.46:0/1849573074 shutdown_connections 2024-04-25T08:48:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 1 -- 172.21.15.46:0/1849573074 wait complete. 2024-04-25T08:48:08.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.504+0000 7f8785e77000 20 client.7239 trim_cache size 0 max 16384 2024-04-25T08:48:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8785e77000 1 Processor -- start 2024-04-25T08:48:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8785e77000 1 -- start start 2024-04-25T08:48:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8785e77000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 0x563330164500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8785e77000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330164a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8785e77000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 0x563330164f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8785e77000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c202920 con 0x5633301082e0 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8785e77000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0aeee0 con 0x5633300be8e0 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8785e77000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c202320 con 0x563330108c00 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330164a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49544/0 (socket says 172.21.15.46:49544) 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8773400640 1 -- 172.21.15.46:0/4229245931 learned_addr learned my addr 172.21.15.46:0/4229245931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8780a00640 1 --2- 172.21.15.46:0/4229245931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 0x563330164f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f877be00640 1 --2- 172.21.15.46:0/4229245931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 0x563330164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8773400640 1 -- 172.21.15.46:0/4229245931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 msgr2=0x563330164f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8773400640 1 --2- 172.21.15.46:0/4229245931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 0x563330164f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8773400640 1 -- 172.21.15.46:0/4229245931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 msgr2=0x563330164500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8773400640 1 --2- 172.21.15.46:0/4229245931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 0x563330164500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8773400640 1 -- 172.21.15.46:0/4229245931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56332fec6fc0 con 0x5633301082e0 2024-04-25T08:48:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f8773400640 1 --2- 172.21.15.46:0/4229245931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330164a40 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c10 tx=0x7f876400b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a0697a2458a071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f876be00640 1 -- 172.21.15.46:0/4229245931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640a8070 con 0x5633301082e0 2024-04-25T08:48:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f876be00640 1 -- 172.21.15.46:0/4229245931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87640c6070 con 0x5633301082e0 2024-04-25T08:48:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.516+0000 7f876be00640 1 -- 172.21.15.46:0/4229245931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764004480 con 0x5633301082e0 2024-04-25T08:48:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- 172.21.15.46:0/4229245931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 msgr2=0x563330164a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 --2- 172.21.15.46:0/4229245931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330164a40 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c10 tx=0x7f876400b070 comp rx=0 tx=0).stop 2024-04-25T08:48:08.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- 172.21.15.46:0/4229245931 shutdown_connections 2024-04-25T08:48:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 --2- 172.21.15.46:0/4229245931 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 0x563330164f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 --2- 172.21.15.46:0/4229245931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330164a40 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 --2- 172.21.15.46:0/4229245931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 0x563330164500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- 172.21.15.46:0/4229245931 >> 172.21.15.46:0/4229245931 conn(0x5633300ba890 msgr2=0x563330107290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- 172.21.15.46:0/4229245931 shutdown_connections 2024-04-25T08:48:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- 172.21.15.46:0/4229245931 wait complete. 2024-04-25T08:48:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- start start 2024-04-25T08:48:08.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 Processor -- start 2024-04-25T08:48:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 0x563330168720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 0x56333010c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332fea5e70 con 0x5633301082e0 2024-04-25T08:48:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332fe93a50 con 0x5633300be8e0 2024-04-25T08:48:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x56332ff0a3a0 con 0x563330108c00 2024-04-25T08:48:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 10 client.0 _ll_register_callbacks cb 0x56332feb7240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8773400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330168c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49558/0 (socket says 172.21.15.46:49558) 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8773400640 1 -- 172.21.15.46:0/50744448 learned_addr learned my addr 172.21.15.46:0/50744448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f877be00640 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 0x563330168720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8780a00640 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 0x56333010c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8773400640 1 -- 172.21.15.46:0/50744448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 msgr2=0x56333010c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8773400640 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 0x56333010c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8773400640 1 -- 172.21.15.46:0/50744448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 msgr2=0x563330168720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8773400640 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 0x563330168720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8773400640 1 -- 172.21.15.46:0/50744448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56332ff649e0 con 0x5633301082e0 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f877be00640 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 0x563330168720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8773400640 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330168c60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c10 tx=0x7f87640b9070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eacd7cab0645755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8769600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8769600640 1 -- 172.21.15.46:0/50744448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87640a8040 con 0x5633301082e0 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8769600640 1 -- 172.21.15.46:0/50744448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f87640e4c90 con 0x5633301082e0 2024-04-25T08:48:08.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- 172.21.15.46:0/50744448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87640c6070 con 0x5633301082e0 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8769600640 1 -- 172.21.15.46:0/50744448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8764003df0 con 0x5633301082e0 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 10 client.7245 fetch_fsmap learned FSMap version 53 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 1 -- 172.21.15.46:0/50744448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x56332feaa630 con 0x5633301082e0 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8769600640 1 -- 172.21.15.46:0/50744448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87640c6070 con 0x5633301082e0 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8769600640 1 -- 172.21.15.46:0/50744448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f87640da070 con 0x5633301082e0 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.520+0000 7f8785e77000 10 client.7245 fetch_fsmap finished waiting for FSMap version 53 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 10 client.7245 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 20 client.7245 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x56333010d370 0x56333010d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 4 client.7245 mds_command: new command op to 6990 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 -- 172.21.15.46:0/50744448 --> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x56332ff185d0 con 0x56333010d370 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f877be00640 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x56333010d370 0x56333010d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f877be00640 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x56333010d370 0x56333010d750 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afabc31eac81c3b3 server_cookie=45236cfd9d04874b in_seq=0 out_seq=0 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8769600640 10 client.7245 ms_handle_connect on v2:172.21.15.46:6833/4112568749 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8769600640 1 -- 172.21.15.46:0/50744448 <== mds.0 v2:172.21.15.46:6833/4112568749 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56332ff185d0 con 0x56333010d370 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8769600640 10 client.7245 handle_command_reply: tid=0 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 client.7245 shutdown 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 -- 172.21.15.46:0/50744448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 msgr2=0x563330168c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330168c60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8764000c10 tx=0x7f87640b9070 comp rx=0 tx=0).stop 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 -- 172.21.15.46:0/50744448 shutdown_connections 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.46:6833/4112568749,v1:172.21.15.46:6835/4112568749] conn(0x56333010d370 0x56333010d750 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563330108c00 0x56333010c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633301082e0 0x563330168c60 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 --2- 172.21.15.46:0/50744448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5633300be8e0 0x563330168720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8769600640 0 client.7245 ms_handle_reset on v2:172.21.15.46:6833/4112568749 2024-04-25T08:48:08.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 -- 172.21.15.46:0/50744448 >> 172.21.15.46:0/50744448 conn(0x5633300ba890 msgr2=0x563330107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 -- 172.21.15.46:0/50744448 shutdown_connections 2024-04-25T08:48:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 1 -- 172.21.15.46:0/50744448 wait complete. 2024-04-25T08:48:08.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8785e77000 20 client.7245 trim_cache size 0 max 16384 2024-04-25T08:48:08.523 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:08.523 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 -- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8758066f60 msgr2=0x7f8758069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8758066f60 0x7f8758069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f87640ae000 tx=0x7f87640a7000 comp rx=0 tx=0).stop 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 -- 172.21.15.46:0/1571867823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c099c60 msgr2=0x7f877c0c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c099c60 0x7f877c0c6030 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f876c00ce00 tx=0x7f876c00a650 comp rx=0 tx=0).stop 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 -- 172.21.15.46:0/1571867823 shutdown_connections 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8758066f60 0x7f8758069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f877c0cf210 0x7f877c0c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f877c09a040 0x7f877c157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 --2- 172.21.15.46:0/1571867823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f877c099c60 0x7f877c0c6030 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 -- 172.21.15.46:0/1571867823 >> 172.21.15.46:0/1571867823 conn(0x7f877c0b3130 msgr2=0x7f877c0cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 -- 172.21.15.46:0/1571867823 shutdown_connections 2024-04-25T08:48:08.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.524+0000 7f8782a00640 1 -- 172.21.15.46:0/1571867823 wait complete. 2024-04-25T08:48:08.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:48:08.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8911c00640 1 Processor -- start 2024-04-25T08:48:08.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8911c00640 1 -- start start 2024-04-25T08:48:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8911c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f890c06bd20 0x7f890c06c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8911c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890c0cc520 con 0x7f890c0cd2e0 2024-04-25T08:48:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8911c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890c0cc800 con 0x7f890c161080 2024-04-25T08:48:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8911c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f890c0cc2f0 con 0x7f890c06bd20 2024-04-25T08:48:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8902000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f890c06bd20 0x7f890c06c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f890b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f890c0cd2e0 0x7f890c099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35916/0 (socket says 172.21.15.46:35916) 2024-04-25T08:48:08.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f890b400640 1 -- 172.21.15.46:0/3483436344 learned_addr learned my addr 172.21.15.46:0/3483436344 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:08.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8902000640 1 -- 172.21.15.46:0/3483436344 >> v1:172.21.15.82:6789/0 conn(0x7f890c161080 legacy=0x7f890c0cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8902000640 1 -- 172.21.15.46:0/3483436344 >> v1:172.21.15.46:6789/0 conn(0x7f890c0cd2e0 legacy=0x7f890c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f8902000640 1 -- 172.21.15.46:0/3483436344 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c0ccae0 con 0x7f890c06bd20 2024-04-25T08:48:08.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.684+0000 7f890a000640 1 -- 172.21.15.46:0/3483436344 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319130896 0 0) 0x7f890c0cc520 con 0x7f890c0cd2e0 2024-04-25T08:48:08.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f8902000640 1 --2- 172.21.15.46:0/3483436344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c06bd20 0x7f890c06c160 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f88f40097c0 tx=0x7f88f401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bf9eb2439a939a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f890a000640 1 -- 172.21.15.46:0/3483436344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40b1070 con 0x7f890c06bd20 2024-04-25T08:48:08.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f890a000640 1 -- 172.21.15.46:0/3483436344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f40ac410 con 0x7f890c06bd20 2024-04-25T08:48:08.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f890a000640 1 -- 172.21.15.46:0/3483436344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40ac7e0 con 0x7f890c06bd20 2024-04-25T08:48:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f8911c00640 1 -- 172.21.15.46:0/3483436344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c06bd20 msgr2=0x7f890c06c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f8911c00640 1 --2- 172.21.15.46:0/3483436344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c06bd20 0x7f890c06c160 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f88f40097c0 tx=0x7f88f401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f8911c00640 1 -- 172.21.15.46:0/3483436344 shutdown_connections 2024-04-25T08:48:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f8911c00640 1 --2- 172.21.15.46:0/3483436344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c06bd20 0x7f890c06c160 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f8911c00640 1 -- 172.21.15.46:0/3483436344 >> 172.21.15.46:0/3483436344 conn(0x7f890c0b31d0 msgr2=0x7f890c0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f8911c00640 1 -- 172.21.15.46:0/3483436344 shutdown_connections 2024-04-25T08:48:08.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.688+0000 7f8911c00640 1 -- 172.21.15.46:0/3483436344 wait complete. 2024-04-25T08:48:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 Processor -- start 2024-04-25T08:48:08.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 -- start start 2024-04-25T08:48:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f890c06bd20 0x7f890c14c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f890c0cd2e0 0x7f890c14b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c161080 0x7f890c1510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c0b29b0 con 0x7f890c161080 2024-04-25T08:48:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c0b2830 con 0x7f890c0cd2e0 2024-04-25T08:48:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c09a110 con 0x7f890c06bd20 2024-04-25T08:48:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c161080 0x7f890c1510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c161080 0x7f890c1510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49586/0 (socket says 172.21.15.46:49586) 2024-04-25T08:48:08.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890b400640 1 -- 172.21.15.46:0/3789817618 learned_addr learned my addr 172.21.15.46:0/3789817618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8902000640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f890c0cd2e0 0x7f890c14b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890aa00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f890c06bd20 0x7f890c14c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890b400640 1 -- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f890c06bd20 msgr2=0x7f890c14c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890b400640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f890c06bd20 0x7f890c14c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890b400640 1 -- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f890c0cd2e0 msgr2=0x7f890c14b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890b400640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f890c0cd2e0 0x7f890c14b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890b400640 1 -- 172.21.15.46:0/3789817618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c151600 con 0x7f890c161080 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890aa00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f890c06bd20 0x7f890c14c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f890b400640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c161080 0x7f890c1510c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f890400c7a0 tx=0x7f890400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8903400640 1 -- 172.21.15.46:0/3789817618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904018070 con 0x7f890c161080 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8903400640 1 -- 172.21.15.46:0/3789817618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890400a640 con 0x7f890c161080 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8903400640 1 -- 172.21.15.46:0/3789817618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400aa70 con 0x7f890c161080 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890c14ca20 con 0x7f890c161080 2024-04-25T08:48:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.692+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890c14cfe0 con 0x7f890c161080 2024-04-25T08:48:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.696+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890c14d7e0 con 0x7f890c161080 2024-04-25T08:48:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.696+0000 7f8903400640 1 -- 172.21.15.46:0/3789817618 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f890401b070 con 0x7f890c161080 2024-04-25T08:48:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.696+0000 7f8903400640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88e8066f60 0x7f88e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.696+0000 7f8903400640 1 -- 172.21.15.46:0/3789817618 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8904084750 con 0x7f890c161080 2024-04-25T08:48:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.696+0000 7f890aa00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88e8066f60 0x7f88e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:08.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.696+0000 7f890aa00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88e8066f60 0x7f88e8069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f88fc000f70 tx=0x7f88fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:08.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.700+0000 7f8903400640 1 -- 172.21.15.46:0/3789817618 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8904055610 con 0x7f890c161080 2024-04-25T08:48:08.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f890c074c60 con 0x7f890c161080 2024-04-25T08:48:08.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8903400640 1 -- 172.21.15.46:0/3789817618 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8904054f10 con 0x7f890c161080 2024-04-25T08:48:08.891 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:08.891 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88e8066f60 msgr2=0x7f88e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88e8066f60 0x7f88e8069420 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f88fc000f70 tx=0x7f88fc009040 comp rx=0 tx=0).stop 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c161080 msgr2=0x7f890c1510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c161080 0x7f890c1510c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f890400c7a0 tx=0x7f890400cc60 comp rx=0 tx=0).stop 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 shutdown_connections 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88e8066f60 0x7f88e8069420 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f890c161080 0x7f890c1510c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f890c0cd2e0 0x7f890c14b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 --2- 172.21.15.46:0/3789817618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f890c06bd20 0x7f890c14c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 >> 172.21.15.46:0/3789817618 conn(0x7f890c0b31d0 msgr2=0x7f890c0cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:08.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 shutdown_connections 2024-04-25T08:48:08.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:08.896+0000 7f8911c00640 1 -- 172.21.15.46:0/3789817618 wait complete. 2024-04-25T08:48:08.911 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:48:08.912 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:48:09.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.056+0000 7f69fde00640 1 Processor -- start 2024-04-25T08:48:09.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.056+0000 7f69fde00640 1 -- start start 2024-04-25T08:48:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69fde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69f815a750 0x7f69f806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69fde00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f80cc9e0 0x7f69f806f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69fde00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f806d5e0 con 0x7f69f80aed10 2024-04-25T08:48:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69fde00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f815ab30 con 0x7f69f815a750 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69fde00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f80ce4f0 con 0x7f69f80cc9e0 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69f6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69f815a750 0x7f69f806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69f6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69f815a750 0x7f69f806ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49588/0 (socket says 172.21.15.46:49588) 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69f6a00640 1 -- 172.21.15.46:0/628261694 learned_addr learned my addr 172.21.15.46:0/628261694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69efe00640 1 --1- 172.21.15.46:0/628261694 >> v1:172.21.15.46:6789/0 conn(0x7f69f80aed10 0x7f69f80b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35924/0 (socket says 172.21.15.46:35924) 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69f7400640 1 --2- 172.21.15.46:0/628261694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f80cc9e0 0x7f69f806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69f6a00640 1 -- 172.21.15.46:0/628261694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f80cc9e0 msgr2=0x7f69f806f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69f6a00640 1 --2- 172.21.15.46:0/628261694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f80cc9e0 0x7f69f806f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69f6a00640 1 -- 172.21.15.46:0/628261694 >> v1:172.21.15.46:6789/0 conn(0x7f69f80aed10 legacy=0x7f69f80b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69f6a00640 1 -- 172.21.15.46:0/628261694 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f80ce720 con 0x7f69f815a750 2024-04-25T08:48:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.060+0000 7f69f6000640 1 -- 172.21.15.46:0/628261694 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735728247 0 0) 0x7f69f806d5e0 con 0x7f69f80aed10 2024-04-25T08:48:09.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69f6a00640 1 --2- 172.21.15.46:0/628261694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f815a750 0x7f69f806ec10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009870 tx=0x7f69e801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a0df6d1ad8bbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:09.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69f6000640 1 -- 172.21.15.46:0/628261694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e80ab650 con 0x7f69f815a750 2024-04-25T08:48:09.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69f6000640 1 -- 172.21.15.46:0/628261694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e80b0070 con 0x7f69f815a750 2024-04-25T08:48:09.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69f6000640 1 -- 172.21.15.46:0/628261694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e80aba10 con 0x7f69f815a750 2024-04-25T08:48:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 -- 172.21.15.46:0/628261694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f815a750 msgr2=0x7f69f806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 --2- 172.21.15.46:0/628261694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f815a750 0x7f69f806ec10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009870 tx=0x7f69e801c560 comp rx=0 tx=0).stop 2024-04-25T08:48:09.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 -- 172.21.15.46:0/628261694 shutdown_connections 2024-04-25T08:48:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 --2- 172.21.15.46:0/628261694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f80cc9e0 0x7f69f806f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 --2- 172.21.15.46:0/628261694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f815a750 0x7f69f806ec10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 -- 172.21.15.46:0/628261694 >> 172.21.15.46:0/628261694 conn(0x7f69f80b12a0 msgr2=0x7f69f80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 -- 172.21.15.46:0/628261694 shutdown_connections 2024-04-25T08:48:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 -- 172.21.15.46:0/628261694 wait complete. 2024-04-25T08:48:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 Processor -- start 2024-04-25T08:48:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.064+0000 7f69fde00640 1 -- start start 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69fde00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f80aed10 0x7f69f8149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69fde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f80cc9e0 0x7f69f8149ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69fde00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f815a750 0x7f69f815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69fde00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f806db00 con 0x7f69f80cc9e0 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69fde00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f806d270 con 0x7f69f80aed10 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69fde00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f806dc80 con 0x7f69f815a750 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f80cc9e0 0x7f69f8149ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f80cc9e0 0x7f69f8149ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49600/0 (socket says 172.21.15.46:49600) 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69efe00640 1 -- 172.21.15.46:0/1624780195 learned_addr learned my addr 172.21.15.46:0/1624780195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69f6a00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f80aed10 0x7f69f8149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69f7400640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f815a750 0x7f69f815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69efe00640 1 -- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f815a750 msgr2=0x7f69f815acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69efe00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f815a750 0x7f69f815acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69efe00640 1 -- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f80aed10 msgr2=0x7f69f8149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69efe00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f80aed10 0x7f69f8149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69efe00640 1 -- 172.21.15.46:0/1624780195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f81500b0 con 0x7f69f80cc9e0 2024-04-25T08:48:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69efe00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f80cc9e0 0x7f69f8149ab0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f69e000c4b0 tx=0x7f69e000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69ef400640 1 -- 172.21.15.46:0/1624780195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0018070 con 0x7f69f80cc9e0 2024-04-25T08:48:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69ef400640 1 -- 172.21.15.46:0/1624780195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e0007500 con 0x7f69f80cc9e0 2024-04-25T08:48:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69ef400640 1 -- 172.21.15.46:0/1624780195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0004a30 con 0x7f69f80cc9e0 2024-04-25T08:48:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f814fdd0 con 0x7f69f80cc9e0 2024-04-25T08:48:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f814a660 con 0x7f69f80cc9e0 2024-04-25T08:48:09.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f8069080 con 0x7f69f80cc9e0 2024-04-25T08:48:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.068+0000 7f69ef400640 1 -- 172.21.15.46:0/1624780195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69e0004bd0 con 0x7f69f80cc9e0 2024-04-25T08:48:09.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.072+0000 7f69ef400640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69d4066f60 0x7f69d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.072+0000 7f69ef400640 1 -- 172.21.15.46:0/1624780195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f69e00844d0 con 0x7f69f80cc9e0 2024-04-25T08:48:09.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.072+0000 7f69f6a00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69d4066f60 0x7f69d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:09.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.072+0000 7f69f6a00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69d4066f60 0x7f69d4069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f69e80af000 tx=0x7f69e80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:09.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.076+0000 7f69ef400640 1 -- 172.21.15.46:0/1624780195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69e0054d60 con 0x7f69f80cc9e0 2024-04-25T08:48:09.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.272+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f69f8153c00 con 0x7f69f80cc9e0 2024-04-25T08:48:09.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69ef400640 1 -- 172.21.15.46:0/1624780195 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f69e0058e00 con 0x7f69f80cc9e0 2024-04-25T08:48:09.269 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:48:09.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69d4066f60 msgr2=0x7f69d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:09.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69d4066f60 0x7f69d4069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f69e80af000 tx=0x7f69e80a7000 comp rx=0 tx=0).stop 2024-04-25T08:48:09.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f80cc9e0 msgr2=0x7f69f8149ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:09.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f80cc9e0 0x7f69f8149ab0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f69e000c4b0 tx=0x7f69e000c970 comp rx=0 tx=0).stop 2024-04-25T08:48:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 shutdown_connections 2024-04-25T08:48:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f69d4066f60 0x7f69d4069420 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f69f815a750 0x7f69f815acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69f80cc9e0 0x7f69f8149ab0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 --2- 172.21.15.46:0/1624780195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69f80aed10 0x7f69f8149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 >> 172.21.15.46:0/1624780195 conn(0x7f69f80b12a0 msgr2=0x7f69f80acbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 shutdown_connections 2024-04-25T08:48:09.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:09.276+0000 7f69fde00640 1 -- 172.21.15.46:0/1624780195 wait complete. 2024-04-25T08:48:09.290 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:48:09.290 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:48:09.291 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:48:09.291 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:48:09.291 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:09.291 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:48:09.291 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:48:09.291 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:48:09.291 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:48:09.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.292 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:48:09.301 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:48:09.301 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:48:09.302 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42113sec preferred_lft 42113sec 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:48:09.303 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:09.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.305 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:48:09.366 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:48:09.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.368 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:48:09.425 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:48:09.425 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:48:09.425 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:48:09.425 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:48:09.425 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:48:09.426 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:48:09.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.426 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:48:09.426 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:48:09.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:48:09.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:48:09.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:48:09.427 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:48:09.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.519 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:48:09.528 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:48:09.528 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:48:09.528 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:48:09.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.528 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:48:09.592 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:48:09.592 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:09.649 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:48:09.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:48:09.650 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:09.700 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:48:09.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:48:09.790 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:48:09.805 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:09.806 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:48:09.806 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:48:09.806 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:48:09.806 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:48:09.816 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:09.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:48:09.816 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:48:09.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:48:09.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:48:09.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:48:09.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:48:09.816 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:48:09.820 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:48:09.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.820 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:09.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.840 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:48:09.891 INFO:teuthology.orchestra.run.smithi046.stdout:client6320 2024-04-25T08:48:09.891 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:09.892 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:09.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:09.940 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T08:48:09.994 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:48:09.994 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T08:48:10.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 Processor -- start 2024-04-25T08:48:10.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- start start 2024-04-25T08:48:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d45f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb6d3c3d30 0x55bb6d4620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bb6d467680 0x55bb6d462600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6d467df0 con 0x55bb6d391a20 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6d467c70 con 0x55bb6d3c3d30 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb6d467f70 con 0x55bb6d467680 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d45f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d45f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49616/0 (socket says 172.21.15.46:49616) 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114200640 1 -- 172.21.15.46:0/1728982430 learned_addr learned my addr 172.21.15.46:0/1728982430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114c00640 1 --2- 172.21.15.46:0/1728982430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bb6d467680 0x55bb6d462600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3113800640 1 --2- 172.21.15.46:0/1728982430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb6d3c3d30 0x55bb6d4620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114200640 1 -- 172.21.15.46:0/1728982430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bb6d467680 msgr2=0x55bb6d462600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114200640 1 --2- 172.21.15.46:0/1728982430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bb6d467680 0x55bb6d462600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114200640 1 -- 172.21.15.46:0/1728982430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb6d3c3d30 msgr2=0x55bb6d4620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114200640 1 --2- 172.21.15.46:0/1728982430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb6d3c3d30 0x55bb6d4620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114200640 1 -- 172.21.15.46:0/1728982430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb6d45fec0 con 0x55bb6d391a20 2024-04-25T08:48:10.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3114200640 1 --2- 172.21.15.46:0/1728982430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d45f980 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f31040097d0 tx=0x7f3104005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb70dea770ccd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3112400640 1 -- 172.21.15.46:0/1728982430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104019070 con 0x55bb6d391a20 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3112400640 1 -- 172.21.15.46:0/1728982430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3104008da0 con 0x55bb6d391a20 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3112400640 1 -- 172.21.15.46:0/1728982430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31040146a0 con 0x55bb6d391a20 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- 172.21.15.46:0/1728982430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 msgr2=0x55bb6d45f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 --2- 172.21.15.46:0/1728982430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d45f980 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f31040097d0 tx=0x7f3104005a10 comp rx=0 tx=0).stop 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- 172.21.15.46:0/1728982430 shutdown_connections 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 --2- 172.21.15.46:0/1728982430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bb6d467680 0x55bb6d462600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 --2- 172.21.15.46:0/1728982430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb6d3c3d30 0x55bb6d4620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 --2- 172.21.15.46:0/1728982430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d45f980 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- 172.21.15.46:0/1728982430 >> 172.21.15.46:0/1728982430 conn(0x55bb6d38fdd0 msgr2=0x55bb6d466390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- 172.21.15.46:0/1728982430 shutdown_connections 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- 172.21.15.46:0/1728982430 wait complete. 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 Processor -- start 2024-04-25T08:48:10.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.096+0000 7f3117013080 1 -- start start 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3117013080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d3d9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3117013080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bb6d3c3d30 0x55bb6d44a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3117013080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb6d467680 0x55bb6d44a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3117013080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6d3dc880 con 0x55bb6d391a20 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3117013080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6d3c2c50 con 0x55bb6d467680 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3117013080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb6d3be490 con 0x55bb6d3c3d30 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d3d9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d3d9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49620/0 (socket says 172.21.15.46:49620) 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114200640 1 -- 172.21.15.46:0/3293484564 learned_addr learned my addr 172.21.15.46:0/3293484564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114c00640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb6d467680 0x55bb6d44a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3113800640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bb6d3c3d30 0x55bb6d44a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114200640 1 -- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bb6d3c3d30 msgr2=0x55bb6d44a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114200640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55bb6d3c3d30 0x55bb6d44a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114200640 1 -- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb6d467680 msgr2=0x55bb6d44a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114200640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bb6d467680 0x55bb6d44a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114200640 1 -- 172.21.15.46:0/3293484564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb6d3d8950 con 0x55bb6d391a20 2024-04-25T08:48:10.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3114200640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d3d9870 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3104000c10 tx=0x7f3104003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3111000640 1 -- 172.21.15.46:0/3293484564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104023020 con 0x55bb6d391a20 2024-04-25T08:48:10.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3111000640 1 -- 172.21.15.46:0/3293484564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310401a040 con 0x55bb6d391a20 2024-04-25T08:48:10.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3111000640 1 -- 172.21.15.46:0/3293484564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104019070 con 0x55bb6d391a20 2024-04-25T08:48:10.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bb6d3d8bf0 con 0x55bb6d391a20 2024-04-25T08:48:10.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb6d3d9130 con 0x55bb6d391a20 2024-04-25T08:48:10.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.100+0000 7f3111000640 1 -- 172.21.15.46:0/3293484564 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3104026020 con 0x55bb6d391a20 2024-04-25T08:48:10.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3111000640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30f4065970 0x7f30f4067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3111000640 1 -- 172.21.15.46:0/3293484564 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3104085b50 con 0x55bb6d391a20 2024-04-25T08:48:10.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55bb6d448500 0x55bb6d4488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3113800640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30f4065970 0x7f30f4067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e79) v9 -- 0x55bb6d448e20 con 0x55bb6d448500 2024-04-25T08:48:10.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3114c00640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55bb6d448500 0x55bb6d4488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3114c00640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55bb6d448500 0x55bb6d4488e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3113800640 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30f4065970 0x7f30f4067e30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f31000ac000 tx=0x7f31000a5000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3114c00640 1 -- 172.21.15.46:0/3293484564 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v79'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f310c002b10 con 0x55bb6d448500 2024-04-25T08:48:10.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55bb6d448500 msgr2=0x55bb6d4488e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55bb6d448500 0x55bb6d4488e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30f4065970 msgr2=0x7f30f4067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30f4065970 0x7f30f4067e30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f31000ac000 tx=0x7f31000a5000 comp rx=0 tx=0).stop 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 msgr2=0x55bb6d3d9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 --2- 172.21.15.46:0/3293484564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb6d391a20 0x55bb6d3d9870 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3104000c10 tx=0x7f3104003030 comp rx=0 tx=0).stop 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3114c00640 1 -- 172.21.15.46:0/3293484564 reap_dead start 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 shutdown_connections 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 >> 172.21.15.46:0/3293484564 conn(0x55bb6d38fdd0 msgr2=0x55bb6d4658b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 shutdown_connections 2024-04-25T08:48:10.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.104+0000 7f3117013080 1 -- 172.21.15.46:0/3293484564 wait complete. 2024-04-25T08:48:10.105 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:48:10.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:48:10.183 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:48:10.281 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:48:10.282 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:10.282 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:10.291 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:10.291 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:10.345 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7aa600640 1 Processor -- start 2024-04-25T08:48:10.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7aa600640 1 -- start start 2024-04-25T08:48:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7aa600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7a406ac10 0x7fd7a406b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7aa600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a40b20d0 con 0x7fd7a415b5d0 2024-04-25T08:48:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7aa600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a415e010 con 0x7fd7a41467a0 2024-04-25T08:48:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7aa600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a415ec20 con 0x7fd7a406ac10 2024-04-25T08:48:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd7a415b5d0 0x7fd7a40b19c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35932/0 (socket says 172.21.15.46:35932) 2024-04-25T08:48:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a3400640 1 -- 172.21.15.46:0/1216595816 learned_addr learned my addr 172.21.15.46:0/1216595816 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd79aa00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7a406ac10 0x7fd7a406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996557363 0 0) 0x7fd7a40b20d0 con 0x7fd7a415b5d0 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd788003530 con 0x7fd7a415b5d0 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3973905373 0 0) 0x7fd7a415e010 con 0x7fd7a41467a0 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a40b20d0 con 0x7fd7a41467a0 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1542661335 0 0) 0x7fd788003530 con 0x7fd7a415b5d0 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7a406ac10 msgr2=0x7fd7a406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7a406ac10 0x7fd7a406b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 >> v1:172.21.15.82:6789/0 conn(0x7fd7a41467a0 legacy=0x7fd7a4146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a415e2f0 con 0x7fd7a415b5d0 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fd794001680 con 0x7fd7a415b5d0 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 >> v1:172.21.15.46:6789/0 conn(0x7fd7a415b5d0 legacy=0x7fd7a40b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd788003b80 0x7fd788003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7a406ac10 0x7fd788004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd788004d50 0x7fd7880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a415ec20 con 0x7fd788004d50 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78800ba80 con 0x7fd788003b80 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78800bc30 con 0x7fd7a406ac10 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a3e00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd788004d50 0x7fd7880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd79aa00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7a406ac10 0x7fd788004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.536+0000 7fd7a3400640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd788003b80 0x7fd788003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a3e00640 1 -- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7a406ac10 msgr2=0x7fd788004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a3e00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd7a406ac10 0x7fd788004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a3e00640 1 -- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd788003b80 msgr2=0x7fd788003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a3e00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd788003b80 0x7fd788003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a3e00640 1 -- 172.21.15.46:0/1216595816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd788003530 con 0x7fd788004d50 2024-04-25T08:48:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a3e00640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd788004d50 0x7fd7880051d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd79c001fd0 tx=0x7fd79c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfc24497d756e49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c001c20 con 0x7fd788004d50 2024-04-25T08:48:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c003ba0 con 0x7fd788004d50 2024-04-25T08:48:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a2a00640 1 -- 172.21.15.46:0/1216595816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0050d0 con 0x7fd788004d50 2024-04-25T08:48:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7aa600640 1 -- 172.21.15.46:0/1216595816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd788004d50 msgr2=0x7fd7880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7aa600640 1 --2- 172.21.15.46:0/1216595816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd788004d50 0x7fd7880051d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd79c001fd0 tx=0x7fd79c01c900 comp rx=0 tx=0).stop 2024-04-25T08:48:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7a3e00640 1 -- 172.21.15.46:0/1216595816 reap_dead start 2024-04-25T08:48:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7aa600640 1 -- 172.21.15.46:0/1216595816 shutdown_connections 2024-04-25T08:48:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7aa600640 1 -- 172.21.15.46:0/1216595816 >> 172.21.15.46:0/1216595816 conn(0x7fd7a4072880 msgr2=0x7fd7a4099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7aa600640 1 -- 172.21.15.46:0/1216595816 shutdown_connections 2024-04-25T08:48:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.540+0000 7fd7aa600640 1 -- 172.21.15.46:0/1216595816 wait complete. 2024-04-25T08:48:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7aa600640 1 Processor -- start 2024-04-25T08:48:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7aa600640 1 -- start start 2024-04-25T08:48:10.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7aa600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd788004d50 0x7fd7a413f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7aa600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7a409b900 0x7fd7a409aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7aa600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a409b0e0 0x7fd7a41f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7aa600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40b3730 con 0x7fd7a409b0e0 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7aa600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a415e8b0 con 0x7fd7a409b900 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7aa600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40b38b0 con 0x7fd788004d50 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a409b0e0 0x7fd7a41f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a409b0e0 0x7fd7a41f3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49634/0 (socket says 172.21.15.46:49634) 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3e00640 1 -- 172.21.15.46:0/4286758396 learned_addr learned my addr 172.21.15.46:0/4286758396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3400640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd788004d50 0x7fd7a413f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd79aa00640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7a409b900 0x7fd7a409aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3e00640 1 -- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd788004d50 msgr2=0x7fd7a413f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3e00640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd788004d50 0x7fd7a413f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3e00640 1 -- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7a409b900 msgr2=0x7fd7a409aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3e00640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7a409b900 0x7fd7a409aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3e00640 1 -- 172.21.15.46:0/4286758396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a4140e40 con 0x7fd7a409b0e0 2024-04-25T08:48:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7a3e00640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a409b0e0 0x7fd7a41f3610 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd79c001fd0 tx=0x7fd788005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd79be00640 1 -- 172.21.15.46:0/4286758396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c001c00 con 0x7fd7a409b0e0 2024-04-25T08:48:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd79be00640 1 -- 172.21.15.46:0/4286758396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c0bc070 con 0x7fd7a409b0e0 2024-04-25T08:48:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.544+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a41410e0 con 0x7fd7a409b0e0 2024-04-25T08:48:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.548+0000 7fd79be00640 1 -- 172.21.15.46:0/4286758396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c003d50 con 0x7fd7a409b0e0 2024-04-25T08:48:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.548+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a41416a0 con 0x7fd7a409b0e0 2024-04-25T08:48:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.548+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a406ccb0 con 0x7fd7a409b0e0 2024-04-25T08:48:10.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.548+0000 7fd79be00640 1 -- 172.21.15.46:0/4286758396 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd79c0b5070 con 0x7fd7a409b0e0 2024-04-25T08:48:10.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.548+0000 7fd79be00640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd780066f90 0x7fd780069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.548+0000 7fd79be00640 1 -- 172.21.15.46:0/4286758396 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd79c11efd0 con 0x7fd7a409b0e0 2024-04-25T08:48:10.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.548+0000 7fd7a3400640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd780066f90 0x7fd780069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.552+0000 7fd7a3400640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd780066f90 0x7fd780069450 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd794001060 tx=0x7fd794001e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.552+0000 7fd79be00640 1 -- 172.21.15.46:0/4286758396 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd79c016610 con 0x7fd7a409b0e0 2024-04-25T08:48:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.716+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7a4069080 con 0x7fd7a409b0e0 2024-04-25T08:48:10.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd79be00640 1 -- 172.21.15.46:0/4286758396 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17024 (secure 0 0 0) 0x7fd79c0efe80 con 0x7fd7a409b0e0 2024-04-25T08:48:10.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:10.713 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:56.883035+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:46:44.505856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:46:45.072095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/960745191":"2024-04-26T08:46:56.883011+0000","172.21.15.46:6833/960745191":"2024-04-26T08:46:56.883011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd780066f90 msgr2=0x7fd780069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd780066f90 0x7fd780069450 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd794001060 tx=0x7fd794001e30 comp rx=0 tx=0).stop 2024-04-25T08:48:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a409b0e0 msgr2=0x7fd7a41f3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a409b0e0 0x7fd7a41f3610 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd79c001fd0 tx=0x7fd788005840 comp rx=0 tx=0).stop 2024-04-25T08:48:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 shutdown_connections 2024-04-25T08:48:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd780066f90 0x7fd780069450 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a409b0e0 0x7fd7a41f3610 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7a409b900 0x7fd7a409aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 --2- 172.21.15.46:0/4286758396 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd788004d50 0x7fd7a413f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 >> 172.21.15.46:0/4286758396 conn(0x7fd7a4072880 msgr2=0x7fd7a406ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 shutdown_connections 2024-04-25T08:48:10.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.720+0000 7fd7aa600640 1 -- 172.21.15.46:0/4286758396 wait complete. 2024-04-25T08:48:10.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:46:44.505856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:46:45.072095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:48:10.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:48:10.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910fa00640 1 Processor -- start 2024-04-25T08:48:10.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910fa00640 1 -- start start 2024-04-25T08:48:10.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910fa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f910806f150 0x7f91080cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910fa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910815e440 con 0x7f9108143990 2024-04-25T08:48:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910fa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910815e6e0 con 0x7f91081440d0 2024-04-25T08:48:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910fa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910815ed30 con 0x7f910806f150 2024-04-25T08:48:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910d000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9108143990 0x7f910806ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35944/0 (socket says 172.21.15.46:35944) 2024-04-25T08:48:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910d000640 1 -- 172.21.15.46:0/2200785599 learned_addr learned my addr 172.21.15.46:0/2200785599 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:10.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910da00640 1 --2- 172.21.15.46:0/2200785599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f910806f150 0x7f91080cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f9107e00640 1 -- 172.21.15.46:0/2200785599 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898869292 0 0) 0x7f910815e440 con 0x7f9108143990 2024-04-25T08:48:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f9107e00640 1 -- 172.21.15.46:0/2200785599 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90ec003530 con 0x7f9108143990 2024-04-25T08:48:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910da00640 1 -- 172.21.15.46:0/2200785599 >> v1:172.21.15.82:6789/0 conn(0x7f91081440d0 legacy=0x7f91080b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910da00640 1 -- 172.21.15.46:0/2200785599 >> v1:172.21.15.46:6789/0 conn(0x7f9108143990 legacy=0x7f910806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.884+0000 7f910da00640 1 -- 172.21.15.46:0/2200785599 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910815e9c0 con 0x7f910806f150 2024-04-25T08:48:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f910da00640 1 --2- 172.21.15.46:0/2200785599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f910806f150 0x7f91080cecc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f910815eeb0 tx=0x7f910001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ba2c82769b6407 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f9107e00640 1 -- 172.21.15.46:0/2200785599 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000b2070 con 0x7f910806f150 2024-04-25T08:48:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f9107e00640 1 -- 172.21.15.46:0/2200785599 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91000a5850 con 0x7f910806f150 2024-04-25T08:48:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f9107e00640 1 -- 172.21.15.46:0/2200785599 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000a5c00 con 0x7f910806f150 2024-04-25T08:48:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f910fa00640 1 -- 172.21.15.46:0/2200785599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f910806f150 msgr2=0x7f91080cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f910fa00640 1 --2- 172.21.15.46:0/2200785599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f910806f150 0x7f91080cecc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f910815eeb0 tx=0x7f910001c990 comp rx=0 tx=0).stop 2024-04-25T08:48:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f910fa00640 1 -- 172.21.15.46:0/2200785599 shutdown_connections 2024-04-25T08:48:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f910fa00640 1 --2- 172.21.15.46:0/2200785599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f910806f150 0x7f91080cecc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f910fa00640 1 -- 172.21.15.46:0/2200785599 >> 172.21.15.46:0/2200785599 conn(0x7f910806ac10 msgr2=0x7f910806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f910fa00640 1 -- 172.21.15.46:0/2200785599 shutdown_connections 2024-04-25T08:48:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.888+0000 7f910fa00640 1 -- 172.21.15.46:0/2200785599 wait complete. 2024-04-25T08:48:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 Processor -- start 2024-04-25T08:48:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 -- start start 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f910806f150 0x7f91080af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9108143990 0x7f9108135fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91081440d0 0x7f91081f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91081686a0 con 0x7f91081440d0 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9108168520 con 0x7f910806f150 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9108168820 con 0x7f9108143990 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91081440d0 0x7f91081f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91081440d0 0x7f91081f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49636/0 (socket says 172.21.15.46:49636) 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910da00640 1 -- 172.21.15.46:0/2225028612 learned_addr learned my addr 172.21.15.46:0/2225028612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:10.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f90ffe00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9108143990 0x7f9108135fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910d000640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f910806f150 0x7f91080af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910da00640 1 -- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9108143990 msgr2=0x7f9108135fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910da00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9108143990 0x7f9108135fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910da00640 1 -- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f910806f150 msgr2=0x7f91080af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910da00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f910806f150 0x7f91080af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910da00640 1 -- 172.21.15.46:0/2225028612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910815da60 con 0x7f91081440d0 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910da00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91081440d0 0x7f91081f6a00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f91000b0040 tx=0x7f9100019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f9105600640 1 -- 172.21.15.46:0/2225028612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000bd020 con 0x7f91081440d0 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f9105600640 1 -- 172.21.15.46:0/2225028612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100003ba0 con 0x7f91081440d0 2024-04-25T08:48:10.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9108136700 con 0x7f91081440d0 2024-04-25T08:48:10.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f9105600640 1 -- 172.21.15.46:0/2225028612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000b2040 con 0x7f91081440d0 2024-04-25T08:48:10.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.892+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9108136cc0 con 0x7f91081440d0 2024-04-25T08:48:10.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.896+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9108069010 con 0x7f91081440d0 2024-04-25T08:48:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.896+0000 7f9105600640 1 -- 172.21.15.46:0/2225028612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91000c0020 con 0x7f91081440d0 2024-04-25T08:48:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.896+0000 7f9105600640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f90e4066f90 0x7f90e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.896+0000 7f9105600640 1 -- 172.21.15.46:0/2225028612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f910011fd80 con 0x7f91081440d0 2024-04-25T08:48:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.896+0000 7f910d000640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f90e4066f90 0x7f90e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:10.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.896+0000 7f910d000640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f90e4066f90 0x7f90e4069450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f90f80011f0 tx=0x7f90f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:10.900+0000 7f9105600640 1 -- 172.21.15.46:0/2225028612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9100123030 con 0x7f91081440d0 2024-04-25T08:48:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f910813cb10 con 0x7f91081440d0 2024-04-25T08:48:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f9105600640 1 -- 172.21.15.46:0/2225028612 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91000c7070 con 0x7f91081440d0 2024-04-25T08:48:11.058 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:48:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f90e4066f90 msgr2=0x7f90e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f90e4066f90 0x7f90e4069450 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f90f80011f0 tx=0x7f90f8009040 comp rx=0 tx=0).stop 2024-04-25T08:48:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91081440d0 msgr2=0x7f91081f6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91081440d0 0x7f91081f6a00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f91000b0040 tx=0x7f9100019730 comp rx=0 tx=0).stop 2024-04-25T08:48:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 shutdown_connections 2024-04-25T08:48:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f90e4066f90 0x7f90e4069450 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91081440d0 0x7f91081f6a00 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9108143990 0x7f9108135fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 --2- 172.21.15.46:0/2225028612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f910806f150 0x7f91080af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 >> 172.21.15.46:0/2225028612 conn(0x7f910806ac10 msgr2=0x7f910815b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 shutdown_connections 2024-04-25T08:48:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.064+0000 7f910fa00640 1 -- 172.21.15.46:0/2225028612 wait complete. 2024-04-25T08:48:11.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:48:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.228+0000 7f102b000640 1 Processor -- start 2024-04-25T08:48:11.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.228+0000 7f102b000640 1 -- start start 2024-04-25T08:48:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f102b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f102415db10 0x7f10240cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f102b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102415def0 con 0x7f1024099a00 2024-04-25T08:48:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f102b000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10240d0b60 con 0x7f1024070430 2024-04-25T08:48:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f102b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10240cbaf0 con 0x7f102415db10 2024-04-25T08:48:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1024099a00 0x7f10240cc540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35952/0 (socket says 172.21.15.46:35952) 2024-04-25T08:48:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029200640 1 -- 172.21.15.46:0/1786327829 learned_addr learned my addr 172.21.15.46:0/1786327829 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029c00640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f102415db10 0x7f10240cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528748281 0 0) 0x7f102415def0 con 0x7f1024099a00 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f100c003530 con 0x7f1024099a00 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 946128487 0 0) 0x7f100c003530 con 0x7f1024099a00 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f102415db10 msgr2=0x7f10240cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f102415db10 0x7f10240cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 >> v1:172.21.15.82:6789/0 conn(0x7f1024070430 legacy=0x7f10240ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240d0e40 con 0x7f1024099a00 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f1010001680 con 0x7f1024099a00 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 >> v1:172.21.15.46:6789/0 conn(0x7f1024099a00 legacy=0x7f10240cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f100c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f102415db10 0x7f100c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f100c004d50 0x7f100c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10240cbaf0 con 0x7f100c0039f0 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c00b9d0 con 0x7f100c004d50 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c00bbd0 con 0x7f102415db10 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029c00640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f100c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029200640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f102415db10 0x7f100c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f102a600640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f100c004d50 0x7f100c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029c00640 1 -- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f102415db10 msgr2=0x7f100c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029c00640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f102415db10 0x7f100c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029c00640 1 -- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f100c004d50 msgr2=0x7f100c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029c00640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f100c004d50 0x7f100c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.232+0000 7f1029c00640 1 -- 172.21.15.46:0/1786327829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c003530 con 0x7f100c0039f0 2024-04-25T08:48:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f1029c00640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f100c003dd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1018009ae0 tx=0x7f101801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14641aa11e906c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180b2070 con 0x7f100c0039f0 2024-04-25T08:48:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10180a55a0 con 0x7f100c0039f0 2024-04-25T08:48:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f1023e00640 1 -- 172.21.15.46:0/1786327829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180096b0 con 0x7f100c0039f0 2024-04-25T08:48:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f102b000640 1 -- 172.21.15.46:0/1786327829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 msgr2=0x7f100c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f102b000640 1 --2- 172.21.15.46:0/1786327829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f100c003dd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f1018009ae0 tx=0x7f101801c9b0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f102a600640 1 -- 172.21.15.46:0/1786327829 reap_dead start 2024-04-25T08:48:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f102b000640 1 -- 172.21.15.46:0/1786327829 shutdown_connections 2024-04-25T08:48:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f102b000640 1 -- 172.21.15.46:0/1786327829 >> 172.21.15.46:0/1786327829 conn(0x7f10240b2f90 msgr2=0x7f10240b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f102b000640 1 -- 172.21.15.46:0/1786327829 shutdown_connections 2024-04-25T08:48:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f102b000640 1 -- 172.21.15.46:0/1786327829 wait complete. 2024-04-25T08:48:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.236+0000 7f102b000640 1 Processor -- start 2024-04-25T08:48:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 -- start start 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f10241588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10241572f0 0x7f10241e77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f10241e7cf0 0x7f10240b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10240d07f0 con 0x7f100c0039f0 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10240cc0b0 con 0x7f10241572f0 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10240cf7d0 con 0x7f10241e7cf0 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f10241588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f10241588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49646/0 (socket says 172.21.15.46:49646) 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029c00640 1 -- 172.21.15.46:0/427402312 learned_addr learned my addr 172.21.15.46:0/427402312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029200640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10241572f0 0x7f10241e77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102a600640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f10241e7cf0 0x7f10240b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029c00640 1 -- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f10241e7cf0 msgr2=0x7f10240b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029c00640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f10241e7cf0 0x7f10240b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029c00640 1 -- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10241572f0 msgr2=0x7f10241e77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029c00640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10241572f0 0x7f10241e77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029c00640 1 -- 172.21.15.46:0/427402312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1024157b50 con 0x7f100c0039f0 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102a600640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f10241e7cf0 0x7f10240b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029200640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10241572f0 0x7f10241e77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1029c00640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f10241588f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1018009ae0 tx=0x7f100c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1021600640 1 -- 172.21.15.46:0/427402312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180b2040 con 0x7f100c0039f0 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1021600640 1 -- 172.21.15.46:0/427402312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101800f000 con 0x7f100c0039f0 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10240b6900 con 0x7f100c0039f0 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f1021600640 1 -- 172.21.15.46:0/427402312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018004240 con 0x7f100c0039f0 2024-04-25T08:48:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10241580f0 con 0x7f100c0039f0 2024-04-25T08:48:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.240+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1024069010 con 0x7f100c0039f0 2024-04-25T08:48:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.244+0000 7f1021600640 1 -- 172.21.15.46:0/427402312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10180c0030 con 0x7f100c0039f0 2024-04-25T08:48:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.244+0000 7f1021600640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1004066f90 0x7f1004069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.244+0000 7f1021600640 1 -- 172.21.15.46:0/427402312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f101811f960 con 0x7f100c0039f0 2024-04-25T08:48:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.244+0000 7f1029200640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1004066f90 0x7f1004069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.244+0000 7f1029200640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1004066f90 0x7f1004069450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f102406cce0 tx=0x7f1010009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.248+0000 7f1021600640 1 -- 172.21.15.46:0/427402312 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10180c7060 con 0x7f100c0039f0 2024-04-25T08:48:11.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.408+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10241e80d0 con 0x7f100c0039f0 2024-04-25T08:48:11.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.408+0000 7f1021600640 1 -- 172.21.15.46:0/427402312 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10180166c0 con 0x7f100c0039f0 2024-04-25T08:48:11.403 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:48:11.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1004066f90 msgr2=0x7f1004069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1004066f90 0x7f1004069450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f102406cce0 tx=0x7f1010009040 comp rx=0 tx=0).stop 2024-04-25T08:48:11.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 msgr2=0x7f10241588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f10241588f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1018009ae0 tx=0x7f100c005840 comp rx=0 tx=0).stop 2024-04-25T08:48:11.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 shutdown_connections 2024-04-25T08:48:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1004066f90 0x7f1004069450 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f10241e7cf0 0x7f10240b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10241572f0 0x7f10241e77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 --2- 172.21.15.46:0/427402312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f100c0039f0 0x7f10241588f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 >> 172.21.15.46:0/427402312 conn(0x7f10240b2f90 msgr2=0x7f10240cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 shutdown_connections 2024-04-25T08:48:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.412+0000 7f102b000640 1 -- 172.21.15.46:0/427402312 wait complete. 2024-04-25T08:48:11.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:48:11.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.572+0000 7fab7bc00640 1 Processor -- start 2024-04-25T08:48:11.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.572+0000 7fab7bc00640 1 -- start start 2024-04-25T08:48:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.572+0000 7fab7bc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fab7413d0b0 0x7fab7415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.572+0000 7fab7bc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 0x7fab740716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.572+0000 7fab7bc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fab7415db50 0x7fab7415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.572+0000 7fab7bc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab74071c10 con 0x7fab7415db50 2024-04-25T08:48:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.572+0000 7fab7bc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab7413d490 con 0x7fab7413d0b0 2024-04-25T08:48:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.572+0000 7fab7bc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7415eab0 con 0x7fab74071270 2024-04-25T08:48:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fab7415db50 0x7fab7415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fab7415db50 0x7fab7415dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49660/0 (socket says 172.21.15.46:49660) 2024-04-25T08:48:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79c00640 1 -- 172.21.15.46:0/2399541288 learned_addr learned my addr 172.21.15.46:0/2399541288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab73e00640 1 --2- 172.21.15.46:0/2399541288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 0x7fab740716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79200640 1 --2- 172.21.15.46:0/2399541288 >> v2:172.21.15.82:3300/0 conn(0x7fab7413d0b0 0x7fab7415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79c00640 1 -- 172.21.15.46:0/2399541288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 msgr2=0x7fab740716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79c00640 1 --2- 172.21.15.46:0/2399541288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 0x7fab740716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79c00640 1 -- 172.21.15.46:0/2399541288 >> v2:172.21.15.82:3300/0 conn(0x7fab7413d0b0 msgr2=0x7fab7415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79c00640 1 --2- 172.21.15.46:0/2399541288 >> v2:172.21.15.82:3300/0 conn(0x7fab7413d0b0 0x7fab7415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79c00640 1 -- 172.21.15.46:0/2399541288 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7415e520 con 0x7fab7415db50 2024-04-25T08:48:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab79c00640 1 --2- 172.21.15.46:0/2399541288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7415db50 0x7fab7415dfe0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fab74072710 tx=0x7fab6c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357505b255c1062 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab73400640 1 -- 172.21.15.46:0/2399541288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0a56e0 con 0x7fab7415db50 2024-04-25T08:48:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab73400640 1 -- 172.21.15.46:0/2399541288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c0a5880 con 0x7fab7415db50 2024-04-25T08:48:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.576+0000 7fab73400640 1 -- 172.21.15.46:0/2399541288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6c0a5c50 con 0x7fab7415db50 2024-04-25T08:48:11.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 -- 172.21.15.46:0/2399541288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7415db50 msgr2=0x7fab7415dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 --2- 172.21.15.46:0/2399541288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7415db50 0x7fab7415dfe0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fab74072710 tx=0x7fab6c01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 -- 172.21.15.46:0/2399541288 shutdown_connections 2024-04-25T08:48:11.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 --2- 172.21.15.46:0/2399541288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7415db50 0x7fab7415dfe0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 --2- 172.21.15.46:0/2399541288 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 0x7fab740716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 --2- 172.21.15.46:0/2399541288 >> v2:172.21.15.82:3300/0 conn(0x7fab7413d0b0 0x7fab7415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 -- 172.21.15.46:0/2399541288 >> 172.21.15.46:0/2399541288 conn(0x7fab7406ac10 msgr2=0x7fab7406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 -- 172.21.15.46:0/2399541288 shutdown_connections 2024-04-25T08:48:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 -- 172.21.15.46:0/2399541288 wait complete. 2024-04-25T08:48:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 Processor -- start 2024-04-25T08:48:11.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 -- start start 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 0x7fab7412c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7413d0b0 0x7fab7415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab7415db50 0x7fab7415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab740b2720 con 0x7fab7413d0b0 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7415aca0 con 0x7fab7415db50 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab7bc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab740b28a0 con 0x7fab74071270 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7413d0b0 0x7fab7415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7413d0b0 0x7fab7415cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49674/0 (socket says 172.21.15.46:49674) 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab73e00640 1 -- 172.21.15.46:0/990098433 learned_addr learned my addr 172.21.15.46:0/990098433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab79c00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab7415db50 0x7fab7415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab79200640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 0x7fab7412c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab73e00640 1 -- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 msgr2=0x7fab7412c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab73e00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 0x7fab7412c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab73e00640 1 -- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab7415db50 msgr2=0x7fab7415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab73e00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab7415db50 0x7fab7415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.580+0000 7fab73e00640 1 -- 172.21.15.46:0/990098433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab7412c5f0 con 0x7fab7413d0b0 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.584+0000 7fab73e00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7413d0b0 0x7fab7415cb00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fab6000d830 tx=0x7fab6000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.584+0000 7fab70c00640 1 -- 172.21.15.46:0/990098433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60018020 con 0x7fab7413d0b0 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.584+0000 7fab70c00640 1 -- 172.21.15.46:0/990098433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab600096a0 con 0x7fab7413d0b0 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.584+0000 7fab70c00640 1 -- 172.21.15.46:0/990098433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60009a50 con 0x7fab7413d0b0 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.584+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab740b14e0 con 0x7fab7413d0b0 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.584+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab740b1aa0 con 0x7fab7413d0b0 2024-04-25T08:48:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.584+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab74069010 con 0x7fab7413d0b0 2024-04-25T08:48:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.584+0000 7fab70c00640 1 -- 172.21.15.46:0/990098433 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab6001b020 con 0x7fab7413d0b0 2024-04-25T08:48:11.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.588+0000 7fab70c00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fab54066f60 0x7fab54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.588+0000 7fab70c00640 1 -- 172.21.15.46:0/990098433 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fab60085ca0 con 0x7fab7413d0b0 2024-04-25T08:48:11.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.588+0000 7fab79200640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fab54066f60 0x7fab54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.588+0000 7fab79200640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fab54066f60 0x7fab54069420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fab68007a60 tx=0x7fab68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.588+0000 7fab70c00640 1 -- 172.21.15.46:0/990098433 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab60022020 con 0x7fab7413d0b0 2024-04-25T08:48:11.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.748+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab740af090 con 0x7fab7413d0b0 2024-04-25T08:48:11.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.748+0000 7fab70c00640 1 -- 172.21.15.46:0/990098433 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab60085070 con 0x7fab7413d0b0 2024-04-25T08:48:11.744 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fab54066f60 msgr2=0x7fab54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fab54066f60 0x7fab54069420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fab68007a60 tx=0x7fab68008040 comp rx=0 tx=0).stop 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7413d0b0 msgr2=0x7fab7415cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7413d0b0 0x7fab7415cb00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fab6000d830 tx=0x7fab6000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 shutdown_connections 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fab54066f60 0x7fab54069420 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab7415db50 0x7fab7415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab7413d0b0 0x7fab7415cb00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 --2- 172.21.15.46:0/990098433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fab74071270 0x7fab7412c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 >> 172.21.15.46:0/990098433 conn(0x7fab7406ac10 msgr2=0x7fab74157260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 shutdown_connections 2024-04-25T08:48:11.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.752+0000 7fab7bc00640 1 -- 172.21.15.46:0/990098433 wait complete. 2024-04-25T08:48:11.765 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:48:11.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5250a00640 1 Processor -- start 2024-04-25T08:48:11.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5250a00640 1 -- start start 2024-04-25T08:48:11.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5250a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f524c15dbd0 0x7f524c0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5250a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 0x7f524c06c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5250a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524c0b5250 con 0x7f524c0b0c30 2024-04-25T08:48:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5250a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c0b5af0 con 0x7f524c15dbd0 2024-04-25T08:48:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5250a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0b5530 con 0x7f524c0713a0 2024-04-25T08:48:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5249e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f524c15dbd0 0x7f524c0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5249e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f524c15dbd0 0x7f524c0b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49688/0 (socket says 172.21.15.46:49688) 2024-04-25T08:48:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.912+0000 7f5249e00640 1 -- 172.21.15.46:0/3293620341 learned_addr learned my addr 172.21.15.46:0/3293620341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f524a800640 1 --2- 172.21.15.46:0/3293620341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 0x7f524c06c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5249e00640 1 -- 172.21.15.46:0/3293620341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 msgr2=0x7f524c06c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:11.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5249e00640 1 --2- 172.21.15.46:0/3293620341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 0x7f524c06c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5249e00640 1 -- 172.21.15.46:0/3293620341 >> v1:172.21.15.82:6789/0 conn(0x7f524c0b0c30 legacy=0x7f524c070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5249e00640 1 -- 172.21.15.46:0/3293620341 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c0b5810 con 0x7f524c15dbd0 2024-04-25T08:48:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5249e00640 1 --2- 172.21.15.46:0/3293620341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c15dbd0 0x7f524c0b06f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f523c009e50 tx=0x7f523c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5894775b94c0ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5249400640 1 -- 172.21.15.46:0/3293620341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c0a5700 con 0x7f524c15dbd0 2024-04-25T08:48:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5249400640 1 -- 172.21.15.46:0/3293620341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c0a58a0 con 0x7f524c15dbd0 2024-04-25T08:48:11.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5249400640 1 -- 172.21.15.46:0/3293620341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c0a5c70 con 0x7f524c15dbd0 2024-04-25T08:48:11.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5250a00640 1 -- 172.21.15.46:0/3293620341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c15dbd0 msgr2=0x7f524c0b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.916+0000 7f5250a00640 1 --2- 172.21.15.46:0/3293620341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c15dbd0 0x7f524c0b06f0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f523c009e50 tx=0x7f523c01ca30 comp rx=0 tx=0).stop 2024-04-25T08:48:11.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 -- 172.21.15.46:0/3293620341 shutdown_connections 2024-04-25T08:48:11.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 --2- 172.21.15.46:0/3293620341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 0x7f524c06c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 --2- 172.21.15.46:0/3293620341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c15dbd0 0x7f524c0b06f0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 -- 172.21.15.46:0/3293620341 >> 172.21.15.46:0/3293620341 conn(0x7f524c06f4f0 msgr2=0x7f524c06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:11.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 -- 172.21.15.46:0/3293620341 shutdown_connections 2024-04-25T08:48:11.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 -- 172.21.15.46:0/3293620341 wait complete. 2024-04-25T08:48:11.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 Processor -- start 2024-04-25T08:48:11.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 -- start start 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 0x7f524c1534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c0b0c30 0x7f524c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c15dbd0 0x7f524c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c06d4e0 con 0x7f524c0b0c30 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c06d360 con 0x7f524c15dbd0 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5250a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c06d660 con 0x7f524c0713a0 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5241400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c0b0c30 0x7f524c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5241400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c0b0c30 0x7f524c15e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49692/0 (socket says 172.21.15.46:49692) 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5249e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 0x7f524c1534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5241400640 1 -- 172.21.15.46:0/3613365104 learned_addr learned my addr 172.21.15.46:0/3613365104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f524a800640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c15dbd0 0x7f524c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5241400640 1 -- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 msgr2=0x7f524c1534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5241400640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 0x7f524c1534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5241400640 1 -- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c15dbd0 msgr2=0x7f524c15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5241400640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c15dbd0 0x7f524c15e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5241400640 1 -- 172.21.15.46:0/3613365104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c160f20 con 0x7f524c0b0c30 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.920+0000 7f5241400640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c0b0c30 0x7f524c15e040 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f523400c420 tx=0x7f523400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5242a00640 1 -- 172.21.15.46:0/3613365104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340052d0 con 0x7f524c0b0c30 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5242a00640 1 -- 172.21.15.46:0/3613365104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5234005470 con 0x7f524c0b0c30 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5242a00640 1 -- 172.21.15.46:0/3613365104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234007260 con 0x7f524c0b0c30 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c15eac0 con 0x7f524c0b0c30 2024-04-25T08:48:11.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c13e520 con 0x7f524c0b0c30 2024-04-25T08:48:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c069080 con 0x7f524c0b0c30 2024-04-25T08:48:11.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5242a00640 1 -- 172.21.15.46:0/3613365104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5234007400 con 0x7f524c0b0c30 2024-04-25T08:48:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5242a00640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5228066f60 0x7f5228069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5242a00640 1 -- 172.21.15.46:0/3613365104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5234084f30 con 0x7f524c0b0c30 2024-04-25T08:48:11.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.924+0000 7f5249e00640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5228066f60 0x7f5228069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:11.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.928+0000 7f5249e00640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5228066f60 0x7f5228069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f523c0ae000 tx=0x7f523c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:11.928+0000 7f5242a00640 1 -- 172.21.15.46:0/3613365104 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52340b0cb0 con 0x7f524c0b0c30 2024-04-25T08:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.104+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f524c0b1690 con 0x7f524c0b0c30 2024-04-25T08:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.104+0000 7f5242a00640 1 -- 172.21.15.46:0/3613365104 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v79) v1 ==== 80+0+120 (secure 0 0 0) 0x7f52340556f0 con 0x7f524c0b0c30 2024-04-25T08:48:12.100 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/960745191 2024-04-26T08:46:56.883011+0000 2024-04-25T08:48:12.100 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6833/960745191 2024-04-26T08:46:56.883011+0000 2024-04-25T08:48:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-04-25T08:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5228066f60 msgr2=0x7f5228069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5228066f60 0x7f5228069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f523c0ae000 tx=0x7f523c0a7070 comp rx=0 tx=0).stop 2024-04-25T08:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c0b0c30 msgr2=0x7f524c15e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c0b0c30 0x7f524c15e040 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f523400c420 tx=0x7f523400c8e0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 shutdown_connections 2024-04-25T08:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5228066f60 0x7f5228069420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f524c15dbd0 0x7f524c15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f524c0b0c30 0x7f524c15e040 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 --2- 172.21.15.46:0/3613365104 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f524c0713a0 0x7f524c1534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 >> 172.21.15.46:0/3613365104 conn(0x7f524c06f4f0 msgr2=0x7f524c15cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 shutdown_connections 2024-04-25T08:48:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.108+0000 7f5250a00640 1 -- 172.21.15.46:0/3613365104 wait complete. 2024-04-25T08:48:12.122 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:48:12.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:12.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:12.148 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:48:12.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:12.148 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:48:12.201 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:48:12.201 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:12.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:48:12.210 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:12.210 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:12.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:48:12.257 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:48:12.257 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:12.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:48:12.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8ca4e00640 1 Processor -- start 2024-04-25T08:48:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8ca4e00640 1 -- start start 2024-04-25T08:48:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8ca4e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca006bc20 0x7f8ca00b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8ca4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ca015c840 0x7f8ca006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8ca4e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ca015f3b0 0x7f8ca015f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8ca4e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca00726a0 con 0x7f8ca015c840 2024-04-25T08:48:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8ca4e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca0072bf0 con 0x7f8ca015f3b0 2024-04-25T08:48:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8ca4e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca006c540 con 0x7f8ca006bc20 2024-04-25T08:48:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8c9d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ca015c840 0x7f8ca006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8c9d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ca015c840 0x7f8ca006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49698/0 (socket says 172.21.15.46:49698) 2024-04-25T08:48:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8c9d400640 1 -- 172.21.15.46:0/2795248624 learned_addr learned my addr 172.21.15.46:0/2795248624 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8c9d400640 1 -- 172.21.15.46:0/2795248624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca006bc20 msgr2=0x7f8ca00b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8c9d400640 1 --2- 172.21.15.46:0/2795248624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca006bc20 0x7f8ca00b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8c9d400640 1 -- 172.21.15.46:0/2795248624 >> v2:172.21.15.82:3300/0 conn(0x7f8ca015f3b0 msgr2=0x7f8ca015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8c9d400640 1 --2- 172.21.15.46:0/2795248624 >> v2:172.21.15.82:3300/0 conn(0x7f8ca015f3b0 0x7f8ca015f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.448+0000 7f8c9d400640 1 -- 172.21.15.46:0/2795248624 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca0072910 con 0x7f8ca015c840 2024-04-25T08:48:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8c9d400640 1 --2- 172.21.15.46:0/2795248624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 0x7f8ca006c000 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00cfc30 tx=0x7f8c8c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0230bd81957bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8c9ca00640 1 -- 172.21.15.46:0/2795248624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0b0070 con 0x7f8ca015c840 2024-04-25T08:48:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8c9ca00640 1 -- 172.21.15.46:0/2795248624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c0ab600 con 0x7f8ca015c840 2024-04-25T08:48:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8c9ca00640 1 -- 172.21.15.46:0/2795248624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0ab9d0 con 0x7f8ca015c840 2024-04-25T08:48:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 -- 172.21.15.46:0/2795248624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 msgr2=0x7f8ca006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/2795248624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 0x7f8ca006c000 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00cfc30 tx=0x7f8c8c01c520 comp rx=0 tx=0).stop 2024-04-25T08:48:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 -- 172.21.15.46:0/2795248624 shutdown_connections 2024-04-25T08:48:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/2795248624 >> v2:172.21.15.82:3300/0 conn(0x7f8ca015f3b0 0x7f8ca015f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/2795248624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 0x7f8ca006c000 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/2795248624 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca006bc20 0x7f8ca00b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 -- 172.21.15.46:0/2795248624 >> 172.21.15.46:0/2795248624 conn(0x7f8ca00cd630 msgr2=0x7f8ca0071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 -- 172.21.15.46:0/2795248624 shutdown_connections 2024-04-25T08:48:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 -- 172.21.15.46:0/2795248624 wait complete. 2024-04-25T08:48:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 Processor -- start 2024-04-25T08:48:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.452+0000 7f8ca4e00640 1 -- start start 2024-04-25T08:48:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8ca4e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ca006bc20 0x7f8ca014e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8ca4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 0x7f8ca01486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8ca4e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca015f3b0 0x7f8ca0148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8ca4e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00ce430 con 0x7f8ca015c840 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8ca4e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca015cc80 con 0x7f8ca006bc20 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8ca4e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00ce5b0 con 0x7f8ca015f3b0 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 0x7f8ca01486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 0x7f8ca01486a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49706/0 (socket says 172.21.15.46:49706) 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9d400640 1 -- 172.21.15.46:0/1758066397 learned_addr learned my addr 172.21.15.46:0/1758066397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9de00640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ca006bc20 0x7f8ca014e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9e800640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca015f3b0 0x7f8ca0148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9d400640 1 -- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca015f3b0 msgr2=0x7f8ca0148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9d400640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca015f3b0 0x7f8ca0148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9d400640 1 -- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ca006bc20 msgr2=0x7f8ca014e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9d400640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ca006bc20 0x7f8ca014e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9d400640 1 -- 172.21.15.46:0/1758066397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca015a320 con 0x7f8ca015c840 2024-04-25T08:48:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9e800640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca015f3b0 0x7f8ca0148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c9d400640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 0x7f8ca01486a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0ae040 tx=0x7f8c8c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c96000640 1 -- 172.21.15.46:0/1758066397 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0ba020 con 0x7f8ca015c840 2024-04-25T08:48:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c96000640 1 -- 172.21.15.46:0/1758066397 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c0b6640 con 0x7f8ca015c840 2024-04-25T08:48:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca015a590 con 0x7f8ca015c840 2024-04-25T08:48:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c96000640 1 -- 172.21.15.46:0/1758066397 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c0b0040 con 0x7f8ca015c840 2024-04-25T08:48:12.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca015ab50 con 0x7f8ca015c840 2024-04-25T08:48:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca0069080 con 0x7f8ca015c840 2024-04-25T08:48:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.456+0000 7f8c96000640 1 -- 172.21.15.46:0/1758066397 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c8c0bd020 con 0x7f8ca015c840 2024-04-25T08:48:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.460+0000 7f8c96000640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c80066f90 0x7f8c80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.460+0000 7f8c96000640 1 -- 172.21.15.46:0/1758066397 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8c8c11f570 con 0x7f8ca015c840 2024-04-25T08:48:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.460+0000 7f8c9de00640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c80066f90 0x7f8c80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.460+0000 7f8c9de00640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c80066f90 0x7f8c80069450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c90006fd0 tx=0x7f8c90008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:12.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.464+0000 7f8c96000640 1 -- 172.21.15.46:0/1758066397 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c8c0c4030 con 0x7f8ca015c840 2024-04-25T08:48:12.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.656+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ca00b36c0 con 0x7f8ca015c840 2024-04-25T08:48:12.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8c96000640 1 -- 172.21.15.46:0/1758066397 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f8c8c016110 con 0x7f8ca015c840 2024-04-25T08:48:12.653 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:12.653 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:47:43.911465+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/4112568749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":4112568749},{"type":"v1","addr":"172.21.15.46:6835","nonce":4112568749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:48:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 53 2024-04-25T08:48:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c80066f90 msgr2=0x7f8c80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c80066f90 0x7f8c80069450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8c90006fd0 tx=0x7f8c90008040 comp rx=0 tx=0).stop 2024-04-25T08:48:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 msgr2=0x7f8ca01486a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 0x7f8ca01486a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0ae040 tx=0x7f8c8c01ad60 comp rx=0 tx=0).stop 2024-04-25T08:48:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 shutdown_connections 2024-04-25T08:48:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c80066f90 0x7f8c80069450 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ca015f3b0 0x7f8ca0148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ca015c840 0x7f8ca01486a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 --2- 172.21.15.46:0/1758066397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ca006bc20 0x7f8ca014e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 >> 172.21.15.46:0/1758066397 conn(0x7f8ca00cd630 msgr2=0x7f8ca014b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 shutdown_connections 2024-04-25T08:48:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.660+0000 7f8ca4e00640 1 -- 172.21.15.46:0/1758066397 wait complete. 2024-04-25T08:48:12.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22d7000640 1 Processor -- start 2024-04-25T08:48:12.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22d7000640 1 -- start start 2024-04-25T08:48:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22d7000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22d015dbd0 0x7f22d00b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22d7000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 0x7f22d0070c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22d7000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f22d00711d0 0x7f22d015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22d7000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d00b5af0 con 0x7f22d015dbd0 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22d7000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d00b5250 con 0x7f22d00711d0 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22d7000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00b5530 con 0x7f22d00b0c30 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22cfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22d015dbd0 0x7f22d00b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22cfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22d015dbd0 0x7f22d00b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49714/0 (socket says 172.21.15.46:49714) 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22cfe00640 1 -- 172.21.15.46:0/3636943950 learned_addr learned my addr 172.21.15.46:0/3636943950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22d5000640 1 --2- 172.21.15.46:0/3636943950 >> v2:172.21.15.82:3300/0 conn(0x7f22d00711d0 0x7f22d015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22c7e00640 1 --2- 172.21.15.46:0/3636943950 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 0x7f22d0070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22cfe00640 1 -- 172.21.15.46:0/3636943950 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 msgr2=0x7f22d0070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22cfe00640 1 --2- 172.21.15.46:0/3636943950 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 0x7f22d0070c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22cfe00640 1 -- 172.21.15.46:0/3636943950 >> v2:172.21.15.82:3300/0 conn(0x7f22d00711d0 msgr2=0x7f22d015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22cfe00640 1 --2- 172.21.15.46:0/3636943950 >> v2:172.21.15.82:3300/0 conn(0x7f22d00711d0 0x7f22d015dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.820+0000 7f22cfe00640 1 -- 172.21.15.46:0/3636943950 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00b5810 con 0x7f22d015dbd0 2024-04-25T08:48:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22cfe00640 1 --2- 172.21.15.46:0/3636943950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 0x7f22d00b06f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f22d006bba0 tx=0x7f22c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc49fc09095ee80d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22cf400640 1 -- 172.21.15.46:0/3636943950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00b0070 con 0x7f22d015dbd0 2024-04-25T08:48:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22cf400640 1 -- 172.21.15.46:0/3636943950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c00ab600 con 0x7f22d015dbd0 2024-04-25T08:48:12.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22cf400640 1 -- 172.21.15.46:0/3636943950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00ab9d0 con 0x7f22d015dbd0 2024-04-25T08:48:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22d7000640 1 -- 172.21.15.46:0/3636943950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 msgr2=0x7f22d00b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22d7000640 1 --2- 172.21.15.46:0/3636943950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 0x7f22d00b06f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f22d006bba0 tx=0x7f22c001c520 comp rx=0 tx=0).stop 2024-04-25T08:48:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22d7000640 1 -- 172.21.15.46:0/3636943950 shutdown_connections 2024-04-25T08:48:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22d7000640 1 --2- 172.21.15.46:0/3636943950 >> v2:172.21.15.82:3300/0 conn(0x7f22d00711d0 0x7f22d015dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22d7000640 1 --2- 172.21.15.46:0/3636943950 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 0x7f22d0070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22d7000640 1 --2- 172.21.15.46:0/3636943950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 0x7f22d00b06f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22d7000640 1 -- 172.21.15.46:0/3636943950 >> 172.21.15.46:0/3636943950 conn(0x7f22d006f4f0 msgr2=0x7f22d006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22d7000640 1 -- 172.21.15.46:0/3636943950 shutdown_connections 2024-04-25T08:48:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.824+0000 7f22d7000640 1 -- 172.21.15.46:0/3636943950 wait complete. 2024-04-25T08:48:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 Processor -- start 2024-04-25T08:48:12.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 -- start start 2024-04-25T08:48:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22d00711d0 0x7f22d00d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 0x7f22d00cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 0x7f22d014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d006cda0 con 0x7f22d015dbd0 2024-04-25T08:48:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d00b1010 con 0x7f22d00711d0 2024-04-25T08:48:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d006cf20 con 0x7f22d00b0c30 2024-04-25T08:48:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 0x7f22d014f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 0x7f22d014f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49718/0 (socket says 172.21.15.46:49718) 2024-04-25T08:48:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d5000640 1 -- 172.21.15.46:0/1264811171 learned_addr learned my addr 172.21.15.46:0/1264811171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22c7e00640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 0x7f22d00cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22cfe00640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22d00711d0 0x7f22d00d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d5000640 1 -- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 msgr2=0x7f22d00cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d5000640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 0x7f22d00cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d5000640 1 -- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22d00711d0 msgr2=0x7f22d00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d5000640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22d00711d0 0x7f22d00d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d5000640 1 -- 172.21.15.46:0/1264811171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d00d1080 con 0x7f22d015dbd0 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22c7e00640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 0x7f22d00cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d5000640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 0x7f22d014f6d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f22c8007e60 tx=0x7f22c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22ccc00640 1 -- 172.21.15.46:0/1264811171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8014620 con 0x7f22d015dbd0 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22ccc00640 1 -- 172.21.15.46:0/1264811171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8019070 con 0x7f22d015dbd0 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22ccc00640 1 -- 172.21.15.46:0/1264811171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8014a20 con 0x7f22d015dbd0 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d00cb2c0 con 0x7f22d015dbd0 2024-04-25T08:48:12.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.828+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d00cad00 con 0x7f22d015dbd0 2024-04-25T08:48:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.832+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d0069080 con 0x7f22d015dbd0 2024-04-25T08:48:12.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.832+0000 7f22ccc00640 1 -- 172.21.15.46:0/1264811171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22c8014bc0 con 0x7f22d015dbd0 2024-04-25T08:48:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.832+0000 7f22ccc00640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22ac066f60 0x7f22ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.832+0000 7f22ccc00640 1 -- 172.21.15.46:0/1264811171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f22c8085a00 con 0x7f22d015dbd0 2024-04-25T08:48:12.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.832+0000 7f22cfe00640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22ac066f60 0x7f22ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:12.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.832+0000 7f22cfe00640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22ac066f60 0x7f22ac069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f22c00ad000 tx=0x7f22c00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:12.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:12.836+0000 7f22ccc00640 1 -- 172.21.15.46:0/1264811171 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22d00cb2c0 con 0x7f22d015dbd0 2024-04-25T08:48:12.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.000+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22d0157c00 con 0x7f22d015dbd0 2024-04-25T08:48:12.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.000+0000 7f22ccc00640 1 -- 172.21.15.46:0/1264811171 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17024 (secure 0 0 0) 0x7f22c8011260 con 0x7f22d015dbd0 2024-04-25T08:48:12.996 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:12.996 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:56.883035+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:46:44.505856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:46:45.072095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/960745191":"2024-04-26T08:46:56.883011+0000","172.21.15.46:6833/960745191":"2024-04-26T08:46:56.883011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22ac066f60 msgr2=0x7f22ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22ac066f60 0x7f22ac069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f22c00ad000 tx=0x7f22c00a6000 comp rx=0 tx=0).stop 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 msgr2=0x7f22d014f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 0x7f22d014f6d0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f22c8007e60 tx=0x7f22c800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 shutdown_connections 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f22ac066f60 0x7f22ac069420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d015dbd0 0x7f22d014f6d0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f22d00b0c30 0x7f22d00cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 --2- 172.21.15.46:0/1264811171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22d00711d0 0x7f22d00d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 >> 172.21.15.46:0/1264811171 conn(0x7f22d006f4f0 msgr2=0x7f22d00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 shutdown_connections 2024-04-25T08:48:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.004+0000 7f22d7000640 1 -- 172.21.15.46:0/1264811171 wait complete. 2024-04-25T08:48:13.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:46:44.505856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:46:45.072095+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:48:13.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:48:13.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e48a00640 1 Processor -- start 2024-04-25T08:48:13.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e48a00640 1 -- start start 2024-04-25T08:48:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e48a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 0x7f6e4415b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e48a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e440b4950 0x7f6e44072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e48a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6e44072e90 0x7f6e4415e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e48a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4415bd80 con 0x7f6e440b4950 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e48a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e440b4590 con 0x7f6e44072e90 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e48a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b0080 con 0x7f6e440afca0 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e39400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e440b4950 0x7f6e44072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e39400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e440b4950 0x7f6e44072950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49724/0 (socket says 172.21.15.46:49724) 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e39400640 1 -- 172.21.15.46:0/1773665758 learned_addr learned my addr 172.21.15.46:0/1773665758 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e42800640 1 --2- 172.21.15.46:0/1773665758 >> v2:172.21.15.82:3300/0 conn(0x7f6e44072e90 0x7f6e4415e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e41e00640 1 --2- 172.21.15.46:0/1773665758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 0x7f6e4415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e39400640 1 -- 172.21.15.46:0/1773665758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 msgr2=0x7f6e4415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e39400640 1 --2- 172.21.15.46:0/1773665758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 0x7f6e4415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e39400640 1 -- 172.21.15.46:0/1773665758 >> v2:172.21.15.82:3300/0 conn(0x7f6e44072e90 msgr2=0x7f6e4415e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:13.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e39400640 1 --2- 172.21.15.46:0/1773665758 >> v2:172.21.15.82:3300/0 conn(0x7f6e44072e90 0x7f6e4415e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.168+0000 7f6e39400640 1 -- 172.21.15.46:0/1773665758 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440b3a10 con 0x7f6e440b4950 2024-04-25T08:48:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e39400640 1 --2- 172.21.15.46:0/1773665758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 0x7f6e44072950 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6e440d15f0 tx=0x7f6e2c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e16b341704b0802 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:13.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e41400640 1 -- 172.21.15.46:0/1773665758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0b0070 con 0x7f6e440b4950 2024-04-25T08:48:13.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e41400640 1 -- 172.21.15.46:0/1773665758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c0a4e50 con 0x7f6e440b4950 2024-04-25T08:48:13.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e41400640 1 -- 172.21.15.46:0/1773665758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0ab820 con 0x7f6e440b4950 2024-04-25T08:48:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e48a00640 1 -- 172.21.15.46:0/1773665758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 msgr2=0x7f6e44072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e48a00640 1 --2- 172.21.15.46:0/1773665758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 0x7f6e44072950 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f6e440d15f0 tx=0x7f6e2c01c400 comp rx=0 tx=0).stop 2024-04-25T08:48:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e48a00640 1 -- 172.21.15.46:0/1773665758 shutdown_connections 2024-04-25T08:48:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e48a00640 1 --2- 172.21.15.46:0/1773665758 >> v2:172.21.15.82:3300/0 conn(0x7f6e44072e90 0x7f6e4415e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e48a00640 1 --2- 172.21.15.46:0/1773665758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 0x7f6e44072950 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e48a00640 1 --2- 172.21.15.46:0/1773665758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 0x7f6e4415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e48a00640 1 -- 172.21.15.46:0/1773665758 >> 172.21.15.46:0/1773665758 conn(0x7f6e4406bd20 msgr2=0x7f6e4406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e48a00640 1 -- 172.21.15.46:0/1773665758 shutdown_connections 2024-04-25T08:48:13.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.172+0000 7f6e48a00640 1 -- 172.21.15.46:0/1773665758 wait complete. 2024-04-25T08:48:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 Processor -- start 2024-04-25T08:48:13.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 -- start start 2024-04-25T08:48:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e44072e90 0x7f6e4415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 0x7f6e4413b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 0x7f6e44157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b2490 con 0x7f6e440b4950 2024-04-25T08:48:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b4db0 con 0x7f6e44072e90 2024-04-25T08:48:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440b3630 con 0x7f6e440afca0 2024-04-25T08:48:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e42800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 0x7f6e44157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e42800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 0x7f6e44157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49740/0 (socket says 172.21.15.46:49740) 2024-04-25T08:48:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e42800640 1 -- 172.21.15.46:0/4212103041 learned_addr learned my addr 172.21.15.46:0/4212103041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e41e00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e44072e90 0x7f6e4415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e39400640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 0x7f6e4413b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e42800640 1 -- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 msgr2=0x7f6e4413b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e42800640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 0x7f6e4413b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e42800640 1 -- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e44072e90 msgr2=0x7f6e4415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e42800640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e44072e90 0x7f6e4415dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e42800640 1 -- 172.21.15.46:0/4212103041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4415ce00 con 0x7f6e440b4950 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e42800640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 0x7f6e44157190 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c007e40 tx=0x7f6e3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e3aa00640 1 -- 172.21.15.46:0/4212103041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c014620 con 0x7f6e440b4950 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e3aa00640 1 -- 172.21.15.46:0/4212103041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c019070 con 0x7f6e440b4950 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e3aa00640 1 -- 172.21.15.46:0/4212103041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c014a20 con 0x7f6e440b4950 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4415d0e0 con 0x7f6e440b4950 2024-04-25T08:48:13.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.176+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4413bc70 con 0x7f6e440b4950 2024-04-25T08:48:13.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.180+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4406ccb0 con 0x7f6e440b4950 2024-04-25T08:48:13.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.180+0000 7f6e3aa00640 1 -- 172.21.15.46:0/4212103041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e3c014bc0 con 0x7f6e440b4950 2024-04-25T08:48:13.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.180+0000 7f6e3aa00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e20066f60 0x7f6e20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.180+0000 7f6e3aa00640 1 -- 172.21.15.46:0/4212103041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6e3c085960 con 0x7f6e440b4950 2024-04-25T08:48:13.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.180+0000 7f6e41e00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e20066f60 0x7f6e20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.180+0000 7f6e41e00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e20066f60 0x7f6e20069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e44156610 tx=0x7f6e34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:13.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.184+0000 7f6e3aa00640 1 -- 172.21.15.46:0/4212103041 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e3c056130 con 0x7f6e440b4950 2024-04-25T08:48:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.348+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e44069010 con 0x7f6e440b4950 2024-04-25T08:48:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.348+0000 7f6e3aa00640 1 -- 172.21.15.46:0/4212103041 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e3c05a1d0 con 0x7f6e440b4950 2024-04-25T08:48:13.345 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e20066f60 msgr2=0x7f6e20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e20066f60 0x7f6e20069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e44156610 tx=0x7f6e34008040 comp rx=0 tx=0).stop 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 msgr2=0x7f6e44157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 0x7f6e44157190 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c007e40 tx=0x7f6e3c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 shutdown_connections 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e20066f60 0x7f6e20069420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e440b4950 0x7f6e44157190 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e440afca0 0x7f6e4413b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 --2- 172.21.15.46:0/4212103041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e44072e90 0x7f6e4415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 >> 172.21.15.46:0/4212103041 conn(0x7f6e4406bd20 msgr2=0x7f6e440d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 shutdown_connections 2024-04-25T08:48:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.352+0000 7f6e48a00640 1 -- 172.21.15.46:0/4212103041 wait complete. 2024-04-25T08:48:13.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:48:13.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e8600640 1 Processor -- start 2024-04-25T08:48:13.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e8600640 1 -- start start 2024-04-25T08:48:13.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e8600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92e0145310 0x7f92e006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e8600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e00cd930 con 0x7f92e006dc70 2024-04-25T08:48:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e8600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e00cdbd0 con 0x7f92e015d090 2024-04-25T08:48:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e8600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e00cdeb0 con 0x7f92e0145310 2024-04-25T08:48:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92e0145310 0x7f92e006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92e0145310 0x7f92e006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49748/0 (socket says 172.21.15.46:49748) 2024-04-25T08:48:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e5200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f92e006dc70 0x7f92e00cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35968/0 (socket says 172.21.15.46:35968) 2024-04-25T08:48:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e5c00640 1 -- 172.21.15.46:0/385572220 learned_addr learned my addr 172.21.15.46:0/385572220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e5c00640 1 -- 172.21.15.46:0/385572220 >> v1:172.21.15.82:6789/0 conn(0x7f92e015d090 legacy=0x7f92e00cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e5c00640 1 -- 172.21.15.46:0/385572220 >> v1:172.21.15.46:6789/0 conn(0x7f92e006dc70 legacy=0x7f92e00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.516+0000 7f92e5c00640 1 -- 172.21.15.46:0/385572220 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00ce190 con 0x7f92e0145310 2024-04-25T08:48:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92e5c00640 1 --2- 172.21.15.46:0/385572220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e0145310 0x7f92e006e0d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009870 tx=0x7f92d401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950e2f515852c67f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92dfe00640 1 -- 172.21.15.46:0/385572220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40a95f0 con 0x7f92e0145310 2024-04-25T08:48:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92dfe00640 1 -- 172.21.15.46:0/385572220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d40a9790 con 0x7f92e0145310 2024-04-25T08:48:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92dfe00640 1 -- 172.21.15.46:0/385572220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d40a9b60 con 0x7f92e0145310 2024-04-25T08:48:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92e8600640 1 -- 172.21.15.46:0/385572220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e0145310 msgr2=0x7f92e006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92e8600640 1 --2- 172.21.15.46:0/385572220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e0145310 0x7f92e006e0d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009870 tx=0x7f92d401c530 comp rx=0 tx=0).stop 2024-04-25T08:48:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92e8600640 1 -- 172.21.15.46:0/385572220 shutdown_connections 2024-04-25T08:48:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92e8600640 1 --2- 172.21.15.46:0/385572220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e0145310 0x7f92e006e0d0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92e8600640 1 -- 172.21.15.46:0/385572220 >> 172.21.15.46:0/385572220 conn(0x7f92e006ac10 msgr2=0x7f92e006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92e8600640 1 -- 172.21.15.46:0/385572220 shutdown_connections 2024-04-25T08:48:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92e8600640 1 -- 172.21.15.46:0/385572220 wait complete. 2024-04-25T08:48:13.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.520+0000 7f92e8600640 1 Processor -- start 2024-04-25T08:48:13.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 -- start start 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0145310 0x7f92e01592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92e015d090 0x7f92e015be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e015c380 0x7f92e00b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e015d8b0 con 0x7f92e015c380 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00cf380 con 0x7f92e0145310 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0169d30 con 0x7f92e015d090 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e015c380 0x7f92e00b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e015c380 0x7f92e00b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49756/0 (socket says 172.21.15.46:49756) 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e6600640 1 -- 172.21.15.46:0/3650045849 learned_addr learned my addr 172.21.15.46:0/3650045849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e5c00640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0145310 0x7f92e01592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e5200640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92e015d090 0x7f92e015be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e6600640 1 -- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92e015d090 msgr2=0x7f92e015be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e6600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92e015d090 0x7f92e015be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e6600640 1 -- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0145310 msgr2=0x7f92e01592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e6600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0145310 0x7f92e01592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e6600640 1 -- 172.21.15.46:0/3650045849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00b4230 con 0x7f92e015c380 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e6600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e015c380 0x7f92e00b34c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c730 tx=0x7f92d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92dd600640 1 -- 172.21.15.46:0/3650045849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8003b80 con 0x7f92e015c380 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92dd600640 1 -- 172.21.15.46:0/3650045849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d8003d20 con 0x7f92e015c380 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92dd600640 1 -- 172.21.15.46:0/3650045849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8005310 con 0x7f92e015c380 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e0134b00 con 0x7f92e015c380 2024-04-25T08:48:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e01350c0 con 0x7f92e015c380 2024-04-25T08:48:13.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.524+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e0069010 con 0x7f92e015c380 2024-04-25T08:48:13.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.528+0000 7f92dd600640 1 -- 172.21.15.46:0/3650045849 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92d80056c0 con 0x7f92e015c380 2024-04-25T08:48:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.528+0000 7f92dd600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92c0066f90 0x7f92c0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.528+0000 7f92dd600640 1 -- 172.21.15.46:0/3650045849 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f92d8085260 con 0x7f92e015c380 2024-04-25T08:48:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.528+0000 7f92e5c00640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92c0066f90 0x7f92c0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.528+0000 7f92e5c00640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92c0066f90 0x7f92c0069450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f92d40ae000 tx=0x7f92d40b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:13.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.532+0000 7f92dd600640 1 -- 172.21.15.46:0/3650045849 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92d8056080 con 0x7f92e015c380 2024-04-25T08:48:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.692+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92e00af5c0 con 0x7f92e015c380 2024-04-25T08:48:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.692+0000 7f92dd600640 1 -- 172.21.15.46:0/3650045849 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7f92d8055980 con 0x7f92e015c380 2024-04-25T08:48:13.687 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:48:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92c0066f90 msgr2=0x7f92c0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92c0066f90 0x7f92c0069450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f92d40ae000 tx=0x7f92d40b3040 comp rx=0 tx=0).stop 2024-04-25T08:48:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e015c380 msgr2=0x7f92e00b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e015c380 0x7f92e00b34c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c730 tx=0x7f92d800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 shutdown_connections 2024-04-25T08:48:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f92c0066f90 0x7f92c0069450 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92e015c380 0x7f92e00b34c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92e015d090 0x7f92e015be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 --2- 172.21.15.46:0/3650045849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0145310 0x7f92e01592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 >> 172.21.15.46:0/3650045849 conn(0x7f92e006ac10 msgr2=0x7f92e006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:13.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 shutdown_connections 2024-04-25T08:48:13.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.696+0000 7f92e8600640 1 -- 172.21.15.46:0/3650045849 wait complete. 2024-04-25T08:48:13.708 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:48:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.856+0000 7fc67d600640 1 Processor -- start 2024-04-25T08:48:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.856+0000 7fc67d600640 1 -- start start 2024-04-25T08:48:13.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67d600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc6780b60c0 0x7fc6780caa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc67806b320 0x7fc67815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6780cafb0 con 0x7fc6780cc640 2024-04-25T08:48:13.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67d600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6780ce210 con 0x7fc6780b60c0 2024-04-25T08:48:13.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780cb250 con 0x7fc67806b320 2024-04-25T08:48:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc66fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6780cc640 0x7fc67806ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35974/0 (socket says 172.21.15.46:35974) 2024-04-25T08:48:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc66fe00640 1 -- 172.21.15.46:0/3248212551 learned_addr learned my addr 172.21.15.46:0/3248212551 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:13.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67cc00640 1 --2- 172.21.15.46:0/3248212551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc67806b320 0x7fc67815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc677e00640 1 --2- 172.21.15.46:0/3248212551 >> v2:172.21.15.82:3300/0 conn(0x7fc6780b60c0 0x7fc6780caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc677400640 1 -- 172.21.15.46:0/3248212551 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969512228 0 0) 0x7fc6780cafb0 con 0x7fc6780cc640 2024-04-25T08:48:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc677400640 1 -- 172.21.15.46:0/3248212551 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc65c003530 con 0x7fc6780cc640 2024-04-25T08:48:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67cc00640 1 -- 172.21.15.46:0/3248212551 >> v2:172.21.15.82:3300/0 conn(0x7fc6780b60c0 msgr2=0x7fc6780caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67cc00640 1 --2- 172.21.15.46:0/3248212551 >> v2:172.21.15.82:3300/0 conn(0x7fc6780b60c0 0x7fc6780caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc677400640 1 -- 172.21.15.46:0/3248212551 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 68460747 0 0) 0x7fc65c003530 con 0x7fc6780cc640 2024-04-25T08:48:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67cc00640 1 -- 172.21.15.46:0/3248212551 >> v1:172.21.15.46:6789/0 conn(0x7fc6780cc640 legacy=0x7fc67806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.860+0000 7fc67cc00640 1 -- 172.21.15.46:0/3248212551 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780cb530 con 0x7fc67806b320 2024-04-25T08:48:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67cc00640 1 --2- 172.21.15.46:0/3248212551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc67806b320 0x7fc67815d0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc670009d90 tx=0x7fc67001f080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9148a8db2d6b40df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc677400640 1 -- 172.21.15.46:0/3248212551 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6700b5070 con 0x7fc67806b320 2024-04-25T08:48:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc677400640 1 -- 172.21.15.46:0/3248212551 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6700a8bd0 con 0x7fc67806b320 2024-04-25T08:48:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc677400640 1 -- 172.21.15.46:0/3248212551 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660001680 con 0x7fc67806b320 2024-04-25T08:48:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 -- 172.21.15.46:0/3248212551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc67806b320 msgr2=0x7fc67815d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 --2- 172.21.15.46:0/3248212551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc67806b320 0x7fc67815d0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc670009d90 tx=0x7fc67001f080 comp rx=0 tx=0).stop 2024-04-25T08:48:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 -- 172.21.15.46:0/3248212551 shutdown_connections 2024-04-25T08:48:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 --2- 172.21.15.46:0/3248212551 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc67806b320 0x7fc67815d0a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 --2- 172.21.15.46:0/3248212551 >> v2:172.21.15.82:3300/0 conn(0x7fc6780b60c0 0x7fc6780caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 -- 172.21.15.46:0/3248212551 >> 172.21.15.46:0/3248212551 conn(0x7fc678099a00 msgr2=0x7fc6780b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:13.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 -- 172.21.15.46:0/3248212551 shutdown_connections 2024-04-25T08:48:13.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 -- 172.21.15.46:0/3248212551 wait complete. 2024-04-25T08:48:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 Processor -- start 2024-04-25T08:48:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.864+0000 7fc67d600640 1 -- start start 2024-04-25T08:48:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67806b320 0x7fc6781f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b60c0 0x7fc6780c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6780cc640 0x7fc6780c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780cbdd0 con 0x7fc6780b60c0 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67d600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6780cbaf0 con 0x7fc67806b320 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67815d5e0 con 0x7fc6780cc640 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc66fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b60c0 0x7fc6780c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc66fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b60c0 0x7fc6780c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49766/0 (socket says 172.21.15.46:49766) 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc66fe00640 1 -- 172.21.15.46:0/1813866177 learned_addr learned my addr 172.21.15.46:0/1813866177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc677e00640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67806b320 0x7fc6781f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67cc00640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6780cc640 0x7fc6780c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc66fe00640 1 -- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6780cc640 msgr2=0x7fc6780c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc66fe00640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6780cc640 0x7fc6780c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc66fe00640 1 -- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67806b320 msgr2=0x7fc6781f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc66fe00640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67806b320 0x7fc6781f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc66fe00640 1 -- 172.21.15.46:0/1813866177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780c99b0 con 0x7fc6780b60c0 2024-04-25T08:48:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc66fe00640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b60c0 0x7fc6780c8d20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc6600017f0 tx=0x7fc66000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc674c00640 1 -- 172.21.15.46:0/1813866177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660005ae0 con 0x7fc6780b60c0 2024-04-25T08:48:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc674c00640 1 -- 172.21.15.46:0/1813866177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc660005ca0 con 0x7fc6780b60c0 2024-04-25T08:48:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6781f41c0 con 0x7fc6780b60c0 2024-04-25T08:48:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc674c00640 1 -- 172.21.15.46:0/1813866177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600072b0 con 0x7fc6780b60c0 2024-04-25T08:48:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6780c46f0 con 0x7fc6780b60c0 2024-04-25T08:48:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc678069010 con 0x7fc6780b60c0 2024-04-25T08:48:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.868+0000 7fc674c00640 1 -- 172.21.15.46:0/1813866177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc660007450 con 0x7fc6780b60c0 2024-04-25T08:48:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.872+0000 7fc674c00640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc654066f60 0x7fc654069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.872+0000 7fc674c00640 1 -- 172.21.15.46:0/1813866177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc660084f40 con 0x7fc6780b60c0 2024-04-25T08:48:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.872+0000 7fc677e00640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc654066f60 0x7fc654069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.872+0000 7fc677e00640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc654066f60 0x7fc654069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc67815bb30 tx=0x7fc668009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:13.876+0000 7fc674c00640 1 -- 172.21.15.46:0/1813866177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6781f41c0 con 0x7fc6780b60c0 2024-04-25T08:48:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.036+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6780c9260 con 0x7fc6780b60c0 2024-04-25T08:48:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.036+0000 7fc674c00640 1 -- 172.21.15.46:0/1813866177 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc660055690 con 0x7fc6780b60c0 2024-04-25T08:48:14.032 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:48:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc654066f60 msgr2=0x7fc654069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc654066f60 0x7fc654069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc67815bb30 tx=0x7fc668009190 comp rx=0 tx=0).stop 2024-04-25T08:48:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b60c0 msgr2=0x7fc6780c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b60c0 0x7fc6780c8d20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc6600017f0 tx=0x7fc66000cbe0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 shutdown_connections 2024-04-25T08:48:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc654066f60 0x7fc654069420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6780cc640 0x7fc6780c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6780b60c0 0x7fc6780c8d20 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 --2- 172.21.15.46:0/1813866177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67806b320 0x7fc6781f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 >> 172.21.15.46:0/1813866177 conn(0x7fc678099a00 msgr2=0x7fc67806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 shutdown_connections 2024-04-25T08:48:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.040+0000 7fc67d600640 1 -- 172.21.15.46:0/1813866177 wait complete. 2024-04-25T08:48:14.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c9800640 1 Processor -- start 2024-04-25T08:48:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c9800640 1 -- start start 2024-04-25T08:48:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a110 0x7fb7c409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c9800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c415f300 con 0x7fb7c415ef20 2024-04-25T08:48:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c9800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c40cd8a0 con 0x7fb7c409a610 2024-04-25T08:48:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c40cddb0 con 0x7fb7c409a110 2024-04-25T08:48:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb7c415ef20 0x7fb7c40cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35986/0 (socket says 172.21.15.46:35986) 2024-04-25T08:48:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c2a00640 1 -- 172.21.15.46:0/1346171504 learned_addr learned my addr 172.21.15.46:0/1346171504 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c3400640 1 --2- 172.21.15.46:0/1346171504 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a110 0x7fb7c409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c2000640 1 -- 172.21.15.46:0/1346171504 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242889229 0 0) 0x7fb7c415f300 con 0x7fb7c415ef20 2024-04-25T08:48:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c2000640 1 -- 172.21.15.46:0/1346171504 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7a8003530 con 0x7fb7c415ef20 2024-04-25T08:48:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c2000640 1 -- 172.21.15.46:0/1346171504 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807563558 0 0) 0x7fb7c40cd8a0 con 0x7fb7c409a610 2024-04-25T08:48:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c2000640 1 -- 172.21.15.46:0/1346171504 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c415f300 con 0x7fb7c409a610 2024-04-25T08:48:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c3400640 1 -- 172.21.15.46:0/1346171504 >> v1:172.21.15.82:6789/0 conn(0x7fb7c409a610 legacy=0x7fb7c4099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c3400640 1 -- 172.21.15.46:0/1346171504 >> v1:172.21.15.46:6789/0 conn(0x7fb7c415ef20 legacy=0x7fb7c40cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.204+0000 7fb7c3400640 1 -- 172.21.15.46:0/1346171504 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c40cc3d0 con 0x7fb7c409a110 2024-04-25T08:48:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c3400640 1 --2- 172.21.15.46:0/1346171504 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a110 0x7fb7c409aa10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009dc0 tx=0x7fb7bc01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa230033173ae373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c2000640 1 -- 172.21.15.46:0/1346171504 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0b2070 con 0x7fb7c409a110 2024-04-25T08:48:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c2000640 1 -- 172.21.15.46:0/1346171504 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc0a5940 con 0x7fb7c409a110 2024-04-25T08:48:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c2000640 1 -- 172.21.15.46:0/1346171504 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0a5d10 con 0x7fb7c409a110 2024-04-25T08:48:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c9800640 1 -- 172.21.15.46:0/1346171504 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a110 msgr2=0x7fb7c409aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c9800640 1 --2- 172.21.15.46:0/1346171504 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a110 0x7fb7c409aa10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009dc0 tx=0x7fb7bc01c990 comp rx=0 tx=0).stop 2024-04-25T08:48:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c9800640 1 -- 172.21.15.46:0/1346171504 shutdown_connections 2024-04-25T08:48:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c9800640 1 --2- 172.21.15.46:0/1346171504 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a110 0x7fb7c409aa10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c9800640 1 -- 172.21.15.46:0/1346171504 >> 172.21.15.46:0/1346171504 conn(0x7fb7c40b3120 msgr2=0x7fb7c40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c9800640 1 -- 172.21.15.46:0/1346171504 shutdown_connections 2024-04-25T08:48:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.208+0000 7fb7c9800640 1 -- 172.21.15.46:0/1346171504 wait complete. 2024-04-25T08:48:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 Processor -- start 2024-04-25T08:48:14.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 -- start start 2024-04-25T08:48:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c409a110 0x7fb7c4156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a610 0x7fb7c40b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7c415ef20 0x7fb7c40b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c40cc190 con 0x7fb7c415ef20 2024-04-25T08:48:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c40cc010 con 0x7fb7c409a110 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c4169890 con 0x7fb7c409a610 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7c415ef20 0x7fb7c40b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7c415ef20 0x7fb7c40b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49770/0 (socket says 172.21.15.46:49770) 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c3400640 1 -- 172.21.15.46:0/3121666501 learned_addr learned my addr 172.21.15.46:0/3121666501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c2a00640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c409a110 0x7fb7c4156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7bbe00640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a610 0x7fb7c40b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c3400640 1 -- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a610 msgr2=0x7fb7c40b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c3400640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a610 0x7fb7c40b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c3400640 1 -- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c409a110 msgr2=0x7fb7c4156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c3400640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c409a110 0x7fb7c4156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c3400640 1 -- 172.21.15.46:0/3121666501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c40b49f0 con 0x7fb7c415ef20 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c3400640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7c415ef20 0x7fb7c40b57b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0195a0 tx=0x7fb7bc019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:14.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7baa00640 1 -- 172.21.15.46:0/3121666501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0b2050 con 0x7fb7c415ef20 2024-04-25T08:48:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7baa00640 1 -- 172.21.15.46:0/3121666501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc0be070 con 0x7fb7c415ef20 2024-04-25T08:48:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7baa00640 1 -- 172.21.15.46:0/3121666501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0ad890 con 0x7fb7c415ef20 2024-04-25T08:48:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c40b4cd0 con 0x7fb7c415ef20 2024-04-25T08:48:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c40b5290 con 0x7fb7c415ef20 2024-04-25T08:48:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.212+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c4069080 con 0x7fb7c415ef20 2024-04-25T08:48:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.216+0000 7fb7baa00640 1 -- 172.21.15.46:0/3121666501 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7bc0ada30 con 0x7fb7c415ef20 2024-04-25T08:48:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.216+0000 7fb7baa00640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7a0066f90 0x7fb7a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.216+0000 7fb7baa00640 1 -- 172.21.15.46:0/3121666501 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb7bc11fd10 con 0x7fb7c415ef20 2024-04-25T08:48:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.216+0000 7fb7c2a00640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7a0066f90 0x7fb7a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.216+0000 7fb7c2a00640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7a0066f90 0x7fb7a0069450 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4000f70 tx=0x7fb7b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:14.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.220+0000 7fb7baa00640 1 -- 172.21.15.46:0/3121666501 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7bc125020 con 0x7fb7c415ef20 2024-04-25T08:48:14.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.412+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7c40b3e60 con 0x7fb7c415ef20 2024-04-25T08:48:14.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7baa00640 1 -- 172.21.15.46:0/3121666501 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fb7bc0c7020 con 0x7fb7c415ef20 2024-04-25T08:48:14.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:14.409 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:47:43.911465+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/4112568749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":4112568749},{"type":"v1","addr":"172.21.15.46:6835","nonce":4112568749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:48:14.410 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 53 2024-04-25T08:48:14.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7a0066f90 msgr2=0x7fb7a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7a0066f90 0x7fb7a0069450 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4000f70 tx=0x7fb7b4009040 comp rx=0 tx=0).stop 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7c415ef20 msgr2=0x7fb7c40b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7c415ef20 0x7fb7c40b57b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0195a0 tx=0x7fb7bc019730 comp rx=0 tx=0).stop 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 shutdown_connections 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb7a0066f90 0x7fb7a0069450 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7c415ef20 0x7fb7c40b57b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7c409a610 0x7fb7c40b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 --2- 172.21.15.46:0/3121666501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7c409a110 0x7fb7c4156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 >> 172.21.15.46:0/3121666501 conn(0x7fb7c40b3120 msgr2=0x7fb7c4099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 shutdown_connections 2024-04-25T08:48:14.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.416+0000 7fb7c9800640 1 -- 172.21.15.46:0/3121666501 wait complete. 2024-04-25T08:48:14.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb4000640 1 Processor -- start 2024-04-25T08:48:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb4000640 1 -- start start 2024-04-25T08:48:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb4000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac15ef20 0x7fdeac0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb4000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdeac099a00 0x7fdeac099eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb4000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeac15f300 con 0x7fdeac0cd6d0 2024-04-25T08:48:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb4000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeac0cddb0 con 0x7fdeac099a00 2024-04-25T08:48:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb4000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac0cc3d0 con 0x7fdeac15ef20 2024-04-25T08:48:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb0c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdeac0cd6d0 0x7fdeac09a610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36002/0 (socket says 172.21.15.46:36002) 2024-04-25T08:48:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb0c00640 1 -- 172.21.15.46:0/822577198 learned_addr learned my addr 172.21.15.46:0/822577198 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb1600640 1 --2- 172.21.15.46:0/822577198 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac15ef20 0x7fdeac0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb2000640 1 --2- 172.21.15.46:0/822577198 >> v2:172.21.15.82:3300/0 conn(0x7fdeac099a00 0x7fdeac099eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeabe00640 1 -- 172.21.15.46:0/822577198 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827082692 0 0) 0x7fdeac15f300 con 0x7fdeac0cd6d0 2024-04-25T08:48:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeabe00640 1 -- 172.21.15.46:0/822577198 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde94003530 con 0x7fdeac0cd6d0 2024-04-25T08:48:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb1600640 1 -- 172.21.15.46:0/822577198 >> v2:172.21.15.82:3300/0 conn(0x7fdeac099a00 msgr2=0x7fdeac099eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb1600640 1 --2- 172.21.15.46:0/822577198 >> v2:172.21.15.82:3300/0 conn(0x7fdeac099a00 0x7fdeac099eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb1600640 1 -- 172.21.15.46:0/822577198 >> v1:172.21.15.46:6789/0 conn(0x7fdeac0cd6d0 legacy=0x7fdeac09a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeabe00640 1 -- 172.21.15.46:0/822577198 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3165738078 0 0) 0x7fde94003530 con 0x7fdeac0cd6d0 2024-04-25T08:48:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.580+0000 7fdeb1600640 1 -- 172.21.15.46:0/822577198 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac0cc6b0 con 0x7fdeac15ef20 2024-04-25T08:48:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeb1600640 1 --2- 172.21.15.46:0/822577198 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac15ef20 0x7fdeac0cd190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdea00097c0 tx=0x7fdea001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0996a51cdc4182a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeabe00640 1 -- 172.21.15.46:0/822577198 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea00b0070 con 0x7fdeac15ef20 2024-04-25T08:48:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeabe00640 1 -- 172.21.15.46:0/822577198 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea00ab400 con 0x7fdeac15ef20 2024-04-25T08:48:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeabe00640 1 -- 172.21.15.46:0/822577198 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98001680 con 0x7fdeac15ef20 2024-04-25T08:48:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeb4000640 1 -- 172.21.15.46:0/822577198 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac15ef20 msgr2=0x7fdeac0cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeb4000640 1 --2- 172.21.15.46:0/822577198 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac15ef20 0x7fdeac0cd190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdea00097c0 tx=0x7fdea001c3c0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeb4000640 1 -- 172.21.15.46:0/822577198 shutdown_connections 2024-04-25T08:48:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeb4000640 1 --2- 172.21.15.46:0/822577198 >> v2:172.21.15.82:3300/0 conn(0x7fdeac099a00 0x7fdeac099eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeb4000640 1 --2- 172.21.15.46:0/822577198 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac15ef20 0x7fdeac0cd190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeb4000640 1 -- 172.21.15.46:0/822577198 >> 172.21.15.46:0/822577198 conn(0x7fdeac0b3120 msgr2=0x7fdeac0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeb4000640 1 -- 172.21.15.46:0/822577198 shutdown_connections 2024-04-25T08:48:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.584+0000 7fdeb4000640 1 -- 172.21.15.46:0/822577198 wait complete. 2024-04-25T08:48:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 Processor -- start 2024-04-25T08:48:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 -- start start 2024-04-25T08:48:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac099a00 0x7fdeac0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeac15ef20 0x7fdeac0b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeac06c730 0x7fdeac0c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac09aea0 con 0x7fdeac06c730 2024-04-25T08:48:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac09ad20 con 0x7fdeac15ef20 2024-04-25T08:48:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeac168880 con 0x7fdeac099a00 2024-04-25T08:48:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeac06c730 0x7fdeac0c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeac06c730 0x7fdeac0c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49786/0 (socket says 172.21.15.46:49786) 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb2000640 1 -- 172.21.15.46:0/1985627264 learned_addr learned my addr 172.21.15.46:0/1985627264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb1600640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac099a00 0x7fdeac0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb0c00640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeac15ef20 0x7fdeac0b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb2000640 1 -- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac099a00 msgr2=0x7fdeac0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb2000640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac099a00 0x7fdeac0cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb2000640 1 -- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeac15ef20 msgr2=0x7fdeac0b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb2000640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeac15ef20 0x7fdeac0b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb2000640 1 -- 172.21.15.46:0/1985627264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac06ccd0 con 0x7fdeac06c730 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb2000640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeac06c730 0x7fdeac0c64d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009d30 tx=0x7fdea400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdea9600640 1 -- 172.21.15.46:0/1985627264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4018020 con 0x7fdeac06c730 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdea9600640 1 -- 172.21.15.46:0/1985627264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea4014480 con 0x7fdeac06c730 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac06cfb0 con 0x7fdeac06c730 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdea9600640 1 -- 172.21.15.46:0/1985627264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4014870 con 0x7fdeac06c730 2024-04-25T08:48:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeac06d570 con 0x7fdeac06c730 2024-04-25T08:48:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.588+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeac069080 con 0x7fdeac06c730 2024-04-25T08:48:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.592+0000 7fdea9600640 1 -- 172.21.15.46:0/1985627264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdea401b020 con 0x7fdeac06c730 2024-04-25T08:48:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.592+0000 7fdea9600640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde8c066f60 0x7fde8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.592+0000 7fdea9600640 1 -- 172.21.15.46:0/1985627264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdea4085c30 con 0x7fdeac06c730 2024-04-25T08:48:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.592+0000 7fdeb1600640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde8c066f60 0x7fde8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.592+0000 7fdeb1600640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde8c066f60 0x7fde8c069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdea00ad000 tx=0x7fdea00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.596+0000 7fdea9600640 1 -- 172.21.15.46:0/1985627264 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdea4022070 con 0x7fdeac06c730 2024-04-25T08:48:14.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.764+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdeac070990 con 0x7fdeac06c730 2024-04-25T08:48:14.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.764+0000 7fdea9600640 1 -- 172.21.15.46:0/1985627264 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17024 (secure 0 0 0) 0x7fdea4010040 con 0x7fdeac06c730 2024-04-25T08:48:14.760 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:14.760 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:56.883035+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:46:44.505856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:46:45.072095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/960745191":"2024-04-26T08:46:56.883011+0000","172.21.15.46:6833/960745191":"2024-04-26T08:46:56.883011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde8c066f60 msgr2=0x7fde8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde8c066f60 0x7fde8c069420 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdea00ad000 tx=0x7fdea00a6000 comp rx=0 tx=0).stop 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeac06c730 msgr2=0x7fdeac0c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeac06c730 0x7fdeac0c64d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009d30 tx=0x7fdea400c490 comp rx=0 tx=0).stop 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 shutdown_connections 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde8c066f60 0x7fde8c069420 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeac06c730 0x7fdeac0c64d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdeac15ef20 0x7fdeac0b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 --2- 172.21.15.46:0/1985627264 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdeac099a00 0x7fdeac0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 >> 172.21.15.46:0/1985627264 conn(0x7fdeac0b3120 msgr2=0x7fdeac154fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:14.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 shutdown_connections 2024-04-25T08:48:14.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.768+0000 7fdeb4000640 1 -- 172.21.15.46:0/1985627264 wait complete. 2024-04-25T08:48:14.781 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:48:14.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa20a600640 1 Processor -- start 2024-04-25T08:48:14.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa20a600640 1 -- start start 2024-04-25T08:48:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa20a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 0x7fa20415d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa20a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2040afd40 0x7fa20415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa20a600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20406ed70 con 0x7fa2040cf520 2024-04-25T08:48:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa20a600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2040afb40 con 0x7fa2040afd40 2024-04-25T08:48:14.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa20a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040d13f0 con 0x7fa2040cf140 2024-04-25T08:48:14.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa1faa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2040afd40 0x7fa20415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa203e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa2040cf520 0x7fa2040ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36010/0 (socket says 172.21.15.46:36010) 2024-04-25T08:48:14.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa203e00640 1 -- 172.21.15.46:0/3842996 learned_addr learned my addr 172.21.15.46:0/3842996 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa203400640 1 --2- 172.21.15.46:0/3842996 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 0x7fa20415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa1faa00640 1 -- 172.21.15.46:0/3842996 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 msgr2=0x7fa20415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa1faa00640 1 --2- 172.21.15.46:0/3842996 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 0x7fa20415d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa1faa00640 1 -- 172.21.15.46:0/3842996 >> v1:172.21.15.46:6789/0 conn(0x7fa2040cf520 legacy=0x7fa2040ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa1faa00640 1 -- 172.21.15.46:0/3842996 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204162530 con 0x7fa2040afd40 2024-04-25T08:48:14.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.928+0000 7fa202a00640 1 -- 172.21.15.46:0/3842996 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830599690 0 0) 0x7fa1fc001120 con 0x7fa2040cf520 2024-04-25T08:48:14.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa1faa00640 1 --2- 172.21.15.46:0/3842996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040afd40 0x7fa20415dad0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0097c0 tx=0x7fa1ec01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3033bf0669e29d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:14.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa202a00640 1 -- 172.21.15.46:0/3842996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0b0070 con 0x7fa2040afd40 2024-04-25T08:48:14.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa202a00640 1 -- 172.21.15.46:0/3842996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec0ab600 con 0x7fa2040afd40 2024-04-25T08:48:14.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa202a00640 1 -- 172.21.15.46:0/3842996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0ab9d0 con 0x7fa2040afd40 2024-04-25T08:48:14.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa20a600640 1 -- 172.21.15.46:0/3842996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040afd40 msgr2=0x7fa20415dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa20a600640 1 --2- 172.21.15.46:0/3842996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040afd40 0x7fa20415dad0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0097c0 tx=0x7fa1ec01c380 comp rx=0 tx=0).stop 2024-04-25T08:48:14.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa20a600640 1 -- 172.21.15.46:0/3842996 shutdown_connections 2024-04-25T08:48:14.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa20a600640 1 --2- 172.21.15.46:0/3842996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040afd40 0x7fa20415dad0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa20a600640 1 --2- 172.21.15.46:0/3842996 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 0x7fa20415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa20a600640 1 -- 172.21.15.46:0/3842996 >> 172.21.15.46:0/3842996 conn(0x7fa2040728c0 msgr2=0x7fa204099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:14.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa20a600640 1 -- 172.21.15.46:0/3842996 shutdown_connections 2024-04-25T08:48:14.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.932+0000 7fa20a600640 1 -- 172.21.15.46:0/3842996 wait complete. 2024-04-25T08:48:14.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 Processor -- start 2024-04-25T08:48:14.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 -- start start 2024-04-25T08:48:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040afd40 0x7fa204158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 0x7fa20415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040cf520 0x7fa20414c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2040af7f0 con 0x7fa2040cf520 2024-04-25T08:48:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa204162010 con 0x7fa2040afd40 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20406b600 con 0x7fa2040cf140 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040cf520 0x7fa20414c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040cf520 0x7fa20414c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49796/0 (socket says 172.21.15.46:49796) 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203e00640 1 -- 172.21.15.46:0/1448952324 learned_addr learned my addr 172.21.15.46:0/1448952324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203400640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040afd40 0x7fa204158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa1faa00640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 0x7fa20415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203e00640 1 -- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 msgr2=0x7fa20415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203e00640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 0x7fa20415eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203e00640 1 -- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040afd40 msgr2=0x7fa204158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203e00640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040afd40 0x7fa204158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203e00640 1 -- 172.21.15.46:0/1448952324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204157d60 con 0x7fa2040cf520 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa203e00640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040cf520 0x7fa20414c9e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c780 tx=0x7fa1fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1448952324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc018070 con 0x7fa2040cf520 2024-04-25T08:48:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1448952324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc0070d0 con 0x7fa2040cf520 2024-04-25T08:48:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1448952324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0074c0 con 0x7fa2040cf520 2024-04-25T08:48:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa204158040 con 0x7fa2040cf520 2024-04-25T08:48:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.936+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa204158600 con 0x7fa2040cf520 2024-04-25T08:48:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.940+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20406ccb0 con 0x7fa2040cf520 2024-04-25T08:48:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.940+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1448952324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1fc0047f0 con 0x7fa2040cf520 2024-04-25T08:48:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.940+0000 7fa1fbe00640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa1e0066f60 0x7fa1e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.940+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1448952324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa1fc084410 con 0x7fa2040cf520 2024-04-25T08:48:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.940+0000 7fa203400640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa1e0066f60 0x7fa1e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.944+0000 7fa203400640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa1e0066f60 0x7fa1e0069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa204159c90 tx=0x7fa1f400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:14.944+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1448952324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa20406ed70 con 0x7fa2040cf520 2024-04-25T08:48:15.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.140+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa204072f90 con 0x7fa2040cf520 2024-04-25T08:48:15.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.140+0000 7fa1fbe00640 1 -- 172.21.15.46:0/1448952324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fa1fc054be0 con 0x7fa2040cf520 2024-04-25T08:48:15.136 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:15.136 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:47:43.911465+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/4112568749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":4112568749},{"type":"v1","addr":"172.21.15.46:6835","nonce":4112568749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:48:15.137 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 53 2024-04-25T08:48:15.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa1e0066f60 msgr2=0x7fa1e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa1e0066f60 0x7fa1e0069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa204159c90 tx=0x7fa1f400b040 comp rx=0 tx=0).stop 2024-04-25T08:48:15.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040cf520 msgr2=0x7fa20414c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040cf520 0x7fa20414c9e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c780 tx=0x7fa1fc00cc40 comp rx=0 tx=0).stop 2024-04-25T08:48:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 shutdown_connections 2024-04-25T08:48:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa1e0066f60 0x7fa1e0069420 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2040cf520 0x7fa20414c9e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2040cf140 0x7fa20415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 --2- 172.21.15.46:0/1448952324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2040afd40 0x7fa204158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 >> 172.21.15.46:0/1448952324 conn(0x7fa2040728c0 msgr2=0x7fa2040adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 shutdown_connections 2024-04-25T08:48:15.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.144+0000 7fa20a600640 1 -- 172.21.15.46:0/1448952324 wait complete. 2024-04-25T08:48:15.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf6b000640 1 Processor -- start 2024-04-25T08:48:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf6b000640 1 -- start start 2024-04-25T08:48:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf6b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcf640ccde0 0x7fcf640cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf6b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6406bdc0 con 0x7fcf64099a00 2024-04-25T08:48:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf6b000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6406c0a0 con 0x7fcf640b2240 2024-04-25T08:48:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf6b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf6406db90 con 0x7fcf640ccde0 2024-04-25T08:48:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf5b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcf640ccde0 0x7fcf640cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf63e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcf64099a00 0x7fcf64099de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36012/0 (socket says 172.21.15.46:36012) 2024-04-25T08:48:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf63e00640 1 -- 172.21.15.46:0/1348512395 learned_addr learned my addr 172.21.15.46:0/1348512395 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:15.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf5b400640 1 -- 172.21.15.46:0/1348512395 >> v1:172.21.15.82:6789/0 conn(0x7fcf640b2240 legacy=0x7fcf640b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf5b400640 1 -- 172.21.15.46:0/1348512395 >> v1:172.21.15.46:6789/0 conn(0x7fcf64099a00 legacy=0x7fcf64099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf5b400640 1 -- 172.21.15.46:0/1348512395 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6406c380 con 0x7fcf640ccde0 2024-04-25T08:48:15.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.304+0000 7fcf63400640 1 -- 172.21.15.46:0/1348512395 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359664632 0 0) 0x7fcf6406bdc0 con 0x7fcf64099a00 2024-04-25T08:48:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf5b400640 1 --2- 172.21.15.46:0/1348512395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640ccde0 0x7fcf640cd290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0097f0 tx=0x7fcf4c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431cb3e21659f0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf63400640 1 -- 172.21.15.46:0/1348512395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0b0070 con 0x7fcf640ccde0 2024-04-25T08:48:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf63400640 1 -- 172.21.15.46:0/1348512395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c0ab600 con 0x7fcf640ccde0 2024-04-25T08:48:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf63400640 1 -- 172.21.15.46:0/1348512395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0ab9d0 con 0x7fcf640ccde0 2024-04-25T08:48:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf6b000640 1 -- 172.21.15.46:0/1348512395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640ccde0 msgr2=0x7fcf640cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf6b000640 1 --2- 172.21.15.46:0/1348512395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640ccde0 0x7fcf640cd290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0097f0 tx=0x7fcf4c01c3e0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf6b000640 1 -- 172.21.15.46:0/1348512395 shutdown_connections 2024-04-25T08:48:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf6b000640 1 --2- 172.21.15.46:0/1348512395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640ccde0 0x7fcf640cd290 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf6b000640 1 -- 172.21.15.46:0/1348512395 >> 172.21.15.46:0/1348512395 conn(0x7fcf640b3120 msgr2=0x7fcf640b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf6b000640 1 -- 172.21.15.46:0/1348512395 shutdown_connections 2024-04-25T08:48:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.308+0000 7fcf6b000640 1 -- 172.21.15.46:0/1348512395 wait complete. 2024-04-25T08:48:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 Processor -- start 2024-04-25T08:48:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 -- start start 2024-04-25T08:48:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf64099a00 0x7fcf6414cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640b2240 0x7fcf64147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf640ccde0 0x7fcf64147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640cd7d0 con 0x7fcf640b2240 2024-04-25T08:48:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640b2da0 con 0x7fcf640ccde0 2024-04-25T08:48:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6414f040 con 0x7fcf64099a00 2024-04-25T08:48:15.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640b2240 0x7fcf64147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640b2240 0x7fcf64147520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49826/0 (socket says 172.21.15.46:49826) 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf5b400640 1 -- 172.21.15.46:0/947373050 learned_addr learned my addr 172.21.15.46:0/947373050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf63e00640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf64099a00 0x7fcf6414cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf69000640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf640ccde0 0x7fcf64147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf5b400640 1 -- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf64099a00 msgr2=0x7fcf6414cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf5b400640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf64099a00 0x7fcf6414cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf5b400640 1 -- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf640ccde0 msgr2=0x7fcf64147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf5b400640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf640ccde0 0x7fcf64147a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf5b400640 1 -- 172.21.15.46:0/947373050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf64149520 con 0x7fcf640b2240 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf63e00640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf64099a00 0x7fcf6414cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf69000640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf640ccde0 0x7fcf64147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf5b400640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640b2240 0x7fcf64147520 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0097f0 tx=0x7fcf4c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf60c00640 1 -- 172.21.15.46:0/947373050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0bb070 con 0x7fcf640b2240 2024-04-25T08:48:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf60c00640 1 -- 172.21.15.46:0/947373050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c0b76a0 con 0x7fcf640b2240 2024-04-25T08:48:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6414ea80 con 0x7fcf640b2240 2024-04-25T08:48:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf60c00640 1 -- 172.21.15.46:0/947373050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0b0040 con 0x7fcf640b2240 2024-04-25T08:48:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf641497c0 con 0x7fcf640b2240 2024-04-25T08:48:15.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.312+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf64069080 con 0x7fcf640b2240 2024-04-25T08:48:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.316+0000 7fcf60c00640 1 -- 172.21.15.46:0/947373050 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf4c0be070 con 0x7fcf640b2240 2024-04-25T08:48:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.316+0000 7fcf60c00640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf40066f90 0x7fcf40069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.316+0000 7fcf60c00640 1 -- 172.21.15.46:0/947373050 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcf4c11f860 con 0x7fcf640b2240 2024-04-25T08:48:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.316+0000 7fcf63e00640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf40066f90 0x7fcf40069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.316+0000 7fcf63e00640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf40066f90 0x7fcf40069450 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcf54000f70 tx=0x7fcf54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.320+0000 7fcf60c00640 1 -- 172.21.15.46:0/947373050 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf4c016090 con 0x7fcf640b2240 2024-04-25T08:48:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.512+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf640cf890 con 0x7fcf640b2240 2024-04-25T08:48:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf60c00640 1 -- 172.21.15.46:0/947373050 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fcf4c0f07a0 con 0x7fcf640b2240 2024-04-25T08:48:15.509 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:15.509 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:46:46.077710+0000","modified":"2024-04-25T08:47:43.911465+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":79,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.46:6835/4112568749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":4112568749},{"type":"v1","addr":"172.21.15.46:6835","nonce":4112568749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:48:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 53 2024-04-25T08:48:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf40066f90 msgr2=0x7fcf40069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf40066f90 0x7fcf40069450 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcf54000f70 tx=0x7fcf54009040 comp rx=0 tx=0).stop 2024-04-25T08:48:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640b2240 msgr2=0x7fcf64147520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640b2240 0x7fcf64147520 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0097f0 tx=0x7fcf4c019180 comp rx=0 tx=0).stop 2024-04-25T08:48:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 shutdown_connections 2024-04-25T08:48:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcf40066f90 0x7fcf40069450 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf640ccde0 0x7fcf64147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf640b2240 0x7fcf64147520 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 --2- 172.21.15.46:0/947373050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcf64099a00 0x7fcf6414cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 >> 172.21.15.46:0/947373050 conn(0x7fcf640b3120 msgr2=0x7fcf6406d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 shutdown_connections 2024-04-25T08:48:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.516+0000 7fcf6b000640 1 -- 172.21.15.46:0/947373050 wait complete. 2024-04-25T08:48:15.531 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7736600640 1 Processor -- start 2024-04-25T08:48:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7736600640 1 -- start start 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7736600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f773006c2b0 0x7f773015f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7736600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300b2da0 con 0x7f77300b29c0 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7736600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77301623a0 con 0x7f7730099a00 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7736600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7730161b80 con 0x7f773006c2b0 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7727e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f773006c2b0 0x7f773015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7727e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f773006c2b0 0x7f773015f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49830/0 (socket says 172.21.15.46:49830) 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7727e00640 1 -- 172.21.15.46:0/2728403314 learned_addr learned my addr 172.21.15.46:0/2728403314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.676+0000 7f7735200640 1 --1- 172.21.15.46:0/2728403314 >> v1:172.21.15.46:6789/0 conn(0x7f77300b29c0 0x7f7730161470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36018/0 (socket says 172.21.15.46:36018) 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7727e00640 1 -- 172.21.15.46:0/2728403314 >> v1:172.21.15.82:6789/0 conn(0x7f7730099a00 legacy=0x7f7730099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7727e00640 1 -- 172.21.15.46:0/2728403314 >> v1:172.21.15.46:6789/0 conn(0x7f77300b29c0 legacy=0x7f7730161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7727e00640 1 -- 172.21.15.46:0/2728403314 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7730161de0 con 0x7f773006c2b0 2024-04-25T08:48:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f772fe00640 1 -- 172.21.15.46:0/2728403314 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225875209 0 0) 0x7f77300b2da0 con 0x7f77300b29c0 2024-04-25T08:48:15.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7727e00640 1 --2- 172.21.15.46:0/2728403314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773006c2b0 0x7f773015f250 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7718009870 tx=0x7f771801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c75345293c045 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f772fe00640 1 -- 172.21.15.46:0/2728403314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77180aa650 con 0x7f773006c2b0 2024-04-25T08:48:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f772fe00640 1 -- 172.21.15.46:0/2728403314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77180af070 con 0x7f773006c2b0 2024-04-25T08:48:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f772fe00640 1 -- 172.21.15.46:0/2728403314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77180aaa10 con 0x7f773006c2b0 2024-04-25T08:48:15.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7736600640 1 -- 172.21.15.46:0/2728403314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773006c2b0 msgr2=0x7f773015f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7736600640 1 --2- 172.21.15.46:0/2728403314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773006c2b0 0x7f773015f250 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7718009870 tx=0x7f771801c4f0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7736600640 1 -- 172.21.15.46:0/2728403314 shutdown_connections 2024-04-25T08:48:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7736600640 1 --2- 172.21.15.46:0/2728403314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773006c2b0 0x7f773015f250 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7736600640 1 -- 172.21.15.46:0/2728403314 >> 172.21.15.46:0/2728403314 conn(0x7f77300b2f90 msgr2=0x7f77300b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.680+0000 7f7736600640 1 -- 172.21.15.46:0/2728403314 shutdown_connections 2024-04-25T08:48:15.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7736600640 1 -- 172.21.15.46:0/2728403314 wait complete. 2024-04-25T08:48:15.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7736600640 1 Processor -- start 2024-04-25T08:48:15.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7736600640 1 -- start start 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7736600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006c2b0 0x7f7730154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7736600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730099a00 0x7f7730150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7736600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300b29c0 0x7f77300c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7736600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773015eab0 con 0x7f77300b29c0 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7736600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773015e930 con 0x7f7730099a00 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7736600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773015ec30 con 0x7f773006c2b0 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300b29c0 0x7f77300c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300b29c0 0x7f77300c4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49836/0 (socket says 172.21.15.46:49836) 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006c2b0 0x7f7730154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735c00640 1 -- 172.21.15.46:0/1378160016 learned_addr learned my addr 172.21.15.46:0/1378160016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7727e00640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730099a00 0x7f7730150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735c00640 1 -- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006c2b0 msgr2=0x7f7730154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735c00640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006c2b0 0x7f7730154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735c00640 1 -- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730099a00 msgr2=0x7f7730150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735c00640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730099a00 0x7f7730150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735c00640 1 -- 172.21.15.46:0/1378160016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7730155270 con 0x7f77300b29c0 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f7735c00640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300b29c0 0x7f77300c4fd0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f772800b9a0 tx=0x7f772800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.684+0000 7f772d600640 1 -- 172.21.15.46:0/1378160016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728019070 con 0x7f77300b29c0 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.688+0000 7f772d600640 1 -- 172.21.15.46:0/1378160016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772800cba0 con 0x7f77300b29c0 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.688+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77300c5ea0 con 0x7f77300b29c0 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.688+0000 7f772d600640 1 -- 172.21.15.46:0/1378160016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728014440 con 0x7f77300b29c0 2024-04-25T08:48:15.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.688+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77300c7360 con 0x7f77300b29c0 2024-04-25T08:48:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.688+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77300c7170 con 0x7f77300b29c0 2024-04-25T08:48:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.688+0000 7f772d600640 1 -- 172.21.15.46:0/1378160016 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f772801c070 con 0x7f77300b29c0 2024-04-25T08:48:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.688+0000 7f772d600640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f770c066f60 0x7f770c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:15.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.688+0000 7f772d600640 1 -- 172.21.15.46:0/1378160016 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7728084d70 con 0x7f77300b29c0 2024-04-25T08:48:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.688+0000 7f7735200640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f770c066f60 0x7f770c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:15.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.692+0000 7f7735200640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f770c066f60 0x7f770c069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7720000f70 tx=0x7f7720007df0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:15.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.692+0000 7f772d600640 1 -- 172.21.15.46:0/1378160016 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7730155270 con 0x7f77300b29c0 2024-04-25T08:48:15.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.856+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7730069010 con 0x7f77300b29c0 2024-04-25T08:48:15.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.856+0000 7f772d600640 1 -- 172.21.15.46:0/1378160016 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17024 (secure 0 0 0) 0x7f7728055540 con 0x7f77300b29c0 2024-04-25T08:48:15.851 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:15.851 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":79,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:46:56.883035+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:46:44.505856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:46:45.072095+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/960745191":"2024-04-26T08:46:56.883011+0000","172.21.15.46:6833/960745191":"2024-04-26T08:46:56.883011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:15.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f770c066f60 msgr2=0x7f770c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f770c066f60 0x7f770c069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7720000f70 tx=0x7f7720007df0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300b29c0 msgr2=0x7f77300c4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:15.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300b29c0 0x7f77300c4fd0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f772800b9a0 tx=0x7f772800be60 comp rx=0 tx=0).stop 2024-04-25T08:48:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 shutdown_connections 2024-04-25T08:48:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f770c066f60 0x7f770c069420 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77300b29c0 0x7f77300c4fd0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7730099a00 0x7f7730150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 --2- 172.21.15.46:0/1378160016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f773006c2b0 0x7f7730154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 >> 172.21.15.46:0/1378160016 conn(0x7f77300b2f90 msgr2=0x7f773006fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 shutdown_connections 2024-04-25T08:48:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:15.860+0000 7f7736600640 1 -- 172.21.15.46:0/1378160016 wait complete. 2024-04-25T08:48:15.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:48:16.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.020+0000 7f4cfd600640 1 Processor -- start 2024-04-25T08:48:16.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.020+0000 7f4cfd600640 1 -- start start 2024-04-25T08:48:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cfd600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 0x7f4cf815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cfd600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80cc700 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cfd600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80cc9e0 con 0x7f4cf80cdc50 2024-04-25T08:48:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cfd600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80cccc0 con 0x7f4cf8159230 2024-04-25T08:48:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cefe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4cf80cbeb0 0x7f4cf80cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36030/0 (socket says 172.21.15.46:36030) 2024-04-25T08:48:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cefe00640 1 -- 172.21.15.46:0/1753456442 learned_addr learned my addr 172.21.15.46:0/1753456442 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cf6a00640 1 --2- 172.21.15.46:0/1753456442 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 0x7f4cf815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cf6000640 1 -- 172.21.15.46:0/1753456442 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464296312 0 0) 0x7f4cf80cc700 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cf6000640 1 -- 172.21.15.46:0/1753456442 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cdc003530 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cf6000640 1 -- 172.21.15.46:0/1753456442 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044808951 0 0) 0x7f4cf80cc9e0 con 0x7f4cf80cdc50 2024-04-25T08:48:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cf6a00640 1 -- 172.21.15.46:0/1753456442 >> v1:172.21.15.82:6789/0 conn(0x7f4cf80cdc50 legacy=0x7f4cf815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cf6a00640 1 -- 172.21.15.46:0/1753456442 >> v1:172.21.15.46:6789/0 conn(0x7f4cf80cbeb0 legacy=0x7f4cf80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.024+0000 7f4cf6a00640 1 -- 172.21.15.46:0/1753456442 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80ccfa0 con 0x7f4cf8159230 2024-04-25T08:48:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cf6a00640 1 --2- 172.21.15.46:0/1753456442 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 0x7f4cf815bdb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80097f0 tx=0x7f4ce801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=814b814d0300176d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cf6000640 1 -- 172.21.15.46:0/1753456442 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80ab640 con 0x7f4cf8159230 2024-04-25T08:48:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cf6000640 1 -- 172.21.15.46:0/1753456442 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce80b0070 con 0x7f4cf8159230 2024-04-25T08:48:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cf6000640 1 -- 172.21.15.46:0/1753456442 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80aba00 con 0x7f4cf8159230 2024-04-25T08:48:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cfd600640 1 -- 172.21.15.46:0/1753456442 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 msgr2=0x7f4cf815bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cfd600640 1 --2- 172.21.15.46:0/1753456442 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 0x7f4cf815bdb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80097f0 tx=0x7f4ce801c420 comp rx=0 tx=0).stop 2024-04-25T08:48:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cfd600640 1 -- 172.21.15.46:0/1753456442 shutdown_connections 2024-04-25T08:48:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cfd600640 1 --2- 172.21.15.46:0/1753456442 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 0x7f4cf815bdb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cfd600640 1 -- 172.21.15.46:0/1753456442 >> 172.21.15.46:0/1753456442 conn(0x7f4cf80b12a0 msgr2=0x7f4cf80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cfd600640 1 -- 172.21.15.46:0/1753456442 shutdown_connections 2024-04-25T08:48:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cfd600640 1 -- 172.21.15.46:0/1753456442 wait complete. 2024-04-25T08:48:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cfd600640 1 Processor -- start 2024-04-25T08:48:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.028+0000 7f4cfd600640 1 -- start start 2024-04-25T08:48:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cfd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf80cbeb0 0x7f4cf8143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cfd600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf80cdc50 0x7f4cf8158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cfd600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 0x7f4cf8158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cfd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf815e120 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cfd600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80cc4a0 con 0x7f4cf80cdc50 2024-04-25T08:48:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cfd600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf815e2a0 con 0x7f4cf8159230 2024-04-25T08:48:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf80cbeb0 0x7f4cf8143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf80cbeb0 0x7f4cf8143d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49844/0 (socket says 172.21.15.46:49844) 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf6a00640 1 -- 172.21.15.46:0/646727477 learned_addr learned my addr 172.21.15.46:0/646727477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cefe00640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf80cdc50 0x7f4cf8158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf7400640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 0x7f4cf8158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf6a00640 1 -- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 msgr2=0x7f4cf8158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf6a00640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 0x7f4cf8158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf6a00640 1 -- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf80cdc50 msgr2=0x7f4cf8158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf6a00640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf80cdc50 0x7f4cf8158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf6a00640 1 -- 172.21.15.46:0/646727477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80aeca0 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cefe00640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf80cdc50 0x7f4cf8158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:16.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cf6a00640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf80cbeb0 0x7f4cf8143d90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80ad040 tx=0x7f4ce801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4ceea00640 1 -- 172.21.15.46:0/646727477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80bf410 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf81410c0 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4ceea00640 1 -- 172.21.15.46:0/646727477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce80bf5b0 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4ceea00640 1 -- 172.21.15.46:0/646727477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80bf9a0 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf812ab30 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf8069080 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.032+0000 7f4ceea00640 1 -- 172.21.15.46:0/646727477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ce80bd020 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.036+0000 7f4ceea00640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cd406b370 0x7f4cd406d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.036+0000 7f4ceea00640 1 -- 172.21.15.46:0/646727477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4ce811f330 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.036+0000 7f4cefe00640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cd406b370 0x7f4cd406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.036+0000 7f4cefe00640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cd406b370 0x7f4cd406d830 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0005890 tx=0x7f4ce0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:16.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.040+0000 7f4ceea00640 1 -- 172.21.15.46:0/646727477 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ce80c6070 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.236+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4cf812b230 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.232 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:48:16.236+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:48:16.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.240+0000 7f4ceea00640 1 -- 172.21.15.46:0/646727477 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4ce80160d0 con 0x7f4cf80cbeb0 2024-04-25T08:48:16.234 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:48:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.240+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cd406b370 msgr2=0x7f4cd406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.240+0000 7f4cfd600640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cd406b370 0x7f4cd406d830 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0005890 tx=0x7f4ce0009040 comp rx=0 tx=0).stop 2024-04-25T08:48:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf80cbeb0 msgr2=0x7f4cf8143d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf80cbeb0 0x7f4cf8143d90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80ad040 tx=0x7f4ce801ac60 comp rx=0 tx=0).stop 2024-04-25T08:48:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 shutdown_connections 2024-04-25T08:48:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cd406b370 0x7f4cd406d830 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4cf8159230 0x7f4cf8158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cf80cdc50 0x7f4cf8158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 --2- 172.21.15.46:0/646727477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf80cbeb0 0x7f4cf8143d90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 >> 172.21.15.46:0/646727477 conn(0x7f4cf80b12a0 msgr2=0x7f4cf80b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 shutdown_connections 2024-04-25T08:48:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.244+0000 7f4cfd600640 1 -- 172.21.15.46:0/646727477 wait complete. 2024-04-25T08:48:16.246 INFO:tasks.ceph.mds.c.smithi046.stderr: -4057> 2024-04-25T08:47:02.893+0000 7fb218a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:48:16.257 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:48:16.294 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:48:16.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.404+0000 7fb2d7600640 1 Processor -- start 2024-04-25T08:48:16.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.404+0000 7fb2d7600640 1 -- start start 2024-04-25T08:48:16.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d7600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2d006f6d0 0x7fb2d006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d7600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb2d00af120 0x7fb2d006c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d7600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d015c0f0 con 0x7fb2d0072600 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d7600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d01563e0 con 0x7fb2d006f6d0 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d7600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d015c3d0 con 0x7fb2d00af120 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2d006f6d0 0x7fb2d006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2d006f6d0 0x7fb2d006bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49862/0 (socket says 172.21.15.46:49862) 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d4c00640 1 -- 172.21.15.46:0/2200388189 learned_addr learned my addr 172.21.15.46:0/2200388189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2cfe00640 1 --2- 172.21.15.46:0/2200388189 >> v2:172.21.15.82:3300/0 conn(0x7fb2d00af120 0x7fb2d006c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d4c00640 1 -- 172.21.15.46:0/2200388189 >> v2:172.21.15.82:3300/0 conn(0x7fb2d00af120 msgr2=0x7fb2d006c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d4c00640 1 --2- 172.21.15.46:0/2200388189 >> v2:172.21.15.82:3300/0 conn(0x7fb2d00af120 0x7fb2d006c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2cf400640 1 -- 172.21.15.46:0/2200388189 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637452727 0 0) 0x7fb2d015c0f0 con 0x7fb2d0072600 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d4c00640 1 -- 172.21.15.46:0/2200388189 >> v1:172.21.15.82:6789/0 conn(0x7fb2d0072600 legacy=0x7fb2d015f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.408+0000 7fb2d4c00640 1 -- 172.21.15.46:0/2200388189 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d015c6b0 con 0x7fb2d006f6d0 2024-04-25T08:48:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d4c00640 1 --2- 172.21.15.46:0/2200388189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f6d0 0x7fb2d006bc20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00d0ce0 tx=0x7fb2c401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3880ffa658547ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2cf400640 1 -- 172.21.15.46:0/2200388189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40b0070 con 0x7fb2d006f6d0 2024-04-25T08:48:16.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2cf400640 1 -- 172.21.15.46:0/2200388189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c40ab600 con 0x7fb2d006f6d0 2024-04-25T08:48:16.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2cf400640 1 -- 172.21.15.46:0/2200388189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c40ab9d0 con 0x7fb2d006f6d0 2024-04-25T08:48:16.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 -- 172.21.15.46:0/2200388189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f6d0 msgr2=0x7fb2d006bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 --2- 172.21.15.46:0/2200388189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f6d0 0x7fb2d006bc20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00d0ce0 tx=0x7fb2c401c520 comp rx=0 tx=0).stop 2024-04-25T08:48:16.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 -- 172.21.15.46:0/2200388189 shutdown_connections 2024-04-25T08:48:16.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 --2- 172.21.15.46:0/2200388189 >> v2:172.21.15.82:3300/0 conn(0x7fb2d00af120 0x7fb2d006c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 --2- 172.21.15.46:0/2200388189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d006f6d0 0x7fb2d006bc20 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 -- 172.21.15.46:0/2200388189 >> 172.21.15.46:0/2200388189 conn(0x7fb2d0071ae0 msgr2=0x7fb2d0071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:16.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 -- 172.21.15.46:0/2200388189 shutdown_connections 2024-04-25T08:48:16.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 -- 172.21.15.46:0/2200388189 wait complete. 2024-04-25T08:48:16.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 Processor -- start 2024-04-25T08:48:16.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.412+0000 7fb2d7600640 1 -- start start 2024-04-25T08:48:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d7600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f6d0 0x7fb2d0165e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d7600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d0072600 0x7fb2d00b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d7600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00af120 0x7fb2d009bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d7600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0168870 con 0x7fb2d00af120 2024-04-25T08:48:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d7600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d006ed60 con 0x7fb2d0072600 2024-04-25T08:48:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d7600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d01689f0 con 0x7fb2d006f6d0 2024-04-25T08:48:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00af120 0x7fb2d009bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00af120 0x7fb2d009bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49868/0 (socket says 172.21.15.46:49868) 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d5600640 1 -- 172.21.15.46:0/460105701 learned_addr learned my addr 172.21.15.46:0/460105701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2cfe00640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d0072600 0x7fb2d00b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d4c00640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f6d0 0x7fb2d0165e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d5600640 1 -- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f6d0 msgr2=0x7fb2d0165e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d5600640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f6d0 0x7fb2d0165e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d5600640 1 -- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d0072600 msgr2=0x7fb2d00b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d5600640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d0072600 0x7fb2d00b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d5600640 1 -- 172.21.15.46:0/460105701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00b3790 con 0x7fb2d00af120 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d5600640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00af120 0x7fb2d009bf60 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800c7b0 tx=0x7fb2c800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2ccc00640 1 -- 172.21.15.46:0/460105701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8003db0 con 0x7fb2d00af120 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2ccc00640 1 -- 172.21.15.46:0/460105701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c8005070 con 0x7fb2d00af120 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2ccc00640 1 -- 172.21.15.46:0/460105701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8005460 con 0x7fb2d00af120 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d00b3a70 con 0x7fb2d00af120 2024-04-25T08:48:16.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d009ba40 con 0x7fb2d00af120 2024-04-25T08:48:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d009bc40 con 0x7fb2d00af120 2024-04-25T08:48:16.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.416+0000 7fb2ccc00640 1 -- 172.21.15.46:0/460105701 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2c8005600 con 0x7fb2d00af120 2024-04-25T08:48:16.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.420+0000 7fb2ccc00640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2b0066f60 0x7fb2b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:16.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.420+0000 7fb2ccc00640 1 -- 172.21.15.46:0/460105701 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fb2c8085260 con 0x7fb2d00af120 2024-04-25T08:48:16.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.420+0000 7fb2d4c00640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2b0066f60 0x7fb2b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:16.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.420+0000 7fb2d4c00640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2b0066f60 0x7fb2b0069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40ad000 tx=0x7fb2c40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:16.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.424+0000 7fb2ccc00640 1 -- 172.21.15.46:0/460105701 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2d009bc40 con 0x7fb2d00af120 2024-04-25T08:48:16.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:16.616+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb2d0166370 con 0x7fb2d00af120 2024-04-25T08:48:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.244+0000 7fb2ccc00640 1 -- 172.21.15.46:0/460105701 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb2c8055f90 con 0x7fb2d00af120 2024-04-25T08:48:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2b0066f60 msgr2=0x7fb2b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2b0066f60 0x7fb2b0069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb2c40ad000 tx=0x7fb2c40a6000 comp rx=0 tx=0).stop 2024-04-25T08:48:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00af120 msgr2=0x7fb2d009bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00af120 0x7fb2d009bf60 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb2c800c7b0 tx=0x7fb2c800cc70 comp rx=0 tx=0).stop 2024-04-25T08:48:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 shutdown_connections 2024-04-25T08:48:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb2b0066f60 0x7fb2b0069420 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2d00af120 0x7fb2d009bf60 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2d0072600 0x7fb2d00b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 --2- 172.21.15.46:0/460105701 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2d006f6d0 0x7fb2d0165e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 >> 172.21.15.46:0/460105701 conn(0x7fb2d0071ae0 msgr2=0x7fb2d015ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 shutdown_connections 2024-04-25T08:48:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.248+0000 7fb2d7600640 1 -- 172.21.15.46:0/460105701 wait complete. 2024-04-25T08:48:17.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:48:17.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.460+0000 7f2658000640 1 Processor -- start 2024-04-25T08:48:17.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.460+0000 7f2658000640 1 -- start start 2024-04-25T08:48:17.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2658000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f265006ac10 0x7f26500c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:17.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2658000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26500d00c0 0x7f26500ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:17.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2658000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26500c7440 con 0x7f26500ce6a0 2024-04-25T08:48:17.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2658000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265006e6c0 con 0x7f26500d00c0 2024-04-25T08:48:17.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2658000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26500cf980 con 0x7f265006ac10 2024-04-25T08:48:17.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2654c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f26500d00c0 0x7f26500ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:17.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2656000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26500ce6a0 0x7f265006aff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36040/0 (socket says 172.21.15.46:36040) 2024-04-25T08:48:17.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2656000640 1 -- 172.21.15.46:0/1016687432 learned_addr learned my addr 172.21.15.46:0/1016687432 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:17.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2655600640 1 --2- 172.21.15.46:0/1016687432 >> v2:172.21.15.82:3300/0 conn(0x7f265006ac10 0x7f26500c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:17.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2654c00640 1 -- 172.21.15.46:0/1016687432 >> v2:172.21.15.82:3300/0 conn(0x7f265006ac10 msgr2=0x7f26500c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:17.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2654c00640 1 --2- 172.21.15.46:0/1016687432 >> v2:172.21.15.82:3300/0 conn(0x7f265006ac10 0x7f26500c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2654c00640 1 -- 172.21.15.46:0/1016687432 >> v1:172.21.15.46:6789/0 conn(0x7f26500ce6a0 legacy=0x7f265006aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:17.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.464+0000 7f2654c00640 1 -- 172.21.15.46:0/1016687432 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265015ddf0 con 0x7f26500d00c0 2024-04-25T08:48:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2654c00640 1 --2- 172.21.15.46:0/1016687432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500d00c0 0x7f26500ce160 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f265006e160 tx=0x7f263c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe45fe6f5194cea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f264fe00640 1 -- 172.21.15.46:0/1016687432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0b0070 con 0x7f26500d00c0 2024-04-25T08:48:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f264fe00640 1 -- 172.21.15.46:0/1016687432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c0ab600 con 0x7f26500d00c0 2024-04-25T08:48:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f264fe00640 1 -- 172.21.15.46:0/1016687432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0ab9d0 con 0x7f26500d00c0 2024-04-25T08:48:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 -- 172.21.15.46:0/1016687432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500d00c0 msgr2=0x7f26500ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 --2- 172.21.15.46:0/1016687432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500d00c0 0x7f26500ce160 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f265006e160 tx=0x7f263c01c3f0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 -- 172.21.15.46:0/1016687432 shutdown_connections 2024-04-25T08:48:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 --2- 172.21.15.46:0/1016687432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500d00c0 0x7f26500ce160 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 --2- 172.21.15.46:0/1016687432 >> v2:172.21.15.82:3300/0 conn(0x7f265006ac10 0x7f26500c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 -- 172.21.15.46:0/1016687432 >> 172.21.15.46:0/1016687432 conn(0x7f26500b21e0 msgr2=0x7f26500b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 -- 172.21.15.46:0/1016687432 shutdown_connections 2024-04-25T08:48:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 -- 172.21.15.46:0/1016687432 wait complete. 2024-04-25T08:48:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 Processor -- start 2024-04-25T08:48:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.468+0000 7f2658000640 1 -- start start 2024-04-25T08:48:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2658000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f265006ac10 0x7f26501568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2658000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500ce6a0 0x7f2650154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2658000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26500d00c0 0x7f265014f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2658000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26500ceab0 con 0x7f26500ce6a0 2024-04-25T08:48:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2658000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26500c6b30 con 0x7f26500d00c0 2024-04-25T08:48:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2658000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26500c62c0 con 0x7f265006ac10 2024-04-25T08:48:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2654c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500ce6a0 0x7f2650154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2654c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500ce6a0 0x7f2650154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49878/0 (socket says 172.21.15.46:49878) 2024-04-25T08:48:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2654c00640 1 -- 172.21.15.46:0/2398288889 learned_addr learned my addr 172.21.15.46:0/2398288889 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2655600640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f265006ac10 0x7f26501568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2656000640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26500d00c0 0x7f265014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2654c00640 1 -- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f265006ac10 msgr2=0x7f26501568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2654c00640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f265006ac10 0x7f26501568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2654c00640 1 -- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26500d00c0 msgr2=0x7f265014f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2654c00640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26500d00c0 0x7f265014f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2654c00640 1 -- 172.21.15.46:0/2398288889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2650151c80 con 0x7f26500ce6a0 2024-04-25T08:48:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2654c00640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500ce6a0 0x7f2650154310 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f263c0ae040 tx=0x7f263c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f264d600640 1 -- 172.21.15.46:0/2398288889 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c0b0040 con 0x7f26500ce6a0 2024-04-25T08:48:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f264d600640 1 -- 172.21.15.46:0/2398288889 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263c0b9070 con 0x7f26500ce6a0 2024-04-25T08:48:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f264d600640 1 -- 172.21.15.46:0/2398288889 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263c006e20 con 0x7f26500ce6a0 2024-04-25T08:48:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265015b2a0 con 0x7f26500ce6a0 2024-04-25T08:48:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265015b860 con 0x7f26500ce6a0 2024-04-25T08:48:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.472+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2650069080 con 0x7f26500ce6a0 2024-04-25T08:48:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.476+0000 7f264d600640 1 -- 172.21.15.46:0/2398288889 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f263c0b4690 con 0x7f26500ce6a0 2024-04-25T08:48:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.476+0000 7f264d600640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2630066f90 0x7f2630069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.476+0000 7f264d600640 1 -- 172.21.15.46:0/2398288889 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f263c11ecc0 con 0x7f26500ce6a0 2024-04-25T08:48:17.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.476+0000 7f2655600640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2630066f90 0x7f2630069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.476+0000 7f2655600640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2630066f90 0x7f2630069450 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2644009b60 tx=0x7f2644009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:17.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.480+0000 7f264d600640 1 -- 172.21.15.46:0/2398288889 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2650151c80 con 0x7f26500ce6a0 2024-04-25T08:48:17.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:17.644+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2650072f70 con 0x7f26500ce6a0 2024-04-25T08:48:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.248+0000 7f264d600640 1 -- 172.21.15.46:0/2398288889 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) v1 ==== 181+0+0 (secure 0 0 0) 0x7f263c016070 con 0x7f26500ce6a0 2024-04-25T08:48:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:48:18.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2630066f90 msgr2=0x7f2630069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2630066f90 0x7f2630069450 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2644009b60 tx=0x7f2644009190 comp rx=0 tx=0).stop 2024-04-25T08:48:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500ce6a0 msgr2=0x7f2650154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500ce6a0 0x7f2650154310 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f263c0ae040 tx=0x7f263c019190 comp rx=0 tx=0).stop 2024-04-25T08:48:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 shutdown_connections 2024-04-25T08:48:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2630066f90 0x7f2630069450 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26500d00c0 0x7f265014f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26500ce6a0 0x7f2650154310 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 --2- 172.21.15.46:0/2398288889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f265006ac10 0x7f26501568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 >> 172.21.15.46:0/2398288889 conn(0x7f26500b21e0 msgr2=0x7f26500cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:18.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 shutdown_connections 2024-04-25T08:48:18.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.252+0000 7f2658000640 1 -- 172.21.15.46:0/2398288889 wait complete. 2024-04-25T08:48:18.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:48:18.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2c0a00640 1 Processor -- start 2024-04-25T08:48:18.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2c0a00640 1 -- start start 2024-04-25T08:48:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2c0a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb2bc06bc20 0x7fb2bc06c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc0ae1a0 0x7fb2bc06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2c0a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc06c540 con 0x7fb2bc15dce0 2024-04-25T08:48:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2c0a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc06ee30 con 0x7fb2bc06bc20 2024-04-25T08:48:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc06f110 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2b1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb2bc15dce0 0x7fb2bc06db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42810/0 (socket says 172.21.15.46:42810) 2024-04-25T08:48:18.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2b1400640 1 -- 172.21.15.46:0/286786889 learned_addr learned my addr 172.21.15.46:0/286786889 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2b9e00640 1 --2- 172.21.15.46:0/286786889 >> v2:172.21.15.82:3300/0 conn(0x7fb2bc06bc20 0x7fb2bc06c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2ba800640 1 --2- 172.21.15.46:0/286786889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc0ae1a0 0x7fb2bc06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2b9400640 1 -- 172.21.15.46:0/286786889 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082299556 0 0) 0x7fb2bc06c540 con 0x7fb2bc15dce0 2024-04-25T08:48:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.428+0000 7fb2b9400640 1 -- 172.21.15.46:0/286786889 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a0003530 con 0x7fb2bc15dce0 2024-04-25T08:48:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2b9400640 1 -- 172.21.15.46:0/286786889 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1277676270 0 0) 0x7fb2a0003530 con 0x7fb2bc15dce0 2024-04-25T08:48:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2b9e00640 1 -- 172.21.15.46:0/286786889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc0ae1a0 msgr2=0x7fb2bc06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:18.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2b9e00640 1 --2- 172.21.15.46:0/286786889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc0ae1a0 0x7fb2bc06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:18.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2b9e00640 1 -- 172.21.15.46:0/286786889 >> v1:172.21.15.46:6789/0 conn(0x7fb2bc15dce0 legacy=0x7fb2bc06db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:18.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2b9e00640 1 -- 172.21.15.46:0/286786889 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc06f3f0 con 0x7fb2bc06bc20 2024-04-25T08:48:18.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2ba800640 1 --2- 172.21.15.46:0/286786889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc0ae1a0 0x7fb2bc06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:48:18.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2b9e00640 1 --2- 172.21.15.46:0/286786889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc06bc20 0x7fb2bc06c000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009d90 tx=0x7fb2ac01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d96064f9a98c8dbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:18.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2b9400640 1 -- 172.21.15.46:0/286786889 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0b2070 con 0x7fb2bc06bc20 2024-04-25T08:48:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2b9400640 1 -- 172.21.15.46:0/286786889 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac0a5620 con 0x7fb2bc06bc20 2024-04-25T08:48:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.432+0000 7fb2b9400640 1 -- 172.21.15.46:0/286786889 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4001680 con 0x7fb2bc06bc20 2024-04-25T08:48:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 -- 172.21.15.46:0/286786889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc06bc20 msgr2=0x7fb2bc06c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 --2- 172.21.15.46:0/286786889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc06bc20 0x7fb2bc06c000 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac009d90 tx=0x7fb2ac01c860 comp rx=0 tx=0).stop 2024-04-25T08:48:18.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 -- 172.21.15.46:0/286786889 shutdown_connections 2024-04-25T08:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 --2- 172.21.15.46:0/286786889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc0ae1a0 0x7fb2bc06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 --2- 172.21.15.46:0/286786889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc06bc20 0x7fb2bc06c000 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 -- 172.21.15.46:0/286786889 >> 172.21.15.46:0/286786889 conn(0x7fb2bc071ae0 msgr2=0x7fb2bc071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 -- 172.21.15.46:0/286786889 shutdown_connections 2024-04-25T08:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 -- 172.21.15.46:0/286786889 wait complete. 2024-04-25T08:48:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 Processor -- start 2024-04-25T08:48:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 -- start start 2024-04-25T08:48:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc06bc20 0x7fb2bc1341a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2bc0ae1a0 0x7fb2bc14d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc15dce0 0x7fb2bc14ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc06eba0 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b3d20 con 0x7fb2bc15dce0 2024-04-25T08:48:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2c0a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc16ae40 con 0x7fb2bc06bc20 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2b1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2bc0ae1a0 0x7fb2bc14d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2b1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2bc0ae1a0 0x7fb2bc14d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35228/0 (socket says 172.21.15.46:35228) 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2b1400640 1 -- 172.21.15.46:0/2427782451 learned_addr learned my addr 172.21.15.46:0/2427782451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2b9e00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc06bc20 0x7fb2bc1341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2ba800640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc15dce0 0x7fb2bc14ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2b1400640 1 -- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc06bc20 msgr2=0x7fb2bc1341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2b1400640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc06bc20 0x7fb2bc1341a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2b1400640 1 -- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc15dce0 msgr2=0x7fb2bc14ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2b1400640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc15dce0 0x7fb2bc14ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2b1400640 1 -- 172.21.15.46:0/2427782451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc149ff0 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.436+0000 7fb2ba800640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc15dce0 0x7fb2bc14ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2b1400640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2bc0ae1a0 0x7fb2bc14d890 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400c750 tx=0x7fb2a400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2b2a00640 1 -- 172.21.15.46:0/2427782451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4005ae0 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2b2a00640 1 -- 172.21.15.46:0/2427782451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a4005ca0 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2b2a00640 1 -- 172.21.15.46:0/2427782451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a40072b0 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc148c90 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc149250 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc067820 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2b2a00640 1 -- 172.21.15.46:0/2427782451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2a4007450 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2b2a00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb298066f60 0x7fb298069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2b2a00640 1 -- 172.21.15.46:0/2427782451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fb2a4084e60 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.440+0000 7fb2b9e00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb298066f60 0x7fb298069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.444+0000 7fb2b9e00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb298066f60 0x7fb298069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0af000 tx=0x7fb2ac0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:18.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.444+0000 7fb2b2a00640 1 -- 172.21.15.46:0/2427782451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2a4055d50 con 0x7fb2bc0ae1a0 2024-04-25T08:48:18.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:18.612+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2bc071f60 con 0x7fb2bc0ae1a0 2024-04-25T08:48:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.252+0000 7fb2b2a00640 1 -- 172.21.15.46:0/2427782451 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb2a4055650 con 0x7fb2bc0ae1a0 2024-04-25T08:48:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-25T08:48:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb298066f60 msgr2=0x7fb298069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb298066f60 0x7fb298069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0af000 tx=0x7fb2ac0a7070 comp rx=0 tx=0).stop 2024-04-25T08:48:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2bc0ae1a0 msgr2=0x7fb2bc14d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2bc0ae1a0 0x7fb2bc14d890 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400c750 tx=0x7fb2a400cc10 comp rx=0 tx=0).stop 2024-04-25T08:48:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 shutdown_connections 2024-04-25T08:48:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb298066f60 0x7fb298069420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2bc15dce0 0x7fb2bc14ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2bc0ae1a0 0x7fb2bc14d890 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 --2- 172.21.15.46:0/2427782451 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb2bc06bc20 0x7fb2bc1341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 >> 172.21.15.46:0/2427782451 conn(0x7fb2bc071ae0 msgr2=0x7fb2bc15e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 shutdown_connections 2024-04-25T08:48:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.256+0000 7fb2c0a00640 1 -- 172.21.15.46:0/2427782451 wait complete. 2024-04-25T08:48:19.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:48:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6774000640 1 Processor -- start 2024-04-25T08:48:19.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6774000640 1 -- start start 2024-04-25T08:48:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6774000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 0x7f676c0af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6774000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f676c06b470 0x7f676c0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6774000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c0b0040 con 0x7f676c0afc20 2024-04-25T08:48:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6774000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f676c06fca0 con 0x7f676c06b470 2024-04-25T08:48:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6774000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c06ff80 con 0x7f676c15d900 2024-04-25T08:48:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6772000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f676c06b470 0x7f676c0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6771600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 0x7f676c0af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6772000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f676c06b470 0x7f676c0d0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:53332/0 (socket says 172.21.15.46:53332) 2024-04-25T08:48:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6772000640 1 -- 172.21.15.46:0/1885042249 learned_addr learned my addr 172.21.15.46:0/1885042249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6772000640 1 -- 172.21.15.46:0/1885042249 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 msgr2=0x7f676c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.428+0000 7f6772000640 1 --2- 172.21.15.46:0/1885042249 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 0x7f676c0af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6772000640 1 -- 172.21.15.46:0/1885042249 >> v1:172.21.15.82:6789/0 conn(0x7f676c0afc20 legacy=0x7f676c06ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6772000640 1 -- 172.21.15.46:0/1885042249 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c070260 con 0x7f676c06b470 2024-04-25T08:48:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6771600640 1 --2- 172.21.15.46:0/1885042249 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 0x7f676c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:48:19.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6772000640 1 --2- 172.21.15.46:0/1885042249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c06b470 0x7f676c0d0210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6764009e50 tx=0x7f676401c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f76fc0f1360d2a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:19.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f676be00640 1 -- 172.21.15.46:0/1885042249 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67640a5750 con 0x7f676c06b470 2024-04-25T08:48:19.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f676be00640 1 -- 172.21.15.46:0/1885042249 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67640a58f0 con 0x7f676c06b470 2024-04-25T08:48:19.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f676be00640 1 -- 172.21.15.46:0/1885042249 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67640a5cc0 con 0x7f676c06b470 2024-04-25T08:48:19.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6774000640 1 -- 172.21.15.46:0/1885042249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c06b470 msgr2=0x7f676c0d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6774000640 1 --2- 172.21.15.46:0/1885042249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c06b470 0x7f676c0d0210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6764009e50 tx=0x7f676401c9a0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6774000640 1 -- 172.21.15.46:0/1885042249 shutdown_connections 2024-04-25T08:48:19.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6774000640 1 --2- 172.21.15.46:0/1885042249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c06b470 0x7f676c0d0210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6774000640 1 --2- 172.21.15.46:0/1885042249 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 0x7f676c0af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6774000640 1 -- 172.21.15.46:0/1885042249 >> 172.21.15.46:0/1885042249 conn(0x7f676c06e3e0 msgr2=0x7f676c06e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:19.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6774000640 1 -- 172.21.15.46:0/1885042249 shutdown_connections 2024-04-25T08:48:19.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.432+0000 7f6774000640 1 -- 172.21.15.46:0/1885042249 wait complete. 2024-04-25T08:48:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 Processor -- start 2024-04-25T08:48:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 -- start start 2024-04-25T08:48:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f676c06b470 0x7f676c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c0afc20 0x7f676c1f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 0x7f676c1fa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0d10e0 con 0x7f676c06b470 2024-04-25T08:48:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0d0f60 con 0x7f676c0afc20 2024-04-25T08:48:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0d1260 con 0x7f676c15d900 2024-04-25T08:48:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6771600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f676c06b470 0x7f676c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6771600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f676c06b470 0x7f676c0cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35232/0 (socket says 172.21.15.46:35232) 2024-04-25T08:48:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6771600640 1 -- 172.21.15.46:0/1326410030 learned_addr learned my addr 172.21.15.46:0/1326410030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6772000640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 0x7f676c1fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6770c00640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c0afc20 0x7f676c1f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6771600640 1 -- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 msgr2=0x7f676c1fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6771600640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 0x7f676c1fa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6771600640 1 -- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c0afc20 msgr2=0x7f676c1f9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6771600640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c0afc20 0x7f676c1f9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6771600640 1 -- 172.21.15.46:0/1326410030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0cc000 con 0x7f676c06b470 2024-04-25T08:48:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6771600640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f676c06b470 0x7f676c0cb5b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f676000e980 tx=0x7f676000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6769600640 1 -- 172.21.15.46:0/1326410030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760017070 con 0x7f676c06b470 2024-04-25T08:48:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6769600640 1 -- 172.21.15.46:0/1326410030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676000ca00 con 0x7f676c06b470 2024-04-25T08:48:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6769600640 1 -- 172.21.15.46:0/1326410030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000cdd0 con 0x7f676c06b470 2024-04-25T08:48:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c0cc2e0 con 0x7f676c06b470 2024-04-25T08:48:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.436+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c139370 con 0x7f676c06b470 2024-04-25T08:48:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.440+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c069010 con 0x7f676c06b470 2024-04-25T08:48:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.440+0000 7f6769600640 1 -- 172.21.15.46:0/1326410030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f676001a070 con 0x7f676c06b470 2024-04-25T08:48:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.440+0000 7f6769600640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f674c066f60 0x7f674c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.440+0000 7f6769600640 1 -- 172.21.15.46:0/1326410030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6760085130 con 0x7f676c06b470 2024-04-25T08:48:19.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.440+0000 7f6770c00640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f674c066f60 0x7f674c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.440+0000 7f6770c00640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f674c066f60 0x7f674c069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f676c150a70 tx=0x7f6758009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:19.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.444+0000 7f6769600640 1 -- 172.21.15.46:0/1326410030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6760055c10 con 0x7f676c06b470 2024-04-25T08:48:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.604+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f676c072890 con 0x7f676c06b470 2024-04-25T08:48:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.608+0000 7f6769600640 1 -- 172.21.15.46:0/1326410030 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6760059cb0 con 0x7f676c06b470 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f674c066f60 msgr2=0x7f674c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f674c066f60 0x7f674c069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f676c150a70 tx=0x7f6758009040 comp rx=0 tx=0).stop 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f676c06b470 msgr2=0x7f676c0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f676c06b470 0x7f676c0cb5b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f676000e980 tx=0x7f676000ee40 comp rx=0 tx=0).stop 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 shutdown_connections 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f674c066f60 0x7f674c069420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f676c15d900 0x7f676c1fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f676c0afc20 0x7f676c1f9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 --2- 172.21.15.46:0/1326410030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f676c06b470 0x7f676c0cb5b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 >> 172.21.15.46:0/1326410030 conn(0x7f676c06e3e0 msgr2=0x7f676c0cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 shutdown_connections 2024-04-25T08:48:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.612+0000 7f6774000640 1 -- 172.21.15.46:0/1326410030 wait complete. 2024-04-25T08:48:19.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T08:48:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc287e00640 1 Processor -- start 2024-04-25T08:48:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc287e00640 1 -- start start 2024-04-25T08:48:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc287e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2880b0be0 0x7fc2880b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc287e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc28806ac10 0x7fc28806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc287e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2880b4f80 con 0x7fc288161420 2024-04-25T08:48:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc287e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2880b5590 con 0x7fc2880b0be0 2024-04-25T08:48:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc287e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28806b5b0 con 0x7fc28806ac10 2024-04-25T08:48:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc286a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2880b0be0 0x7fc2880b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc287400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc288161420 0x7fc2881618b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42814/0 (socket says 172.21.15.46:42814) 2024-04-25T08:48:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc287400640 1 -- 172.21.15.46:0/692190061 learned_addr learned my addr 172.21.15.46:0/692190061 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc27e000640 1 --2- 172.21.15.46:0/692190061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc28806ac10 0x7fc28806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc286a00640 1 -- 172.21.15.46:0/692190061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc28806ac10 msgr2=0x7fc28806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc286a00640 1 --2- 172.21.15.46:0/692190061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc28806ac10 0x7fc28806b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc286a00640 1 -- 172.21.15.46:0/692190061 >> v1:172.21.15.46:6789/0 conn(0x7fc288161420 legacy=0x7fc2881618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc286a00640 1 -- 172.21.15.46:0/692190061 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880b1b40 con 0x7fc2880b0be0 2024-04-25T08:48:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.772+0000 7fc286000640 1 -- 172.21.15.46:0/692190061 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773353525 0 0) 0x7fc2880b4f80 con 0x7fc288161420 2024-04-25T08:48:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc286a00640 1 --2- 172.21.15.46:0/692190061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2880b0be0 0x7fc2880b3380 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc2780097c0 tx=0x7fc27801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe30240a1c7b537 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc286000640 1 -- 172.21.15.46:0/692190061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780ab600 con 0x7fc2880b0be0 2024-04-25T08:48:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc286000640 1 -- 172.21.15.46:0/692190061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780b0070 con 0x7fc2880b0be0 2024-04-25T08:48:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc286000640 1 -- 172.21.15.46:0/692190061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780ab9c0 con 0x7fc2880b0be0 2024-04-25T08:48:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc287e00640 1 -- 172.21.15.46:0/692190061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2880b0be0 msgr2=0x7fc2880b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc287e00640 1 --2- 172.21.15.46:0/692190061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2880b0be0 0x7fc2880b3380 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc2780097c0 tx=0x7fc27801c3e0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc287e00640 1 -- 172.21.15.46:0/692190061 shutdown_connections 2024-04-25T08:48:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc287e00640 1 --2- 172.21.15.46:0/692190061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc28806ac10 0x7fc28806b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc287e00640 1 --2- 172.21.15.46:0/692190061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2880b0be0 0x7fc2880b3380 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc287e00640 1 -- 172.21.15.46:0/692190061 >> 172.21.15.46:0/692190061 conn(0x7fc288099a00 msgr2=0x7fc2880b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc287e00640 1 -- 172.21.15.46:0/692190061 shutdown_connections 2024-04-25T08:48:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.776+0000 7fc287e00640 1 -- 172.21.15.46:0/692190061 wait complete. 2024-04-25T08:48:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 Processor -- start 2024-04-25T08:48:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 -- start start 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806ac10 0x7fc28813add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2880b0be0 0x7fc28813b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc288161420 0x7fc288149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880b0fc0 con 0x7fc28806ac10 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880b38c0 con 0x7fc288161420 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880b1140 con 0x7fc2880b0be0 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc286a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806ac10 0x7fc28813add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc286a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806ac10 0x7fc28813add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35236/0 (socket says 172.21.15.46:35236) 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc286a00640 1 -- 172.21.15.46:0/1038587944 learned_addr learned my addr 172.21.15.46:0/1038587944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc27e000640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2880b0be0 0x7fc28813b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287400640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc288161420 0x7fc288149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc286a00640 1 -- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2880b0be0 msgr2=0x7fc28813b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc286a00640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2880b0be0 0x7fc28813b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc286a00640 1 -- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc288161420 msgr2=0x7fc288149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc286a00640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc288161420 0x7fc288149a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc286a00640 1 -- 172.21.15.46:0/1038587944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880b1420 con 0x7fc28806ac10 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc286a00640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806ac10 0x7fc28813add0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc27801a010 tx=0x7fc27801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc27f400640 1 -- 172.21.15.46:0/1038587944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780bf610 con 0x7fc28806ac10 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc27f400640 1 -- 172.21.15.46:0/1038587944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780bf7d0 con 0x7fc28806ac10 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc27f400640 1 -- 172.21.15.46:0/1038587944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780bfbc0 con 0x7fc28806ac10 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc288143260 con 0x7fc28806ac10 2024-04-25T08:48:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc288143820 con 0x7fc28806ac10 2024-04-25T08:48:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.780+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288069010 con 0x7fc28806ac10 2024-04-25T08:48:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.784+0000 7fc27f400640 1 -- 172.21.15.46:0/1038587944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2780bd020 con 0x7fc28806ac10 2024-04-25T08:48:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.784+0000 7fc27f400640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc264066f60 0x7fc264069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.784+0000 7fc27f400640 1 -- 172.21.15.46:0/1038587944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc27811f120 con 0x7fc28806ac10 2024-04-25T08:48:19.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.784+0000 7fc27e000640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc264066f60 0x7fc264069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:19.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.784+0000 7fc27e000640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc264066f60 0x7fc264069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc270003fd0 tx=0x7fc2700039a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:19.788+0000 7fc27f400640 1 -- 172.21.15.46:0/1038587944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2780c6070 con 0x7fc28806ac10 2024-04-25T08:48:20.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.004+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7fc28813ea20 con 0x7fc28806ac10 2024-04-25T08:48:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.260+0000 7fc27f400640 1 -- 172.21.15.46:0/1038587944 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v304) v1 ==== 166+0+0 (secure 0 0 0) 0x7fc278016090 con 0x7fc28806ac10 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc264066f60 msgr2=0x7fc264069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc264066f60 0x7fc264069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc270003fd0 tx=0x7fc2700039a0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806ac10 msgr2=0x7fc28813add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806ac10 0x7fc28813add0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc27801a010 tx=0x7fc27801ac20 comp rx=0 tx=0).stop 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 shutdown_connections 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc264066f60 0x7fc264069420 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc288161420 0x7fc288149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2880b0be0 0x7fc28813b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 --2- 172.21.15.46:0/1038587944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc28806ac10 0x7fc28813add0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 >> 172.21.15.46:0/1038587944 conn(0x7fc288099a00 msgr2=0x7fc2880d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 shutdown_connections 2024-04-25T08:48:20.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.264+0000 7fc287e00640 1 -- 172.21.15.46:0/1038587944 wait complete. 2024-04-25T08:48:20.277 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T08:48:20.277 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T08:48:20.277 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:48:20.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.424+0000 7f648ce00640 1 Processor -- start 2024-04-25T08:48:20.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.424+0000 7f648ce00640 1 -- start start 2024-04-25T08:48:20.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.424+0000 7f648ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6488159230 0x7f648815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.424+0000 7f648ce00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64880cdc50 0x7f648815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.424+0000 7f648ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64880cc700 con 0x7f64880cbeb0 2024-04-25T08:48:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.424+0000 7f648ce00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64880cc9e0 con 0x7f6488159230 2024-04-25T08:48:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.424+0000 7f648ce00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64880cccc0 con 0x7f64880cdc50 2024-04-25T08:48:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.424+0000 7f6485e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6488159230 0x7f648815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.424+0000 7f647d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f64880cbeb0 0x7f64880cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42820/0 (socket says 172.21.15.46:42820) 2024-04-25T08:48:20.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f647d400640 1 -- 172.21.15.46:0/2266779199 learned_addr learned my addr 172.21.15.46:0/2266779199 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:20.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6486800640 1 --2- 172.21.15.46:0/2266779199 >> v2:172.21.15.82:3300/0 conn(0x7f64880cdc50 0x7f648815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485400640 1 -- 172.21.15.46:0/2266779199 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331622586 0 0) 0x7f64880cc700 con 0x7f64880cbeb0 2024-04-25T08:48:20.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485400640 1 -- 172.21.15.46:0/2266779199 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f646c003530 con 0x7f64880cbeb0 2024-04-25T08:48:20.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485e00640 1 -- 172.21.15.46:0/2266779199 >> v2:172.21.15.82:3300/0 conn(0x7f64880cdc50 msgr2=0x7f648815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:20.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485e00640 1 --2- 172.21.15.46:0/2266779199 >> v2:172.21.15.82:3300/0 conn(0x7f64880cdc50 0x7f648815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485e00640 1 -- 172.21.15.46:0/2266779199 >> v1:172.21.15.46:6789/0 conn(0x7f64880cbeb0 legacy=0x7f64880cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485e00640 1 -- 172.21.15.46:0/2266779199 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880ccfa0 con 0x7f6488159230 2024-04-25T08:48:20.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485e00640 1 --2- 172.21.15.46:0/2266779199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 0x7f648815bdb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f64880b2340 tx=0x7f647801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f173e7ae2001d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:20.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485400640 1 -- 172.21.15.46:0/2266779199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64780b1070 con 0x7f6488159230 2024-04-25T08:48:20.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485400640 1 -- 172.21.15.46:0/2266779199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64780a8050 con 0x7f6488159230 2024-04-25T08:48:20.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.428+0000 7f6485400640 1 -- 172.21.15.46:0/2266779199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64780ac860 con 0x7f6488159230 2024-04-25T08:48:20.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 -- 172.21.15.46:0/2266779199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 msgr2=0x7f648815bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 --2- 172.21.15.46:0/2266779199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 0x7f648815bdb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f64880b2340 tx=0x7f647801c3c0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 -- 172.21.15.46:0/2266779199 shutdown_connections 2024-04-25T08:48:20.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 --2- 172.21.15.46:0/2266779199 >> v2:172.21.15.82:3300/0 conn(0x7f64880cdc50 0x7f648815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 --2- 172.21.15.46:0/2266779199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 0x7f648815bdb0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 -- 172.21.15.46:0/2266779199 >> 172.21.15.46:0/2266779199 conn(0x7f64880b12a0 msgr2=0x7f64880b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:20.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 -- 172.21.15.46:0/2266779199 shutdown_connections 2024-04-25T08:48:20.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 -- 172.21.15.46:0/2266779199 wait complete. 2024-04-25T08:48:20.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 Processor -- start 2024-04-25T08:48:20.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 -- start start 2024-04-25T08:48:20.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64880cbeb0 0x7f648813de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdc50 0x7f64880ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 0x7f64880ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648815df50 con 0x7f6488159230 2024-04-25T08:48:20.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64880cc4e0 con 0x7f64880cdc50 2024-04-25T08:48:20.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f648ce00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648815e0d0 con 0x7f64880cbeb0 2024-04-25T08:48:20.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6486800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 0x7f64880ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6486800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 0x7f64880ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35254/0 (socket says 172.21.15.46:35254) 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6486800640 1 -- 172.21.15.46:0/3589120589 learned_addr learned my addr 172.21.15.46:0/3589120589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6485e00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64880cbeb0 0x7f648813de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f647d400640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdc50 0x7f64880ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6486800640 1 -- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64880cbeb0 msgr2=0x7f648813de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6486800640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64880cbeb0 0x7f648813de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6486800640 1 -- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdc50 msgr2=0x7f64880ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6486800640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdc50 0x7f64880ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6486800640 1 -- 172.21.15.46:0/3589120589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880b27d0 con 0x7f6488159230 2024-04-25T08:48:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f647d400640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdc50 0x7f64880ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.432+0000 7f6486800640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 0x7f64880ad850 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f648000d600 tx=0x7f648000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f647ea00640 1 -- 172.21.15.46:0/3589120589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800125f0 con 0x7f6488159230 2024-04-25T08:48:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f647ea00640 1 -- 172.21.15.46:0/3589120589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6480019070 con 0x7f6488159230 2024-04-25T08:48:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f647ea00640 1 -- 172.21.15.46:0/3589120589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800129f0 con 0x7f6488159230 2024-04-25T08:48:20.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64880b2ab0 con 0x7f6488159230 2024-04-25T08:48:20.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64880b3070 con 0x7f6488159230 2024-04-25T08:48:20.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6488069080 con 0x7f6488159230 2024-04-25T08:48:20.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f647ea00640 1 -- 172.21.15.46:0/3589120589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6480012b90 con 0x7f6488159230 2024-04-25T08:48:20.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f647ea00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6464066f60 0x7f6464069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f647ea00640 1 -- 172.21.15.46:0/3589120589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f64800852c0 con 0x7f6488159230 2024-04-25T08:48:20.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.436+0000 7f6485e00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6464066f60 0x7f6464069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.440+0000 7f6485e00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6464066f60 0x7f6464069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f64780ae000 tx=0x7f64780a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:20.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.440+0000 7f647ea00640 1 -- 172.21.15.46:0/3589120589 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64800563e0 con 0x7f6488159230 2024-04-25T08:48:20.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.604+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64880b3240 con 0x7f6488159230 2024-04-25T08:48:20.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.604+0000 7f647ea00640 1 -- 172.21.15.46:0/3589120589 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+13374 (secure 0 0 0) 0x7f6480055ce0 con 0x7f6488159230 2024-04-25T08:48:20.601 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:20.601 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:48:19.253524+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/4112568749":"2024-04-26T08:48:16.242929+0000","172.21.15.46:6833/4112568749":"2024-04-26T08:48:16.242929+0000","172.21.15.46:6835/960745191":"2024-04-26T08:46:56.883011+0000","172.21.15.46:6833/960745191":"2024-04-26T08:46:56.883011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6464066f60 msgr2=0x7f6464069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6464066f60 0x7f6464069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f64780ae000 tx=0x7f64780a7000 comp rx=0 tx=0).stop 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 msgr2=0x7f64880ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 0x7f64880ad850 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f648000d600 tx=0x7f648000dac0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 shutdown_connections 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6464066f60 0x7f6464069420 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6488159230 0x7f64880ad850 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64880cdc50 0x7f64880ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 --2- 172.21.15.46:0/3589120589 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64880cbeb0 0x7f648813de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 >> 172.21.15.46:0/3589120589 conn(0x7f64880b12a0 msgr2=0x7f648806d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 shutdown_connections 2024-04-25T08:48:20.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.608+0000 7f648ce00640 1 -- 172.21.15.46:0/3589120589 wait complete. 2024-04-25T08:48:20.624 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:48:20.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:48:20.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:48:20.765 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:48:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f9165000640 1 Processor -- start 2024-04-25T08:48:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f9165000640 1 -- start start 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f9165000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9160162380 0x7f91600cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f9165000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916006b610 con 0x7f91600ce220 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f9165000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916006f980 con 0x7f916006b230 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f9165000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91600cd400 con 0x7f9160162380 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f915e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9160162380 0x7f91600cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f915e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9160162380 0x7f91600cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35256/0 (socket says 172.21.15.46:35256) 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f915e800640 1 -- 172.21.15.46:0/2464055441 learned_addr learned my addr 172.21.15.46:0/2464055441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f9157e00640 1 --1- 172.21.15.46:0/2464055441 >> v1:172.21.15.46:6789/0 conn(0x7f91600ce220 0x7f9160161c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42828/0 (socket says 172.21.15.46:42828) 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f915e800640 1 -- 172.21.15.46:0/2464055441 >> v1:172.21.15.82:6789/0 conn(0x7f916006b230 legacy=0x7f91600cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f915e800640 1 -- 172.21.15.46:0/2464055441 >> v1:172.21.15.46:6789/0 conn(0x7f91600ce220 legacy=0x7f9160161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.776+0000 7f915e800640 1 -- 172.21.15.46:0/2464055441 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916006fc60 con 0x7f9160162380 2024-04-25T08:48:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f915e800640 1 --2- 172.21.15.46:0/2464055441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9160162380 0x7f91600cf890 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f9158009e50 tx=0x7f915801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea8d76283087abb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f915d400640 1 -- 172.21.15.46:0/2464055441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580a57a0 con 0x7f9160162380 2024-04-25T08:48:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f915d400640 1 -- 172.21.15.46:0/2464055441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91580a5940 con 0x7f9160162380 2024-04-25T08:48:20.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f915d400640 1 -- 172.21.15.46:0/2464055441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580a5d10 con 0x7f9160162380 2024-04-25T08:48:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f9165000640 1 -- 172.21.15.46:0/2464055441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9160162380 msgr2=0x7f91600cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f9165000640 1 --2- 172.21.15.46:0/2464055441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9160162380 0x7f91600cf890 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f9158009e50 tx=0x7f915801c8e0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f9165000640 1 -- 172.21.15.46:0/2464055441 shutdown_connections 2024-04-25T08:48:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f9165000640 1 --2- 172.21.15.46:0/2464055441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9160162380 0x7f91600cf890 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f9165000640 1 -- 172.21.15.46:0/2464055441 >> 172.21.15.46:0/2464055441 conn(0x7f9160099a00 msgr2=0x7f91600b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f9165000640 1 -- 172.21.15.46:0/2464055441 shutdown_connections 2024-04-25T08:48:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f9165000640 1 -- 172.21.15.46:0/2464055441 wait complete. 2024-04-25T08:48:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f9165000640 1 Processor -- start 2024-04-25T08:48:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.780+0000 7f9165000640 1 -- start start 2024-04-25T08:48:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9165000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916006b230 0x7f916015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9165000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9160162380 0x7f91600cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9165000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91600cb0c0 0x7f91600c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9165000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91600cff50 con 0x7f91600cb0c0 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9165000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91600cfdd0 con 0x7f9160162380 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9165000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91600d00d0 con 0x7f916006b230 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91600cb0c0 0x7f91600c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91600cb0c0 0x7f91600c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35266/0 (socket says 172.21.15.46:35266) 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915e800640 1 -- 172.21.15.46:0/2392280532 learned_addr learned my addr 172.21.15.46:0/2392280532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9157e00640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9160162380 0x7f91600cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915de00640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916006b230 0x7f916015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915e800640 1 -- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916006b230 msgr2=0x7f916015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915e800640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916006b230 0x7f916015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915e800640 1 -- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9160162380 msgr2=0x7f91600cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915e800640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9160162380 0x7f91600cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915e800640 1 -- 172.21.15.46:0/2392280532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91600c5510 con 0x7f91600cb0c0 2024-04-25T08:48:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f915e800640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91600cb0c0 0x7f91600c81d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f91580af040 tx=0x7f9158019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9156000640 1 -- 172.21.15.46:0/2392280532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580bc020 con 0x7f91600cb0c0 2024-04-25T08:48:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9156000640 1 -- 172.21.15.46:0/2392280532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91580b8690 con 0x7f91600cb0c0 2024-04-25T08:48:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9156000640 1 -- 172.21.15.46:0/2392280532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580b8a80 con 0x7f91600cb0c0 2024-04-25T08:48:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91600c57f0 con 0x7f91600cb0c0 2024-04-25T08:48:20.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91600c5db0 con 0x7f91600cb0c0 2024-04-25T08:48:20.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.784+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91600cb540 con 0x7f91600cb0c0 2024-04-25T08:48:20.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.788+0000 7f9156000640 1 -- 172.21.15.46:0/2392280532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91580bf020 con 0x7f91600cb0c0 2024-04-25T08:48:20.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.788+0000 7f9156000640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f913c066f90 0x7f913c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:20.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.788+0000 7f9156000640 1 -- 172.21.15.46:0/2392280532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f915811ff10 con 0x7f91600cb0c0 2024-04-25T08:48:20.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.788+0000 7f915de00640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f913c066f90 0x7f913c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:20.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.788+0000 7f915de00640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f913c066f90 0x7f913c069450 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9150001810 tx=0x7f9150009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:20.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.792+0000 7f9156000640 1 -- 172.21.15.46:0/2392280532 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91580c6030 con 0x7f91600cb0c0 2024-04-25T08:48:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.952+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9160069010 con 0x7f91600cb0c0 2024-04-25T08:48:20.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.952+0000 7f9156000640 1 -- 172.21.15.46:0/2392280532 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9158016620 con 0x7f91600cb0c0 2024-04-25T08:48:20.947 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:48:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f913c066f90 msgr2=0x7f913c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f913c066f90 0x7f913c069450 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9150001810 tx=0x7f9150009040 comp rx=0 tx=0).stop 2024-04-25T08:48:20.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91600cb0c0 msgr2=0x7f91600c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91600cb0c0 0x7f91600c81d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f91580af040 tx=0x7f9158019680 comp rx=0 tx=0).stop 2024-04-25T08:48:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 shutdown_connections 2024-04-25T08:48:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f913c066f90 0x7f913c069450 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91600cb0c0 0x7f91600c81d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9160162380 0x7f91600cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 --2- 172.21.15.46:0/2392280532 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916006b230 0x7f916015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 >> 172.21.15.46:0/2392280532 conn(0x7f9160099a00 msgr2=0x7f91600af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 shutdown_connections 2024-04-25T08:48:20.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:20.956+0000 7f9165000640 1 -- 172.21.15.46:0/2392280532 wait complete. 2024-04-25T08:48:20.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T08:48:21.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.116+0000 7fd6fdc00640 1 Processor -- start 2024-04-25T08:48:21.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.116+0000 7fd6fdc00640 1 -- start start 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.116+0000 7fd6fdc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd6f813e540 0x7fd6f80cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6fdc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6f8071ea0 0x7fd6f80cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6fdc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f80710e0 con 0x7fd6f8140e50 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6fdc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f80713c0 con 0x7fd6f8071ea0 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6fdc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f80716a0 con 0x7fd6f813e540 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6ef400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6f8071ea0 0x7fd6f80cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6ef400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6f8071ea0 0x7fd6f80cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35270/0 (socket says 172.21.15.46:35270) 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6ef400640 1 -- 172.21.15.46:0/1047642165 learned_addr learned my addr 172.21.15.46:0/1047642165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6fd200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd6f8140e50 0x7fd6f8141300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42834/0 (socket says 172.21.15.46:42834) 2024-04-25T08:48:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6f7e00640 1 --2- 172.21.15.46:0/1047642165 >> v2:172.21.15.82:3300/0 conn(0x7fd6f813e540 0x7fd6f80cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:21.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6ef400640 1 -- 172.21.15.46:0/1047642165 >> v2:172.21.15.82:3300/0 conn(0x7fd6f813e540 msgr2=0x7fd6f80cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:21.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6ef400640 1 --2- 172.21.15.46:0/1047642165 >> v2:172.21.15.82:3300/0 conn(0x7fd6f813e540 0x7fd6f80cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:21.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6ef400640 1 -- 172.21.15.46:0/1047642165 >> v1:172.21.15.46:6789/0 conn(0x7fd6f8140e50 legacy=0x7fd6f8141300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:21.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6ef400640 1 -- 172.21.15.46:0/1047642165 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f8071980 con 0x7fd6f8071ea0 2024-04-25T08:48:21.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6f7400640 1 -- 172.21.15.46:0/1047642165 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234783145 0 0) 0x7fd6f80710e0 con 0x7fd6f8140e50 2024-04-25T08:48:21.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6ef400640 1 --2- 172.21.15.46:0/1047642165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8071ea0 0x7fd6f80cd750 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd6f806bbe0 tx=0x7fd6e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20cbbcffbce8b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:21.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6f7400640 1 -- 172.21.15.46:0/1047642165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00b1070 con 0x7fd6f8071ea0 2024-04-25T08:48:21.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6f7400640 1 -- 172.21.15.46:0/1047642165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e00ac600 con 0x7fd6f8071ea0 2024-04-25T08:48:21.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.120+0000 7fd6f7400640 1 -- 172.21.15.46:0/1047642165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00ac9d0 con 0x7fd6f8071ea0 2024-04-25T08:48:21.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 -- 172.21.15.46:0/1047642165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8071ea0 msgr2=0x7fd6f80cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:21.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 --2- 172.21.15.46:0/1047642165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8071ea0 0x7fd6f80cd750 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd6f806bbe0 tx=0x7fd6e001c490 comp rx=0 tx=0).stop 2024-04-25T08:48:21.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 -- 172.21.15.46:0/1047642165 shutdown_connections 2024-04-25T08:48:21.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 --2- 172.21.15.46:0/1047642165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8071ea0 0x7fd6f80cd750 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 --2- 172.21.15.46:0/1047642165 >> v2:172.21.15.82:3300/0 conn(0x7fd6f813e540 0x7fd6f80cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 -- 172.21.15.46:0/1047642165 >> 172.21.15.46:0/1047642165 conn(0x7fd6f8066fa0 msgr2=0x7fd6f806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 -- 172.21.15.46:0/1047642165 shutdown_connections 2024-04-25T08:48:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 -- 172.21.15.46:0/1047642165 wait complete. 2024-04-25T08:48:21.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 Processor -- start 2024-04-25T08:48:21.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 -- start start 2024-04-25T08:48:21.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd6f8071ea0 0x7fd6f813d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:21.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f813e540 0x7fd6f8136af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:21.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8140e50 0x7fd6f81582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:21.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f813f870 con 0x7fd6f8140e50 2024-04-25T08:48:21.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f813f6f0 con 0x7fd6f813e540 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fdc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f813f9f0 con 0x7fd6f8071ea0 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8140e50 0x7fd6f81582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8140e50 0x7fd6f81582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35284/0 (socket says 172.21.15.46:35284) 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fd200640 1 -- 172.21.15.46:0/533969282 learned_addr learned my addr 172.21.15.46:0/533969282 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6f7e00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd6f8071ea0 0x7fd6f813d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6ef400640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f813e540 0x7fd6f8136af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fd200640 1 -- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd6f8071ea0 msgr2=0x7fd6f813d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.124+0000 7fd6fd200640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd6f8071ea0 0x7fd6f813d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6fd200640 1 -- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f813e540 msgr2=0x7fd6f8136af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6fd200640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f813e540 0x7fd6f8136af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6fd200640 1 -- 172.21.15.46:0/533969282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f813c0a0 con 0x7fd6f8140e50 2024-04-25T08:48:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6f7e00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd6f8071ea0 0x7fd6f813d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6fd200640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8140e50 0x7fd6f81582a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c3e0 tx=0x7fd6f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6f4c00640 1 -- 172.21.15.46:0/533969282 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00074e0 con 0x7fd6f8140e50 2024-04-25T08:48:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6f4c00640 1 -- 172.21.15.46:0/533969282 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00047f0 con 0x7fd6f8140e50 2024-04-25T08:48:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f813c380 con 0x7fd6f8140e50 2024-04-25T08:48:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6f4c00640 1 -- 172.21.15.46:0/533969282 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0004be0 con 0x7fd6f8140e50 2024-04-25T08:48:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f813c940 con 0x7fd6f8140e50 2024-04-25T08:48:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f80cca10 con 0x7fd6f8140e50 2024-04-25T08:48:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6f4c00640 1 -- 172.21.15.46:0/533969282 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6f0011070 con 0x7fd6f8140e50 2024-04-25T08:48:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6f4c00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd6d4066f90 0x7fd6d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6f4c00640 1 -- 172.21.15.46:0/533969282 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd6f00848c0 con 0x7fd6f8140e50 2024-04-25T08:48:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.128+0000 7fd6f7e00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd6d4066f90 0x7fd6d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:21.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.132+0000 7fd6f7e00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd6d4066f90 0x7fd6d4069450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8003fd0 tx=0x7fd6e80039e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:21.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.132+0000 7fd6f4c00640 1 -- 172.21.15.46:0/533969282 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6f0055a80 con 0x7fd6f8140e50 2024-04-25T08:48:21.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:21.352+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fd6f815f410 con 0x7fd6f8140e50 2024-04-25T08:48:22.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.268+0000 7fd6f4c00640 1 -- 172.21.15.46:0/533969282 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v306) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd6f0055380 con 0x7fd6f8140e50 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd6d4066f90 msgr2=0x7fd6d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd6d4066f90 0x7fd6d4069450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8003fd0 tx=0x7fd6e80039e0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8140e50 msgr2=0x7fd6f81582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8140e50 0x7fd6f81582a0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000c3e0 tx=0x7fd6f000c8a0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 shutdown_connections 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd6d4066f90 0x7fd6d4069450 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6f8140e50 0x7fd6f81582a0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6f813e540 0x7fd6f8136af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 --2- 172.21.15.46:0/533969282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd6f8071ea0 0x7fd6f813d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 >> 172.21.15.46:0/533969282 conn(0x7fd6f8066fa0 msgr2=0x7fd6f813a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 shutdown_connections 2024-04-25T08:48:22.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.272+0000 7fd6fdc00640 1 -- 172.21.15.46:0/533969282 wait complete. 2024-04-25T08:48:22.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:48:22.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.444+0000 7fa719e00640 1 Processor -- start 2024-04-25T08:48:22.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.444+0000 7fa719e00640 1 -- start start 2024-04-25T08:48:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.444+0000 7fa719e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa71406b470 0x7fa7141688f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.444+0000 7fa719e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71406fca0 con 0x7fa7140afdf0 2024-04-25T08:48:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.444+0000 7fa719e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71406ff80 con 0x7fa71415d900 2024-04-25T08:48:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.444+0000 7fa719e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa714070260 con 0x7fa71406b470 2024-04-25T08:48:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa70a000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa7140afdf0 0x7fa71406ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42840/0 (socket says 172.21.15.46:42840) 2024-04-25T08:48:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa70a000640 1 -- 172.21.15.46:0/2493754047 learned_addr learned my addr 172.21.15.46:0/2493754047 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa713400640 1 --2- 172.21.15.46:0/2493754047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa71406b470 0x7fa7141688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa712000640 1 -- 172.21.15.46:0/2493754047 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401102708 0 0) 0x7fa71406fca0 con 0x7fa7140afdf0 2024-04-25T08:48:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa712000640 1 -- 172.21.15.46:0/2493754047 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8003530 con 0x7fa7140afdf0 2024-04-25T08:48:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa712000640 1 -- 172.21.15.46:0/2493754047 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884482488 0 0) 0x7fa71406ff80 con 0x7fa71415d900 2024-04-25T08:48:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa712000640 1 -- 172.21.15.46:0/2493754047 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa71406fca0 con 0x7fa71415d900 2024-04-25T08:48:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa713400640 1 -- 172.21.15.46:0/2493754047 >> v1:172.21.15.82:6789/0 conn(0x7fa71415d900 legacy=0x7fa7140af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa713400640 1 -- 172.21.15.46:0/2493754047 >> v1:172.21.15.46:6789/0 conn(0x7fa7140afdf0 legacy=0x7fa71406ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:22.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa713400640 1 -- 172.21.15.46:0/2493754047 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa714070540 con 0x7fa71406b470 2024-04-25T08:48:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.448+0000 7fa713400640 1 --2- 172.21.15.46:0/2493754047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa71406b470 0x7fa7141688f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0097f0 tx=0x7fa70c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3bd04a62e081622 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa712000640 1 -- 172.21.15.46:0/2493754047 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0b0070 con 0x7fa71406b470 2024-04-25T08:48:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa712000640 1 -- 172.21.15.46:0/2493754047 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c0ab600 con 0x7fa71406b470 2024-04-25T08:48:22.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa712000640 1 -- 172.21.15.46:0/2493754047 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0ab9d0 con 0x7fa71406b470 2024-04-25T08:48:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 -- 172.21.15.46:0/2493754047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa71406b470 msgr2=0x7fa7141688f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 --2- 172.21.15.46:0/2493754047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa71406b470 0x7fa7141688f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0097f0 tx=0x7fa70c01c450 comp rx=0 tx=0).stop 2024-04-25T08:48:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 -- 172.21.15.46:0/2493754047 shutdown_connections 2024-04-25T08:48:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 --2- 172.21.15.46:0/2493754047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa71406b470 0x7fa7141688f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 -- 172.21.15.46:0/2493754047 >> 172.21.15.46:0/2493754047 conn(0x7fa71406e3e0 msgr2=0x7fa71406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 -- 172.21.15.46:0/2493754047 shutdown_connections 2024-04-25T08:48:22.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 -- 172.21.15.46:0/2493754047 wait complete. 2024-04-25T08:48:22.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 Processor -- start 2024-04-25T08:48:22.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 -- start start 2024-04-25T08:48:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa71406b470 0x7fa71415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7140afdf0 0x7fa7140d0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71415d900 0x7fa714147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71416b7d0 con 0x7fa71415d900 2024-04-25T08:48:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71416b650 con 0x7fa71406b470 2024-04-25T08:48:22.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa719e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71416b950 con 0x7fa7140afdf0 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa713400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71415d900 0x7fa714147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa713400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71415d900 0x7fa714147f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35292/0 (socket says 172.21.15.46:35292) 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa713400640 1 -- 172.21.15.46:0/3918133753 learned_addr learned my addr 172.21.15.46:0/3918133753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa712a00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa71406b470 0x7fa71415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa70a000640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7140afdf0 0x7fa7140d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa713400640 1 -- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7140afdf0 msgr2=0x7fa7140d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa713400640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7140afdf0 0x7fa7140d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa713400640 1 -- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa71406b470 msgr2=0x7fa71415af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.452+0000 7fa713400640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa71406b470 0x7fa71415af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa713400640 1 -- 172.21.15.46:0/3918133753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa714158470 con 0x7fa71415d900 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa713400640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71415d900 0x7fa714147f90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0097f0 tx=0x7fa70c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa70be00640 1 -- 172.21.15.46:0/3918133753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0b0040 con 0x7fa71415d900 2024-04-25T08:48:22.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa70be00640 1 -- 172.21.15.46:0/3918133753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c0b9070 con 0x7fa71415d900 2024-04-25T08:48:22.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa70be00640 1 -- 172.21.15.46:0/3918133753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0b4650 con 0x7fa71415d900 2024-04-25T08:48:22.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71415b750 con 0x7fa71415d900 2024-04-25T08:48:22.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7141589e0 con 0x7fa71415d900 2024-04-25T08:48:22.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa714069010 con 0x7fa71415d900 2024-04-25T08:48:22.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa70be00640 1 -- 172.21.15.46:0/3918133753 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa70c0b47f0 con 0x7fa71415d900 2024-04-25T08:48:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa70be00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6f0066fb0 0x7fa6f0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa70be00640 1 -- 172.21.15.46:0/3918133753 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa70c11e990 con 0x7fa71415d900 2024-04-25T08:48:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.456+0000 7fa712a00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6f0066fb0 0x7fa6f0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.460+0000 7fa712a00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6f0066fb0 0x7fa6f0069470 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa704001720 tx=0x7fa704009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:22.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.460+0000 7fa70be00640 1 -- 172.21.15.46:0/3918133753 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa70c0160d0 con 0x7fa71415d900 2024-04-25T08:48:22.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.624+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa714072890 con 0x7fa71415d900 2024-04-25T08:48:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa70be00640 1 -- 172.21.15.46:0/3918133753 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa70c0ef4e0 con 0x7fa71415d900 2024-04-25T08:48:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6f0066fb0 msgr2=0x7fa6f0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6f0066fb0 0x7fa6f0069470 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa704001720 tx=0x7fa704009040 comp rx=0 tx=0).stop 2024-04-25T08:48:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71415d900 msgr2=0x7fa714147f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71415d900 0x7fa714147f90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0097f0 tx=0x7fa70c0191f0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 shutdown_connections 2024-04-25T08:48:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6f0066fb0 0x7fa6f0069470 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71415d900 0x7fa714147f90 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7140afdf0 0x7fa7140d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 --2- 172.21.15.46:0/3918133753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa71406b470 0x7fa71415af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:22.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 >> 172.21.15.46:0/3918133753 conn(0x7fa71406e3e0 msgr2=0x7fa714150660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:22.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 shutdown_connections 2024-04-25T08:48:22.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:22.628+0000 7fa719e00640 1 -- 172.21.15.46:0/3918133753 wait complete. 2024-04-25T08:48:22.643 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:48:22.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:48:22.723 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:48:22.833 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:48:22.834 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:22.834 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:22.842 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:48:22.843 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:48:22.843 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:22.852 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:48:22.852 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:22.852 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:22.892 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:48:22.892 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:48:22.892 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:22.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:48:22.902 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:48:22.902 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.080+0000 7f7fa3e00640 1 Processor -- start 2024-04-25T08:48:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.080+0000 7f7fa3e00640 1 -- start start 2024-04-25T08:48:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa415f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7fa4099a00 0x7f7fa4099e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40b2da0 con 0x7f7fa406c2b0 2024-04-25T08:48:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa3e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa4161b80 con 0x7f7fa4099a00 2024-04-25T08:48:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa409a370 con 0x7f7fa40b29c0 2024-04-25T08:48:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7f9a000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7fa406c2b0 0x7f7fa4161470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42856/0 (socket says 172.21.15.46:42856) 2024-04-25T08:48:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7f9a000640 1 -- 172.21.15.46:0/3568973617 learned_addr learned my addr 172.21.15.46:0/3568973617 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/3568973617 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa3400640 1 --2- 172.21.15.46:0/3568973617 >> v2:172.21.15.82:3300/0 conn(0x7f7fa4099a00 0x7f7fa4099e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa2000640 1 -- 172.21.15.46:0/3568973617 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748960310 0 0) 0x7f7fa40b2da0 con 0x7f7fa406c2b0 2024-04-25T08:48:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa2000640 1 -- 172.21.15.46:0/3568973617 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f88003530 con 0x7f7fa406c2b0 2024-04-25T08:48:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa2a00640 1 -- 172.21.15.46:0/3568973617 >> v2:172.21.15.82:3300/0 conn(0x7f7fa4099a00 msgr2=0x7f7fa4099e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/3568973617 >> v2:172.21.15.82:3300/0 conn(0x7f7fa4099a00 0x7f7fa4099e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa2a00640 1 -- 172.21.15.46:0/3568973617 >> v1:172.21.15.46:6789/0 conn(0x7f7fa406c2b0 legacy=0x7f7fa4161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa2000640 1 -- 172.21.15.46:0/3568973617 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17983927 0 0) 0x7f7f88003530 con 0x7f7fa406c2b0 2024-04-25T08:48:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.084+0000 7f7fa2a00640 1 -- 172.21.15.46:0/3568973617 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa41623a0 con 0x7f7fa40b29c0 2024-04-25T08:48:23.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/3568973617 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa415f250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097c0 tx=0x7f7f9401c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc5c10690f068ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa2000640 1 -- 172.21.15.46:0/3568973617 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f940b0070 con 0x7f7fa40b29c0 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa2000640 1 -- 172.21.15.46:0/3568973617 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f940ab400 con 0x7f7fa40b29c0 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa2000640 1 -- 172.21.15.46:0/3568973617 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c001a00 con 0x7f7fa40b29c0 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 -- 172.21.15.46:0/3568973617 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 msgr2=0x7f7fa415f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/3568973617 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa415f250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097c0 tx=0x7f7f9401c390 comp rx=0 tx=0).stop 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 -- 172.21.15.46:0/3568973617 shutdown_connections 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/3568973617 >> v2:172.21.15.82:3300/0 conn(0x7f7fa4099a00 0x7f7fa4099e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/3568973617 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa415f250 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 -- 172.21.15.46:0/3568973617 >> 172.21.15.46:0/3568973617 conn(0x7f7fa40b2f90 msgr2=0x7f7fa40b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 -- 172.21.15.46:0/3568973617 shutdown_connections 2024-04-25T08:48:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 -- 172.21.15.46:0/3568973617 wait complete. 2024-04-25T08:48:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 Processor -- start 2024-04-25T08:48:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 -- start start 2024-04-25T08:48:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa406c2b0 0x7f7fa40c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.088+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa4099a00 0x7f7fa41f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa41507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa415e930 con 0x7f7fa406c2b0 2024-04-25T08:48:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa40cfc90 con 0x7f7fa4099a00 2024-04-25T08:48:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa415eab0 con 0x7f7fa40b29c0 2024-04-25T08:48:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa406c2b0 0x7f7fa40c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa406c2b0 0x7f7fa40c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35300/0 (socket says 172.21.15.46:35300) 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa2a00640 1 -- 172.21.15.46:0/1721705296 learned_addr learned my addr 172.21.15.46:0/1721705296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7f9a000640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa4099a00 0x7f7fa41f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa3400640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa41507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa2a00640 1 -- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 msgr2=0x7f7fa41507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa41507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa2a00640 1 -- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa4099a00 msgr2=0x7f7fa41f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa4099a00 0x7f7fa41f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa2a00640 1 -- 172.21.15.46:0/1721705296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa41612b0 con 0x7f7fa406c2b0 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7f9a000640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa4099a00 0x7f7fa41f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa3400640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa41507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa2a00640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa406c2b0 0x7f7fa40c6700 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097c0 tx=0x7f7f94019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7f9b400640 1 -- 172.21.15.46:0/1721705296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f940b0040 con 0x7f7fa406c2b0 2024-04-25T08:48:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7f9b400640 1 -- 172.21.15.46:0/1721705296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f940ba070 con 0x7f7fa406c2b0 2024-04-25T08:48:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7f9b400640 1 -- 172.21.15.46:0/1721705296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94006e20 con 0x7f7fa406c2b0 2024-04-25T08:48:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa40c8380 con 0x7f7fa406c2b0 2024-04-25T08:48:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa4152400 con 0x7f7fa406c2b0 2024-04-25T08:48:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa4069010 con 0x7f7fa406c2b0 2024-04-25T08:48:23.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.092+0000 7f7f9b400640 1 -- 172.21.15.46:0/1721705296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f940b4650 con 0x7f7fa406c2b0 2024-04-25T08:48:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.096+0000 7f7f9b400640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.096+0000 7f7f9b400640 1 -- 172.21.15.46:0/1721705296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7f9411eac0 con 0x7f7fa406c2b0 2024-04-25T08:48:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.096+0000 7f7f9a000640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.096+0000 7f7f9a000640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00d210 tx=0x7f7f8c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.100+0000 7f7f9b400640 1 -- 172.21.15.46:0/1721705296 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f94015fb0 con 0x7f7fa406c2b0 2024-04-25T08:48:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.292+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fa40b3ea0 con 0x7f7fa406c2b0 2024-04-25T08:48:23.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7f9b400640 1 -- 172.21.15.46:0/1721705296 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f7f940efc80 con 0x7f7fa406c2b0 2024-04-25T08:48:23.289 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:23.289 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6526,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/4009791610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4009791610},{"type":"v1","addr":"172.21.15.82:6836","nonce":4009791610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6529,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1169650336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1169650336},{"type":"v1","addr":"172.21.15.82:6837","nonce":1169650336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":6771,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/3235544280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3235544280},{"type":"v1","addr":"172.21.15.46:6834","nonce":3235544280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7383,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2752419900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2752419900},{"type":"v1","addr":"172.21.15.46:6835","nonce":2752419900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[]} 2024-04-25T08:48:23.289 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 55 2024-04-25T08:48:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 msgr2=0x7f7f80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00d210 tx=0x7f7f8c009040 comp rx=0 tx=0).stop 2024-04-25T08:48:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa406c2b0 msgr2=0x7f7fa40c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa406c2b0 0x7f7fa40c6700 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7f940097c0 tx=0x7f7f94019130 comp rx=0 tx=0).stop 2024-04-25T08:48:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 shutdown_connections 2024-04-25T08:48:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7f80066f60 0x7f7f80069420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7fa40b29c0 0x7f7fa41507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fa4099a00 0x7f7fa41f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1721705296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fa406c2b0 0x7f7fa40c6700 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 >> 172.21.15.46:0/1721705296 conn(0x7f7fa40b2f90 msgr2=0x7f7fa4073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 shutdown_connections 2024-04-25T08:48:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.296+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1721705296 wait complete. 2024-04-25T08:48:23.311 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:48:23.311 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:48:23.311 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:48:23.311 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:48:23.311 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:48:23.311 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:48:23.312 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:48:23.312 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:48:23.312 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:48:23.312 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:48:23.312 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:48:23.312 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:48:23.312 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:48:23.313 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:48:23.313 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:48:23.313 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:48:23.322 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:48:23.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:48:23.324 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:48:23.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:48:23.325 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:48:23.325 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:48:23.326 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:48:23.327 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:48:23.369 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:48:23.369 DEBUG:teuthology.parallel:result is None 2024-04-25T08:48:23.370 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:48:23.371 DEBUG:teuthology.parallel:result is None 2024-04-25T08:48:23.371 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:48:23.371 DEBUG:teuthology.parallel:result is None 2024-04-25T08:48:23.372 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:48:23.373 DEBUG:teuthology.parallel:result is None 2024-04-25T08:48:23.373 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:48:23.455 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-04-25T08:48:23.457 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:48:23.458 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-04-25T08:48:23.459 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:48:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.524+0000 7f1c38000640 1 Processor -- start 2024-04-25T08:48:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.524+0000 7f1c38000640 1 -- start start 2024-04-25T08:48:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.524+0000 7f1c38000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 0x7f1c300cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.524+0000 7f1c38000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c300b2240 0x7f1c300b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.524+0000 7f1c38000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3006bdc0 con 0x7f1c30099a00 2024-04-25T08:48:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.524+0000 7f1c38000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3006db90 con 0x7f1c300b2240 2024-04-25T08:48:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.524+0000 7f1c38000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3006c0a0 con 0x7f1c300ccde0 2024-04-25T08:48:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c35600640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1c30099a00 0x7f1c30099de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:47714/0 (socket says 172.21.15.46:47714) 2024-04-25T08:48:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c35600640 1 -- 172.21.15.46:0/2657961817 learned_addr learned my addr 172.21.15.46:0/2657961817 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c34c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 0x7f1c300cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c36000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c300b2240 0x7f1c300b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c2fe00640 1 -- 172.21.15.46:0/2657961817 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416659594 0 0) 0x7f1c3006bdc0 con 0x7f1c30099a00 2024-04-25T08:48:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c2fe00640 1 -- 172.21.15.46:0/2657961817 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c18003530 con 0x7f1c30099a00 2024-04-25T08:48:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c36000640 1 -- 172.21.15.46:0/2657961817 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 msgr2=0x7f1c300cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c36000640 1 --2- 172.21.15.46:0/2657961817 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 0x7f1c300cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c36000640 1 -- 172.21.15.46:0/2657961817 >> v1:172.21.15.82:6789/0 conn(0x7f1c30099a00 legacy=0x7f1c30099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c36000640 1 -- 172.21.15.46:0/2657961817 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3006c380 con 0x7f1c300b2240 2024-04-25T08:48:23.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c36000640 1 --2- 172.21.15.46:0/2657961817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 0x7f1c300b26d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009d90 tx=0x7f1c2801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c20a3a9552f28598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:23.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c2fe00640 1 -- 172.21.15.46:0/2657961817 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c280b2070 con 0x7f1c300b2240 2024-04-25T08:48:23.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c2fe00640 1 -- 172.21.15.46:0/2657961817 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c280a5900 con 0x7f1c300b2240 2024-04-25T08:48:23.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.528+0000 7f1c2fe00640 1 -- 172.21.15.46:0/2657961817 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c280a5cd0 con 0x7f1c300b2240 2024-04-25T08:48:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 -- 172.21.15.46:0/2657961817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 msgr2=0x7f1c300b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 --2- 172.21.15.46:0/2657961817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 0x7f1c300b26d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009d90 tx=0x7f1c2801c8b0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 -- 172.21.15.46:0/2657961817 shutdown_connections 2024-04-25T08:48:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 --2- 172.21.15.46:0/2657961817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 0x7f1c300b26d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 --2- 172.21.15.46:0/2657961817 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 0x7f1c300cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 -- 172.21.15.46:0/2657961817 >> 172.21.15.46:0/2657961817 conn(0x7f1c300b3120 msgr2=0x7f1c300b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 -- 172.21.15.46:0/2657961817 shutdown_connections 2024-04-25T08:48:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 -- 172.21.15.46:0/2657961817 wait complete. 2024-04-25T08:48:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 Processor -- start 2024-04-25T08:48:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 -- start start 2024-04-25T08:48:23.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c30099a00 0x7f1c30152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 0x7f1c3015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 0x7f1c300cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c300cd920 con 0x7f1c30099a00 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c300cd7a0 con 0x7f1c300b2240 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c38000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3015dd10 con 0x7f1c300ccde0 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c35600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c30099a00 0x7f1c30152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c35600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c30099a00 0x7f1c30152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35328/0 (socket says 172.21.15.46:35328) 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c35600640 1 -- 172.21.15.46:0/1824426900 learned_addr learned my addr 172.21.15.46:0/1824426900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c34c00640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 0x7f1c3015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c36000640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 0x7f1c300cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c35600640 1 -- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 msgr2=0x7f1c300cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c35600640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 0x7f1c300cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c35600640 1 -- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 msgr2=0x7f1c3015e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c35600640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 0x7f1c3015e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c35600640 1 -- 172.21.15.46:0/1824426900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3015ead0 con 0x7f1c30099a00 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c34c00640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 0x7f1c3015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.532+0000 7f1c36000640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 0x7f1c300cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:23.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c35600640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c30099a00 0x7f1c30152b40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400c3f0 tx=0x7f1c2400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:23.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c2d600640 1 -- 172.21.15.46:0/1824426900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24017070 con 0x7f1c30099a00 2024-04-25T08:48:23.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c2d600640 1 -- 172.21.15.46:0/1824426900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c24005cf0 con 0x7f1c30099a00 2024-04-25T08:48:23.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c2d600640 1 -- 172.21.15.46:0/1824426900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c240072f0 con 0x7f1c30099a00 2024-04-25T08:48:23.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3015edb0 con 0x7f1c30099a00 2024-04-25T08:48:23.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3015f370 con 0x7f1c30099a00 2024-04-25T08:48:23.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c30069080 con 0x7f1c30099a00 2024-04-25T08:48:23.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c2d600640 1 -- 172.21.15.46:0/1824426900 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c2401a070 con 0x7f1c30099a00 2024-04-25T08:48:23.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c2d600640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c10066f60 0x7f1c10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:23.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c2d600640 1 -- 172.21.15.46:0/1824426900 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f1c240843f0 con 0x7f1c30099a00 2024-04-25T08:48:23.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.536+0000 7f1c34c00640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c10066f60 0x7f1c10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:23.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.540+0000 7f1c34c00640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c10066f60 0x7f1c10069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1c300ce570 tx=0x7f1c1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:23.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.540+0000 7f1c2d600640 1 -- 172.21.15.46:0/1824426900 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c24055480 con 0x7f1c30099a00 2024-04-25T08:48:23.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:23.716+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1c30156590 con 0x7f1c30099a00 2024-04-25T08:48:24.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c2d600640 1 -- 172.21.15.46:0/1824426900 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v84) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1c24054d80 con 0x7f1c30099a00 2024-04-25T08:48:24.281 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-25T08:48:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c10066f60 msgr2=0x7f1c10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c10066f60 0x7f1c10069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1c300ce570 tx=0x7f1c1c008040 comp rx=0 tx=0).stop 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c30099a00 msgr2=0x7f1c30152b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c30099a00 0x7f1c30152b40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400c3f0 tx=0x7f1c2400c8b0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 shutdown_connections 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c10066f60 0x7f1c10069420 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c300ccde0 0x7f1c300cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c300b2240 0x7f1c3015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 --2- 172.21.15.46:0/1824426900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c30099a00 0x7f1c30152b40 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.288+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 >> 172.21.15.46:0/1824426900 conn(0x7f1c300b3120 msgr2=0x7f1c300ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.292+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 shutdown_connections 2024-04-25T08:48:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.292+0000 7f1c38000640 1 -- 172.21.15.46:0/1824426900 wait complete. 2024-04-25T08:48:24.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:48:24.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.460+0000 7f17d2800640 1 Processor -- start 2024-04-25T08:48:24.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17d2800640 1 -- start start 2024-04-25T08:48:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17d2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc15bdb0 0x7f17cc0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17d2800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc0cc700 con 0x7f17cc159230 2024-04-25T08:48:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17d2800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc0cc9e0 con 0x7f17cc0cda80 2024-04-25T08:48:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17d2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0cccc0 con 0x7f17cc15bdb0 2024-04-25T08:48:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17cb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f17cc159230 0x7f17cc0cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42898/0 (socket says 172.21.15.46:42898) 2024-04-25T08:48:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17cb400640 1 -- 172.21.15.46:0/2629093509 learned_addr learned my addr 172.21.15.46:0/2629093509 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430184444 0 0) 0x7f17cc0cc700 con 0x7f17cc159230 2024-04-25T08:48:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17b0003530 con 0x7f17cc159230 2024-04-25T08:48:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17c3e00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc15bdb0 0x7f17cc0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4172280624 0 0) 0x7f17b0003530 con 0x7f17cc159230 2024-04-25T08:48:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc15bdb0 msgr2=0x7f17cc0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc15bdb0 0x7f17cc0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 >> v1:172.21.15.82:6789/0 conn(0x7f17cc0cda80 legacy=0x7f17cc15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0ccfa0 con 0x7f17cc159230 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f17bc001680 con 0x7f17cc159230 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17c3e00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc15bdb0 0x7f17cc0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 >> v1:172.21.15.46:6789/0 conn(0x7f17cc159230 legacy=0x7f17cc0cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc15bdb0 0x7f17b00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b0003fe0 0x7f17b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b0004d50 0x7f17b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0cccc0 con 0x7f17b0004d50 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b000bbc0 con 0x7f17b0003fe0 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b000bdc0 con 0x7f17cc15bdb0 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17cbe00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b0004d50 0x7f17b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17cb400640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc15bdb0 0x7f17b00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17c3e00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b0003fe0 0x7f17b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17cbe00640 1 -- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc15bdb0 msgr2=0x7f17b00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17cbe00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc15bdb0 0x7f17b00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17cbe00640 1 -- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b0003fe0 msgr2=0x7f17b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17cbe00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b0003fe0 0x7f17b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.464+0000 7f17cbe00640 1 -- 172.21.15.46:0/2629093509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0003530 con 0x7f17b0004d50 2024-04-25T08:48:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17cbe00640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b0004d50 0x7f17b00051d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f17c4001e70 tx=0x7f17c401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92967543dd821390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4001a90 con 0x7f17b0004d50 2024-04-25T08:48:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c4003b80 con 0x7f17b0004d50 2024-04-25T08:48:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17caa00640 1 -- 172.21.15.46:0/2629093509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40050d0 con 0x7f17b0004d50 2024-04-25T08:48:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17d2800640 1 -- 172.21.15.46:0/2629093509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b0004d50 msgr2=0x7f17b00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17d2800640 1 --2- 172.21.15.46:0/2629093509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b0004d50 0x7f17b00051d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f17c4001e70 tx=0x7f17c401c780 comp rx=0 tx=0).stop 2024-04-25T08:48:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17cbe00640 1 -- 172.21.15.46:0/2629093509 reap_dead start 2024-04-25T08:48:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17d2800640 1 -- 172.21.15.46:0/2629093509 shutdown_connections 2024-04-25T08:48:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17d2800640 1 -- 172.21.15.46:0/2629093509 >> 172.21.15.46:0/2629093509 conn(0x7f17cc0b12a0 msgr2=0x7f17cc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17d2800640 1 -- 172.21.15.46:0/2629093509 shutdown_connections 2024-04-25T08:48:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.468+0000 7f17d2800640 1 -- 172.21.15.46:0/2629093509 wait complete. 2024-04-25T08:48:24.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 Processor -- start 2024-04-25T08:48:24.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 -- start start 2024-04-25T08:48:24.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17cc13c350 0x7f17cc06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc13c730 0x7f17cc1423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17cc13cc50 0x7f17cc142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc159790 con 0x7f17cc13c350 2024-04-25T08:48:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc159610 con 0x7f17cc13cc50 2024-04-25T08:48:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0cd540 con 0x7f17cc13c730 2024-04-25T08:48:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17cc13c350 0x7f17cc06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17cc13c350 0x7f17cc06eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35352/0 (socket says 172.21.15.46:35352) 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cb400640 1 -- 172.21.15.46:0/795301205 learned_addr learned my addr 172.21.15.46:0/795301205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17c3e00640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc13c730 0x7f17cc1423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cbe00640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17cc13cc50 0x7f17cc142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cb400640 1 -- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc13c730 msgr2=0x7f17cc1423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cb400640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc13c730 0x7f17cc1423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cb400640 1 -- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17cc13cc50 msgr2=0x7f17cc142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cb400640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17cc13cc50 0x7f17cc142920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cb400640 1 -- 172.21.15.46:0/795301205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc06f530 con 0x7f17cc13c350 2024-04-25T08:48:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17c3e00640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc13c730 0x7f17cc1423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17cb400640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17cc13c350 0x7f17cc06eff0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f17bc012540 tx=0x7f17bc012a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17c3400640 1 -- 172.21.15.46:0/795301205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc01c070 con 0x7f17cc13c350 2024-04-25T08:48:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17c3400640 1 -- 172.21.15.46:0/795301205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc014810 con 0x7f17cc13c350 2024-04-25T08:48:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17c3400640 1 -- 172.21.15.46:0/795301205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc014c00 con 0x7f17cc13c350 2024-04-25T08:48:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc13e560 con 0x7f17cc13c350 2024-04-25T08:48:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.472+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc13eb20 con 0x7f17cc13c350 2024-04-25T08:48:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.476+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc13f0e0 con 0x7f17cc13c350 2024-04-25T08:48:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.476+0000 7f17c3400640 1 -- 172.21.15.46:0/795301205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17bc0153e0 con 0x7f17cc13c350 2024-04-25T08:48:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.476+0000 7f17c3400640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17a8066f90 0x7f17a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.476+0000 7f17c3400640 1 -- 172.21.15.46:0/795301205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f17bc087870 con 0x7f17cc13c350 2024-04-25T08:48:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.476+0000 7f17c3e00640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17a8066f90 0x7f17a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.476+0000 7f17c3e00640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17a8066f90 0x7f17a8069450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f17cc1430c0 tx=0x7f17b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:24.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.480+0000 7f17c3400640 1 -- 172.21.15.46:0/795301205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17bc058350 con 0x7f17cc13c350 2024-04-25T08:48:24.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.700+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f17cc069080 con 0x7f17cc13c350 2024-04-25T08:48:24.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.700+0000 7f17c3400640 1 -- 172.21.15.46:0/795301205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f17bc05c3f0 con 0x7f17cc13c350 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmq+gDGRAAtB5ObM/zelMmCZ4cJPghdQ==", 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:48:24.695 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmyIaAJRAA9zodtgftP7faNth4UPyCTA==", 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmFahWIBAAORHmzDLXPjuANF+/W2wKpQ==", 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmRhrPLBAATXOHl1jGJYmIlnYk/JPcOw==", 2024-04-25T08:48:24.697 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA0FypmaHiqEhAAYzasCl8t+pls5IHe9tCP4g==", 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:48:24.698 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA3FypmOGTuGhAA37F+qRI/+pj5dkLc3TBIUQ==", 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-25T08:48:24.699 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA6Fypm/YbNJRAAP1QyYDLY/xFvRz63R+H/Pw==", 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA9FypmiT8ALhAAX/2CJJeJ+2SGNhwah8NTBQ==", 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBDFypmyG88JBAAq1065gzLIIR/6fYaw0bBTg==", 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBGFypmNir3KxAAszCXJJ1KFtCusDZhBHAZSQ==", 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBJFypm4r7LNxAA42A07ntI6OOgblk+4i1Yiw==", 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBNFypmG1fOBhAAOKRgw04jIo8H3u2hWIZe2g==", 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q==", 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-25T08:48:24.703 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmxPyHOhAADBuigZlq3GHURsJkB0nnyA==", 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.704 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypm2PP0NhAAixV/kkY7eFPXUW/nJDOCOA==", 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAzFypmaVDtBBAAKnWk9CxZzg2r6T/IycuFEg==", 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:48:24.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAxFypm2W2MOBAAN7TDQiZXFV6+pWMTar95sg==", 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmzhptFBAAlFQZQn1GSseEguy5zfw2CA==", 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmAY6CEBAA2qHNALNTRKmCJRZatDqAvw==", 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:48:24.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:48:24.708 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:48:24.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:24.708 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-25T08:48:24.708 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:48:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.700+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17a8066f90 msgr2=0x7f17a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.700+0000 7f17d2800640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17a8066f90 0x7f17a8069450 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f17cc1430c0 tx=0x7f17b4008040 comp rx=0 tx=0).stop 2024-04-25T08:48:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.700+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17cc13c350 msgr2=0x7f17cc06eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.700+0000 7f17d2800640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17cc13c350 0x7f17cc06eff0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f17bc012540 tx=0x7f17bc012a00 comp rx=0 tx=0).stop 2024-04-25T08:48:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.704+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 shutdown_connections 2024-04-25T08:48:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.704+0000 7f17d2800640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f17a8066f90 0x7f17a8069450 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.704+0000 7f17d2800640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17cc13cc50 0x7f17cc142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.704+0000 7f17d2800640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f17cc13c730 0x7f17cc1423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.704+0000 7f17d2800640 1 --2- 172.21.15.46:0/795301205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17cc13c350 0x7f17cc06eff0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.704+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 >> 172.21.15.46:0/795301205 conn(0x7f17cc0b12a0 msgr2=0x7f17cc15f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.704+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 shutdown_connections 2024-04-25T08:48:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.704+0000 7f17d2800640 1 -- 172.21.15.46:0/795301205 wait complete. 2024-04-25T08:48:24.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:24.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f9082c00640 1 Processor -- start 2024-04-25T08:48:24.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f9082c00640 1 -- start start 2024-04-25T08:48:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f9082c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f907c0ccde0 0x7f907c0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f9082c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f907c0b2240 0x7f907c0b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f9082c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907c06bdc0 con 0x7f907c099a00 2024-04-25T08:48:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f9082c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907c06db90 con 0x7f907c0ccde0 2024-04-25T08:48:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f9082c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907c06c0a0 con 0x7f907c0b2240 2024-04-25T08:48:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f907b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f907c0ccde0 0x7f907c0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f907b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f907c0ccde0 0x7f907c0cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35358/0 (socket says 172.21.15.46:35358) 2024-04-25T08:48:24.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f907b400640 1 -- 172.21.15.46:0/2517255932 learned_addr learned my addr 172.21.15.46:0/2517255932 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f9080c00640 1 --2- 172.21.15.46:0/2517255932 >> v2:172.21.15.82:3300/0 conn(0x7f907c0b2240 0x7f907c0b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f907b400640 1 -- 172.21.15.46:0/2517255932 >> v2:172.21.15.82:3300/0 conn(0x7f907c0b2240 msgr2=0x7f907c0b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f907b400640 1 --2- 172.21.15.46:0/2517255932 >> v2:172.21.15.82:3300/0 conn(0x7f907c0b2240 0x7f907c0b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f907b400640 1 -- 172.21.15.46:0/2517255932 >> v1:172.21.15.82:6789/0 conn(0x7f907c099a00 legacy=0x7f907c099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:24.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.864+0000 7f907b400640 1 -- 172.21.15.46:0/2517255932 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c06c380 con 0x7f907c0ccde0 2024-04-25T08:48:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f907b400640 1 --2- 172.21.15.46:0/2517255932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 0x7f907c0cd290 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f9068009870 tx=0x7f906801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3937a8900914a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f907aa00640 1 -- 172.21.15.46:0/2517255932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90680b0070 con 0x7f907c0ccde0 2024-04-25T08:48:24.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f907aa00640 1 -- 172.21.15.46:0/2517255932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90680ab600 con 0x7f907c0ccde0 2024-04-25T08:48:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f907aa00640 1 -- 172.21.15.46:0/2517255932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90680ab9d0 con 0x7f907c0ccde0 2024-04-25T08:48:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f9082c00640 1 -- 172.21.15.46:0/2517255932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 msgr2=0x7f907c0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f9082c00640 1 --2- 172.21.15.46:0/2517255932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 0x7f907c0cd290 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f9068009870 tx=0x7f906801c420 comp rx=0 tx=0).stop 2024-04-25T08:48:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f9082c00640 1 -- 172.21.15.46:0/2517255932 shutdown_connections 2024-04-25T08:48:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f9082c00640 1 --2- 172.21.15.46:0/2517255932 >> v2:172.21.15.82:3300/0 conn(0x7f907c0b2240 0x7f907c0b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f9082c00640 1 --2- 172.21.15.46:0/2517255932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 0x7f907c0cd290 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f9082c00640 1 -- 172.21.15.46:0/2517255932 >> 172.21.15.46:0/2517255932 conn(0x7f907c0b3120 msgr2=0x7f907c0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f9082c00640 1 -- 172.21.15.46:0/2517255932 shutdown_connections 2024-04-25T08:48:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.868+0000 7f9082c00640 1 -- 172.21.15.46:0/2517255932 wait complete. 2024-04-25T08:48:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 Processor -- start 2024-04-25T08:48:24.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 -- start start 2024-04-25T08:48:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f907c099a00 0x7f907c153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f907c0b2240 0x7f907c14d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 0x7f907c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c0cd7d0 con 0x7f907c0ccde0 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c15dd10 con 0x7f907c0b2240 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907c158360 con 0x7f907c099a00 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9080c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 0x7f907c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9080c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 0x7f907c15e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35370/0 (socket says 172.21.15.46:35370) 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9080c00640 1 -- 172.21.15.46:0/2651197650 learned_addr learned my addr 172.21.15.46:0/2651197650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f907be00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f907c099a00 0x7f907c153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f907b400640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f907c0b2240 0x7f907c14d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9080c00640 1 -- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f907c099a00 msgr2=0x7f907c153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9080c00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f907c099a00 0x7f907c153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9080c00640 1 -- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f907c0b2240 msgr2=0x7f907c14d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9080c00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f907c0b2240 0x7f907c14d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9080c00640 1 -- 172.21.15.46:0/2651197650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c155770 con 0x7f907c0ccde0 2024-04-25T08:48:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9080c00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 0x7f907c15e040 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f9074007e40 tx=0x7f907400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:24.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9073e00640 1 -- 172.21.15.46:0/2651197650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074014620 con 0x7f907c0ccde0 2024-04-25T08:48:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9073e00640 1 -- 172.21.15.46:0/2651197650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074019070 con 0x7f907c0ccde0 2024-04-25T08:48:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9073e00640 1 -- 172.21.15.46:0/2651197650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074014a20 con 0x7f907c0ccde0 2024-04-25T08:48:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907c154300 con 0x7f907c0ccde0 2024-04-25T08:48:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907c151940 con 0x7f907c0ccde0 2024-04-25T08:48:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.872+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907c069080 con 0x7f907c0ccde0 2024-04-25T08:48:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.876+0000 7f9073e00640 1 -- 172.21.15.46:0/2651197650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9074014bc0 con 0x7f907c0ccde0 2024-04-25T08:48:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.876+0000 7f9073e00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f905c066f60 0x7f905c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.876+0000 7f9073e00640 1 -- 172.21.15.46:0/2651197650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f90740855e0 con 0x7f907c0ccde0 2024-04-25T08:48:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.876+0000 7f907be00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f905c066f60 0x7f905c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.876+0000 7f907be00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f905c066f60 0x7f905c069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f907c09a2a0 tx=0x7f906c007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:24.880+0000 7f9073e00640 1 -- 172.21.15.46:0/2651197650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f907c155770 con 0x7f907c0ccde0 2024-04-25T08:48:25.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.044+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f907c0cedf0 con 0x7f907c0ccde0 2024-04-25T08:48:25.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.044+0000 7f9073e00640 1 -- 172.21.15.46:0/2651197650 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13117 (secure 0 0 0) 0x7f9074056160 con 0x7f907c0ccde0 2024-04-25T08:48:25.039 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:25.039 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":86,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:48:24.302315+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f905c066f60 msgr2=0x7f905c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f905c066f60 0x7f905c069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f907c09a2a0 tx=0x7f906c007e40 comp rx=0 tx=0).stop 2024-04-25T08:48:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 msgr2=0x7f907c15e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 0x7f907c15e040 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f9074007e40 tx=0x7f907400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 shutdown_connections 2024-04-25T08:48:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f905c066f60 0x7f905c069420 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f907c0ccde0 0x7f907c15e040 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f907c0b2240 0x7f907c14d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 --2- 172.21.15.46:0/2651197650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f907c099a00 0x7f907c153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 >> 172.21.15.46:0/2651197650 conn(0x7f907c0b3120 msgr2=0x7f907c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 shutdown_connections 2024-04-25T08:48:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.048+0000 7f9082c00640 1 -- 172.21.15.46:0/2651197650 wait complete. 2024-04-25T08:48:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:48:25.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:48:25.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9c600640 1 Processor -- start 2024-04-25T08:48:25.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9c600640 1 -- start start 2024-04-25T08:48:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9c600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a94143990 0x7f2a940cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9c600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a9406f150 0x7f2a940b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9c600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9415e440 con 0x7f2a941440d0 2024-04-25T08:48:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9c600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9415ed30 con 0x7f2a9406f150 2024-04-25T08:48:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9c600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9415e6e0 con 0x7f2a94143990 2024-04-25T08:48:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a9406f150 0x7f2a940b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a9406f150 0x7f2a940b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35372/0 (socket says 172.21.15.46:35372) 2024-04-25T08:48:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a99c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a94143990 0x7f2a940cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9a600640 1 -- 172.21.15.46:0/2971744089 learned_addr learned my addr 172.21.15.46:0/2971744089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9a600640 1 -- 172.21.15.46:0/2971744089 >> v2:172.21.15.82:3300/0 conn(0x7f2a94143990 msgr2=0x7f2a940cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9a600640 1 --2- 172.21.15.46:0/2971744089 >> v2:172.21.15.82:3300/0 conn(0x7f2a94143990 0x7f2a940cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9a600640 1 -- 172.21.15.46:0/2971744089 >> v1:172.21.15.82:6789/0 conn(0x7f2a941440d0 legacy=0x7f2a9406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.212+0000 7f2a9a600640 1 -- 172.21.15.46:0/2971744089 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9415e9c0 con 0x7f2a9406f150 2024-04-25T08:48:25.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a9a600640 1 --2- 172.21.15.46:0/2971744089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 0x7f2a940b2720 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f2a9406b9d0 tx=0x7f2a8c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709bc14f40f34b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a93e00640 1 -- 172.21.15.46:0/2971744089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0a56c0 con 0x7f2a9406f150 2024-04-25T08:48:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a93e00640 1 -- 172.21.15.46:0/2971744089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c0a5860 con 0x7f2a9406f150 2024-04-25T08:48:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a93e00640 1 -- 172.21.15.46:0/2971744089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c0a5c30 con 0x7f2a9406f150 2024-04-25T08:48:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a9c600640 1 -- 172.21.15.46:0/2971744089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 msgr2=0x7f2a940b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a9c600640 1 --2- 172.21.15.46:0/2971744089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 0x7f2a940b2720 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f2a9406b9d0 tx=0x7f2a8c01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a9c600640 1 -- 172.21.15.46:0/2971744089 shutdown_connections 2024-04-25T08:48:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a9c600640 1 --2- 172.21.15.46:0/2971744089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 0x7f2a940b2720 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a9c600640 1 --2- 172.21.15.46:0/2971744089 >> v2:172.21.15.82:3300/0 conn(0x7f2a94143990 0x7f2a940cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a9c600640 1 -- 172.21.15.46:0/2971744089 >> 172.21.15.46:0/2971744089 conn(0x7f2a9406ac10 msgr2=0x7f2a9406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:25.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a9c600640 1 -- 172.21.15.46:0/2971744089 shutdown_connections 2024-04-25T08:48:25.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.216+0000 7f2a9c600640 1 -- 172.21.15.46:0/2971744089 wait complete. 2024-04-25T08:48:25.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 Processor -- start 2024-04-25T08:48:25.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 -- start start 2024-04-25T08:48:25.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 0x7f2a9415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a94143990 0x7f2a9415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a941440d0 0x7f2a94131930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940b35f0 con 0x7f2a9406f150 2024-04-25T08:48:25.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940b3470 con 0x7f2a94143990 2024-04-25T08:48:25.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940b3770 con 0x7f2a941440d0 2024-04-25T08:48:25.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 0x7f2a9415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 0x7f2a9415dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35382/0 (socket says 172.21.15.46:35382) 2024-04-25T08:48:25.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99c00640 1 -- 172.21.15.46:0/1680978547 learned_addr learned my addr 172.21.15.46:0/1680978547 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99200640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a94143990 0x7f2a9415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99c00640 1 -- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a941440d0 msgr2=0x7f2a94131930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99c00640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a941440d0 0x7f2a94131930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99c00640 1 -- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a94143990 msgr2=0x7f2a9415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99c00640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a94143990 0x7f2a9415c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99c00640 1 -- 172.21.15.46:0/1680978547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9415ca10 con 0x7f2a9406f150 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99200640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a94143990 0x7f2a9415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a99c00640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 0x7f2a9415dd50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800e930 tx=0x7f2a8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a91600640 1 -- 172.21.15.46:0/1680978547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88017070 con 0x7f2a9406f150 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a91600640 1 -- 172.21.15.46:0/1680978547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8800ca00 con 0x7f2a9406f150 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a91600640 1 -- 172.21.15.46:0/1680978547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800cdb0 con 0x7f2a9406f150 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9415ccb0 con 0x7f2a9406f150 2024-04-25T08:48:25.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.220+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9415d270 con 0x7f2a9406f150 2024-04-25T08:48:25.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.224+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a94069010 con 0x7f2a9406f150 2024-04-25T08:48:25.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.224+0000 7f2a91600640 1 -- 172.21.15.46:0/1680978547 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a8801a070 con 0x7f2a9406f150 2024-04-25T08:48:25.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.224+0000 7f2a91600640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2a74066f60 0x7f2a74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.224+0000 7f2a91600640 1 -- 172.21.15.46:0/1680978547 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2a88084ea0 con 0x7f2a9406f150 2024-04-25T08:48:25.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.224+0000 7f2a99200640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2a74066f60 0x7f2a74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.224+0000 7f2a99200640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2a74066f60 0x7f2a74069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2a94142960 tx=0x7f2a80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:25.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.228+0000 7f2a91600640 1 -- 172.21.15.46:0/1680978547 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a88056130 con 0x7f2a9406f150 2024-04-25T08:48:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.392+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a9413acb0 con 0x7f2a9406f150 2024-04-25T08:48:25.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.392+0000 7f2a91600640 1 -- 172.21.15.46:0/1680978547 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a88055a30 con 0x7f2a9406f150 2024-04-25T08:48:25.388 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:48:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2a74066f60 msgr2=0x7f2a74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2a74066f60 0x7f2a74069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2a94142960 tx=0x7f2a80009040 comp rx=0 tx=0).stop 2024-04-25T08:48:25.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 msgr2=0x7f2a9415dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 0x7f2a9415dd50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800e930 tx=0x7f2a8800edf0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 shutdown_connections 2024-04-25T08:48:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2a74066f60 0x7f2a74069420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2a941440d0 0x7f2a94131930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a94143990 0x7f2a9415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 --2- 172.21.15.46:0/1680978547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9406f150 0x7f2a9415dd50 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 >> 172.21.15.46:0/1680978547 conn(0x7f2a9406ac10 msgr2=0x7f2a94130610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 shutdown_connections 2024-04-25T08:48:25.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.396+0000 7f2a9c600640 1 -- 172.21.15.46:0/1680978547 wait complete. 2024-04-25T08:48:25.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:48:25.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60fc00640 1 Processor -- start 2024-04-25T08:48:25.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60fc00640 1 -- start start 2024-04-25T08:48:25.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60fc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc608099a00 0x7fc608099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60fc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 0x7fc6080cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60fc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc6080b2240 0x7fc6080b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60fc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60809a320 con 0x7fc608099a00 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60fc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60806bdc0 con 0x7fc6080b2240 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60fc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60806c0a0 con 0x7fc6080ccde0 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc608099a00 0x7fc608099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc608099a00 0x7fc608099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35394/0 (socket says 172.21.15.46:35394) 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60d200640 1 -- 172.21.15.46:0/1047061946 learned_addr learned my addr 172.21.15.46:0/1047061946 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60dc00640 1 --2- 172.21.15.46:0/1047061946 >> v2:172.21.15.82:3300/0 conn(0x7fc6080b2240 0x7fc6080b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc5ffe00640 1 --2- 172.21.15.46:0/1047061946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 0x7fc6080cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60d200640 1 -- 172.21.15.46:0/1047061946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 msgr2=0x7fc6080cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:25.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60d200640 1 --2- 172.21.15.46:0/1047061946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 0x7fc6080cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60d200640 1 -- 172.21.15.46:0/1047061946 >> v2:172.21.15.82:3300/0 conn(0x7fc6080b2240 msgr2=0x7fc6080b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:25.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60d200640 1 --2- 172.21.15.46:0/1047061946 >> v2:172.21.15.82:3300/0 conn(0x7fc6080b2240 0x7fc6080b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.556+0000 7fc60d200640 1 -- 172.21.15.46:0/1047061946 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60806c380 con 0x7fc608099a00 2024-04-25T08:48:25.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60d200640 1 --2- 172.21.15.46:0/1047061946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608099a00 0x7fc608099de0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc6080b3cd0 tx=0x7fc5f801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69284626018b2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:25.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc607e00640 1 -- 172.21.15.46:0/1047061946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80b1070 con 0x7fc608099a00 2024-04-25T08:48:25.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc607e00640 1 -- 172.21.15.46:0/1047061946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f80ac410 con 0x7fc608099a00 2024-04-25T08:48:25.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc607e00640 1 -- 172.21.15.46:0/1047061946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80ac7e0 con 0x7fc608099a00 2024-04-25T08:48:25.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60fc00640 1 -- 172.21.15.46:0/1047061946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608099a00 msgr2=0x7fc608099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60fc00640 1 --2- 172.21.15.46:0/1047061946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608099a00 0x7fc608099de0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc6080b3cd0 tx=0x7fc5f801c460 comp rx=0 tx=0).stop 2024-04-25T08:48:25.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60fc00640 1 -- 172.21.15.46:0/1047061946 shutdown_connections 2024-04-25T08:48:25.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60fc00640 1 --2- 172.21.15.46:0/1047061946 >> v2:172.21.15.82:3300/0 conn(0x7fc6080b2240 0x7fc6080b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60fc00640 1 --2- 172.21.15.46:0/1047061946 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 0x7fc6080cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60fc00640 1 --2- 172.21.15.46:0/1047061946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608099a00 0x7fc608099de0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60fc00640 1 -- 172.21.15.46:0/1047061946 >> 172.21.15.46:0/1047061946 conn(0x7fc6080b3120 msgr2=0x7fc6080b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:25.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60fc00640 1 -- 172.21.15.46:0/1047061946 shutdown_connections 2024-04-25T08:48:25.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.560+0000 7fc60fc00640 1 -- 172.21.15.46:0/1047061946 wait complete. 2024-04-25T08:48:25.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 Processor -- start 2024-04-25T08:48:25.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 -- start start 2024-04-25T08:48:25.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc608099a00 0x7fc6080c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080b2240 0x7fc6080cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 0x7fc60814e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080cd920 con 0x7fc6080b2240 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080cd7a0 con 0x7fc608099a00 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60815dd10 con 0x7fc6080ccde0 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc5ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080b2240 0x7fc6080cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc608099a00 0x7fc6080c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc5ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080b2240 0x7fc6080cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35404/0 (socket says 172.21.15.46:35404) 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc5ffe00640 1 -- 172.21.15.46:0/3193338191 learned_addr learned my addr 172.21.15.46:0/3193338191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60dc00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 0x7fc60814e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc5ffe00640 1 -- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 msgr2=0x7fc60814e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc5ffe00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 0x7fc60814e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc5ffe00640 1 -- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc608099a00 msgr2=0x7fc6080c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc5ffe00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc608099a00 0x7fc6080c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc5ffe00640 1 -- 172.21.15.46:0/3193338191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080cb5d0 con 0x7fc6080b2240 2024-04-25T08:48:25.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60d200640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc608099a00 0x7fc6080c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:48:25.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc5ffe00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080b2240 0x7fc6080cad10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000e870 tx=0x7fc5f000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:25.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc605600640 1 -- 172.21.15.46:0/3193338191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0017070 con 0x7fc6080b2240 2024-04-25T08:48:25.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc605600640 1 -- 172.21.15.46:0/3193338191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f000c940 con 0x7fc6080b2240 2024-04-25T08:48:25.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc605600640 1 -- 172.21.15.46:0/3193338191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f000ccf0 con 0x7fc6080b2240 2024-04-25T08:48:25.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc608153c60 con 0x7fc6080b2240 2024-04-25T08:48:25.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.564+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc608154220 con 0x7fc6080b2240 2024-04-25T08:48:25.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.568+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc608069080 con 0x7fc6080b2240 2024-04-25T08:48:25.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.568+0000 7fc605600640 1 -- 172.21.15.46:0/3193338191 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5f001a070 con 0x7fc6080b2240 2024-04-25T08:48:25.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.568+0000 7fc605600640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc5e4066f60 0x7fc5e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.568+0000 7fc605600640 1 -- 172.21.15.46:0/3193338191 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc5f0084fc0 con 0x7fc6080b2240 2024-04-25T08:48:25.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.568+0000 7fc60d200640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc5e4066f60 0x7fc5e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.568+0000 7fc60d200640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc5e4066f60 0x7fc5e4069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80ae000 tx=0x7fc5f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:25.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.572+0000 7fc605600640 1 -- 172.21.15.46:0/3193338191 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5f0055bd0 con 0x7fc6080b2240 2024-04-25T08:48:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc6080d0f20 con 0x7fc6080b2240 2024-04-25T08:48:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc605600640 1 -- 172.21.15.46:0/3193338191 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v86) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc5f0059c70 con 0x7fc6080b2240 2024-04-25T08:48:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:25.733 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:48:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:48:25.733 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-25T08:48:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-25T08:48:25.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:25.734 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:48:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc5e4066f60 msgr2=0x7fc5e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc5e4066f60 0x7fc5e4069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80ae000 tx=0x7fc5f8008040 comp rx=0 tx=0).stop 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080b2240 msgr2=0x7fc6080cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080b2240 0x7fc6080cad10 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000e870 tx=0x7fc5f000ec70 comp rx=0 tx=0).stop 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 shutdown_connections 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc5e4066f60 0x7fc5e4069420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6080ccde0 0x7fc60814e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6080b2240 0x7fc6080cad10 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 --2- 172.21.15.46:0/3193338191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc608099a00 0x7fc6080c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 >> 172.21.15.46:0/3193338191 conn(0x7fc6080b3120 msgr2=0x7fc6080ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.736+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 shutdown_connections 2024-04-25T08:48:25.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.740+0000 7fc60fc00640 1 -- 172.21.15.46:0/3193338191 wait complete. 2024-04-25T08:48:25.752 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:48:25.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:48:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc9a2e00640 1 Processor -- start 2024-04-25T08:48:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc9a2e00640 1 -- start start 2024-04-25T08:48:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc9a2e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc9a2e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc99c15c680 0x7fc99c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc9a2e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c06d330 con 0x7fc99c071720 2024-04-25T08:48:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc9a2e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99c071b80 con 0x7fc99c15c680 2024-04-25T08:48:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc9a2e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c15cce0 con 0x7fc99c13e6b0 2024-04-25T08:48:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc99be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc99be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c0711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:40414/0 (socket says 172.21.15.46:40414) 2024-04-25T08:48:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc99be00640 1 -- 172.21.15.46:0/804428162 learned_addr learned my addr 172.21.15.46:0/804428162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc9a0e00640 1 --2- 172.21.15.46:0/804428162 >> v2:172.21.15.82:3300/0 conn(0x7fc99c15c680 0x7fc99c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc99be00640 1 -- 172.21.15.46:0/804428162 >> v2:172.21.15.82:3300/0 conn(0x7fc99c15c680 msgr2=0x7fc99c06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc99be00640 1 --2- 172.21.15.46:0/804428162 >> v2:172.21.15.82:3300/0 conn(0x7fc99c15c680 0x7fc99c06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc99be00640 1 -- 172.21.15.46:0/804428162 >> v1:172.21.15.82:6789/0 conn(0x7fc99c071720 legacy=0x7fc99c15bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc99aa00640 1 -- 172.21.15.46:0/804428162 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147564504 0 0) 0x7fc99c06d330 con 0x7fc99c071720 2024-04-25T08:48:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.904+0000 7fc99be00640 1 -- 172.21.15.46:0/804428162 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c06d610 con 0x7fc99c13e6b0 2024-04-25T08:48:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc99be00640 1 --2- 172.21.15.46:0/804428162 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c0711e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ede12744fbbd1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc99aa00640 1 -- 172.21.15.46:0/804428162 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0b0070 con 0x7fc99c13e6b0 2024-04-25T08:48:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc99aa00640 1 -- 172.21.15.46:0/804428162 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c0ab600 con 0x7fc99c13e6b0 2024-04-25T08:48:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc99aa00640 1 -- 172.21.15.46:0/804428162 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0ab9d0 con 0x7fc99c13e6b0 2024-04-25T08:48:25.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc9a2e00640 1 -- 172.21.15.46:0/804428162 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 msgr2=0x7fc99c0711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc9a2e00640 1 --2- 172.21.15.46:0/804428162 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c0711e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c01c440 comp rx=0 tx=0).stop 2024-04-25T08:48:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc9a2e00640 1 -- 172.21.15.46:0/804428162 shutdown_connections 2024-04-25T08:48:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc9a2e00640 1 --2- 172.21.15.46:0/804428162 >> v2:172.21.15.82:3300/0 conn(0x7fc99c15c680 0x7fc99c06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc9a2e00640 1 --2- 172.21.15.46:0/804428162 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c0711e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc9a2e00640 1 -- 172.21.15.46:0/804428162 >> 172.21.15.46:0/804428162 conn(0x7fc99c067050 msgr2=0x7fc99c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc9a2e00640 1 -- 172.21.15.46:0/804428162 shutdown_connections 2024-04-25T08:48:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.908+0000 7fc9a2e00640 1 -- 172.21.15.46:0/804428162 wait complete. 2024-04-25T08:48:25.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 Processor -- start 2024-04-25T08:48:25.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 -- start start 2024-04-25T08:48:25.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c071720 0x7fc99c151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c1f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99c15c680 0x7fc99c138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c15ca90 con 0x7fc99c15c680 2024-04-25T08:48:25.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c168380 con 0x7fc99c071720 2024-04-25T08:48:25.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c168500 con 0x7fc99c13e6b0 2024-04-25T08:48:25.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99c15c680 0x7fc99c138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99c15c680 0x7fc99c138a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35414/0 (socket says 172.21.15.46:35414) 2024-04-25T08:48:25.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a0e00640 1 -- 172.21.15.46:0/2045035524 learned_addr learned my addr 172.21.15.46:0/2045035524 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:25.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc99be00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c071720 0x7fc99c151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc99b400640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c1f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a0e00640 1 -- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 msgr2=0x7fc99c1f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a0e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c1f7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a0e00640 1 -- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c071720 msgr2=0x7fc99c151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:25.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a0e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c071720 0x7fc99c151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:25.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a0e00640 1 -- 172.21.15.46:0/2045035524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c1f8e70 con 0x7fc99c15c680 2024-04-25T08:48:25.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a0e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99c15c680 0x7fc99c138a60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc994009dd0 tx=0x7fc99400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:25.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc993e00640 1 -- 172.21.15.46:0/2045035524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994018020 con 0x7fc99c15c680 2024-04-25T08:48:25.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc993e00640 1 -- 172.21.15.46:0/2045035524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994014480 con 0x7fc99c15c680 2024-04-25T08:48:25.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc993e00640 1 -- 172.21.15.46:0/2045035524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994014870 con 0x7fc99c15c680 2024-04-25T08:48:25.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99c1f88b0 con 0x7fc99c15c680 2024-04-25T08:48:25.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c151930 con 0x7fc99c15c680 2024-04-25T08:48:25.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.912+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99c069080 con 0x7fc99c15c680 2024-04-25T08:48:25.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.916+0000 7fc993e00640 1 -- 172.21.15.46:0/2045035524 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc99401b020 con 0x7fc99c15c680 2024-04-25T08:48:25.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.916+0000 7fc993e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc97c066f90 0x7fc97c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:25.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.916+0000 7fc993e00640 1 -- 172.21.15.46:0/2045035524 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc994085af0 con 0x7fc99c15c680 2024-04-25T08:48:25.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.916+0000 7fc99be00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc97c066f90 0x7fc97c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:25.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.916+0000 7fc99be00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc97c066f90 0x7fc97c069450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0ad000 tx=0x7fc98c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:25.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:25.920+0000 7fc993e00640 1 -- 172.21.15.46:0/2045035524 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc994022030 con 0x7fc99c15c680 2024-04-25T08:48:26.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.080+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc99c0cf240 con 0x7fc99c15c680 2024-04-25T08:48:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc993e00640 1 -- 172.21.15.46:0/2045035524 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v87) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc994010040 con 0x7fc99c15c680 2024-04-25T08:48:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-25T08:48:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc97c066f90 msgr2=0x7fc97c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:26.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc97c066f90 0x7fc97c069450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc98c0ad000 tx=0x7fc98c0a6000 comp rx=0 tx=0).stop 2024-04-25T08:48:26.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99c15c680 msgr2=0x7fc99c138a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:26.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99c15c680 0x7fc99c138a60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc994009dd0 tx=0x7fc99400c490 comp rx=0 tx=0).stop 2024-04-25T08:48:26.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 shutdown_connections 2024-04-25T08:48:26.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc97c066f90 0x7fc97c069450 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc99c15c680 0x7fc99c138a60 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc99c13e6b0 0x7fc99c1f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 --2- 172.21.15.46:0/2045035524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c071720 0x7fc99c151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 >> 172.21.15.46:0/2045035524 conn(0x7fc99c067050 msgr2=0x7fc99c0b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.308+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 shutdown_connections 2024-04-25T08:48:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.312+0000 7fc9a2e00640 1 -- 172.21.15.46:0/2045035524 wait complete. 2024-04-25T08:48:26.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:48:26.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.480+0000 7f28dfe00640 1 Processor -- start 2024-04-25T08:48:26.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.480+0000 7f28dfe00640 1 -- start start 2024-04-25T08:48:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28e006e1e0 0x7f28e006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dfe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 0x7f28e015e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dfe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e006eb00 con 0x7f28e015b140 2024-04-25T08:48:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dfe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e013fbb0 con 0x7f28e006e1e0 2024-04-25T08:48:26.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dfe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e013fe90 con 0x7f28e015b5c0 2024-04-25T08:48:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28e006e1e0 0x7f28e006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28e006e1e0 0x7f28e006e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35430/0 (socket says 172.21.15.46:35430) 2024-04-25T08:48:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dea00640 1 -- 172.21.15.46:0/487259373 learned_addr learned my addr 172.21.15.46:0/487259373 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28df400640 1 --2- 172.21.15.46:0/487259373 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 0x7f28e015e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:26.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dea00640 1 -- 172.21.15.46:0/487259373 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 msgr2=0x7f28e015e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dea00640 1 --2- 172.21.15.46:0/487259373 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 0x7f28e015e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dea00640 1 -- 172.21.15.46:0/487259373 >> v1:172.21.15.82:6789/0 conn(0x7f28e015b140 legacy=0x7f28e015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:26.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.484+0000 7f28dea00640 1 -- 172.21.15.46:0/487259373 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0140170 con 0x7f28e006e1e0 2024-04-25T08:48:26.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dea00640 1 --2- 172.21.15.46:0/487259373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 0x7f28e006e5c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009870 tx=0x7f28d001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7889adaa4bac36ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:26.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28de000640 1 -- 172.21.15.46:0/487259373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d00b2070 con 0x7f28e006e1e0 2024-04-25T08:48:26.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28de000640 1 -- 172.21.15.46:0/487259373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d00a9070 con 0x7f28e006e1e0 2024-04-25T08:48:26.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28de000640 1 -- 172.21.15.46:0/487259373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d00ad880 con 0x7f28e006e1e0 2024-04-25T08:48:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 -- 172.21.15.46:0/487259373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 msgr2=0x7f28e006e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 --2- 172.21.15.46:0/487259373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 0x7f28e006e5c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009870 tx=0x7f28d001c560 comp rx=0 tx=0).stop 2024-04-25T08:48:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 -- 172.21.15.46:0/487259373 shutdown_connections 2024-04-25T08:48:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 --2- 172.21.15.46:0/487259373 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 0x7f28e015e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 --2- 172.21.15.46:0/487259373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 0x7f28e006e5c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 -- 172.21.15.46:0/487259373 >> 172.21.15.46:0/487259373 conn(0x7f28e0066fa0 msgr2=0x7f28e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 -- 172.21.15.46:0/487259373 shutdown_connections 2024-04-25T08:48:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 -- 172.21.15.46:0/487259373 wait complete. 2024-04-25T08:48:26.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 Processor -- start 2024-04-25T08:48:26.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 -- start start 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 0x7f28e01390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28e015b140 0x7f28e0157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 0x7f28e0155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e013d5c0 con 0x7f28e006e1e0 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e013d740 con 0x7f28e015b140 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dfe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01395f0 con 0x7f28e015b5c0 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 0x7f28e01390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 0x7f28e01390b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35446/0 (socket says 172.21.15.46:35446) 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.488+0000 7f28dea00640 1 -- 172.21.15.46:0/3712405112 learned_addr learned my addr 172.21.15.46:0/3712405112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28d6000640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28e015b140 0x7f28e0157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28df400640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 0x7f28e0155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28dea00640 1 -- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 msgr2=0x7f28e0155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28dea00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 0x7f28e0155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28dea00640 1 -- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28e015b140 msgr2=0x7f28e0157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28dea00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28e015b140 0x7f28e0157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28dea00640 1 -- 172.21.15.46:0/3712405112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0153030 con 0x7f28e006e1e0 2024-04-25T08:48:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28df400640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 0x7f28e0155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28d6000640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28e015b140 0x7f28e0157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:26.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28dea00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 0x7f28e01390b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f28d00b0040 tx=0x7f28d001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28d7e00640 1 -- 172.21.15.46:0/3712405112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d00b2040 con 0x7f28e006e1e0 2024-04-25T08:48:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28d7e00640 1 -- 172.21.15.46:0/3712405112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d00be070 con 0x7f28e006e1e0 2024-04-25T08:48:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e0156950 con 0x7f28e006e1e0 2024-04-25T08:48:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28d7e00640 1 -- 172.21.15.46:0/3712405112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d00b8420 con 0x7f28e006e1e0 2024-04-25T08:48:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e01535d0 con 0x7f28e006e1e0 2024-04-25T08:48:26.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e0069010 con 0x7f28e006e1e0 2024-04-25T08:48:26.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.492+0000 7f28d7e00640 1 -- 172.21.15.46:0/3712405112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28d00ab070 con 0x7f28e006e1e0 2024-04-25T08:48:26.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.496+0000 7f28d7e00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28bc066f60 0x7f28bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:26.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.496+0000 7f28d7e00640 1 -- 172.21.15.46:0/3712405112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f28d011fb70 con 0x7f28e006e1e0 2024-04-25T08:48:26.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.496+0000 7f28d6000640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28bc066f60 0x7f28bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:26.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.496+0000 7f28d6000640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28bc066f60 0x7f28bc069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f28e013c480 tx=0x7f28c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:26.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.500+0000 7f28d7e00640 1 -- 172.21.15.46:0/3712405112 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28d0016150 con 0x7f28e006e1e0 2024-04-25T08:48:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:26.664+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f28e0151350 con 0x7f28e006e1e0 2024-04-25T08:48:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.312+0000 7f28d7e00640 1 -- 172.21.15.46:0/3712405112 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) v1 ==== 170+0+0 (secure 0 0 0) 0x7f28d00f0bb0 con 0x7f28e006e1e0 2024-04-25T08:48:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-25T08:48:27.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28bc066f60 msgr2=0x7f28bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:27.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28bc066f60 0x7f28bc069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f28e013c480 tx=0x7f28c8009040 comp rx=0 tx=0).stop 2024-04-25T08:48:27.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 msgr2=0x7f28e01390b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:27.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 0x7f28e01390b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f28d00b0040 tx=0x7f28d001ada0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 shutdown_connections 2024-04-25T08:48:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f28bc066f60 0x7f28bc069420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f28e015b5c0 0x7f28e0155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28e015b140 0x7f28e0157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 --2- 172.21.15.46:0/3712405112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e006e1e0 0x7f28e01390b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 >> 172.21.15.46:0/3712405112 conn(0x7f28e0066fa0 msgr2=0x7f28e013ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 shutdown_connections 2024-04-25T08:48:27.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.316+0000 7f28dfe00640 1 -- 172.21.15.46:0/3712405112 wait complete. 2024-04-25T08:48:27.355 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:48:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.504+0000 7f0bc2c00640 1 Processor -- start 2024-04-25T08:48:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.504+0000 7f0bc2c00640 1 -- start start 2024-04-25T08:48:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc2c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0bbc1440d0 0x7f0bbc0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc2c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0bbc06f150 0x7f0bbc0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc2c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bbc15e440 con 0x7f0bbc143990 2024-04-25T08:48:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc2c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bbc15ed30 con 0x7f0bbc06f150 2024-04-25T08:48:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc2c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bbc15e6e0 con 0x7f0bbc1440d0 2024-04-25T08:48:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0bbc06f150 0x7f0bbc0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0bbc06f150 0x7f0bbc0b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35460/0 (socket says 172.21.15.46:35460) 2024-04-25T08:48:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc0c00640 1 -- 172.21.15.46:0/2188753068 learned_addr learned my addr 172.21.15.46:0/2188753068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bb3e00640 1 --2- 172.21.15.46:0/2188753068 >> v2:172.21.15.82:3300/0 conn(0x7f0bbc1440d0 0x7f0bbc0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc0c00640 1 -- 172.21.15.46:0/2188753068 >> v2:172.21.15.82:3300/0 conn(0x7f0bbc1440d0 msgr2=0x7f0bbc0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc0c00640 1 --2- 172.21.15.46:0/2188753068 >> v2:172.21.15.82:3300/0 conn(0x7f0bbc1440d0 0x7f0bbc0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc0c00640 1 -- 172.21.15.46:0/2188753068 >> v1:172.21.15.82:6789/0 conn(0x7f0bbc143990 legacy=0x7f0bbc06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.508+0000 7f0bc0c00640 1 -- 172.21.15.46:0/2188753068 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc15e9c0 con 0x7f0bbc06f150 2024-04-25T08:48:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc0c00640 1 --2- 172.21.15.46:0/2188753068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 0x7f0bbc0b2720 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc070c80 tx=0x7f0bb401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23feb5ddbee3944 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bbb400640 1 -- 172.21.15.46:0/2188753068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40a56c0 con 0x7f0bbc06f150 2024-04-25T08:48:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bbb400640 1 -- 172.21.15.46:0/2188753068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb40a5860 con 0x7f0bbc06f150 2024-04-25T08:48:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bbb400640 1 -- 172.21.15.46:0/2188753068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb40a5c30 con 0x7f0bbc06f150 2024-04-25T08:48:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 -- 172.21.15.46:0/2188753068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 msgr2=0x7f0bbc0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 --2- 172.21.15.46:0/2188753068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 0x7f0bbc0b2720 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc070c80 tx=0x7f0bb401c9f0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 -- 172.21.15.46:0/2188753068 shutdown_connections 2024-04-25T08:48:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 --2- 172.21.15.46:0/2188753068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 0x7f0bbc0b2720 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 --2- 172.21.15.46:0/2188753068 >> v2:172.21.15.82:3300/0 conn(0x7f0bbc1440d0 0x7f0bbc0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 -- 172.21.15.46:0/2188753068 >> 172.21.15.46:0/2188753068 conn(0x7f0bbc06ac10 msgr2=0x7f0bbc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 -- 172.21.15.46:0/2188753068 shutdown_connections 2024-04-25T08:48:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 -- 172.21.15.46:0/2188753068 wait complete. 2024-04-25T08:48:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 Processor -- start 2024-04-25T08:48:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.512+0000 7f0bc2c00640 1 -- start start 2024-04-25T08:48:27.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc2c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 0x7f0bbc15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:27.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc2c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bbc143990 0x7f0bbc132a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:27.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc2c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bbc1440d0 0x7f0bbc15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:27.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc2c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0b35f0 con 0x7f0bbc06f150 2024-04-25T08:48:27.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc2c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0b3470 con 0x7f0bbc143990 2024-04-25T08:48:27.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc2c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0b3770 con 0x7f0bbc1440d0 2024-04-25T08:48:27.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bbbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 0x7f0bbc15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:27.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bbbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 0x7f0bbc15a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35468/0 (socket says 172.21.15.46:35468) 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bbbe00640 1 -- 172.21.15.46:0/4047344186 learned_addr learned my addr 172.21.15.46:0/4047344186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc0c00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bbc1440d0 0x7f0bbc15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bb3e00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bbc143990 0x7f0bbc132a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bbbe00640 1 -- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bbc1440d0 msgr2=0x7f0bbc15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bbbe00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bbc1440d0 0x7f0bbc15bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bbbe00640 1 -- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bbc143990 msgr2=0x7f0bbc132a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bbbe00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bbc143990 0x7f0bbc132a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bbbe00640 1 -- 172.21.15.46:0/4047344186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc13b4d0 con 0x7f0bbc06f150 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc0c00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bbc1440d0 0x7f0bbc15bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bbbe00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 0x7f0bbc15a010 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c450 tx=0x7f0bac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bb8c00640 1 -- 172.21.15.46:0/4047344186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac016020 con 0x7f0bbc06f150 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bb8c00640 1 -- 172.21.15.46:0/4047344186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac0052d0 con 0x7f0bbc06f150 2024-04-25T08:48:27.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bb8c00640 1 -- 172.21.15.46:0/4047344186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac007260 con 0x7f0bbc06f150 2024-04-25T08:48:27.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc13b7b0 con 0x7f0bbc06f150 2024-04-25T08:48:27.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bbc15a550 con 0x7f0bbc06f150 2024-04-25T08:48:27.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bbc069010 con 0x7f0bbc06f150 2024-04-25T08:48:27.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.516+0000 7f0bb8c00640 1 -- 172.21.15.46:0/4047344186 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bac007400 con 0x7f0bbc06f150 2024-04-25T08:48:27.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.520+0000 7f0bb8c00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b98066f60 0x7f0b98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:27.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.520+0000 7f0bb8c00640 1 -- 172.21.15.46:0/4047344186 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f0bac084ae0 con 0x7f0bbc06f150 2024-04-25T08:48:27.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.520+0000 7f0bb3e00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b98066f60 0x7f0b98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:27.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.520+0000 7f0bb3e00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b98066f60 0x7f0b98069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ae690 tx=0x7f0ba4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:27.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.524+0000 7f0bb8c00640 1 -- 172.21.15.46:0/4047344186 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bac020020 con 0x7f0bbc06f150 2024-04-25T08:48:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:27.724+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0bbc06b200 con 0x7f0bbc06f150 2024-04-25T08:48:28.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.316+0000 7f0bb8c00640 1 -- 172.21.15.46:0/4047344186 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0bac005480 con 0x7f0bbc06f150 2024-04-25T08:48:28.311 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T08:48:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.324+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b98066f60 msgr2=0x7f0b98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.324+0000 7f0bc2c00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b98066f60 0x7f0b98069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ae690 tx=0x7f0ba4008040 comp rx=0 tx=0).stop 2024-04-25T08:48:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.324+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 msgr2=0x7f0bbc15a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.324+0000 7f0bc2c00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 0x7f0bbc15a010 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c450 tx=0x7f0bac00c910 comp rx=0 tx=0).stop 2024-04-25T08:48:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.328+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 shutdown_connections 2024-04-25T08:48:28.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.328+0000 7f0bc2c00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b98066f60 0x7f0b98069420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.328+0000 7f0bc2c00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0bbc1440d0 0x7f0bbc15bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.328+0000 7f0bc2c00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bbc143990 0x7f0bbc132a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.328+0000 7f0bc2c00640 1 --2- 172.21.15.46:0/4047344186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bbc06f150 0x7f0bbc15a010 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.328+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 >> 172.21.15.46:0/4047344186 conn(0x7f0bbc06ac10 msgr2=0x7f0bbc130610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.328+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 shutdown_connections 2024-04-25T08:48:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.328+0000 7f0bc2c00640 1 -- 172.21.15.46:0/4047344186 wait complete. 2024-04-25T08:48:28.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c1e600640 1 Processor -- start 2024-04-25T08:48:28.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c1e600640 1 -- start start 2024-04-25T08:48:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c1e600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2c1815ef20 0x7f2c180cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c1e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c180cd6d0 0x7f2c1809a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c1e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1815f300 con 0x7f2c1809ab50 2024-04-25T08:48:28.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c1e600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c180cddb0 con 0x7f2c1815ef20 2024-04-25T08:48:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c1e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180cc3d0 con 0x7f2c180cd6d0 2024-04-25T08:48:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c17e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2c1809ab50 0x7f2c18099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50810/0 (socket says 172.21.15.46:50810) 2024-04-25T08:48:28.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c17e00640 1 -- 172.21.15.46:0/2312701475 learned_addr learned my addr 172.21.15.46:0/2312701475 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c0ea00640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c180cd6d0 0x7f2c1809a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c17400640 1 --2- 172.21.15.46:0/2312701475 >> v2:172.21.15.82:3300/0 conn(0x7f2c1815ef20 0x7f2c180cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182556628 0 0) 0x7f2c1815f300 con 0x7f2c1809ab50 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bfc003530 con 0x7f2c1809ab50 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3067163816 0 0) 0x7f2bfc003530 con 0x7f2c1809ab50 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c180cd6d0 msgr2=0x7f2c1809a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2c180cd6d0 0x7f2c1809a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 >> v2:172.21.15.82:3300/0 conn(0x7f2c1815ef20 msgr2=0x7f2c180cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 --2- 172.21.15.46:0/2312701475 >> v2:172.21.15.82:3300/0 conn(0x7f2c1815ef20 0x7f2c180cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c180cc6b0 con 0x7f2c1809ab50 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f2c10001680 con 0x7f2c1809ab50 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 >> v1:172.21.15.46:6789/0 conn(0x7f2c1809ab50 legacy=0x7f2c18099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.520+0000 7f2c16a00640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc0039c0 0x7f2bfc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c16a00640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bfc004810 0x7f2bfc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c16a00640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bfc005190 0x7f2bfc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180cddb0 con 0x7f2bfc0039c0 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180cc3d0 con 0x7f2bfc005190 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc004610 con 0x7f2bfc004810 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c17400640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc0039c0 0x7f2bfc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c17e00640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bfc005190 0x7f2bfc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c0ea00640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bfc004810 0x7f2bfc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c17400640 1 -- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bfc004810 msgr2=0x7f2bfc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c17400640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bfc004810 0x7f2bfc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c17400640 1 -- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bfc005190 msgr2=0x7f2bfc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c17400640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bfc005190 0x7f2bfc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c17400640 1 -- 172.21.15.46:0/2312701475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc003530 con 0x7f2bfc0039c0 2024-04-25T08:48:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c17400640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc0039c0 0x7f2bfc003de0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009ae0 tx=0x7f2c0801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e488e1bc1b27d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c080b2070 con 0x7f2bfc0039c0 2024-04-25T08:48:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c080a5650 con 0x7f2bfc0039c0 2024-04-25T08:48:28.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.524+0000 7f2c16a00640 1 -- 172.21.15.46:0/2312701475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c080096b0 con 0x7f2bfc0039c0 2024-04-25T08:48:28.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 -- 172.21.15.46:0/2312701475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc0039c0 msgr2=0x7f2bfc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 --2- 172.21.15.46:0/2312701475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bfc0039c0 0x7f2bfc003de0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009ae0 tx=0x7f2c0801ca80 comp rx=0 tx=0).stop 2024-04-25T08:48:28.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c17e00640 1 -- 172.21.15.46:0/2312701475 reap_dead start 2024-04-25T08:48:28.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 -- 172.21.15.46:0/2312701475 shutdown_connections 2024-04-25T08:48:28.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 -- 172.21.15.46:0/2312701475 >> 172.21.15.46:0/2312701475 conn(0x7f2c180b3120 msgr2=0x7f2c180b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:28.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 -- 172.21.15.46:0/2312701475 shutdown_connections 2024-04-25T08:48:28.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 -- 172.21.15.46:0/2312701475 wait complete. 2024-04-25T08:48:28.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 Processor -- start 2024-04-25T08:48:28.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 -- start start 2024-04-25T08:48:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bfc0039c0 0x7f2c180b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c181f8590 0x7f2c181f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181f8100 0x7f2c18161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180c5b10 con 0x7f2c181f8590 2024-04-25T08:48:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180c5990 con 0x7f2c181f8100 2024-04-25T08:48:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c1e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1809a110 con 0x7f2bfc0039c0 2024-04-25T08:48:28.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c0ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c181f8590 0x7f2c181f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c0ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c181f8590 0x7f2c181f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36758/0 (socket says 172.21.15.46:36758) 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c0ea00640 1 -- 172.21.15.46:0/3488955812 learned_addr learned my addr 172.21.15.46:0/3488955812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c17400640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bfc0039c0 0x7f2c180b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c17e00640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181f8100 0x7f2c18161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c0ea00640 1 -- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bfc0039c0 msgr2=0x7f2c180b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c0ea00640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bfc0039c0 0x7f2c180b63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c0ea00640 1 -- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181f8100 msgr2=0x7f2c18161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c0ea00640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181f8100 0x7f2c18161830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c0ea00640 1 -- 172.21.15.46:0/3488955812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c180b5320 con 0x7f2c181f8590 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.528+0000 7f2c17e00640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181f8100 0x7f2c18161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c0ea00640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c181f8590 0x7f2c181f7bc0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009f90 tx=0x7f2c0000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c0fe00640 1 -- 172.21.15.46:0/3488955812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000fea0 con 0x7f2c181f8590 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c0fe00640 1 -- 172.21.15.46:0/3488955812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c00015630 con 0x7f2c181f8590 2024-04-25T08:48:28.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c0fe00640 1 -- 172.21.15.46:0/3488955812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c00015a20 con 0x7f2c181f8590 2024-04-25T08:48:28.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c180b5600 con 0x7f2c181f8590 2024-04-25T08:48:28.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c180b5bc0 con 0x7f2c181f8590 2024-04-25T08:48:28.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c18069080 con 0x7f2c181f8590 2024-04-25T08:48:28.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c0fe00640 1 -- 172.21.15.46:0/3488955812 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c00015bc0 con 0x7f2c181f8590 2024-04-25T08:48:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c0fe00640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2bf4066f60 0x7f2bf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c0fe00640 1 -- 172.21.15.46:0/3488955812 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2c00086320 con 0x7f2c181f8590 2024-04-25T08:48:28.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.532+0000 7f2c17400640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2bf4066f60 0x7f2bf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.536+0000 7f2c17400640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2bf4066f60 0x7f2bf4069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2c080af000 tx=0x7f2c080046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:28.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.536+0000 7f2c0fe00640 1 -- 172.21.15.46:0/3488955812 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c00019030 con 0x7f2c181f8590 2024-04-25T08:48:28.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.700+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c180b71b0 con 0x7f2c181f8590 2024-04-25T08:48:28.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.700+0000 7f2c0fe00640 1 -- 172.21.15.46:0/3488955812 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16850 (secure 0 0 0) 0x7f2c00010040 con 0x7f2c181f8590 2024-04-25T08:48:28.695 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:28.695 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:48:27.728360+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:48:26.132790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:48:26.714570+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:28.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2bf4066f60 msgr2=0x7f2bf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2bf4066f60 0x7f2bf4069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2c080af000 tx=0x7f2c080046d0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c181f8590 msgr2=0x7f2c181f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c181f8590 0x7f2c181f7bc0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009f90 tx=0x7f2c0000f580 comp rx=0 tx=0).stop 2024-04-25T08:48:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 shutdown_connections 2024-04-25T08:48:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2bf4066f60 0x7f2bf4069420 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c181f8100 0x7f2c18161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c181f8590 0x7f2c181f7bc0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 --2- 172.21.15.46:0/3488955812 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bfc0039c0 0x7f2c180b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 >> 172.21.15.46:0/3488955812 conn(0x7f2c180b3120 msgr2=0x7f2c180c75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 shutdown_connections 2024-04-25T08:48:28.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.704+0000 7f2c1e600640 1 -- 172.21.15.46:0/3488955812 wait complete. 2024-04-25T08:48:28.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:28.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f7985400640 1 Processor -- start 2024-04-25T08:48:28.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f7985400640 1 -- start start 2024-04-25T08:48:28.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f7985400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f79801611d0 0x7f798006c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f7985400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79800cfba0 con 0x7f798006cc70 2024-04-25T08:48:28.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f7985400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79800cfe80 con 0x7f79800d0960 2024-04-25T08:48:28.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f7985400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79800d0160 con 0x7f79801611d0 2024-04-25T08:48:28.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f7975e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f798006cc70 0x7f79800cbd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50818/0 (socket says 172.21.15.46:50818) 2024-04-25T08:48:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f7975e00640 1 -- 172.21.15.46:0/1170898616 learned_addr learned my addr 172.21.15.46:0/1170898616 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f797e800640 1 --2- 172.21.15.46:0/1170898616 >> v2:172.21.15.82:3300/0 conn(0x7f79801611d0 0x7f798006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f797de00640 1 -- 172.21.15.46:0/1170898616 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631457540 0 0) 0x7f79800cfba0 con 0x7f798006cc70 2024-04-25T08:48:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f797de00640 1 -- 172.21.15.46:0/1170898616 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7964003530 con 0x7f798006cc70 2024-04-25T08:48:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f797de00640 1 -- 172.21.15.46:0/1170898616 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392768696 0 0) 0x7f79800cfe80 con 0x7f79800d0960 2024-04-25T08:48:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f797de00640 1 -- 172.21.15.46:0/1170898616 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79800cfba0 con 0x7f79800d0960 2024-04-25T08:48:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f797e800640 1 -- 172.21.15.46:0/1170898616 >> v1:172.21.15.82:6789/0 conn(0x7f79800d0960 legacy=0x7f7980099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f797e800640 1 -- 172.21.15.46:0/1170898616 >> v1:172.21.15.46:6789/0 conn(0x7f798006cc70 legacy=0x7f79800cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.864+0000 7f797e800640 1 -- 172.21.15.46:0/1170898616 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79800d0440 con 0x7f79801611d0 2024-04-25T08:48:28.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f797e800640 1 --2- 172.21.15.46:0/1170898616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79801611d0 0x7f798006c730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f79700097c0 tx=0x7f797001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9e55eb5fbc73e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f797de00640 1 -- 172.21.15.46:0/1170898616 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79700b1070 con 0x7f79801611d0 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f797de00640 1 -- 172.21.15.46:0/1170898616 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79700ac410 con 0x7f79801611d0 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f797de00640 1 -- 172.21.15.46:0/1170898616 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79700ac7e0 con 0x7f79801611d0 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f7985400640 1 -- 172.21.15.46:0/1170898616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79801611d0 msgr2=0x7f798006c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f7985400640 1 --2- 172.21.15.46:0/1170898616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79801611d0 0x7f798006c730 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f79700097c0 tx=0x7f797001c410 comp rx=0 tx=0).stop 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f7985400640 1 -- 172.21.15.46:0/1170898616 shutdown_connections 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f7985400640 1 --2- 172.21.15.46:0/1170898616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79801611d0 0x7f798006c730 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f7985400640 1 -- 172.21.15.46:0/1170898616 >> 172.21.15.46:0/1170898616 conn(0x7f79800d1700 msgr2=0x7f79800d1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f7985400640 1 -- 172.21.15.46:0/1170898616 shutdown_connections 2024-04-25T08:48:28.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.868+0000 7f7985400640 1 -- 172.21.15.46:0/1170898616 wait complete. 2024-04-25T08:48:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 Processor -- start 2024-04-25T08:48:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 -- start start 2024-04-25T08:48:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798006cc70 0x7f798015f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0960 0x7f7980155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79801611d0 0x7f7980070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798009a7d0 con 0x7f79800d0960 2024-04-25T08:48:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798009a650 con 0x7f798006cc70 2024-04-25T08:48:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798009a110 con 0x7f79801611d0 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7975e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0960 0x7f7980155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7975e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0960 0x7f7980155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36762/0 (socket says 172.21.15.46:36762) 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7975e00640 1 -- 172.21.15.46:0/4016906586 learned_addr learned my addr 172.21.15.46:0/4016906586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f797e800640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798006cc70 0x7f798015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f797f200640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79801611d0 0x7f7980070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7975e00640 1 -- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79801611d0 msgr2=0x7f7980070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7975e00640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79801611d0 0x7f7980070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7975e00640 1 -- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798006cc70 msgr2=0x7f798015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7975e00640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798006cc70 0x7f798015f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7975e00640 1 -- 172.21.15.46:0/4016906586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798015fcc0 con 0x7f79800d0960 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7975e00640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0960 0x7f7980155e80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f796800cb50 tx=0x7f796800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7977400640 1 -- 172.21.15.46:0/4016906586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968017070 con 0x7f79800d0960 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7977400640 1 -- 172.21.15.46:0/4016906586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796800ad40 con 0x7f79800d0960 2024-04-25T08:48:28.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7977400640 1 -- 172.21.15.46:0/4016906586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968012620 con 0x7f79800d0960 2024-04-25T08:48:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79800b4f10 con 0x7f79800d0960 2024-04-25T08:48:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.872+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79800b54d0 con 0x7f79800d0960 2024-04-25T08:48:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.876+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798009ae00 con 0x7f79800d0960 2024-04-25T08:48:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.876+0000 7f7977400640 1 -- 172.21.15.46:0/4016906586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f796801a070 con 0x7f79800d0960 2024-04-25T08:48:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.876+0000 7f7977400640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.876+0000 7f7977400640 1 -- 172.21.15.46:0/4016906586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7968084ed0 con 0x7f79800d0960 2024-04-25T08:48:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.876+0000 7f797e800640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:28.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.876+0000 7f797e800640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f79700ae000 tx=0x7f7970008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:28.880+0000 7f7977400640 1 -- 172.21.15.46:0/4016906586 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7968055700 con 0x7f79800d0960 2024-04-25T08:48:29.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.040+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79800d2530 con 0x7f79800d0960 2024-04-25T08:48:29.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7977400640 1 -- 172.21.15.46:0/4016906586 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16850 (secure 0 0 0) 0x7f79680597a0 con 0x7f79800d0960 2024-04-25T08:48:29.037 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:29.037 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:48:27.728360+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:48:26.132790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:48:26.714570+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:29.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 msgr2=0x7f795c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:29.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f79700ae000 tx=0x7f7970008040 comp rx=0 tx=0).stop 2024-04-25T08:48:29.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0960 msgr2=0x7f7980155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:29.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0960 0x7f7980155e80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f796800cb50 tx=0x7f796800a430 comp rx=0 tx=0).stop 2024-04-25T08:48:29.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 shutdown_connections 2024-04-25T08:48:29.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f795c066f60 0x7f795c069420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f79801611d0 0x7f7980070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79800d0960 0x7f7980155e80 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 --2- 172.21.15.46:0/4016906586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f798006cc70 0x7f798015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 >> 172.21.15.46:0/4016906586 conn(0x7f79800d1700 msgr2=0x7f79800cc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:29.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 shutdown_connections 2024-04-25T08:48:29.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.044+0000 7f7985400640 1 -- 172.21.15.46:0/4016906586 wait complete. 2024-04-25T08:48:29.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.208+0000 7f7e55600640 1 Processor -- start 2024-04-25T08:48:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.208+0000 7f7e55600640 1 -- start start 2024-04-25T08:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.208+0000 7f7e55600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e5006bf30 0x7f7e500cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.208+0000 7f7e55600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5009acb0 con 0x7f7e5009a410 2024-04-25T08:48:29.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.208+0000 7f7e55600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e500d0aa0 con 0x7f7e5006c730 2024-04-25T08:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.208+0000 7f7e55600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e500b3dc0 con 0x7f7e5006bf30 2024-04-25T08:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e47e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7e5009a410 0x7f7e500d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50826/0 (socket says 172.21.15.46:50826) 2024-04-25T08:48:29.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e47e00640 1 -- 172.21.15.46:0/2471031312 learned_addr learned my addr 172.21.15.46:0/2471031312 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e54c00640 1 --2- 172.21.15.46:0/2471031312 >> v2:172.21.15.82:3300/0 conn(0x7f7e5006bf30 0x7f7e500cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703327340 0 0) 0x7f7e5009acb0 con 0x7f7e5009a410 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e34003530 con 0x7f7e5009a410 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 114791791 0 0) 0x7f7e34003530 con 0x7f7e5009a410 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 >> v2:172.21.15.82:3300/0 conn(0x7f7e5006bf30 msgr2=0x7f7e500cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 --2- 172.21.15.46:0/2471031312 >> v2:172.21.15.82:3300/0 conn(0x7f7e5006bf30 0x7f7e500cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 >> v1:172.21.15.82:6789/0 conn(0x7f7e5006c730 legacy=0x7f7e5006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5009a9d0 con 0x7f7e5009a410 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f7e38001680 con 0x7f7e5009a410 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 >> v1:172.21.15.46:6789/0 conn(0x7f7e5009a410 legacy=0x7f7e500d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e340039c0 0x7f7e34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:29.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e34004810 0x7f7e34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e340051d0 0x7f7e3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3400c040 con 0x7f7e340051d0 2024-04-25T08:48:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3400c1f0 con 0x7f7e340039c0 2024-04-25T08:48:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3400c3f0 con 0x7f7e34004810 2024-04-25T08:48:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e54c00640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e340051d0 0x7f7e3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e4fe00640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e340039c0 0x7f7e34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e47e00640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e34004810 0x7f7e34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:29.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e54c00640 1 -- 172.21.15.46:0/2471031312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e34004810 msgr2=0x7f7e34004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e54c00640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e34004810 0x7f7e34004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e54c00640 1 -- 172.21.15.46:0/2471031312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e340039c0 msgr2=0x7f7e34003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e54c00640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e340039c0 0x7f7e34003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.212+0000 7f7e54c00640 1 -- 172.21.15.46:0/2471031312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34003530 con 0x7f7e340051d0 2024-04-25T08:48:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e54c00640 1 -- 172.21.15.46:0/2471031312 reap_dead start 2024-04-25T08:48:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e54c00640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e340051d0 0x7f7e3400b9d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009fc0 tx=0x7f7e4801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad074d50c0c7a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48009c10 con 0x7f7e340051d0 2024-04-25T08:48:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e480a5d10 con 0x7f7e340051d0 2024-04-25T08:48:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e4f400640 1 -- 172.21.15.46:0/2471031312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48003be0 con 0x7f7e340051d0 2024-04-25T08:48:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e55600640 1 -- 172.21.15.46:0/2471031312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e340051d0 msgr2=0x7f7e3400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:29.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e55600640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e340051d0 0x7f7e3400b9d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009fc0 tx=0x7f7e4801ccc0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e55600640 1 -- 172.21.15.46:0/2471031312 shutdown_connections 2024-04-25T08:48:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e55600640 1 --2- 172.21.15.46:0/2471031312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e340051d0 0x7f7e3400b9d0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e55600640 1 -- 172.21.15.46:0/2471031312 >> 172.21.15.46:0/2471031312 conn(0x7f7e500b2f50 msgr2=0x7f7e500b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e55600640 1 -- 172.21.15.46:0/2471031312 shutdown_connections 2024-04-25T08:48:29.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e55600640 1 -- 172.21.15.46:0/2471031312 wait complete. 2024-04-25T08:48:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e55600640 1 Processor -- start 2024-04-25T08:48:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.216+0000 7f7e55600640 1 -- start start 2024-04-25T08:48:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e55600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e340051d0 0x7f7e50153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e55600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e500b6840 0x7f7e501e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e55600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e500b6cc0 0x7f7e501e7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e55600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e500cfb90 con 0x7f7e500b6840 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e55600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e500cfe70 con 0x7f7e500b6cc0 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e55600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e500b57a0 con 0x7f7e340051d0 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e500b6840 0x7f7e501e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e500b6840 0x7f7e501e71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36782/0 (socket says 172.21.15.46:36782) 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e47e00640 1 -- 172.21.15.46:0/2565097655 learned_addr learned my addr 172.21.15.46:0/2565097655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e4fe00640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e340051d0 0x7f7e50153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e54c00640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e500b6cc0 0x7f7e501e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e47e00640 1 -- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e340051d0 msgr2=0x7f7e50153550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e47e00640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e340051d0 0x7f7e50153550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e47e00640 1 -- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e500b6cc0 msgr2=0x7f7e501e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e47e00640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e500b6cc0 0x7f7e501e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e47e00640 1 -- 172.21.15.46:0/2565097655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e500b5a80 con 0x7f7e500b6840 2024-04-25T08:48:29.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e54c00640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e500b6cc0 0x7f7e501e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e47e00640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e500b6840 0x7f7e501e71d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7e38007290 tx=0x7f7e38007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e4cc00640 1 -- 172.21.15.46:0/2565097655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800f940 con 0x7f7e500b6840 2024-04-25T08:48:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e4cc00640 1 -- 172.21.15.46:0/2565097655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e38018070 con 0x7f7e500b6840 2024-04-25T08:48:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e4cc00640 1 -- 172.21.15.46:0/2565097655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3800fd40 con 0x7f7e500b6840 2024-04-25T08:48:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e500b5d60 con 0x7f7e500b6840 2024-04-25T08:48:29.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e500b6320 con 0x7f7e500b6840 2024-04-25T08:48:29.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e50069010 con 0x7f7e500b6840 2024-04-25T08:48:29.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.220+0000 7f7e4cc00640 1 -- 172.21.15.46:0/2565097655 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e3801d460 con 0x7f7e500b6840 2024-04-25T08:48:29.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.224+0000 7f7e4cc00640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e2c066f90 0x7f7e2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:29.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.224+0000 7f7e4cc00640 1 -- 172.21.15.46:0/2565097655 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e380851d0 con 0x7f7e500b6840 2024-04-25T08:48:29.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.224+0000 7f7e4fe00640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e2c066f90 0x7f7e2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:29.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.224+0000 7f7e4fe00640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e2c066f90 0x7f7e2c069450 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e40000c10 tx=0x7f7e40009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:29.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.228+0000 7f7e4cc00640 1 -- 172.21.15.46:0/2565097655 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e38056050 con 0x7f7e500b6840 2024-04-25T08:48:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:29.428+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7e500b64f0 con 0x7f7e500b6840 2024-04-25T08:48:30.286 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:48:30.292+0000 7f0b53000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:48:30.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e4cc00640 1 -- 172.21.15.46:0/2565097655 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7e38055950 con 0x7f7e500b6840 2024-04-25T08:48:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e2c066f90 msgr2=0x7f7e2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e2c066f90 0x7f7e2c069450 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e40000c10 tx=0x7f7e40009040 comp rx=0 tx=0).stop 2024-04-25T08:48:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e500b6840 msgr2=0x7f7e501e71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e500b6840 0x7f7e501e71d0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f7e38007290 tx=0x7f7e38007750 comp rx=0 tx=0).stop 2024-04-25T08:48:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 shutdown_connections 2024-04-25T08:48:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7e2c066f90 0x7f7e2c069450 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:30.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e500b6cc0 0x7f7e501e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e500b6840 0x7f7e501e71d0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 --2- 172.21.15.46:0/2565097655 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e340051d0 0x7f7e50153550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 >> 172.21.15.46:0/2565097655 conn(0x7f7e500b2f50 msgr2=0x7f7e5006f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 shutdown_connections 2024-04-25T08:48:30.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.344+0000 7f7e55600640 1 -- 172.21.15.46:0/2565097655 wait complete. 2024-04-25T08:48:30.363 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:48:30.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:48:30.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.516+0000 7f1f27c00640 1 Processor -- start 2024-04-25T08:48:30.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.516+0000 7f1f27c00640 1 -- start start 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f27c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2006dc70 0x7f1f200cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f27c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f200cd930 con 0x7f1f20145310 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f27c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f200cdbd0 con 0x7f1f200cf1b0 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f27c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200cdeb0 con 0x7f1f2006dc70 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f25200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1f20145310 0x7f1f2006e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50842/0 (socket says 172.21.15.46:50842) 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f25200640 1 -- 172.21.15.46:0/3743506389 learned_addr learned my addr 172.21.15.46:0/3743506389 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f17e00640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2006dc70 0x7f1f200cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610954976 0 0) 0x7f1f200cd930 con 0x7f1f20145310 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f04003530 con 0x7f1f20145310 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995665840 0 0) 0x7f1f200cdbd0 con 0x7f1f200cf1b0 2024-04-25T08:48:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f200cd930 con 0x7f1f200cf1b0 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 711161545 0 0) 0x7f1f04003530 con 0x7f1f20145310 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2006dc70 msgr2=0x7f1f200cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2006dc70 0x7f1f200cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 >> v1:172.21.15.82:6789/0 conn(0x7f1f200cf1b0 legacy=0x7f1f2015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200ce190 con 0x7f1f20145310 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f1f10001680 con 0x7f1f20145310 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f17e00640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2006dc70 0x7f1f200cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 >> v1:172.21.15.46:6789/0 conn(0x7f1f20145310 legacy=0x7f1f2006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04003bf0 0x7f1f04003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f04004810 0x7f1f04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f040051d0 0x7f1f0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200cdeb0 con 0x7f1f04003bf0 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0400bf10 con 0x7f1f04004810 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0400c0c0 con 0x7f1f040051d0 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f25200640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04003bf0 0x7f1f04003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f17e00640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f04004810 0x7f1f04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f25c00640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f040051d0 0x7f1f0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f25200640 1 -- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f040051d0 msgr2=0x7f1f0400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f25200640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f040051d0 0x7f1f0400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f25200640 1 -- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f04004810 msgr2=0x7f1f04004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f25200640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f04004810 0x7f1f04004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:30.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.520+0000 7f1f25200640 1 -- 172.21.15.46:0/3743506389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f04003530 con 0x7f1f04003bf0 2024-04-25T08:48:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f25200640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04003bf0 0x7f1f04003ff0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007fc0 tx=0x7f1f1001cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d429713488f6846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10007c10 con 0x7f1f04003bf0 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f100a9070 con 0x7f1f04003bf0 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f1fe00640 1 -- 172.21.15.46:0/3743506389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10005d20 con 0x7f1f04003bf0 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f27c00640 1 -- 172.21.15.46:0/3743506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04003bf0 msgr2=0x7f1f04003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f27c00640 1 --2- 172.21.15.46:0/3743506389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f04003bf0 0x7f1f04003ff0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007fc0 tx=0x7f1f1001cb50 comp rx=0 tx=0).stop 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f25c00640 1 -- 172.21.15.46:0/3743506389 reap_dead start 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f27c00640 1 -- 172.21.15.46:0/3743506389 shutdown_connections 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f27c00640 1 -- 172.21.15.46:0/3743506389 >> 172.21.15.46:0/3743506389 conn(0x7f1f2006ac10 msgr2=0x7f1f2006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f27c00640 1 -- 172.21.15.46:0/3743506389 shutdown_connections 2024-04-25T08:48:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.524+0000 7f1f27c00640 1 -- 172.21.15.46:0/3743506389 wait complete. 2024-04-25T08:48:30.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 Processor -- start 2024-04-25T08:48:30.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 -- start start 2024-04-25T08:48:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f04003bf0 0x7f1f200ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2013d1b0 0x7f1f20143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f20143e60 0x7f1f20071270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:30.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20167510 con 0x7f1f20143e60 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2015d9b0 con 0x7f1f04003bf0 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20167690 con 0x7f1f2013d1b0 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f20143e60 0x7f1f20071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f20143e60 0x7f1f20071270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36792/0 (socket says 172.21.15.46:36792) 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25c00640 1 -- 172.21.15.46:0/1858497971 learned_addr learned my addr 172.21.15.46:0/1858497971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f17e00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2013d1b0 0x7f1f20143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25200640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f04003bf0 0x7f1f200ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25c00640 1 -- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2013d1b0 msgr2=0x7f1f20143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25c00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2013d1b0 0x7f1f20143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25c00640 1 -- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f04003bf0 msgr2=0x7f1f200ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25c00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f04003bf0 0x7f1f200ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25c00640 1 -- 172.21.15.46:0/1858497971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2013f480 con 0x7f1f20143e60 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f25c00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f20143e60 0x7f1f20071270 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1f18007fd0 tx=0x7f1f180021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f1d600640 1 -- 172.21.15.46:0/1858497971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18019070 con 0x7f1f20143e60 2024-04-25T08:48:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f1d600640 1 -- 172.21.15.46:0/1858497971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f18006d70 con 0x7f1f20143e60 2024-04-25T08:48:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f1d600640 1 -- 172.21.15.46:0/1858497971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18003ba0 con 0x7f1f20143e60 2024-04-25T08:48:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2013f760 con 0x7f1f20143e60 2024-04-25T08:48:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2013e500 con 0x7f1f20143e60 2024-04-25T08:48:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.528+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f20069010 con 0x7f1f20143e60 2024-04-25T08:48:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.532+0000 7f1f1d600640 1 -- 172.21.15.46:0/1858497971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f1800d7a0 con 0x7f1f20143e60 2024-04-25T08:48:30.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.532+0000 7f1f1d600640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1efc066f90 0x7f1efc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:30.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.532+0000 7f1f1d600640 1 -- 172.21.15.46:0/1858497971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f18084620 con 0x7f1f20143e60 2024-04-25T08:48:30.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.532+0000 7f1f25200640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1efc066f90 0x7f1efc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:30.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.532+0000 7f1f25200640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1efc066f90 0x7f1efc069450 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1f100ab000 tx=0x7f1f100b1040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.536+0000 7f1f1d600640 1 -- 172.21.15.46:0/1858497971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f18054e40 con 0x7f1f20143e60 2024-04-25T08:48:30.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:30.732+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1f2015b740 con 0x7f1f20143e60 2024-04-25T08:48:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.348+0000 7f1f1d600640 1 -- 172.21.15.46:0/1858497971 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f18058ee0 con 0x7f1f20143e60 2024-04-25T08:48:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1efc066f90 msgr2=0x7f1efc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1efc066f90 0x7f1efc069450 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1f100ab000 tx=0x7f1f100b1040 comp rx=0 tx=0).stop 2024-04-25T08:48:31.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f20143e60 msgr2=0x7f1f20071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f20143e60 0x7f1f20071270 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f1f18007fd0 tx=0x7f1f180021f0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 shutdown_connections 2024-04-25T08:48:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1efc066f90 0x7f1efc069450 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f20143e60 0x7f1f20071270 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f2013d1b0 0x7f1f20143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 --2- 172.21.15.46:0/1858497971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f04003bf0 0x7f1f200ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 >> 172.21.15.46:0/1858497971 conn(0x7f1f2006ac10 msgr2=0x7f1f201420c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 shutdown_connections 2024-04-25T08:48:31.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.352+0000 7f1f27c00640 1 -- 172.21.15.46:0/1858497971 wait complete. 2024-04-25T08:48:31.370 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:48:31.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:31.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.520+0000 7fecf9a00640 1 Processor -- start 2024-04-25T08:48:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.520+0000 7fecf9a00640 1 -- start start 2024-04-25T08:48:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecf9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 0x7fecf40b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecf9a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fecf40cc150 0x7fecf40cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecf9a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf40b25a0 con 0x7fecf40ce250 2024-04-25T08:48:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecf9a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf40b2eb0 con 0x7fecf40cc150 2024-04-25T08:48:31.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecf9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40cc6f0 con 0x7fecf4157b00 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecea000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fecf40cc150 0x7fecf40cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecea000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fecf40cc150 0x7fecf40cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:33346/0 (socket says 172.21.15.46:33346) 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecea000640 1 -- 172.21.15.46:0/3417298211 learned_addr learned my addr 172.21.15.46:0/3417298211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecf2a00640 1 --2- 172.21.15.46:0/3417298211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 0x7fecf40b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecea000640 1 -- 172.21.15.46:0/3417298211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 msgr2=0x7fecf40b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecea000640 1 --2- 172.21.15.46:0/3417298211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 0x7fecf40b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecea000640 1 -- 172.21.15.46:0/3417298211 >> v1:172.21.15.82:6789/0 conn(0x7fecf40ce250 legacy=0x7fecf415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecea000640 1 -- 172.21.15.46:0/3417298211 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40b2860 con 0x7fecf40cc150 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecf2000640 1 -- 172.21.15.46:0/3417298211 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000704956 0 0) 0x7fecf40b25a0 con 0x7fecf40ce250 2024-04-25T08:48:31.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.524+0000 7fecf2a00640 1 --2- 172.21.15.46:0/3417298211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 0x7fecf40b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:48:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecea000640 1 --2- 172.21.15.46:0/3417298211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf40cc150 0x7fecf40cdd10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00af60 tx=0x7fecdc01da70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac0a4932fe81ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf2000640 1 -- 172.21.15.46:0/3417298211 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00a2d0 con 0x7fecf40cc150 2024-04-25T08:48:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf2000640 1 -- 172.21.15.46:0/3417298211 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc0a67e0 con 0x7fecf40cc150 2024-04-25T08:48:31.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf2000640 1 -- 172.21.15.46:0/3417298211 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0a6bb0 con 0x7fecf40cc150 2024-04-25T08:48:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 -- 172.21.15.46:0/3417298211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf40cc150 msgr2=0x7fecf40cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 --2- 172.21.15.46:0/3417298211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf40cc150 0x7fecf40cdd10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00af60 tx=0x7fecdc01da70 comp rx=0 tx=0).stop 2024-04-25T08:48:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 -- 172.21.15.46:0/3417298211 shutdown_connections 2024-04-25T08:48:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 --2- 172.21.15.46:0/3417298211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf40cc150 0x7fecf40cdd10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 --2- 172.21.15.46:0/3417298211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 0x7fecf40b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 -- 172.21.15.46:0/3417298211 >> 172.21.15.46:0/3417298211 conn(0x7fecf40b12a0 msgr2=0x7fecf40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 -- 172.21.15.46:0/3417298211 shutdown_connections 2024-04-25T08:48:31.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 -- 172.21.15.46:0/3417298211 wait complete. 2024-04-25T08:48:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 Processor -- start 2024-04-25T08:48:31.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.528+0000 7fecf9a00640 1 -- start start 2024-04-25T08:48:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecf40cc150 0x7fecf4153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf40ce250 0x7fecf41509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf9a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 0x7fecf4156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf415aef0 con 0x7fecf40cc150 2024-04-25T08:48:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf9a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf415ad70 con 0x7fecf40ce250 2024-04-25T08:48:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf9a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf415b070 con 0x7fecf4157b00 2024-04-25T08:48:31.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecf40cc150 0x7fecf4153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecf40cc150 0x7fecf4153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36802/0 (socket says 172.21.15.46:36802) 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf2a00640 1 -- 172.21.15.46:0/2631813597 learned_addr learned my addr 172.21.15.46:0/2631813597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf3400640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 0x7fecf4156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecea000640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf40ce250 0x7fecf41509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf2a00640 1 -- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 msgr2=0x7fecf4156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf2a00640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 0x7fecf4156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf2a00640 1 -- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf40ce250 msgr2=0x7fecf41509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf2a00640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf40ce250 0x7fecf41509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf2a00640 1 -- 172.21.15.46:0/2631813597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf4153a40 con 0x7fecf40cc150 2024-04-25T08:48:31.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf2a00640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecf40cc150 0x7fecf4153460 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fece400e980 tx=0x7fece400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7feceb400640 1 -- 172.21.15.46:0/2631813597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4017070 con 0x7fecf40cc150 2024-04-25T08:48:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7feceb400640 1 -- 172.21.15.46:0/2631813597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece400ca00 con 0x7fecf40cc150 2024-04-25T08:48:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf4153d20 con 0x7fecf40cc150 2024-04-25T08:48:31.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7feceb400640 1 -- 172.21.15.46:0/2631813597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece400cdd0 con 0x7fecf40cc150 2024-04-25T08:48:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf41542e0 con 0x7fecf40cc150 2024-04-25T08:48:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.532+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf4069080 con 0x7fecf40cc150 2024-04-25T08:48:31.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.536+0000 7feceb400640 1 -- 172.21.15.46:0/2631813597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fece401a070 con 0x7fecf40cc150 2024-04-25T08:48:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.536+0000 7feceb400640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecd0066f60 0x7fecd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.536+0000 7feceb400640 1 -- 172.21.15.46:0/2631813597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fece4085150 con 0x7fecf40cc150 2024-04-25T08:48:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.536+0000 7fecea000640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecd0066f60 0x7fecd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.536+0000 7fecea000640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecd0066f60 0x7fecd0069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0af000 tx=0x7fecdc0a8070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:31.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.540+0000 7feceb400640 1 -- 172.21.15.46:0/2631813597 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fece4055f30 con 0x7fecf40cc150 2024-04-25T08:48:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecf40cdda0 con 0x7fecf40cc150 2024-04-25T08:48:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7feceb400640 1 -- 172.21.15.46:0/2631813597 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fece4055830 con 0x7fecf40cc150 2024-04-25T08:48:31.746 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:31.746 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:48:30.739960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1269610178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1269610178},{"type":"v1","addr":"172.21.15.46:6835","nonce":1269610178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:48:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 62 2024-04-25T08:48:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecd0066f60 msgr2=0x7fecd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecd0066f60 0x7fecd0069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0af000 tx=0x7fecdc0a8070 comp rx=0 tx=0).stop 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecf40cc150 msgr2=0x7fecf4153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecf40cc150 0x7fecf4153460 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fece400e980 tx=0x7fece400ee40 comp rx=0 tx=0).stop 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 shutdown_connections 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecd0066f60 0x7fecd0069420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecf4157b00 0x7fecf4156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecf40ce250 0x7fecf41509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 --2- 172.21.15.46:0/2631813597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecf40cc150 0x7fecf4153460 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 >> 172.21.15.46:0/2631813597 conn(0x7fecf40b12a0 msgr2=0x7fecf4073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 shutdown_connections 2024-04-25T08:48:31.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.752+0000 7fecf9a00640 1 -- 172.21.15.46:0/2631813597 wait complete. 2024-04-25T08:48:31.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:31.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.912+0000 7f5384e00640 1 Processor -- start 2024-04-25T08:48:31.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.912+0000 7f5384e00640 1 -- start start 2024-04-25T08:48:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f5384e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5380145050 0x7f5380070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f5384e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53800d1870 0x7f53800b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f5384e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53800d1e50 con 0x7f53800711f0 2024-04-25T08:48:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f5384e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538015be40 con 0x7f53800d1870 2024-04-25T08:48:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f5384e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5380071630 con 0x7f5380145050 2024-04-25T08:48:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53800d1870 0x7f53800b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53800d1870 0x7f53800b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36806/0 (socket says 172.21.15.46:36806) 2024-04-25T08:48:31.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537e800640 1 -- 172.21.15.46:0/2570839150 learned_addr learned my addr 172.21.15.46:0/2570839150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:31.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537e800640 1 -- 172.21.15.46:0/2570839150 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5380145050 msgr2=0x7f5380070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:31.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537de00640 1 --2- 172.21.15.46:0/2570839150 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5380145050 0x7f5380070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537e800640 1 --2- 172.21.15.46:0/2570839150 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5380145050 0x7f5380070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537e800640 1 -- 172.21.15.46:0/2570839150 >> v1:172.21.15.82:6789/0 conn(0x7f53800711f0 legacy=0x7f53800d1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:31.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537e800640 1 -- 172.21.15.46:0/2570839150 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53800d2130 con 0x7f53800d1870 2024-04-25T08:48:31.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537de00640 1 --2- 172.21.15.46:0/2570839150 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5380145050 0x7f5380070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:31.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.916+0000 7f537e800640 1 --2- 172.21.15.46:0/2570839150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53800d1870 0x7f53800b5760 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f53780098a0 tx=0x7f537801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe36a43a71d93f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:31.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f537d400640 1 -- 172.21.15.46:0/2570839150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780aa5f0 con 0x7f53800d1870 2024-04-25T08:48:31.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f537d400640 1 -- 172.21.15.46:0/2570839150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53780af070 con 0x7f53800d1870 2024-04-25T08:48:31.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f537d400640 1 -- 172.21.15.46:0/2570839150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780aa9b0 con 0x7f53800d1870 2024-04-25T08:48:31.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 -- 172.21.15.46:0/2570839150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53800d1870 msgr2=0x7f53800b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 --2- 172.21.15.46:0/2570839150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53800d1870 0x7f53800b5760 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f53780098a0 tx=0x7f537801c4d0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 -- 172.21.15.46:0/2570839150 shutdown_connections 2024-04-25T08:48:31.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 --2- 172.21.15.46:0/2570839150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53800d1870 0x7f53800b5760 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 --2- 172.21.15.46:0/2570839150 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5380145050 0x7f5380070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 -- 172.21.15.46:0/2570839150 >> 172.21.15.46:0/2570839150 conn(0x7f53800b33a0 msgr2=0x7f53800b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:31.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 -- 172.21.15.46:0/2570839150 shutdown_connections 2024-04-25T08:48:31.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 -- 172.21.15.46:0/2570839150 wait complete. 2024-04-25T08:48:31.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 Processor -- start 2024-04-25T08:48:31.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 -- start start 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53800711f0 0x7f538013cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53800d1870 0x7f538006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5380145050 0x7f538013bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53800b6630 con 0x7f5380145050 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53800b64b0 con 0x7f53800d1870 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5384e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53800b67b0 con 0x7f53800711f0 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f537e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5380145050 0x7f538013bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f537e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5380145050 0x7f538013bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36822/0 (socket says 172.21.15.46:36822) 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f537e800640 1 -- 172.21.15.46:0/2829971126 learned_addr learned my addr 172.21.15.46:0/2829971126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f537de00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53800711f0 0x7f538013cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.920+0000 7f5375400640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53800d1870 0x7f538006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f537e800640 1 -- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53800711f0 msgr2=0x7f538013cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f537e800640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53800711f0 0x7f538013cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f537e800640 1 -- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53800d1870 msgr2=0x7f538006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:31.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f537e800640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53800d1870 0x7f538006d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:31.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f537e800640 1 -- 172.21.15.46:0/2829971126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538013d1f0 con 0x7f5380145050 2024-04-25T08:48:31.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f537e800640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5380145050 0x7f538013bc30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f53780ac040 tx=0x7f537801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:31.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f5376a00640 1 -- 172.21.15.46:0/2829971126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780bd5f0 con 0x7f5380145050 2024-04-25T08:48:31.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f5376a00640 1 -- 172.21.15.46:0/2829971126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53780bd790 con 0x7f5380145050 2024-04-25T08:48:31.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f5376a00640 1 -- 172.21.15.46:0/2829971126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780bdb80 con 0x7f5380145050 2024-04-25T08:48:31.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538013d4d0 con 0x7f5380145050 2024-04-25T08:48:31.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538013da90 con 0x7f5380145050 2024-04-25T08:48:31.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5380069080 con 0x7f5380145050 2024-04-25T08:48:31.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f5376a00640 1 -- 172.21.15.46:0/2829971126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53780b33f0 con 0x7f5380145050 2024-04-25T08:48:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f5376a00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f535c066f60 0x7f535c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.924+0000 7f5376a00640 1 -- 172.21.15.46:0/2829971126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f537811fec0 con 0x7f5380145050 2024-04-25T08:48:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.928+0000 7f537de00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f535c066f60 0x7f535c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:31.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.928+0000 7f537de00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f535c066f60 0x7f535c069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5370006fd0 tx=0x7f5370008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:31.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:31.928+0000 7f5376a00640 1 -- 172.21.15.46:0/2829971126 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53780c3060 con 0x7f5380145050 2024-04-25T08:48:32.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f538015f0f0 con 0x7f5380145050 2024-04-25T08:48:32.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5376a00640 1 -- 172.21.15.46:0/2829971126 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16837 (secure 0 0 0) 0x7f53780160f0 con 0x7f5380145050 2024-04-25T08:48:32.091 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:32.091 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":90,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:48:29.334944+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:48:26.132790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:48:26.714570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f535c066f60 msgr2=0x7f535c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f535c066f60 0x7f535c069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5370006fd0 tx=0x7f5370008040 comp rx=0 tx=0).stop 2024-04-25T08:48:32.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5380145050 msgr2=0x7f538013bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5380145050 0x7f538013bc30 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f53780ac040 tx=0x7f537801ad10 comp rx=0 tx=0).stop 2024-04-25T08:48:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 shutdown_connections 2024-04-25T08:48:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f535c066f60 0x7f535c069420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5380145050 0x7f538013bc30 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53800d1870 0x7f538006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 --2- 172.21.15.46:0/2829971126 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f53800711f0 0x7f538013cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 >> 172.21.15.46:0/2829971126 conn(0x7f53800b33a0 msgr2=0x7f53800b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 shutdown_connections 2024-04-25T08:48:32.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.096+0000 7f5384e00640 1 -- 172.21.15.46:0/2829971126 wait complete. 2024-04-25T08:48:32.112 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:48:32.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:48:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f2585e00640 1 Processor -- start 2024-04-25T08:48:32.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f2585e00640 1 -- start start 2024-04-25T08:48:32.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f2585e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2580160f50 0x7f258015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f2585e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f258006f110 0x7f25800ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f2585e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2580161330 con 0x7f25800adc50 2024-04-25T08:48:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f2585e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258006d680 con 0x7f2580160f50 2024-04-25T08:48:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f2585e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25800ae810 con 0x7f258006f110 2024-04-25T08:48:32.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f257ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2580160f50 0x7f258015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f257ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2580160f50 0x7f258015f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36824/0 (socket says 172.21.15.46:36824) 2024-04-25T08:48:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f257ea00640 1 -- 172.21.15.46:0/209811882 learned_addr learned my addr 172.21.15.46:0/209811882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f257f400640 1 --2- 172.21.15.46:0/209811882 >> v2:172.21.15.82:3300/0 conn(0x7f258006f110 0x7f25800ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f257ea00640 1 -- 172.21.15.46:0/209811882 >> v2:172.21.15.82:3300/0 conn(0x7f258006f110 msgr2=0x7f25800ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f257ea00640 1 --2- 172.21.15.46:0/209811882 >> v2:172.21.15.82:3300/0 conn(0x7f258006f110 0x7f25800ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f257ea00640 1 -- 172.21.15.46:0/209811882 >> v1:172.21.15.82:6789/0 conn(0x7f25800adc50 legacy=0x7f258006ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:32.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.260+0000 7f257ea00640 1 -- 172.21.15.46:0/209811882 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25800aeaf0 con 0x7f2580160f50 2024-04-25T08:48:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f257ea00640 1 --2- 172.21.15.46:0/209811882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2580160f50 0x7f258015f1a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f25800cba60 tx=0x7f257001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568f42df54a3127e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f257e000640 1 -- 172.21.15.46:0/209811882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700ab5f0 con 0x7f2580160f50 2024-04-25T08:48:32.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f257e000640 1 -- 172.21.15.46:0/209811882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25700b0070 con 0x7f2580160f50 2024-04-25T08:48:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f257e000640 1 -- 172.21.15.46:0/209811882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700ab9b0 con 0x7f2580160f50 2024-04-25T08:48:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f2585e00640 1 -- 172.21.15.46:0/209811882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2580160f50 msgr2=0x7f258015f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f2585e00640 1 --2- 172.21.15.46:0/209811882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2580160f50 0x7f258015f1a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f25800cba60 tx=0x7f257001c4e0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f2585e00640 1 -- 172.21.15.46:0/209811882 shutdown_connections 2024-04-25T08:48:32.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f2585e00640 1 --2- 172.21.15.46:0/209811882 >> v2:172.21.15.82:3300/0 conn(0x7f258006f110 0x7f25800ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f2585e00640 1 --2- 172.21.15.46:0/209811882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2580160f50 0x7f258015f1a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f2585e00640 1 -- 172.21.15.46:0/209811882 >> 172.21.15.46:0/209811882 conn(0x7f25800b12a0 msgr2=0x7f25800b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f2585e00640 1 -- 172.21.15.46:0/209811882 shutdown_connections 2024-04-25T08:48:32.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.264+0000 7f2585e00640 1 -- 172.21.15.46:0/209811882 wait complete. 2024-04-25T08:48:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 Processor -- start 2024-04-25T08:48:32.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 -- start start 2024-04-25T08:48:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258006f110 0x7f2580138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25800adc50 0x7f2580141940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2580160f50 0x7f25800cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258006e4a0 con 0x7f258006f110 2024-04-25T08:48:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258006e320 con 0x7f25800adc50 2024-04-25T08:48:32.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258006e620 con 0x7f2580160f50 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258006f110 0x7f2580138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258006f110 0x7f2580138d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36836/0 (socket says 172.21.15.46:36836) 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257ea00640 1 -- 172.21.15.46:0/4077015734 learned_addr learned my addr 172.21.15.46:0/4077015734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2576000640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25800adc50 0x7f2580141940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257f400640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2580160f50 0x7f25800cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257ea00640 1 -- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2580160f50 msgr2=0x7f25800cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257ea00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2580160f50 0x7f25800cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257ea00640 1 -- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25800adc50 msgr2=0x7f2580141940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257ea00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25800adc50 0x7f2580141940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257ea00640 1 -- 172.21.15.46:0/4077015734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580143330 con 0x7f258006f110 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f257ea00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258006f110 0x7f2580138d50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f25700ad040 tx=0x7f257001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2577e00640 1 -- 172.21.15.46:0/4077015734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700abe60 con 0x7f258006f110 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2577e00640 1 -- 172.21.15.46:0/4077015734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25700be5f0 con 0x7f258006f110 2024-04-25T08:48:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2580142290 con 0x7f258006f110 2024-04-25T08:48:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2577e00640 1 -- 172.21.15.46:0/4077015734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700be9c0 con 0x7f258006f110 2024-04-25T08:48:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2580142850 con 0x7f258006f110 2024-04-25T08:48:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.268+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2580069080 con 0x7f258006f110 2024-04-25T08:48:32.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.272+0000 7f2577e00640 1 -- 172.21.15.46:0/4077015734 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25700b43f0 con 0x7f258006f110 2024-04-25T08:48:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.272+0000 7f2577e00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f255c066f60 0x7f255c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.272+0000 7f2577e00640 1 -- 172.21.15.46:0/4077015734 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f257011f5e0 con 0x7f258006f110 2024-04-25T08:48:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.272+0000 7f2576000640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f255c066f60 0x7f255c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.272+0000 7f2576000640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f255c066f60 0x7f255c069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2580143bd0 tx=0x7f2568009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.276+0000 7f2577e00640 1 -- 172.21.15.46:0/4077015734 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25700c4060 con 0x7f258006f110 2024-04-25T08:48:32.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.436+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2580073db0 con 0x7f255c066f60 2024-04-25T08:48:32.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.444+0000 7f2577e00640 1 -- 172.21.15.46:0/4077015734 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252074 (secure 0 0 0) 0x7f2580073db0 con 0x7f255c066f60 2024-04-25T08:48:32.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:32.443 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:48:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f255c066f60 msgr2=0x7f255c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f255c066f60 0x7f255c069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2580143bd0 tx=0x7f2568009040 comp rx=0 tx=0).stop 2024-04-25T08:48:32.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258006f110 msgr2=0x7f2580138d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258006f110 0x7f2580138d50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f25700ad040 tx=0x7f257001ad20 comp rx=0 tx=0).stop 2024-04-25T08:48:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 shutdown_connections 2024-04-25T08:48:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f255c066f60 0x7f255c069420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2580160f50 0x7f25800cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25800adc50 0x7f2580141940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 --2- 172.21.15.46:0/4077015734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258006f110 0x7f2580138d50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 >> 172.21.15.46:0/4077015734 conn(0x7f25800b12a0 msgr2=0x7f2580140cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 shutdown_connections 2024-04-25T08:48:32.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.448+0000 7f2585e00640 1 -- 172.21.15.46:0/4077015734 wait complete. 2024-04-25T08:48:32.460 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-04-25T08:48:32.347326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259356,"kb_used_data":9896,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715364,"statfs":{"total":773094113280,"available":772828532736,"internally_reserved":0,"allocated":10133504,"data_stored":5271572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001256"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342679+0000","last_change":"2024-04-25T08:48:28.355459+0000","last_active":"2024-04-25T08:48:29.342679+0000","last_peered":"2024-04-25T08:48:29.342679+0000","last_clean":"2024-04-25T08:48:29.342679+0000","last_became_active":"2024-04-25T08:48:28.354742+0000","last_became_peered":"2024-04-25T08:48:28.354742+0000","last_unstale":"2024-04-25T08:48:29.342679+0000","last_undegraded":"2024-04-25T08:48:29.342679+0000","last_fullsized":"2024-04-25T08:48:29.342679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:18.648662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538928+0000","last_change":"2024-04-25T08:48:28.352992+0000","last_active":"2024-04-25T08:48:29.538928+0000","last_peered":"2024-04-25T08:48:29.538928+0000","last_clean":"2024-04-25T08:48:29.538928+0000","last_became_active":"2024-04-25T08:48:28.352699+0000","last_became_peered":"2024-04-25T08:48:28.352699+0000","last_unstale":"2024-04-25T08:48:29.538928+0000","last_undegraded":"2024-04-25T08:48:29.538928+0000","last_fullsized":"2024-04-25T08:48:29.538928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:24.977760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342391+0000","last_change":"2024-04-25T08:48:28.358791+0000","last_active":"2024-04-25T08:48:29.342391+0000","last_peered":"2024-04-25T08:48:29.342391+0000","last_clean":"2024-04-25T08:48:29.342391+0000","last_became_active":"2024-04-25T08:48:28.355698+0000","last_became_peered":"2024-04-25T08:48:28.355698+0000","last_unstale":"2024-04-25T08:48:29.342391+0000","last_undegraded":"2024-04-25T08:48:29.342391+0000","last_fullsized":"2024-04-25T08:48:29.342391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:41.718549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963347+0000","last_change":"2024-04-25T08:48:28.347034+0000","last_active":"2024-04-25T08:48:29.963347+0000","last_peered":"2024-04-25T08:48:29.963347+0000","last_clean":"2024-04-25T08:48:29.963347+0000","last_became_active":"2024-04-25T08:48:28.345969+0000","last_became_peered":"2024-04-25T08:48:28.345969+0000","last_unstale":"2024-04-25T08:48:29.963347+0000","last_undegraded":"2024-04-25T08:48:29.963347+0000","last_fullsized":"2024-04-25T08:48:29.963347+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:01.607154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347243+0000","last_change":"2024-04-25T08:48:28.339924+0000","last_active":"2024-04-25T08:48:29.347243+0000","last_peered":"2024-04-25T08:48:29.347243+0000","last_clean":"2024-04-25T08:48:29.347243+0000","last_became_active":"2024-04-25T08:48:28.339636+0000","last_became_peered":"2024-04-25T08:48:28.339636+0000","last_unstale":"2024-04-25T08:48:29.347243+0000","last_undegraded":"2024-04-25T08:48:29.347243+0000","last_fullsized":"2024-04-25T08:48:29.347243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:06.711222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508273+0000","last_change":"2024-04-25T08:48:28.337302+0000","last_active":"2024-04-25T08:48:30.508273+0000","last_peered":"2024-04-25T08:48:30.508273+0000","last_clean":"2024-04-25T08:48:30.508273+0000","last_became_active":"2024-04-25T08:48:28.337008+0000","last_became_peered":"2024-04-25T08:48:28.337008+0000","last_unstale":"2024-04-25T08:48:30.508273+0000","last_undegraded":"2024-04-25T08:48:30.508273+0000","last_fullsized":"2024-04-25T08:48:30.508273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:05.971261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584075+0000","last_change":"2024-04-25T08:48:28.356045+0000","last_active":"2024-04-25T08:48:31.584075+0000","last_peered":"2024-04-25T08:48:31.584075+0000","last_clean":"2024-04-25T08:48:31.584075+0000","last_became_active":"2024-04-25T08:48:28.355708+0000","last_became_peered":"2024-04-25T08:48:28.355708+0000","last_unstale":"2024-04-25T08:48:31.584075+0000","last_undegraded":"2024-04-25T08:48:31.584075+0000","last_fullsized":"2024-04-25T08:48:31.584075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:53.946686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962671+0000","last_change":"2024-04-25T08:48:28.339621+0000","last_active":"2024-04-25T08:48:29.962671+0000","last_peered":"2024-04-25T08:48:29.962671+0000","last_clean":"2024-04-25T08:48:29.962671+0000","last_became_active":"2024-04-25T08:48:28.339332+0000","last_became_peered":"2024-04-25T08:48:28.339332+0000","last_unstale":"2024-04-25T08:48:29.962671+0000","last_undegraded":"2024-04-25T08:48:29.962671+0000","last_fullsized":"2024-04-25T08:48:29.962671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:19.457397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347240+0000","last_change":"2024-04-25T08:48:28.346973+0000","last_active":"2024-04-25T08:48:29.347240+0000","last_peered":"2024-04-25T08:48:29.347240+0000","last_clean":"2024-04-25T08:48:29.347240+0000","last_became_active":"2024-04-25T08:48:28.345326+0000","last_became_peered":"2024-04-25T08:48:28.345326+0000","last_unstale":"2024-04-25T08:48:29.347240+0000","last_undegraded":"2024-04-25T08:48:29.347240+0000","last_fullsized":"2024-04-25T08:48:29.347240+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:19.698931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.345235+0000","last_change":"2024-04-25T08:48:28.340190+0000","last_active":"2024-04-25T08:48:29.345235+0000","last_peered":"2024-04-25T08:48:29.345235+0000","last_clean":"2024-04-25T08:48:29.345235+0000","last_became_active":"2024-04-25T08:48:28.339891+0000","last_became_peered":"2024-04-25T08:48:28.339891+0000","last_unstale":"2024-04-25T08:48:29.345235+0000","last_undegraded":"2024-04-25T08:48:29.345235+0000","last_fullsized":"2024-04-25T08:48:29.345235+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:18.175106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347383+0000","last_change":"2024-04-25T08:48:28.352521+0000","last_active":"2024-04-25T08:48:29.347383+0000","last_peered":"2024-04-25T08:48:29.347383+0000","last_clean":"2024-04-25T08:48:29.347383+0000","last_became_active":"2024-04-25T08:48:28.351876+0000","last_became_peered":"2024-04-25T08:48:28.351876+0000","last_unstale":"2024-04-25T08:48:29.347383+0000","last_undegraded":"2024-04-25T08:48:29.347383+0000","last_fullsized":"2024-04-25T08:48:29.347383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:01.607154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344987+0000","last_change":"2024-04-25T08:48:28.362256+0000","last_active":"2024-04-25T08:48:29.344987+0000","last_peered":"2024-04-25T08:48:29.344987+0000","last_clean":"2024-04-25T08:48:29.344987+0000","last_became_active":"2024-04-25T08:48:28.361983+0000","last_became_peered":"2024-04-25T08:48:28.361983+0000","last_unstale":"2024-04-25T08:48:29.344987+0000","last_undegraded":"2024-04-25T08:48:29.344987+0000","last_fullsized":"2024-04-25T08:48:29.344987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:21.105685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963315+0000","last_change":"2024-04-25T08:48:28.348932+0000","last_active":"2024-04-25T08:48:29.963315+0000","last_peered":"2024-04-25T08:48:29.963315+0000","last_clean":"2024-04-25T08:48:29.963315+0000","last_became_active":"2024-04-25T08:48:28.348052+0000","last_became_peered":"2024-04-25T08:48:28.348052+0000","last_unstale":"2024-04-25T08:48:29.963315+0000","last_undegraded":"2024-04-25T08:48:29.963315+0000","last_fullsized":"2024-04-25T08:48:29.963315+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:48.911885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347314+0000","last_change":"2024-04-25T08:48:28.349373+0000","last_active":"2024-04-25T08:48:29.347314+0000","last_peered":"2024-04-25T08:48:29.347314+0000","last_clean":"2024-04-25T08:48:29.347314+0000","last_became_active":"2024-04-25T08:48:28.349078+0000","last_became_peered":"2024-04-25T08:48:28.349078+0000","last_unstale":"2024-04-25T08:48:29.347314+0000","last_undegraded":"2024-04-25T08:48:29.347314+0000","last_fullsized":"2024-04-25T08:48:29.347314+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:48.911885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962981+0000","last_change":"2024-04-25T08:48:28.357145+0000","last_active":"2024-04-25T08:48:29.962981+0000","last_peered":"2024-04-25T08:48:29.962981+0000","last_clean":"2024-04-25T08:48:29.962981+0000","last_became_active":"2024-04-25T08:48:28.356861+0000","last_became_peered":"2024-04-25T08:48:28.356861+0000","last_unstale":"2024-04-25T08:48:29.962981+0000","last_undegraded":"2024-04-25T08:48:29.962981+0000","last_fullsized":"2024-04-25T08:48:29.962981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:19.698931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508237+0000","last_change":"2024-04-25T08:48:28.356515+0000","last_active":"2024-04-25T08:48:30.508237+0000","last_peered":"2024-04-25T08:48:30.508237+0000","last_clean":"2024-04-25T08:48:30.508237+0000","last_became_active":"2024-04-25T08:48:28.355293+0000","last_became_peered":"2024-04-25T08:48:28.355293+0000","last_unstale":"2024-04-25T08:48:30.508237+0000","last_undegraded":"2024-04-25T08:48:30.508237+0000","last_fullsized":"2024-04-25T08:48:30.508237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:53.265772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508751+0000","last_change":"2024-04-25T08:48:28.353509+0000","last_active":"2024-04-25T08:48:30.508751+0000","last_peered":"2024-04-25T08:48:30.508751+0000","last_clean":"2024-04-25T08:48:30.508751+0000","last_became_active":"2024-04-25T08:48:28.353242+0000","last_became_peered":"2024-04-25T08:48:28.353242+0000","last_unstale":"2024-04-25T08:48:30.508751+0000","last_undegraded":"2024-04-25T08:48:30.508751+0000","last_fullsized":"2024-04-25T08:48:30.508751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:58.782234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342550+0000","last_change":"2024-04-25T08:48:28.358625+0000","last_active":"2024-04-25T08:48:29.342550+0000","last_peered":"2024-04-25T08:48:29.342550+0000","last_clean":"2024-04-25T08:48:29.342550+0000","last_became_active":"2024-04-25T08:48:28.358361+0000","last_became_peered":"2024-04-25T08:48:28.358361+0000","last_unstale":"2024-04-25T08:48:29.342550+0000","last_undegraded":"2024-04-25T08:48:29.342550+0000","last_fullsized":"2024-04-25T08:48:29.342550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:04.197012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347283+0000","last_change":"2024-04-25T08:48:28.341057+0000","last_active":"2024-04-25T08:48:29.347283+0000","last_peered":"2024-04-25T08:48:29.347283+0000","last_clean":"2024-04-25T08:48:29.347283+0000","last_became_active":"2024-04-25T08:48:28.340710+0000","last_became_peered":"2024-04-25T08:48:28.340710+0000","last_unstale":"2024-04-25T08:48:29.347283+0000","last_undegraded":"2024-04-25T08:48:29.347283+0000","last_fullsized":"2024-04-25T08:48:29.347283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:11.473349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963254+0000","last_change":"2024-04-25T08:48:28.356979+0000","last_active":"2024-04-25T08:48:29.963254+0000","last_peered":"2024-04-25T08:48:29.963254+0000","last_clean":"2024-04-25T08:48:29.963254+0000","last_became_active":"2024-04-25T08:48:28.356645+0000","last_became_peered":"2024-04-25T08:48:28.356645+0000","last_unstale":"2024-04-25T08:48:29.963254+0000","last_undegraded":"2024-04-25T08:48:29.963254+0000","last_fullsized":"2024-04-25T08:48:29.963254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:11.473349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.346802+0000","last_change":"2024-04-25T08:48:28.337712+0000","last_active":"2024-04-25T08:48:29.346802+0000","last_peered":"2024-04-25T08:48:29.346802+0000","last_clean":"2024-04-25T08:48:29.346802+0000","last_became_active":"2024-04-25T08:48:28.337374+0000","last_became_peered":"2024-04-25T08:48:28.337374+0000","last_unstale":"2024-04-25T08:48:29.346802+0000","last_undegraded":"2024-04-25T08:48:29.346802+0000","last_fullsized":"2024-04-25T08:48:29.346802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:07.494358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.346735+0000","last_change":"2024-04-25T08:48:28.335918+0000","last_active":"2024-04-25T08:48:29.346735+0000","last_peered":"2024-04-25T08:48:29.346735+0000","last_clean":"2024-04-25T08:48:29.346735+0000","last_became_active":"2024-04-25T08:48:28.335302+0000","last_became_peered":"2024-04-25T08:48:28.335302+0000","last_unstale":"2024-04-25T08:48:29.346735+0000","last_undegraded":"2024-04-25T08:48:29.346735+0000","last_fullsized":"2024-04-25T08:48:29.346735+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:19.457397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156467+0000","last_change":"2024-04-25T08:48:28.342192+0000","last_active":"2024-04-25T08:48:31.156467+0000","last_peered":"2024-04-25T08:48:31.156467+0000","last_clean":"2024-04-25T08:48:31.156467+0000","last_became_active":"2024-04-25T08:48:28.341893+0000","last_became_peered":"2024-04-25T08:48:28.341893+0000","last_unstale":"2024-04-25T08:48:31.156467+0000","last_undegraded":"2024-04-25T08:48:31.156467+0000","last_fullsized":"2024-04-25T08:48:31.156467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:11.473349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342334+0000","last_change":"2024-04-25T08:48:28.353808+0000","last_active":"2024-04-25T08:48:29.342334+0000","last_peered":"2024-04-25T08:48:29.342334+0000","last_clean":"2024-04-25T08:48:29.342334+0000","last_became_active":"2024-04-25T08:48:28.353433+0000","last_became_peered":"2024-04-25T08:48:28.353433+0000","last_unstale":"2024-04-25T08:48:29.342334+0000","last_undegraded":"2024-04-25T08:48:29.342334+0000","last_fullsized":"2024-04-25T08:48:29.342334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:53.946686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538960+0000","last_change":"2024-04-25T08:48:28.346131+0000","last_active":"2024-04-25T08:48:29.538960+0000","last_peered":"2024-04-25T08:48:29.538960+0000","last_clean":"2024-04-25T08:48:29.538960+0000","last_became_active":"2024-04-25T08:48:28.345776+0000","last_became_peered":"2024-04-25T08:48:28.345776+0000","last_unstale":"2024-04-25T08:48:29.538960+0000","last_undegraded":"2024-04-25T08:48:29.538960+0000","last_fullsized":"2024-04-25T08:48:29.538960+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:26.594635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347324+0000","last_change":"2024-04-25T08:48:28.341255+0000","last_active":"2024-04-25T08:48:29.347324+0000","last_peered":"2024-04-25T08:48:29.347324+0000","last_clean":"2024-04-25T08:48:29.347324+0000","last_became_active":"2024-04-25T08:48:28.340957+0000","last_became_peered":"2024-04-25T08:48:28.340957+0000","last_unstale":"2024-04-25T08:48:29.347324+0000","last_undegraded":"2024-04-25T08:48:29.347324+0000","last_fullsized":"2024-04-25T08:48:29.347324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:51.751147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.345148+0000","last_change":"2024-04-25T08:48:28.348878+0000","last_active":"2024-04-25T08:48:29.345148+0000","last_peered":"2024-04-25T08:48:29.345148+0000","last_clean":"2024-04-25T08:48:29.345148+0000","last_became_active":"2024-04-25T08:48:28.348528+0000","last_became_peered":"2024-04-25T08:48:28.348528+0000","last_unstale":"2024-04-25T08:48:29.345148+0000","last_undegraded":"2024-04-25T08:48:29.345148+0000","last_fullsized":"2024-04-25T08:48:29.345148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:43.378571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963439+0000","last_change":"2024-04-25T08:48:28.339673+0000","last_active":"2024-04-25T08:48:29.963439+0000","last_peered":"2024-04-25T08:48:29.963439+0000","last_clean":"2024-04-25T08:48:29.963439+0000","last_became_active":"2024-04-25T08:48:28.339365+0000","last_became_peered":"2024-04-25T08:48:28.339365+0000","last_unstale":"2024-04-25T08:48:29.963439+0000","last_undegraded":"2024-04-25T08:48:29.963439+0000","last_fullsized":"2024-04-25T08:48:29.963439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:32.313567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584264+0000","last_change":"2024-04-25T08:48:28.344242+0000","last_active":"2024-04-25T08:48:31.584264+0000","last_peered":"2024-04-25T08:48:31.584264+0000","last_clean":"2024-04-25T08:48:31.584264+0000","last_became_active":"2024-04-25T08:48:28.343965+0000","last_became_peered":"2024-04-25T08:48:28.343965+0000","last_unstale":"2024-04-25T08:48:31.584264+0000","last_undegraded":"2024-04-25T08:48:31.584264+0000","last_fullsized":"2024-04-25T08:48:31.584264+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:41.718549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347140+0000","last_change":"2024-04-25T08:48:28.345531+0000","last_active":"2024-04-25T08:48:29.347140+0000","last_peered":"2024-04-25T08:48:29.347140+0000","last_clean":"2024-04-25T08:48:29.347140+0000","last_became_active":"2024-04-25T08:48:28.345214+0000","last_became_peered":"2024-04-25T08:48:28.345214+0000","last_unstale":"2024-04-25T08:48:29.347140+0000","last_undegraded":"2024-04-25T08:48:29.347140+0000","last_fullsized":"2024-04-25T08:48:29.347140+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:20.723874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538592+0000","last_change":"2024-04-25T08:48:28.344939+0000","last_active":"2024-04-25T08:48:29.538592+0000","last_peered":"2024-04-25T08:48:29.538592+0000","last_clean":"2024-04-25T08:48:29.538592+0000","last_became_active":"2024-04-25T08:48:28.344298+0000","last_became_peered":"2024-04-25T08:48:28.344298+0000","last_unstale":"2024-04-25T08:48:29.538592+0000","last_undegraded":"2024-04-25T08:48:29.538592+0000","last_fullsized":"2024-04-25T08:48:29.538592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:01.607154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508280+0000","last_change":"2024-04-25T08:48:28.355576+0000","last_active":"2024-04-25T08:48:30.508280+0000","last_peered":"2024-04-25T08:48:30.508280+0000","last_clean":"2024-04-25T08:48:30.508280+0000","last_became_active":"2024-04-25T08:48:28.354979+0000","last_became_peered":"2024-04-25T08:48:28.354979+0000","last_unstale":"2024-04-25T08:48:30.508280+0000","last_undegraded":"2024-04-25T08:48:30.508280+0000","last_fullsized":"2024-04-25T08:48:30.508280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:26.509829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508821+0000","last_change":"2024-04-25T08:48:28.350985+0000","last_active":"2024-04-25T08:48:30.508821+0000","last_peered":"2024-04-25T08:48:30.508821+0000","last_clean":"2024-04-25T08:48:30.508821+0000","last_became_active":"2024-04-25T08:48:28.350675+0000","last_became_peered":"2024-04-25T08:48:28.350675+0000","last_unstale":"2024-04-25T08:48:30.508821+0000","last_undegraded":"2024-04-25T08:48:30.508821+0000","last_fullsized":"2024-04-25T08:48:30.508821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:42.579967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156690+0000","last_change":"2024-04-25T08:48:27.344548+0000","last_active":"2024-04-25T08:48:31.156690+0000","last_peered":"2024-04-25T08:48:31.156690+0000","last_clean":"2024-04-25T08:48:31.156690+0000","last_became_active":"2024-04-25T08:48:27.343372+0000","last_became_peered":"2024-04-25T08:48:27.343372+0000","last_unstale":"2024-04-25T08:48:31.156690+0000","last_undegraded":"2024-04-25T08:48:31.156690+0000","last_fullsized":"2024-04-25T08:48:31.156690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:12.383863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508608+0000","last_change":"2024-04-25T08:48:28.342015+0000","last_active":"2024-04-25T08:48:30.508608+0000","last_peered":"2024-04-25T08:48:30.508608+0000","last_clean":"2024-04-25T08:48:30.508608+0000","last_became_active":"2024-04-25T08:48:28.341743+0000","last_became_peered":"2024-04-25T08:48:28.341743+0000","last_unstale":"2024-04-25T08:48:30.508608+0000","last_undegraded":"2024-04-25T08:48:30.508608+0000","last_fullsized":"2024-04-25T08:48:30.508608+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:00.804254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347393+0000","last_change":"2024-04-25T08:48:27.338445+0000","last_active":"2024-04-25T08:48:29.347393+0000","last_peered":"2024-04-25T08:48:29.347393+0000","last_clean":"2024-04-25T08:48:29.347393+0000","last_became_active":"2024-04-25T08:48:27.337954+0000","last_became_peered":"2024-04-25T08:48:27.337954+0000","last_unstale":"2024-04-25T08:48:29.347393+0000","last_undegraded":"2024-04-25T08:48:29.347393+0000","last_fullsized":"2024-04-25T08:48:29.347393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:52.412255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584359+0000","last_change":"2024-04-25T08:48:28.351053+0000","last_active":"2024-04-25T08:48:31.584359+0000","last_peered":"2024-04-25T08:48:31.584359+0000","last_clean":"2024-04-25T08:48:31.584359+0000","last_became_active":"2024-04-25T08:48:28.350593+0000","last_became_peered":"2024-04-25T08:48:28.350593+0000","last_unstale":"2024-04-25T08:48:31.584359+0000","last_undegraded":"2024-04-25T08:48:31.584359+0000","last_fullsized":"2024-04-25T08:48:31.584359+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:19.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584209+0000","last_change":"2024-04-25T08:48:27.346078+0000","last_active":"2024-04-25T08:48:31.584209+0000","last_peered":"2024-04-25T08:48:31.584209+0000","last_clean":"2024-04-25T08:48:31.584209+0000","last_became_active":"2024-04-25T08:48:27.345640+0000","last_became_peered":"2024-04-25T08:48:27.345640+0000","last_unstale":"2024-04-25T08:48:31.584209+0000","last_undegraded":"2024-04-25T08:48:31.584209+0000","last_fullsized":"2024-04-25T08:48:31.584209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:23.974715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508914+0000","last_change":"2024-04-25T08:48:28.349781+0000","last_active":"2024-04-25T08:48:30.508914+0000","last_peered":"2024-04-25T08:48:30.508914+0000","last_clean":"2024-04-25T08:48:30.508914+0000","last_became_active":"2024-04-25T08:48:28.348987+0000","last_became_peered":"2024-04-25T08:48:28.348987+0000","last_unstale":"2024-04-25T08:48:30.508914+0000","last_undegraded":"2024-04-25T08:48:30.508914+0000","last_fullsized":"2024-04-25T08:48:30.508914+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.323544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156426+0000","last_change":"2024-04-25T08:48:27.339219+0000","last_active":"2024-04-25T08:48:31.156426+0000","last_peered":"2024-04-25T08:48:31.156426+0000","last_clean":"2024-04-25T08:48:31.156426+0000","last_became_active":"2024-04-25T08:48:27.338860+0000","last_became_peered":"2024-04-25T08:48:27.338860+0000","last_unstale":"2024-04-25T08:48:31.156426+0000","last_undegraded":"2024-04-25T08:48:31.156426+0000","last_fullsized":"2024-04-25T08:48:31.156426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:35.932619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962709+0000","last_change":"2024-04-25T08:48:28.356994+0000","last_active":"2024-04-25T08:48:29.962709+0000","last_peered":"2024-04-25T08:48:29.962709+0000","last_clean":"2024-04-25T08:48:29.962709+0000","last_became_active":"2024-04-25T08:48:28.356707+0000","last_became_peered":"2024-04-25T08:48:28.356707+0000","last_unstale":"2024-04-25T08:48:29.962709+0000","last_undegraded":"2024-04-25T08:48:29.962709+0000","last_fullsized":"2024-04-25T08:48:29.962709+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:07.494358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.346905+0000","last_change":"2024-04-25T08:48:27.340351+0000","last_active":"2024-04-25T08:48:29.346905+0000","last_peered":"2024-04-25T08:48:29.346905+0000","last_clean":"2024-04-25T08:48:29.346905+0000","last_became_active":"2024-04-25T08:48:27.339980+0000","last_became_peered":"2024-04-25T08:48:27.339980+0000","last_unstale":"2024-04-25T08:48:29.346905+0000","last_undegraded":"2024-04-25T08:48:29.346905+0000","last_fullsized":"2024-04-25T08:48:29.346905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:07.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156534+0000","last_change":"2024-04-25T08:48:28.345518+0000","last_active":"2024-04-25T08:48:31.156534+0000","last_peered":"2024-04-25T08:48:31.156534+0000","last_clean":"2024-04-25T08:48:31.156534+0000","last_became_active":"2024-04-25T08:48:28.345217+0000","last_became_peered":"2024-04-25T08:48:28.345217+0000","last_unstale":"2024-04-25T08:48:31.156534+0000","last_undegraded":"2024-04-25T08:48:31.156534+0000","last_fullsized":"2024-04-25T08:48:31.156534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:48.911885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963091+0000","last_change":"2024-04-25T08:48:27.340987+0000","last_active":"2024-04-25T08:48:29.963091+0000","last_peered":"2024-04-25T08:48:29.963091+0000","last_clean":"2024-04-25T08:48:29.963091+0000","last_became_active":"2024-04-25T08:48:27.339959+0000","last_became_peered":"2024-04-25T08:48:27.339959+0000","last_unstale":"2024-04-25T08:48:29.963091+0000","last_undegraded":"2024-04-25T08:48:29.963091+0000","last_fullsized":"2024-04-25T08:48:29.963091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:36.942846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508461+0000","last_change":"2024-04-25T08:48:28.337697+0000","last_active":"2024-04-25T08:48:30.508461+0000","last_peered":"2024-04-25T08:48:30.508461+0000","last_clean":"2024-04-25T08:48:30.508461+0000","last_became_active":"2024-04-25T08:48:28.337382+0000","last_became_peered":"2024-04-25T08:48:28.337382+0000","last_unstale":"2024-04-25T08:48:30.508461+0000","last_undegraded":"2024-04-25T08:48:30.508461+0000","last_fullsized":"2024-04-25T08:48:30.508461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.864948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342266+0000","last_change":"2024-04-25T08:48:27.339259+0000","last_active":"2024-04-25T08:48:29.342266+0000","last_peered":"2024-04-25T08:48:29.342266+0000","last_clean":"2024-04-25T08:48:29.342266+0000","last_became_active":"2024-04-25T08:48:27.338960+0000","last_became_peered":"2024-04-25T08:48:27.338960+0000","last_unstale":"2024-04-25T08:48:29.342266+0000","last_undegraded":"2024-04-25T08:48:29.342266+0000","last_fullsized":"2024-04-25T08:48:29.342266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:24.287644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963327+0000","last_change":"2024-04-25T08:48:28.348669+0000","last_active":"2024-04-25T08:48:29.963327+0000","last_peered":"2024-04-25T08:48:29.963327+0000","last_clean":"2024-04-25T08:48:29.963327+0000","last_became_active":"2024-04-25T08:48:28.346047+0000","last_became_peered":"2024-04-25T08:48:28.346047+0000","last_unstale":"2024-04-25T08:48:29.963327+0000","last_undegraded":"2024-04-25T08:48:29.963327+0000","last_fullsized":"2024-04-25T08:48:29.963327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:51.751147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584028+0000","last_change":"2024-04-25T08:48:27.350508+0000","last_active":"2024-04-25T08:48:31.584028+0000","last_peered":"2024-04-25T08:48:31.584028+0000","last_clean":"2024-04-25T08:48:31.584028+0000","last_became_active":"2024-04-25T08:48:27.350229+0000","last_became_peered":"2024-04-25T08:48:27.350229+0000","last_unstale":"2024-04-25T08:48:31.584028+0000","last_undegraded":"2024-04-25T08:48:31.584028+0000","last_fullsized":"2024-04-25T08:48:31.584028+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:00.604109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347485+0000","last_change":"2024-04-25T08:48:28.355948+0000","last_active":"2024-04-25T08:48:29.347485+0000","last_peered":"2024-04-25T08:48:29.347485+0000","last_clean":"2024-04-25T08:48:29.347485+0000","last_became_active":"2024-04-25T08:48:28.355617+0000","last_became_peered":"2024-04-25T08:48:28.355617+0000","last_unstale":"2024-04-25T08:48:29.347485+0000","last_undegraded":"2024-04-25T08:48:29.347485+0000","last_fullsized":"2024-04-25T08:48:29.347485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:24.977760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.539000+0000","last_change":"2024-04-25T08:48:27.342956+0000","last_active":"2024-04-25T08:48:29.539000+0000","last_peered":"2024-04-25T08:48:29.539000+0000","last_clean":"2024-04-25T08:48:29.539000+0000","last_became_active":"2024-04-25T08:48:27.342647+0000","last_became_peered":"2024-04-25T08:48:27.342647+0000","last_unstale":"2024-04-25T08:48:29.539000+0000","last_undegraded":"2024-04-25T08:48:29.539000+0000","last_fullsized":"2024-04-25T08:48:29.539000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:10.524868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584439+0000","last_change":"2024-04-25T08:48:28.343183+0000","last_active":"2024-04-25T08:48:31.584439+0000","last_peered":"2024-04-25T08:48:31.584439+0000","last_clean":"2024-04-25T08:48:31.584439+0000","last_became_active":"2024-04-25T08:48:28.342775+0000","last_became_peered":"2024-04-25T08:48:28.342775+0000","last_unstale":"2024-04-25T08:48:31.584439+0000","last_undegraded":"2024-04-25T08:48:31.584439+0000","last_fullsized":"2024-04-25T08:48:31.584439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:04.197012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342698+0000","last_change":"2024-04-25T08:48:27.339364+0000","last_active":"2024-04-25T08:48:29.342698+0000","last_peered":"2024-04-25T08:48:29.342698+0000","last_clean":"2024-04-25T08:48:29.342698+0000","last_became_active":"2024-04-25T08:48:27.338534+0000","last_became_peered":"2024-04-25T08:48:27.338534+0000","last_unstale":"2024-04-25T08:48:29.342698+0000","last_undegraded":"2024-04-25T08:48:29.342698+0000","last_fullsized":"2024-04-25T08:48:29.342698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:25.591591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342650+0000","last_change":"2024-04-25T08:48:28.362972+0000","last_active":"2024-04-25T08:48:29.342650+0000","last_peered":"2024-04-25T08:48:29.342650+0000","last_clean":"2024-04-25T08:48:29.342650+0000","last_became_active":"2024-04-25T08:48:28.362578+0000","last_became_peered":"2024-04-25T08:48:28.362578+0000","last_unstale":"2024-04-25T08:48:29.342650+0000","last_undegraded":"2024-04-25T08:48:29.342650+0000","last_fullsized":"2024-04-25T08:48:29.342650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:40.770992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347444+0000","last_change":"2024-04-25T08:48:27.345486+0000","last_active":"2024-04-25T08:48:29.347444+0000","last_peered":"2024-04-25T08:48:29.347444+0000","last_clean":"2024-04-25T08:48:29.347444+0000","last_became_active":"2024-04-25T08:48:27.345228+0000","last_became_peered":"2024-04-25T08:48:27.345228+0000","last_unstale":"2024-04-25T08:48:29.347444+0000","last_undegraded":"2024-04-25T08:48:29.347444+0000","last_fullsized":"2024-04-25T08:48:29.347444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:20.590733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963016+0000","last_change":"2024-04-25T08:48:28.346177+0000","last_active":"2024-04-25T08:48:29.963016+0000","last_peered":"2024-04-25T08:48:29.963016+0000","last_clean":"2024-04-25T08:48:29.963016+0000","last_became_active":"2024-04-25T08:48:28.345873+0000","last_became_peered":"2024-04-25T08:48:28.345873+0000","last_unstale":"2024-04-25T08:48:29.963016+0000","last_undegraded":"2024-04-25T08:48:29.963016+0000","last_fullsized":"2024-04-25T08:48:29.963016+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:06.711222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508753+0000","last_change":"2024-04-25T08:48:27.344069+0000","last_active":"2024-04-25T08:48:30.508753+0000","last_peered":"2024-04-25T08:48:30.508753+0000","last_clean":"2024-04-25T08:48:30.508753+0000","last_became_active":"2024-04-25T08:48:27.343784+0000","last_became_peered":"2024-04-25T08:48:27.343784+0000","last_unstale":"2024-04-25T08:48:30.508753+0000","last_undegraded":"2024-04-25T08:48:30.508753+0000","last_fullsized":"2024-04-25T08:48:30.508753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:39.767947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962918+0000","last_change":"2024-04-25T08:48:28.337161+0000","last_active":"2024-04-25T08:48:29.962918+0000","last_peered":"2024-04-25T08:48:29.962918+0000","last_clean":"2024-04-25T08:48:29.962918+0000","last_became_active":"2024-04-25T08:48:28.336886+0000","last_became_peered":"2024-04-25T08:48:28.336886+0000","last_unstale":"2024-04-25T08:48:29.962918+0000","last_undegraded":"2024-04-25T08:48:29.962918+0000","last_fullsized":"2024-04-25T08:48:29.962918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:20.723874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584165+0000","last_change":"2024-04-25T08:48:27.350653+0000","last_active":"2024-04-25T08:48:31.584165+0000","last_peered":"2024-04-25T08:48:31.584165+0000","last_clean":"2024-04-25T08:48:31.584165+0000","last_became_active":"2024-04-25T08:48:27.350317+0000","last_became_peered":"2024-04-25T08:48:27.350317+0000","last_unstale":"2024-04-25T08:48:31.584165+0000","last_undegraded":"2024-04-25T08:48:31.584165+0000","last_fullsized":"2024-04-25T08:48:31.584165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:25.591591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.346667+0000","last_change":"2024-04-25T08:48:28.346681+0000","last_active":"2024-04-25T08:48:29.346667+0000","last_peered":"2024-04-25T08:48:29.346667+0000","last_clean":"2024-04-25T08:48:29.346667+0000","last_became_active":"2024-04-25T08:48:28.346413+0000","last_became_peered":"2024-04-25T08:48:28.346413+0000","last_unstale":"2024-04-25T08:48:29.346667+0000","last_undegraded":"2024-04-25T08:48:29.346667+0000","last_fullsized":"2024-04-25T08:48:29.346667+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:11.527913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508338+0000","last_change":"2024-04-25T08:48:27.348338+0000","last_active":"2024-04-25T08:48:30.508338+0000","last_peered":"2024-04-25T08:48:30.508338+0000","last_clean":"2024-04-25T08:48:30.508338+0000","last_became_active":"2024-04-25T08:48:27.348071+0000","last_became_peered":"2024-04-25T08:48:27.348071+0000","last_unstale":"2024-04-25T08:48:30.508338+0000","last_undegraded":"2024-04-25T08:48:30.508338+0000","last_fullsized":"2024-04-25T08:48:30.508338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.448083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344695+0000","last_change":"2024-04-25T08:48:28.346243+0000","last_active":"2024-04-25T08:48:29.344695+0000","last_peered":"2024-04-25T08:48:29.344695+0000","last_clean":"2024-04-25T08:48:29.344695+0000","last_became_active":"2024-04-25T08:48:28.345966+0000","last_became_peered":"2024-04-25T08:48:28.345966+0000","last_unstale":"2024-04-25T08:48:29.344695+0000","last_undegraded":"2024-04-25T08:48:29.344695+0000","last_fullsized":"2024-04-25T08:48:29.344695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:51.925566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962805+0000","last_change":"2024-04-25T08:48:27.335201+0000","last_active":"2024-04-25T08:48:29.962805+0000","last_peered":"2024-04-25T08:48:29.962805+0000","last_clean":"2024-04-25T08:48:29.962805+0000","last_became_active":"2024-04-25T08:48:27.334915+0000","last_became_peered":"2024-04-25T08:48:27.334915+0000","last_unstale":"2024-04-25T08:48:29.962805+0000","last_undegraded":"2024-04-25T08:48:29.962805+0000","last_fullsized":"2024-04-25T08:48:29.962805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:52.412255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344999+0000","last_change":"2024-04-25T08:48:28.346974+0000","last_active":"2024-04-25T08:48:29.344999+0000","last_peered":"2024-04-25T08:48:29.344999+0000","last_clean":"2024-04-25T08:48:29.344999+0000","last_became_active":"2024-04-25T08:48:28.346750+0000","last_became_peered":"2024-04-25T08:48:28.346750+0000","last_unstale":"2024-04-25T08:48:29.344999+0000","last_undegraded":"2024-04-25T08:48:29.344999+0000","last_fullsized":"2024-04-25T08:48:29.344999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.035383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963050+0000","last_change":"2024-04-25T08:48:27.340062+0000","last_active":"2024-04-25T08:48:29.963050+0000","last_peered":"2024-04-25T08:48:29.963050+0000","last_clean":"2024-04-25T08:48:29.963050+0000","last_became_active":"2024-04-25T08:48:27.339497+0000","last_became_peered":"2024-04-25T08:48:27.339497+0000","last_unstale":"2024-04-25T08:48:29.963050+0000","last_undegraded":"2024-04-25T08:48:29.963050+0000","last_fullsized":"2024-04-25T08:48:29.963050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:45.203569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508676+0000","last_change":"2024-04-25T08:48:28.338377+0000","last_active":"2024-04-25T08:48:30.508676+0000","last_peered":"2024-04-25T08:48:30.508676+0000","last_clean":"2024-04-25T08:48:30.508676+0000","last_became_active":"2024-04-25T08:48:28.337857+0000","last_became_peered":"2024-04-25T08:48:28.337857+0000","last_unstale":"2024-04-25T08:48:30.508676+0000","last_undegraded":"2024-04-25T08:48:30.508676+0000","last_fullsized":"2024-04-25T08:48:30.508676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:19.865786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156750+0000","last_change":"2024-04-25T08:48:27.343770+0000","last_active":"2024-04-25T08:48:31.156750+0000","last_peered":"2024-04-25T08:48:31.156750+0000","last_clean":"2024-04-25T08:48:31.156750+0000","last_became_active":"2024-04-25T08:48:27.343177+0000","last_became_peered":"2024-04-25T08:48:27.343177+0000","last_unstale":"2024-04-25T08:48:31.156750+0000","last_undegraded":"2024-04-25T08:48:31.156750+0000","last_fullsized":"2024-04-25T08:48:31.156750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:21.224599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156616+0000","last_change":"2024-04-25T08:48:28.348120+0000","last_active":"2024-04-25T08:48:31.156616+0000","last_peered":"2024-04-25T08:48:31.156616+0000","last_clean":"2024-04-25T08:48:31.156616+0000","last_became_active":"2024-04-25T08:48:28.347806+0000","last_became_peered":"2024-04-25T08:48:28.347806+0000","last_unstale":"2024-04-25T08:48:31.156616+0000","last_undegraded":"2024-04-25T08:48:31.156616+0000","last_fullsized":"2024-04-25T08:48:31.156616+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:51.751147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584321+0000","last_change":"2024-04-25T08:48:27.346516+0000","last_active":"2024-04-25T08:48:31.584321+0000","last_peered":"2024-04-25T08:48:31.584321+0000","last_clean":"2024-04-25T08:48:31.584321+0000","last_became_active":"2024-04-25T08:48:27.346121+0000","last_became_peered":"2024-04-25T08:48:27.346121+0000","last_unstale":"2024-04-25T08:48:31.584321+0000","last_undegraded":"2024-04-25T08:48:31.584321+0000","last_fullsized":"2024-04-25T08:48:31.584321+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:24.287644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347428+0000","last_change":"2024-04-25T08:48:28.342996+0000","last_active":"2024-04-25T08:48:29.347428+0000","last_peered":"2024-04-25T08:48:29.347428+0000","last_clean":"2024-04-25T08:48:29.347428+0000","last_became_active":"2024-04-25T08:48:28.342682+0000","last_became_peered":"2024-04-25T08:48:28.342682+0000","last_unstale":"2024-04-25T08:48:29.347428+0000","last_undegraded":"2024-04-25T08:48:29.347428+0000","last_fullsized":"2024-04-25T08:48:29.347428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:32.313567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347528+0000","last_change":"2024-04-25T08:48:27.340030+0000","last_active":"2024-04-25T08:48:29.347528+0000","last_peered":"2024-04-25T08:48:29.347528+0000","last_clean":"2024-04-25T08:48:29.347528+0000","last_became_active":"2024-04-25T08:48:27.339664+0000","last_became_peered":"2024-04-25T08:48:27.339664+0000","last_unstale":"2024-04-25T08:48:29.347528+0000","last_undegraded":"2024-04-25T08:48:29.347528+0000","last_fullsized":"2024-04-25T08:48:29.347528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:43.149870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508605+0000","last_change":"2024-04-25T08:48:28.335968+0000","last_active":"2024-04-25T08:48:30.508605+0000","last_peered":"2024-04-25T08:48:30.508605+0000","last_clean":"2024-04-25T08:48:30.508605+0000","last_became_active":"2024-04-25T08:48:28.335689+0000","last_became_peered":"2024-04-25T08:48:28.335689+0000","last_unstale":"2024-04-25T08:48:30.508605+0000","last_undegraded":"2024-04-25T08:48:30.508605+0000","last_fullsized":"2024-04-25T08:48:30.508605+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:30.409699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538644+0000","last_change":"2024-04-25T08:48:27.338924+0000","last_active":"2024-04-25T08:48:29.538644+0000","last_peered":"2024-04-25T08:48:29.538644+0000","last_clean":"2024-04-25T08:48:29.538644+0000","last_became_active":"2024-04-25T08:48:27.337616+0000","last_became_peered":"2024-04-25T08:48:27.337616+0000","last_unstale":"2024-04-25T08:48:29.538644+0000","last_undegraded":"2024-04-25T08:48:29.538644+0000","last_fullsized":"2024-04-25T08:48:29.538644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:21.224599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538780+0000","last_change":"2024-04-25T08:48:28.358445+0000","last_active":"2024-04-25T08:48:29.538780+0000","last_peered":"2024-04-25T08:48:29.538780+0000","last_clean":"2024-04-25T08:48:29.538780+0000","last_became_active":"2024-04-25T08:48:28.358170+0000","last_became_peered":"2024-04-25T08:48:28.358170+0000","last_unstale":"2024-04-25T08:48:29.538780+0000","last_undegraded":"2024-04-25T08:48:29.538780+0000","last_fullsized":"2024-04-25T08:48:29.538780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:32.313567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344872+0000","last_change":"2024-04-25T08:48:27.347646+0000","last_active":"2024-04-25T08:48:29.344872+0000","last_peered":"2024-04-25T08:48:29.344872+0000","last_clean":"2024-04-25T08:48:29.344872+0000","last_became_active":"2024-04-25T08:48:27.346992+0000","last_became_peered":"2024-04-25T08:48:27.346992+0000","last_unstale":"2024-04-25T08:48:29.344872+0000","last_undegraded":"2024-04-25T08:48:29.344872+0000","last_fullsized":"2024-04-25T08:48:29.344872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:12.132746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538955+0000","last_change":"2024-04-25T08:48:28.346752+0000","last_active":"2024-04-25T08:48:29.538955+0000","last_peered":"2024-04-25T08:48:29.538955+0000","last_clean":"2024-04-25T08:48:29.538955+0000","last_became_active":"2024-04-25T08:48:28.346471+0000","last_became_peered":"2024-04-25T08:48:28.346471+0000","last_unstale":"2024-04-25T08:48:29.538955+0000","last_undegraded":"2024-04-25T08:48:29.538955+0000","last_fullsized":"2024-04-25T08:48:29.538955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:25.290689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538659+0000","last_change":"2024-04-25T08:48:27.347782+0000","last_active":"2024-04-25T08:48:29.538659+0000","last_peered":"2024-04-25T08:48:29.538659+0000","last_clean":"2024-04-25T08:48:29.538659+0000","last_became_active":"2024-04-25T08:48:27.347506+0000","last_became_peered":"2024-04-25T08:48:27.347506+0000","last_unstale":"2024-04-25T08:48:29.538659+0000","last_undegraded":"2024-04-25T08:48:29.538659+0000","last_fullsized":"2024-04-25T08:48:29.538659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:57.632606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508679+0000","last_change":"2024-04-25T08:48:28.344296+0000","last_active":"2024-04-25T08:48:30.508679+0000","last_peered":"2024-04-25T08:48:30.508679+0000","last_clean":"2024-04-25T08:48:30.508679+0000","last_became_active":"2024-04-25T08:48:28.343965+0000","last_became_peered":"2024-04-25T08:48:28.343965+0000","last_unstale":"2024-04-25T08:48:30.508679+0000","last_undegraded":"2024-04-25T08:48:30.508679+0000","last_fullsized":"2024-04-25T08:48:30.508679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:28.074730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344668+0000","last_change":"2024-04-25T08:48:27.343639+0000","last_active":"2024-04-25T08:48:29.344668+0000","last_peered":"2024-04-25T08:48:29.344668+0000","last_clean":"2024-04-25T08:48:29.344668+0000","last_became_active":"2024-04-25T08:48:27.343377+0000","last_became_peered":"2024-04-25T08:48:27.343377+0000","last_unstale":"2024-04-25T08:48:29.344668+0000","last_undegraded":"2024-04-25T08:48:29.344668+0000","last_fullsized":"2024-04-25T08:48:29.344668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:55.281125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156667+0000","last_change":"2024-04-25T08:48:28.342848+0000","last_active":"2024-04-25T08:48:31.156667+0000","last_peered":"2024-04-25T08:48:31.156667+0000","last_clean":"2024-04-25T08:48:31.156667+0000","last_became_active":"2024-04-25T08:48:28.342553+0000","last_became_peered":"2024-04-25T08:48:28.342553+0000","last_unstale":"2024-04-25T08:48:31.156667+0000","last_undegraded":"2024-04-25T08:48:31.156667+0000","last_fullsized":"2024-04-25T08:48:31.156667+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:01.607154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584445+0000","last_change":"2024-04-25T08:48:27.343682+0000","last_active":"2024-04-25T08:48:31.584445+0000","last_peered":"2024-04-25T08:48:31.584445+0000","last_clean":"2024-04-25T08:48:31.584445+0000","last_became_active":"2024-04-25T08:48:27.343362+0000","last_became_peered":"2024-04-25T08:48:27.343362+0000","last_unstale":"2024-04-25T08:48:31.584445+0000","last_undegraded":"2024-04-25T08:48:31.584445+0000","last_fullsized":"2024-04-25T08:48:31.584445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:40.814921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508751+0000","last_change":"2024-04-25T08:48:28.335886+0000","last_active":"2024-04-25T08:48:30.508751+0000","last_peered":"2024-04-25T08:48:30.508751+0000","last_clean":"2024-04-25T08:48:30.508751+0000","last_became_active":"2024-04-25T08:48:28.335390+0000","last_became_peered":"2024-04-25T08:48:28.335390+0000","last_unstale":"2024-04-25T08:48:30.508751+0000","last_undegraded":"2024-04-25T08:48:30.508751+0000","last_fullsized":"2024-04-25T08:48:30.508751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:19.147713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347554+0000","last_change":"2024-04-25T08:48:27.339407+0000","last_active":"2024-04-25T08:48:29.347554+0000","last_peered":"2024-04-25T08:48:29.347554+0000","last_clean":"2024-04-25T08:48:29.347554+0000","last_became_active":"2024-04-25T08:48:27.339138+0000","last_became_peered":"2024-04-25T08:48:27.339138+0000","last_unstale":"2024-04-25T08:48:29.347554+0000","last_undegraded":"2024-04-25T08:48:29.347554+0000","last_fullsized":"2024-04-25T08:48:29.347554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:45.203569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508978+0000","last_change":"2024-04-25T08:48:28.343574+0000","last_active":"2024-04-25T08:48:30.508978+0000","last_peered":"2024-04-25T08:48:30.508978+0000","last_clean":"2024-04-25T08:48:30.508978+0000","last_became_active":"2024-04-25T08:48:28.343318+0000","last_became_peered":"2024-04-25T08:48:28.343318+0000","last_unstale":"2024-04-25T08:48:30.508978+0000","last_undegraded":"2024-04-25T08:48:30.508978+0000","last_fullsized":"2024-04-25T08:48:30.508978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:07.718365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347586+0000","last_change":"2024-04-25T08:48:27.345473+0000","last_active":"2024-04-25T08:48:29.347586+0000","last_peered":"2024-04-25T08:48:29.347586+0000","last_clean":"2024-04-25T08:48:29.347586+0000","last_became_active":"2024-04-25T08:48:27.344880+0000","last_became_peered":"2024-04-25T08:48:27.344880+0000","last_unstale":"2024-04-25T08:48:29.347586+0000","last_undegraded":"2024-04-25T08:48:29.347586+0000","last_fullsized":"2024-04-25T08:48:29.347586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:36.942846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.539013+0000","last_change":"2024-04-25T08:48:28.342117+0000","last_active":"2024-04-25T08:48:29.539013+0000","last_peered":"2024-04-25T08:48:29.539013+0000","last_clean":"2024-04-25T08:48:29.539013+0000","last_became_active":"2024-04-25T08:48:28.341824+0000","last_became_peered":"2024-04-25T08:48:28.341824+0000","last_unstale":"2024-04-25T08:48:29.539013+0000","last_undegraded":"2024-04-25T08:48:29.539013+0000","last_fullsized":"2024-04-25T08:48:29.539013+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:41.817966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156573+0000","last_change":"2024-04-25T08:48:27.342749+0000","last_active":"2024-04-25T08:48:31.156573+0000","last_peered":"2024-04-25T08:48:31.156573+0000","last_clean":"2024-04-25T08:48:31.156573+0000","last_became_active":"2024-04-25T08:48:27.342482+0000","last_became_peered":"2024-04-25T08:48:27.342482+0000","last_unstale":"2024-04-25T08:48:31.156573+0000","last_undegraded":"2024-04-25T08:48:31.156573+0000","last_fullsized":"2024-04-25T08:48:31.156573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:00.606870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344687+0000","last_change":"2024-04-25T08:48:28.341528+0000","last_active":"2024-04-25T08:48:29.344687+0000","last_peered":"2024-04-25T08:48:29.344687+0000","last_clean":"2024-04-25T08:48:29.344687+0000","last_became_active":"2024-04-25T08:48:28.341135+0000","last_became_peered":"2024-04-25T08:48:28.341135+0000","last_unstale":"2024-04-25T08:48:29.344687+0000","last_undegraded":"2024-04-25T08:48:29.344687+0000","last_fullsized":"2024-04-25T08:48:29.344687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:29.731744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508857+0000","last_change":"2024-04-25T08:48:27.343141+0000","last_active":"2024-04-25T08:48:30.508857+0000","last_peered":"2024-04-25T08:48:30.508857+0000","last_clean":"2024-04-25T08:48:30.508857+0000","last_became_active":"2024-04-25T08:48:27.342849+0000","last_became_peered":"2024-04-25T08:48:27.342849+0000","last_unstale":"2024-04-25T08:48:30.508857+0000","last_undegraded":"2024-04-25T08:48:30.508857+0000","last_fullsized":"2024-04-25T08:48:30.508857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:31.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342402+0000","last_change":"2024-04-25T08:48:28.363801+0000","last_active":"2024-04-25T08:48:29.342402+0000","last_peered":"2024-04-25T08:48:29.342402+0000","last_clean":"2024-04-25T08:48:29.342402+0000","last_became_active":"2024-04-25T08:48:28.363518+0000","last_became_peered":"2024-04-25T08:48:28.363518+0000","last_unstale":"2024-04-25T08:48:29.342402+0000","last_undegraded":"2024-04-25T08:48:29.342402+0000","last_fullsized":"2024-04-25T08:48:29.342402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:19.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538832+0000","last_change":"2024-04-25T08:48:27.341133+0000","last_active":"2024-04-25T08:48:29.538832+0000","last_peered":"2024-04-25T08:48:29.538832+0000","last_clean":"2024-04-25T08:48:29.538832+0000","last_became_active":"2024-04-25T08:48:27.340827+0000","last_became_peered":"2024-04-25T08:48:27.340827+0000","last_unstale":"2024-04-25T08:48:29.538832+0000","last_undegraded":"2024-04-25T08:48:29.538832+0000","last_fullsized":"2024-04-25T08:48:29.538832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:12.383863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156360+0000","last_change":"2024-04-25T08:48:28.350536+0000","last_active":"2024-04-25T08:48:31.156360+0000","last_peered":"2024-04-25T08:48:31.156360+0000","last_clean":"2024-04-25T08:48:31.156360+0000","last_became_active":"2024-04-25T08:48:28.350167+0000","last_became_peered":"2024-04-25T08:48:28.350167+0000","last_unstale":"2024-04-25T08:48:31.156360+0000","last_undegraded":"2024-04-25T08:48:31.156360+0000","last_fullsized":"2024-04-25T08:48:31.156360+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:06.711222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156533+0000","last_change":"2024-04-25T08:48:27.343760+0000","last_active":"2024-04-25T08:48:31.156533+0000","last_peered":"2024-04-25T08:48:31.156533+0000","last_clean":"2024-04-25T08:48:31.156533+0000","last_became_active":"2024-04-25T08:48:27.343391+0000","last_became_peered":"2024-04-25T08:48:27.343391+0000","last_unstale":"2024-04-25T08:48:31.156533+0000","last_undegraded":"2024-04-25T08:48:31.156533+0000","last_fullsized":"2024-04-25T08:48:31.156533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:25.679160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508807+0000","last_change":"2024-04-25T08:48:28.352574+0000","last_active":"2024-04-25T08:48:30.508807+0000","last_peered":"2024-04-25T08:48:30.508807+0000","last_clean":"2024-04-25T08:48:30.508807+0000","last_became_active":"2024-04-25T08:48:28.352270+0000","last_became_peered":"2024-04-25T08:48:28.352270+0000","last_unstale":"2024-04-25T08:48:30.508807+0000","last_undegraded":"2024-04-25T08:48:30.508807+0000","last_fullsized":"2024-04-25T08:48:30.508807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:22.862805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584065+0000","last_change":"2024-04-25T08:48:27.346720+0000","last_active":"2024-04-25T08:48:31.584065+0000","last_peered":"2024-04-25T08:48:31.584065+0000","last_clean":"2024-04-25T08:48:31.584065+0000","last_became_active":"2024-04-25T08:48:27.346424+0000","last_became_peered":"2024-04-25T08:48:27.346424+0000","last_unstale":"2024-04-25T08:48:31.584065+0000","last_undegraded":"2024-04-25T08:48:31.584065+0000","last_fullsized":"2024-04-25T08:48:31.584065+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:31.310522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584500+0000","last_change":"2024-04-25T08:48:28.356293+0000","last_active":"2024-04-25T08:48:31.584500+0000","last_peered":"2024-04-25T08:48:31.584500+0000","last_clean":"2024-04-25T08:48:31.584500+0000","last_became_active":"2024-04-25T08:48:28.355695+0000","last_became_peered":"2024-04-25T08:48:28.355695+0000","last_unstale":"2024-04-25T08:48:31.584500+0000","last_undegraded":"2024-04-25T08:48:31.584500+0000","last_fullsized":"2024-04-25T08:48:31.584500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:40.770992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963473+0000","last_change":"2024-04-25T08:48:27.337423+0000","last_active":"2024-04-25T08:48:29.963473+0000","last_peered":"2024-04-25T08:48:29.963473+0000","last_clean":"2024-04-25T08:48:29.963473+0000","last_became_active":"2024-04-25T08:48:27.337127+0000","last_became_peered":"2024-04-25T08:48:27.337127+0000","last_unstale":"2024-04-25T08:48:29.963473+0000","last_undegraded":"2024-04-25T08:48:29.963473+0000","last_fullsized":"2024-04-25T08:48:29.963473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:07.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":301,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584278+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:48:31.584278+0000","last_peered":"2024-04-25T08:48:31.584278+0000","last_clean":"2024-04-25T08:48:31.584278+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:48:31.584278+0000","last_undegraded":"2024-04-25T08:48:31.584278+0000","last_fullsized":"2024-04-25T08:48:31.584278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1712128,"data_stored":1101176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1130496,"data_stored":512968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1110016,"data_stored":510752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1118208,"data_stored":510912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1110016,"data_stored":510752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1110016,"data_stored":510752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":510822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1103438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:48:32.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:48:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.648+0000 7fdf6b000640 1 Processor -- start 2024-04-25T08:48:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf6b000640 1 -- start start 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf6b000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdf640afd40 0x7fdf640d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf6b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6406ed70 con 0x7fdf6415dca0 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf6b000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf64162530 con 0x7fdf640cf140 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf6b000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf640afb40 con 0x7fdf640afd40 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf5b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdf6415dca0 0x7fdf640d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50848/0 (socket says 172.21.15.46:50848) 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf5b400640 1 -- 172.21.15.46:0/216016711 learned_addr learned my addr 172.21.15.46:0/216016711 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf69000640 1 --2- 172.21.15.46:0/216016711 >> v2:172.21.15.82:3300/0 conn(0x7fdf640afd40 0x7fdf640d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720751711 0 0) 0x7fdf4c001120 con 0x7fdf6415dca0 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf48003530 con 0x7fdf6415dca0 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2968630216 0 0) 0x7fdf48003530 con 0x7fdf6415dca0 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 >> v2:172.21.15.82:3300/0 conn(0x7fdf640afd40 msgr2=0x7fdf640d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 --2- 172.21.15.46:0/216016711 >> v2:172.21.15.82:3300/0 conn(0x7fdf640afd40 0x7fdf640d0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 >> v1:172.21.15.82:6789/0 conn(0x7fdf640cf140 legacy=0x7fdf6415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64161770 con 0x7fdf6415dca0 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fdf4c001850 con 0x7fdf6415dca0 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 >> v1:172.21.15.46:6789/0 conn(0x7fdf6415dca0 legacy=0x7fdf640d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf48004810 0x7fdf48004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf480051d0 0x7fdf4800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640afb40 con 0x7fdf480039c0 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4800c170 con 0x7fdf480051d0 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4800c370 con 0x7fdf48004810 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63e00640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf48003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf5b400640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf48004810 0x7fdf48004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf69000640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf480051d0 0x7fdf4800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63e00640 1 -- 172.21.15.46:0/216016711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf48004810 msgr2=0x7fdf48004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:32.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63e00640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf48004810 0x7fdf48004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63e00640 1 -- 172.21.15.46:0/216016711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf480051d0 msgr2=0x7fdf4800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63e00640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf480051d0 0x7fdf4800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf69000640 1 -- 172.21.15.46:0/216016711 reap_dead start 2024-04-25T08:48:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.652+0000 7fdf63e00640 1 -- 172.21.15.46:0/216016711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf48003530 con 0x7fdf480039c0 2024-04-25T08:48:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf63e00640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf48003de0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fdf54001780 tx=0x7fdf5401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55d49080959f1b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540013a0 con 0x7fdf480039c0 2024-04-25T08:48:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf540052f0 con 0x7fdf480039c0 2024-04-25T08:48:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf63400640 1 -- 172.21.15.46:0/216016711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54003840 con 0x7fdf480039c0 2024-04-25T08:48:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf6b000640 1 -- 172.21.15.46:0/216016711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 msgr2=0x7fdf48003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf6b000640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf48003de0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fdf54001780 tx=0x7fdf5401c400 comp rx=0 tx=0).stop 2024-04-25T08:48:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf6b000640 1 -- 172.21.15.46:0/216016711 shutdown_connections 2024-04-25T08:48:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf6b000640 1 --2- 172.21.15.46:0/216016711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf48003de0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf6b000640 1 -- 172.21.15.46:0/216016711 >> 172.21.15.46:0/216016711 conn(0x7fdf640728c0 msgr2=0x7fdf64099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf6b000640 1 -- 172.21.15.46:0/216016711 shutdown_connections 2024-04-25T08:48:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.656+0000 7fdf6b000640 1 -- 172.21.15.46:0/216016711 wait complete. 2024-04-25T08:48:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 Processor -- start 2024-04-25T08:48:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 -- start start 2024-04-25T08:48:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf6415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf641f8020 0x7fdf641f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf640b1340 0x7fdf640b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6415d9a0 con 0x7fdf480039c0 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf640af7f0 con 0x7fdf641f8020 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6415db20 con 0x7fdf640b1340 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf6415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf63e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf6415f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36850/0 (socket says 172.21.15.46:36850) 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf63e00640 1 -- 172.21.15.46:0/4254239086 learned_addr learned my addr 172.21.15.46:0/4254239086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf69000640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf640b1340 0x7fdf640b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf5b400640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf641f8020 0x7fdf641f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf63e00640 1 -- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf640b1340 msgr2=0x7fdf640b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf63e00640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf640b1340 0x7fdf640b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf63e00640 1 -- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf641f8020 msgr2=0x7fdf641f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf63e00640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf641f8020 0x7fdf641f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf63e00640 1 -- 172.21.15.46:0/4254239086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64160af0 con 0x7fdf480039c0 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf5b400640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf641f8020 0x7fdf641f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf69000640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf640b1340 0x7fdf640b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf63e00640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf6415f5a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fdf5401c230 tx=0x7fdf48005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf60c00640 1 -- 172.21.15.46:0/4254239086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540b8020 con 0x7fdf480039c0 2024-04-25T08:48:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf60c00640 1 -- 172.21.15.46:0/4254239086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf540054a0 con 0x7fdf480039c0 2024-04-25T08:48:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf60c00640 1 -- 172.21.15.46:0/4254239086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf540013a0 con 0x7fdf480039c0 2024-04-25T08:48:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf64160dd0 con 0x7fdf480039c0 2024-04-25T08:48:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6415f080 con 0x7fdf480039c0 2024-04-25T08:48:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.660+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf64069080 con 0x7fdf480039c0 2024-04-25T08:48:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.664+0000 7fdf60c00640 1 -- 172.21.15.46:0/4254239086 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf540bb020 con 0x7fdf480039c0 2024-04-25T08:48:32.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.664+0000 7fdf60c00640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf40066f60 0x7fdf40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:32.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.664+0000 7fdf60c00640 1 -- 172.21.15.46:0/4254239086 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdf5411efa0 con 0x7fdf480039c0 2024-04-25T08:48:32.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.664+0000 7fdf5b400640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf40066f60 0x7fdf40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:32.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.664+0000 7fdf5b400640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf40066f60 0x7fdf40069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c001060 tx=0x7fdf4c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.668+0000 7fdf60c00640 1 -- 172.21.15.46:0/4254239086 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf540c2030 con 0x7fdf480039c0 2024-04-25T08:48:32.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.824+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf6415ca80 con 0x7fdf40066f60 2024-04-25T08:48:32.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.832+0000 7fdf60c00640 1 -- 172.21.15.46:0/4254239086 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252074 (secure 0 0 0) 0x7fdf6415ca80 con 0x7fdf40066f60 2024-04-25T08:48:32.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:32.830 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:48:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf40066f60 msgr2=0x7fdf40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf40066f60 0x7fdf40069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c001060 tx=0x7fdf4c009040 comp rx=0 tx=0).stop 2024-04-25T08:48:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 msgr2=0x7fdf6415f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf6415f5a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fdf5401c230 tx=0x7fdf48005840 comp rx=0 tx=0).stop 2024-04-25T08:48:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 shutdown_connections 2024-04-25T08:48:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdf40066f60 0x7fdf40069420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdf640b1340 0x7fdf640b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf641f8020 0x7fdf641f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 --2- 172.21.15.46:0/4254239086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf480039c0 0x7fdf6415f5a0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 >> 172.21.15.46:0/4254239086 conn(0x7fdf640728c0 msgr2=0x7fdf640d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 shutdown_connections 2024-04-25T08:48:32.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:32.836+0000 7fdf6b000640 1 -- 172.21.15.46:0/4254239086 wait complete. 2024-04-25T08:48:32.848 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-04-25T08:48:32.347326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259356,"kb_used_data":9896,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715364,"statfs":{"total":773094113280,"available":772828532736,"internally_reserved":0,"allocated":10133504,"data_stored":5271572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001256"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342679+0000","last_change":"2024-04-25T08:48:28.355459+0000","last_active":"2024-04-25T08:48:29.342679+0000","last_peered":"2024-04-25T08:48:29.342679+0000","last_clean":"2024-04-25T08:48:29.342679+0000","last_became_active":"2024-04-25T08:48:28.354742+0000","last_became_peered":"2024-04-25T08:48:28.354742+0000","last_unstale":"2024-04-25T08:48:29.342679+0000","last_undegraded":"2024-04-25T08:48:29.342679+0000","last_fullsized":"2024-04-25T08:48:29.342679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:18.648662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538928+0000","last_change":"2024-04-25T08:48:28.352992+0000","last_active":"2024-04-25T08:48:29.538928+0000","last_peered":"2024-04-25T08:48:29.538928+0000","last_clean":"2024-04-25T08:48:29.538928+0000","last_became_active":"2024-04-25T08:48:28.352699+0000","last_became_peered":"2024-04-25T08:48:28.352699+0000","last_unstale":"2024-04-25T08:48:29.538928+0000","last_undegraded":"2024-04-25T08:48:29.538928+0000","last_fullsized":"2024-04-25T08:48:29.538928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:24.977760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342391+0000","last_change":"2024-04-25T08:48:28.358791+0000","last_active":"2024-04-25T08:48:29.342391+0000","last_peered":"2024-04-25T08:48:29.342391+0000","last_clean":"2024-04-25T08:48:29.342391+0000","last_became_active":"2024-04-25T08:48:28.355698+0000","last_became_peered":"2024-04-25T08:48:28.355698+0000","last_unstale":"2024-04-25T08:48:29.342391+0000","last_undegraded":"2024-04-25T08:48:29.342391+0000","last_fullsized":"2024-04-25T08:48:29.342391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:41.718549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963347+0000","last_change":"2024-04-25T08:48:28.347034+0000","last_active":"2024-04-25T08:48:29.963347+0000","last_peered":"2024-04-25T08:48:29.963347+0000","last_clean":"2024-04-25T08:48:29.963347+0000","last_became_active":"2024-04-25T08:48:28.345969+0000","last_became_peered":"2024-04-25T08:48:28.345969+0000","last_unstale":"2024-04-25T08:48:29.963347+0000","last_undegraded":"2024-04-25T08:48:29.963347+0000","last_fullsized":"2024-04-25T08:48:29.963347+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:01.607154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347243+0000","last_change":"2024-04-25T08:48:28.339924+0000","last_active":"2024-04-25T08:48:29.347243+0000","last_peered":"2024-04-25T08:48:29.347243+0000","last_clean":"2024-04-25T08:48:29.347243+0000","last_became_active":"2024-04-25T08:48:28.339636+0000","last_became_peered":"2024-04-25T08:48:28.339636+0000","last_unstale":"2024-04-25T08:48:29.347243+0000","last_undegraded":"2024-04-25T08:48:29.347243+0000","last_fullsized":"2024-04-25T08:48:29.347243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:06.711222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508273+0000","last_change":"2024-04-25T08:48:28.337302+0000","last_active":"2024-04-25T08:48:30.508273+0000","last_peered":"2024-04-25T08:48:30.508273+0000","last_clean":"2024-04-25T08:48:30.508273+0000","last_became_active":"2024-04-25T08:48:28.337008+0000","last_became_peered":"2024-04-25T08:48:28.337008+0000","last_unstale":"2024-04-25T08:48:30.508273+0000","last_undegraded":"2024-04-25T08:48:30.508273+0000","last_fullsized":"2024-04-25T08:48:30.508273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:05.971261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584075+0000","last_change":"2024-04-25T08:48:28.356045+0000","last_active":"2024-04-25T08:48:31.584075+0000","last_peered":"2024-04-25T08:48:31.584075+0000","last_clean":"2024-04-25T08:48:31.584075+0000","last_became_active":"2024-04-25T08:48:28.355708+0000","last_became_peered":"2024-04-25T08:48:28.355708+0000","last_unstale":"2024-04-25T08:48:31.584075+0000","last_undegraded":"2024-04-25T08:48:31.584075+0000","last_fullsized":"2024-04-25T08:48:31.584075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:53.946686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962671+0000","last_change":"2024-04-25T08:48:28.339621+0000","last_active":"2024-04-25T08:48:29.962671+0000","last_peered":"2024-04-25T08:48:29.962671+0000","last_clean":"2024-04-25T08:48:29.962671+0000","last_became_active":"2024-04-25T08:48:28.339332+0000","last_became_peered":"2024-04-25T08:48:28.339332+0000","last_unstale":"2024-04-25T08:48:29.962671+0000","last_undegraded":"2024-04-25T08:48:29.962671+0000","last_fullsized":"2024-04-25T08:48:29.962671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:19.457397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347240+0000","last_change":"2024-04-25T08:48:28.346973+0000","last_active":"2024-04-25T08:48:29.347240+0000","last_peered":"2024-04-25T08:48:29.347240+0000","last_clean":"2024-04-25T08:48:29.347240+0000","last_became_active":"2024-04-25T08:48:28.345326+0000","last_became_peered":"2024-04-25T08:48:28.345326+0000","last_unstale":"2024-04-25T08:48:29.347240+0000","last_undegraded":"2024-04-25T08:48:29.347240+0000","last_fullsized":"2024-04-25T08:48:29.347240+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:19.698931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.345235+0000","last_change":"2024-04-25T08:48:28.340190+0000","last_active":"2024-04-25T08:48:29.345235+0000","last_peered":"2024-04-25T08:48:29.345235+0000","last_clean":"2024-04-25T08:48:29.345235+0000","last_became_active":"2024-04-25T08:48:28.339891+0000","last_became_peered":"2024-04-25T08:48:28.339891+0000","last_unstale":"2024-04-25T08:48:29.345235+0000","last_undegraded":"2024-04-25T08:48:29.345235+0000","last_fullsized":"2024-04-25T08:48:29.345235+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:18.175106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347383+0000","last_change":"2024-04-25T08:48:28.352521+0000","last_active":"2024-04-25T08:48:29.347383+0000","last_peered":"2024-04-25T08:48:29.347383+0000","last_clean":"2024-04-25T08:48:29.347383+0000","last_became_active":"2024-04-25T08:48:28.351876+0000","last_became_peered":"2024-04-25T08:48:28.351876+0000","last_unstale":"2024-04-25T08:48:29.347383+0000","last_undegraded":"2024-04-25T08:48:29.347383+0000","last_fullsized":"2024-04-25T08:48:29.347383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:01.607154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344987+0000","last_change":"2024-04-25T08:48:28.362256+0000","last_active":"2024-04-25T08:48:29.344987+0000","last_peered":"2024-04-25T08:48:29.344987+0000","last_clean":"2024-04-25T08:48:29.344987+0000","last_became_active":"2024-04-25T08:48:28.361983+0000","last_became_peered":"2024-04-25T08:48:28.361983+0000","last_unstale":"2024-04-25T08:48:29.344987+0000","last_undegraded":"2024-04-25T08:48:29.344987+0000","last_fullsized":"2024-04-25T08:48:29.344987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:21.105685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963315+0000","last_change":"2024-04-25T08:48:28.348932+0000","last_active":"2024-04-25T08:48:29.963315+0000","last_peered":"2024-04-25T08:48:29.963315+0000","last_clean":"2024-04-25T08:48:29.963315+0000","last_became_active":"2024-04-25T08:48:28.348052+0000","last_became_peered":"2024-04-25T08:48:28.348052+0000","last_unstale":"2024-04-25T08:48:29.963315+0000","last_undegraded":"2024-04-25T08:48:29.963315+0000","last_fullsized":"2024-04-25T08:48:29.963315+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:48.911885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347314+0000","last_change":"2024-04-25T08:48:28.349373+0000","last_active":"2024-04-25T08:48:29.347314+0000","last_peered":"2024-04-25T08:48:29.347314+0000","last_clean":"2024-04-25T08:48:29.347314+0000","last_became_active":"2024-04-25T08:48:28.349078+0000","last_became_peered":"2024-04-25T08:48:28.349078+0000","last_unstale":"2024-04-25T08:48:29.347314+0000","last_undegraded":"2024-04-25T08:48:29.347314+0000","last_fullsized":"2024-04-25T08:48:29.347314+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:48.911885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962981+0000","last_change":"2024-04-25T08:48:28.357145+0000","last_active":"2024-04-25T08:48:29.962981+0000","last_peered":"2024-04-25T08:48:29.962981+0000","last_clean":"2024-04-25T08:48:29.962981+0000","last_became_active":"2024-04-25T08:48:28.356861+0000","last_became_peered":"2024-04-25T08:48:28.356861+0000","last_unstale":"2024-04-25T08:48:29.962981+0000","last_undegraded":"2024-04-25T08:48:29.962981+0000","last_fullsized":"2024-04-25T08:48:29.962981+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:19.698931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508237+0000","last_change":"2024-04-25T08:48:28.356515+0000","last_active":"2024-04-25T08:48:30.508237+0000","last_peered":"2024-04-25T08:48:30.508237+0000","last_clean":"2024-04-25T08:48:30.508237+0000","last_became_active":"2024-04-25T08:48:28.355293+0000","last_became_peered":"2024-04-25T08:48:28.355293+0000","last_unstale":"2024-04-25T08:48:30.508237+0000","last_undegraded":"2024-04-25T08:48:30.508237+0000","last_fullsized":"2024-04-25T08:48:30.508237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:53.265772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508751+0000","last_change":"2024-04-25T08:48:28.353509+0000","last_active":"2024-04-25T08:48:30.508751+0000","last_peered":"2024-04-25T08:48:30.508751+0000","last_clean":"2024-04-25T08:48:30.508751+0000","last_became_active":"2024-04-25T08:48:28.353242+0000","last_became_peered":"2024-04-25T08:48:28.353242+0000","last_unstale":"2024-04-25T08:48:30.508751+0000","last_undegraded":"2024-04-25T08:48:30.508751+0000","last_fullsized":"2024-04-25T08:48:30.508751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:58.782234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342550+0000","last_change":"2024-04-25T08:48:28.358625+0000","last_active":"2024-04-25T08:48:29.342550+0000","last_peered":"2024-04-25T08:48:29.342550+0000","last_clean":"2024-04-25T08:48:29.342550+0000","last_became_active":"2024-04-25T08:48:28.358361+0000","last_became_peered":"2024-04-25T08:48:28.358361+0000","last_unstale":"2024-04-25T08:48:29.342550+0000","last_undegraded":"2024-04-25T08:48:29.342550+0000","last_fullsized":"2024-04-25T08:48:29.342550+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:04.197012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347283+0000","last_change":"2024-04-25T08:48:28.341057+0000","last_active":"2024-04-25T08:48:29.347283+0000","last_peered":"2024-04-25T08:48:29.347283+0000","last_clean":"2024-04-25T08:48:29.347283+0000","last_became_active":"2024-04-25T08:48:28.340710+0000","last_became_peered":"2024-04-25T08:48:28.340710+0000","last_unstale":"2024-04-25T08:48:29.347283+0000","last_undegraded":"2024-04-25T08:48:29.347283+0000","last_fullsized":"2024-04-25T08:48:29.347283+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:11.473349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963254+0000","last_change":"2024-04-25T08:48:28.356979+0000","last_active":"2024-04-25T08:48:29.963254+0000","last_peered":"2024-04-25T08:48:29.963254+0000","last_clean":"2024-04-25T08:48:29.963254+0000","last_became_active":"2024-04-25T08:48:28.356645+0000","last_became_peered":"2024-04-25T08:48:28.356645+0000","last_unstale":"2024-04-25T08:48:29.963254+0000","last_undegraded":"2024-04-25T08:48:29.963254+0000","last_fullsized":"2024-04-25T08:48:29.963254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:11.473349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.346802+0000","last_change":"2024-04-25T08:48:28.337712+0000","last_active":"2024-04-25T08:48:29.346802+0000","last_peered":"2024-04-25T08:48:29.346802+0000","last_clean":"2024-04-25T08:48:29.346802+0000","last_became_active":"2024-04-25T08:48:28.337374+0000","last_became_peered":"2024-04-25T08:48:28.337374+0000","last_unstale":"2024-04-25T08:48:29.346802+0000","last_undegraded":"2024-04-25T08:48:29.346802+0000","last_fullsized":"2024-04-25T08:48:29.346802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:07.494358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.346735+0000","last_change":"2024-04-25T08:48:28.335918+0000","last_active":"2024-04-25T08:48:29.346735+0000","last_peered":"2024-04-25T08:48:29.346735+0000","last_clean":"2024-04-25T08:48:29.346735+0000","last_became_active":"2024-04-25T08:48:28.335302+0000","last_became_peered":"2024-04-25T08:48:28.335302+0000","last_unstale":"2024-04-25T08:48:29.346735+0000","last_undegraded":"2024-04-25T08:48:29.346735+0000","last_fullsized":"2024-04-25T08:48:29.346735+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:19.457397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156467+0000","last_change":"2024-04-25T08:48:28.342192+0000","last_active":"2024-04-25T08:48:31.156467+0000","last_peered":"2024-04-25T08:48:31.156467+0000","last_clean":"2024-04-25T08:48:31.156467+0000","last_became_active":"2024-04-25T08:48:28.341893+0000","last_became_peered":"2024-04-25T08:48:28.341893+0000","last_unstale":"2024-04-25T08:48:31.156467+0000","last_undegraded":"2024-04-25T08:48:31.156467+0000","last_fullsized":"2024-04-25T08:48:31.156467+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:11.473349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342334+0000","last_change":"2024-04-25T08:48:28.353808+0000","last_active":"2024-04-25T08:48:29.342334+0000","last_peered":"2024-04-25T08:48:29.342334+0000","last_clean":"2024-04-25T08:48:29.342334+0000","last_became_active":"2024-04-25T08:48:28.353433+0000","last_became_peered":"2024-04-25T08:48:28.353433+0000","last_unstale":"2024-04-25T08:48:29.342334+0000","last_undegraded":"2024-04-25T08:48:29.342334+0000","last_fullsized":"2024-04-25T08:48:29.342334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:53.946686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538960+0000","last_change":"2024-04-25T08:48:28.346131+0000","last_active":"2024-04-25T08:48:29.538960+0000","last_peered":"2024-04-25T08:48:29.538960+0000","last_clean":"2024-04-25T08:48:29.538960+0000","last_became_active":"2024-04-25T08:48:28.345776+0000","last_became_peered":"2024-04-25T08:48:28.345776+0000","last_unstale":"2024-04-25T08:48:29.538960+0000","last_undegraded":"2024-04-25T08:48:29.538960+0000","last_fullsized":"2024-04-25T08:48:29.538960+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:26.594635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347324+0000","last_change":"2024-04-25T08:48:28.341255+0000","last_active":"2024-04-25T08:48:29.347324+0000","last_peered":"2024-04-25T08:48:29.347324+0000","last_clean":"2024-04-25T08:48:29.347324+0000","last_became_active":"2024-04-25T08:48:28.340957+0000","last_became_peered":"2024-04-25T08:48:28.340957+0000","last_unstale":"2024-04-25T08:48:29.347324+0000","last_undegraded":"2024-04-25T08:48:29.347324+0000","last_fullsized":"2024-04-25T08:48:29.347324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:51.751147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.345148+0000","last_change":"2024-04-25T08:48:28.348878+0000","last_active":"2024-04-25T08:48:29.345148+0000","last_peered":"2024-04-25T08:48:29.345148+0000","last_clean":"2024-04-25T08:48:29.345148+0000","last_became_active":"2024-04-25T08:48:28.348528+0000","last_became_peered":"2024-04-25T08:48:28.348528+0000","last_unstale":"2024-04-25T08:48:29.345148+0000","last_undegraded":"2024-04-25T08:48:29.345148+0000","last_fullsized":"2024-04-25T08:48:29.345148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:43.378571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963439+0000","last_change":"2024-04-25T08:48:28.339673+0000","last_active":"2024-04-25T08:48:29.963439+0000","last_peered":"2024-04-25T08:48:29.963439+0000","last_clean":"2024-04-25T08:48:29.963439+0000","last_became_active":"2024-04-25T08:48:28.339365+0000","last_became_peered":"2024-04-25T08:48:28.339365+0000","last_unstale":"2024-04-25T08:48:29.963439+0000","last_undegraded":"2024-04-25T08:48:29.963439+0000","last_fullsized":"2024-04-25T08:48:29.963439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:32.313567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584264+0000","last_change":"2024-04-25T08:48:28.344242+0000","last_active":"2024-04-25T08:48:31.584264+0000","last_peered":"2024-04-25T08:48:31.584264+0000","last_clean":"2024-04-25T08:48:31.584264+0000","last_became_active":"2024-04-25T08:48:28.343965+0000","last_became_peered":"2024-04-25T08:48:28.343965+0000","last_unstale":"2024-04-25T08:48:31.584264+0000","last_undegraded":"2024-04-25T08:48:31.584264+0000","last_fullsized":"2024-04-25T08:48:31.584264+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:41.718549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347140+0000","last_change":"2024-04-25T08:48:28.345531+0000","last_active":"2024-04-25T08:48:29.347140+0000","last_peered":"2024-04-25T08:48:29.347140+0000","last_clean":"2024-04-25T08:48:29.347140+0000","last_became_active":"2024-04-25T08:48:28.345214+0000","last_became_peered":"2024-04-25T08:48:28.345214+0000","last_unstale":"2024-04-25T08:48:29.347140+0000","last_undegraded":"2024-04-25T08:48:29.347140+0000","last_fullsized":"2024-04-25T08:48:29.347140+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:20.723874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538592+0000","last_change":"2024-04-25T08:48:28.344939+0000","last_active":"2024-04-25T08:48:29.538592+0000","last_peered":"2024-04-25T08:48:29.538592+0000","last_clean":"2024-04-25T08:48:29.538592+0000","last_became_active":"2024-04-25T08:48:28.344298+0000","last_became_peered":"2024-04-25T08:48:28.344298+0000","last_unstale":"2024-04-25T08:48:29.538592+0000","last_undegraded":"2024-04-25T08:48:29.538592+0000","last_fullsized":"2024-04-25T08:48:29.538592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:01.607154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508280+0000","last_change":"2024-04-25T08:48:28.355576+0000","last_active":"2024-04-25T08:48:30.508280+0000","last_peered":"2024-04-25T08:48:30.508280+0000","last_clean":"2024-04-25T08:48:30.508280+0000","last_became_active":"2024-04-25T08:48:28.354979+0000","last_became_peered":"2024-04-25T08:48:28.354979+0000","last_unstale":"2024-04-25T08:48:30.508280+0000","last_undegraded":"2024-04-25T08:48:30.508280+0000","last_fullsized":"2024-04-25T08:48:30.508280+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:26.509829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508821+0000","last_change":"2024-04-25T08:48:28.350985+0000","last_active":"2024-04-25T08:48:30.508821+0000","last_peered":"2024-04-25T08:48:30.508821+0000","last_clean":"2024-04-25T08:48:30.508821+0000","last_became_active":"2024-04-25T08:48:28.350675+0000","last_became_peered":"2024-04-25T08:48:28.350675+0000","last_unstale":"2024-04-25T08:48:30.508821+0000","last_undegraded":"2024-04-25T08:48:30.508821+0000","last_fullsized":"2024-04-25T08:48:30.508821+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:42.579967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"89'4","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156690+0000","last_change":"2024-04-25T08:48:27.344548+0000","last_active":"2024-04-25T08:48:31.156690+0000","last_peered":"2024-04-25T08:48:31.156690+0000","last_clean":"2024-04-25T08:48:31.156690+0000","last_became_active":"2024-04-25T08:48:27.343372+0000","last_became_peered":"2024-04-25T08:48:27.343372+0000","last_unstale":"2024-04-25T08:48:31.156690+0000","last_undegraded":"2024-04-25T08:48:31.156690+0000","last_fullsized":"2024-04-25T08:48:31.156690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:12.383863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508608+0000","last_change":"2024-04-25T08:48:28.342015+0000","last_active":"2024-04-25T08:48:30.508608+0000","last_peered":"2024-04-25T08:48:30.508608+0000","last_clean":"2024-04-25T08:48:30.508608+0000","last_became_active":"2024-04-25T08:48:28.341743+0000","last_became_peered":"2024-04-25T08:48:28.341743+0000","last_unstale":"2024-04-25T08:48:30.508608+0000","last_undegraded":"2024-04-25T08:48:30.508608+0000","last_fullsized":"2024-04-25T08:48:30.508608+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:00.804254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347393+0000","last_change":"2024-04-25T08:48:27.338445+0000","last_active":"2024-04-25T08:48:29.347393+0000","last_peered":"2024-04-25T08:48:29.347393+0000","last_clean":"2024-04-25T08:48:29.347393+0000","last_became_active":"2024-04-25T08:48:27.337954+0000","last_became_peered":"2024-04-25T08:48:27.337954+0000","last_unstale":"2024-04-25T08:48:29.347393+0000","last_undegraded":"2024-04-25T08:48:29.347393+0000","last_fullsized":"2024-04-25T08:48:29.347393+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:52.412255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584359+0000","last_change":"2024-04-25T08:48:28.351053+0000","last_active":"2024-04-25T08:48:31.584359+0000","last_peered":"2024-04-25T08:48:31.584359+0000","last_clean":"2024-04-25T08:48:31.584359+0000","last_became_active":"2024-04-25T08:48:28.350593+0000","last_became_peered":"2024-04-25T08:48:28.350593+0000","last_unstale":"2024-04-25T08:48:31.584359+0000","last_undegraded":"2024-04-25T08:48:31.584359+0000","last_fullsized":"2024-04-25T08:48:31.584359+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:19.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584209+0000","last_change":"2024-04-25T08:48:27.346078+0000","last_active":"2024-04-25T08:48:31.584209+0000","last_peered":"2024-04-25T08:48:31.584209+0000","last_clean":"2024-04-25T08:48:31.584209+0000","last_became_active":"2024-04-25T08:48:27.345640+0000","last_became_peered":"2024-04-25T08:48:27.345640+0000","last_unstale":"2024-04-25T08:48:31.584209+0000","last_undegraded":"2024-04-25T08:48:31.584209+0000","last_fullsized":"2024-04-25T08:48:31.584209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:23.974715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508914+0000","last_change":"2024-04-25T08:48:28.349781+0000","last_active":"2024-04-25T08:48:30.508914+0000","last_peered":"2024-04-25T08:48:30.508914+0000","last_clean":"2024-04-25T08:48:30.508914+0000","last_became_active":"2024-04-25T08:48:28.348987+0000","last_became_peered":"2024-04-25T08:48:28.348987+0000","last_unstale":"2024-04-25T08:48:30.508914+0000","last_undegraded":"2024-04-25T08:48:30.508914+0000","last_fullsized":"2024-04-25T08:48:30.508914+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.323544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156426+0000","last_change":"2024-04-25T08:48:27.339219+0000","last_active":"2024-04-25T08:48:31.156426+0000","last_peered":"2024-04-25T08:48:31.156426+0000","last_clean":"2024-04-25T08:48:31.156426+0000","last_became_active":"2024-04-25T08:48:27.338860+0000","last_became_peered":"2024-04-25T08:48:27.338860+0000","last_unstale":"2024-04-25T08:48:31.156426+0000","last_undegraded":"2024-04-25T08:48:31.156426+0000","last_fullsized":"2024-04-25T08:48:31.156426+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:35.932619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962709+0000","last_change":"2024-04-25T08:48:28.356994+0000","last_active":"2024-04-25T08:48:29.962709+0000","last_peered":"2024-04-25T08:48:29.962709+0000","last_clean":"2024-04-25T08:48:29.962709+0000","last_became_active":"2024-04-25T08:48:28.356707+0000","last_became_peered":"2024-04-25T08:48:28.356707+0000","last_unstale":"2024-04-25T08:48:29.962709+0000","last_undegraded":"2024-04-25T08:48:29.962709+0000","last_fullsized":"2024-04-25T08:48:29.962709+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:07.494358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.346905+0000","last_change":"2024-04-25T08:48:27.340351+0000","last_active":"2024-04-25T08:48:29.346905+0000","last_peered":"2024-04-25T08:48:29.346905+0000","last_clean":"2024-04-25T08:48:29.346905+0000","last_became_active":"2024-04-25T08:48:27.339980+0000","last_became_peered":"2024-04-25T08:48:27.339980+0000","last_unstale":"2024-04-25T08:48:29.346905+0000","last_undegraded":"2024-04-25T08:48:29.346905+0000","last_fullsized":"2024-04-25T08:48:29.346905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:07.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156534+0000","last_change":"2024-04-25T08:48:28.345518+0000","last_active":"2024-04-25T08:48:31.156534+0000","last_peered":"2024-04-25T08:48:31.156534+0000","last_clean":"2024-04-25T08:48:31.156534+0000","last_became_active":"2024-04-25T08:48:28.345217+0000","last_became_peered":"2024-04-25T08:48:28.345217+0000","last_unstale":"2024-04-25T08:48:31.156534+0000","last_undegraded":"2024-04-25T08:48:31.156534+0000","last_fullsized":"2024-04-25T08:48:31.156534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:48.911885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963091+0000","last_change":"2024-04-25T08:48:27.340987+0000","last_active":"2024-04-25T08:48:29.963091+0000","last_peered":"2024-04-25T08:48:29.963091+0000","last_clean":"2024-04-25T08:48:29.963091+0000","last_became_active":"2024-04-25T08:48:27.339959+0000","last_became_peered":"2024-04-25T08:48:27.339959+0000","last_unstale":"2024-04-25T08:48:29.963091+0000","last_undegraded":"2024-04-25T08:48:29.963091+0000","last_fullsized":"2024-04-25T08:48:29.963091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:36.942846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508461+0000","last_change":"2024-04-25T08:48:28.337697+0000","last_active":"2024-04-25T08:48:30.508461+0000","last_peered":"2024-04-25T08:48:30.508461+0000","last_clean":"2024-04-25T08:48:30.508461+0000","last_became_active":"2024-04-25T08:48:28.337382+0000","last_became_peered":"2024-04-25T08:48:28.337382+0000","last_unstale":"2024-04-25T08:48:30.508461+0000","last_undegraded":"2024-04-25T08:48:30.508461+0000","last_fullsized":"2024-04-25T08:48:30.508461+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:14.864948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342266+0000","last_change":"2024-04-25T08:48:27.339259+0000","last_active":"2024-04-25T08:48:29.342266+0000","last_peered":"2024-04-25T08:48:29.342266+0000","last_clean":"2024-04-25T08:48:29.342266+0000","last_became_active":"2024-04-25T08:48:27.338960+0000","last_became_peered":"2024-04-25T08:48:27.338960+0000","last_unstale":"2024-04-25T08:48:29.342266+0000","last_undegraded":"2024-04-25T08:48:29.342266+0000","last_fullsized":"2024-04-25T08:48:29.342266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:24.287644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963327+0000","last_change":"2024-04-25T08:48:28.348669+0000","last_active":"2024-04-25T08:48:29.963327+0000","last_peered":"2024-04-25T08:48:29.963327+0000","last_clean":"2024-04-25T08:48:29.963327+0000","last_became_active":"2024-04-25T08:48:28.346047+0000","last_became_peered":"2024-04-25T08:48:28.346047+0000","last_unstale":"2024-04-25T08:48:29.963327+0000","last_undegraded":"2024-04-25T08:48:29.963327+0000","last_fullsized":"2024-04-25T08:48:29.963327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:51.751147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584028+0000","last_change":"2024-04-25T08:48:27.350508+0000","last_active":"2024-04-25T08:48:31.584028+0000","last_peered":"2024-04-25T08:48:31.584028+0000","last_clean":"2024-04-25T08:48:31.584028+0000","last_became_active":"2024-04-25T08:48:27.350229+0000","last_became_peered":"2024-04-25T08:48:27.350229+0000","last_unstale":"2024-04-25T08:48:31.584028+0000","last_undegraded":"2024-04-25T08:48:31.584028+0000","last_fullsized":"2024-04-25T08:48:31.584028+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:00.604109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347485+0000","last_change":"2024-04-25T08:48:28.355948+0000","last_active":"2024-04-25T08:48:29.347485+0000","last_peered":"2024-04-25T08:48:29.347485+0000","last_clean":"2024-04-25T08:48:29.347485+0000","last_became_active":"2024-04-25T08:48:28.355617+0000","last_became_peered":"2024-04-25T08:48:28.355617+0000","last_unstale":"2024-04-25T08:48:29.347485+0000","last_undegraded":"2024-04-25T08:48:29.347485+0000","last_fullsized":"2024-04-25T08:48:29.347485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:24.977760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.539000+0000","last_change":"2024-04-25T08:48:27.342956+0000","last_active":"2024-04-25T08:48:29.539000+0000","last_peered":"2024-04-25T08:48:29.539000+0000","last_clean":"2024-04-25T08:48:29.539000+0000","last_became_active":"2024-04-25T08:48:27.342647+0000","last_became_peered":"2024-04-25T08:48:27.342647+0000","last_unstale":"2024-04-25T08:48:29.539000+0000","last_undegraded":"2024-04-25T08:48:29.539000+0000","last_fullsized":"2024-04-25T08:48:29.539000+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:10.524868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584439+0000","last_change":"2024-04-25T08:48:28.343183+0000","last_active":"2024-04-25T08:48:31.584439+0000","last_peered":"2024-04-25T08:48:31.584439+0000","last_clean":"2024-04-25T08:48:31.584439+0000","last_became_active":"2024-04-25T08:48:28.342775+0000","last_became_peered":"2024-04-25T08:48:28.342775+0000","last_unstale":"2024-04-25T08:48:31.584439+0000","last_undegraded":"2024-04-25T08:48:31.584439+0000","last_fullsized":"2024-04-25T08:48:31.584439+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:04.197012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342698+0000","last_change":"2024-04-25T08:48:27.339364+0000","last_active":"2024-04-25T08:48:29.342698+0000","last_peered":"2024-04-25T08:48:29.342698+0000","last_clean":"2024-04-25T08:48:29.342698+0000","last_became_active":"2024-04-25T08:48:27.338534+0000","last_became_peered":"2024-04-25T08:48:27.338534+0000","last_unstale":"2024-04-25T08:48:29.342698+0000","last_undegraded":"2024-04-25T08:48:29.342698+0000","last_fullsized":"2024-04-25T08:48:29.342698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:25.591591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342650+0000","last_change":"2024-04-25T08:48:28.362972+0000","last_active":"2024-04-25T08:48:29.342650+0000","last_peered":"2024-04-25T08:48:29.342650+0000","last_clean":"2024-04-25T08:48:29.342650+0000","last_became_active":"2024-04-25T08:48:28.362578+0000","last_became_peered":"2024-04-25T08:48:28.362578+0000","last_unstale":"2024-04-25T08:48:29.342650+0000","last_undegraded":"2024-04-25T08:48:29.342650+0000","last_fullsized":"2024-04-25T08:48:29.342650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:40.770992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"89'3","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347444+0000","last_change":"2024-04-25T08:48:27.345486+0000","last_active":"2024-04-25T08:48:29.347444+0000","last_peered":"2024-04-25T08:48:29.347444+0000","last_clean":"2024-04-25T08:48:29.347444+0000","last_became_active":"2024-04-25T08:48:27.345228+0000","last_became_peered":"2024-04-25T08:48:27.345228+0000","last_unstale":"2024-04-25T08:48:29.347444+0000","last_undegraded":"2024-04-25T08:48:29.347444+0000","last_fullsized":"2024-04-25T08:48:29.347444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:20.590733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963016+0000","last_change":"2024-04-25T08:48:28.346177+0000","last_active":"2024-04-25T08:48:29.963016+0000","last_peered":"2024-04-25T08:48:29.963016+0000","last_clean":"2024-04-25T08:48:29.963016+0000","last_became_active":"2024-04-25T08:48:28.345873+0000","last_became_peered":"2024-04-25T08:48:28.345873+0000","last_unstale":"2024-04-25T08:48:29.963016+0000","last_undegraded":"2024-04-25T08:48:29.963016+0000","last_fullsized":"2024-04-25T08:48:29.963016+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:06.711222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"89'6","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508753+0000","last_change":"2024-04-25T08:48:27.344069+0000","last_active":"2024-04-25T08:48:30.508753+0000","last_peered":"2024-04-25T08:48:30.508753+0000","last_clean":"2024-04-25T08:48:30.508753+0000","last_became_active":"2024-04-25T08:48:27.343784+0000","last_became_peered":"2024-04-25T08:48:27.343784+0000","last_unstale":"2024-04-25T08:48:30.508753+0000","last_undegraded":"2024-04-25T08:48:30.508753+0000","last_fullsized":"2024-04-25T08:48:30.508753+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:39.767947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962918+0000","last_change":"2024-04-25T08:48:28.337161+0000","last_active":"2024-04-25T08:48:29.962918+0000","last_peered":"2024-04-25T08:48:29.962918+0000","last_clean":"2024-04-25T08:48:29.962918+0000","last_became_active":"2024-04-25T08:48:28.336886+0000","last_became_peered":"2024-04-25T08:48:28.336886+0000","last_unstale":"2024-04-25T08:48:29.962918+0000","last_undegraded":"2024-04-25T08:48:29.962918+0000","last_fullsized":"2024-04-25T08:48:29.962918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:20.723874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584165+0000","last_change":"2024-04-25T08:48:27.350653+0000","last_active":"2024-04-25T08:48:31.584165+0000","last_peered":"2024-04-25T08:48:31.584165+0000","last_clean":"2024-04-25T08:48:31.584165+0000","last_became_active":"2024-04-25T08:48:27.350317+0000","last_became_peered":"2024-04-25T08:48:27.350317+0000","last_unstale":"2024-04-25T08:48:31.584165+0000","last_undegraded":"2024-04-25T08:48:31.584165+0000","last_fullsized":"2024-04-25T08:48:31.584165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:25.591591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.346667+0000","last_change":"2024-04-25T08:48:28.346681+0000","last_active":"2024-04-25T08:48:29.346667+0000","last_peered":"2024-04-25T08:48:29.346667+0000","last_clean":"2024-04-25T08:48:29.346667+0000","last_became_active":"2024-04-25T08:48:28.346413+0000","last_became_peered":"2024-04-25T08:48:28.346413+0000","last_unstale":"2024-04-25T08:48:29.346667+0000","last_undegraded":"2024-04-25T08:48:29.346667+0000","last_fullsized":"2024-04-25T08:48:29.346667+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:11.527913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508338+0000","last_change":"2024-04-25T08:48:27.348338+0000","last_active":"2024-04-25T08:48:30.508338+0000","last_peered":"2024-04-25T08:48:30.508338+0000","last_clean":"2024-04-25T08:48:30.508338+0000","last_became_active":"2024-04-25T08:48:27.348071+0000","last_became_peered":"2024-04-25T08:48:27.348071+0000","last_unstale":"2024-04-25T08:48:30.508338+0000","last_undegraded":"2024-04-25T08:48:30.508338+0000","last_fullsized":"2024-04-25T08:48:30.508338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:51.448083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344695+0000","last_change":"2024-04-25T08:48:28.346243+0000","last_active":"2024-04-25T08:48:29.344695+0000","last_peered":"2024-04-25T08:48:29.344695+0000","last_clean":"2024-04-25T08:48:29.344695+0000","last_became_active":"2024-04-25T08:48:28.345966+0000","last_became_peered":"2024-04-25T08:48:28.345966+0000","last_unstale":"2024-04-25T08:48:29.344695+0000","last_undegraded":"2024-04-25T08:48:29.344695+0000","last_fullsized":"2024-04-25T08:48:29.344695+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:51.925566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.962805+0000","last_change":"2024-04-25T08:48:27.335201+0000","last_active":"2024-04-25T08:48:29.962805+0000","last_peered":"2024-04-25T08:48:29.962805+0000","last_clean":"2024-04-25T08:48:29.962805+0000","last_became_active":"2024-04-25T08:48:27.334915+0000","last_became_peered":"2024-04-25T08:48:27.334915+0000","last_unstale":"2024-04-25T08:48:29.962805+0000","last_undegraded":"2024-04-25T08:48:29.962805+0000","last_fullsized":"2024-04-25T08:48:29.962805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:52.412255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344999+0000","last_change":"2024-04-25T08:48:28.346974+0000","last_active":"2024-04-25T08:48:29.344999+0000","last_peered":"2024-04-25T08:48:29.344999+0000","last_clean":"2024-04-25T08:48:29.344999+0000","last_became_active":"2024-04-25T08:48:28.346750+0000","last_became_peered":"2024-04-25T08:48:28.346750+0000","last_unstale":"2024-04-25T08:48:29.344999+0000","last_undegraded":"2024-04-25T08:48:29.344999+0000","last_fullsized":"2024-04-25T08:48:29.344999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:42.035383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"89'8","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963050+0000","last_change":"2024-04-25T08:48:27.340062+0000","last_active":"2024-04-25T08:48:29.963050+0000","last_peered":"2024-04-25T08:48:29.963050+0000","last_clean":"2024-04-25T08:48:29.963050+0000","last_became_active":"2024-04-25T08:48:27.339497+0000","last_became_peered":"2024-04-25T08:48:27.339497+0000","last_unstale":"2024-04-25T08:48:29.963050+0000","last_undegraded":"2024-04-25T08:48:29.963050+0000","last_fullsized":"2024-04-25T08:48:29.963050+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:45.203569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508676+0000","last_change":"2024-04-25T08:48:28.338377+0000","last_active":"2024-04-25T08:48:30.508676+0000","last_peered":"2024-04-25T08:48:30.508676+0000","last_clean":"2024-04-25T08:48:30.508676+0000","last_became_active":"2024-04-25T08:48:28.337857+0000","last_became_peered":"2024-04-25T08:48:28.337857+0000","last_unstale":"2024-04-25T08:48:30.508676+0000","last_undegraded":"2024-04-25T08:48:30.508676+0000","last_fullsized":"2024-04-25T08:48:30.508676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:19.865786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156750+0000","last_change":"2024-04-25T08:48:27.343770+0000","last_active":"2024-04-25T08:48:31.156750+0000","last_peered":"2024-04-25T08:48:31.156750+0000","last_clean":"2024-04-25T08:48:31.156750+0000","last_became_active":"2024-04-25T08:48:27.343177+0000","last_became_peered":"2024-04-25T08:48:27.343177+0000","last_unstale":"2024-04-25T08:48:31.156750+0000","last_undegraded":"2024-04-25T08:48:31.156750+0000","last_fullsized":"2024-04-25T08:48:31.156750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:21.224599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156616+0000","last_change":"2024-04-25T08:48:28.348120+0000","last_active":"2024-04-25T08:48:31.156616+0000","last_peered":"2024-04-25T08:48:31.156616+0000","last_clean":"2024-04-25T08:48:31.156616+0000","last_became_active":"2024-04-25T08:48:28.347806+0000","last_became_peered":"2024-04-25T08:48:28.347806+0000","last_unstale":"2024-04-25T08:48:31.156616+0000","last_undegraded":"2024-04-25T08:48:31.156616+0000","last_fullsized":"2024-04-25T08:48:31.156616+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:51.751147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584321+0000","last_change":"2024-04-25T08:48:27.346516+0000","last_active":"2024-04-25T08:48:31.584321+0000","last_peered":"2024-04-25T08:48:31.584321+0000","last_clean":"2024-04-25T08:48:31.584321+0000","last_became_active":"2024-04-25T08:48:27.346121+0000","last_became_peered":"2024-04-25T08:48:27.346121+0000","last_unstale":"2024-04-25T08:48:31.584321+0000","last_undegraded":"2024-04-25T08:48:31.584321+0000","last_fullsized":"2024-04-25T08:48:31.584321+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:24.287644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347428+0000","last_change":"2024-04-25T08:48:28.342996+0000","last_active":"2024-04-25T08:48:29.347428+0000","last_peered":"2024-04-25T08:48:29.347428+0000","last_clean":"2024-04-25T08:48:29.347428+0000","last_became_active":"2024-04-25T08:48:28.342682+0000","last_became_peered":"2024-04-25T08:48:28.342682+0000","last_unstale":"2024-04-25T08:48:29.347428+0000","last_undegraded":"2024-04-25T08:48:29.347428+0000","last_fullsized":"2024-04-25T08:48:29.347428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:32.313567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"89'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347528+0000","last_change":"2024-04-25T08:48:27.340030+0000","last_active":"2024-04-25T08:48:29.347528+0000","last_peered":"2024-04-25T08:48:29.347528+0000","last_clean":"2024-04-25T08:48:29.347528+0000","last_became_active":"2024-04-25T08:48:27.339664+0000","last_became_peered":"2024-04-25T08:48:27.339664+0000","last_unstale":"2024-04-25T08:48:29.347528+0000","last_undegraded":"2024-04-25T08:48:29.347528+0000","last_fullsized":"2024-04-25T08:48:29.347528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:43.149870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508605+0000","last_change":"2024-04-25T08:48:28.335968+0000","last_active":"2024-04-25T08:48:30.508605+0000","last_peered":"2024-04-25T08:48:30.508605+0000","last_clean":"2024-04-25T08:48:30.508605+0000","last_became_active":"2024-04-25T08:48:28.335689+0000","last_became_peered":"2024-04-25T08:48:28.335689+0000","last_unstale":"2024-04-25T08:48:30.508605+0000","last_undegraded":"2024-04-25T08:48:30.508605+0000","last_fullsized":"2024-04-25T08:48:30.508605+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:30.409699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538644+0000","last_change":"2024-04-25T08:48:27.338924+0000","last_active":"2024-04-25T08:48:29.538644+0000","last_peered":"2024-04-25T08:48:29.538644+0000","last_clean":"2024-04-25T08:48:29.538644+0000","last_became_active":"2024-04-25T08:48:27.337616+0000","last_became_peered":"2024-04-25T08:48:27.337616+0000","last_unstale":"2024-04-25T08:48:29.538644+0000","last_undegraded":"2024-04-25T08:48:29.538644+0000","last_fullsized":"2024-04-25T08:48:29.538644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:21.224599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538780+0000","last_change":"2024-04-25T08:48:28.358445+0000","last_active":"2024-04-25T08:48:29.538780+0000","last_peered":"2024-04-25T08:48:29.538780+0000","last_clean":"2024-04-25T08:48:29.538780+0000","last_became_active":"2024-04-25T08:48:28.358170+0000","last_became_peered":"2024-04-25T08:48:28.358170+0000","last_unstale":"2024-04-25T08:48:29.538780+0000","last_undegraded":"2024-04-25T08:48:29.538780+0000","last_fullsized":"2024-04-25T08:48:29.538780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:32.313567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344872+0000","last_change":"2024-04-25T08:48:27.347646+0000","last_active":"2024-04-25T08:48:29.344872+0000","last_peered":"2024-04-25T08:48:29.344872+0000","last_clean":"2024-04-25T08:48:29.344872+0000","last_became_active":"2024-04-25T08:48:27.346992+0000","last_became_peered":"2024-04-25T08:48:27.346992+0000","last_unstale":"2024-04-25T08:48:29.344872+0000","last_undegraded":"2024-04-25T08:48:29.344872+0000","last_fullsized":"2024-04-25T08:48:29.344872+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:12.132746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538955+0000","last_change":"2024-04-25T08:48:28.346752+0000","last_active":"2024-04-25T08:48:29.538955+0000","last_peered":"2024-04-25T08:48:29.538955+0000","last_clean":"2024-04-25T08:48:29.538955+0000","last_became_active":"2024-04-25T08:48:28.346471+0000","last_became_peered":"2024-04-25T08:48:28.346471+0000","last_unstale":"2024-04-25T08:48:29.538955+0000","last_undegraded":"2024-04-25T08:48:29.538955+0000","last_fullsized":"2024-04-25T08:48:29.538955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:25.290689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538659+0000","last_change":"2024-04-25T08:48:27.347782+0000","last_active":"2024-04-25T08:48:29.538659+0000","last_peered":"2024-04-25T08:48:29.538659+0000","last_clean":"2024-04-25T08:48:29.538659+0000","last_became_active":"2024-04-25T08:48:27.347506+0000","last_became_peered":"2024-04-25T08:48:27.347506+0000","last_unstale":"2024-04-25T08:48:29.538659+0000","last_undegraded":"2024-04-25T08:48:29.538659+0000","last_fullsized":"2024-04-25T08:48:29.538659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:57.632606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508679+0000","last_change":"2024-04-25T08:48:28.344296+0000","last_active":"2024-04-25T08:48:30.508679+0000","last_peered":"2024-04-25T08:48:30.508679+0000","last_clean":"2024-04-25T08:48:30.508679+0000","last_became_active":"2024-04-25T08:48:28.343965+0000","last_became_peered":"2024-04-25T08:48:28.343965+0000","last_unstale":"2024-04-25T08:48:30.508679+0000","last_undegraded":"2024-04-25T08:48:30.508679+0000","last_fullsized":"2024-04-25T08:48:30.508679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:28.074730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344668+0000","last_change":"2024-04-25T08:48:27.343639+0000","last_active":"2024-04-25T08:48:29.344668+0000","last_peered":"2024-04-25T08:48:29.344668+0000","last_clean":"2024-04-25T08:48:29.344668+0000","last_became_active":"2024-04-25T08:48:27.343377+0000","last_became_peered":"2024-04-25T08:48:27.343377+0000","last_unstale":"2024-04-25T08:48:29.344668+0000","last_undegraded":"2024-04-25T08:48:29.344668+0000","last_fullsized":"2024-04-25T08:48:29.344668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:55.281125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156667+0000","last_change":"2024-04-25T08:48:28.342848+0000","last_active":"2024-04-25T08:48:31.156667+0000","last_peered":"2024-04-25T08:48:31.156667+0000","last_clean":"2024-04-25T08:48:31.156667+0000","last_became_active":"2024-04-25T08:48:28.342553+0000","last_became_peered":"2024-04-25T08:48:28.342553+0000","last_unstale":"2024-04-25T08:48:31.156667+0000","last_undegraded":"2024-04-25T08:48:31.156667+0000","last_fullsized":"2024-04-25T08:48:31.156667+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:01.607154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"89'2","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584445+0000","last_change":"2024-04-25T08:48:27.343682+0000","last_active":"2024-04-25T08:48:31.584445+0000","last_peered":"2024-04-25T08:48:31.584445+0000","last_clean":"2024-04-25T08:48:31.584445+0000","last_became_active":"2024-04-25T08:48:27.343362+0000","last_became_peered":"2024-04-25T08:48:27.343362+0000","last_unstale":"2024-04-25T08:48:31.584445+0000","last_undegraded":"2024-04-25T08:48:31.584445+0000","last_fullsized":"2024-04-25T08:48:31.584445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:40.814921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508751+0000","last_change":"2024-04-25T08:48:28.335886+0000","last_active":"2024-04-25T08:48:30.508751+0000","last_peered":"2024-04-25T08:48:30.508751+0000","last_clean":"2024-04-25T08:48:30.508751+0000","last_became_active":"2024-04-25T08:48:28.335390+0000","last_became_peered":"2024-04-25T08:48:28.335390+0000","last_unstale":"2024-04-25T08:48:30.508751+0000","last_undegraded":"2024-04-25T08:48:30.508751+0000","last_fullsized":"2024-04-25T08:48:30.508751+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:19.147713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347554+0000","last_change":"2024-04-25T08:48:27.339407+0000","last_active":"2024-04-25T08:48:29.347554+0000","last_peered":"2024-04-25T08:48:29.347554+0000","last_clean":"2024-04-25T08:48:29.347554+0000","last_became_active":"2024-04-25T08:48:27.339138+0000","last_became_peered":"2024-04-25T08:48:27.339138+0000","last_unstale":"2024-04-25T08:48:29.347554+0000","last_undegraded":"2024-04-25T08:48:29.347554+0000","last_fullsized":"2024-04-25T08:48:29.347554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:45.203569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508978+0000","last_change":"2024-04-25T08:48:28.343574+0000","last_active":"2024-04-25T08:48:30.508978+0000","last_peered":"2024-04-25T08:48:30.508978+0000","last_clean":"2024-04-25T08:48:30.508978+0000","last_became_active":"2024-04-25T08:48:28.343318+0000","last_became_peered":"2024-04-25T08:48:28.343318+0000","last_unstale":"2024-04-25T08:48:30.508978+0000","last_undegraded":"2024-04-25T08:48:30.508978+0000","last_fullsized":"2024-04-25T08:48:30.508978+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:07.718365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.347586+0000","last_change":"2024-04-25T08:48:27.345473+0000","last_active":"2024-04-25T08:48:29.347586+0000","last_peered":"2024-04-25T08:48:29.347586+0000","last_clean":"2024-04-25T08:48:29.347586+0000","last_became_active":"2024-04-25T08:48:27.344880+0000","last_became_peered":"2024-04-25T08:48:27.344880+0000","last_unstale":"2024-04-25T08:48:29.347586+0000","last_undegraded":"2024-04-25T08:48:29.347586+0000","last_fullsized":"2024-04-25T08:48:29.347586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:36.942846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.539013+0000","last_change":"2024-04-25T08:48:28.342117+0000","last_active":"2024-04-25T08:48:29.539013+0000","last_peered":"2024-04-25T08:48:29.539013+0000","last_clean":"2024-04-25T08:48:29.539013+0000","last_became_active":"2024-04-25T08:48:28.341824+0000","last_became_peered":"2024-04-25T08:48:28.341824+0000","last_unstale":"2024-04-25T08:48:29.539013+0000","last_undegraded":"2024-04-25T08:48:29.539013+0000","last_fullsized":"2024-04-25T08:48:29.539013+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:41.817966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156573+0000","last_change":"2024-04-25T08:48:27.342749+0000","last_active":"2024-04-25T08:48:31.156573+0000","last_peered":"2024-04-25T08:48:31.156573+0000","last_clean":"2024-04-25T08:48:31.156573+0000","last_became_active":"2024-04-25T08:48:27.342482+0000","last_became_peered":"2024-04-25T08:48:27.342482+0000","last_unstale":"2024-04-25T08:48:31.156573+0000","last_undegraded":"2024-04-25T08:48:31.156573+0000","last_fullsized":"2024-04-25T08:48:31.156573+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:00.606870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.344687+0000","last_change":"2024-04-25T08:48:28.341528+0000","last_active":"2024-04-25T08:48:29.344687+0000","last_peered":"2024-04-25T08:48:29.344687+0000","last_clean":"2024-04-25T08:48:29.344687+0000","last_became_active":"2024-04-25T08:48:28.341135+0000","last_became_peered":"2024-04-25T08:48:28.341135+0000","last_unstale":"2024-04-25T08:48:29.344687+0000","last_undegraded":"2024-04-25T08:48:29.344687+0000","last_fullsized":"2024-04-25T08:48:29.344687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:29.731744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508857+0000","last_change":"2024-04-25T08:48:27.343141+0000","last_active":"2024-04-25T08:48:30.508857+0000","last_peered":"2024-04-25T08:48:30.508857+0000","last_clean":"2024-04-25T08:48:30.508857+0000","last_became_active":"2024-04-25T08:48:27.342849+0000","last_became_peered":"2024-04-25T08:48:27.342849+0000","last_unstale":"2024-04-25T08:48:30.508857+0000","last_undegraded":"2024-04-25T08:48:30.508857+0000","last_fullsized":"2024-04-25T08:48:30.508857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:31.215480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.342402+0000","last_change":"2024-04-25T08:48:28.363801+0000","last_active":"2024-04-25T08:48:29.342402+0000","last_peered":"2024-04-25T08:48:29.342402+0000","last_clean":"2024-04-25T08:48:29.342402+0000","last_became_active":"2024-04-25T08:48:28.363518+0000","last_became_peered":"2024-04-25T08:48:28.363518+0000","last_unstale":"2024-04-25T08:48:29.342402+0000","last_undegraded":"2024-04-25T08:48:29.342402+0000","last_fullsized":"2024-04-25T08:48:29.342402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:19.279166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.538832+0000","last_change":"2024-04-25T08:48:27.341133+0000","last_active":"2024-04-25T08:48:29.538832+0000","last_peered":"2024-04-25T08:48:29.538832+0000","last_clean":"2024-04-25T08:48:29.538832+0000","last_became_active":"2024-04-25T08:48:27.340827+0000","last_became_peered":"2024-04-25T08:48:27.340827+0000","last_unstale":"2024-04-25T08:48:29.538832+0000","last_undegraded":"2024-04-25T08:48:29.538832+0000","last_fullsized":"2024-04-25T08:48:29.538832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:12.383863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156360+0000","last_change":"2024-04-25T08:48:28.350536+0000","last_active":"2024-04-25T08:48:31.156360+0000","last_peered":"2024-04-25T08:48:31.156360+0000","last_clean":"2024-04-25T08:48:31.156360+0000","last_became_active":"2024-04-25T08:48:28.350167+0000","last_became_peered":"2024-04-25T08:48:28.350167+0000","last_unstale":"2024-04-25T08:48:31.156360+0000","last_undegraded":"2024-04-25T08:48:31.156360+0000","last_fullsized":"2024-04-25T08:48:31.156360+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:06.711222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.156533+0000","last_change":"2024-04-25T08:48:27.343760+0000","last_active":"2024-04-25T08:48:31.156533+0000","last_peered":"2024-04-25T08:48:31.156533+0000","last_clean":"2024-04-25T08:48:31.156533+0000","last_became_active":"2024-04-25T08:48:27.343391+0000","last_became_peered":"2024-04-25T08:48:27.343391+0000","last_unstale":"2024-04-25T08:48:31.156533+0000","last_undegraded":"2024-04-25T08:48:31.156533+0000","last_fullsized":"2024-04-25T08:48:31.156533+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:25.679160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:30.508807+0000","last_change":"2024-04-25T08:48:28.352574+0000","last_active":"2024-04-25T08:48:30.508807+0000","last_peered":"2024-04-25T08:48:30.508807+0000","last_clean":"2024-04-25T08:48:30.508807+0000","last_became_active":"2024-04-25T08:48:28.352270+0000","last_became_peered":"2024-04-25T08:48:28.352270+0000","last_unstale":"2024-04-25T08:48:30.508807+0000","last_undegraded":"2024-04-25T08:48:30.508807+0000","last_fullsized":"2024-04-25T08:48:30.508807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:22.862805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584065+0000","last_change":"2024-04-25T08:48:27.346720+0000","last_active":"2024-04-25T08:48:31.584065+0000","last_peered":"2024-04-25T08:48:31.584065+0000","last_clean":"2024-04-25T08:48:31.584065+0000","last_became_active":"2024-04-25T08:48:27.346424+0000","last_became_peered":"2024-04-25T08:48:27.346424+0000","last_unstale":"2024-04-25T08:48:31.584065+0000","last_undegraded":"2024-04-25T08:48:31.584065+0000","last_fullsized":"2024-04-25T08:48:31.584065+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:31.310522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584500+0000","last_change":"2024-04-25T08:48:28.356293+0000","last_active":"2024-04-25T08:48:31.584500+0000","last_peered":"2024-04-25T08:48:31.584500+0000","last_clean":"2024-04-25T08:48:31.584500+0000","last_became_active":"2024-04-25T08:48:28.355695+0000","last_became_peered":"2024-04-25T08:48:28.355695+0000","last_unstale":"2024-04-25T08:48:31.584500+0000","last_undegraded":"2024-04-25T08:48:31.584500+0000","last_fullsized":"2024-04-25T08:48:31.584500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:27.312709+0000","last_clean_scrub_stamp":"2024-04-25T08:48:27.312709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:40.770992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:29.963473+0000","last_change":"2024-04-25T08:48:27.337423+0000","last_active":"2024-04-25T08:48:29.963473+0000","last_peered":"2024-04-25T08:48:29.963473+0000","last_clean":"2024-04-25T08:48:29.963473+0000","last_became_active":"2024-04-25T08:48:27.337127+0000","last_became_peered":"2024-04-25T08:48:27.337127+0000","last_unstale":"2024-04-25T08:48:29.963473+0000","last_undegraded":"2024-04-25T08:48:29.963473+0000","last_fullsized":"2024-04-25T08:48:29.963473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:48:26.309664+0000","last_clean_scrub_stamp":"2024-04-25T08:48:26.309664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:07.134909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":301,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-25T08:48:31.584278+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:48:31.584278+0000","last_peered":"2024-04-25T08:48:31.584278+0000","last_clean":"2024-04-25T08:48:31.584278+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:48:31.584278+0000","last_undegraded":"2024-04-25T08:48:31.584278+0000","last_fullsized":"2024-04-25T08:48:31.584278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574929,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1712128,"data_stored":1101176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574929,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1130496,"data_stored":512968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1110016,"data_stored":510752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1118208,"data_stored":510912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574927,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1110016,"data_stored":510752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1110016,"data_stored":510752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":510822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1103438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:48:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:48:32.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:48:33.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.048+0000 7ff6d2e00640 1 Processor -- start 2024-04-25T08:48:33.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.048+0000 7ff6d2e00640 1 -- start start 2024-04-25T08:48:33.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d2e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc1440d0 0x7ff6cc0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d2e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff6cc06f150 0x7ff6cc0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d2e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc15e440 con 0x7ff6cc143990 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d2e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc15ed30 con 0x7ff6cc06f150 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d2e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc15e6e0 con 0x7ff6cc1440d0 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6cbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff6cc143990 0x7ff6cc06ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50850/0 (socket says 172.21.15.46:50850) 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6cbe00640 1 -- 172.21.15.46:0/207939911 learned_addr learned my addr 172.21.15.46:0/207939911 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d0e00640 1 --2- 172.21.15.46:0/207939911 >> v2:172.21.15.82:3300/0 conn(0x7ff6cc06f150 0x7ff6cc0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6c3400640 1 --2- 172.21.15.46:0/207939911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc1440d0 0x7ff6cc0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6cb400640 1 -- 172.21.15.46:0/207939911 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638137535 0 0) 0x7ff6cc15e440 con 0x7ff6cc143990 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6cb400640 1 -- 172.21.15.46:0/207939911 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6b0003530 con 0x7ff6cc143990 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d0e00640 1 -- 172.21.15.46:0/207939911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc1440d0 msgr2=0x7ff6cc0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d0e00640 1 --2- 172.21.15.46:0/207939911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc1440d0 0x7ff6cc0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d0e00640 1 -- 172.21.15.46:0/207939911 >> v1:172.21.15.46:6789/0 conn(0x7ff6cc143990 legacy=0x7ff6cc06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.052+0000 7ff6d0e00640 1 -- 172.21.15.46:0/207939911 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc15e9c0 con 0x7ff6cc06f150 2024-04-25T08:48:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d0e00640 1 --2- 172.21.15.46:0/207939911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6cc06f150 0x7ff6cc0b2720 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009d90 tx=0x7ff6c401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c346bc7ea4ea90dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6cb400640 1 -- 172.21.15.46:0/207939911 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c40b2070 con 0x7ff6cc06f150 2024-04-25T08:48:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6cb400640 1 -- 172.21.15.46:0/207939911 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c40a57a0 con 0x7ff6cc06f150 2024-04-25T08:48:33.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6cb400640 1 -- 172.21.15.46:0/207939911 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c40a5b70 con 0x7ff6cc06f150 2024-04-25T08:48:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 -- 172.21.15.46:0/207939911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6cc06f150 msgr2=0x7ff6cc0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 --2- 172.21.15.46:0/207939911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6cc06f150 0x7ff6cc0b2720 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009d90 tx=0x7ff6c401c8e0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 -- 172.21.15.46:0/207939911 shutdown_connections 2024-04-25T08:48:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 --2- 172.21.15.46:0/207939911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6cc06f150 0x7ff6cc0b2720 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 --2- 172.21.15.46:0/207939911 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc1440d0 0x7ff6cc0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 -- 172.21.15.46:0/207939911 >> 172.21.15.46:0/207939911 conn(0x7ff6cc06ac10 msgr2=0x7ff6cc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 -- 172.21.15.46:0/207939911 shutdown_connections 2024-04-25T08:48:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 -- 172.21.15.46:0/207939911 wait complete. 2024-04-25T08:48:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 Processor -- start 2024-04-25T08:48:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 -- start start 2024-04-25T08:48:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc06f150 0x7ff6cc142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6cc143990 0x7ff6cc15b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6cc1440d0 0x7ff6cc139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0b35f0 con 0x7ff6cc143990 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0b3470 con 0x7ff6cc1440d0 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6d2e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0b3770 con 0x7ff6cc06f150 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6cc143990 0x7ff6cc15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6cc143990 0x7ff6cc15b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36854/0 (socket says 172.21.15.46:36854) 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.056+0000 7ff6c3400640 1 -- 172.21.15.46:0/4159999861 learned_addr learned my addr 172.21.15.46:0/4159999861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6cbe00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc06f150 0x7ff6cc142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6d0e00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6cc1440d0 0x7ff6cc139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c3400640 1 -- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc06f150 msgr2=0x7ff6cc142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c3400640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc06f150 0x7ff6cc142730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c3400640 1 -- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6cc1440d0 msgr2=0x7ff6cc139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c3400640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6cc1440d0 0x7ff6cc139770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c3400640 1 -- 172.21.15.46:0/4159999861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc15c100 con 0x7ff6cc143990 2024-04-25T08:48:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6cbe00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc06f150 0x7ff6cc142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c3400640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6cc143990 0x7ff6cc15b800 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400e870 tx=0x7ff6b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c8c00640 1 -- 172.21.15.46:0/4159999861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4017070 con 0x7ff6cc143990 2024-04-25T08:48:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c8c00640 1 -- 172.21.15.46:0/4159999861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b400c940 con 0x7ff6cc143990 2024-04-25T08:48:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc15c3e0 con 0x7ff6cc143990 2024-04-25T08:48:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c8c00640 1 -- 172.21.15.46:0/4159999861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400ccf0 con 0x7ff6cc143990 2024-04-25T08:48:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc15c9a0 con 0x7ff6cc143990 2024-04-25T08:48:33.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc069010 con 0x7ff6cc143990 2024-04-25T08:48:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c8c00640 1 -- 172.21.15.46:0/4159999861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6b401a070 con 0x7ff6cc143990 2024-04-25T08:48:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c8c00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6a8066f60 0x7ff6a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.060+0000 7ff6c8c00640 1 -- 172.21.15.46:0/4159999861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6b40854f0 con 0x7ff6cc143990 2024-04-25T08:48:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.064+0000 7ff6cbe00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6a8066f60 0x7ff6a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.064+0000 7ff6cbe00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6a8066f60 0x7ff6a8069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc005890 tx=0x7ff6bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:33.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.064+0000 7ff6c8c00640 1 -- 172.21.15.46:0/4159999861 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff6b40562d0 con 0x7ff6cc143990 2024-04-25T08:48:33.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.284+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff6cc0ceeb0 con 0x7ff6cc143990 2024-04-25T08:48:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6c8c00640 1 -- 172.21.15.46:0/4159999861 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff6b4055bd0 con 0x7ff6cc143990 2024-04-25T08:48:33.281 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-25T08:48:33.281 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q== 2024-04-25T08:48:33.281 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-25T08:48:33.281 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-25T08:48:33.281 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:48:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-25T08:48:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6a8066f60 msgr2=0x7ff6a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6a8066f60 0x7ff6a8069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc005890 tx=0x7ff6bc009040 comp rx=0 tx=0).stop 2024-04-25T08:48:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6cc143990 msgr2=0x7ff6cc15b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6cc143990 0x7ff6cc15b800 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400e870 tx=0x7ff6b400ec70 comp rx=0 tx=0).stop 2024-04-25T08:48:33.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 shutdown_connections 2024-04-25T08:48:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6a8066f60 0x7ff6a8069420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6cc1440d0 0x7ff6cc139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6cc143990 0x7ff6cc15b800 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 --2- 172.21.15.46:0/4159999861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff6cc06f150 0x7ff6cc142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 >> 172.21.15.46:0/4159999861 conn(0x7ff6cc06ac10 msgr2=0x7ff6cc0b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 shutdown_connections 2024-04-25T08:48:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.288+0000 7ff6d2e00640 1 -- 172.21.15.46:0/4159999861 wait complete. 2024-04-25T08:48:33.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:33.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.448+0000 7f4987a00640 1 Processor -- start 2024-04-25T08:48:33.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.448+0000 7f4987a00640 1 -- start start 2024-04-25T08:48:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4987a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49800afd40 0x7f49800d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4987a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498006ed70 con 0x7f49800cf140 2024-04-25T08:48:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4987a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4980162530 con 0x7f498015dca0 2024-04-25T08:48:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4987a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49800afb40 con 0x7f49800afd40 2024-04-25T08:48:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4985a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49800afd40 0x7f49800d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4985a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49800afd40 0x7f49800d0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36860/0 (socket says 172.21.15.46:36860) 2024-04-25T08:48:33.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4985a00640 1 -- 172.21.15.46:0/2210740001 learned_addr learned my addr 172.21.15.46:0/2210740001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4985a00640 1 -- 172.21.15.46:0/2210740001 >> v1:172.21.15.82:6789/0 conn(0x7f498015dca0 legacy=0x7f49800d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4985a00640 1 -- 172.21.15.46:0/2210740001 >> v1:172.21.15.46:6789/0 conn(0x7f49800cf140 legacy=0x7f498015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.452+0000 7f4985a00640 1 -- 172.21.15.46:0/2210740001 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980161770 con 0x7f49800afd40 2024-04-25T08:48:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4985a00640 1 --2- 172.21.15.46:0/2210740001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800afd40 0x7f49800d0af0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f4978009e50 tx=0x7f497801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c682ab8c435443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f497fe00640 1 -- 172.21.15.46:0/2210740001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780a5860 con 0x7f49800afd40 2024-04-25T08:48:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f497fe00640 1 -- 172.21.15.46:0/2210740001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49780a5a00 con 0x7f49800afd40 2024-04-25T08:48:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f497fe00640 1 -- 172.21.15.46:0/2210740001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780a5dd0 con 0x7f49800afd40 2024-04-25T08:48:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4987a00640 1 -- 172.21.15.46:0/2210740001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800afd40 msgr2=0x7f49800d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4987a00640 1 --2- 172.21.15.46:0/2210740001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800afd40 0x7f49800d0af0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f4978009e50 tx=0x7f497801c9a0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4987a00640 1 -- 172.21.15.46:0/2210740001 shutdown_connections 2024-04-25T08:48:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4987a00640 1 --2- 172.21.15.46:0/2210740001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800afd40 0x7f49800d0af0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4987a00640 1 -- 172.21.15.46:0/2210740001 >> 172.21.15.46:0/2210740001 conn(0x7f49800728c0 msgr2=0x7f4980099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4987a00640 1 -- 172.21.15.46:0/2210740001 shutdown_connections 2024-04-25T08:48:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4987a00640 1 -- 172.21.15.46:0/2210740001 wait complete. 2024-04-25T08:48:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4987a00640 1 Processor -- start 2024-04-25T08:48:33.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.456+0000 7f4987a00640 1 -- start start 2024-04-25T08:48:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4987a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49800afd40 0x7f498014d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4987a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800cf140 0x7f4980153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4987a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498015dca0 0x7f4980151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4987a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800af7f0 con 0x7f49800cf140 2024-04-25T08:48:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4987a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800d1440 con 0x7f498015dca0 2024-04-25T08:48:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4987a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498006b600 con 0x7f49800afd40 2024-04-25T08:48:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4977e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800cf140 0x7f4980153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4977e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800cf140 0x7f4980153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36864/0 (socket says 172.21.15.46:36864) 2024-04-25T08:48:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4977e00640 1 -- 172.21.15.46:0/2649656465 learned_addr learned my addr 172.21.15.46:0/2649656465 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4985a00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498015dca0 0x7f4980151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4985000640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49800afd40 0x7f498014d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4977e00640 1 -- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49800afd40 msgr2=0x7f498014d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4977e00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49800afd40 0x7f498014d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4977e00640 1 -- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498015dca0 msgr2=0x7f4980151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4977e00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498015dca0 0x7f4980151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4977e00640 1 -- 172.21.15.46:0/2649656465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49801545c0 con 0x7f49800cf140 2024-04-25T08:48:33.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4977e00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800cf140 0x7f4980153b50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f496800c340 tx=0x7f496800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f497d600640 1 -- 172.21.15.46:0/2649656465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49680052d0 con 0x7f49800cf140 2024-04-25T08:48:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f497d600640 1 -- 172.21.15.46:0/2649656465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968005470 con 0x7f49800cf140 2024-04-25T08:48:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49801548a0 con 0x7f49800cf140 2024-04-25T08:48:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f497d600640 1 -- 172.21.15.46:0/2649656465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968007260 con 0x7f49800cf140 2024-04-25T08:48:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498014dc00 con 0x7f49800cf140 2024-04-25T08:48:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4980069080 con 0x7f49800cf140 2024-04-25T08:48:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.460+0000 7f497d600640 1 -- 172.21.15.46:0/2649656465 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4968007400 con 0x7f49800cf140 2024-04-25T08:48:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.464+0000 7f497d600640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f495c066f60 0x7f495c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.464+0000 7f497d600640 1 -- 172.21.15.46:0/2649656465 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49680845c0 con 0x7f49800cf140 2024-04-25T08:48:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.464+0000 7f4985000640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f495c066f60 0x7f495c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.464+0000 7f4985000640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f495c066f60 0x7f495c069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4980137870 tx=0x7f4970009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.468+0000 7f497d600640 1 -- 172.21.15.46:0/2649656465 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4968054de0 con 0x7f49800cf140 2024-04-25T08:48:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f498006e6a0 con 0x7f49800cf140 2024-04-25T08:48:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f497d600640 1 -- 172.21.15.46:0/2649656465 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4968058e80 con 0x7f49800cf140 2024-04-25T08:48:33.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:33.663 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:48:30.739960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1269610178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1269610178},{"type":"v1","addr":"172.21.15.46:6835","nonce":1269610178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:48:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 62 2024-04-25T08:48:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f495c066f60 msgr2=0x7f495c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f495c066f60 0x7f495c069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4980137870 tx=0x7f4970009040 comp rx=0 tx=0).stop 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800cf140 msgr2=0x7f4980153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800cf140 0x7f4980153b50 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f496800c340 tx=0x7f496800c800 comp rx=0 tx=0).stop 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 shutdown_connections 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f495c066f60 0x7f495c069420 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f498015dca0 0x7f4980151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49800cf140 0x7f4980153b50 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 --2- 172.21.15.46:0/2649656465 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49800afd40 0x7f498014d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 >> 172.21.15.46:0/2649656465 conn(0x7f49800728c0 msgr2=0x7f498015d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.668+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 shutdown_connections 2024-04-25T08:48:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:33.672+0000 7f4987a00640 1 -- 172.21.15.46:0/2649656465 wait complete. 2024-04-25T08:48:34.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:48:27.728697+0000', 'modified': '2024-04-25T08:48:30.739960+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7479': {'gid': 7479, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/1269610178', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1269610178}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1269610178}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:48:34.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:48:34.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:48:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.236+0000 7f6e94000640 1 Processor -- start 2024-04-25T08:48:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e94000640 1 -- start start 2024-04-25T08:48:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e94000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6e8c15a750 0x7f6e8c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e94000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e8c0cc9e0 0x7f6e8c06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e94000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e8c06d5e0 con 0x7f6e8c0aed10 2024-04-25T08:48:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e94000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8c15ab30 con 0x7f6e8c0cc9e0 2024-04-25T08:48:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e94000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8c0ce4f0 con 0x7f6e8c15a750 2024-04-25T08:48:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e92000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e8c0cc9e0 0x7f6e8c06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e92000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6e8c0cc9e0 0x7f6e8c06f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36868/0 (socket says 172.21.15.46:36868) 2024-04-25T08:48:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e92000640 1 -- 172.21.15.46:0/3410030304 learned_addr learned my addr 172.21.15.46:0/3410030304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e91600640 1 --2- 172.21.15.46:0/3410030304 >> v2:172.21.15.82:3300/0 conn(0x7f6e8c15a750 0x7f6e8c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e92000640 1 -- 172.21.15.46:0/3410030304 >> v2:172.21.15.82:3300/0 conn(0x7f6e8c15a750 msgr2=0x7f6e8c06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e92000640 1 --2- 172.21.15.46:0/3410030304 >> v2:172.21.15.82:3300/0 conn(0x7f6e8c15a750 0x7f6e8c06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e92000640 1 -- 172.21.15.46:0/3410030304 >> v1:172.21.15.82:6789/0 conn(0x7f6e8c0aed10 legacy=0x7f6e8c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.240+0000 7f6e92000640 1 -- 172.21.15.46:0/3410030304 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c0ce720 con 0x7f6e8c0cc9e0 2024-04-25T08:48:34.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e92000640 1 --2- 172.21.15.46:0/3410030304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0cc9e0 0x7f6e8c06f150 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c04ffd0 tx=0x7f6e8401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502ce062a3cc9ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e8be00640 1 -- 172.21.15.46:0/3410030304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e840a57a0 con 0x7f6e8c0cc9e0 2024-04-25T08:48:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e8be00640 1 -- 172.21.15.46:0/3410030304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e840a5940 con 0x7f6e8c0cc9e0 2024-04-25T08:48:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e8be00640 1 -- 172.21.15.46:0/3410030304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e840a5d10 con 0x7f6e8c0cc9e0 2024-04-25T08:48:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 -- 172.21.15.46:0/3410030304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0cc9e0 msgr2=0x7f6e8c06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 --2- 172.21.15.46:0/3410030304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0cc9e0 0x7f6e8c06f150 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c04ffd0 tx=0x7f6e8401c9f0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 -- 172.21.15.46:0/3410030304 shutdown_connections 2024-04-25T08:48:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 --2- 172.21.15.46:0/3410030304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0cc9e0 0x7f6e8c06f150 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 --2- 172.21.15.46:0/3410030304 >> v2:172.21.15.82:3300/0 conn(0x7f6e8c15a750 0x7f6e8c06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 -- 172.21.15.46:0/3410030304 >> 172.21.15.46:0/3410030304 conn(0x7f6e8c0b12a0 msgr2=0x7f6e8c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 -- 172.21.15.46:0/3410030304 shutdown_connections 2024-04-25T08:48:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 -- 172.21.15.46:0/3410030304 wait complete. 2024-04-25T08:48:34.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 Processor -- start 2024-04-25T08:48:34.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.244+0000 7f6e94000640 1 -- start start 2024-04-25T08:48:34.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e94000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0aed10 0x7f6e8c151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e94000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e8c0cc9e0 0x7f6e8c14ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e94000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e8c15a750 0x7f6e8c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e94000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c06db00 con 0x7f6e8c0aed10 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e94000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c06d270 con 0x7f6e8c15a750 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e94000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c06dc80 con 0x7f6e8c0cc9e0 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e91600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0aed10 0x7f6e8c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e91600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0aed10 0x7f6e8c151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36872/0 (socket says 172.21.15.46:36872) 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e91600640 1 -- 172.21.15.46:0/626467554 learned_addr learned my addr 172.21.15.46:0/626467554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e92000640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e8c15a750 0x7f6e8c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e90c00640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e8c0cc9e0 0x7f6e8c14ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e91600640 1 -- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e8c0cc9e0 msgr2=0x7f6e8c14ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e91600640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e8c0cc9e0 0x7f6e8c14ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e91600640 1 -- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e8c15a750 msgr2=0x7f6e8c14f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e91600640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e8c15a750 0x7f6e8c14f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e91600640 1 -- 172.21.15.46:0/626467554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c154030 con 0x7f6e8c0aed10 2024-04-25T08:48:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e91600640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0aed10 0x7f6e8c151460 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000d5f0 tx=0x7f6e8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e89600640 1 -- 172.21.15.46:0/626467554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e800125f0 con 0x7f6e8c0aed10 2024-04-25T08:48:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e89600640 1 -- 172.21.15.46:0/626467554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80019070 con 0x7f6e8c0aed10 2024-04-25T08:48:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e89600640 1 -- 172.21.15.46:0/626467554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e800129b0 con 0x7f6e8c0aed10 2024-04-25T08:48:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e94000640 1 -- 172.21.15.46:0/626467554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8c15ba60 con 0x7f6e8c0aed10 2024-04-25T08:48:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.248+0000 7f6e94000640 1 -- 172.21.15.46:0/626467554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c15c020 con 0x7f6e8c0aed10 2024-04-25T08:48:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e89600640 1 -- 172.21.15.46:0/626467554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e80012b50 con 0x7f6e8c0aed10 2024-04-25T08:48:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e89600640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e6c066f60 0x7f6e6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e89600640 1 -- 172.21.15.46:0/626467554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e800856b0 con 0x7f6e8c0aed10 2024-04-25T08:48:34.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e90c00640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e6c066f60 0x7f6e6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 Processor -- start 2024-04-25T08:48:34.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 -- start start 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e90c00640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e6c066f60 0x7f6e6c069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c1439c0 tx=0x7f6e78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fcd490 0x55e5e8f82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8f83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8e665f0 con 0x55e5e8fccb70 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8dcdda0 con 0x55e5e8f83340 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8d56150 con 0x55e5e8fcd490 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e91600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e91600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36882/0 (socket says 172.21.15.46:36882) 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e91600640 1 -- 172.21.15.46:0/1234416158 learned_addr learned my addr 172.21.15.46:0/1234416158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e91600640 1 -- 172.21.15.46:0/1234416158 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fcd490 msgr2=0x55e5e8f82e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e91600640 1 --2- 172.21.15.46:0/1234416158 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fcd490 0x55e5e8f82e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e91600640 1 -- 172.21.15.46:0/1234416158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 msgr2=0x55e5e8f83720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e91600640 1 --2- 172.21.15.46:0/1234416158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8f83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e91600640 1 -- 172.21.15.46:0/1234416158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e8d6fca0 con 0x55e5e8fccb70 2024-04-25T08:48:34.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e91600640 1 --2- 172.21.15.46:0/1234416158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fccf50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6e800838f0 tx=0x7f6e80055820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a56c146c1fcf942 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e7e000640 1 -- 172.21.15.46:0/1234416158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8005e4a0 con 0x55e5e8fccb70 2024-04-25T08:48:34.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e7e000640 1 -- 172.21.15.46:0/1234416158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80012e40 con 0x55e5e8fccb70 2024-04-25T08:48:34.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e7e000640 1 -- 172.21.15.46:0/1234416158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80054090 con 0x55e5e8fccb70 2024-04-25T08:48:34.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 -- 172.21.15.46:0/1234416158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 msgr2=0x55e5e8fccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.252+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1234416158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fccf50 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6e800838f0 tx=0x7f6e80055820 comp rx=0 tx=0).stop 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- 172.21.15.46:0/1234416158 shutdown_connections 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1234416158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8f83720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1234416158 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fcd490 0x55e5e8f82e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1234416158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fccf50 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- 172.21.15.46:0/1234416158 >> 172.21.15.46:0/1234416158 conn(0x55e5e8f852a0 msgr2=0x55e5e8f7d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- 172.21.15.46:0/1234416158 shutdown_connections 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- 172.21.15.46:0/1234416158 wait complete. 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- start start 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 Processor -- start 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8fd4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fd4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fcd490 0x55e5e8fd9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8d88990 con 0x55e5e8fccb70 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8dc9e40 con 0x55e5e8f83340 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8d59900 con 0x55e5e8fcd490 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 10 client.0 _ll_register_callbacks cb 0x55e5e8e811c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e90c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fd4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e90c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fd4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36894/0 (socket says 172.21.15.46:36894) 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e90c00640 1 -- 172.21.15.46:0/498842897 learned_addr learned my addr 172.21.15.46:0/498842897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e91600640 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8fd4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e92000640 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fcd490 0x55e5e8fd9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e90c00640 1 -- 172.21.15.46:0/498842897 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fcd490 msgr2=0x55e5e8fd9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e90c00640 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fcd490 0x55e5e8fd9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e90c00640 1 -- 172.21.15.46:0/498842897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 msgr2=0x55e5e8fd4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e90c00640 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8fd4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e90c00640 1 -- 172.21.15.46:0/498842897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5e8d86630 con 0x55e5e8fccb70 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e91600640 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8fd4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e90c00640 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fd4690 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6e78000f70 tx=0x7f6e7800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91cd4d4333e3ff77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e72a00640 1 -- 172.21.15.46:0/498842897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e780038d0 con 0x55e5e8fccb70 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- 172.21.15.46:0/498842897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e80012e40 con 0x55e5e8fccb70 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e72a00640 1 -- 172.21.15.46:0/498842897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e78023c60 con 0x55e5e8fccb70 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e72a00640 1 -- 172.21.15.46:0/498842897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e78023e60 con 0x55e5e8fccb70 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 10 client.7593 fetch_fsmap learned FSMap version 62 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- 172.21.15.46:0/498842897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55e5e8d563a0 con 0x55e5e8fccb70 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e72a00640 1 -- 172.21.15.46:0/498842897 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80012e40 con 0x55e5e8fccb70 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e72a00640 1 -- 172.21.15.46:0/498842897 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e7802a050 con 0x55e5e8fccb70 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 10 client.7593 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 10 client.7593 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:48:34.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 20 client.7593 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] conn(0x55e5e8fdbd20 0x55e5e8fde110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 4 client.7593 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:48:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.256+0000 7f6e9759e000 1 -- 172.21.15.46:0/498842897 --> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5e8d69a50 con 0x55e5e8fdbd20 2024-04-25T08:48:34.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e91600640 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] conn(0x55e5e8fdbd20 0x55e5e8fde110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e91600640 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] conn(0x55e5e8fdbd20 0x55e5e8fde110 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5562577f0134a5e4 server_cookie=c8b0f4a09058363e in_seq=0 out_seq=0 2024-04-25T08:48:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e72a00640 10 client.7593 ms_handle_connect on v2:172.21.15.46:6834/1269610178 2024-04-25T08:48:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e72a00640 1 -- 172.21.15.46:0/498842897 <== mds.0 v2:172.21.15.46:6834/1269610178 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5e8d69a50 con 0x55e5e8fdbd20 2024-04-25T08:48:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e72a00640 10 client.7593 handle_command_reply: tid=0 2024-04-25T08:48:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 client.7593 shutdown 2024-04-25T08:48:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 -- 172.21.15.46:0/498842897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 msgr2=0x55e5e8fd4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fd4690 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6e78000f70 tx=0x7f6e7800eb20 comp rx=0 tx=0).stop 2024-04-25T08:48:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 -- 172.21.15.46:0/498842897 shutdown_connections 2024-04-25T08:48:34.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] conn(0x55e5e8fdbd20 0x55e5e8fde110 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fcd490 0x55e5e8fd9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e8fd4690 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 --2- 172.21.15.46:0/498842897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8fd4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 -- 172.21.15.46:0/498842897 >> 172.21.15.46:0/498842897 conn(0x55e5e8f852a0 msgr2=0x55e5e8f846c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e72a00640 0 client.7593 ms_handle_reset on v2:172.21.15.46:6834/1269610178 2024-04-25T08:48:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 -- 172.21.15.46:0/498842897 shutdown_connections 2024-04-25T08:48:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 1 -- 172.21.15.46:0/498842897 wait complete. 2024-04-25T08:48:34.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.260+0000 7f6e9759e000 20 client.7593 trim_cache size 0 max 16384 2024-04-25T08:48:34.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e9759e000 1 Processor -- start 2024-04-25T08:48:34.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e9759e000 1 -- start start 2024-04-25T08:48:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8fcbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fccb70 0x55e5e8fcc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fcd490 0x55e5e8fdd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e9759e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c1fe1a0 con 0x55e5e8fcd490 2024-04-25T08:48:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e9759e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c15d370 con 0x55e5e8f83340 2024-04-25T08:48:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e9759e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c15d970 con 0x55e5e8fccb70 2024-04-25T08:48:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e92000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fcd490 0x55e5e8fdd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e92000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fcd490 0x55e5e8fdd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36896/0 (socket says 172.21.15.46:36896) 2024-04-25T08:48:34.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e92000640 1 -- 172.21.15.46:0/2912708425 learned_addr learned my addr 172.21.15.46:0/2912708425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e92000640 1 -- 172.21.15.46:0/2912708425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fccb70 msgr2=0x55e5e8fcc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e92000640 1 --2- 172.21.15.46:0/2912708425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fccb70 0x55e5e8fcc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e92000640 1 -- 172.21.15.46:0/2912708425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 msgr2=0x55e5e8fcbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e92000640 1 --2- 172.21.15.46:0/2912708425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8fcbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e92000640 1 -- 172.21.15.46:0/2912708425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e8f83170 con 0x55e5e8fcd490 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e92000640 1 --2- 172.21.15.46:0/2912708425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fcd490 0x55e5e8fdd110 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6e84019600 tx=0x7f6e84004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0dffb8e5ea5427 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e7d600640 1 -- 172.21.15.46:0/2912708425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e840bc070 con 0x55e5e8fcd490 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e7d600640 1 -- 172.21.15.46:0/2912708425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e840a57a0 con 0x55e5e8fcd490 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.276+0000 7f6e7d600640 1 -- 172.21.15.46:0/2912708425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e840a5b50 con 0x55e5e8fcd490 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- 172.21.15.46:0/2912708425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fcd490 msgr2=0x55e5e8fdd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 --2- 172.21.15.46:0/2912708425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fcd490 0x55e5e8fdd110 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6e84019600 tx=0x7f6e84004a50 comp rx=0 tx=0).stop 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- 172.21.15.46:0/2912708425 shutdown_connections 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 --2- 172.21.15.46:0/2912708425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fcd490 0x55e5e8fdd110 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 --2- 172.21.15.46:0/2912708425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8fccb70 0x55e5e8fcc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 --2- 172.21.15.46:0/2912708425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8f83340 0x55e5e8fcbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- 172.21.15.46:0/2912708425 >> 172.21.15.46:0/2912708425 conn(0x55e5e8f852a0 msgr2=0x55e5e8f7d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- 172.21.15.46:0/2912708425 shutdown_connections 2024-04-25T08:48:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- 172.21.15.46:0/2912708425 wait complete. 2024-04-25T08:48:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- start start 2024-04-25T08:48:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 Processor -- start 2024-04-25T08:48:34.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8f83340 0x55e5e8fe0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e902dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8fcd490 0x55e5e902e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8dd23d0 con 0x55e5e8fccb70 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8e89d50 con 0x55e5e8fcd490 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e8d68980 con 0x55e5e8f83340 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 10 client.0 _ll_register_callbacks cb 0x7f6e80055b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e90c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e902dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e90c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e902dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36906/0 (socket says 172.21.15.46:36906) 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e90c00640 1 -- 172.21.15.46:0/1380994174 learned_addr learned my addr 172.21.15.46:0/1380994174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e91600640 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8f83340 0x55e5e8fe0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e92000640 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8fcd490 0x55e5e902e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e90c00640 1 -- 172.21.15.46:0/1380994174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8f83340 msgr2=0x55e5e8fe0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e90c00640 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8f83340 0x55e5e8fe0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e90c00640 1 -- 172.21.15.46:0/1380994174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8fcd490 msgr2=0x55e5e902e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e90c00640 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8fcd490 0x55e5e902e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e90c00640 1 -- 172.21.15.46:0/1380994174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5e8e4f600 con 0x55e5e8fccb70 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e90c00640 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e902dba0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6e7802dad0 tx=0x7f6e78005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c61fcba58d0674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e72a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e72a00640 1 -- 172.21.15.46:0/1380994174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7803a060 con 0x55e5e8fccb70 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e72a00640 1 -- 172.21.15.46:0/1380994174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6e78003c30 con 0x55e5e8fccb70 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- 172.21.15.46:0/1380994174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e840a57a0 con 0x55e5e8fccb70 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e72a00640 1 -- 172.21.15.46:0/1380994174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e780038d0 con 0x55e5e8fccb70 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 10 client.7599 fetch_fsmap learned FSMap version 62 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.280+0000 7f6e9759e000 1 -- 172.21.15.46:0/1380994174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55e5e8d629b0 con 0x55e5e8fccb70 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e72a00640 1 -- 172.21.15.46:0/1380994174 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e840a57a0 con 0x55e5e8fccb70 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e72a00640 1 -- 172.21.15.46:0/1380994174 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6e7802a050 con 0x55e5e8fccb70 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 10 client.7599 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 10 client.7599 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 20 client.7599 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] conn(0x55e5e8fe0760 0x55e5e902e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 4 client.7599 mds_command: new command op to 7479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 -- 172.21.15.46:0/1380994174 --> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e5e8ddb2e0 con 0x55e5e8fe0760 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e91600640 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] conn(0x55e5e8fe0760 0x55e5e902e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e91600640 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] conn(0x55e5e8fe0760 0x55e5e902e620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af104c73f1d8364b server_cookie=1d57fff9836d0a5f in_seq=0 out_seq=0 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e72a00640 10 client.7599 ms_handle_connect on v2:172.21.15.46:6834/1269610178 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e72a00640 1 -- 172.21.15.46:0/1380994174 <== mds.0 v2:172.21.15.46:6834/1269610178 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e5e8ddb2e0 con 0x55e5e8fe0760 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e72a00640 10 client.7599 handle_command_reply: tid=0 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 client.7599 shutdown 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 -- 172.21.15.46:0/1380994174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 msgr2=0x55e5e902dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e902dba0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6e7802dad0 tx=0x7f6e78005000 comp rx=0 tx=0).stop 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 -- 172.21.15.46:0/1380994174 shutdown_connections 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.46:6834/1269610178,v1:172.21.15.46:6835/1269610178] conn(0x55e5e8fe0760 0x55e5e902e620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e5e8fcd490 0x55e5e902e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e5e8fccb70 0x55e5e902dba0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 --2- 172.21.15.46:0/1380994174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e5e8f83340 0x55e5e8fe0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e72a00640 0 client.7599 ms_handle_reset on v2:172.21.15.46:6834/1269610178 2024-04-25T08:48:34.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 -- 172.21.15.46:0/1380994174 >> 172.21.15.46:0/1380994174 conn(0x55e5e8f852a0 msgr2=0x55e5e8fcbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:34.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 -- 172.21.15.46:0/1380994174 shutdown_connections 2024-04-25T08:48:34.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 1 -- 172.21.15.46:0/1380994174 wait complete. 2024-04-25T08:48:34.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e9759e000 20 client.7599 trim_cache size 0 max 16384 2024-04-25T08:48:34.282 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:48:34.282 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:48:34.282 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7479, 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.9664197799999998, 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 62, 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 90, 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 90, 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 10.813372737 2024-04-25T08:48:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:48:34.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 -- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e6c066f60 msgr2=0x7f6e6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e6c066f60 0x7f6e6c069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c1439c0 tx=0x7f6e78009040 comp rx=0 tx=0).stop 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 -- 172.21.15.46:0/626467554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0aed10 msgr2=0x7f6e8c151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0aed10 0x7f6e8c151460 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6e8000d5f0 tx=0x7f6e8000dab0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 -- 172.21.15.46:0/626467554 shutdown_connections 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6e6c066f60 0x7f6e6c069420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e8c15a750 0x7f6e8c14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6e8c0cc9e0 0x7f6e8c14ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 --2- 172.21.15.46:0/626467554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e8c0aed10 0x7f6e8c151460 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 -- 172.21.15.46:0/626467554 >> 172.21.15.46:0/626467554 conn(0x7f6e8c0b12a0 msgr2=0x7f6e8c0afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 -- 172.21.15.46:0/626467554 shutdown_connections 2024-04-25T08:48:34.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.284+0000 7f6e94000640 1 -- 172.21.15.46:0/626467554 wait complete. 2024-04-25T08:48:34.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:48:34.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.444+0000 7fa5e7400640 1 Processor -- start 2024-04-25T08:48:34.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.444+0000 7fa5e7400640 1 -- start start 2024-04-25T08:48:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e7400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa5e015bdb0 0x7fa5e00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e7400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e00cda80 0x7fa5e015d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e7400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e00cc700 con 0x7fa5e0159230 2024-04-25T08:48:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e7400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e00cc9e0 con 0x7fa5e015bdb0 2024-04-25T08:48:34.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e7400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e00cccc0 con 0x7fa5e00cda80 2024-04-25T08:48:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5d7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa5e015bdb0 0x7fa5e00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e4a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa5e0159230 0x7fa5e00cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:50710/0 (socket says 172.21.15.46:50710) 2024-04-25T08:48:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e4a00640 1 -- 172.21.15.46:0/3904949623 learned_addr learned my addr 172.21.15.46:0/3904949623 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e5400640 1 --2- 172.21.15.46:0/3904949623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e00cda80 0x7fa5e015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5dfe00640 1 -- 172.21.15.46:0/3904949623 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742343124 0 0) 0x7fa5e00cc700 con 0x7fa5e0159230 2024-04-25T08:48:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5dfe00640 1 -- 172.21.15.46:0/3904949623 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5c4003530 con 0x7fa5e0159230 2024-04-25T08:48:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e5400640 1 -- 172.21.15.46:0/3904949623 >> v2:172.21.15.82:3300/0 conn(0x7fa5e015bdb0 msgr2=0x7fa5e00cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:34.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e5400640 1 --2- 172.21.15.46:0/3904949623 >> v2:172.21.15.82:3300/0 conn(0x7fa5e015bdb0 0x7fa5e00cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e5400640 1 -- 172.21.15.46:0/3904949623 >> v1:172.21.15.82:6789/0 conn(0x7fa5e0159230 legacy=0x7fa5e00cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5e5400640 1 -- 172.21.15.46:0/3904949623 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e00ccfa0 con 0x7fa5e00cda80 2024-04-25T08:48:34.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.448+0000 7fa5d7e00640 1 --2- 172.21.15.46:0/3904949623 >> v2:172.21.15.82:3300/0 conn(0x7fa5e015bdb0 0x7fa5e00cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:48:34.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e5400640 1 --2- 172.21.15.46:0/3904949623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e00cda80 0x7fa5e015d200 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009d90 tx=0x7fa5d801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=766f74dec507a731 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5dfe00640 1 -- 172.21.15.46:0/3904949623 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80b2070 con 0x7fa5e00cda80 2024-04-25T08:48:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5dfe00640 1 -- 172.21.15.46:0/3904949623 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d80a57a0 con 0x7fa5e00cda80 2024-04-25T08:48:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5dfe00640 1 -- 172.21.15.46:0/3904949623 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80a5b70 con 0x7fa5e00cda80 2024-04-25T08:48:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 -- 172.21.15.46:0/3904949623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e00cda80 msgr2=0x7fa5e015d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 --2- 172.21.15.46:0/3904949623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e00cda80 0x7fa5e015d200 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8009d90 tx=0x7fa5d801c8e0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 -- 172.21.15.46:0/3904949623 shutdown_connections 2024-04-25T08:48:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 --2- 172.21.15.46:0/3904949623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e00cda80 0x7fa5e015d200 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 --2- 172.21.15.46:0/3904949623 >> v2:172.21.15.82:3300/0 conn(0x7fa5e015bdb0 0x7fa5e00cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 -- 172.21.15.46:0/3904949623 >> 172.21.15.46:0/3904949623 conn(0x7fa5e00b12a0 msgr2=0x7fa5e00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 -- 172.21.15.46:0/3904949623 shutdown_connections 2024-04-25T08:48:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 -- 172.21.15.46:0/3904949623 wait complete. 2024-04-25T08:48:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 Processor -- start 2024-04-25T08:48:34.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 -- start start 2024-04-25T08:48:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5e00cda80 0x7fa5e013e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e0159230 0x7fa5e012ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e015bdb0 0x7fa5e0142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e015e320 con 0x7fa5e00cda80 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e015e1a0 con 0x7fa5e0159230 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.452+0000 7fa5e7400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e015e4a0 con 0x7fa5e015bdb0 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5e00cda80 0x7fa5e013e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5e00cda80 0x7fa5e013e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36918/0 (socket says 172.21.15.46:36918) 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e4a00640 1 -- 172.21.15.46:0/3716499607 learned_addr learned my addr 172.21.15.46:0/3716499607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5d7e00640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e0159230 0x7fa5e012ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e5400640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e015bdb0 0x7fa5e0142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e4a00640 1 -- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e015bdb0 msgr2=0x7fa5e0142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e4a00640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e015bdb0 0x7fa5e0142e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e4a00640 1 -- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e0159230 msgr2=0x7fa5e012ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e4a00640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e0159230 0x7fa5e012ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e4a00640 1 -- 172.21.15.46:0/3716499607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e01438f0 con 0x7fa5e00cda80 2024-04-25T08:48:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e4a00640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5e00cda80 0x7fa5e013e8b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000c3c0 tx=0x7fa5d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5dd600640 1 -- 172.21.15.46:0/3716499607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0017070 con 0x7fa5e00cda80 2024-04-25T08:48:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5dd600640 1 -- 172.21.15.46:0/3716499607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d0005cf0 con 0x7fa5e00cda80 2024-04-25T08:48:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e0143bd0 con 0x7fa5e00cda80 2024-04-25T08:48:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5dd600640 1 -- 172.21.15.46:0/3716499607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d00072f0 con 0x7fa5e00cda80 2024-04-25T08:48:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e0144190 con 0x7fa5e00cda80 2024-04-25T08:48:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e0069080 con 0x7fa5e00cda80 2024-04-25T08:48:34.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5dd600640 1 -- 172.21.15.46:0/3716499607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5d001a070 con 0x7fa5e00cda80 2024-04-25T08:48:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5dd600640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5bc066f60 0x7fa5bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.456+0000 7fa5dd600640 1 -- 172.21.15.46:0/3716499607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5d0084520 con 0x7fa5e00cda80 2024-04-25T08:48:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.460+0000 7fa5d7e00640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5bc066f60 0x7fa5bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.460+0000 7fa5d7e00640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5bc066f60 0x7fa5bc069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa5e01f78f0 tx=0x7fa5c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:34.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.460+0000 7fa5dd600640 1 -- 172.21.15.46:0/3716499607 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5d0055410 con 0x7fa5e00cda80 2024-04-25T08:48:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa5e0166550 con 0x7fa5e00cda80 2024-04-25T08:48:34.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5dd600640 1 -- 172.21.15.46:0/3716499607 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa5d0054d10 con 0x7fa5e00cda80 2024-04-25T08:48:34.655 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:48:34.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5bc066f60 msgr2=0x7fa5bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5bc066f60 0x7fa5bc069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa5e01f78f0 tx=0x7fa5c8008040 comp rx=0 tx=0).stop 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5e00cda80 msgr2=0x7fa5e013e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5e00cda80 0x7fa5e013e8b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000c3c0 tx=0x7fa5d000c880 comp rx=0 tx=0).stop 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 shutdown_connections 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5bc066f60 0x7fa5bc069420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5e015bdb0 0x7fa5e0142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5e0159230 0x7fa5e012ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 --2- 172.21.15.46:0/3716499607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5e00cda80 0x7fa5e013e8b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.660+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 >> 172.21.15.46:0/3716499607 conn(0x7fa5e00b12a0 msgr2=0x7fa5e00b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.664+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 shutdown_connections 2024-04-25T08:48:34.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:34.664+0000 7fa5e7400640 1 -- 172.21.15.46:0/3716499607 wait complete. 2024-04-25T08:48:34.676 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:48:34.676 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:48:34.676 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:48:34.676 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:48:34.676 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:34.676 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:48:34.676 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:48:34.676 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:48:34.676 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:48:34.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:34.677 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:48:34.687 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:48:34.687 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:48:34.687 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:48:34.687 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:34.687 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:48:34.687 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42088sec preferred_lft 42088sec 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:34.688 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:48:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:48:34.690 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:48:34.690 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:34.690 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:48:34.690 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:48:34.690 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:48:34.690 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:34.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:34.690 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:48:34.750 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:48:34.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:34.752 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:48:34.809 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:48:34.810 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:48:34.810 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:48:34.810 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:48:34.810 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:48:34.810 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:48:34.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:34.810 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:48:34.810 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:48:34.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:48:34.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:48:34.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:48:34.811 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:48:34.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:34.902 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:48:34.912 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:48:34.912 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:48:34.912 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:48:34.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:34.913 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:48:34.976 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:48:34.976 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:35.024 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:48:35.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:48:35.024 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:35.076 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:48:35.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:35.076 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:35.166 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:35.166 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:48:35.166 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:48:35.181 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:35.181 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:48:35.181 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:48:35.182 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:48:35.182 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:48:35.190 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:35.191 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:48:35.191 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:48:35.191 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:48:35.191 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:48:35.191 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:48:35.191 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:48:35.191 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:48:35.194 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:48:35.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:35.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:35.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:35.215 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:48:35.272 INFO:teuthology.orchestra.run.smithi046.stdout:client7273 2024-04-25T08:48:35.272 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:35.272 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:35.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:35.329 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-25T08:48:35.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:35.387 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-25T08:48:35.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:35.439 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-25T08:48:35.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:35.490 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-25T08:48:35.546 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:48:35.546 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:48:35.546 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:48:35.546 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:48:35.546 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-25T08:48:35.546 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:48:35.654 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-25T08:48:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:48:35.663 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:35.663 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:35.713 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:35.713 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:35.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:48:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d7ac00640 1 Processor -- start 2024-04-25T08:48:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d7ac00640 1 -- start start 2024-04-25T08:48:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d7ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d74099c60 0x7f8d740b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:35.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d7ac00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8d7409a040 0x7f8d740b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d7ac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8d7415b0d0 0x7f8d7415b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d7ac00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d740cc080 con 0x7f8d7409a040 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d7ac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d740cc360 con 0x7f8d7415b0d0 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d7ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740cc640 con 0x7f8d74099c60 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d78e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8d7409a040 0x7f8d740b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d78e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8d7409a040 0x7f8d740b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36934/0 (socket says 172.21.15.46:36934) 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d78e00640 1 -- 172.21.15.46:0/4236017271 learned_addr learned my addr 172.21.15.46:0/4236017271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d79800640 1 --2- 172.21.15.46:0/4236017271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d74099c60 0x7f8d740b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d7a200640 1 --2- 172.21.15.46:0/4236017271 >> v2:172.21.15.82:3300/0 conn(0x7f8d7415b0d0 0x7f8d7415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d78e00640 1 -- 172.21.15.46:0/4236017271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d74099c60 msgr2=0x7f8d740b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d78e00640 1 --2- 172.21.15.46:0/4236017271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d74099c60 0x7f8d740b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d78e00640 1 -- 172.21.15.46:0/4236017271 >> v2:172.21.15.82:3300/0 conn(0x7f8d7415b0d0 msgr2=0x7f8d7415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d78e00640 1 --2- 172.21.15.46:0/4236017271 >> v2:172.21.15.82:3300/0 conn(0x7f8d7415b0d0 0x7f8d7415b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.960+0000 7f8d78e00640 1 -- 172.21.15.46:0/4236017271 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d740cc920 con 0x7f8d7409a040 2024-04-25T08:48:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d78e00640 1 --2- 172.21.15.46:0/4236017271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7409a040 0x7f8d740b2950 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8d740d03c0 tx=0x7f8d6001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cc7a5057b34b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d73e00640 1 -- 172.21.15.46:0/4236017271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600ab650 con 0x7f8d7409a040 2024-04-25T08:48:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d73e00640 1 -- 172.21.15.46:0/4236017271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d600b0070 con 0x7f8d7409a040 2024-04-25T08:48:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d73e00640 1 -- 172.21.15.46:0/4236017271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600aba10 con 0x7f8d7409a040 2024-04-25T08:48:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 -- 172.21.15.46:0/4236017271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7409a040 msgr2=0x7f8d740b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/4236017271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7409a040 0x7f8d740b2950 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8d740d03c0 tx=0x7f8d6001c4f0 comp rx=0 tx=0).stop 2024-04-25T08:48:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 -- 172.21.15.46:0/4236017271 shutdown_connections 2024-04-25T08:48:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/4236017271 >> v2:172.21.15.82:3300/0 conn(0x7f8d7415b0d0 0x7f8d7415b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/4236017271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7409a040 0x7f8d740b2950 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/4236017271 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d74099c60 0x7f8d740b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 -- 172.21.15.46:0/4236017271 >> 172.21.15.46:0/4236017271 conn(0x7f8d740d1480 msgr2=0x7f8d740d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 -- 172.21.15.46:0/4236017271 shutdown_connections 2024-04-25T08:48:35.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 -- 172.21.15.46:0/4236017271 wait complete. 2024-04-25T08:48:35.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 Processor -- start 2024-04-25T08:48:35.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.964+0000 7f8d7ac00640 1 -- start start 2024-04-25T08:48:35.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d74099c60 0x7f8d740caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d7409a040 0x7f8d740c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7415b0d0 0x7f8d741f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b1cb0 con 0x7f8d7415b0d0 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b1b30 con 0x7f8d74099c60 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d740b1e30 con 0x7f8d7409a040 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7415b0d0 0x7f8d741f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7415b0d0 0x7f8d741f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36942/0 (socket says 172.21.15.46:36942) 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d79800640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d74099c60 0x7f8d740caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d78e00640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d7409a040 0x7f8d740c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7a200640 1 -- 172.21.15.46:0/3363006292 learned_addr learned my addr 172.21.15.46:0/3363006292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7a200640 1 -- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d7409a040 msgr2=0x7f8d740c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7a200640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d7409a040 0x7f8d740c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7a200640 1 -- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d74099c60 msgr2=0x7f8d740caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7a200640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d74099c60 0x7f8d740caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7a200640 1 -- 172.21.15.46:0/3363006292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d740cb450 con 0x7f8d7415b0d0 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7a200640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7415b0d0 0x7f8d741f6100 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c007e60 tx=0x7f8d6c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d71600640 1 -- 172.21.15.46:0/3363006292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c014420 con 0x7f8d7415b0d0 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d71600640 1 -- 172.21.15.46:0/3363006292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c019070 con 0x7f8d7415b0d0 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d71600640 1 -- 172.21.15.46:0/3363006292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c014820 con 0x7f8d7415b0d0 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7415d190 con 0x7f8d7415b0d0 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7415da30 con 0x7f8d7415b0d0 2024-04-25T08:48:35.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.968+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d74069010 con 0x7f8d7415b0d0 2024-04-25T08:48:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.972+0000 7f8d71600640 1 -- 172.21.15.46:0/3363006292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d6c0149c0 con 0x7f8d7415b0d0 2024-04-25T08:48:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.972+0000 7f8d71600640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d54066f90 0x7f8d54069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:35.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.972+0000 7f8d71600640 1 -- 172.21.15.46:0/3363006292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d6c085840 con 0x7f8d7415b0d0 2024-04-25T08:48:35.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.972+0000 7f8d79800640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d54066f90 0x7f8d54069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:35.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.972+0000 7f8d79800640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d54066f90 0x7f8d54069450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007a60 tx=0x7f8d68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:35.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:35.976+0000 7f8d71600640 1 -- 172.21.15.46:0/3363006292 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d6c056060 con 0x7f8d7415b0d0 2024-04-25T08:48:36.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.140+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d740c7720 con 0x7f8d7415b0d0 2024-04-25T08:48:36.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.140+0000 7f8d71600640 1 -- 172.21.15.46:0/3363006292 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16837 (secure 0 0 0) 0x7f8d6c05a100 con 0x7f8d7415b0d0 2024-04-25T08:48:36.135 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:36.136 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":90,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:48:29.334944+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:48:26.132790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:48:26.714570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:48:36.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d54066f90 msgr2=0x7f8d54069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d54066f90 0x7f8d54069450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8d68007a60 tx=0x7f8d68008040 comp rx=0 tx=0).stop 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7415b0d0 msgr2=0x7f8d741f6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7415b0d0 0x7f8d741f6100 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c007e60 tx=0x7f8d6c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 shutdown_connections 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8d54066f90 0x7f8d54069450 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d7415b0d0 0x7f8d741f6100 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8d7409a040 0x7f8d740c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 --2- 172.21.15.46:0/3363006292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d74099c60 0x7f8d740caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 >> 172.21.15.46:0/3363006292 conn(0x7f8d740d1480 msgr2=0x7f8d7406bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 shutdown_connections 2024-04-25T08:48:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.144+0000 7f8d7ac00640 1 -- 172.21.15.46:0/3363006292 wait complete. 2024-04-25T08:48:36.157 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:48:26.132790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:48:26.714570+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:48:36.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:48:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c7a00640 1 Processor -- start 2024-04-25T08:48:36.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c7a00640 1 -- start start 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c7a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86c0160f70 0x7f86c00d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c7a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006bd20 0x7f86c0162130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c7a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f86c006c3e0 0x7f86c006c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c7a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c006cd90 con 0x7f86c0160f70 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c7a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c0162670 con 0x7f86c006c3e0 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c7a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00cfa40 con 0x7f86c006bd20 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86c0160f70 0x7f86c00d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86c0160f70 0x7f86c00d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36944/0 (socket says 172.21.15.46:36944) 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5000640 1 -- 172.21.15.46:0/1154956211 learned_addr learned my addr 172.21.15.46:0/1154956211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86b7e00640 1 --2- 172.21.15.46:0/1154956211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006bd20 0x7f86c0162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5a00640 1 --2- 172.21.15.46:0/1154956211 >> v2:172.21.15.82:3300/0 conn(0x7f86c006c3e0 0x7f86c006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5000640 1 -- 172.21.15.46:0/1154956211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006bd20 msgr2=0x7f86c0162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5000640 1 --2- 172.21.15.46:0/1154956211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006bd20 0x7f86c0162130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5000640 1 -- 172.21.15.46:0/1154956211 >> v2:172.21.15.82:3300/0 conn(0x7f86c006c3e0 msgr2=0x7f86c006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5000640 1 --2- 172.21.15.46:0/1154956211 >> v2:172.21.15.82:3300/0 conn(0x7f86c006c3e0 0x7f86c006c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5000640 1 -- 172.21.15.46:0/1154956211 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c00cfce0 con 0x7f86c0160f70 2024-04-25T08:48:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86b7e00640 1 --2- 172.21.15.46:0/1154956211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006bd20 0x7f86c0162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.308+0000 7f86c5a00640 1 --2- 172.21.15.46:0/1154956211 >> v2:172.21.15.82:3300/0 conn(0x7f86c006c3e0 0x7f86c006c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c5000640 1 --2- 172.21.15.46:0/1154956211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 0x7f86c00d0d80 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f86c0072b00 tx=0x7f86b001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bb4d94e0da73ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86bfe00640 1 -- 172.21.15.46:0/1154956211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00b5070 con 0x7f86c0160f70 2024-04-25T08:48:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86bfe00640 1 -- 172.21.15.46:0/1154956211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b00a89d0 con 0x7f86c0160f70 2024-04-25T08:48:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86bfe00640 1 -- 172.21.15.46:0/1154956211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00a8da0 con 0x7f86c0160f70 2024-04-25T08:48:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c7a00640 1 -- 172.21.15.46:0/1154956211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 msgr2=0x7f86c00d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c7a00640 1 --2- 172.21.15.46:0/1154956211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 0x7f86c00d0d80 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f86c0072b00 tx=0x7f86b001ed10 comp rx=0 tx=0).stop 2024-04-25T08:48:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c7a00640 1 -- 172.21.15.46:0/1154956211 shutdown_connections 2024-04-25T08:48:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c7a00640 1 --2- 172.21.15.46:0/1154956211 >> v2:172.21.15.82:3300/0 conn(0x7f86c006c3e0 0x7f86c006c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c7a00640 1 --2- 172.21.15.46:0/1154956211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006bd20 0x7f86c0162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c7a00640 1 --2- 172.21.15.46:0/1154956211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 0x7f86c00d0d80 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c7a00640 1 -- 172.21.15.46:0/1154956211 >> 172.21.15.46:0/1154956211 conn(0x7f86c00b3130 msgr2=0x7f86c00b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c7a00640 1 -- 172.21.15.46:0/1154956211 shutdown_connections 2024-04-25T08:48:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.312+0000 7f86c7a00640 1 -- 172.21.15.46:0/1154956211 wait complete. 2024-04-25T08:48:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 Processor -- start 2024-04-25T08:48:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 -- start start 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c006bd20 0x7f86c015f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006c3e0 0x7f86c0159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 0x7f86c0159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00d0580 con 0x7f86c0160f70 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00d02a0 con 0x7f86c006bd20 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00d0860 con 0x7f86c006c3e0 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 0x7f86c0159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86b7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006c3e0 0x7f86c0159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c006bd20 0x7f86c015f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c006bd20 0x7f86c015f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:33532/0 (socket says 172.21.15.46:33532) 2024-04-25T08:48:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5000640 1 -- 172.21.15.46:0/763076680 learned_addr learned my addr 172.21.15.46:0/763076680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86b7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006c3e0 0x7f86c0159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:41118/0 (socket says 172.21.15.46:41118) 2024-04-25T08:48:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 0x7f86c0159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36958/0 (socket says 172.21.15.46:36958) 2024-04-25T08:48:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5000640 1 -- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006c3e0 msgr2=0x7f86c0159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5000640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006c3e0 0x7f86c0159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5000640 1 -- 172.21.15.46:0/763076680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 msgr2=0x7f86c0159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5000640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 0x7f86c0159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5000640 1 -- 172.21.15.46:0/763076680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c015e4e0 con 0x7f86c006bd20 2024-04-25T08:48:36.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5a00640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 0x7f86c0159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:48:36.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c5000640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c006bd20 0x7f86c015f580 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f86b00b3040 tx=0x7f86b001bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:36.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86bd600640 1 -- 172.21.15.46:0/763076680 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00c0070 con 0x7f86c006bd20 2024-04-25T08:48:36.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86bd600640 1 -- 172.21.15.46:0/763076680 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b00b07e0 con 0x7f86c006bd20 2024-04-25T08:48:36.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c015e7c0 con 0x7f86c006bd20 2024-04-25T08:48:36.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86bd600640 1 -- 172.21.15.46:0/763076680 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00b5040 con 0x7f86c006bd20 2024-04-25T08:48:36.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.316+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c015ed80 con 0x7f86c006bd20 2024-04-25T08:48:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.320+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c0069010 con 0x7f86c006bd20 2024-04-25T08:48:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.320+0000 7f86bd600640 1 -- 172.21.15.46:0/763076680 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f86b00c3070 con 0x7f86c006bd20 2024-04-25T08:48:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.320+0000 7f86bd600640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f869c066fb0 0x7f869c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.320+0000 7f86bd600640 1 -- 172.21.15.46:0/763076680 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f86b0122580 con 0x7f86c006bd20 2024-04-25T08:48:36.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.320+0000 7f86b7e00640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f869c066fb0 0x7f869c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.320+0000 7f86b7e00640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f869c066fb0 0x7f869c069470 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f86c00c8630 tx=0x7f86a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:36.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.324+0000 7f86bd600640 1 -- 172.21.15.46:0/763076680 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f86b00ca020 con 0x7f86c006bd20 2024-04-25T08:48:36.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.492+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86c0160230 con 0x7f86c006bd20 2024-04-25T08:48:36.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.492+0000 7f86bd600640 1 -- 172.21.15.46:0/763076680 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7f86b0018870 con 0x7f86c006bd20 2024-04-25T08:48:36.488 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:48:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f869c066fb0 msgr2=0x7f869c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f869c066fb0 0x7f869c069470 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f86c00c8630 tx=0x7f86a8008040 comp rx=0 tx=0).stop 2024-04-25T08:48:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c006bd20 msgr2=0x7f86c015f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c006bd20 0x7f86c015f580 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f86b00b3040 tx=0x7f86b001bab0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 shutdown_connections 2024-04-25T08:48:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f869c066fb0 0x7f869c069470 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c0160f70 0x7f86c0159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f86c006c3e0 0x7f86c0159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 --2- 172.21.15.46:0/763076680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f86c006bd20 0x7f86c015f580 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 >> 172.21.15.46:0/763076680 conn(0x7f86c00b3130 msgr2=0x7f86c00b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 shutdown_connections 2024-04-25T08:48:36.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.496+0000 7f86c7a00640 1 -- 172.21.15.46:0/763076680 wait complete. 2024-04-25T08:48:36.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:48:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2de2800640 1 Processor -- start 2024-04-25T08:48:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2de2800640 1 -- start start 2024-04-25T08:48:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2de2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc15dae0 0x7f2ddc072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2de2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ddc06ea60 0x7f2ddc0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2de2800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc0b3850 con 0x7f2ddc06e620 2024-04-25T08:48:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2de2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ddc06e190 con 0x7f2ddc06ea60 2024-04-25T08:48:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2de2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0b3af0 con 0x7f2ddc15dae0 2024-04-25T08:48:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2dd2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2ddc06e620 0x7f2ddc06bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50882/0 (socket says 172.21.15.46:50882) 2024-04-25T08:48:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2dd2a00640 1 -- 172.21.15.46:0/997405671 learned_addr learned my addr 172.21.15.46:0/997405671 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2ddbe00640 1 --2- 172.21.15.46:0/997405671 >> v2:172.21.15.82:3300/0 conn(0x7f2ddc06ea60 0x7f2ddc0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2ddb400640 1 --2- 172.21.15.46:0/997405671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc15dae0 0x7f2ddc072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2ddaa00640 1 -- 172.21.15.46:0/997405671 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693000219 0 0) 0x7f2ddc0b3850 con 0x7f2ddc06e620 2024-04-25T08:48:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2ddaa00640 1 -- 172.21.15.46:0/997405671 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dc0003530 con 0x7f2ddc06e620 2024-04-25T08:48:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2ddbe00640 1 -- 172.21.15.46:0/997405671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc15dae0 msgr2=0x7f2ddc072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2ddbe00640 1 --2- 172.21.15.46:0/997405671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc15dae0 0x7f2ddc072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2ddbe00640 1 -- 172.21.15.46:0/997405671 >> v1:172.21.15.46:6789/0 conn(0x7f2ddc06e620 legacy=0x7f2ddc06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2ddbe00640 1 -- 172.21.15.46:0/997405671 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc0b3dd0 con 0x7f2ddc06ea60 2024-04-25T08:48:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.660+0000 7f2ddb400640 1 --2- 172.21.15.46:0/997405671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc15dae0 0x7f2ddc072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:48:36.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2ddbe00640 1 --2- 172.21.15.46:0/997405671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ddc06ea60 0x7f2ddc0acdb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc06e310 tx=0x7f2dd401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cbc2a69181aab50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2ddaa00640 1 -- 172.21.15.46:0/997405671 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40a7070 con 0x7f2ddc06ea60 2024-04-25T08:48:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2ddaa00640 1 -- 172.21.15.46:0/997405671 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd40b0070 con 0x7f2ddc06ea60 2024-04-25T08:48:36.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2ddaa00640 1 -- 172.21.15.46:0/997405671 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40ab890 con 0x7f2ddc06ea60 2024-04-25T08:48:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2de2800640 1 -- 172.21.15.46:0/997405671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ddc06ea60 msgr2=0x7f2ddc0acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2de2800640 1 --2- 172.21.15.46:0/997405671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ddc06ea60 0x7f2ddc0acdb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc06e310 tx=0x7f2dd401c470 comp rx=0 tx=0).stop 2024-04-25T08:48:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2de2800640 1 -- 172.21.15.46:0/997405671 shutdown_connections 2024-04-25T08:48:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2de2800640 1 --2- 172.21.15.46:0/997405671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ddc06ea60 0x7f2ddc0acdb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2de2800640 1 --2- 172.21.15.46:0/997405671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc15dae0 0x7f2ddc072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2de2800640 1 -- 172.21.15.46:0/997405671 >> 172.21.15.46:0/997405671 conn(0x7f2ddc071ae0 msgr2=0x7f2ddc071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2de2800640 1 -- 172.21.15.46:0/997405671 shutdown_connections 2024-04-25T08:48:36.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.664+0000 7f2de2800640 1 -- 172.21.15.46:0/997405671 wait complete. 2024-04-25T08:48:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 Processor -- start 2024-04-25T08:48:36.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 -- start start 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ddc06e620 0x7f2ddc148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc06ea60 0x7f2ddc0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15dae0 0x7f2ddc0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc06f6d0 con 0x7f2ddc15dae0 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc15dec0 con 0x7f2ddc06e620 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc06f850 con 0x7f2ddc06ea60 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15dae0 0x7f2ddc0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15dae0 0x7f2ddc0af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36972/0 (socket says 172.21.15.46:36972) 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddbe00640 1 -- 172.21.15.46:0/1850006431 learned_addr learned my addr 172.21.15.46:0/1850006431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:36.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2dd2a00640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc06ea60 0x7f2ddc0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddb400640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ddc06e620 0x7f2ddc148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddbe00640 1 -- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc06ea60 msgr2=0x7f2ddc0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddbe00640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc06ea60 0x7f2ddc0aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddbe00640 1 -- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ddc06e620 msgr2=0x7f2ddc148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddbe00640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ddc06e620 0x7f2ddc148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddbe00640 1 -- 172.21.15.46:0/1850006431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc151e10 con 0x7f2ddc15dae0 2024-04-25T08:48:36.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2ddbe00640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15dae0 0x7f2ddc0af0b0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40ad040 tx=0x7f2dd401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2dd3e00640 1 -- 172.21.15.46:0/1850006431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40bf410 con 0x7f2ddc15dae0 2024-04-25T08:48:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2dd3e00640 1 -- 172.21.15.46:0/1850006431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd40bf5b0 con 0x7f2ddc15dae0 2024-04-25T08:48:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2dd3e00640 1 -- 172.21.15.46:0/1850006431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd40bf980 con 0x7f2ddc15dae0 2024-04-25T08:48:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ddc136b80 con 0x7f2ddc15dae0 2024-04-25T08:48:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ddc137140 con 0x7f2ddc15dae0 2024-04-25T08:48:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.668+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ddc069010 con 0x7f2ddc15dae0 2024-04-25T08:48:36.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.672+0000 7f2dd3e00640 1 -- 172.21.15.46:0/1850006431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2dd40bd020 con 0x7f2ddc15dae0 2024-04-25T08:48:36.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.672+0000 7f2dd3e00640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2db8066f90 0x7f2db8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.672+0000 7f2dd3e00640 1 -- 172.21.15.46:0/1850006431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2dd411f4c0 con 0x7f2ddc15dae0 2024-04-25T08:48:36.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.672+0000 7f2ddb400640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2db8066f90 0x7f2db8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.672+0000 7f2ddb400640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2db8066f90 0x7f2db8069450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009580 tx=0x7f2dcc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:36.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.676+0000 7f2dd3e00640 1 -- 172.21.15.46:0/1850006431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2dd40c6030 con 0x7f2ddc15dae0 2024-04-25T08:48:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.836+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ddc152d80 con 0x7f2ddc15dae0 2024-04-25T08:48:36.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.836+0000 7f2dd3e00640 1 -- 172.21.15.46:0/1850006431 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2dd4016090 con 0x7f2ddc15dae0 2024-04-25T08:48:36.831 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:48:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2db8066f90 msgr2=0x7f2db8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2db8066f90 0x7f2db8069450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009580 tx=0x7f2dcc009190 comp rx=0 tx=0).stop 2024-04-25T08:48:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15dae0 msgr2=0x7f2ddc0af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15dae0 0x7f2ddc0af0b0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2dd40ad040 tx=0x7f2dd401acb0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 shutdown_connections 2024-04-25T08:48:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2db8066f90 0x7f2db8069450 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15dae0 0x7f2ddc0af0b0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2ddc06ea60 0x7f2ddc0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 --2- 172.21.15.46:0/1850006431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ddc06e620 0x7f2ddc148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 >> 172.21.15.46:0/1850006431 conn(0x7f2ddc071ae0 msgr2=0x7f2ddc0b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 shutdown_connections 2024-04-25T08:48:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:36.840+0000 7f2de2800640 1 -- 172.21.15.46:0/1850006431 wait complete. 2024-04-25T08:48:36.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:48:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.000+0000 7f8323200640 1 Processor -- start 2024-04-25T08:48:36.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.000+0000 7f8323200640 1 -- start start 2024-04-25T08:48:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8323200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f832415d9a0 0x7f83240aee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8323200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83240b4720 0x7f83240b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8323200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8324067060 con 0x7f83240669a0 2024-04-25T08:48:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8323200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83240b3c60 con 0x7f832415d9a0 2024-04-25T08:48:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8323200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8324066e00 con 0x7f83240b4720 2024-04-25T08:48:36.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f831be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f83240669a0 0x7f83240b4010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50886/0 (socket says 172.21.15.46:50886) 2024-04-25T08:48:36.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f831be00640 1 -- 172.21.15.46:0/1990567527 learned_addr learned my addr 172.21.15.46:0/1990567527 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:36.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8321e00640 1 --2- 172.21.15.46:0/1990567527 >> v2:172.21.15.82:3300/0 conn(0x7f832415d9a0 0x7f83240aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8322800640 1 --2- 172.21.15.46:0/1990567527 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83240b4720 0x7f83240b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:36.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8321400640 1 -- 172.21.15.46:0/1990567527 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143339724 0 0) 0x7f8324067060 con 0x7f83240669a0 2024-04-25T08:48:36.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8321400640 1 -- 172.21.15.46:0/1990567527 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8308003530 con 0x7f83240669a0 2024-04-25T08:48:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8322800640 1 -- 172.21.15.46:0/1990567527 >> v2:172.21.15.82:3300/0 conn(0x7f832415d9a0 msgr2=0x7f83240aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8322800640 1 --2- 172.21.15.46:0/1990567527 >> v2:172.21.15.82:3300/0 conn(0x7f832415d9a0 0x7f83240aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8322800640 1 -- 172.21.15.46:0/1990567527 >> v1:172.21.15.46:6789/0 conn(0x7f83240669a0 legacy=0x7f83240b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:36.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.004+0000 7f8322800640 1 -- 172.21.15.46:0/1990567527 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83240ae090 con 0x7f83240b4720 2024-04-25T08:48:37.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8322800640 1 --2- 172.21.15.46:0/1990567527 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83240b4720 0x7f83240b3720 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f831c009d90 tx=0x7f831c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a063b9e364fb3aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:37.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8321400640 1 -- 172.21.15.46:0/1990567527 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0b2070 con 0x7f83240b4720 2024-04-25T08:48:37.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8321400640 1 -- 172.21.15.46:0/1990567527 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c0a5920 con 0x7f83240b4720 2024-04-25T08:48:37.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8321400640 1 -- 172.21.15.46:0/1990567527 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c0a5cf0 con 0x7f83240b4720 2024-04-25T08:48:37.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 -- 172.21.15.46:0/1990567527 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83240b4720 msgr2=0x7f83240b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 --2- 172.21.15.46:0/1990567527 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83240b4720 0x7f83240b3720 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f831c009d90 tx=0x7f831c01c970 comp rx=0 tx=0).stop 2024-04-25T08:48:37.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 -- 172.21.15.46:0/1990567527 shutdown_connections 2024-04-25T08:48:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 --2- 172.21.15.46:0/1990567527 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f83240b4720 0x7f83240b3720 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 --2- 172.21.15.46:0/1990567527 >> v2:172.21.15.82:3300/0 conn(0x7f832415d9a0 0x7f83240aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 -- 172.21.15.46:0/1990567527 >> 172.21.15.46:0/1990567527 conn(0x7f83240cfc30 msgr2=0x7f83240d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:37.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 -- 172.21.15.46:0/1990567527 shutdown_connections 2024-04-25T08:48:37.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 -- 172.21.15.46:0/1990567527 wait complete. 2024-04-25T08:48:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 Processor -- start 2024-04-25T08:48:37.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.008+0000 7f8323200640 1 -- start start 2024-04-25T08:48:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8323200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240669a0 0x7f8324155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8323200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240b4720 0x7f8324142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8323200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832415d9a0 0x7f832406b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8323200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832415df00 con 0x7f83240669a0 2024-04-25T08:48:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8323200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832415dd80 con 0x7f83240b4720 2024-04-25T08:48:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8323200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832406e150 con 0x7f832415d9a0 2024-04-25T08:48:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8321e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240669a0 0x7f8324155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8321e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240669a0 0x7f8324155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36978/0 (socket says 172.21.15.46:36978) 2024-04-25T08:48:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8321e00640 1 -- 172.21.15.46:0/72482485 learned_addr learned my addr 172.21.15.46:0/72482485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8322800640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832415d9a0 0x7f832406b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f831be00640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240b4720 0x7f8324142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8321e00640 1 -- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832415d9a0 msgr2=0x7f832406b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8321e00640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832415d9a0 0x7f832406b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8321e00640 1 -- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240b4720 msgr2=0x7f8324142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8321e00640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240b4720 0x7f8324142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8321e00640 1 -- 172.21.15.46:0/72482485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324158260 con 0x7f83240669a0 2024-04-25T08:48:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8321e00640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240669a0 0x7f8324155780 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f831400e870 tx=0x7f831400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f831a000640 1 -- 172.21.15.46:0/72482485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314017070 con 0x7f83240669a0 2024-04-25T08:48:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f831a000640 1 -- 172.21.15.46:0/72482485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831400c940 con 0x7f83240669a0 2024-04-25T08:48:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f831a000640 1 -- 172.21.15.46:0/72482485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831400ccf0 con 0x7f83240669a0 2024-04-25T08:48:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8324142a90 con 0x7f83240669a0 2024-04-25T08:48:37.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83241f66e0 con 0x7f83240669a0 2024-04-25T08:48:37.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8324152bb0 con 0x7f83240669a0 2024-04-25T08:48:37.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.012+0000 7f831a000640 1 -- 172.21.15.46:0/72482485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f831401a070 con 0x7f83240669a0 2024-04-25T08:48:37.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.016+0000 7f831a000640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f60 0x7f8300069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.016+0000 7f831a000640 1 -- 172.21.15.46:0/72482485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8314084ff0 con 0x7f83240669a0 2024-04-25T08:48:37.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.016+0000 7f831be00640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f60 0x7f8300069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.016+0000 7f831be00640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f60 0x7f8300069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8324155cc0 tx=0x7f830c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:37.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.020+0000 7f831a000640 1 -- 172.21.15.46:0/72482485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8324158260 con 0x7f83240669a0 2024-04-25T08:48:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8324069010 con 0x7f83240669a0 2024-04-25T08:48:37.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f831a000640 1 -- 172.21.15.46:0/72482485 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8314055f00 con 0x7f83240669a0 2024-04-25T08:48:37.174 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:48:37.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f60 msgr2=0x7f8300069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f60 0x7f8300069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8324155cc0 tx=0x7f830c009040 comp rx=0 tx=0).stop 2024-04-25T08:48:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240669a0 msgr2=0x7f8324155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240669a0 0x7f8324155780 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f831400e870 tx=0x7f831400ec70 comp rx=0 tx=0).stop 2024-04-25T08:48:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 shutdown_connections 2024-04-25T08:48:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8300066f60 0x7f8300069420 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f832415d9a0 0x7f832406b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83240b4720 0x7f8324142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 --2- 172.21.15.46:0/72482485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83240669a0 0x7f8324155780 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 >> 172.21.15.46:0/72482485 conn(0x7f83240cfc30 msgr2=0x7f8324073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:37.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 shutdown_connections 2024-04-25T08:48:37.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.180+0000 7f8323200640 1 -- 172.21.15.46:0/72482485 wait complete. 2024-04-25T08:48:37.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:48:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f549f200640 1 Processor -- start 2024-04-25T08:48:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f549f200640 1 -- start start 2024-04-25T08:48:37.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f549f200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5498099a00 0x7f5498099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f549f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f54980cffc0 0x7f549806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f549f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549809a320 con 0x7f5498070390 2024-04-25T08:48:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f549f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54980d07e0 con 0x7f54980cffc0 2024-04-25T08:48:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f549f200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54980d0a30 con 0x7f5498099a00 2024-04-25T08:48:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f548f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f54980cffc0 0x7f549806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f548f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f54980cffc0 0x7f549806fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36984/0 (socket says 172.21.15.46:36984) 2024-04-25T08:48:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f549d200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5498070390 0x7f54980cc1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50894/0 (socket says 172.21.15.46:50894) 2024-04-25T08:48:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f548f400640 1 -- 172.21.15.46:0/1335747531 learned_addr learned my addr 172.21.15.46:0/1335747531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f5497e00640 1 --2- 172.21.15.46:0/1335747531 >> v2:172.21.15.82:3300/0 conn(0x7f5498099a00 0x7f5498099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f548f400640 1 -- 172.21.15.46:0/1335747531 >> v2:172.21.15.82:3300/0 conn(0x7f5498099a00 msgr2=0x7f5498099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f548f400640 1 --2- 172.21.15.46:0/1335747531 >> v2:172.21.15.82:3300/0 conn(0x7f5498099a00 0x7f5498099de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f548f400640 1 -- 172.21.15.46:0/1335747531 >> v1:172.21.15.46:6789/0 conn(0x7f5498070390 legacy=0x7f54980cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f548f400640 1 -- 172.21.15.46:0/1335747531 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54980d0d10 con 0x7f54980cffc0 2024-04-25T08:48:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.344+0000 7f5497400640 1 -- 172.21.15.46:0/1335747531 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965562054 0 0) 0x7f549809a320 con 0x7f5498070390 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f548f400640 1 --2- 172.21.15.46:0/1335747531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54980cffc0 0x7f549806fe50 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5498071700 tx=0x7f548001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd27f05d5da3092 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f5497400640 1 -- 172.21.15.46:0/1335747531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800ab610 con 0x7f54980cffc0 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f5497400640 1 -- 172.21.15.46:0/1335747531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54800b0070 con 0x7f54980cffc0 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f5497400640 1 -- 172.21.15.46:0/1335747531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800ab9d0 con 0x7f54980cffc0 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f549f200640 1 -- 172.21.15.46:0/1335747531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54980cffc0 msgr2=0x7f549806fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f549f200640 1 --2- 172.21.15.46:0/1335747531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54980cffc0 0x7f549806fe50 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f5498071700 tx=0x7f548001c520 comp rx=0 tx=0).stop 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f549f200640 1 -- 172.21.15.46:0/1335747531 shutdown_connections 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f549f200640 1 --2- 172.21.15.46:0/1335747531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54980cffc0 0x7f549806fe50 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f549f200640 1 --2- 172.21.15.46:0/1335747531 >> v2:172.21.15.82:3300/0 conn(0x7f5498099a00 0x7f5498099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f549f200640 1 -- 172.21.15.46:0/1335747531 >> 172.21.15.46:0/1335747531 conn(0x7f54980d16b0 msgr2=0x7f54980d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:37.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f549f200640 1 -- 172.21.15.46:0/1335747531 shutdown_connections 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.348+0000 7f549f200640 1 -- 172.21.15.46:0/1335747531 wait complete. 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 Processor -- start 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 -- start start 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5498070390 0x7f54980c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5498099a00 0x7f54980b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54980cffc0 0x7f54980b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980b0f60 con 0x7f5498070390 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980b0de0 con 0x7f54980cffc0 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54980d0400 con 0x7f5498099a00 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5497e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5498070390 0x7f54980c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5497e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5498070390 0x7f54980c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36998/0 (socket says 172.21.15.46:36998) 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5497e00640 1 -- 172.21.15.46:0/2573534317 learned_addr learned my addr 172.21.15.46:0/2573534317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f548f400640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5498099a00 0x7f54980b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549d200640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54980cffc0 0x7f54980b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5497e00640 1 -- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5498099a00 msgr2=0x7f54980b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5497e00640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5498099a00 0x7f54980b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5497e00640 1 -- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54980cffc0 msgr2=0x7f54980b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5497e00640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54980cffc0 0x7f54980b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5497e00640 1 -- 172.21.15.46:0/2573534317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54980c5980 con 0x7f5498070390 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549d200640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54980cffc0 0x7f54980b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5497e00640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5498070390 0x7f54980c51e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5488007ef0 tx=0x7f548800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5494c00640 1 -- 172.21.15.46:0/2573534317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488014620 con 0x7f5498070390 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5494c00640 1 -- 172.21.15.46:0/2573534317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5488019070 con 0x7f5498070390 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f5494c00640 1 -- 172.21.15.46:0/2573534317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488014a00 con 0x7f5498070390 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5498151490 con 0x7f5498070390 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.352+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5498151a50 con 0x7f5498070390 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.356+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5498069080 con 0x7f5498070390 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.356+0000 7f5494c00640 1 -- 172.21.15.46:0/2573534317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5488014ba0 con 0x7f5498070390 2024-04-25T08:48:37.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.356+0000 7f5494c00640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5474066f60 0x7f5474069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.356+0000 7f5494c00640 1 -- 172.21.15.46:0/2573534317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5488085a60 con 0x7f5498070390 2024-04-25T08:48:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.356+0000 7f548f400640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5474066f60 0x7f5474069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.356+0000 7f548f400640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5474066f60 0x7f5474069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f54800af000 tx=0x7f54800a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:37.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.360+0000 7f5494c00640 1 -- 172.21.15.46:0/2573534317 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54980c5980 con 0x7f5498070390 2024-04-25T08:48:37.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f54980b7630 con 0x7f5498070390 2024-04-25T08:48:37.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f5494c00640 1 -- 172.21.15.46:0/2573534317 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v90) v1 ==== 80+0+0 (secure 0 0 0) 0x7f54880568d0 con 0x7f5498070390 2024-04-25T08:48:37.529 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5474066f60 msgr2=0x7f5474069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5474066f60 0x7f5474069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f54800af000 tx=0x7f54800a7000 comp rx=0 tx=0).stop 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5498070390 msgr2=0x7f54980c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5498070390 0x7f54980c51e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f5488007ef0 tx=0x7f548800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 shutdown_connections 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5474066f60 0x7f5474069420 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54980cffc0 0x7f54980b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5498099a00 0x7f54980b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 --2- 172.21.15.46:0/2573534317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5498070390 0x7f54980c51e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 >> 172.21.15.46:0/2573534317 conn(0x7f54980d16b0 msgr2=0x7f54980af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 shutdown_connections 2024-04-25T08:48:37.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.536+0000 7f549f200640 1 -- 172.21.15.46:0/2573534317 wait complete. 2024-04-25T08:48:37.550 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:48:37.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:37.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:37.604 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:48:37.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:37.604 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:48:37.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:37.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.816+0000 7f3495400640 1 Processor -- start 2024-04-25T08:48:37.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f3495400640 1 -- start start 2024-04-25T08:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f3495400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f34900af3f0 0x7f34900b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f3495400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349006ee30 con 0x7f349006db40 2024-04-25T08:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f3495400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349015e180 con 0x7f349006f630 2024-04-25T08:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f3495400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34900b3ca0 con 0x7f34900af3f0 2024-04-25T08:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f3487e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f349006db40 0x7f349006e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50904/0 (socket says 172.21.15.46:50904) 2024-04-25T08:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f3487e00640 1 -- 172.21.15.46:0/3586794632 learned_addr learned my addr 172.21.15.46:0/3586794632 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348e800640 1 --2- 172.21.15.46:0/3586794632 >> v2:172.21.15.82:3300/0 conn(0x7f34900af3f0 0x7f34900b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211551351 0 0) 0x7f349006ee30 con 0x7f349006db40 2024-04-25T08:48:37.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3474003530 con 0x7f349006db40 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2933645322 0 0) 0x7f3474003530 con 0x7f349006db40 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 >> v2:172.21.15.82:3300/0 conn(0x7f34900af3f0 msgr2=0x7f34900b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 --2- 172.21.15.46:0/3586794632 >> v2:172.21.15.82:3300/0 conn(0x7f34900af3f0 0x7f34900b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 >> v1:172.21.15.82:6789/0 conn(0x7f349006f630 legacy=0x7f349006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349015e460 con 0x7f349006db40 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f3478001680 con 0x7f349006db40 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 >> v1:172.21.15.46:6789/0 conn(0x7f349006db40 legacy=0x7f349006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34740039c0 0x7f3474003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3474004810 0x7f3474004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740051d0 0x7f347400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34900b3ca0 con 0x7f34740039c0 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347400c070 con 0x7f34740051d0 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347400c270 con 0x7f3474004810 2024-04-25T08:48:37.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348e800640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34740039c0 0x7f3474003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348f200640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740051d0 0x7f347400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f3487e00640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3474004810 0x7f3474004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348e800640 1 -- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3474004810 msgr2=0x7f3474004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348e800640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3474004810 0x7f3474004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348e800640 1 -- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740051d0 msgr2=0x7f347400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348e800640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740051d0 0x7f347400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348e800640 1 -- 172.21.15.46:0/3586794632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3474003530 con 0x7f34740039c0 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f3487e00640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3474004810 0x7f3474004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348f200640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740051d0 0x7f347400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.820+0000 7f348f200640 1 -- 172.21.15.46:0/3586794632 reap_dead start 2024-04-25T08:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f348e800640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34740039c0 0x7f3474003de0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f3480009ae0 tx=0x7f348001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8c41827213214b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:37.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34800096b0 con 0x7f34740039c0 2024-04-25T08:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34800a6450 con 0x7f34740039c0 2024-04-25T08:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f348de00640 1 -- 172.21.15.46:0/3586794632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34800a6840 con 0x7f34740039c0 2024-04-25T08:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f3495400640 1 -- 172.21.15.46:0/3586794632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34740039c0 msgr2=0x7f3474003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f3495400640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34740039c0 0x7f3474003de0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f3480009ae0 tx=0x7f348001ca70 comp rx=0 tx=0).stop 2024-04-25T08:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f3495400640 1 -- 172.21.15.46:0/3586794632 shutdown_connections 2024-04-25T08:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f3495400640 1 --2- 172.21.15.46:0/3586794632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34740039c0 0x7f3474003de0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f3495400640 1 -- 172.21.15.46:0/3586794632 >> 172.21.15.46:0/3586794632 conn(0x7f34900cfd90 msgr2=0x7f34900d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f3495400640 1 -- 172.21.15.46:0/3586794632 shutdown_connections 2024-04-25T08:48:37.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.824+0000 7f3495400640 1 -- 172.21.15.46:0/3586794632 wait complete. 2024-04-25T08:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 Processor -- start 2024-04-25T08:48:37.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 -- start start 2024-04-25T08:48:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740039c0 0x7f349015bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34900ce0d0 0x7f34900ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f349014a420 0x7f349014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349006e010 con 0x7f349014a420 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34900af7d0 con 0x7f34740039c0 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349006e190 con 0x7f34900ce0d0 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f349014a420 0x7f349014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f349014a420 0x7f349014a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37014/0 (socket says 172.21.15.46:37014) 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348f200640 1 -- 172.21.15.46:0/2570430037 learned_addr learned my addr 172.21.15.46:0/2570430037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3487e00640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34900ce0d0 0x7f34900ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348e800640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740039c0 0x7f349015bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348f200640 1 -- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34900ce0d0 msgr2=0x7f34900ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348f200640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34900ce0d0 0x7f34900ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348f200640 1 -- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740039c0 msgr2=0x7f349015bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348f200640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740039c0 0x7f349015bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:37.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348f200640 1 -- 172.21.15.46:0/2570430037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34900cec90 con 0x7f349014a420 2024-04-25T08:48:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f348f200640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f349014a420 0x7f349014a8a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f349006fad0 tx=0x7f348800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3486a00640 1 -- 172.21.15.46:0/2570430037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488019070 con 0x7f349014a420 2024-04-25T08:48:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3486a00640 1 -- 172.21.15.46:0/2570430037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3488003c50 con 0x7f349014a420 2024-04-25T08:48:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34900cef70 con 0x7f349014a420 2024-04-25T08:48:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3486a00640 1 -- 172.21.15.46:0/2570430037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34880050d0 con 0x7f349014a420 2024-04-25T08:48:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.828+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349015c1e0 con 0x7f349014a420 2024-04-25T08:48:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.832+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3490159250 con 0x7f349014a420 2024-04-25T08:48:37.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.832+0000 7f3486a00640 1 -- 172.21.15.46:0/2570430037 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f348801c070 con 0x7f349014a420 2024-04-25T08:48:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.832+0000 7f3486a00640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f346c066f60 0x7f346c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.832+0000 7f3486a00640 1 -- 172.21.15.46:0/2570430037 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3488084720 con 0x7f349014a420 2024-04-25T08:48:37.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.832+0000 7f348e800640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f346c066f60 0x7f346c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:37.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.832+0000 7f348e800640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f346c066f60 0x7f346c069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f34800ad000 tx=0x7f34800b2040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:37.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:37.836+0000 7f3486a00640 1 -- 172.21.15.46:0/2570430037 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3490159250 con 0x7f349014a420 2024-04-25T08:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3490069080 con 0x7f349014a420 2024-04-25T08:48:38.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3486a00640 1 -- 172.21.15.46:0/2570430037 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f3488054eb0 con 0x7f349014a420 2024-04-25T08:48:38.026 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:38.026 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:48:30.739960+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/1269610178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1269610178},{"type":"v1","addr":"172.21.15.46:6835","nonce":1269610178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:48:38.026 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 62 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f346c066f60 msgr2=0x7f346c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f346c066f60 0x7f346c069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f34800ad000 tx=0x7f34800b2040 comp rx=0 tx=0).stop 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f349014a420 msgr2=0x7f349014a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f349014a420 0x7f349014a8a0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f349006fad0 tx=0x7f348800e960 comp rx=0 tx=0).stop 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 shutdown_connections 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f346c066f60 0x7f346c069420 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f349014a420 0x7f349014a8a0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f34900ce0d0 0x7f34900ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 --2- 172.21.15.46:0/2570430037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34740039c0 0x7f349015bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 >> 172.21.15.46:0/2570430037 conn(0x7f34900cfd90 msgr2=0x7f34900ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 shutdown_connections 2024-04-25T08:48:38.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.032+0000 7f3495400640 1 -- 172.21.15.46:0/2570430037 wait complete. 2024-04-25T08:48:38.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:48:38.218 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-25T08:48:38.219 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:48:38.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:48:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ee00640 1 Processor -- start 2024-04-25T08:48:38.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ee00640 1 -- start start 2024-04-25T08:48:38.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ee00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f730815d710 0x7f73080b6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f73080b9530 0x7f73080ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73080d1ff0 con 0x7f730815daf0 2024-04-25T08:48:38.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73080b72b0 con 0x7f73080b9530 2024-04-25T08:48:38.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ee00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73080b7590 con 0x7f730815d710 2024-04-25T08:48:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f7307400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f730815daf0 0x7f73080d18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32858/0 (socket says 172.21.15.46:32858) 2024-04-25T08:48:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f7307400640 1 -- 172.21.15.46:0/3921931963 learned_addr learned my addr 172.21.15.46:0/3921931963 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ce00640 1 --2- 172.21.15.46:0/3921931963 >> v2:172.21.15.46:3300/0 conn(0x7f73080b9530 0x7f73080ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f7307e00640 1 --2- 172.21.15.46:0/3921931963 >> v2:172.21.15.82:3300/0 conn(0x7f730815d710 0x7f73080b6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ce00640 1 -- 172.21.15.46:0/3921931963 >> v2:172.21.15.82:3300/0 conn(0x7f730815d710 msgr2=0x7f73080b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ce00640 1 --2- 172.21.15.46:0/3921931963 >> v2:172.21.15.82:3300/0 conn(0x7f730815d710 0x7f73080b6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ce00640 1 -- 172.21.15.46:0/3921931963 >> v1:172.21.15.46:6789/0 conn(0x7f730815daf0 legacy=0x7f73080d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.368+0000 7f730ce00640 1 -- 172.21.15.46:0/3921931963 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080b7870 con 0x7f73080b9530 2024-04-25T08:48:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f730ce00640 1 --2- 172.21.15.46:0/3921931963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73080b9530 0x7f73080ce550 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7300009d90 tx=0x7f730001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919f8b45bc657527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f7306a00640 1 -- 172.21.15.46:0/3921931963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000b2070 con 0x7f73080b9530 2024-04-25T08:48:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f7306a00640 1 -- 172.21.15.46:0/3921931963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73000a58d0 con 0x7f73080b9530 2024-04-25T08:48:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f7306a00640 1 -- 172.21.15.46:0/3921931963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000a5ca0 con 0x7f73080b9530 2024-04-25T08:48:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f730ee00640 1 -- 172.21.15.46:0/3921931963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73080b9530 msgr2=0x7f73080ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f730ee00640 1 --2- 172.21.15.46:0/3921931963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73080b9530 0x7f73080ce550 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7300009d90 tx=0x7f730001c900 comp rx=0 tx=0).stop 2024-04-25T08:48:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f730ee00640 1 -- 172.21.15.46:0/3921931963 shutdown_connections 2024-04-25T08:48:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f730ee00640 1 --2- 172.21.15.46:0/3921931963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73080b9530 0x7f73080ce550 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f730ee00640 1 --2- 172.21.15.46:0/3921931963 >> v2:172.21.15.82:3300/0 conn(0x7f730815d710 0x7f73080b6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f730ee00640 1 -- 172.21.15.46:0/3921931963 >> 172.21.15.46:0/3921931963 conn(0x7f7308072770 msgr2=0x7f7308072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f730ee00640 1 -- 172.21.15.46:0/3921931963 shutdown_connections 2024-04-25T08:48:38.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.372+0000 7f730ee00640 1 -- 172.21.15.46:0/3921931963 wait complete. 2024-04-25T08:48:38.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ee00640 1 Processor -- start 2024-04-25T08:48:38.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ee00640 1 -- start start 2024-04-25T08:48:38.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f73080b9530 0x7f730815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f730815d710 0x7f730814fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f730815daf0 0x7f73081500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730816a620 con 0x7f730815daf0 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080b7060 con 0x7f730815d710 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730816a7a0 con 0x7f73080b9530 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f730815daf0 0x7f73081500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f730815daf0 0x7f73081500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60678/0 (socket says 172.21.15.46:60678) 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ce00640 1 -- 172.21.15.46:0/2711838423 learned_addr learned my addr 172.21.15.46:0/2711838423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f7307400640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f730815d710 0x7f730814fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f7307e00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f73080b9530 0x7f730815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ce00640 1 -- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f73080b9530 msgr2=0x7f730815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ce00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f73080b9530 0x7f730815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ce00640 1 -- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f730815d710 msgr2=0x7f730814fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ce00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f730815d710 0x7f730814fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ce00640 1 -- 172.21.15.46:0/2711838423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308160d20 con 0x7f730815daf0 2024-04-25T08:48:38.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.376+0000 7f730ce00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f730815daf0 0x7f73081500a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7300009d90 tx=0x7f73000196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.380+0000 7f72ffe00640 1 -- 172.21.15.46:0/2711838423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000b2050 con 0x7f730815daf0 2024-04-25T08:48:38.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.380+0000 7f72ffe00640 1 -- 172.21.15.46:0/2711838423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73000be070 con 0x7f730815daf0 2024-04-25T08:48:38.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.380+0000 7f72ffe00640 1 -- 172.21.15.46:0/2711838423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000ada90 con 0x7f730815daf0 2024-04-25T08:48:38.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.380+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730815fc80 con 0x7f730815daf0 2024-04-25T08:48:38.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.380+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7308160240 con 0x7f730815daf0 2024-04-25T08:48:38.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.380+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7308069080 con 0x7f730815daf0 2024-04-25T08:48:38.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.380+0000 7f72ffe00640 1 -- 172.21.15.46:0/2711838423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f73000adc30 con 0x7f730815daf0 2024-04-25T08:48:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.384+0000 7f72ffe00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e8066f90 0x7f72e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.384+0000 7f72ffe00640 1 -- 172.21.15.46:0/2711838423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f73001202d0 con 0x7f730815daf0 2024-04-25T08:48:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.384+0000 7f7307e00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e8066f90 0x7f72e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.384+0000 7f7307e00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e8066f90 0x7f72e8069450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f730814e520 tx=0x7f72f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.388+0000 7f72ffe00640 1 -- 172.21.15.46:0/2711838423 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7300016520 con 0x7f730815daf0 2024-04-25T08:48:38.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.588+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f73080680e0 con 0x7f730815daf0 2024-04-25T08:48:38.584 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:48:38.588+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:48:38.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.592+0000 7f72ffe00640 1 -- 172.21.15.46:0/2711838423 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f73000f0a60 con 0x7f730815daf0 2024-04-25T08:48:38.586 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:48:38.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e8066f90 msgr2=0x7f72e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e8066f90 0x7f72e8069450 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f730814e520 tx=0x7f72f8008040 comp rx=0 tx=0).stop 2024-04-25T08:48:38.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f730815daf0 msgr2=0x7f73081500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f730815daf0 0x7f73081500a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7300009d90 tx=0x7f73000196a0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 shutdown_connections 2024-04-25T08:48:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f72e8066f90 0x7f72e8069450 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f730815daf0 0x7f73081500a0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f730815d710 0x7f730814fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 --2- 172.21.15.46:0/2711838423 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f73080b9530 0x7f730815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 >> 172.21.15.46:0/2711838423 conn(0x7f7308072770 msgr2=0x7f73081542d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 shutdown_connections 2024-04-25T08:48:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.596+0000 7f730ee00640 1 -- 172.21.15.46:0/2711838423 wait complete. 2024-04-25T08:48:38.598 INFO:tasks.ceph.mds.c.smithi046.stderr: -4873> 2024-04-25T08:48:30.292+0000 7f0b53000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:48:38.612 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-25T08:48:38.648 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:48:38.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7fd6a080 1 Processor -- start 2024-04-25T08:48:38.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7fd6a080 1 -- start start 2024-04-25T08:48:38.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7fd6a080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b44bf0 0x556310c19460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7fd6a080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310b8f830 0x556310c199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7fd6a080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310c1a680 0x556310caece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7fd6a080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556310ae99f0 con 0x556310c1a680 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7fd6a080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556310aed950 con 0x556310b8f830 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7fd6a080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556310b8fc10 con 0x556310b44bf0 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310c1a680 0x556310caece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310c1a680 0x556310caece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60708/0 (socket says 172.21.15.46:60708) 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7da00640 1 -- 172.21.15.46:0/2401873612 learned_addr learned my addr 172.21.15.46:0/2401873612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7da00640 1 -- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b44bf0 msgr2=0x556310c19460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7d000640 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b44bf0 0x556310c19460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd77e00640 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310b8f830 0x556310c199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7da00640 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b44bf0 0x556310c19460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7da00640 1 -- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310b8f830 msgr2=0x556310c199a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7da00640 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310b8f830 0x556310c199a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7da00640 1 -- 172.21.15.46:0/2401873612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556310c19ee0 con 0x556310c1a680 2024-04-25T08:48:38.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd77e00640 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310b8f830 0x556310c199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:38.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7d000640 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b44bf0 0x556310c19460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:38.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.664+0000 7fbd7da00640 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310c1a680 0x556310caece0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbd780ae040 tx=0x7fbd780033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512ba4f24c071f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd76a00640 1 -- 172.21.15.46:0/2401873612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd780b0040 con 0x556310c1a680 2024-04-25T08:48:38.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd76a00640 1 -- 172.21.15.46:0/2401873612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd780b5640 con 0x556310c1a680 2024-04-25T08:48:38.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd76a00640 1 -- 172.21.15.46:0/2401873612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd780b59f0 con 0x556310c1a680 2024-04-25T08:48:38.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/2401873612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310c1a680 msgr2=0x556310caece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310c1a680 0x556310caece0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fbd780ae040 tx=0x7fbd780033b0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/2401873612 shutdown_connections 2024-04-25T08:48:38.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310c1a680 0x556310caece0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310b8f830 0x556310c199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 --2- 172.21.15.46:0/2401873612 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b44bf0 0x556310c19460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/2401873612 >> 172.21.15.46:0/2401873612 conn(0x556310b42dd0 msgr2=0x556310c17480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/2401873612 shutdown_connections 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/2401873612 wait complete. 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 Processor -- start 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- start start 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310b44bf0 0x556310b88a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b8f830 0x556310c15e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310c1a680 0x556310b892e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556310c151f0 con 0x556310b44bf0 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556310b727e0 con 0x556310c1a680 2024-04-25T08:48:38.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556310b72a80 con 0x556310b8f830 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310b44bf0 0x556310b88a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310b44bf0 0x556310b88a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60722/0 (socket says 172.21.15.46:60722) 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7d000640 1 -- 172.21.15.46:0/285461564 learned_addr learned my addr 172.21.15.46:0/285461564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7da00640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310c1a680 0x556310b892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd77e00640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b8f830 0x556310c15e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7d000640 1 -- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b8f830 msgr2=0x556310c15e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7d000640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556310b8f830 0x556310c15e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7d000640 1 -- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310c1a680 msgr2=0x556310b892e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7d000640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556310c1a680 0x556310b892e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7d000640 1 -- 172.21.15.46:0/285461564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556310c16de0 con 0x556310b44bf0 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7d000640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310b44bf0 0x556310b88a20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000cf10 tx=0x7fbd7000a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd75600640 1 -- 172.21.15.46:0/285461564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000e660 con 0x556310b44bf0 2024-04-25T08:48:38.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd75600640 1 -- 172.21.15.46:0/285461564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7000e800 con 0x556310b44bf0 2024-04-25T08:48:38.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd75600640 1 -- 172.21.15.46:0/285461564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000ebd0 con 0x556310b44bf0 2024-04-25T08:48:38.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.668+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556310c16600 con 0x556310b44bf0 2024-04-25T08:48:38.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556310c16b40 con 0x556310b44bf0 2024-04-25T08:48:38.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd75600640 1 -- 172.21.15.46:0/285461564 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd70016400 con 0x556310b44bf0 2024-04-25T08:48:38.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd75600640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c065940 0x7fbd5c067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd75600640 1 -- 172.21.15.46:0/285461564 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbd700842d0 con 0x556310b44bf0 2024-04-25T08:48:38.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd7fd6a080 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556310b89820 0x556310b875e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e91) v9 -- 0x556310b87b20 con 0x556310b89820 2024-04-25T08:48:38.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd77e00640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c065940 0x7fbd5c067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd7da00640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556310b89820 0x556310b875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd7da00640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556310b89820 0x556310b875e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.672+0000 7fbd77e00640 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c065940 0x7fbd5c067e00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbd68001020 tx=0x7fbd680018d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7da00640 1 -- 172.21.15.46:0/285461564 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v91'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fbd780b57f0 con 0x556310b89820 2024-04-25T08:48:38.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556310b89820 msgr2=0x556310b875e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556310b89820 0x556310b875e0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c065940 msgr2=0x7fbd5c067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd5c065940 0x7fbd5c067e00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbd68001020 tx=0x7fbd680018d0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310b44bf0 msgr2=0x556310b88a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 --2- 172.21.15.46:0/285461564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556310b44bf0 0x556310b88a20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000cf10 tx=0x7fbd7000a6c0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7da00640 1 -- 172.21.15.46:0/285461564 reap_dead start 2024-04-25T08:48:38.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 shutdown_connections 2024-04-25T08:48:38.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 >> 172.21.15.46:0/285461564 conn(0x556310b42dd0 msgr2=0x556310b41520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 shutdown_connections 2024-04-25T08:48:38.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.676+0000 7fbd7fd6a080 1 -- 172.21.15.46:0/285461564 wait complete. 2024-04-25T08:48:38.677 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-25T08:48:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 -- start start 2024-04-25T08:48:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 Processor -- start 2024-04-25T08:48:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564163257800 0x564163411b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564163257400 0x564163411600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564163257000 0x564163411080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564164026180 con 0x564163257800 2024-04-25T08:48:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641632b6900 con 0x564163257400 2024-04-25T08:48:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5641632b6480 con 0x564163257000 2024-04-25T08:48:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564163257800 0x564163411b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564163257800 0x564163411b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60746/0 (socket says 172.21.15.46:60746) 2024-04-25T08:48:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4200640 1 -- 172.21.15.46:0/893937389 learned_addr learned my addr 172.21.15.46:0/893937389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe3800640 1 --2- 172.21.15.46:0/893937389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564163257400 0x564163411600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/893937389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564163257000 0x564163411080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4200640 1 -- 172.21.15.46:0/893937389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564163257000 msgr2=0x564163411080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4200640 1 --2- 172.21.15.46:0/893937389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564163257000 0x564163411080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4200640 1 -- 172.21.15.46:0/893937389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564163257400 msgr2=0x564163411600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4200640 1 --2- 172.21.15.46:0/893937389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564163257400 0x564163411600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4200640 1 -- 172.21.15.46:0/893937389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641632eeb40 con 0x564163257800 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe4200640 1 --2- 172.21.15.46:0/893937389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564163257800 0x564163411b80 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x5641640b2a20 tx=0x56416334a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b714a75e7da085cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe1000640 1 -- 172.21.15.46:0/893937389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56416328c9c0 con 0x564163257800 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe1000640 1 -- 172.21.15.46:0/893937389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641632a01c0 con 0x564163257800 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe1000640 1 -- 172.21.15.46:0/893937389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56416328c4e0 con 0x564163257800 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 -- 172.21.15.46:0/893937389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641632eed20 con 0x564163257800 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 1 -- 172.21.15.46:0/893937389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5641632eef00 con 0x564163257800 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 4 waiting for MDS map... 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe1000640 1 -- 172.21.15.46:0/893937389 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x564164172200 con 0x564163257800 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe1000640 1 -- 172.21.15.46:0/893937389 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 2723+0+0 (secure 0 0 0) 0x5641632ad180 con 0x564163257800 2024-04-25T08:48:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.772+0000 7fcbe60c1400 4 Got MDS map 63 2024-04-25T08:48:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 10 main: JournalTool::main 2024-04-25T08:48:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 4 main: JournalTool: connecting to RADOS... 2024-04-25T08:48:38.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 Processor -- start 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 -- start start 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56416401f800 0x564163410b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401fc00 0x564163410580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce000 0x564163412100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641632b6180 con 0x5641641ce000 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564164026d80 con 0x56416401f800 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564164026f00 con 0x56416401fc00 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe3800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce000 0x564163412100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe4200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401fc00 0x564163410580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56416401f800 0x564163410b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe3800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce000 0x564163412100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60760/0 (socket says 172.21.15.46:60760) 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe3800640 1 -- 172.21.15.46:0/556321195 learned_addr learned my addr 172.21.15.46:0/556321195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe3800640 1 -- 172.21.15.46:0/556321195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401fc00 msgr2=0x564163410580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe3800640 1 --2- 172.21.15.46:0/556321195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401fc00 0x564163410580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe3800640 1 -- 172.21.15.46:0/556321195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56416401f800 msgr2=0x564163410b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe3800640 1 --2- 172.21.15.46:0/556321195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56416401f800 0x564163410b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe3800640 1 -- 172.21.15.46:0/556321195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641632ee1e0 con 0x5641641ce000 2024-04-25T08:48:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe3800640 1 --2- 172.21.15.46:0/556321195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce000 0x564163412100 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x5641640168d0 tx=0x56416402cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f00e9662dbe745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbde800640 1 -- 172.21.15.46:0/556321195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56416328c9c0 con 0x5641641ce000 2024-04-25T08:48:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbde800640 1 -- 172.21.15.46:0/556321195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641632a0c40 con 0x5641641ce000 2024-04-25T08:48:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbde800640 1 -- 172.21.15.46:0/556321195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56416328c4e0 con 0x5641641ce000 2024-04-25T08:48:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 -- 172.21.15.46:0/556321195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce000 msgr2=0x564163412100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/556321195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce000 0x564163412100 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x5641640168d0 tx=0x56416402cc00 comp rx=0 tx=0).stop 2024-04-25T08:48:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 -- 172.21.15.46:0/556321195 shutdown_connections 2024-04-25T08:48:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/556321195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce000 0x564163412100 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/556321195 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401fc00 0x564163410580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/556321195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56416401f800 0x564163410b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 -- 172.21.15.46:0/556321195 >> 172.21.15.46:0/556321195 conn(0x564163256c00 msgr2=0x564163285800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 -- 172.21.15.46:0/556321195 shutdown_connections 2024-04-25T08:48:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.776+0000 7fcbe60c1400 1 -- 172.21.15.46:0/556321195 wait complete. 2024-04-25T08:48:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 Processor -- start 2024-04-25T08:48:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 -- start start 2024-04-25T08:48:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56416401fc00 0x564163410b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401f800 0x564163410580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cf800 0x564163412100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564164027200 con 0x56416401fc00 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564164027080 con 0x5641641cf800 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564164027380 con 0x56416401f800 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56416401fc00 0x564163410b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56416401fc00 0x564163410b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60772/0 (socket says 172.21.15.46:60772) 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4c00640 1 -- 172.21.15.46:0/554628413 learned_addr learned my addr 172.21.15.46:0/554628413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4200640 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401f800 0x564163410580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe3800640 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cf800 0x564163412100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4c00640 1 -- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401f800 msgr2=0x564163410580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401f800 0x564163410580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4c00640 1 -- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cf800 msgr2=0x564163412100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cf800 0x564163412100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4c00640 1 -- 172.21.15.46:0/554628413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641632efa40 con 0x56416401fc00 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe3800640 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cf800 0x564163412100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56416401fc00 0x564163410b00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x5641640175c0 tx=0x56416402cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbdca00640 1 -- 172.21.15.46:0/554628413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56416328c9c0 con 0x56416401fc00 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbdca00640 1 -- 172.21.15.46:0/554628413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641632a0540 con 0x56416401fc00 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 -- 172.21.15.46:0/554628413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5641632efc20 con 0x56416401fc00 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbdca00640 1 -- 172.21.15.46:0/554628413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56416328c4e0 con 0x56416401fc00 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 1 -- 172.21.15.46:0/554628413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641632efa40 con 0x56416401fc00 2024-04-25T08:48:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbe60c1400 4 main: JournalTool: resolving pool 13 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.780+0000 7fcbdca00640 1 -- 172.21.15.46:0/554628413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5641641c8c00 con 0x56416401fc00 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbdca00640 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x564163583400 0x56416366c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbdca00640 1 -- 172.21.15.46:0/554628413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x564163588a00 con 0x56416401fc00 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 4 main: JournalTool: creating IoCtx.. 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 4 main: Executing for rank 0 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe4200640 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x564163583400 0x56416366c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe4200640 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x564163583400 0x56416366c000 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x5641640b3410 tx=0x56416334a360 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 -- start start 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 Processor -- start 2024-04-25T08:48:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5641641cf000 0x564163412680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cec00 0x564163412c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce800 0x564163413180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564164026f00 con 0x5641641ce800 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564164026d80 con 0x5641641cec00 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564164027800 con 0x5641641cf000 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe3800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce800 0x564163413180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe3800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce800 0x564163413180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60780/0 (socket says 172.21.15.46:60780) 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe3800640 1 -- 172.21.15.46:0/1890893224 learned_addr learned my addr 172.21.15.46:0/1890893224 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe4200640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cec00 0x564163412c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5641641cf000 0x564163412680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe3800640 1 -- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5641641cf000 msgr2=0x564163412680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe3800640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5641641cf000 0x564163412680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe3800640 1 -- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cec00 msgr2=0x564163412c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe3800640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cec00 0x564163412c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe3800640 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641636c6780 con 0x5641641ce800 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe3800640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce800 0x564163413180 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x564163625560 tx=0x56416334a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124410109be56aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbd8e00640 1 -- 172.21.15.46:0/1890893224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56416328c000 con 0x5641641ce800 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbd8e00640 1 -- 172.21.15.46:0/1890893224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5641632a0c40 con 0x5641641ce800 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbd8e00640 1 -- 172.21.15.46:0/1890893224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5641635bc4e0 con 0x5641641ce800 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5641636c6960 con 0x5641641ce800 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5641636c6b40 con 0x5641641ce800 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 4 waiting for MDS map... 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbd8e00640 1 -- 172.21.15.46:0/1890893224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x5641636e0000 con 0x5641641ce800 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbd8e00640 1 -- 172.21.15.46:0/1890893224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 2723+0+0 (secure 0 0 0) 0x5641636f0000 con 0x5641641ce800 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.784+0000 7fcbe60c1400 4 Got MDS map 63 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5641641ce400 0x564163413700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) v9 -- 0x5641641ce000 con 0x5641641ce400 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5641641ce400 0x564163413700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5641641ce400 0x564163413700 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=7692a4c7d0b7bbaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe4c00640 1 -- 172.21.15.46:0/1890893224 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x5641633d2000 con 0x5641641ce400 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5641636ea400 0x56416366c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91) v9 -- 0x5641636ea800 con 0x5641636ea400 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe4200640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5641636ea400 0x56416366c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe4200640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5641636ea400 0x56416366c580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=2f5e0ec0d2bdbaad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe4200640 1 -- 172.21.15.46:0/1890893224 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x5641633d2500 con 0x5641636ea400 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbd7000640 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91) v9 -- 0x564163704000 con 0x5641636ea400 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbd7000640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x564163704400 0x56416366ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbd7000640 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e91) v9 -- 0x564163704800 con 0x564163704400 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe4200640 1 -- 172.21.15.46:0/1890893224 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x5641633d2500 con 0x5641636ea400 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x564163704400 0x56416366ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.788+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x564163704400 0x56416366ec00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=5f5f5f01df30e501 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.792+0000 7fcbe4c00640 1 -- 172.21.15.46:0/1890893224 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5641633d2000 con 0x564163704400 2024-04-25T08:48:38.789 INFO:teuthology.orchestra.run.smithi046.stdout:old journal was 4212018~2544 2024-04-25T08:48:38.789 INFO:teuthology.orchestra.run.smithi046.stdout:new journal start will be 8388608 (4174046 bytes past old end) 2024-04-25T08:48:38.790 INFO:teuthology.orchestra.run.smithi046.stdout:writing journal head 2024-04-25T08:48:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.792+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) v9 -- 0x5641636eac00 con 0x5641636ea400 2024-04-25T08:48:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe4200640 1 -- 172.21.15.46:0/1890893224 <== osd.0 v2:172.21.15.46:6800/3448226631 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v91'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5641633d2500 con 0x5641636ea400 2024-04-25T08:48:38.790 INFO:teuthology.orchestra.run.smithi046.stdout:writing EResetJournal entry 2024-04-25T08:48:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) v9 -- 0x5641636eb000 con 0x564163704400 2024-04-25T08:48:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x5641636eb400 0x56416366cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) v9 -- 0x5641636eb800 con 0x5641636eb400 2024-04-25T08:48:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) v9 -- 0x5641636ebc00 con 0x5641641ce400 2024-04-25T08:48:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x564163704c00 0x56416366d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) v9 -- 0x564163705000 con 0x564163704c00 2024-04-25T08:48:38.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) v9 -- 0x564163705400 con 0x564163704400 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x564163705800 0x56416366d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) v9 -- 0x564163705c00 con 0x564163705800 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe3800640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x564163704c00 0x56416366d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe4200640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x5641636eb400 0x56416366cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x564163705800 0x56416366d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe3800640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x564163704c00 0x56416366d080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=a0714c6a5add0667 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe4200640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x5641636eb400 0x56416366cb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=891d5def2990cc46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe4c00640 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x564163705800 0x56416366d600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=9f9a9cc640ed5423 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe4c00640 1 -- 172.21.15.46:0/1890893224 <== osd.4 v2:172.21.15.82:6800/2293006647 2 ==== osd_op_reply(6 200.00000002 [delete] v91'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5641633d2000 con 0x564163704400 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe3800640 1 -- 172.21.15.46:0/1890893224 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(9 200.00000005 [delete] v91'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5641633d2780 con 0x564163704c00 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe4c00640 1 -- 172.21.15.46:0/1890893224 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== osd_op_reply(8 200.00000004 [delete] v91'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5641633d2000 con 0x5641641ce400 2024-04-25T08:48:38.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.796+0000 7fcbe4c00640 1 -- 172.21.15.46:0/1890893224 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(11 200.00000007 [delete] v91'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5641633d2000 con 0x564163705800 2024-04-25T08:48:38.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.800+0000 7fcbe4c00640 1 -- 172.21.15.46:0/1890893224 <== osd.4 v2:172.21.15.82:6800/2293006647 3 ==== osd_op_reply(10 200.00000006 [delete] v91'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5641633d2000 con 0x564163704400 2024-04-25T08:48:38.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.800+0000 7fcbe4200640 1 -- 172.21.15.46:0/1890893224 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(7 200.00000003 [delete] v91'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5641633d2500 con 0x5641636eb400 2024-04-25T08:48:38.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.800+0000 7fcbd7000640 1 -- 172.21.15.46:0/1890893224 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e91) v9 -- 0x5641636eb000 con 0x564163704400 2024-04-25T08:48:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe4c00640 1 -- 172.21.15.46:0/1890893224 <== osd.4 v2:172.21.15.82:6800/2293006647 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v91'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5641633d2000 con 0x564163704400 2024-04-25T08:48:38.797 INFO:teuthology.orchestra.run.smithi046.stdout:done 2024-04-25T08:48:38.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5641636ea400 msgr2=0x56416366c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5641636ea400 0x56416366c580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x564163704c00 msgr2=0x56416366d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x564163704c00 0x56416366d080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x564163704400 msgr2=0x56416366ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x564163704400 0x56416366ec00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x5641636eb400 msgr2=0x56416366cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x5641636eb400 0x56416366cb00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5641641ce400 msgr2=0x564163413700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5641641ce400 0x564163413700 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x564163705800 msgr2=0x56416366d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x564163705800 0x56416366d600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe3800640 1 -- 172.21.15.46:0/1890893224 reap_dead start 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce800 msgr2=0x564163413180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce800 0x564163413180 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x564163625560 tx=0x56416334a180 comp rx=0 tx=0).stop 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 shutdown_connections 2024-04-25T08:48:38.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/1890893224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641641ce800 0x564163413180 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 >> 172.21.15.46:0/1890893224 conn(0x5641641cf400 msgr2=0x5641636c2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 shutdown_connections 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/1890893224 wait complete. 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x564163583400 msgr2=0x56416366c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x564163583400 0x56416366c000 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x5641640b3410 tx=0x56416334a360 comp rx=0 tx=0).stop 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/554628413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56416401fc00 msgr2=0x564163410b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56416401fc00 0x564163410b00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x5641640175c0 tx=0x56416402cd20 comp rx=0 tx=0).stop 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/554628413 shutdown_connections 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x564163583400 0x56416366c000 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5641641cf800 0x564163412100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56416401f800 0x564163410580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/554628413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56416401fc00 0x564163410b00 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/554628413 >> 172.21.15.46:0/554628413 conn(0x564163256c00 msgr2=0x564163285800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/554628413 shutdown_connections 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/554628413 wait complete. 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/893937389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564163257800 msgr2=0x564163411b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/893937389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564163257800 0x564163411b80 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x5641640b2a20 tx=0x56416334a000 comp rx=0 tx=0).stop 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/893937389 shutdown_connections 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/893937389 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564163257000 0x564163411080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/893937389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564163257400 0x564163411600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 --2- 172.21.15.46:0/893937389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564163257800 0x564163411b80 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/893937389 >> 172.21.15.46:0/893937389 conn(0x564163257c00 msgr2=0x564163285000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/893937389 shutdown_connections 2024-04-25T08:48:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.804+0000 7fcbe60c1400 1 -- 172.21.15.46:0/893937389 wait complete. 2024-04-25T08:48:38.804 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.127207, result: old journal was 4212018~2544 new journal start will be 8388608 (4174046 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-25T08:48:38.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T08:48:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.952+0000 7f2590800640 1 Processor -- start 2024-04-25T08:48:38.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.952+0000 7f2590800640 1 -- start start 2024-04-25T08:48:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.952+0000 7f2590800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f258806b230 0x7f25880cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.952+0000 7f2590800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f25880ce220 0x7f2588161c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f2590800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258806b610 con 0x7f25880b2ce0 2024-04-25T08:48:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f2590800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258806f980 con 0x7f25880ce220 2024-04-25T08:48:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f2590800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258806fc60 con 0x7f258806b230 2024-04-25T08:48:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f25880ce220 0x7f2588161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f25880ce220 0x7f2588161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60792/0 (socket says 172.21.15.46:60792) 2024-04-25T08:48:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258d400640 1 -- 172.21.15.46:0/2323853678 learned_addr learned my addr 172.21.15.46:0/2323853678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258de00640 1 --2- 172.21.15.46:0/2323853678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f258806b230 0x7f25880cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258d400640 1 -- 172.21.15.46:0/2323853678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f258806b230 msgr2=0x7f25880cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258d400640 1 --2- 172.21.15.46:0/2323853678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f258806b230 0x7f25880cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258d400640 1 -- 172.21.15.46:0/2323853678 >> v1:172.21.15.82:6789/0 conn(0x7f25880b2ce0 legacy=0x7f25880cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258d400640 1 -- 172.21.15.46:0/2323853678 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258806ff40 con 0x7f25880ce220 2024-04-25T08:48:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258d400640 1 --2- 172.21.15.46:0/2323853678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25880ce220 0x7f2588161c70 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2578009870 tx=0x7f257801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29958a940b2478bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258ca00640 1 -- 172.21.15.46:0/2323853678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780b0070 con 0x7f25880ce220 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258ca00640 1 -- 172.21.15.46:0/2323853678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25780ab600 con 0x7f25880ce220 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.956+0000 7f258ca00640 1 -- 172.21.15.46:0/2323853678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780ab9d0 con 0x7f25880ce220 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 -- 172.21.15.46:0/2323853678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25880ce220 msgr2=0x7f2588161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 --2- 172.21.15.46:0/2323853678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25880ce220 0x7f2588161c70 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2578009870 tx=0x7f257801c410 comp rx=0 tx=0).stop 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 -- 172.21.15.46:0/2323853678 shutdown_connections 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 --2- 172.21.15.46:0/2323853678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25880ce220 0x7f2588161c70 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 --2- 172.21.15.46:0/2323853678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f258806b230 0x7f25880cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 -- 172.21.15.46:0/2323853678 >> 172.21.15.46:0/2323853678 conn(0x7f2588099a00 msgr2=0x7f25880b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 -- 172.21.15.46:0/2323853678 shutdown_connections 2024-04-25T08:48:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 -- 172.21.15.46:0/2323853678 wait complete. 2024-04-25T08:48:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 Processor -- start 2024-04-25T08:48:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 -- start start 2024-04-25T08:48:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258806b230 0x7f25880add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25880b2ce0 0x7f25880ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258815b700 0x7f258815bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2588162330 con 0x7f258815b700 2024-04-25T08:48:38.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25881621b0 con 0x7f258806b230 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f2590800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25881624b0 con 0x7f25880b2ce0 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f258e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258815b700 0x7f258815bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f258e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258815b700 0x7f258815bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60806/0 (socket says 172.21.15.46:60806) 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f258e800640 1 -- 172.21.15.46:0/1166620021 learned_addr learned my addr 172.21.15.46:0/1166620021 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f258d400640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25880b2ce0 0x7f25880ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.960+0000 7f258de00640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258806b230 0x7f25880add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f258e800640 1 -- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25880b2ce0 msgr2=0x7f25880ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f258e800640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25880b2ce0 0x7f25880ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f258e800640 1 -- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258806b230 msgr2=0x7f25880add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f258e800640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258806b230 0x7f25880add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f258e800640 1 -- 172.21.15.46:0/1166620021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25881608e0 con 0x7f258815b700 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f258d400640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25880b2ce0 0x7f25880ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f258e800640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258815b700 0x7f258815bb90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f258400c730 tx=0x7f258400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f2582000640 1 -- 172.21.15.46:0/1166620021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584003b80 con 0x7f258815b700 2024-04-25T08:48:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f2582000640 1 -- 172.21.15.46:0/1166620021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2584003d20 con 0x7f258815b700 2024-04-25T08:48:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f2582000640 1 -- 172.21.15.46:0/1166620021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584005310 con 0x7f258815b700 2024-04-25T08:48:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2588158080 con 0x7f258815b700 2024-04-25T08:48:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2588158640 con 0x7f258815b700 2024-04-25T08:48:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2588069010 con 0x7f258815b700 2024-04-25T08:48:38.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f2582000640 1 -- 172.21.15.46:0/1166620021 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25840056c0 con 0x7f258815b700 2024-04-25T08:48:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f2582000640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f256c066f90 0x7f256c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f2582000640 1 -- 172.21.15.46:0/1166620021 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f25840851a0 con 0x7f258815b700 2024-04-25T08:48:38.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.964+0000 7f258de00640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f256c066f90 0x7f256c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:38.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.968+0000 7f258de00640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f256c066f90 0x7f256c069450 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f258815cdc0 tx=0x7f257c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:38.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:38.968+0000 7f2582000640 1 -- 172.21.15.46:0/1166620021 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25840558e0 con 0x7f258815b700 2024-04-25T08:48:39.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.164+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f25880ca8f0 con 0x7f258815b700 2024-04-25T08:48:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.592+0000 7f2582000640 1 -- 172.21.15.46:0/1166620021 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v64) v1 ==== 162+0+0 (secure 0 0 0) 0x7f2584059980 con 0x7f258815b700 2024-04-25T08:48:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T08:48:39.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f256c066f90 msgr2=0x7f256c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f256c066f90 0x7f256c069450 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f258815cdc0 tx=0x7f257c009190 comp rx=0 tx=0).stop 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258815b700 msgr2=0x7f258815bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258815b700 0x7f258815bb90 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f258400c730 tx=0x7f258400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 shutdown_connections 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f256c066f90 0x7f256c069450 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f258815b700 0x7f258815bb90 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25880b2ce0 0x7f25880ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 --2- 172.21.15.46:0/1166620021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f258806b230 0x7f25880add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 >> 172.21.15.46:0/1166620021 conn(0x7f2588099a00 msgr2=0x7f25880b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 shutdown_connections 2024-04-25T08:48:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.596+0000 7f2590800640 1 -- 172.21.15.46:0/1166620021 wait complete. 2024-04-25T08:48:39.614 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.760+0000 7f0491000640 1 Processor -- start 2024-04-25T08:48:39.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.760+0000 7f0491000640 1 -- start start 2024-04-25T08:48:39.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.760+0000 7f0491000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f048c159230 0x7f048c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.760+0000 7f0491000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cdc50 0x7f048c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.760+0000 7f0491000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c0cc700 con 0x7f048c0cbeb0 2024-04-25T08:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.760+0000 7f0491000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c0cc9e0 con 0x7f048c159230 2024-04-25T08:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.760+0000 7f0491000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0cccc0 con 0x7f048c0cdc50 2024-04-25T08:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f048c159230 0x7f048c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f048c159230 0x7f048c15bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60818/0 (socket says 172.21.15.46:60818) 2024-04-25T08:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489e00640 1 -- 172.21.15.46:0/1763997744 learned_addr learned my addr 172.21.15.46:0/1763997744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0481400640 1 --1- 172.21.15.46:0/1763997744 >> v1:172.21.15.46:6789/0 conn(0x7f048c0cbeb0 0x7f048c0cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32890/0 (socket says 172.21.15.46:32890) 2024-04-25T08:48:39.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f048a800640 1 --2- 172.21.15.46:0/1763997744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cdc50 0x7f048c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:39.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489e00640 1 -- 172.21.15.46:0/1763997744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cdc50 msgr2=0x7f048c15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489e00640 1 --2- 172.21.15.46:0/1763997744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cdc50 0x7f048c15d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489e00640 1 -- 172.21.15.46:0/1763997744 >> v1:172.21.15.46:6789/0 conn(0x7f048c0cbeb0 legacy=0x7f048c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489e00640 1 -- 172.21.15.46:0/1763997744 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c0ccfa0 con 0x7f048c159230 2024-04-25T08:48:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489400640 1 -- 172.21.15.46:0/1763997744 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526958589 0 0) 0x7f048c0cc700 con 0x7f048c0cbeb0 2024-04-25T08:48:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f048a800640 1 --2- 172.21.15.46:0/1763997744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cdc50 0x7f048c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489e00640 1 --2- 172.21.15.46:0/1763997744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c159230 0x7f048c15bdb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f047c009870 tx=0x7f047c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadf611c1346d717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489400640 1 -- 172.21.15.46:0/1763997744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0aa620 con 0x7f048c159230 2024-04-25T08:48:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489400640 1 -- 172.21.15.46:0/1763997744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c0af070 con 0x7f048c159230 2024-04-25T08:48:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.764+0000 7f0489400640 1 -- 172.21.15.46:0/1763997744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0aa9e0 con 0x7f048c159230 2024-04-25T08:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 -- 172.21.15.46:0/1763997744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c159230 msgr2=0x7f048c15bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 --2- 172.21.15.46:0/1763997744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c159230 0x7f048c15bdb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f047c009870 tx=0x7f047c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 -- 172.21.15.46:0/1763997744 shutdown_connections 2024-04-25T08:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 --2- 172.21.15.46:0/1763997744 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cdc50 0x7f048c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 --2- 172.21.15.46:0/1763997744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c159230 0x7f048c15bdb0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 -- 172.21.15.46:0/1763997744 >> 172.21.15.46:0/1763997744 conn(0x7f048c0b12a0 msgr2=0x7f048c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 -- 172.21.15.46:0/1763997744 shutdown_connections 2024-04-25T08:48:39.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 -- 172.21.15.46:0/1763997744 wait complete. 2024-04-25T08:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 Processor -- start 2024-04-25T08:48:39.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 -- start start 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cbeb0 0x7f048c1402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c0cdc50 0x7f048c140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f048c159230 0x7f048c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c15e0d0 con 0x7f048c0cdc50 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c15df50 con 0x7f048c159230 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0491000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c15e250 con 0x7f048c0cbeb0 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0481400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c0cdc50 0x7f048c140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0481400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c0cdc50 0x7f048c140890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60834/0 (socket says 172.21.15.46:60834) 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0481400640 1 -- 172.21.15.46:0/2947103284 learned_addr learned my addr 172.21.15.46:0/2947103284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:39.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0489e00640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cbeb0 0x7f048c1402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f048a800640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f048c159230 0x7f048c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0481400640 1 -- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cbeb0 msgr2=0x7f048c1402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0481400640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cbeb0 0x7f048c1402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0481400640 1 -- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f048c159230 msgr2=0x7f048c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0481400640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f048c159230 0x7f048c0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f0481400640 1 -- 172.21.15.46:0/2947103284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c141190 con 0x7f048c0cdc50 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.768+0000 7f048a800640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f048c159230 0x7f048c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0481400640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c0cdc50 0x7f048c140890 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f04740011f0 tx=0x7f047400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0482a00640 1 -- 172.21.15.46:0/2947103284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474018070 con 0x7f048c0cdc50 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0482a00640 1 -- 172.21.15.46:0/2947103284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04740074e0 con 0x7f048c0cdc50 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0482a00640 1 -- 172.21.15.46:0/2947103284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0474004a10 con 0x7f048c0cdc50 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c141470 con 0x7f048c0cdc50 2024-04-25T08:48:39.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c141a30 con 0x7f048c0cdc50 2024-04-25T08:48:39.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c069080 con 0x7f048c0cdc50 2024-04-25T08:48:39.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0482a00640 1 -- 172.21.15.46:0/2947103284 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0474004bb0 con 0x7f048c0cdc50 2024-04-25T08:48:39.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0482a00640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:39.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0482a00640 1 -- 172.21.15.46:0/2947103284 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0474084680 con 0x7f048c0cdc50 2024-04-25T08:48:39.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.772+0000 7f0489e00640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:39.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.776+0000 7f0489e00640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f047c0ae000 tx=0x7f047c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:39.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.776+0000 7f0482a00640 1 -- 172.21.15.46:0/2947103284 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0474055410 con 0x7f048c0cdc50 2024-04-25T08:48:39.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.972+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f048c0723d0 con 0x7f048c0cdc50 2024-04-25T08:48:39.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.972+0000 7f0482a00640 1 -- 172.21.15.46:0/2947103284 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0474054d10 con 0x7f048c0cdc50 2024-04-25T08:48:39.968 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:39.968 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:48:39.601355+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7656},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7656":{"gid":7656,"name":"c","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/2777781817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2777781817},{"type":"v1","addr":"172.21.15.46:6835","nonce":2777781817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:48:39.969 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 65 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 msgr2=0x7f0468069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f047c0ae000 tx=0x7f047c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c0cdc50 msgr2=0x7f048c140890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c0cdc50 0x7f048c140890 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f04740011f0 tx=0x7f047400c9b0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 shutdown_connections 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0468066f60 0x7f0468069420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f048c159230 0x7f048c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f048c0cdc50 0x7f048c140890 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 --2- 172.21.15.46:0/2947103284 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f048c0cbeb0 0x7f048c1402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:39.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 >> 172.21.15.46:0/2947103284 conn(0x7f048c0b12a0 msgr2=0x7f048c15fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 shutdown_connections 2024-04-25T08:48:39.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:39.976+0000 7f0491000640 1 -- 172.21.15.46:0/2947103284 wait complete. 2024-04-25T08:48:40.391 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:48:27.728697+0000', 'modified': '2024-04-25T08:48:39.601355+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 91, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7656}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7656': {'gid': 7656, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/2777781817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2777781817}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2777781817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:48:40.391 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7656:up:replay 2024-04-25T08:48:41.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f4007400640 1 Processor -- start 2024-04-25T08:48:41.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f4007400640 1 -- start start 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f4007400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40000b2d00 0x7f400006b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f4007400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f40000c5450 0x7f40000c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f4007400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40000af7f0 con 0x7f400015dca0 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f4007400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40000afad0 con 0x7f40000b2d00 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f4007400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40000c6b40 con 0x7f40000c5450 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f3ff7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40000b2d00 0x7f400006b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f3ff7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40000b2d00 0x7f400006b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60838/0 (socket says 172.21.15.46:60838) 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f3ff7e00640 1 -- 172.21.15.46:0/1058928453 learned_addr learned my addr 172.21.15.46:0/1058928453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f4005400640 1 --2- 172.21.15.46:0/1058928453 >> v2:172.21.15.82:3300/0 conn(0x7f40000c5450 0x7f40000c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f3ff7e00640 1 -- 172.21.15.46:0/1058928453 >> v2:172.21.15.82:3300/0 conn(0x7f40000c5450 msgr2=0x7f40000c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f3ff7e00640 1 --2- 172.21.15.46:0/1058928453 >> v2:172.21.15.82:3300/0 conn(0x7f40000c5450 0x7f40000c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f3ff7e00640 1 -- 172.21.15.46:0/1058928453 >> v1:172.21.15.82:6789/0 conn(0x7f400015dca0 legacy=0x7f400006ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:41.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.552+0000 7f3ff7e00640 1 -- 172.21.15.46:0/1058928453 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40000c7340 con 0x7f40000b2d00 2024-04-25T08:48:41.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f3ff7e00640 1 --2- 172.21.15.46:0/1058928453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 0x7f400006b320 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f40000cdfd0 tx=0x7f3fe801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a3daee58676a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f3fffe00640 1 -- 172.21.15.46:0/1058928453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80ab600 con 0x7f40000b2d00 2024-04-25T08:48:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f3fffe00640 1 -- 172.21.15.46:0/1058928453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe80b0070 con 0x7f40000b2d00 2024-04-25T08:48:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f3fffe00640 1 -- 172.21.15.46:0/1058928453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80ab9c0 con 0x7f40000b2d00 2024-04-25T08:48:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f4007400640 1 -- 172.21.15.46:0/1058928453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 msgr2=0x7f400006b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f4007400640 1 --2- 172.21.15.46:0/1058928453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 0x7f400006b320 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f40000cdfd0 tx=0x7f3fe801c4f0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f4007400640 1 -- 172.21.15.46:0/1058928453 shutdown_connections 2024-04-25T08:48:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f4007400640 1 --2- 172.21.15.46:0/1058928453 >> v2:172.21.15.82:3300/0 conn(0x7f40000c5450 0x7f40000c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f4007400640 1 --2- 172.21.15.46:0/1058928453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 0x7f400006b320 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f4007400640 1 -- 172.21.15.46:0/1058928453 >> 172.21.15.46:0/1058928453 conn(0x7f40000b21e0 msgr2=0x7f40000b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f4007400640 1 -- 172.21.15.46:0/1058928453 shutdown_connections 2024-04-25T08:48:41.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.556+0000 7f4007400640 1 -- 172.21.15.46:0/1058928453 wait complete. 2024-04-25T08:48:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 Processor -- start 2024-04-25T08:48:41.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 -- start start 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 0x7f4000156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40000c5450 0x7f40001530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f400015dca0 0x7f40001513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400016a8d0 con 0x7f40000b2d00 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400016aa50 con 0x7f40000c5450 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400016abd0 con 0x7f400015dca0 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4004a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 0x7f4000156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4004a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 0x7f4000156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60852/0 (socket says 172.21.15.46:60852) 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4004a00640 1 -- 172.21.15.46:0/980149341 learned_addr learned my addr 172.21.15.46:0/980149341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f3ff7e00640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40000c5450 0x7f40001530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:41.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4005400640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f400015dca0 0x7f40001513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4004a00640 1 -- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f400015dca0 msgr2=0x7f40001513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4004a00640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f400015dca0 0x7f40001513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4004a00640 1 -- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40000c5450 msgr2=0x7f40001530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4004a00640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40000c5450 0x7f40001530c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4004a00640 1 -- 172.21.15.46:0/980149341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4000152060 con 0x7f40000b2d00 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f3ff7e00640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40000c5450 0x7f40001530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4004a00640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 0x7f4000156a80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c370 tx=0x7f3ff000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f3ffd600640 1 -- 172.21.15.46:0/980149341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00052d0 con 0x7f40000b2d00 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f3ffd600640 1 -- 172.21.15.46:0/980149341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0005470 con 0x7f40000b2d00 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f3ffd600640 1 -- 172.21.15.46:0/980149341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0007260 con 0x7f40000b2d00 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4000152300 con 0x7f40000b2d00 2024-04-25T08:48:41.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.560+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40001528c0 con 0x7f40000b2d00 2024-04-25T08:48:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.564+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4000069080 con 0x7f40000b2d00 2024-04-25T08:48:41.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.564+0000 7f3ffd600640 1 -- 172.21.15.46:0/980149341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ff0007400 con 0x7f40000b2d00 2024-04-25T08:48:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.564+0000 7f3ffd600640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fdc066f60 0x7f3fdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.564+0000 7f3ffd600640 1 -- 172.21.15.46:0/980149341 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3ff00846b0 con 0x7f40000b2d00 2024-04-25T08:48:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.564+0000 7f3ff7e00640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fdc066f60 0x7f3fdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:41.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.564+0000 7f3ff7e00640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fdc066f60 0x7f3fdc069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80af000 tx=0x7f3fe80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:41.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.568+0000 7f3ffd600640 1 -- 172.21.15.46:0/980149341 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ff00554d0 con 0x7f40000b2d00 2024-04-25T08:48:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40000b2900 con 0x7f40000b2d00 2024-04-25T08:48:41.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f3ffd600640 1 -- 172.21.15.46:0/980149341 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f3ff0054dd0 con 0x7f40000b2d00 2024-04-25T08:48:41.761 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:41.762 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:48:40.609877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7656},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7656":{"gid":7656,"name":"c","rank":0,"incarnation":65,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/2777781817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2777781817},{"type":"v1","addr":"172.21.15.46:6835","nonce":2777781817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:48:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 67 2024-04-25T08:48:41.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fdc066f60 msgr2=0x7f3fdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fdc066f60 0x7f3fdc069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80af000 tx=0x7f3fe80a7000 comp rx=0 tx=0).stop 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 msgr2=0x7f4000156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 0x7f4000156a80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c370 tx=0x7f3ff000c830 comp rx=0 tx=0).stop 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 shutdown_connections 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3fdc066f60 0x7f3fdc069420 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f400015dca0 0x7f40001513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40000c5450 0x7f40001530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 --2- 172.21.15.46:0/980149341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40000b2d00 0x7f4000156a80 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 >> 172.21.15.46:0/980149341 conn(0x7f40000b21e0 msgr2=0x7f40000b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 shutdown_connections 2024-04-25T08:48:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:41.768+0000 7f4007400640 1 -- 172.21.15.46:0/980149341 wait complete. 2024-04-25T08:48:42.186 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:48:27.728697+0000', 'modified': '2024-04-25T08:48:40.609877+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 91, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7656}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7656': {'gid': 7656, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/2777781817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2777781817}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2777781817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:48:42.186 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7656:up:rejoin 2024-04-25T08:48:43.187 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:48:43.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed59c00640 1 Processor -- start 2024-04-25T08:48:43.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed59c00640 1 -- start start 2024-04-25T08:48:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed59c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed5406f320 0x7fed54162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed59c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed59c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed540b35b0 con 0x7fed5415a750 2024-04-25T08:48:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed59c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5415ab30 con 0x7fed5406f320 2024-04-25T08:48:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed59c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed540ce4f0 con 0x7fed540aed10 2024-04-25T08:48:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed4a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fed5406f320 0x7fed54162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed52a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fed5415a750 0x7fed5406ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32900/0 (socket says 172.21.15.46:32900) 2024-04-25T08:48:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed52a00640 1 -- 172.21.15.46:0/1762697553 learned_addr learned my addr 172.21.15.46:0/1762697553 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed53400640 1 --2- 172.21.15.46:0/1762697553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed4a000640 1 -- 172.21.15.46:0/1762697553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 msgr2=0x7fed54164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed4a000640 1 --2- 172.21.15.46:0/1762697553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed4a000640 1 -- 172.21.15.46:0/1762697553 >> v1:172.21.15.46:6789/0 conn(0x7fed5415a750 legacy=0x7fed5406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed4a000640 1 -- 172.21.15.46:0/1762697553 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5406d5e0 con 0x7fed5406f320 2024-04-25T08:48:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed53400640 1 --2- 172.21.15.46:0/1762697553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.348+0000 7fed52000640 1 -- 172.21.15.46:0/1762697553 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113229333 0 0) 0x7fed540b35b0 con 0x7fed5415a750 2024-04-25T08:48:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed4a000640 1 --2- 172.21.15.46:0/1762697553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5406f320 0x7fed54162200 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0097c0 tx=0x7fed3c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397971e042618aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed52000640 1 -- 172.21.15.46:0/1762697553 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0b2070 con 0x7fed5406f320 2024-04-25T08:48:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed52000640 1 -- 172.21.15.46:0/1762697553 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c0ad640 con 0x7fed5406f320 2024-04-25T08:48:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed52000640 1 -- 172.21.15.46:0/1762697553 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0ada10 con 0x7fed5406f320 2024-04-25T08:48:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed59c00640 1 -- 172.21.15.46:0/1762697553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5406f320 msgr2=0x7fed54162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed59c00640 1 --2- 172.21.15.46:0/1762697553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5406f320 0x7fed54162200 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0097c0 tx=0x7fed3c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed59c00640 1 -- 172.21.15.46:0/1762697553 shutdown_connections 2024-04-25T08:48:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed59c00640 1 --2- 172.21.15.46:0/1762697553 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed59c00640 1 --2- 172.21.15.46:0/1762697553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5406f320 0x7fed54162200 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed59c00640 1 -- 172.21.15.46:0/1762697553 >> 172.21.15.46:0/1762697553 conn(0x7fed540b12a0 msgr2=0x7fed540b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed59c00640 1 -- 172.21.15.46:0/1762697553 shutdown_connections 2024-04-25T08:48:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.352+0000 7fed59c00640 1 -- 172.21.15.46:0/1762697553 wait complete. 2024-04-25T08:48:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 Processor -- start 2024-04-25T08:48:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 -- start start 2024-04-25T08:48:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed5406f320 0x7fed5414d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5415a750 0x7fed54159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5406d270 con 0x7fed5415a750 2024-04-25T08:48:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed540cf200 con 0x7fed5406f320 2024-04-25T08:48:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed541674b0 con 0x7fed540aed10 2024-04-25T08:48:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed53400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5415a750 0x7fed54159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed53400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5415a750 0x7fed54159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60866/0 (socket says 172.21.15.46:60866) 2024-04-25T08:48:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed53400640 1 -- 172.21.15.46:0/2480994557 learned_addr learned my addr 172.21.15.46:0/2480994557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed52a00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed5406f320 0x7fed5414d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed4a000640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed53400640 1 -- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 msgr2=0x7fed54146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed53400640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed53400640 1 -- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed5406f320 msgr2=0x7fed5414d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed53400640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed5406f320 0x7fed5414d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed53400640 1 -- 172.21.15.46:0/2480994557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed54163130 con 0x7fed5415a750 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed52a00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed5406f320 0x7fed5414d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed4a000640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed53400640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5415a750 0x7fed54159290 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00cef0 tx=0x7fed4c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed4b400640 1 -- 172.21.15.46:0/2480994557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c01a070 con 0x7fed5415a750 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed4b400640 1 -- 172.21.15.46:0/2480994557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c00d860 con 0x7fed5415a750 2024-04-25T08:48:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed4b400640 1 -- 172.21.15.46:0/2480994557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c00dc50 con 0x7fed5415a750 2024-04-25T08:48:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed54163410 con 0x7fed5415a750 2024-04-25T08:48:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.356+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed54163f40 con 0x7fed5415a750 2024-04-25T08:48:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.360+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed54069080 con 0x7fed5415a750 2024-04-25T08:48:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.360+0000 7fed4b400640 1 -- 172.21.15.46:0/2480994557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed4c01d070 con 0x7fed5415a750 2024-04-25T08:48:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.360+0000 7fed4b400640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed30066f90 0x7fed30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.360+0000 7fed4b400640 1 -- 172.21.15.46:0/2480994557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fed4c085b20 con 0x7fed5415a750 2024-04-25T08:48:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.360+0000 7fed52a00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed30066f90 0x7fed30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.360+0000 7fed52a00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed30066f90 0x7fed30069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fed44000f70 tx=0x7fed44007cd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.364+0000 7fed4b400640 1 -- 172.21.15.46:0/2480994557 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed4c056970 con 0x7fed5415a750 2024-04-25T08:48:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.560+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed541647b0 con 0x7fed5415a750 2024-04-25T08:48:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.560+0000 7fed4b400640 1 -- 172.21.15.46:0/2480994557 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fed4c056270 con 0x7fed5415a750 2024-04-25T08:48:43.556 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:48:43.556 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:48:41.615064+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7656},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7656":{"gid":7656,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/2777781817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2777781817},{"type":"v1","addr":"172.21.15.46:6835","nonce":2777781817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:48:43.556 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 68 2024-04-25T08:48:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed30066f90 msgr2=0x7fed30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed30066f90 0x7fed30069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fed44000f70 tx=0x7fed44007cd0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5415a750 msgr2=0x7fed54159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5415a750 0x7fed54159290 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fed4c00cef0 tx=0x7fed4c00a710 comp rx=0 tx=0).stop 2024-04-25T08:48:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 shutdown_connections 2024-04-25T08:48:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fed30066f90 0x7fed30069450 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed5415a750 0x7fed54159290 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fed540aed10 0x7fed54146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 --2- 172.21.15.46:0/2480994557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed5406f320 0x7fed5414d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 >> 172.21.15.46:0/2480994557 conn(0x7fed540b12a0 msgr2=0x7fed54150fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 shutdown_connections 2024-04-25T08:48:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:43.564+0000 7fed59c00640 1 -- 172.21.15.46:0/2480994557 wait complete. 2024-04-25T08:48:43.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:48:27.728697+0000', 'modified': '2024-04-25T08:48:41.615064+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 91, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7656}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7656': {'gid': 7656, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/2777781817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 2777781817}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2777781817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:48:43.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:48:43.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:48:44.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2213e00640 1 Processor -- start 2024-04-25T08:48:44.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2213e00640 1 -- start start 2024-04-25T08:48:44.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2213e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f221406ac10 0x7f221406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2213e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 0x7f2214145690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2213e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22140b4f80 con 0x7f22140b0be0 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2213e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22140b5590 con 0x7f221406ac10 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2213e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221406b5b0 con 0x7f2214145200 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f220a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f221406ac10 0x7f221406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2212a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f22140b0be0 0x7f22140b3380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32916/0 (socket says 172.21.15.46:32916) 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2212a00640 1 -- 172.21.15.46:0/823599342 learned_addr learned my addr 172.21.15.46:0/823599342 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f220a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f221406ac10 0x7f221406b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60874/0 (socket says 172.21.15.46:60874) 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2213400640 1 --2- 172.21.15.46:0/823599342 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 0x7f2214145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f220a000640 1 -- 172.21.15.46:0/823599342 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 msgr2=0x7f2214145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f220a000640 1 --2- 172.21.15.46:0/823599342 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 0x7f2214145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f220a000640 1 -- 172.21.15.46:0/823599342 >> v1:172.21.15.46:6789/0 conn(0x7f22140b0be0 legacy=0x7f22140b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f220a000640 1 -- 172.21.15.46:0/823599342 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22140b1b40 con 0x7f221406ac10 2024-04-25T08:48:44.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2212000640 1 -- 172.21.15.46:0/823599342 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415392276 0 0) 0x7f22140b4f80 con 0x7f22140b0be0 2024-04-25T08:48:44.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.132+0000 7f2213400640 1 --2- 172.21.15.46:0/823599342 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 0x7f2214145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f220a000640 1 --2- 172.21.15.46:0/823599342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221406ac10 0x7f221406b070 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009d90 tx=0x7f21fc01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283a97291c4a596a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2212000640 1 -- 172.21.15.46:0/823599342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0b2070 con 0x7f221406ac10 2024-04-25T08:48:44.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2212000640 1 -- 172.21.15.46:0/823599342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc0a5990 con 0x7f221406ac10 2024-04-25T08:48:44.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2212000640 1 -- 172.21.15.46:0/823599342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0a5d60 con 0x7f221406ac10 2024-04-25T08:48:44.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2213e00640 1 -- 172.21.15.46:0/823599342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221406ac10 msgr2=0x7f221406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2213e00640 1 --2- 172.21.15.46:0/823599342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221406ac10 0x7f221406b070 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009d90 tx=0x7f21fc01c8b0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2213e00640 1 -- 172.21.15.46:0/823599342 shutdown_connections 2024-04-25T08:48:44.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2213e00640 1 --2- 172.21.15.46:0/823599342 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 0x7f2214145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2213e00640 1 --2- 172.21.15.46:0/823599342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221406ac10 0x7f221406b070 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2213e00640 1 -- 172.21.15.46:0/823599342 >> 172.21.15.46:0/823599342 conn(0x7f2214099a00 msgr2=0x7f22140b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:44.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2213e00640 1 -- 172.21.15.46:0/823599342 shutdown_connections 2024-04-25T08:48:44.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.136+0000 7f2213e00640 1 -- 172.21.15.46:0/823599342 wait complete. 2024-04-25T08:48:44.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 Processor -- start 2024-04-25T08:48:44.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 -- start start 2024-04-25T08:48:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f221406ac10 0x7f22140add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22140b0be0 0x7f2214160970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 0x7f221415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22140b1140 con 0x7f22140b0be0 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22140b0fc0 con 0x7f221406ac10 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22140b1760 con 0x7f2214145200 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22140b0be0 0x7f2214160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22140b0be0 0x7f2214160970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60886/0 (socket says 172.21.15.46:60886) 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220a000640 1 -- 172.21.15.46:0/796887086 learned_addr learned my addr 172.21.15.46:0/796887086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2212a00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f221406ac10 0x7f22140add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213400640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 0x7f221415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220a000640 1 -- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 msgr2=0x7f221415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220a000640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 0x7f221415c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220a000640 1 -- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f221406ac10 msgr2=0x7f22140add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220a000640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f221406ac10 0x7f22140add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220a000640 1 -- 172.21.15.46:0/796887086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221415c8b0 con 0x7f22140b0be0 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220a000640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22140b0be0 0x7f2214160970 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009d90 tx=0x7f21fc019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220be00640 1 -- 172.21.15.46:0/796887086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0b2050 con 0x7f22140b0be0 2024-04-25T08:48:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220be00640 1 -- 172.21.15.46:0/796887086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc0be070 con 0x7f22140b0be0 2024-04-25T08:48:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f220be00640 1 -- 172.21.15.46:0/796887086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0adad0 con 0x7f22140b0be0 2024-04-25T08:48:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 -- 172.21.15.46:0/796887086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22140ae470 con 0x7f22140b0be0 2024-04-25T08:48:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.140+0000 7f2213e00640 1 -- 172.21.15.46:0/796887086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221415e650 con 0x7f22140b0be0 2024-04-25T08:48:44.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f220be00640 1 -- 172.21.15.46:0/796887086 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22141463e0 con 0x7f22140b0be0 2024-04-25T08:48:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f220be00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f21f0066f60 0x7f21f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f220be00640 1 -- 172.21.15.46:0/796887086 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f21fc1203d0 con 0x7f22140b0be0 2024-04-25T08:48:44.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f2212a00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f21f0066f60 0x7f21f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f221d1e6000 1 Processor -- start 2024-04-25T08:48:44.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f221d1e6000 1 -- start start 2024-04-25T08:48:44.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f2212a00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f21f0066f60 0x7f21f0069420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2204000f70 tx=0x7f2204007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b690cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b68c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b68c3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f221d1e6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b67aa5f0 con 0x55d4b68c2f30 2024-04-25T08:48:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f221d1e6000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b6711da0 con 0x55d4b690d240 2024-04-25T08:48:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f221d1e6000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4b669a150 con 0x55d4b690c920 2024-04-25T08:48:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f2213400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b68c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f2213400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b68c3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60890/0 (socket says 172.21.15.46:60890) 2024-04-25T08:48:44.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f2213400640 1 -- 172.21.15.46:0/3638012995 learned_addr learned my addr 172.21.15.46:0/3638012995 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.144+0000 7f2212a00640 1 --2- 172.21.15.46:0/3638012995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b690cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f220a000640 1 --2- 172.21.15.46:0/3638012995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b68c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2213400640 1 -- 172.21.15.46:0/3638012995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 msgr2=0x55d4b690cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2213400640 1 --2- 172.21.15.46:0/3638012995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b690cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2213400640 1 -- 172.21.15.46:0/3638012995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 msgr2=0x55d4b68c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2213400640 1 --2- 172.21.15.46:0/3638012995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b68c29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2213400640 1 -- 172.21.15.46:0/3638012995 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4b66b3ca0 con 0x55d4b68c2f30 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2213400640 1 --2- 172.21.15.46:0/3638012995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b68c3310 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f220c00caa0 tx=0x7f220c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8db03dac27297d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2203e00640 1 -- 172.21.15.46:0/3638012995 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c018020 con 0x55d4b68c2f30 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2203e00640 1 -- 172.21.15.46:0/3638012995 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c00eb70 con 0x55d4b68c2f30 2024-04-25T08:48:44.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2203e00640 1 -- 172.21.15.46:0/3638012995 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c013450 con 0x55d4b68c2f30 2024-04-25T08:48:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 -- 172.21.15.46:0/3638012995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 msgr2=0x55d4b68c3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 --2- 172.21.15.46:0/3638012995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b68c3310 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f220c00caa0 tx=0x7f220c00cf60 comp rx=0 tx=0).stop 2024-04-25T08:48:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 -- 172.21.15.46:0/3638012995 shutdown_connections 2024-04-25T08:48:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 --2- 172.21.15.46:0/3638012995 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b68c3310 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 --2- 172.21.15.46:0/3638012995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b68c29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 --2- 172.21.15.46:0/3638012995 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b690cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 -- 172.21.15.46:0/3638012995 >> 172.21.15.46:0/3638012995 conn(0x55d4b68c92a0 msgr2=0x55d4b68c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 -- 172.21.15.46:0/3638012995 shutdown_connections 2024-04-25T08:48:44.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 -- 172.21.15.46:0/3638012995 wait complete. 2024-04-25T08:48:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 -- start start 2024-04-25T08:48:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 Processor -- start 2024-04-25T08:48:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b68c2f30 0x55d4b68f9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690c920 0x55d4b68c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b690d240 0x55d4b68c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b66cc990 con 0x55d4b690d240 2024-04-25T08:48:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b670de40 con 0x55d4b690c920 2024-04-25T08:48:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4b669d900 con 0x55d4b68c2f30 2024-04-25T08:48:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f221d1e6000 10 client.0 _ll_register_callbacks cb 0x55d4b67c51c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2213400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b690d240 0x55d4b68c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2213400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b690d240 0x55d4b68c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60896/0 (socket says 172.21.15.46:60896) 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.148+0000 7f2213400640 1 -- 172.21.15.46:0/2511329534 learned_addr learned my addr 172.21.15.46:0/2511329534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2213400640 1 -- 172.21.15.46:0/2511329534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b68c2f30 msgr2=0x55d4b68f9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2213400640 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b68c2f30 0x55d4b68f9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2213400640 1 -- 172.21.15.46:0/2511329534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690c920 msgr2=0x55d4b68c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2213400640 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690c920 0x55d4b68c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2213400640 1 -- 172.21.15.46:0/2511329534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4b66ca630 con 0x55d4b690d240 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2213400640 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b690d240 0x55d4b68c6fe0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f220c017040 tx=0x7f220c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f052c3fbd23a637e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 1 -- 172.21.15.46:0/2511329534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c023070 con 0x55d4b690d240 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 1 -- 172.21.15.46:0/2511329534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f220c028c90 con 0x55d4b690d240 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 -- 172.21.15.46:0/2511329534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f220c00eb70 con 0x55d4b690d240 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 1 -- 172.21.15.46:0/2511329534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f220c028e90 con 0x55d4b690d240 2024-04-25T08:48:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 10 client.7725 fetch_fsmap learned FSMap version 68 2024-04-25T08:48:44.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 -- 172.21.15.46:0/2511329534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55d4b669a3a0 con 0x55d4b690d240 2024-04-25T08:48:44.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 1 -- 172.21.15.46:0/2511329534 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c00eb70 con 0x55d4b690d240 2024-04-25T08:48:44.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 1 -- 172.21.15.46:0/2511329534 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f220c003a30 con 0x55d4b690d240 2024-04-25T08:48:44.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 10 client.7725 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T08:48:44.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 10 client.7725 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:48:44.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 20 client.7725 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:44.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d4b691bcd0 0x55d4b691e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 4 client.7725 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:48:44.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 -- 172.21.15.46:0/2511329534 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4b66ada50 con 0x55d4b691bcd0 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2212a00640 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d4b691bcd0 0x55d4b691e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2212a00640 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d4b691bcd0 0x55d4b691e0c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdcb16a4c94ed129 server_cookie=26581d40ac7120c6 in_seq=0 out_seq=0 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 10 client.7725 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 1 -- 172.21.15.46:0/2511329534 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d4b66ada50 con 0x55d4b691bcd0 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 10 client.7725 handle_command_reply: tid=0 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 client.7725 shutdown 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 -- 172.21.15.46:0/2511329534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b690d240 msgr2=0x55d4b68c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b690d240 0x55d4b68c6fe0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f220c017040 tx=0x7f220c004740 comp rx=0 tx=0).stop 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 -- 172.21.15.46:0/2511329534 shutdown_connections 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d4b691bcd0 0x55d4b691e0c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b690d240 0x55d4b68c6fe0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690c920 0x55d4b68c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 --2- 172.21.15.46:0/2511329534 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b68c2f30 0x55d4b68f9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f2200c00640 0 client.7725 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 -- 172.21.15.46:0/2511329534 >> 172.21.15.46:0/2511329534 conn(0x55d4b68c92a0 msgr2=0x55d4b68c19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 -- 172.21.15.46:0/2511329534 shutdown_connections 2024-04-25T08:48:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 1 -- 172.21.15.46:0/2511329534 wait complete. 2024-04-25T08:48:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.152+0000 7f221d1e6000 20 client.7725 trim_cache size 0 max 16384 2024-04-25T08:48:44.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.168+0000 7f221d1e6000 1 Processor -- start 2024-04-25T08:48:44.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.168+0000 7f221d1e6000 1 -- start start 2024-04-25T08:48:44.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b691c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b691cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2214202030 con 0x55d4b68c2f30 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221415fe20 con 0x55d4b690d240 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2214201a30 con 0x55d4b690c920 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2212a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2212a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60912/0 (socket says 172.21.15.46:60912) 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2212a00640 1 -- 172.21.15.46:0/3581065241 learned_addr learned my addr 172.21.15.46:0/3581065241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f220a000640 1 --2- 172.21.15.46:0/3581065241 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b691c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2213400640 1 --2- 172.21.15.46:0/3581065241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b691cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2212a00640 1 -- 172.21.15.46:0/3581065241 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 msgr2=0x55d4b691c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2212a00640 1 --2- 172.21.15.46:0/3581065241 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b691c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2212a00640 1 -- 172.21.15.46:0/3581065241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 msgr2=0x55d4b691cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2212a00640 1 --2- 172.21.15.46:0/3581065241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b691cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2212a00640 1 -- 172.21.15.46:0/3581065241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4b690d070 con 0x55d4b68c2f30 2024-04-25T08:48:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2212a00640 1 --2- 172.21.15.46:0/3581065241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691c0b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55d4b6710e30 tx=0x7f2204011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1b900c24ae7909 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2203400640 1 -- 172.21.15.46:0/3581065241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204029070 con 0x55d4b68c2f30 2024-04-25T08:48:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2203400640 1 -- 172.21.15.46:0/3581065241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22040071f0 con 0x55d4b68c2f30 2024-04-25T08:48:44.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f2203400640 1 -- 172.21.15.46:0/3581065241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22040138f0 con 0x55d4b68c2f30 2024-04-25T08:48:44.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- 172.21.15.46:0/3581065241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 msgr2=0x55d4b691c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- 172.21.15.46:0/3581065241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691c0b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x55d4b6710e30 tx=0x7f2204011000 comp rx=0 tx=0).stop 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- 172.21.15.46:0/3581065241 shutdown_connections 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- 172.21.15.46:0/3581065241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b691cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- 172.21.15.46:0/3581065241 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b691c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- 172.21.15.46:0/3581065241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691c0b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- 172.21.15.46:0/3581065241 >> 172.21.15.46:0/3581065241 conn(0x55d4b68c92a0 msgr2=0x55d4b690aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- 172.21.15.46:0/3581065241 shutdown_connections 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- 172.21.15.46:0/3581065241 wait complete. 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- start start 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 Processor -- start 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b691fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b6950fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b67163d0 con 0x55d4b68c2f30 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4b67cdd50 con 0x55d4b690d240 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d4b66ac980 con 0x55d4b690c920 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.172+0000 7f221d1e6000 10 client.0 _ll_register_callbacks cb 0x7f220c00c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2212a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2212a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60916/0 (socket says 172.21.15.46:60916) 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2212a00640 1 -- 172.21.15.46:0/4113678390 learned_addr learned my addr 172.21.15.46:0/4113678390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213400640 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b6950fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f220a000640 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b691fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2212a00640 1 -- 172.21.15.46:0/4113678390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 msgr2=0x55d4b691fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2212a00640 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b691fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2212a00640 1 -- 172.21.15.46:0/4113678390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 msgr2=0x55d4b6950fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2212a00640 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b6950fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2212a00640 1 -- 172.21.15.46:0/4113678390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4b6793600 con 0x55d4b68c2f30 2024-04-25T08:48:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213400640 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b6950fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2212a00640 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691f6d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2204027040 tx=0x7f2204000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d07f4a6367fb420 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 1 -- 172.21.15.46:0/4113678390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204033060 con 0x55d4b68c2f30 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 1 -- 172.21.15.46:0/4113678390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2204024280 con 0x55d4b68c2f30 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 -- 172.21.15.46:0/4113678390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4b671f2e0 con 0x55d4b68c2f30 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 1 -- 172.21.15.46:0/4113678390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2204039050 con 0x55d4b68c2f30 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 10 client.7731 fetch_fsmap learned FSMap version 68 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 -- 172.21.15.46:0/4113678390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55d4b66a69b0 con 0x55d4b68c2f30 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 1 -- 172.21.15.46:0/4113678390 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d4b671f2e0 con 0x55d4b68c2f30 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 1 -- 172.21.15.46:0/4113678390 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f220403c020 con 0x55d4b68c2f30 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 10 client.7731 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 10 client.7731 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 20 client.7731 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d4b6951520 0x55d4b6951900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 4 client.7731 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:48:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 -- 172.21.15.46:0/4113678390 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d4b6719630 con 0x55d4b6951520 2024-04-25T08:48:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f220a000640 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d4b6951520 0x55d4b6951900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f220a000640 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d4b6951520 0x55d4b6951900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39438d561edb1e4 server_cookie=8b246a335ae167a8 in_seq=0 out_seq=0 2024-04-25T08:48:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 10 client.7731 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 1 -- 172.21.15.46:0/4113678390 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d4b6719630 con 0x55d4b6951520 2024-04-25T08:48:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 10 client.7731 handle_command_reply: tid=0 2024-04-25T08:48:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 client.7731 shutdown 2024-04-25T08:48:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 -- 172.21.15.46:0/4113678390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 msgr2=0x55d4b691f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691f6d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2204027040 tx=0x7f2204000ee0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 -- 172.21.15.46:0/4113678390 shutdown_connections 2024-04-25T08:48:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d4b6951520 0x55d4b6951900 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d4b690d240 0x55d4b6950fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d4b690c920 0x55d4b691fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 --2- 172.21.15.46:0/4113678390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4b68c2f30 0x55d4b691f6d0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2200c00640 0 client.7731 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 -- 172.21.15.46:0/4113678390 >> 172.21.15.46:0/4113678390 conn(0x55d4b68c92a0 msgr2=0x55d4b690aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 -- 172.21.15.46:0/4113678390 shutdown_connections 2024-04-25T08:48:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 1 -- 172.21.15.46:0/4113678390 wait complete. 2024-04-25T08:48:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f221d1e6000 20 client.7731 trim_cache size 0 max 16384 2024-04-25T08:48:44.176 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7656, 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5748646949999996, 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 68, 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 91, 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.5195898899999998 2024-04-25T08:48:44.177 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:48:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 -- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f21f0066f60 msgr2=0x7f21f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f21f0066f60 0x7f21f0069420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2204000f70 tx=0x7f2204007b60 comp rx=0 tx=0).stop 2024-04-25T08:48:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 -- 172.21.15.46:0/796887086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22140b0be0 msgr2=0x7f2214160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22140b0be0 0x7f2214160970 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009d90 tx=0x7f21fc019650 comp rx=0 tx=0).stop 2024-04-25T08:48:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 -- 172.21.15.46:0/796887086 shutdown_connections 2024-04-25T08:48:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f21f0066f60 0x7f21f0069420 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2214145200 0x7f221415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22140b0be0 0x7f2214160970 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 --2- 172.21.15.46:0/796887086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f221406ac10 0x7f22140add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.176+0000 7f2213e00640 1 -- 172.21.15.46:0/796887086 >> 172.21.15.46:0/796887086 conn(0x7f2214099a00 msgr2=0x7f221406f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.180+0000 7f2213e00640 1 -- 172.21.15.46:0/796887086 shutdown_connections 2024-04-25T08:48:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.180+0000 7f2213e00640 1 -- 172.21.15.46:0/796887086 wait complete. 2024-04-25T08:48:44.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:48:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8f200640 1 Processor -- start 2024-04-25T08:48:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8f200640 1 -- start start 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8f200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0f8814b8c0 0x7f0f880b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8806ac10 0x7f0f8806b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f881634a0 0x7f0f88163930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f8806b590 con 0x7f0f881634a0 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8f200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f88162d10 con 0x7f0f8814b8c0 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880b5010 con 0x7f0f8806ac10 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f881634a0 0x7f0f88163930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f881634a0 0x7f0f88163930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60932/0 (socket says 172.21.15.46:60932) 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8e800640 1 -- 172.21.15.46:0/339937369 learned_addr learned my addr 172.21.15.46:0/339937369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8d400640 1 --2- 172.21.15.46:0/339937369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8806ac10 0x7f0f8806b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8de00640 1 --2- 172.21.15.46:0/339937369 >> v2:172.21.15.82:3300/0 conn(0x7f0f8814b8c0 0x7f0f880b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8e800640 1 -- 172.21.15.46:0/339937369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8806ac10 msgr2=0x7f0f8806b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8e800640 1 --2- 172.21.15.46:0/339937369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8806ac10 0x7f0f8806b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8e800640 1 -- 172.21.15.46:0/339937369 >> v2:172.21.15.82:3300/0 conn(0x7f0f8814b8c0 msgr2=0x7f0f880b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8e800640 1 --2- 172.21.15.46:0/339937369 >> v2:172.21.15.82:3300/0 conn(0x7f0f8814b8c0 0x7f0f880b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.340+0000 7f0f8e800640 1 -- 172.21.15.46:0/339937369 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f880b52f0 con 0x7f0f881634a0 2024-04-25T08:48:44.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8e800640 1 --2- 172.21.15.46:0/339937369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 0x7f0f88163930 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0f880d14a0 tx=0x7f0f8401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da29f486885dd29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8ca00640 1 -- 172.21.15.46:0/339937369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840b4070 con 0x7f0f881634a0 2024-04-25T08:48:44.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8ca00640 1 -- 172.21.15.46:0/339937369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f840af400 con 0x7f0f881634a0 2024-04-25T08:48:44.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8ca00640 1 -- 172.21.15.46:0/339937369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840af7d0 con 0x7f0f881634a0 2024-04-25T08:48:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8f200640 1 -- 172.21.15.46:0/339937369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 msgr2=0x7f0f88163930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8f200640 1 --2- 172.21.15.46:0/339937369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 0x7f0f88163930 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0f880d14a0 tx=0x7f0f8401f1e0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8f200640 1 -- 172.21.15.46:0/339937369 shutdown_connections 2024-04-25T08:48:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8f200640 1 --2- 172.21.15.46:0/339937369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 0x7f0f88163930 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8f200640 1 --2- 172.21.15.46:0/339937369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8806ac10 0x7f0f8806b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8f200640 1 --2- 172.21.15.46:0/339937369 >> v2:172.21.15.82:3300/0 conn(0x7f0f8814b8c0 0x7f0f880b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8f200640 1 -- 172.21.15.46:0/339937369 >> 172.21.15.46:0/339937369 conn(0x7f0f88099a00 msgr2=0x7f0f880b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8f200640 1 -- 172.21.15.46:0/339937369 shutdown_connections 2024-04-25T08:48:44.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.344+0000 7f0f8f200640 1 -- 172.21.15.46:0/339937369 wait complete. 2024-04-25T08:48:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 Processor -- start 2024-04-25T08:48:44.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 -- start start 2024-04-25T08:48:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8806ac10 0x7f0f88142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8814b8c0 0x7f0f880ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 0x7f0f881450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880b1d30 con 0x7f0f881634a0 2024-04-25T08:48:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880b55d0 con 0x7f0f8806ac10 2024-04-25T08:48:44.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880b1eb0 con 0x7f0f8814b8c0 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 0x7f0f881450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8806ac10 0x7f0f88142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 0x7f0f881450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60948/0 (socket says 172.21.15.46:60948) 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8e800640 1 -- 172.21.15.46:0/1542250305 learned_addr learned my addr 172.21.15.46:0/1542250305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8d400640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8814b8c0 0x7f0f880ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8e800640 1 -- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8814b8c0 msgr2=0x7f0f880ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8e800640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8814b8c0 0x7f0f880ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8e800640 1 -- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8806ac10 msgr2=0x7f0f88142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8e800640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8806ac10 0x7f0f88142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8e800640 1 -- 172.21.15.46:0/1542250305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f880b1ac0 con 0x7f0f881634a0 2024-04-25T08:48:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8e800640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 0x7f0f881450a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0f840b2040 tx=0x7f0f8401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f82000640 1 -- 172.21.15.46:0/1542250305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840b4040 con 0x7f0f881634a0 2024-04-25T08:48:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f82000640 1 -- 172.21.15.46:0/1542250305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f840090d0 con 0x7f0f881634a0 2024-04-25T08:48:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f82000640 1 -- 172.21.15.46:0/1542250305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84009510 con 0x7f0f881634a0 2024-04-25T08:48:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f880ae0f0 con 0x7f0f881634a0 2024-04-25T08:48:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.348+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f880ae6b0 con 0x7f0f881634a0 2024-04-25T08:48:44.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.352+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f88069010 con 0x7f0f881634a0 2024-04-25T08:48:44.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.352+0000 7f0f82000640 1 -- 172.21.15.46:0/1542250305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f840c3020 con 0x7f0f881634a0 2024-04-25T08:48:44.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.352+0000 7f0f82000640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f6c066f60 0x7f0f6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:44.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.352+0000 7f0f82000640 1 -- 172.21.15.46:0/1542250305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0f84122de0 con 0x7f0f881634a0 2024-04-25T08:48:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.352+0000 7f0f8de00640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f6c066f60 0x7f0f6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.356+0000 7f0f8de00640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f6c066f60 0x7f0f6c069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009b60 tx=0x7f0f7c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:44.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.356+0000 7f0f82000640 1 -- 172.21.15.46:0/1542250305 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f840ca020 con 0x7f0f881634a0 2024-04-25T08:48:44.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.552+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f8806da80 con 0x7f0f881634a0 2024-04-25T08:48:44.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f82000640 1 -- 172.21.15.46:0/1542250305 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f84018d40 con 0x7f0f881634a0 2024-04-25T08:48:44.551 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:48:44.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f6c066f60 msgr2=0x7f0f6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f6c066f60 0x7f0f6c069420 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009b60 tx=0x7f0f7c009190 comp rx=0 tx=0).stop 2024-04-25T08:48:44.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 msgr2=0x7f0f881450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:44.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 0x7f0f881450a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0f840b2040 tx=0x7f0f8401bf80 comp rx=0 tx=0).stop 2024-04-25T08:48:44.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 shutdown_connections 2024-04-25T08:48:44.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0f6c066f60 0x7f0f6c069420 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f881634a0 0x7f0f881450a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0f8814b8c0 0x7f0f880ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 --2- 172.21.15.46:0/1542250305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8806ac10 0x7f0f88142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:44.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 >> 172.21.15.46:0/1542250305 conn(0x7f0f88099a00 msgr2=0x7f0f88149890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:44.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 shutdown_connections 2024-04-25T08:48:44.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:44.556+0000 7f0f8f200640 1 -- 172.21.15.46:0/1542250305 wait complete. 2024-04-25T08:48:44.569 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:48:44.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:48:44.569 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:48:44.569 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:48:44.569 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:44.570 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:48:44.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:48:44.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:48:44.570 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:48:44.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:44.570 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:48:44.580 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:48:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:48:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:48:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:48:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:44.580 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:48:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:44.580 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42078sec preferred_lft 42078sec 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:48:44.581 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:48:44.582 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:44.583 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:48:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:48:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:48:44.583 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:48:44.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:44.583 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:48:44.643 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:48:44.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:44.646 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:48:44.698 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:48:44.698 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:48:44.698 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:48:44.698 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:48:44.698 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:48:44.698 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:48:44.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:44.699 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:48:44.699 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:48:44.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:48:44.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:48:44.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:48:44.699 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:48:44.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:44.794 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:48:44.803 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:48:44.803 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:48:44.805 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:48:44.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:44.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:48:44.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:48:44.872 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:44.924 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:48:44.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:48:44.924 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:44.976 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:48:44.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:44.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:45.069 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:45.069 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:48:45.069 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:48:45.084 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:45.084 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:48:45.085 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:48:45.085 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:48:45.085 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:48:45.093 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:48:45.093 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:48:45.093 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:48:45.093 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:48:45.093 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:48:45.094 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:48:45.094 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:48:45.094 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:48:45.097 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:48:45.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:45.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:48:45.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:45.118 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:48:45.175 INFO:teuthology.orchestra.run.smithi046.stdout:client7740 2024-04-25T08:48:45.176 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:48:45.176 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:48:45.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:45.224 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-25T08:48:45.279 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:48:45.284+0000 7f9045200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T08:48:45.280 INFO:teuthology.orchestra.run.smithi046.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-25T08:48:45.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:48:45.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:45.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-25T08:48:45.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f961d000640 1 Processor -- start 2024-04-25T08:48:45.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f961d000640 1 -- start start 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f961d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180ccc60 0x7f9618099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f961d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96180cd0c0 0x7f9618099f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f961d000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96180cc1c0 con 0x7f96180b05c0 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f961d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96180cc7d0 con 0x7f96180cd0c0 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f961d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961809ab20 con 0x7f96180ccc60 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f9617e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96180cd0c0 0x7f9618099f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f9617e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96180cd0c0 0x7f9618099f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60950/0 (socket says 172.21.15.46:60950) 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f9617e00640 1 -- 172.21.15.46:0/1718147061 learned_addr learned my addr 172.21.15.46:0/1718147061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f960fe00640 1 --2- 172.21.15.46:0/1718147061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180ccc60 0x7f9618099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f9617e00640 1 -- 172.21.15.46:0/1718147061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180ccc60 msgr2=0x7f9618099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f9617e00640 1 --2- 172.21.15.46:0/1718147061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180ccc60 0x7f9618099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f9617e00640 1 -- 172.21.15.46:0/1718147061 >> v1:172.21.15.82:6789/0 conn(0x7f96180b05c0 legacy=0x7f96180cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.476+0000 7f9617e00640 1 -- 172.21.15.46:0/1718147061 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96180cc460 con 0x7f96180cd0c0 2024-04-25T08:48:45.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f9617e00640 1 --2- 172.21.15.46:0/1718147061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180cd0c0 0x7f9618099f40 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9610009e50 tx=0x7f961001d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadd91c465a783c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f9616a00640 1 -- 172.21.15.46:0/1718147061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100a5810 con 0x7f96180cd0c0 2024-04-25T08:48:45.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f9616a00640 1 -- 172.21.15.46:0/1718147061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96100a59b0 con 0x7f96180cd0c0 2024-04-25T08:48:45.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f9616a00640 1 -- 172.21.15.46:0/1718147061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100a5d80 con 0x7f96180cd0c0 2024-04-25T08:48:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 -- 172.21.15.46:0/1718147061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180cd0c0 msgr2=0x7f9618099f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 --2- 172.21.15.46:0/1718147061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180cd0c0 0x7f9618099f40 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f9610009e50 tx=0x7f961001d490 comp rx=0 tx=0).stop 2024-04-25T08:48:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 -- 172.21.15.46:0/1718147061 shutdown_connections 2024-04-25T08:48:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 --2- 172.21.15.46:0/1718147061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180cd0c0 0x7f9618099f40 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 --2- 172.21.15.46:0/1718147061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180ccc60 0x7f9618099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 -- 172.21.15.46:0/1718147061 >> 172.21.15.46:0/1718147061 conn(0x7f96180b2f70 msgr2=0x7f96180b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 -- 172.21.15.46:0/1718147061 shutdown_connections 2024-04-25T08:48:45.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 -- 172.21.15.46:0/1718147061 wait complete. 2024-04-25T08:48:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 Processor -- start 2024-04-25T08:48:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.480+0000 7f961d000640 1 -- start start 2024-04-25T08:48:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f961d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180b05c0 0x7f961814ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f961d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180ccc60 0x7f96180caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f961d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96180cd0c0 0x7f961806f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f961d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180b0b20 con 0x7f96180ccc60 2024-04-25T08:48:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f961d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96180b09a0 con 0x7f96180cd0c0 2024-04-25T08:48:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f961d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9618168920 con 0x7f96180b05c0 2024-04-25T08:48:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180ccc60 0x7f96180caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180ccc60 0x7f96180caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60958/0 (socket says 172.21.15.46:60958) 2024-04-25T08:48:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960fe00640 1 -- 172.21.15.46:0/630707070 learned_addr learned my addr 172.21.15.46:0/630707070 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f9617e00640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96180cd0c0 0x7f961806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f9617400640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180b05c0 0x7f961814ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960fe00640 1 -- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180b05c0 msgr2=0x7f961814ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960fe00640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180b05c0 0x7f961814ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960fe00640 1 -- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96180cd0c0 msgr2=0x7f961806f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960fe00640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96180cd0c0 0x7f961806f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960fe00640 1 -- 172.21.15.46:0/630707070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96180c9640 con 0x7f96180ccc60 2024-04-25T08:48:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960fe00640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180ccc60 0x7f96180caf30 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f960000e960 tx=0x7f960000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960f400640 1 -- 172.21.15.46:0/630707070 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600017070 con 0x7f96180ccc60 2024-04-25T08:48:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960f400640 1 -- 172.21.15.46:0/630707070 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960000ca00 con 0x7f96180ccc60 2024-04-25T08:48:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f960f400640 1 -- 172.21.15.46:0/630707070 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960000cdb0 con 0x7f96180ccc60 2024-04-25T08:48:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f961d000640 1 -- 172.21.15.46:0/630707070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96180c98b0 con 0x7f96180ccc60 2024-04-25T08:48:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.484+0000 7f961d000640 1 -- 172.21.15.46:0/630707070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96180c9e70 con 0x7f96180ccc60 2024-04-25T08:48:45.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960f400640 1 -- 172.21.15.46:0/630707070 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f960001a070 con 0x7f96180ccc60 2024-04-25T08:48:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960f400640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95f4066f60 0x7f95f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960f400640 1 -- 172.21.15.46:0/630707070 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f96000855a0 con 0x7f96180ccc60 2024-04-25T08:48:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9617400640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95f4066f60 0x7f95f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 Processor -- start 2024-04-25T08:48:45.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 -- start start 2024-04-25T08:48:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9617400640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95f4066f60 0x7f95f4069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f96180cf660 tx=0x7f9608009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f9854c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9834ffd0 con 0x555f9854cad0 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f982d8170 con 0x555f98530f10 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f9834c070 con 0x555f9854c1b0 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985309d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60964/0 (socket says 172.21.15.46:60964) 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960fe00640 1 -- 172.21.15.46:0/3827358835 learned_addr learned my addr 172.21.15.46:0/3827358835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9617400640 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f9854c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9617e00640 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960fe00640 1 -- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 msgr2=0x555f9854c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960fe00640 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f9854c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960fe00640 1 -- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 msgr2=0x555f985312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960fe00640 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985312f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960fe00640 1 -- 172.21.15.46:0/3827358835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f98396d20 con 0x555f9854cad0 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9617400640 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f9854c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9617e00640 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f960fe00640 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985309d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9600055a70 tx=0x7f9600055c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b13485d319b20f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9607e00640 1 -- 172.21.15.46:0/3827358835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600021040 con 0x555f9854cad0 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9607e00640 1 -- 172.21.15.46:0/3827358835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960005c0d0 con 0x555f9854cad0 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9607e00640 1 -- 172.21.15.46:0/3827358835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96000523e0 con 0x555f9854cad0 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 -- 172.21.15.46:0/3827358835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 msgr2=0x555f985309d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985309d0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9600055a70 tx=0x7f9600055c70 comp rx=0 tx=0).stop 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 -- 172.21.15.46:0/3827358835 shutdown_connections 2024-04-25T08:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985309d0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 --2- 172.21.15.46:0/3827358835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f9854c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.488+0000 7f9621a45000 1 -- 172.21.15.46:0/3827358835 >> 172.21.15.46:0/3827358835 conn(0x555f98508710 msgr2=0x555f98508af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 -- 172.21.15.46:0/3827358835 shutdown_connections 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 -- 172.21.15.46:0/3827358835 wait complete. 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 -- start start 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 Processor -- start 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f98530f10 0x555f98553720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f9854c1b0 0x555f98553c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9840dc10 con 0x555f9854cad0 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9830a680 con 0x555f9854c1b0 2024-04-25T08:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f98354600 con 0x555f98530f10 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 10 client.0 _ll_register_callbacks cb 0x555f98405080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985312f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60968/0 (socket says 172.21.15.46:60968) 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617e00640 1 -- 172.21.15.46:0/3551424131 learned_addr learned my addr 172.21.15.46:0/3551424131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f960fe00640 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f9854c1b0 0x555f98553c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617400640 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f98530f10 0x555f98553720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617e00640 1 -- 172.21.15.46:0/3551424131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f98530f10 msgr2=0x555f98553720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617e00640 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f98530f10 0x555f98553720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617e00640 1 -- 172.21.15.46:0/3551424131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f9854c1b0 msgr2=0x555f98553c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617e00640 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f9854c1b0 0x555f98553c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617e00640 1 -- 172.21.15.46:0/3551424131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f98327d30 con 0x555f9854cad0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9617e00640 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985312f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f961001a660 tx=0x7f961001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93af9ae18d068cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9604c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9604c00640 1 -- 172.21.15.46:0/3551424131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100a5910 con 0x555f9854cad0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9604c00640 1 -- 172.21.15.46:0/3551424131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f96100c2cb0 con 0x555f9854cad0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 -- 172.21.15.46:0/3551424131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f960005c0d0 con 0x555f9854cad0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9604c00640 1 -- 172.21.15.46:0/3551424131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96100a5ab0 con 0x555f9854cad0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 10 client.7752 fetch_fsmap learned FSMap version 68 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 -- 172.21.15.46:0/3551424131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x555f982d83c0 con 0x555f9854cad0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9604c00640 1 -- 172.21.15.46:0/3551424131 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960005c0d0 con 0x555f9854cad0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9604c00640 1 -- 172.21.15.46:0/3551424131 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f96100c9050 con 0x555f9854cad0 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 10 client.7752 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 10 client.7752 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 20 client.7752 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555f98532060 0x555f98532470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 4 client.7752 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.492+0000 7f9621a45000 1 -- 172.21.15.46:0/3551424131 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f982ebe00 con 0x555f98532060 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9617400640 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555f98532060 0x555f98532470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9604c00640 10 client.7752 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9617400640 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555f98532060 0x555f98532470 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b2ed3abb1dc4c62 server_cookie=993b4b11a29051ed in_seq=0 out_seq=0 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9604c00640 1 -- 172.21.15.46:0/3551424131 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555f982ebe00 con 0x555f98532060 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9604c00640 10 client.7752 handle_command_reply: tid=0 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 client.7752 shutdown 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 -- 172.21.15.46:0/3551424131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 msgr2=0x555f985312f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985312f0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f961001a660 tx=0x7f961001ca30 comp rx=0 tx=0).stop 2024-04-25T08:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 -- 172.21.15.46:0/3551424131 shutdown_connections 2024-04-25T08:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555f98532060 0x555f98532470 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f985312f0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f9854c1b0 0x555f98553c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 --2- 172.21.15.46:0/3551424131 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f98530f10 0x555f98553720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9604c00640 0 client.7752 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 -- 172.21.15.46:0/3551424131 >> 172.21.15.46:0/3551424131 conn(0x555f98508710 msgr2=0x555f98500950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 -- 172.21.15.46:0/3551424131 shutdown_connections 2024-04-25T08:48:45.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 1 -- 172.21.15.46:0/3551424131 wait complete. 2024-04-25T08:48:45.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.496+0000 7f9621a45000 20 client.7752 trim_cache size 0 max 16384 2024-04-25T08:48:45.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 Processor -- start 2024-04-25T08:48:45.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 -- start start 2024-04-25T08:48:45.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f98507c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f9854a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f9854aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9618203370 con 0x555f9854cad0 2024-04-25T08:48:45.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96182025d0 con 0x555f98530f10 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961806c930 con 0x555f9854c1b0 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f9854aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f9854aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60974/0 (socket says 172.21.15.46:60974) 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617e00640 1 -- 172.21.15.46:0/3052414061 learned_addr learned my addr 172.21.15.46:0/3052414061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617400640 1 --2- 172.21.15.46:0/3052414061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f98507c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f960fe00640 1 --2- 172.21.15.46:0/3052414061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f9854a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617e00640 1 -- 172.21.15.46:0/3052414061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 msgr2=0x555f9854a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617e00640 1 --2- 172.21.15.46:0/3052414061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f9854a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617e00640 1 -- 172.21.15.46:0/3052414061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 msgr2=0x555f98507c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617e00640 1 --2- 172.21.15.46:0/3052414061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f98507c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617e00640 1 -- 172.21.15.46:0/3052414061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f983d20f0 con 0x555f9854cad0 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9617e00640 1 --2- 172.21.15.46:0/3052414061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f9854aa70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9610002240 tx=0x7f96100a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cc576ffbd15a87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9607400640 1 -- 172.21.15.46:0/3052414061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100a5810 con 0x555f9854cad0 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9607400640 1 -- 172.21.15.46:0/3052414061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96100039a0 con 0x555f9854cad0 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9607400640 1 -- 172.21.15.46:0/3052414061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9610003d90 con 0x555f9854cad0 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 -- 172.21.15.46:0/3052414061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 msgr2=0x555f9854aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 --2- 172.21.15.46:0/3052414061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f9854aa70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9610002240 tx=0x7f96100a7070 comp rx=0 tx=0).stop 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 -- 172.21.15.46:0/3052414061 shutdown_connections 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 --2- 172.21.15.46:0/3052414061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f9854aa70 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 --2- 172.21.15.46:0/3052414061 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f9854a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 --2- 172.21.15.46:0/3052414061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f98507c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 -- 172.21.15.46:0/3052414061 >> 172.21.15.46:0/3052414061 conn(0x555f98508710 msgr2=0x555f9852ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 -- 172.21.15.46:0/3052414061 shutdown_connections 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.512+0000 7f9621a45000 1 -- 172.21.15.46:0/3052414061 wait complete. 2024-04-25T08:48:45.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 -- start start 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 Processor -- start 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f985acb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f98501570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f9840aeb0 con 0x555f9854cad0 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9618062200 con 0x555f98530f10 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555f982f6d40 con 0x555f9854c1b0 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 10 client.0 _ll_register_callbacks cb 0x555f982edb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f98501570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f98501570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60986/0 (socket says 172.21.15.46:60986) 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617e00640 1 -- 172.21.15.46:0/3748040904 learned_addr learned my addr 172.21.15.46:0/3748040904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617400640 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f960fe00640 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f985acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617e00640 1 -- 172.21.15.46:0/3748040904 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 msgr2=0x555f985acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617e00640 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f985acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617e00640 1 -- 172.21.15.46:0/3748040904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 msgr2=0x555f985ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617e00640 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617e00640 1 -- 172.21.15.46:0/3748040904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555f982e4d20 con 0x555f9854cad0 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617400640 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617e00640 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f98501570 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f961001a660 tx=0x7f961001b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3dfd9fa0b5467c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9604c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9604c00640 1 -- 172.21.15.46:0/3748040904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96100cbbd0 con 0x555f9854cad0 2024-04-25T08:48:45.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9604c00640 1 -- 172.21.15.46:0/3748040904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f96100cbd70 con 0x555f9854cad0 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 -- 172.21.15.46:0/3748040904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96100039a0 con 0x555f9854cad0 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9604c00640 1 -- 172.21.15.46:0/3748040904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96100db050 con 0x555f9854cad0 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 10 client.7758 fetch_fsmap learned FSMap version 68 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 -- 172.21.15.46:0/3748040904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x555f982f70c0 con 0x555f9854cad0 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9604c00640 1 -- 172.21.15.46:0/3748040904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96100039a0 con 0x555f9854cad0 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9604c00640 1 -- 172.21.15.46:0/3748040904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f96100de020 con 0x555f9854cad0 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 10 client.7758 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 10 client.7758 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 20 client.7758 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555f98501f00 0x555f98502310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 4 client.7758 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9621a45000 1 -- 172.21.15.46:0/3748040904 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x555f9835d510 con 0x555f98501f00 2024-04-25T08:48:45.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617400640 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555f98501f00 0x555f98502310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9617400640 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555f98501f00 0x555f98502310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2edafbff9c9c2e server_cookie=a9dbf13626051708 in_seq=0 out_seq=0 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.516+0000 7f9604c00640 10 client.7758 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9604c00640 1 -- 172.21.15.46:0/3748040904 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555f9835d510 con 0x555f98501f00 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9604c00640 10 client.7758 handle_command_reply: tid=0 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 client.7758 shutdown 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 -- 172.21.15.46:0/3748040904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 msgr2=0x555f98501570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f98501570 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f961001a660 tx=0x7f961001b270 comp rx=0 tx=0).stop 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 -- 172.21.15.46:0/3748040904 shutdown_connections 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555f98501f00 0x555f98502310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555f9854cad0 0x555f98501570 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555f9854c1b0 0x555f985acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 --2- 172.21.15.46:0/3748040904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f98530f10 0x555f985ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 -- 172.21.15.46:0/3748040904 >> 172.21.15.46:0/3748040904 conn(0x555f98508710 msgr2=0x555f98530640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9604c00640 0 client.7758 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:45.520 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9045200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T08:48:35.444502+0000 1=1+0) n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x557050529b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T08:48:35.444502+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T08:48:45.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 -- 172.21.15.46:0/3748040904 shutdown_connections 2024-04-25T08:48:45.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 1 -- 172.21.15.46:0/3748040904 wait complete. 2024-04-25T08:48:45.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f9621a45000 20 client.7758 trim_cache size 0 max 16384 2024-04-25T08:48:45.521 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:48:45.521 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:48:45.521 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "7807bba0-bafe-4690-b466-fef3939b18b7", 2024-04-25T08:48:45.521 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:48:45.521 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:48:45.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 -- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95f4066f60 msgr2=0x7f95f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95f4066f60 0x7f95f4069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f96180cf660 tx=0x7f9608009040 comp rx=0 tx=0).stop 2024-04-25T08:48:45.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 -- 172.21.15.46:0/630707070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180ccc60 msgr2=0x7f96180caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180ccc60 0x7f96180caf30 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f960000e960 tx=0x7f960000ee20 comp rx=0 tx=0).stop 2024-04-25T08:48:45.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 -- 172.21.15.46:0/630707070 shutdown_connections 2024-04-25T08:48:45.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f95f4066f60 0x7f95f4069420 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96180cd0c0 0x7f961806f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96180ccc60 0x7f96180caf30 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 --2- 172.21.15.46:0/630707070 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f96180b05c0 0x7f961814ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 -- 172.21.15.46:0/630707070 >> 172.21.15.46:0/630707070 conn(0x7f96180b2f70 msgr2=0x7f96180aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 -- 172.21.15.46:0/630707070 shutdown_connections 2024-04-25T08:48:45.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.520+0000 7f961d000640 1 -- 172.21.15.46:0/630707070 wait complete. 2024-04-25T08:48:45.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:48:45.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:48:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f1951200640 1 Processor -- start 2024-04-25T08:48:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f1951200640 1 -- start start 2024-04-25T08:48:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f1951200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f194c072960 0x7f194c0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f1951200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f194c0b3200 0x7f194c06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f1951200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0b39a0 con 0x7f194c06bc20 2024-04-25T08:48:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f1951200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c0b3750 con 0x7f194c0b3200 2024-04-25T08:48:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f1951200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c072720 con 0x7f194c072960 2024-04-25T08:48:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f194be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f194c0b3200 0x7f194c06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f194be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f194c0b3200 0x7f194c06ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60996/0 (socket says 172.21.15.46:60996) 2024-04-25T08:48:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f194be00640 1 -- 172.21.15.46:0/217633289 learned_addr learned my addr 172.21.15.46:0/217633289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f1942a00640 1 --2- 172.21.15.46:0/217633289 >> v2:172.21.15.82:3300/0 conn(0x7f194c072960 0x7f194c0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f194be00640 1 -- 172.21.15.46:0/217633289 >> v2:172.21.15.82:3300/0 conn(0x7f194c072960 msgr2=0x7f194c0acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f194be00640 1 --2- 172.21.15.46:0/217633289 >> v2:172.21.15.82:3300/0 conn(0x7f194c072960 0x7f194c0acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f194be00640 1 -- 172.21.15.46:0/217633289 >> v1:172.21.15.82:6789/0 conn(0x7f194c06bc20 legacy=0x7f194c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.684+0000 7f194be00640 1 -- 172.21.15.46:0/217633289 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0b3c80 con 0x7f194c0b3200 2024-04-25T08:48:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f194be00640 1 --2- 172.21.15.46:0/217633289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c0b3200 0x7f194c06ed80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f194c0b4330 tx=0x7f194401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439dfd2e643b0540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f194aa00640 1 -- 172.21.15.46:0/217633289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440ab610 con 0x7f194c0b3200 2024-04-25T08:48:45.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f194aa00640 1 -- 172.21.15.46:0/217633289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19440b0070 con 0x7f194c0b3200 2024-04-25T08:48:45.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f194aa00640 1 -- 172.21.15.46:0/217633289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440ab9d0 con 0x7f194c0b3200 2024-04-25T08:48:45.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f1951200640 1 -- 172.21.15.46:0/217633289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c0b3200 msgr2=0x7f194c06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f1951200640 1 --2- 172.21.15.46:0/217633289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c0b3200 0x7f194c06ed80 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f194c0b4330 tx=0x7f194401c520 comp rx=0 tx=0).stop 2024-04-25T08:48:45.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f1951200640 1 -- 172.21.15.46:0/217633289 shutdown_connections 2024-04-25T08:48:45.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f1951200640 1 --2- 172.21.15.46:0/217633289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c0b3200 0x7f194c06ed80 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f1951200640 1 --2- 172.21.15.46:0/217633289 >> v2:172.21.15.82:3300/0 conn(0x7f194c072960 0x7f194c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f1951200640 1 -- 172.21.15.46:0/217633289 >> 172.21.15.46:0/217633289 conn(0x7f194c071970 msgr2=0x7f194c071d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f1951200640 1 -- 172.21.15.46:0/217633289 shutdown_connections 2024-04-25T08:48:45.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.688+0000 7f1951200640 1 -- 172.21.15.46:0/217633289 wait complete. 2024-04-25T08:48:45.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 Processor -- start 2024-04-25T08:48:45.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 -- start start 2024-04-25T08:48:45.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06bc20 0x7f194c14abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c072960 0x7f194c15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c0b3200 0x7f194c14f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15dc70 con 0x7f194c072960 2024-04-25T08:48:45.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0ad280 con 0x7f194c06bc20 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c15ddf0 con 0x7f194c0b3200 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1942a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c072960 0x7f194c15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1942a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c072960 0x7f194c15c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60998/0 (socket says 172.21.15.46:60998) 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1942a00640 1 -- 172.21.15.46:0/1979073363 learned_addr learned my addr 172.21.15.46:0/1979073363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f194b400640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06bc20 0x7f194c14abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f194be00640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c0b3200 0x7f194c14f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1942a00640 1 -- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c0b3200 msgr2=0x7f194c14f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1942a00640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c0b3200 0x7f194c14f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1942a00640 1 -- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06bc20 msgr2=0x7f194c14abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1942a00640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06bc20 0x7f194c14abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1942a00640 1 -- 172.21.15.46:0/1979073363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c04f030 con 0x7f194c072960 2024-04-25T08:48:45.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1942a00640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c072960 0x7f194c15c500 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f193400e930 tx=0x7f193400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1943e00640 1 -- 172.21.15.46:0/1979073363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934017070 con 0x7f194c072960 2024-04-25T08:48:45.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1943e00640 1 -- 172.21.15.46:0/1979073363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193400ca00 con 0x7f194c072960 2024-04-25T08:48:45.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1943e00640 1 -- 172.21.15.46:0/1979073363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400cdb0 con 0x7f194c072960 2024-04-25T08:48:45.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 -- 172.21.15.46:0/1979073363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c14b110 con 0x7f194c072960 2024-04-25T08:48:45.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.692+0000 7f1951200640 1 -- 172.21.15.46:0/1979073363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c15b3f0 con 0x7f194c072960 2024-04-25T08:48:45.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.696+0000 7f1943e00640 1 -- 172.21.15.46:0/1979073363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f193401a070 con 0x7f194c072960 2024-04-25T08:48:45.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.696+0000 7f1943e00640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1928066f60 0x7f1928069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.696+0000 7f1943e00640 1 -- 172.21.15.46:0/1979073363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1934085190 con 0x7f194c072960 2024-04-25T08:48:45.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.696+0000 7f1955ce0000 1 Processor -- start 2024-04-25T08:48:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.696+0000 7f194b400640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1928066f60 0x7f1928069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.696+0000 7f1955ce0000 1 -- start start 2024-04-25T08:48:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194b400640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1928066f60 0x7f1928069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f194c0cd730 tx=0x7f193c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 0x55d54ffbfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffc02f0 0x55d54ff75b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ff76480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54fdc2ff0 con 0x55d54ff760a0 2024-04-25T08:48:45.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54fd4b150 con 0x55d54ffc02f0 2024-04-25T08:48:45.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d54fdbf090 con 0x55d54ffbf9d0 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ff76480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ff76480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32782/0 (socket says 172.21.15.46:32782) 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194be00640 1 -- 172.21.15.46:0/1641509487 learned_addr learned my addr 172.21.15.46:0/1641509487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1942a00640 1 --2- 172.21.15.46:0/1641509487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffc02f0 0x55d54ff75b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194b400640 1 --2- 172.21.15.46:0/1641509487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 0x55d54ffbfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194be00640 1 -- 172.21.15.46:0/1641509487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 msgr2=0x55d54ffbfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194be00640 1 --2- 172.21.15.46:0/1641509487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 0x55d54ffbfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194be00640 1 -- 172.21.15.46:0/1641509487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffc02f0 msgr2=0x55d54ff75b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194be00640 1 --2- 172.21.15.46:0/1641509487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffc02f0 0x55d54ff75b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194be00640 1 -- 172.21.15.46:0/1641509487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d54ff590d0 con 0x55d54ff760a0 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f194be00640 1 --2- 172.21.15.46:0/1641509487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ff76480 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f1944019130 tx=0x7f1944004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d653b08bc78c18ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f193be00640 1 -- 172.21.15.46:0/1641509487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440ab900 con 0x55d54ff760a0 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f193be00640 1 -- 172.21.15.46:0/1641509487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19440b0040 con 0x55d54ff760a0 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f193be00640 1 -- 172.21.15.46:0/1641509487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440abca0 con 0x55d54ff760a0 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 -- 172.21.15.46:0/1641509487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 msgr2=0x55d54ff76480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1641509487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ff76480 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f1944019130 tx=0x7f1944004740 comp rx=0 tx=0).stop 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 -- 172.21.15.46:0/1641509487 shutdown_connections 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1641509487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ff76480 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1641509487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffc02f0 0x55d54ff75b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1641509487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 0x55d54ffbfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 -- 172.21.15.46:0/1641509487 >> 172.21.15.46:0/1641509487 conn(0x55d54ff72020 msgr2=0x55d54ff74440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 -- 172.21.15.46:0/1641509487 shutdown_connections 2024-04-25T08:48:45.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 -- 172.21.15.46:0/1641509487 wait complete. 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 -- start start 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.700+0000 7f1955ce0000 1 Processor -- start 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ffa47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffbf9d0 0x55d54ffa4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 0x55d54ffa5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54fd87250 con 0x55d54ff760a0 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54fd7d690 con 0x55d54ffbf9d0 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d54fd4ba40 con 0x55d54ffc02f0 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 10 client.0 _ll_register_callbacks cb 0x55d54ff60e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ffa47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ffa47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32784/0 (socket says 172.21.15.46:32784) 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194b400640 1 -- 172.21.15.46:0/547147676 learned_addr learned my addr 172.21.15.46:0/547147676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194be00640 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 0x55d54ffa5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1942a00640 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffbf9d0 0x55d54ffa4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194b400640 1 -- 172.21.15.46:0/547147676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 msgr2=0x55d54ffa5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194b400640 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 0x55d54ffa5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194b400640 1 -- 172.21.15.46:0/547147676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffbf9d0 msgr2=0x55d54ffa4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194b400640 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffbf9d0 0x55d54ffa4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194b400640 1 -- 172.21.15.46:0/547147676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d54fe701c0 con 0x55d54ff760a0 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1942a00640 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffbf9d0 0x55d54ffa4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f194b400640 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ffa47f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f193c014a00 tx=0x7f193c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df5a5d35cdd8a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:45.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 1 -- 172.21.15.46:0/547147676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c025020 con 0x55d54ff760a0 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 1 -- 172.21.15.46:0/547147676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f193c013b40 con 0x55d54ff760a0 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- 172.21.15.46:0/547147676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19440b0040 con 0x55d54ff760a0 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 1 -- 172.21.15.46:0/547147676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f193c02f050 con 0x55d54ff760a0 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 10 client.7770 fetch_fsmap learned FSMap version 68 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- 172.21.15.46:0/547147676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55d54fe6c170 con 0x55d54ff760a0 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 1 -- 172.21.15.46:0/547147676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19440b0040 con 0x55d54ff760a0 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 1 -- 172.21.15.46:0/547147676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f193c032020 con 0x55d54ff760a0 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 10 client.7770 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 10 client.7770 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 20 client.7770 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d54ffcfc80 0x55d54ffd2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 4 client.7770 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:48:45.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- 172.21.15.46:0/547147676 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d54fd5ec60 con 0x55d54ffcfc80 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1942a00640 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d54ffcfc80 0x55d54ffd2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1942a00640 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d54ffcfc80 0x55d54ffd2070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64e3896799175e26 server_cookie=725124e59157d858 in_seq=0 out_seq=0 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 10 client.7770 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 1 -- 172.21.15.46:0/547147676 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d54fd5ec60 con 0x55d54ffcfc80 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 10 client.7770 handle_command_reply: tid=0 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 client.7770 shutdown 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- 172.21.15.46:0/547147676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 msgr2=0x55d54ffa47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ffa47f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f193c014a00 tx=0x7f193c001260 comp rx=0 tx=0).stop 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- 172.21.15.46:0/547147676 shutdown_connections 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d54ffcfc80 0x55d54ffd2070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 0x55d54ffa5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1938c00640 0 client.7770 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ffbf9d0 0x55d54ffa4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 --2- 172.21.15.46:0/547147676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ff760a0 0x55d54ffa47f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- 172.21.15.46:0/547147676 >> 172.21.15.46:0/547147676 conn(0x55d54ff72020 msgr2=0x55d54ff74ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- 172.21.15.46:0/547147676 shutdown_connections 2024-04-25T08:48:45.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.704+0000 7f1955ce0000 1 -- 172.21.15.46:0/547147676 wait complete. 2024-04-25T08:48:45.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.708+0000 7f1955ce0000 20 client.7770 trim_cache size 0 max 16384 2024-04-25T08:48:45.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.720+0000 7f1955ce0000 1 Processor -- start 2024-04-25T08:48:45.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.720+0000 7f1955ce0000 1 -- start start 2024-04-25T08:48:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.720+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 0x55d54ffbee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.720+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffbf9d0 0x55d54ff75420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.720+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 0x55d54ff75960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.720+0000 7f1955ce0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c203370 con 0x55d54ffbf9d0 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.720+0000 7f1955ce0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c202560 con 0x55d54ff760a0 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.720+0000 7f1955ce0000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c14c0a0 con 0x55d54ffc02f0 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffbf9d0 0x55d54ff75420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffbf9d0 0x55d54ff75420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32792/0 (socket says 172.21.15.46:32792) 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 -- 172.21.15.46:0/931512138 learned_addr learned my addr 172.21.15.46:0/931512138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194b400640 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 0x55d54ffbee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194be00640 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 0x55d54ff75960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 -- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 msgr2=0x55d54ff75960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 0x55d54ff75960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 -- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 msgr2=0x55d54ffbee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 0x55d54ffbee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 -- 172.21.15.46:0/931512138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d54fe09de0 con 0x55d54ffbf9d0 2024-04-25T08:48:45.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194be00640 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 0x55d54ff75960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194b400640 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 0x55d54ffbee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffbf9d0 0x55d54ff75420 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f19340559c0 tx=0x7f19340833b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5acbf71ea062d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f193b400640 1 -- 172.21.15.46:0/931512138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934071070 con 0x55d54ffbf9d0 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f193b400640 1 -- 172.21.15.46:0/931512138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193407ee50 con 0x55d54ffbf9d0 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f193b400640 1 -- 172.21.15.46:0/931512138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934021040 con 0x55d54ffbf9d0 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 -- 172.21.15.46:0/931512138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffbf9d0 msgr2=0x55d54ff75420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffbf9d0 0x55d54ff75420 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f19340559c0 tx=0x7f19340833b0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 -- 172.21.15.46:0/931512138 shutdown_connections 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffc02f0 0x55d54ff75960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffbf9d0 0x55d54ff75420 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 --2- 172.21.15.46:0/931512138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 0x55d54ffbee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 -- 172.21.15.46:0/931512138 >> 172.21.15.46:0/931512138 conn(0x55d54ff72020 msgr2=0x55d54ffbe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 -- 172.21.15.46:0/931512138 shutdown_connections 2024-04-25T08:48:45.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 -- 172.21.15.46:0/931512138 wait complete. 2024-04-25T08:48:45.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 -- start start 2024-04-25T08:48:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 Processor -- start 2024-04-25T08:48:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 0x55d550020520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 0x55d550020a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffc02f0 0x55d54ffd2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d54fd5de00 con 0x55d54ffc02f0 2024-04-25T08:48:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0cce70 con 0x55d54ff760a0 2024-04-25T08:48:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d54fdc2300 con 0x55d54ffbf9d0 2024-04-25T08:48:45.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1955ce0000 10 client.0 _ll_register_callbacks cb 0x55d54ff41040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:48:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffc02f0 0x55d54ffd2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffc02f0 0x55d54ffd2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32802/0 (socket says 172.21.15.46:32802) 2024-04-25T08:48:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194be00640 1 -- 172.21.15.46:0/1874403184 learned_addr learned my addr 172.21.15.46:0/1874403184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:48:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f1942a00640 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 0x55d550020a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194b400640 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 0x55d550020520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194be00640 1 -- 172.21.15.46:0/1874403184 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 msgr2=0x55d550020a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194be00640 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 0x55d550020a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194be00640 1 -- 172.21.15.46:0/1874403184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 msgr2=0x55d550020520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:48:45.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.724+0000 7f194be00640 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 0x55d550020520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f194be00640 1 -- 172.21.15.46:0/1874403184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d54fd648e0 con 0x55d54ffc02f0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f194be00640 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffc02f0 0x55d54ffd2380 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f194401ca00 tx=0x7f194401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568c4196ca1c9a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 1 -- 172.21.15.46:0/1874403184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440cf060 con 0x55d54ffc02f0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 1 -- 172.21.15.46:0/1874403184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f19440ab840 con 0x55d54ffc02f0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 -- 172.21.15.46:0/1874403184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f193407ee50 con 0x55d54ffc02f0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 1 -- 172.21.15.46:0/1874403184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19440d5050 con 0x55d54ffc02f0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 10 client.7776 fetch_fsmap learned FSMap version 68 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 -- 172.21.15.46:0/1874403184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) v3 -- 0x55d54fe1c8f0 con 0x55d54ffc02f0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 1 -- 172.21.15.46:0/1874403184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193407ee50 con 0x55d54ffc02f0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 1 -- 172.21.15.46:0/1874403184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 68) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f19440d8020 con 0x55d54ffc02f0 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 10 client.7776 fetch_fsmap finished waiting for FSMap version 68 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 10 client.7776 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 20 client.7776 populate_metadata read hostname 'smithi046' 2024-04-25T08:48:45.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d54ffd2d40 0x55d54ffd3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 4 client.7776 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 -- 172.21.15.46:0/1874403184 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d54fdd0530 con 0x55d54ffd2d40 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f194b400640 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d54ffd2d40 0x55d54ffd3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f194b400640 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d54ffd2d40 0x55d54ffd3150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd2e4bcaa03464a server_cookie=e3cd0eb147ce8df7 in_seq=0 out_seq=0 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 10 client.7776 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 1 -- 172.21.15.46:0/1874403184 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x55d54fdd0530 con 0x55d54ffd2d40 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 10 client.7776 handle_command_reply: tid=0 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 client.7776 shutdown 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 -- 172.21.15.46:0/1874403184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffc02f0 msgr2=0x55d54ffd2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffc02f0 0x55d54ffd2380 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f194401ca00 tx=0x7f194401ad60 comp rx=0 tx=0).stop 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 -- 172.21.15.46:0/1874403184 shutdown_connections 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55d54ffd2d40 0x55d54ffd3150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d54ffc02f0 0x55d54ffd2380 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d54ffbf9d0 0x55d550020a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 --2- 172.21.15.46:0/1874403184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d54ff760a0 0x55d550020520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1938c00640 0 client.7776 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 -- 172.21.15.46:0/1874403184 >> 172.21.15.46:0/1874403184 conn(0x55d54ff72020 msgr2=0x55d54ffbe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 -- 172.21.15.46:0/1874403184 shutdown_connections 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 1 -- 172.21.15.46:0/1874403184 wait complete. 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1955ce0000 20 client.7776 trim_cache size 0 max 16384 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stdout: "7807bba0-bafe-4690-b466-fef3939b18b7": { 2024-04-25T08:48:45.729 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir", 2024-04-25T08:48:45.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "7807bba0-bafe-4690-b466-fef3939b18b7", 2024-04-25T08:48:45.730 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-04-25T08:48:45.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:45.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:48:45.730 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 -- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1928066f60 msgr2=0x7f1928069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1928066f60 0x7f1928069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f194c0cd730 tx=0x7f193c009040 comp rx=0 tx=0).stop 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 -- 172.21.15.46:0/1979073363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c072960 msgr2=0x7f194c15c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c072960 0x7f194c15c500 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f193400e930 tx=0x7f193400edf0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 -- 172.21.15.46:0/1979073363 shutdown_connections 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1928066f60 0x7f1928069420 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f194c0b3200 0x7f194c14f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f194c072960 0x7f194c15c500 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 --2- 172.21.15.46:0/1979073363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f194c06bc20 0x7f194c14abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 -- 172.21.15.46:0/1979073363 >> 172.21.15.46:0/1979073363 conn(0x7f194c071970 msgr2=0x7f194c0ad6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 -- 172.21.15.46:0/1979073363 shutdown_connections 2024-04-25T08:48:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:48:45.728+0000 7f1951200640 1 -- 172.21.15.46:0/1979073363 wait complete. 2024-04-25T08:48:45.743 INFO:tasks.cephfs.filesystem:scrub status for tag:7807bba0-bafe-4690-b466-fef3939b18b7 - {'path': '/dir', 'tag': '7807bba0-bafe-4690-b466-fef3939b18b7', 'options': 'recursive'} 2024-04-25T08:48:49.624 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:48:49.632+0000 7fccb4a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:48:50.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:48:50.827 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:49:15.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:15.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:49:15.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e4e600640 1 Processor -- start 2024-04-25T08:49:15.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e4e600640 1 -- start start 2024-04-25T08:49:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e4e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 0x7f8e4809a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e4e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e480cf7a0 con 0x7f8e48161250 2024-04-25T08:49:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e4e600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4806d230 con 0x7f8e4809a110 2024-04-25T08:49:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e4e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4806ddb0 con 0x7f8e4806cd30 2024-04-25T08:49:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e47400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8e48161250 0x7f8e48099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46770/0 (socket says 172.21.15.46:46770) 2024-04-25T08:49:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e47400640 1 -- 172.21.15.46:0/3608671009 learned_addr learned my addr 172.21.15.46:0/3608671009 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e47e00640 1 --2- 172.21.15.46:0/3608671009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 0x7f8e4809a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e46a00640 1 -- 172.21.15.46:0/3608671009 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620799415 0 0) 0x7f8e480cf7a0 con 0x7f8e48161250 2024-04-25T08:49:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e46a00640 1 -- 172.21.15.46:0/3608671009 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e2c003530 con 0x7f8e48161250 2024-04-25T08:49:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e46a00640 1 -- 172.21.15.46:0/3608671009 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896914377 0 0) 0x7f8e4806d230 con 0x7f8e4809a110 2024-04-25T08:49:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e46a00640 1 -- 172.21.15.46:0/3608671009 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e480cf7a0 con 0x7f8e4809a110 2024-04-25T08:49:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e47e00640 1 -- 172.21.15.46:0/3608671009 >> v1:172.21.15.82:6789/0 conn(0x7f8e4809a110 legacy=0x7f8e480cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e46a00640 1 -- 172.21.15.46:0/3608671009 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 541502043 0 0) 0x7f8e2c003530 con 0x7f8e48161250 2024-04-25T08:49:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e47e00640 1 -- 172.21.15.46:0/3608671009 >> v1:172.21.15.46:6789/0 conn(0x7f8e48161250 legacy=0x7f8e48099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.904+0000 7f8e47e00640 1 -- 172.21.15.46:0/3608671009 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4806d510 con 0x7f8e4806cd30 2024-04-25T08:49:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e47e00640 1 --2- 172.21.15.46:0/3608671009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 0x7f8e4809a7e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009dc0 tx=0x7f8e4001c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e2d37551c63824b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e46a00640 1 -- 172.21.15.46:0/3608671009 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400b2070 con 0x7f8e4806cd30 2024-04-25T08:49:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e46a00640 1 -- 172.21.15.46:0/3608671009 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e400a56c0 con 0x7f8e4806cd30 2024-04-25T08:49:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e46a00640 1 -- 172.21.15.46:0/3608671009 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38001680 con 0x7f8e4806cd30 2024-04-25T08:49:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e4e600640 1 -- 172.21.15.46:0/3608671009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 msgr2=0x7f8e4809a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e4e600640 1 --2- 172.21.15.46:0/3608671009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 0x7f8e4809a7e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009dc0 tx=0x7f8e4001c880 comp rx=0 tx=0).stop 2024-04-25T08:49:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e4e600640 1 -- 172.21.15.46:0/3608671009 shutdown_connections 2024-04-25T08:49:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e4e600640 1 --2- 172.21.15.46:0/3608671009 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 0x7f8e4809a7e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e4e600640 1 -- 172.21.15.46:0/3608671009 >> 172.21.15.46:0/3608671009 conn(0x7f8e480b3170 msgr2=0x7f8e480b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e4e600640 1 -- 172.21.15.46:0/3608671009 shutdown_connections 2024-04-25T08:49:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.908+0000 7f8e4e600640 1 -- 172.21.15.46:0/3608671009 wait complete. 2024-04-25T08:49:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e4e600640 1 Processor -- start 2024-04-25T08:49:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e4e600640 1 -- start start 2024-04-25T08:49:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e4e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 0x7f8e48154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e4e600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4809a110 0x7f8e4815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e4e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e48161250 0x7f8e48156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e4e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480d0ce0 con 0x7f8e48161250 2024-04-25T08:49:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e4e600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480d0b60 con 0x7f8e4809a110 2024-04-25T08:49:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e4e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480d0e60 con 0x7f8e4806cd30 2024-04-25T08:49:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e48161250 0x7f8e48156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e48161250 0x7f8e48156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54524/0 (socket says 172.21.15.46:54524) 2024-04-25T08:49:15.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47e00640 1 -- 172.21.15.46:0/716992475 learned_addr learned my addr 172.21.15.46:0/716992475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:15.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4809a110 0x7f8e4815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47400640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 0x7f8e48154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47e00640 1 -- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 msgr2=0x7f8e48154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47e00640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 0x7f8e48154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47e00640 1 -- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4809a110 msgr2=0x7f8e4815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47e00640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4809a110 0x7f8e4815eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47e00640 1 -- 172.21.15.46:0/716992475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480c7130 con 0x7f8e48161250 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e47e00640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e48161250 0x7f8e48156900 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8e400b0040 tx=0x7f8e40019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e3f400640 1 -- 172.21.15.46:0/716992475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400b2040 con 0x7f8e48161250 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e3f400640 1 -- 172.21.15.46:0/716992475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e40003bc0 con 0x7f8e48161250 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e3f400640 1 -- 172.21.15.46:0/716992475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40009450 con 0x7f8e48161250 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.912+0000 7f8e4e600640 1 -- 172.21.15.46:0/716992475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e480c7410 con 0x7f8e48161250 2024-04-25T08:49:15.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.916+0000 7f8e4e600640 1 -- 172.21.15.46:0/716992475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e480c79d0 con 0x7f8e48161250 2024-04-25T08:49:15.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.916+0000 7f8e3f400640 1 -- 172.21.15.46:0/716992475 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e40004030 con 0x7f8e48161250 2024-04-25T08:49:15.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.916+0000 7f8e3f400640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e24066f50 0x7f8e24069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.916+0000 7f8e3f400640 1 -- 172.21.15.46:0/716992475 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8e40120270 con 0x7f8e48161250 2024-04-25T08:49:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.916+0000 7f8e47400640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e24066f50 0x7f8e24069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 Processor -- start 2024-04-25T08:49:15.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- start start 2024-04-25T08:49:15.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e47400640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e24066f50 0x7f8e24069410 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8e38001000 tx=0x7f8e38001700 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 0x555e38630db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386312f0 0x555e385e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e385e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e38433ff0 con 0x555e386312f0 2024-04-25T08:49:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e383bc150 con 0x555e385e70a0 2024-04-25T08:49:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e38430090 con 0x555e386309d0 2024-04-25T08:49:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386312f0 0x555e385e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386312f0 0x555e385e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54534/0 (socket says 172.21.15.46:54534) 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e3fe00640 1 -- 172.21.15.46:0/1249559715 learned_addr learned my addr 172.21.15.46:0/1249559715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e47e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e385e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e47400640 1 --2- 172.21.15.46:0/1249559715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 0x555e38630db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e47e00640 1 -- 172.21.15.46:0/1249559715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 msgr2=0x555e38630db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e47e00640 1 --2- 172.21.15.46:0/1249559715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 0x555e38630db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e47e00640 1 -- 172.21.15.46:0/1249559715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386312f0 msgr2=0x555e385e6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e47e00640 1 --2- 172.21.15.46:0/1249559715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386312f0 0x555e385e6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e47e00640 1 -- 172.21.15.46:0/1249559715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e385ca0d0 con 0x555e385e70a0 2024-04-25T08:49:15.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e47e00640 1 --2- 172.21.15.46:0/1249559715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e385e7480 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8e4011e540 tx=0x7f8e400f05d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7b806f446c9cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e37e00640 1 -- 172.21.15.46:0/1249559715 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400f4a70 con 0x555e385e70a0 2024-04-25T08:49:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e37e00640 1 -- 172.21.15.46:0/1249559715 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e400f9030 con 0x555e385e70a0 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e37e00640 1 -- 172.21.15.46:0/1249559715 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400ed040 con 0x555e385e70a0 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1249559715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 msgr2=0x555e385e7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1249559715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e385e7480 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8e4011e540 tx=0x7f8e400f05d0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1249559715 shutdown_connections 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1249559715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e385e7480 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1249559715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386312f0 0x555e385e6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1249559715 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 0x555e38630db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1249559715 >> 172.21.15.46:0/1249559715 conn(0x555e385e3020 msgr2=0x555e385e5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1249559715 shutdown_connections 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1249559715 wait complete. 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 -- start start 2024-04-25T08:49:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.920+0000 7f8e51aaa000 1 Processor -- start 2024-04-25T08:49:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e385e70a0 0x555e386157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 0x555e38615d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 0x555e38616270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e383f8250 con 0x555e385e70a0 2024-04-25T08:49:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e383ee690 con 0x555e386312f0 2024-04-25T08:49:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e383bca40 con 0x555e386309d0 2024-04-25T08:49:15.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 10 client.0 _ll_register_callbacks cb 0x555e385d1e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e385e70a0 0x555e386157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e385e70a0 0x555e386157f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54536/0 (socket says 172.21.15.46:54536) 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47400640 1 -- 172.21.15.46:0/2644926136 learned_addr learned my addr 172.21.15.46:0/2644926136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47e00640 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 0x555e38616270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 0x555e38615d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47400640 1 -- 172.21.15.46:0/2644926136 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 msgr2=0x555e38615d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47400640 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 0x555e38615d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47400640 1 -- 172.21.15.46:0/2644926136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 msgr2=0x555e38616270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47400640 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 0x555e38616270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47400640 1 -- 172.21.15.46:0/2644926136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e384e11c0 con 0x555e385e70a0 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e47400640 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e385e70a0 0x555e386157f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8e4809f600 tx=0x7f8e380017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e62eeeca8dd1acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e34c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e34c00640 1 -- 172.21.15.46:0/2644926136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38025020 con 0x555e385e70a0 2024-04-25T08:49:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e34c00640 1 -- 172.21.15.46:0/2644926136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8e38013b10 con 0x555e385e70a0 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 -- 172.21.15.46:0/2644926136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e400f9030 con 0x555e385e70a0 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e34c00640 1 -- 172.21.15.46:0/2644926136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e3802f050 con 0x555e385e70a0 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 10 client.7788 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 -- 172.21.15.46:0/2644926136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x555e384dd170 con 0x555e385e70a0 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e34c00640 1 -- 172.21.15.46:0/2644926136 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e400f9030 con 0x555e385e70a0 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e34c00640 1 -- 172.21.15.46:0/2644926136 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e38032020 con 0x555e385e70a0 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 10 client.7788 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 10 client.7788 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 20 client.7788 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555e38640c80 0x555e38643070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 4 client.7788 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e51aaa000 1 -- 172.21.15.46:0/2644926136 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e383cfc60 con 0x555e38640c80 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555e38640c80 0x555e38643070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e34c00640 10 client.7788 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.924+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555e38640c80 0x555e38643070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd48d4cb7c21138 server_cookie=87f1d9108dfe8742 in_seq=0 out_seq=0 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e34c00640 1 -- 172.21.15.46:0/2644926136 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e383cfc60 con 0x555e38640c80 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e34c00640 10 client.7788 handle_command_reply: tid=0 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 client.7788 shutdown 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 -- 172.21.15.46:0/2644926136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e385e70a0 msgr2=0x555e386157f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e385e70a0 0x555e386157f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8e4809f600 tx=0x7f8e380017b0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 -- 172.21.15.46:0/2644926136 shutdown_connections 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555e38640c80 0x555e38643070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 0x555e38616270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386309d0 0x555e38615d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/2644926136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e385e70a0 0x555e386157f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e34c00640 0 client.7788 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 -- 172.21.15.46:0/2644926136 >> 172.21.15.46:0/2644926136 conn(0x555e385e3020 msgr2=0x555e385e5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:15.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 -- 172.21.15.46:0/2644926136 shutdown_connections 2024-04-25T08:49:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 1 -- 172.21.15.46:0/2644926136 wait complete. 2024-04-25T08:49:15.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.928+0000 7f8e51aaa000 20 client.7788 trim_cache size 0 max 16384 2024-04-25T08:49:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.940+0000 7f8e51aaa000 1 Processor -- start 2024-04-25T08:49:15.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- start start 2024-04-25T08:49:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e3862fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e385e6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386312f0 0x555e385e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48203220 con 0x555e386309d0 2024-04-25T08:49:15.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e482023a0 con 0x555e385e70a0 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48201850 con 0x555e386312f0 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e385e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e385e6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54552/0 (socket says 172.21.15.46:54552) 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e3fe00640 1 -- 172.21.15.46:0/3475253577 learned_addr learned my addr 172.21.15.46:0/3475253577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e47400640 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e3862fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e47e00640 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386312f0 0x555e385e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e3fe00640 1 -- 172.21.15.46:0/3475253577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386312f0 msgr2=0x555e385e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386312f0 0x555e385e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e3fe00640 1 -- 172.21.15.46:0/3475253577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 msgr2=0x555e3862fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e3862fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e3fe00640 1 -- 172.21.15.46:0/3475253577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e3847ade0 con 0x555e386309d0 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e47400640 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e3862fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e385e6420 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8e30005630 tx=0x7f8e30009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744c2607157a528f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:15.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e37400640 1 -- 172.21.15.46:0/3475253577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30004ec0 con 0x555e386309d0 2024-04-25T08:49:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e37400640 1 -- 172.21.15.46:0/3475253577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e30016920 con 0x555e386309d0 2024-04-25T08:49:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e37400640 1 -- 172.21.15.46:0/3475253577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30016cd0 con 0x555e386309d0 2024-04-25T08:49:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- 172.21.15.46:0/3475253577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 msgr2=0x555e385e6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e385e6420 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8e30005630 tx=0x7f8e30009040 comp rx=0 tx=0).stop 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- 172.21.15.46:0/3475253577 shutdown_connections 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e386312f0 0x555e385e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e385e6420 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/3475253577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e385e70a0 0x555e3862fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- 172.21.15.46:0/3475253577 >> 172.21.15.46:0/3475253577 conn(0x555e385e3020 msgr2=0x555e3862f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- 172.21.15.46:0/3475253577 shutdown_connections 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- 172.21.15.46:0/3475253577 wait complete. 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 -- start start 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.944+0000 7f8e51aaa000 1 Processor -- start 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e385e70a0 0x555e38691310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e38691850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 0x555e38643380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e383cee00 con 0x555e386309d0 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e48201dd0 con 0x555e386312f0 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e38433300 con 0x555e385e70a0 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 10 client.0 _ll_register_callbacks cb 0x555e385b2040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e38691850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e38691850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54568/0 (socket says 172.21.15.46:54568) 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e3fe00640 1 -- 172.21.15.46:0/1427238468 learned_addr learned my addr 172.21.15.46:0/1427238468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e47400640 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e385e70a0 0x555e38691310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e47e00640 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 0x555e38643380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e3fe00640 1 -- 172.21.15.46:0/1427238468 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e385e70a0 msgr2=0x555e38691310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e385e70a0 0x555e38691310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e3fe00640 1 -- 172.21.15.46:0/1427238468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 msgr2=0x555e38643380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 0x555e38643380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e3fe00640 1 -- 172.21.15.46:0/1427238468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e383d58e0 con 0x555e386309d0 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e3fe00640 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e38691850 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8e30008fd0 tx=0x7f8e38012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b91b115c5bc32c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 1 -- 172.21.15.46:0/1427238468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e30032060 con 0x555e386309d0 2024-04-25T08:49:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 1 -- 172.21.15.46:0/1427238468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8e30016ce0 con 0x555e386309d0 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1427238468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e30016920 con 0x555e386309d0 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 1 -- 172.21.15.46:0/1427238468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e30038050 con 0x555e386309d0 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 10 client.7794 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1427238468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x555e3848d8f0 con 0x555e386309d0 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 1 -- 172.21.15.46:0/1427238468 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e30016920 con 0x555e386309d0 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 1 -- 172.21.15.46:0/1427238468 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e3003b020 con 0x555e386309d0 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 10 client.7794 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 10 client.7794 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 20 client.7794 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555e38643fb0 0x555e386443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 4 client.7794 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1427238468 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e38441530 con 0x555e38643fb0 2024-04-25T08:49:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e47400640 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555e38643fb0 0x555e386443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e47400640 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555e38643fb0 0x555e386443c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cf8888e8dc5133 server_cookie=4ddee819133538b3 in_seq=0 out_seq=0 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 10 client.7794 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 1 -- 172.21.15.46:0/1427238468 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555e38441530 con 0x555e38643fb0 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 10 client.7794 handle_command_reply: tid=0 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 client.7794 shutdown 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1427238468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 msgr2=0x555e38691850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e38691850 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8e30008fd0 tx=0x7f8e38012000 comp rx=0 tx=0).stop 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1427238468 shutdown_connections 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555e38643fb0 0x555e386443c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e386312f0 0x555e38643380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e386309d0 0x555e38691850 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 --2- 172.21.15.46:0/1427238468 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e385e70a0 0x555e38691310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e34c00640 0 client.7794 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1427238468 >> 172.21.15.46:0/1427238468 conn(0x555e385e3020 msgr2=0x555e3862f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1427238468 shutdown_connections 2024-04-25T08:49:15.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.948+0000 7f8e51aaa000 1 -- 172.21.15.46:0/1427238468 wait complete. 2024-04-25T08:49:15.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e51aaa000 20 client.7794 trim_cache size 0 max 16384 2024-04-25T08:49:15.948 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:49:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:49:15.948 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:49:15.948 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:49:15.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 -- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e24066f50 msgr2=0x7f8e24069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e24066f50 0x7f8e24069410 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8e38001000 tx=0x7f8e38001700 comp rx=0 tx=0).stop 2024-04-25T08:49:15.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 -- 172.21.15.46:0/716992475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e48161250 msgr2=0x7f8e48156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:15.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e48161250 0x7f8e48156900 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8e400b0040 tx=0x7f8e40019620 comp rx=0 tx=0).stop 2024-04-25T08:49:15.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 -- 172.21.15.46:0/716992475 shutdown_connections 2024-04-25T08:49:15.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e24066f50 0x7f8e24069410 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e48161250 0x7f8e48156900 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4809a110 0x7f8e4815eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 --2- 172.21.15.46:0/716992475 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e4806cd30 0x7f8e48154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 -- 172.21.15.46:0/716992475 >> 172.21.15.46:0/716992475 conn(0x7f8e480b3170 msgr2=0x7f8e48070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 -- 172.21.15.46:0/716992475 shutdown_connections 2024-04-25T08:49:15.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:15.952+0000 7f8e4e600640 1 -- 172.21.15.46:0/716992475 wait complete. 2024-04-25T08:49:15.964 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:49:15.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:49:16.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f7600640 1 Processor -- start 2024-04-25T08:49:16.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f7600640 1 -- start start 2024-04-25T08:49:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f7600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18f009a110 0x7f18f009aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f7600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f015f300 con 0x7f18f009a610 2024-04-25T08:49:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f7600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f00cd8a0 con 0x7f18f015ef20 2024-04-25T08:49:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f7600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f00cddb0 con 0x7f18f009a110 2024-04-25T08:49:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18e7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f18f009a610 0x7f18f0099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46778/0 (socket says 172.21.15.46:46778) 2024-04-25T08:49:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f5600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18f009a110 0x7f18f009aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18e7e00640 1 -- 172.21.15.46:0/2172778820 learned_addr learned my addr 172.21.15.46:0/2172778820 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18efe00640 1 -- 172.21.15.46:0/2172778820 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406506721 0 0) 0x7f18f015f300 con 0x7f18f009a610 2024-04-25T08:49:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18efe00640 1 -- 172.21.15.46:0/2172778820 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d4003530 con 0x7f18f009a610 2024-04-25T08:49:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f5600640 1 -- 172.21.15.46:0/2172778820 >> v1:172.21.15.82:6789/0 conn(0x7f18f015ef20 legacy=0x7f18f00cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18efe00640 1 -- 172.21.15.46:0/2172778820 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327854341 0 0) 0x7f18f00cd8a0 con 0x7f18f015ef20 2024-04-25T08:49:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f5600640 1 -- 172.21.15.46:0/2172778820 >> v1:172.21.15.46:6789/0 conn(0x7f18f009a610 legacy=0x7f18f0099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.112+0000 7f18f5600640 1 -- 172.21.15.46:0/2172778820 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f00cc3d0 con 0x7f18f009a110 2024-04-25T08:49:16.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18f5600640 1 --2- 172.21.15.46:0/2172778820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a110 0x7f18f009aa10 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009d90 tx=0x7f18e801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894137580c2e5fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18efe00640 1 -- 172.21.15.46:0/2172778820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e80b2070 con 0x7f18f009a110 2024-04-25T08:49:16.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18efe00640 1 -- 172.21.15.46:0/2172778820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e80a5920 con 0x7f18f009a110 2024-04-25T08:49:16.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18efe00640 1 -- 172.21.15.46:0/2172778820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e80a5cf0 con 0x7f18f009a110 2024-04-25T08:49:16.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18f7600640 1 -- 172.21.15.46:0/2172778820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a110 msgr2=0x7f18f009aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18f7600640 1 --2- 172.21.15.46:0/2172778820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a110 0x7f18f009aa10 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009d90 tx=0x7f18e801c970 comp rx=0 tx=0).stop 2024-04-25T08:49:16.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18f7600640 1 -- 172.21.15.46:0/2172778820 shutdown_connections 2024-04-25T08:49:16.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18f7600640 1 --2- 172.21.15.46:0/2172778820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a110 0x7f18f009aa10 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18f7600640 1 -- 172.21.15.46:0/2172778820 >> 172.21.15.46:0/2172778820 conn(0x7f18f00b3120 msgr2=0x7f18f00b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18f7600640 1 -- 172.21.15.46:0/2172778820 shutdown_connections 2024-04-25T08:49:16.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.116+0000 7f18f7600640 1 -- 172.21.15.46:0/2172778820 wait complete. 2024-04-25T08:49:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 Processor -- start 2024-04-25T08:49:16.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 -- start start 2024-04-25T08:49:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18f009a110 0x7f18f0154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a610 0x7f18f00c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f015ef20 0x7f18f00caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f00cc190 con 0x7f18f009a610 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f00cc010 con 0x7f18f015ef20 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f0169890 con 0x7f18f009a110 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a610 0x7f18f00c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a610 0x7f18f00c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54588/0 (socket says 172.21.15.46:54588) 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18e7e00640 1 -- 172.21.15.46:0/3937268377 learned_addr learned my addr 172.21.15.46:0/3937268377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f4c00640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18f009a110 0x7f18f0154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f5600640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f015ef20 0x7f18f00caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18e7e00640 1 -- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18f009a110 msgr2=0x7f18f0154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18e7e00640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18f009a110 0x7f18f0154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18e7e00640 1 -- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f015ef20 msgr2=0x7f18f00caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18e7e00640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f015ef20 0x7f18f00caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18e7e00640 1 -- 172.21.15.46:0/3937268377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f00b4fc0 con 0x7f18f009a610 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f5600640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f015ef20 0x7f18f00caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18e7e00640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a610 0x7f18f00c8ef0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c450 tx=0x7f18d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18ed600640 1 -- 172.21.15.46:0/3937268377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8017070 con 0x7f18f009a610 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18ed600640 1 -- 172.21.15.46:0/3937268377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d8005cf0 con 0x7f18f009a610 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18ed600640 1 -- 172.21.15.46:0/3937268377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8007290 con 0x7f18f009a610 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f00b5260 con 0x7f18f009a610 2024-04-25T08:49:16.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.120+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f00b5820 con 0x7f18f009a610 2024-04-25T08:49:16.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.124+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f0069080 con 0x7f18f009a610 2024-04-25T08:49:16.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.124+0000 7f18ed600640 1 -- 172.21.15.46:0/3937268377 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18d801a070 con 0x7f18f009a610 2024-04-25T08:49:16.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.124+0000 7f18ed600640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18cc066f60 0x7f18cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.124+0000 7f18ed600640 1 -- 172.21.15.46:0/3937268377 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f18d8084730 con 0x7f18f009a610 2024-04-25T08:49:16.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.124+0000 7f18f4c00640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18cc066f60 0x7f18cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.124+0000 7f18f4c00640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18cc066f60 0x7f18cc069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f18e0000f70 tx=0x7f18e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.128+0000 7f18ed600640 1 -- 172.21.15.46:0/3937268377 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18d8055450 con 0x7f18f009a610 2024-04-25T08:49:16.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18f00d0700 con 0x7f18f009a610 2024-04-25T08:49:16.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18ed600640 1 -- 172.21.15.46:0/3937268377 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f18d8054d50 con 0x7f18f009a610 2024-04-25T08:49:16.318 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:16.318 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:48:49.616083+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7656},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7656":{"gid":7656,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6835/2777781817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2777781817},{"type":"v1","addr":"172.21.15.46:6835","nonce":2777781817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:49:16.319 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2024-04-25T08:49:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18cc066f60 msgr2=0x7f18cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18cc066f60 0x7f18cc069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f18e0000f70 tx=0x7f18e0009040 comp rx=0 tx=0).stop 2024-04-25T08:49:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a610 msgr2=0x7f18f00c8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a610 0x7f18f00c8ef0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c450 tx=0x7f18d800c910 comp rx=0 tx=0).stop 2024-04-25T08:49:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 shutdown_connections 2024-04-25T08:49:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f18cc066f60 0x7f18cc069420 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18f015ef20 0x7f18f00caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18f009a610 0x7f18f00c8ef0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 --2- 172.21.15.46:0/3937268377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f18f009a110 0x7f18f0154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 >> 172.21.15.46:0/3937268377 conn(0x7f18f00b3120 msgr2=0x7f18f006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 shutdown_connections 2024-04-25T08:49:16.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.324+0000 7f18f7600640 1 -- 172.21.15.46:0/3937268377 wait complete. 2024-04-25T08:49:16.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:49:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.488+0000 7f491c400640 1 Processor -- start 2024-04-25T08:49:16.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f491c400640 1 -- start start 2024-04-25T08:49:16.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f491c400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4914145310 0x7f491406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f491c400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f491406dc70 0x7f49140cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f491c400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f49140cf1b0 0x7f491415d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f491c400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49140cd930 con 0x7f4914145310 2024-04-25T08:49:16.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f491c400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49140cdbd0 con 0x7f49140cf1b0 2024-04-25T08:49:16.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f491c400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49140cdeb0 con 0x7f491406dc70 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f4919a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4914145310 0x7f491406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f4919a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4914145310 0x7f491406e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54600/0 (socket says 172.21.15.46:54600) 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f4919a00640 1 -- 172.21.15.46:0/3051965284 learned_addr learned my addr 172.21.15.46:0/3051965284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f491a400640 1 --2- 172.21.15.46:0/3051965284 >> v2:172.21.15.82:3300/0 conn(0x7f49140cf1b0 0x7f491415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f4919000640 1 --2- 172.21.15.46:0/3051965284 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f491406dc70 0x7f49140cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f4919a00640 1 -- 172.21.15.46:0/3051965284 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f491406dc70 msgr2=0x7f49140cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f4919a00640 1 --2- 172.21.15.46:0/3051965284 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f491406dc70 0x7f49140cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f4919a00640 1 -- 172.21.15.46:0/3051965284 >> v2:172.21.15.82:3300/0 conn(0x7f49140cf1b0 msgr2=0x7f491415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f4919a00640 1 --2- 172.21.15.46:0/3051965284 >> v2:172.21.15.82:3300/0 conn(0x7f49140cf1b0 0x7f491415d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.492+0000 7f4919a00640 1 -- 172.21.15.46:0/3051965284 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49140ce190 con 0x7f4914145310 2024-04-25T08:49:16.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f4919a00640 1 --2- 172.21.15.46:0/3051965284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4914145310 0x7f491406e0d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f49140cd3f0 tx=0x7f490801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcbe81b74b78aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f4913e00640 1 -- 172.21.15.46:0/3051965284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080b1070 con 0x7f4914145310 2024-04-25T08:49:16.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f4913e00640 1 -- 172.21.15.46:0/3051965284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49080ac620 con 0x7f4914145310 2024-04-25T08:49:16.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f4913e00640 1 -- 172.21.15.46:0/3051965284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080ac9f0 con 0x7f4914145310 2024-04-25T08:49:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 -- 172.21.15.46:0/3051965284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4914145310 msgr2=0x7f491406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 --2- 172.21.15.46:0/3051965284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4914145310 0x7f491406e0d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f49140cd3f0 tx=0x7f490801c4f0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 -- 172.21.15.46:0/3051965284 shutdown_connections 2024-04-25T08:49:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 --2- 172.21.15.46:0/3051965284 >> v2:172.21.15.82:3300/0 conn(0x7f49140cf1b0 0x7f491415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 --2- 172.21.15.46:0/3051965284 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f491406dc70 0x7f49140cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 --2- 172.21.15.46:0/3051965284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4914145310 0x7f491406e0d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 -- 172.21.15.46:0/3051965284 >> 172.21.15.46:0/3051965284 conn(0x7f491406ac10 msgr2=0x7f491406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 -- 172.21.15.46:0/3051965284 shutdown_connections 2024-04-25T08:49:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 -- 172.21.15.46:0/3051965284 wait complete. 2024-04-25T08:49:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 Processor -- start 2024-04-25T08:49:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.496+0000 7f491c400640 1 -- start start 2024-04-25T08:49:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491c400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49140cf1b0 0x7f4914156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491c400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4914145310 0x7f49140b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491c400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491413a010 0x7f49140af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491c400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49140713f0 con 0x7f491413a010 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491c400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4914071270 con 0x7f4914145310 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491c400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4914071570 con 0x7f49140cf1b0 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491413a010 0x7f49140af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491413a010 0x7f49140af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54604/0 (socket says 172.21.15.46:54604) 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491a400640 1 -- 172.21.15.46:0/357344664 learned_addr learned my addr 172.21.15.46:0/357344664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f4919a00640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49140cf1b0 0x7f4914156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f4919000640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4914145310 0x7f49140b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491a400640 1 -- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49140cf1b0 msgr2=0x7f4914156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491a400640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49140cf1b0 0x7f4914156460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491a400640 1 -- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4914145310 msgr2=0x7f49140b3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491a400640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4914145310 0x7f49140b3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491a400640 1 -- 172.21.15.46:0/357344664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49140b43f0 con 0x7f491413a010 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491a400640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491413a010 0x7f49140af9a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f490c007e60 tx=0x7f490c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f4911600640 1 -- 172.21.15.46:0/357344664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c014620 con 0x7f491413a010 2024-04-25T08:49:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f4911600640 1 -- 172.21.15.46:0/357344664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c019070 con 0x7f491413a010 2024-04-25T08:49:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491c400640 1 -- 172.21.15.46:0/357344664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491413b670 con 0x7f491413a010 2024-04-25T08:49:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f4911600640 1 -- 172.21.15.46:0/357344664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c014a00 con 0x7f491413a010 2024-04-25T08:49:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.500+0000 7f491c400640 1 -- 172.21.15.46:0/357344664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491413a8b0 con 0x7f491413a010 2024-04-25T08:49:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4911600640 1 -- 172.21.15.46:0/357344664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f490c014ba0 con 0x7f491413a010 2024-04-25T08:49:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4911600640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f48f4066f60 0x7f48f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4911600640 1 -- 172.21.15.46:0/357344664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f490c0859d0 con 0x7f491413a010 2024-04-25T08:49:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f48f4066f60 0x7f48f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f491f8c3000 1 Processor -- start 2024-04-25T08:49:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f491f8c3000 1 -- start start 2024-04-25T08:49:16.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f48f4066f60 0x7f48f4069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f49080ae000 tx=0x7f49080a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1ec76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1e7d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1e7dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f491f8c3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d1d64ba0 con 0x55e7d1ec72e0 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f491f8c3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d1cca090 con 0x55e7d1e7d8e0 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f491f8c3000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7d1c52160 con 0x55e7d1ec7c00 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1ec76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1ec76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54608/0 (socket says 172.21.15.46:54608) 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 -- 172.21.15.46:0/80941023 learned_addr learned my addr 172.21.15.46:0/80941023 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919000640 1 --2- 172.21.15.46:0/80941023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1e7d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f491a400640 1 --2- 172.21.15.46:0/80941023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1e7dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 -- 172.21.15.46:0/80941023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 msgr2=0x55e7d1e7d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 --2- 172.21.15.46:0/80941023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1e7d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 -- 172.21.15.46:0/80941023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 msgr2=0x55e7d1e7dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 --2- 172.21.15.46:0/80941023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1e7dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.504+0000 7f4919a00640 1 -- 172.21.15.46:0/80941023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7d1d73c20 con 0x55e7d1ec72e0 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919a00640 1 --2- 172.21.15.46:0/80941023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1ec76c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f490801aba0 tx=0x7f4908004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acc7f5fc44c036d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4906000640 1 -- 172.21.15.46:0/80941023 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080b1040 con 0x55e7d1ec72e0 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4906000640 1 -- 172.21.15.46:0/80941023 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49080c7070 con 0x55e7d1ec72e0 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4906000640 1 -- 172.21.15.46:0/80941023 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49080ace60 con 0x55e7d1ec72e0 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 -- 172.21.15.46:0/80941023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 msgr2=0x55e7d1ec76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 --2- 172.21.15.46:0/80941023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1ec76c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f490801aba0 tx=0x7f4908004740 comp rx=0 tx=0).stop 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 -- 172.21.15.46:0/80941023 shutdown_connections 2024-04-25T08:49:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 --2- 172.21.15.46:0/80941023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1e7dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 --2- 172.21.15.46:0/80941023 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1e7d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 --2- 172.21.15.46:0/80941023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1ec76c0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 -- 172.21.15.46:0/80941023 >> 172.21.15.46:0/80941023 conn(0x55e7d1e79890 msgr2=0x55e7d1e7bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 -- 172.21.15.46:0/80941023 shutdown_connections 2024-04-25T08:49:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 -- 172.21.15.46:0/80941023 wait complete. 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 -- start start 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 Processor -- start 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1ece640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1eceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1eb5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d1cc6130 con 0x55e7d1ec72e0 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d1c8e130 con 0x55e7d1e7d8e0 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7d1d88300 con 0x55e7d1ec7c00 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491f8c3000 10 client.0 _ll_register_callbacks cb 0x55e7d1e68bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1eceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1eceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54624/0 (socket says 172.21.15.46:54624) 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919000640 1 -- 172.21.15.46:0/2191338325 learned_addr learned my addr 172.21.15.46:0/2191338325 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f491a400640 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1eb5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919a00640 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1ece640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919000640 1 -- 172.21.15.46:0/2191338325 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 msgr2=0x55e7d1eb5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919000640 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1eb5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919000640 1 -- 172.21.15.46:0/2191338325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 msgr2=0x55e7d1ece640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919000640 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1ece640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919000640 1 -- 172.21.15.46:0/2191338325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7d1ca7060 con 0x55e7d1ec72e0 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.508+0000 7f4919000640 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1eceb80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4900013680 tx=0x7f4900013b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6495a36d25e3b3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 1 -- 172.21.15.46:0/2191338325 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900019070 con 0x55e7d1ec72e0 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 1 -- 172.21.15.46:0/2191338325 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f490001fd60 con 0x55e7d1ec72e0 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 -- 172.21.15.46:0/2191338325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49080c7070 con 0x55e7d1ec72e0 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 1 -- 172.21.15.46:0/2191338325 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49000118a0 con 0x55e7d1ec72e0 2024-04-25T08:49:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 10 client.7815 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 -- 172.21.15.46:0/2191338325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55e7d1c6b7c0 con 0x55e7d1ec72e0 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 1 -- 172.21.15.46:0/2191338325 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49080c7070 con 0x55e7d1ec72e0 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 1 -- 172.21.15.46:0/2191338325 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4900011a40 con 0x55e7d1ec72e0 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 10 client.7815 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 10 client.7815 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 20 client.7815 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55e7d1eb5e00 0x55e7d1eb61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 4 client.7815 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 -- 172.21.15.46:0/2191338325 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7d1c65cd0 con 0x55e7d1eb5e00 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f4919a00640 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55e7d1eb5e00 0x55e7d1eb61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f4919a00640 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55e7d1eb5e00 0x55e7d1eb61e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae64873fda6a78d server_cookie=4c08cecc7c5fc134 in_seq=0 out_seq=0 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 10 client.7815 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 1 -- 172.21.15.46:0/2191338325 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e7d1c65cd0 con 0x55e7d1eb5e00 2024-04-25T08:49:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 10 client.7815 handle_command_reply: tid=0 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 client.7815 shutdown 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 -- 172.21.15.46:0/2191338325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 msgr2=0x55e7d1eceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1eceb80 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4900013680 tx=0x7f4900013b40 comp rx=0 tx=0).stop 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 -- 172.21.15.46:0/2191338325 shutdown_connections 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55e7d1eb5e00 0x55e7d1eb61e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1eb5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1eceb80 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2191338325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1ece640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f48faa00640 0 client.7815 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 -- 172.21.15.46:0/2191338325 >> 172.21.15.46:0/2191338325 conn(0x55e7d1e79890 msgr2=0x55e7d1e7c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 -- 172.21.15.46:0/2191338325 shutdown_connections 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 1 -- 172.21.15.46:0/2191338325 wait complete. 2024-04-25T08:49:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.512+0000 7f491f8c3000 20 client.7815 trim_cache size 0 max 16384 2024-04-25T08:49:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.524+0000 7f491f8c3000 1 Processor -- start 2024-04-25T08:49:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.524+0000 7f491f8c3000 1 -- start start 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.524+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1e7cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1e7bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1e7c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4914202cf0 con 0x55e7d1ec72e0 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49141f7c20 con 0x55e7d1e7d8e0 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49142026f0 con 0x55e7d1ec7c00 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1e7bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1e7bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54628/0 (socket says 172.21.15.46:54628) 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919000640 1 -- 172.21.15.46:0/1245810593 learned_addr learned my addr 172.21.15.46:0/1245810593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919a00640 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1e7cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491a400640 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1e7c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919000640 1 -- 172.21.15.46:0/1245810593 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 msgr2=0x55e7d1e7c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919000640 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1e7c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919000640 1 -- 172.21.15.46:0/1245810593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 msgr2=0x55e7d1e7cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919000640 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1e7cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919000640 1 -- 172.21.15.46:0/1245810593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7d1c85fc0 con 0x55e7d1ec72e0 2024-04-25T08:49:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919a00640 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1e7cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4919000640 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1e7bc80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4900013b80 tx=0x7f4900009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3d197963b65d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4905600640 1 -- 172.21.15.46:0/1245810593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49000072d0 con 0x55e7d1ec72e0 2024-04-25T08:49:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4905600640 1 -- 172.21.15.46:0/1245810593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4900000e50 con 0x55e7d1ec72e0 2024-04-25T08:49:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f4905600640 1 -- 172.21.15.46:0/1245810593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490001e4b0 con 0x55e7d1ec72e0 2024-04-25T08:49:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- 172.21.15.46:0/1245810593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 msgr2=0x55e7d1e7bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1e7bc80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4900013b80 tx=0x7f4900009000 comp rx=0 tx=0).stop 2024-04-25T08:49:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- 172.21.15.46:0/1245810593 shutdown_connections 2024-04-25T08:49:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1ec7c00 0x55e7d1e7c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec72e0 0x55e7d1e7bc80 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 --2- 172.21.15.46:0/1245810593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1e7d8e0 0x55e7d1e7cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- 172.21.15.46:0/1245810593 >> 172.21.15.46:0/1245810593 conn(0x55e7d1e79890 msgr2=0x55e7d1ec6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- 172.21.15.46:0/1245810593 shutdown_connections 2024-04-25T08:49:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- 172.21.15.46:0/1245810593 wait complete. 2024-04-25T08:49:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- start start 2024-04-25T08:49:16.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 Processor -- start 2024-04-25T08:49:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1e7d8e0 0x55e7d1f27720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1ec72e0 0x55e7d1f27c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec7c00 0x55e7d1ecb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d1c64e70 con 0x55e7d1ec7c00 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d1c52a50 con 0x55e7d1ec72e0 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7d1cc93a0 con 0x55e7d1e7d8e0 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.528+0000 7f491f8c3000 10 client.0 _ll_register_callbacks cb 0x55e7d1c76240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec7c00 0x55e7d1ecb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f4919a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1e7d8e0 0x55e7d1f27720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f4919000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1ec72e0 0x55e7d1f27c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f4919a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1e7d8e0 0x55e7d1f27720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:43558/0 (socket says 172.21.15.46:43558) 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f4919a00640 1 -- 172.21.15.46:0/2590062566 learned_addr learned my addr 172.21.15.46:0/2590062566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f4919000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1ec72e0 0x55e7d1f27c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:50286/0 (socket says 172.21.15.46:50286) 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491a400640 1 -- 172.21.15.46:0/2590062566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1e7d8e0 msgr2=0x55e7d1f27720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491a400640 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1e7d8e0 0x55e7d1f27720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491a400640 1 -- 172.21.15.46:0/2590062566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1ec72e0 msgr2=0x55e7d1f27c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491a400640 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1ec72e0 0x55e7d1f27c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491a400640 1 -- 172.21.15.46:0/2590062566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7d1d239e0 con 0x55e7d1ec7c00 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f4919000640 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1ec72e0 0x55e7d1f27c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491a400640 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec7c00 0x55e7d1ecb9b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f490c083ca0 tx=0x7f490c05a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945652ed0d65a335 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 1 -- 172.21.15.46:0/2590062566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c055780 con 0x55e7d1ec7c00 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 1 -- 172.21.15.46:0/2590062566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f490c05c160 con 0x55e7d1ec7c00 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 -- 172.21.15.46:0/2590062566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4900000e50 con 0x55e7d1ec7c00 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 1 -- 172.21.15.46:0/2590062566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f490c0544f0 con 0x55e7d1ec7c00 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 10 client.7821 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 -- 172.21.15.46:0/2590062566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55e7d1c69630 con 0x55e7d1ec7c00 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 1 -- 172.21.15.46:0/2590062566 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4900000e50 con 0x55e7d1ec7c00 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 1 -- 172.21.15.46:0/2590062566 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f490c051e80 con 0x55e7d1ec7c00 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 10 client.7821 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 10 client.7821 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:49:16.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 20 client.7821 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55e7d1f2e580 0x55e7d1f30970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 4 client.7821 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:49:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 -- 172.21.15.46:0/2590062566 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55e7d1cd75d0 con 0x55e7d1f2e580 2024-04-25T08:49:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f4919a00640 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55e7d1f2e580 0x55e7d1f30970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f4919a00640 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55e7d1f2e580 0x55e7d1f30970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d53068d566c0ce server_cookie=9d7b4165181a44ec in_seq=0 out_seq=0 2024-04-25T08:49:16.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 10 client.7821 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 1 -- 172.21.15.46:0/2590062566 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+420 (crc 0 0 0) 0x55e7d1cd75d0 con 0x55e7d1f2e580 2024-04-25T08:49:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 10 client.7821 handle_command_reply: tid=0 2024-04-25T08:49:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 client.7821 shutdown 2024-04-25T08:49:16.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 -- 172.21.15.46:0/2590062566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec7c00 msgr2=0x55e7d1ecb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec7c00 0x55e7d1ecb9b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f490c083ca0 tx=0x7f490c05a1d0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 -- 172.21.15.46:0/2590062566 shutdown_connections 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55e7d1f2e580 0x55e7d1f30970 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e7d1ec7c00 0x55e7d1ecb9b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e7d1ec72e0 0x55e7d1f27c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 --2- 172.21.15.46:0/2590062566 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e7d1e7d8e0 0x55e7d1f27720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f48faa00640 0 client.7821 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 -- 172.21.15.46:0/2590062566 >> 172.21.15.46:0/2590062566 conn(0x55e7d1e79890 msgr2=0x55e7d1ec6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 -- 172.21.15.46:0/2590062566 shutdown_connections 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 1 -- 172.21.15.46:0/2590062566 wait complete. 2024-04-25T08:49:16.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491f8c3000 20 client.7821 trim_cache size 0 max 16384 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 1541640988, 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/testdir/hardlink" 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 3873852791, 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627776, 2024-04-25T08:49:16.532 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir" 2024-04-25T08:49:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:49:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:49:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "dir_frag", 2024-04-25T08:49:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4230945268, 2024-04-25T08:49:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627776, 2024-04-25T08:49:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "frag": "*", 2024-04-25T08:49:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir" 2024-04-25T08:49:16.533 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:49:16.533 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:49:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491c400640 1 -- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f48f4066f60 msgr2=0x7f48f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491c400640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f48f4066f60 0x7f48f4069420 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f49080ae000 tx=0x7f49080a7000 comp rx=0 tx=0).stop 2024-04-25T08:49:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491c400640 1 -- 172.21.15.46:0/357344664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491413a010 msgr2=0x7f49140af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491c400640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491413a010 0x7f49140af9a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f490c007e60 tx=0x7f490c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491c400640 1 -- 172.21.15.46:0/357344664 shutdown_connections 2024-04-25T08:49:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.532+0000 7f491c400640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f48f4066f60 0x7f48f4069420 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.536+0000 7f491c400640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f491413a010 0x7f49140af9a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.536+0000 7f491c400640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4914145310 0x7f49140b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.536+0000 7f491c400640 1 --2- 172.21.15.46:0/357344664 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f49140cf1b0 0x7f4914156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.536+0000 7f491c400640 1 -- 172.21.15.46:0/357344664 >> 172.21.15.46:0/357344664 conn(0x7f491406ac10 msgr2=0x7f4914159230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.536+0000 7f491c400640 1 -- 172.21.15.46:0/357344664 shutdown_connections 2024-04-25T08:49:16.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.536+0000 7f491c400640 1 -- 172.21.15.46:0/357344664 wait complete. 2024-04-25T08:49:16.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:49:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6acec00640 1 Processor -- start 2024-04-25T08:49:16.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6acec00640 1 -- start start 2024-04-25T08:49:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6acec00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6ac815d940 0x7f6ac80b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6acec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ac80af4a0 0x7f6ac80ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6acec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 0x7f6ac806f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6acec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac80b5820 con 0x7f6ac80af4a0 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6acec00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac80b3b50 con 0x7f6ac815d940 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6acec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac8070070 con 0x7f6ac806ee30 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6abfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ac80af4a0 0x7f6ac80ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6ac7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6ac815d940 0x7f6ac80b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6abfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ac80af4a0 0x7f6ac80ad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54648/0 (socket says 172.21.15.46:54648) 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6abfe00640 1 -- 172.21.15.46:0/3213411352 learned_addr learned my addr 172.21.15.46:0/3213411352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6accc00640 1 --2- 172.21.15.46:0/3213411352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 0x7f6ac806f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6abfe00640 1 -- 172.21.15.46:0/3213411352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 msgr2=0x7f6ac806f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6abfe00640 1 --2- 172.21.15.46:0/3213411352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 0x7f6ac806f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6abfe00640 1 -- 172.21.15.46:0/3213411352 >> v2:172.21.15.82:3300/0 conn(0x7f6ac815d940 msgr2=0x7f6ac80b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6abfe00640 1 --2- 172.21.15.46:0/3213411352 >> v2:172.21.15.82:3300/0 conn(0x7f6ac815d940 0x7f6ac80b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.696+0000 7f6abfe00640 1 -- 172.21.15.46:0/3213411352 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac8070350 con 0x7f6ac80af4a0 2024-04-25T08:49:16.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6abfe00640 1 --2- 172.21.15.46:0/3213411352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac80af4a0 0x7f6ac80ad4c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8071920 tx=0x7f6ab001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3887a517c0b1be39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6ac7400640 1 -- 172.21.15.46:0/3213411352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00b0070 con 0x7f6ac80af4a0 2024-04-25T08:49:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6ac7400640 1 -- 172.21.15.46:0/3213411352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab00ab600 con 0x7f6ac80af4a0 2024-04-25T08:49:16.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6ac7400640 1 -- 172.21.15.46:0/3213411352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab00ab9d0 con 0x7f6ac80af4a0 2024-04-25T08:49:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 -- 172.21.15.46:0/3213411352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac80af4a0 msgr2=0x7f6ac80ad4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 --2- 172.21.15.46:0/3213411352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac80af4a0 0x7f6ac80ad4c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8071920 tx=0x7f6ab001c400 comp rx=0 tx=0).stop 2024-04-25T08:49:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 -- 172.21.15.46:0/3213411352 shutdown_connections 2024-04-25T08:49:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 --2- 172.21.15.46:0/3213411352 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 0x7f6ac806f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 --2- 172.21.15.46:0/3213411352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac80af4a0 0x7f6ac80ad4c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 --2- 172.21.15.46:0/3213411352 >> v2:172.21.15.82:3300/0 conn(0x7f6ac815d940 0x7f6ac80b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 -- 172.21.15.46:0/3213411352 >> 172.21.15.46:0/3213411352 conn(0x7f6ac80b5a00 msgr2=0x7f6ac80b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 -- 172.21.15.46:0/3213411352 shutdown_connections 2024-04-25T08:49:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 -- 172.21.15.46:0/3213411352 wait complete. 2024-04-25T08:49:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.700+0000 7f6acec00640 1 Processor -- start 2024-04-25T08:49:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 -- start start 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 0x7f6ac814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac80af4a0 0x7f6ac815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac815d940 0x7f6ac815ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac815dea0 con 0x7f6ac815d940 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac815dd20 con 0x7f6ac80af4a0 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac80ada00 con 0x7f6ac806ee30 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6accc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac815d940 0x7f6ac815ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6accc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac815d940 0x7f6ac815ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54662/0 (socket says 172.21.15.46:54662) 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6accc00640 1 -- 172.21.15.46:0/3043164957 learned_addr learned my addr 172.21.15.46:0/3043164957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6ac7e00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 0x7f6ac814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6abfe00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac80af4a0 0x7f6ac815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6accc00640 1 -- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 msgr2=0x7f6ac814a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6accc00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 0x7f6ac814a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6accc00640 1 -- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac80af4a0 msgr2=0x7f6ac815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6accc00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac80af4a0 0x7f6ac815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6accc00640 1 -- 172.21.15.46:0/3043164957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac81595c0 con 0x7f6ac815d940 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6abfe00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac80af4a0 0x7f6ac815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6accc00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac815d940 0x7f6ac815ec30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000efc0 tx=0x7f6ac000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6ac4c00640 1 -- 172.21.15.46:0/3043164957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0018020 con 0x7f6ac815d940 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6ac4c00640 1 -- 172.21.15.46:0/3043164957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac0014480 con 0x7f6ac815d940 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac81598a0 con 0x7f6ac815d940 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6ac4c00640 1 -- 172.21.15.46:0/3043164957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0014870 con 0x7f6ac815d940 2024-04-25T08:49:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac8159e60 con 0x7f6ac815d940 2024-04-25T08:49:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.704+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac8074c60 con 0x7f6ac815d940 2024-04-25T08:49:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.708+0000 7f6ac4c00640 1 -- 172.21.15.46:0/3043164957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ac001b020 con 0x7f6ac815d940 2024-04-25T08:49:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.708+0000 7f6ac4c00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6aa4066f60 0x7f6aa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.708+0000 7f6ac4c00640 1 -- 172.21.15.46:0/3043164957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6ac0086560 con 0x7f6ac815d940 2024-04-25T08:49:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.708+0000 7f6ac7e00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6aa4066f60 0x7f6aa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.712+0000 7f6ac7e00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6aa4066f60 0x7f6aa4069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80d15a0 tx=0x7f6ab8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.712+0000 7f6ac4c00640 1 -- 172.21.15.46:0/3043164957 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ac0022030 con 0x7f6ac815d940 2024-04-25T08:49:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.904+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6ac8168450 con 0x7f6ac815d940 2024-04-25T08:49:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.904+0000 7f6ac4c00640 1 -- 172.21.15.46:0/3043164957 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f6ac0010040 con 0x7f6ac815d940 2024-04-25T08:49:16.900 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:16.900 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T08:49:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6aa4066f60 msgr2=0x7f6aa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6aa4066f60 0x7f6aa4069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80d15a0 tx=0x7f6ab8008040 comp rx=0 tx=0).stop 2024-04-25T08:49:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac815d940 msgr2=0x7f6ac815ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac815d940 0x7f6ac815ec30 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000efc0 tx=0x7f6ac000c490 comp rx=0 tx=0).stop 2024-04-25T08:49:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 shutdown_connections 2024-04-25T08:49:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6aa4066f60 0x7f6aa4069420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ac815d940 0x7f6ac815ec30 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac80af4a0 0x7f6ac815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 --2- 172.21.15.46:0/3043164957 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ac806ee30 0x7f6ac814a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 >> 172.21.15.46:0/3043164957 conn(0x7f6ac80b5a00 msgr2=0x7f6ac80b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 shutdown_connections 2024-04-25T08:49:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:16.908+0000 7f6acec00640 1 -- 172.21.15.46:0/3043164957 wait complete. 2024-04-25T08:49:16.920 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:49:16.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:16.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T08:49:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece4600640 1 Processor -- start 2024-04-25T08:49:17.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece4600640 1 -- start start 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece4600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecdc15f190 0x7fecdc099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece4600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fecdc071280 0x7fecdc099f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece4600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecdc09aba0 con 0x7fecdc06f4d0 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece4600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc0ae3e0 con 0x7fecdc15f190 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece4600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc0cc740 con 0x7fecdc071280 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fecdc15f190 0x7fecdc099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece2600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fecdc06f4d0 0x7fecdc06f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46788/0 (socket says 172.21.15.46:46788) 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece2600640 1 -- 172.21.15.46:0/148698442 learned_addr learned my addr 172.21.15.46:0/148698442 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece1c00640 1 -- 172.21.15.46:0/148698442 >> v2:172.21.15.82:3300/0 conn(0x7fecdc071280 msgr2=0x7fecdc099f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece1200640 1 --2- 172.21.15.46:0/148698442 >> v2:172.21.15.82:3300/0 conn(0x7fecdc071280 0x7fecdc099f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece1c00640 1 --2- 172.21.15.46:0/148698442 >> v2:172.21.15.82:3300/0 conn(0x7fecdc071280 0x7fecdc099f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece1c00640 1 -- 172.21.15.46:0/148698442 >> v1:172.21.15.46:6789/0 conn(0x7fecdc06f4d0 legacy=0x7fecdc06f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fecdbe00640 1 -- 172.21.15.46:0/148698442 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110279281 0 0) 0x7fecdc09aba0 con 0x7fecdc06f4d0 2024-04-25T08:49:17.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece1c00640 1 -- 172.21.15.46:0/148698442 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc0cc3d0 con 0x7fecdc15f190 2024-04-25T08:49:17.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.072+0000 7fece1200640 1 --2- 172.21.15.46:0/148698442 >> v2:172.21.15.82:3300/0 conn(0x7fecdc071280 0x7fecdc099f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:17.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fece1c00640 1 --2- 172.21.15.46:0/148698442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc15f190 0x7fecdc099a00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0b7720 tx=0x7fecd001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7793833c0ccd663e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fecdbe00640 1 -- 172.21.15.46:0/148698442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00b0070 con 0x7fecdc15f190 2024-04-25T08:49:17.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fecdbe00640 1 -- 172.21.15.46:0/148698442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd00ab600 con 0x7fecdc15f190 2024-04-25T08:49:17.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fecdbe00640 1 -- 172.21.15.46:0/148698442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00ab9d0 con 0x7fecdc15f190 2024-04-25T08:49:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fece4600640 1 -- 172.21.15.46:0/148698442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc15f190 msgr2=0x7fecdc099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fece4600640 1 --2- 172.21.15.46:0/148698442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc15f190 0x7fecdc099a00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0b7720 tx=0x7fecd001c3f0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fece4600640 1 -- 172.21.15.46:0/148698442 shutdown_connections 2024-04-25T08:49:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fece4600640 1 --2- 172.21.15.46:0/148698442 >> v2:172.21.15.82:3300/0 conn(0x7fecdc071280 0x7fecdc099f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fece4600640 1 --2- 172.21.15.46:0/148698442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc15f190 0x7fecdc099a00 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fece4600640 1 -- 172.21.15.46:0/148698442 >> 172.21.15.46:0/148698442 conn(0x7fecdc0b3160 msgr2=0x7fecdc0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fece4600640 1 -- 172.21.15.46:0/148698442 shutdown_connections 2024-04-25T08:49:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.076+0000 7fece4600640 1 -- 172.21.15.46:0/148698442 wait complete. 2024-04-25T08:49:17.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 Processor -- start 2024-04-25T08:49:17.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 -- start start 2024-04-25T08:49:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecdc06f4d0 0x7fecdc1f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc071280 0x7fecdc1f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc15f190 0x7fecdc15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0cdd60 con 0x7fecdc071280 2024-04-25T08:49:17.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0cdbe0 con 0x7fecdc15f190 2024-04-25T08:49:17.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc168ee0 con 0x7fecdc06f4d0 2024-04-25T08:49:17.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc071280 0x7fecdc1f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc071280 0x7fecdc1f74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54690/0 (socket says 172.21.15.46:54690) 2024-04-25T08:49:17.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece1200640 1 -- 172.21.15.46:0/1285988265 learned_addr learned my addr 172.21.15.46:0/1285988265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece1200640 1 -- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecdc06f4d0 msgr2=0x7fecdc1f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece2600640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc15f190 0x7fecdc15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece1200640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecdc06f4d0 0x7fecdc1f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece1200640 1 -- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc15f190 msgr2=0x7fecdc15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece1200640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc15f190 0x7fecdc15cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece1200640 1 -- 172.21.15.46:0/1285988265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc1f8230 con 0x7fecdc071280 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece2600640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc15f190 0x7fecdc15cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece1200640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc071280 0x7fecdc1f74b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fecc800c820 tx=0x7fecc800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fecd9600640 1 -- 172.21.15.46:0/1285988265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8015070 con 0x7fecdc071280 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fecd9600640 1 -- 172.21.15.46:0/1285988265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc8007a30 con 0x7fecdc071280 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fecd9600640 1 -- 172.21.15.46:0/1285988265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8007e40 con 0x7fecdc071280 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 -- 172.21.15.46:0/1285988265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc0c87e0 con 0x7fecdc071280 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.080+0000 7fece4600640 1 -- 172.21.15.46:0/1285988265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc0c9080 con 0x7fecdc071280 2024-04-25T08:49:17.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fecd9600640 1 -- 172.21.15.46:0/1285988265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fecc8018070 con 0x7fecdc071280 2024-04-25T08:49:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fecd9600640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecbc066f90 0x7fecbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fecd9600640 1 -- 172.21.15.46:0/1285988265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fecc8085260 con 0x7fecdc071280 2024-04-25T08:49:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece7b9d000 1 Processor -- start 2024-04-25T08:49:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1c00640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecbc066f90 0x7fecbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece7b9d000 1 -- start start 2024-04-25T08:49:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1c00640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecbc066f90 0x7fecbc069450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fecd00ad000 tx=0x7fecd0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1bc90 0x55c995f1c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995ed1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 0x55c995ed2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece7b9d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c995db8f40 con 0x55c995f1c5b0 2024-04-25T08:49:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece7b9d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c995d20070 con 0x55c995ed2290 2024-04-25T08:49:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece7b9d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c995ca81a0 con 0x55c995f1bc90 2024-04-25T08:49:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995ed1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995ed1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54704/0 (socket says 172.21.15.46:54704) 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1200640 1 -- 172.21.15.46:0/56685699 learned_addr learned my addr 172.21.15.46:0/56685699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1c00640 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1bc90 0x55c995f1c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece2600640 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 0x55c995ed2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1200640 1 -- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1bc90 msgr2=0x55c995f1c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1200640 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1bc90 0x55c995f1c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1200640 1 -- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 msgr2=0x55c995ed2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1200640 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 0x55c995ed2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1200640 1 -- 172.21.15.46:0/56685699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c995ca6510 con 0x55c995f1c5b0 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1c00640 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1bc90 0x55c995f1c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece2600640 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 0x55c995ed2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fece1200640 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995ed1d50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fecc805da60 tx=0x7fecc80558b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d36a631c913afc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fecce000640 1 -- 172.21.15.46:0/56685699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc801f040 con 0x55c995f1c5b0 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fecce000640 1 -- 172.21.15.46:0/56685699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc800ce10 con 0x55c995f1c5b0 2024-04-25T08:49:17.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.084+0000 7fecce000640 1 -- 172.21.15.46:0/56685699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8051e60 con 0x55c995f1c5b0 2024-04-25T08:49:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- 172.21.15.46:0/56685699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 msgr2=0x55c995ed1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995ed1d50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fecc805da60 tx=0x7fecc80558b0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- 172.21.15.46:0/56685699 shutdown_connections 2024-04-25T08:49:17.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 0x55c995ed2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995ed1d50 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 --2- 172.21.15.46:0/56685699 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1bc90 0x55c995f1c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- 172.21.15.46:0/56685699 >> 172.21.15.46:0/56685699 conn(0x55c995ece240 msgr2=0x55c995ed0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- 172.21.15.46:0/56685699 shutdown_connections 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- 172.21.15.46:0/56685699 wait complete. 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- start start 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 Processor -- start 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995ed2290 0x55c995edce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 0x55c995edd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995edd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c995cda6c0 con 0x55c995ed2290 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c995d1c110 con 0x55c995f1bc90 2024-04-25T08:49:17.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c995dd6f20 con 0x55c995f1c5b0 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 10 client.0 _ll_register_callbacks cb 0x55c995ebd570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995ed2290 0x55c995edce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995ed2290 0x55c995edce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54714/0 (socket says 172.21.15.46:54714) 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1c00640 1 -- 172.21.15.46:0/3186562346 learned_addr learned my addr 172.21.15.46:0/3186562346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece2600640 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995edd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1200640 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 0x55c995edd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1c00640 1 -- 172.21.15.46:0/3186562346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 msgr2=0x55c995edd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1c00640 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995edd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1c00640 1 -- 172.21.15.46:0/3186562346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 msgr2=0x55c995edd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1c00640 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 0x55c995edd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1c00640 1 -- 172.21.15.46:0/3186562346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c995ca83b0 con 0x55c995ed2290 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece2600640 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995edd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece1c00640 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995ed2290 0x55c995edce30 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fecd0019000 tx=0x7fecd00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca76dd1559a4ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fecc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fecc2a00640 1 -- 172.21.15.46:0/3186562346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00b0040 con 0x55c995ed2290 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fece7b9d000 1 -- 172.21.15.46:0/3186562346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecc800ce10 con 0x55c995ed2290 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.088+0000 7fecc2a00640 1 -- 172.21.15.46:0/3186562346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fecd00b65f0 con 0x55c995ed2290 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fecc2a00640 1 -- 172.21.15.46:0/3186562346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd00ce050 con 0x55c995ed2290 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 10 client.7842 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 -- 172.21.15.46:0/3186562346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c995da1e00 con 0x55c995ed2290 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fecc2a00640 1 -- 172.21.15.46:0/3186562346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc800ce10 con 0x55c995ed2290 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fecc2a00640 1 -- 172.21.15.46:0/3186562346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fecd00d1020 con 0x55c995ed2290 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 10 client.7842 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 10 client.7842 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 20 client.7842 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c995ede150 0x55c995f2d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 4 client.7842 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:49:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 -- 172.21.15.46:0/3186562346 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c995cbbd60 con 0x55c995ede150 2024-04-25T08:49:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece1200640 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c995ede150 0x55c995f2d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece1200640 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c995ede150 0x55c995f2d410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b6b11c517327b9 server_cookie=267a69efc398d87f in_seq=0 out_seq=0 2024-04-25T08:49:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fecc2a00640 10 client.7842 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fecc2a00640 1 -- 172.21.15.46:0/3186562346 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c995cbbd60 con 0x55c995ede150 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fecc2a00640 10 client.7842 handle_command_reply: tid=0 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 client.7842 shutdown 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 -- 172.21.15.46:0/3186562346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995ed2290 msgr2=0x55c995edce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995ed2290 0x55c995edce30 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fecd0019000 tx=0x7fecd00046d0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 -- 172.21.15.46:0/3186562346 shutdown_connections 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c995ede150 0x55c995f2d410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995edd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 0x55c995edd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fecc2a00640 0 client.7842 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 --2- 172.21.15.46:0/3186562346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995ed2290 0x55c995edce30 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 -- 172.21.15.46:0/3186562346 >> 172.21.15.46:0/3186562346 conn(0x55c995ece240 msgr2=0x55c995ed0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 -- 172.21.15.46:0/3186562346 shutdown_connections 2024-04-25T08:49:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 1 -- 172.21.15.46:0/3186562346 wait complete. 2024-04-25T08:49:17.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.092+0000 7fece7b9d000 20 client.7842 trim_cache size 0 max 16384 2024-04-25T08:49:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 Processor -- start 2024-04-25T08:49:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 -- start start 2024-04-25T08:49:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 0x55c995ed1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1bc90 0x55c995ed1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995f2d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc203740 con 0x55c995f1bc90 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc2028a0 con 0x55c995ed2290 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1faa30 con 0x55c995f1c5b0 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1bc90 0x55c995ed1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1bc90 0x55c995ed1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54724/0 (socket says 172.21.15.46:54724) 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1200640 1 -- 172.21.15.46:0/4141534670 learned_addr learned my addr 172.21.15.46:0/4141534670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1c00640 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 0x55c995ed1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece2600640 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995f2d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1200640 1 -- 172.21.15.46:0/4141534670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 msgr2=0x55c995f2d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1200640 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995f2d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1200640 1 -- 172.21.15.46:0/4141534670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 msgr2=0x55c995ed1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1200640 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 0x55c995ed1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1200640 1 -- 172.21.15.46:0/4141534670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c995d77c50 con 0x55c995f1bc90 2024-04-25T08:49:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece2600640 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995f2d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece1200640 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1bc90 0x55c995ed1b80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fecc80554f0 tx=0x7fecc801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ff57a7b9aab93e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7feccd600640 1 -- 172.21.15.46:0/4141534670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc801f040 con 0x55c995f1bc90 2024-04-25T08:49:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7feccd600640 1 -- 172.21.15.46:0/4141534670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc8053890 con 0x55c995f1bc90 2024-04-25T08:49:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7feccd600640 1 -- 172.21.15.46:0/4141534670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8051e60 con 0x55c995f1bc90 2024-04-25T08:49:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 -- 172.21.15.46:0/4141534670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1bc90 msgr2=0x55c995ed1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.108+0000 7fece7b9d000 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1bc90 0x55c995ed1b80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fecc80554f0 tx=0x7fecc801c000 comp rx=0 tx=0).stop 2024-04-25T08:49:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- 172.21.15.46:0/4141534670 shutdown_connections 2024-04-25T08:49:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995f1c5b0 0x55c995f2d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1bc90 0x55c995ed1b80 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 --2- 172.21.15.46:0/4141534670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995ed2290 0x55c995ed1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- 172.21.15.46:0/4141534670 >> 172.21.15.46:0/4141534670 conn(0x55c995ece240 msgr2=0x55c995ed6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- 172.21.15.46:0/4141534670 shutdown_connections 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- 172.21.15.46:0/4141534670 wait complete. 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- start start 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 Processor -- start 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995ed2290 0x55c995f7c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 0x55c995f7c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995f37ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c995ed7b70 con 0x55c995f1c5b0 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc202490 con 0x55c995f1bc90 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c995d1f380 con 0x55c995ed2290 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 10 client.0 _ll_register_callbacks cb 0x7fecc8051b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995f37ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995f37ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54726/0 (socket says 172.21.15.46:54726) 2024-04-25T08:49:17.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece2600640 1 -- 172.21.15.46:0/2702273398 learned_addr learned my addr 172.21.15.46:0/2702273398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece1c00640 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995ed2290 0x55c995f7c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece1200640 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 0x55c995f7c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece2600640 1 -- 172.21.15.46:0/2702273398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995ed2290 msgr2=0x55c995f7c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece2600640 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995ed2290 0x55c995f7c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece2600640 1 -- 172.21.15.46:0/2702273398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 msgr2=0x55c995f7c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece2600640 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 0x55c995f7c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece2600640 1 -- 172.21.15.46:0/2702273398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c995ce8120 con 0x55c995f1c5b0 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece2600640 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995f37ee0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fecd4014960 tx=0x7fecd4014e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48a0cd014798ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fecc2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fecc2a00640 1 -- 172.21.15.46:0/2702273398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd401a070 con 0x55c995f1c5b0 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fecc2a00640 1 -- 172.21.15.46:0/2702273398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fecd4020d60 con 0x55c995f1c5b0 2024-04-25T08:49:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- 172.21.15.46:0/2702273398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecc8053890 con 0x55c995f1c5b0 2024-04-25T08:49:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fecc2a00640 1 -- 172.21.15.46:0/2702273398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd4012ab0 con 0x55c995f1c5b0 2024-04-25T08:49:17.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 10 client.7848 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- 172.21.15.46:0/2702273398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c995cc7170 con 0x55c995f1c5b0 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fecc2a00640 1 -- 172.21.15.46:0/2702273398 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc8053890 con 0x55c995f1c5b0 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fecc2a00640 1 -- 172.21.15.46:0/2702273398 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fecd4012c50 con 0x55c995f1c5b0 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 10 client.7848 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 10 client.7848 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 20 client.7848 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c995f38c70 0x55c995f39080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 4 client.7848 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.112+0000 7fece7b9d000 1 -- 172.21.15.46:0/2702273398 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55c995d2d5b0 con 0x55c995f38c70 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece1c00640 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c995f38c70 0x55c995f39080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece1c00640 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c995f38c70 0x55c995f39080 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb6914eaf243b93 server_cookie=64efe946f1ba734e in_seq=0 out_seq=0 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fecc2a00640 10 client.7848 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:17.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fecc2a00640 1 -- 172.21.15.46:0/2702273398 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c995d2d5b0 con 0x55c995f38c70 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fecc2a00640 10 client.7848 handle_command_reply: tid=0 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 client.7848 shutdown 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 -- 172.21.15.46:0/2702273398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 msgr2=0x55c995f37ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995f37ee0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fecd4014960 tx=0x7fecd4014e20 comp rx=0 tx=0).stop 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 -- 172.21.15.46:0/2702273398 shutdown_connections 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c995f38c70 0x55c995f39080 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c995f1c5b0 0x55c995f37ee0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c995f1bc90 0x55c995f7c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 --2- 172.21.15.46:0/2702273398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c995ed2290 0x55c995f7c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 -- 172.21.15.46:0/2702273398 >> 172.21.15.46:0/2702273398 conn(0x55c995ece240 msgr2=0x55c995ed0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fecc2a00640 0 client.7848 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 -- 172.21.15.46:0/2702273398 shutdown_connections 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 1 -- 172.21.15.46:0/2702273398 wait complete. 2024-04-25T08:49:17.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece7b9d000 20 client.7848 trim_cache size 0 max 16384 2024-04-25T08:49:17.114 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:49:17.116+0000 7f9045200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-25T08:48:35.444502+0000 1=1+0)->f(v1 m2024-04-25T08:48:35.444502+0000) n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1)->n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x557050529b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T08:48:35.444502+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T08:49:17.114 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:49:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:49:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "cc042e3e-b984-4bbc-a564-8387eeb83b47", 2024-04-25T08:49:17.114 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:49:17.114 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:49:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 -- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecbc066f90 msgr2=0x7fecbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecbc066f90 0x7fecbc069450 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fecd00ad000 tx=0x7fecd0008040 comp rx=0 tx=0).stop 2024-04-25T08:49:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 -- 172.21.15.46:0/1285988265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc071280 msgr2=0x7fecdc1f74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc071280 0x7fecdc1f74b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fecc800c820 tx=0x7fecc800cce0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 -- 172.21.15.46:0/1285988265 shutdown_connections 2024-04-25T08:49:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fecbc066f90 0x7fecbc069450 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc15f190 0x7fecdc15cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecdc071280 0x7fecdc1f74b0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 --2- 172.21.15.46:0/1285988265 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecdc06f4d0 0x7fecdc1f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 -- 172.21.15.46:0/1285988265 >> 172.21.15.46:0/1285988265 conn(0x7fecdc0b3160 msgr2=0x7fecdc0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 -- 172.21.15.46:0/1285988265 shutdown_connections 2024-04-25T08:49:17.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.116+0000 7fece4600640 1 -- 172.21.15.46:0/1285988265 wait complete. 2024-04-25T08:49:17.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:17.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:49:17.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2544400640 1 Processor -- start 2024-04-25T08:49:17.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2544400640 1 -- start start 2024-04-25T08:49:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2544400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c143990 0x7f253c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2544400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f253c06f150 0x7f253c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2544400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c15e440 con 0x7f253c1440d0 2024-04-25T08:49:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2544400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c15ed30 con 0x7f253c06f150 2024-04-25T08:49:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2544400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15e6e0 con 0x7f253c143990 2024-04-25T08:49:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2541000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f253c1440d0 0x7f253c06ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46790/0 (socket says 172.21.15.46:46790) 2024-04-25T08:49:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2541000640 1 -- 172.21.15.46:0/1660339636 learned_addr learned my addr 172.21.15.46:0/1660339636 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2541a00640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c143990 0x7f253c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577875490 0 0) 0x7f253c15e440 con 0x7f253c1440d0 2024-04-25T08:49:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2542400640 1 --2- 172.21.15.46:0/1660339636 >> v2:172.21.15.82:3300/0 conn(0x7f253c06f150 0x7f253c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2524003530 con 0x7f253c1440d0 2024-04-25T08:49:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3690038529 0 0) 0x7f2524003530 con 0x7f253c1440d0 2024-04-25T08:49:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c143990 msgr2=0x7f253c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c143990 0x7f253c0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 >> v2:172.21.15.82:3300/0 conn(0x7f253c06f150 msgr2=0x7f253c0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 --2- 172.21.15.46:0/1660339636 >> v2:172.21.15.82:3300/0 conn(0x7f253c06f150 0x7f253c0b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c15e9c0 con 0x7f253c1440d0 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f2528001680 con 0x7f253c1440d0 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 >> v1:172.21.15.46:6789/0 conn(0x7f253c1440d0 legacy=0x7f253c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c143990 0x7f2524003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2524004000 0x7f2524004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2524004d50 0x7f25240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15ed30 con 0x7f2524004d50 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15e6e0 con 0x7f2524004000 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252400bad0 con 0x7f253c143990 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2542400640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2524004d50 0x7f25240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2541000640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2524004000 0x7f2524004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.280+0000 7f2541a00640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c143990 0x7f2524003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2542400640 1 -- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c143990 msgr2=0x7f2524003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2542400640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c143990 0x7f2524003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2542400640 1 -- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2524004000 msgr2=0x7f2524004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2542400640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2524004000 0x7f2524004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2542400640 1 -- 172.21.15.46:0/1660339636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524003530 con 0x7f2524004d50 2024-04-25T08:49:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2542400640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2524004d50 0x7f25240051d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2534009fc0 tx=0x7f253401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5367b44c2d6b804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340b2070 con 0x7f2524004d50 2024-04-25T08:49:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25340a5cd0 con 0x7f2524004d50 2024-04-25T08:49:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f253be00640 1 -- 172.21.15.46:0/1660339636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534009c10 con 0x7f2524004d50 2024-04-25T08:49:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2544400640 1 -- 172.21.15.46:0/1660339636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2524004d50 msgr2=0x7f25240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2544400640 1 --2- 172.21.15.46:0/1660339636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2524004d50 0x7f25240051d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2534009fc0 tx=0x7f253401ccc0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2542400640 1 -- 172.21.15.46:0/1660339636 reap_dead start 2024-04-25T08:49:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2544400640 1 -- 172.21.15.46:0/1660339636 shutdown_connections 2024-04-25T08:49:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2544400640 1 -- 172.21.15.46:0/1660339636 >> 172.21.15.46:0/1660339636 conn(0x7f253c06ac10 msgr2=0x7f253c06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2544400640 1 -- 172.21.15.46:0/1660339636 shutdown_connections 2024-04-25T08:49:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.284+0000 7f2544400640 1 -- 172.21.15.46:0/1660339636 wait complete. 2024-04-25T08:49:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 Processor -- start 2024-04-25T08:49:17.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 -- start start 2024-04-25T08:49:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c13ddd0 0x7f253c13b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c13bde0 0x7f253c142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c142560 0x7f253c1f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c143d70 con 0x7f253c142560 2024-04-25T08:49:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15f1c0 con 0x7f253c13bde0 2024-04-25T08:49:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c143ef0 con 0x7f253c13ddd0 2024-04-25T08:49:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2542400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c142560 0x7f253c1f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2542400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c142560 0x7f253c1f7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54754/0 (socket says 172.21.15.46:54754) 2024-04-25T08:49:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2542400640 1 -- 172.21.15.46:0/4235078605 learned_addr learned my addr 172.21.15.46:0/4235078605 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2541000640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c13bde0 0x7f253c142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2541a00640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c13ddd0 0x7f253c13b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2542400640 1 -- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c13ddd0 msgr2=0x7f253c13b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2542400640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c13ddd0 0x7f253c13b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2542400640 1 -- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c13bde0 msgr2=0x7f253c142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2542400640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c13bde0 0x7f253c142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2542400640 1 -- 172.21.15.46:0/4235078605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c13c360 con 0x7f253c142560 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2541000640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c13bde0 0x7f253c142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2541a00640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c13ddd0 0x7f253c13b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2542400640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c142560 0x7f253c1f7000 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f25340b0040 tx=0x7f2534009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2539600640 1 -- 172.21.15.46:0/4235078605 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340b2040 con 0x7f253c142560 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2539600640 1 -- 172.21.15.46:0/4235078605 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25340a5ad0 con 0x7f253c142560 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2539600640 1 -- 172.21.15.46:0/4235078605 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534009c10 con 0x7f253c142560 2024-04-25T08:49:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 -- 172.21.15.46:0/4235078605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c13c640 con 0x7f253c142560 2024-04-25T08:49:17.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.288+0000 7f2544400640 1 -- 172.21.15.46:0/4235078605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c13cc00 con 0x7f253c142560 2024-04-25T08:49:17.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2539600640 1 -- 172.21.15.46:0/4235078605 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25340ae930 con 0x7f253c142560 2024-04-25T08:49:17.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2539600640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2539600640 1 -- 172.21.15.46:0/4235078605 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f253411f1e0 con 0x7f253c142560 2024-04-25T08:49:17.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2547863000 1 Processor -- start 2024-04-25T08:49:17.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2541a00640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2547863000 1 -- start start 2024-04-25T08:49:17.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2541a00640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2530009ad0 tx=0x7f2530008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2547863000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4cd9d0 0x555a9c4cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2547863000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 0x555a9c483b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2547863000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4840a0 0x555a9c484480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2547863000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a9c2d0ff0 con 0x555a9c4cd9d0 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2547863000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a9c259150 con 0x555a9c4840a0 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.292+0000 7f2547863000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a9c2cd090 con 0x555a9c4ce2f0 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4cd9d0 0x555a9c4cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4cd9d0 0x555a9c4cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54764/0 (socket says 172.21.15.46:54764) 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541a00640 1 -- 172.21.15.46:0/3059717450 learned_addr learned my addr 172.21.15.46:0/3059717450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2542400640 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4840a0 0x555a9c484480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541000640 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 0x555a9c483b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541a00640 1 -- 172.21.15.46:0/3059717450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 msgr2=0x555a9c483b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541a00640 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 0x555a9c483b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541a00640 1 -- 172.21.15.46:0/3059717450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4840a0 msgr2=0x555a9c484480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541a00640 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4840a0 0x555a9c484480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541a00640 1 -- 172.21.15.46:0/3059717450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a9c4670d0 con 0x555a9c4cd9d0 2024-04-25T08:49:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2542400640 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4840a0 0x555a9c484480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2541a00640 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4cd9d0 0x555a9c4cddb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2530013df0 tx=0x7f2530016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4717580d83c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f252e000640 1 -- 172.21.15.46:0/3059717450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530029070 con 0x555a9c4cd9d0 2024-04-25T08:49:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f252e000640 1 -- 172.21.15.46:0/3059717450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253001f050 con 0x555a9c4cd9d0 2024-04-25T08:49:17.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f252e000640 1 -- 172.21.15.46:0/3059717450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530003da0 con 0x555a9c4cd9d0 2024-04-25T08:49:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 -- 172.21.15.46:0/3059717450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4cd9d0 msgr2=0x555a9c4cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4cd9d0 0x555a9c4cddb0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2530013df0 tx=0x7f2530016000 comp rx=0 tx=0).stop 2024-04-25T08:49:17.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 -- 172.21.15.46:0/3059717450 shutdown_connections 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4840a0 0x555a9c484480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 0x555a9c483b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 --2- 172.21.15.46:0/3059717450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4cd9d0 0x555a9c4cddb0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 -- 172.21.15.46:0/3059717450 >> 172.21.15.46:0/3059717450 conn(0x555a9c480020 msgr2=0x555a9c482440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 -- 172.21.15.46:0/3059717450 shutdown_connections 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 -- 172.21.15.46:0/3059717450 wait complete. 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 -- start start 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 Processor -- start 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 0x555a9c4b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a9c295250 con 0x555a9c4840a0 2024-04-25T08:49:17.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a9c28b690 con 0x555a9c4ce2f0 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.296+0000 7f2547863000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a9c259a40 con 0x555a9c4cd9d0 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 10 client.0 _ll_register_callbacks cb 0x555a9c46ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4b27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54774/0 (socket says 172.21.15.46:54774) 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541a00640 1 -- 172.21.15.46:0/1373456848 learned_addr learned my addr 172.21.15.46:0/1373456848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541000640 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 0x555a9c4b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2542400640 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541a00640 1 -- 172.21.15.46:0/1373456848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 msgr2=0x555a9c4b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541a00640 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 0x555a9c4b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541a00640 1 -- 172.21.15.46:0/1373456848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 msgr2=0x555a9c4b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541a00640 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541a00640 1 -- 172.21.15.46:0/1373456848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a9c37e1c0 con 0x555a9c4840a0 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2542400640 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541a00640 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4b27f0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2530008010 tx=0x7f2530004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dded5be29caa4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 1 -- 172.21.15.46:0/1373456848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530033020 con 0x555a9c4840a0 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 1 -- 172.21.15.46:0/1373456848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f253002f730 con 0x555a9c4840a0 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 -- 172.21.15.46:0/1373456848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f253001f050 con 0x555a9c4840a0 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 1 -- 172.21.15.46:0/1373456848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2530029040 con 0x555a9c4840a0 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 10 client.7860 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 -- 172.21.15.46:0/1373456848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x555a9c37a170 con 0x555a9c4840a0 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 1 -- 172.21.15.46:0/1373456848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253001f050 con 0x555a9c4840a0 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 1 -- 172.21.15.46:0/1373456848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f253002f930 con 0x555a9c4840a0 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 10 client.7860 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 10 client.7860 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 20 client.7860 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555a9c4ddc80 0x555a9c4e0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 4 client.7860 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:49:17.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 -- 172.21.15.46:0/1373456848 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a9c26cc60 con 0x555a9c4ddc80 2024-04-25T08:49:17.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541000640 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555a9c4ddc80 0x555a9c4e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2541000640 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555a9c4ddc80 0x555a9c4e0070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67e2e99a25230752 server_cookie=1fe967912977c6f2 in_seq=0 out_seq=0 2024-04-25T08:49:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 10 client.7860 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 1 -- 172.21.15.46:0/1373456848 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a9c26cc60 con 0x555a9c4ddc80 2024-04-25T08:49:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 10 client.7860 handle_command_reply: tid=0 2024-04-25T08:49:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 client.7860 shutdown 2024-04-25T08:49:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 -- 172.21.15.46:0/1373456848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 msgr2=0x555a9c4b27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4b27f0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2530008010 tx=0x7f2530004740 comp rx=0 tx=0).stop 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 -- 172.21.15.46:0/1373456848 shutdown_connections 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555a9c4ddc80 0x555a9c4e0070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 0x555a9c4b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 --2- 172.21.15.46:0/1373456848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4b27f0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2522a00640 0 client.7860 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 -- 172.21.15.46:0/1373456848 >> 172.21.15.46:0/1373456848 conn(0x555a9c480020 msgr2=0x555a9c482ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 -- 172.21.15.46:0/1373456848 shutdown_connections 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 1 -- 172.21.15.46:0/1373456848 wait complete. 2024-04-25T08:49:17.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.300+0000 7f2547863000 20 client.7860 trim_cache size 0 max 16384 2024-04-25T08:49:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.316+0000 7f2547863000 1 Processor -- start 2024-04-25T08:49:17.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.316+0000 7f2547863000 1 -- start start 2024-04-25T08:49:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c52a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4cd9d0 0x555a9c52a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 0x555a9c52ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c200fa0 con 0x555a9c4840a0 2024-04-25T08:49:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c200120 con 0x555a9c4cd9d0 2024-04-25T08:49:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1ff550 con 0x555a9c4ce2f0 2024-04-25T08:49:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c52a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c52a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54788/0 (socket says 172.21.15.46:54788) 2024-04-25T08:49:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541a00640 1 -- 172.21.15.46:0/4052728592 learned_addr learned my addr 172.21.15.46:0/4052728592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2542400640 1 --2- 172.21.15.46:0/4052728592 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 0x555a9c52ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541000640 1 --2- 172.21.15.46:0/4052728592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4cd9d0 0x555a9c52a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541a00640 1 -- 172.21.15.46:0/4052728592 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 msgr2=0x555a9c52ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541a00640 1 --2- 172.21.15.46:0/4052728592 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 0x555a9c52ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541a00640 1 -- 172.21.15.46:0/4052728592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4cd9d0 msgr2=0x555a9c52a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541a00640 1 --2- 172.21.15.46:0/4052728592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4cd9d0 0x555a9c52a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541a00640 1 -- 172.21.15.46:0/4052728592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a9c317de0 con 0x555a9c4840a0 2024-04-25T08:49:17.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2541a00640 1 --2- 172.21.15.46:0/4052728592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c52a0f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2530009830 tx=0x7f253002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa4b7a8dd9aeb9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f252d600640 1 -- 172.21.15.46:0/4052728592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530047070 con 0x555a9c4840a0 2024-04-25T08:49:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f252d600640 1 -- 172.21.15.46:0/4052728592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253001f040 con 0x555a9c4840a0 2024-04-25T08:49:17.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f252d600640 1 -- 172.21.15.46:0/4052728592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530033070 con 0x555a9c4840a0 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- 172.21.15.46:0/4052728592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 msgr2=0x555a9c52a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- 172.21.15.46:0/4052728592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c52a0f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2530009830 tx=0x7f253002d000 comp rx=0 tx=0).stop 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- 172.21.15.46:0/4052728592 shutdown_connections 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- 172.21.15.46:0/4052728592 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4ce2f0 0x555a9c52ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- 172.21.15.46:0/4052728592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4cd9d0 0x555a9c52a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- 172.21.15.46:0/4052728592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c52a0f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- 172.21.15.46:0/4052728592 >> 172.21.15.46:0/4052728592 conn(0x555a9c480020 msgr2=0x555a9c4cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- 172.21.15.46:0/4052728592 shutdown_connections 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- 172.21.15.46:0/4052728592 wait complete. 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- start start 2024-04-25T08:49:17.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 Processor -- start 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4e0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 0x555a9c4e08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4e0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a9c26be00 con 0x555a9c4840a0 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1ffb50 con 0x555a9c4ce2f0 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a9c2d0300 con 0x555a9c4cd9d0 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.320+0000 7f2547863000 10 client.0 _ll_register_callbacks cb 0x555a9c44f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4e0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54798/0 (socket says 172.21.15.46:54798) 2024-04-25T08:49:17.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541a00640 1 -- 172.21.15.46:0/510985380 learned_addr learned my addr 172.21.15.46:0/510985380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541000640 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 0x555a9c4e08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2542400640 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4e0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541a00640 1 -- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 msgr2=0x555a9c4e08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541a00640 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 0x555a9c4e08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541a00640 1 -- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 msgr2=0x555a9c4e0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541a00640 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4e0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541a00640 1 -- 172.21.15.46:0/510985380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a9c2728e0 con 0x555a9c4840a0 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541000640 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 0x555a9c4e08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2542400640 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4e0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541a00640 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4e0380 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f25300047c0 tx=0x7f2530009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38508e91803e679f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 1 -- 172.21.15.46:0/510985380 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530047070 con 0x555a9c4840a0 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 1 -- 172.21.15.46:0/510985380 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f253004ecf0 con 0x555a9c4840a0 2024-04-25T08:49:17.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 -- 172.21.15.46:0/510985380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f253001f040 con 0x555a9c4840a0 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 1 -- 172.21.15.46:0/510985380 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2530055050 con 0x555a9c4840a0 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 10 client.7866 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 -- 172.21.15.46:0/510985380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x555a9c32a8f0 con 0x555a9c4840a0 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 1 -- 172.21.15.46:0/510985380 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253001f040 con 0x555a9c4840a0 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 1 -- 172.21.15.46:0/510985380 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2530058020 con 0x555a9c4840a0 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 10 client.7866 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 10 client.7866 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 20 client.7866 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555a9c4e17f0 0x555a9c535f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 4 client.7866 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 -- 172.21.15.46:0/510985380 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555a9c2de530 con 0x555a9c4e17f0 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541000640 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555a9c4e17f0 0x555a9c535f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 10 client.7866 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:17.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2541000640 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555a9c4e17f0 0x555a9c535f60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41abed8f7644224b server_cookie=bfe9bb33945e8b14 in_seq=0 out_seq=0 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 1 -- 172.21.15.46:0/510985380 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x555a9c2de530 con 0x555a9c4e17f0 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 10 client.7866 handle_command_reply: tid=0 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 client.7866 shutdown 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 -- 172.21.15.46:0/510985380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 msgr2=0x555a9c4e0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4e0380 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f25300047c0 tx=0x7f2530009f90 comp rx=0 tx=0).stop 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 -- 172.21.15.46:0/510985380 shutdown_connections 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x555a9c4e17f0 0x555a9c535f60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555a9c4ce2f0 0x555a9c4e0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555a9c4cd9d0 0x555a9c4e08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 --2- 172.21.15.46:0/510985380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a9c4840a0 0x555a9c4e0380 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 -- 172.21.15.46:0/510985380 >> 172.21.15.46:0/510985380 conn(0x555a9c480020 msgr2=0x555a9c4cc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2522a00640 0 client.7866 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 -- 172.21.15.46:0/510985380 shutdown_connections 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 1 -- 172.21.15.46:0/510985380 wait complete. 2024-04-25T08:49:17.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.324+0000 7f2547863000 20 client.7866 trim_cache size 0 max 16384 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "cc042e3e-b984-4bbc-a564-8387eeb83b47": { 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir", 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "cc042e3e-b984-4bbc-a564-8387eeb83b47", 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:49:17.324 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:49:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 -- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 msgr2=0x7f251c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2530009ad0 tx=0x7f2530008040 comp rx=0 tx=0).stop 2024-04-25T08:49:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 -- 172.21.15.46:0/4235078605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c142560 msgr2=0x7f253c1f7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c142560 0x7f253c1f7000 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f25340b0040 tx=0x7f2534009750 comp rx=0 tx=0).stop 2024-04-25T08:49:17.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 -- 172.21.15.46:0/4235078605 shutdown_connections 2024-04-25T08:49:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f251c066f60 0x7f251c069420 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c142560 0x7f253c1f7000 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f253c13bde0 0x7f253c142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 --2- 172.21.15.46:0/4235078605 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f253c13ddd0 0x7f253c13b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 -- 172.21.15.46:0/4235078605 >> 172.21.15.46:0/4235078605 conn(0x7f253c06ac10 msgr2=0x7f253c15cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 -- 172.21.15.46:0/4235078605 shutdown_connections 2024-04-25T08:49:17.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:17.328+0000 7f2544400640 1 -- 172.21.15.46:0/4235078605 wait complete. 2024-04-25T08:49:17.339 INFO:tasks.cephfs.filesystem:scrub status for tag:cc042e3e-b984-4bbc-a564-8387eeb83b47 - {'path': '/dir', 'tag': 'cc042e3e-b984-4bbc-a564-8387eeb83b47', 'options': 'recursive,repair'} 2024-04-25T08:49:20.888 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:49:20.891 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:49:47.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:47.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:49:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24cda00640 1 Processor -- start 2024-04-25T08:49:47.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24cda00640 1 -- start start 2024-04-25T08:49:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24cda00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815d090 0x7f24c815d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24cda00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c80cd930 con 0x7f24c806dc70 2024-04-25T08:49:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24cda00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c80cdbd0 con 0x7f24c8145310 2024-04-25T08:49:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24cda00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80cdeb0 con 0x7f24c815d090 2024-04-25T08:49:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24be000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f24c806dc70 0x7f24c80cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49270/0 (socket says 172.21.15.46:49270) 2024-04-25T08:49:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24be000640 1 -- 172.21.15.46:0/289247881 learned_addr learned my addr 172.21.15.46:0/289247881 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:47.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c7400640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815d090 0x7f24c815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049755412 0 0) 0x7f24c80cd930 con 0x7f24c806dc70 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24ac003530 con 0x7f24c806dc70 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3199758194 0 0) 0x7f24ac003530 con 0x7f24c806dc70 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815d090 msgr2=0x7f24c815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815d090 0x7f24c815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 >> v1:172.21.15.82:6789/0 conn(0x7f24c8145310 legacy=0x7f24c806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c80ce190 con 0x7f24c806dc70 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f24b0001a00 con 0x7f24c806dc70 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 >> v1:172.21.15.46:6789/0 conn(0x7f24c806dc70 legacy=0x7f24c80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24ac003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815d090 0x7f24ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac004d50 0x7f24ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80cdeb0 con 0x7f24ac003a60 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac00ba80 con 0x7f24ac004d50 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac00bc30 con 0x7f24c815d090 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c6a00640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24ac003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24c7400640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac004d50 0x7f24ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.503+0000 7f24be000640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815d090 0x7f24ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c6a00640 1 -- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815d090 msgr2=0x7f24ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c6a00640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815d090 0x7f24ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c6a00640 1 -- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac004d50 msgr2=0x7f24ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c6a00640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24ac004d50 0x7f24ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c6a00640 1 -- 172.21.15.46:0/289247881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac003530 con 0x7f24ac003a60 2024-04-25T08:49:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c6a00640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24ac003e40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f24b8001910 tx=0x7f24b801c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b0ea3d8d02ef3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8001510 con 0x7f24ac003a60 2024-04-25T08:49:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b80b0070 con 0x7f24ac003a60 2024-04-25T08:49:47.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c6000640 1 -- 172.21.15.46:0/289247881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80a7070 con 0x7f24ac003a60 2024-04-25T08:49:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24cda00640 1 -- 172.21.15.46:0/289247881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 msgr2=0x7f24ac003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24cda00640 1 --2- 172.21.15.46:0/289247881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24ac003e40 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f24b8001910 tx=0x7f24b801c550 comp rx=0 tx=0).stop 2024-04-25T08:49:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24c7400640 1 -- 172.21.15.46:0/289247881 reap_dead start 2024-04-25T08:49:47.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24cda00640 1 -- 172.21.15.46:0/289247881 shutdown_connections 2024-04-25T08:49:47.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.507+0000 7f24cda00640 1 -- 172.21.15.46:0/289247881 >> 172.21.15.46:0/289247881 conn(0x7f24c806ac10 msgr2=0x7f24c806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:47.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 -- 172.21.15.46:0/289247881 shutdown_connections 2024-04-25T08:49:47.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 -- 172.21.15.46:0/289247881 wait complete. 2024-04-25T08:49:47.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 Processor -- start 2024-04-25T08:49:47.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 -- start start 2024-04-25T08:49:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24c81f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24c80b3ee0 0x7f24c81592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815e440 0x7f24c815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8167510 con 0x7f24ac003a60 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c815d8b0 con 0x7f24c80b3ee0 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24cda00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8167690 con 0x7f24c815e440 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24c81f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24c81f7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35058/0 (socket says 172.21.15.46:35058) 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c6a00640 1 -- 172.21.15.46:0/1682776135 learned_addr learned my addr 172.21.15.46:0/1682776135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24be000640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24c80b3ee0 0x7f24c81592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c7400640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815e440 0x7f24c815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c6a00640 1 -- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815e440 msgr2=0x7f24c815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c6a00640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815e440 0x7f24c815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c6a00640 1 -- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24c80b3ee0 msgr2=0x7f24c81592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c6a00640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24c80b3ee0 0x7f24c81592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c6a00640 1 -- 172.21.15.46:0/1682776135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c81f7b50 con 0x7f24ac003a60 2024-04-25T08:49:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c7400640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815e440 0x7f24c815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.511+0000 7f24c6a00640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24c81f7610 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f24b8000c10 tx=0x7f24ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24bf400640 1 -- 172.21.15.46:0/1682776135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8005380 con 0x7f24ac003a60 2024-04-25T08:49:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24bf400640 1 -- 172.21.15.46:0/1682776135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b80ab8b0 con 0x7f24ac003a60 2024-04-25T08:49:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24cda00640 1 -- 172.21.15.46:0/1682776135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c80ac880 con 0x7f24ac003a60 2024-04-25T08:49:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24bf400640 1 -- 172.21.15.46:0/1682776135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80abc60 con 0x7f24ac003a60 2024-04-25T08:49:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24cda00640 1 -- 172.21.15.46:0/1682776135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c80ace40 con 0x7f24ac003a60 2024-04-25T08:49:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24bf400640 1 -- 172.21.15.46:0/1682776135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f24b80067e0 con 0x7f24ac003a60 2024-04-25T08:49:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24bf400640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f24a4066f60 0x7f24a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24bf400640 1 -- 172.21.15.46:0/1682776135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f24b811fdb0 con 0x7f24ac003a60 2024-04-25T08:49:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24be000640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f24a4066f60 0x7f24a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24d0f02000 1 Processor -- start 2024-04-25T08:49:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.515+0000 7f24d0f02000 1 -- start start 2024-04-25T08:49:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24be000640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f24a4066f60 0x7f24a4069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f24b00013e0 tx=0x7f24b0006040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93384f9d0 0x55c93384fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338502f0 0x55c933805b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c9338060a0 0x55c933806480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c933652ff0 con 0x55c93384f9d0 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9335db150 con 0x55c9338502f0 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c93364f090 con 0x55c9338060a0 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93384f9d0 0x55c93384fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93384f9d0 0x55c93384fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35072/0 (socket says 172.21.15.46:35072) 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24c6a00640 1 -- 172.21.15.46:0/3556916307 learned_addr learned my addr 172.21.15.46:0/3556916307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24c6a00640 1 -- 172.21.15.46:0/3556916307 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c9338060a0 msgr2=0x55c933806480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24c6a00640 1 --2- 172.21.15.46:0/3556916307 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c9338060a0 0x55c933806480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24c6a00640 1 -- 172.21.15.46:0/3556916307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338502f0 msgr2=0x55c933805b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24c6a00640 1 --2- 172.21.15.46:0/3556916307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338502f0 0x55c933805b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24c6a00640 1 -- 172.21.15.46:0/3556916307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9337e90d0 con 0x55c93384f9d0 2024-04-25T08:49:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24c6a00640 1 --2- 172.21.15.46:0/3556916307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93384f9d0 0x55c93384fdb0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f24b80f86d0 tx=0x7f24b801c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7301857185bb6585 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24b7400640 1 -- 172.21.15.46:0/3556916307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80f8c00 con 0x55c93384f9d0 2024-04-25T08:49:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24b7400640 1 -- 172.21.15.46:0/3556916307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8108070 con 0x55c93384f9d0 2024-04-25T08:49:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24b7400640 1 -- 172.21.15.46:0/3556916307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80ee590 con 0x55c93384f9d0 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 -- 172.21.15.46:0/3556916307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93384f9d0 msgr2=0x55c93384fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 --2- 172.21.15.46:0/3556916307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93384f9d0 0x55c93384fdb0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f24b80f86d0 tx=0x7f24b801c550 comp rx=0 tx=0).stop 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 -- 172.21.15.46:0/3556916307 shutdown_connections 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 --2- 172.21.15.46:0/3556916307 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c9338060a0 0x55c933806480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 --2- 172.21.15.46:0/3556916307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338502f0 0x55c933805b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 --2- 172.21.15.46:0/3556916307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c93384f9d0 0x55c93384fdb0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 -- 172.21.15.46:0/3556916307 >> 172.21.15.46:0/3556916307 conn(0x55c933802020 msgr2=0x55c933804440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 -- 172.21.15.46:0/3556916307 shutdown_connections 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 -- 172.21.15.46:0/3556916307 wait complete. 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 -- start start 2024-04-25T08:49:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.519+0000 7f24d0f02000 1 Processor -- start 2024-04-25T08:49:47.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 0x55c9338347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 0x55c933834d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933835270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c933617250 con 0x55c9338502f0 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c93360d690 con 0x55c9338060a0 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9335dba40 con 0x55c93384f9d0 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 10 client.0 _ll_register_callbacks cb 0x55c9337f0e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933835270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 0x55c9338347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933835270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35074/0 (socket says 172.21.15.46:35074) 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c7400640 1 -- 172.21.15.46:0/1942800016 learned_addr learned my addr 172.21.15.46:0/1942800016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24be000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 0x55c933834d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c7400640 1 -- 172.21.15.46:0/1942800016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 msgr2=0x55c933834d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c7400640 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 0x55c933834d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c7400640 1 -- 172.21.15.46:0/1942800016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 msgr2=0x55c9338347f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c7400640 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 0x55c9338347f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c7400640 1 -- 172.21.15.46:0/1942800016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9337001c0 con 0x55c9338502f0 2024-04-25T08:49:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c7400640 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933835270 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f24c80710e0 tx=0x7f24c0014a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e42982b535852b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 1 -- 172.21.15.46:0/1942800016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0004020 con 0x55c9338502f0 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 1 -- 172.21.15.46:0/1942800016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f24c00041c0 con 0x55c9338502f0 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 -- 172.21.15.46:0/1942800016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24b8108070 con 0x55c9338502f0 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 1 -- 172.21.15.46:0/1942800016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c00043c0 con 0x55c9338502f0 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 10 client.7878 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 -- 172.21.15.46:0/1942800016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c9336fc170 con 0x55c9338502f0 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 1 -- 172.21.15.46:0/1942800016 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8108070 con 0x55c9338502f0 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 1 -- 172.21.15.46:0/1942800016 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24c0016700 con 0x55c9338502f0 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 10 client.7878 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 10 client.7878 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 20 client.7878 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c93385fc80 0x55c933862070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 4 client.7878 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 -- 172.21.15.46:0/1942800016 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9335eec60 con 0x55c93385fc80 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c6a00640 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c93385fc80 0x55c933862070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24c6a00640 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c93385fc80 0x55c933862070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7359bbc343070f94 server_cookie=657cd08034c77558 in_seq=0 out_seq=0 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 10 client.7878 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 1 -- 172.21.15.46:0/1942800016 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9335eec60 con 0x55c93385fc80 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 10 client.7878 handle_command_reply: tid=0 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 client.7878 shutdown 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 -- 172.21.15.46:0/1942800016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 msgr2=0x55c933835270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933835270 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f24c80710e0 tx=0x7f24c0014a10 comp rx=0 tx=0).stop 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 -- 172.21.15.46:0/1942800016 shutdown_connections 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c93385fc80 0x55c933862070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933835270 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 0x55c933834d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1942800016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 0x55c9338347f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24abe00640 0 client.7878 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:47.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.523+0000 7f24d0f02000 1 -- 172.21.15.46:0/1942800016 >> 172.21.15.46:0/1942800016 conn(0x55c933802020 msgr2=0x55c933804ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:47.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.527+0000 7f24d0f02000 1 -- 172.21.15.46:0/1942800016 shutdown_connections 2024-04-25T08:49:47.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.527+0000 7f24d0f02000 1 -- 172.21.15.46:0/1942800016 wait complete. 2024-04-25T08:49:47.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.527+0000 7f24d0f02000 20 client.7878 trim_cache size 0 max 16384 2024-04-25T08:49:47.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.539+0000 7f24d0f02000 1 Processor -- start 2024-04-25T08:49:47.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.539+0000 7f24d0f02000 1 -- start start 2024-04-25T08:49:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.539+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c9338060a0 0x55c9338ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.539+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c93384f9d0 0x55c9338ac630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c9338acb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8204170 con 0x55c9338502f0 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8203350 con 0x55c93384f9d0 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8202780 con 0x55c9338060a0 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c9338acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c9338acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35090/0 (socket says 172.21.15.46:35090) 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 -- 172.21.15.46:0/1634145849 learned_addr learned my addr 172.21.15.46:0/1634145849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c6a00640 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c9338060a0 0x55c9338ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24be000640 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c93384f9d0 0x55c9338ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 -- 172.21.15.46:0/1634145849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c9338060a0 msgr2=0x55c9338ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c9338060a0 0x55c9338ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 -- 172.21.15.46:0/1634145849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c93384f9d0 msgr2=0x55c9338ac630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c93384f9d0 0x55c9338ac630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 -- 172.21.15.46:0/1634145849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c933699de0 con 0x55c9338502f0 2024-04-25T08:49:47.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24be000640 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c93384f9d0 0x55c9338ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c9338acb70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f24c0014a50 tx=0x7f24c0014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1247af25f78647e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24b6a00640 1 -- 172.21.15.46:0/1634145849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0009d20 con 0x55c9338502f0 2024-04-25T08:49:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24b6a00640 1 -- 172.21.15.46:0/1634145849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c002f070 con 0x55c9338502f0 2024-04-25T08:49:47.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24b6a00640 1 -- 172.21.15.46:0/1634145849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c002a680 con 0x55c9338502f0 2024-04-25T08:49:47.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- 172.21.15.46:0/1634145849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 msgr2=0x55c9338acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c9338acb70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f24c0014a50 tx=0x7f24c0014f90 comp rx=0 tx=0).stop 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- 172.21.15.46:0/1634145849 shutdown_connections 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c9338acb70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c93384f9d0 0x55c9338ac630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 --2- 172.21.15.46:0/1634145849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c9338060a0 0x55c9338ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- 172.21.15.46:0/1634145849 >> 172.21.15.46:0/1634145849 conn(0x55c933802020 msgr2=0x55c93385cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- 172.21.15.46:0/1634145849 shutdown_connections 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- 172.21.15.46:0/1634145849 wait complete. 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- start start 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 Processor -- start 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 0x55c933862380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 0x55c9338628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933862e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9335ede00 con 0x55c9338502f0 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c8202d80 con 0x55c9338060a0 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c933652300 con 0x55c93384f9d0 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24d0f02000 10 client.0 _ll_register_callbacks cb 0x55c9337d1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933862e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933862e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35106/0 (socket says 172.21.15.46:35106) 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c7400640 1 -- 172.21.15.46:0/2236716987 learned_addr learned my addr 172.21.15.46:0/2236716987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24be000640 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 0x55c9338628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.543+0000 7f24c6a00640 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 0x55c933862380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24c7400640 1 -- 172.21.15.46:0/2236716987 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 msgr2=0x55c9338628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24c7400640 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 0x55c9338628c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24c7400640 1 -- 172.21.15.46:0/2236716987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 msgr2=0x55c933862380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24c7400640 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 0x55c933862380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24c7400640 1 -- 172.21.15.46:0/2236716987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9335f48e0 con 0x55c9338502f0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24c7400640 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933862e00 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f24c002c040 tx=0x7f24c0009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c670ef832affeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 1 -- 172.21.15.46:0/2236716987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0039020 con 0x55c9338502f0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 1 -- 172.21.15.46:0/2236716987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f24c002ac40 con 0x55c9338502f0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 -- 172.21.15.46:0/2236716987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24c002f070 con 0x55c9338502f0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 1 -- 172.21.15.46:0/2236716987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24c002ae60 con 0x55c9338502f0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 10 client.7884 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 -- 172.21.15.46:0/2236716987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c9336ac8f0 con 0x55c9338502f0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 1 -- 172.21.15.46:0/2236716987 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c002f070 con 0x55c9338502f0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 1 -- 172.21.15.46:0/2236716987 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f24c00059a0 con 0x55c9338502f0 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 10 client.7884 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 10 client.7884 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:47.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 20 client.7884 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:47.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c933863790 0x55c9338b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 4 client.7884 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:49:47.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 -- 172.21.15.46:0/2236716987 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c933660530 con 0x55c933863790 2024-04-25T08:49:47.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24c6a00640 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c933863790 0x55c9338b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 10 client.7884 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:47.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24c6a00640 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c933863790 0x55c9338b7f60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50ef621de4a0746 server_cookie=6cdd2a7ba4ad321c in_seq=0 out_seq=0 2024-04-25T08:49:47.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 1 -- 172.21.15.46:0/2236716987 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c933660530 con 0x55c933863790 2024-04-25T08:49:47.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 10 client.7884 handle_command_reply: tid=0 2024-04-25T08:49:47.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 client.7884 shutdown 2024-04-25T08:49:47.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 -- 172.21.15.46:0/2236716987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 msgr2=0x55c933862e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933862e00 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f24c002c040 tx=0x7f24c0009190 comp rx=0 tx=0).stop 2024-04-25T08:49:47.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 -- 172.21.15.46:0/2236716987 shutdown_connections 2024-04-25T08:49:47.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x55c933863790 0x55c9338b7f60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c9338502f0 0x55c933862e00 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c93384f9d0 0x55c9338628c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 --2- 172.21.15.46:0/2236716987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c9338060a0 0x55c933862380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24abe00640 0 client.7884 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 -- 172.21.15.46:0/2236716987 >> 172.21.15.46:0/2236716987 conn(0x55c933802020 msgr2=0x55c93384e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 -- 172.21.15.46:0/2236716987 shutdown_connections 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 1 -- 172.21.15.46:0/2236716987 wait complete. 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24d0f02000 20 client.7884 trim_cache size 0 max 16384 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:49:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:49:47.548 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:49:47.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 -- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f24a4066f60 msgr2=0x7f24a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f24a4066f60 0x7f24a4069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f24b00013e0 tx=0x7f24b0006040 comp rx=0 tx=0).stop 2024-04-25T08:49:47.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 -- 172.21.15.46:0/1682776135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 msgr2=0x7f24c81f7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24c81f7610 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f24b8000c10 tx=0x7f24ac005840 comp rx=0 tx=0).stop 2024-04-25T08:49:47.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 -- 172.21.15.46:0/1682776135 shutdown_connections 2024-04-25T08:49:47.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f24a4066f60 0x7f24a4069420 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f24c815e440 0x7f24c815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24c80b3ee0 0x7f24c81592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 --2- 172.21.15.46:0/1682776135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24ac003a60 0x7f24c81f7610 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 -- 172.21.15.46:0/1682776135 >> 172.21.15.46:0/1682776135 conn(0x7f24c806ac10 msgr2=0x7f24c80cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:47.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 -- 172.21.15.46:0/1682776135 shutdown_connections 2024-04-25T08:49:47.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.547+0000 7f24cda00640 1 -- 172.21.15.46:0/1682776135 wait complete. 2024-04-25T08:49:47.563 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:49:47.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:49:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fba600640 1 Processor -- start 2024-04-25T08:49:47.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fba600640 1 -- start start 2024-04-25T08:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fba600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb415ef20 0x7f5fb40cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fba600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5fb40cd6d0 0x7f5fb409a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fba600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb415f300 con 0x7f5fb409ab50 2024-04-25T08:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fba600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb40cddb0 con 0x7f5fb40cd6d0 2024-04-25T08:49:47.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fba600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40cc3d0 con 0x7f5fb415ef20 2024-04-25T08:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fabe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5fb40cd6d0 0x7f5fb409a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fb3400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb415ef20 0x7f5fb40cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fb3e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5fb409ab50 0x7f5fb4099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:42432/0 (socket says 172.21.15.46:42432) 2024-04-25T08:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fb3e00640 1 -- 172.21.15.46:0/1269763071 learned_addr learned my addr 172.21.15.46:0/1269763071 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:47.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fb3400640 1 -- 172.21.15.46:0/1269763071 >> v2:172.21.15.82:3300/0 conn(0x7f5fb40cd6d0 msgr2=0x7f5fb409a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fb3400640 1 --2- 172.21.15.46:0/1269763071 >> v2:172.21.15.82:3300/0 conn(0x7f5fb40cd6d0 0x7f5fb409a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fb3400640 1 -- 172.21.15.46:0/1269763071 >> v1:172.21.15.82:6789/0 conn(0x7f5fb409ab50 legacy=0x7f5fb4099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fb2a00640 1 -- 172.21.15.46:0/1269763071 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011704916 0 0) 0x7f5fb415f300 con 0x7f5fb409ab50 2024-04-25T08:49:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.711+0000 7f5fb3400640 1 -- 172.21.15.46:0/1269763071 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb40cc6b0 con 0x7f5fb415ef20 2024-04-25T08:49:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fb3400640 1 --2- 172.21.15.46:0/1269763071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb415ef20 0x7f5fb40cd190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097c0 tx=0x7f5fa401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14b3a0ebe773863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fb2a00640 1 -- 172.21.15.46:0/1269763071 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40a7070 con 0x7f5fb415ef20 2024-04-25T08:49:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fb2a00640 1 -- 172.21.15.46:0/1269763071 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa40b0070 con 0x7f5fb415ef20 2024-04-25T08:49:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fb2a00640 1 -- 172.21.15.46:0/1269763071 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40ab890 con 0x7f5fb415ef20 2024-04-25T08:49:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fba600640 1 -- 172.21.15.46:0/1269763071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb415ef20 msgr2=0x7f5fb40cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fba600640 1 --2- 172.21.15.46:0/1269763071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb415ef20 0x7f5fb40cd190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40097c0 tx=0x7f5fa401c470 comp rx=0 tx=0).stop 2024-04-25T08:49:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fba600640 1 -- 172.21.15.46:0/1269763071 shutdown_connections 2024-04-25T08:49:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fba600640 1 --2- 172.21.15.46:0/1269763071 >> v2:172.21.15.82:3300/0 conn(0x7f5fb40cd6d0 0x7f5fb409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fba600640 1 --2- 172.21.15.46:0/1269763071 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb415ef20 0x7f5fb40cd190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fba600640 1 -- 172.21.15.46:0/1269763071 >> 172.21.15.46:0/1269763071 conn(0x7f5fb40b3120 msgr2=0x7f5fb40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fba600640 1 -- 172.21.15.46:0/1269763071 shutdown_connections 2024-04-25T08:49:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.715+0000 7f5fba600640 1 -- 172.21.15.46:0/1269763071 wait complete. 2024-04-25T08:49:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 Processor -- start 2024-04-25T08:49:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 -- start start 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb409ab50 0x7f5fb40caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb415ef20 0x7f5fb41f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb414faf0 0x7f5fb40c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40c5b10 con 0x7f5fb414faf0 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40c5990 con 0x7f5fb415ef20 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb409a110 con 0x7f5fb409ab50 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb414faf0 0x7f5fb40c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb414faf0 0x7f5fb40c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35120/0 (socket says 172.21.15.46:35120) 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3e00640 1 -- 172.21.15.46:0/4290408134 learned_addr learned my addr 172.21.15.46:0/4290408134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3400640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb409ab50 0x7f5fb40caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fabe00640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb415ef20 0x7f5fb41f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3e00640 1 -- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb409ab50 msgr2=0x7f5fb40caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3e00640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb409ab50 0x7f5fb40caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3e00640 1 -- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb415ef20 msgr2=0x7f5fb41f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3e00640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb415ef20 0x7f5fb41f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3e00640 1 -- 172.21.15.46:0/4290408134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb4150290 con 0x7f5fb414faf0 2024-04-25T08:49:47.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fb3e00640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb414faf0 0x7f5fb40c8770 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00c810 tx=0x7f5fac00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fab400640 1 -- 172.21.15.46:0/4290408134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac018070 con 0x7f5fb414faf0 2024-04-25T08:49:47.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fab400640 1 -- 172.21.15.46:0/4290408134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac00a640 con 0x7f5fb414faf0 2024-04-25T08:49:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fab400640 1 -- 172.21.15.46:0/4290408134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00aa30 con 0x7f5fb414faf0 2024-04-25T08:49:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb4150570 con 0x7f5fb414faf0 2024-04-25T08:49:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.719+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb406ca10 con 0x7f5fb414faf0 2024-04-25T08:49:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.723+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb4069080 con 0x7f5fb414faf0 2024-04-25T08:49:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.723+0000 7f5fab400640 1 -- 172.21.15.46:0/4290408134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fac01b070 con 0x7f5fb414faf0 2024-04-25T08:49:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.723+0000 7f5fab400640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f90066f90 0x7f5f90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.723+0000 7f5fab400640 1 -- 172.21.15.46:0/4290408134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5fac084960 con 0x7f5fb414faf0 2024-04-25T08:49:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.723+0000 7f5fb3400640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f90066f90 0x7f5f90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.723+0000 7f5fb3400640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f90066f90 0x7f5f90069450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40af000 tx=0x7f5fa40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.727+0000 7f5fab400640 1 -- 172.21.15.46:0/4290408134 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5fac055680 con 0x7f5fb414faf0 2024-04-25T08:49:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fb40d0700 con 0x7f5fb414faf0 2024-04-25T08:49:47.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fab400640 1 -- 172.21.15.46:0/4290408134 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f5fac054f80 con 0x7f5fb414faf0 2024-04-25T08:49:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:47.918 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:48:49.616083+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7656},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7656":{"gid":7656,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6835/2777781817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2777781817},{"type":"v1","addr":"172.21.15.46:6835","nonce":2777781817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:49:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2024-04-25T08:49:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f90066f90 msgr2=0x7f5f90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f90066f90 0x7f5f90069450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5fa40af000 tx=0x7f5fa40a7000 comp rx=0 tx=0).stop 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb414faf0 msgr2=0x7f5fb40c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb414faf0 0x7f5fb40c8770 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00c810 tx=0x7f5fac00ccd0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 shutdown_connections 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f90066f90 0x7f5f90069450 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb414faf0 0x7f5fb40c8770 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fb415ef20 0x7f5fb41f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 --2- 172.21.15.46:0/4290408134 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fb409ab50 0x7f5fb40caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 >> 172.21.15.46:0/4290408134 conn(0x7f5fb40b3120 msgr2=0x7f5fb406f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 shutdown_connections 2024-04-25T08:49:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:47.923+0000 7f5fba600640 1 -- 172.21.15.46:0/4290408134 wait complete. 2024-04-25T08:49:47.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:49:48.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67ce200640 1 Processor -- start 2024-04-25T08:49:48.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67ce200640 1 -- start start 2024-04-25T08:49:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67ce200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c813d0b0 0x7f67c815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67ce200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c815eab0 con 0x7f67c815e260 2024-04-25T08:49:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67ce200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c815e7d0 con 0x7f67c8071270 2024-04-25T08:49:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67ce200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c813d490 con 0x7f67c813d0b0 2024-04-25T08:49:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f67c815e260 0x7f67c80b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35580/0 (socket says 172.21.15.46:35580) 2024-04-25T08:49:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c7e00640 1 -- 172.21.15.46:0/3676051985 learned_addr learned my addr 172.21.15.46:0/3676051985 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c7400640 1 --2- 172.21.15.46:0/3676051985 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c813d0b0 0x7f67c815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c6a00640 1 -- 172.21.15.46:0/3676051985 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223229129 0 0) 0x7f67c815eab0 con 0x7f67c815e260 2024-04-25T08:49:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c6a00640 1 -- 172.21.15.46:0/3676051985 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ac003530 con 0x7f67c815e260 2024-04-25T08:49:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c6a00640 1 -- 172.21.15.46:0/3676051985 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831841460 0 0) 0x7f67c815e7d0 con 0x7f67c8071270 2024-04-25T08:49:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c6a00640 1 -- 172.21.15.46:0/3676051985 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c815eab0 con 0x7f67c8071270 2024-04-25T08:49:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c7400640 1 -- 172.21.15.46:0/3676051985 >> v1:172.21.15.82:6789/0 conn(0x7f67c8071270 legacy=0x7f67c815db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c7400640 1 -- 172.21.15.46:0/3676051985 >> v1:172.21.15.46:6789/0 conn(0x7f67c815e260 legacy=0x7f67c80b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.087+0000 7f67c7400640 1 -- 172.21.15.46:0/3676051985 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c815a6e0 con 0x7f67c813d0b0 2024-04-25T08:49:48.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67c7400640 1 --2- 172.21.15.46:0/3676051985 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c813d0b0 0x7f67c815b1c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f67b800aed0 tx=0x7f67c80b3640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16bb630975d8d637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67c6a00640 1 -- 172.21.15.46:0/3676051985 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80a5740 con 0x7f67c813d0b0 2024-04-25T08:49:48.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67c6a00640 1 -- 172.21.15.46:0/3676051985 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b80a58e0 con 0x7f67c813d0b0 2024-04-25T08:49:48.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67c6a00640 1 -- 172.21.15.46:0/3676051985 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80a5cb0 con 0x7f67c813d0b0 2024-04-25T08:49:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67ce200640 1 -- 172.21.15.46:0/3676051985 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c813d0b0 msgr2=0x7f67c815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67ce200640 1 --2- 172.21.15.46:0/3676051985 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c813d0b0 0x7f67c815b1c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f67b800aed0 tx=0x7f67c80b3640 comp rx=0 tx=0).stop 2024-04-25T08:49:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67ce200640 1 -- 172.21.15.46:0/3676051985 shutdown_connections 2024-04-25T08:49:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67ce200640 1 --2- 172.21.15.46:0/3676051985 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c813d0b0 0x7f67c815b1c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67ce200640 1 -- 172.21.15.46:0/3676051985 >> 172.21.15.46:0/3676051985 conn(0x7f67c806ac10 msgr2=0x7f67c806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67ce200640 1 -- 172.21.15.46:0/3676051985 shutdown_connections 2024-04-25T08:49:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.091+0000 7f67ce200640 1 -- 172.21.15.46:0/3676051985 wait complete. 2024-04-25T08:49:48.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 Processor -- start 2024-04-25T08:49:48.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 -- start start 2024-04-25T08:49:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c8071270 0x7f67c813bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c813d0b0 0x7f67c80af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c815e260 0x7f67c815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80cc2f0 con 0x7f67c813d0b0 2024-04-25T08:49:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80cc170 con 0x7f67c815e260 2024-04-25T08:49:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c80cc470 con 0x7f67c8071270 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c813d0b0 0x7f67c80af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c813d0b0 0x7f67c80af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45138/0 (socket says 172.21.15.46:45138) 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bea00640 1 -- 172.21.15.46:0/2941694040 learned_addr learned my addr 172.21.15.46:0/2941694040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67c7400640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c8071270 0x7f67c813bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67c7e00640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c815e260 0x7f67c815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bea00640 1 -- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c8071270 msgr2=0x7f67c813bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bea00640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c8071270 0x7f67c813bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bea00640 1 -- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c815e260 msgr2=0x7f67c815bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bea00640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c815e260 0x7f67c815bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bea00640 1 -- 172.21.15.46:0/2941694040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c815ca00 con 0x7f67c813d0b0 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67c7400640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c8071270 0x7f67c813bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bea00640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c813d0b0 0x7f67c80af2b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c500 tx=0x7f67b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bfe00640 1 -- 172.21.15.46:0/2941694040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0017070 con 0x7f67c813d0b0 2024-04-25T08:49:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bfe00640 1 -- 172.21.15.46:0/2941694040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b0005310 con 0x7f67c813d0b0 2024-04-25T08:49:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 -- 172.21.15.46:0/2941694040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c815cca0 con 0x7f67c813d0b0 2024-04-25T08:49:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67bfe00640 1 -- 172.21.15.46:0/2941694040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b00054c0 con 0x7f67c813d0b0 2024-04-25T08:49:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.095+0000 7f67ce200640 1 -- 172.21.15.46:0/2941694040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c815c4d0 con 0x7f67c813d0b0 2024-04-25T08:49:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.099+0000 7f67bfe00640 1 -- 172.21.15.46:0/2941694040 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67b001a070 con 0x7f67c813d0b0 2024-04-25T08:49:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.099+0000 7f67bfe00640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67a4066f60 0x7f67a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.099+0000 7f67bfe00640 1 -- 172.21.15.46:0/2941694040 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f67b00845b0 con 0x7f67c813d0b0 2024-04-25T08:49:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.099+0000 7f67c7400640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67a4066f60 0x7f67a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.099+0000 7f67d1722000 1 Processor -- start 2024-04-25T08:49:48.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.099+0000 7f67d1722000 1 -- start start 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67a4066f60 0x7f67a4069420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f67b80ae000 tx=0x7f67b8004ac0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 0x564b58cd86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58cd8c00 0x564b58c8e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58c8e8e0 0x564b58c8ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b58b75ba0 con 0x564b58c8e8e0 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b58adb090 con 0x564b58cd8c00 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b58a63160 con 0x564b58cd82e0 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 0x564b58cd86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 0x564b58cd86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:44396/0 (socket says 172.21.15.46:44396) 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 -- 172.21.15.46:0/1636604809 learned_addr learned my addr 172.21.15.46:0/1636604809 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58c8e8e0 0x564b58c8ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67bea00640 1 --2- 172.21.15.46:0/1636604809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58cd8c00 0x564b58c8e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 -- 172.21.15.46:0/1636604809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58cd8c00 msgr2=0x564b58c8e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 --2- 172.21.15.46:0/1636604809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58cd8c00 0x564b58c8e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 -- 172.21.15.46:0/1636604809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58c8e8e0 msgr2=0x564b58c8ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 --2- 172.21.15.46:0/1636604809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58c8e8e0 0x564b58c8ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 -- 172.21.15.46:0/1636604809 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b58b84c20 con 0x564b58cd82e0 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7e00640 1 --2- 172.21.15.46:0/1636604809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58c8e8e0 0x564b58c8ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67c7400640 1 --2- 172.21.15.46:0/1636604809 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 0x564b58cd86c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67b80af040 tx=0x7f67b8004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c828d845427d30bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67b7e00640 1 -- 172.21.15.46:0/1636604809 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80b6940 con 0x564b58cd82e0 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67b7e00640 1 -- 172.21.15.46:0/1636604809 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b80b6ae0 con 0x564b58cd82e0 2024-04-25T08:49:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67b7e00640 1 -- 172.21.15.46:0/1636604809 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80c1410 con 0x564b58cd82e0 2024-04-25T08:49:48.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 -- 172.21.15.46:0/1636604809 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 msgr2=0x564b58cd86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 --2- 172.21.15.46:0/1636604809 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 0x564b58cd86c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67b80af040 tx=0x7f67b8004a50 comp rx=0 tx=0).stop 2024-04-25T08:49:48.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 -- 172.21.15.46:0/1636604809 shutdown_connections 2024-04-25T08:49:48.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 --2- 172.21.15.46:0/1636604809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58c8e8e0 0x564b58c8ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 --2- 172.21.15.46:0/1636604809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58cd8c00 0x564b58c8e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 --2- 172.21.15.46:0/1636604809 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 0x564b58cd86c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 -- 172.21.15.46:0/1636604809 >> 172.21.15.46:0/1636604809 conn(0x564b58c8a890 msgr2=0x564b58c8cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 -- 172.21.15.46:0/1636604809 shutdown_connections 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 -- 172.21.15.46:0/1636604809 wait complete. 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 -- start start 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.103+0000 7f67d1722000 1 Processor -- start 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58cdf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58cdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 0x564b58cc6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b58ad7130 con 0x564b58cd82e0 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b58a9f130 con 0x564b58c8e8e0 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b58b99300 con 0x564b58cd8c00 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 10 client.0 _ll_register_callbacks cb 0x564b58c79bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58cdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58cdfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45150/0 (socket says 172.21.15.46:45150) 2024-04-25T08:49:48.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67bea00640 1 -- 172.21.15.46:0/450838251 learned_addr learned my addr 172.21.15.46:0/450838251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67c7400640 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58cdf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67c7e00640 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 0x564b58cc6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67bea00640 1 -- 172.21.15.46:0/450838251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 msgr2=0x564b58cc6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67bea00640 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 0x564b58cc6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67bea00640 1 -- 172.21.15.46:0/450838251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 msgr2=0x564b58cdf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67bea00640 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58cdf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67bea00640 1 -- 172.21.15.46:0/450838251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b58ab8060 con 0x564b58cd82e0 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67c7e00640 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 0x564b58cc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67bea00640 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58cdfb80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f67b0082880 tx=0x7f67b00546b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efddfdd76f146ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 1 -- 172.21.15.46:0/450838251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b006c020 con 0x564b58cd82e0 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 1 -- 172.21.15.46:0/450838251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f67b005d060 con 0x564b58cd82e0 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 -- 172.21.15.46:0/450838251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67b80b6ae0 con 0x564b58cd82e0 2024-04-25T08:49:48.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 1 -- 172.21.15.46:0/450838251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b005d2f0 con 0x564b58cd82e0 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 10 client.7899 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 -- 172.21.15.46:0/450838251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x564b58a7c7c0 con 0x564b58cd82e0 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 1 -- 172.21.15.46:0/450838251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b80b6ae0 con 0x564b58cd82e0 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 1 -- 172.21.15.46:0/450838251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67b0068ba0 con 0x564b58cd82e0 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 10 client.7899 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 10 client.7899 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 20 client.7899 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x564b58cc6e30 0x564b58cc7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 4 client.7899 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 -- 172.21.15.46:0/450838251 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b58a76cd0 con 0x564b58cc6e30 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67c7400640 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x564b58cc6e30 0x564b58cc7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67c7400640 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x564b58cc6e30 0x564b58cc7210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2634cd7090649e82 server_cookie=1259c3e0bd2f1684 in_seq=0 out_seq=0 2024-04-25T08:49:48.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 10 client.7899 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:48.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 1 -- 172.21.15.46:0/450838251 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b58a76cd0 con 0x564b58cc6e30 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 10 client.7899 handle_command_reply: tid=0 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 client.7899 shutdown 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 -- 172.21.15.46:0/450838251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 msgr2=0x564b58cdfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58cdfb80 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f67b0082880 tx=0x7f67b00546b0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 -- 172.21.15.46:0/450838251 shutdown_connections 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x564b58cc6e30 0x564b58cc7210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 0x564b58cc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58cdfb80 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 --2- 172.21.15.46:0/450838251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58cdf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67d1722000 1 -- 172.21.15.46:0/450838251 >> 172.21.15.46:0/450838251 conn(0x564b58c8a890 msgr2=0x564b58c8d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.107+0000 7f67b4c00640 0 client.7899 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:48.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.111+0000 7f67d1722000 1 -- 172.21.15.46:0/450838251 shutdown_connections 2024-04-25T08:49:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.111+0000 7f67d1722000 1 -- 172.21.15.46:0/450838251 wait complete. 2024-04-25T08:49:48.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.111+0000 7f67d1722000 20 client.7899 trim_cache size 0 max 16384 2024-04-25T08:49:48.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.119+0000 7f67d1722000 1 Processor -- start 2024-04-25T08:49:48.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- start start 2024-04-25T08:49:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58cc51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 0x564b58cc5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd8c00 0x564b58c8cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c82027c0 con 0x564b58cd8c00 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c813e330 con 0x564b58c8e8e0 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c8159d30 con 0x564b58cd82e0 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd8c00 0x564b58c8cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd8c00 0x564b58c8cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45160/0 (socket says 172.21.15.46:45160) 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7e00640 1 -- 172.21.15.46:0/2896197477 learned_addr learned my addr 172.21.15.46:0/2896197477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7e00640 1 -- 172.21.15.46:0/2896197477 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 msgr2=0x564b58cc5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7e00640 1 --2- 172.21.15.46:0/2896197477 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 0x564b58cc5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7400640 1 --2- 172.21.15.46:0/2896197477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58cc51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7e00640 1 -- 172.21.15.46:0/2896197477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 msgr2=0x564b58cc51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7e00640 1 --2- 172.21.15.46:0/2896197477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58cc51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7e00640 1 -- 172.21.15.46:0/2896197477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b58a96fc0 con 0x564b58cd8c00 2024-04-25T08:49:48.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7400640 1 --2- 172.21.15.46:0/2896197477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58cc51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:48.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67c7e00640 1 --2- 172.21.15.46:0/2896197477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd8c00 0x564b58c8cc80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007340 tx=0x7f67c0007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d677a38db74a5d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67b7400640 1 -- 172.21.15.46:0/2896197477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0014b30 con 0x564b58cd8c00 2024-04-25T08:49:48.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67b7400640 1 -- 172.21.15.46:0/2896197477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c0014cf0 con 0x564b58cd8c00 2024-04-25T08:49:48.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67b7400640 1 -- 172.21.15.46:0/2896197477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0014ea0 con 0x564b58cd8c00 2024-04-25T08:49:48.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- 172.21.15.46:0/2896197477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd8c00 msgr2=0x564b58c8cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 --2- 172.21.15.46:0/2896197477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd8c00 0x564b58c8cc80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f67c0007340 tx=0x7f67c0007cc0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- 172.21.15.46:0/2896197477 shutdown_connections 2024-04-25T08:49:48.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 --2- 172.21.15.46:0/2896197477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd8c00 0x564b58c8cc80 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 --2- 172.21.15.46:0/2896197477 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd82e0 0x564b58cc5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 --2- 172.21.15.46:0/2896197477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58cc51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- 172.21.15.46:0/2896197477 >> 172.21.15.46:0/2896197477 conn(0x564b58c8a890 msgr2=0x564b58ce5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- 172.21.15.46:0/2896197477 shutdown_connections 2024-04-25T08:49:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- 172.21.15.46:0/2896197477 wait complete. 2024-04-25T08:49:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 -- start start 2024-04-25T08:49:48.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.123+0000 7f67d1722000 1 Processor -- start 2024-04-25T08:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58d37d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58d38280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 0x564b58cdc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b58a75e70 con 0x564b58cd82e0 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b58a63a50 con 0x564b58c8e8e0 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b58ada3a0 con 0x564b58cd8c00 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 10 client.0 _ll_register_callbacks cb 0x564b58a87240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58d38280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58d38280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45176/0 (socket says 172.21.15.46:45176) 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67bea00640 1 -- 172.21.15.46:0/3929628679 learned_addr learned my addr 172.21.15.46:0/3929628679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67c7400640 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58d37d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67c7e00640 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 0x564b58cdc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67bea00640 1 -- 172.21.15.46:0/3929628679 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 msgr2=0x564b58cdc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67bea00640 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 0x564b58cdc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67bea00640 1 -- 172.21.15.46:0/3929628679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 msgr2=0x564b58d37d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67bea00640 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58d37d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67bea00640 1 -- 172.21.15.46:0/3929628679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b58b349e0 con 0x564b58cd82e0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67c7400640 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58d37d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67bea00640 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58d38280 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f67b0001330 tx=0x7f67b0082750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fee955c317b1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 1 -- 172.21.15.46:0/3929628679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b006c030 con 0x564b58cd82e0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 1 -- 172.21.15.46:0/3929628679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f67b00540e0 con 0x564b58cd82e0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- 172.21.15.46:0/3929628679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67c0014cf0 con 0x564b58cd82e0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 1 -- 172.21.15.46:0/3929628679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67b00547d0 con 0x564b58cd82e0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 10 client.7905 fetch_fsmap learned FSMap version 69 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- 172.21.15.46:0/3929628679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x564b58a7a630 con 0x564b58cd82e0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 1 -- 172.21.15.46:0/3929628679 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c0014cf0 con 0x564b58cd82e0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 1 -- 172.21.15.46:0/3929628679 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f67b00840c0 con 0x564b58cd82e0 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 10 client.7905 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 10 client.7905 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:49:48.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 20 client.7905 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x564b58cdca00 0x564b58cdcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 4 client.7905 mds_command: new command op to 7656 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- 172.21.15.46:0/3929628679 --> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x564b58ae85d0 con 0x564b58cdca00 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67c7400640 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x564b58cdca00 0x564b58cdcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67c7400640 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x564b58cdca00 0x564b58cdcde0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25cb545ef8f3aa87 server_cookie=6a8bfbbb6e496c3c in_seq=0 out_seq=0 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 10 client.7905 ms_handle_connect on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 1 -- 172.21.15.46:0/3929628679 <== mds.0 v2:172.21.15.46:6834/2777781817 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+144 (crc 0 0 0) 0x564b58ae85d0 con 0x564b58cdca00 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 10 client.7905 handle_command_reply: tid=0 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 client.7905 shutdown 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- 172.21.15.46:0/3929628679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 msgr2=0x564b58d38280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58d38280 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f67b0001330 tx=0x7f67b0082750 comp rx=0 tx=0).stop 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- 172.21.15.46:0/3929628679 shutdown_connections 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.46:6834/2777781817,v1:172.21.15.46:6835/2777781817] conn(0x564b58cdca00 0x564b58cdcde0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564b58cd8c00 0x564b58cdc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564b58cd82e0 0x564b58d38280 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 --2- 172.21.15.46:0/3929628679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b58c8e8e0 0x564b58d37d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67b4c00640 0 client.7905 ms_handle_reset on v2:172.21.15.46:6834/2777781817 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- 172.21.15.46:0/3929628679 >> 172.21.15.46:0/3929628679 conn(0x564b58c8a890 msgr2=0x564b58ce5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- 172.21.15.46:0/3929628679 shutdown_connections 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 1 -- 172.21.15.46:0/3929628679 wait complete. 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.127+0000 7f67d1722000 20 client.7905 trim_cache size 0 max 16384 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:49:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-04-25T08:49:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 1541640988, 2024-04-25T08:49:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-04-25T08:49:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/testdir/hardlink" 2024-04-25T08:49:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:49:48.129 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 -- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67a4066f60 msgr2=0x7f67a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67a4066f60 0x7f67a4069420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f67b80ae000 tx=0x7f67b8004ac0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 -- 172.21.15.46:0/2941694040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c813d0b0 msgr2=0x7f67c80af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c813d0b0 0x7f67c80af2b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f67b000c500 tx=0x7f67b000c9c0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 -- 172.21.15.46:0/2941694040 shutdown_connections 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f67a4066f60 0x7f67a4069420 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67c815e260 0x7f67c815bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67c813d0b0 0x7f67c80af2b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 --2- 172.21.15.46:0/2941694040 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f67c8071270 0x7f67c813bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 -- 172.21.15.46:0/2941694040 >> 172.21.15.46:0/2941694040 conn(0x7f67c806ac10 msgr2=0x7f67c80ada00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 -- 172.21.15.46:0/2941694040 shutdown_connections 2024-04-25T08:49:48.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.131+0000 7f67ce200640 1 -- 172.21.15.46:0/2941694040 wait complete. 2024-04-25T08:49:48.144 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:49:48.144 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:49:48.153 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:49:48.153 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:49:48.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:49:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.399+0000 7fd9e8400640 1 Processor -- start 2024-04-25T08:49:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.399+0000 7fd9e8400640 1 -- start start 2024-04-25T08:49:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e8400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9e015a750 0x7fd9e006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e8400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd9e00cc9e0 0x7fd9e006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e8400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e006d5e0 con 0x7fd9e00aed10 2024-04-25T08:49:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e8400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e015ab30 con 0x7fd9e015a750 2024-04-25T08:49:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e8400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e00ce4f0 con 0x7fd9e00cc9e0 2024-04-25T08:49:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9e015a750 0x7fd9e006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9e015a750 0x7fd9e006ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45192/0 (socket says 172.21.15.46:45192) 2024-04-25T08:49:48.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e5a00640 1 -- 172.21.15.46:0/4080429482 learned_addr learned my addr 172.21.15.46:0/4080429482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:48.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e6400640 1 --2- 172.21.15.46:0/4080429482 >> v2:172.21.15.82:3300/0 conn(0x7fd9e00cc9e0 0x7fd9e006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e5a00640 1 -- 172.21.15.46:0/4080429482 >> v2:172.21.15.82:3300/0 conn(0x7fd9e00cc9e0 msgr2=0x7fd9e006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e5a00640 1 --2- 172.21.15.46:0/4080429482 >> v2:172.21.15.82:3300/0 conn(0x7fd9e00cc9e0 0x7fd9e006f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e5a00640 1 -- 172.21.15.46:0/4080429482 >> v1:172.21.15.82:6789/0 conn(0x7fd9e00aed10 legacy=0x7fd9e00b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e5a00640 1 -- 172.21.15.46:0/4080429482 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00ce720 con 0x7fd9e015a750 2024-04-25T08:49:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.403+0000 7fd9e5a00640 1 --2- 172.21.15.46:0/4080429482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 0x7fd9e006ec10 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0050030 tx=0x7fd9d401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0fe48fc48de17d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9dfe00640 1 -- 172.21.15.46:0/4080429482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d40b4070 con 0x7fd9e015a750 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9dfe00640 1 -- 172.21.15.46:0/4080429482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d40a7bb0 con 0x7fd9e015a750 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9dfe00640 1 -- 172.21.15.46:0/4080429482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d40af440 con 0x7fd9e015a750 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 -- 172.21.15.46:0/4080429482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 msgr2=0x7fd9e006ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 --2- 172.21.15.46:0/4080429482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 0x7fd9e006ec10 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0050030 tx=0x7fd9d401ebf0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 -- 172.21.15.46:0/4080429482 shutdown_connections 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 --2- 172.21.15.46:0/4080429482 >> v2:172.21.15.82:3300/0 conn(0x7fd9e00cc9e0 0x7fd9e006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 --2- 172.21.15.46:0/4080429482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 0x7fd9e006ec10 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 -- 172.21.15.46:0/4080429482 >> 172.21.15.46:0/4080429482 conn(0x7fd9e00b12a0 msgr2=0x7fd9e00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 -- 172.21.15.46:0/4080429482 shutdown_connections 2024-04-25T08:49:48.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 -- 172.21.15.46:0/4080429482 wait complete. 2024-04-25T08:49:48.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 Processor -- start 2024-04-25T08:49:48.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 -- start start 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9e00aed10 0x7fd9e0139490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e00cc9e0 0x7fd9e015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 0x7fd9e00b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e006db00 con 0x7fd9e015a750 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e006d270 con 0x7fd9e00cc9e0 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e8400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e006dc80 con 0x7fd9e00aed10 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 0x7fd9e00b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 0x7fd9e00b0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45208/0 (socket says 172.21.15.46:45208) 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e6400640 1 -- 172.21.15.46:0/898295086 learned_addr learned my addr 172.21.15.46:0/898295086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e5a00640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9e00aed10 0x7fd9e0139490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.407+0000 7fd9e5000640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e00cc9e0 0x7fd9e015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e6400640 1 -- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9e00aed10 msgr2=0x7fd9e0139490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e6400640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9e00aed10 0x7fd9e0139490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e6400640 1 -- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e00cc9e0 msgr2=0x7fd9e015bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e6400640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e00cc9e0 0x7fd9e015bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e6400640 1 -- 172.21.15.46:0/898295086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e01399d0 con 0x7fd9e015a750 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e6400640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 0x7fd9e00b0270 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8007e40 tx=0x7fd9d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9dd600640 1 -- 172.21.15.46:0/898295086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8014620 con 0x7fd9e015a750 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9dd600640 1 -- 172.21.15.46:0/898295086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d8019070 con 0x7fd9e015a750 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e015c490 con 0x7fd9e015a750 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9dd600640 1 -- 172.21.15.46:0/898295086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8014a20 con 0x7fd9e015a750 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e015ca50 con 0x7fd9e015a750 2024-04-25T08:49:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e015cc90 con 0x7fd9e015a750 2024-04-25T08:49:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9dd600640 1 -- 172.21.15.46:0/898295086 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9d8014bc0 con 0x7fd9e015a750 2024-04-25T08:49:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9dd600640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd9c0066f60 0x7fd9c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9dd600640 1 -- 172.21.15.46:0/898295086 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd9d80859c0 con 0x7fd9e015a750 2024-04-25T08:49:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.411+0000 7fd9e5a00640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd9c0066f60 0x7fd9c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.415+0000 7fd9e5a00640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd9c0066f60 0x7fd9c0069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40b1000 tx=0x7fd9d40a9070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.415+0000 7fd9dd600640 1 -- 172.21.15.46:0/898295086 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9e015cc90 con 0x7fd9e015a750 2024-04-25T08:49:48.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.579+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9e0069080 con 0x7fd9e015a750 2024-04-25T08:49:48.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.579+0000 7fd9dd600640 1 -- 172.21.15.46:0/898295086 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17159 (secure 0 0 0) 0x7fd9d8056160 con 0x7fd9e015a750 2024-04-25T08:49:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:48.575 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:49:03.672984+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:48:26.132790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:48:26.714570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1269610178":"2024-04-26T08:48:38.594941+0000","172.21.15.46:6834/1269610178":"2024-04-26T08:48:38.594941+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:49:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd9c0066f60 msgr2=0x7fd9c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd9c0066f60 0x7fd9c0069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40b1000 tx=0x7fd9d40a9070 comp rx=0 tx=0).stop 2024-04-25T08:49:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 msgr2=0x7fd9e00b0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 0x7fd9e00b0270 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8007e40 tx=0x7fd9d800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 shutdown_connections 2024-04-25T08:49:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd9c0066f60 0x7fd9c0069420 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9e015a750 0x7fd9e00b0270 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e00cc9e0 0x7fd9e015bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 --2- 172.21.15.46:0/898295086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9e00aed10 0x7fd9e0139490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 >> 172.21.15.46:0/898295086 conn(0x7fd9e00b12a0 msgr2=0x7fd9e015b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 shutdown_connections 2024-04-25T08:49:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.583+0000 7fd9e8400640 1 -- 172.21.15.46:0/898295086 wait complete. 2024-04-25T08:49:48.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:48:26.132790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:48:26.714570+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:49:48.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:49:48.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f857b800640 1 Processor -- start 2024-04-25T08:49:48.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f857b800640 1 -- start start 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f857b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f857413d0b0 0x7f857415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f857b800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8574071270 0x7f85740716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f857b800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857415eab0 con 0x7f857415db50 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f857b800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8574071c10 con 0x7f8574071270 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f857b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857413d490 con 0x7f857413d0b0 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f856be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8574071270 0x7f85740716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f856be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8574071270 0x7f85740716d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45220/0 (socket says 172.21.15.46:45220) 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f8579800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f857415db50 0x7f857415dfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35594/0 (socket says 172.21.15.46:35594) 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f856be00640 1 -- 172.21.15.46:0/1576994024 learned_addr learned my addr 172.21.15.46:0/1576994024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f8578e00640 1 --2- 172.21.15.46:0/1576994024 >> v2:172.21.15.82:3300/0 conn(0x7f857413d0b0 0x7f857415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f856be00640 1 -- 172.21.15.46:0/1576994024 >> v2:172.21.15.82:3300/0 conn(0x7f857413d0b0 msgr2=0x7f857415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:48.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f856be00640 1 --2- 172.21.15.46:0/1576994024 >> v2:172.21.15.82:3300/0 conn(0x7f857413d0b0 0x7f857415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f856be00640 1 -- 172.21.15.46:0/1576994024 >> v1:172.21.15.46:6789/0 conn(0x7f857415db50 legacy=0x7f857415dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f856be00640 1 -- 172.21.15.46:0/1576994024 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857415e7d0 con 0x7f8574071270 2024-04-25T08:49:48.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.747+0000 7f8573e00640 1 -- 172.21.15.46:0/1576994024 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380304578 0 0) 0x7f857415eab0 con 0x7f857415db50 2024-04-25T08:49:48.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f856be00640 1 --2- 172.21.15.46:0/1576994024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8574071270 0x7f85740716d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8574072710 tx=0x7f855c01ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab5a99d4e4f0d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f8573e00640 1 -- 172.21.15.46:0/1576994024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c0b4070 con 0x7f8574071270 2024-04-25T08:49:48.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f8573e00640 1 -- 172.21.15.46:0/1576994024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855c0a7b30 con 0x7f8574071270 2024-04-25T08:49:48.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f8573e00640 1 -- 172.21.15.46:0/1576994024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c0af440 con 0x7f8574071270 2024-04-25T08:49:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f857b800640 1 -- 172.21.15.46:0/1576994024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8574071270 msgr2=0x7f85740716d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f857b800640 1 --2- 172.21.15.46:0/1576994024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8574071270 0x7f85740716d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f8574072710 tx=0x7f855c01ebf0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f857b800640 1 -- 172.21.15.46:0/1576994024 shutdown_connections 2024-04-25T08:49:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f857b800640 1 --2- 172.21.15.46:0/1576994024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8574071270 0x7f85740716d0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f857b800640 1 --2- 172.21.15.46:0/1576994024 >> v2:172.21.15.82:3300/0 conn(0x7f857413d0b0 0x7f857415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f857b800640 1 -- 172.21.15.46:0/1576994024 >> 172.21.15.46:0/1576994024 conn(0x7f857406ac10 msgr2=0x7f857406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f857b800640 1 -- 172.21.15.46:0/1576994024 shutdown_connections 2024-04-25T08:49:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.751+0000 7f857b800640 1 -- 172.21.15.46:0/1576994024 wait complete. 2024-04-25T08:49:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 Processor -- start 2024-04-25T08:49:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 -- start start 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8574071270 0x7f8574138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857413d0b0 0x7f8574139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857415db50 0x7f8574155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85740b28a0 con 0x7f857415db50 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85740b2720 con 0x7f857413d0b0 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85740b2a20 con 0x7f8574071270 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8579800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857415db50 0x7f8574155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8579800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857415db50 0x7f8574155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45222/0 (socket says 172.21.15.46:45222) 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8579800640 1 -- 172.21.15.46:0/103312737 learned_addr learned my addr 172.21.15.46:0/103312737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f856be00640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857413d0b0 0x7f8574139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8578e00640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8574071270 0x7f8574138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8579800640 1 -- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8574071270 msgr2=0x7f8574138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8579800640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8574071270 0x7f8574138860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8579800640 1 -- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857413d0b0 msgr2=0x7f8574139d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8579800640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857413d0b0 0x7f8574139d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8579800640 1 -- 172.21.15.46:0/103312737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857413a290 con 0x7f857415db50 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8578e00640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8574071270 0x7f8574138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8579800640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857415db50 0x7f8574155e90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c700 tx=0x7f856c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8571600640 1 -- 172.21.15.46:0/103312737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c018070 con 0x7f857415db50 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8571600640 1 -- 172.21.15.46:0/103312737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c00ce10 con 0x7f857415db50 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f8571600640 1 -- 172.21.15.46:0/103312737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c00a860 con 0x7f857415db50 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8574138fd0 con 0x7f857415db50 2024-04-25T08:49:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8574139550 con 0x7f857415db50 2024-04-25T08:49:48.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.755+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8574069010 con 0x7f857415db50 2024-04-25T08:49:48.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.759+0000 7f8571600640 1 -- 172.21.15.46:0/103312737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f856c00aa00 con 0x7f857415db50 2024-04-25T08:49:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.759+0000 7f8571600640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8550066f90 0x7f8550069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.759+0000 7f8571600640 1 -- 172.21.15.46:0/103312737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f856c0847b0 con 0x7f857415db50 2024-04-25T08:49:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.759+0000 7f8578e00640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8550066f90 0x7f8550069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:48.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.759+0000 7f8578e00640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8550066f90 0x7f8550069450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8564007a70 tx=0x7f8564008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:48.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.763+0000 7f8571600640 1 -- 172.21.15.46:0/103312737 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f857413a290 con 0x7f857415db50 2024-04-25T08:49:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.923+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85740ad950 con 0x7f857415db50 2024-04-25T08:49:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f8571600640 1 -- 172.21.15.46:0/103312737 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f856c054e80 con 0x7f857415db50 2024-04-25T08:49:48.921 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:49:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8550066f90 msgr2=0x7f8550069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8550066f90 0x7f8550069450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8564007a70 tx=0x7f8564008040 comp rx=0 tx=0).stop 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857415db50 msgr2=0x7f8574155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857415db50 0x7f8574155e90 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c700 tx=0x7f856c00cbc0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 shutdown_connections 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8550066f90 0x7f8550069450 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f857415db50 0x7f8574155e90 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f857413d0b0 0x7f8574139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 --2- 172.21.15.46:0/103312737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8574071270 0x7f8574138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 >> 172.21.15.46:0/103312737 conn(0x7f857406ac10 msgr2=0x7f857415df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 shutdown_connections 2024-04-25T08:49:48.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:48.927+0000 7f857b800640 1 -- 172.21.15.46:0/103312737 wait complete. 2024-04-25T08:49:48.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:49:49.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac73e00640 1 Processor -- start 2024-04-25T08:49:49.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac73e00640 1 -- start start 2024-04-25T08:49:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac73e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac6c15dae0 0x7fac6c072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac6c06e620 0x7fac6c06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac73e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6c0b3850 con 0x7fac6c06ea60 2024-04-25T08:49:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac73e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6c06e190 con 0x7fac6c06e620 2024-04-25T08:49:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac73e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6c0b3af0 con 0x7fac6c15dae0 2024-04-25T08:49:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac70a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fac6c06e620 0x7fac6c06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac71e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fac6c06ea60 0x7fac6c0acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35596/0 (socket says 172.21.15.46:35596) 2024-04-25T08:49:49.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac71e00640 1 -- 172.21.15.46:0/3872731399 learned_addr learned my addr 172.21.15.46:0/3872731399 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac71400640 1 --2- 172.21.15.46:0/3872731399 >> v2:172.21.15.82:3300/0 conn(0x7fac6c15dae0 0x7fac6c072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac70a00640 1 -- 172.21.15.46:0/3872731399 >> v2:172.21.15.82:3300/0 conn(0x7fac6c15dae0 msgr2=0x7fac6c072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac70a00640 1 --2- 172.21.15.46:0/3872731399 >> v2:172.21.15.82:3300/0 conn(0x7fac6c15dae0 0x7fac6c072600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac70a00640 1 -- 172.21.15.46:0/3872731399 >> v1:172.21.15.46:6789/0 conn(0x7fac6c06ea60 legacy=0x7fac6c0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac70a00640 1 -- 172.21.15.46:0/3872731399 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c0b3dd0 con 0x7fac6c06e620 2024-04-25T08:49:49.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.091+0000 7fac6be00640 1 -- 172.21.15.46:0/3872731399 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748454841 0 0) 0x7fac6c0b3850 con 0x7fac6c06ea60 2024-04-25T08:49:49.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac70a00640 1 --2- 172.21.15.46:0/3872731399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c06e620 0x7fac6c06bc20 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fac6c06e310 tx=0x7fac5801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a8ff6b5f8e2c5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:49.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac6be00640 1 -- 172.21.15.46:0/3872731399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580b0070 con 0x7fac6c06e620 2024-04-25T08:49:49.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac6be00640 1 -- 172.21.15.46:0/3872731399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac580ab600 con 0x7fac6c06e620 2024-04-25T08:49:49.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac6be00640 1 -- 172.21.15.46:0/3872731399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580ab9d0 con 0x7fac6c06e620 2024-04-25T08:49:49.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac73e00640 1 -- 172.21.15.46:0/3872731399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c06e620 msgr2=0x7fac6c06bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac73e00640 1 --2- 172.21.15.46:0/3872731399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c06e620 0x7fac6c06bc20 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fac6c06e310 tx=0x7fac5801c430 comp rx=0 tx=0).stop 2024-04-25T08:49:49.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac73e00640 1 -- 172.21.15.46:0/3872731399 shutdown_connections 2024-04-25T08:49:49.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac73e00640 1 --2- 172.21.15.46:0/3872731399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c06e620 0x7fac6c06bc20 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac73e00640 1 --2- 172.21.15.46:0/3872731399 >> v2:172.21.15.82:3300/0 conn(0x7fac6c15dae0 0x7fac6c072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac73e00640 1 -- 172.21.15.46:0/3872731399 >> 172.21.15.46:0/3872731399 conn(0x7fac6c071ae0 msgr2=0x7fac6c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:49.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac73e00640 1 -- 172.21.15.46:0/3872731399 shutdown_connections 2024-04-25T08:49:49.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.095+0000 7fac73e00640 1 -- 172.21.15.46:0/3872731399 wait complete. 2024-04-25T08:49:49.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 Processor -- start 2024-04-25T08:49:49.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 -- start start 2024-04-25T08:49:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac6c06e620 0x7fac6c1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6c06ea60 0x7fac6c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c15dae0 0x7fac6c14b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c15dec0 con 0x7fac6c15dae0 2024-04-25T08:49:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c06c570 con 0x7fac6c06ea60 2024-04-25T08:49:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c06f6d0 con 0x7fac6c06e620 2024-04-25T08:49:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c15dae0 0x7fac6c14b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c15dae0 0x7fac6c14b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45248/0 (socket says 172.21.15.46:45248) 2024-04-25T08:49:49.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71e00640 1 -- 172.21.15.46:0/3635875982 learned_addr learned my addr 172.21.15.46:0/3635875982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac70a00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6c06ea60 0x7fac6c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71e00640 1 -- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac6c06e620 msgr2=0x7fac6c1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71400640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac6c06e620 0x7fac6c1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71e00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac6c06e620 0x7fac6c1586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71e00640 1 -- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6c06ea60 msgr2=0x7fac6c155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71e00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6c06ea60 0x7fac6c155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71e00640 1 -- 172.21.15.46:0/3635875982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c14be00 con 0x7fac6c15dae0 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac70a00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6c06ea60 0x7fac6c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71400640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac6c06e620 0x7fac6c1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac71e00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c15dae0 0x7fac6c14b8c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c7b0 tx=0x7fac6400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac69600640 1 -- 172.21.15.46:0/3635875982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64018070 con 0x7fac6c15dae0 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac69600640 1 -- 172.21.15.46:0/3635875982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6400a640 con 0x7fac6c15dae0 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c158df0 con 0x7fac6c15dae0 2024-04-25T08:49:49.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac69600640 1 -- 172.21.15.46:0/3635875982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6400aa70 con 0x7fac6c15dae0 2024-04-25T08:49:49.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.099+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c1593b0 con 0x7fac6c15dae0 2024-04-25T08:49:49.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.103+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6c069010 con 0x7fac6c15dae0 2024-04-25T08:49:49.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.103+0000 7fac69600640 1 -- 172.21.15.46:0/3635875982 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac6401b070 con 0x7fac6c15dae0 2024-04-25T08:49:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.103+0000 7fac69600640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fac4c066f90 0x7fac4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.103+0000 7fac69600640 1 -- 172.21.15.46:0/3635875982 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fac640847f0 con 0x7fac6c15dae0 2024-04-25T08:49:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.103+0000 7fac71400640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fac4c066f90 0x7fac4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.103+0000 7fac71400640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fac4c066f90 0x7fac4c069450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fac6c1508a0 tx=0x7fac60008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:49.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.107+0000 7fac69600640 1 -- 172.21.15.46:0/3635875982 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac640555a0 con 0x7fac6c15dae0 2024-04-25T08:49:49.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fac6c15a3c0 con 0x7fac6c15dae0 2024-04-25T08:49:49.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac69600640 1 -- 172.21.15.46:0/3635875982 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fac64054ea0 con 0x7fac6c15dae0 2024-04-25T08:49:49.267 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:49:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fac4c066f90 msgr2=0x7fac4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fac4c066f90 0x7fac4c069450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fac6c1508a0 tx=0x7fac60008040 comp rx=0 tx=0).stop 2024-04-25T08:49:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c15dae0 msgr2=0x7fac6c14b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c15dae0 0x7fac6c14b8c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c7b0 tx=0x7fac6400cc70 comp rx=0 tx=0).stop 2024-04-25T08:49:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 shutdown_connections 2024-04-25T08:49:49.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fac4c066f90 0x7fac4c069450 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac6c15dae0 0x7fac6c14b8c0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6c06ea60 0x7fac6c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 --2- 172.21.15.46:0/3635875982 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fac6c06e620 0x7fac6c1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 >> 172.21.15.46:0/3635875982 conn(0x7fac6c071ae0 msgr2=0x7fac6c152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 shutdown_connections 2024-04-25T08:49:49.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.271+0000 7fac73e00640 1 -- 172.21.15.46:0/3635875982 wait complete. 2024-04-25T08:49:49.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:49:49.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.435+0000 7f8bf7600640 1 Processor -- start 2024-04-25T08:49:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.435+0000 7f8bf7600640 1 -- start start 2024-04-25T08:49:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf7600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf0143990 0x7f8bf00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf7600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8bf006f150 0x7f8bf00b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf7600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf015e440 con 0x7f8bf01440d0 2024-04-25T08:49:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf7600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf015ed30 con 0x7f8bf006f150 2024-04-25T08:49:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf7600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf015e6e0 con 0x7f8bf0143990 2024-04-25T08:49:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8be7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8bf01440d0 0x7f8bf006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35610/0 (socket says 172.21.15.46:35610) 2024-04-25T08:49:49.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8be7e00640 1 -- 172.21.15.46:0/3121551182 learned_addr learned my addr 172.21.15.46:0/3121551182 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf5600640 1 --2- 172.21.15.46:0/3121551182 >> v2:172.21.15.46:3300/0 conn(0x7f8bf006f150 0x7f8bf00b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf4c00640 1 --2- 172.21.15.46:0/3121551182 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf0143990 0x7f8bf00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8befe00640 1 -- 172.21.15.46:0/3121551182 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487135222 0 0) 0x7f8bf015e440 con 0x7f8bf01440d0 2024-04-25T08:49:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8befe00640 1 -- 172.21.15.46:0/3121551182 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bd4003530 con 0x7f8bf01440d0 2024-04-25T08:49:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf5600640 1 -- 172.21.15.46:0/3121551182 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf0143990 msgr2=0x7f8bf00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf5600640 1 --2- 172.21.15.46:0/3121551182 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf0143990 0x7f8bf00cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf5600640 1 -- 172.21.15.46:0/3121551182 >> v1:172.21.15.46:6789/0 conn(0x7f8bf01440d0 legacy=0x7f8bf006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.439+0000 7f8bf5600640 1 -- 172.21.15.46:0/3121551182 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf015e9c0 con 0x7f8bf006f150 2024-04-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf5600640 1 --2- 172.21.15.46:0/3121551182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf006f150 0x7f8bf00b2720 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009e10 tx=0x7f8be801ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbe30a88be3099d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8befe00640 1 -- 172.21.15.46:0/3121551182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be80b4070 con 0x7f8bf006f150 2024-04-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8befe00640 1 -- 172.21.15.46:0/3121551182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be80a7c40 con 0x7f8bf006f150 2024-04-25T08:49:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8befe00640 1 -- 172.21.15.46:0/3121551182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be80af440 con 0x7f8bf006f150 2024-04-25T08:49:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 -- 172.21.15.46:0/3121551182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf006f150 msgr2=0x7f8bf00b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 --2- 172.21.15.46:0/3121551182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf006f150 0x7f8bf00b2720 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f8be8009e10 tx=0x7f8be801ed70 comp rx=0 tx=0).stop 2024-04-25T08:49:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 -- 172.21.15.46:0/3121551182 shutdown_connections 2024-04-25T08:49:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 --2- 172.21.15.46:0/3121551182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf006f150 0x7f8bf00b2720 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 --2- 172.21.15.46:0/3121551182 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf0143990 0x7f8bf00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 -- 172.21.15.46:0/3121551182 >> 172.21.15.46:0/3121551182 conn(0x7f8bf006ac10 msgr2=0x7f8bf006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 -- 172.21.15.46:0/3121551182 shutdown_connections 2024-04-25T08:49:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 -- 172.21.15.46:0/3121551182 wait complete. 2024-04-25T08:49:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 Processor -- start 2024-04-25T08:49:49.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.443+0000 7f8bf7600640 1 -- start start 2024-04-25T08:49:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf7600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf006f150 0x7f8bf015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf7600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf0143990 0x7f8bf0149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf7600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf01440d0 0x7f8bf0159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf7600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00b35f0 con 0x7f8bf0143990 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf7600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00b3470 con 0x7f8bf006f150 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf7600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00b3770 con 0x7f8bf01440d0 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8be7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf0143990 0x7f8bf0149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8be7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf0143990 0x7f8bf0149a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45256/0 (socket says 172.21.15.46:45256) 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8be7e00640 1 -- 172.21.15.46:0/3341770049 learned_addr learned my addr 172.21.15.46:0/3341770049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf5600640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf01440d0 0x7f8bf0159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf4c00640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf006f150 0x7f8bf015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8be7e00640 1 -- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf01440d0 msgr2=0x7f8bf0159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8be7e00640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf01440d0 0x7f8bf0159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8be7e00640 1 -- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf006f150 msgr2=0x7f8bf015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8be7e00640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf006f150 0x7f8bf015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8be7e00640 1 -- 172.21.15.46:0/3341770049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf014a1d0 con 0x7f8bf0143990 2024-04-25T08:49:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf4c00640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf006f150 0x7f8bf015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8be7e00640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf0143990 0x7f8bf0149a60 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800c3f0 tx=0x7f8bd800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bed600640 1 -- 172.21.15.46:0/3341770049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8017070 con 0x7f8bf0143990 2024-04-25T08:49:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bed600640 1 -- 172.21.15.46:0/3341770049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd8005cf0 con 0x7f8bf0143990 2024-04-25T08:49:49.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf014a4b0 con 0x7f8bf0143990 2024-04-25T08:49:49.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bed600640 1 -- 172.21.15.46:0/3341770049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80072f0 con 0x7f8bf0143990 2024-04-25T08:49:49.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf0159da0 con 0x7f8bf0143990 2024-04-25T08:49:49.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf0069010 con 0x7f8bf0143990 2024-04-25T08:49:49.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.447+0000 7f8bed600640 1 -- 172.21.15.46:0/3341770049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bd801a070 con 0x7f8bf0143990 2024-04-25T08:49:49.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.451+0000 7f8bed600640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8bcc066f60 0x7f8bcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.451+0000 7f8bed600640 1 -- 172.21.15.46:0/3341770049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8bd8084620 con 0x7f8bf0143990 2024-04-25T08:49:49.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.451+0000 7f8bf4c00640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8bcc066f60 0x7f8bcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.451+0000 7f8bf4c00640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8bcc066f60 0x7f8bcc069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8bf01478a0 tx=0x7f8be0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:49.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.455+0000 7f8bed600640 1 -- 172.21.15.46:0/3341770049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bd8054d80 con 0x7f8bf0143990 2024-04-25T08:49:49.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.615+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8bf015b520 con 0x7f8bf0143990 2024-04-25T08:49:49.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.615+0000 7f8bed600640 1 -- 172.21.15.46:0/3341770049 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8bd8058e20 con 0x7f8bf0143990 2024-04-25T08:49:49.612 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:49:49.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8bcc066f60 msgr2=0x7f8bcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8bcc066f60 0x7f8bcc069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8bf01478a0 tx=0x7f8be0008040 comp rx=0 tx=0).stop 2024-04-25T08:49:49.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf0143990 msgr2=0x7f8bf0149a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf0143990 0x7f8bf0149a60 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800c3f0 tx=0x7f8bd800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 shutdown_connections 2024-04-25T08:49:49.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8bcc066f60 0x7f8bcc069420 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8bf01440d0 0x7f8bf0159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bf0143990 0x7f8bf0149a60 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 --2- 172.21.15.46:0/3341770049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf006f150 0x7f8bf015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 >> 172.21.15.46:0/3341770049 conn(0x7f8bf006ac10 msgr2=0x7f8bf00b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:49.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 shutdown_connections 2024-04-25T08:49:49.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.619+0000 7f8bf7600640 1 -- 172.21.15.46:0/3341770049 wait complete. 2024-04-25T08:49:49.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:49:49.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddd4a00640 1 Processor -- start 2024-04-25T08:49:49.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddd4a00640 1 -- start start 2024-04-25T08:49:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddd4a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 0x7fddd015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddd4a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fddd00cbeb0 0x7fddd00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddd4a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddd00cda80 0x7fddd015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddd4a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd00cc700 con 0x7fddd00cda80 2024-04-25T08:49:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddd4a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd00cc9e0 con 0x7fddd00cbeb0 2024-04-25T08:49:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddd4a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00cccc0 con 0x7fddd0159230 2024-04-25T08:49:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddce800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddd00cda80 0x7fddd015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddce800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fddd00cda80 0x7fddd015f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45272/0 (socket says 172.21.15.46:45272) 2024-04-25T08:49:49.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddce800640 1 -- 172.21.15.46:0/3860510493 learned_addr learned my addr 172.21.15.46:0/3860510493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddc7e00640 1 --2- 172.21.15.46:0/3860510493 >> v2:172.21.15.82:3300/0 conn(0x7fddd00cbeb0 0x7fddd00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddcde00640 1 --2- 172.21.15.46:0/3860510493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 0x7fddd015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddce800640 1 -- 172.21.15.46:0/3860510493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 msgr2=0x7fddd015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:49.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddce800640 1 --2- 172.21.15.46:0/3860510493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 0x7fddd015bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddce800640 1 -- 172.21.15.46:0/3860510493 >> v2:172.21.15.82:3300/0 conn(0x7fddd00cbeb0 msgr2=0x7fddd00cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddce800640 1 --2- 172.21.15.46:0/3860510493 >> v2:172.21.15.82:3300/0 conn(0x7fddd00cbeb0 0x7fddd00cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddce800640 1 -- 172.21.15.46:0/3860510493 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd00ccfa0 con 0x7fddd00cda80 2024-04-25T08:49:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddcde00640 1 --2- 172.21.15.46:0/3860510493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 0x7fddd015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:49.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.783+0000 7fddc7e00640 1 --2- 172.21.15.46:0/3860510493 >> v2:172.21.15.82:3300/0 conn(0x7fddd00cbeb0 0x7fddd00cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:49.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddce800640 1 --2- 172.21.15.46:0/3860510493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 0x7fddd015f3e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fddd0050030 tx=0x7fddc801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1006564bdffe280f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddcd400640 1 -- 172.21.15.46:0/3860510493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc80ac020 con 0x7fddd00cda80 2024-04-25T08:49:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddcd400640 1 -- 172.21.15.46:0/3860510493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc80a6440 con 0x7fddd00cda80 2024-04-25T08:49:49.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddcd400640 1 -- 172.21.15.46:0/3860510493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc80a6810 con 0x7fddd00cda80 2024-04-25T08:49:49.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddd4a00640 1 -- 172.21.15.46:0/3860510493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 msgr2=0x7fddd015f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3860510493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 0x7fddd015f3e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fddd0050030 tx=0x7fddc801c6a0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddd4a00640 1 -- 172.21.15.46:0/3860510493 shutdown_connections 2024-04-25T08:49:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3860510493 >> v2:172.21.15.82:3300/0 conn(0x7fddd00cbeb0 0x7fddd00cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3860510493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 0x7fddd015f3e0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3860510493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 0x7fddd015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddd4a00640 1 -- 172.21.15.46:0/3860510493 >> 172.21.15.46:0/3860510493 conn(0x7fddd00b12a0 msgr2=0x7fddd00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddd4a00640 1 -- 172.21.15.46:0/3860510493 shutdown_connections 2024-04-25T08:49:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.787+0000 7fddd4a00640 1 -- 172.21.15.46:0/3860510493 wait complete. 2024-04-25T08:49:49.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 Processor -- start 2024-04-25T08:49:49.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 -- start start 2024-04-25T08:49:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd00cbeb0 0x7fddd013dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 0x7fddd0163080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 0x7fddd015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd015d200 con 0x7fddd00cda80 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00cc4a0 con 0x7fddd00cbeb0 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd015d380 con 0x7fddd0159230 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 0x7fddd0163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 0x7fddd0163080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45280/0 (socket says 172.21.15.46:45280) 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc7e00640 1 -- 172.21.15.46:0/3827820391 learned_addr learned my addr 172.21.15.46:0/3827820391 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddcde00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd00cbeb0 0x7fddd013dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddce800640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 0x7fddd015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc7e00640 1 -- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 msgr2=0x7fddd015a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc7e00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 0x7fddd015a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc7e00640 1 -- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd00cbeb0 msgr2=0x7fddd013dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc7e00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd00cbeb0 0x7fddd013dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc7e00640 1 -- 172.21.15.46:0/3827820391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd013fb90 con 0x7fddd00cda80 2024-04-25T08:49:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc7e00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 0x7fddd0163080 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c7c0 tx=0x7fddb800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc6000640 1 -- 172.21.15.46:0/3827820391 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8015070 con 0x7fddd00cda80 2024-04-25T08:49:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc6000640 1 -- 172.21.15.46:0/3827820391 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8007a30 con 0x7fddd00cda80 2024-04-25T08:49:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd013fed0 con 0x7fddd00cda80 2024-04-25T08:49:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddc6000640 1 -- 172.21.15.46:0/3827820391 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb8007e00 con 0x7fddd00cda80 2024-04-25T08:49:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.791+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd0140710 con 0x7fddd00cda80 2024-04-25T08:49:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.795+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd0069080 con 0x7fddd00cda80 2024-04-25T08:49:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.795+0000 7fddc6000640 1 -- 172.21.15.46:0/3827820391 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fddb8018070 con 0x7fddd00cda80 2024-04-25T08:49:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.795+0000 7fddc6000640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddac066f60 0x7fddac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:49.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.795+0000 7fddc6000640 1 -- 172.21.15.46:0/3827820391 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fddb80851e0 con 0x7fddd00cda80 2024-04-25T08:49:49.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.795+0000 7fddcde00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddac066f60 0x7fddac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:49.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.795+0000 7fddcde00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddac066f60 0x7fddac069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fddd015b670 tx=0x7fddc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.799+0000 7fddc6000640 1 -- 172.21.15.46:0/3827820391 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fddd013fed0 con 0x7fddd00cda80 2024-04-25T08:49:49.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fddd0166550 con 0x7fddd00cda80 2024-04-25T08:49:49.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddc6000640 1 -- 172.21.15.46:0/3827820391 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) v1 ==== 80+0+122 (secure 0 0 0) 0x7fddb80558b0 con 0x7fddd00cda80 2024-04-25T08:49:49.970 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/1269610178 2024-04-26T08:48:38.594941+0000 2024-04-25T08:49:49.970 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/1269610178 2024-04-26T08:48:38.594941+0000 2024-04-25T08:49:49.970 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddac066f60 msgr2=0x7fddac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddac066f60 0x7fddac069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fddd015b670 tx=0x7fddc0008040 comp rx=0 tx=0).stop 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 msgr2=0x7fddd0163080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 0x7fddd0163080 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fddb800c7c0 tx=0x7fddb800cc80 comp rx=0 tx=0).stop 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 shutdown_connections 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddac066f60 0x7fddac069420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddd0159230 0x7fddd015a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddd00cda80 0x7fddd0163080 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 --2- 172.21.15.46:0/3827820391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd00cbeb0 0x7fddd013dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 >> 172.21.15.46:0/3827820391 conn(0x7fddd00b12a0 msgr2=0x7fddd006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:49.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 shutdown_connections 2024-04-25T08:49:49.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:49.975+0000 7fddd4a00640 1 -- 172.21.15.46:0/3827820391 wait complete. 2024-04-25T08:49:49.991 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:49:49.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:49.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:49:50.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:49:50.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:50.018 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:49:50.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:49:50.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f42c00640 1 Processor -- start 2024-04-25T08:49:50.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f42c00640 1 -- start start 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f42c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f3c06ac10 0x7f4f3c06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f42c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c1467a0 0x7f4f3c146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f42c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3c0b20d0 con 0x7f4f3c15b5d0 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f42c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f3c15ec20 con 0x7f4f3c06ac10 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f42c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c15e010 con 0x7f4f3c1467a0 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f33e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f3c06ac10 0x7f4f3c06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f33e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f3c06ac10 0x7f4f3c06b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45286/0 (socket says 172.21.15.46:45286) 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f33e00640 1 -- 172.21.15.46:0/2034597671 learned_addr learned my addr 172.21.15.46:0/2034597671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f40c00640 1 --2- 172.21.15.46:0/2034597671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c1467a0 0x7f4f3c146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f33e00640 1 -- 172.21.15.46:0/2034597671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c1467a0 msgr2=0x7f4f3c146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f33e00640 1 --2- 172.21.15.46:0/2034597671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c1467a0 0x7f4f3c146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f33e00640 1 -- 172.21.15.46:0/2034597671 >> v1:172.21.15.82:6789/0 conn(0x7f4f3c15b5d0 legacy=0x7f4f3c0b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f33e00640 1 -- 172.21.15.46:0/2034597671 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c15e2f0 con 0x7f4f3c06ac10 2024-04-25T08:49:50.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.267+0000 7f4f40c00640 1 --2- 172.21.15.46:0/2034597671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c1467a0 0x7f4f3c146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f33e00640 1 --2- 172.21.15.46:0/2034597671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 0x7f4f3c06b0c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009870 tx=0x7f4f2401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e257943a3faf2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f3b400640 1 -- 172.21.15.46:0/2034597671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f240b2070 con 0x7f4f3c06ac10 2024-04-25T08:49:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f3b400640 1 -- 172.21.15.46:0/2034597671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f240a9070 con 0x7f4f3c06ac10 2024-04-25T08:49:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f3b400640 1 -- 172.21.15.46:0/2034597671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f240ad880 con 0x7f4f3c06ac10 2024-04-25T08:49:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f42c00640 1 -- 172.21.15.46:0/2034597671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 msgr2=0x7f4f3c06b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f42c00640 1 --2- 172.21.15.46:0/2034597671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 0x7f4f3c06b0c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009870 tx=0x7f4f2401c560 comp rx=0 tx=0).stop 2024-04-25T08:49:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f42c00640 1 -- 172.21.15.46:0/2034597671 shutdown_connections 2024-04-25T08:49:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f42c00640 1 --2- 172.21.15.46:0/2034597671 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c1467a0 0x7f4f3c146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f42c00640 1 --2- 172.21.15.46:0/2034597671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 0x7f4f3c06b0c0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f42c00640 1 -- 172.21.15.46:0/2034597671 >> 172.21.15.46:0/2034597671 conn(0x7f4f3c072880 msgr2=0x7f4f3c099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f42c00640 1 -- 172.21.15.46:0/2034597671 shutdown_connections 2024-04-25T08:49:50.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f42c00640 1 -- 172.21.15.46:0/2034597671 wait complete. 2024-04-25T08:49:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.271+0000 7f4f42c00640 1 Processor -- start 2024-04-25T08:49:50.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 -- start start 2024-04-25T08:49:50.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 0x7f4f3c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f3c1467a0 0x7f4f3c1395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c15b5d0 0x7f4f3c139b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c0b38b0 con 0x7f4f3c06ac10 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c0b3730 con 0x7f4f3c1467a0 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c15c430 con 0x7f4f3c15b5d0 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f3be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 0x7f4f3c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f3be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 0x7f4f3c157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45288/0 (socket says 172.21.15.46:45288) 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f3be00640 1 -- 172.21.15.46:0/2635606981 learned_addr learned my addr 172.21.15.46:0/2635606981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f40c00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c15b5d0 0x7f4f3c139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f33e00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f3c1467a0 0x7f4f3c1395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f3be00640 1 -- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c15b5d0 msgr2=0x7f4f3c139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f3be00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c15b5d0 0x7f4f3c139b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f3be00640 1 -- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f3c1467a0 msgr2=0x7f4f3c1395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f3be00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f3c1467a0 0x7f4f3c1395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f3be00640 1 -- 172.21.15.46:0/2635606981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c13b180 con 0x7f4f3c06ac10 2024-04-25T08:49:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f40c00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c15b5d0 0x7f4f3c139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f3be00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 0x7f4f3c157890 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00c3c0 tx=0x7f4f2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f38c00640 1 -- 172.21.15.46:0/2635606981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c017070 con 0x7f4f3c06ac10 2024-04-25T08:49:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f38c00640 1 -- 172.21.15.46:0/2635606981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c0052d0 con 0x7f4f3c06ac10 2024-04-25T08:49:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c13b460 con 0x7f4f3c06ac10 2024-04-25T08:49:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f38c00640 1 -- 172.21.15.46:0/2635606981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c003840 con 0x7f4f3c06ac10 2024-04-25T08:49:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c13ba20 con 0x7f4f3c06ac10 2024-04-25T08:49:50.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.275+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3c06ccb0 con 0x7f4f3c06ac10 2024-04-25T08:49:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.279+0000 7f4f38c00640 1 -- 172.21.15.46:0/2635606981 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f2c01a070 con 0x7f4f3c06ac10 2024-04-25T08:49:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.279+0000 7f4f38c00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f18066f60 0x7f4f18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.279+0000 7f4f38c00640 1 -- 172.21.15.46:0/2635606981 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f4f2c084530 con 0x7f4f3c06ac10 2024-04-25T08:49:50.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.279+0000 7f4f33e00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f18066f60 0x7f4f18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.279+0000 7f4f33e00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f18066f60 0x7f4f18069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f240af000 tx=0x7f4f24008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:50.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.283+0000 7f4f38c00640 1 -- 172.21.15.46:0/2635606981 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f2c005480 con 0x7f4f3c06ac10 2024-04-25T08:49:50.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f3c06d360 con 0x7f4f3c06ac10 2024-04-25T08:49:50.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f38c00640 1 -- 172.21.15.46:0/2635606981 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f4f2c054c90 con 0x7f4f3c06ac10 2024-04-25T08:49:50.478 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:50.478 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:48:49.616083+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":91,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7656},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7656":{"gid":7656,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6835/2777781817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":2777781817},{"type":"v1","addr":"172.21.15.46:6835","nonce":2777781817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:49:50.478 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2024-04-25T08:49:50.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f18066f60 msgr2=0x7f4f18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f18066f60 0x7f4f18069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f240af000 tx=0x7f4f24008040 comp rx=0 tx=0).stop 2024-04-25T08:49:50.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 msgr2=0x7f4f3c157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 0x7f4f3c157890 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00c3c0 tx=0x7f4f2c00c880 comp rx=0 tx=0).stop 2024-04-25T08:49:50.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 shutdown_connections 2024-04-25T08:49:50.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f18066f60 0x7f4f18069420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f3c15b5d0 0x7f4f3c139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f3c1467a0 0x7f4f3c1395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 --2- 172.21.15.46:0/2635606981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f3c06ac10 0x7f4f3c157890 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 >> 172.21.15.46:0/2635606981 conn(0x7f4f3c072880 msgr2=0x7f4f3c13f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:50.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 shutdown_connections 2024-04-25T08:49:50.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.483+0000 7f4f42c00640 1 -- 172.21.15.46:0/2635606981 wait complete. 2024-04-25T08:49:50.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:49:50.668 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-25T08:49:50.669 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:49:50.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:49:50.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.815+0000 7f7ce6800640 1 Processor -- start 2024-04-25T08:49:50.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.815+0000 7f7ce6800640 1 -- start start 2024-04-25T08:49:50.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7ce6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ce00b2240 0x7f7ce00b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7ce6800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce006bdc0 con 0x7f7ce00ccde0 2024-04-25T08:49:50.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7ce6800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce006c0a0 con 0x7f7ce0099a00 2024-04-25T08:49:50.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7ce6800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce006db90 con 0x7f7ce00b2240 2024-04-25T08:49:50.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7cd6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7ce00ccde0 0x7f7ce00cd290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35620/0 (socket says 172.21.15.46:35620) 2024-04-25T08:49:50.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7cd6a00640 1 -- 172.21.15.46:0/3743469452 learned_addr learned my addr 172.21.15.46:0/3743469452 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:50.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7cdfe00640 1 --2- 172.21.15.46:0/3743469452 >> v2:172.21.15.46:3300/0 conn(0x7f7ce00b2240 0x7f7ce00b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7cdea00640 1 -- 172.21.15.46:0/3743469452 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080563012 0 0) 0x7f7ce006bdc0 con 0x7f7ce00ccde0 2024-04-25T08:49:50.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7cdea00640 1 -- 172.21.15.46:0/3743469452 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cc4003530 con 0x7f7ce00ccde0 2024-04-25T08:49:50.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7cdfe00640 1 -- 172.21.15.46:0/3743469452 >> v1:172.21.15.82:6789/0 conn(0x7f7ce0099a00 legacy=0x7f7ce0099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7cdea00640 1 -- 172.21.15.46:0/3743469452 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144555791 0 0) 0x7f7ce006c0a0 con 0x7f7ce0099a00 2024-04-25T08:49:50.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7cdfe00640 1 -- 172.21.15.46:0/3743469452 >> v1:172.21.15.46:6789/0 conn(0x7f7ce00ccde0 legacy=0x7f7ce00cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.819+0000 7f7cdfe00640 1 -- 172.21.15.46:0/3743469452 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce006c380 con 0x7f7ce00b2240 2024-04-25T08:49:50.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7cdfe00640 1 --2- 172.21.15.46:0/3743469452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00b2240 0x7f7ce00b2690 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80097f0 tx=0x7f7cd801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41713a115b72c5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:50.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7cdea00640 1 -- 172.21.15.46:0/3743469452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80b0070 con 0x7f7ce00b2240 2024-04-25T08:49:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7cdea00640 1 -- 172.21.15.46:0/3743469452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd80ab600 con 0x7f7ce00b2240 2024-04-25T08:49:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7cdea00640 1 -- 172.21.15.46:0/3743469452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80ab9b0 con 0x7f7ce00b2240 2024-04-25T08:49:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 -- 172.21.15.46:0/3743469452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00b2240 msgr2=0x7f7ce00b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 --2- 172.21.15.46:0/3743469452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00b2240 0x7f7ce00b2690 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80097f0 tx=0x7f7cd801c410 comp rx=0 tx=0).stop 2024-04-25T08:49:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 -- 172.21.15.46:0/3743469452 shutdown_connections 2024-04-25T08:49:50.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 --2- 172.21.15.46:0/3743469452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00b2240 0x7f7ce00b2690 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 -- 172.21.15.46:0/3743469452 >> 172.21.15.46:0/3743469452 conn(0x7f7ce00b3120 msgr2=0x7f7ce00b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:50.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 -- 172.21.15.46:0/3743469452 shutdown_connections 2024-04-25T08:49:50.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 -- 172.21.15.46:0/3743469452 wait complete. 2024-04-25T08:49:50.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 Processor -- start 2024-04-25T08:49:50.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 -- start start 2024-04-25T08:49:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce0099a00 0x7f7ce00ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce00b2240 0x7f7ce0154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00ccde0 0x7f7ce0155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce015de90 con 0x7f7ce00ccde0 2024-04-25T08:49:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce015dd10 con 0x7f7ce0099a00 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7ce6800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce016be10 con 0x7f7ce00b2240 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.823+0000 7f7cdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00ccde0 0x7f7ce0155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00ccde0 0x7f7ce0155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45306/0 (socket says 172.21.15.46:45306) 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cdfe00640 1 -- 172.21.15.46:0/250154403 learned_addr learned my addr 172.21.15.46:0/250154403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cd6a00640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce00b2240 0x7f7ce0154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cdf400640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce0099a00 0x7f7ce00ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cdfe00640 1 -- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce00b2240 msgr2=0x7f7ce0154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cdfe00640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce00b2240 0x7f7ce0154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cdfe00640 1 -- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce0099a00 msgr2=0x7f7ce00ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cdfe00640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce0099a00 0x7f7ce00ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:50.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cdfe00640 1 -- 172.21.15.46:0/250154403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce00cab10 con 0x7f7ce00ccde0 2024-04-25T08:49:50.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cdfe00640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00ccde0 0x7f7ce0155090 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8019020 tx=0x7f7cd801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:50.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cd7e00640 1 -- 172.21.15.46:0/250154403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80b0050 con 0x7f7ce00ccde0 2024-04-25T08:49:50.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cd7e00640 1 -- 172.21.15.46:0/250154403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd80bc070 con 0x7f7ce00ccde0 2024-04-25T08:49:50.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cd7e00640 1 -- 172.21.15.46:0/250154403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8006e20 con 0x7f7ce00ccde0 2024-04-25T08:49:50.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce00cadf0 con 0x7f7ce00ccde0 2024-04-25T08:49:50.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce00c63e0 con 0x7f7ce00ccde0 2024-04-25T08:49:50.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce00c61d0 con 0x7f7ce00ccde0 2024-04-25T08:49:50.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cd7e00640 1 -- 172.21.15.46:0/250154403 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cd80b5810 con 0x7f7ce00ccde0 2024-04-25T08:49:50.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.827+0000 7f7cd7e00640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cbc066f60 0x7f7cbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:50.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.831+0000 7f7cd7e00640 1 -- 172.21.15.46:0/250154403 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f7cd811f820 con 0x7f7ce00ccde0 2024-04-25T08:49:50.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.831+0000 7f7cdf400640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cbc066f60 0x7f7cbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:50.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.831+0000 7f7cdf400640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cbc066f60 0x7f7cbc069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0007fd0 tx=0x7f7cd0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:50.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:50.835+0000 7f7cd7e00640 1 -- 172.21.15.46:0/250154403 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7cd8016090 con 0x7f7ce00ccde0 2024-04-25T08:49:50.947 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:49:50.950 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:49:51.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.031+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7ce0069080 con 0x7f7ce00ccde0 2024-04-25T08:49:51.025 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:49:51.031+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:49:51.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.031+0000 7f7cd7e00640 1 -- 172.21.15.46:0/250154403 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7cd80eff80 con 0x7f7ce00ccde0 2024-04-25T08:49:51.027 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cbc066f60 msgr2=0x7f7cbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cbc066f60 0x7f7cbc069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0007fd0 tx=0x7f7cd0009040 comp rx=0 tx=0).stop 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00ccde0 msgr2=0x7f7ce0155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00ccde0 0x7f7ce0155090 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8019020 tx=0x7f7cd801ac50 comp rx=0 tx=0).stop 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 shutdown_connections 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7cbc066f60 0x7f7cbc069420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ce00ccde0 0x7f7ce0155090 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7ce00b2240 0x7f7ce0154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 --2- 172.21.15.46:0/250154403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ce0099a00 0x7f7ce00ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 >> 172.21.15.46:0/250154403 conn(0x7f7ce00b3120 msgr2=0x7f7ce00cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 shutdown_connections 2024-04-25T08:49:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.035+0000 7f7ce6800640 1 -- 172.21.15.46:0/250154403 wait complete. 2024-04-25T08:49:51.046 INFO:tasks.ceph.mds.c.smithi046.stderr: -4258> 2024-04-25T08:48:45.284+0000 7f9045200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T08:49:51.047 INFO:tasks.ceph.mds.c.smithi046.stderr: -4100> 2024-04-25T08:48:45.520+0000 7f9045200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-04-25T08:48:35.444502+0000 1=1+0) n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x557050529b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T08:48:35.444502+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T08:49:51.054 INFO:tasks.ceph.mds.c.smithi046.stderr: -2241> 2024-04-25T08:49:17.116+0000 7f9045200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-04-25T08:48:35.444502+0000 1=1+0)->f(v1 m2024-04-25T08:48:35.444502+0000) n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1)->n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x557050529b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T08:48:35.444502+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T08:48:35.548501+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T08:49:51.062 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T08:49:51.099 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:49:51.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.147+0000 7fac2e7c38c0 1 -- start start 2024-04-25T08:49:51.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.147+0000 7fac2e7c38c0 1 Processor -- start 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556ace2bd800 0x556ace476580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556ace2bd400 0x556ace476b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ace2bd000 0x556ace477080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf088180 con 0x556ace2bd000 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf088300 con 0x556ace2bd800 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556acf088480 con 0x556ace2bd400 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ace2bd000 0x556ace477080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ace2bd000 0x556ace477080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45328/0 (socket says 172.21.15.46:45328) 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2d200640 1 -- 172.21.15.46:0/3108934450 learned_addr learned my addr 172.21.15.46:0/3108934450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2c800640 1 --2- 172.21.15.46:0/3108934450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556ace2bd800 0x556ace476580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2be00640 1 --2- 172.21.15.46:0/3108934450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556ace2bd400 0x556ace476b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2d200640 1 -- 172.21.15.46:0/3108934450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556ace2bd400 msgr2=0x556ace476b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2d200640 1 --2- 172.21.15.46:0/3108934450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556ace2bd400 0x556ace476b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2d200640 1 -- 172.21.15.46:0/3108934450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556ace2bd800 msgr2=0x556ace476580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2d200640 1 --2- 172.21.15.46:0/3108934450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556ace2bd800 0x556ace476580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2d200640 1 -- 172.21.15.46:0/3108934450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ace2e21e0 con 0x556ace2bd000 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2d200640 1 --2- 172.21.15.46:0/3108934450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ace2bd000 0x556ace477080 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x556acf0fd0b0 tx=0x556ace35e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230172ce5c2b40c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac29600640 1 -- 172.21.15.46:0/3108934450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556ace2f29c0 con 0x556ace2bd000 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac29600640 1 -- 172.21.15.46:0/3108934450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556ace306e00 con 0x556ace2bd000 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac29600640 1 -- 172.21.15.46:0/3108934450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556ace2f24e0 con 0x556ace2bd000 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3108934450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ace2e34a0 con 0x556ace2bd000 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3108934450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556ace2e23c0 con 0x556ace2bd000 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 4 waiting for MDS map... 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac29600640 1 -- 172.21.15.46:0/3108934450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x556acf08c000 con 0x556ace2bd000 2024-04-25T08:49:51.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac29600640 1 -- 172.21.15.46:0/3108934450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 70) v1 ==== 2723+0+0 (secure 0 0 0) 0x556ace313180 con 0x556ace2bd000 2024-04-25T08:49:51.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 4 Got MDS map 70 2024-04-25T08:49:51.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 4 datascan.main: Using metadata pool output 2024-04-25T08:49:51.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 4 datascan.main: connecting to RADOS... 2024-04-25T08:49:51.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 Processor -- start 2024-04-25T08:49:51.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 -- start start 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 0x556ace477600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace477b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556acf242000 0x556ace478100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf088600 con 0x556acf081c00 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf088780 con 0x556acf081800 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2e7c38c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556acf088900 con 0x556acf242000 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace477b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace477b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45340/0 (socket says 172.21.15.46:45340) 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2c800640 1 -- 172.21.15.46:0/3239849426 learned_addr learned my addr 172.21.15.46:0/3239849426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.151+0000 7fac2be00640 1 --2- 172.21.15.46:0/3239849426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 0x556ace477600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 -- 172.21.15.46:0/3239849426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556acf242000 msgr2=0x556ace478100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 --2- 172.21.15.46:0/3239849426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556acf242000 0x556ace478100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 -- 172.21.15.46:0/3239849426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 msgr2=0x556ace477600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 --2- 172.21.15.46:0/3239849426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 0x556ace477600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 -- 172.21.15.46:0/3239849426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ace2e21e0 con 0x556acf081c00 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 --2- 172.21.15.46:0/3239849426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace477b80 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x556acf1bad80 tx=0x556acf1e4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1706c4519e6acd54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac26e00640 1 -- 172.21.15.46:0/3239849426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556ace2f29c0 con 0x556acf081c00 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac26e00640 1 -- 172.21.15.46:0/3239849426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556ace306540 con 0x556acf081c00 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac26e00640 1 -- 172.21.15.46:0/3239849426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556ace2f24e0 con 0x556acf081c00 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3239849426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 msgr2=0x556ace477b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3239849426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace477b80 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x556acf1bad80 tx=0x556acf1e4c00 comp rx=0 tx=0).stop 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3239849426 shutdown_connections 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3239849426 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556acf242000 0x556ace478100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3239849426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace477b80 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3239849426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 0x556ace477600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3239849426 >> 172.21.15.46:0/3239849426 conn(0x556ace2bcc00 msgr2=0x556ace2eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3239849426 shutdown_connections 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3239849426 wait complete. 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 Processor -- start 2024-04-25T08:49:51.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- start start 2024-04-25T08:49:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace479180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 0x556ace478c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556acf243800 0x556ace478680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf089c80 con 0x556acf081c00 2024-04-25T08:49:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556acf089b00 con 0x556acf081800 2024-04-25T08:49:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556acf089980 con 0x556acf243800 2024-04-25T08:49:51.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace479180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace479180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45348/0 (socket says 172.21.15.46:45348) 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 learned_addr learned my addr 172.21.15.46:0/3999670103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 0x556ace478c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556acf243800 0x556ace478680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556acf243800 msgr2=0x556ace478680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556acf243800 0x556ace478680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 msgr2=0x556ace479180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace479180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ace2e32c0 con 0x556acf081800 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556acf081c00 0x556ace479180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2c800640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 0x556ace478c00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x556acf1baf30 tx=0x556ace376e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac25000640 1 -- 172.21.15.46:0/3999670103 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556ace2f2000 con 0x556acf081800 2024-04-25T08:49:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac25000640 1 -- 172.21.15.46:0/3999670103 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556ace306a80 con 0x556acf081800 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac25000640 1 -- 172.21.15.46:0/3999670103 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556ace2f29c0 con 0x556acf081800 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556ace2e3a40 con 0x556acf081800 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ace2e3c20 con 0x556acf081800 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.155+0000 7fac2e7c38c0 4 datascan.init: resolving metadata pool 13 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac25000640 1 -- 172.21.15.46:0/3999670103 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x556ace5e6c00 con 0x556acf081800 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac25000640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x556acf244c00 0x556ace479700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac25000640 1 -- 172.21.15.46:0/3999670103 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x556ace5e4a00 con 0x556acf081800 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac2e7c38c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac2e7c38c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac2e7c38c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x556acf243400 0x556ace478100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556acf243000 con 0x556acf243400 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x556acf244c00 0x556ace479700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x556acf243400 0x556ace478100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x556acf243400 0x556ace478100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.159+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x556acf244c00 0x556ace479700 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x556ace626de0 tx=0x556ace6f03c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.163+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.2 v2:172.21.15.46:6808/3102391403 1 ==== osd_op_reply(1 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf243400 2024-04-25T08:49:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.163+0000 7fac26400640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x556acf245800 0x556ace6fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.163+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556acf245c00 con 0x556acf245800 2024-04-25T08:49:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.163+0000 7fac2d200640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x556acf245800 0x556ace6fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.163+0000 7fac2d200640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x556acf245800 0x556ace6fa000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(2 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac26e00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556ace706000 0x556ace6fa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace706400 con 0x556ace706000 2024-04-25T08:49:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac2c800640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556ace706000 0x556ace6fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac2c800640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556ace706000 0x556ace6fa580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(3 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace706c00 con 0x556acf245800 2024-04-25T08:49:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== osd_op_reply(4 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace706800 con 0x556acf245800 2024-04-25T08:49:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 3 ==== osd_op_reply(5 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac26400640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x556ace707000 0x556ace6fab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace707400 con 0x556ace707000 2024-04-25T08:49:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x556ace707000 0x556ace6fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.167+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x556ace707000 0x556ace6fab00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== osd_op_reply(6 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556ace707000 2024-04-25T08:49:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace707800 con 0x556ace706000 2024-04-25T08:49:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 2 ==== osd_op_reply(7 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710000 con 0x556ace706000 2024-04-25T08:49:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 3 ==== osd_op_reply(8 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac26e00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x556ace707c00 0x556ace6fb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711000 con 0x556ace707c00 2024-04-25T08:49:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac2d200640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x556ace707c00 0x556ace6fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac2d200640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x556ace707c00 0x556ace6fb600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(9 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace707c00 2024-04-25T08:49:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710400 con 0x556ace706000 2024-04-25T08:49:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 4 ==== osd_op_reply(10 [pgnls start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac26e00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x556ace711400 0x556ace6fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711800 con 0x556ace711400 2024-04-25T08:49:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.171+0000 7fac2c800640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x556ace711400 0x556ace6fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2c800640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x556ace711400 0x556ace6fbb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== osd_op_reply(11 [pgnls start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710800 con 0x556acf243400 2024-04-25T08:49:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.2 v2:172.21.15.46:6808/3102391403 2 ==== osd_op_reply(12 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf243400 2024-04-25T08:49:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711c00 con 0x556acf245800 2024-04-25T08:49:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 4 ==== osd_op_reply(13 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710c00 con 0x556ace706000 2024-04-25T08:49:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 5 ==== osd_op_reply(14 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace706400 con 0x556acf245800 2024-04-25T08:49:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 5 ==== osd_op_reply(15 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710000 con 0x556ace706000 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 6 ==== osd_op_reply(16 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace707800 con 0x556ace707c00 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== osd_op_reply(17 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace707c00 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710400 con 0x556ace711400 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 2 ==== osd_op_reply(18 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace706c00 con 0x556acf243400 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.2 v2:172.21.15.46:6808/3102391403 3 ==== osd_op_reply(19 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf243400 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711000 con 0x556ace711400 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 3 ==== osd_op_reply(20 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.175+0000 7fac26e00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x556acf245c00 0x556ace6fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.179+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556acf081400 con 0x556acf245c00 2024-04-25T08:49:51.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.179+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x556acf245c00 0x556ace6fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.179+0000 7fac2be00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x556acf245c00 0x556ace6fc100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.179+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(21 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf245c00 2024-04-25T08:49:51.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.179+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace706800 con 0x556ace711400 2024-04-25T08:49:51.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.179+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 4 ==== osd_op_reply(22 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.179+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711800 con 0x556acf245800 2024-04-25T08:49:51.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.179+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 6 ==== osd_op_reply(23 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.179+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace707400 con 0x556acf245800 2024-04-25T08:49:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 7 ==== osd_op_reply(24 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710c00 con 0x556acf245800 2024-04-25T08:49:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 8 ==== osd_op_reply(25 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace706c00 con 0x556ace711400 2024-04-25T08:49:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 5 ==== osd_op_reply(26 [pgnls start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710000 con 0x556ace707000 2024-04-25T08:49:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.3 v2:172.21.15.46:6824/1647449001 2 ==== osd_op_reply(27 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556ace707000 2024-04-25T08:49:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710800 con 0x556acf243400 2024-04-25T08:49:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.2 v2:172.21.15.46:6808/3102391403 4 ==== osd_op_reply(28 [pgnls start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf243400 2024-04-25T08:49:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710400 con 0x556ace711400 2024-04-25T08:49:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 6 ==== osd_op_reply(29 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711c00 con 0x556ace707000 2024-04-25T08:49:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.3 v2:172.21.15.46:6824/1647449001 3 ==== osd_op_reply(30 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556ace707000 2024-04-25T08:49:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac26e00640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x556ace711000 0x556ace6fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556acf243000 con 0x556ace711000 2024-04-25T08:49:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2d200640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x556ace711000 0x556ace6fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2d200640 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x556ace711000 0x556ace6fc680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(31 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace711000 2024-04-25T08:49:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.183+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711800 con 0x556acf245c00 2024-04-25T08:49:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.5 v2:172.21.15.82:6816/541310136 2 ==== osd_op_reply(32 [pgnls start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf245c00 2024-04-25T08:49:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710000 con 0x556ace707000 2024-04-25T08:49:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.3 v2:172.21.15.46:6824/1647449001 4 ==== osd_op_reply(33 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556ace707000 2024-04-25T08:49:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710c00 con 0x556acf245c00 2024-04-25T08:49:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.5 v2:172.21.15.82:6816/541310136 3 ==== osd_op_reply(34 [pgnls start_epoch 94 out=49b] v90'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf245c00 2024-04-25T08:49:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710800 con 0x556ace711400 2024-04-25T08:49:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 7 ==== osd_op_reply(35 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace706400 con 0x556ace706000 2024-04-25T08:49:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 7 ==== osd_op_reply(36 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711c00 con 0x556acf245c00 2024-04-25T08:49:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.5 v2:172.21.15.82:6816/541310136 4 ==== osd_op_reply(37 [pgnls start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf245c00 2024-04-25T08:49:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace707800 con 0x556ace707c00 2024-04-25T08:49:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.6 v2:172.21.15.82:6823/1178952145 3 ==== osd_op_reply(38 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace707c00 2024-04-25T08:49:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711800 con 0x556ace707c00 2024-04-25T08:49:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.6 v2:172.21.15.82:6823/1178952145 4 ==== osd_op_reply(39 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace707c00 2024-04-25T08:49:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace707400 con 0x556ace711000 2024-04-25T08:49:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.7 v2:172.21.15.82:6808/1420997799 2 ==== osd_op_reply(40 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace711000 2024-04-25T08:49:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556acf081400 con 0x556acf245800 2024-04-25T08:49:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 9 ==== osd_op_reply(41 [pgnls start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace720000 con 0x556acf243400 2024-04-25T08:49:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.2 v2:172.21.15.46:6808/3102391403 5 ==== osd_op_reply(42 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf243400 2024-04-25T08:49:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.187+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace722000 con 0x556ace711000 2024-04-25T08:49:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.7 v2:172.21.15.82:6808/1420997799 3 ==== osd_op_reply(43 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace711000 2024-04-25T08:49:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace720400 con 0x556ace711400 2024-04-25T08:49:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 8 ==== osd_op_reply(44 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace722400 con 0x556ace707c00 2024-04-25T08:49:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.6 v2:172.21.15.82:6823/1178952145 5 ==== osd_op_reply(45 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace707c00 2024-04-25T08:49:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace720800 con 0x556ace711400 2024-04-25T08:49:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 9 ==== osd_op_reply(46 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace722800 con 0x556acf245800 2024-04-25T08:49:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 10 ==== osd_op_reply(47 [pgnls start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace720c00 con 0x556ace707000 2024-04-25T08:49:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.3 v2:172.21.15.46:6824/1647449001 5 ==== osd_op_reply(48 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556ace707000 2024-04-25T08:49:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace722c00 con 0x556acf245800 2024-04-25T08:49:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 11 ==== osd_op_reply(49 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace721000 con 0x556ace711000 2024-04-25T08:49:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.7 v2:172.21.15.82:6808/1420997799 4 ==== osd_op_reply(50 [pgnls start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace711000 2024-04-25T08:49:51.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace723000 con 0x556acf243400 2024-04-25T08:49:51.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.2 v2:172.21.15.46:6808/3102391403 6 ==== osd_op_reply(51 [pgnls start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf243400 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace721400 con 0x556ace706000 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 8 ==== osd_op_reply(52 [pgnls start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.191+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace723400 con 0x556acf245800 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 12 ==== osd_op_reply(53 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace721800 con 0x556ace711400 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 10 ==== osd_op_reply(54 [pgnls start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace723800 con 0x556ace706000 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 9 ==== osd_op_reply(55 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace721c00 con 0x556ace711400 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 11 ==== osd_op_reply(56 [pgnls start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace723c00 con 0x556acf245c00 2024-04-25T08:49:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.5 v2:172.21.15.82:6816/541310136 5 ==== osd_op_reply(57 [pgnls start_epoch 94 out=81b] v90'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556ace3cc000 con 0x556acf245c00 2024-04-25T08:49:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x556acf242c00 con 0x556acf245c00 2024-04-25T08:49:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.5 v2:172.21.15.82:6816/541310136 6 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556ace3cc000 con 0x556acf245c00 2024-04-25T08:49:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac2e7c38c0 10 datascan.operator(): handling object 1099511627777.0 2024-04-25T08:49:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.195+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v9 -- 0x556acf242800 con 0x556acf245c00 2024-04-25T08:49:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.5 v2:172.21.15.82:6816/541310136 7 ==== osd_op_reply(59 10000000001.00000000 [call] v94'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556ace3cc000 con 0x556acf245c00 2024-04-25T08:49:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556acf242400 con 0x556acf245c00 2024-04-25T08:49:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.5 v2:172.21.15.82:6816/541310136 8 ==== osd_op_reply(60 [pgnls start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556acf245c00 2024-04-25T08:49:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710000 con 0x556ace706000 2024-04-25T08:49:51.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.1 v2:172.21.15.46:6809/2552657765 10 ==== osd_op_reply(61 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace706000 2024-04-25T08:49:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace720000 con 0x556ace711400 2024-04-25T08:49:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 <== osd.4 v2:172.21.15.82:6800/2293006647 12 ==== osd_op_reply(62 [pgnls start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc780 con 0x556ace711400 2024-04-25T08:49:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace710c00 con 0x556ace711000 2024-04-25T08:49:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.7 v2:172.21.15.82:6808/1420997799 5 ==== osd_op_reply(63 [pgnls start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556ace711000 2024-04-25T08:49:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace720c00 con 0x556acf245800 2024-04-25T08:49:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 13 ==== osd_op_reply(64 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac26e00640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace711c00 con 0x556acf245800 2024-04-25T08:49:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2d200640 1 -- 172.21.15.46:0/3999670103 <== osd.0 v2:172.21.15.46:6800/3448226631 14 ==== osd_op_reply(65 [pgnls start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc500 con 0x556acf245800 2024-04-25T08:49:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac26400640 1 -- 172.21.15.46:0/3999670103 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x556ace707800 con 0x556ace707000 2024-04-25T08:49:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2be00640 1 -- 172.21.15.46:0/3999670103 <== osd.3 v2:172.21.15.46:6824/1647449001 6 ==== osd_op_reply(66 [pgnls start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556ace3cc000 con 0x556ace707000 2024-04-25T08:49:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x556acf245800 msgr2=0x556ace6fa000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x556acf245800 0x556ace6fa000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556ace706000 msgr2=0x556ace6fa580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556ace706000 0x556ace6fa580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x556acf243400 msgr2=0x556ace478100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x556acf243400 0x556ace478100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x556ace707000 msgr2=0x556ace6fab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x556ace707000 0x556ace6fab00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x556ace711400 msgr2=0x556ace6fbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x556ace711400 0x556ace6fbb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x556acf245c00 msgr2=0x556ace6fc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x556acf245c00 0x556ace6fc100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x556ace707c00 msgr2=0x556ace6fb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x556ace707c00 0x556ace6fb600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x556ace711000 msgr2=0x556ace6fc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x556ace711000 0x556ace6fc680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 reap_dead start 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 reap_dead start 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 reap_dead start 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x556acf244c00 msgr2=0x556ace479700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2c800640 1 -- 172.21.15.46:0/3999670103 reap_dead start 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x556acf244c00 0x556ace479700 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x556ace626de0 tx=0x556ace6f03c0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 msgr2=0x556ace478c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 0x556ace478c00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x556acf1baf30 tx=0x556ace376e40 comp rx=0 tx=0).stop 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 shutdown_connections 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x556acf244c00 0x556ace479700 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3999670103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556acf081800 0x556ace478c00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 >> 172.21.15.46:0/3999670103 conn(0x556ace2bcc00 msgr2=0x556ace2eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 shutdown_connections 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3999670103 wait complete. 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3108934450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ace2bd000 msgr2=0x556ace477080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3108934450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ace2bd000 0x556ace477080 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x556acf0fd0b0 tx=0x556ace35e000 comp rx=0 tx=0).stop 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3108934450 shutdown_connections 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3108934450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ace2bd000 0x556ace477080 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3108934450 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556ace2bd400 0x556ace476b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 --2- 172.21.15.46:0/3108934450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556ace2bd800 0x556ace476580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.199+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3108934450 >> 172.21.15.46:0/3108934450 conn(0x556ace2bdc00 msgr2=0x556ace2eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.203+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3108934450 shutdown_connections 2024-04-25T08:49:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.203+0000 7fac2e7c38c0 1 -- 172.21.15.46:0/3108934450 wait complete. 2024-04-25T08:49:51.201 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.139659, result: 2024-04-25T08:49:51.202 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-25T08:49:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.295+0000 7f5b50a548c0 1 -- start start 2024-04-25T08:49:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.295+0000 7f5b50a548c0 1 Processor -- start 2024-04-25T08:49:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.295+0000 7f5b50a548c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c242db800 0x563c24494580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.295+0000 7f5b50a548c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c242db400 0x563c24494b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.295+0000 7f5b50a548c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c242db000 0x563c24495080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.295+0000 7f5b50a548c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c251da180 con 0x563c242db800 2024-04-25T08:49:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.295+0000 7f5b50a548c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c251da300 con 0x563c242db000 2024-04-25T08:49:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.295+0000 7f5b50a548c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c251da480 con 0x563c242db400 2024-04-25T08:49:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c242db800 0x563c24494580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c242db800 0x563c24494580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45356/0 (socket says 172.21.15.46:45356) 2024-04-25T08:49:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 -- 172.21.15.46:0/560455717 learned_addr learned my addr 172.21.15.46:0/560455717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 -- 172.21.15.46:0/560455717 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c242db400 msgr2=0x563c24494b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/560455717 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c242db400 0x563c24494b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 -- 172.21.15.46:0/560455717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c242db000 msgr2=0x563c24495080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4f600640 1 --2- 172.21.15.46:0/560455717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c242db000 0x563c24495080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/560455717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c242db000 0x563c24495080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 -- 172.21.15.46:0/560455717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c243001e0 con 0x563c242db800 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4f600640 1 --2- 172.21.15.46:0/560455717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c242db000 0x563c24495080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/560455717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c242db800 0x563c24494580 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x563c25127110 tx=0x563c243d8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27cdadf3c1b0b81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ba00640 1 -- 172.21.15.46:0/560455717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c243109c0 con 0x563c242db800 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ba00640 1 -- 172.21.15.46:0/560455717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563c243241c0 con 0x563c242db800 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ba00640 1 -- 172.21.15.46:0/560455717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c243104e0 con 0x563c242db800 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 -- 172.21.15.46:0/560455717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c243012c0 con 0x563c242db800 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 -- 172.21.15.46:0/560455717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563c243003c0 con 0x563c242db800 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 4 waiting for MDS map... 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ba00640 1 -- 172.21.15.46:0/560455717 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x563c251e8200 con 0x563c242db800 2024-04-25T08:49:51.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ba00640 1 -- 172.21.15.46:0/560455717 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 70) v1 ==== 2723+0+0 (secure 0 0 0) 0x563c24331180 con 0x563c242db800 2024-04-25T08:49:51.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 4 Got MDS map 70 2024-04-25T08:49:51.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 4 datascan.main: Using metadata pool output 2024-04-25T08:49:51.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 4 datascan.main: connecting to RADOS... 2024-04-25T08:49:51.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 Processor -- start 2024-04-25T08:49:51.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 -- start start 2024-04-25T08:49:51.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c250b7800 0x563c24495600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c250b7c00 0x563c24495b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25256000 0x563c24496100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c251da600 con 0x563c25256000 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c251da780 con 0x563c250b7800 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b50a548c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c251da900 con 0x563c250b7c00 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25256000 0x563c24496100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4f600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c250b7800 0x563c24495600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25256000 0x563c24496100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45364/0 (socket says 172.21.15.46:45364) 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4e200640 1 -- 172.21.15.46:0/136966807 learned_addr learned my addr 172.21.15.46:0/136966807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.299+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/136966807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c250b7c00 0x563c24495b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 -- 172.21.15.46:0/136966807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c250b7c00 msgr2=0x563c24495b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 --2- 172.21.15.46:0/136966807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c250b7c00 0x563c24495b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 -- 172.21.15.46:0/136966807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c250b7800 msgr2=0x563c24495600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 --2- 172.21.15.46:0/136966807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c250b7800 0x563c24495600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 -- 172.21.15.46:0/136966807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c243001e0 con 0x563c25256000 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 --2- 172.21.15.46:0/136966807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25256000 0x563c24496100 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x563c251f4780 tx=0x563c25204cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dec9efa1bca328 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b49200640 1 -- 172.21.15.46:0/136966807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c243109c0 con 0x563c25256000 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b49200640 1 -- 172.21.15.46:0/136966807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563c24324c40 con 0x563c25256000 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b49200640 1 -- 172.21.15.46:0/136966807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c243104e0 con 0x563c25256000 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- 172.21.15.46:0/136966807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25256000 msgr2=0x563c24496100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/136966807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25256000 0x563c24496100 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x563c251f4780 tx=0x563c25204cc0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- 172.21.15.46:0/136966807 shutdown_connections 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/136966807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25256000 0x563c24496100 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/136966807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c250b7c00 0x563c24495b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/136966807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c250b7800 0x563c24495600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- 172.21.15.46:0/136966807 >> 172.21.15.46:0/136966807 conn(0x563c242dac00 msgr2=0x563c24309800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- 172.21.15.46:0/136966807 shutdown_connections 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- 172.21.15.46:0/136966807 wait complete. 2024-04-25T08:49:51.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 Processor -- start 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- start start 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c250b7c00 0x563c24497180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c250b7800 0x563c24496c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25257800 0x563c24496680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c251dbc80 con 0x563c25257800 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c251dbb00 con 0x563c250b7800 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c251db980 con 0x563c250b7c00 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25257800 0x563c24496680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4f600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c250b7c00 0x563c24497180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25257800 0x563c24496680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45374/0 (socket says 172.21.15.46:45374) 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 learned_addr learned my addr 172.21.15.46:0/3972390617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c250b7800 0x563c24496c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c250b7c00 msgr2=0x563c24497180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c250b7c00 0x563c24497180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c250b7800 msgr2=0x563c24496c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c250b7800 0x563c24496c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c243014a0 con 0x563c25257800 2024-04-25T08:49:51.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b4e200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25257800 0x563c24496680 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x563c251f58c0 tx=0x563c243d81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b47400640 1 -- 172.21.15.46:0/3972390617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c24310000 con 0x563c25257800 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b47400640 1 -- 172.21.15.46:0/3972390617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563c24324a80 con 0x563c25257800 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b47400640 1 -- 172.21.15.46:0/3972390617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c243109c0 con 0x563c25257800 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.303+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563c24301a40 con 0x563c25257800 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c24301c20 con 0x563c25257800 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b50a548c0 4 datascan.init: resolving metadata pool 13 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b47400640 1 -- 172.21.15.46:0/3972390617 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x563c250b0c00 con 0x563c25257800 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b47400640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563c250ae800 0x563c24497700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b47400640 1 -- 172.21.15.46:0/3972390617 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x563c25259400 con 0x563c25257800 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b50a548c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b50a548c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b50a548c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563c25257400 0x563c24496100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25257000 con 0x563c25257400 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b4f600640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563c250ae800 0x563c24497700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563c25257400 0x563c24496100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563c25257400 0x563c24496100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b4f600640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563c250ae800 0x563c24497700 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x563c243b7080 tx=0x563c246fc000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563c25256c00 0x563c24495b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25256800 con 0x563c25256c00 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.307+0000 7f5b4f600640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563c25256c00 0x563c24495b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4f600640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563c25256c00 0x563c24495b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563c25256400 0x563c24495600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25256000 con 0x563c25256400 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563c25256400 0x563c24495600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563c25256400 0x563c24495600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250af800 con 0x563c25257400 2024-04-25T08:49:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b49200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563c250afc00 0x563c24712000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716000 con 0x563c250afc00 2024-04-25T08:49:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4e200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563c250afc00 0x563c24712000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4e200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563c250afc00 0x563c24712000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716800 con 0x563c25257400 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.311+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716400 con 0x563c25257400 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b48800640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563c24716c00 0x563c24712580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25257000 con 0x563c24716c00 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4f600640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563c24716c00 0x563c24712580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4f600640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563c24716c00 0x563c24712580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716800 con 0x563c250afc00 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25256000 con 0x563c250afc00 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717000 con 0x563c25256c00 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250b7400 con 0x563c250afc00 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b49200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563c250af800 0x563c24712b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25256000 con 0x563c250af800 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563c250af800 0x563c24712b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4ec00640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563c250af800 0x563c24712b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.315+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716000 con 0x563c25256400 2024-04-25T08:49:51.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25256800 con 0x563c25257400 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716800 con 0x563c250afc00 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250af400 con 0x563c25257400 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717000 con 0x563c250afc00 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250b7400 con 0x563c25256c00 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25257000 con 0x563c250af800 2024-04-25T08:49:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716400 con 0x563c25256400 2024-04-25T08:49:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716000 con 0x563c250af800 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b49200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563c25256000 0x563c24713080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25256800 con 0x563c25256000 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4e200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563c25256000 0x563c24713080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4e200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563c25256000 0x563c24713080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.319+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717400 con 0x563c250af800 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25257000 con 0x563c25257400 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717800 con 0x563c25257400 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250af400 con 0x563c25257400 2024-04-25T08:49:51.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717c00 con 0x563c250af800 2024-04-25T08:49:51.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716400 con 0x563c24716c00 2024-04-25T08:49:51.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717c00 con 0x563c25256400 2024-04-25T08:49:51.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716000 con 0x563c250af800 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c000 con 0x563c24716c00 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b49200640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563c24717400 0x563c24713600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717800 con 0x563c24717400 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4f600640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563c24717400 0x563c24713600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.323+0000 7f5b4f600640 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563c24717400 0x563c24713600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24717400 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c400 con 0x563c25256000 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250af400 con 0x563c24716c00 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c800 con 0x563c25256000 2024-04-25T08:49:51.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 94 out=49b] v90'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25257000 con 0x563c250af800 2024-04-25T08:49:51.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472cc00 con 0x563c250afc00 2024-04-25T08:49:51.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250af400 con 0x563c25256000 2024-04-25T08:49:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d000 con 0x563c25256c00 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716800 con 0x563c25256c00 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.327+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d400 con 0x563c24717400 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.7 v2:172.21.15.82:6808/1420997799 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24717400 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717000 con 0x563c25257400 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d800 con 0x563c25256400 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716400 con 0x563c24717400 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.7 v2:172.21.15.82:6808/1420997799 3 ==== osd_op_reply(45 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24717400 2024-04-25T08:49:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c400 con 0x563c250af800 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717800 con 0x563c25256c00 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c800 con 0x563c250af800 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250b7400 con 0x563c25257400 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c000 con 0x563c24716c00 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25256800 con 0x563c25257400 2024-04-25T08:49:51.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472dc00 con 0x563c24717400 2024-04-25T08:49:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.7 v2:172.21.15.82:6808/1420997799 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24717400 2024-04-25T08:49:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.331+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d000 con 0x563c25256400 2024-04-25T08:49:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250b7400 con 0x563c250afc00 2024-04-25T08:49:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716800 con 0x563c25257400 2024-04-25T08:49:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d400 con 0x563c250af800 2024-04-25T08:49:51.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716400 con 0x563c250afc00 2024-04-25T08:49:51.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c000 con 0x563c250af800 2024-04-25T08:49:51.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717800 con 0x563c25256000 2024-04-25T08:49:51.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 94 out=81b] v94'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24740000 con 0x563c25256400 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717c00 con 0x563c25257400 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d800 con 0x563c250afc00 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 10 ==== osd_op_reply(62 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716000 con 0x563c25257400 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c400 con 0x563c25257400 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.335+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717000 con 0x563c24716c00 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c800 con 0x563c250afc00 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25257000 con 0x563c250afc00 2024-04-25T08:49:51.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 12 ==== osd_op_reply(67 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d000 con 0x563c25256c00 2024-04-25T08:49:51.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d400 con 0x563c250afc00 2024-04-25T08:49:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 13 ==== osd_op_reply(69 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716800 con 0x563c250af800 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c000 con 0x563c25256400 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717c00 con 0x563c25257400 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c400 con 0x563c250afc00 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 14 ==== osd_op_reply(73 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716000 con 0x563c25257400 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472cc00 con 0x563c250afc00 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.339+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716400 con 0x563c25256c00 2024-04-25T08:49:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d800 con 0x563c250af800 2024-04-25T08:49:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717800 con 0x563c25256400 2024-04-25T08:49:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24748000 con 0x563c250af800 2024-04-25T08:49:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250b7400 con 0x563c25256000 2024-04-25T08:49:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 6 ==== osd_op_reply(80 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24748400 con 0x563c250af800 2024-04-25T08:49:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250af400 con 0x563c25257400 2024-04-25T08:49:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.343+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24748800 con 0x563c25257400 2024-04-25T08:49:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24740000 con 0x563c25257400 2024-04-25T08:49:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24748c00 con 0x563c250af800 2024-04-25T08:49:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25256800 con 0x563c24716c00 2024-04-25T08:49:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24749000 con 0x563c25256400 2024-04-25T08:49:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24748000 con 0x563c250af800 2024-04-25T08:49:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24749400 con 0x563c24716c00 2024-04-25T08:49:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24748400 con 0x563c24717400 2024-04-25T08:49:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.7 v2:172.21.15.82:6808/1420997799 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24717400 2024-04-25T08:49:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24749800 con 0x563c25256000 2024-04-25T08:49:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24748800 con 0x563c24716c00 2024-04-25T08:49:51.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.347+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24749c00 con 0x563c25256000 2024-04-25T08:49:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 8 ==== osd_op_reply(93 [pgnls-filter start_epoch 94 out=49b] v90'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250af400 con 0x563c250af800 2024-04-25T08:49:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25257000 con 0x563c250afc00 2024-04-25T08:49:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 16 ==== osd_op_reply(95 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716800 con 0x563c25256000 2024-04-25T08:49:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 9 ==== osd_op_reply(96 [pgnls-filter start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c25256800 con 0x563c25256c00 2024-04-25T08:49:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717c00 con 0x563c25256c00 2024-04-25T08:49:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717000 con 0x563c24717400 2024-04-25T08:49:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.7 v2:172.21.15.82:6808/1420997799 6 ==== osd_op_reply(99 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24717400 2024-04-25T08:49:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716000 con 0x563c25257400 2024-04-25T08:49:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 94 out=49b] v90'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24716400 con 0x563c25256400 2024-04-25T08:49:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.351+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24717800 con 0x563c24717400 2024-04-25T08:49:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.7 v2:172.21.15.82:6808/1420997799 7 ==== osd_op_reply(102 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24717400 2024-04-25T08:49:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c800 con 0x563c250af800 2024-04-25T08:49:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c000 con 0x563c25256c00 2024-04-25T08:49:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d400 con 0x563c250af800 2024-04-25T08:49:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d800 con 0x563c25257400 2024-04-25T08:49:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c400 con 0x563c24716c00 2024-04-25T08:49:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24749800 con 0x563c25257400 2024-04-25T08:49:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472cc00 con 0x563c24717400 2024-04-25T08:49:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.7 v2:172.21.15.82:6808/1420997799 8 ==== osd_op_reply(109 [pgnls-filter start_epoch 94 out=49b] v90'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24717400 2024-04-25T08:49:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24749400 con 0x563c25256400 2024-04-25T08:49:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.2 v2:172.21.15.46:6808/3102391403 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 94 out=49b] v90'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25256400 2024-04-25T08:49:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472dc00 con 0x563c250afc00 2024-04-25T08:49:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 94 out=49b] v90'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24748400 con 0x563c25257400 2024-04-25T08:49:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.355+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d000 con 0x563c250af800 2024-04-25T08:49:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c000 con 0x563c250afc00 2024-04-25T08:49:51.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 18 ==== osd_op_reply(114 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c250b7400 con 0x563c250af800 2024-04-25T08:49:51.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 94 out=49b] v90'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c400 con 0x563c25256000 2024-04-25T08:49:51.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 10 ==== osd_op_reply(116 [pgnls-filter start_epoch 94 out=81b] v94'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 10 datascan.operator(): handling object 10000000001.0 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24740800 con 0x563c25256000 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 11 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24740c00 con 0x563c250afc00 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 19 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24741000 con 0x563c25256c00 2024-04-25T08:49:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.6 v2:172.21.15.82:6823/1178952145 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+520 (crc 0 0 0) 0x563c2438e500 con 0x563c25256c00 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 20 datascan.get_frag_of: frag is * 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24741400 con 0x563c250afc00 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 20 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24741800 con 0x563c250afc00 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 21 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 20 datascan.read_dentry: file_head not found in result 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-25T08:49:51.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.359+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24741c00 con 0x563c250afc00 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 22 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v94'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24749c00 con 0x563c25256000 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.5 v2:172.21.15.82:6816/541310136 12 ==== osd_op_reply(123 [pgnls-filter start_epoch 94 out=49b] v90'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c25256000 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472cc00 con 0x563c250afc00 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 <== osd.1 v2:172.21.15.46:6809/2552657765 23 ==== osd_op_reply(124 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e780 con 0x563c250afc00 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24749000 con 0x563c250af800 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.4 v2:172.21.15.82:6800/2293006647 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 94 out=49b] v90'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c250af800 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472c800 con 0x563c24717400 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.7 v2:172.21.15.82:6808/1420997799 9 ==== osd_op_reply(126 [pgnls-filter start_epoch 94 out=49b] v90'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24717400 2024-04-25T08:49:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24748000 con 0x563c25257400 2024-04-25T08:49:51.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b48800640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c2472d400 con 0x563c25257400 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4ec00640 1 -- 172.21.15.46:0/3972390617 <== osd.0 v2:172.21.15.46:6800/3448226631 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 94 out=49b] v90'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e000 con 0x563c25257400 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b49200640 1 -- 172.21.15.46:0/3972390617 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x563c24749800 con 0x563c24716c00 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4f600640 1 -- 172.21.15.46:0/3972390617 <== osd.3 v2:172.21.15.46:6824/1647449001 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 94 out=49b] v90'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563c2438e500 con 0x563c24716c00 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563c25257400 msgr2=0x563c24496100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563c25257400 0x563c24496100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563c250afc00 msgr2=0x563c24712000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563c250afc00 0x563c24712000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563c25256400 msgr2=0x563c24495600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563c25256400 0x563c24495600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563c24716c00 msgr2=0x563c24712580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563c24716c00 0x563c24712580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563c250af800 msgr2=0x563c24712b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563c250af800 0x563c24712b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563c25256000 msgr2=0x563c24713080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563c25256000 0x563c24713080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563c25256c00 msgr2=0x563c24495b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563c25256c00 0x563c24495b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563c24717400 msgr2=0x563c24713600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563c24717400 0x563c24713600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 reap_dead start 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 reap_dead start 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 reap_dead start 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.363+0000 7f5b4e200640 1 -- 172.21.15.46:0/3972390617 reap_dead start 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563c250ae800 msgr2=0x563c24497700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563c250ae800 0x563c24497700 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x563c243b7080 tx=0x563c246fc000 comp rx=0 tx=0).stop 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25257800 msgr2=0x563c24496680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25257800 0x563c24496680 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x563c251f58c0 tx=0x563c243d81e0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 shutdown_connections 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563c250ae800 0x563c24497700 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/3972390617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c25257800 0x563c24496680 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 >> 172.21.15.46:0/3972390617 conn(0x563c242dac00 msgr2=0x563c24309800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 shutdown_connections 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/3972390617 wait complete. 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/560455717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c242db800 msgr2=0x563c24494580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/560455717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c242db800 0x563c24494580 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x563c25127110 tx=0x563c243d8000 comp rx=0 tx=0).stop 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/560455717 shutdown_connections 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/560455717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c242db000 0x563c24495080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/560455717 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563c242db400 0x563c24494b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 --2- 172.21.15.46:0/560455717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c242db800 0x563c24494580 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/560455717 >> 172.21.15.46:0/560455717 conn(0x563c242dbc00 msgr2=0x563c24309000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/560455717 shutdown_connections 2024-04-25T08:49:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.367+0000 7f5b50a548c0 1 -- 172.21.15.46:0/560455717 wait complete. 2024-04-25T08:49:51.367 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.165269, result: 2024-04-25T08:49:51.367 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T08:49:51.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 -- start start 2024-04-25T08:49:51.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 Processor -- start 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b80fb3800 0x557b8116c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b80fb3400 0x557b8116cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b80fb3000 0x557b8116d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b81ea6180 con 0x557b80fb3400 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b81ea6300 con 0x557b80fb3800 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b81ea6480 con 0x557b80fb3000 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b80fb3400 0x557b8116cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b80fb3400 0x557b8116cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45394/0 (socket says 172.21.15.46:45394) 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedace00640 1 -- 172.21.15.46:0/3223663347 learned_addr learned my addr 172.21.15.46:0/3223663347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedae200640 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b80fb3000 0x557b8116d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedad800640 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b80fb3800 0x557b8116c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedace00640 1 -- 172.21.15.46:0/3223663347 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b80fb3000 msgr2=0x557b8116d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedace00640 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b80fb3000 0x557b8116d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedace00640 1 -- 172.21.15.46:0/3223663347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b80fb3800 msgr2=0x557b8116c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedace00640 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b80fb3800 0x557b8116c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedace00640 1 -- 172.21.15.46:0/3223663347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b80fd81e0 con 0x557b80fb3400 2024-04-25T08:49:51.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedae200640 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b80fb3000 0x557b8116d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:51.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedace00640 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b80fb3400 0x557b8116cb00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x557b81df38c0 tx=0x557b81054000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f1775469dfd2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedaa600640 1 -- 172.21.15.46:0/3223663347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b80fe89c0 con 0x557b80fb3400 2024-04-25T08:49:51.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedaa600640 1 -- 172.21.15.46:0/3223663347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b80ffce00 con 0x557b80fb3400 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedaa600640 1 -- 172.21.15.46:0/3223663347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b80fe84e0 con 0x557b80fb3400 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3223663347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b80fd94a0 con 0x557b80fb3400 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3223663347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557b80fd83c0 con 0x557b80fb3400 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 4 waiting for MDS map... 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedaa600640 1 -- 172.21.15.46:0/3223663347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x557b81ec4000 con 0x557b80fb3400 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedaa600640 1 -- 172.21.15.46:0/3223663347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 70) v1 ==== 2723+0+0 (secure 0 0 0) 0x557b81009180 con 0x557b80fb3400 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 4 Got MDS map 70 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 4 datascan.main: Using metadata pool output 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.459+0000 7fedb024a8c0 4 datascan.main: connecting to RADOS... 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 Processor -- start 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 -- start start 2024-04-25T08:49:51.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b81d83800 0x557b8116d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b81d83c00 0x557b8116db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81f2a000 0x557b8116e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b81ea6600 con 0x557b81f2a000 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b81ea6780 con 0x557b81d83800 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b81ea6900 con 0x557b81d83c00 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81f2a000 0x557b8116e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81f2a000 0x557b8116e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45402/0 (socket says 172.21.15.46:45402) 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedace00640 1 -- 172.21.15.46:0/2725834768 learned_addr learned my addr 172.21.15.46:0/2725834768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedae200640 1 --2- 172.21.15.46:0/2725834768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b81d83800 0x557b8116d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedad800640 1 --2- 172.21.15.46:0/2725834768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b81d83c00 0x557b8116db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedace00640 1 -- 172.21.15.46:0/2725834768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b81d83c00 msgr2=0x557b8116db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedace00640 1 --2- 172.21.15.46:0/2725834768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b81d83c00 0x557b8116db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedace00640 1 -- 172.21.15.46:0/2725834768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b81d83800 msgr2=0x557b8116d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedace00640 1 --2- 172.21.15.46:0/2725834768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b81d83800 0x557b8116d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedace00640 1 -- 172.21.15.46:0/2725834768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b80fd81e0 con 0x557b81f2a000 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedace00640 1 --2- 172.21.15.46:0/2725834768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81f2a000 0x557b8116e100 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x557b81df3ce0 tx=0x557b810543c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ba9ab1451bf0f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7feda7e00640 1 -- 172.21.15.46:0/2725834768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b80fe8b60 con 0x557b81f2a000 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7feda7e00640 1 -- 172.21.15.46:0/2725834768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b80ffcc40 con 0x557b81f2a000 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7feda7e00640 1 -- 172.21.15.46:0/2725834768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b80fe8340 con 0x557b81f2a000 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 -- 172.21.15.46:0/2725834768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81f2a000 msgr2=0x557b8116e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/2725834768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81f2a000 0x557b8116e100 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x557b81df3ce0 tx=0x557b810543c0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 -- 172.21.15.46:0/2725834768 shutdown_connections 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/2725834768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81f2a000 0x557b8116e100 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/2725834768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b81d83c00 0x557b8116db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/2725834768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b81d83800 0x557b8116d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 -- 172.21.15.46:0/2725834768 >> 172.21.15.46:0/2725834768 conn(0x557b80fb2c00 msgr2=0x557b80fe1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 -- 172.21.15.46:0/2725834768 shutdown_connections 2024-04-25T08:49:51.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 -- 172.21.15.46:0/2725834768 wait complete. 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.463+0000 7fedb024a8c0 1 Processor -- start 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 1 -- start start 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81d83c00 0x557b8116f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b81d83800 0x557b8116ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b81f2b800 0x557b8116e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b81ea7500 con 0x557b81d83c00 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b81ea6a80 con 0x557b81d83800 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b81ea7680 con 0x557b81f2b800 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedae200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81d83c00 0x557b8116f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedae200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81d83c00 0x557b8116f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45414/0 (socket says 172.21.15.46:45414) 2024-04-25T08:49:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 learned_addr learned my addr 172.21.15.46:0/3942407167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedad800640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b81d83800 0x557b8116ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b81f2b800 0x557b8116e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b81f2b800 msgr2=0x557b8116e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b81f2b800 0x557b8116e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b81d83800 msgr2=0x557b8116ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b81d83800 0x557b8116ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b80fd92c0 con 0x557b81d83c00 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81d83c00 0x557b8116f180 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x557b81ec7d10 tx=0x557b81ed4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7feda6000640 1 -- 172.21.15.46:0/3942407167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b80fe8b60 con 0x557b81d83c00 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7feda6000640 1 -- 172.21.15.46:0/3942407167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b80ffca80 con 0x557b81d83c00 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7feda6000640 1 -- 172.21.15.46:0/3942407167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b80fe8340 con 0x557b81d83c00 2024-04-25T08:49:51.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557b80fd9a40 con 0x557b81d83c00 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b80fd9c20 con 0x557b81d83c00 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7fedb024a8c0 4 datascan.init: resolving metadata pool 13 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.467+0000 7feda6000640 1 -- 172.21.15.46:0/3942407167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x557b81f36c00 con 0x557b81d83c00 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7feda6000640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x557b812f2c00 0x557b8116f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7feda6000640 1 -- 172.21.15.46:0/3942407167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x557b81f34e00 con 0x557b81d83c00 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedb024a8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedb024a8c0 4 datascan.main: resolving metadata pool 13 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557b81f2b400 0x557b8116e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2b000 con 0x557b81f2b400 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedad800640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x557b812f2c00 0x557b8116f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557b81f2b400 0x557b8116e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557b81f2b400 0x557b8116e100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedad800640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x557b812f2c00 0x557b8116f700 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x557b81df37a0 tx=0x557b813e0000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(1 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x557b81f2b000 0x557b813e6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2bc00 con 0x557b81f2b000 2024-04-25T08:49:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x557b81f2b000 0x557b813e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x557b81f2b000 0x557b813e6000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(2 [pgnls start_epoch 94 out=177b] v94'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x557b81f2bc00 0x557b813e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b812f3c00 con 0x557b81f2bc00 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedad800640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x557b81f2bc00 0x557b813e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedad800640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x557b81f2bc00 0x557b813e6580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.2 v2:172.21.15.46:6808/3102391403 1 ==== osd_op_reply(3 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2780 con 0x557b81f2bc00 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.471+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b812f3c00 con 0x557b81f2b400 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 2 ==== osd_op_reply(4 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x557b812f3c00 0x557b813e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81d83400 con 0x557b812f3c00 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x557b812f3c00 0x557b813e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x557b812f3c00 0x557b813e6b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(5 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x557b81d83400 0x557b813e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee000 con 0x557b81d83400 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x557b81d83400 0x557b813e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x557b81d83400 0x557b813e7080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(6 [pgnls start_epoch 94 out=149b] v93'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x557b810c2500 con 0x557b81d83400 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee000 con 0x557b81f2b000 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 2 ==== osd_op_reply(7 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee000 con 0x557b812f3c00 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== osd_op_reply(8 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557b813ee000 0x557b813e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee400 con 0x557b813ee000 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.475+0000 7fedad800640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557b813ee000 0x557b813e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedad800640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557b813ee000 0x557b813e7600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(9 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2780 con 0x557b813ee000 2024-04-25T08:49:51.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee400 con 0x557b81d83400 2024-04-25T08:49:51.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== osd_op_reply(10 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81d83400 2024-04-25T08:49:51.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee400 con 0x557b81f2bc00 2024-04-25T08:49:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.2 v2:172.21.15.46:6808/3102391403 2 ==== osd_op_reply(11 [pgnls start_epoch 94 out=81b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557b810c2780 con 0x557b81f2bc00 2024-04-25T08:49:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee400 con 0x557b81f2b000 2024-04-25T08:49:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 3 ==== osd_op_reply(12 [pgnls start_epoch 94 out=49b] v91'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x557b813ee400 0x557b813e8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ef400 con 0x557b813ee400 2024-04-25T08:49:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x557b813ee400 0x557b813e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x557b813ee400 0x557b813e8680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== osd_op_reply(13 [pgnls start_epoch 94 out=77b] v93'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedace00640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x557b813ef400 0x557b813e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ef400 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x557b813ef400 0x557b813e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedae200640 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x557b813ef400 0x557b813e7b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== osd_op_reply(14 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b813ef400 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.479+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b81f2b400 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 3 ==== osd_op_reply(15 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ee400 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 2 ==== osd_op_reply(16 [pgnls start_epoch 94 out=49b] v91'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b812f3c00 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 3 ==== osd_op_reply(17 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b81f2b000 2024-04-25T08:49:51.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 4 ==== osd_op_reply(18 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ee000 2024-04-25T08:49:51.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.5 v2:172.21.15.82:6816/541310136 2 ==== osd_op_reply(19 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2780 con 0x557b813ee000 2024-04-25T08:49:51.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ef400 2024-04-25T08:49:51.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.3 v2:172.21.15.46:6824/1647449001 2 ==== osd_op_reply(20 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b813ef400 2024-04-25T08:49:51.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ee400 2024-04-25T08:49:51.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 3 ==== osd_op_reply(21 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ee400 2024-04-25T08:49:51.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 4 ==== osd_op_reply(22 [pgnls start_epoch 94 out=97b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ee400 2024-04-25T08:49:51.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 5 ==== osd_op_reply(23 [pgnls start_epoch 94 out=74b] v89'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b81f2b400 2024-04-25T08:49:51.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 4 ==== osd_op_reply(24 [pgnls start_epoch 94 out=73b] v93'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.483+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b81d83400 2024-04-25T08:49:51.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.0 v2:172.21.15.46:6800/3448226631 3 ==== osd_op_reply(25 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81d83400 2024-04-25T08:49:51.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b81f2b400 2024-04-25T08:49:51.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 5 ==== osd_op_reply(26 [pgnls start_epoch 94 out=98b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ee000 2024-04-25T08:49:51.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.5 v2:172.21.15.82:6816/541310136 3 ==== osd_op_reply(27 [pgnls start_epoch 94 out=76b] v93'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x557b810c2780 con 0x557b813ee000 2024-04-25T08:49:51.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ee400 2024-04-25T08:49:51.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 6 ==== osd_op_reply(28 [pgnls start_epoch 94 out=73b] v93'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b81f2b400 2024-04-25T08:49:51.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 6 ==== osd_op_reply(29 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b813ee000 2024-04-25T08:49:51.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.5 v2:172.21.15.82:6816/541310136 4 ==== osd_op_reply(30 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2780 con 0x557b813ee000 2024-04-25T08:49:51.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b812f3c00 2024-04-25T08:49:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 4 ==== osd_op_reply(31 [pgnls start_epoch 94 out=73b] v89'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b812f3c00 2024-04-25T08:49:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 5 ==== osd_op_reply(32 [pgnls start_epoch 94 out=101b] v93'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T08:49:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2ac00 con 0x557b81f2b000 2024-04-25T08:49:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T08:49:51.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2a800 con 0x557b81f2b000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2a400 con 0x557b81f2b000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.487+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 20 datascan.scan_links: injected ino detected 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2a000 con 0x557b81f2b000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2ac00 con 0x557b81f2b000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T08:49:51.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T08:49:51.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T08:49:51.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T08:49:51.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T08:49:51.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T08:49:51.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2a800 con 0x557b81d83400 2024-04-25T08:49:51.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.0 v2:172.21.15.46:6800/3448226631 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81d83400 2024-04-25T08:49:51.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T08:49:51.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2a400 con 0x557b812f3c00 2024-04-25T08:49:51.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-25T08:49:51.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2a000 con 0x557b81f2bc00 2024-04-25T08:49:51.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.2 v2:172.21.15.46:6808/3102391403 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x557b810c2780 con 0x557b81f2bc00 2024-04-25T08:49:51.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T08:49:51.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T08:49:51.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T08:49:51.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813efc00 con 0x557b81f2b400 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ef800 con 0x557b813ee400 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140a000 con 0x557b81f2b400 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T08:49:51.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140a400 con 0x557b81f2b400 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140a800 con 0x557b81f2b400 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T08:49:51.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.491+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140ac00 con 0x557b812f3c00 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b400 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 11 ==== osd_op_reply(47 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b000 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 10 ==== osd_op_reply(48 [pgnls start_epoch 94 out=177b] v94'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2bc00 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.2 v2:172.21.15.46:6808/3102391403 4 ==== osd_op_reply(49 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2780 con 0x557b81f2bc00 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b400 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 12 ==== osd_op_reply(50 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b812f3c00 2024-04-25T08:49:51.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 8 ==== osd_op_reply(51 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81d83400 2024-04-25T08:49:51.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.0 v2:172.21.15.46:6800/3448226631 5 ==== osd_op_reply(52 [pgnls start_epoch 94 out=149b] v93'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x557b810c2500 con 0x557b81d83400 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b000 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 11 ==== osd_op_reply(53 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b812f3c00 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 9 ==== osd_op_reply(54 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.495+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee000 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.5 v2:172.21.15.82:6816/541310136 5 ==== osd_op_reply(55 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2780 con 0x557b813ee000 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81d83400 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.0 v2:172.21.15.46:6800/3448226631 6 ==== osd_op_reply(56 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81d83400 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2bc00 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.2 v2:172.21.15.46:6808/3102391403 5 ==== osd_op_reply(57 [pgnls start_epoch 94 out=81b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557b810c2780 con 0x557b81f2bc00 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b000 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 12 ==== osd_op_reply(58 [pgnls start_epoch 94 out=49b] v91'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee400 2024-04-25T08:49:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 8 ==== osd_op_reply(59 [pgnls start_epoch 94 out=77b] v93'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ef400 2024-04-25T08:49:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.3 v2:172.21.15.46:6824/1647449001 3 ==== osd_op_reply(60 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b813ef400 2024-04-25T08:49:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b400 2024-04-25T08:49:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 13 ==== osd_op_reply(61 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee400 2024-04-25T08:49:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 9 ==== osd_op_reply(62 [pgnls start_epoch 94 out=49b] v91'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b812f3c00 2024-04-25T08:49:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 10 ==== osd_op_reply(63 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b000 2024-04-25T08:49:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 13 ==== osd_op_reply(64 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.499+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee000 2024-04-25T08:49:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.5 v2:172.21.15.82:6816/541310136 6 ==== osd_op_reply(65 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2780 con 0x557b813ee000 2024-04-25T08:49:51.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ef400 2024-04-25T08:49:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.3 v2:172.21.15.46:6824/1647449001 4 ==== osd_op_reply(66 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b813ef400 2024-04-25T08:49:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee400 2024-04-25T08:49:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 10 ==== osd_op_reply(67 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee400 2024-04-25T08:49:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 11 ==== osd_op_reply(68 [pgnls start_epoch 94 out=97b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee400 2024-04-25T08:49:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 12 ==== osd_op_reply(69 [pgnls start_epoch 94 out=74b] v89'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b400 2024-04-25T08:49:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 14 ==== osd_op_reply(70 [pgnls start_epoch 94 out=73b] v93'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81d83400 2024-04-25T08:49:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.0 v2:172.21.15.46:6800/3448226631 7 ==== osd_op_reply(71 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2500 con 0x557b81d83400 2024-04-25T08:49:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.503+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b400 2024-04-25T08:49:51.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 15 ==== osd_op_reply(72 [pgnls start_epoch 94 out=98b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee000 2024-04-25T08:49:51.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.5 v2:172.21.15.82:6816/541310136 7 ==== osd_op_reply(73 [pgnls start_epoch 94 out=76b] v93'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x557b810c2780 con 0x557b813ee000 2024-04-25T08:49:51.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee400 2024-04-25T08:49:51.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 13 ==== osd_op_reply(74 [pgnls start_epoch 94 out=73b] v93'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b81f2b400 2024-04-25T08:49:51.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 16 ==== osd_op_reply(75 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b813ee000 2024-04-25T08:49:51.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.5 v2:172.21.15.82:6816/541310136 8 ==== osd_op_reply(76 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b810c2780 con 0x557b813ee000 2024-04-25T08:49:51.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b812f3c00 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 11 ==== osd_op_reply(77 [pgnls start_epoch 94 out=73b] v89'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b000 con 0x557b812f3c00 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 12 ==== osd_op_reply(78 [pgnls start_epoch 94 out=101b] v93'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b400 con 0x557b81f2b000 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b800 con 0x557b81f2b000 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140bc00 con 0x557b81f2b000 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 20 datascan.scan_links: injected ino detected 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140a000 con 0x557b81f2b000 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ef000 con 0x557b81f2b000 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T08:49:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.507+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813eec00 con 0x557b81d83400 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.0 v2:172.21.15.46:6800/3448226631 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2500 con 0x557b81d83400 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b812f3c00 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-25T08:49:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813efc00 con 0x557b81f2bc00 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 <== osd.2 v2:172.21.15.46:6808/3102391403 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x557b810c2780 con 0x557b81f2bc00 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ef800 con 0x557b81f2b400 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2a400 con 0x557b813ee400 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T08:49:51.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ef000 con 0x557b81f2b400 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b400 con 0x557b81f2b400 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140b800 con 0x557b81f2b400 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140bc00 con 0x557b812f3c00 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.6 v2:172.21.15.82:6823/1178952145 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x557b810c2000 con 0x557b812f3c00 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b8140a000 con 0x557b81f2b000 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.511+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2ac00 con 0x557b81f2b000 2024-04-25T08:49:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v94'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T08:49:51.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T08:49:51.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-25T08:49:51.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-25T08:49:51.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2a800 con 0x557b81f2b000 2024-04-25T08:49:51.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T08:49:51.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 20 datascan.scan_links: first is now 2 2024-04-25T08:49:51.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ee800 con 0x557b81f2b000 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedae200640 1 -- 172.21.15.46:0/3942407167 <== osd.1 v2:172.21.15.46:6809/2552657765 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v94'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557b810c2500 con 0x557b81f2b000 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 10 datascan.scan_links: updating inotable 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ef800 con 0x557b81f2b400 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.515+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b813ef000 con 0x557b81f2b400 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.7 v2:172.21.15.82:6808/1420997799 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v94'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x557b810c2000 con 0x557b81f2b400 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 10 datascan.scan_links: updating snaptable 2024-04-25T08:49:51.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v9 -- 0x557b81f2a400 con 0x557b813ee400 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedace00640 1 -- 172.21.15.46:0/3942407167 <== osd.4 v2:172.21.15.82:6800/2293006647 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x557b810c2000 con 0x557b813ee400 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x557b81d83400 msgr2=0x557b813e7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x557b81d83400 0x557b813e7080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x557b81f2b000 msgr2=0x557b813e6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x557b81f2b000 0x557b813e6000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x557b81f2bc00 msgr2=0x557b813e6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x557b81f2bc00 0x557b813e6580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x557b813ef400 msgr2=0x557b813e7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x557b813ef400 0x557b813e7b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x557b813ee400 msgr2=0x557b813e8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x557b813ee400 0x557b813e8680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557b813ee000 msgr2=0x557b813e7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x557b813ee000 0x557b813e7600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 reap_dead start 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x557b812f3c00 msgr2=0x557b813e6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x557b812f3c00 0x557b813e6b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557b81f2b400 msgr2=0x557b8116e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x557b81f2b400 0x557b8116e100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 reap_dead start 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedad800640 1 -- 172.21.15.46:0/3942407167 reap_dead start 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x557b812f2c00 msgr2=0x557b8116f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x557b812f2c00 0x557b8116f700 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x557b81df37a0 tx=0x557b813e0000 comp rx=0 tx=0).stop 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81d83c00 msgr2=0x557b8116f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81d83c00 0x557b8116f180 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x557b81ec7d10 tx=0x557b81ed4c00 comp rx=0 tx=0).stop 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 shutdown_connections 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x557b812f2c00 0x557b8116f700 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3942407167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b81d83c00 0x557b8116f180 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 >> 172.21.15.46:0/3942407167 conn(0x557b80fb2c00 msgr2=0x557b80fe1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 shutdown_connections 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3942407167 wait complete. 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3223663347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b80fb3400 msgr2=0x557b8116cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b80fb3400 0x557b8116cb00 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x557b81df38c0 tx=0x557b81054000 comp rx=0 tx=0).stop 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3223663347 shutdown_connections 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x557b80fb3000 0x557b8116d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557b80fb3400 0x557b8116cb00 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 --2- 172.21.15.46:0/3223663347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557b80fb3800 0x557b8116c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3223663347 >> 172.21.15.46:0/3223663347 conn(0x557b80fb3c00 msgr2=0x557b80fe1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3223663347 shutdown_connections 2024-04-25T08:49:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.519+0000 7fedb024a8c0 1 -- 172.21.15.46:0/3223663347 wait complete. 2024-04-25T08:49:51.523 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.156243, result: 2024-04-25T08:49:51.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T08:49:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.715+0000 7fd538e00640 1 Processor -- start 2024-04-25T08:49:51.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.715+0000 7fd538e00640 1 -- start start 2024-04-25T08:49:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd538e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd53406f150 0x7fd5340cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd538e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53415e440 con 0x7fd5341440d0 2024-04-25T08:49:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd538e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53415e6e0 con 0x7fd534143990 2024-04-25T08:49:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd538e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53415ed30 con 0x7fd53406f150 2024-04-25T08:49:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd52be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd5341440d0 0x7fd5340b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35658/0 (socket says 172.21.15.46:35658) 2024-04-25T08:49:51.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd52be00640 1 -- 172.21.15.46:0/1147698540 learned_addr learned my addr 172.21.15.46:0/1147698540 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd532800640 1 --2- 172.21.15.46:0/1147698540 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd53406f150 0x7fd5340cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd531400640 1 -- 172.21.15.46:0/1147698540 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617890720 0 0) 0x7fd53415e440 con 0x7fd5341440d0 2024-04-25T08:49:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd531400640 1 -- 172.21.15.46:0/1147698540 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd518003530 con 0x7fd5341440d0 2024-04-25T08:49:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd531400640 1 -- 172.21.15.46:0/1147698540 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321634442 0 0) 0x7fd53415e6e0 con 0x7fd534143990 2024-04-25T08:49:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd531400640 1 -- 172.21.15.46:0/1147698540 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd53415e440 con 0x7fd534143990 2024-04-25T08:49:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd532800640 1 -- 172.21.15.46:0/1147698540 >> v1:172.21.15.82:6789/0 conn(0x7fd534143990 legacy=0x7fd53406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd532800640 1 -- 172.21.15.46:0/1147698540 >> v1:172.21.15.46:6789/0 conn(0x7fd5341440d0 legacy=0x7fd5340b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.719+0000 7fd532800640 1 -- 172.21.15.46:0/1147698540 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53415e9c0 con 0x7fd53406f150 2024-04-25T08:49:51.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd532800640 1 --2- 172.21.15.46:0/1147698540 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd53406f150 0x7fd5340cecc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd53415eeb0 tx=0x7fd52c01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c3ee314b862763b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd531400640 1 -- 172.21.15.46:0/1147698540 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0b2070 con 0x7fd53406f150 2024-04-25T08:49:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd531400640 1 -- 172.21.15.46:0/1147698540 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52c0a5850 con 0x7fd53406f150 2024-04-25T08:49:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd531400640 1 -- 172.21.15.46:0/1147698540 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c0a5c20 con 0x7fd53406f150 2024-04-25T08:49:51.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd538e00640 1 -- 172.21.15.46:0/1147698540 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd53406f150 msgr2=0x7fd5340cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd538e00640 1 --2- 172.21.15.46:0/1147698540 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd53406f150 0x7fd5340cecc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd53415eeb0 tx=0x7fd52c01c990 comp rx=0 tx=0).stop 2024-04-25T08:49:51.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd538e00640 1 -- 172.21.15.46:0/1147698540 shutdown_connections 2024-04-25T08:49:51.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd538e00640 1 --2- 172.21.15.46:0/1147698540 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd53406f150 0x7fd5340cecc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd538e00640 1 -- 172.21.15.46:0/1147698540 >> 172.21.15.46:0/1147698540 conn(0x7fd53406ac10 msgr2=0x7fd53406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:51.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd538e00640 1 -- 172.21.15.46:0/1147698540 shutdown_connections 2024-04-25T08:49:51.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd538e00640 1 -- 172.21.15.46:0/1147698540 wait complete. 2024-04-25T08:49:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd538e00640 1 Processor -- start 2024-04-25T08:49:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.723+0000 7fd538e00640 1 -- start start 2024-04-25T08:49:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd538e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd53406f150 0x7fd534138b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd538e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd534143990 0x7fd534141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd538e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5341440d0 0x7fd5340b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd538e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340713f0 con 0x7fd53406f150 2024-04-25T08:49:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd538e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd534071270 con 0x7fd534143990 2024-04-25T08:49:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd538e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd534071570 con 0x7fd5341440d0 2024-04-25T08:49:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd531e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd53406f150 0x7fd534138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd531e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd53406f150 0x7fd534138b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45418/0 (socket says 172.21.15.46:45418) 2024-04-25T08:49:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd531e00640 1 -- 172.21.15.46:0/3461326833 learned_addr learned my addr 172.21.15.46:0/3461326833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd52be00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd534143990 0x7fd534141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd532800640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5341440d0 0x7fd5340b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd531e00640 1 -- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5341440d0 msgr2=0x7fd5340b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd531e00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5341440d0 0x7fd5340b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd531e00640 1 -- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd534143990 msgr2=0x7fd534141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd531e00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd534143990 0x7fd534141520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd531e00640 1 -- 172.21.15.46:0/3461326833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd534138910 con 0x7fd53406f150 2024-04-25T08:49:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd531e00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd53406f150 0x7fd534138b40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c530 tx=0x7fd52400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd52a000640 1 -- 172.21.15.46:0/3461326833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240052d0 con 0x7fd53406f150 2024-04-25T08:49:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd52a000640 1 -- 172.21.15.46:0/3461326833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd524005490 con 0x7fd53406f150 2024-04-25T08:49:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd52a000640 1 -- 172.21.15.46:0/3461326833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd524012850 con 0x7fd53406f150 2024-04-25T08:49:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5340aff20 con 0x7fd53406f150 2024-04-25T08:49:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5340b04a0 con 0x7fd53406f150 2024-04-25T08:49:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd534069010 con 0x7fd53406f150 2024-04-25T08:49:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.727+0000 7fd52a000640 1 -- 172.21.15.46:0/3461326833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5240129f0 con 0x7fd53406f150 2024-04-25T08:49:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.731+0000 7fd52a000640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd510066f90 0x7fd510069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.731+0000 7fd52a000640 1 -- 172.21.15.46:0/3461326833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd524084e70 con 0x7fd53406f150 2024-04-25T08:49:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.731+0000 7fd52be00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd510066f90 0x7fd510069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:51.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.731+0000 7fd52be00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd510066f90 0x7fd510069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd534142cd0 tx=0x7fd51c007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:51.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.735+0000 7fd52a000640 1 -- 172.21.15.46:0/3461326833 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5240554f0 con 0x7fd53406f150 2024-04-25T08:49:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:51.931+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fd53406eb10 con 0x7fd53406f150 2024-04-25T08:49:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd52a000640 1 -- 172.21.15.46:0/3461326833 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v71) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd524059590 con 0x7fd53406f150 2024-04-25T08:49:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T08:49:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd510066f90 msgr2=0x7fd510069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd510066f90 0x7fd510069450 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd534142cd0 tx=0x7fd51c007d40 comp rx=0 tx=0).stop 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd53406f150 msgr2=0x7fd534138b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd53406f150 0x7fd534138b40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c530 tx=0x7fd52400c9f0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 shutdown_connections 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd510066f90 0x7fd510069450 secure :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd534142cd0 tx=0x7fd51c007d40 comp rx=0 tx=0).stop 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd5341440d0 0x7fd5340b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd534143990 0x7fd534141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 --2- 172.21.15.46:0/3461326833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd53406f150 0x7fd534138b40 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 >> 172.21.15.46:0/3461326833 conn(0x7fd53406ac10 msgr2=0x7fd5340ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 shutdown_connections 2024-04-25T08:49:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.035+0000 7fd538e00640 1 -- 172.21.15.46:0/3461326833 wait complete. 2024-04-25T08:49:52.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:49:52.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.199+0000 7f7c0d000640 1 Processor -- start 2024-04-25T08:49:52.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.199+0000 7f7c0d000640 1 -- start start 2024-04-25T08:49:52.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c0d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c0815ef20 0x7f7c080cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c0d000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c08099a00 0x7f7c08099eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c0d000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0815f300 con 0x7f7c080cd6d0 2024-04-25T08:49:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c0d000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c080cddb0 con 0x7f7c08099a00 2024-04-25T08:49:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c0d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080cc3d0 con 0x7f7c0815ef20 2024-04-25T08:49:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c05e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c0815ef20 0x7f7c080cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c06800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c08099a00 0x7f7c08099eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:52.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7bffe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7c080cd6d0 0x7f7c0809a610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:51232/0 (socket says 172.21.15.46:51232) 2024-04-25T08:49:52.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7bffe00640 1 -- 172.21.15.46:0/1415711999 learned_addr learned my addr 172.21.15.46:0/1415711999 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:52.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c06800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c08099a00 0x7f7c08099eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:42888/0 (socket says 172.21.15.46:42888) 2024-04-25T08:49:52.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c06800640 1 -- 172.21.15.46:0/1415711999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c0815ef20 msgr2=0x7f7c080cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:52.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c06800640 1 --2- 172.21.15.46:0/1415711999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c0815ef20 0x7f7c080cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c06800640 1 -- 172.21.15.46:0/1415711999 >> v1:172.21.15.82:6789/0 conn(0x7f7c080cd6d0 legacy=0x7f7c0809a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:52.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.203+0000 7f7c06800640 1 -- 172.21.15.46:0/1415711999 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c080cc6b0 con 0x7f7c08099a00 2024-04-25T08:49:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c06800640 1 --2- 172.21.15.46:0/1415711999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c08099a00 0x7f7c08099eb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009d90 tx=0x7f7c0001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff55565af9df5f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c05400640 1 -- 172.21.15.46:0/1415711999 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c000b2070 con 0x7f7c08099a00 2024-04-25T08:49:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c05400640 1 -- 172.21.15.46:0/1415711999 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c000a5920 con 0x7f7c08099a00 2024-04-25T08:49:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c05400640 1 -- 172.21.15.46:0/1415711999 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c000a5cf0 con 0x7f7c08099a00 2024-04-25T08:49:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 -- 172.21.15.46:0/1415711999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c08099a00 msgr2=0x7f7c08099eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 --2- 172.21.15.46:0/1415711999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c08099a00 0x7f7c08099eb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009d90 tx=0x7f7c0001c970 comp rx=0 tx=0).stop 2024-04-25T08:49:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 -- 172.21.15.46:0/1415711999 shutdown_connections 2024-04-25T08:49:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 --2- 172.21.15.46:0/1415711999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c08099a00 0x7f7c08099eb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 --2- 172.21.15.46:0/1415711999 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c0815ef20 0x7f7c080cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 -- 172.21.15.46:0/1415711999 >> 172.21.15.46:0/1415711999 conn(0x7f7c080b3120 msgr2=0x7f7c080b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 -- 172.21.15.46:0/1415711999 shutdown_connections 2024-04-25T08:49:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 -- 172.21.15.46:0/1415711999 wait complete. 2024-04-25T08:49:52.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 Processor -- start 2024-04-25T08:49:52.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.207+0000 7f7c0d000640 1 -- start start 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c0d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c08099a00 0x7f7c0814eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c0d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0815ef20 0x7f7c080b6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c0d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c0815f4d0 0x7f7c0815f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c0d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0809aea0 con 0x7f7c0815f4d0 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c0d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0809ad20 con 0x7f7c0815ef20 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c0d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c08168880 con 0x7f7c08099a00 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c05e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c08099a00 0x7f7c0814eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c06800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c0815f4d0 0x7f7c0815f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7bffe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0815ef20 0x7f7c080b6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c05e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c08099a00 0x7f7c0814eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:44604/0 (socket says 172.21.15.46:44604) 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c06800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c0815f4d0 0x7f7c0815f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45430/0 (socket says 172.21.15.46:45430) 2024-04-25T08:49:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c05e00640 1 -- 172.21.15.46:0/1205371941 learned_addr learned my addr 172.21.15.46:0/1205371941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c06800640 1 -- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c08099a00 msgr2=0x7f7c0814eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c06800640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c08099a00 0x7f7c0814eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c06800640 1 -- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0815ef20 msgr2=0x7f7c080b6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c06800640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0815ef20 0x7f7c080b6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c06800640 1 -- 172.21.15.46:0/1205371941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c080b5ad0 con 0x7f7c0815f4d0 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c05e00640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c08099a00 0x7f7c0814eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7bffe00640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0815ef20 0x7f7c080b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c06800640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c0815f4d0 0x7f7c0815f960 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7c000ac000 tx=0x7f7c00019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7bfe000640 1 -- 172.21.15.46:0/1205371941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c000b2050 con 0x7f7c0815f4d0 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7bfe000640 1 -- 172.21.15.46:0/1205371941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c000b3040 con 0x7f7c0815f4d0 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0814f020 con 0x7f7c0815f4d0 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7bfe000640 1 -- 172.21.15.46:0/1205371941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c000ad870 con 0x7f7c0815f4d0 2024-04-25T08:49:52.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c080b6050 con 0x7f7c0815f4d0 2024-04-25T08:49:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c08069080 con 0x7f7c0815f4d0 2024-04-25T08:49:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.211+0000 7f7bfe000640 1 -- 172.21.15.46:0/1205371941 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c000c0020 con 0x7f7c0815f4d0 2024-04-25T08:49:52.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.215+0000 7f7bfe000640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7be4066f60 0x7f7be4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:52.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.215+0000 7f7bfe000640 1 -- 172.21.15.46:0/1205371941 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f7c0011f800 con 0x7f7c0815f4d0 2024-04-25T08:49:52.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.215+0000 7f7c05e00640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7be4066f60 0x7f7be4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:52.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.215+0000 7f7c05e00640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7be4066f60 0x7f7be4069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8011310 tx=0x7f7bf8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:52.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.219+0000 7f7bfe000640 1 -- 172.21.15.46:0/1205371941 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c000c7020 con 0x7f7c0815f4d0 2024-04-25T08:49:52.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.415+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c080cbc10 con 0x7f7c0815f4d0 2024-04-25T08:49:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.415+0000 7f7bfe000640 1 -- 172.21.15.46:0/1205371941 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f7c00016560 con 0x7f7c0815f4d0 2024-04-25T08:49:52.412 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:52.412 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:49:52.042682+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"c","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/1358086076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1358086076},{"type":"v1","addr":"172.21.15.46:6835","nonce":1358086076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:49:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 72 2024-04-25T08:49:52.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7be4066f60 msgr2=0x7f7be4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7be4066f60 0x7f7be4069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8011310 tx=0x7f7bf8009190 comp rx=0 tx=0).stop 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c0815f4d0 msgr2=0x7f7c0815f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c0815f4d0 0x7f7c0815f960 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7c000ac000 tx=0x7f7c00019710 comp rx=0 tx=0).stop 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 shutdown_connections 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7be4066f60 0x7f7be4069420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c0815f4d0 0x7f7c0815f960 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0815ef20 0x7f7c080b6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 --2- 172.21.15.46:0/1205371941 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7c08099a00 0x7f7c0814eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 >> 172.21.15.46:0/1205371941 conn(0x7f7c080b3120 msgr2=0x7f7c0809a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 shutdown_connections 2024-04-25T08:49:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:52.419+0000 7f7c0d000640 1 -- 172.21.15.46:0/1205371941 wait complete. 2024-04-25T08:49:52.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:48:27.728697+0000', 'modified': '2024-04-25T08:49:52.042682+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7962}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7962': {'gid': 7962, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/1358086076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1358086076}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1358086076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:49:52.836 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7962:up:replay 2024-04-25T08:49:53.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:49:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.991+0000 7f0ace000640 1 Processor -- start 2024-04-25T08:49:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.991+0000 7f0ace000640 1 -- start start 2024-04-25T08:49:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ace000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac8099a00 0x7f0ac8099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ace000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ac80b2240 0x7f0ac80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ace000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac809a320 con 0x7f0ac80ccde0 2024-04-25T08:49:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ace000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac806db90 con 0x7f0ac80b2240 2024-04-25T08:49:53.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ace000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac806bdc0 con 0x7f0ac8099a00 2024-04-25T08:49:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0abea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0ac80ccde0 0x7f0ac80cd260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35662/0 (socket says 172.21.15.46:35662) 2024-04-25T08:49:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0abea00640 1 -- 172.21.15.46:0/1369013369 learned_addr learned my addr 172.21.15.46:0/1369013369 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ac7e00640 1 --2- 172.21.15.46:0/1369013369 >> v2:172.21.15.46:3300/0 conn(0x7f0ac80b2240 0x7f0ac80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ac7400640 1 --2- 172.21.15.46:0/1369013369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac8099a00 0x7f0ac8099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:53.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ac6a00640 1 -- 172.21.15.46:0/1369013369 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631136587 0 0) 0x7f0ac809a320 con 0x7f0ac80ccde0 2024-04-25T08:49:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ac6a00640 1 -- 172.21.15.46:0/1369013369 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aac003530 con 0x7f0ac80ccde0 2024-04-25T08:49:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ac7e00640 1 -- 172.21.15.46:0/1369013369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac8099a00 msgr2=0x7f0ac8099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ac7e00640 1 --2- 172.21.15.46:0/1369013369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac8099a00 0x7f0ac8099de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ac7e00640 1 -- 172.21.15.46:0/1369013369 >> v1:172.21.15.46:6789/0 conn(0x7f0ac80ccde0 legacy=0x7f0ac80cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ac7e00640 1 -- 172.21.15.46:0/1369013369 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac806c0a0 con 0x7f0ac80b2240 2024-04-25T08:49:53.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.995+0000 7f0ac7400640 1 --2- 172.21.15.46:0/1369013369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac8099a00 0x7f0ac8099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ac7e00640 1 --2- 172.21.15.46:0/1369013369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 0x7f0ac80b26d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009d90 tx=0x7f0ac001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d886f951ab2855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ac6a00640 1 -- 172.21.15.46:0/1369013369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00b2070 con 0x7f0ac80b2240 2024-04-25T08:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ac6a00640 1 -- 172.21.15.46:0/1369013369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac00a5890 con 0x7f0ac80b2240 2024-04-25T08:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ac6a00640 1 -- 172.21.15.46:0/1369013369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00a5c60 con 0x7f0ac80b2240 2024-04-25T08:49:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 -- 172.21.15.46:0/1369013369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 msgr2=0x7f0ac80b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 --2- 172.21.15.46:0/1369013369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 0x7f0ac80b26d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009d90 tx=0x7f0ac001c8e0 comp rx=0 tx=0).stop 2024-04-25T08:49:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 -- 172.21.15.46:0/1369013369 shutdown_connections 2024-04-25T08:49:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 --2- 172.21.15.46:0/1369013369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 0x7f0ac80b26d0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 --2- 172.21.15.46:0/1369013369 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac8099a00 0x7f0ac8099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 -- 172.21.15.46:0/1369013369 >> 172.21.15.46:0/1369013369 conn(0x7f0ac80b3120 msgr2=0x7f0ac80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 -- 172.21.15.46:0/1369013369 shutdown_connections 2024-04-25T08:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 -- 172.21.15.46:0/1369013369 wait complete. 2024-04-25T08:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 Processor -- start 2024-04-25T08:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 -- start start 2024-04-25T08:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:53.999+0000 7f0ace000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8099a00 0x7f0ac8158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ace000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 0x7f0ac80ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ace000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac80ccde0 0x7f0ac80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ace000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac815dd10 con 0x7f0ac80b2240 2024-04-25T08:49:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ace000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80cdb80 con 0x7f0ac8099a00 2024-04-25T08:49:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ace000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac815de90 con 0x7f0ac80ccde0 2024-04-25T08:49:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 0x7f0ac80ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 0x7f0ac80ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45442/0 (socket says 172.21.15.46:45442) 2024-04-25T08:49:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abea00640 1 -- 172.21.15.46:0/1037912482 learned_addr learned my addr 172.21.15.46:0/1037912482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ac7e00640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac80ccde0 0x7f0ac80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ac7400640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8099a00 0x7f0ac8158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abea00640 1 -- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac80ccde0 msgr2=0x7f0ac80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abea00640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac80ccde0 0x7f0ac80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abea00640 1 -- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8099a00 msgr2=0x7f0ac8158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abea00640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8099a00 0x7f0ac8158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abea00640 1 -- 172.21.15.46:0/1037912482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac8159320 con 0x7f0ac80b2240 2024-04-25T08:49:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ac7400640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8099a00 0x7f0ac8158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abea00640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 0x7f0ac80ced50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c450 tx=0x7f0ab000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abfe00640 1 -- 172.21.15.46:0/1037912482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0017070 con 0x7f0ac80b2240 2024-04-25T08:49:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abfe00640 1 -- 172.21.15.46:0/1037912482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0005cf0 con 0x7f0ac80b2240 2024-04-25T08:49:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abfe00640 1 -- 172.21.15.46:0/1037912482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0007290 con 0x7f0ac80b2240 2024-04-25T08:49:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac815a230 con 0x7f0ac80b2240 2024-04-25T08:49:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac815ad10 con 0x7f0ac80b2240 2024-04-25T08:49:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac815a020 con 0x7f0ac80b2240 2024-04-25T08:49:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.003+0000 7f0abfe00640 1 -- 172.21.15.46:0/1037912482 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ab001a070 con 0x7f0ac80b2240 2024-04-25T08:49:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.007+0000 7f0abfe00640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0aa4066f60 0x7f0aa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.007+0000 7f0abfe00640 1 -- 172.21.15.46:0/1037912482 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f0ab0084610 con 0x7f0ac80b2240 2024-04-25T08:49:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.007+0000 7f0ac7400640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0aa4066f60 0x7f0aa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.007+0000 7f0ac7400640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0aa4066f60 0x7f0aa4069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80b7480 tx=0x7f0ab8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.007+0000 7f0abfe00640 1 -- 172.21.15.46:0/1037912482 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ab0054c80 con 0x7f0ac80b2240 2024-04-25T08:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.203+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ac8069080 con 0x7f0ac80b2240 2024-04-25T08:49:54.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0abfe00640 1 -- 172.21.15.46:0/1037912482 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0ab0058d20 con 0x7f0ac80b2240 2024-04-25T08:49:54.201 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:54.201 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:49:53.051153+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"c","rank":0,"incarnation":72,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/1358086076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1358086076},{"type":"v1","addr":"172.21.15.46:6835","nonce":1358086076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:49:54.201 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 74 2024-04-25T08:49:54.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0aa4066f60 msgr2=0x7f0aa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0aa4066f60 0x7f0aa4069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80b7480 tx=0x7f0ab8008040 comp rx=0 tx=0).stop 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 msgr2=0x7f0ac80ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 0x7f0ac80ced50 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c450 tx=0x7f0ab000c910 comp rx=0 tx=0).stop 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 shutdown_connections 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0aa4066f60 0x7f0aa4069420 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ac80ccde0 0x7f0ac80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ac80b2240 0x7f0ac80ced50 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 --2- 172.21.15.46:0/1037912482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ac8099a00 0x7f0ac8158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 >> 172.21.15.46:0/1037912482 conn(0x7f0ac80b3120 msgr2=0x7f0ac8073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 shutdown_connections 2024-04-25T08:49:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:54.207+0000 7f0ace000640 1 -- 172.21.15.46:0/1037912482 wait complete. 2024-04-25T08:49:54.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:48:27.728697+0000', 'modified': '2024-04-25T08:49:53.051153+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7962}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7962': {'gid': 7962, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/1358086076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1358086076}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1358086076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:49:54.625 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7962:up:rejoin 2024-04-25T08:49:55.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:49:55.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6ee1800640 1 Processor -- start 2024-04-25T08:49:55.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6ee1800640 1 -- start start 2024-04-25T08:49:55.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6ee1800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6edc09a110 0x7f6edc09aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:55.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6ee1800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6edc15f300 con 0x7f6edc15ef20 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6ee1800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6edc0cd8a0 con 0x7f6edc09a610 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6ee1800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6edc0cddb0 con 0x7f6edc09a110 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6edaa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6edc15ef20 0x7f6edc0cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35668/0 (socket says 172.21.15.46:35668) 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6edaa00640 1 -- 172.21.15.46:0/2534321836 learned_addr learned my addr 172.21.15.46:0/2534321836 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6edb400640 1 --2- 172.21.15.46:0/2534321836 >> v2:172.21.15.82:3300/0 conn(0x7f6edc09a110 0x7f6edc09aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626739736 0 0) 0x7f6edc15f300 con 0x7f6edc15ef20 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec0003530 con 0x7f6edc15ef20 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2345118432 0 0) 0x7f6ec0003530 con 0x7f6edc15ef20 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 >> v2:172.21.15.82:3300/0 conn(0x7f6edc09a110 msgr2=0x7f6edc09aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 --2- 172.21.15.46:0/2534321836 >> v2:172.21.15.82:3300/0 conn(0x7f6edc09a110 0x7f6edc09aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 >> v1:172.21.15.82:6789/0 conn(0x7f6edc09a610 legacy=0x7f6edc099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc0cc3d0 con 0x7f6edc15ef20 2024-04-25T08:49:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f6ecc001680 con 0x7f6edc15ef20 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 >> v1:172.21.15.46:6789/0 conn(0x7f6edc15ef20 legacy=0x7f6edc0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ec00039c0 0x7f6ec0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec0004810 0x7f6ec0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6ec000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0cddb0 con 0x7f6ec00051d0 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec000c070 con 0x7f6ec0004810 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.783+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec000c270 con 0x7f6ec00039c0 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6edb400640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6ec000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6edaa00640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ec00039c0 0x7f6ec0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec0004810 0x7f6ec0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6edb400640 1 -- 172.21.15.46:0/2534321836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ec00039c0 msgr2=0x7f6ec0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6edb400640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ec00039c0 0x7f6ec0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6edb400640 1 -- 172.21.15.46:0/2534321836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec0004810 msgr2=0x7f6ec0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6edb400640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ec0004810 0x7f6ec0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6edb400640 1 -- 172.21.15.46:0/2534321836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec0003530 con 0x7f6ec00051d0 2024-04-25T08:49:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6edb400640 1 -- 172.21.15.46:0/2534321836 reap_dead start 2024-04-25T08:49:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6edb400640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6ec000b9d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6edc071e30 tx=0x7f6ed401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb7596373bf8b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40b2070 con 0x7f6ec00051d0 2024-04-25T08:49:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed40a5c70 con 0x7f6ec00051d0 2024-04-25T08:49:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.787+0000 7f6eda000640 1 -- 172.21.15.46:0/2534321836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4009c10 con 0x7f6ec00051d0 2024-04-25T08:49:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 -- 172.21.15.46:0/2534321836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 msgr2=0x7f6ec000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6ec000b9d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6edc071e30 tx=0x7f6ed401cc60 comp rx=0 tx=0).stop 2024-04-25T08:49:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 -- 172.21.15.46:0/2534321836 shutdown_connections 2024-04-25T08:49:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 --2- 172.21.15.46:0/2534321836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6ec000b9d0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 -- 172.21.15.46:0/2534321836 >> 172.21.15.46:0/2534321836 conn(0x7f6edc0b3120 msgr2=0x7f6edc0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 -- 172.21.15.46:0/2534321836 shutdown_connections 2024-04-25T08:49:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 -- 172.21.15.46:0/2534321836 wait complete. 2024-04-25T08:49:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 Processor -- start 2024-04-25T08:49:55.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 -- start start 2024-04-25T08:49:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6edc150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:55.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6edc153220 0x7f6edc14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6edc0b5090 0x7f6edc0c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0ccc70 con 0x7f6ec00051d0 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0cc990 con 0x7f6edc0b5090 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ee1800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc0cc010 con 0x7f6edc153220 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6edc150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6edc150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45460/0 (socket says 172.21.15.46:45460) 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edaa00640 1 -- 172.21.15.46:0/364797768 learned_addr learned my addr 172.21.15.46:0/364797768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6edc153220 0x7f6edc14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edb400640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6edc0b5090 0x7f6edc0c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edaa00640 1 -- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6edc153220 msgr2=0x7f6edc14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edaa00640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6edc153220 0x7f6edc14faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edaa00640 1 -- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6edc0b5090 msgr2=0x7f6edc0c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edaa00640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6edc0b5090 0x7f6edc0c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edaa00640 1 -- 172.21.15.46:0/364797768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc150ad0 con 0x7f6ec00051d0 2024-04-25T08:49:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.791+0000 7f6edaa00640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6edc150590 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc001060 tx=0x7f6ecc007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ed2a00640 1 -- 172.21.15.46:0/364797768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00f9c0 con 0x7f6ec00051d0 2024-04-25T08:49:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ed2a00640 1 -- 172.21.15.46:0/364797768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc018070 con 0x7f6ec00051d0 2024-04-25T08:49:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ed2a00640 1 -- 172.21.15.46:0/364797768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00fdc0 con 0x7f6ec00051d0 2024-04-25T08:49:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc0b5990 con 0x7f6ec00051d0 2024-04-25T08:49:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc0ca110 con 0x7f6ec00051d0 2024-04-25T08:49:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6edc069080 con 0x7f6ec00051d0 2024-04-25T08:49:55.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ed2a00640 1 -- 172.21.15.46:0/364797768 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ecc01d460 con 0x7f6ec00051d0 2024-04-25T08:49:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ed2a00640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6eb8066f60 0x7f6eb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ed2a00640 1 -- 172.21.15.46:0/364797768 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f6ecc0853f0 con 0x7f6ec00051d0 2024-04-25T08:49:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.795+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6eb8066f60 0x7f6eb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.799+0000 7f6ed3e00640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6eb8066f60 0x7f6eb8069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0b6ef0 tx=0x7f6ec4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:55.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.799+0000 7f6ed2a00640 1 -- 172.21.15.46:0/364797768 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ecc0560b0 con 0x7f6ec00051d0 2024-04-25T08:49:55.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.991+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6edc0d0700 con 0x7f6ec00051d0 2024-04-25T08:49:55.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ed2a00640 1 -- 172.21.15.46:0/364797768 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f6ecc0559b0 con 0x7f6ec00051d0 2024-04-25T08:49:55.988 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:49:55.989 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:49:54.058324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1358086076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1358086076},{"type":"v1","addr":"172.21.15.46:6835","nonce":1358086076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:49:55.989 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 75 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6eb8066f60 msgr2=0x7f6eb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6eb8066f60 0x7f6eb8069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0b6ef0 tx=0x7f6ec4009040 comp rx=0 tx=0).stop 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 msgr2=0x7f6edc150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6edc150590 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc001060 tx=0x7f6ecc007720 comp rx=0 tx=0).stop 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 shutdown_connections 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6eb8066f60 0x7f6eb8069420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6edc0b5090 0x7f6edc0c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6edc153220 0x7f6edc14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 --2- 172.21.15.46:0/364797768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ec00051d0 0x7f6edc150590 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 >> 172.21.15.46:0/364797768 conn(0x7f6edc0b3120 msgr2=0x7f6edc073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 shutdown_connections 2024-04-25T08:49:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:55.995+0000 7f6ee1800640 1 -- 172.21.15.46:0/364797768 wait complete. 2024-04-25T08:49:56.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:48:27.728697+0000', 'modified': '2024-04-25T08:49:54.058324+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7962}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7962': {'gid': 7962, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/1358086076', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1358086076}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1358086076}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:49:56.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:49:56.413 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:49:56.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4640000640 1 Processor -- start 2024-04-25T08:49:56.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4640000640 1 -- start start 2024-04-25T08:49:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4640000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46380af3f0 0x7f46380b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4640000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f463806f630 0x7f46380b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4640000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463815e180 con 0x7f463806db40 2024-04-25T08:49:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4640000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46380b3ca0 con 0x7f463806f630 2024-04-25T08:49:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4640000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463815e460 con 0x7f46380af3f0 2024-04-25T08:49:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f463e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f463806f630 0x7f46380b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f463e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f463806f630 0x7f46380b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45472/0 (socket says 172.21.15.46:45472) 2024-04-25T08:49:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f463e000640 1 -- 172.21.15.46:0/1909517613 learned_addr learned my addr 172.21.15.46:0/1909517613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:56.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f463d600640 1 --2- 172.21.15.46:0/1909517613 >> v2:172.21.15.82:3300/0 conn(0x7f46380af3f0 0x7f46380b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f463e000640 1 -- 172.21.15.46:0/1909517613 >> v2:172.21.15.82:3300/0 conn(0x7f46380af3f0 msgr2=0x7f46380b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f463e000640 1 --2- 172.21.15.46:0/1909517613 >> v2:172.21.15.82:3300/0 conn(0x7f46380af3f0 0x7f46380b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f463e000640 1 -- 172.21.15.46:0/1909517613 >> v1:172.21.15.82:6789/0 conn(0x7f463806db40 legacy=0x7f463806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f463e000640 1 -- 172.21.15.46:0/1909517613 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463815e740 con 0x7f463806f630 2024-04-25T08:49:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f463e000640 1 --2- 172.21.15.46:0/1909517613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463806f630 0x7f46380b3380 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f46380b5310 tx=0x7f463001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30ec28f8fd714f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4637e00640 1 -- 172.21.15.46:0/1909517613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300a5590 con 0x7f463806f630 2024-04-25T08:49:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4637e00640 1 -- 172.21.15.46:0/1909517613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46300a5730 con 0x7f463806f630 2024-04-25T08:49:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4637e00640 1 -- 172.21.15.46:0/1909517613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300a5b00 con 0x7f463806f630 2024-04-25T08:49:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4640000640 1 -- 172.21.15.46:0/1909517613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463806f630 msgr2=0x7f46380b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.567+0000 7f4640000640 1 --2- 172.21.15.46:0/1909517613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463806f630 0x7f46380b3380 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f46380b5310 tx=0x7f463001c9b0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 -- 172.21.15.46:0/1909517613 shutdown_connections 2024-04-25T08:49:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 --2- 172.21.15.46:0/1909517613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f463806f630 0x7f46380b3380 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 --2- 172.21.15.46:0/1909517613 >> v2:172.21.15.82:3300/0 conn(0x7f46380af3f0 0x7f46380b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 -- 172.21.15.46:0/1909517613 >> 172.21.15.46:0/1909517613 conn(0x7f46380cfd90 msgr2=0x7f46380d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 -- 172.21.15.46:0/1909517613 shutdown_connections 2024-04-25T08:49:56.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 -- 172.21.15.46:0/1909517613 wait complete. 2024-04-25T08:49:56.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 Processor -- start 2024-04-25T08:49:56.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 -- start start 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f463806db40 0x7f4638142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463806f630 0x7f4638151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380af3f0 0x7f46381571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463806bda0 con 0x7f46380af3f0 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463806bc20 con 0x7f463806db40 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f4640000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463816a6f0 con 0x7f463806f630 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380af3f0 0x7f46381571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380af3f0 0x7f46381571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45476/0 (socket says 172.21.15.46:45476) 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463e000640 1 -- 172.21.15.46:0/1924783645 learned_addr learned my addr 172.21.15.46:0/1924783645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463cc00640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463806f630 0x7f4638151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463d600640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f463806db40 0x7f4638142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463e000640 1 -- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463806f630 msgr2=0x7f4638151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463e000640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463806f630 0x7f4638151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463e000640 1 -- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f463806db40 msgr2=0x7f4638142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463e000640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f463806db40 0x7f4638142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463e000640 1 -- 172.21.15.46:0/1924783645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380b10e0 con 0x7f46380af3f0 2024-04-25T08:49:56.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.571+0000 7f463e000640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380af3f0 0x7f46381571e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f463001a5e0 tx=0x7f4630019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.575+0000 7f4635600640 1 -- 172.21.15.46:0/1924783645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630009510 con 0x7f46380af3f0 2024-04-25T08:49:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.575+0000 7f4635600640 1 -- 172.21.15.46:0/1924783645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46300be070 con 0x7f46380af3f0 2024-04-25T08:49:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.575+0000 7f4635600640 1 -- 172.21.15.46:0/1924783645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300b88d0 con 0x7f46380af3f0 2024-04-25T08:49:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.575+0000 7f4640000640 1 -- 172.21.15.46:0/1924783645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46380b13c0 con 0x7f46380af3f0 2024-04-25T08:49:56.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.575+0000 7f4640000640 1 -- 172.21.15.46:0/1924783645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4638159b80 con 0x7f46380af3f0 2024-04-25T08:49:56.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.575+0000 7f4635600640 1 -- 172.21.15.46:0/1924783645 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46300b8a70 con 0x7f46380af3f0 2024-04-25T08:49:56.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.575+0000 7f4635600640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4618066f60 0x7f4618069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4635600640 1 -- 172.21.15.46:0/1924783645 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f46301203f0 con 0x7f46380af3f0 2024-04-25T08:49:56.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4618066f60 0x7f4618069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 Processor -- start 2024-04-25T08:49:56.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 -- start start 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4618066f60 0x7f4618069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f462c0079c0 tx=0x7f462c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2af69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2af7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e29de5f0 con 0x5652e2af6f30 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e2945da0 con 0x5652e2b40920 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652e28ce150 con 0x5652e2b41240 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2af7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b40d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:42970/0 (socket says 172.21.15.46:42970) 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 -- 172.21.15.46:0/4164975643 learned_addr learned my addr 172.21.15.46:0/4164975643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:56.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463cc00640 1 --2- 172.21.15.46:0/4164975643 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2af69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 -- 172.21.15.46:0/4164975643 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 msgr2=0x5652e2af69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 --2- 172.21.15.46:0/4164975643 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2af69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 -- 172.21.15.46:0/4164975643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 msgr2=0x5652e2af7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 --2- 172.21.15.46:0/4164975643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2af7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 -- 172.21.15.46:0/4164975643 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e28e7ca0 con 0x5652e2b40920 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463e000640 1 --2- 172.21.15.46:0/4164975643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2af7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f463d600640 1 --2- 172.21.15.46:0/4164975643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b40d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f462c010ac0 tx=0x7f462c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f90845cf3822b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f462a000640 1 -- 172.21.15.46:0/4164975643 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c01c050 con 0x5652e2b40920 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f462a000640 1 -- 172.21.15.46:0/4164975643 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c026070 con 0x5652e2b40920 2024-04-25T08:49:56.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f462a000640 1 -- 172.21.15.46:0/4164975643 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c016e10 con 0x5652e2b40920 2024-04-25T08:49:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 -- 172.21.15.46:0/4164975643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 msgr2=0x5652e2b40d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 --2- 172.21.15.46:0/4164975643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b40d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f462c010ac0 tx=0x7f462c013000 comp rx=0 tx=0).stop 2024-04-25T08:49:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 -- 172.21.15.46:0/4164975643 shutdown_connections 2024-04-25T08:49:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 --2- 172.21.15.46:0/4164975643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2af7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 --2- 172.21.15.46:0/4164975643 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2af69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 --2- 172.21.15.46:0/4164975643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b40d00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 -- 172.21.15.46:0/4164975643 >> 172.21.15.46:0/4164975643 conn(0x5652e2afd2a0 msgr2=0x5652e2af52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 -- 172.21.15.46:0/4164975643 shutdown_connections 2024-04-25T08:49:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.579+0000 7f4643462000 1 -- 172.21.15.46:0/4164975643 wait complete. 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 -- start start 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 Processor -- start 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2af6f30 0x5652e2b47c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2b41240 0x5652e2afaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e2900990 con 0x5652e2b41240 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e2941e40 con 0x5652e2b40920 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652e28d1900 con 0x5652e2af6f30 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 10 client.0 _ll_register_callbacks cb 0x5652e29f91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2af6f30 0x5652e2b47c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2af6f30 0x5652e2b47c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:44668/0 (socket says 172.21.15.46:44668) 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463d600640 1 -- 172.21.15.46:0/755532307 learned_addr learned my addr 172.21.15.46:0/755532307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463e000640 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2b41240 0x5652e2afaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463d600640 1 -- 172.21.15.46:0/755532307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 msgr2=0x5652e2b481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463d600640 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463d600640 1 -- 172.21.15.46:0/755532307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2b41240 msgr2=0x5652e2afaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463d600640 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2b41240 0x5652e2afaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463d600640 1 -- 172.21.15.46:0/755532307 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652e28fe630 con 0x5652e2af6f30 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463e000640 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2b41240 0x5652e2afaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f463d600640 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2af6f30 0x5652e2b47c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f462c016850 tx=0x7f462c004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1387a5a60f314c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f461ea00640 10 client.0 ms_handle_connect on v2:172.21.15.82:3301/0 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f461ea00640 1 -- 172.21.15.46:0/755532307 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c02f070 con 0x5652e2af6f30 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f461ea00640 1 -- 172.21.15.46:0/755532307 <== mon.2 v2:172.21.15.82:3301/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f462c005bb0 con 0x5652e2af6f30 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 -- 172.21.15.46:0/755532307 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f462c026070 con 0x5652e2af6f30 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f461ea00640 1 -- 172.21.15.46:0/755532307 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f462c01c020 con 0x5652e2af6f30 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 10 client.6860 fetch_fsmap learned FSMap version 75 2024-04-25T08:49:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 -- 172.21.15.46:0/755532307 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5652e28ce3a0 con 0x5652e2af6f30 2024-04-25T08:49:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f461ea00640 1 -- 172.21.15.46:0/755532307 <== mon.2 v2:172.21.15.82:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c026070 con 0x5652e2af6f30 2024-04-25T08:49:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f461ea00640 1 -- 172.21.15.46:0/755532307 <== mon.2 v2:172.21.15.82:3301/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f462c0218a0 con 0x5652e2af6f30 2024-04-25T08:49:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 10 client.6860 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:49:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 10 client.6860 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:49:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 20 client.6860 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5652e2afc160 0x5652e2b51c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 4 client.6860 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.583+0000 7f4643462000 1 -- 172.21.15.46:0/755532307 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652e28e1a50 con 0x5652e2afc160 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f463cc00640 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5652e2afc160 0x5652e2b51c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f463cc00640 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5652e2afc160 0x5652e2b51c70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2753f468d456cb58 server_cookie=62e44be968d7da08 in_seq=0 out_seq=0 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f461ea00640 10 client.6860 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f461ea00640 1 -- 172.21.15.46:0/755532307 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652e28e1a50 con 0x5652e2afc160 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f461ea00640 10 client.6860 handle_command_reply: tid=0 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 client.6860 shutdown 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 -- 172.21.15.46:0/755532307 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2af6f30 msgr2=0x5652e2b47c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2af6f30 0x5652e2b47c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f462c016850 tx=0x7f462c004740 comp rx=0 tx=0).stop 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 -- 172.21.15.46:0/755532307 shutdown_connections 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5652e2afc160 0x5652e2b51c70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2b41240 0x5652e2afaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 --2- 172.21.15.46:0/755532307 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2af6f30 0x5652e2b47c80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 -- 172.21.15.46:0/755532307 >> 172.21.15.46:0/755532307 conn(0x5652e2afd2a0 msgr2=0x5652e2af59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:56.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f461ea00640 0 client.6860 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:56.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 -- 172.21.15.46:0/755532307 shutdown_connections 2024-04-25T08:49:56.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 1 -- 172.21.15.46:0/755532307 wait complete. 2024-04-25T08:49:56.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.587+0000 7f4643462000 20 client.6860 trim_cache size 0 max 16384 2024-04-25T08:49:56.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 Processor -- start 2024-04-25T08:49:56.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- start start 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2b9c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b9cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2b9d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4638202710 con 0x5652e2af6f30 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463815adc0 con 0x5652e2b40920 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463815b3c0 con 0x5652e2b41240 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2b9c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2b9c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45506/0 (socket says 172.21.15.46:45506) 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 -- 172.21.15.46:0/985789186 learned_addr learned my addr 172.21.15.46:0/985789186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 -- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 msgr2=0x5652e2b9d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463e000640 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2b9d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463cc00640 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b9cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2b9d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 -- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 msgr2=0x5652e2b9cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b9cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 -- 172.21.15.46:0/985789186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e2b41070 con 0x5652e2af6f30 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463e000640 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2b9d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463cc00640 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b9cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2b9c750 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f462c0047c0 tx=0x7f462c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a617c9f62f4f780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4629600640 1 -- 172.21.15.46:0/985789186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c02f040 con 0x5652e2af6f30 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4629600640 1 -- 172.21.15.46:0/985789186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c026070 con 0x5652e2af6f30 2024-04-25T08:49:56.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4629600640 1 -- 172.21.15.46:0/985789186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c005df0 con 0x5652e2af6f30 2024-04-25T08:49:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- 172.21.15.46:0/985789186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 msgr2=0x5652e2b9c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2b9c750 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f462c0047c0 tx=0x7f462c02c000 comp rx=0 tx=0).stop 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- 172.21.15.46:0/985789186 shutdown_connections 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2b9d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2b9cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- 172.21.15.46:0/985789186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2b9c750 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- 172.21.15.46:0/985789186 >> 172.21.15.46:0/985789186 conn(0x5652e2afd2a0 msgr2=0x5652e2afa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- 172.21.15.46:0/985789186 shutdown_connections 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- 172.21.15.46:0/985789186 wait complete. 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- start start 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 Processor -- start 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2ba1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2ba2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2b85ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e294a3d0 con 0x5652e2af6f30 2024-04-25T08:49:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e2a01d50 con 0x5652e2b40920 2024-04-25T08:49:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652e28e0980 con 0x5652e2b41240 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 10 client.0 _ll_register_callbacks cb 0x7f462c0076d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2ba1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2ba1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45508/0 (socket says 172.21.15.46:45508) 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 -- 172.21.15.46:0/2608488874 learned_addr learned my addr 172.21.15.46:0/2608488874 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463cc00640 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2ba2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463e000640 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2b85ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 -- 172.21.15.46:0/2608488874 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 msgr2=0x5652e2b85ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2b85ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 -- 172.21.15.46:0/2608488874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 msgr2=0x5652e2ba2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2ba2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 -- 172.21.15.46:0/2608488874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652e29c7600 con 0x5652e2af6f30 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f463d600640 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2ba1e10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f462c021d20 tx=0x7f462c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865d341f089f4354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f461ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f461ea00640 1 -- 172.21.15.46:0/2608488874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c02f040 con 0x5652e2af6f30 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f461ea00640 1 -- 172.21.15.46:0/2608488874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f462c034650 con 0x5652e2af6f30 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- 172.21.15.46:0/2608488874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f462c026070 con 0x5652e2af6f30 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f461ea00640 1 -- 172.21.15.46:0/2608488874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f462c050050 con 0x5652e2af6f30 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 10 client.8046 fetch_fsmap learned FSMap version 75 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- 172.21.15.46:0/2608488874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5652e28da9b0 con 0x5652e2af6f30 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f461ea00640 1 -- 172.21.15.46:0/2608488874 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c026070 con 0x5652e2af6f30 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f461ea00640 1 -- 172.21.15.46:0/2608488874 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f462c053020 con 0x5652e2af6f30 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 10 client.8046 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 10 client.8046 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 20 client.8046 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:56.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5652e2b86020 0x5652e2b86400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 4 client.8046 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.607+0000 7f4643462000 1 -- 172.21.15.46:0/2608488874 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5652e29532e0 con 0x5652e2b86020 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f463cc00640 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5652e2b86020 0x5652e2b86400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f463cc00640 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5652e2b86020 0x5652e2b86400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df7ee8698ba0e262 server_cookie=5698357ceb5fcc8e in_seq=0 out_seq=0 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f461ea00640 10 client.8046 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f461ea00640 1 -- 172.21.15.46:0/2608488874 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5652e29532e0 con 0x5652e2b86020 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f461ea00640 10 client.8046 handle_command_reply: tid=0 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 client.8046 shutdown 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 -- 172.21.15.46:0/2608488874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 msgr2=0x5652e2ba1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2ba1e10 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f462c021d20 tx=0x7f462c016000 comp rx=0 tx=0).stop 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 -- 172.21.15.46:0/2608488874 shutdown_connections 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5652e2b86020 0x5652e2b86400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5652e2b41240 0x5652e2b85ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5652e2b40920 0x5652e2ba2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f461ea00640 0 client.8046 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 --2- 172.21.15.46:0/2608488874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e2af6f30 0x5652e2ba1e10 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 -- 172.21.15.46:0/2608488874 >> 172.21.15.46:0/2608488874 conn(0x5652e2afd2a0 msgr2=0x5652e2b48870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 -- 172.21.15.46:0/2608488874 shutdown_connections 2024-04-25T08:49:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 1 -- 172.21.15.46:0/2608488874 wait complete. 2024-04-25T08:49:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4643462000 20 client.8046 trim_cache size 0 max 16384 2024-04-25T08:49:56.613 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:49:56.613 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:49:56.613 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7962, 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5679903670000002, 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 75, 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 94, 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 94, 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.5017383039999999 2024-04-25T08:49:56.614 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:49:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 -- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4618066f60 msgr2=0x7f4618069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4618066f60 0x7f4618069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f462c0079c0 tx=0x7f462c008040 comp rx=0 tx=0).stop 2024-04-25T08:49:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 -- 172.21.15.46:0/1924783645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380af3f0 msgr2=0x7f46381571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380af3f0 0x7f46381571e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f463001a5e0 tx=0x7f4630019750 comp rx=0 tx=0).stop 2024-04-25T08:49:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 -- 172.21.15.46:0/1924783645 shutdown_connections 2024-04-25T08:49:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4618066f60 0x7f4618069420 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46380af3f0 0x7f46381571e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f463806f630 0x7f4638151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 --2- 172.21.15.46:0/1924783645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f463806db40 0x7f4638142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 -- 172.21.15.46:0/1924783645 >> 172.21.15.46:0/1924783645 conn(0x7f46380cfd90 msgr2=0x7f46380b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 -- 172.21.15.46:0/1924783645 shutdown_connections 2024-04-25T08:49:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.611+0000 7f4640000640 1 -- 172.21.15.46:0/1924783645 wait complete. 2024-04-25T08:49:56.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:49:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92c2000640 1 Processor -- start 2024-04-25T08:49:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92c2000640 1 -- start start 2024-04-25T08:49:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92c2000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92bc15bdb0 0x7f92bc0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92c2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92bc0cda80 0x7f92bc15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92c2000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc0cc700 con 0x7f92bc159230 2024-04-25T08:49:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92c2000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc0cded0 con 0x7f92bc0cda80 2024-04-25T08:49:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92c2000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc0cc9e0 con 0x7f92bc15bdb0 2024-04-25T08:49:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92bc0cda80 0x7f92bc15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92bc0cda80 0x7f92bc15d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45510/0 (socket says 172.21.15.46:45510) 2024-04-25T08:49:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92bbe00640 1 -- 172.21.15.46:0/1060759997 learned_addr learned my addr 172.21.15.46:0/1060759997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92b3e00640 1 --2- 172.21.15.46:0/1060759997 >> v2:172.21.15.82:3300/0 conn(0x7f92bc15bdb0 0x7f92bc0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92bbe00640 1 -- 172.21.15.46:0/1060759997 >> v2:172.21.15.82:3300/0 conn(0x7f92bc15bdb0 msgr2=0x7f92bc0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92bbe00640 1 --2- 172.21.15.46:0/1060759997 >> v2:172.21.15.82:3300/0 conn(0x7f92bc15bdb0 0x7f92bc0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92bbe00640 1 -- 172.21.15.46:0/1060759997 >> v1:172.21.15.82:6789/0 conn(0x7f92bc159230 legacy=0x7f92bc0cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92bbe00640 1 -- 172.21.15.46:0/1060759997 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0cccc0 con 0x7f92bc0cda80 2024-04-25T08:49:56.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92bbe00640 1 --2- 172.21.15.46:0/1060759997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 0x7f92bc15d200 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f92bc050000 tx=0x7f92b401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a34f2a68b7bb0f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92baa00640 1 -- 172.21.15.46:0/1060759997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40a5710 con 0x7f92bc0cda80 2024-04-25T08:49:56.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92baa00640 1 -- 172.21.15.46:0/1060759997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b40a58b0 con 0x7f92bc0cda80 2024-04-25T08:49:56.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.775+0000 7f92baa00640 1 -- 172.21.15.46:0/1060759997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b40a5c80 con 0x7f92bc0cda80 2024-04-25T08:49:56.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- 172.21.15.46:0/1060759997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 msgr2=0x7f92bc15d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 --2- 172.21.15.46:0/1060759997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 0x7f92bc15d200 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f92bc050000 tx=0x7f92b401c960 comp rx=0 tx=0).stop 2024-04-25T08:49:56.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- 172.21.15.46:0/1060759997 shutdown_connections 2024-04-25T08:49:56.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 --2- 172.21.15.46:0/1060759997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 0x7f92bc15d200 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 --2- 172.21.15.46:0/1060759997 >> v2:172.21.15.82:3300/0 conn(0x7f92bc15bdb0 0x7f92bc0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- 172.21.15.46:0/1060759997 >> 172.21.15.46:0/1060759997 conn(0x7f92bc0b12a0 msgr2=0x7f92bc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:56.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- 172.21.15.46:0/1060759997 shutdown_connections 2024-04-25T08:49:56.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- 172.21.15.46:0/1060759997 wait complete. 2024-04-25T08:49:56.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 Processor -- start 2024-04-25T08:49:56.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- start start 2024-04-25T08:49:56.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 0x7f92bc143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc159230 0x7f92bc13d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92bc15bdb0 0x7f92bc13e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc15e0d0 con 0x7f92bc0cda80 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc15df50 con 0x7f92bc159230 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc15e250 con 0x7f92bc15bdb0 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 0x7f92bc143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 0x7f92bc143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45514/0 (socket says 172.21.15.46:45514) 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bb400640 1 -- 172.21.15.46:0/922688367 learned_addr learned my addr 172.21.15.46:0/922688367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bbe00640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92bc15bdb0 0x7f92bc13e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92b3e00640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc159230 0x7f92bc13d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bb400640 1 -- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92bc15bdb0 msgr2=0x7f92bc13e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bb400640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92bc15bdb0 0x7f92bc13e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bb400640 1 -- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc159230 msgr2=0x7f92bc13d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bb400640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc159230 0x7f92bc13d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bb400640 1 -- 172.21.15.46:0/922688367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0ae430 con 0x7f92bc0cda80 2024-04-25T08:49:56.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92b3e00640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc159230 0x7f92bc13d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92bb400640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 0x7f92bc143850 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c420 tx=0x7f92ac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92b3400640 1 -- 172.21.15.46:0/922688367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0052d0 con 0x7f92bc0cda80 2024-04-25T08:49:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92b3400640 1 -- 172.21.15.46:0/922688367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac005470 con 0x7f92bc0cda80 2024-04-25T08:49:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92b3400640 1 -- 172.21.15.46:0/922688367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac007260 con 0x7f92bc0cda80 2024-04-25T08:49:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc13d850 con 0x7f92bc0cda80 2024-04-25T08:49:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc13de10 con 0x7f92bc0cda80 2024-04-25T08:49:56.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc069080 con 0x7f92bc0cda80 2024-04-25T08:49:56.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92b3400640 1 -- 172.21.15.46:0/922688367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92ac007400 con 0x7f92bc0cda80 2024-04-25T08:49:56.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92b3400640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9298066f60 0x7f9298069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:56.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92b3400640 1 -- 172.21.15.46:0/922688367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f92ac0847d0 con 0x7f92bc0cda80 2024-04-25T08:49:56.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.783+0000 7f92b3e00640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9298066f60 0x7f9298069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:56.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.791+0000 7f92b3e00640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9298066f60 0x7f9298069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f92bc070b90 tx=0x7f92a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:56.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.791+0000 7f92b3400640 1 -- 172.21.15.46:0/922688367 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92ac054e40 con 0x7f92bc0cda80 2024-04-25T08:49:56.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.987+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f92bc0723d0 con 0x7f92bc0cda80 2024-04-25T08:49:56.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92b3400640 1 -- 172.21.15.46:0/922688367 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7f92ac058ee0 con 0x7f92bc0cda80 2024-04-25T08:49:56.980 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:49:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9298066f60 msgr2=0x7f9298069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9298066f60 0x7f9298069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f92bc070b90 tx=0x7f92a4008040 comp rx=0 tx=0).stop 2024-04-25T08:49:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 msgr2=0x7f92bc143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:56.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 0x7f92bc143850 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00c420 tx=0x7f92ac00c8e0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 shutdown_connections 2024-04-25T08:49:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9298066f60 0x7f9298069420 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92bc15bdb0 0x7f92bc13e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92bc159230 0x7f92bc13d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 --2- 172.21.15.46:0/922688367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92bc0cda80 0x7f92bc143850 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 >> 172.21.15.46:0/922688367 conn(0x7f92bc0b12a0 msgr2=0x7f92bc15f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 shutdown_connections 2024-04-25T08:49:56.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:56.991+0000 7f92c2000640 1 -- 172.21.15.46:0/922688367 wait complete. 2024-04-25T08:49:57.002 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:49:57.002 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:49:57.002 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:49:57.002 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:49:57.003 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:49:57.003 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:49:57.003 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:49:57.003 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:49:57.003 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:49:57.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.003 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:49:57.012 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:49:57.013 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42006sec preferred_lft 42006sec 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:49:57.014 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:49:57.015 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:49:57.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.017 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:49:57.082 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:49:57.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.085 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:49:57.141 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:49:57.142 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:49:57.142 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:49:57.142 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:49:57.142 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:49:57.142 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:49:57.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.142 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:49:57.143 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:49:57.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:49:57.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:49:57.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:49:57.143 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:49:57.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.234 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:49:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:49:57.244 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:49:57.244 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:49:57.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:49:57.307 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:49:57.308 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:49:57.360 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:49:57.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:49:57.360 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:49:57.411 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:49:57.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:49:57.505 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:49:57.505 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:49:57.505 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:49:57.520 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:49:57.521 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:49:57.521 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:49:57.521 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:49:57.521 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:49:57.530 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:49:57.530 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:49:57.530 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:49:57.530 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:49:57.530 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:49:57.530 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:49:57.531 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:49:57.531 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:49:57.535 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:49:57.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:49:57.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.555 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:49:57.607 INFO:teuthology.orchestra.run.smithi046.stdout:client6869 2024-04-25T08:49:57.607 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:49:57.607 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:49:57.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T08:49:57.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.855+0000 7f2cb6000640 1 Processor -- start 2024-04-25T08:49:57.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.855+0000 7f2cb6000640 1 -- start start 2024-04-25T08:49:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2cb6000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2cb0140ee0 0x7f2cb006e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2cb6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0141580 0x7f2cb006e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2cb6000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb00b2e60 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2cb6000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb01412c0 con 0x7f2cb0140ee0 2024-04-25T08:49:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2cb6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00b30d0 con 0x7f2cb0141580 2024-04-25T08:49:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2ca6a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0141580 0x7f2cb006e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2caf400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2cb0140ee0 0x7f2cb006e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2caf400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2cb0140ee0 0x7f2cb006e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:43030/0 (socket says 172.21.15.46:43030) 2024-04-25T08:49:57.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2caf400640 1 -- 172.21.15.46:0/467544930 learned_addr learned my addr 172.21.15.46:0/467544930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:57.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2caea00640 1 -- 172.21.15.46:0/467544930 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504956919 0 0) 0x7f2cb00b2e60 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2ca6a00640 1 -- 172.21.15.46:0/467544930 >> v2:172.21.15.82:3300/0 conn(0x7f2cb0140ee0 msgr2=0x7f2cb006e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:57.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/467544930 >> v2:172.21.15.82:3300/0 conn(0x7f2cb0140ee0 0x7f2cb006e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2ca6a00640 1 -- 172.21.15.46:0/467544930 >> v1:172.21.15.82:6789/0 conn(0x7f2cb00ce4d0 legacy=0x7f2cb00ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.859+0000 7f2ca6a00640 1 -- 172.21.15.46:0/467544930 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb00b33b0 con 0x7f2cb0141580 2024-04-25T08:49:57.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/467544930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0141580 0x7f2cb006e850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c980097c0 tx=0x7f2c9801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b64f90d56a2a6a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:57.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2caea00640 1 -- 172.21.15.46:0/467544930 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980aa620 con 0x7f2cb0141580 2024-04-25T08:49:57.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2caea00640 1 -- 172.21.15.46:0/467544930 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c980af070 con 0x7f2cb0141580 2024-04-25T08:49:57.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2caea00640 1 -- 172.21.15.46:0/467544930 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980aa9e0 con 0x7f2cb0141580 2024-04-25T08:49:57.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 -- 172.21.15.46:0/467544930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0141580 msgr2=0x7f2cb006e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 --2- 172.21.15.46:0/467544930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0141580 0x7f2cb006e850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2c980097c0 tx=0x7f2c9801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 -- 172.21.15.46:0/467544930 shutdown_connections 2024-04-25T08:49:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 --2- 172.21.15.46:0/467544930 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0141580 0x7f2cb006e850 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 --2- 172.21.15.46:0/467544930 >> v2:172.21.15.82:3300/0 conn(0x7f2cb0140ee0 0x7f2cb006e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 -- 172.21.15.46:0/467544930 >> 172.21.15.46:0/467544930 conn(0x7f2cb0067050 msgr2=0x7f2cb006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 -- 172.21.15.46:0/467544930 shutdown_connections 2024-04-25T08:49:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 -- 172.21.15.46:0/467544930 wait complete. 2024-04-25T08:49:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 Processor -- start 2024-04-25T08:49:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.863+0000 7f2cb6000640 1 -- start start 2024-04-25T08:49:57.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2cb6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00ce4d0 0x7f2cb00af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2cb6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0140ee0 0x7f2cb0143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2cb6000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb0141580 0x7f2cb01585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2cb6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00b4750 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2cb6000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00b45d0 con 0x7f2cb0141580 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2cb6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00b48d0 con 0x7f2cb0140ee0 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2caf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00ce4d0 0x7f2cb00af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2caf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00ce4d0 0x7f2cb00af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45528/0 (socket says 172.21.15.46:45528) 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2caf400640 1 -- 172.21.15.46:0/2066080305 learned_addr learned my addr 172.21.15.46:0/2066080305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0140ee0 0x7f2cb0143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2cafe00640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb0141580 0x7f2cb01585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2caf400640 1 -- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0140ee0 msgr2=0x7f2cb0143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2caf400640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0140ee0 0x7f2cb0143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2caf400640 1 -- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb0141580 msgr2=0x7f2cb01585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2caf400640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb0141580 0x7f2cb01585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2caf400640 1 -- 172.21.15.46:0/2066080305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb00af8c0 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2caf400640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00ce4d0 0x7f2cb00af100 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000e7f0 tx=0x7f2ca000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2ca7e00640 1 -- 172.21.15.46:0/2066080305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0017070 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2ca7e00640 1 -- 172.21.15.46:0/2066080305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca000c940 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2ca7e00640 1 -- 172.21.15.46:0/2066080305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000cd30 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2cb6000640 1 -- 172.21.15.46:0/2066080305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb0142d60 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.867+0000 7f2cb6000640 1 -- 172.21.15.46:0/2066080305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb0159060 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2ca7e00640 1 -- 172.21.15.46:0/2066080305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ca001a070 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2ca7e00640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c8c066f60 0x7f2c8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2ca7e00640 1 -- 172.21.15.46:0/2066080305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f2ca00855a0 con 0x7f2cb00ce4d0 2024-04-25T08:49:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cb9547000 1 Processor -- start 2024-04-25T08:49:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c8c066f60 0x7f2c8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cb9547000 1 -- start start 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c8c066f60 0x7f2c8c069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2c980ae000 tx=0x7f2c980a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x5559363b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 0x555936367d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936368670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cb9547000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55593624ef40 con 0x555936368290 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cb9547000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559361b6070 con 0x5559363b25b0 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cb9547000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55593613e1a0 con 0x5559363b1c90 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936368670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936368670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45542/0 (socket says 172.21.15.46:45542) 2024-04-25T08:49:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cafe00640 1 -- 172.21.15.46:0/970104840 learned_addr learned my addr 172.21.15.46:0/970104840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/970104840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 0x555936367d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2caf400640 1 --2- 172.21.15.46:0/970104840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x5559363b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cafe00640 1 -- 172.21.15.46:0/970104840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 msgr2=0x5559363b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cafe00640 1 --2- 172.21.15.46:0/970104840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x5559363b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cafe00640 1 -- 172.21.15.46:0/970104840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 msgr2=0x555936367d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cafe00640 1 --2- 172.21.15.46:0/970104840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 0x555936367d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cafe00640 1 -- 172.21.15.46:0/970104840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55593613c510 con 0x555936368290 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2cafe00640 1 --2- 172.21.15.46:0/970104840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936368670 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8001e60 tx=0x7f2ca800f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b92f5254bf3e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2c9fe00640 1 -- 172.21.15.46:0/970104840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8002070 con 0x555936368290 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2c9fe00640 1 -- 172.21.15.46:0/970104840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8018070 con 0x555936368290 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.871+0000 7f2c9fe00640 1 -- 172.21.15.46:0/970104840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8003ba0 con 0x555936368290 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- 172.21.15.46:0/970104840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 msgr2=0x555936368670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 --2- 172.21.15.46:0/970104840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936368670 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8001e60 tx=0x7f2ca800f5a0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- 172.21.15.46:0/970104840 shutdown_connections 2024-04-25T08:49:57.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 --2- 172.21.15.46:0/970104840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936368670 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 --2- 172.21.15.46:0/970104840 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 0x555936367d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 --2- 172.21.15.46:0/970104840 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x5559363b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- 172.21.15.46:0/970104840 >> 172.21.15.46:0/970104840 conn(0x555936364240 msgr2=0x555936366630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- 172.21.15.46:0/970104840 shutdown_connections 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- 172.21.15.46:0/970104840 wait complete. 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- start start 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 Processor -- start 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936372e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x555936373370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 0x5559363738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559361706c0 con 0x555936368290 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559361b2110 con 0x5559363b25b0 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559361b5380 con 0x5559363b1c90 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 10 client.0 _ll_register_callbacks cb 0x555936353570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:57.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2caf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936372e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2caf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936372e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45552/0 (socket says 172.21.15.46:45552) 2024-04-25T08:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2caf400640 1 -- 172.21.15.46:0/1321702907 learned_addr learned my addr 172.21.15.46:0/1321702907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x555936373370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cafe00640 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 0x5559363738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2caf400640 1 -- 172.21.15.46:0/1321702907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 msgr2=0x555936373370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2caf400640 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x555936373370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2caf400640 1 -- 172.21.15.46:0/1321702907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 msgr2=0x5559363738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:57.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2caf400640 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 0x5559363738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2caf400640 1 -- 172.21.15.46:0/1321702907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55593613e3b0 con 0x555936368290 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2caf400640 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936372e30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000c520 tx=0x7f2ca0055880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19598156214c0a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2c9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1321702907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca006d020 con 0x555936368290 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1321702907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f2ca005add0 con 0x555936368290 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- 172.21.15.46:0/1321702907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ca8018070 con 0x555936368290 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1321702907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca0072050 con 0x555936368290 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 10 client.8061 fetch_fsmap learned FSMap version 75 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 1 -- 172.21.15.46:0/1321702907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x555936237e00 con 0x555936368290 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1321702907 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8018070 con 0x555936368290 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1321702907 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2ca005dde0 con 0x555936368290 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 10 client.8061 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:49:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.875+0000 7f2cb9547000 10 client.8061 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 20 client.8061 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555936374150 0x5559363c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 4 client.8061 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 -- 172.21.15.46:0/1321702907 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555936151d60 con 0x555936374150 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555936374150 0x5559363c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555936374150 0x5559363c3410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c615c43a65b7c270 server_cookie=8ea4d3c01402d35c in_seq=0 out_seq=0 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2c9cc00640 10 client.8061 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1321702907 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555936151d60 con 0x555936374150 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2c9cc00640 10 client.8061 handle_command_reply: tid=0 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 client.8061 shutdown 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 -- 172.21.15.46:0/1321702907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 msgr2=0x555936372e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936372e30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000c520 tx=0x7f2ca0055880 comp rx=0 tx=0).stop 2024-04-25T08:49:57.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 -- 172.21.15.46:0/1321702907 shutdown_connections 2024-04-25T08:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555936374150 0x5559363c3410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559363b25b0 0x5559363738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x555936373370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1321702907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555936368290 0x555936372e30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2c9cc00640 0 client.8061 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 -- 172.21.15.46:0/1321702907 >> 172.21.15.46:0/1321702907 conn(0x555936364240 msgr2=0x555936366ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 -- 172.21.15.46:0/1321702907 shutdown_connections 2024-04-25T08:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 1 -- 172.21.15.46:0/1321702907 wait complete. 2024-04-25T08:49:57.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.879+0000 7f2cb9547000 20 client.8061 trim_cache size 0 max 16384 2024-04-25T08:49:57.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 Processor -- start 2024-04-25T08:49:57.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 -- start start 2024-04-25T08:49:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 0x55593640dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x55593640e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x55593640e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb0203290 con 0x5559363b25b0 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb0202470 con 0x555936368290 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb00ad3a0 con 0x5559363b1c90 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x55593640e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x55593640e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45564/0 (socket says 172.21.15.46:45564) 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cafe00640 1 -- 172.21.15.46:0/764966125 learned_addr learned my addr 172.21.15.46:0/764966125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2caf400640 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 0x55593640dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x55593640e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cafe00640 1 -- 172.21.15.46:0/764966125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 msgr2=0x55593640e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cafe00640 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x55593640e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cafe00640 1 -- 172.21.15.46:0/764966125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 msgr2=0x55593640dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cafe00640 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 0x55593640dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cafe00640 1 -- 172.21.15.46:0/764966125 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55593620dc50 con 0x5559363b25b0 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2caf400640 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 0x55593640dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cafe00640 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x55593640e760 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8014000 tx=0x7f2ca800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e560b3e8cbc2d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2c9f400640 1 -- 172.21.15.46:0/764966125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8025070 con 0x5559363b25b0 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2c9f400640 1 -- 172.21.15.46:0/764966125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8018050 con 0x5559363b25b0 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2c9f400640 1 -- 172.21.15.46:0/764966125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80050d0 con 0x5559363b25b0 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 -- 172.21.15.46:0/764966125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 msgr2=0x55593640e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x55593640e760 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8014000 tx=0x7f2ca800a000 comp rx=0 tx=0).stop 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 -- 172.21.15.46:0/764966125 shutdown_connections 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x55593640e760 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x55593640e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 --2- 172.21.15.46:0/764966125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 0x55593640dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 -- 172.21.15.46:0/764966125 >> 172.21.15.46:0/764966125 conn(0x555936364240 msgr2=0x55593636c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.895+0000 7f2cb9547000 1 -- 172.21.15.46:0/764966125 shutdown_connections 2024-04-25T08:49:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 -- 172.21.15.46:0/764966125 wait complete. 2024-04-25T08:49:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 -- start start 2024-04-25T08:49:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 Processor -- start 2024-04-25T08:49:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 0x555936411c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x5559364121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x5559363cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555936158bd0 con 0x5559363b25b0 2024-04-25T08:49:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb0201ea0 con 0x555936368290 2024-04-25T08:49:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559361b5380 con 0x5559363b1c90 2024-04-25T08:49:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 10 client.0 _ll_register_callbacks cb 0x7f2ca8001900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x5559363cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x5559363cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45574/0 (socket says 172.21.15.46:45574) 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cafe00640 1 -- 172.21.15.46:0/1441822226 learned_addr learned my addr 172.21.15.46:0/1441822226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x5559364121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2caf400640 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 0x555936411c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cafe00640 1 -- 172.21.15.46:0/1441822226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 msgr2=0x5559364121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cafe00640 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x5559364121b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cafe00640 1 -- 172.21.15.46:0/1441822226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 msgr2=0x555936411c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cafe00640 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 0x555936411c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cafe00640 1 -- 172.21.15.46:0/1441822226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55593617e120 con 0x5559363b25b0 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2ca6a00640 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x5559364121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cafe00640 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x5559363cce90 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009010 tx=0x7f2ca80015d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d095b50ae6ee5ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2c9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1441822226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8025040 con 0x5559363b25b0 2024-04-25T08:49:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 -- 172.21.15.46:0/1441822226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ca8018050 con 0x5559363b25b0 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1441822226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f2ca8005270 con 0x5559363b25b0 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1441822226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ca8038050 con 0x5559363b25b0 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 10 client.8067 fetch_fsmap learned FSMap version 75 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 -- 172.21.15.46:0/1441822226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55593615d170 con 0x5559363b25b0 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1441822226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8018050 con 0x5559363b25b0 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1441822226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2ca803b020 con 0x5559363b25b0 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 10 client.8067 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 10 client.8067 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 20 client.8067 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5559363cd880 0x5559363cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 4 client.8067 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2cb9547000 1 -- 172.21.15.46:0/1441822226 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x5559361c35b0 con 0x5559363cd880 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.899+0000 7f2caf400640 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5559363cd880 0x5559363cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2caf400640 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5559363cd880 0x5559363cdc90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ed7c470541b18d server_cookie=c487a9cabb1d0e1d in_seq=0 out_seq=0 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2c9cc00640 10 client.8067 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2c9cc00640 1 -- 172.21.15.46:0/1441822226 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5559361c35b0 con 0x5559363cd880 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2c9cc00640 10 client.8067 handle_command_reply: tid=0 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 client.8067 shutdown 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 -- 172.21.15.46:0/1441822226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 msgr2=0x5559363cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x5559363cce90 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8009010 tx=0x7f2ca80015d0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 -- 172.21.15.46:0/1441822226 shutdown_connections 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5559363cd880 0x5559363cdc90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559363b25b0 0x5559363cce90 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5559363b1c90 0x5559364121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 --2- 172.21.15.46:0/1441822226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555936368290 0x555936411c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 -- 172.21.15.46:0/1441822226 >> 172.21.15.46:0/1441822226 conn(0x555936364240 msgr2=0x55593636cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2c9cc00640 0 client.8067 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 -- 172.21.15.46:0/1441822226 shutdown_connections 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 1 -- 172.21.15.46:0/1441822226 wait complete. 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb9547000 20 client.8067 trim_cache size 0 max 16384 2024-04-25T08:49:57.899 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:49:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:49:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "6ab70f18-ef8b-486f-a8c3-fe16de913b10", 2024-04-25T08:49:57.900 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:49:57.900 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 -- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c8c066f60 msgr2=0x7f2c8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c8c066f60 0x7f2c8c069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2c980ae000 tx=0x7f2c980a7000 comp rx=0 tx=0).stop 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 -- 172.21.15.46:0/2066080305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00ce4d0 msgr2=0x7f2cb00af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00ce4d0 0x7f2cb00af100 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000e7f0 tx=0x7f2ca000ecb0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 -- 172.21.15.46:0/2066080305 shutdown_connections 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2c8c066f60 0x7f2c8c069420 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cb0141580 0x7f2cb01585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2cb0140ee0 0x7f2cb0143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 --2- 172.21.15.46:0/2066080305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb00ce4d0 0x7f2cb00af100 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:57.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 -- 172.21.15.46:0/2066080305 >> 172.21.15.46:0/2066080305 conn(0x7f2cb0067050 msgr2=0x7f2cb0071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 -- 172.21.15.46:0/2066080305 shutdown_connections 2024-04-25T08:49:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:57.903+0000 7f2cb6000640 1 -- 172.21.15.46:0/2066080305 wait complete. 2024-04-25T08:49:57.902 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:49:57.903+0000 7fe460200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f082812100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:49:57.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:49:57.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:49:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22fc00640 1 Processor -- start 2024-04-25T08:49:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22fc00640 1 -- start start 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22fc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff22806dc70 0x7ff2280cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22fc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2280cf1b0 0x7ff22815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22fc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2280cd930 con 0x7ff228145310 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22fc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2280cdbd0 con 0x7ff2280cf1b0 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22fc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280cdeb0 con 0x7ff22806dc70 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22dc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2280cf1b0 0x7ff22815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22dc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2280cf1b0 0x7ff22815d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33758/0 (socket says 172.21.15.46:33758) 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22dc00640 1 -- 172.21.15.46:0/3922773540 learned_addr learned my addr 172.21.15.46:0/3922773540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff21fe00640 1 --2- 172.21.15.46:0/3922773540 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff22806dc70 0x7ff2280cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22dc00640 1 -- 172.21.15.46:0/3922773540 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff22806dc70 msgr2=0x7ff2280cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22dc00640 1 --2- 172.21.15.46:0/3922773540 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff22806dc70 0x7ff2280cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22dc00640 1 -- 172.21.15.46:0/3922773540 >> v1:172.21.15.82:6789/0 conn(0x7ff228145310 legacy=0x7ff22806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.067+0000 7ff22dc00640 1 -- 172.21.15.46:0/3922773540 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2280ce190 con 0x7ff2280cf1b0 2024-04-25T08:49:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff22dc00640 1 --2- 172.21.15.46:0/3922773540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280cf1b0 0x7ff22815d090 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff220009e50 tx=0x7ff22001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc447d88780cf7db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff227e00640 1 -- 172.21.15.46:0/3922773540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2200a57e0 con 0x7ff2280cf1b0 2024-04-25T08:49:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff227e00640 1 -- 172.21.15.46:0/3922773540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2200a5980 con 0x7ff2280cf1b0 2024-04-25T08:49:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff227e00640 1 -- 172.21.15.46:0/3922773540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2200a5d50 con 0x7ff2280cf1b0 2024-04-25T08:49:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff22fc00640 1 -- 172.21.15.46:0/3922773540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280cf1b0 msgr2=0x7ff22815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff22fc00640 1 --2- 172.21.15.46:0/3922773540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280cf1b0 0x7ff22815d090 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff220009e50 tx=0x7ff22001ca00 comp rx=0 tx=0).stop 2024-04-25T08:49:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff22fc00640 1 -- 172.21.15.46:0/3922773540 shutdown_connections 2024-04-25T08:49:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff22fc00640 1 --2- 172.21.15.46:0/3922773540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280cf1b0 0x7ff22815d090 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff22fc00640 1 --2- 172.21.15.46:0/3922773540 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff22806dc70 0x7ff2280cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff22fc00640 1 -- 172.21.15.46:0/3922773540 >> 172.21.15.46:0/3922773540 conn(0x7ff22806ac10 msgr2=0x7ff22806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff22fc00640 1 -- 172.21.15.46:0/3922773540 shutdown_connections 2024-04-25T08:49:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.071+0000 7ff22fc00640 1 -- 172.21.15.46:0/3922773540 wait complete. 2024-04-25T08:49:58.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 Processor -- start 2024-04-25T08:49:58.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 -- start start 2024-04-25T08:49:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2280cf1b0 0x7ff2280af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff228145310 0x7ff2280b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280b40c0 0x7ff228159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2281688a0 con 0x7ff2280b40c0 2024-04-25T08:49:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff228168720 con 0x7ff2280cf1b0 2024-04-25T08:49:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff228168a20 con 0x7ff228145310 2024-04-25T08:49:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280b40c0 0x7ff228159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280b40c0 0x7ff228159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33772/0 (socket says 172.21.15.46:33772) 2024-04-25T08:49:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22dc00640 1 -- 172.21.15.46:0/645563250 learned_addr learned my addr 172.21.15.46:0/645563250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff21fe00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff228145310 0x7ff2280b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22d200640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2280cf1b0 0x7ff2280af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22dc00640 1 -- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff228145310 msgr2=0x7ff2280b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22dc00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff228145310 0x7ff2280b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22dc00640 1 -- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2280cf1b0 msgr2=0x7ff2280af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22dc00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2280cf1b0 0x7ff2280af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22dc00640 1 -- 172.21.15.46:0/645563250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2280b16a0 con 0x7ff2280b40c0 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22d200640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2280cf1b0 0x7ff2280af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22dc00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280b40c0 0x7ff228159ca0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff2200af040 tx=0x7ff2200197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff225600640 1 -- 172.21.15.46:0/645563250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2200bc020 con 0x7ff2280b40c0 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff225600640 1 -- 172.21.15.46:0/645563250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2200b8690 con 0x7ff2280b40c0 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff225600640 1 -- 172.21.15.46:0/645563250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2200b8a80 con 0x7ff2280b40c0 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 -- 172.21.15.46:0/645563250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2280b26f0 con 0x7ff2280b40c0 2024-04-25T08:49:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.075+0000 7ff22fc00640 1 -- 172.21.15.46:0/645563250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2280b2cb0 con 0x7ff2280b40c0 2024-04-25T08:49:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff225600640 1 -- 172.21.15.46:0/645563250 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2200bf020 con 0x7ff2280b40c0 2024-04-25T08:49:58.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff225600640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff204066f90 0x7ff204069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff225600640 1 -- 172.21.15.46:0/645563250 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7ff220120420 con 0x7ff2280b40c0 2024-04-25T08:49:58.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff2331a5000 1 Processor -- start 2024-04-25T08:49:58.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff22d200640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff204066f90 0x7ff204069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff2331a5000 1 -- start start 2024-04-25T08:49:58.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff22d200640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff204066f90 0x7ff204069450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff2280b00a0 tx=0x7ff218009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 0x555e084a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08459b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 0x555e0845a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff2331a5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e082a6ff0 con 0x555e084a42f0 2024-04-25T08:49:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff2331a5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0822f150 con 0x555e0845a0a0 2024-04-25T08:49:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff2331a5000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e082a3090 con 0x555e084a39d0 2024-04-25T08:49:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08459b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08459b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33782/0 (socket says 172.21.15.46:33782) 2024-04-25T08:49:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.079+0000 7ff21fe00640 1 -- 172.21.15.46:0/514464598 learned_addr learned my addr 172.21.15.46:0/514464598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff22dc00640 1 --2- 172.21.15.46:0/514464598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 0x555e0845a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff22d200640 1 --2- 172.21.15.46:0/514464598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 0x555e084a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff21fe00640 1 -- 172.21.15.46:0/514464598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 msgr2=0x555e084a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:58.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff21fe00640 1 --2- 172.21.15.46:0/514464598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 0x555e084a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff21fe00640 1 -- 172.21.15.46:0/514464598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 msgr2=0x555e0845a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff21fe00640 1 --2- 172.21.15.46:0/514464598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 0x555e0845a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff21fe00640 1 -- 172.21.15.46:0/514464598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e0843d0d0 con 0x555e084a42f0 2024-04-25T08:49:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff21fe00640 1 --2- 172.21.15.46:0/514464598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08459b60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff210007940 tx=0x7ff210007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca7ed6a770aeefd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff21d600640 1 -- 172.21.15.46:0/514464598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210018020 con 0x555e084a42f0 2024-04-25T08:49:58.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff21d600640 1 -- 172.21.15.46:0/514464598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff210004300 con 0x555e084a42f0 2024-04-25T08:49:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff21d600640 1 -- 172.21.15.46:0/514464598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210010b30 con 0x555e084a42f0 2024-04-25T08:49:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 -- 172.21.15.46:0/514464598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 msgr2=0x555e08459b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 --2- 172.21.15.46:0/514464598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08459b60 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7ff210007940 tx=0x7ff210007e00 comp rx=0 tx=0).stop 2024-04-25T08:49:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 -- 172.21.15.46:0/514464598 shutdown_connections 2024-04-25T08:49:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 --2- 172.21.15.46:0/514464598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 0x555e0845a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 --2- 172.21.15.46:0/514464598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08459b60 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 --2- 172.21.15.46:0/514464598 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 0x555e084a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 -- 172.21.15.46:0/514464598 >> 172.21.15.46:0/514464598 conn(0x555e08456020 msgr2=0x555e08458440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:58.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 -- 172.21.15.46:0/514464598 shutdown_connections 2024-04-25T08:49:58.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 -- 172.21.15.46:0/514464598 wait complete. 2024-04-25T08:49:58.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 -- start start 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 Processor -- start 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e0845a0a0 0x555e084887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a39d0 0x555e08488d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08489270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e0826b250 con 0x555e084a42f0 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e08261690 con 0x555e084a39d0 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e0822fa40 con 0x555e0845a0a0 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff2331a5000 10 client.0 _ll_register_callbacks cb 0x555e08444e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff22dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08489270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff22dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08489270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33792/0 (socket says 172.21.15.46:33792) 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.083+0000 7ff22dc00640 1 -- 172.21.15.46:0/3284104544 learned_addr learned my addr 172.21.15.46:0/3284104544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff21fe00640 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a39d0 0x555e08488d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22d200640 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e0845a0a0 0x555e084887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22dc00640 1 -- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e0845a0a0 msgr2=0x555e084887f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22dc00640 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e0845a0a0 0x555e084887f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22dc00640 1 -- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a39d0 msgr2=0x555e08488d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22dc00640 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a39d0 0x555e08488d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22dc00640 1 -- 172.21.15.46:0/3284104544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e083541c0 con 0x555e084a42f0 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22d200640 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e0845a0a0 0x555e084887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff21fe00640 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a39d0 0x555e08488d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22dc00640 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08489270 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff22001cee0 tx=0x7ff2200f04f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e268b7fb087960d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 1 -- 172.21.15.46:0/3284104544 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220108070 con 0x555e084a42f0 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 1 -- 172.21.15.46:0/3284104544 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7ff2200ed200 con 0x555e084a42f0 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 -- 172.21.15.46:0/3284104544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff210004300 con 0x555e084a42f0 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 1 -- 172.21.15.46:0/3284104544 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2200fe050 con 0x555e084a42f0 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 10 client.8079 fetch_fsmap learned FSMap version 75 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 -- 172.21.15.46:0/3284104544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x555e08350170 con 0x555e084a42f0 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 1 -- 172.21.15.46:0/3284104544 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff210004300 con 0x555e084a42f0 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 1 -- 172.21.15.46:0/3284104544 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff2200c9060 con 0x555e084a42f0 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 10 client.8079 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 10 client.8079 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 20 client.8079 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555e084b3c80 0x555e084b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 4 client.8079 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 -- 172.21.15.46:0/3284104544 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e08242c60 con 0x555e084b3c80 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22d200640 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555e084b3c80 0x555e084b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff22d200640 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555e084b3c80 0x555e084b6070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf128e2a5a55d6 server_cookie=4ea9d97fa769ca2 in_seq=0 out_seq=0 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 10 client.8079 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:58.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 1 -- 172.21.15.46:0/3284104544 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e08242c60 con 0x555e084b3c80 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 10 client.8079 handle_command_reply: tid=0 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 client.8079 shutdown 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 -- 172.21.15.46:0/3284104544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 msgr2=0x555e08489270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08489270 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff22001cee0 tx=0x7ff2200f04f0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 -- 172.21.15.46:0/3284104544 shutdown_connections 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555e084b3c80 0x555e084b6070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a42f0 0x555e08489270 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a39d0 0x555e08488d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 --2- 172.21.15.46:0/3284104544 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e0845a0a0 0x555e084887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff216000640 0 client.8079 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 -- 172.21.15.46:0/3284104544 >> 172.21.15.46:0/3284104544 conn(0x555e08456020 msgr2=0x555e08458ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 -- 172.21.15.46:0/3284104544 shutdown_connections 2024-04-25T08:49:58.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 1 -- 172.21.15.46:0/3284104544 wait complete. 2024-04-25T08:49:58.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.087+0000 7ff2331a5000 20 client.8079 trim_cache size 0 max 16384 2024-04-25T08:49:58.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff2331a5000 1 Processor -- start 2024-04-25T08:49:58.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff2331a5000 1 -- start start 2024-04-25T08:49:58.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 0x555e08500300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a39d0 0x555e08500840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a42f0 0x555e08500d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff2331a5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff228203b80 con 0x555e084a39d0 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff2331a5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff228202d30 con 0x555e0845a0a0 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff2331a5000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280aaec0 con 0x555e084a42f0 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a39d0 0x555e08500840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff21fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a39d0 0x555e08500840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33796/0 (socket says 172.21.15.46:33796) 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff21fe00640 1 -- 172.21.15.46:0/1722400902 learned_addr learned my addr 172.21.15.46:0/1722400902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff22d200640 1 --2- 172.21.15.46:0/1722400902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 0x555e08500300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff22dc00640 1 --2- 172.21.15.46:0/1722400902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a42f0 0x555e08500d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff21fe00640 1 -- 172.21.15.46:0/1722400902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a42f0 msgr2=0x555e08500d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:58.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff21fe00640 1 --2- 172.21.15.46:0/1722400902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a42f0 0x555e08500d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff21fe00640 1 -- 172.21.15.46:0/1722400902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 msgr2=0x555e08500300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:58.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff21fe00640 1 --2- 172.21.15.46:0/1722400902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 0x555e08500300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.103+0000 7ff21fe00640 1 -- 172.21.15.46:0/1722400902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e082edde0 con 0x555e084a39d0 2024-04-25T08:49:58.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff21fe00640 1 --2- 172.21.15.46:0/1722400902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a39d0 0x555e08500840 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff210006b20 tx=0x7ff210009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33476d96dfbaaa8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff21cc00640 1 -- 172.21.15.46:0/1722400902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210018050 con 0x555e084a39d0 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff21cc00640 1 -- 172.21.15.46:0/1722400902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff210004300 con 0x555e084a39d0 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff21cc00640 1 -- 172.21.15.46:0/1722400902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff210004510 con 0x555e084a39d0 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 -- 172.21.15.46:0/1722400902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a39d0 msgr2=0x555e08500840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 --2- 172.21.15.46:0/1722400902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a39d0 0x555e08500840 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff210006b20 tx=0x7ff210009000 comp rx=0 tx=0).stop 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 -- 172.21.15.46:0/1722400902 shutdown_connections 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 --2- 172.21.15.46:0/1722400902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a42f0 0x555e08500d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 --2- 172.21.15.46:0/1722400902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e084a39d0 0x555e08500840 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 --2- 172.21.15.46:0/1722400902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e0845a0a0 0x555e08500300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 -- 172.21.15.46:0/1722400902 >> 172.21.15.46:0/1722400902 conn(0x555e08456020 msgr2=0x555e084b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 -- 172.21.15.46:0/1722400902 shutdown_connections 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 -- 172.21.15.46:0/1722400902 wait complete. 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 -- start start 2024-04-25T08:49:58.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 Processor -- start 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e0845a0a0 0x555e084b6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 0x555e084b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a42f0 0x555e084b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e08241e00 con 0x555e0845a0a0 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2280ab4c0 con 0x555e084a42f0 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e082a6300 con 0x555e084a39d0 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff2331a5000 10 client.0 _ll_register_callbacks cb 0x555e08425040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e0845a0a0 0x555e084b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e0845a0a0 0x555e084b6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33806/0 (socket says 172.21.15.46:33806) 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22d200640 1 -- 172.21.15.46:0/510907588 learned_addr learned my addr 172.21.15.46:0/510907588 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff21fe00640 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 0x555e084b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22dc00640 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a42f0 0x555e084b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22d200640 1 -- 172.21.15.46:0/510907588 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 msgr2=0x555e084b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:58.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22d200640 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 0x555e084b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22d200640 1 -- 172.21.15.46:0/510907588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a42f0 msgr2=0x555e084b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:49:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22d200640 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a42f0 0x555e084b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22d200640 1 -- 172.21.15.46:0/510907588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e082488e0 con 0x555e0845a0a0 2024-04-25T08:49:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22dc00640 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a42f0 0x555e084b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:49:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff22d200640 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e0845a0a0 0x555e084b6770 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff218005400 tx=0x7ff218000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be45f7cc4e0a16fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:49:58.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.107+0000 7ff216000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:49:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff216000640 1 -- 172.21.15.46:0/510907588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff218003d20 con 0x555e0845a0a0 2024-04-25T08:49:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff216000640 1 -- 172.21.15.46:0/510907588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7ff218010040 con 0x555e0845a0a0 2024-04-25T08:49:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 -- 172.21.15.46:0/510907588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff210004300 con 0x555e0845a0a0 2024-04-25T08:49:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff216000640 1 -- 172.21.15.46:0/510907588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff218036070 con 0x555e0845a0a0 2024-04-25T08:49:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 10 client.8085 fetch_fsmap learned FSMap version 75 2024-04-25T08:49:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 -- 172.21.15.46:0/510907588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x555e083008f0 con 0x555e0845a0a0 2024-04-25T08:49:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff216000640 1 -- 172.21.15.46:0/510907588 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff210004300 con 0x555e0845a0a0 2024-04-25T08:49:58.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff216000640 1 -- 172.21.15.46:0/510907588 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff21802d020 con 0x555e0845a0a0 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 10 client.8085 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 10 client.8085 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 20 client.8085 populate_metadata read hostname 'smithi046' 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555e08509f50 0x555e0850c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 4 client.8085 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 -- 172.21.15.46:0/510907588 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e082b4530 con 0x555e08509f50 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff21fe00640 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555e08509f50 0x555e0850c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff21fe00640 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555e08509f50 0x555e0850c340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5feb6db53332fff4 server_cookie=687c908471f74739 in_seq=0 out_seq=0 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff216000640 10 client.8085 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff216000640 1 -- 172.21.15.46:0/510907588 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x555e082b4530 con 0x555e08509f50 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff216000640 10 client.8085 handle_command_reply: tid=0 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 client.8085 shutdown 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 -- 172.21.15.46:0/510907588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e0845a0a0 msgr2=0x555e084b6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e0845a0a0 0x555e084b6770 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff218005400 tx=0x7ff218000bd0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 -- 172.21.15.46:0/510907588 shutdown_connections 2024-04-25T08:49:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x555e08509f50 0x555e0850c340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e084a42f0 0x555e084b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555e084a39d0 0x555e084b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff216000640 0 client.8085 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:49:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 --2- 172.21.15.46:0/510907588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555e0845a0a0 0x555e084b6770 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 -- 172.21.15.46:0/510907588 >> 172.21.15.46:0/510907588 conn(0x555e08456020 msgr2=0x555e084a20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 -- 172.21.15.46:0/510907588 shutdown_connections 2024-04-25T08:49:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 1 -- 172.21.15.46:0/510907588 wait complete. 2024-04-25T08:49:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff2331a5000 20 client.8085 trim_cache size 0 max 16384 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout: "6ab70f18-ef8b-486f-a8c3-fe16de913b10": { 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir", 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "6ab70f18-ef8b-486f-a8c3-fe16de913b10", 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:49:58.107 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:49:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 -- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff204066f90 msgr2=0x7ff204069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff204066f90 0x7ff204069450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff2280b00a0 tx=0x7ff218009040 comp rx=0 tx=0).stop 2024-04-25T08:49:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 -- 172.21.15.46:0/645563250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280b40c0 msgr2=0x7ff228159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:49:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280b40c0 0x7ff228159ca0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff2200af040 tx=0x7ff2200197a0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 -- 172.21.15.46:0/645563250 shutdown_connections 2024-04-25T08:49:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff204066f90 0x7ff204069450 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2280b40c0 0x7ff228159ca0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff228145310 0x7ff2280b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 --2- 172.21.15.46:0/645563250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2280cf1b0 0x7ff2280af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:49:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 -- 172.21.15.46:0/645563250 >> 172.21.15.46:0/645563250 conn(0x7ff22806ac10 msgr2=0x7ff2280cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:49:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 -- 172.21.15.46:0/645563250 shutdown_connections 2024-04-25T08:49:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:49:58.111+0000 7ff22fc00640 1 -- 172.21.15.46:0/645563250 wait complete. 2024-04-25T08:49:58.122 INFO:tasks.cephfs.filesystem:scrub status for tag:6ab70f18-ef8b-486f-a8c3-fe16de913b10 - {'path': '/dir', 'tag': '6ab70f18-ef8b-486f-a8c3-fe16de913b10', 'options': 'recursive,repair'} 2024-04-25T08:50:21.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:50:21.022 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:50:28.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:28.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:50:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85cca00640 1 Processor -- start 2024-04-25T08:50:28.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85cca00640 1 -- start start 2024-04-25T08:50:28.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85cca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85c8145310 0x7f85c806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85cca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f85c806dc70 0x7f85c80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85cca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80cd930 con 0x7f85c80cf1b0 2024-04-25T08:50:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85cca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80cdbd0 con 0x7f85c806dc70 2024-04-25T08:50:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85cca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80cdeb0 con 0x7f85c8145310 2024-04-25T08:50:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85bfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f85c806dc70 0x7f85c80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85bfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f85c806dc70 0x7f85c80cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37860/0 (socket says 172.21.15.46:37860) 2024-04-25T08:50:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85c6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f85c80cf1b0 0x7f85c815d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57164/0 (socket says 172.21.15.46:57164) 2024-04-25T08:50:28.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85bfe00640 1 -- 172.21.15.46:0/199680786 learned_addr learned my addr 172.21.15.46:0/199680786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85bfe00640 1 -- 172.21.15.46:0/199680786 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85c8145310 msgr2=0x7f85c806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85bfe00640 1 --2- 172.21.15.46:0/199680786 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85c8145310 0x7f85c806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85bfe00640 1 -- 172.21.15.46:0/199680786 >> v1:172.21.15.46:6789/0 conn(0x7f85c80cf1b0 legacy=0x7f85c815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.291+0000 7f85bfe00640 1 -- 172.21.15.46:0/199680786 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80ce190 con 0x7f85c806dc70 2024-04-25T08:50:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85bfe00640 1 --2- 172.21.15.46:0/199680786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c806dc70 0x7f85c80cec70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009870 tx=0x7f85b001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354485d6764c2d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85c5400640 1 -- 172.21.15.46:0/199680786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00a95f0 con 0x7f85c806dc70 2024-04-25T08:50:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85c5400640 1 -- 172.21.15.46:0/199680786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b00a9790 con 0x7f85c806dc70 2024-04-25T08:50:28.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85c5400640 1 -- 172.21.15.46:0/199680786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00a9b60 con 0x7f85c806dc70 2024-04-25T08:50:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 -- 172.21.15.46:0/199680786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c806dc70 msgr2=0x7f85c80cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 --2- 172.21.15.46:0/199680786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c806dc70 0x7f85c80cec70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f85b0009870 tx=0x7f85b001c530 comp rx=0 tx=0).stop 2024-04-25T08:50:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 -- 172.21.15.46:0/199680786 shutdown_connections 2024-04-25T08:50:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 --2- 172.21.15.46:0/199680786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c806dc70 0x7f85c80cec70 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 --2- 172.21.15.46:0/199680786 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85c8145310 0x7f85c806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 -- 172.21.15.46:0/199680786 >> 172.21.15.46:0/199680786 conn(0x7f85c806ac10 msgr2=0x7f85c806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 -- 172.21.15.46:0/199680786 shutdown_connections 2024-04-25T08:50:28.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 -- 172.21.15.46:0/199680786 wait complete. 2024-04-25T08:50:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 Processor -- start 2024-04-25T08:50:28.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.295+0000 7f85cca00640 1 -- start start 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85cca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85c80cf1b0 0x7f85c8157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85cca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c8145310 0x7f85c81411b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85cca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85c80b2820 0x7f85c81546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85cca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c815d920 con 0x7f85c8145310 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85cca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c815d7a0 con 0x7f85c80b2820 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85cca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8168510 con 0x7f85c80cf1b0 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c8145310 0x7f85c81411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c8145310 0x7f85c81411b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37872/0 (socket says 172.21.15.46:37872) 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85bfe00640 1 -- 172.21.15.46:0/1806548253 learned_addr learned my addr 172.21.15.46:0/1806548253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85c5e00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85c80cf1b0 0x7f85c8157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85c6800640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85c80b2820 0x7f85c81546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85bfe00640 1 -- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85c80cf1b0 msgr2=0x7f85c8157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85bfe00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85c80cf1b0 0x7f85c8157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85bfe00640 1 -- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85c80b2820 msgr2=0x7f85c81546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85bfe00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85c80b2820 0x7f85c81546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85bfe00640 1 -- 172.21.15.46:0/1806548253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80b2fc0 con 0x7f85c8145310 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85c6800640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85c80b2820 0x7f85c81546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85bfe00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c8145310 0x7f85c81411b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f85b00ab040 tx=0x7f85b001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85be000640 1 -- 172.21.15.46:0/1806548253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00b1800 con 0x7f85c8145310 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85be000640 1 -- 172.21.15.46:0/1806548253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b00b19c0 con 0x7f85c8145310 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85be000640 1 -- 172.21.15.46:0/1806548253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00b1db0 con 0x7f85c8145310 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85cca00640 1 -- 172.21.15.46:0/1806548253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c80b32a0 con 0x7f85c8145310 2024-04-25T08:50:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85cca00640 1 -- 172.21.15.46:0/1806548253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c80b3860 con 0x7f85c8145310 2024-04-25T08:50:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.299+0000 7f85be000640 1 -- 172.21.15.46:0/1806548253 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85b00a8350 con 0x7f85c8145310 2024-04-25T08:50:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85be000640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f85a4066f60 0x7f85a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85be000640 1 -- 172.21.15.46:0/1806548253 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f85b011f8a0 con 0x7f85c8145310 2024-04-25T08:50:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f85a4066f60 0x7f85a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85cfdd7000 1 Processor -- start 2024-04-25T08:50:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85cfdd7000 1 -- start start 2024-04-25T08:50:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f85a4066f60 0x7f85a4069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f85b8006fd0 tx=0x7f85b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1aabdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aac2f0 0x5590a1a61b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 0x5590a1a62480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a18aeff0 con 0x5590a1aab9d0 2024-04-25T08:50:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a1837150 con 0x5590a1a620a0 2024-04-25T08:50:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590a18ab090 con 0x5590a1aac2f0 2024-04-25T08:50:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1aabdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1aabdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37886/0 (socket says 172.21.15.46:37886) 2024-04-25T08:50:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 -- 172.21.15.46:0/3610937981 learned_addr learned my addr 172.21.15.46:0/3610937981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c6800640 1 --2- 172.21.15.46:0/3610937981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 0x5590a1a62480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85bfe00640 1 --2- 172.21.15.46:0/3610937981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aac2f0 0x5590a1a61b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 -- 172.21.15.46:0/3610937981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aac2f0 msgr2=0x5590a1a61b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 --2- 172.21.15.46:0/3610937981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aac2f0 0x5590a1a61b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 -- 172.21.15.46:0/3610937981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 msgr2=0x5590a1a62480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 --2- 172.21.15.46:0/3610937981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 0x5590a1a62480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 -- 172.21.15.46:0/3610937981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a1a450d0 con 0x5590a1aab9d0 2024-04-25T08:50:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85c5e00640 1 --2- 172.21.15.46:0/3610937981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1aabdb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f85b800d860 tx=0x7f85b8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deab5bd10d8833f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85b6a00640 1 -- 172.21.15.46:0/3610937981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b801a040 con 0x5590a1aab9d0 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85b6a00640 1 -- 172.21.15.46:0/3610937981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8023070 con 0x5590a1aab9d0 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.303+0000 7f85b6a00640 1 -- 172.21.15.46:0/3610937981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8014990 con 0x5590a1aab9d0 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3610937981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 msgr2=0x5590a1aabdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3610937981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1aabdb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f85b800d860 tx=0x7f85b8010000 comp rx=0 tx=0).stop 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3610937981 shutdown_connections 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3610937981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 0x5590a1a62480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3610937981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aac2f0 0x5590a1a61b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3610937981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1aabdb0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3610937981 >> 172.21.15.46:0/3610937981 conn(0x5590a1a5e020 msgr2=0x5590a1a60440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3610937981 shutdown_connections 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3610937981 wait complete. 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- start start 2024-04-25T08:50:28.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 Processor -- start 2024-04-25T08:50:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 0x5590a1a907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1a90d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 0x5590a1a91270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a1873250 con 0x5590a1aab9d0 2024-04-25T08:50:28.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a1869690 con 0x5590a1aac2f0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590a1837a40 con 0x5590a1a620a0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 10 client.0 _ll_register_callbacks cb 0x5590a1a4ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1a90d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1a90d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37894/0 (socket says 172.21.15.46:37894) 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85bfe00640 1 -- 172.21.15.46:0/1105836380 learned_addr learned my addr 172.21.15.46:0/1105836380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85c5e00640 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 0x5590a1a907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85c6800640 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 0x5590a1a91270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85bfe00640 1 -- 172.21.15.46:0/1105836380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 msgr2=0x5590a1a907f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85bfe00640 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 0x5590a1a907f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85bfe00640 1 -- 172.21.15.46:0/1105836380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 msgr2=0x5590a1a91270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85bfe00640 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 0x5590a1a91270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85bfe00640 1 -- 172.21.15.46:0/1105836380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590a195c1c0 con 0x5590a1aab9d0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85bfe00640 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1a90d30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f85b001a160 tx=0x7f85b00f0560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2bfb087a2eba37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85ab400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85ab400640 1 -- 172.21.15.46:0/1105836380 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00f8570 con 0x5590a1aab9d0 2024-04-25T08:50:28.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85ab400640 1 -- 172.21.15.46:0/1105836380 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f85b0119de0 con 0x5590a1aab9d0 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- 172.21.15.46:0/1105836380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85b8023070 con 0x5590a1aab9d0 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85ab400640 1 -- 172.21.15.46:0/1105836380 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85b00f60b0 con 0x5590a1aab9d0 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 10 client.8100 fetch_fsmap learned FSMap version 75 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 1 -- 172.21.15.46:0/1105836380 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5590a1958170 con 0x5590a1aab9d0 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85ab400640 1 -- 172.21.15.46:0/1105836380 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b8023070 con 0x5590a1aab9d0 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85ab400640 1 -- 172.21.15.46:0/1105836380 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85b00ef2b0 con 0x5590a1aab9d0 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 10 client.8100 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.307+0000 7f85cfdd7000 10 client.8100 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 20 client.8100 populate_metadata read hostname 'smithi046' 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5590a1abbc80 0x5590a1abe070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 4 client.8100 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 -- 172.21.15.46:0/1105836380 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590a184ac60 con 0x5590a1abbc80 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85c5e00640 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5590a1abbc80 0x5590a1abe070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85c5e00640 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5590a1abbc80 0x5590a1abe070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca30ab72d9d0c821 server_cookie=822591ffd8944365 in_seq=0 out_seq=0 2024-04-25T08:50:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85ab400640 10 client.8100 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85ab400640 1 -- 172.21.15.46:0/1105836380 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590a184ac60 con 0x5590a1abbc80 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85ab400640 10 client.8100 handle_command_reply: tid=0 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 client.8100 shutdown 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 -- 172.21.15.46:0/1105836380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 msgr2=0x5590a1a90d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1a90d30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f85b001a160 tx=0x7f85b00f0560 comp rx=0 tx=0).stop 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 -- 172.21.15.46:0/1105836380 shutdown_connections 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5590a1abbc80 0x5590a1abe070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 0x5590a1a91270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1a90d30 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/1105836380 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 0x5590a1a907f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85ab400640 0 client.8100 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 -- 172.21.15.46:0/1105836380 >> 172.21.15.46:0/1105836380 conn(0x5590a1a5e020 msgr2=0x5590a1a60ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 -- 172.21.15.46:0/1105836380 shutdown_connections 2024-04-25T08:50:28.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 1 -- 172.21.15.46:0/1105836380 wait complete. 2024-04-25T08:50:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.311+0000 7f85cfdd7000 20 client.8100 trim_cache size 0 max 16384 2024-04-25T08:50:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 Processor -- start 2024-04-25T08:50:28.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 -- start start 2024-04-25T08:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 0x5590a1b08300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1b08840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 0x5590a1b08d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8203690 con 0x5590a1aab9d0 2024-04-25T08:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8202870 con 0x5590a1aac2f0 2024-04-25T08:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8156810 con 0x5590a1a620a0 2024-04-25T08:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1b08840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1b08840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37900/0 (socket says 172.21.15.46:37900) 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85bfe00640 1 -- 172.21.15.46:0/3112313434 learned_addr learned my addr 172.21.15.46:0/3112313434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85c5e00640 1 --2- 172.21.15.46:0/3112313434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 0x5590a1b08300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85c6800640 1 --2- 172.21.15.46:0/3112313434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 0x5590a1b08d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85bfe00640 1 -- 172.21.15.46:0/3112313434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 msgr2=0x5590a1b08300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85bfe00640 1 --2- 172.21.15.46:0/3112313434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 0x5590a1b08300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85bfe00640 1 -- 172.21.15.46:0/3112313434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 msgr2=0x5590a1b08d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85bfe00640 1 --2- 172.21.15.46:0/3112313434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 0x5590a1b08d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85bfe00640 1 -- 172.21.15.46:0/3112313434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a18f5de0 con 0x5590a1aab9d0 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85bfe00640 1 --2- 172.21.15.46:0/3112313434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1b08840 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f85b001abe0 tx=0x7f85b00be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bedd19ac513fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85b6000640 1 -- 172.21.15.46:0/3112313434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00f8570 con 0x5590a1aab9d0 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85b6000640 1 -- 172.21.15.46:0/3112313434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b00f6250 con 0x5590a1aab9d0 2024-04-25T08:50:28.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85b6000640 1 -- 172.21.15.46:0/3112313434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b00ef9a0 con 0x5590a1aab9d0 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3112313434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 msgr2=0x5590a1b08840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3112313434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1b08840 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f85b001abe0 tx=0x7f85b00be000 comp rx=0 tx=0).stop 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3112313434 shutdown_connections 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3112313434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1aac2f0 0x5590a1b08d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3112313434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aab9d0 0x5590a1b08840 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3112313434 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1a620a0 0x5590a1b08300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3112313434 >> 172.21.15.46:0/3112313434 conn(0x5590a1a5e020 msgr2=0x5590a1ab8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3112313434 shutdown_connections 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.327+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3112313434 wait complete. 2024-04-25T08:50:28.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 -- start start 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 Processor -- start 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 0x5590a1abe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aab9d0 0x5590a1abe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aac2f0 0x5590a1abee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a1849e00 con 0x5590a1aac2f0 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8156e10 con 0x5590a1a620a0 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590a18ae300 con 0x5590a1aab9d0 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 10 client.0 _ll_register_callbacks cb 0x5590a1a2d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aac2f0 0x5590a1abee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aac2f0 0x5590a1abee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37916/0 (socket says 172.21.15.46:37916) 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c6800640 1 -- 172.21.15.46:0/3774936391 learned_addr learned my addr 172.21.15.46:0/3774936391 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85bfe00640 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aab9d0 0x5590a1abe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c5e00640 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 0x5590a1abe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c6800640 1 -- 172.21.15.46:0/3774936391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aab9d0 msgr2=0x5590a1abe8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c6800640 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aab9d0 0x5590a1abe8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c6800640 1 -- 172.21.15.46:0/3774936391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 msgr2=0x5590a1abe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c6800640 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 0x5590a1abe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c6800640 1 -- 172.21.15.46:0/3774936391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590a18508e0 con 0x5590a1aac2f0 2024-04-25T08:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85c6800640 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aac2f0 0x5590a1abee00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f85c00072e0 tx=0x7f85c00077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d324e33ee8ff91a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85ab400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85ab400640 1 -- 172.21.15.46:0/3774936391 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0013a90 con 0x5590a1aac2f0 2024-04-25T08:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3774936391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85b00f6250 con 0x5590a1aac2f0 2024-04-25T08:50:28.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85ab400640 1 -- 172.21.15.46:0/3774936391 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f85c0013c30 con 0x5590a1aac2f0 2024-04-25T08:50:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85ab400640 1 -- 172.21.15.46:0/3774936391 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85c0024050 con 0x5590a1aac2f0 2024-04-25T08:50:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 10 client.8106 fetch_fsmap learned FSMap version 75 2024-04-25T08:50:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3774936391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5590a19088f0 con 0x5590a1aac2f0 2024-04-25T08:50:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85ab400640 1 -- 172.21.15.46:0/3774936391 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b00f6250 con 0x5590a1aac2f0 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85ab400640 1 -- 172.21.15.46:0/3774936391 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85c0027020 con 0x5590a1aac2f0 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 10 client.8106 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 10 client.8106 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 20 client.8106 populate_metadata read hostname 'smithi046' 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5590a1abf9e0 0x5590a1b13f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 4 client.8106 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.331+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3774936391 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590a18bc530 con 0x5590a1abf9e0 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85c5e00640 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5590a1abf9e0 0x5590a1b13f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85ab400640 10 client.8106 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85c5e00640 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5590a1abf9e0 0x5590a1b13f60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=791f9f1db1921d29 server_cookie=726eb7803be98a45 in_seq=0 out_seq=0 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85ab400640 1 -- 172.21.15.46:0/3774936391 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5590a18bc530 con 0x5590a1abf9e0 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85ab400640 10 client.8106 handle_command_reply: tid=0 2024-04-25T08:50:28.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 client.8106 shutdown 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3774936391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aac2f0 msgr2=0x5590a1abee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aac2f0 0x5590a1abee00 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f85c00072e0 tx=0x7f85c00077a0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3774936391 shutdown_connections 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x5590a1abf9e0 0x5590a1b13f60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590a1aac2f0 0x5590a1abee00 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5590a1aab9d0 0x5590a1abe8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 --2- 172.21.15.46:0/3774936391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590a1a620a0 0x5590a1abe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85ab400640 0 client.8106 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3774936391 >> 172.21.15.46:0/3774936391 conn(0x5590a1a5e020 msgr2=0x5590a1aaa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3774936391 shutdown_connections 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 1 -- 172.21.15.46:0/3774936391 wait complete. 2024-04-25T08:50:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cfdd7000 20 client.8106 trim_cache size 0 max 16384 2024-04-25T08:50:28.330 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:50:28.330 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:50:28.330 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:50:28.330 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 -- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f85a4066f60 msgr2=0x7f85a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f85a4066f60 0x7f85a4069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f85b8006fd0 tx=0x7f85b8008040 comp rx=0 tx=0).stop 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 -- 172.21.15.46:0/1806548253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c8145310 msgr2=0x7f85c81411b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c8145310 0x7f85c81411b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f85b00ab040 tx=0x7f85b001ad70 comp rx=0 tx=0).stop 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 -- 172.21.15.46:0/1806548253 shutdown_connections 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f85a4066f60 0x7f85a4069420 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85c80b2820 0x7f85c81546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85c8145310 0x7f85c81411b0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 --2- 172.21.15.46:0/1806548253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f85c80cf1b0 0x7f85c8157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 -- 172.21.15.46:0/1806548253 >> 172.21.15.46:0/1806548253 conn(0x7f85c806ac10 msgr2=0x7f85c8142f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 -- 172.21.15.46:0/1806548253 shutdown_connections 2024-04-25T08:50:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.335+0000 7f85cca00640 1 -- 172.21.15.46:0/1806548253 wait complete. 2024-04-25T08:50:28.347 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:50:28.347 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e35600640 1 Processor -- start 2024-04-25T08:50:28.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e35600640 1 -- start start 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e35600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 0x7f3e300aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e35600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e3006aff0 0x7f3e3015c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e35600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3e3015cd40 0x7f3e3015e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e35600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3015ded0 con 0x7f3e3006aff0 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e35600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e3015d2c0 con 0x7f3e3015cd40 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e35600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3015d5a0 con 0x7f3e3006ac10 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e26000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e3006aff0 0x7f3e3015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e26000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e3006aff0 0x7f3e3015c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37922/0 (socket says 172.21.15.46:37922) 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e26000640 1 -- 172.21.15.46:0/685279028 learned_addr learned my addr 172.21.15.46:0/685279028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e2ea00640 1 --2- 172.21.15.46:0/685279028 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 0x7f3e300aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e2f400640 1 --2- 172.21.15.46:0/685279028 >> v2:172.21.15.82:3300/0 conn(0x7f3e3015cd40 0x7f3e3015e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e26000640 1 -- 172.21.15.46:0/685279028 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 msgr2=0x7f3e300aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e26000640 1 --2- 172.21.15.46:0/685279028 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 0x7f3e300aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e26000640 1 -- 172.21.15.46:0/685279028 >> v2:172.21.15.82:3300/0 conn(0x7f3e3015cd40 msgr2=0x7f3e3015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e26000640 1 --2- 172.21.15.46:0/685279028 >> v2:172.21.15.82:3300/0 conn(0x7f3e3015cd40 0x7f3e3015e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.499+0000 7f3e26000640 1 -- 172.21.15.46:0/685279028 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3015d880 con 0x7f3e3006aff0 2024-04-25T08:50:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e26000640 1 --2- 172.21.15.46:0/685279028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3006aff0 0x7f3e3015c800 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3e300686d0 tx=0x7f3e1801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8351156e752b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e2e000640 1 -- 172.21.15.46:0/685279028 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e180ab5f0 con 0x7f3e3006aff0 2024-04-25T08:50:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e2e000640 1 -- 172.21.15.46:0/685279028 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e180b0070 con 0x7f3e3006aff0 2024-04-25T08:50:28.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e2e000640 1 -- 172.21.15.46:0/685279028 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e180ab9b0 con 0x7f3e3006aff0 2024-04-25T08:50:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e35600640 1 -- 172.21.15.46:0/685279028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3006aff0 msgr2=0x7f3e3015c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e35600640 1 --2- 172.21.15.46:0/685279028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3006aff0 0x7f3e3015c800 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f3e300686d0 tx=0x7f3e1801c520 comp rx=0 tx=0).stop 2024-04-25T08:50:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e35600640 1 -- 172.21.15.46:0/685279028 shutdown_connections 2024-04-25T08:50:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e35600640 1 --2- 172.21.15.46:0/685279028 >> v2:172.21.15.82:3300/0 conn(0x7f3e3015cd40 0x7f3e3015e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e35600640 1 --2- 172.21.15.46:0/685279028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3006aff0 0x7f3e3015c800 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e35600640 1 --2- 172.21.15.46:0/685279028 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 0x7f3e300aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e35600640 1 -- 172.21.15.46:0/685279028 >> 172.21.15.46:0/685279028 conn(0x7f3e30072880 msgr2=0x7f3e30099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e35600640 1 -- 172.21.15.46:0/685279028 shutdown_connections 2024-04-25T08:50:28.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.503+0000 7f3e35600640 1 -- 172.21.15.46:0/685279028 wait complete. 2024-04-25T08:50:28.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 Processor -- start 2024-04-25T08:50:28.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 -- start start 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 0x7f3e30138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e3006aff0 0x7f3e30133580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3015cd40 0x7f3e3013cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3015f2f0 con 0x7f3e3015cd40 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3006b450 con 0x7f3e3006aff0 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3015f470 con 0x7f3e3006ac10 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3015cd40 0x7f3e3013cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3015cd40 0x7f3e3013cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37934/0 (socket says 172.21.15.46:37934) 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2f400640 1 -- 172.21.15.46:0/3127169867 learned_addr learned my addr 172.21.15.46:0/3127169867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2ea00640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 0x7f3e30138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e26000640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e3006aff0 0x7f3e30133580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2f400640 1 -- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 msgr2=0x7f3e30138040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2f400640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 0x7f3e30138040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2f400640 1 -- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e3006aff0 msgr2=0x7f3e30133580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2f400640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e3006aff0 0x7f3e30133580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2f400640 1 -- 172.21.15.46:0/3127169867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e30132550 con 0x7f3e3015cd40 2024-04-25T08:50:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e2f400640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3015cd40 0x7f3e3013cf00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800d590 tx=0x7f3e2800da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e27400640 1 -- 172.21.15.46:0/3127169867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e280125f0 con 0x7f3e3015cd40 2024-04-25T08:50:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e27400640 1 -- 172.21.15.46:0/3127169867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e28019070 con 0x7f3e3015cd40 2024-04-25T08:50:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e27400640 1 -- 172.21.15.46:0/3127169867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e280129f0 con 0x7f3e3015cd40 2024-04-25T08:50:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e301327c0 con 0x7f3e3015cd40 2024-04-25T08:50:28.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e30133060 con 0x7f3e3015cd40 2024-04-25T08:50:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.507+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e30132990 con 0x7f3e3015cd40 2024-04-25T08:50:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.511+0000 7f3e27400640 1 -- 172.21.15.46:0/3127169867 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e28012b90 con 0x7f3e3015cd40 2024-04-25T08:50:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.511+0000 7f3e27400640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e0c066f60 0x7f3e0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.511+0000 7f3e27400640 1 -- 172.21.15.46:0/3127169867 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f3e28085610 con 0x7f3e3015cd40 2024-04-25T08:50:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.511+0000 7f3e2ea00640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e0c066f60 0x7f3e0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.511+0000 7f3e2ea00640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e0c066f60 0x7f3e0c069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3e20003ab0 tx=0x7f3e20009830 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.515+0000 7f3e27400640 1 -- 172.21.15.46:0/3127169867 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e28055c80 con 0x7f3e3015cd40 2024-04-25T08:50:28.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e30069080 con 0x7f3e3015cd40 2024-04-25T08:50:28.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e27400640 1 -- 172.21.15.46:0/3127169867 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f3e28059d20 con 0x7f3e3015cd40 2024-04-25T08:50:28.701 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:28.701 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:49:54.058324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1358086076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1358086076},{"type":"v1","addr":"172.21.15.46:6835","nonce":1358086076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:50:28.701 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 75 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e0c066f60 msgr2=0x7f3e0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e0c066f60 0x7f3e0c069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3e20003ab0 tx=0x7f3e20009830 comp rx=0 tx=0).stop 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3015cd40 msgr2=0x7f3e3013cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3015cd40 0x7f3e3013cf00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800d590 tx=0x7f3e2800da50 comp rx=0 tx=0).stop 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 shutdown_connections 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e0c066f60 0x7f3e0c069420 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e3015cd40 0x7f3e3013cf00 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e3006aff0 0x7f3e30133580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 --2- 172.21.15.46:0/3127169867 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e3006ac10 0x7f3e30138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 >> 172.21.15.46:0/3127169867 conn(0x7f3e30072880 msgr2=0x7f3e300b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 shutdown_connections 2024-04-25T08:50:28.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.711+0000 7f3e35600640 1 -- 172.21.15.46:0/3127169867 wait complete. 2024-04-25T08:50:28.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:50:28.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ec800640 1 Processor -- start 2024-04-25T08:50:28.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ec800640 1 -- start start 2024-04-25T08:50:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ec800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8e40cf340 0x7fd8e4166680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ec800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e415f2f0 con 0x7fd8e40cf890 2024-04-25T08:50:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ec800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e415f5d0 con 0x7fd8e40cc180 2024-04-25T08:50:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ec800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e415f8b0 con 0x7fd8e40cf340 2024-04-25T08:50:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8e9e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd8e40cf890 0x7fd8e40b2810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57166/0 (socket says 172.21.15.46:57166) 2024-04-25T08:50:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8e9e00640 1 -- 172.21.15.46:0/3927908644 learned_addr learned my addr 172.21.15.46:0/3927908644 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:28.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ea800640 1 --2- 172.21.15.46:0/3927908644 >> v2:172.21.15.46:3300/0 conn(0x7fd8e40cf340 0x7fd8e4166680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8e8a00640 1 -- 172.21.15.46:0/3927908644 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543792985 0 0) 0x7fd8e415f2f0 con 0x7fd8e40cf890 2024-04-25T08:50:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ea800640 1 -- 172.21.15.46:0/3927908644 >> v1:172.21.15.82:6789/0 conn(0x7fd8e40cc180 legacy=0x7fd8e40cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ea800640 1 -- 172.21.15.46:0/3927908644 >> v1:172.21.15.46:6789/0 conn(0x7fd8e40cf890 legacy=0x7fd8e40b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.875+0000 7fd8ea800640 1 -- 172.21.15.46:0/3927908644 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e415fb90 con 0x7fd8e40cf340 2024-04-25T08:50:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8ea800640 1 --2- 172.21.15.46:0/3927908644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 0x7fd8e4166680 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009d90 tx=0x7fd8e001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a95f2e18aa55c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8e8a00640 1 -- 172.21.15.46:0/3927908644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00b2070 con 0x7fd8e40cf340 2024-04-25T08:50:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8e8a00640 1 -- 172.21.15.46:0/3927908644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e00a5840 con 0x7fd8e40cf340 2024-04-25T08:50:28.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8e8a00640 1 -- 172.21.15.46:0/3927908644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00a5c10 con 0x7fd8e40cf340 2024-04-25T08:50:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8ec800640 1 -- 172.21.15.46:0/3927908644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 msgr2=0x7fd8e4166680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8ec800640 1 --2- 172.21.15.46:0/3927908644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 0x7fd8e4166680 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009d90 tx=0x7fd8e001c970 comp rx=0 tx=0).stop 2024-04-25T08:50:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8ec800640 1 -- 172.21.15.46:0/3927908644 shutdown_connections 2024-04-25T08:50:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8ec800640 1 --2- 172.21.15.46:0/3927908644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 0x7fd8e4166680 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8ec800640 1 -- 172.21.15.46:0/3927908644 >> 172.21.15.46:0/3927908644 conn(0x7fd8e4099a00 msgr2=0x7fd8e40b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8ec800640 1 -- 172.21.15.46:0/3927908644 shutdown_connections 2024-04-25T08:50:28.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.879+0000 7fd8ec800640 1 -- 172.21.15.46:0/3927908644 wait complete. 2024-04-25T08:50:28.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 Processor -- start 2024-04-25T08:50:28.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 -- start start 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8e40cc180 0x7fd8e41665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 0x7fd8e415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e40cf890 0x7fd8e40c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e4169b80 con 0x7fd8e40cf340 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e4169a00 con 0x7fd8e40cf890 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e4169d00 con 0x7fd8e40cc180 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 0x7fd8e415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 0x7fd8e415eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37950/0 (socket says 172.21.15.46:37950) 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9400640 1 -- 172.21.15.46:0/2318747868 learned_addr learned my addr 172.21.15.46:0/2318747868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8e40cc180 0x7fd8e41665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ea800640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e40cf890 0x7fd8e40c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9400640 1 -- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8e40cc180 msgr2=0x7fd8e41665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9400640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8e40cc180 0x7fd8e41665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9400640 1 -- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e40cf890 msgr2=0x7fd8e40c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9400640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e40cf890 0x7fd8e40c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9400640 1 -- 172.21.15.46:0/2318747868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4153ed0 con 0x7fd8e40cf340 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8e40cc180 0x7fd8e41665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ea800640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e40cf890 0x7fd8e40c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8e9400640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 0x7fd8e415eac0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c400 tx=0x7fd8d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8de000640 1 -- 172.21.15.46:0/2318747868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4017070 con 0x7fd8e40cf340 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8de000640 1 -- 172.21.15.46:0/2318747868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d40052d0 con 0x7fd8e40cf340 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8de000640 1 -- 172.21.15.46:0/2318747868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4003840 con 0x7fd8e40cf340 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 -- 172.21.15.46:0/2318747868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e41649f0 con 0x7fd8e40cf340 2024-04-25T08:50:28.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.883+0000 7fd8ec800640 1 -- 172.21.15.46:0/2318747868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e415d060 con 0x7fd8e40cf340 2024-04-25T08:50:28.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8de000640 1 -- 172.21.15.46:0/2318747868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8d401a070 con 0x7fd8e40cf340 2024-04-25T08:50:28.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8de000640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8c8066f60 0x7fd8c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8de000640 1 -- 172.21.15.46:0/2318747868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd8d4084740 con 0x7fd8e40cf340 2024-04-25T08:50:28.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8c8066f60 0x7fd8c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8efcfa000 1 Processor -- start 2024-04-25T08:50:28.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8efcfa000 1 -- start start 2024-04-25T08:50:28.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8c8066f60 0x7fd8c8069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8000f70 tx=0x7fd8d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead22e0 0x55cd5ead26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea883a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ea888e0 0x55cd5ea88cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e96fba0 con 0x55cd5ead2c00 2024-04-25T08:50:28.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e8d5090 con 0x55cd5ead22e0 2024-04-25T08:50:28.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e85d160 con 0x55cd5ea888e0 2024-04-25T08:50:28.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea883a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8e9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea883a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37952/0 (socket says 172.21.15.46:37952) 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8e9400640 1 -- 172.21.15.46:0/3932038558 learned_addr learned my addr 172.21.15.46:0/3932038558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/3932038558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead22e0 0x55cd5ead26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.887+0000 7fd8ea800640 1 --2- 172.21.15.46:0/3932038558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ea888e0 0x55cd5ea88cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9400640 1 -- 172.21.15.46:0/3932038558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ea888e0 msgr2=0x55cd5ea88cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9400640 1 --2- 172.21.15.46:0/3932038558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ea888e0 0x55cd5ea88cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9400640 1 -- 172.21.15.46:0/3932038558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead22e0 msgr2=0x55cd5ead26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9400640 1 --2- 172.21.15.46:0/3932038558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead22e0 0x55cd5ead26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9400640 1 -- 172.21.15.46:0/3932038558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5e97ec20 con 0x55cd5ead2c00 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9400640 1 --2- 172.21.15.46:0/3932038558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea883a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400cfd0 tx=0x7fd8d4054790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ebb6a82c9e1bf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8d2a00640 1 -- 172.21.15.46:0/3932038558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4052da0 con 0x55cd5ead2c00 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8d2a00640 1 -- 172.21.15.46:0/3932038558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4053000 con 0x55cd5ead2c00 2024-04-25T08:50:28.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8d2a00640 1 -- 172.21.15.46:0/3932038558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4054150 con 0x55cd5ead2c00 2024-04-25T08:50:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3932038558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 msgr2=0x55cd5ea883a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/3932038558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea883a0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400cfd0 tx=0x7fd8d4054790 comp rx=0 tx=0).stop 2024-04-25T08:50:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3932038558 shutdown_connections 2024-04-25T08:50:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/3932038558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ea888e0 0x55cd5ea88cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/3932038558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea883a0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/3932038558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead22e0 0x55cd5ead26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3932038558 >> 172.21.15.46:0/3932038558 conn(0x55cd5ea84890 msgr2=0x55cd5ea86c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3932038558 shutdown_connections 2024-04-25T08:50:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3932038558 wait complete. 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 -- start start 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 Processor -- start 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5ead9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5ead9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 0x55cd5eac0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e8d1130 con 0x55cd5ea888e0 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e899130 con 0x55cd5ead2c00 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e993300 con 0x55cd5ead22e0 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8efcfa000 10 client.0 _ll_register_callbacks cb 0x55cd5ea73bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5ead9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5ead9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37958/0 (socket says 172.21.15.46:37958) 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9e00640 1 -- 172.21.15.46:0/1027490518 learned_addr learned my addr 172.21.15.46:0/1027490518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9400640 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5ead9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8ea800640 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 0x55cd5eac0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9e00640 1 -- 172.21.15.46:0/1027490518 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 msgr2=0x55cd5ead9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5ead9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9e00640 1 -- 172.21.15.46:0/1027490518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 msgr2=0x55cd5eac0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 0x55cd5eac0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.891+0000 7fd8e9e00640 1 -- 172.21.15.46:0/1027490518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd5e8b2060 con 0x55cd5ea888e0 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5ead9640 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8005b50 tx=0x7fd8d8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7251da4a6e5ef68b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 1 -- 172.21.15.46:0/1027490518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8027060 con 0x55cd5ea888e0 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 1 -- 172.21.15.46:0/1027490518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd8d800de40 con 0x55cd5ea888e0 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 -- 172.21.15.46:0/1027490518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8d4053000 con 0x55cd5ea888e0 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 1 -- 172.21.15.46:0/1027490518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8d8012620 con 0x55cd5ea888e0 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 10 client.8127 fetch_fsmap learned FSMap version 75 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 -- 172.21.15.46:0/1027490518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55cd5e8767c0 con 0x55cd5ea888e0 2024-04-25T08:50:28.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 1 -- 172.21.15.46:0/1027490518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4053000 con 0x55cd5ea888e0 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 1 -- 172.21.15.46:0/1027490518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8d80127c0 con 0x55cd5ea888e0 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 10 client.8127 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 10 client.8127 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 20 client.8127 populate_metadata read hostname 'smithi046' 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x55cd5eac0e00 0x55cd5eac11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 4 client.8127 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 -- 172.21.15.46:0/1027490518 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd5e870cd0 con 0x55cd5eac0e00 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8e9400640 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x55cd5eac0e00 0x55cd5eac11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8e9400640 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x55cd5eac0e00 0x55cd5eac11e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47bc2282020bcfc2 server_cookie=6a721971a80224bd in_seq=0 out_seq=0 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 10 client.8127 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 1 -- 172.21.15.46:0/1027490518 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd5e870cd0 con 0x55cd5eac0e00 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 10 client.8127 handle_command_reply: tid=0 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 client.8127 shutdown 2024-04-25T08:50:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 -- 172.21.15.46:0/1027490518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 msgr2=0x55cd5ead9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5ead9640 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8005b50 tx=0x7fd8d8000ee0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 -- 172.21.15.46:0/1027490518 shutdown_connections 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x55cd5eac0e00 0x55cd5eac11e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 0x55cd5eac0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5ead9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/1027490518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5ead9640 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8c7400640 0 client.8127 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 -- 172.21.15.46:0/1027490518 >> 172.21.15.46:0/1027490518 conn(0x55cd5ea84890 msgr2=0x55cd5ea87510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 -- 172.21.15.46:0/1027490518 shutdown_connections 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 1 -- 172.21.15.46:0/1027490518 wait complete. 2024-04-25T08:50:28.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.895+0000 7fd8efcfa000 20 client.8127 trim_cache size 0 max 16384 2024-04-25T08:50:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.907+0000 7fd8efcfa000 1 Processor -- start 2024-04-25T08:50:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.907+0000 7fd8efcfa000 1 -- start start 2024-04-25T08:50:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.907+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ea888e0 0x55cd5ead1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.907+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5ead17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.907+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea86c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.907+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e42037a0 con 0x55cd5ead2c00 2024-04-25T08:50:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.907+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e4202990 con 0x55cd5ea888e0 2024-04-25T08:50:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.907+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e40b0b90 con 0x55cd5ead22e0 2024-04-25T08:50:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea86c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea86c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37966/0 (socket says 172.21.15.46:37966) 2024-04-25T08:50:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8ea800640 1 -- 172.21.15.46:0/3600308437 learned_addr learned my addr 172.21.15.46:0/3600308437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8e9400640 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5ead17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ea888e0 0x55cd5ead1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8ea800640 1 -- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 msgr2=0x55cd5ead17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8ea800640 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5ead17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8ea800640 1 -- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ea888e0 msgr2=0x55cd5ead1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8ea800640 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ea888e0 0x55cd5ead1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8ea800640 1 -- 172.21.15.46:0/3600308437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5e890fc0 con 0x55cd5ead2c00 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8e9400640 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5ead17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ea888e0 0x55cd5ead1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8ea800640 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea86c80 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0019580 tx=0x7fd8e0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec5d790228339a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8d2000640 1 -- 172.21.15.46:0/3600308437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00bf030 con 0x55cd5ead2c00 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8d2000640 1 -- 172.21.15.46:0/3600308437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e00b3040 con 0x55cd5ead2c00 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8d2000640 1 -- 172.21.15.46:0/3600308437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00b2040 con 0x55cd5ead2c00 2024-04-25T08:50:28.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3600308437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 msgr2=0x55cd5ea86c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea86c80 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0019580 tx=0x7fd8e0004ac0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3600308437 shutdown_connections 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ead2c00 0x55cd5ea86c80 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5ead17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/3600308437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ea888e0 0x55cd5ead1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3600308437 >> 172.21.15.46:0/3600308437 conn(0x55cd5ea84890 msgr2=0x55cd5eadfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3600308437 shutdown_connections 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 -- 172.21.15.46:0/3600308437 wait complete. 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 -- start start 2024-04-25T08:50:28.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 Processor -- start 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5eb31f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5eb32490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 0x55cd5ead6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e86fe70 con 0x55cd5ea888e0 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e85da50 con 0x55cd5ead2c00 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5e8d43a0 con 0x55cd5ead22e0 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8efcfa000 10 client.0 _ll_register_callbacks cb 0x55cd5e881240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.911+0000 7fd8e9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5eb31f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5eb31f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37972/0 (socket says 172.21.15.46:37972) 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9e00640 1 -- 172.21.15.46:0/492252523 learned_addr learned my addr 172.21.15.46:0/492252523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:28.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9400640 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5eb32490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8ea800640 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 0x55cd5ead6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9e00640 1 -- 172.21.15.46:0/492252523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 msgr2=0x55cd5eb32490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5eb32490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9e00640 1 -- 172.21.15.46:0/492252523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 msgr2=0x55cd5ead6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 0x55cd5ead6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9e00640 1 -- 172.21.15.46:0/492252523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd5e92e9e0 con 0x55cd5ea888e0 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9e00640 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5eb31f50 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80070a0 tx=0x7fd8d8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ae25244b58d361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 1 -- 172.21.15.46:0/492252523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d800de40 con 0x55cd5ea888e0 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 1 -- 172.21.15.46:0/492252523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd8d8037d80 con 0x55cd5ea888e0 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 -- 172.21.15.46:0/492252523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8e00b3040 con 0x55cd5ea888e0 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 1 -- 172.21.15.46:0/492252523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8d8021660 con 0x55cd5ea888e0 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 10 client.8133 fetch_fsmap learned FSMap version 75 2024-04-25T08:50:28.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 -- 172.21.15.46:0/492252523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55cd5e874630 con 0x55cd5ea888e0 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 1 -- 172.21.15.46:0/492252523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e00b3040 con 0x55cd5ea888e0 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 1 -- 172.21.15.46:0/492252523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd8d803e020 con 0x55cd5ea888e0 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 10 client.8133 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 10 client.8133 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 20 client.8133 populate_metadata read hostname 'smithi046' 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x55cd5ead6a00 0x55cd5ead6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 4 client.8133 mds_command: new command op to 7962 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 -- 172.21.15.46:0/492252523 --> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55cd5e8e25d0 con 0x55cd5ead6a00 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9400640 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x55cd5ead6a00 0x55cd5ead6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 10 client.8133 ms_handle_connect on v2:172.21.15.46:6834/1358086076 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8e9400640 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x55cd5ead6a00 0x55cd5ead6de0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a7a650ee955174 server_cookie=a62bd54476d06af1 in_seq=0 out_seq=0 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 1 -- 172.21.15.46:0/492252523 <== mds.0 v2:172.21.15.46:6834/1358086076 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cd5e8e25d0 con 0x55cd5ead6a00 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 10 client.8133 handle_command_reply: tid=0 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 client.8133 shutdown 2024-04-25T08:50:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 -- 172.21.15.46:0/492252523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 msgr2=0x55cd5eb31f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5eb31f50 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80070a0 tx=0x7fd8d8012000 comp rx=0 tx=0).stop 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 -- 172.21.15.46:0/492252523 shutdown_connections 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.46:6834/1358086076,v1:172.21.15.46:6835/1358086076] conn(0x55cd5ead6a00 0x55cd5ead6de0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cd5ead2c00 0x55cd5ead6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55cd5ead22e0 0x55cd5eb32490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8c7400640 0 client.8133 ms_handle_reset on v2:172.21.15.46:6834/1358086076 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 --2- 172.21.15.46:0/492252523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cd5ea888e0 0x55cd5eb31f50 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 -- 172.21.15.46:0/492252523 >> 172.21.15.46:0/492252523 conn(0x55cd5ea84890 msgr2=0x55cd5ead1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 -- 172.21.15.46:0/492252523 shutdown_connections 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 1 -- 172.21.15.46:0/492252523 wait complete. 2024-04-25T08:50:28.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8efcfa000 20 client.8133 trim_cache size 0 max 16384 2024-04-25T08:50:28.912 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:28.912 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-04-25T08:50:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8ec800640 1 -- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8c8066f60 msgr2=0x7fd8c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8ec800640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8c8066f60 0x7fd8c8069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8000f70 tx=0x7fd8d8009040 comp rx=0 tx=0).stop 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8ec800640 1 -- 172.21.15.46:0/2318747868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 msgr2=0x7fd8e415eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.915+0000 7fd8ec800640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 0x7fd8e415eac0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c400 tx=0x7fd8d400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.919+0000 7fd8ec800640 1 -- 172.21.15.46:0/2318747868 shutdown_connections 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.919+0000 7fd8ec800640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8c8066f60 0x7fd8c8069420 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.919+0000 7fd8ec800640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e40cf890 0x7fd8e40c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.919+0000 7fd8ec800640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8e40cf340 0x7fd8e415eac0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.919+0000 7fd8ec800640 1 --2- 172.21.15.46:0/2318747868 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8e40cc180 0x7fd8e41665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.919+0000 7fd8ec800640 1 -- 172.21.15.46:0/2318747868 >> 172.21.15.46:0/2318747868 conn(0x7fd8e4099a00 msgr2=0x7fd8e40d0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.919+0000 7fd8ec800640 1 -- 172.21.15.46:0/2318747868 shutdown_connections 2024-04-25T08:50:28.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:28.919+0000 7fd8ec800640 1 -- 172.21.15.46:0/2318747868 wait complete. 2024-04-25T08:50:28.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:50:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6c0000640 1 Processor -- start 2024-04-25T08:50:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6c0000640 1 -- start start 2024-04-25T08:50:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6c0000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa6b80ada20 0x7fa6b8153f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6c0000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b815f160 con 0x7fa6b806c330 2024-04-25T08:50:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6c0000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b80adf60 con 0x7fa6b80701c0 2024-04-25T08:50:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6c0000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b815e250 con 0x7fa6b80ada20 2024-04-25T08:50:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6bcc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa6b806c330 0x7fa6b80ad310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57168/0 (socket says 172.21.15.46:57168) 2024-04-25T08:50:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6bcc00640 1 -- 172.21.15.46:0/3988504892 learned_addr learned my addr 172.21.15.46:0/3988504892 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6be000640 1 --2- 172.21.15.46:0/3988504892 >> v2:172.21.15.46:3300/0 conn(0x7fa6b80ada20 0x7fa6b8153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6be000640 1 -- 172.21.15.46:0/3988504892 >> v1:172.21.15.82:6789/0 conn(0x7fa6b80701c0 legacy=0x7fa6b806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6be000640 1 -- 172.21.15.46:0/3988504892 >> v1:172.21.15.46:6789/0 conn(0x7fa6b806c330 legacy=0x7fa6b80ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.079+0000 7fa6be000640 1 -- 172.21.15.46:0/3988504892 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b80ae240 con 0x7fa6b80ada20 2024-04-25T08:50:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6be000640 1 --2- 172.21.15.46:0/3988504892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b80ada20 0x7fa6b8153f60 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009d90 tx=0x7fa6b001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71c14e34cd13d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6b7e00640 1 -- 172.21.15.46:0/3988504892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00b2070 con 0x7fa6b80ada20 2024-04-25T08:50:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6b7e00640 1 -- 172.21.15.46:0/3988504892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b00a57f0 con 0x7fa6b80ada20 2024-04-25T08:50:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6b7e00640 1 -- 172.21.15.46:0/3988504892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00a5bc0 con 0x7fa6b80ada20 2024-04-25T08:50:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6c0000640 1 -- 172.21.15.46:0/3988504892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b80ada20 msgr2=0x7fa6b8153f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6c0000640 1 --2- 172.21.15.46:0/3988504892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b80ada20 0x7fa6b8153f60 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009d90 tx=0x7fa6b001c930 comp rx=0 tx=0).stop 2024-04-25T08:50:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6c0000640 1 -- 172.21.15.46:0/3988504892 shutdown_connections 2024-04-25T08:50:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6c0000640 1 --2- 172.21.15.46:0/3988504892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b80ada20 0x7fa6b8153f60 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6c0000640 1 -- 172.21.15.46:0/3988504892 >> 172.21.15.46:0/3988504892 conn(0x7fa6b8071b00 msgr2=0x7fa6b8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6c0000640 1 -- 172.21.15.46:0/3988504892 shutdown_connections 2024-04-25T08:50:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.083+0000 7fa6c0000640 1 -- 172.21.15.46:0/3988504892 wait complete. 2024-04-25T08:50:29.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 Processor -- start 2024-04-25T08:50:29.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 -- start start 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b806c330 0x7fa6b814a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b80701c0 0x7fa6b806b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6b80ada20 0x7fa6b806bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b816ba60 con 0x7fa6b806c330 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b80ae520 con 0x7fa6b80701c0 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b816bbe0 con 0x7fa6b80ada20 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b806c330 0x7fa6b814a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b806c330 0x7fa6b814a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37988/0 (socket says 172.21.15.46:37988) 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bd600640 1 -- 172.21.15.46:0/1994716688 learned_addr learned my addr 172.21.15.46:0/1994716688 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bcc00640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b80701c0 0x7fa6b806b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6be000640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6b80ada20 0x7fa6b806bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bd600640 1 -- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6b80ada20 msgr2=0x7fa6b806bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bd600640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6b80ada20 0x7fa6b806bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bd600640 1 -- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b80701c0 msgr2=0x7fa6b806b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bd600640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b80701c0 0x7fa6b806b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bd600640 1 -- 172.21.15.46:0/1994716688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b814ae80 con 0x7fa6b806c330 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6be000640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6b80ada20 0x7fa6b806bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bcc00640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b80701c0 0x7fa6b806b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6bd600640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b806c330 0x7fa6b814a940 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c450 tx=0x7fa6ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6b5600640 1 -- 172.21.15.46:0/1994716688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac011790 con 0x7fa6b806c330 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6b5600640 1 -- 172.21.15.46:0/1994716688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac017070 con 0x7fa6b806c330 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b81559f0 con 0x7fa6b806c330 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6b5600640 1 -- 172.21.15.46:0/1994716688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac011b90 con 0x7fa6b806c330 2024-04-25T08:50:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b8155fb0 con 0x7fa6b806c330 2024-04-25T08:50:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.087+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b8074c60 con 0x7fa6b806c330 2024-04-25T08:50:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.091+0000 7fa6b5600640 1 -- 172.21.15.46:0/1994716688 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6ac011d30 con 0x7fa6b806c330 2024-04-25T08:50:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.091+0000 7fa6b5600640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa698066f60 0x7fa698069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.091+0000 7fa6b5600640 1 -- 172.21.15.46:0/1994716688 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fa6ac085160 con 0x7fa6b806c330 2024-04-25T08:50:29.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.091+0000 7fa6bcc00640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa698066f60 0x7fa698069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.091+0000 7fa6bcc00640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa698066f60 0x7fa698069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8153a60 tx=0x7fa6a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.095+0000 7fa6b5600640 1 -- 172.21.15.46:0/1994716688 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6ac055e20 con 0x7fa6b806c330 2024-04-25T08:50:29.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.291+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa6b80729a0 con 0x7fa6b806c330 2024-04-25T08:50:29.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.291+0000 7fa6b5600640 1 -- 172.21.15.46:0/1994716688 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa6ac055720 con 0x7fa6b806c330 2024-04-25T08:50:29.283 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:29.283 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:50:29.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa698066f60 msgr2=0x7fa698069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa698066f60 0x7fa698069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8153a60 tx=0x7fa6a4009040 comp rx=0 tx=0).stop 2024-04-25T08:50:29.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b806c330 msgr2=0x7fa6b814a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b806c330 0x7fa6b814a940 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c450 tx=0x7fa6ac00c910 comp rx=0 tx=0).stop 2024-04-25T08:50:29.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 shutdown_connections 2024-04-25T08:50:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa698066f60 0x7fa698069420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa6b80ada20 0x7fa6b806bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6b80701c0 0x7fa6b806b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 --2- 172.21.15.46:0/1994716688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6b806c330 0x7fa6b814a940 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 >> 172.21.15.46:0/1994716688 conn(0x7fa6b8071b00 msgr2=0x7fa6b806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 shutdown_connections 2024-04-25T08:50:29.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.295+0000 7fa6c0000640 1 -- 172.21.15.46:0/1994716688 wait complete. 2024-04-25T08:50:29.304 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:50:29.304 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:50:29.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:50:29.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:50:29.444 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:50:29.444 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:29.444 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:29.453 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:29.453 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:29.504 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb7c00640 1 Processor -- start 2024-04-25T08:50:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb7c00640 1 -- start start 2024-04-25T08:50:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb7c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb015a750 0x7fecb006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb7c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fecb00aed10 0x7fecb006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb7c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb006d5e0 con 0x7fecb00cc9e0 2024-04-25T08:50:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb7c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb015ab30 con 0x7fecb00aed10 2024-04-25T08:50:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb7c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00ce4f0 con 0x7fecb015a750 2024-04-25T08:50:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb5c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fecb00cc9e0 0x7fecb00b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57182/0 (socket says 172.21.15.46:57182) 2024-04-25T08:50:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb5c00640 1 -- 172.21.15.46:0/2053596474 learned_addr learned my addr 172.21.15.46:0/2053596474 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb5200640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb015a750 0x7fecb006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7feca7e00640 1 --2- 172.21.15.46:0/2053596474 >> v2:172.21.15.82:3300/0 conn(0x7fecb00aed10 0x7fecb006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433081231 0 0) 0x7fecb006d5e0 con 0x7fecb00cc9e0 2024-04-25T08:50:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec94003530 con 0x7fecb00cc9e0 2024-04-25T08:50:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1709881937 0 0) 0x7fec94003530 con 0x7fecb00cc9e0 2024-04-25T08:50:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb015a750 msgr2=0x7fecb006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb015a750 0x7fecb006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 >> v2:172.21.15.82:3300/0 conn(0x7fecb00aed10 msgr2=0x7fecb006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 --2- 172.21.15.46:0/2053596474 >> v2:172.21.15.82:3300/0 conn(0x7fecb00aed10 0x7fecb006f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00ce720 con 0x7fecb00cc9e0 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7feca8001a00 con 0x7fecb00cc9e0 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 >> v1:172.21.15.46:6789/0 conn(0x7fecb00cc9e0 legacy=0x7fecb00b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb015a750 0x7fec940039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec94003f40 0x7fec94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94004d50 0x7fec940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9400b9d0 con 0x7fec94003f40 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9400bb50 con 0x7fec94004d50 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9400bcd0 con 0x7fecb015a750 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7feca7e00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec94003f40 0x7fec94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb5200640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb015a750 0x7fec940039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.703+0000 7fecb5c00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94004d50 0x7fec940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7feca7e00640 1 -- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb015a750 msgr2=0x7fec940039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7feca7e00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb015a750 0x7fec940039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7feca7e00640 1 -- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94004d50 msgr2=0x7fec940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7feca7e00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94004d50 0x7fec940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7feca7e00640 1 -- 172.21.15.46:0/2053596474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec94003530 con 0x7fec94003f40 2024-04-25T08:50:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecb5c00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94004d50 0x7fec940051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7feca7e00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec94003f40 0x7fec94004810 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fec98009ae0 tx=0x7fec9801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b6ff68f399e190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec980b2070 con 0x7fec94003f40 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec980a56a0 con 0x7fec94003f40 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecafe00640 1 -- 172.21.15.46:0/2053596474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec980096b0 con 0x7fec94003f40 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecb7c00640 1 -- 172.21.15.46:0/2053596474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec94003f40 msgr2=0x7fec94004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecb7c00640 1 --2- 172.21.15.46:0/2053596474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec94003f40 0x7fec94004810 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fec98009ae0 tx=0x7fec9801ca50 comp rx=0 tx=0).stop 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecb5c00640 1 -- 172.21.15.46:0/2053596474 reap_dead start 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecb7c00640 1 -- 172.21.15.46:0/2053596474 shutdown_connections 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecb7c00640 1 -- 172.21.15.46:0/2053596474 >> 172.21.15.46:0/2053596474 conn(0x7fecb00b12a0 msgr2=0x7fecb00b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecb7c00640 1 -- 172.21.15.46:0/2053596474 shutdown_connections 2024-04-25T08:50:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.707+0000 7fecb7c00640 1 -- 172.21.15.46:0/2053596474 wait complete. 2024-04-25T08:50:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb7c00640 1 Processor -- start 2024-04-25T08:50:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb7c00640 1 -- start start 2024-04-25T08:50:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb7c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94003f40 0x7fecb01e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb7c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb01e4e10 0x7fecb014c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb7c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb006db00 0x7fecb006df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb7c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cece0 con 0x7fecb006db00 2024-04-25T08:50:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb7c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb006d8c0 con 0x7fec94003f40 2024-04-25T08:50:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb7c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cf200 con 0x7fecb01e4e10 2024-04-25T08:50:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb006db00 0x7fecb006df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb006db00 0x7fecb006df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37996/0 (socket says 172.21.15.46:37996) 2024-04-25T08:50:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5c00640 1 -- 172.21.15.46:0/2327218657 learned_addr learned my addr 172.21.15.46:0/2327218657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7feca7e00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb01e4e10 0x7fecb014c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5200640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94003f40 0x7fecb01e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5c00640 1 -- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb01e4e10 msgr2=0x7fecb014c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5c00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb01e4e10 0x7fecb014c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5c00640 1 -- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94003f40 msgr2=0x7fecb01e2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5c00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94003f40 0x7fecb01e2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5c00640 1 -- 172.21.15.46:0/2327218657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb01e4a10 con 0x7fecb006db00 2024-04-25T08:50:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7feca7e00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb01e4e10 0x7fecb014c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb5c00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb006db00 0x7fecb006df80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7feca8002240 tx=0x7feca8011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecad600640 1 -- 172.21.15.46:0/2327218657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8010040 con 0x7fecb006db00 2024-04-25T08:50:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecad600640 1 -- 172.21.15.46:0/2327218657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca801a070 con 0x7fecb006db00 2024-04-25T08:50:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecad600640 1 -- 172.21.15.46:0/2327218657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca800fac0 con 0x7fecb006db00 2024-04-25T08:50:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.711+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb01e5290 con 0x7fecb006db00 2024-04-25T08:50:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.715+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb01e3560 con 0x7fecb006db00 2024-04-25T08:50:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.715+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb0069080 con 0x7fecb006db00 2024-04-25T08:50:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.715+0000 7fecad600640 1 -- 172.21.15.46:0/2327218657 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feca800fc60 con 0x7fecb006db00 2024-04-25T08:50:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.715+0000 7fecad600640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec8c066f90 0x7fec8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.715+0000 7fecad600640 1 -- 172.21.15.46:0/2327218657 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7feca8088100 con 0x7fecb006db00 2024-04-25T08:50:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.715+0000 7fecb5200640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec8c066f90 0x7fec8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:29.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.715+0000 7fecb5200640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec8c066f90 0x7fec8c069450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7feca0009a40 tx=0x7feca0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.719+0000 7fecad600640 1 -- 172.21.15.46:0/2327218657 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fecb01e5290 con 0x7fecb006db00 2024-04-25T08:50:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.883+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecb00b0180 con 0x7fecb006db00 2024-04-25T08:50:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.883+0000 7fecad600640 1 -- 172.21.15.46:0/2327218657 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17289 (secure 0 0 0) 0x7fecb01e4a10 con 0x7fecb006db00 2024-04-25T08:50:29.875 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:29.875 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:49:51.036629+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:48:26.132790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:48:26.714570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6834/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6835/1269610178":"2024-04-26T08:48:38.594941+0000","172.21.15.46:6834/1269610178":"2024-04-26T08:48:38.594941+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:29.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec8c066f90 msgr2=0x7fec8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec8c066f90 0x7fec8c069450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7feca0009a40 tx=0x7feca0008040 comp rx=0 tx=0).stop 2024-04-25T08:50:29.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb006db00 msgr2=0x7fecb006df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:29.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb006db00 0x7fecb006df80 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7feca8002240 tx=0x7feca8011c20 comp rx=0 tx=0).stop 2024-04-25T08:50:29.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 shutdown_connections 2024-04-25T08:50:29.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fec8c066f90 0x7fec8c069450 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fecb006db00 0x7fecb006df80 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fecb01e4e10 0x7fecb014c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 --2- 172.21.15.46:0/2327218657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec94003f40 0x7fecb01e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:29.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 >> 172.21.15.46:0/2327218657 conn(0x7fecb00b12a0 msgr2=0x7fecb015af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:29.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 shutdown_connections 2024-04-25T08:50:29.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:29.887+0000 7fecb7c00640 1 -- 172.21.15.46:0/2327218657 wait complete. 2024-04-25T08:50:29.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:48:26.132790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:48:26.714570+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:50:29.897 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:50:30.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f159ba00640 1 Processor -- start 2024-04-25T08:50:30.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f159ba00640 1 -- start start 2024-04-25T08:50:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f159ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f15940b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f159ba00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f15940cf340 0x7f159415bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f159ba00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15940b2d50 con 0x7f15940cc180 2024-04-25T08:50:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f159ba00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f159415f2f0 con 0x7f15940cf340 2024-04-25T08:50:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f159ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159415f5d0 con 0x7f15940cf890 2024-04-25T08:50:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f1599a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f15940cf340 0x7f159415bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f1599000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f15940b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f1599000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f15940b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36024/0 (socket says 172.21.15.46:36024) 2024-04-25T08:50:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f1599000640 1 -- 172.21.15.46:0/3439823585 learned_addr learned my addr 172.21.15.46:0/3439823585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f158be00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f15940cc180 0x7f15940cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:57570/0 (socket says 172.21.15.46:57570) 2024-04-25T08:50:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f1599000640 1 -- 172.21.15.46:0/3439823585 >> v2:172.21.15.82:3300/0 conn(0x7f15940cf340 msgr2=0x7f159415bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f1599000640 1 --2- 172.21.15.46:0/3439823585 >> v2:172.21.15.82:3300/0 conn(0x7f15940cf340 0x7f159415bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f1599000640 1 -- 172.21.15.46:0/3439823585 >> v1:172.21.15.82:6789/0 conn(0x7f15940cc180 legacy=0x7f15940cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.051+0000 7f1599000640 1 -- 172.21.15.46:0/3439823585 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159415f8b0 con 0x7f15940cf890 2024-04-25T08:50:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f1599000640 1 --2- 172.21.15.46:0/3439823585 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f15940b2810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1584009870 tx=0x7f158401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f50a26ed06e22dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f1593e00640 1 -- 172.21.15.46:0/3439823585 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15840a9610 con 0x7f15940cf890 2024-04-25T08:50:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f1593e00640 1 -- 172.21.15.46:0/3439823585 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15840a97b0 con 0x7f15940cf890 2024-04-25T08:50:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f1593e00640 1 -- 172.21.15.46:0/3439823585 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15840a9b80 con 0x7f15940cf890 2024-04-25T08:50:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f159ba00640 1 -- 172.21.15.46:0/3439823585 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 msgr2=0x7f15940b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f159ba00640 1 --2- 172.21.15.46:0/3439823585 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f15940b2810 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1584009870 tx=0x7f158401c530 comp rx=0 tx=0).stop 2024-04-25T08:50:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f159ba00640 1 -- 172.21.15.46:0/3439823585 shutdown_connections 2024-04-25T08:50:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f159ba00640 1 --2- 172.21.15.46:0/3439823585 >> v2:172.21.15.82:3300/0 conn(0x7f15940cf340 0x7f159415bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f159ba00640 1 --2- 172.21.15.46:0/3439823585 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f15940b2810 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f159ba00640 1 -- 172.21.15.46:0/3439823585 >> 172.21.15.46:0/3439823585 conn(0x7f1594099a00 msgr2=0x7f15940b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f159ba00640 1 -- 172.21.15.46:0/3439823585 shutdown_connections 2024-04-25T08:50:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.055+0000 7f159ba00640 1 -- 172.21.15.46:0/3439823585 wait complete. 2024-04-25T08:50:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 Processor -- start 2024-04-25T08:50:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 -- start start 2024-04-25T08:50:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15940cc180 0x7f15940c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15940cf340 0x7f1594165990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f159415a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159415cc50 con 0x7f15940cc180 2024-04-25T08:50:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159415cad0 con 0x7f15940cf340 2024-04-25T08:50:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1594168690 con 0x7f15940cf890 2024-04-25T08:50:30.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15940cc180 0x7f15940c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15940cc180 0x7f15940c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37998/0 (socket says 172.21.15.46:37998) 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599000640 1 -- 172.21.15.46:0/2037450910 learned_addr learned my addr 172.21.15.46:0/2037450910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f158be00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15940cf340 0x7f1594165990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599a00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f159415a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599000640 1 -- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 msgr2=0x7f159415a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599000640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f159415a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599000640 1 -- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15940cf340 msgr2=0x7f1594165990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599000640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15940cf340 0x7f1594165990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599000640 1 -- 172.21.15.46:0/2037450910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15941648f0 con 0x7f15940cc180 2024-04-25T08:50:30.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1599000640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15940cc180 0x7f15940c2fe0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f15840ab040 tx=0x7f158401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1591600640 1 -- 172.21.15.46:0/2037450910 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15840a9610 con 0x7f15940cc180 2024-04-25T08:50:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1591600640 1 -- 172.21.15.46:0/2037450910 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15840bc670 con 0x7f15940cc180 2024-04-25T08:50:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f1591600640 1 -- 172.21.15.46:0/2037450910 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15840bca60 con 0x7f15940cc180 2024-04-25T08:50:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1594164bd0 con 0x7f15940cc180 2024-04-25T08:50:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.059+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1594165190 con 0x7f15940cc180 2024-04-25T08:50:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.063+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1594069010 con 0x7f15940cc180 2024-04-25T08:50:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.063+0000 7f1591600640 1 -- 172.21.15.46:0/2037450910 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15840b13f0 con 0x7f15940cc180 2024-04-25T08:50:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.063+0000 7f1591600640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1570066f60 0x7f1570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.063+0000 7f1591600640 1 -- 172.21.15.46:0/2037450910 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f158411f700 con 0x7f15940cc180 2024-04-25T08:50:30.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.063+0000 7f158be00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1570066f60 0x7f1570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.063+0000 7f158be00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1570066f60 0x7f1570069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f15941fa1f0 tx=0x7f157c007e30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:30.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.067+0000 7f1591600640 1 -- 172.21.15.46:0/2037450910 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f15840c3030 con 0x7f15940cc180 2024-04-25T08:50:30.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1594073db0 con 0x7f15940cc180 2024-04-25T08:50:30.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f1591600640 1 -- 172.21.15.46:0/2037450910 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15840ad020 con 0x7f15940cc180 2024-04-25T08:50:30.223 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:50:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1570066f60 msgr2=0x7f1570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1570066f60 0x7f1570069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f15941fa1f0 tx=0x7f157c007e30 comp rx=0 tx=0).stop 2024-04-25T08:50:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15940cc180 msgr2=0x7f15940c2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15940cc180 0x7f15940c2fe0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f15840ab040 tx=0x7f158401ad70 comp rx=0 tx=0).stop 2024-04-25T08:50:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 shutdown_connections 2024-04-25T08:50:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1570066f60 0x7f1570069420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f15940cf890 0x7f159415a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15940cf340 0x7f1594165990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 --2- 172.21.15.46:0/2037450910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15940cc180 0x7f15940c2fe0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.231+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 >> 172.21.15.46:0/2037450910 conn(0x7f1594099a00 msgr2=0x7f1594155230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:30.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.235+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 shutdown_connections 2024-04-25T08:50:30.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.235+0000 7f159ba00640 1 -- 172.21.15.46:0/2037450910 wait complete. 2024-04-25T08:50:30.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:50:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a78800640 1 Processor -- start 2024-04-25T08:50:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a78800640 1 -- start start 2024-04-25T08:50:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a78800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a7006dc70 0x7f3a700cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a78800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a700cf1b0 0x7f3a7015d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a78800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a700cd930 con 0x7f3a70145310 2024-04-25T08:50:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a78800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a700cdbd0 con 0x7f3a7006dc70 2024-04-25T08:50:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a78800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a700cdeb0 con 0x7f3a700cf1b0 2024-04-25T08:50:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a75400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a7006dc70 0x7f3a700cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a75400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a7006dc70 0x7f3a700cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38006/0 (socket says 172.21.15.46:38006) 2024-04-25T08:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a75e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3a70145310 0x7f3a7006e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57192/0 (socket says 172.21.15.46:57192) 2024-04-25T08:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.395+0000 7f3a75400640 1 -- 172.21.15.46:0/3166969804 learned_addr learned my addr 172.21.15.46:0/3166969804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a76800640 1 --2- 172.21.15.46:0/3166969804 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a700cf1b0 0x7f3a7015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a75400640 1 -- 172.21.15.46:0/3166969804 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a700cf1b0 msgr2=0x7f3a7015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a75400640 1 --2- 172.21.15.46:0/3166969804 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a700cf1b0 0x7f3a7015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a75400640 1 -- 172.21.15.46:0/3166969804 >> v1:172.21.15.46:6789/0 conn(0x7f3a70145310 legacy=0x7f3a7006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a75400640 1 -- 172.21.15.46:0/3166969804 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a700ce190 con 0x7f3a7006dc70 2024-04-25T08:50:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a74a00640 1 -- 172.21.15.46:0/3166969804 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919452866 0 0) 0x7f3a700cd930 con 0x7f3a70145310 2024-04-25T08:50:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a75400640 1 --2- 172.21.15.46:0/3166969804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7006dc70 0x7f3a700cec70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009870 tx=0x7f3a6001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c2fa524674ea4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a74a00640 1 -- 172.21.15.46:0/3166969804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600ab640 con 0x7f3a7006dc70 2024-04-25T08:50:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a74a00640 1 -- 172.21.15.46:0/3166969804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a600b0070 con 0x7f3a7006dc70 2024-04-25T08:50:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a74a00640 1 -- 172.21.15.46:0/3166969804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600aba00 con 0x7f3a7006dc70 2024-04-25T08:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a78800640 1 -- 172.21.15.46:0/3166969804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7006dc70 msgr2=0x7f3a700cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.399+0000 7f3a78800640 1 --2- 172.21.15.46:0/3166969804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7006dc70 0x7f3a700cec70 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009870 tx=0x7f3a6001c530 comp rx=0 tx=0).stop 2024-04-25T08:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 -- 172.21.15.46:0/3166969804 shutdown_connections 2024-04-25T08:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 --2- 172.21.15.46:0/3166969804 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a700cf1b0 0x7f3a7015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 --2- 172.21.15.46:0/3166969804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a7006dc70 0x7f3a700cec70 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 -- 172.21.15.46:0/3166969804 >> 172.21.15.46:0/3166969804 conn(0x7f3a7006ac10 msgr2=0x7f3a7006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:30.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 -- 172.21.15.46:0/3166969804 shutdown_connections 2024-04-25T08:50:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 -- 172.21.15.46:0/3166969804 wait complete. 2024-04-25T08:50:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 Processor -- start 2024-04-25T08:50:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 -- start start 2024-04-25T08:50:30.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a700cf1b0 0x7f3a700b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a70145310 0x7f3a7013de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a7013c0a0 0x7f3a700add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a701688a0 con 0x7f3a700cf1b0 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a70168720 con 0x7f3a70145310 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a78800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a70168a20 con 0x7f3a7013c0a0 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a700cf1b0 0x7f3a700b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a700cf1b0 0x7f3a700b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38010/0 (socket says 172.21.15.46:38010) 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75e00640 1 -- 172.21.15.46:0/4008375440 learned_addr learned my addr 172.21.15.46:0/4008375440 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75400640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a70145310 0x7f3a7013de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a76800640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a7013c0a0 0x7f3a700add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75e00640 1 -- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a7013c0a0 msgr2=0x7f3a700add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75e00640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a7013c0a0 0x7f3a700add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75e00640 1 -- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a70145310 msgr2=0x7f3a7013de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75e00640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a70145310 0x7f3a7013de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75e00640 1 -- 172.21.15.46:0/4008375440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7013c530 con 0x7f3a700cf1b0 2024-04-25T08:50:30.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a76800640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a7013c0a0 0x7f3a700add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:30.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.403+0000 7f3a75e00640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a700cf1b0 0x7f3a700b0ed0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400c480 tx=0x7f3a6400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a6a000640 1 -- 172.21.15.46:0/4008375440 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64018070 con 0x7f3a700cf1b0 2024-04-25T08:50:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a6a000640 1 -- 172.21.15.46:0/4008375440 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64007500 con 0x7f3a700cf1b0 2024-04-25T08:50:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a6a000640 1 -- 172.21.15.46:0/4008375440 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64004a30 con 0x7f3a700cf1b0 2024-04-25T08:50:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7013e550 con 0x7f3a700cf1b0 2024-04-25T08:50:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7013eb10 con 0x7f3a700cf1b0 2024-04-25T08:50:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a70069010 con 0x7f3a700cf1b0 2024-04-25T08:50:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a6a000640 1 -- 172.21.15.46:0/4008375440 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a64004bd0 con 0x7f3a700cf1b0 2024-04-25T08:50:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a6a000640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a54066f60 0x7f3a54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a6a000640 1 -- 172.21.15.46:0/4008375440 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f3a64084560 con 0x7f3a700cf1b0 2024-04-25T08:50:30.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.407+0000 7f3a75400640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a54066f60 0x7f3a54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.411+0000 7f3a75400640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a54066f60 0x7f3a54069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3a600af000 tx=0x7f3a600a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:30.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.411+0000 7f3a6a000640 1 -- 172.21.15.46:0/4008375440 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a64054c60 con 0x7f3a700cf1b0 2024-04-25T08:50:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.575+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a7006e0f0 con 0x7f3a700cf1b0 2024-04-25T08:50:30.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.575+0000 7f3a6a000640 1 -- 172.21.15.46:0/4008375440 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a64058d00 con 0x7f3a700cf1b0 2024-04-25T08:50:30.566 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:50:30.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.575+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a54066f60 msgr2=0x7f3a54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.575+0000 7f3a78800640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a54066f60 0x7f3a54069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3a600af000 tx=0x7f3a600a7000 comp rx=0 tx=0).stop 2024-04-25T08:50:30.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.575+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a700cf1b0 msgr2=0x7f3a700b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.575+0000 7f3a78800640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a700cf1b0 0x7f3a700b0ed0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f3a6400c480 tx=0x7f3a6400c940 comp rx=0 tx=0).stop 2024-04-25T08:50:30.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.579+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 shutdown_connections 2024-04-25T08:50:30.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.579+0000 7f3a78800640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3a54066f60 0x7f3a54069420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.579+0000 7f3a78800640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3a7013c0a0 0x7f3a700add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.579+0000 7f3a78800640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a70145310 0x7f3a7013de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.579+0000 7f3a78800640 1 --2- 172.21.15.46:0/4008375440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a700cf1b0 0x7f3a700b0ed0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.579+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 >> 172.21.15.46:0/4008375440 conn(0x7f3a7006ac10 msgr2=0x7f3a7015a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:30.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.579+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 shutdown_connections 2024-04-25T08:50:30.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.579+0000 7f3a78800640 1 -- 172.21.15.46:0/4008375440 wait complete. 2024-04-25T08:50:30.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:50:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.739+0000 7f00c3e00640 1 Processor -- start 2024-04-25T08:50:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.739+0000 7f00c3e00640 1 -- start start 2024-04-25T08:50:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c40b2770 0x7f00c40cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00c4099a00 0x7f00c4099e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c40cfe30 con 0x7f00c40b2b50 2024-04-25T08:50:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c409a370 con 0x7f00c4099a00 2024-04-25T08:50:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c40d0110 con 0x7f00c40b2770 2024-04-25T08:50:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00c4099a00 0x7f00c4099e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00ba000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f00c40b2b50 0x7f00c406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57198/0 (socket says 172.21.15.46:57198) 2024-04-25T08:50:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00c4099a00 0x7f00c4099e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38024/0 (socket says 172.21.15.46:38024) 2024-04-25T08:50:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00ba000640 1 -- 172.21.15.46:0/2500049976 learned_addr learned my addr 172.21.15.46:0/2500049976 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c2a00640 1 --2- 172.21.15.46:0/2500049976 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c40b2770 0x7f00c40cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3400640 1 -- 172.21.15.46:0/2500049976 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c40b2770 msgr2=0x7f00c40cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3400640 1 --2- 172.21.15.46:0/2500049976 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c40b2770 0x7f00c40cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3400640 1 -- 172.21.15.46:0/2500049976 >> v1:172.21.15.46:6789/0 conn(0x7f00c40b2b50 legacy=0x7f00c406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c3400640 1 -- 172.21.15.46:0/2500049976 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c40d03f0 con 0x7f00c4099a00 2024-04-25T08:50:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c2000640 1 -- 172.21.15.46:0/2500049976 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652147887 0 0) 0x7f00c40cfe30 con 0x7f00c40b2b50 2024-04-25T08:50:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.743+0000 7f00c2a00640 1 --2- 172.21.15.46:0/2500049976 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c40b2770 0x7f00c40cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3400640 1 --2- 172.21.15.46:0/2500049976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4099a00 0x7f00c4099e30 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0097c0 tx=0x7f00bc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495d64f7ffab698a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c2000640 1 -- 172.21.15.46:0/2500049976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0a7070 con 0x7f00c4099a00 2024-04-25T08:50:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c2000640 1 -- 172.21.15.46:0/2500049976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc0b0070 con 0x7f00c4099a00 2024-04-25T08:50:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c2000640 1 -- 172.21.15.46:0/2500049976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0ab890 con 0x7f00c4099a00 2024-04-25T08:50:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 -- 172.21.15.46:0/2500049976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4099a00 msgr2=0x7f00c4099e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 --2- 172.21.15.46:0/2500049976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4099a00 0x7f00c4099e30 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0097c0 tx=0x7f00bc01c470 comp rx=0 tx=0).stop 2024-04-25T08:50:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 -- 172.21.15.46:0/2500049976 shutdown_connections 2024-04-25T08:50:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 --2- 172.21.15.46:0/2500049976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c4099a00 0x7f00c4099e30 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 --2- 172.21.15.46:0/2500049976 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c40b2770 0x7f00c40cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 -- 172.21.15.46:0/2500049976 >> 172.21.15.46:0/2500049976 conn(0x7f00c40b3000 msgr2=0x7f00c40b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 -- 172.21.15.46:0/2500049976 shutdown_connections 2024-04-25T08:50:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 -- 172.21.15.46:0/2500049976 wait complete. 2024-04-25T08:50:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 Processor -- start 2024-04-25T08:50:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 -- start start 2024-04-25T08:50:30.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c4099a00 0x7f00c4160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c40b2770 0x7f00c40ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00c40b2b50 0x7f00c40c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c415dd20 con 0x7f00c40b2770 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c415dba0 con 0x7f00c40b2b50 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00c3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c415dea0 con 0x7f00c4099a00 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00ba000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c40b2770 0x7f00c40ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00ba000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c40b2770 0x7f00c40ccb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38030/0 (socket says 172.21.15.46:38030) 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.747+0000 7f00ba000640 1 -- 172.21.15.46:0/135531703 learned_addr learned my addr 172.21.15.46:0/135531703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00c3400640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00c40b2b50 0x7f00c40c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00c2a00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c4099a00 0x7f00c4160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00ba000640 1 -- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c4099a00 msgr2=0x7f00c4160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00ba000640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c4099a00 0x7f00c4160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00ba000640 1 -- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00c40b2b50 msgr2=0x7f00c40c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00ba000640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00c40b2b50 0x7f00c40c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00ba000640 1 -- 172.21.15.46:0/135531703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c415ff20 con 0x7f00c40b2770 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00ba000640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c40b2770 0x7f00c40ccb50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0011f0 tx=0x7f00ac00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00bbe00640 1 -- 172.21.15.46:0/135531703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac018070 con 0x7f00c40b2770 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00bbe00640 1 -- 172.21.15.46:0/135531703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac0074e0 con 0x7f00c40b2770 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00bbe00640 1 -- 172.21.15.46:0/135531703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac004a10 con 0x7f00c40b2770 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c40c9190 con 0x7f00c40b2770 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c41558f0 con 0x7f00c40b2770 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c4069010 con 0x7f00c40b2770 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00bbe00640 1 -- 172.21.15.46:0/135531703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00ac004bb0 con 0x7f00c40b2770 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00bbe00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00a0066f90 0x7f00a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.751+0000 7f00bbe00640 1 -- 172.21.15.46:0/135531703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f00ac084520 con 0x7f00c40b2770 2024-04-25T08:50:30.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.755+0000 7f00c2a00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00a0066f90 0x7f00a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:30.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.755+0000 7f00c2a00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00a0066f90 0x7f00a0069450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009fd0 tx=0x7f00b40097f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:30.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.755+0000 7f00bbe00640 1 -- 172.21.15.46:0/135531703 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00ac054c20 con 0x7f00c40b2770 2024-04-25T08:50:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00c40cf710 con 0x7f00c40b2770 2024-04-25T08:50:30.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00bbe00640 1 -- 172.21.15.46:0/135531703 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00ac058cc0 con 0x7f00c40b2770 2024-04-25T08:50:30.909 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:50:30.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00a0066f90 msgr2=0x7f00a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00a0066f90 0x7f00a0069450 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009fd0 tx=0x7f00b40097f0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c40b2770 msgr2=0x7f00c40ccb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c40b2770 0x7f00c40ccb50 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0011f0 tx=0x7f00ac00c9b0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 shutdown_connections 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f00a0066f90 0x7f00a0069450 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00c40b2b50 0x7f00c40c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00c40b2770 0x7f00c40ccb50 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 --2- 172.21.15.46:0/135531703 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f00c4099a00 0x7f00c4160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 >> 172.21.15.46:0/135531703 conn(0x7f00c40b3000 msgr2=0x7f00c40736d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 shutdown_connections 2024-04-25T08:50:30.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:30.919+0000 7f00c3e00640 1 -- 172.21.15.46:0/135531703 wait complete. 2024-04-25T08:50:30.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:50:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5ac1400640 1 Processor -- start 2024-04-25T08:50:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5ac1400640 1 -- start start 2024-04-25T08:50:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5ac1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5abc15daf0 0x7f5abc0d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5ac1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5abc0d1e20 0x7f5abc0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5ac1400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc0b7020 con 0x7f5abc15d710 2024-04-25T08:50:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5ac1400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0b72d0 con 0x7f5abc0d1e20 2024-04-25T08:50:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5ac1400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0b7590 con 0x7f5abc15daf0 2024-04-25T08:50:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5abb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5abc0d1e20 0x7f5abc0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5abb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5abc0d1e20 0x7f5abc0ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38042/0 (socket says 172.21.15.46:38042) 2024-04-25T08:50:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5aba800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5abc15d710 0x7f5abc0b6910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57204/0 (socket says 172.21.15.46:57204) 2024-04-25T08:50:31.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5abb200640 1 -- 172.21.15.46:0/249369415 learned_addr learned my addr 172.21.15.46:0/249369415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5ab3e00640 1 --2- 172.21.15.46:0/249369415 >> v2:172.21.15.82:3300/0 conn(0x7f5abc15daf0 0x7f5abc0d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5abb200640 1 -- 172.21.15.46:0/249369415 >> v2:172.21.15.82:3300/0 conn(0x7f5abc15daf0 msgr2=0x7f5abc0d18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5abb200640 1 --2- 172.21.15.46:0/249369415 >> v2:172.21.15.82:3300/0 conn(0x7f5abc15daf0 0x7f5abc0d18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5abb200640 1 -- 172.21.15.46:0/249369415 >> v1:172.21.15.46:6789/0 conn(0x7f5abc15d710 legacy=0x7f5abc0b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5abb200640 1 -- 172.21.15.46:0/249369415 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0b7870 con 0x7f5abc0d1e20 2024-04-25T08:50:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.083+0000 7f5ab3e00640 1 --2- 172.21.15.46:0/249369415 >> v2:172.21.15.82:3300/0 conn(0x7f5abc15daf0 0x7f5abc0d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5abb200640 1 --2- 172.21.15.46:0/249369415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc0d1e20 0x7f5abc0ce550 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009e50 tx=0x7f5ab401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e41d4fc0f063e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ab9e00640 1 -- 172.21.15.46:0/249369415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40a5880 con 0x7f5abc0d1e20 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ab9e00640 1 -- 172.21.15.46:0/249369415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab40a5a20 con 0x7f5abc0d1e20 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ab9e00640 1 -- 172.21.15.46:0/249369415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40a5df0 con 0x7f5abc0d1e20 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ac1400640 1 -- 172.21.15.46:0/249369415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc0d1e20 msgr2=0x7f5abc0ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ac1400640 1 --2- 172.21.15.46:0/249369415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc0d1e20 0x7f5abc0ce550 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009e50 tx=0x7f5ab401c9c0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ac1400640 1 -- 172.21.15.46:0/249369415 shutdown_connections 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ac1400640 1 --2- 172.21.15.46:0/249369415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc0d1e20 0x7f5abc0ce550 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ac1400640 1 --2- 172.21.15.46:0/249369415 >> v2:172.21.15.82:3300/0 conn(0x7f5abc15daf0 0x7f5abc0d18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ac1400640 1 -- 172.21.15.46:0/249369415 >> 172.21.15.46:0/249369415 conn(0x7f5abc072770 msgr2=0x7f5abc072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ac1400640 1 -- 172.21.15.46:0/249369415 shutdown_connections 2024-04-25T08:50:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.087+0000 7f5ac1400640 1 -- 172.21.15.46:0/249369415 wait complete. 2024-04-25T08:50:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 Processor -- start 2024-04-25T08:50:31.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 -- start start 2024-04-25T08:50:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5abc15d710 0x7f5abc153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc15daf0 0x7f5abc160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5abc152a50 0x7f5abc14d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc16a620 con 0x7f5abc15daf0 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0ceea0 con 0x7f5abc15d710 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc16a7a0 con 0x7f5abc152a50 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc15daf0 0x7f5abc160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc15daf0 0x7f5abc160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38058/0 (socket says 172.21.15.46:38058) 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab3e00640 1 -- 172.21.15.46:0/786438697 learned_addr learned my addr 172.21.15.46:0/786438697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5aba800640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5abc15d710 0x7f5abc153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5abb200640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5abc152a50 0x7f5abc14d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab3e00640 1 -- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5abc152a50 msgr2=0x7f5abc14d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab3e00640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5abc152a50 0x7f5abc14d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab3e00640 1 -- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5abc15d710 msgr2=0x7f5abc153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab3e00640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5abc15d710 0x7f5abc153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab3e00640 1 -- 172.21.15.46:0/786438697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc153cf0 con 0x7f5abc15daf0 2024-04-25T08:50:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab3e00640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc15daf0 0x7f5abc160790 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400d800 tx=0x7f5aa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:31.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab2a00640 1 -- 172.21.15.46:0/786438697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa40096a0 con 0x7f5abc15daf0 2024-04-25T08:50:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab2a00640 1 -- 172.21.15.46:0/786438697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa4019070 con 0x7f5abc15daf0 2024-04-25T08:50:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ab2a00640 1 -- 172.21.15.46:0/786438697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4009a80 con 0x7f5abc15daf0 2024-04-25T08:50:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc152f00 con 0x7f5abc15daf0 2024-04-25T08:50:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.091+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc15fcb0 con 0x7f5abc15daf0 2024-04-25T08:50:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.095+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc069080 con 0x7f5abc15daf0 2024-04-25T08:50:31.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.095+0000 7f5ab2a00640 1 -- 172.21.15.46:0/786438697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5aa4009c20 con 0x7f5abc15daf0 2024-04-25T08:50:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.095+0000 7f5ab2a00640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a98066f60 0x7f5a98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.095+0000 7f5ab2a00640 1 -- 172.21.15.46:0/786438697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5aa4086250 con 0x7f5abc15daf0 2024-04-25T08:50:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.095+0000 7f5aba800640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a98066f60 0x7f5a98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.095+0000 7f5aba800640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a98066f60 0x7f5a98069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5abc1571c0 tx=0x7f5aac007cf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:31.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.099+0000 7f5ab2a00640 1 -- 172.21.15.46:0/786438697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5aa4056f90 con 0x7f5abc15daf0 2024-04-25T08:50:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.271+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5abc15a3d0 con 0x7f5abc15daf0 2024-04-25T08:50:31.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.271+0000 7f5ab2a00640 1 -- 172.21.15.46:0/786438697 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v94) v1 ==== 80+0+244 (secure 0 0 0) 0x7f5aa4056890 con 0x7f5abc15daf0 2024-04-25T08:50:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/2777781817 2024-04-26T08:49:51.036604+0000 2024-04-25T08:50:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/2777781817 2024-04-26T08:49:51.036604+0000 2024-04-25T08:50:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/1269610178 2024-04-26T08:48:38.594941+0000 2024-04-25T08:50:31.264 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/1269610178 2024-04-26T08:48:38.594941+0000 2024-04-25T08:50:31.264 INFO:teuthology.orchestra.run.smithi046.stderr:listed 4 entries 2024-04-25T08:50:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a98066f60 msgr2=0x7f5a98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a98066f60 0x7f5a98069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5abc1571c0 tx=0x7f5aac007cf0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc15daf0 msgr2=0x7f5abc160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc15daf0 0x7f5abc160790 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400d800 tx=0x7f5aa400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 shutdown_connections 2024-04-25T08:50:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5a98066f60 0x7f5a98069420 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5abc152a50 0x7f5abc14d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5abc15daf0 0x7f5abc160790 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 --2- 172.21.15.46:0/786438697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5abc15d710 0x7f5abc153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 >> 172.21.15.46:0/786438697 conn(0x7f5abc072770 msgr2=0x7f5abc15c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 shutdown_connections 2024-04-25T08:50:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.275+0000 7f5ac1400640 1 -- 172.21.15.46:0/786438697 wait complete. 2024-04-25T08:50:31.285 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:50:31.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:31.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:31.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:50:31.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:31.313 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:50:31.368 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:50:31.368 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:31.378 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:50:31.378 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:31.378 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:31.425 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:50:31.425 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:50:31.425 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:31.436 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:50:31.436 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:31.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.619+0000 7f335b200640 1 Processor -- start 2024-04-25T08:50:31.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f335b200640 1 -- start start 2024-04-25T08:50:31.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f335b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f33540cd920 0x7f3354140e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f335b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33540710e0 con 0x7f3354071ea0 2024-04-25T08:50:31.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f335b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33540713c0 con 0x7f335413e540 2024-04-25T08:50:31.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f335b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33540716a0 con 0x7f33540cd920 2024-04-25T08:50:31.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f335a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3354071ea0 0x7f3354141390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57218/0 (socket says 172.21.15.46:57218) 2024-04-25T08:50:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f335a800640 1 -- 172.21.15.46:0/405557990 learned_addr learned my addr 172.21.15.46:0/405557990 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f3359400640 1 --2- 172.21.15.46:0/405557990 >> v2:172.21.15.82:3300/0 conn(0x7f33540cd920 0x7f3354140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f3358a00640 1 -- 172.21.15.46:0/405557990 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135983327 0 0) 0x7f33540710e0 con 0x7f3354071ea0 2024-04-25T08:50:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f3358a00640 1 -- 172.21.15.46:0/405557990 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f333c003530 con 0x7f3354071ea0 2024-04-25T08:50:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f3358a00640 1 -- 172.21.15.46:0/405557990 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90127771 0 0) 0x7f33540713c0 con 0x7f335413e540 2024-04-25T08:50:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f3358a00640 1 -- 172.21.15.46:0/405557990 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33540710e0 con 0x7f335413e540 2024-04-25T08:50:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f3359400640 1 -- 172.21.15.46:0/405557990 >> v1:172.21.15.82:6789/0 conn(0x7f335413e540 legacy=0x7f33540cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f3359400640 1 -- 172.21.15.46:0/405557990 >> v1:172.21.15.46:6789/0 conn(0x7f3354071ea0 legacy=0x7f3354141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.623+0000 7f3359400640 1 -- 172.21.15.46:0/405557990 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3354071980 con 0x7f33540cd920 2024-04-25T08:50:31.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f3359400640 1 --2- 172.21.15.46:0/405557990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33540cd920 0x7f3354140e50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f33440097c0 tx=0x7f334401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee06965e2cdd5776 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:31.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f3358a00640 1 -- 172.21.15.46:0/405557990 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440ab640 con 0x7f33540cd920 2024-04-25T08:50:31.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f3358a00640 1 -- 172.21.15.46:0/405557990 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33440b0070 con 0x7f33540cd920 2024-04-25T08:50:31.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f3358a00640 1 -- 172.21.15.46:0/405557990 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440ab9e0 con 0x7f33540cd920 2024-04-25T08:50:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f335b200640 1 -- 172.21.15.46:0/405557990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33540cd920 msgr2=0x7f3354140e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f335b200640 1 --2- 172.21.15.46:0/405557990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33540cd920 0x7f3354140e50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f33440097c0 tx=0x7f334401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f335b200640 1 -- 172.21.15.46:0/405557990 shutdown_connections 2024-04-25T08:50:31.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f335b200640 1 --2- 172.21.15.46:0/405557990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33540cd920 0x7f3354140e50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f335b200640 1 -- 172.21.15.46:0/405557990 >> 172.21.15.46:0/405557990 conn(0x7f3354066fa0 msgr2=0x7f335406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:31.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f335b200640 1 -- 172.21.15.46:0/405557990 shutdown_connections 2024-04-25T08:50:31.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.627+0000 7f335b200640 1 -- 172.21.15.46:0/405557990 wait complete. 2024-04-25T08:50:31.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 Processor -- start 2024-04-25T08:50:31.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 -- start start 2024-04-25T08:50:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3354071ea0 0x7f33540b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33540cd920 0x7f3354137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f335413e540 0x7f33540b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335413f8b0 con 0x7f3354071ea0 2024-04-25T08:50:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335413f730 con 0x7f335413e540 2024-04-25T08:50:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335413fa30 con 0x7f33540cd920 2024-04-25T08:50:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3354071ea0 0x7f33540b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3354071ea0 0x7f33540b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38062/0 (socket says 172.21.15.46:38062) 2024-04-25T08:50:31.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359e00640 1 -- 172.21.15.46:0/411263351 learned_addr learned my addr 172.21.15.46:0/411263351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359400640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33540cd920 0x7f3354137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335a800640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f335413e540 0x7f33540b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359e00640 1 -- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33540cd920 msgr2=0x7f3354137e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359e00640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33540cd920 0x7f3354137e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359e00640 1 -- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f335413e540 msgr2=0x7f33540b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359e00640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f335413e540 0x7f33540b24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359e00640 1 -- 172.21.15.46:0/411263351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33540b2f40 con 0x7f3354071ea0 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f3359e00640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3354071ea0 0x7f33540b3fe0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f334800c460 tx=0x7f334800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f334e000640 1 -- 172.21.15.46:0/411263351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348017070 con 0x7f3354071ea0 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f334e000640 1 -- 172.21.15.46:0/411263351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33480052f0 con 0x7f3354071ea0 2024-04-25T08:50:31.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33540b3220 con 0x7f3354071ea0 2024-04-25T08:50:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f334e000640 1 -- 172.21.15.46:0/411263351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3348003840 con 0x7f3354071ea0 2024-04-25T08:50:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.631+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33540b37e0 con 0x7f3354071ea0 2024-04-25T08:50:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.635+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3354069010 con 0x7f3354071ea0 2024-04-25T08:50:31.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.635+0000 7f334e000640 1 -- 172.21.15.46:0/411263351 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f334801a070 con 0x7f3354071ea0 2024-04-25T08:50:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.635+0000 7f334e000640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3338066f90 0x7f3338069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.635+0000 7f334e000640 1 -- 172.21.15.46:0/411263351 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f33480845d0 con 0x7f3354071ea0 2024-04-25T08:50:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.635+0000 7f3359400640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3338066f90 0x7f3338069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.635+0000 7f3359400640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3338066f90 0x7f3338069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f33440af000 tx=0x7f33440b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.639+0000 7f334e000640 1 -- 172.21.15.46:0/411263351 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3348054cd0 con 0x7f3354071ea0 2024-04-25T08:50:31.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f335406ed80 con 0x7f3354071ea0 2024-04-25T08:50:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f334e000640 1 -- 172.21.15.46:0/411263351 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f3348058d70 con 0x7f3354071ea0 2024-04-25T08:50:31.825 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:31.825 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:49:54.058324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1358086076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1358086076},{"type":"v1","addr":"172.21.15.46:6835","nonce":1358086076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:50:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 75 2024-04-25T08:50:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3338066f90 msgr2=0x7f3338069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3338066f90 0x7f3338069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f33440af000 tx=0x7f33440b6040 comp rx=0 tx=0).stop 2024-04-25T08:50:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3354071ea0 msgr2=0x7f33540b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3354071ea0 0x7f33540b3fe0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f334800c460 tx=0x7f334800c920 comp rx=0 tx=0).stop 2024-04-25T08:50:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 shutdown_connections 2024-04-25T08:50:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3338066f90 0x7f3338069450 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f335413e540 0x7f33540b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f33540cd920 0x7f3354137e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 --2- 172.21.15.46:0/411263351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3354071ea0 0x7f33540b3fe0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 >> 172.21.15.46:0/411263351 conn(0x7f3354066fa0 msgr2=0x7f335413bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 shutdown_connections 2024-04-25T08:50:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.835+0000 7f335b200640 1 -- 172.21.15.46:0/411263351 wait complete. 2024-04-25T08:50:31.847 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.995+0000 7f1c6bc00640 1 Processor -- start 2024-04-25T08:50:31.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.995+0000 7f1c6bc00640 1 -- start start 2024-04-25T08:50:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.995+0000 7f1c6bc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c640cf140 0x7f1c6415d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c6bc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf520 0x7f1c6415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c6bc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6406ed70 con 0x7f1c640afd40 2024-04-25T08:50:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c6bc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c640afb40 con 0x7f1c640cf140 2024-04-25T08:50:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c6bc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640d13f0 con 0x7f1c640cf520 2024-04-25T08:50:31.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c63e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c640afd40 0x7f1c640ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57222/0 (socket says 172.21.15.46:57222) 2024-04-25T08:50:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c63e00640 1 -- 172.21.15.46:0/802814902 learned_addr learned my addr 172.21.15.46:0/802814902 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c69200640 1 --2- 172.21.15.46:0/802814902 >> v2:172.21.15.82:3300/0 conn(0x7f1c640cf140 0x7f1c6415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c69c00640 1 --2- 172.21.15.46:0/802814902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf520 0x7f1c6415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c63400640 1 -- 172.21.15.46:0/802814902 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667872551 0 0) 0x7f1c50001120 con 0x7f1c640afd40 2024-04-25T08:50:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c63400640 1 -- 172.21.15.46:0/802814902 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c4c003530 con 0x7f1c640afd40 2024-04-25T08:50:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c69c00640 1 -- 172.21.15.46:0/802814902 >> v2:172.21.15.82:3300/0 conn(0x7f1c640cf140 msgr2=0x7f1c6415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c69c00640 1 --2- 172.21.15.46:0/802814902 >> v2:172.21.15.82:3300/0 conn(0x7f1c640cf140 0x7f1c6415d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c69c00640 1 -- 172.21.15.46:0/802814902 >> v1:172.21.15.46:6789/0 conn(0x7f1c640afd40 legacy=0x7f1c640ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c69c00640 1 -- 172.21.15.46:0/802814902 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c64162530 con 0x7f1c640cf520 2024-04-25T08:50:31.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:31.999+0000 7f1c69c00640 1 --2- 172.21.15.46:0/802814902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf520 0x7f1c6415dad0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009d90 tx=0x7f1c5c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=879526dc840267db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:31.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c63400640 1 -- 172.21.15.46:0/802814902 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0b2070 con 0x7f1c640cf520 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c63400640 1 -- 172.21.15.46:0/802814902 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c0a57d0 con 0x7f1c640cf520 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c63400640 1 -- 172.21.15.46:0/802814902 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0a5ba0 con 0x7f1c640cf520 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 -- 172.21.15.46:0/802814902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf520 msgr2=0x7f1c6415dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 --2- 172.21.15.46:0/802814902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf520 0x7f1c6415dad0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009d90 tx=0x7f1c5c01c8e0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 -- 172.21.15.46:0/802814902 shutdown_connections 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 --2- 172.21.15.46:0/802814902 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf520 0x7f1c6415dad0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 --2- 172.21.15.46:0/802814902 >> v2:172.21.15.82:3300/0 conn(0x7f1c640cf140 0x7f1c6415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 -- 172.21.15.46:0/802814902 >> 172.21.15.46:0/802814902 conn(0x7f1c640728c0 msgr2=0x7f1c64099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 -- 172.21.15.46:0/802814902 shutdown_connections 2024-04-25T08:50:31.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 -- 172.21.15.46:0/802814902 wait complete. 2024-04-25T08:50:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 Processor -- start 2024-04-25T08:50:31.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 -- start start 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c640afd40 0x7f1c64148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf140 0x7f1c641520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c640cf520 0x7f1c6415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640af7f0 con 0x7f1c640afd40 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c64162010 con 0x7f1c640cf520 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c6bc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6406b600 con 0x7f1c640cf140 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c69200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c640afd40 0x7f1c64148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c69200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c640afd40 0x7f1c64148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38074/0 (socket says 172.21.15.46:38074) 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c69200640 1 -- 172.21.15.46:0/381121189 learned_addr learned my addr 172.21.15.46:0/381121189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:31.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c63e00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf140 0x7f1c641520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c69200640 1 -- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf140 msgr2=0x7f1c641520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c69c00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c640cf520 0x7f1c6415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c69200640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf140 0x7f1c641520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.003+0000 7f1c69200640 1 -- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c640cf520 msgr2=0x7f1c6415d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c69200640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c640cf520 0x7f1c6415d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c69200640 1 -- 172.21.15.46:0/381121189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6415d020 con 0x7f1c640afd40 2024-04-25T08:50:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c63e00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf140 0x7f1c641520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c69c00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c640cf520 0x7f1c6415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:31.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c69200640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c640afd40 0x7f1c64148b80 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e8a0 tx=0x7f1c5800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c60c00640 1 -- 172.21.15.46:0/381121189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58017070 con 0x7f1c640afd40 2024-04-25T08:50:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c60c00640 1 -- 172.21.15.46:0/381121189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5800c940 con 0x7f1c640afd40 2024-04-25T08:50:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c60c00640 1 -- 172.21.15.46:0/381121189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800ccf0 con 0x7f1c640afd40 2024-04-25T08:50:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6415c780 con 0x7f1c640afd40 2024-04-25T08:50:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6415cd40 con 0x7f1c640afd40 2024-04-25T08:50:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6406ccb0 con 0x7f1c640afd40 2024-04-25T08:50:32.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c60c00640 1 -- 172.21.15.46:0/381121189 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c5801a070 con 0x7f1c640afd40 2024-04-25T08:50:32.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c60c00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c44066f60 0x7f1c44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.007+0000 7f1c60c00640 1 -- 172.21.15.46:0/381121189 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f1c58085150 con 0x7f1c640afd40 2024-04-25T08:50:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.011+0000 7f1c63e00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c44066f60 0x7f1c44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.011+0000 7f1c63e00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c44066f60 0x7f1c44069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1c64153d20 tx=0x7f1c50009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.015+0000 7f1c60c00640 1 -- 172.21.15.46:0/381121189 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c58055f40 con 0x7f1c640afd40 2024-04-25T08:50:32.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.179+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c64072f90 con 0x7f1c640afd40 2024-04-25T08:50:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.179+0000 7f1c60c00640 1 -- 172.21.15.46:0/381121189 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17289 (secure 0 0 0) 0x7f1c58055840 con 0x7f1c640afd40 2024-04-25T08:50:32.170 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:32.170 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:49:51.036629+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:48:26.132790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:48:26.714570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6834/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6835/1269610178":"2024-04-26T08:48:38.594941+0000","172.21.15.46:6834/1269610178":"2024-04-26T08:48:38.594941+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c44066f60 msgr2=0x7f1c44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c44066f60 0x7f1c44069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1c64153d20 tx=0x7f1c50009040 comp rx=0 tx=0).stop 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c640afd40 msgr2=0x7f1c64148b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c640afd40 0x7f1c64148b80 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e8a0 tx=0x7f1c5800eca0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 shutdown_connections 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1c44066f60 0x7f1c44069420 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c640cf520 0x7f1c6415d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1c640cf140 0x7f1c641520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 --2- 172.21.15.46:0/381121189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c640afd40 0x7f1c64148b80 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 >> 172.21.15.46:0/381121189 conn(0x7f1c640728c0 msgr2=0x7f1c64149820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 shutdown_connections 2024-04-25T08:50:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.183+0000 7f1c6bc00640 1 -- 172.21.15.46:0/381121189 wait complete. 2024-04-25T08:50:32.193 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:48:26.132790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:48:26.714570+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:50:32.194 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:50:32.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0527c00640 1 Processor -- start 2024-04-25T08:50:32.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0527c00640 1 -- start start 2024-04-25T08:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0527c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0520145310 0x7f052006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0527c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f052015d090 0x7f052015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0527c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05200cd930 con 0x7f052006dc70 2024-04-25T08:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0527c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05200cdbd0 con 0x7f052015d090 2024-04-25T08:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0527c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05200cdeb0 con 0x7f0520145310 2024-04-25T08:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0525c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f052015d090 0x7f052015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0525c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f052015d090 0x7f052015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38078/0 (socket says 172.21.15.46:38078) 2024-04-25T08:50:32.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0525c00640 1 -- 172.21.15.46:0/120845170 learned_addr learned my addr 172.21.15.46:0/120845170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0525200640 1 --2- 172.21.15.46:0/120845170 >> v2:172.21.15.82:3300/0 conn(0x7f0520145310 0x7f052006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0525c00640 1 -- 172.21.15.46:0/120845170 >> v2:172.21.15.82:3300/0 conn(0x7f0520145310 msgr2=0x7f052006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0525c00640 1 --2- 172.21.15.46:0/120845170 >> v2:172.21.15.82:3300/0 conn(0x7f0520145310 0x7f052006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0525c00640 1 -- 172.21.15.46:0/120845170 >> v1:172.21.15.82:6789/0 conn(0x7f052006dc70 legacy=0x7f05200cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0525c00640 1 -- 172.21.15.46:0/120845170 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200ce190 con 0x7f052015d090 2024-04-25T08:50:32.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.347+0000 7f0525200640 1 --2- 172.21.15.46:0/120845170 >> v2:172.21.15.82:3300/0 conn(0x7f0520145310 0x7f052006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f0525c00640 1 --2- 172.21.15.46:0/120845170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052015d090 0x7f052015d4e0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0520072530 tx=0x7f051801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b08805faa4b869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:32.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f051f400640 1 -- 172.21.15.46:0/120845170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05180a57a0 con 0x7f052015d090 2024-04-25T08:50:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f051f400640 1 -- 172.21.15.46:0/120845170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05180a5940 con 0x7f052015d090 2024-04-25T08:50:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f051f400640 1 -- 172.21.15.46:0/120845170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05180a5d10 con 0x7f052015d090 2024-04-25T08:50:32.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f0527c00640 1 -- 172.21.15.46:0/120845170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052015d090 msgr2=0x7f052015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f0527c00640 1 --2- 172.21.15.46:0/120845170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052015d090 0x7f052015d4e0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0520072530 tx=0x7f051801c9f0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f0527c00640 1 -- 172.21.15.46:0/120845170 shutdown_connections 2024-04-25T08:50:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f0527c00640 1 --2- 172.21.15.46:0/120845170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052015d090 0x7f052015d4e0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f0527c00640 1 --2- 172.21.15.46:0/120845170 >> v2:172.21.15.82:3300/0 conn(0x7f0520145310 0x7f052006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f0527c00640 1 -- 172.21.15.46:0/120845170 >> 172.21.15.46:0/120845170 conn(0x7f052006ac10 msgr2=0x7f052006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f0527c00640 1 -- 172.21.15.46:0/120845170 shutdown_connections 2024-04-25T08:50:32.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.351+0000 7f0527c00640 1 -- 172.21.15.46:0/120845170 wait complete. 2024-04-25T08:50:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 Processor -- start 2024-04-25T08:50:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 -- start start 2024-04-25T08:50:32.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0520145310 0x7f05200acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f052015d090 0x7f05200b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0520158df0 0x7f0520158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520168510 con 0x7f0520145310 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05200cf380 con 0x7f0520158df0 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0520168690 con 0x7f052015d090 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0520145310 0x7f05200acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0520145310 0x7f05200acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38086/0 (socket says 172.21.15.46:38086) 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525200640 1 -- 172.21.15.46:0/1809763322 learned_addr learned my addr 172.21.15.46:0/1809763322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525c00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0520158df0 0x7f0520158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f051fe00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f052015d090 0x7f05200b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525200640 1 -- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f052015d090 msgr2=0x7f05200b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525200640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f052015d090 0x7f05200b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525200640 1 -- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0520158df0 msgr2=0x7f0520158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525200640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0520158df0 0x7f0520158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525200640 1 -- 172.21.15.46:0/1809763322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200ad2c0 con 0x7f0520145310 2024-04-25T08:50:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0525200640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0520145310 0x7f05200acbc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f051400e840 tx=0x7f051400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f051cc00640 1 -- 172.21.15.46:0/1809763322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514017070 con 0x7f0520145310 2024-04-25T08:50:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f051cc00640 1 -- 172.21.15.46:0/1809763322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051400c8d0 con 0x7f0520145310 2024-04-25T08:50:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f051cc00640 1 -- 172.21.15.46:0/1809763322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051400f040 con 0x7f0520145310 2024-04-25T08:50:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05200ad5a0 con 0x7f0520145310 2024-04-25T08:50:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.355+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05200adb60 con 0x7f0520145310 2024-04-25T08:50:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.359+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0520069010 con 0x7f0520145310 2024-04-25T08:50:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.359+0000 7f051cc00640 1 -- 172.21.15.46:0/1809763322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f051401a070 con 0x7f0520145310 2024-04-25T08:50:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.359+0000 7f051cc00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0500066f90 0x7f0500069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.359+0000 7f051cc00640 1 -- 172.21.15.46:0/1809763322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f0514084b80 con 0x7f0520145310 2024-04-25T08:50:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.359+0000 7f051fe00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0500066f90 0x7f0500069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.359+0000 7f051fe00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0500066f90 0x7f0500069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f05200aea20 tx=0x7f050c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:32.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.363+0000 7f051cc00640 1 -- 172.21.15.46:0/1809763322 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0514055990 con 0x7f0520145310 2024-04-25T08:50:32.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.523+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0520073aa0 con 0x7f0520145310 2024-04-25T08:50:32.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.523+0000 7f051cc00640 1 -- 172.21.15.46:0/1809763322 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0514055290 con 0x7f0520145310 2024-04-25T08:50:32.515 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:50:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0500066f90 msgr2=0x7f0500069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0500066f90 0x7f0500069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f05200aea20 tx=0x7f050c009040 comp rx=0 tx=0).stop 2024-04-25T08:50:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0520145310 msgr2=0x7f05200acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0520145310 0x7f05200acbc0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f051400e840 tx=0x7f051400ed00 comp rx=0 tx=0).stop 2024-04-25T08:50:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 shutdown_connections 2024-04-25T08:50:32.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0500066f90 0x7f0500069450 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0520158df0 0x7f0520158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f052015d090 0x7f05200b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 --2- 172.21.15.46:0/1809763322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0520145310 0x7f05200acbc0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 >> 172.21.15.46:0/1809763322 conn(0x7f052006ac10 msgr2=0x7f0520071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 shutdown_connections 2024-04-25T08:50:32.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.527+0000 7f0527c00640 1 -- 172.21.15.46:0/1809763322 wait complete. 2024-04-25T08:50:32.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:50:32.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9af5a00640 1 Processor -- start 2024-04-25T08:50:32.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9af5a00640 1 -- start start 2024-04-25T08:50:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9af5a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9af009a110 0x7f9af00b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9af5a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 0x7f9af00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9af5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af00cce10 con 0x7f9af015efb0 2024-04-25T08:50:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9af5a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af009abb0 con 0x7f9af009a110 2024-04-25T08:50:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9af5a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af015f390 con 0x7f9af009a740 2024-04-25T08:50:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9ae7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9af009a110 0x7f9af00b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9aeea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9af015efb0 0x7f9af0099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57226/0 (socket says 172.21.15.46:57226) 2024-04-25T08:50:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9ae7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9af009a110 0x7f9af00b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38098/0 (socket says 172.21.15.46:38098) 2024-04-25T08:50:32.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9aeea00640 1 -- 172.21.15.46:0/4195791806 learned_addr learned my addr 172.21.15.46:0/4195791806 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9aef400640 1 --2- 172.21.15.46:0/4195791806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 0x7f9af00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9ae7e00640 1 -- 172.21.15.46:0/4195791806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 msgr2=0x7f9af00cf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9ae7e00640 1 --2- 172.21.15.46:0/4195791806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 0x7f9af00cf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9ae7e00640 1 -- 172.21.15.46:0/4195791806 >> v1:172.21.15.46:6789/0 conn(0x7f9af015efb0 legacy=0x7f9af0099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9ae7e00640 1 -- 172.21.15.46:0/4195791806 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af00cd0f0 con 0x7f9af009a110 2024-04-25T08:50:32.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.691+0000 7f9aee000640 1 -- 172.21.15.46:0/4195791806 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589310702 0 0) 0x7f9af00cce10 con 0x7f9af015efb0 2024-04-25T08:50:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9ae7e00640 1 --2- 172.21.15.46:0/4195791806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af009a110 0x7f9af00b23f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80097c0 tx=0x7f9ad801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26269938c79433c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9aee000640 1 -- 172.21.15.46:0/4195791806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad80b0070 con 0x7f9af009a110 2024-04-25T08:50:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9aee000640 1 -- 172.21.15.46:0/4195791806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad80ab600 con 0x7f9af009a110 2024-04-25T08:50:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9aee000640 1 -- 172.21.15.46:0/4195791806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad80ab9d0 con 0x7f9af009a110 2024-04-25T08:50:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9af5a00640 1 -- 172.21.15.46:0/4195791806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af009a110 msgr2=0x7f9af00b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9af5a00640 1 --2- 172.21.15.46:0/4195791806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af009a110 0x7f9af00b23f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80097c0 tx=0x7f9ad801c410 comp rx=0 tx=0).stop 2024-04-25T08:50:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9af5a00640 1 -- 172.21.15.46:0/4195791806 shutdown_connections 2024-04-25T08:50:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9af5a00640 1 --2- 172.21.15.46:0/4195791806 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 0x7f9af00cf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9af5a00640 1 --2- 172.21.15.46:0/4195791806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af009a110 0x7f9af00b23f0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9af5a00640 1 -- 172.21.15.46:0/4195791806 >> 172.21.15.46:0/4195791806 conn(0x7f9af00b3120 msgr2=0x7f9af00b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9af5a00640 1 -- 172.21.15.46:0/4195791806 shutdown_connections 2024-04-25T08:50:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.695+0000 7f9af5a00640 1 -- 172.21.15.46:0/4195791806 wait complete. 2024-04-25T08:50:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 Processor -- start 2024-04-25T08:50:32.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 -- start start 2024-04-25T08:50:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af009a110 0x7f9af014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 0x7f9af00c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af015efb0 0x7f9af0152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00d0560 con 0x7f9af015efb0 2024-04-25T08:50:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00d03e0 con 0x7f9af009a110 2024-04-25T08:50:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af00d06e0 con 0x7f9af009a740 2024-04-25T08:50:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af015efb0 0x7f9af0152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af015efb0 0x7f9af0152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38114/0 (socket says 172.21.15.46:38114) 2024-04-25T08:50:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9ae7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 0x7f9af00c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aef400640 1 -- 172.21.15.46:0/266828732 learned_addr learned my addr 172.21.15.46:0/266828732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aeea00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af009a110 0x7f9af014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aef400640 1 -- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 msgr2=0x7f9af00c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aef400640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 0x7f9af00c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aef400640 1 -- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af009a110 msgr2=0x7f9af014d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aef400640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af009a110 0x7f9af014d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aef400640 1 -- 172.21.15.46:0/266828732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af0150d50 con 0x7f9af015efb0 2024-04-25T08:50:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9ae7e00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 0x7f9af00c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9aef400640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af015efb0 0x7f9af0152e40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009d30 tx=0x7f9ae800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9ae6a00640 1 -- 172.21.15.46:0/266828732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8018020 con 0x7f9af015efb0 2024-04-25T08:50:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9ae6a00640 1 -- 172.21.15.46:0/266828732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8014480 con 0x7f9af015efb0 2024-04-25T08:50:32.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9ae6a00640 1 -- 172.21.15.46:0/266828732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8014870 con 0x7f9af015efb0 2024-04-25T08:50:32.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af0151030 con 0x7f9af015efb0 2024-04-25T08:50:32.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.699+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af00b45d0 con 0x7f9af015efb0 2024-04-25T08:50:32.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.703+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af0151200 con 0x7f9af015efb0 2024-04-25T08:50:32.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.703+0000 7f9ae6a00640 1 -- 172.21.15.46:0/266828732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ae801b020 con 0x7f9af015efb0 2024-04-25T08:50:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.703+0000 7f9ae6a00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9acc066f60 0x7f9acc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.703+0000 7f9ae6a00640 1 -- 172.21.15.46:0/266828732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f9ae8085c10 con 0x7f9af015efb0 2024-04-25T08:50:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.703+0000 7f9aeea00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9acc066f60 0x7f9acc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.703+0000 7f9aeea00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9acc066f60 0x7f9acc069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0000f70 tx=0x7f9ae0007df0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:32.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.707+0000 7f9ae6a00640 1 -- 172.21.15.46:0/266828732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ae8022070 con 0x7f9af015efb0 2024-04-25T08:50:32.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.867+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9af0069010 con 0x7f9af015efb0 2024-04-25T08:50:32.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.867+0000 7f9ae6a00640 1 -- 172.21.15.46:0/266828732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9ae8010040 con 0x7f9af015efb0 2024-04-25T08:50:32.859 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:50:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9acc066f60 msgr2=0x7f9acc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9acc066f60 0x7f9acc069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0000f70 tx=0x7f9ae0007df0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af015efb0 msgr2=0x7f9af0152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af015efb0 0x7f9af0152e40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009d30 tx=0x7f9ae800c490 comp rx=0 tx=0).stop 2024-04-25T08:50:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 shutdown_connections 2024-04-25T08:50:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9acc066f60 0x7f9acc069420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af015efb0 0x7f9af0152e40 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9af009a740 0x7f9af00c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 --2- 172.21.15.46:0/266828732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9af009a110 0x7f9af014d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 >> 172.21.15.46:0/266828732 conn(0x7f9af00b3120 msgr2=0x7f9af00c69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 shutdown_connections 2024-04-25T08:50:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:32.871+0000 7f9af5a00640 1 -- 172.21.15.46:0/266828732 wait complete. 2024-04-25T08:50:32.880 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:50:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.031+0000 7f92bd400640 1 Processor -- start 2024-04-25T08:50:33.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.031+0000 7f92bd400640 1 -- start start 2024-04-25T08:50:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92bd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92b8143990 0x7f92b80cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92bd400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92b81440d0 0x7f92b806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92bd400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b815e440 con 0x7f92b806ef80 2024-04-25T08:50:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92bd400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b815ed30 con 0x7f92b8143990 2024-04-25T08:50:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92bd400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b80b3730 con 0x7f92b81440d0 2024-04-25T08:50:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92b6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92b8143990 0x7f92b80cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92b7200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f92b806ef80 0x7f92b80cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57238/0 (socket says 172.21.15.46:57238) 2024-04-25T08:50:33.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92b7200640 1 -- 172.21.15.46:0/3057627594 learned_addr learned my addr 172.21.15.46:0/3057627594 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92ade00640 1 --2- 172.21.15.46:0/3057627594 >> v2:172.21.15.82:3300/0 conn(0x7f92b81440d0 0x7f92b806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92b6800640 1 -- 172.21.15.46:0/3057627594 >> v2:172.21.15.82:3300/0 conn(0x7f92b81440d0 msgr2=0x7f92b806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92b6800640 1 --2- 172.21.15.46:0/3057627594 >> v2:172.21.15.82:3300/0 conn(0x7f92b81440d0 0x7f92b806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92b6800640 1 -- 172.21.15.46:0/3057627594 >> v1:172.21.15.46:6789/0 conn(0x7f92b806ef80 legacy=0x7f92b80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92b5e00640 1 -- 172.21.15.46:0/3057627594 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601701398 0 0) 0x7f92b815e440 con 0x7f92b806ef80 2024-04-25T08:50:33.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.035+0000 7f92b6800640 1 -- 172.21.15.46:0/3057627594 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b815e6e0 con 0x7f92b8143990 2024-04-25T08:50:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92b6800640 1 --2- 172.21.15.46:0/3057627594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b8143990 0x7f92b80cecc0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f92b8070c80 tx=0x7f92a801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c34dfcb15ec226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92b5e00640 1 -- 172.21.15.46:0/3057627594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80b0070 con 0x7f92b8143990 2024-04-25T08:50:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92b5e00640 1 -- 172.21.15.46:0/3057627594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a80ab600 con 0x7f92b8143990 2024-04-25T08:50:33.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92b5e00640 1 -- 172.21.15.46:0/3057627594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80ab9d0 con 0x7f92b8143990 2024-04-25T08:50:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 -- 172.21.15.46:0/3057627594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b8143990 msgr2=0x7f92b80cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 --2- 172.21.15.46:0/3057627594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b8143990 0x7f92b80cecc0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f92b8070c80 tx=0x7f92a801c480 comp rx=0 tx=0).stop 2024-04-25T08:50:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 -- 172.21.15.46:0/3057627594 shutdown_connections 2024-04-25T08:50:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 --2- 172.21.15.46:0/3057627594 >> v2:172.21.15.82:3300/0 conn(0x7f92b81440d0 0x7f92b806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 --2- 172.21.15.46:0/3057627594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b8143990 0x7f92b80cecc0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 -- 172.21.15.46:0/3057627594 >> 172.21.15.46:0/3057627594 conn(0x7f92b806ac10 msgr2=0x7f92b806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 -- 172.21.15.46:0/3057627594 shutdown_connections 2024-04-25T08:50:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 -- 172.21.15.46:0/3057627594 wait complete. 2024-04-25T08:50:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 Processor -- start 2024-04-25T08:50:33.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.039+0000 7f92bd400640 1 -- start start 2024-04-25T08:50:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92bd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b806ef80 0x7f92b8159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92bd400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92b8143990 0x7f92b81fa720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92bd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b81440d0 0x7f92b815adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92bd400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8143d70 con 0x7f92b806ef80 2024-04-25T08:50:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92bd400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b806f390 con 0x7f92b81440d0 2024-04-25T08:50:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92bd400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8143ef0 con 0x7f92b8143990 2024-04-25T08:50:33.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b806ef80 0x7f92b8159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b806ef80 0x7f92b8159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38120/0 (socket says 172.21.15.46:38120) 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b6800640 1 -- 172.21.15.46:0/1585529567 learned_addr learned my addr 172.21.15.46:0/1585529567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92ade00640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92b8143990 0x7f92b81fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b7200640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b81440d0 0x7f92b815adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b6800640 1 -- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92b8143990 msgr2=0x7f92b81fa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b6800640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92b8143990 0x7f92b81fa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b6800640 1 -- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b81440d0 msgr2=0x7f92b815adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b6800640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b81440d0 0x7f92b815adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b6800640 1 -- 172.21.15.46:0/1585529567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b815b640 con 0x7f92b806ef80 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92ade00640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92b8143990 0x7f92b81fa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92b6800640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b806ef80 0x7f92b8159f10 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f92a801a0b0 tx=0x7f92a801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:33.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92af400640 1 -- 172.21.15.46:0/1585529567 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80b0050 con 0x7f92b806ef80 2024-04-25T08:50:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92af400640 1 -- 172.21.15.46:0/1585529567 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a80bc070 con 0x7f92b806ef80 2024-04-25T08:50:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92af400640 1 -- 172.21.15.46:0/1585529567 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80b5830 con 0x7f92b806ef80 2024-04-25T08:50:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b815b920 con 0x7f92b806ef80 2024-04-25T08:50:33.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b81fa200 con 0x7f92b806ef80 2024-04-25T08:50:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b8069010 con 0x7f92b806ef80 2024-04-25T08:50:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.043+0000 7f92af400640 1 -- 172.21.15.46:0/1585529567 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92a80b59d0 con 0x7f92b806ef80 2024-04-25T08:50:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.047+0000 7f92af400640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9294066f90 0x7f9294069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.047+0000 7f92af400640 1 -- 172.21.15.46:0/1585529567 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f92a811fc80 con 0x7f92b806ef80 2024-04-25T08:50:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.047+0000 7f92ade00640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9294066f90 0x7f9294069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.047+0000 7f92ade00640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9294066f90 0x7f9294069450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f92b8139a90 tx=0x7f92a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:33.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.051+0000 7f92af400640 1 -- 172.21.15.46:0/1585529567 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92a8016070 con 0x7f92b806ef80 2024-04-25T08:50:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92b80b4490 con 0x7f92b806ef80 2024-04-25T08:50:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92af400640 1 -- 172.21.15.46:0/1585529567 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f92a80f0940 con 0x7f92b806ef80 2024-04-25T08:50:33.205 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:50:33.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9294066f90 msgr2=0x7f9294069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9294066f90 0x7f9294069450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f92b8139a90 tx=0x7f92a0008040 comp rx=0 tx=0).stop 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b806ef80 msgr2=0x7f92b8159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b806ef80 0x7f92b8159f10 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f92a801a0b0 tx=0x7f92a801acc0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 shutdown_connections 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9294066f90 0x7f9294069450 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b81440d0 0x7f92b815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92b8143990 0x7f92b81fa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 --2- 172.21.15.46:0/1585529567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92b806ef80 0x7f92b8159f10 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 >> 172.21.15.46:0/1585529567 conn(0x7f92b806ac10 msgr2=0x7f92b81407c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 shutdown_connections 2024-04-25T08:50:33.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.215+0000 7f92bd400640 1 -- 172.21.15.46:0/1585529567 wait complete. 2024-04-25T08:50:33.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:33.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6adba00640 1 Processor -- start 2024-04-25T08:50:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6adba00640 1 -- start start 2024-04-25T08:50:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6adba00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6ad4099a00 0x7f6ad4099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6adba00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad409a320 con 0x7f6ad40b2240 2024-04-25T08:50:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6adba00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad406bdc0 con 0x7f6ad40ccde0 2024-04-25T08:50:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6adba00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad406db90 con 0x7f6ad4099a00 2024-04-25T08:50:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad9a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6ad40b2240 0x7f6ad40b2670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57240/0 (socket says 172.21.15.46:57240) 2024-04-25T08:50:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad9a00640 1 -- 172.21.15.46:0/3229660006 learned_addr learned my addr 172.21.15.46:0/3229660006 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323458281 0 0) 0x7f6ad409a320 con 0x7f6ad40b2240 2024-04-25T08:50:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad9000640 1 --2- 172.21.15.46:0/3229660006 >> v2:172.21.15.82:3300/0 conn(0x7f6ad4099a00 0x7f6ad4099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab8003530 con 0x7f6ad40b2240 2024-04-25T08:50:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 388970109 0 0) 0x7f6ab8003530 con 0x7f6ad40b2240 2024-04-25T08:50:33.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 >> v2:172.21.15.82:3300/0 conn(0x7f6ad4099a00 msgr2=0x7f6ad4099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad3400640 1 --2- 172.21.15.46:0/3229660006 >> v2:172.21.15.82:3300/0 conn(0x7f6ad4099a00 0x7f6ad4099de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 >> v1:172.21.15.82:6789/0 conn(0x7f6ad40ccde0 legacy=0x7f6ad40cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad406c0a0 con 0x7f6ad40b2240 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f6acc001680 con 0x7f6ad40b2240 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.379+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 >> v1:172.21.15.46:6789/0 conn(0x7f6ad40b2240 legacy=0x7f6ad40b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3400640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab8003a10 0x7f6ab8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3400640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ab8004810 0x7f6ab8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3400640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ab8005220 0x7f6ab800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad406db90 con 0x7f6ab8003a10 2024-04-25T08:50:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab800c170 con 0x7f6ab8004810 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab800c370 con 0x7f6ab8005220 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad9000640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab8003a10 0x7f6ab8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3e00640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ab8004810 0x7f6ab8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad9a00640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ab8005220 0x7f6ab800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad9000640 1 -- 172.21.15.46:0/3229660006 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ab8005220 msgr2=0x7f6ab800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad9000640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ab8005220 0x7f6ab800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad9000640 1 -- 172.21.15.46:0/3229660006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ab8004810 msgr2=0x7f6ab8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad9000640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ab8004810 0x7f6ab8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad9000640 1 -- 172.21.15.46:0/3229660006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab8003530 con 0x7f6ab8003a10 2024-04-25T08:50:33.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad9a00640 1 -- 172.21.15.46:0/3229660006 reap_dead start 2024-04-25T08:50:33.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad9000640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab8003a10 0x7f6ab8003e30 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009f90 tx=0x7f6ac401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40590a782a6ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:33.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac40b2070 con 0x7f6ab8003a10 2024-04-25T08:50:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac40a5b10 con 0x7f6ab8003a10 2024-04-25T08:50:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.383+0000 7f6ad3400640 1 -- 172.21.15.46:0/3229660006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4009c10 con 0x7f6ab8003a10 2024-04-25T08:50:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 -- 172.21.15.46:0/3229660006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab8003a10 msgr2=0x7f6ab8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab8003a10 0x7f6ab8003e30 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009f90 tx=0x7f6ac401cc10 comp rx=0 tx=0).stop 2024-04-25T08:50:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 -- 172.21.15.46:0/3229660006 shutdown_connections 2024-04-25T08:50:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 --2- 172.21.15.46:0/3229660006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab8003a10 0x7f6ab8003e30 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 -- 172.21.15.46:0/3229660006 >> 172.21.15.46:0/3229660006 conn(0x7f6ad40b3120 msgr2=0x7f6ad40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 -- 172.21.15.46:0/3229660006 shutdown_connections 2024-04-25T08:50:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 -- 172.21.15.46:0/3229660006 wait complete. 2024-04-25T08:50:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 Processor -- start 2024-04-25T08:50:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 -- start start 2024-04-25T08:50:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ab8003a10 0x7f6ad40af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad40aed70 0x7f6ad41f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4161750 0x7f6ad4161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad415dd10 con 0x7f6ad4161750 2024-04-25T08:50:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad40cdb80 con 0x7f6ad40aed70 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6adba00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad415de90 con 0x7f6ab8003a10 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4161750 0x7f6ad4161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4161750 0x7f6ad4161bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38148/0 (socket says 172.21.15.46:38148) 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9a00640 1 -- 172.21.15.46:0/2827699650 learned_addr learned my addr 172.21.15.46:0/2827699650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9000640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ab8003a10 0x7f6ad40af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad3e00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad40aed70 0x7f6ad41f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9a00640 1 -- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ab8003a10 msgr2=0x7f6ad40af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9a00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ab8003a10 0x7f6ad40af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9a00640 1 -- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad40aed70 msgr2=0x7f6ad41f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9a00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad40aed70 0x7f6ad41f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9a00640 1 -- 172.21.15.46:0/2827699650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad40aff80 con 0x7f6ad4161750 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.387+0000 7f6ad9a00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4161750 0x7f6ad4161bd0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40c8680 tx=0x7f6acc00e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6ad0c00640 1 -- 172.21.15.46:0/2827699650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc019070 con 0x7f6ad4161750 2024-04-25T08:50:33.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6ad0c00640 1 -- 172.21.15.46:0/2827699650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc010510 con 0x7f6ad4161750 2024-04-25T08:50:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6ad0c00640 1 -- 172.21.15.46:0/2827699650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc010900 con 0x7f6ad4161750 2024-04-25T08:50:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad40b0260 con 0x7f6ad4161750 2024-04-25T08:50:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad40b0820 con 0x7f6ad4161750 2024-04-25T08:50:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad4069080 con 0x7f6ad4161750 2024-04-25T08:50:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6ad0c00640 1 -- 172.21.15.46:0/2827699650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6acc01c070 con 0x7f6ad4161750 2024-04-25T08:50:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6ad0c00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ab4066f60 0x7f6ab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6ad0c00640 1 -- 172.21.15.46:0/2827699650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f6acc084880 con 0x7f6ad4161750 2024-04-25T08:50:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.391+0000 7f6ad9000640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ab4066f60 0x7f6ab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.395+0000 7f6ad9000640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ab4066f60 0x7f6ab4069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40af000 tx=0x7f6ac40a7040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:33.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.395+0000 7f6ad0c00640 1 -- 172.21.15.46:0/2827699650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6acc054ef0 con 0x7f6ad4161750 2024-04-25T08:50:33.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ad406e3a0 con 0x7f6ad4161750 2024-04-25T08:50:33.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6ad0c00640 1 -- 172.21.15.46:0/2827699650 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f6acc058f90 con 0x7f6ad4161750 2024-04-25T08:50:33.582 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:33.582 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:49:54.058324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1358086076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1358086076},{"type":"v1","addr":"172.21.15.46:6835","nonce":1358086076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:50:33.582 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 75 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ab4066f60 msgr2=0x7f6ab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ab4066f60 0x7f6ab4069420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40af000 tx=0x7f6ac40a7040 comp rx=0 tx=0).stop 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4161750 msgr2=0x7f6ad4161bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4161750 0x7f6ad4161bd0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40c8680 tx=0x7f6acc00e880 comp rx=0 tx=0).stop 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 shutdown_connections 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6ab4066f60 0x7f6ab4069420 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ad4161750 0x7f6ad4161bd0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ad40aed70 0x7f6ad41f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 --2- 172.21.15.46:0/2827699650 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6ab8003a10 0x7f6ad40af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 >> 172.21.15.46:0/2827699650 conn(0x7f6ad40b3120 msgr2=0x7f6ad406f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 shutdown_connections 2024-04-25T08:50:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.591+0000 7f6adba00640 1 -- 172.21.15.46:0/2827699650 wait complete. 2024-04-25T08:50:33.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:33.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.751+0000 7ff41a200640 1 Processor -- start 2024-04-25T08:50:33.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.751+0000 7ff41a200640 1 -- start start 2024-04-25T08:50:33.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff41a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff41406f320 0x7ff414162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff41a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4140aed10 0x7ff414164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff41a200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4140b35b0 con 0x7ff41415a750 2024-04-25T08:50:33.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff41a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41415ab30 con 0x7ff41406f320 2024-04-25T08:50:33.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff41a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4140ce4f0 con 0x7ff4140aed10 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff413400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff41415a750 0x7ff41406ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57252/0 (socket says 172.21.15.46:57252) 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff413400640 1 -- 172.21.15.46:0/108675738 learned_addr learned my addr 172.21.15.46:0/108675738 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff40aa00640 1 --2- 172.21.15.46:0/108675738 >> v2:172.21.15.46:3300/0 conn(0x7ff41406f320 0x7ff414162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff413e00640 1 --2- 172.21.15.46:0/108675738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4140aed10 0x7ff414164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff412a00640 1 -- 172.21.15.46:0/108675738 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678931637 0 0) 0x7ff4140b35b0 con 0x7ff41415a750 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff412a00640 1 -- 172.21.15.46:0/108675738 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3f8003530 con 0x7ff41415a750 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff40aa00640 1 -- 172.21.15.46:0/108675738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4140aed10 msgr2=0x7ff414164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff40aa00640 1 --2- 172.21.15.46:0/108675738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4140aed10 0x7ff414164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff40aa00640 1 -- 172.21.15.46:0/108675738 >> v1:172.21.15.46:6789/0 conn(0x7ff41415a750 legacy=0x7ff41406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.755+0000 7ff40aa00640 1 -- 172.21.15.46:0/108675738 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41406d5e0 con 0x7ff41406f320 2024-04-25T08:50:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff40aa00640 1 --2- 172.21.15.46:0/108675738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41406f320 0x7ff414162200 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0097c0 tx=0x7ff3fc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c764f98b4ee5a032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff412a00640 1 -- 172.21.15.46:0/108675738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0b2070 con 0x7ff41406f320 2024-04-25T08:50:33.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff412a00640 1 -- 172.21.15.46:0/108675738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc0ad640 con 0x7ff41406f320 2024-04-25T08:50:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff412a00640 1 -- 172.21.15.46:0/108675738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0ad9f0 con 0x7ff41406f320 2024-04-25T08:50:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 -- 172.21.15.46:0/108675738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41406f320 msgr2=0x7ff414162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 --2- 172.21.15.46:0/108675738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41406f320 0x7ff414162200 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0097c0 tx=0x7ff3fc01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 -- 172.21.15.46:0/108675738 shutdown_connections 2024-04-25T08:50:33.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 --2- 172.21.15.46:0/108675738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4140aed10 0x7ff414164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 --2- 172.21.15.46:0/108675738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff41406f320 0x7ff414162200 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 -- 172.21.15.46:0/108675738 >> 172.21.15.46:0/108675738 conn(0x7ff4140b12a0 msgr2=0x7ff4140b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 -- 172.21.15.46:0/108675738 shutdown_connections 2024-04-25T08:50:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 -- 172.21.15.46:0/108675738 wait complete. 2024-04-25T08:50:33.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 Processor -- start 2024-04-25T08:50:33.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 -- start start 2024-04-25T08:50:33.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff41406f320 0x7ff4140a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4140aed10 0x7ff41415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff41415a750 0x7ff41413b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41406d270 con 0x7ff4140aed10 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4140cf200 con 0x7ff41406f320 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.759+0000 7ff41a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4141674b0 con 0x7ff41415a750 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4140aed10 0x7ff41415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4140aed10 0x7ff41415d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38164/0 (socket says 172.21.15.46:38164) 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40aa00640 1 -- 172.21.15.46:0/1009789329 learned_addr learned my addr 172.21.15.46:0/1009789329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff413e00640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff41415a750 0x7ff41413b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff413400640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff41406f320 0x7ff4140a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40aa00640 1 -- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff41415a750 msgr2=0x7ff41413b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40aa00640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff41415a750 0x7ff41413b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40aa00640 1 -- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff41406f320 msgr2=0x7ff4140a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40aa00640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff41406f320 0x7ff4140a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40aa00640 1 -- 172.21.15.46:0/1009789329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41414df00 con 0x7ff4140aed10 2024-04-25T08:50:33.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff413400640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff41406f320 0x7ff4140a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:33.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40aa00640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4140aed10 0x7ff41415d6a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0b0040 tx=0x7ff3fc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:33.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40be00640 1 -- 172.21.15.46:0/1009789329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0b2040 con 0x7ff4140aed10 2024-04-25T08:50:33.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40be00640 1 -- 172.21.15.46:0/1009789329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc0bc070 con 0x7ff4140aed10 2024-04-25T08:50:33.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40be00640 1 -- 172.21.15.46:0/1009789329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc006e20 con 0x7ff4140aed10 2024-04-25T08:50:33.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41413c520 con 0x7ff4140aed10 2024-04-25T08:50:33.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff414139630 con 0x7ff4140aed10 2024-04-25T08:50:33.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff414069080 con 0x7ff4140aed10 2024-04-25T08:50:33.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40be00640 1 -- 172.21.15.46:0/1009789329 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3fc0b6630 con 0x7ff4140aed10 2024-04-25T08:50:33.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40be00640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff3f0066f60 0x7ff3f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:33.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.763+0000 7ff40be00640 1 -- 172.21.15.46:0/1009789329 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7ff3fc11f230 con 0x7ff4140aed10 2024-04-25T08:50:33.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.767+0000 7ff413400640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff3f0066f60 0x7ff3f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:33.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.767+0000 7ff413400640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff3f0066f60 0x7ff3f0069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff404005890 tx=0x7ff404009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:33.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.767+0000 7ff40be00640 1 -- 172.21.15.46:0/1009789329 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3fc016090 con 0x7ff4140aed10 2024-04-25T08:50:33.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.931+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4140b32f0 con 0x7ff4140aed10 2024-04-25T08:50:33.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff40be00640 1 -- 172.21.15.46:0/1009789329 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17289 (secure 0 0 0) 0x7ff3fc0efef0 con 0x7ff4140aed10 2024-04-25T08:50:33.925 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:33.925 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:49:51.036629+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:48:26.132790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:48:26.714570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6834/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6835/1269610178":"2024-04-26T08:48:38.594941+0000","172.21.15.46:6834/1269610178":"2024-04-26T08:48:38.594941+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff3f0066f60 msgr2=0x7ff3f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff3f0066f60 0x7ff3f0069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff404005890 tx=0x7ff404009040 comp rx=0 tx=0).stop 2024-04-25T08:50:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4140aed10 msgr2=0x7ff41415d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4140aed10 0x7ff41415d6a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0b0040 tx=0x7ff3fc019240 comp rx=0 tx=0).stop 2024-04-25T08:50:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 shutdown_connections 2024-04-25T08:50:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff3f0066f60 0x7ff3f0069420 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff41415a750 0x7ff41413b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4140aed10 0x7ff41415d6a0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 --2- 172.21.15.46:0/1009789329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff41406f320 0x7ff4140a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 >> 172.21.15.46:0/1009789329 conn(0x7ff4140b12a0 msgr2=0x7ff4140b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 shutdown_connections 2024-04-25T08:50:33.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:33.935+0000 7ff41a200640 1 -- 172.21.15.46:0/1009789329 wait complete. 2024-04-25T08:50:33.946 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:50:33.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.095+0000 7fc437600640 1 Processor -- start 2024-04-25T08:50:34.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.095+0000 7fc437600640 1 -- start start 2024-04-25T08:50:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc437600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc43006f320 0x7fc430162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc437600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4300aed10 0x7fc430164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc437600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4300b35b0 con 0x7fc43015a750 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc437600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43015ab30 con 0x7fc4300aed10 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc437600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4300ce4f0 con 0x7fc43006f320 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc435600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4300aed10 0x7fc430164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc435600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4300aed10 0x7fc430164750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38168/0 (socket says 172.21.15.46:38168) 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc435600640 1 -- 172.21.15.46:0/907067745 learned_addr learned my addr 172.21.15.46:0/907067745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc42fe00640 1 --2- 172.21.15.46:0/907067745 >> v2:172.21.15.82:3300/0 conn(0x7fc43006f320 0x7fc430162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc435600640 1 -- 172.21.15.46:0/907067745 >> v2:172.21.15.82:3300/0 conn(0x7fc43006f320 msgr2=0x7fc430162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc435600640 1 --2- 172.21.15.46:0/907067745 >> v2:172.21.15.82:3300/0 conn(0x7fc43006f320 0x7fc430162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc435600640 1 -- 172.21.15.46:0/907067745 >> v1:172.21.15.82:6789/0 conn(0x7fc43015a750 legacy=0x7fc43006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.099+0000 7fc435600640 1 -- 172.21.15.46:0/907067745 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43006d5e0 con 0x7fc4300aed10 2024-04-25T08:50:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc435600640 1 --2- 172.21.15.46:0/907067745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 0x7fc430164750 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc4300cdca0 tx=0x7fc42801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3c03e11d5699e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc42f400640 1 -- 172.21.15.46:0/907067745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280a5760 con 0x7fc4300aed10 2024-04-25T08:50:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc42f400640 1 -- 172.21.15.46:0/907067745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4280a5900 con 0x7fc4300aed10 2024-04-25T08:50:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc42f400640 1 -- 172.21.15.46:0/907067745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280a5cd0 con 0x7fc4300aed10 2024-04-25T08:50:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 -- 172.21.15.46:0/907067745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 msgr2=0x7fc430164750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 --2- 172.21.15.46:0/907067745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 0x7fc430164750 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc4300cdca0 tx=0x7fc42801c9b0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 -- 172.21.15.46:0/907067745 shutdown_connections 2024-04-25T08:50:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 --2- 172.21.15.46:0/907067745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 0x7fc430164750 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 --2- 172.21.15.46:0/907067745 >> v2:172.21.15.82:3300/0 conn(0x7fc43006f320 0x7fc430162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 -- 172.21.15.46:0/907067745 >> 172.21.15.46:0/907067745 conn(0x7fc4300b12a0 msgr2=0x7fc4300b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 -- 172.21.15.46:0/907067745 shutdown_connections 2024-04-25T08:50:34.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 -- 172.21.15.46:0/907067745 wait complete. 2024-04-25T08:50:34.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 Processor -- start 2024-04-25T08:50:34.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.103+0000 7fc437600640 1 -- start start 2024-04-25T08:50:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc437600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc43006f320 0x7fc43014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc437600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 0x7fc43014f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc437600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc43015a750 0x7fc43014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc437600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43006d270 con 0x7fc4300aed10 2024-04-25T08:50:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc437600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4300cf200 con 0x7fc43015a750 2024-04-25T08:50:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc437600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4301674b0 con 0x7fc43006f320 2024-04-25T08:50:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 0x7fc43014f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 0x7fc43014f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38180/0 (socket says 172.21.15.46:38180) 2024-04-25T08:50:34.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42fe00640 1 -- 172.21.15.46:0/2581783431 learned_addr learned my addr 172.21.15.46:0/2581783431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc435600640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc43015a750 0x7fc43014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc434c00640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc43006f320 0x7fc43014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42fe00640 1 -- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc43006f320 msgr2=0x7fc43014dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42fe00640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc43006f320 0x7fc43014dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42fe00640 1 -- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc43015a750 msgr2=0x7fc43014fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42fe00640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc43015a750 0x7fc43014fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42fe00640 1 -- 172.21.15.46:0/2581783431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43015b9b0 con 0x7fc4300aed10 2024-04-25T08:50:34.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42fe00640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 0x7fc43014f6c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00d800 tx=0x7fc41c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42cc00640 1 -- 172.21.15.46:0/2581783431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00b6c0 con 0x7fc4300aed10 2024-04-25T08:50:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42cc00640 1 -- 172.21.15.46:0/2581783431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c019070 con 0x7fc4300aed10 2024-04-25T08:50:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42cc00640 1 -- 172.21.15.46:0/2581783431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c00ba60 con 0x7fc4300aed10 2024-04-25T08:50:34.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43015bc90 con 0x7fc4300aed10 2024-04-25T08:50:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43015c250 con 0x7fc4300aed10 2024-04-25T08:50:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc430069080 con 0x7fc4300aed10 2024-04-25T08:50:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.107+0000 7fc42cc00640 1 -- 172.21.15.46:0/2581783431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc41c00bc00 con 0x7fc4300aed10 2024-04-25T08:50:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.111+0000 7fc42cc00640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc410066f60 0x7fc410069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.111+0000 7fc42cc00640 1 -- 172.21.15.46:0/2581783431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fc41c0857c0 con 0x7fc4300aed10 2024-04-25T08:50:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.111+0000 7fc434c00640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc410066f60 0x7fc410069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.111+0000 7fc434c00640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc410066f60 0x7fc410069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc4301572d0 tx=0x7fc424008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.115+0000 7fc42cc00640 1 -- 172.21.15.46:0/2581783431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc41c056480 con 0x7fc4300aed10 2024-04-25T08:50:34.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc430157ab0 con 0x7fc4300aed10 2024-04-25T08:50:34.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc42cc00640 1 -- 172.21.15.46:0/2581783431 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fc41c055d80 con 0x7fc4300aed10 2024-04-25T08:50:34.301 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:34.301 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:49:54.058324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1358086076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1358086076},{"type":"v1","addr":"172.21.15.46:6835","nonce":1358086076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:50:34.301 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 75 2024-04-25T08:50:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc410066f60 msgr2=0x7fc410069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc410066f60 0x7fc410069420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc4301572d0 tx=0x7fc424008040 comp rx=0 tx=0).stop 2024-04-25T08:50:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 msgr2=0x7fc43014f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 0x7fc43014f6c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc41c00d800 tx=0x7fc41c00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 shutdown_connections 2024-04-25T08:50:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc410066f60 0x7fc410069420 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc43015a750 0x7fc43014fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4300aed10 0x7fc43014f6c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 --2- 172.21.15.46:0/2581783431 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc43006f320 0x7fc43014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 >> 172.21.15.46:0/2581783431 conn(0x7fc4300b12a0 msgr2=0x7fc4300ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 shutdown_connections 2024-04-25T08:50:34.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.311+0000 7fc437600640 1 -- 172.21.15.46:0/2581783431 wait complete. 2024-04-25T08:50:34.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:34.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.471+0000 7f9a35200640 1 Processor -- start 2024-04-25T08:50:34.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.471+0000 7f9a35200640 1 -- start start 2024-04-25T08:50:34.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.471+0000 7f9a35200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a30160f50 0x7f9a3015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.471+0000 7f9a35200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a300adc50 0x7f9a300ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.471+0000 7f9a35200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a3006ea00 0x7f9a3006ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.471+0000 7f9a35200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3006d680 con 0x7f9a30160f50 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.471+0000 7f9a35200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a30161330 con 0x7f9a300adc50 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.471+0000 7f9a35200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a300ae810 con 0x7f9a3006ea00 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a30160f50 0x7f9a3015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a30160f50 0x7f9a3015f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38196/0 (socket says 172.21.15.46:38196) 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2de00640 1 -- 172.21.15.46:0/1028613807 learned_addr learned my addr 172.21.15.46:0/1028613807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2e800640 1 --2- 172.21.15.46:0/1028613807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a3006ea00 0x7f9a3006ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a27e00640 1 --2- 172.21.15.46:0/1028613807 >> v2:172.21.15.82:3300/0 conn(0x7f9a300adc50 0x7f9a300ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2de00640 1 -- 172.21.15.46:0/1028613807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a3006ea00 msgr2=0x7f9a3006ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2de00640 1 --2- 172.21.15.46:0/1028613807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a3006ea00 0x7f9a3006ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2de00640 1 -- 172.21.15.46:0/1028613807 >> v2:172.21.15.82:3300/0 conn(0x7f9a300adc50 msgr2=0x7f9a300ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:34.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2de00640 1 --2- 172.21.15.46:0/1028613807 >> v2:172.21.15.82:3300/0 conn(0x7f9a300adc50 0x7f9a300ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2de00640 1 -- 172.21.15.46:0/1028613807 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a300aeaf0 con 0x7f9a30160f50 2024-04-25T08:50:34.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2de00640 1 --2- 172.21.15.46:0/1028613807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 0x7f9a3015f1a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9a30147800 tx=0x7f9a2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8346134ece5f8a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:34.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2d400640 1 -- 172.21.15.46:0/1028613807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a200b0070 con 0x7f9a30160f50 2024-04-25T08:50:34.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2d400640 1 -- 172.21.15.46:0/1028613807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a200ab600 con 0x7f9a30160f50 2024-04-25T08:50:34.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.475+0000 7f9a2d400640 1 -- 172.21.15.46:0/1028613807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a200ab9d0 con 0x7f9a30160f50 2024-04-25T08:50:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 -- 172.21.15.46:0/1028613807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 msgr2=0x7f9a3015f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 --2- 172.21.15.46:0/1028613807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 0x7f9a3015f1a0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9a30147800 tx=0x7f9a2001c520 comp rx=0 tx=0).stop 2024-04-25T08:50:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 -- 172.21.15.46:0/1028613807 shutdown_connections 2024-04-25T08:50:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 --2- 172.21.15.46:0/1028613807 >> v2:172.21.15.82:3300/0 conn(0x7f9a300adc50 0x7f9a300ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 --2- 172.21.15.46:0/1028613807 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a3006ea00 0x7f9a3006ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 --2- 172.21.15.46:0/1028613807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 0x7f9a3015f1a0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 -- 172.21.15.46:0/1028613807 >> 172.21.15.46:0/1028613807 conn(0x7f9a300b12a0 msgr2=0x7f9a300b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 -- 172.21.15.46:0/1028613807 shutdown_connections 2024-04-25T08:50:34.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 -- 172.21.15.46:0/1028613807 wait complete. 2024-04-25T08:50:34.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 Processor -- start 2024-04-25T08:50:34.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 -- start start 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3006ea00 0x7f9a30152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a300adc50 0x7f9a30148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 0x7f9a30155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3006e4a0 con 0x7f9a30160f50 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3006e320 con 0x7f9a3006ea00 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a35200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3006e620 con 0x7f9a300adc50 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a2e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 0x7f9a30155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a2e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 0x7f9a30155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38204/0 (socket says 172.21.15.46:38204) 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a2e800640 1 -- 172.21.15.46:0/3690267885 learned_addr learned my addr 172.21.15.46:0/3690267885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a2de00640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3006ea00 0x7f9a30152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a27e00640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a300adc50 0x7f9a30148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a2e800640 1 -- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a300adc50 msgr2=0x7f9a30148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a2e800640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a300adc50 0x7f9a30148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a2e800640 1 -- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3006ea00 msgr2=0x7f9a30152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a2e800640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3006ea00 0x7f9a30152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.479+0000 7f9a2e800640 1 -- 172.21.15.46:0/3690267885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a301492b0 con 0x7f9a30160f50 2024-04-25T08:50:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a2e800640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 0x7f9a30155050 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9a28007d20 tx=0x7f9a2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a26000640 1 -- 172.21.15.46:0/3690267885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28014420 con 0x7f9a30160f50 2024-04-25T08:50:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a26000640 1 -- 172.21.15.46:0/3690267885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28019070 con 0x7f9a30160f50 2024-04-25T08:50:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a26000640 1 -- 172.21.15.46:0/3690267885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28014820 con 0x7f9a30160f50 2024-04-25T08:50:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a300cc170 con 0x7f9a30160f50 2024-04-25T08:50:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a300cc730 con 0x7f9a30160f50 2024-04-25T08:50:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a30069080 con 0x7f9a30160f50 2024-04-25T08:50:34.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a26000640 1 -- 172.21.15.46:0/3690267885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a280149c0 con 0x7f9a30160f50 2024-04-25T08:50:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a26000640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a0c066f60 0x7f9a0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a26000640 1 -- 172.21.15.46:0/3690267885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f9a28085960 con 0x7f9a30160f50 2024-04-25T08:50:34.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.483+0000 7f9a2de00640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a0c066f60 0x7f9a0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.487+0000 7f9a2de00640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a0c066f60 0x7f9a0c069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9a200ad000 tx=0x7f9a200a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.487+0000 7f9a26000640 1 -- 172.21.15.46:0/3690267885 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a301492b0 con 0x7f9a30160f50 2024-04-25T08:50:34.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.683+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a30073750 con 0x7f9a30160f50 2024-04-25T08:50:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.683+0000 7f9a26000640 1 -- 172.21.15.46:0/3690267885 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f9a28056060 con 0x7f9a30160f50 2024-04-25T08:50:34.676 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:34.676 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:48:27.728697+0000","modified":"2024-04-25T08:49:54.058324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7962},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7962":{"gid":7962,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1358086076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1358086076},{"type":"v1","addr":"172.21.15.46:6835","nonce":1358086076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:50:34.676 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 75 2024-04-25T08:50:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a0c066f60 msgr2=0x7f9a0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a0c066f60 0x7f9a0c069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9a200ad000 tx=0x7f9a200a6000 comp rx=0 tx=0).stop 2024-04-25T08:50:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 msgr2=0x7f9a30155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 0x7f9a30155050 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9a28007d20 tx=0x7f9a2800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 shutdown_connections 2024-04-25T08:50:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a0c066f60 0x7f9a0c069420 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a30160f50 0x7f9a30155050 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a300adc50 0x7f9a30148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 --2- 172.21.15.46:0/3690267885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3006ea00 0x7f9a30152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 >> 172.21.15.46:0/3690267885 conn(0x7f9a300b12a0 msgr2=0x7f9a300b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 shutdown_connections 2024-04-25T08:50:34.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.687+0000 7f9a35200640 1 -- 172.21.15.46:0/3690267885 wait complete. 2024-04-25T08:50:34.698 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:34.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d35400640 1 Processor -- start 2024-04-25T08:50:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d35400640 1 -- start start 2024-04-25T08:50:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d35400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9d3015ef20 0x7f9d300cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d35400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d30099a00 0x7f9d30099eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d35400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3015f300 con 0x7f9d300cd6d0 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d35400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d300cddb0 con 0x7f9d3015ef20 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d35400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300cc3d0 con 0x7f9d30099a00 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d2e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9d3015ef20 0x7f9d300cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d2e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9d3015ef20 0x7f9d300cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38208/0 (socket says 172.21.15.46:38208) 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.847+0000 7f9d2e800640 1 -- 172.21.15.46:0/434247275 learned_addr learned my addr 172.21.15.46:0/434247275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.851+0000 7f9d2f200640 1 --2- 172.21.15.46:0/434247275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d30099a00 0x7f9d30099eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.851+0000 7f9d2e800640 1 -- 172.21.15.46:0/434247275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d30099a00 msgr2=0x7f9d30099eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.851+0000 7f9d2e800640 1 --2- 172.21.15.46:0/434247275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d30099a00 0x7f9d30099eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.851+0000 7f9d2e800640 1 -- 172.21.15.46:0/434247275 >> v1:172.21.15.82:6789/0 conn(0x7f9d300cd6d0 legacy=0x7f9d3009a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.851+0000 7f9d2e800640 1 -- 172.21.15.46:0/434247275 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d300cc6b0 con 0x7f9d3015ef20 2024-04-25T08:50:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.851+0000 7f9d2e800640 1 --2- 172.21.15.46:0/434247275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3015ef20 0x7f9d300cd190 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009870 tx=0x7f9d2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94015df6fb7cf1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.851+0000 7f9d2de00640 1 -- 172.21.15.46:0/434247275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200b1070 con 0x7f9d3015ef20 2024-04-25T08:50:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.851+0000 7f9d2de00640 1 -- 172.21.15.46:0/434247275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d200ac630 con 0x7f9d3015ef20 2024-04-25T08:50:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.851+0000 7f9d2de00640 1 -- 172.21.15.46:0/434247275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200a8050 con 0x7f9d3015ef20 2024-04-25T08:50:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- 172.21.15.46:0/434247275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3015ef20 msgr2=0x7f9d300cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 --2- 172.21.15.46:0/434247275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3015ef20 0x7f9d300cd190 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009870 tx=0x7f9d2001c520 comp rx=0 tx=0).stop 2024-04-25T08:50:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- 172.21.15.46:0/434247275 shutdown_connections 2024-04-25T08:50:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 --2- 172.21.15.46:0/434247275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d30099a00 0x7f9d30099eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 --2- 172.21.15.46:0/434247275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d3015ef20 0x7f9d300cd190 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- 172.21.15.46:0/434247275 >> 172.21.15.46:0/434247275 conn(0x7f9d300b3120 msgr2=0x7f9d300b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- 172.21.15.46:0/434247275 shutdown_connections 2024-04-25T08:50:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- 172.21.15.46:0/434247275 wait complete. 2024-04-25T08:50:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 Processor -- start 2024-04-25T08:50:34.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- start start 2024-04-25T08:50:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d30099a00 0x7f9d300b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3015ef20 0x7f9d30158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d3009d880 0x7f9d3009dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3009aea0 con 0x7f9d30099a00 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3009ad20 con 0x7f9d3015ef20 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30168880 con 0x7f9d3009d880 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d30099a00 0x7f9d300b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d30099a00 0x7f9d300b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38212/0 (socket says 172.21.15.46:38212) 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2e800640 1 -- 172.21.15.46:0/645259113 learned_addr learned my addr 172.21.15.46:0/645259113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2f200640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d3009d880 0x7f9d3009dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d25e00640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3015ef20 0x7f9d30158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2e800640 1 -- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d3009d880 msgr2=0x7f9d3009dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2e800640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d3009d880 0x7f9d3009dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2e800640 1 -- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3015ef20 msgr2=0x7f9d30158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2e800640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3015ef20 0x7f9d30158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2e800640 1 -- 172.21.15.46:0/645259113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d300b50f0 con 0x7f9d30099a00 2024-04-25T08:50:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d2e800640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d30099a00 0x7f9d300b4aa0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9d200af040 tx=0x7f9d200192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:34.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d27400640 1 -- 172.21.15.46:0/645259113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200b1040 con 0x7f9d30099a00 2024-04-25T08:50:34.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d27400640 1 -- 172.21.15.46:0/645259113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d200ba070 con 0x7f9d30099a00 2024-04-25T08:50:34.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d27400640 1 -- 172.21.15.46:0/645259113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d200a8040 con 0x7f9d30099a00 2024-04-25T08:50:34.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.855+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d300ca200 con 0x7f9d30099a00 2024-04-25T08:50:34.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.859+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d300ca7c0 con 0x7f9d30099a00 2024-04-25T08:50:34.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.859+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d30069080 con 0x7f9d30099a00 2024-04-25T08:50:34.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.859+0000 7f9d27400640 1 -- 172.21.15.46:0/645259113 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d200b5680 con 0x7f9d30099a00 2024-04-25T08:50:34.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.859+0000 7f9d27400640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d0c066f60 0x7f9d0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:34.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.859+0000 7f9d27400640 1 -- 172.21.15.46:0/645259113 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f9d2011f0e0 con 0x7f9d30099a00 2024-04-25T08:50:34.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.859+0000 7f9d25e00640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d0c066f60 0x7f9d0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:34.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.859+0000 7f9d25e00640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d0c066f60 0x7f9d0c069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9d301f7ed0 tx=0x7f9d180094d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:34.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:34.863+0000 7f9d27400640 1 -- 172.21.15.46:0/645259113 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d20016110 con 0x7f9d30099a00 2024-04-25T08:50:35.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d300d0700 con 0x7f9d30099a00 2024-04-25T08:50:35.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d27400640 1 -- 172.21.15.46:0/645259113 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17289 (secure 0 0 0) 0x7f9d200efda0 con 0x7f9d30099a00 2024-04-25T08:50:35.018 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:35.018 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:49:51.036629+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:48:26.132790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:48:26.714570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.a","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":5}]},{"pgid":"14.29","mappings":[{"from":1,"to":7}]},{"pgid":"14.2d","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6834/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6835/1269610178":"2024-04-26T08:48:38.594941+0000","172.21.15.46:6834/1269610178":"2024-04-26T08:48:38.594941+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d0c066f60 msgr2=0x7f9d0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d0c066f60 0x7f9d0c069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9d301f7ed0 tx=0x7f9d180094d0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d30099a00 msgr2=0x7f9d300b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d30099a00 0x7f9d300b4aa0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9d200af040 tx=0x7f9d200192c0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 shutdown_connections 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9d0c066f60 0x7f9d0c069420 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9d3009d880 0x7f9d3009dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d3015ef20 0x7f9d30158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 --2- 172.21.15.46:0/645259113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d30099a00 0x7f9d300b4aa0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 >> 172.21.15.46:0/645259113 conn(0x7f9d300b3120 msgr2=0x7f9d3015aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:35.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 shutdown_connections 2024-04-25T08:50:35.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.027+0000 7f9d35400640 1 -- 172.21.15.46:0/645259113 wait complete. 2024-04-25T08:50:35.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:50:35.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.191+0000 7f5faf000640 1 Processor -- start 2024-04-25T08:50:35.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.191+0000 7f5faf000640 1 -- start start 2024-04-25T08:50:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.191+0000 7f5faf000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa8099a00 0x7f5fa8099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.191+0000 7f5faf000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5fa80cffc0 0x7f5fa806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5faf000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa809a320 con 0x7f5fa8070390 2024-04-25T08:50:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5faf000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa80d07e0 con 0x7f5fa80cffc0 2024-04-25T08:50:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5faf000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80d0a30 con 0x7f5fa8099a00 2024-04-25T08:50:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa8099a00 0x7f5fa8099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5f9f400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5fa80cffc0 0x7f5fa806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5fa7e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa8099a00 0x7f5fa8099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36238/0 (socket says 172.21.15.46:36238) 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5fa7e00640 1 -- 172.21.15.46:0/227722433 learned_addr learned my addr 172.21.15.46:0/227722433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5fa7e00640 1 -- 172.21.15.46:0/227722433 >> v2:172.21.15.82:3300/0 conn(0x7f5fa80cffc0 msgr2=0x7f5fa806fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5fa7e00640 1 --2- 172.21.15.46:0/227722433 >> v2:172.21.15.82:3300/0 conn(0x7f5fa80cffc0 0x7f5fa806fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5fa7e00640 1 -- 172.21.15.46:0/227722433 >> v1:172.21.15.82:6789/0 conn(0x7f5fa8070390 legacy=0x7f5fa80cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5fa7e00640 1 -- 172.21.15.46:0/227722433 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80d0d10 con 0x7f5fa8099a00 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.195+0000 7f5fa7e00640 1 --2- 172.21.15.46:0/227722433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa8099a00 0x7f5fa8099de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009e50 tx=0x7f5f9801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90df19230a5c8d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5fa7400640 1 -- 172.21.15.46:0/227722433 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f980a5810 con 0x7f5fa8099a00 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5fa7400640 1 -- 172.21.15.46:0/227722433 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f980a59b0 con 0x7f5fa8099a00 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5fa7400640 1 -- 172.21.15.46:0/227722433 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f980a5d80 con 0x7f5fa8099a00 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 -- 172.21.15.46:0/227722433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa8099a00 msgr2=0x7f5fa8099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 --2- 172.21.15.46:0/227722433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa8099a00 0x7f5fa8099de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5f98009e50 tx=0x7f5f9801ca30 comp rx=0 tx=0).stop 2024-04-25T08:50:35.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 -- 172.21.15.46:0/227722433 shutdown_connections 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 --2- 172.21.15.46:0/227722433 >> v2:172.21.15.82:3300/0 conn(0x7f5fa80cffc0 0x7f5fa806fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 --2- 172.21.15.46:0/227722433 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa8099a00 0x7f5fa8099de0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 -- 172.21.15.46:0/227722433 >> 172.21.15.46:0/227722433 conn(0x7f5fa80d16b0 msgr2=0x7f5fa80d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 -- 172.21.15.46:0/227722433 shutdown_connections 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 -- 172.21.15.46:0/227722433 wait complete. 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 Processor -- start 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 -- start start 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa8070390 0x7f5fa8145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fa8099a00 0x7f5fa806cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa80cffc0 0x7f5fa80b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80b0f60 con 0x7f5fa8099a00 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80b0de0 con 0x7f5fa8070390 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5faf000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80d03a0 con 0x7f5fa80cffc0 2024-04-25T08:50:35.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5f9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fa8099a00 0x7f5fa806cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5f9f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fa8099a00 0x7f5fa806cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38216/0 (socket says 172.21.15.46:38216) 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5f9f400640 1 -- 172.21.15.46:0/3436432794 learned_addr learned my addr 172.21.15.46:0/3436432794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5fa7e00640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa8070390 0x7f5fa8145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5fad000640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa80cffc0 0x7f5fa80b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5f9f400640 1 -- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa80cffc0 msgr2=0x7f5fa80b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5f9f400640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa80cffc0 0x7f5fa80b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5f9f400640 1 -- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa8070390 msgr2=0x7f5fa8145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5f9f400640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa8070390 0x7f5fa8145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5f9f400640 1 -- 172.21.15.46:0/3436432794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa81580a0 con 0x7f5fa8099a00 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.199+0000 7f5fa7e00640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa8070390 0x7f5fa8145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5f9f400640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fa8099a00 0x7f5fa806cc20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000e870 tx=0x7f5f9000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5fa4c00640 1 -- 172.21.15.46:0/3436432794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f90017070 con 0x7f5fa8099a00 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5fa4c00640 1 -- 172.21.15.46:0/3436432794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9000c940 con 0x7f5fa8099a00 2024-04-25T08:50:35.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5fa4c00640 1 -- 172.21.15.46:0/3436432794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9000ccf0 con 0x7f5fa8099a00 2024-04-25T08:50:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa8158380 con 0x7f5fa8099a00 2024-04-25T08:50:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa806d160 con 0x7f5fa8099a00 2024-04-25T08:50:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa8069080 con 0x7f5fa8099a00 2024-04-25T08:50:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5fa4c00640 1 -- 172.21.15.46:0/3436432794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f9001a070 con 0x7f5fa8099a00 2024-04-25T08:50:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5fa4c00640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f84066f60 0x7f5f84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5fa4c00640 1 -- 172.21.15.46:0/3436432794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5f90085620 con 0x7f5fa8099a00 2024-04-25T08:50:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.203+0000 7f5fa7e00640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f84066f60 0x7f5f84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.207+0000 7f5fa7e00640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f84066f60 0x7f5f84069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5f980ae000 tx=0x7f5f980a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:35.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.207+0000 7f5fa4c00640 1 -- 172.21.15.46:0/3436432794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f900562f0 con 0x7f5fa8099a00 2024-04-25T08:50:35.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.407+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5fa8070f80 con 0x7f5fa8099a00 2024-04-25T08:50:35.400 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:50:35.411+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:50:35.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.411+0000 7f5fa4c00640 1 -- 172.21.15.46:0/3436432794 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5f90055bf0 con 0x7f5fa8099a00 2024-04-25T08:50:35.403 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:50:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f84066f60 msgr2=0x7f5f84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f84066f60 0x7f5f84069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5f980ae000 tx=0x7f5f980a7070 comp rx=0 tx=0).stop 2024-04-25T08:50:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fa8099a00 msgr2=0x7f5fa806cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fa8099a00 0x7f5fa806cc20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5f9000e870 tx=0x7f5f9000ec70 comp rx=0 tx=0).stop 2024-04-25T08:50:35.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 shutdown_connections 2024-04-25T08:50:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5f84066f60 0x7f5f84069420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5fa80cffc0 0x7f5fa80b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fa8099a00 0x7f5fa806cc20 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 --2- 172.21.15.46:0/3436432794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa8070390 0x7f5fa8145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 >> 172.21.15.46:0/3436432794 conn(0x7f5fa80d16b0 msgr2=0x7f5fa80cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 shutdown_connections 2024-04-25T08:50:35.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.415+0000 7f5faf000640 1 -- 172.21.15.46:0/3436432794 wait complete. 2024-04-25T08:50:35.414 INFO:tasks.ceph.mds.c.smithi046.stderr: -1948> 2024-04-25T08:49:57.903+0000 7fe460200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f082812100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:50:35.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:50:35.458 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:50:35.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.579+0000 7f1fd4600640 1 Processor -- start 2024-04-25T08:50:35.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.579+0000 7f1fd4600640 1 -- start start 2024-04-25T08:50:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd4600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1fcc143990 0x7f1fcc0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd4600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1fcc1440d0 0x7f1fcc06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd4600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc06ef80 0x7f1fcc0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd4600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc15ed30 con 0x7f1fcc143990 2024-04-25T08:50:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd4600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc15e440 con 0x7f1fcc1440d0 2024-04-25T08:50:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd4600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15e6e0 con 0x7f1fcc06ef80 2024-04-25T08:50:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1fcc143990 0x7f1fcc0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1fcc143990 0x7f1fcc0cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38242/0 (socket says 172.21.15.46:38242) 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1c00640 1 -- 172.21.15.46:0/2417941491 learned_addr learned my addr 172.21.15.46:0/2417941491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1c00640 1 -- 172.21.15.46:0/2417941491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc06ef80 msgr2=0x7f1fcc0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1200640 1 --2- 172.21.15.46:0/2417941491 >> v2:172.21.15.82:3300/0 conn(0x7f1fcc1440d0 0x7f1fcc06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd2600640 1 --2- 172.21.15.46:0/2417941491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc06ef80 0x7f1fcc0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1c00640 1 --2- 172.21.15.46:0/2417941491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc06ef80 0x7f1fcc0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1c00640 1 -- 172.21.15.46:0/2417941491 >> v2:172.21.15.82:3300/0 conn(0x7f1fcc1440d0 msgr2=0x7f1fcc06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1c00640 1 --2- 172.21.15.46:0/2417941491 >> v2:172.21.15.82:3300/0 conn(0x7f1fcc1440d0 0x7f1fcc06ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1c00640 1 -- 172.21.15.46:0/2417941491 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc15e9c0 con 0x7f1fcc143990 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd1200640 1 --2- 172.21.15.46:0/2417941491 >> v2:172.21.15.82:3300/0 conn(0x7f1fcc1440d0 0x7f1fcc06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:35.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.583+0000 7f1fd2600640 1 --2- 172.21.15.46:0/2417941491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc06ef80 0x7f1fcc0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd1c00640 1 --2- 172.21.15.46:0/2417941491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc143990 0x7f1fcc0cecc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc070c80 tx=0x7f1fc001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da80102cb159669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fcbe00640 1 -- 172.21.15.46:0/2417941491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00b0070 con 0x7f1fcc143990 2024-04-25T08:50:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fcbe00640 1 -- 172.21.15.46:0/2417941491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc00ab600 con 0x7f1fcc143990 2024-04-25T08:50:35.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fcbe00640 1 -- 172.21.15.46:0/2417941491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00ab9d0 con 0x7f1fcc143990 2024-04-25T08:50:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 -- 172.21.15.46:0/2417941491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc143990 msgr2=0x7f1fcc0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 --2- 172.21.15.46:0/2417941491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc143990 0x7f1fcc0cecc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc070c80 tx=0x7f1fc001c260 comp rx=0 tx=0).stop 2024-04-25T08:50:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 -- 172.21.15.46:0/2417941491 shutdown_connections 2024-04-25T08:50:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 --2- 172.21.15.46:0/2417941491 >> v2:172.21.15.82:3300/0 conn(0x7f1fcc1440d0 0x7f1fcc06ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 --2- 172.21.15.46:0/2417941491 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc06ef80 0x7f1fcc0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 --2- 172.21.15.46:0/2417941491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc143990 0x7f1fcc0cecc0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 -- 172.21.15.46:0/2417941491 >> 172.21.15.46:0/2417941491 conn(0x7f1fcc06ac10 msgr2=0x7f1fcc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 -- 172.21.15.46:0/2417941491 shutdown_connections 2024-04-25T08:50:35.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 -- 172.21.15.46:0/2417941491 wait complete. 2024-04-25T08:50:35.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 Processor -- start 2024-04-25T08:50:35.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.587+0000 7f1fd4600640 1 -- start start 2024-04-25T08:50:35.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd4600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fcc06ef80 0x7f1fcc13ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd4600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc143990 0x7f1fcc137170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd4600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc1440d0 0x7f1fcc15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd4600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc143ef0 con 0x7f1fcc1440d0 2024-04-25T08:50:35.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd4600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc143d70 con 0x7f1fcc06ef80 2024-04-25T08:50:35.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd4600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc071270 con 0x7f1fcc143990 2024-04-25T08:50:35.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc1440d0 0x7f1fcc15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc1440d0 0x7f1fcc15d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38244/0 (socket says 172.21.15.46:38244) 2024-04-25T08:50:35.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd2600640 1 -- 172.21.15.46:0/619613831 learned_addr learned my addr 172.21.15.46:0/619613831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:35.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd1c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fcc06ef80 0x7f1fcc13ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd1200640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc143990 0x7f1fcc137170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd2600640 1 -- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc143990 msgr2=0x7f1fcc137170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd2600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc143990 0x7f1fcc137170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd2600640 1 -- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fcc06ef80 msgr2=0x7f1fcc13ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd2600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fcc06ef80 0x7f1fcc13ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd2600640 1 -- 172.21.15.46:0/619613831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc13d210 con 0x7f1fcc1440d0 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd2600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc1440d0 0x7f1fcc15d0a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400cef0 tx=0x7f1fc400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fc9600640 1 -- 172.21.15.46:0/619613831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc401a070 con 0x7f1fcc1440d0 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fc9600640 1 -- 172.21.15.46:0/619613831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc400d860 con 0x7f1fcc1440d0 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fc9600640 1 -- 172.21.15.46:0/619613831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400dc50 con 0x7f1fcc1440d0 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc13bc30 con 0x7f1fcc1440d0 2024-04-25T08:50:35.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc13c1f0 con 0x7f1fcc1440d0 2024-04-25T08:50:35.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.591+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc13c9c0 con 0x7f1fcc1440d0 2024-04-25T08:50:35.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.595+0000 7f1fc9600640 1 -- 172.21.15.46:0/619613831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1fc401d070 con 0x7f1fcc1440d0 2024-04-25T08:50:35.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.595+0000 7f1fc9600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1fac066f60 0x7f1fac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:35.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.595+0000 7f1fc9600640 1 -- 172.21.15.46:0/619613831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f1fc4085b90 con 0x7f1fcc1440d0 2024-04-25T08:50:35.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.595+0000 7f1fd1c00640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1fac066f60 0x7f1fac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:35.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.595+0000 7f1fd1c00640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1fac066f60 0x7f1fac069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00ad000 tx=0x7f1fc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.599+0000 7f1fc9600640 1 -- 172.21.15.46:0/619613831 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1fcc13d210 con 0x7f1fcc1440d0 2024-04-25T08:50:35.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:35.795+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1fcc0b4750 con 0x7f1fcc1440d0 2024-04-25T08:50:36.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.415+0000 7f1fc9600640 1 -- 172.21.15.46:0/619613831 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1fc4056270 con 0x7f1fcc1440d0 2024-04-25T08:50:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1fac066f60 msgr2=0x7f1fac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1fac066f60 0x7f1fac069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00ad000 tx=0x7f1fc0008040 comp rx=0 tx=0).stop 2024-04-25T08:50:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc1440d0 msgr2=0x7f1fcc15d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc1440d0 0x7f1fcc15d0a0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400cef0 tx=0x7f1fc400a710 comp rx=0 tx=0).stop 2024-04-25T08:50:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 shutdown_connections 2024-04-25T08:50:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1fac066f60 0x7f1fac069420 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fcc1440d0 0x7f1fcc15d0a0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1fcc143990 0x7f1fcc137170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 --2- 172.21.15.46:0/619613831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fcc06ef80 0x7f1fcc13ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 >> 172.21.15.46:0/619613831 conn(0x7f1fcc06ac10 msgr2=0x7f1fcc130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 shutdown_connections 2024-04-25T08:50:36.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.419+0000 7f1fd4600640 1 -- 172.21.15.46:0/619613831 wait complete. 2024-04-25T08:50:36.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:50:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad1d600640 1 Processor -- start 2024-04-25T08:50:36.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad1d600640 1 -- start start 2024-04-25T08:50:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad1d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad1815dce0 0x7fad180ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad1d600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fad18072600 0x7fad180ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad1d600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1806ee30 con 0x7fad1806bc20 2024-04-25T08:50:36.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad1d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1806f110 con 0x7fad1815dce0 2024-04-25T08:50:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad1d600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1806f3f0 con 0x7fad18072600 2024-04-25T08:50:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad0fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad1815dce0 0x7fad180ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad0fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad1815dce0 0x7fad180ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38246/0 (socket says 172.21.15.46:38246) 2024-04-25T08:50:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad0fe00640 1 -- 172.21.15.46:0/3089404698 learned_addr learned my addr 172.21.15.46:0/3089404698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad17400640 1 --2- 172.21.15.46:0/3089404698 >> v2:172.21.15.82:3300/0 conn(0x7fad18072600 0x7fad180ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad0fe00640 1 -- 172.21.15.46:0/3089404698 >> v2:172.21.15.82:3300/0 conn(0x7fad18072600 msgr2=0x7fad180ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad0fe00640 1 --2- 172.21.15.46:0/3089404698 >> v2:172.21.15.82:3300/0 conn(0x7fad18072600 0x7fad180ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad0fe00640 1 -- 172.21.15.46:0/3089404698 >> v1:172.21.15.82:6789/0 conn(0x7fad1806bc20 legacy=0x7fad1806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:36.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.631+0000 7fad0fe00640 1 -- 172.21.15.46:0/3089404698 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1806f6d0 con 0x7fad1815dce0 2024-04-25T08:50:36.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad0fe00640 1 --2- 172.21.15.46:0/3089404698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1815dce0 0x7fad180ad190 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fad180671e0 tx=0x7fad0001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eaba638f3adfb0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad16000640 1 -- 172.21.15.46:0/3089404698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000a7070 con 0x7fad1815dce0 2024-04-25T08:50:36.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad16000640 1 -- 172.21.15.46:0/3089404698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad000b0070 con 0x7fad1815dce0 2024-04-25T08:50:36.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad16000640 1 -- 172.21.15.46:0/3089404698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000ab890 con 0x7fad1815dce0 2024-04-25T08:50:36.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad1d600640 1 -- 172.21.15.46:0/3089404698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1815dce0 msgr2=0x7fad180ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:36.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad1d600640 1 --2- 172.21.15.46:0/3089404698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1815dce0 0x7fad180ad190 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fad180671e0 tx=0x7fad0001c4b0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad1d600640 1 -- 172.21.15.46:0/3089404698 shutdown_connections 2024-04-25T08:50:36.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad1d600640 1 --2- 172.21.15.46:0/3089404698 >> v2:172.21.15.82:3300/0 conn(0x7fad18072600 0x7fad180ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad1d600640 1 --2- 172.21.15.46:0/3089404698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad1815dce0 0x7fad180ad190 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad1d600640 1 -- 172.21.15.46:0/3089404698 >> 172.21.15.46:0/3089404698 conn(0x7fad18071ae0 msgr2=0x7fad18071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:36.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad1d600640 1 -- 172.21.15.46:0/3089404698 shutdown_connections 2024-04-25T08:50:36.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad1d600640 1 -- 172.21.15.46:0/3089404698 wait complete. 2024-04-25T08:50:36.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.635+0000 7fad1d600640 1 Processor -- start 2024-04-25T08:50:36.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 -- start start 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad1806bc20 0x7fad18144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad18072600 0x7fad181431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1815dce0 0x7fad18140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1816a630 con 0x7fad18072600 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad180b3d20 con 0x7fad1815dce0 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1816a7b0 con 0x7fad1806bc20 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad18072600 0x7fad181431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad18072600 0x7fad181431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38260/0 (socket says 172.21.15.46:38260) 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0fe00640 1 -- 172.21.15.46:0/4153023075 learned_addr learned my addr 172.21.15.46:0/4153023075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:36.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad17400640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1815dce0 0x7fad18140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad16a00640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad1806bc20 0x7fad18144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0fe00640 1 -- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad1806bc20 msgr2=0x7fad18144a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0fe00640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad1806bc20 0x7fad18144a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0fe00640 1 -- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1815dce0 msgr2=0x7fad18140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0fe00640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1815dce0 0x7fad18140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0fe00640 1 -- 172.21.15.46:0/4153023075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad18143910 con 0x7fad18072600 2024-04-25T08:50:36.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0fe00640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad18072600 0x7fad181431a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fad0001ab60 tx=0x7fad0001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:36.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0ea00640 1 -- 172.21.15.46:0/4153023075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000be5f0 con 0x7fad18072600 2024-04-25T08:50:36.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0ea00640 1 -- 172.21.15.46:0/4153023075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad000be7b0 con 0x7fad18072600 2024-04-25T08:50:36.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad0ea00640 1 -- 172.21.15.46:0/4153023075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000beba0 con 0x7fad18072600 2024-04-25T08:50:36.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad18143bf0 con 0x7fad18072600 2024-04-25T08:50:36.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad181416e0 con 0x7fad18072600 2024-04-25T08:50:36.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.639+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad18067820 con 0x7fad18072600 2024-04-25T08:50:36.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.643+0000 7fad0ea00640 1 -- 172.21.15.46:0/4153023075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad000b43f0 con 0x7fad18072600 2024-04-25T08:50:36.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.643+0000 7fad0ea00640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:36.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.643+0000 7fad0ea00640 1 -- 172.21.15.46:0/4153023075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6475+0+0 (secure 0 0 0) 0x7fad0011f670 con 0x7fad18072600 2024-04-25T08:50:36.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.643+0000 7fad16a00640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:36.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.643+0000 7fad16a00640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fad18143110 tx=0x7fad08007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:36.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.647+0000 7fad0ea00640 1 -- 172.21.15.46:0/4153023075 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad000160d0 con 0x7fad18072600 2024-04-25T08:50:36.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:36.811+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad18144110 con 0x7fad18072600 2024-04-25T08:50:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.415+0000 7fad0ea00640 1 -- 172.21.15.46:0/4153023075 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 181+0+0 (secure 0 0 0) 0x7fad18144110 con 0x7fad18072600 2024-04-25T08:50:37.407 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:50:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.419+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facf4066f60 msgr2=0x7facf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:37.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.419+0000 7fad1d600640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facf4066f60 0x7facf4069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fad18143110 tx=0x7fad08007e40 comp rx=0 tx=0).stop 2024-04-25T08:50:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.419+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad18072600 msgr2=0x7fad181431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.419+0000 7fad1d600640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad18072600 0x7fad181431a0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fad0001ab60 tx=0x7fad0001acf0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.423+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 shutdown_connections 2024-04-25T08:50:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.423+0000 7fad1d600640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7facf4066f60 0x7facf4069420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.423+0000 7fad1d600640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1815dce0 0x7fad18140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.423+0000 7fad1d600640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad18072600 0x7fad181431a0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.423+0000 7fad1d600640 1 --2- 172.21.15.46:0/4153023075 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fad1806bc20 0x7fad18144a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.423+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 >> 172.21.15.46:0/4153023075 conn(0x7fad18071ae0 msgr2=0x7fad180aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.423+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 shutdown_connections 2024-04-25T08:50:37.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.423+0000 7fad1d600640 1 -- 172.21.15.46:0/4153023075 wait complete. 2024-04-25T08:50:37.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:50:37.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.595+0000 7fddf3c00640 1 Processor -- start 2024-04-25T08:50:37.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.595+0000 7fddf3c00640 1 -- start start 2024-04-25T08:50:37.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf3c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf3c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fddec141580 0x7fddec0cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf3c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddec0b2e60 con 0x7fddec140ee0 2024-04-25T08:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf3c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddec1412c0 con 0x7fddec141580 2024-04-25T08:50:37.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf3c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec0cef50 con 0x7fddec06ea20 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf1c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fddec141580 0x7fddec0cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddebe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf1c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fddec141580 0x7fddec0cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:35192/0 (socket says 172.21.15.46:35192) 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf1c00640 1 -- 172.21.15.46:0/3721018928 learned_addr learned my addr 172.21.15.46:0/3721018928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddebe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec0ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36298/0 (socket says 172.21.15.46:36298) 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf1c00640 1 -- 172.21.15.46:0/3721018928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 msgr2=0x7fddec0ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf1c00640 1 --2- 172.21.15.46:0/3721018928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec0ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf1c00640 1 -- 172.21.15.46:0/3721018928 >> v1:172.21.15.82:6789/0 conn(0x7fddec140ee0 legacy=0x7fddec06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddf1c00640 1 -- 172.21.15.46:0/3721018928 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec0b30d0 con 0x7fddec141580 2024-04-25T08:50:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.599+0000 7fddebe00640 1 --2- 172.21.15.46:0/3721018928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec0ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf1c00640 1 --2- 172.21.15.46:0/3721018928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 0x7fddec0cea10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdde400d180 tx=0x7fdde401ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71d06f1e8f1e774e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:37.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddeb400640 1 -- 172.21.15.46:0/3721018928 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde40a7ce0 con 0x7fddec141580 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddeb400640 1 -- 172.21.15.46:0/3721018928 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde400b0d0 con 0x7fddec141580 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddeb400640 1 -- 172.21.15.46:0/3721018928 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4009600 con 0x7fddec141580 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 -- 172.21.15.46:0/3721018928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 msgr2=0x7fddec0cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 --2- 172.21.15.46:0/3721018928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 0x7fddec0cea10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdde400d180 tx=0x7fdde401ec10 comp rx=0 tx=0).stop 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 -- 172.21.15.46:0/3721018928 shutdown_connections 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 --2- 172.21.15.46:0/3721018928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 0x7fddec0cea10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 --2- 172.21.15.46:0/3721018928 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec0ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 -- 172.21.15.46:0/3721018928 >> 172.21.15.46:0/3721018928 conn(0x7fddec067050 msgr2=0x7fddec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 -- 172.21.15.46:0/3721018928 shutdown_connections 2024-04-25T08:50:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 -- 172.21.15.46:0/3721018928 wait complete. 2024-04-25T08:50:37.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 Processor -- start 2024-04-25T08:50:37.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.603+0000 7fddf3c00640 1 -- start start 2024-04-25T08:50:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf3c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec13eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf3c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddec140ee0 0x7fddec15ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf3c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 0x7fddec149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf3c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec0b4540 con 0x7fddec140ee0 2024-04-25T08:50:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf3c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddec0b43c0 con 0x7fddec141580 2024-04-25T08:50:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf3c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddec0b46c0 con 0x7fddec06ea20 2024-04-25T08:50:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddec140ee0 0x7fddec15ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddec140ee0 0x7fddec15ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38266/0 (socket says 172.21.15.46:38266) 2024-04-25T08:50:37.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddebe00640 1 -- 172.21.15.46:0/2330725719 learned_addr learned my addr 172.21.15.46:0/2330725719 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf1c00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 0x7fddec149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf1200640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec13eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddebe00640 1 -- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 msgr2=0x7fddec13eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddebe00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec13eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddebe00640 1 -- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 msgr2=0x7fddec149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddebe00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 0x7fddec149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddebe00640 1 -- 172.21.15.46:0/2330725719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddec15b420 con 0x7fddec140ee0 2024-04-25T08:50:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf1c00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 0x7fddec149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:37.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddebe00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddec140ee0 0x7fddec15ac80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fddd800ebe0 tx=0x7fddd800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fdde8c00640 1 -- 172.21.15.46:0/2330725719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8017070 con 0x7fddec140ee0 2024-04-25T08:50:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fdde8c00640 1 -- 172.21.15.46:0/2330725719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd800cc30 con 0x7fddec140ee0 2024-04-25T08:50:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fdde8c00640 1 -- 172.21.15.46:0/2330725719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8013470 con 0x7fddec140ee0 2024-04-25T08:50:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddec15b700 con 0x7fddec140ee0 2024-04-25T08:50:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddec14a210 con 0x7fddec140ee0 2024-04-25T08:50:37.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddec069080 con 0x7fddec140ee0 2024-04-25T08:50:37.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.607+0000 7fdde8c00640 1 -- 172.21.15.46:0/2330725719 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fddd801a070 con 0x7fddec140ee0 2024-04-25T08:50:37.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.611+0000 7fdde8c00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddcc066f60 0x7fddcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:37.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.611+0000 7fdde8c00640 1 -- 172.21.15.46:0/2330725719 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6037+0+0 (secure 0 0 0) 0x7fddd8085540 con 0x7fddec140ee0 2024-04-25T08:50:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.611+0000 7fddf1200640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddcc066f60 0x7fddcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:37.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.611+0000 7fddf1200640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddcc066f60 0x7fddcc069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fddec143b70 tx=0x7fdde0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:37.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.615+0000 7fdde8c00640 1 -- 172.21.15.46:0/2330725719 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fddd8056360 con 0x7fddec140ee0 2024-04-25T08:50:37.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:37.775+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fddec06ac10 con 0x7fddec140ee0 2024-04-25T08:50:38.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.419+0000 7fdde8c00640 1 -- 172.21.15.46:0/2330725719 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 169+0+0 (secure 0 0 0) 0x7fddd8055c60 con 0x7fddec140ee0 2024-04-25T08:50:38.410 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddcc066f60 msgr2=0x7fddcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddcc066f60 0x7fddcc069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fddec143b70 tx=0x7fdde0009040 comp rx=0 tx=0).stop 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddec140ee0 msgr2=0x7fddec15ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddec140ee0 0x7fddec15ac80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fddd800ebe0 tx=0x7fddd800c520 comp rx=0 tx=0).stop 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 shutdown_connections 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fddcc066f60 0x7fddcc069420 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddec141580 0x7fddec149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fddec140ee0 0x7fddec15ac80 secure :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fddd800ebe0 tx=0x7fddd800c520 comp rx=0 tx=0).stop 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 --2- 172.21.15.46:0/2330725719 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fddec06ea20 0x7fddec13eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 >> 172.21.15.46:0/2330725719 conn(0x7fddec067050 msgr2=0x7fddec0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:38.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 shutdown_connections 2024-04-25T08:50:38.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.423+0000 7fddf3c00640 1 -- 172.21.15.46:0/2330725719 wait complete. 2024-04-25T08:50:38.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:50:38.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92c4c00640 1 Processor -- start 2024-04-25T08:50:38.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92c4c00640 1 -- start start 2024-04-25T08:50:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92c4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92c009a410 0x7f92c006bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92c4c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92c00d0390 0x7f92c00cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92c4c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c009acb0 con 0x7f92c006c730 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92c4c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00b3dc0 con 0x7f92c009a410 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92c4c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c00cf680 con 0x7f92c00d0390 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92b6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92c009a410 0x7f92c006bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92b6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92c009a410 0x7f92c006bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43066/0 (socket says 172.21.15.46:43066) 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92b6a00640 1 -- 172.21.15.46:0/1410130482 learned_addr learned my addr 172.21.15.46:0/1410130482 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92bfe00640 1 --2- 172.21.15.46:0/1410130482 >> v2:172.21.15.82:3300/0 conn(0x7f92c00d0390 0x7f92c00cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92b6a00640 1 -- 172.21.15.46:0/1410130482 >> v2:172.21.15.82:3300/0 conn(0x7f92c00d0390 msgr2=0x7f92c00cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92b6a00640 1 --2- 172.21.15.46:0/1410130482 >> v2:172.21.15.82:3300/0 conn(0x7f92c00d0390 0x7f92c00cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92b6a00640 1 -- 172.21.15.46:0/1410130482 >> v1:172.21.15.82:6789/0 conn(0x7f92c006c730 legacy=0x7f92c006cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:38.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.599+0000 7f92b6a00640 1 -- 172.21.15.46:0/1410130482 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c009a9d0 con 0x7f92c009a410 2024-04-25T08:50:38.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92b6a00640 1 --2- 172.21.15.46:0/1410130482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c009a410 0x7f92c006bf30 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009870 tx=0x7f92a801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95ee9ea27266f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:38.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92bea00640 1 -- 172.21.15.46:0/1410130482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80b1070 con 0x7f92c009a410 2024-04-25T08:50:38.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92bea00640 1 -- 172.21.15.46:0/1410130482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a80ac620 con 0x7f92c009a410 2024-04-25T08:50:38.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92bea00640 1 -- 172.21.15.46:0/1410130482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80ac9f0 con 0x7f92c009a410 2024-04-25T08:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92c4c00640 1 -- 172.21.15.46:0/1410130482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c009a410 msgr2=0x7f92c006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92c4c00640 1 --2- 172.21.15.46:0/1410130482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c009a410 0x7f92c006bf30 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009870 tx=0x7f92a801c4b0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92c4c00640 1 -- 172.21.15.46:0/1410130482 shutdown_connections 2024-04-25T08:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92c4c00640 1 --2- 172.21.15.46:0/1410130482 >> v2:172.21.15.82:3300/0 conn(0x7f92c00d0390 0x7f92c00cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92c4c00640 1 --2- 172.21.15.46:0/1410130482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c009a410 0x7f92c006bf30 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92c4c00640 1 -- 172.21.15.46:0/1410130482 >> 172.21.15.46:0/1410130482 conn(0x7f92c00b2f50 msgr2=0x7f92c00b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92c4c00640 1 -- 172.21.15.46:0/1410130482 shutdown_connections 2024-04-25T08:50:38.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.603+0000 7f92c4c00640 1 -- 172.21.15.46:0/1410130482 wait complete. 2024-04-25T08:50:38.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 Processor -- start 2024-04-25T08:50:38.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 -- start start 2024-04-25T08:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c006c730 0x7f92c0152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c009a410 0x7f92c014d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92c00d0390 0x7f92c00b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c015dd60 con 0x7f92c006c730 2024-04-25T08:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c00cc400 con 0x7f92c009a410 2024-04-25T08:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c016b040 con 0x7f92c00d0390 2024-04-25T08:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c006c730 0x7f92c0152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c006c730 0x7f92c0152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43076/0 (socket says 172.21.15.46:43076) 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92bf400640 1 -- 172.21.15.46:0/3505969523 learned_addr learned my addr 172.21.15.46:0/3505969523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92bf400640 1 -- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92c00d0390 msgr2=0x7f92c00b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92bf400640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92c00d0390 0x7f92c00b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92bf400640 1 -- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c009a410 msgr2=0x7f92c014d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92bf400640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c009a410 0x7f92c014d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92bf400640 1 -- 172.21.15.46:0/3505969523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c014e690 con 0x7f92c006c730 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92bf400640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c006c730 0x7f92c0152890 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f92b000ba50 tx=0x7f92b000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92b7e00640 1 -- 172.21.15.46:0/3505969523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0019070 con 0x7f92c006c730 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92b7e00640 1 -- 172.21.15.46:0/3505969523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b000cc20 con 0x7f92c006c730 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c014e970 con 0x7f92c006c730 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92b7e00640 1 -- 172.21.15.46:0/3505969523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0014680 con 0x7f92c006c730 2024-04-25T08:50:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c00ae3f0 con 0x7f92c006c730 2024-04-25T08:50:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.607+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c0069010 con 0x7f92c006c730 2024-04-25T08:50:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.611+0000 7f92b7e00640 1 -- 172.21.15.46:0/3505969523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92b001c070 con 0x7f92c006c730 2024-04-25T08:50:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.611+0000 7f92b7e00640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f929c066f60 0x7f929c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.611+0000 7f92b7e00640 1 -- 172.21.15.46:0/3505969523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f92b0084d30 con 0x7f92c006c730 2024-04-25T08:50:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.611+0000 7f92b6a00640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f929c066f60 0x7f929c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.611+0000 7f92b6a00640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f929c066f60 0x7f929c069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f92a80ae000 tx=0x7f92a8004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:38.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.615+0000 7f92b7e00640 1 -- 172.21.15.46:0/3505969523 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92b0055700 con 0x7f92c006c730 2024-04-25T08:50:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.779+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f92c00cb820 con 0x7f92c006c730 2024-04-25T08:50:38.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.783+0000 7f92b7e00640 1 -- 172.21.15.46:0/3505969523 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f92b00597a0 con 0x7f92c006c730 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f929c066f60 msgr2=0x7f929c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f929c066f60 0x7f929c069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f92a80ae000 tx=0x7f92a8004740 comp rx=0 tx=0).stop 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c006c730 msgr2=0x7f92c0152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c006c730 0x7f92c0152890 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f92b000ba50 tx=0x7f92b000bf10 comp rx=0 tx=0).stop 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 shutdown_connections 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f929c066f60 0x7f929c069420 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f92c00d0390 0x7f92c00b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92c009a410 0x7f92c014d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 --2- 172.21.15.46:0/3505969523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92c006c730 0x7f92c0152890 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 >> 172.21.15.46:0/3505969523 conn(0x7f92c00b2f50 msgr2=0x7f92c00b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 shutdown_connections 2024-04-25T08:50:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.787+0000 7f92c4c00640 1 -- 172.21.15.46:0/3505969523 wait complete. 2024-04-25T08:50:38.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T08:50:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb8e00640 1 Processor -- start 2024-04-25T08:50:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb8e00640 1 -- start start 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb8e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1eb415dbd0 0x7f1eb40b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 0x7f1eb4070c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1eb40711d0 0x7f1eb415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb8e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb40b5af0 con 0x7f1eb40711d0 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb8e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb40b5250 con 0x7f1eb415dbd0 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb8e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40b5530 con 0x7f1eb40b0c30 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1eb40711d0 0x7f1eb415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1eb40711d0 0x7f1eb415dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43092/0 (socket says 172.21.15.46:43092) 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb2800640 1 -- 172.21.15.46:0/2853221275 learned_addr learned my addr 172.21.15.46:0/2853221275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:38.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1ea9400640 1 --2- 172.21.15.46:0/2853221275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 0x7f1eb4070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.947+0000 7f1eb1e00640 1 --2- 172.21.15.46:0/2853221275 >> v2:172.21.15.82:3300/0 conn(0x7f1eb415dbd0 0x7f1eb40b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb2800640 1 -- 172.21.15.46:0/2853221275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 msgr2=0x7f1eb4070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:38.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb2800640 1 --2- 172.21.15.46:0/2853221275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 0x7f1eb4070c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb2800640 1 -- 172.21.15.46:0/2853221275 >> v2:172.21.15.82:3300/0 conn(0x7f1eb415dbd0 msgr2=0x7f1eb40b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:38.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb2800640 1 --2- 172.21.15.46:0/2853221275 >> v2:172.21.15.82:3300/0 conn(0x7f1eb415dbd0 0x7f1eb40b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb2800640 1 -- 172.21.15.46:0/2853221275 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40b5810 con 0x7f1eb40711d0 2024-04-25T08:50:38.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb2800640 1 --2- 172.21.15.46:0/2853221275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb40711d0 0x7f1eb415dfb0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4072e70 tx=0x7f1eac01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417580de0cea23ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb1400640 1 -- 172.21.15.46:0/2853221275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0a5670 con 0x7f1eb40711d0 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb1400640 1 -- 172.21.15.46:0/2853221275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac0a5810 con 0x7f1eb40711d0 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb1400640 1 -- 172.21.15.46:0/2853221275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0a5be0 con 0x7f1eb40711d0 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb8e00640 1 -- 172.21.15.46:0/2853221275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb40711d0 msgr2=0x7f1eb415dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/2853221275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb40711d0 0x7f1eb415dfb0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4072e70 tx=0x7f1eac01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb8e00640 1 -- 172.21.15.46:0/2853221275 shutdown_connections 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/2853221275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb40711d0 0x7f1eb415dfb0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/2853221275 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 0x7f1eb4070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/2853221275 >> v2:172.21.15.82:3300/0 conn(0x7f1eb415dbd0 0x7f1eb40b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb8e00640 1 -- 172.21.15.46:0/2853221275 >> 172.21.15.46:0/2853221275 conn(0x7f1eb406f4f0 msgr2=0x7f1eb406f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb8e00640 1 -- 172.21.15.46:0/2853221275 shutdown_connections 2024-04-25T08:50:38.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.951+0000 7f1eb8e00640 1 -- 172.21.15.46:0/2853221275 wait complete. 2024-04-25T08:50:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb8e00640 1 Processor -- start 2024-04-25T08:50:38.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb8e00640 1 -- start start 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eb40711d0 0x7f1eb4151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 0x7f1eb4157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb415dbd0 0x7f1eb414d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb406cda0 con 0x7f1eb415dbd0 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb8e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb40b1010 con 0x7f1eb40711d0 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb8e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb406cf20 con 0x7f1eb40b0c30 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb415dbd0 0x7f1eb414d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb415dbd0 0x7f1eb414d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43108/0 (socket says 172.21.15.46:43108) 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb2800640 1 -- 172.21.15.46:0/1174539464 learned_addr learned my addr 172.21.15.46:0/1174539464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb1e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eb40711d0 0x7f1eb4151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1ea9400640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 0x7f1eb4157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb2800640 1 -- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 msgr2=0x7f1eb4157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb2800640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 0x7f1eb4157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb2800640 1 -- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eb40711d0 msgr2=0x7f1eb4151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb2800640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eb40711d0 0x7f1eb4151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb2800640 1 -- 172.21.15.46:0/1174539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb414e4c0 con 0x7f1eb415dbd0 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb1e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eb40711d0 0x7f1eb4151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb2800640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb415dbd0 0x7f1eb414d670 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1eac01b0a0 tx=0x7f1eac019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eaaa00640 1 -- 172.21.15.46:0/1174539464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac0a5670 con 0x7f1eb415dbd0 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eaaa00640 1 -- 172.21.15.46:0/1174539464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eac0be070 con 0x7f1eb415dbd0 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eaaa00640 1 -- 172.21.15.46:0/1174539464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eac009510 con 0x7f1eb415dbd0 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.955+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb414f880 con 0x7f1eb415dbd0 2024-04-25T08:50:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.959+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb414fe00 con 0x7f1eb415dbd0 2024-04-25T08:50:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.959+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb4069080 con 0x7f1eb415dbd0 2024-04-25T08:50:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.959+0000 7f1eaaa00640 1 -- 172.21.15.46:0/1174539464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1eac0b8ae0 con 0x7f1eb415dbd0 2024-04-25T08:50:38.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.959+0000 7f1eaaa00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e90066f60 0x7f1e90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.959+0000 7f1eaaa00640 1 -- 172.21.15.46:0/1174539464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f1eac120300 con 0x7f1eb415dbd0 2024-04-25T08:50:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.959+0000 7f1eb1e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e90066f60 0x7f1e90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.959+0000 7f1eb1e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e90066f60 0x7f1e90069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1eb41552f0 tx=0x7f1ea4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:38.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:38.963+0000 7f1eaaa00640 1 -- 172.21.15.46:0/1174539464 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1eac0165e0 con 0x7f1eb415dbd0 2024-04-25T08:50:39.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.187+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f1eb4154360 con 0x7f1eb415dbd0 2024-04-25T08:50:39.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.419+0000 7f1eaaa00640 1 -- 172.21.15.46:0/1174539464 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v403) v1 ==== 167+0+0 (secure 0 0 0) 0x7f1eac0f1460 con 0x7f1eb415dbd0 2024-04-25T08:50:39.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e90066f60 msgr2=0x7f1e90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e90066f60 0x7f1e90069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1eb41552f0 tx=0x7f1ea4008040 comp rx=0 tx=0).stop 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb415dbd0 msgr2=0x7f1eb414d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb415dbd0 0x7f1eb414d670 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1eac01b0a0 tx=0x7f1eac019790 comp rx=0 tx=0).stop 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 shutdown_connections 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e90066f60 0x7f1e90069420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eb415dbd0 0x7f1eb414d670 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1eb40b0c30 0x7f1eb4157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 --2- 172.21.15.46:0/1174539464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1eb40711d0 0x7f1eb4151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 >> 172.21.15.46:0/1174539464 conn(0x7f1eb406f4f0 msgr2=0x7f1eb415fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 shutdown_connections 2024-04-25T08:50:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.423+0000 7f1eb8e00640 1 -- 172.21.15.46:0/1174539464 wait complete. 2024-04-25T08:50:39.434 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T08:50:39.434 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:50:39.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.583+0000 7f616ac00640 1 Processor -- start 2024-04-25T08:50:39.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.583+0000 7f616ac00640 1 -- start start 2024-04-25T08:50:39.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f616ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f616415a750 0x7f616406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f616ac00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61640aed10 0x7f616406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f616ac00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616406d5e0 con 0x7f61640cc9e0 2024-04-25T08:50:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f616ac00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616415ab30 con 0x7f61640aed10 2024-04-25T08:50:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f616ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640ce4f0 con 0x7f616415a750 2024-04-25T08:50:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f615be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61640aed10 0x7f616406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f615be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61640aed10 0x7f616406f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43122/0 (socket says 172.21.15.46:43122) 2024-04-25T08:50:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f615be00640 1 -- 172.21.15.46:0/3618304073 learned_addr learned my addr 172.21.15.46:0/3618304073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f6163e00640 1 --2- 172.21.15.46:0/3618304073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f616415a750 0x7f616406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f615be00640 1 -- 172.21.15.46:0/3618304073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f616415a750 msgr2=0x7f616406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:39.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f615be00640 1 --2- 172.21.15.46:0/3618304073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f616415a750 0x7f616406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f615be00640 1 -- 172.21.15.46:0/3618304073 >> v1:172.21.15.82:6789/0 conn(0x7f61640cc9e0 legacy=0x7f61640b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f615be00640 1 -- 172.21.15.46:0/3618304073 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640ce720 con 0x7f61640aed10 2024-04-25T08:50:39.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f6163e00640 1 --2- 172.21.15.46:0/3618304073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f616415a750 0x7f616406ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:39.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.587+0000 7f615be00640 1 --2- 172.21.15.46:0/3618304073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 0x7f616406f150 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f614c009870 tx=0x7f614c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a36cbfe9e3ad141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:39.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f6163400640 1 -- 172.21.15.46:0/3618304073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c0ab650 con 0x7f61640aed10 2024-04-25T08:50:39.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f6163400640 1 -- 172.21.15.46:0/3618304073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c0b0070 con 0x7f61640aed10 2024-04-25T08:50:39.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f6163400640 1 -- 172.21.15.46:0/3618304073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c0aba10 con 0x7f61640aed10 2024-04-25T08:50:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 -- 172.21.15.46:0/3618304073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 msgr2=0x7f616406f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 --2- 172.21.15.46:0/3618304073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 0x7f616406f150 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f614c009870 tx=0x7f614c01c560 comp rx=0 tx=0).stop 2024-04-25T08:50:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 -- 172.21.15.46:0/3618304073 shutdown_connections 2024-04-25T08:50:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 --2- 172.21.15.46:0/3618304073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 0x7f616406f150 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 --2- 172.21.15.46:0/3618304073 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f616415a750 0x7f616406ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 -- 172.21.15.46:0/3618304073 >> 172.21.15.46:0/3618304073 conn(0x7f61640b12a0 msgr2=0x7f61640b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 -- 172.21.15.46:0/3618304073 shutdown_connections 2024-04-25T08:50:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 -- 172.21.15.46:0/3618304073 wait complete. 2024-04-25T08:50:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 Processor -- start 2024-04-25T08:50:39.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 -- start start 2024-04-25T08:50:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 0x7f616415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61640cc9e0 0x7f6164144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f616415a750 0x7f61640afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164167230 con 0x7f61640aed10 2024-04-25T08:50:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616406d270 con 0x7f616415a750 2024-04-25T08:50:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f616ac00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61641673b0 con 0x7f61640cc9e0 2024-04-25T08:50:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f6163e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 0x7f616415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f6163e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 0x7f616415cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43130/0 (socket says 172.21.15.46:43130) 2024-04-25T08:50:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f6163e00640 1 -- 172.21.15.46:0/3890171277 learned_addr learned my addr 172.21.15.46:0/3890171277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.591+0000 7f615be00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61640cc9e0 0x7f6164144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6168c00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f616415a750 0x7f61640afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6163e00640 1 -- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61640cc9e0 msgr2=0x7f6164144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6163e00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61640cc9e0 0x7f6164144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6163e00640 1 -- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f616415a750 msgr2=0x7f61640afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6163e00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f616415a750 0x7f61640afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6163e00640 1 -- 172.21.15.46:0/3890171277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640ad8e0 con 0x7f61640aed10 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6168c00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f616415a750 0x7f61640afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6163e00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 0x7f616415cd30 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f615400d800 tx=0x7f615400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6160c00640 1 -- 172.21.15.46:0/3890171277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154018020 con 0x7f61640aed10 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6160c00640 1 -- 172.21.15.46:0/3890171277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61540096c0 con 0x7f61640aed10 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6160c00640 1 -- 172.21.15.46:0/3890171277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154009ab0 con 0x7f61640aed10 2024-04-25T08:50:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61640adbc0 con 0x7f61640aed10 2024-04-25T08:50:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61640ae180 con 0x7f61640aed10 2024-04-25T08:50:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6164069080 con 0x7f61640aed10 2024-04-25T08:50:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6160c00640 1 -- 172.21.15.46:0/3890171277 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f615401b020 con 0x7f61640aed10 2024-04-25T08:50:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6160c00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6140066f90 0x7f6140069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.595+0000 7f6160c00640 1 -- 172.21.15.46:0/3890171277 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f6154085410 con 0x7f61640aed10 2024-04-25T08:50:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.599+0000 7f615be00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6140066f90 0x7f6140069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.599+0000 7f615be00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6140066f90 0x7f6140069450 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f614c0af000 tx=0x7f614c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.599+0000 7f6160c00640 1 -- 172.21.15.46:0/3890171277 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6154022020 con 0x7f61640aed10 2024-04-25T08:50:39.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61640739b0 con 0x7f61640aed10 2024-04-25T08:50:39.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f6160c00640 1 -- 172.21.15.46:0/3890171277 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+13506 (secure 0 0 0) 0x7f61540564d0 con 0x7f61640aed10 2024-04-25T08:50:39.758 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:39.758 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:50:38.418845+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1358086076":"2024-04-26T08:50:35.411611+0000","172.21.15.46:6834/1358086076":"2024-04-26T08:50:35.411611+0000","172.21.15.46:6835/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6834/2777781817":"2024-04-26T08:49:51.036604+0000","172.21.15.46:6835/1269610178":"2024-04-26T08:48:38.594941+0000","172.21.15.46:6834/1269610178":"2024-04-26T08:48:38.594941+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6140066f90 msgr2=0x7f6140069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6140066f90 0x7f6140069450 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f614c0af000 tx=0x7f614c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:50:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 msgr2=0x7f616415cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 0x7f616415cd30 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f615400d800 tx=0x7f615400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 shutdown_connections 2024-04-25T08:50:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6140066f90 0x7f6140069450 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f616415a750 0x7f61640afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f61640cc9e0 0x7f6164144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 --2- 172.21.15.46:0/3890171277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61640aed10 0x7f616415cd30 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 >> 172.21.15.46:0/3890171277 conn(0x7f61640b12a0 msgr2=0x7f6164152750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 shutdown_connections 2024-04-25T08:50:39.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.767+0000 7f616ac00640 1 -- 172.21.15.46:0/3890171277 wait complete. 2024-04-25T08:50:39.779 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:50:39.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:50:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.931+0000 7f9bc6e00640 1 Processor -- start 2024-04-25T08:50:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.931+0000 7f9bc6e00640 1 -- start start 2024-04-25T08:50:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bc6e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9bc0145310 0x7f9bc006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bc6e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc015d090 0x7f9bc015d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bc6e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc00cd930 con 0x7f9bc006dc70 2024-04-25T08:50:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bc6e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc00cdbd0 con 0x7f9bc0145310 2024-04-25T08:50:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bc6e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc00cdeb0 con 0x7f9bc015d090 2024-04-25T08:50:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bbfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9bc0145310 0x7f9bc006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bc4e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc015d090 0x7f9bc015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bbfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9bc0145310 0x7f9bc006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:52120/0 (socket says 172.21.15.46:52120) 2024-04-25T08:50:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bbfe00640 1 -- 172.21.15.46:0/4100342492 learned_addr learned my addr 172.21.15.46:0/4100342492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bbfe00640 1 -- 172.21.15.46:0/4100342492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc015d090 msgr2=0x7f9bc015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bbfe00640 1 --2- 172.21.15.46:0/4100342492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc015d090 0x7f9bc015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bbfe00640 1 -- 172.21.15.46:0/4100342492 >> v1:172.21.15.82:6789/0 conn(0x7f9bc006dc70 legacy=0x7f9bc00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bbfe00640 1 -- 172.21.15.46:0/4100342492 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc00ce190 con 0x7f9bc0145310 2024-04-25T08:50:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.935+0000 7f9bbfe00640 1 --2- 172.21.15.46:0/4100342492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc0145310 0x7f9bc006e0d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009e50 tx=0x7f9bb001ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=499692845281383c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bbf400640 1 -- 172.21.15.46:0/4100342492 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00a57f0 con 0x7f9bc0145310 2024-04-25T08:50:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bbf400640 1 -- 172.21.15.46:0/4100342492 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb00a5990 con 0x7f9bc0145310 2024-04-25T08:50:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bbf400640 1 -- 172.21.15.46:0/4100342492 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00a5d60 con 0x7f9bc0145310 2024-04-25T08:50:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 -- 172.21.15.46:0/4100342492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc0145310 msgr2=0x7f9bc006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 --2- 172.21.15.46:0/4100342492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc0145310 0x7f9bc006e0d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009e50 tx=0x7f9bb001ca30 comp rx=0 tx=0).stop 2024-04-25T08:50:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 -- 172.21.15.46:0/4100342492 shutdown_connections 2024-04-25T08:50:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 --2- 172.21.15.46:0/4100342492 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc015d090 0x7f9bc015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 --2- 172.21.15.46:0/4100342492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc0145310 0x7f9bc006e0d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 -- 172.21.15.46:0/4100342492 >> 172.21.15.46:0/4100342492 conn(0x7f9bc006ac10 msgr2=0x7f9bc006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 -- 172.21.15.46:0/4100342492 shutdown_connections 2024-04-25T08:50:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 -- 172.21.15.46:0/4100342492 wait complete. 2024-04-25T08:50:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 Processor -- start 2024-04-25T08:50:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 -- start start 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc0145310 0x7f9bc00b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc015d090 0x7f9bc0158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc00b2970 0x7f9bc00a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc0168510 con 0x7f9bc0145310 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc00cf380 con 0x7f9bc015d090 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc6e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc0168690 con 0x7f9bc00b2970 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bbfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc0145310 0x7f9bc00b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bbfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc0145310 0x7f9bc00b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43132/0 (socket says 172.21.15.46:43132) 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bbfe00640 1 -- 172.21.15.46:0/3812858557 learned_addr learned my addr 172.21.15.46:0/3812858557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.939+0000 7f9bc4e00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc00b2970 0x7f9bc00a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bb7400640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc015d090 0x7f9bc0158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbfe00640 1 -- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc00b2970 msgr2=0x7f9bc00a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbfe00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc00b2970 0x7f9bc00a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbfe00640 1 -- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc015d090 msgr2=0x7f9bc0158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbfe00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc015d090 0x7f9bc0158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbfe00640 1 -- 172.21.15.46:0/3812858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc00b30f0 con 0x7f9bc0145310 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bb7400640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc015d090 0x7f9bc0158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbfe00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc0145310 0x7f9bc00b3eb0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00af040 tx=0x7f9bb00197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbcc00640 1 -- 172.21.15.46:0/3812858557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00b8470 con 0x7f9bc0145310 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbcc00640 1 -- 172.21.15.46:0/3812858557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb00b8650 con 0x7f9bc0145310 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbcc00640 1 -- 172.21.15.46:0/3812858557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00b8a40 con 0x7f9bc0145310 2024-04-25T08:50:39.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc00b33d0 con 0x7f9bc0145310 2024-04-25T08:50:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc00b3990 con 0x7f9bc0145310 2024-04-25T08:50:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc0069010 con 0x7f9bc0145310 2024-04-25T08:50:39.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbcc00640 1 -- 172.21.15.46:0/3812858557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bb00c0020 con 0x7f9bc0145310 2024-04-25T08:50:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbcc00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b9c066f60 0x7f9b9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.943+0000 7f9bbcc00640 1 -- 172.21.15.46:0/3812858557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f9bb011fa70 con 0x7f9bc0145310 2024-04-25T08:50:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.947+0000 7f9bb7400640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b9c066f60 0x7f9b9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:39.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.947+0000 7f9bb7400640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b9c066f60 0x7f9b9c069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0144770 tx=0x7f9ba8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:39.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:39.947+0000 7f9bbcc00640 1 -- 172.21.15.46:0/3812858557 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bb0016620 con 0x7f9bc0145310 2024-04-25T08:50:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9bc015c4a0 con 0x7f9bc0145310 2024-04-25T08:50:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bbcc00640 1 -- 172.21.15.46:0/3812858557 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bb00f0aa0 con 0x7f9bc0145310 2024-04-25T08:50:40.105 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:50:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b9c066f60 msgr2=0x7f9b9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b9c066f60 0x7f9b9c069420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0144770 tx=0x7f9ba8009040 comp rx=0 tx=0).stop 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc0145310 msgr2=0x7f9bc00b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc0145310 0x7f9bc00b3eb0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00af040 tx=0x7f9bb00197d0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 shutdown_connections 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9b9c066f60 0x7f9b9c069420 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bc00b2970 0x7f9bc00a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bc015d090 0x7f9bc0158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 --2- 172.21.15.46:0/3812858557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bc0145310 0x7f9bc00b3eb0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 >> 172.21.15.46:0/3812858557 conn(0x7f9bc006ac10 msgr2=0x7f9bc00af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 shutdown_connections 2024-04-25T08:50:40.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.115+0000 7f9bc6e00640 1 -- 172.21.15.46:0/3812858557 wait complete. 2024-04-25T08:50:40.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T08:50:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.275+0000 7f2d43800640 1 Processor -- start 2024-04-25T08:50:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.275+0000 7f2d43800640 1 -- start start 2024-04-25T08:50:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d43800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2d3c159230 0x7f2d3c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d43800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 0x7f2d3c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d43800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d3c0cc700 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d43800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d3c0cc9e0 con 0x7f2d3c159230 2024-04-25T08:50:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d43800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0cccc0 con 0x7f2d3c0cdc50 2024-04-25T08:50:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d33e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2d3c0cbeb0 0x7f2d3c0cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53788/0 (socket says 172.21.15.46:53788) 2024-04-25T08:50:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d33e00640 1 -- 172.21.15.46:0/3058596596 learned_addr learned my addr 172.21.15.46:0/3058596596 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d41800640 1 --2- 172.21.15.46:0/3058596596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 0x7f2d3c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d40e00640 1 --2- 172.21.15.46:0/3058596596 >> v2:172.21.15.82:3300/0 conn(0x7f2d3c159230 0x7f2d3c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d3be00640 1 -- 172.21.15.46:0/3058596596 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688762486 0 0) 0x7f2d3c0cc700 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d3be00640 1 -- 172.21.15.46:0/3058596596 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d20003530 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d40e00640 1 -- 172.21.15.46:0/3058596596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 msgr2=0x7f2d3c15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d40e00640 1 --2- 172.21.15.46:0/3058596596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 0x7f2d3c15d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d40e00640 1 -- 172.21.15.46:0/3058596596 >> v1:172.21.15.46:6789/0 conn(0x7f2d3c0cbeb0 legacy=0x7f2d3c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d40e00640 1 -- 172.21.15.46:0/3058596596 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c0ccfa0 con 0x7f2d3c159230 2024-04-25T08:50:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.279+0000 7f2d41800640 1 --2- 172.21.15.46:0/3058596596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 0x7f2d3c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d40e00640 1 --2- 172.21.15.46:0/3058596596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c159230 0x7f2d3c15bdb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0097c0 tx=0x7f2d2c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daa6d93095620a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d3be00640 1 -- 172.21.15.46:0/3058596596 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0ab600 con 0x7f2d3c159230 2024-04-25T08:50:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d3be00640 1 -- 172.21.15.46:0/3058596596 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c0b0070 con 0x7f2d3c159230 2024-04-25T08:50:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d3be00640 1 -- 172.21.15.46:0/3058596596 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0ab9c0 con 0x7f2d3c159230 2024-04-25T08:50:40.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 -- 172.21.15.46:0/3058596596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c159230 msgr2=0x7f2d3c15bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 --2- 172.21.15.46:0/3058596596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c159230 0x7f2d3c15bdb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0097c0 tx=0x7f2d2c01c3e0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 -- 172.21.15.46:0/3058596596 shutdown_connections 2024-04-25T08:50:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 --2- 172.21.15.46:0/3058596596 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 0x7f2d3c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 --2- 172.21.15.46:0/3058596596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c159230 0x7f2d3c15bdb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 -- 172.21.15.46:0/3058596596 >> 172.21.15.46:0/3058596596 conn(0x7f2d3c0b12a0 msgr2=0x7f2d3c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 -- 172.21.15.46:0/3058596596 shutdown_connections 2024-04-25T08:50:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 -- 172.21.15.46:0/3058596596 wait complete. 2024-04-25T08:50:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 Processor -- start 2024-04-25T08:50:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.283+0000 7f2d43800640 1 -- start start 2024-04-25T08:50:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d43800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c0cbeb0 0x7f2d3c0ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d43800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 0x7f2d3c06dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d43800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d3c159230 0x7f2d3c1f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d43800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c15df50 con 0x7f2d3c159230 2024-04-25T08:50:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d43800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c0cc4e0 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d43800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3c15e0d0 con 0x7f2d3c0cdc50 2024-04-25T08:50:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d41800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d3c159230 0x7f2d3c1f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d33e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 0x7f2d3c06dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d40e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c0cbeb0 0x7f2d3c0ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d41800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d3c159230 0x7f2d3c1f6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43146/0 (socket says 172.21.15.46:43146) 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d41800640 1 -- 172.21.15.46:0/254253294 learned_addr learned my addr 172.21.15.46:0/254253294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d40e00640 1 -- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 msgr2=0x7f2d3c06dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d40e00640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 0x7f2d3c06dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d40e00640 1 -- 172.21.15.46:0/254253294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d3c159230 msgr2=0x7f2d3c1f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d40e00640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d3c159230 0x7f2d3c1f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d40e00640 1 -- 172.21.15.46:0/254253294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c164900 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d41800640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d3c159230 0x7f2d3c1f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d40e00640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c0cbeb0 0x7f2d3c0ade40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0ac000 tx=0x7f2d2c01ac20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d39600640 1 -- 172.21.15.46:0/254253294 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0bf610 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d39600640 1 -- 172.21.15.46:0/254253294 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2c0bf7f0 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d39600640 1 -- 172.21.15.46:0/254253294 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0bfbe0 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3c164bc0 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3c165180 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3c069080 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.287+0000 7f2d39600640 1 -- 172.21.15.46:0/254253294 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d2c0bd020 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.291+0000 7f2d39600640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d18066f60 0x7f2d18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.291+0000 7f2d39600640 1 -- 172.21.15.46:0/254253294 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f2d2c11f320 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.291+0000 7f2d33e00640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d18066f60 0x7f2d18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.291+0000 7f2d33e00640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d18066f60 0x7f2d18069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c159e40 tx=0x7f2d24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.295+0000 7f2d39600640 1 -- 172.21.15.46:0/254253294 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d2c016090 con 0x7f2d3c0cbeb0 2024-04-25T08:50:40.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:40.519+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f2d3c0723d0 con 0x7f2d3c0cbeb0 2024-04-25T08:50:41.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.431+0000 7f2d39600640 1 -- 172.21.15.46:0/254253294 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v405) v1 ==== 149+0+0 (secure 0 0 0) 0x7f2d2c0efd90 con 0x7f2d3c0cbeb0 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d18066f60 msgr2=0x7f2d18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d18066f60 0x7f2d18069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c159e40 tx=0x7f2d24009040 comp rx=0 tx=0).stop 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c0cbeb0 msgr2=0x7f2d3c0ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c0cbeb0 0x7f2d3c0ade40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0ac000 tx=0x7f2d2c01ac20 comp rx=0 tx=0).stop 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 shutdown_connections 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d18066f60 0x7f2d18069420 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d3c159230 0x7f2d3c1f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d3c0cdc50 0x7f2d3c06dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 --2- 172.21.15.46:0/254253294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d3c0cbeb0 0x7f2d3c0ade40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 >> 172.21.15.46:0/254253294 conn(0x7f2d3c0b12a0 msgr2=0x7f2d3c143d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 shutdown_connections 2024-04-25T08:50:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.435+0000 7f2d43800640 1 -- 172.21.15.46:0/254253294 wait complete. 2024-04-25T08:50:41.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:50:41.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d79c00640 1 Processor -- start 2024-04-25T08:50:41.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d79c00640 1 -- start start 2024-04-25T08:50:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d79c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d74144f60 0x7f6d7415c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d79c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6d740717c0 0x7f6d7406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d79c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7406df20 con 0x7f6d7406d0d0 2024-04-25T08:50:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d79c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7415c2e0 con 0x7f6d74144f60 2024-04-25T08:50:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d79c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7406dc40 con 0x7f6d740717c0 2024-04-25T08:50:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d74144f60 0x7f6d7415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d74144f60 0x7f6d7415c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43158/0 (socket says 172.21.15.46:43158) 2024-04-25T08:50:41.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d73e00640 1 -- 172.21.15.46:0/2691685654 learned_addr learned my addr 172.21.15.46:0/2691685654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d79200640 1 --2- 172.21.15.46:0/2691685654 >> v2:172.21.15.82:3300/0 conn(0x7f6d740717c0 0x7f6d7406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d73e00640 1 -- 172.21.15.46:0/2691685654 >> v2:172.21.15.82:3300/0 conn(0x7f6d740717c0 msgr2=0x7f6d7406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d73e00640 1 --2- 172.21.15.46:0/2691685654 >> v2:172.21.15.82:3300/0 conn(0x7f6d740717c0 0x7f6d7406d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d73e00640 1 -- 172.21.15.46:0/2691685654 >> v1:172.21.15.82:6789/0 conn(0x7f6d7406d0d0 legacy=0x7f6d740710b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.607+0000 7f6d73e00640 1 -- 172.21.15.46:0/2691685654 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7415bc90 con 0x7f6d74144f60 2024-04-25T08:50:41.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d73e00640 1 --2- 172.21.15.46:0/2691685654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 0x7f6d7415c770 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6d7415c460 tx=0x7f6d6401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f357db5f4cab13da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:41.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d73400640 1 -- 172.21.15.46:0/2691685654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d640b0070 con 0x7f6d74144f60 2024-04-25T08:50:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d73400640 1 -- 172.21.15.46:0/2691685654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d640ab600 con 0x7f6d74144f60 2024-04-25T08:50:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d73400640 1 -- 172.21.15.46:0/2691685654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d640ab9d0 con 0x7f6d74144f60 2024-04-25T08:50:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d79c00640 1 -- 172.21.15.46:0/2691685654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 msgr2=0x7f6d7415c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d79c00640 1 --2- 172.21.15.46:0/2691685654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 0x7f6d7415c770 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6d7415c460 tx=0x7f6d6401c450 comp rx=0 tx=0).stop 2024-04-25T08:50:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d79c00640 1 -- 172.21.15.46:0/2691685654 shutdown_connections 2024-04-25T08:50:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d79c00640 1 --2- 172.21.15.46:0/2691685654 >> v2:172.21.15.82:3300/0 conn(0x7f6d740717c0 0x7f6d7406d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d79c00640 1 --2- 172.21.15.46:0/2691685654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 0x7f6d7415c770 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d79c00640 1 -- 172.21.15.46:0/2691685654 >> 172.21.15.46:0/2691685654 conn(0x7f6d74066fa0 msgr2=0x7f6d7406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d79c00640 1 -- 172.21.15.46:0/2691685654 shutdown_connections 2024-04-25T08:50:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.611+0000 7f6d79c00640 1 -- 172.21.15.46:0/2691685654 wait complete. 2024-04-25T08:50:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 Processor -- start 2024-04-25T08:50:41.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 -- start start 2024-04-25T08:50:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d7406d0d0 0x7f6d74141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d740717c0 0x7f6d741f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 0x7f6d7415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7415fd50 con 0x7f6d74144f60 2024-04-25T08:50:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7415fbd0 con 0x7f6d740717c0 2024-04-25T08:50:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7415fed0 con 0x7f6d7406d0d0 2024-04-25T08:50:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 0x7f6d7415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 0x7f6d7415ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43160/0 (socket says 172.21.15.46:43160) 2024-04-25T08:50:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79200640 1 -- 172.21.15.46:0/558225962 learned_addr learned my addr 172.21.15.46:0/558225962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d73e00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d7406d0d0 0x7f6d74141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d6b400640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d740717c0 0x7f6d741f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79200640 1 -- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d7406d0d0 msgr2=0x7f6d74141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79200640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d7406d0d0 0x7f6d74141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79200640 1 -- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d740717c0 msgr2=0x7f6d741f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79200640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d740717c0 0x7f6d741f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79200640 1 -- 172.21.15.46:0/558225962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d741415f0 con 0x7f6d74144f60 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79200640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 0x7f6d7415ae30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c007e40 tx=0x7f6d6c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d70c00640 1 -- 172.21.15.46:0/558225962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c014620 con 0x7f6d74144f60 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d70c00640 1 -- 172.21.15.46:0/558225962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c019070 con 0x7f6d74144f60 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d70c00640 1 -- 172.21.15.46:0/558225962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c014a20 con 0x7f6d74144f60 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d74142410 con 0x7f6d74144f60 2024-04-25T08:50:41.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.615+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7415b370 con 0x7f6d74144f60 2024-04-25T08:50:41.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.619+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d74069010 con 0x7f6d74144f60 2024-04-25T08:50:41.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.619+0000 7f6d70c00640 1 -- 172.21.15.46:0/558225962 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d6c014bc0 con 0x7f6d74144f60 2024-04-25T08:50:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.619+0000 7f6d70c00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6d50066f60 0x7f6d50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.619+0000 7f6d70c00640 1 -- 172.21.15.46:0/558225962 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f6d6c085780 con 0x7f6d74144f60 2024-04-25T08:50:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.619+0000 7f6d73e00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6d50066f60 0x7f6d50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:41.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.619+0000 7f6d73e00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6d50066f60 0x7f6d50069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6d640ad000 tx=0x7f6d64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:41.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.623+0000 7f6d70c00640 1 -- 172.21.15.46:0/558225962 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d6c056840 con 0x7f6d74144f60 2024-04-25T08:50:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.787+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6d740721c0 con 0x7f6d74144f60 2024-04-25T08:50:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d70c00640 1 -- 172.21.15.46:0/558225962 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6d6c05c020 con 0x7f6d74144f60 2024-04-25T08:50:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6d50066f60 msgr2=0x7f6d50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6d50066f60 0x7f6d50069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6d640ad000 tx=0x7f6d64008040 comp rx=0 tx=0).stop 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 msgr2=0x7f6d7415ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 0x7f6d7415ae30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c007e40 tx=0x7f6d6c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 shutdown_connections 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6d50066f60 0x7f6d50069420 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d74144f60 0x7f6d7415ae30 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d740717c0 0x7f6d741f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 --2- 172.21.15.46:0/558225962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6d7406d0d0 0x7f6d74141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 >> 172.21.15.46:0/558225962 conn(0x7f6d74066fa0 msgr2=0x7f6d7413a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 shutdown_connections 2024-04-25T08:50:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:41.791+0000 7f6d79c00640 1 -- 172.21.15.46:0/558225962 wait complete. 2024-04-25T08:50:41.801 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:50:41.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:50:41.902 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:50:41.996 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:50:41.996 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:41.996 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:42.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:50:42.005 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:50:42.005 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:42.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:50:42.015 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:42.015 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:42.059 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:50:42.059 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:50:42.059 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:42.068 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:50:42.068 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:50:42.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f0dc00640 1 Processor -- start 2024-04-25T08:50:42.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f0dc00640 1 -- start start 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f0dc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f08099a00 0x7f3f08099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f0dc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806c190 0x7f3f08161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f0dc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f080cf5b0 con 0x7f3f0806bd20 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f0dc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0806d060 con 0x7f3f08099a00 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f0dc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0809a320 con 0x7f3f0806c190 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f06a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f08099a00 0x7f3f08099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f06a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f08099a00 0x7f3f08099de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43168/0 (socket says 172.21.15.46:43168) 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3effe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3f0806bd20 0x7f3f081616b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53800/0 (socket says 172.21.15.46:53800) 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f06a00640 1 -- 172.21.15.46:0/1574968314 learned_addr learned my addr 172.21.15.46:0/1574968314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f07400640 1 --2- 172.21.15.46:0/1574968314 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806c190 0x7f3f08161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f06a00640 1 -- 172.21.15.46:0/1574968314 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806c190 msgr2=0x7f3f08161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f06a00640 1 --2- 172.21.15.46:0/1574968314 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806c190 0x7f3f08161dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f06a00640 1 -- 172.21.15.46:0/1574968314 >> v1:172.21.15.46:6789/0 conn(0x7f3f0806bd20 legacy=0x7f3f081616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f06a00640 1 -- 172.21.15.46:0/1574968314 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0806fa90 con 0x7f3f08099a00 2024-04-25T08:50:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f06000640 1 -- 172.21.15.46:0/1574968314 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260642085 0 0) 0x7f3f080cf5b0 con 0x7f3f0806bd20 2024-04-25T08:50:42.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.251+0000 7f3f07400640 1 --2- 172.21.15.46:0/1574968314 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806c190 0x7f3f08161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:42.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f06a00640 1 --2- 172.21.15.46:0/1574968314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f08099a00 0x7f3f08099de0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009870 tx=0x7f3ef801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61cb0ca32c63690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f06000640 1 -- 172.21.15.46:0/1574968314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80ab640 con 0x7f3f08099a00 2024-04-25T08:50:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f06000640 1 -- 172.21.15.46:0/1574968314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef80b0070 con 0x7f3f08099a00 2024-04-25T08:50:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f06000640 1 -- 172.21.15.46:0/1574968314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80aba00 con 0x7f3f08099a00 2024-04-25T08:50:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f0dc00640 1 -- 172.21.15.46:0/1574968314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f08099a00 msgr2=0x7f3f08099de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f0dc00640 1 --2- 172.21.15.46:0/1574968314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f08099a00 0x7f3f08099de0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009870 tx=0x7f3ef801c530 comp rx=0 tx=0).stop 2024-04-25T08:50:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f0dc00640 1 -- 172.21.15.46:0/1574968314 shutdown_connections 2024-04-25T08:50:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f0dc00640 1 --2- 172.21.15.46:0/1574968314 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806c190 0x7f3f08161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f0dc00640 1 --2- 172.21.15.46:0/1574968314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f08099a00 0x7f3f08099de0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f0dc00640 1 -- 172.21.15.46:0/1574968314 >> 172.21.15.46:0/1574968314 conn(0x7f3f080d16b0 msgr2=0x7f3f080d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f0dc00640 1 -- 172.21.15.46:0/1574968314 shutdown_connections 2024-04-25T08:50:42.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.255+0000 7f3f0dc00640 1 -- 172.21.15.46:0/1574968314 wait complete. 2024-04-25T08:50:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 Processor -- start 2024-04-25T08:50:42.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 -- start start 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806bd20 0x7f3f080c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f0806c190 0x7f3f0814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f08099a00 0x7f3f08150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0809a750 con 0x7f3f0806c190 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0806fd50 con 0x7f3f08099a00 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f08071280 con 0x7f3f0806bd20 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3effe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f0806c190 0x7f3f0814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3effe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f0806c190 0x7f3f0814fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43178/0 (socket says 172.21.15.46:43178) 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3effe00640 1 -- 172.21.15.46:0/2319544978 learned_addr learned my addr 172.21.15.46:0/2319544978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:42.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f06a00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806bd20 0x7f3f080c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f07400640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f08099a00 0x7f3f08150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3effe00640 1 -- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806bd20 msgr2=0x7f3f080c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3effe00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806bd20 0x7f3f080c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3effe00640 1 -- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f08099a00 msgr2=0x7f3f08150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3effe00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f08099a00 0x7f3f08150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3effe00640 1 -- 172.21.15.46:0/2319544978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f081509c0 con 0x7f3f0806c190 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f06a00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806bd20 0x7f3f080c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f07400640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f08099a00 0x7f3f08150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3effe00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f0806c190 0x7f3f0814fce0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c4b0 tx=0x7f3ef000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3efea00640 1 -- 172.21.15.46:0/2319544978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0018070 con 0x7f3f0806c190 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3efea00640 1 -- 172.21.15.46:0/2319544978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef0007500 con 0x7f3f0806c190 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3efea00640 1 -- 172.21.15.46:0/2319544978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0004a30 con 0x7f3f0806c190 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f081fad00 con 0x7f3f0806c190 2024-04-25T08:50:42.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f081fb2c0 con 0x7f3f0806c190 2024-04-25T08:50:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.259+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f081fb660 con 0x7f3f0806c190 2024-04-25T08:50:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.263+0000 7f3efea00640 1 -- 172.21.15.46:0/2319544978 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ef0004bd0 con 0x7f3f0806c190 2024-04-25T08:50:42.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.263+0000 7f3efea00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3ee4066f60 0x7f3ee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.263+0000 7f3efea00640 1 -- 172.21.15.46:0/2319544978 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f3ef0084180 con 0x7f3f0806c190 2024-04-25T08:50:42.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.263+0000 7f3f06a00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3ee4066f60 0x7f3ee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.263+0000 7f3f06a00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3ee4066f60 0x7f3ee4069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80af000 tx=0x7f3ef80a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:42.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.267+0000 7f3efea00640 1 -- 172.21.15.46:0/2319544978 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ef00551b0 con 0x7f3f0806c190 2024-04-25T08:50:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.463+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f08069080 con 0x7f3f0806c190 2024-04-25T08:50:42.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.463+0000 7f3efea00640 1 -- 172.21.15.46:0/2319544978 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f3ef0054ab0 con 0x7f3f0806c190 2024-04-25T08:50:42.455 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:42.455 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1642732503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1642732503},{"type":"v1","addr":"172.21.15.82:6837","nonce":1642732503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2191739588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2191739588},{"type":"v1","addr":"172.21.15.82:6835","nonce":2191739588}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2881292494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2881292494},{"type":"v1","addr":"172.21.15.46:6833","nonce":2881292494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8256,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/313325433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":313325433},{"type":"v1","addr":"172.21.15.46:6835","nonce":313325433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[]} 2024-04-25T08:50:42.455 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 77 2024-04-25T08:50:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3ee4066f60 msgr2=0x7f3ee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3ee4066f60 0x7f3ee4069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80af000 tx=0x7f3ef80a7000 comp rx=0 tx=0).stop 2024-04-25T08:50:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f0806c190 msgr2=0x7f3f0814fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f0806c190 0x7f3f0814fce0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000c4b0 tx=0x7f3ef000c970 comp rx=0 tx=0).stop 2024-04-25T08:50:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 shutdown_connections 2024-04-25T08:50:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3ee4066f60 0x7f3ee4069420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f08099a00 0x7f3f08150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f0806c190 0x7f3f0814fce0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 --2- 172.21.15.46:0/2319544978 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3f0806bd20 0x7f3f080c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 >> 172.21.15.46:0/2319544978 conn(0x7f3f080d16b0 msgr2=0x7f3f081614d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 shutdown_connections 2024-04-25T08:50:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.467+0000 7f3f0dc00640 1 -- 172.21.15.46:0/2319544978 wait complete. 2024-04-25T08:50:42.477 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:50:42.477 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:50:42.477 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:50:42.477 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:50:42.478 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:50:42.478 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:50:42.478 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:50:42.478 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:50:42.478 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:50:42.478 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:50:42.479 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:50:42.479 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:50:42.479 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:50:42.479 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:50:42.479 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:50:42.479 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:50:42.489 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:50:42.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:50:42.491 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:50:42.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:50:42.492 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:50:42.492 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:50:42.493 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:50:42.493 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:50:42.539 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:50:42.539 DEBUG:teuthology.parallel:result is None 2024-04-25T08:50:42.539 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:50:42.540 DEBUG:teuthology.parallel:result is None 2024-04-25T08:50:42.540 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:50:42.540 DEBUG:teuthology.parallel:result is None 2024-04-25T08:50:42.541 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:50:42.541 DEBUG:teuthology.parallel:result is None 2024-04-25T08:50:42.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:50:42.625 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:50:42.626 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-04-25T08:50:42.626 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:50:42.628 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-04-25T08:50:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.695+0000 7f64f6200640 1 Processor -- start 2024-04-25T08:50:42.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64f6200640 1 -- start start 2024-04-25T08:50:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64f6200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 0x7f64f00b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64f6200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f006bdc0 con 0x7f64f0099a00 2024-04-25T08:50:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64f6200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f006c0a0 con 0x7f64f00ccde0 2024-04-25T08:50:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64f6200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f006db90 con 0x7f64f00b2240 2024-04-25T08:50:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64ef400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f64f0099a00 0x7f64f0099de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53814/0 (socket says 172.21.15.46:53814) 2024-04-25T08:50:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64ef400640 1 -- 172.21.15.46:0/36081948 learned_addr learned my addr 172.21.15.46:0/36081948 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64efe00640 1 --2- 172.21.15.46:0/36081948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 0x7f64f00b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64eea00640 1 -- 172.21.15.46:0/36081948 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247589887 0 0) 0x7f64f006bdc0 con 0x7f64f0099a00 2024-04-25T08:50:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64eea00640 1 -- 172.21.15.46:0/36081948 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d4003530 con 0x7f64f0099a00 2024-04-25T08:50:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64eea00640 1 -- 172.21.15.46:0/36081948 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760345431 0 0) 0x7f64f006c0a0 con 0x7f64f00ccde0 2024-04-25T08:50:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64eea00640 1 -- 172.21.15.46:0/36081948 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f006bdc0 con 0x7f64f00ccde0 2024-04-25T08:50:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64efe00640 1 -- 172.21.15.46:0/36081948 >> v1:172.21.15.82:6789/0 conn(0x7f64f00ccde0 legacy=0x7f64f00cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64efe00640 1 -- 172.21.15.46:0/36081948 >> v1:172.21.15.46:6789/0 conn(0x7f64f0099a00 legacy=0x7f64f0099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.699+0000 7f64efe00640 1 -- 172.21.15.46:0/36081948 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f006c380 con 0x7f64f00b2240 2024-04-25T08:50:42.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64efe00640 1 --2- 172.21.15.46:0/36081948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 0x7f64f00b26d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009df0 tx=0x7f64e801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29a37ff5b3be4b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64eea00640 1 -- 172.21.15.46:0/36081948 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e80b2070 con 0x7f64f00b2240 2024-04-25T08:50:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64eea00640 1 -- 172.21.15.46:0/36081948 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e80a5840 con 0x7f64f00b2240 2024-04-25T08:50:42.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64eea00640 1 -- 172.21.15.46:0/36081948 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e80a5c10 con 0x7f64f00b2240 2024-04-25T08:50:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64f6200640 1 -- 172.21.15.46:0/36081948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 msgr2=0x7f64f00b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64f6200640 1 --2- 172.21.15.46:0/36081948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 0x7f64f00b26d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009df0 tx=0x7f64e801c900 comp rx=0 tx=0).stop 2024-04-25T08:50:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64f6200640 1 -- 172.21.15.46:0/36081948 shutdown_connections 2024-04-25T08:50:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64f6200640 1 --2- 172.21.15.46:0/36081948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 0x7f64f00b26d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64f6200640 1 -- 172.21.15.46:0/36081948 >> 172.21.15.46:0/36081948 conn(0x7f64f00b3120 msgr2=0x7f64f00b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:42.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64f6200640 1 -- 172.21.15.46:0/36081948 shutdown_connections 2024-04-25T08:50:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64f6200640 1 -- 172.21.15.46:0/36081948 wait complete. 2024-04-25T08:50:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64f6200640 1 Processor -- start 2024-04-25T08:50:42.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.703+0000 7f64f6200640 1 -- start start 2024-04-25T08:50:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64f6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f0099a00 0x7f64f00c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64f6200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 0x7f64f0153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64f6200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00ccde0 0x7f64f0153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64f6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f015de90 con 0x7f64f0099a00 2024-04-25T08:50:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64f6200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f015dd10 con 0x7f64f00ccde0 2024-04-25T08:50:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64f6200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f016be10 con 0x7f64f00b2240 2024-04-25T08:50:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f0099a00 0x7f64f00c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f0099a00 0x7f64f00c50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43218/0 (socket says 172.21.15.46:43218) 2024-04-25T08:50:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64ef400640 1 -- 172.21.15.46:0/356348772 learned_addr learned my addr 172.21.15.46:0/356348772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64efe00640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00ccde0 0x7f64f0153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64e7e00640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 0x7f64f0153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64ef400640 1 -- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 msgr2=0x7f64f0153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64ef400640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 0x7f64f0153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64ef400640 1 -- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00ccde0 msgr2=0x7f64f0153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64ef400640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00ccde0 0x7f64f0153190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64ef400640 1 -- 172.21.15.46:0/356348772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f01542f0 con 0x7f64f0099a00 2024-04-25T08:50:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64ef400640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f0099a00 0x7f64f00c50b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c450 tx=0x7f64e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64e7400640 1 -- 172.21.15.46:0/356348772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0017070 con 0x7f64f0099a00 2024-04-25T08:50:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64e7400640 1 -- 172.21.15.46:0/356348772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e0005cf0 con 0x7f64f0099a00 2024-04-25T08:50:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64e7400640 1 -- 172.21.15.46:0/356348772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e00072b0 con 0x7f64f0099a00 2024-04-25T08:50:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f01545d0 con 0x7f64f0099a00 2024-04-25T08:50:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f01536d0 con 0x7f64f0099a00 2024-04-25T08:50:42.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.707+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f0069080 con 0x7f64f0099a00 2024-04-25T08:50:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.711+0000 7f64e7400640 1 -- 172.21.15.46:0/356348772 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64e001a070 con 0x7f64f0099a00 2024-04-25T08:50:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.711+0000 7f64e7400640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.711+0000 7f64e7400640 1 -- 172.21.15.46:0/356348772 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f64e00841e0 con 0x7f64f0099a00 2024-04-25T08:50:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.711+0000 7f64e7e00640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.711+0000 7f64e7e00640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f64f006c070 tx=0x7f64d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.715+0000 7f64e7400640 1 -- 172.21.15.46:0/356348772 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64e0054c50 con 0x7f64f0099a00 2024-04-25T08:50:42.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:42.887+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f64f009a1d0 con 0x7f64f0099a00 2024-04-25T08:50:43.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.807+0000 7f64e7400640 1 -- 172.21.15.46:0/356348772 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v102) v1 ==== 97+0+0 (secure 0 0 0) 0x7f64e0058cf0 con 0x7f64f0099a00 2024-04-25T08:50:43.799 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 msgr2=0x7f64cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f64f006c070 tx=0x7f64d8009040 comp rx=0 tx=0).stop 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f0099a00 msgr2=0x7f64f00c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f0099a00 0x7f64f00c50b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c450 tx=0x7f64e000c910 comp rx=0 tx=0).stop 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 shutdown_connections 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f64cc066f60 0x7f64cc069420 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64f00ccde0 0x7f64f0153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f64f00b2240 0x7f64f0153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 --2- 172.21.15.46:0/356348772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64f0099a00 0x7f64f00c50b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 >> 172.21.15.46:0/356348772 conn(0x7f64f00b3120 msgr2=0x7f64f009aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:43.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 shutdown_connections 2024-04-25T08:50:43.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.811+0000 7f64f6200640 1 -- 172.21.15.46:0/356348772 wait complete. 2024-04-25T08:50:43.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:50:43.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa01e400640 1 Processor -- start 2024-04-25T08:50:43.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa01e400640 1 -- start start 2024-04-25T08:50:43.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa01e400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa018159230 0x7fa01815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:43.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa01e400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0180cbeb0 0x7fa0180cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:43.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa01e400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0180cc700 con 0x7fa0180cda80 2024-04-25T08:50:43.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa01e400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0180cc9e0 con 0x7fa018159230 2024-04-25T08:50:43.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa01e400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cccc0 con 0x7fa0180cbeb0 2024-04-25T08:50:43.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa017e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa0180cda80 0x7fa01815f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53822/0 (socket says 172.21.15.46:53822) 2024-04-25T08:50:43.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa017e00640 1 -- 172.21.15.46:0/2735249358 learned_addr learned my addr 172.21.15.46:0/2735249358 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa017400640 1 --2- 172.21.15.46:0/2735249358 >> v2:172.21.15.82:3300/0 conn(0x7fa018159230 0x7fa01815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa00fe00640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0180cbeb0 0x7fa0180cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119859250 0 0) 0x7fa0180cc700 con 0x7fa0180cda80 2024-04-25T08:50:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ffc003530 con 0x7fa0180cda80 2024-04-25T08:50:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1763695957 0 0) 0x7f9ffc003530 con 0x7fa0180cda80 2024-04-25T08:50:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0180cbeb0 msgr2=0x7fa0180cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa016a00640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0180cbeb0 0x7fa0180cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 >> v2:172.21.15.82:3300/0 conn(0x7fa018159230 msgr2=0x7fa01815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:43.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.983+0000 7fa016a00640 1 --2- 172.21.15.46:0/2735249358 >> v2:172.21.15.82:3300/0 conn(0x7fa018159230 0x7fa01815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180ccfa0 con 0x7fa0180cda80 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fa010001a00 con 0x7fa0180cda80 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 >> v1:172.21.15.46:6789/0 conn(0x7fa0180cda80 legacy=0x7fa01815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7f9ffc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ffc004810 0x7f9ffc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0180cbeb0 0x7f9ffc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cc9e0 con 0x7f9ffc0039c0 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cccc0 con 0x7f9ffc004810 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc00b9d0 con 0x7fa0180cbeb0 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa017400640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7f9ffc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa00fe00640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ffc004810 0x7f9ffc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa017e00640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0180cbeb0 0x7f9ffc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa017400640 1 -- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0180cbeb0 msgr2=0x7f9ffc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa017400640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0180cbeb0 0x7f9ffc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa017400640 1 -- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ffc004810 msgr2=0x7f9ffc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa017400640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ffc004810 0x7f9ffc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa017400640 1 -- 172.21.15.46:0/2735249358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc003530 con 0x7f9ffc0039c0 2024-04-25T08:50:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa017e00640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa0180cbeb0 0x7f9ffc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:43.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa017400640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7f9ffc003de0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa008009b10 tx=0x7fa00801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57d9c991d74462b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080096b0 con 0x7f9ffc0039c0 2024-04-25T08:50:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0080a64c0 con 0x7f9ffc0039c0 2024-04-25T08:50:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.987+0000 7fa016a00640 1 -- 172.21.15.46:0/2735249358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080a68b0 con 0x7f9ffc0039c0 2024-04-25T08:50:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 -- 172.21.15.46:0/2735249358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 msgr2=0x7f9ffc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:43.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 --2- 172.21.15.46:0/2735249358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7f9ffc003de0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fa008009b10 tx=0x7fa00801cab0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017e00640 1 -- 172.21.15.46:0/2735249358 reap_dead start 2024-04-25T08:50:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 -- 172.21.15.46:0/2735249358 shutdown_connections 2024-04-25T08:50:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 -- 172.21.15.46:0/2735249358 >> 172.21.15.46:0/2735249358 conn(0x7fa0180b12a0 msgr2=0x7fa0180b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 -- 172.21.15.46:0/2735249358 shutdown_connections 2024-04-25T08:50:43.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 -- 172.21.15.46:0/2735249358 wait complete. 2024-04-25T08:50:43.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 Processor -- start 2024-04-25T08:50:43.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 -- start start 2024-04-25T08:50:43.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7fa01813c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01806d250 0x7fa01806d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa01815a3c0 0x7fa01815a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa018166770 con 0x7f9ffc0039c0 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cc4a0 con 0x7fa01806d250 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa01e400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0181668f0 con 0x7fa01815a3c0 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7fa01813c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7fa01813c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43236/0 (socket says 172.21.15.46:43236) 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017400640 1 -- 172.21.15.46:0/790454198 learned_addr learned my addr 172.21.15.46:0/790454198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017e00640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa01815a3c0 0x7fa01815a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa00fe00640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01806d250 0x7fa01806d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017400640 1 -- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa01815a3c0 msgr2=0x7fa01815a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa01815a3c0 0x7fa01815a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017400640 1 -- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01806d250 msgr2=0x7fa01806d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01806d250 0x7fa01806d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.991+0000 7fa017400640 1 -- 172.21.15.46:0/790454198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01806dcd0 con 0x7f9ffc0039c0 2024-04-25T08:50:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.995+0000 7fa017400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7fa01813c720 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fa00801c8e0 tx=0x7f9ffc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.995+0000 7fa00f400640 1 -- 172.21.15.46:0/790454198 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080be020 con 0x7f9ffc0039c0 2024-04-25T08:50:43.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.995+0000 7fa00f400640 1 -- 172.21.15.46:0/790454198 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00800f000 con 0x7f9ffc0039c0 2024-04-25T08:50:43.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.995+0000 7fa00f400640 1 -- 172.21.15.46:0/790454198 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080a6ad0 con 0x7f9ffc0039c0 2024-04-25T08:50:43.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.995+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01806dfb0 con 0x7f9ffc0039c0 2024-04-25T08:50:43.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.995+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01806e570 con 0x7f9ffc0039c0 2024-04-25T08:50:43.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.995+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa018069080 con 0x7f9ffc0039c0 2024-04-25T08:50:43.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.995+0000 7fa00f400640 1 -- 172.21.15.46:0/790454198 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0080c1020 con 0x7f9ffc0039c0 2024-04-25T08:50:43.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.995+0000 7fa00f400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ff4066f60 0x7f9ff4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.999+0000 7fa00f400640 1 -- 172.21.15.46:0/790454198 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa00811f8a0 con 0x7f9ffc0039c0 2024-04-25T08:50:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.999+0000 7fa00fe00640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ff4066f60 0x7f9ff4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.999+0000 7fa00fe00640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ff4066f60 0x7f9ff4069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa000009be0 tx=0x7fa000009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:43.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:43.999+0000 7fa00f400640 1 -- 172.21.15.46:0/790454198 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0080c8030 con 0x7f9ffc0039c0 2024-04-25T08:50:44.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.219+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa0180723d0 con 0x7f9ffc0039c0 2024-04-25T08:50:44.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.219+0000 7fa00f400640 1 -- 172.21.15.46:0/790454198 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa008016800 con 0x7f9ffc0039c0 2024-04-25T08:50:44.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:44.211 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:50:44.211 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-25T08:50:44.211 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmq+gDGRAAtB5ObM/zelMmCZ4cJPghdQ==", 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmyIaAJRAA9zodtgftP7faNth4UPyCTA==", 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:50:44.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmFahWIBAAORHmzDLXPjuANF+/W2wKpQ==", 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmRhrPLBAATXOHl1jGJYmIlnYk/JPcOw==", 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA0FypmaHiqEhAAYzasCl8t+pls5IHe9tCP4g==", 2024-04-25T08:50:44.214 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA3FypmOGTuGhAA37F+qRI/+pj5dkLc3TBIUQ==", 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.215 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA6Fypm/YbNJRAAP1QyYDLY/xFvRz63R+H/Pw==", 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA9FypmiT8ALhAAX/2CJJeJ+2SGNhwah8NTBQ==", 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:50:44.216 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBDFypmyG88JBAAq1065gzLIIR/6fYaw0bBTg==", 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBGFypmNir3KxAAszCXJJ1KFtCusDZhBHAZSQ==", 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBJFypm4r7LNxAA42A07ntI6OOgblk+4i1Yiw==", 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.218 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBNFypmG1fOBhAAOKRgw04jIo8H3u2hWIZe2g==", 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.219 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q==", 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmxPyHOhAADBuigZlq3GHURsJkB0nnyA==", 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:50:44.220 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypm2PP0NhAAixV/kkY7eFPXUW/nJDOCOA==", 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.221 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAzFypmaVDtBBAAKnWk9CxZzg2r6T/IycuFEg==", 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAxFypm2W2MOBAAN7TDQiZXFV6+pWMTar95sg==", 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:50:44.222 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmzhptFBAAlFQZQn1GSseEguy5zfw2CA==", 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:44.223 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-25T08:50:44.224 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmAY6CEBAA2qHNALNTRKmCJRZatDqAvw==", 2024-04-25T08:50:44.224 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:50:44.224 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:50:44.224 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:50:44.224 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:50:44.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.224 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:44.224 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-25T08:50:44.224 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:50:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ff4066f60 msgr2=0x7f9ff4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ff4066f60 0x7f9ff4069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa000009be0 tx=0x7fa000009190 comp rx=0 tx=0).stop 2024-04-25T08:50:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 msgr2=0x7fa01813c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7fa01813c720 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fa00801c8e0 tx=0x7f9ffc005840 comp rx=0 tx=0).stop 2024-04-25T08:50:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 shutdown_connections 2024-04-25T08:50:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9ff4066f60 0x7f9ff4069420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa01815a3c0 0x7fa01815a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa01806d250 0x7fa01806d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 --2- 172.21.15.46:0/790454198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc0039c0 0x7fa01813c720 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 >> 172.21.15.46:0/790454198 conn(0x7fa0180b12a0 msgr2=0x7fa01815c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 shutdown_connections 2024-04-25T08:50:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.223+0000 7fa01e400640 1 -- 172.21.15.46:0/790454198 wait complete. 2024-04-25T08:50:44.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:44.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.383+0000 7f0266000640 1 Processor -- start 2024-04-25T08:50:44.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.383+0000 7f0266000640 1 -- start start 2024-04-25T08:50:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0266000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f02600cf140 0x7f026015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0266000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02600afd40 0x7f026015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0266000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 0x7f02600ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0266000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02600afb40 con 0x7f02600afd40 2024-04-25T08:50:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0266000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026006ed70 con 0x7f02600cf140 2024-04-25T08:50:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0266000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0260162530 con 0x7f02600cf520 2024-04-25T08:50:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0256a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02600afd40 0x7f026015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0256a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f02600afd40 0x7f026015dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43250/0 (socket says 172.21.15.46:43250) 2024-04-25T08:50:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0256a00640 1 -- 172.21.15.46:0/1924656541 learned_addr learned my addr 172.21.15.46:0/1924656541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f025f400640 1 --2- 172.21.15.46:0/1924656541 >> v2:172.21.15.82:3300/0 conn(0x7f02600cf140 0x7f026015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f025fe00640 1 --2- 172.21.15.46:0/1924656541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 0x7f02600ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0256a00640 1 -- 172.21.15.46:0/1924656541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 msgr2=0x7f02600ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0256a00640 1 --2- 172.21.15.46:0/1924656541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 0x7f02600ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0256a00640 1 -- 172.21.15.46:0/1924656541 >> v2:172.21.15.82:3300/0 conn(0x7f02600cf140 msgr2=0x7f026015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0256a00640 1 --2- 172.21.15.46:0/1924656541 >> v2:172.21.15.82:3300/0 conn(0x7f02600cf140 0x7f026015d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0256a00640 1 -- 172.21.15.46:0/1924656541 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0260161770 con 0x7f02600afd40 2024-04-25T08:50:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.387+0000 7f0256a00640 1 --2- 172.21.15.46:0/1924656541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600afd40 0x7f026015dad0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0260073310 tx=0x7f024801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef3588058a3b4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f025ea00640 1 -- 172.21.15.46:0/1924656541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480b0070 con 0x7f02600afd40 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f025ea00640 1 -- 172.21.15.46:0/1924656541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02480ab600 con 0x7f02600afd40 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f025ea00640 1 -- 172.21.15.46:0/1924656541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480ab9d0 con 0x7f02600afd40 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 -- 172.21.15.46:0/1924656541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600afd40 msgr2=0x7f026015dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 --2- 172.21.15.46:0/1924656541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600afd40 0x7f026015dad0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0260073310 tx=0x7f024801c1d0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 -- 172.21.15.46:0/1924656541 shutdown_connections 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 --2- 172.21.15.46:0/1924656541 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 0x7f02600ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 --2- 172.21.15.46:0/1924656541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600afd40 0x7f026015dad0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 --2- 172.21.15.46:0/1924656541 >> v2:172.21.15.82:3300/0 conn(0x7f02600cf140 0x7f026015d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 -- 172.21.15.46:0/1924656541 >> 172.21.15.46:0/1924656541 conn(0x7f02600728c0 msgr2=0x7f0260099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 -- 172.21.15.46:0/1924656541 shutdown_connections 2024-04-25T08:50:44.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 -- 172.21.15.46:0/1924656541 wait complete. 2024-04-25T08:50:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 Processor -- start 2024-04-25T08:50:44.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 -- start start 2024-04-25T08:50:44.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02600afd40 0x7f0260152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600cf140 0x7f02601495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 0x7f02600b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026006b600 con 0x7f02600cf140 2024-04-25T08:50:44.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02600af7f0 con 0x7f02600afd40 2024-04-25T08:50:44.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0266000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026006b780 con 0x7f02600cf520 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0256a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600cf140 0x7f02601495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0256a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600cf140 0x7f02601495b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43252/0 (socket says 172.21.15.46:43252) 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f0256a00640 1 -- 172.21.15.46:0/1681617493 learned_addr learned my addr 172.21.15.46:0/1681617493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f025fe00640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 0x7f02600b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.391+0000 7f025f400640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02600afd40 0x7f0260152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0256a00640 1 -- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 msgr2=0x7f02600b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0256a00640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 0x7f02600b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0256a00640 1 -- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02600afd40 msgr2=0x7f0260152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0256a00640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02600afd40 0x7f0260152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0256a00640 1 -- 172.21.15.46:0/1681617493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0260152b20 con 0x7f02600cf140 2024-04-25T08:50:44.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0256a00640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600cf140 0x7f02601495b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f024801ca70 tx=0x7f0248018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0257e00640 1 -- 172.21.15.46:0/1681617493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480b0040 con 0x7f02600cf140 2024-04-25T08:50:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0257e00640 1 -- 172.21.15.46:0/1681617493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02480b4640 con 0x7f02600cf140 2024-04-25T08:50:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0257e00640 1 -- 172.21.15.46:0/1681617493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480b4a30 con 0x7f02600cf140 2024-04-25T08:50:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02600b4fc0 con 0x7f02600cf140 2024-04-25T08:50:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02600b5580 con 0x7f02600cf140 2024-04-25T08:50:44.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026006ccb0 con 0x7f02600cf140 2024-04-25T08:50:44.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0257e00640 1 -- 172.21.15.46:0/1681617493 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02480ab4c0 con 0x7f02600cf140 2024-04-25T08:50:44.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0257e00640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f023c066f60 0x7f023c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.395+0000 7f0257e00640 1 -- 172.21.15.46:0/1681617493 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f024811e810 con 0x7f02600cf140 2024-04-25T08:50:44.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.399+0000 7f025f400640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f023c066f60 0x7f023c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.399+0000 7f025f400640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f023c066f60 0x7f023c069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f02600b4c10 tx=0x7f0250008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:44.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.399+0000 7f0257e00640 1 -- 172.21.15.46:0/1681617493 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0248016110 con 0x7f02600cf140 2024-04-25T08:50:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.563+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0260072f90 con 0x7f02600cf140 2024-04-25T08:50:44.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.563+0000 7f0257e00640 1 -- 172.21.15.46:0/1681617493 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+13118 (secure 0 0 0) 0x7f02480ef540 con 0x7f02600cf140 2024-04-25T08:50:44.556 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:44.556 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":102,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:50:43.808034+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f023c066f60 msgr2=0x7f023c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f023c066f60 0x7f023c069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f02600b4c10 tx=0x7f0250008040 comp rx=0 tx=0).stop 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600cf140 msgr2=0x7f02601495b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600cf140 0x7f02601495b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f024801ca70 tx=0x7f0248018f70 comp rx=0 tx=0).stop 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 shutdown_connections 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f023c066f60 0x7f023c069420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f02600cf520 0x7f02600b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02600cf140 0x7f02601495b0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 --2- 172.21.15.46:0/1681617493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02600afd40 0x7f0260152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 >> 172.21.15.46:0/1681617493 conn(0x7f02600728c0 msgr2=0x7f0260157010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 shutdown_connections 2024-04-25T08:50:44.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.567+0000 7f0266000640 1 -- 172.21.15.46:0/1681617493 wait complete. 2024-04-25T08:50:44.577 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:50:44.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:50:44.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.731+0000 7fde76000640 1 Processor -- start 2024-04-25T08:50:44.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.731+0000 7fde76000640 1 -- start start 2024-04-25T08:50:44.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.731+0000 7fde76000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 0x7fde700cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.731+0000 7fde76000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde7006f150 0x7fde700b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.731+0000 7fde76000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde7015e440 con 0x7fde70143990 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.731+0000 7fde76000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7015ed30 con 0x7fde7006f150 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.731+0000 7fde76000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7015e6e0 con 0x7fde701440d0 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde7006f150 0x7fde700b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde7006f150 0x7fde700b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43258/0 (socket says 172.21.15.46:43258) 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6fe00640 1 -- 172.21.15.46:0/1561414653 learned_addr learned my addr 172.21.15.46:0/1561414653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fde70143990 0x7fde7006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53838/0 (socket says 172.21.15.46:53838) 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde67e00640 1 --2- 172.21.15.46:0/1561414653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 0x7fde700cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6fe00640 1 -- 172.21.15.46:0/1561414653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 msgr2=0x7fde700cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6fe00640 1 --2- 172.21.15.46:0/1561414653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 0x7fde700cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6fe00640 1 -- 172.21.15.46:0/1561414653 >> v1:172.21.15.46:6789/0 conn(0x7fde70143990 legacy=0x7fde7006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6fe00640 1 -- 172.21.15.46:0/1561414653 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7015e9c0 con 0x7fde7006f150 2024-04-25T08:50:44.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6ea00640 1 -- 172.21.15.46:0/1561414653 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279888415 0 0) 0x7fde7015e440 con 0x7fde70143990 2024-04-25T08:50:44.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6fe00640 1 --2- 172.21.15.46:0/1561414653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7006f150 0x7fde700b2720 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fde68009e50 tx=0x7fde6801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9837cfe60f87322a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:44.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6ea00640 1 -- 172.21.15.46:0/1561414653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde680a5660 con 0x7fde7006f150 2024-04-25T08:50:44.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6ea00640 1 -- 172.21.15.46:0/1561414653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde680a5800 con 0x7fde7006f150 2024-04-25T08:50:44.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.735+0000 7fde6ea00640 1 -- 172.21.15.46:0/1561414653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde680a5bd0 con 0x7fde7006f150 2024-04-25T08:50:44.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 -- 172.21.15.46:0/1561414653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7006f150 msgr2=0x7fde700b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 --2- 172.21.15.46:0/1561414653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7006f150 0x7fde700b2720 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fde68009e50 tx=0x7fde6801c9a0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 -- 172.21.15.46:0/1561414653 shutdown_connections 2024-04-25T08:50:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 --2- 172.21.15.46:0/1561414653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde7006f150 0x7fde700b2720 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 --2- 172.21.15.46:0/1561414653 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 0x7fde700cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 -- 172.21.15.46:0/1561414653 >> 172.21.15.46:0/1561414653 conn(0x7fde7006ac10 msgr2=0x7fde7006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 -- 172.21.15.46:0/1561414653 shutdown_connections 2024-04-25T08:50:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 -- 172.21.15.46:0/1561414653 wait complete. 2024-04-25T08:50:44.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 Processor -- start 2024-04-25T08:50:44.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 -- start start 2024-04-25T08:50:44.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7006f150 0x7fde701578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde70143990 0x7fde70159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 0x7fde70136790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde700b35f0 con 0x7fde70143990 2024-04-25T08:50:44.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde700b3470 con 0x7fde7006f150 2024-04-25T08:50:44.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde76000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde700b3770 con 0x7fde701440d0 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde70143990 0x7fde70159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde70143990 0x7fde70159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43268/0 (socket says 172.21.15.46:43268) 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde67e00640 1 -- 172.21.15.46:0/3111489837 learned_addr learned my addr 172.21.15.46:0/3111489837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde6f400640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7006f150 0x7fde701578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde6fe00640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 0x7fde70136790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde67e00640 1 -- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 msgr2=0x7fde70136790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde67e00640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 0x7fde70136790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde67e00640 1 -- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7006f150 msgr2=0x7fde701578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde67e00640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7006f150 0x7fde701578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde67e00640 1 -- 172.21.15.46:0/3111489837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70137070 con 0x7fde70143990 2024-04-25T08:50:44.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.739+0000 7fde67e00640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde70143990 0x7fde70159660 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e960 tx=0x7fde5800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:44.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde67400640 1 -- 172.21.15.46:0/3111489837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58017070 con 0x7fde70143990 2024-04-25T08:50:44.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde67400640 1 -- 172.21.15.46:0/3111489837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5800ca00 con 0x7fde70143990 2024-04-25T08:50:44.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde67400640 1 -- 172.21.15.46:0/3111489837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5800cdb0 con 0x7fde70143990 2024-04-25T08:50:44.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde70137350 con 0x7fde70143990 2024-04-25T08:50:44.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde70159ba0 con 0x7fde70143990 2024-04-25T08:50:44.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde70069010 con 0x7fde70143990 2024-04-25T08:50:44.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde67400640 1 -- 172.21.15.46:0/3111489837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde5801a070 con 0x7fde70143990 2024-04-25T08:50:44.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde67400640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde4c066f90 0x7fde4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:44.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde67400640 1 -- 172.21.15.46:0/3111489837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fde580851a0 con 0x7fde70143990 2024-04-25T08:50:44.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde6f400640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde4c066f90 0x7fde4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:44.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.743+0000 7fde6f400640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde4c066f90 0x7fde4c069450 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde60007fd0 tx=0x7fde60007d60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:44.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.747+0000 7fde67400640 1 -- 172.21.15.46:0/3111489837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde58055d20 con 0x7fde70143990 2024-04-25T08:50:44.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde70138730 con 0x7fde70143990 2024-04-25T08:50:44.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde67400640 1 -- 172.21.15.46:0/3111489837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde58059dc0 con 0x7fde70143990 2024-04-25T08:50:44.902 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde4c066f90 msgr2=0x7fde4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde4c066f90 0x7fde4c069450 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fde60007fd0 tx=0x7fde60007d60 comp rx=0 tx=0).stop 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde70143990 msgr2=0x7fde70159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde70143990 0x7fde70159660 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e960 tx=0x7fde5800ee20 comp rx=0 tx=0).stop 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 shutdown_connections 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fde4c066f90 0x7fde4c069450 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fde701440d0 0x7fde70136790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde70143990 0x7fde70159660 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 --2- 172.21.15.46:0/3111489837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde7006f150 0x7fde701578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 >> 172.21.15.46:0/3111489837 conn(0x7fde7006ac10 msgr2=0x7fde7013b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 shutdown_connections 2024-04-25T08:50:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:44.911+0000 7fde76000640 1 -- 172.21.15.46:0/3111489837 wait complete. 2024-04-25T08:50:44.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:50:45.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0da000640 1 Processor -- start 2024-04-25T08:50:45.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0da000640 1 -- start start 2024-04-25T08:50:45.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0da000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff0d415a750 0x7ff0d406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0da000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0d40cc9e0 0x7ff0d406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0da000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d406d5e0 con 0x7ff0d40aed10 2024-04-25T08:50:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0da000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d415ab30 con 0x7ff0d415a750 2024-04-25T08:50:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0da000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40ce4f0 con 0x7ff0d40cc9e0 2024-04-25T08:50:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff0d40aed10 0x7ff0d40b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53852/0 (socket says 172.21.15.46:53852) 2024-04-25T08:50:45.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2a00640 1 -- 172.21.15.46:0/688863754 learned_addr learned my addr 172.21.15.46:0/688863754 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d3e00640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0d40cc9e0 0x7ff0d406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d3400640 1 --2- 172.21.15.46:0/688863754 >> v2:172.21.15.82:3300/0 conn(0x7ff0d415a750 0x7ff0d406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722987726 0 0) 0x7ff0d406d5e0 con 0x7ff0d40aed10 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b8003530 con 0x7ff0d40aed10 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1260734133 0 0) 0x7ff0b8003530 con 0x7ff0d40aed10 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0d40cc9e0 msgr2=0x7ff0d406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0d40cc9e0 0x7ff0d406f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 >> v2:172.21.15.82:3300/0 conn(0x7ff0d415a750 msgr2=0x7ff0d406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 --2- 172.21.15.46:0/688863754 >> v2:172.21.15.82:3300/0 conn(0x7ff0d415a750 0x7ff0d406ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d40ce720 con 0x7ff0d40aed10 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7ff0c0001680 con 0x7ff0d40aed10 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 >> v1:172.21.15.46:6789/0 conn(0x7ff0d40aed10 legacy=0x7ff0d40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0b80039c0 0x7ff0b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b8004810 0x7ff0b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0d40cc9e0 0x7ff0b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b800b9d0 con 0x7ff0b8004810 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b800bb50 con 0x7ff0b80039c0 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b800bcd0 con 0x7ff0d40cc9e0 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2a00640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b8004810 0x7ff0b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d3400640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0b80039c0 0x7ff0b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d3e00640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0d40cc9e0 0x7ff0b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2a00640 1 -- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0d40cc9e0 msgr2=0x7ff0b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2a00640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0d40cc9e0 0x7ff0b80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2a00640 1 -- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0b80039c0 msgr2=0x7ff0b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:45.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2a00640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0b80039c0 0x7ff0b8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.075+0000 7ff0d2a00640 1 -- 172.21.15.46:0/688863754 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b8003530 con 0x7ff0b8004810 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0d2a00640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b8004810 0x7ff0b8004c90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40af180 tx=0x7ff0c001f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fa02c173b12d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0007c90 con 0x7ff0b8004810 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c00ac020 con 0x7ff0b8004810 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0d2000640 1 -- 172.21.15.46:0/688863754 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00a8e30 con 0x7ff0b8004810 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0da000640 1 -- 172.21.15.46:0/688863754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b8004810 msgr2=0x7ff0b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0da000640 1 --2- 172.21.15.46:0/688863754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0b8004810 0x7ff0b8004c90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40af180 tx=0x7ff0c001f2b0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0d3e00640 1 -- 172.21.15.46:0/688863754 reap_dead start 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0da000640 1 -- 172.21.15.46:0/688863754 shutdown_connections 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0da000640 1 -- 172.21.15.46:0/688863754 >> 172.21.15.46:0/688863754 conn(0x7ff0d40b12a0 msgr2=0x7ff0d40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0da000640 1 -- 172.21.15.46:0/688863754 shutdown_connections 2024-04-25T08:50:45.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.079+0000 7ff0da000640 1 -- 172.21.15.46:0/688863754 wait complete. 2024-04-25T08:50:45.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 Processor -- start 2024-04-25T08:50:45.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 -- start start 2024-04-25T08:50:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0b8004810 0x7ff0d414b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d413eae0 0x7ff0d4148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0d4149710 0x7ff0d4142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40cece0 con 0x7ff0d413eae0 2024-04-25T08:50:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d406d8c0 con 0x7ff0d4149710 2024-04-25T08:50:45.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d40cf200 con 0x7ff0b8004810 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d413eae0 0x7ff0d4148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d413eae0 0x7ff0d4148230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43288/0 (socket says 172.21.15.46:43288) 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d2a00640 1 -- 172.21.15.46:0/2124295570 learned_addr learned my addr 172.21.15.46:0/2124295570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d3400640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0b8004810 0x7ff0d414b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d3e00640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0d4149710 0x7ff0d4142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d2a00640 1 -- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0b8004810 msgr2=0x7ff0d414b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d2a00640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0b8004810 0x7ff0d414b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d2a00640 1 -- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0d4149710 msgr2=0x7ff0d4142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d2a00640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0d4149710 0x7ff0d4142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d2a00640 1 -- 172.21.15.46:0/2124295570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d414c750 con 0x7ff0d413eae0 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d3400640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0b8004810 0x7ff0d414b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:45.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0d2a00640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d413eae0 0x7ff0d4148230 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0007040 tx=0x7ff0b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0cb400640 1 -- 172.21.15.46:0/2124295570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0004ab0 con 0x7ff0d413eae0 2024-04-25T08:50:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0cb400640 1 -- 172.21.15.46:0/2124295570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0004c50 con 0x7ff0d413eae0 2024-04-25T08:50:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d414a200 con 0x7ff0d413eae0 2024-04-25T08:50:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0cb400640 1 -- 172.21.15.46:0/2124295570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00070d0 con 0x7ff0d413eae0 2024-04-25T08:50:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d414c190 con 0x7ff0d413eae0 2024-04-25T08:50:45.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.083+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d4069080 con 0x7ff0d413eae0 2024-04-25T08:50:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.087+0000 7ff0cb400640 1 -- 172.21.15.46:0/2124295570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0c0007270 con 0x7ff0d413eae0 2024-04-25T08:50:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.087+0000 7ff0cb400640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff0b4066f90 0x7ff0b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.087+0000 7ff0cb400640 1 -- 172.21.15.46:0/2124295570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff0c01224d0 con 0x7ff0d413eae0 2024-04-25T08:50:45.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.087+0000 7ff0d3400640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff0b4066f90 0x7ff0b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.087+0000 7ff0d3400640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff0b4066f90 0x7ff0b4069450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009be0 tx=0x7ff0c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:45.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.091+0000 7ff0cb400640 1 -- 172.21.15.46:0/2124295570 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff0d414a200 con 0x7ff0d413eae0 2024-04-25T08:50:45.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff0d40b25b0 con 0x7ff0d413eae0 2024-04-25T08:50:45.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0cb400640 1 -- 172.21.15.46:0/2124295570 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v102) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff0c0019020 con 0x7ff0d413eae0 2024-04-25T08:50:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:45.245 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:50:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:50:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-25T08:50:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-25T08:50:45.245 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:50:45.245 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:50:45.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff0b4066f90 msgr2=0x7ff0b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff0b4066f90 0x7ff0b4069450 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009be0 tx=0x7ff0c4008040 comp rx=0 tx=0).stop 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d413eae0 msgr2=0x7ff0d4148230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d413eae0 0x7ff0d4148230 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0007040 tx=0x7ff0b8005840 comp rx=0 tx=0).stop 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 shutdown_connections 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff0b4066f90 0x7ff0b4069450 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0d4149710 0x7ff0d4142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0d413eae0 0x7ff0d4148230 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 --2- 172.21.15.46:0/2124295570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff0b8004810 0x7ff0d414b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 >> 172.21.15.46:0/2124295570 conn(0x7ff0d40b12a0 msgr2=0x7ff0d406e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 shutdown_connections 2024-04-25T08:50:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.255+0000 7ff0da000640 1 -- 172.21.15.46:0/2124295570 wait complete. 2024-04-25T08:50:45.265 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:50:45.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:50:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.419+0000 7f84f4800640 1 Processor -- start 2024-04-25T08:50:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.419+0000 7f84f4800640 1 -- start start 2024-04-25T08:50:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f4800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f84ec13e6b0 0x7f84ec0711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f4800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec15bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f4800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ec06d330 con 0x7f84ec15c4b0 2024-04-25T08:50:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f4800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84ec15cce0 con 0x7f84ec13e6b0 2024-04-25T08:50:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f4800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec06d610 con 0x7f84ec071720 2024-04-25T08:50:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f1e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f84ec13e6b0 0x7f84ec0711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f1400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec15bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f1e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f84ec13e6b0 0x7f84ec0711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:52330/0 (socket says 172.21.15.46:52330) 2024-04-25T08:50:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f1e00640 1 -- 172.21.15.46:0/2126315757 learned_addr learned my addr 172.21.15.46:0/2126315757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f1400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec15bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:54236/0 (socket says 172.21.15.46:54236) 2024-04-25T08:50:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f1400640 1 -- 172.21.15.46:0/2126315757 >> v2:172.21.15.82:3300/0 conn(0x7f84ec13e6b0 msgr2=0x7f84ec0711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:45.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f1400640 1 --2- 172.21.15.46:0/2126315757 >> v2:172.21.15.82:3300/0 conn(0x7f84ec13e6b0 0x7f84ec0711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f0a00640 1 -- 172.21.15.46:0/2126315757 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467702238 0 0) 0x7f84ec06d330 con 0x7f84ec15c4b0 2024-04-25T08:50:45.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f1400640 1 -- 172.21.15.46:0/2126315757 >> v1:172.21.15.82:6789/0 conn(0x7f84ec15c4b0 legacy=0x7f84ec0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.423+0000 7f84f1400640 1 -- 172.21.15.46:0/2126315757 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec06d8f0 con 0x7f84ec071720 2024-04-25T08:50:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f1400640 1 --2- 172.21.15.46:0/2126315757 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec15bf70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0097c0 tx=0x7f84dc01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4cf188d3d70480f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f0a00640 1 -- 172.21.15.46:0/2126315757 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0b0070 con 0x7f84ec071720 2024-04-25T08:50:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f0a00640 1 -- 172.21.15.46:0/2126315757 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc0ab600 con 0x7f84ec071720 2024-04-25T08:50:45.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f0a00640 1 -- 172.21.15.46:0/2126315757 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0ab9d0 con 0x7f84ec071720 2024-04-25T08:50:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 -- 172.21.15.46:0/2126315757 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 msgr2=0x7f84ec15bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 --2- 172.21.15.46:0/2126315757 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec15bf70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0097c0 tx=0x7f84dc01c380 comp rx=0 tx=0).stop 2024-04-25T08:50:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 -- 172.21.15.46:0/2126315757 shutdown_connections 2024-04-25T08:50:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 --2- 172.21.15.46:0/2126315757 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec15bf70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 --2- 172.21.15.46:0/2126315757 >> v2:172.21.15.82:3300/0 conn(0x7f84ec13e6b0 0x7f84ec0711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 -- 172.21.15.46:0/2126315757 >> 172.21.15.46:0/2126315757 conn(0x7f84ec067050 msgr2=0x7f84ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:45.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 -- 172.21.15.46:0/2126315757 shutdown_connections 2024-04-25T08:50:45.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 -- 172.21.15.46:0/2126315757 wait complete. 2024-04-25T08:50:45.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 Processor -- start 2024-04-25T08:50:45.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 -- start start 2024-04-25T08:50:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ec13e6b0 0x7f84ec165b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84ec15c4b0 0x7f84ec13a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec168380 con 0x7f84ec13e6b0 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec168500 con 0x7f84ec15c4b0 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.427+0000 7f84f4800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec168680 con 0x7f84ec071720 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ec13e6b0 0x7f84ec165b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ec13e6b0 0x7f84ec165b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43294/0 (socket says 172.21.15.46:43294) 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1400640 1 -- 172.21.15.46:0/848648917 learned_addr learned my addr 172.21.15.46:0/848648917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f2800640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84ec15c4b0 0x7f84ec13a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1400640 1 -- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 msgr2=0x7f84ec155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1400640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1400640 1 -- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84ec15c4b0 msgr2=0x7f84ec13a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1400640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84ec15c4b0 0x7f84ec13a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1400640 1 -- 172.21.15.46:0/848648917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec160280 con 0x7f84ec13e6b0 2024-04-25T08:50:45.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1e00640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:45.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f1400640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ec13e6b0 0x7f84ec165b10 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f84dc019fb0 tx=0x7f84dc019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:45.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84e6000640 1 -- 172.21.15.46:0/848648917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0b8020 con 0x7f84ec13e6b0 2024-04-25T08:50:45.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84e6000640 1 -- 172.21.15.46:0/848648917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc0b4410 con 0x7f84ec13e6b0 2024-04-25T08:50:45.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ec160560 con 0x7f84ec13e6b0 2024-04-25T08:50:45.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84e6000640 1 -- 172.21.15.46:0/848648917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0b0040 con 0x7f84ec13e6b0 2024-04-25T08:50:45.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ec139b10 con 0x7f84ec13e6b0 2024-04-25T08:50:45.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ec069080 con 0x7f84ec13e6b0 2024-04-25T08:50:45.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.431+0000 7f84e6000640 1 -- 172.21.15.46:0/848648917 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84dc0bb020 con 0x7f84ec13e6b0 2024-04-25T08:50:45.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.435+0000 7f84e6000640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f84d0066f60 0x7f84d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:45.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.435+0000 7f84e6000640 1 -- 172.21.15.46:0/848648917 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f84dc11e780 con 0x7f84ec13e6b0 2024-04-25T08:50:45.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.435+0000 7f84f1e00640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f84d0066f60 0x7f84d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:45.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.435+0000 7f84f1e00640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f84d0066f60 0x7f84d0069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f84ec15b220 tx=0x7f84e0009240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:45.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.439+0000 7f84e6000640 1 -- 172.21.15.46:0/848648917 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84dc0c2030 con 0x7f84ec13e6b0 2024-04-25T08:50:45.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:45.599+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f84ec15bc90 con 0x7f84ec13e6b0 2024-04-25T08:50:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.443+0000 7f84e6000640 1 -- 172.21.15.46:0/848648917 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v103) v1 ==== 138+0+0 (secure 0 0 0) 0x7f84dc016090 con 0x7f84ec13e6b0 2024-04-25T08:50:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-25T08:50:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f84d0066f60 msgr2=0x7f84d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f84d0066f60 0x7f84d0069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f84ec15b220 tx=0x7f84e0009240 comp rx=0 tx=0).stop 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ec13e6b0 msgr2=0x7f84ec165b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ec13e6b0 0x7f84ec165b10 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f84dc019fb0 tx=0x7f84dc019120 comp rx=0 tx=0).stop 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 shutdown_connections 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f84d0066f60 0x7f84d0069420 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84ec15c4b0 0x7f84ec13a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84ec13e6b0 0x7f84ec165b10 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 --2- 172.21.15.46:0/848648917 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f84ec071720 0x7f84ec155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 >> 172.21.15.46:0/848648917 conn(0x7f84ec067050 msgr2=0x7f84ec0cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 shutdown_connections 2024-04-25T08:50:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.447+0000 7f84f4800640 1 -- 172.21.15.46:0/848648917 wait complete. 2024-04-25T08:50:46.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:50:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.627+0000 7efdf0400640 1 Processor -- start 2024-04-25T08:50:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.627+0000 7efdf0400640 1 -- start start 2024-04-25T08:50:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdf0400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde815d900 0x7efde80af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdf0400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde80b0040 con 0x7efde80afc20 2024-04-25T08:50:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdf0400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde806fca0 con 0x7efde806b470 2024-04-25T08:50:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdf0400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde806ff80 con 0x7efde815d900 2024-04-25T08:50:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efded000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efde80afc20 0x7efde806ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53864/0 (socket says 172.21.15.46:53864) 2024-04-25T08:50:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efded000640 1 -- 172.21.15.46:0/733289026 learned_addr learned my addr 172.21.15.46:0/733289026 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdeda00640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde815d900 0x7efde80af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:46.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245926024 0 0) 0x7efde80b0040 con 0x7efde80afc20 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd0003530 con 0x7efde80afc20 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3294200046 0 0) 0x7efdd0003530 con 0x7efde80afc20 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde815d900 msgr2=0x7efde80af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde815d900 0x7efde80af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 >> v1:172.21.15.82:6789/0 conn(0x7efde806b470 legacy=0x7efde80d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde8070260 con 0x7efde80afc20 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7efdd4001680 con 0x7efde80afc20 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 >> v1:172.21.15.46:6789/0 conn(0x7efde80afc20 legacy=0x7efde806ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efdd0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdd0004810 0x7efdd0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde815d900 0x7efdd0005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde806ff80 con 0x7efdd0003a60 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd000bb90 con 0x7efdd0004810 2024-04-25T08:50:46.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd000bd90 con 0x7efde815d900 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdeda00640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efdd0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efded000640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdd0004810 0x7efdd0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdee400640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde815d900 0x7efdd0005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdeda00640 1 -- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde815d900 msgr2=0x7efdd0005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdeda00640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde815d900 0x7efdd0005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdeda00640 1 -- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdd0004810 msgr2=0x7efdd0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdeda00640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdd0004810 0x7efdd0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdeda00640 1 -- 172.21.15.46:0/733289026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0003530 con 0x7efdd0003a60 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.631+0000 7efdee400640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde815d900 0x7efdd0005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:46.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdeda00640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efdd0003e80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7efddc009ab0 tx=0x7efddc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70ee6a00a555336 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0096b0 con 0x7efdd0003a60 2024-04-25T08:50:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc0a6430 con 0x7efdd0003a60 2024-04-25T08:50:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efde7e00640 1 -- 172.21.15.46:0/733289026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0a6820 con 0x7efdd0003a60 2024-04-25T08:50:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdf0400640 1 -- 172.21.15.46:0/733289026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 msgr2=0x7efdd0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdf0400640 1 --2- 172.21.15.46:0/733289026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efdd0003e80 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7efddc009ab0 tx=0x7efddc01ca50 comp rx=0 tx=0).stop 2024-04-25T08:50:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdee400640 1 -- 172.21.15.46:0/733289026 reap_dead start 2024-04-25T08:50:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdf0400640 1 -- 172.21.15.46:0/733289026 shutdown_connections 2024-04-25T08:50:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdf0400640 1 -- 172.21.15.46:0/733289026 >> 172.21.15.46:0/733289026 conn(0x7efde806e3e0 msgr2=0x7efde806e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdf0400640 1 -- 172.21.15.46:0/733289026 shutdown_connections 2024-04-25T08:50:46.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdf0400640 1 -- 172.21.15.46:0/733289026 wait complete. 2024-04-25T08:50:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdf0400640 1 Processor -- start 2024-04-25T08:50:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.635+0000 7efdf0400640 1 -- start start 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdf0400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efde80b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdf0400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde8154cf0 0x7efde8155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdf0400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde814dcb0 0x7efde814e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdf0400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80d12b0 con 0x7efdd0003a60 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdf0400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80d1130 con 0x7efde814dcb0 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdf0400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde80d1430 con 0x7efde8154cf0 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdeda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efde80b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdeda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efde80b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43310/0 (socket says 172.21.15.46:43310) 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdeda00640 1 -- 172.21.15.46:0/4144520851 learned_addr learned my addr 172.21.15.46:0/4144520851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:46.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efded000640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde8154cf0 0x7efde8155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdee400640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde814dcb0 0x7efde814e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdeda00640 1 -- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde8154cf0 msgr2=0x7efde8155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdeda00640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde8154cf0 0x7efde8155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdeda00640 1 -- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde814dcb0 msgr2=0x7efde814e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdeda00640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde814dcb0 0x7efde814e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdeda00640 1 -- 172.21.15.46:0/4144520851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde814e650 con 0x7efdd0003a60 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efded000640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde8154cf0 0x7efde8155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdee400640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde814dcb0 0x7efde814e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdeda00640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efde80b2a80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efddc009ab0 tx=0x7efddc0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efde5600640 1 -- 172.21.15.46:0/4144520851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0a6c60 con 0x7efdd0003a60 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efde5600640 1 -- 172.21.15.46:0/4144520851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc0a6e00 con 0x7efdd0003a60 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde81557b0 con 0x7efdd0003a60 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efde5600640 1 -- 172.21.15.46:0/4144520851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0b9660 con 0x7efdd0003a60 2024-04-25T08:50:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde8155d70 con 0x7efdd0003a60 2024-04-25T08:50:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.639+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde8069010 con 0x7efdd0003a60 2024-04-25T08:50:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.643+0000 7efde5600640 1 -- 172.21.15.46:0/4144520851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efddc0c2440 con 0x7efdd0003a60 2024-04-25T08:50:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.643+0000 7efde5600640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdc8066f90 0x7efdc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.643+0000 7efde5600640 1 -- 172.21.15.46:0/4144520851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5619+0+0 (secure 0 0 0) 0x7efddc11f180 con 0x7efdd0003a60 2024-04-25T08:50:46.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.643+0000 7efded000640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdc8066f90 0x7efdc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:46.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.643+0000 7efded000640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdc8066f90 0x7efdc8069450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efde8156e00 tx=0x7efdd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.647+0000 7efde5600640 1 -- 172.21.15.46:0/4144520851 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efddc0ac020 con 0x7efdd0003a60 2024-04-25T08:50:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:46.807+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efde8072890 con 0x7efdd0003a60 2024-04-25T08:50:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.451+0000 7efde5600640 1 -- 172.21.15.46:0/4144520851 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v104) v1 ==== 170+0+0 (secure 0 0 0) 0x7efddc016760 con 0x7efdd0003a60 2024-04-25T08:50:47.444 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-25T08:50:47.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.455+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdc8066f90 msgr2=0x7efdc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:47.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.455+0000 7efdf0400640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdc8066f90 0x7efdc8069450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efde8156e00 tx=0x7efdd4009040 comp rx=0 tx=0).stop 2024-04-25T08:50:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.455+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 msgr2=0x7efde80b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.455+0000 7efdf0400640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efde80b2a80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7efddc009ab0 tx=0x7efddc0197f0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.459+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 shutdown_connections 2024-04-25T08:50:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.459+0000 7efdf0400640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdc8066f90 0x7efdc8069450 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.459+0000 7efdf0400640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde814dcb0 0x7efde814e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.459+0000 7efdf0400640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efde8154cf0 0x7efde8155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.459+0000 7efdf0400640 1 --2- 172.21.15.46:0/4144520851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdd0003a60 0x7efde80b2a80 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.459+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 >> 172.21.15.46:0/4144520851 conn(0x7efde806e3e0 msgr2=0x7efde80d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.459+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 shutdown_connections 2024-04-25T08:50:47.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.459+0000 7efdf0400640 1 -- 172.21.15.46:0/4144520851 wait complete. 2024-04-25T08:50:47.487 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:50:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f361ec00640 1 Processor -- start 2024-04-25T08:50:47.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f361ec00640 1 -- start start 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f361ec00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3618143990 0x7f36180cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f361ec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f36181440d0 0x7f361806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f361ec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361815e440 con 0x7f361806ef80 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f361ec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361815ed30 con 0x7f36181440d0 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f361ec00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36180b3730 con 0x7f3618143990 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f360f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f36181440d0 0x7f361806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f361cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f361806ef80 0x7f36180cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53874/0 (socket says 172.21.15.46:53874) 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f361cc00640 1 -- 172.21.15.46:0/3904704512 learned_addr learned my addr 172.21.15.46:0/3904704512 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f360f400640 1 --2- 172.21.15.46:0/3904704512 >> v2:172.21.15.46:3300/0 conn(0x7f36181440d0 0x7f361806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43324/0 (socket says 172.21.15.46:43324) 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f3617e00640 1 --2- 172.21.15.46:0/3904704512 >> v2:172.21.15.82:3300/0 conn(0x7f3618143990 0x7f36180cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f360f400640 1 -- 172.21.15.46:0/3904704512 >> v2:172.21.15.82:3300/0 conn(0x7f3618143990 msgr2=0x7f36180cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f360f400640 1 --2- 172.21.15.46:0/3904704512 >> v2:172.21.15.82:3300/0 conn(0x7f3618143990 0x7f36180cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f360f400640 1 -- 172.21.15.46:0/3904704512 >> v1:172.21.15.46:6789/0 conn(0x7f361806ef80 legacy=0x7f36180cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f360f400640 1 -- 172.21.15.46:0/3904704512 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361815e6e0 con 0x7f36181440d0 2024-04-25T08:50:47.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.643+0000 7f3617e00640 1 --2- 172.21.15.46:0/3904704512 >> v2:172.21.15.82:3300/0 conn(0x7f3618143990 0x7f36180cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f360f400640 1 --2- 172.21.15.46:0/3904704512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36181440d0 0x7f361806ea40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f361806b9d0 tx=0x7f360001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b14f49c8d3c2ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f3617400640 1 -- 172.21.15.46:0/3904704512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000b0070 con 0x7f36181440d0 2024-04-25T08:50:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f3617400640 1 -- 172.21.15.46:0/3904704512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36000ab600 con 0x7f36181440d0 2024-04-25T08:50:47.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f3617400640 1 -- 172.21.15.46:0/3904704512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000ab9d0 con 0x7f36181440d0 2024-04-25T08:50:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f361ec00640 1 -- 172.21.15.46:0/3904704512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36181440d0 msgr2=0x7f361806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f361ec00640 1 --2- 172.21.15.46:0/3904704512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36181440d0 0x7f361806ea40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f361806b9d0 tx=0x7f360001c480 comp rx=0 tx=0).stop 2024-04-25T08:50:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f361ec00640 1 -- 172.21.15.46:0/3904704512 shutdown_connections 2024-04-25T08:50:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f361ec00640 1 --2- 172.21.15.46:0/3904704512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36181440d0 0x7f361806ea40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f361ec00640 1 --2- 172.21.15.46:0/3904704512 >> v2:172.21.15.82:3300/0 conn(0x7f3618143990 0x7f36180cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f361ec00640 1 -- 172.21.15.46:0/3904704512 >> 172.21.15.46:0/3904704512 conn(0x7f361806ac10 msgr2=0x7f361806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f361ec00640 1 -- 172.21.15.46:0/3904704512 shutdown_connections 2024-04-25T08:50:47.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.647+0000 7f361ec00640 1 -- 172.21.15.46:0/3904704512 wait complete. 2024-04-25T08:50:47.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 Processor -- start 2024-04-25T08:50:47.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 -- start start 2024-04-25T08:50:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f361806ef80 0x7f36180b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618143990 0x7f36180afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36181440d0 0x7f3618159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3618143d70 con 0x7f3618143990 2024-04-25T08:50:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361806f390 con 0x7f36181440d0 2024-04-25T08:50:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3618143ef0 con 0x7f361806ef80 2024-04-25T08:50:47.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f360f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618143990 0x7f36180afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f360f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618143990 0x7f36180afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43340/0 (socket says 172.21.15.46:43340) 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f360f400640 1 -- 172.21.15.46:0/1238932141 learned_addr learned my addr 172.21.15.46:0/1238932141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f3617e00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f361806ef80 0x7f36180b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361cc00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36181440d0 0x7f3618159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f360f400640 1 -- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f361806ef80 msgr2=0x7f36180b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f360f400640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f361806ef80 0x7f36180b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f360f400640 1 -- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36181440d0 msgr2=0x7f3618159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f360f400640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36181440d0 0x7f3618159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f360f400640 1 -- 172.21.15.46:0/1238932141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180b04d0 con 0x7f3618143990 2024-04-25T08:50:47.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f360f400640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618143990 0x7f36180afd80 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f36000ae040 tx=0x7f3600019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f3614c00640 1 -- 172.21.15.46:0/1238932141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000b0040 con 0x7f3618143990 2024-04-25T08:50:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f3614c00640 1 -- 172.21.15.46:0/1238932141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36000ba070 con 0x7f3618143990 2024-04-25T08:50:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f3614c00640 1 -- 172.21.15.46:0/1238932141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36000b4870 con 0x7f3618143990 2024-04-25T08:50:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36180b07b0 con 0x7f3618143990 2024-04-25T08:50:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.651+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36180ab730 con 0x7f3618143990 2024-04-25T08:50:47.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.655+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36180b0cd0 con 0x7f3618143990 2024-04-25T08:50:47.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.655+0000 7f3614c00640 1 -- 172.21.15.46:0/1238932141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36000ab480 con 0x7f3618143990 2024-04-25T08:50:47.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.655+0000 7f3614c00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35f4066f60 0x7f35f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:47.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.655+0000 7f3614c00640 1 -- 172.21.15.46:0/1238932141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f360011f6c0 con 0x7f3618143990 2024-04-25T08:50:47.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.655+0000 7f3617e00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35f4066f60 0x7f35f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:47.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.659+0000 7f3617e00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35f4066f60 0x7f35f4069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f360800d620 tx=0x7f3608007810 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:47.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.659+0000 7f3614c00640 1 -- 172.21.15.46:0/1238932141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f36180b0cd0 con 0x7f3618143990 2024-04-25T08:50:47.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:47.855+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3618069010 con 0x7f3618143990 2024-04-25T08:50:48.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.471+0000 7f3614c00640 1 -- 172.21.15.46:0/1238932141 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v79) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3600016070 con 0x7f3618143990 2024-04-25T08:50:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T08:50:48.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.479+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35f4066f60 msgr2=0x7f35f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:48.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.479+0000 7f361ec00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35f4066f60 0x7f35f4069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f360800d620 tx=0x7f3608007810 comp rx=0 tx=0).stop 2024-04-25T08:50:48.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.479+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618143990 msgr2=0x7f36180afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:48.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.479+0000 7f361ec00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618143990 0x7f36180afd80 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f36000ae040 tx=0x7f3600019220 comp rx=0 tx=0).stop 2024-04-25T08:50:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.483+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 shutdown_connections 2024-04-25T08:50:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.483+0000 7f361ec00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35f4066f60 0x7f35f4069420 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.483+0000 7f361ec00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36181440d0 0x7f3618159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.483+0000 7f361ec00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3618143990 0x7f36180afd80 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.483+0000 7f361ec00640 1 --2- 172.21.15.46:0/1238932141 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f361806ef80 0x7f36180b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.483+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 >> 172.21.15.46:0/1238932141 conn(0x7f361806ac10 msgr2=0x7f361806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.487+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 shutdown_connections 2024-04-25T08:50:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.491+0000 7f361ec00640 1 -- 172.21.15.46:0/1238932141 wait complete. 2024-04-25T08:50:48.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.663+0000 7fc3dd600640 1 Processor -- start 2024-04-25T08:50:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.663+0000 7fc3dd600640 1 -- start start 2024-04-25T08:50:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.663+0000 7fc3dd600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc3d815bdb0 0x7fc3d80cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.663+0000 7fc3dd600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80cda80 0x7fc3d815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.663+0000 7fc3dd600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d80cc700 con 0x7fc3d8159230 2024-04-25T08:50:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.663+0000 7fc3dd600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d80cc9e0 con 0x7fc3d815bdb0 2024-04-25T08:50:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.663+0000 7fc3dd600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80cccc0 con 0x7fc3d80cda80 2024-04-25T08:50:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.663+0000 7fc3d6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc3d8159230 0x7fc3d80cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44998/0 (socket says 172.21.15.46:44998) 2024-04-25T08:50:48.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.663+0000 7fc3d6a00640 1 -- 172.21.15.46:0/3338579861 learned_addr learned my addr 172.21.15.46:0/3338579861 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d7400640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80cda80 0x7fc3d815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3cfe00640 1 --2- 172.21.15.46:0/3338579861 >> v2:172.21.15.82:3300/0 conn(0x7fc3d815bdb0 0x7fc3d80cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295089172 0 0) 0x7fc3d80cc700 con 0x7fc3d8159230 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3bc003530 con 0x7fc3d8159230 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3025863976 0 0) 0x7fc3bc003530 con 0x7fc3d8159230 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80cda80 msgr2=0x7fc3d815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80cda80 0x7fc3d815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 >> v2:172.21.15.82:3300/0 conn(0x7fc3d815bdb0 msgr2=0x7fc3d80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 --2- 172.21.15.46:0/3338579861 >> v2:172.21.15.82:3300/0 conn(0x7fc3d815bdb0 0x7fc3d80cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d80ccfa0 con 0x7fc3d8159230 2024-04-25T08:50:48.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fc3c8001680 con 0x7fc3d8159230 2024-04-25T08:50:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 >> v1:172.21.15.46:6789/0 conn(0x7fc3d8159230 legacy=0x7fc3d80cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80cda80 0x7fc3bc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3bc003f40 0x7fc3bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3bc004d50 0x7fc3bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80cc9e0 con 0x7fc3bc004d50 2024-04-25T08:50:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80cccc0 con 0x7fc3bc003f40 2024-04-25T08:50:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc00ba20 con 0x7fc3d80cda80 2024-04-25T08:50:48.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d7400640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3bc004d50 0x7fc3bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3cfe00640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3bc003f40 0x7fc3bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d6a00640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80cda80 0x7fc3bc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d7400640 1 -- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80cda80 msgr2=0x7fc3bc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d7400640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80cda80 0x7fc3bc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d7400640 1 -- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3bc003f40 msgr2=0x7fc3bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d7400640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3bc003f40 0x7fc3bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.667+0000 7fc3d7400640 1 -- 172.21.15.46:0/3338579861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc003530 con 0x7fc3bc004d50 2024-04-25T08:50:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3d7400640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3bc004d50 0x7fc3bc0051d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000e170 tx=0x7fc3d001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b3370890fbe48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00b2070 con 0x7fc3bc004d50 2024-04-25T08:50:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d00a5c70 con 0x7fc3bc004d50 2024-04-25T08:50:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3d6000640 1 -- 172.21.15.46:0/3338579861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0009c10 con 0x7fc3bc004d50 2024-04-25T08:50:48.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3dd600640 1 -- 172.21.15.46:0/3338579861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3bc004d50 msgr2=0x7fc3bc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3dd600640 1 --2- 172.21.15.46:0/3338579861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3bc004d50 0x7fc3bc0051d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000e170 tx=0x7fc3d001cc60 comp rx=0 tx=0).stop 2024-04-25T08:50:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3d7400640 1 -- 172.21.15.46:0/3338579861 reap_dead start 2024-04-25T08:50:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3dd600640 1 -- 172.21.15.46:0/3338579861 shutdown_connections 2024-04-25T08:50:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3dd600640 1 -- 172.21.15.46:0/3338579861 >> 172.21.15.46:0/3338579861 conn(0x7fc3d80b12a0 msgr2=0x7fc3d80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3dd600640 1 -- 172.21.15.46:0/3338579861 shutdown_connections 2024-04-25T08:50:48.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3dd600640 1 -- 172.21.15.46:0/3338579861 wait complete. 2024-04-25T08:50:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3dd600640 1 Processor -- start 2024-04-25T08:50:48.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.671+0000 7fc3dd600640 1 -- start start 2024-04-25T08:50:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3dd600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3d81580b0 0x7fc3d81438d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d8142090 0x7fc3d80a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3dd600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80a9c00 0x7fc3d80a8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:48.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3dd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d8159790 con 0x7fc3d8142090 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3dd600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d8159610 con 0x7fc3d81580b0 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3dd600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d815d200 con 0x7fc3d80a9c00 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d8142090 0x7fc3d80a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d8142090 0x7fc3d80a96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40038/0 (socket says 172.21.15.46:40038) 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3d6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3d81580b0 0x7fc3d81438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cfe00640 1 -- 172.21.15.46:0/2219517991 learned_addr learned my addr 172.21.15.46:0/2219517991 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3d7400640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80a9c00 0x7fc3d80a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cfe00640 1 -- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80a9c00 msgr2=0x7fc3d80a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cfe00640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80a9c00 0x7fc3d80a8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cfe00640 1 -- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3d81580b0 msgr2=0x7fc3d81438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cfe00640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3d81580b0 0x7fc3d81438d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cfe00640 1 -- 172.21.15.46:0/2219517991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8142800 con 0x7fc3d8142090 2024-04-25T08:50:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cfe00640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d8142090 0x7fc3d80a96c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009bb0 tx=0x7fc3c000b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cea00640 1 -- 172.21.15.46:0/2219517991 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000bd20 con 0x7fc3d8142090 2024-04-25T08:50:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cea00640 1 -- 172.21.15.46:0/2219517991 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c0020070 con 0x7fc3d8142090 2024-04-25T08:50:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cea00640 1 -- 172.21.15.46:0/2219517991 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c001b640 con 0x7fc3d8142090 2024-04-25T08:50:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d8142ae0 con 0x7fc3d8142090 2024-04-25T08:50:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d81430a0 con 0x7fc3d8142090 2024-04-25T08:50:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d8069080 con 0x7fc3d8142090 2024-04-25T08:50:48.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.675+0000 7fc3cea00640 1 -- 172.21.15.46:0/2219517991 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3c001b7e0 con 0x7fc3d8142090 2024-04-25T08:50:48.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.679+0000 7fc3cea00640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3b4066f60 0x7fc3b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.679+0000 7fc3cea00640 1 -- 172.21.15.46:0/2219517991 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc3c0089770 con 0x7fc3d8142090 2024-04-25T08:50:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.679+0000 7fc3d6a00640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3b4066f60 0x7fc3b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:48.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.679+0000 7fc3d6a00640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3b4066f60 0x7fc3b4069420 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005280 tx=0x7fc3c8007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:48.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.683+0000 7fc3cea00640 1 -- 172.21.15.46:0/2219517991 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3c005a5f0 con 0x7fc3d8142090 2024-04-25T08:50:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.843+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3d80aa010 con 0x7fc3d8142090 2024-04-25T08:50:48.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3cea00640 1 -- 172.21.15.46:0/2219517991 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16849 (secure 0 0 0) 0x7fc3c0059ef0 con 0x7fc3d8142090 2024-04-25T08:50:48.836 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:48.837 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":105,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:50:47.859671+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:50:45.647599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:50:46.857198+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3b4066f60 msgr2=0x7fc3b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3b4066f60 0x7fc3b4069420 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8005280 tx=0x7fc3c8007900 comp rx=0 tx=0).stop 2024-04-25T08:50:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d8142090 msgr2=0x7fc3d80a96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d8142090 0x7fc3d80a96c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0009bb0 tx=0x7fc3c000b140 comp rx=0 tx=0).stop 2024-04-25T08:50:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 shutdown_connections 2024-04-25T08:50:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc3b4066f60 0x7fc3b4069420 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc3d80a9c00 0x7fc3d80a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d8142090 0x7fc3d80a96c0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 --2- 172.21.15.46:0/2219517991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3d81580b0 0x7fc3d81438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 >> 172.21.15.46:0/2219517991 conn(0x7fc3d80b12a0 msgr2=0x7fc3d80af260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 shutdown_connections 2024-04-25T08:50:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:48.847+0000 7fc3dd600640 1 -- 172.21.15.46:0/2219517991 wait complete. 2024-04-25T08:50:48.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.007+0000 7f35dee00640 1 Processor -- start 2024-04-25T08:50:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.007+0000 7f35dee00640 1 -- start start 2024-04-25T08:50:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dee00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35d815a750 0x7f35d806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35d80cc9e0 0x7f35d806f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d806d5e0 con 0x7f35d80aed10 2024-04-25T08:50:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d815ab30 con 0x7f35d80cc9e0 2024-04-25T08:50:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dee00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d80ce4f0 con 0x7f35d815a750 2024-04-25T08:50:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35d80cc9e0 0x7f35d806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35d80cc9e0 0x7f35d806f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40046/0 (socket says 172.21.15.46:40046) 2024-04-25T08:50:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dce00640 1 -- 172.21.15.46:0/2280211825 learned_addr learned my addr 172.21.15.46:0/2280211825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35d7e00640 1 --2- 172.21.15.46:0/2280211825 >> v2:172.21.15.82:3300/0 conn(0x7f35d815a750 0x7f35d806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dce00640 1 -- 172.21.15.46:0/2280211825 >> v2:172.21.15.82:3300/0 conn(0x7f35d815a750 msgr2=0x7f35d806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dce00640 1 --2- 172.21.15.46:0/2280211825 >> v2:172.21.15.82:3300/0 conn(0x7f35d815a750 0x7f35d806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dce00640 1 -- 172.21.15.46:0/2280211825 >> v1:172.21.15.82:6789/0 conn(0x7f35d80aed10 legacy=0x7f35d80b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dce00640 1 -- 172.21.15.46:0/2280211825 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d80ce720 con 0x7f35d80cc9e0 2024-04-25T08:50:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35dce00640 1 --2- 172.21.15.46:0/2280211825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 0x7f35d806f150 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f35d804ffd0 tx=0x7f35d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e930a42d4c203da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:49.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.011+0000 7f35d7400640 1 -- 172.21.15.46:0/2280211825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d00a57a0 con 0x7f35d80cc9e0 2024-04-25T08:50:49.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35d7400640 1 -- 172.21.15.46:0/2280211825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d00a5940 con 0x7f35d80cc9e0 2024-04-25T08:50:49.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35d7400640 1 -- 172.21.15.46:0/2280211825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d00a5d10 con 0x7f35d80cc9e0 2024-04-25T08:50:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 -- 172.21.15.46:0/2280211825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 msgr2=0x7f35d806f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 --2- 172.21.15.46:0/2280211825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 0x7f35d806f150 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f35d804ffd0 tx=0x7f35d001c9f0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 -- 172.21.15.46:0/2280211825 shutdown_connections 2024-04-25T08:50:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 --2- 172.21.15.46:0/2280211825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 0x7f35d806f150 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 --2- 172.21.15.46:0/2280211825 >> v2:172.21.15.82:3300/0 conn(0x7f35d815a750 0x7f35d806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 -- 172.21.15.46:0/2280211825 >> 172.21.15.46:0/2280211825 conn(0x7f35d80b12a0 msgr2=0x7f35d80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 -- 172.21.15.46:0/2280211825 shutdown_connections 2024-04-25T08:50:49.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 -- 172.21.15.46:0/2280211825 wait complete. 2024-04-25T08:50:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 Processor -- start 2024-04-25T08:50:49.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 -- start start 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35d80aed10 0x7f35d814b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 0x7f35d815c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35d815a750 0x7f35d80ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8165220 con 0x7f35d80cc9e0 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d806d270 con 0x7f35d815a750 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d81653a0 con 0x7f35d80aed10 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 0x7f35d815c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 0x7f35d815c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40054/0 (socket says 172.21.15.46:40054) 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35cf400640 1 -- 172.21.15.46:0/2509158100 learned_addr learned my addr 172.21.15.46:0/2509158100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:49.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35dce00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35d815a750 0x7f35d80ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35d7e00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35d80aed10 0x7f35d814b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35cf400640 1 -- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35d80aed10 msgr2=0x7f35d814b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35cf400640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35d80aed10 0x7f35d814b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35cf400640 1 -- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35d815a750 msgr2=0x7f35d80ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35cf400640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35d815a750 0x7f35d80ad8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.015+0000 7f35cf400640 1 -- 172.21.15.46:0/2509158100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d81fa9f0 con 0x7f35d80cc9e0 2024-04-25T08:50:49.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35cf400640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 0x7f35d815c9b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c430 tx=0x7f35c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35d4c00640 1 -- 172.21.15.46:0/2509158100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c00052d0 con 0x7f35d80cc9e0 2024-04-25T08:50:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35d4c00640 1 -- 172.21.15.46:0/2509158100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c0005470 con 0x7f35d80cc9e0 2024-04-25T08:50:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35d4c00640 1 -- 172.21.15.46:0/2509158100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0007260 con 0x7f35d80cc9e0 2024-04-25T08:50:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d814b8a0 con 0x7f35d80cc9e0 2024-04-25T08:50:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d81f9de0 con 0x7f35d80cc9e0 2024-04-25T08:50:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d8069080 con 0x7f35d80cc9e0 2024-04-25T08:50:49.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35d4c00640 1 -- 172.21.15.46:0/2509158100 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35c0007400 con 0x7f35d80cc9e0 2024-04-25T08:50:49.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35d4c00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35b4066f60 0x7f35b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.019+0000 7f35d4c00640 1 -- 172.21.15.46:0/2509158100 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f35c00846c0 con 0x7f35d80cc9e0 2024-04-25T08:50:49.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.023+0000 7f35d7e00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35b4066f60 0x7f35b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.023+0000 7f35d7e00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35b4066f60 0x7f35b4069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f35c80079c0 tx=0x7f35c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:49.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.027+0000 7f35d4c00640 1 -- 172.21.15.46:0/2509158100 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35d814b8a0 con 0x7f35d80cc9e0 2024-04-25T08:50:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.187+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35d806f100 con 0x7f35d80cc9e0 2024-04-25T08:50:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35d4c00640 1 -- 172.21.15.46:0/2509158100 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16849 (secure 0 0 0) 0x7f35c0054e50 con 0x7f35d80cc9e0 2024-04-25T08:50:49.180 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:49.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":105,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:50:47.859671+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:50:45.647599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:50:46.857198+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:49.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35b4066f60 msgr2=0x7f35b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35b4066f60 0x7f35b4069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f35c80079c0 tx=0x7f35c8008040 comp rx=0 tx=0).stop 2024-04-25T08:50:49.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 msgr2=0x7f35d815c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 0x7f35d815c9b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c430 tx=0x7f35c000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 shutdown_connections 2024-04-25T08:50:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f35b4066f60 0x7f35b4069420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35d815a750 0x7f35d80ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d80cc9e0 0x7f35d815c9b0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 --2- 172.21.15.46:0/2509158100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f35d80aed10 0x7f35d814b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 >> 172.21.15.46:0/2509158100 conn(0x7f35d80b12a0 msgr2=0x7f35d80cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 shutdown_connections 2024-04-25T08:50:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.191+0000 7f35dee00640 1 -- 172.21.15.46:0/2509158100 wait complete. 2024-04-25T08:50:49.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:50:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff38f200640 1 Processor -- start 2024-04-25T08:50:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff38f200640 1 -- start start 2024-04-25T08:50:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff38f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff388143990 0x7ff3880cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff38f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff38806f150 0x7ff3880b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff38f200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38815e440 con 0x7ff3881440d0 2024-04-25T08:50:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff38f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38815ed30 con 0x7ff388143990 2024-04-25T08:50:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff38f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38815e6e0 con 0x7ff38806f150 2024-04-25T08:50:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff387e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff388143990 0x7ff3880cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff387e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff388143990 0x7ff3880cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40056/0 (socket says 172.21.15.46:40056) 2024-04-25T08:50:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff387e00640 1 -- 172.21.15.46:0/2023712734 learned_addr learned my addr 172.21.15.46:0/2023712734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff38d200640 1 --2- 172.21.15.46:0/2023712734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff38806f150 0x7ff3880b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff387e00640 1 -- 172.21.15.46:0/2023712734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff38806f150 msgr2=0x7ff3880b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff387e00640 1 --2- 172.21.15.46:0/2023712734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff38806f150 0x7ff3880b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff387e00640 1 -- 172.21.15.46:0/2023712734 >> v1:172.21.15.82:6789/0 conn(0x7ff3881440d0 legacy=0x7ff38806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.355+0000 7ff387e00640 1 -- 172.21.15.46:0/2023712734 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38815e9c0 con 0x7ff388143990 2024-04-25T08:50:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff387e00640 1 --2- 172.21.15.46:0/2023712734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff388143990 0x7ff3880cecc0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff38815eeb0 tx=0x7ff37801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7709b2255ef871b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff387400640 1 -- 172.21.15.46:0/2023712734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780b0070 con 0x7ff388143990 2024-04-25T08:50:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff387400640 1 -- 172.21.15.46:0/2023712734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3780ab600 con 0x7ff388143990 2024-04-25T08:50:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff387400640 1 -- 172.21.15.46:0/2023712734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3780ab9d0 con 0x7ff388143990 2024-04-25T08:50:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff38f200640 1 -- 172.21.15.46:0/2023712734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff388143990 msgr2=0x7ff3880cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff38f200640 1 --2- 172.21.15.46:0/2023712734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff388143990 0x7ff3880cecc0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff38815eeb0 tx=0x7ff37801c540 comp rx=0 tx=0).stop 2024-04-25T08:50:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff38f200640 1 -- 172.21.15.46:0/2023712734 shutdown_connections 2024-04-25T08:50:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff38f200640 1 --2- 172.21.15.46:0/2023712734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff38806f150 0x7ff3880b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff38f200640 1 --2- 172.21.15.46:0/2023712734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff388143990 0x7ff3880cecc0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff38f200640 1 -- 172.21.15.46:0/2023712734 >> 172.21.15.46:0/2023712734 conn(0x7ff38806ac10 msgr2=0x7ff38806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff38f200640 1 -- 172.21.15.46:0/2023712734 shutdown_connections 2024-04-25T08:50:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.359+0000 7ff38f200640 1 -- 172.21.15.46:0/2023712734 wait complete. 2024-04-25T08:50:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 Processor -- start 2024-04-25T08:50:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 -- start start 2024-04-25T08:50:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38806f150 0x7ff388142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff388143990 0x7ff38815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3881440d0 0x7ff3880b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3880b35f0 con 0x7ff3881440d0 2024-04-25T08:50:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3880b3470 con 0x7ff38806f150 2024-04-25T08:50:49.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3880b3770 con 0x7ff388143990 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3881440d0 0x7ff3880b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3881440d0 0x7ff3880b3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40066/0 (socket says 172.21.15.46:40066) 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38d200640 1 -- 172.21.15.46:0/2394496496 learned_addr learned my addr 172.21.15.46:0/2394496496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff387e00640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38806f150 0x7ff388142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff37f400640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff388143990 0x7ff38815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38d200640 1 -- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff388143990 msgr2=0x7ff38815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38d200640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff388143990 0x7ff38815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38d200640 1 -- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38806f150 msgr2=0x7ff388142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38d200640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38806f150 0x7ff388142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38d200640 1 -- 172.21.15.46:0/2394496496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff388143070 con 0x7ff3881440d0 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff37f400640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff388143990 0x7ff38815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38d200640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3881440d0 0x7ff3880b3920 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff380007e70 tx=0x7ff38000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff384c00640 1 -- 172.21.15.46:0/2394496496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380014620 con 0x7ff3881440d0 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff384c00640 1 -- 172.21.15.46:0/2394496496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff380019070 con 0x7ff3881440d0 2024-04-25T08:50:49.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff384c00640 1 -- 172.21.15.46:0/2394496496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380014a20 con 0x7ff3881440d0 2024-04-25T08:50:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff388142d90 con 0x7ff3881440d0 2024-04-25T08:50:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.363+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3881fa0f0 con 0x7ff3881440d0 2024-04-25T08:50:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.367+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff388069010 con 0x7ff3881440d0 2024-04-25T08:50:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.367+0000 7ff384c00640 1 -- 172.21.15.46:0/2394496496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff380014bc0 con 0x7ff3881440d0 2024-04-25T08:50:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.367+0000 7ff384c00640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff364066f90 0x7ff364069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.367+0000 7ff384c00640 1 -- 172.21.15.46:0/2394496496 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff380085a50 con 0x7ff3881440d0 2024-04-25T08:50:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.367+0000 7ff387e00640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff364066f90 0x7ff364069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.367+0000 7ff387e00640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff364066f90 0x7ff364069450 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff388070c80 tx=0x7ff378009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.371+0000 7ff384c00640 1 -- 172.21.15.46:0/2394496496 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff380056830 con 0x7ff3881440d0 2024-04-25T08:50:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:49.575+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff388137df0 con 0x7ff3881440d0 2024-04-25T08:50:50.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff384c00640 1 -- 172.21.15.46:0/2394496496 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff380056130 con 0x7ff3881440d0 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff364066f90 msgr2=0x7ff364069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff364066f90 0x7ff364069450 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff388070c80 tx=0x7ff378009040 comp rx=0 tx=0).stop 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3881440d0 msgr2=0x7ff3880b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3881440d0 0x7ff3880b3920 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff380007e70 tx=0x7ff38000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 shutdown_connections 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff364066f90 0x7ff364069450 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3881440d0 0x7ff3880b3920 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff388143990 0x7ff38815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 --2- 172.21.15.46:0/2394496496 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38806f150 0x7ff388142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 >> 172.21.15.46:0/2394496496 conn(0x7ff38806ac10 msgr2=0x7ff3880b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 shutdown_connections 2024-04-25T08:50:50.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.479+0000 7ff38f200640 1 -- 172.21.15.46:0/2394496496 wait complete. 2024-04-25T08:50:50.494 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:50:50.494 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:50:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdef0600640 1 Processor -- start 2024-04-25T08:50:50.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdef0600640 1 -- start start 2024-04-25T08:50:50.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdef0600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee8143990 0x7fdee80cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdef0600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee81440d0 0x7fdee806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdef0600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdee806ef80 0x7fdee80cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdef0600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee815ed30 con 0x7fdee81440d0 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdef0600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee815e440 con 0x7fdee806ef80 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdef0600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee815e6e0 con 0x7fdee8143990 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeed200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee81440d0 0x7fdee806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeed200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee81440d0 0x7fdee806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40072/0 (socket says 172.21.15.46:40072) 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeed200640 1 -- 172.21.15.46:0/1258072667 learned_addr learned my addr 172.21.15.46:0/1258072667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeee600640 1 --2- 172.21.15.46:0/1258072667 >> v2:172.21.15.82:3300/0 conn(0x7fdee806ef80 0x7fdee80cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeedc00640 1 --2- 172.21.15.46:0/1258072667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee8143990 0x7fdee80cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeed200640 1 -- 172.21.15.46:0/1258072667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee8143990 msgr2=0x7fdee80cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:50.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeed200640 1 --2- 172.21.15.46:0/1258072667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee8143990 0x7fdee80cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeed200640 1 -- 172.21.15.46:0/1258072667 >> v2:172.21.15.82:3300/0 conn(0x7fdee806ef80 msgr2=0x7fdee80cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:50.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeed200640 1 --2- 172.21.15.46:0/1258072667 >> v2:172.21.15.82:3300/0 conn(0x7fdee806ef80 0x7fdee80cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeed200640 1 -- 172.21.15.46:0/1258072667 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee815e9c0 con 0x7fdee81440d0 2024-04-25T08:50:50.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.651+0000 7fdeee600640 1 --2- 172.21.15.46:0/1258072667 >> v2:172.21.15.82:3300/0 conn(0x7fdee806ef80 0x7fdee80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:50.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdeed200640 1 --2- 172.21.15.46:0/1258072667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee81440d0 0x7fdee806ea40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fdee806f700 tx=0x7fded401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb5148509daad6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:50.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdee7e00640 1 -- 172.21.15.46:0/1258072667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40b0070 con 0x7fdee81440d0 2024-04-25T08:50:50.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdee7e00640 1 -- 172.21.15.46:0/1258072667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded40ab600 con 0x7fdee81440d0 2024-04-25T08:50:50.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdee7e00640 1 -- 172.21.15.46:0/1258072667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40ab9d0 con 0x7fdee81440d0 2024-04-25T08:50:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdef0600640 1 -- 172.21.15.46:0/1258072667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee81440d0 msgr2=0x7fdee806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdef0600640 1 --2- 172.21.15.46:0/1258072667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee81440d0 0x7fdee806ea40 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fdee806f700 tx=0x7fded401c260 comp rx=0 tx=0).stop 2024-04-25T08:50:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdef0600640 1 -- 172.21.15.46:0/1258072667 shutdown_connections 2024-04-25T08:50:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdef0600640 1 --2- 172.21.15.46:0/1258072667 >> v2:172.21.15.82:3300/0 conn(0x7fdee806ef80 0x7fdee80cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdef0600640 1 --2- 172.21.15.46:0/1258072667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee81440d0 0x7fdee806ea40 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdef0600640 1 --2- 172.21.15.46:0/1258072667 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee8143990 0x7fdee80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdef0600640 1 -- 172.21.15.46:0/1258072667 >> 172.21.15.46:0/1258072667 conn(0x7fdee806ac10 msgr2=0x7fdee806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdef0600640 1 -- 172.21.15.46:0/1258072667 shutdown_connections 2024-04-25T08:50:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.655+0000 7fdef0600640 1 -- 172.21.15.46:0/1258072667 wait complete. 2024-04-25T08:50:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdef0600640 1 Processor -- start 2024-04-25T08:50:50.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdef0600640 1 -- start start 2024-04-25T08:50:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdef0600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee806ef80 0x7fdee8159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdef0600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee8143990 0x7fdee813c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdef0600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee81440d0 0x7fdee81406e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdef0600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8143ef0 con 0x7fdee8143990 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdef0600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8143d70 con 0x7fdee81440d0 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdef0600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8071270 con 0x7fdee806ef80 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdeed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee8143990 0x7fdee813c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdeed200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee8143990 0x7fdee813c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40082/0 (socket says 172.21.15.46:40082) 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdeed200640 1 -- 172.21.15.46:0/724231399 learned_addr learned my addr 172.21.15.46:0/724231399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdeedc00640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee806ef80 0x7fdee8159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.659+0000 7fdeee600640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee81440d0 0x7fdee81406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdeed200640 1 -- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee806ef80 msgr2=0x7fdee8159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdeed200640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee806ef80 0x7fdee8159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdeed200640 1 -- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee81440d0 msgr2=0x7fdee81406e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdeed200640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee81440d0 0x7fdee81406e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdeed200640 1 -- 172.21.15.46:0/724231399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80ccf90 con 0x7fdee8143990 2024-04-25T08:50:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdeed200640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee8143990 0x7fdee813c570 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fded401cb00 tx=0x7fded401aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdee5600640 1 -- 172.21.15.46:0/724231399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40ba020 con 0x7fdee8143990 2024-04-25T08:50:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee813cf30 con 0x7fdee8143990 2024-04-25T08:50:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdee5600640 1 -- 172.21.15.46:0/724231399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded40b6420 con 0x7fdee8143990 2024-04-25T08:50:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdee5600640 1 -- 172.21.15.46:0/724231399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded40b0040 con 0x7fdee8143990 2024-04-25T08:50:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee813d4f0 con 0x7fdee8143990 2024-04-25T08:50:50.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee8069010 con 0x7fdee8143990 2024-04-25T08:50:50.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdee5600640 1 -- 172.21.15.46:0/724231399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fded40bd020 con 0x7fdee8143990 2024-04-25T08:50:50.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdee5600640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec8066f60 0x7fdec8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:50.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdee5600640 1 -- 172.21.15.46:0/724231399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fded411f370 con 0x7fdee8143990 2024-04-25T08:50:50.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.663+0000 7fdeedc00640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec8066f60 0x7fdec8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:50.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.667+0000 7fdeedc00640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec8066f60 0x7fdec8069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdee815ddf0 tx=0x7fdedc0077b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:50.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.667+0000 7fdee5600640 1 -- 172.21.15.46:0/724231399 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fded40c4030 con 0x7fdee8143990 2024-04-25T08:50:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:50.867+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdee8141280 con 0x7fdee8143990 2024-04-25T08:50:51.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:50:51.202 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:50:51.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.479+0000 7fdee5600640 1 -- 172.21.15.46:0/724231399 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) v1 ==== 122+0+0 (secure 0 0 0) 0x7fded4016110 con 0x7fdee8143990 2024-04-25T08:50:51.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec8066f60 msgr2=0x7fdec8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec8066f60 0x7fdec8069420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdee815ddf0 tx=0x7fdedc0077b0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee8143990 msgr2=0x7fdee813c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee8143990 0x7fdee813c570 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fded401cb00 tx=0x7fded401aaa0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 shutdown_connections 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdec8066f60 0x7fdec8069420 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdee81440d0 0x7fdee81406e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee8143990 0x7fdee813c570 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 --2- 172.21.15.46:0/724231399 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdee806ef80 0x7fdee8159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 >> 172.21.15.46:0/724231399 conn(0x7fdee806ac10 msgr2=0x7fdee8139ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 shutdown_connections 2024-04-25T08:50:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.483+0000 7fdef0600640 1 -- 172.21.15.46:0/724231399 wait complete. 2024-04-25T08:50:51.497 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:50:51.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b29000640 1 Processor -- start 2024-04-25T08:50:51.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b29000640 1 -- start start 2024-04-25T08:50:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b29000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b2415b980 0x7f6b240cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b29000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ceee0 0x7f6b240cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b29000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2415bd60 con 0x7f6b240ce580 2024-04-25T08:50:51.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b29000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b240cd420 con 0x7f6b2415b980 2024-04-25T08:50:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b29000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240cd700 con 0x7f6b240ceee0 2024-04-25T08:50:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b21e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b2415b980 0x7f6b240cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b21e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b2415b980 0x7f6b240cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40090/0 (socket says 172.21.15.46:40090) 2024-04-25T08:50:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b21e00640 1 -- 172.21.15.46:0/2792374949 learned_addr learned my addr 172.21.15.46:0/2792374949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:51.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b22800640 1 --2- 172.21.15.46:0/2792374949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ceee0 0x7f6b240cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b21e00640 1 -- 172.21.15.46:0/2792374949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ceee0 msgr2=0x7f6b240cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b21e00640 1 --2- 172.21.15.46:0/2792374949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ceee0 0x7f6b240cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b21e00640 1 -- 172.21.15.46:0/2792374949 >> v1:172.21.15.82:6789/0 conn(0x7f6b240ce580 legacy=0x7f6b240b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b21e00640 1 -- 172.21.15.46:0/2792374949 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b240cd9e0 con 0x7f6b2415b980 2024-04-25T08:50:51.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.655+0000 7f6b22800640 1 --2- 172.21.15.46:0/2792374949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ceee0 0x7f6b240cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:51.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b21e00640 1 --2- 172.21.15.46:0/2792374949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 0x7f6b240cce40 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009870 tx=0x7f6b1401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730702915658f0cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b21400640 1 -- 172.21.15.46:0/2792374949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140b1070 con 0x7f6b2415b980 2024-04-25T08:50:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b21400640 1 -- 172.21.15.46:0/2792374949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b140ac410 con 0x7f6b2415b980 2024-04-25T08:50:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b21400640 1 -- 172.21.15.46:0/2792374949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b140ac7e0 con 0x7f6b2415b980 2024-04-25T08:50:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b29000640 1 -- 172.21.15.46:0/2792374949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 msgr2=0x7f6b240cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b29000640 1 --2- 172.21.15.46:0/2792374949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 0x7f6b240cce40 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009870 tx=0x7f6b1401c520 comp rx=0 tx=0).stop 2024-04-25T08:50:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b29000640 1 -- 172.21.15.46:0/2792374949 shutdown_connections 2024-04-25T08:50:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b29000640 1 --2- 172.21.15.46:0/2792374949 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ceee0 0x7f6b240cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b29000640 1 --2- 172.21.15.46:0/2792374949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 0x7f6b240cce40 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b29000640 1 -- 172.21.15.46:0/2792374949 >> 172.21.15.46:0/2792374949 conn(0x7f6b240728c0 msgr2=0x7f6b24099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b29000640 1 -- 172.21.15.46:0/2792374949 shutdown_connections 2024-04-25T08:50:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.659+0000 7f6b29000640 1 -- 172.21.15.46:0/2792374949 wait complete. 2024-04-25T08:50:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 Processor -- start 2024-04-25T08:50:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 -- start start 2024-04-25T08:50:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ce580 0x7f6b241f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b240ceee0 0x7f6b2414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 0x7f6b24156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240d1190 con 0x7f6b2415b980 2024-04-25T08:50:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240d1010 con 0x7f6b240ceee0 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b240d1310 con 0x7f6b240ce580 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b22800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 0x7f6b24156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b22800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 0x7f6b24156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40106/0 (socket says 172.21.15.46:40106) 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b22800640 1 -- 172.21.15.46:0/1587521323 learned_addr learned my addr 172.21.15.46:0/1587521323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b19400640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b240ceee0 0x7f6b2414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b21e00640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ce580 0x7f6b241f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b22800640 1 -- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ce580 msgr2=0x7f6b241f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b22800640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ce580 0x7f6b241f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b22800640 1 -- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b240ceee0 msgr2=0x7f6b2414dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b22800640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b240ceee0 0x7f6b2414dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b22800640 1 -- 172.21.15.46:0/1587521323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2415c0d0 con 0x7f6b2415b980 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b19400640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b240ceee0 0x7f6b2414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b22800640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 0x7f6b24156fd0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00cf20 tx=0x7f6b1c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:51.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b1aa00640 1 -- 172.21.15.46:0/1587521323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c01a070 con 0x7f6b2415b980 2024-04-25T08:50:51.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b1aa00640 1 -- 172.21.15.46:0/1587521323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c00d840 con 0x7f6b2415b980 2024-04-25T08:50:51.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2415c3b0 con 0x7f6b2415b980 2024-04-25T08:50:51.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b1aa00640 1 -- 172.21.15.46:0/1587521323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00dc30 con 0x7f6b2415b980 2024-04-25T08:50:51.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.663+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2415c970 con 0x7f6b2415b980 2024-04-25T08:50:51.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.667+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b24069080 con 0x7f6b2415b980 2024-04-25T08:50:51.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.667+0000 7f6b1aa00640 1 -- 172.21.15.46:0/1587521323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b1c01d070 con 0x7f6b2415b980 2024-04-25T08:50:51.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.667+0000 7f6b1aa00640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b00066f60 0x7f6b00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:51.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.667+0000 7f6b1aa00640 1 -- 172.21.15.46:0/1587521323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b1c0859b0 con 0x7f6b2415b980 2024-04-25T08:50:51.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.667+0000 7f6b21e00640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b00066f60 0x7f6b00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:51.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.667+0000 7f6b21e00640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b00066f60 0x7f6b00069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6b140ae000 tx=0x7f6b14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:51.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.671+0000 7f6b1aa00640 1 -- 172.21.15.46:0/1587521323 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b1c0561d0 con 0x7f6b2415b980 2024-04-25T08:50:51.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.863+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b240affb0 con 0x7f6b2415b980 2024-04-25T08:50:51.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b1aa00640 1 -- 172.21.15.46:0/1587521323 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f6b1c05a270 con 0x7f6b2415b980 2024-04-25T08:50:51.856 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:51.856 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:50:50.870175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8334":{"gid":8334,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/623970467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":623970467},{"type":"v1","addr":"172.21.15.46:6835","nonce":623970467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:50:51.857 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 83 2024-04-25T08:50:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b00066f60 msgr2=0x7f6b00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b00066f60 0x7f6b00069420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6b140ae000 tx=0x7f6b14008040 comp rx=0 tx=0).stop 2024-04-25T08:50:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 msgr2=0x7f6b24156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 0x7f6b24156fd0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00cf20 tx=0x7f6b1c00a7d0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 shutdown_connections 2024-04-25T08:50:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b00066f60 0x7f6b00069420 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b2415b980 0x7f6b24156fd0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b240ceee0 0x7f6b2414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 --2- 172.21.15.46:0/1587521323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b240ce580 0x7f6b241f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 >> 172.21.15.46:0/1587521323 conn(0x7f6b240728c0 msgr2=0x7f6b240af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 shutdown_connections 2024-04-25T08:50:51.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:51.867+0000 7f6b29000640 1 -- 172.21.15.46:0/1587521323 wait complete. 2024-04-25T08:50:51.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:52.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.027+0000 7f8713800640 1 Processor -- start 2024-04-25T08:50:52.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.027+0000 7f8713800640 1 -- start start 2024-04-25T08:50:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.027+0000 7f8713800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c159230 0x7f870c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.027+0000 7f8713800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f870c0cdc50 0x7f870c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8713800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870c0cc700 con 0x7f870c0cbeb0 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8713800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870c0cc9e0 con 0x7f870c0cdc50 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8713800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0cccc0 con 0x7f870c159230 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8711800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f870c0cdc50 0x7f870c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f870be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f870c0cbeb0 0x7f870c0cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45014/0 (socket says 172.21.15.46:45014) 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f870be00640 1 -- 172.21.15.46:0/2192693164 learned_addr learned my addr 172.21.15.46:0/2192693164 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8710e00640 1 --2- 172.21.15.46:0/2192693164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c159230 0x7f870c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f870b400640 1 -- 172.21.15.46:0/2192693164 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324449239 0 0) 0x7f870c0cc700 con 0x7f870c0cbeb0 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f870b400640 1 -- 172.21.15.46:0/2192693164 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86f4003530 con 0x7f870c0cbeb0 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8711800640 1 -- 172.21.15.46:0/2192693164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c159230 msgr2=0x7f870c15bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8711800640 1 --2- 172.21.15.46:0/2192693164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c159230 0x7f870c15bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8711800640 1 -- 172.21.15.46:0/2192693164 >> v1:172.21.15.46:6789/0 conn(0x7f870c0cbeb0 legacy=0x7f870c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8711800640 1 -- 172.21.15.46:0/2192693164 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c0ccfa0 con 0x7f870c0cdc50 2024-04-25T08:50:52.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.031+0000 7f8711800640 1 --2- 172.21.15.46:0/2192693164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c0cdc50 0x7f870c15d200 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8704009d90 tx=0x7f870401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547857dd7fc11bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:52.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f870b400640 1 -- 172.21.15.46:0/2192693164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87040b2070 con 0x7f870c0cdc50 2024-04-25T08:50:52.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f870b400640 1 -- 172.21.15.46:0/2192693164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87040a5990 con 0x7f870c0cdc50 2024-04-25T08:50:52.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f870b400640 1 -- 172.21.15.46:0/2192693164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87040a5d40 con 0x7f870c0cdc50 2024-04-25T08:50:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 -- 172.21.15.46:0/2192693164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c0cdc50 msgr2=0x7f870c15d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 --2- 172.21.15.46:0/2192693164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c0cdc50 0x7f870c15d200 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8704009d90 tx=0x7f870401c8b0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 -- 172.21.15.46:0/2192693164 shutdown_connections 2024-04-25T08:50:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 --2- 172.21.15.46:0/2192693164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c0cdc50 0x7f870c15d200 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 --2- 172.21.15.46:0/2192693164 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c159230 0x7f870c15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 -- 172.21.15.46:0/2192693164 >> 172.21.15.46:0/2192693164 conn(0x7f870c0b12a0 msgr2=0x7f870c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 -- 172.21.15.46:0/2192693164 shutdown_connections 2024-04-25T08:50:52.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 -- 172.21.15.46:0/2192693164 wait complete. 2024-04-25T08:50:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 Processor -- start 2024-04-25T08:50:52.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 -- start start 2024-04-25T08:50:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f870c0cbeb0 0x7f870c140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c0cdc50 0x7f870c165f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c159230 0x7f870c15b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c15df50 con 0x7f870c159230 2024-04-25T08:50:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0cc4a0 con 0x7f870c0cbeb0 2024-04-25T08:50:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8713800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c15e0d0 con 0x7f870c0cdc50 2024-04-25T08:50:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8711800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c159230 0x7f870c15b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8711800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c159230 0x7f870c15b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40114/0 (socket says 172.21.15.46:40114) 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8711800640 1 -- 172.21.15.46:0/505460623 learned_addr learned my addr 172.21.15.46:0/505460623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f8710e00640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f870c0cbeb0 0x7f870c140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.035+0000 7f870be00640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c0cdc50 0x7f870c165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8711800640 1 -- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c0cdc50 msgr2=0x7f870c165f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8711800640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c0cdc50 0x7f870c165f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8711800640 1 -- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f870c0cbeb0 msgr2=0x7f870c140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8711800640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f870c0cbeb0 0x7f870c140e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8711800640 1 -- 172.21.15.46:0/505460623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c143b60 con 0x7f870c159230 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8710e00640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f870c0cbeb0 0x7f870c140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8711800640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c159230 0x7f870c15b060 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f87040b0040 tx=0x7f8704019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8708c00640 1 -- 172.21.15.46:0/505460623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87040b2050 con 0x7f870c159230 2024-04-25T08:50:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8708c00640 1 -- 172.21.15.46:0/505460623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87040be070 con 0x7f870c159230 2024-04-25T08:50:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8708c00640 1 -- 172.21.15.46:0/505460623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87040ada50 con 0x7f870c159230 2024-04-25T08:50:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c143e40 con 0x7f870c159230 2024-04-25T08:50:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c144400 con 0x7f870c159230 2024-04-25T08:50:52.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c069080 con 0x7f870c159230 2024-04-25T08:50:52.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.039+0000 7f8708c00640 1 -- 172.21.15.46:0/505460623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87040adbf0 con 0x7f870c159230 2024-04-25T08:50:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.043+0000 7f8708c00640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86ec066f90 0x7f86ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.043+0000 7f8708c00640 1 -- 172.21.15.46:0/505460623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f870411fbd0 con 0x7f870c159230 2024-04-25T08:50:52.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.043+0000 7f8710e00640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86ec066f90 0x7f86ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.043+0000 7f8710e00640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86ec066f90 0x7f86ec069450 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8700009b10 tx=0x7f8700008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:52.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.047+0000 7f8708c00640 1 -- 172.21.15.46:0/505460623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87040c7020 con 0x7f870c159230 2024-04-25T08:50:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f870c0723d0 con 0x7f870c159230 2024-04-25T08:50:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8708c00640 1 -- 172.21.15.46:0/505460623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16836 (secure 0 0 0) 0x7f8704016560 con 0x7f870c159230 2024-04-25T08:50:52.202 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:52.202 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:50:49.457661+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:50:45.647599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:50:46.857198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86ec066f90 msgr2=0x7f86ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86ec066f90 0x7f86ec069450 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8700009b10 tx=0x7f8700008040 comp rx=0 tx=0).stop 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c159230 msgr2=0x7f870c15b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c159230 0x7f870c15b060 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f87040b0040 tx=0x7f8704019650 comp rx=0 tx=0).stop 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 shutdown_connections 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f86ec066f90 0x7f86ec069450 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f870c159230 0x7f870c15b060 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f870c0cdc50 0x7f870c165f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 --2- 172.21.15.46:0/505460623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f870c0cbeb0 0x7f870c140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 >> 172.21.15.46:0/505460623 conn(0x7f870c0b12a0 msgr2=0x7f870c0cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 shutdown_connections 2024-04-25T08:50:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.211+0000 7f8713800640 1 -- 172.21.15.46:0/505460623 wait complete. 2024-04-25T08:50:52.223 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:50:52.223 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:50:52.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd91a600640 1 Processor -- start 2024-04-25T08:50:52.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd91a600640 1 -- start start 2024-04-25T08:50:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd91a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91409a610 0x7fd91409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd91a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd914099a00 0x7fd914099e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd91a600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91415f300 con 0x7fd91415ef20 2024-04-25T08:50:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd91a600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140cd8a0 con 0x7fd914099a00 2024-04-25T08:50:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd91a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140cddb0 con 0x7fd91409a610 2024-04-25T08:50:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd913e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd914099a00 0x7fd914099e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd913400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd91415ef20 0x7fd9140cd190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45026/0 (socket says 172.21.15.46:45026) 2024-04-25T08:50:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd913400640 1 -- 172.21.15.46:0/1946300287 learned_addr learned my addr 172.21.15.46:0/1946300287 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd913e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd914099a00 0x7fd914099e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40128/0 (socket says 172.21.15.46:40128) 2024-04-25T08:50:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd90aa00640 1 --2- 172.21.15.46:0/1946300287 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91409a610 0x7fd91409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd913e00640 1 -- 172.21.15.46:0/1946300287 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91409a610 msgr2=0x7fd91409aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd913e00640 1 --2- 172.21.15.46:0/1946300287 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91409a610 0x7fd91409aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd913e00640 1 -- 172.21.15.46:0/1946300287 >> v1:172.21.15.46:6789/0 conn(0x7fd91415ef20 legacy=0x7fd9140cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd913e00640 1 -- 172.21.15.46:0/1946300287 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140cc3d0 con 0x7fd914099a00 2024-04-25T08:50:52.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.375+0000 7fd912a00640 1 -- 172.21.15.46:0/1946300287 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078028972 0 0) 0x7fd91415f300 con 0x7fd91415ef20 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd913e00640 1 --2- 172.21.15.46:0/1946300287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd914099a00 0x7fd914099e90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0097c0 tx=0x7fd90c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2251c2dbc3c116 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd912a00640 1 -- 172.21.15.46:0/1946300287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0b0070 con 0x7fd914099a00 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd912a00640 1 -- 172.21.15.46:0/1946300287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c0ab600 con 0x7fd914099a00 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd912a00640 1 -- 172.21.15.46:0/1946300287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0ab9d0 con 0x7fd914099a00 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd91a600640 1 -- 172.21.15.46:0/1946300287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd914099a00 msgr2=0x7fd914099e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd91a600640 1 --2- 172.21.15.46:0/1946300287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd914099a00 0x7fd914099e90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fd90c0097c0 tx=0x7fd90c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd91a600640 1 -- 172.21.15.46:0/1946300287 shutdown_connections 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd91a600640 1 --2- 172.21.15.46:0/1946300287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd914099a00 0x7fd914099e90 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd91a600640 1 --2- 172.21.15.46:0/1946300287 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91409a610 0x7fd91409aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd91a600640 1 -- 172.21.15.46:0/1946300287 >> 172.21.15.46:0/1946300287 conn(0x7fd9140b3120 msgr2=0x7fd9140b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd91a600640 1 -- 172.21.15.46:0/1946300287 shutdown_connections 2024-04-25T08:50:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.379+0000 7fd91a600640 1 -- 172.21.15.46:0/1946300287 wait complete. 2024-04-25T08:50:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 Processor -- start 2024-04-25T08:50:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 -- start start 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd914099a00 0x7fd9140b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd91409a610 0x7fd914151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91415ef20 0x7fd914070a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140cc190 con 0x7fd91409a610 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140cc010 con 0x7fd914099a00 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd914168880 con 0x7fd91415ef20 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd91409a610 0x7fd914151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd91409a610 0x7fd914151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40140/0 (socket says 172.21.15.46:40140) 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90aa00640 1 -- 172.21.15.46:0/4060606866 learned_addr learned my addr 172.21.15.46:0/4060606866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd913e00640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91415ef20 0x7fd914070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd913400640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd914099a00 0x7fd9140b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90aa00640 1 -- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91415ef20 msgr2=0x7fd914070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90aa00640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91415ef20 0x7fd914070a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90aa00640 1 -- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd914099a00 msgr2=0x7fd9140b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90aa00640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd914099a00 0x7fd9140b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90aa00640 1 -- 172.21.15.46:0/4060606866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140b4c20 con 0x7fd91409a610 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd913e00640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91415ef20 0x7fd914070a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90aa00640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd91409a610 0x7fd914151fa0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc007e70 tx=0x7fd8fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90be00640 1 -- 172.21.15.46:0/4060606866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc014620 con 0x7fd91409a610 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90be00640 1 -- 172.21.15.46:0/4060606866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc019070 con 0x7fd91409a610 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9140b4ea0 con 0x7fd91409a610 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd90be00640 1 -- 172.21.15.46:0/4060606866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc014a00 con 0x7fd91409a610 2024-04-25T08:50:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9140b5440 con 0x7fd91409a610 2024-04-25T08:50:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.383+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd914069080 con 0x7fd91409a610 2024-04-25T08:50:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.387+0000 7fd90be00640 1 -- 172.21.15.46:0/4060606866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8fc014ba0 con 0x7fd91409a610 2024-04-25T08:50:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.387+0000 7fd90be00640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f0066f90 0x7fd8f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.387+0000 7fd90be00640 1 -- 172.21.15.46:0/4060606866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8fc085a00 con 0x7fd91409a610 2024-04-25T08:50:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.387+0000 7fd913400640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f0066f90 0x7fd8f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.387+0000 7fd913400640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f0066f90 0x7fd8f0069450 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd904000f70 tx=0x7fd904007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.391+0000 7fd90be00640 1 -- 172.21.15.46:0/4060606866 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9140b4c20 con 0x7fd91409a610 2024-04-25T08:50:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.555+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9140d0700 con 0x7fd8f0066f90 2024-04-25T08:50:52.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.563+0000 7fd90be00640 1 -- 172.21.15.46:0/4060606866 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7fd9140d0700 con 0x7fd8f0066f90 2024-04-25T08:50:52.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:52.560 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:50:52.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f0066f90 msgr2=0x7fd8f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f0066f90 0x7fd8f0069450 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd904000f70 tx=0x7fd904007b60 comp rx=0 tx=0).stop 2024-04-25T08:50:52.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd91409a610 msgr2=0x7fd914151fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd91409a610 0x7fd914151fa0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc007e70 tx=0x7fd8fc00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 shutdown_connections 2024-04-25T08:50:52.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f0066f90 0x7fd8f0069450 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd91415ef20 0x7fd914070a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd91409a610 0x7fd914151fa0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 --2- 172.21.15.46:0/4060606866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd914099a00 0x7fd9140b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 >> 172.21.15.46:0/4060606866 conn(0x7fd9140b3120 msgr2=0x7fd9140c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:52.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 shutdown_connections 2024-04-25T08:50:52.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.567+0000 7fd91a600640 1 -- 172.21.15.46:0/4060606866 wait complete. 2024-04-25T08:50:52.578 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-25T08:50:52.385129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260952,"kb_used_data":11496,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713768,"statfs":{"total":773094113280,"available":772826898432,"internally_reserved":0,"allocated":11771904,"data_stored":6080828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.935186"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463695+0000","last_change":"2024-04-25T08:50:48.480492+0000","last_active":"2024-04-25T08:50:49.463695+0000","last_peered":"2024-04-25T08:50:49.463695+0000","last_clean":"2024-04-25T08:50:49.463695+0000","last_became_active":"2024-04-25T08:50:48.480001+0000","last_became_peered":"2024-04-25T08:50:48.480001+0000","last_unstale":"2024-04-25T08:50:49.463695+0000","last_undegraded":"2024-04-25T08:50:49.463695+0000","last_fullsized":"2024-04-25T08:50:49.463695+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:23.222694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663660+0000","last_change":"2024-04-25T08:50:48.478572+0000","last_active":"2024-04-25T08:50:50.663660+0000","last_peered":"2024-04-25T08:50:50.663660+0000","last_clean":"2024-04-25T08:50:50.663660+0000","last_became_active":"2024-04-25T08:50:48.478286+0000","last_became_peered":"2024-04-25T08:50:48.478286+0000","last_unstale":"2024-04-25T08:50:50.663660+0000","last_undegraded":"2024-04-25T08:50:50.663660+0000","last_fullsized":"2024-04-25T08:50:50.663660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:00.051922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463567+0000","last_change":"2024-04-25T08:50:48.495025+0000","last_active":"2024-04-25T08:50:49.463567+0000","last_peered":"2024-04-25T08:50:49.463567+0000","last_clean":"2024-04-25T08:50:49.463567+0000","last_became_active":"2024-04-25T08:50:48.494746+0000","last_became_peered":"2024-04-25T08:50:48.494746+0000","last_unstale":"2024-04-25T08:50:49.463567+0000","last_undegraded":"2024-04-25T08:50:49.463567+0000","last_fullsized":"2024-04-25T08:50:49.463567+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:45.190214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462565+0000","last_change":"2024-04-25T08:50:48.475449+0000","last_active":"2024-04-25T08:50:49.462565+0000","last_peered":"2024-04-25T08:50:49.462565+0000","last_clean":"2024-04-25T08:50:49.462565+0000","last_became_active":"2024-04-25T08:50:48.475137+0000","last_became_peered":"2024-04-25T08:50:48.475137+0000","last_unstale":"2024-04-25T08:50:49.462565+0000","last_undegraded":"2024-04-25T08:50:49.462565+0000","last_fullsized":"2024-04-25T08:50:49.462565+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:25.480050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660663+0000","last_change":"2024-04-25T08:50:48.467088+0000","last_active":"2024-04-25T08:50:50.660663+0000","last_peered":"2024-04-25T08:50:50.660663+0000","last_clean":"2024-04-25T08:50:50.660663+0000","last_became_active":"2024-04-25T08:50:48.466790+0000","last_became_peered":"2024-04-25T08:50:48.466790+0000","last_unstale":"2024-04-25T08:50:50.660663+0000","last_undegraded":"2024-04-25T08:50:50.660663+0000","last_fullsized":"2024-04-25T08:50:50.660663+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:38.926179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663184+0000","last_change":"2024-04-25T08:50:48.486891+0000","last_active":"2024-04-25T08:50:50.663184+0000","last_peered":"2024-04-25T08:50:50.663184+0000","last_clean":"2024-04-25T08:50:50.663184+0000","last_became_active":"2024-04-25T08:50:48.485703+0000","last_became_peered":"2024-04-25T08:50:48.485703+0000","last_unstale":"2024-04-25T08:50:50.663184+0000","last_undegraded":"2024-04-25T08:50:50.663184+0000","last_fullsized":"2024-04-25T08:50:50.663184+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:52.126434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663317+0000","last_change":"2024-04-25T08:50:48.476906+0000","last_active":"2024-04-25T08:50:50.663317+0000","last_peered":"2024-04-25T08:50:50.663317+0000","last_clean":"2024-04-25T08:50:50.663317+0000","last_became_active":"2024-04-25T08:50:48.476148+0000","last_became_peered":"2024-04-25T08:50:48.476148+0000","last_unstale":"2024-04-25T08:50:50.663317+0000","last_undegraded":"2024-04-25T08:50:50.663317+0000","last_fullsized":"2024-04-25T08:50:50.663317+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:19.646246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.659996+0000","last_change":"2024-04-25T08:50:48.482855+0000","last_active":"2024-04-25T08:50:50.659996+0000","last_peered":"2024-04-25T08:50:50.659996+0000","last_clean":"2024-04-25T08:50:50.659996+0000","last_became_active":"2024-04-25T08:50:48.482682+0000","last_became_peered":"2024-04-25T08:50:48.482682+0000","last_unstale":"2024-04-25T08:50:50.659996+0000","last_undegraded":"2024-04-25T08:50:50.659996+0000","last_fullsized":"2024-04-25T08:50:50.659996+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:21.014431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663763+0000","last_change":"2024-04-25T08:50:48.481048+0000","last_active":"2024-04-25T08:50:50.663763+0000","last_peered":"2024-04-25T08:50:50.663763+0000","last_clean":"2024-04-25T08:50:50.663763+0000","last_became_active":"2024-04-25T08:50:48.480757+0000","last_became_peered":"2024-04-25T08:50:48.480757+0000","last_unstale":"2024-04-25T08:50:50.663763+0000","last_undegraded":"2024-04-25T08:50:50.663763+0000","last_fullsized":"2024-04-25T08:50:50.663763+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:48.132422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661393+0000","last_change":"2024-04-25T08:50:48.476393+0000","last_active":"2024-04-25T08:50:50.661393+0000","last_peered":"2024-04-25T08:50:50.661393+0000","last_clean":"2024-04-25T08:50:50.661393+0000","last_became_active":"2024-04-25T08:50:48.476101+0000","last_became_peered":"2024-04-25T08:50:48.476101+0000","last_unstale":"2024-04-25T08:50:50.661393+0000","last_undegraded":"2024-04-25T08:50:50.661393+0000","last_fullsized":"2024-04-25T08:50:50.661393+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:19.646246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490631+0000","last_change":"2024-04-25T08:50:48.485607+0000","last_active":"2024-04-25T08:50:49.490631+0000","last_peered":"2024-04-25T08:50:49.490631+0000","last_clean":"2024-04-25T08:50:49.490631+0000","last_became_active":"2024-04-25T08:50:48.485245+0000","last_became_peered":"2024-04-25T08:50:48.485245+0000","last_unstale":"2024-04-25T08:50:49.490631+0000","last_undegraded":"2024-04-25T08:50:49.490631+0000","last_fullsized":"2024-04-25T08:50:49.490631+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:19.646246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466107+0000","last_change":"2024-04-25T08:50:48.479907+0000","last_active":"2024-04-25T08:50:49.466107+0000","last_peered":"2024-04-25T08:50:49.466107+0000","last_clean":"2024-04-25T08:50:49.466107+0000","last_became_active":"2024-04-25T08:50:48.479391+0000","last_became_peered":"2024-04-25T08:50:48.479391+0000","last_unstale":"2024-04-25T08:50:49.466107+0000","last_undegraded":"2024-04-25T08:50:49.466107+0000","last_fullsized":"2024-04-25T08:50:49.466107+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:07.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462270+0000","last_change":"2024-04-25T08:50:48.487454+0000","last_active":"2024-04-25T08:50:49.462270+0000","last_peered":"2024-04-25T08:50:49.462270+0000","last_clean":"2024-04-25T08:50:49.462270+0000","last_became_active":"2024-04-25T08:50:48.487185+0000","last_became_peered":"2024-04-25T08:50:48.487185+0000","last_unstale":"2024-04-25T08:50:49.462270+0000","last_undegraded":"2024-04-25T08:50:49.462270+0000","last_fullsized":"2024-04-25T08:50:49.462270+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:31.314109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663186+0000","last_change":"2024-04-25T08:50:48.480452+0000","last_active":"2024-04-25T08:50:50.663186+0000","last_peered":"2024-04-25T08:50:50.663186+0000","last_clean":"2024-04-25T08:50:50.663186+0000","last_became_active":"2024-04-25T08:50:48.480218+0000","last_became_peered":"2024-04-25T08:50:48.480218+0000","last_unstale":"2024-04-25T08:50:50.663186+0000","last_undegraded":"2024-04-25T08:50:50.663186+0000","last_fullsized":"2024-04-25T08:50:50.663186+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:31.314109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662915+0000","last_change":"2024-04-25T08:50:48.491310+0000","last_active":"2024-04-25T08:50:50.662915+0000","last_peered":"2024-04-25T08:50:50.662915+0000","last_clean":"2024-04-25T08:50:50.662915+0000","last_became_active":"2024-04-25T08:50:48.490954+0000","last_became_peered":"2024-04-25T08:50:48.490954+0000","last_unstale":"2024-04-25T08:50:50.662915+0000","last_undegraded":"2024-04-25T08:50:50.662915+0000","last_fullsized":"2024-04-25T08:50:50.662915+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:39.031265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662534+0000","last_change":"2024-04-25T08:50:48.483454+0000","last_active":"2024-04-25T08:50:50.662534+0000","last_peered":"2024-04-25T08:50:50.662534+0000","last_clean":"2024-04-25T08:50:50.662534+0000","last_became_active":"2024-04-25T08:50:48.483119+0000","last_became_peered":"2024-04-25T08:50:48.483119+0000","last_unstale":"2024-04-25T08:50:50.662534+0000","last_undegraded":"2024-04-25T08:50:50.662534+0000","last_fullsized":"2024-04-25T08:50:50.662534+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:02.994114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463703+0000","last_change":"2024-04-25T08:50:48.484656+0000","last_active":"2024-04-25T08:50:49.463703+0000","last_peered":"2024-04-25T08:50:49.463703+0000","last_clean":"2024-04-25T08:50:49.463703+0000","last_became_active":"2024-04-25T08:50:48.484324+0000","last_became_peered":"2024-04-25T08:50:48.484324+0000","last_unstale":"2024-04-25T08:50:49.463703+0000","last_undegraded":"2024-04-25T08:50:49.463703+0000","last_fullsized":"2024-04-25T08:50:49.463703+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:25.144536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663773+0000","last_change":"2024-04-25T08:50:48.486940+0000","last_active":"2024-04-25T08:50:50.663773+0000","last_peered":"2024-04-25T08:50:50.663773+0000","last_clean":"2024-04-25T08:50:50.663773+0000","last_became_active":"2024-04-25T08:50:48.486396+0000","last_became_peered":"2024-04-25T08:50:48.486396+0000","last_unstale":"2024-04-25T08:50:50.663773+0000","last_undegraded":"2024-04-25T08:50:50.663773+0000","last_fullsized":"2024-04-25T08:50:50.663773+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:43.918413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463588+0000","last_change":"2024-04-25T08:50:48.477793+0000","last_active":"2024-04-25T08:50:49.463588+0000","last_peered":"2024-04-25T08:50:49.463588+0000","last_clean":"2024-04-25T08:50:49.463588+0000","last_became_active":"2024-04-25T08:50:48.477502+0000","last_became_peered":"2024-04-25T08:50:48.477502+0000","last_unstale":"2024-04-25T08:50:49.463588+0000","last_undegraded":"2024-04-25T08:50:49.463588+0000","last_fullsized":"2024-04-25T08:50:49.463588+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:51.145418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465924+0000","last_change":"2024-04-25T08:50:48.473067+0000","last_active":"2024-04-25T08:50:49.465924+0000","last_peered":"2024-04-25T08:50:49.465924+0000","last_clean":"2024-04-25T08:50:49.465924+0000","last_became_active":"2024-04-25T08:50:48.472748+0000","last_became_peered":"2024-04-25T08:50:48.472748+0000","last_unstale":"2024-04-25T08:50:49.465924+0000","last_undegraded":"2024-04-25T08:50:49.465924+0000","last_fullsized":"2024-04-25T08:50:49.465924+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:40.772018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661092+0000","last_change":"2024-04-25T08:50:48.488160+0000","last_active":"2024-04-25T08:50:50.661092+0000","last_peered":"2024-04-25T08:50:50.661092+0000","last_clean":"2024-04-25T08:50:50.661092+0000","last_became_active":"2024-04-25T08:50:48.487856+0000","last_became_peered":"2024-04-25T08:50:48.487856+0000","last_unstale":"2024-04-25T08:50:50.661092+0000","last_undegraded":"2024-04-25T08:50:50.661092+0000","last_fullsized":"2024-04-25T08:50:50.661092+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:50.544561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660897+0000","last_change":"2024-04-25T08:50:48.492748+0000","last_active":"2024-04-25T08:50:50.660897+0000","last_peered":"2024-04-25T08:50:50.660897+0000","last_clean":"2024-04-25T08:50:50.660897+0000","last_became_active":"2024-04-25T08:50:48.492330+0000","last_became_peered":"2024-04-25T08:50:48.492330+0000","last_unstale":"2024-04-25T08:50:50.660897+0000","last_undegraded":"2024-04-25T08:50:50.660897+0000","last_fullsized":"2024-04-25T08:50:50.660897+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:30.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463289+0000","last_change":"2024-04-25T08:50:48.488353+0000","last_active":"2024-04-25T08:50:49.463289+0000","last_peered":"2024-04-25T08:50:49.463289+0000","last_clean":"2024-04-25T08:50:49.463289+0000","last_became_active":"2024-04-25T08:50:48.487815+0000","last_became_peered":"2024-04-25T08:50:48.487815+0000","last_unstale":"2024-04-25T08:50:49.463289+0000","last_undegraded":"2024-04-25T08:50:49.463289+0000","last_fullsized":"2024-04-25T08:50:49.463289+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:47.577495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660187+0000","last_change":"2024-04-25T08:50:48.467030+0000","last_active":"2024-04-25T08:50:50.660187+0000","last_peered":"2024-04-25T08:50:50.660187+0000","last_clean":"2024-04-25T08:50:50.660187+0000","last_became_active":"2024-04-25T08:50:48.466258+0000","last_became_peered":"2024-04-25T08:50:48.466258+0000","last_unstale":"2024-04-25T08:50:50.660187+0000","last_undegraded":"2024-04-25T08:50:50.660187+0000","last_fullsized":"2024-04-25T08:50:50.660187+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:03.616967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466080+0000","last_change":"2024-04-25T08:50:48.489451+0000","last_active":"2024-04-25T08:50:49.466080+0000","last_peered":"2024-04-25T08:50:49.466080+0000","last_clean":"2024-04-25T08:50:49.466080+0000","last_became_active":"2024-04-25T08:50:48.489150+0000","last_became_peered":"2024-04-25T08:50:48.489150+0000","last_unstale":"2024-04-25T08:50:49.466080+0000","last_undegraded":"2024-04-25T08:50:49.466080+0000","last_fullsized":"2024-04-25T08:50:49.466080+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:43.918413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465959+0000","last_change":"2024-04-25T08:50:48.481618+0000","last_active":"2024-04-25T08:50:49.465959+0000","last_peered":"2024-04-25T08:50:49.465959+0000","last_clean":"2024-04-25T08:50:49.465959+0000","last_became_active":"2024-04-25T08:50:48.481275+0000","last_became_peered":"2024-04-25T08:50:48.481275+0000","last_unstale":"2024-04-25T08:50:49.465959+0000","last_undegraded":"2024-04-25T08:50:49.465959+0000","last_fullsized":"2024-04-25T08:50:49.465959+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:00.051922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490732+0000","last_change":"2024-04-25T08:50:48.490030+0000","last_active":"2024-04-25T08:50:49.490732+0000","last_peered":"2024-04-25T08:50:49.490732+0000","last_clean":"2024-04-25T08:50:49.490732+0000","last_became_active":"2024-04-25T08:50:48.489676+0000","last_became_peered":"2024-04-25T08:50:48.489676+0000","last_unstale":"2024-04-25T08:50:49.490732+0000","last_undegraded":"2024-04-25T08:50:49.490732+0000","last_fullsized":"2024-04-25T08:50:49.490732+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:40.772018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663434+0000","last_change":"2024-04-25T08:50:48.487672+0000","last_active":"2024-04-25T08:50:50.663434+0000","last_peered":"2024-04-25T08:50:50.663434+0000","last_clean":"2024-04-25T08:50:50.663434+0000","last_became_active":"2024-04-25T08:50:48.487381+0000","last_became_peered":"2024-04-25T08:50:48.487381+0000","last_unstale":"2024-04-25T08:50:50.663434+0000","last_undegraded":"2024-04-25T08:50:50.663434+0000","last_fullsized":"2024-04-25T08:50:50.663434+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.676510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661018+0000","last_change":"2024-04-25T08:50:48.481642+0000","last_active":"2024-04-25T08:50:50.661018+0000","last_peered":"2024-04-25T08:50:50.661018+0000","last_clean":"2024-04-25T08:50:50.661018+0000","last_became_active":"2024-04-25T08:50:48.481347+0000","last_became_peered":"2024-04-25T08:50:48.481347+0000","last_unstale":"2024-04-25T08:50:50.661018+0000","last_undegraded":"2024-04-25T08:50:50.661018+0000","last_fullsized":"2024-04-25T08:50:50.661018+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:46.644691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466093+0000","last_change":"2024-04-25T08:50:48.489807+0000","last_active":"2024-04-25T08:50:49.466093+0000","last_peered":"2024-04-25T08:50:49.466093+0000","last_clean":"2024-04-25T08:50:49.466093+0000","last_became_active":"2024-04-25T08:50:48.489100+0000","last_became_peered":"2024-04-25T08:50:48.489100+0000","last_unstale":"2024-04-25T08:50:49.466093+0000","last_undegraded":"2024-04-25T08:50:49.466093+0000","last_fullsized":"2024-04-25T08:50:49.466093+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:48.132422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662801+0000","last_change":"2024-04-25T08:50:48.476136+0000","last_active":"2024-04-25T08:50:50.662801+0000","last_peered":"2024-04-25T08:50:50.662801+0000","last_clean":"2024-04-25T08:50:50.662801+0000","last_became_active":"2024-04-25T08:50:48.475543+0000","last_became_peered":"2024-04-25T08:50:48.475543+0000","last_unstale":"2024-04-25T08:50:50.662801+0000","last_undegraded":"2024-04-25T08:50:50.662801+0000","last_fullsized":"2024-04-25T08:50:50.662801+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:04.379008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660036+0000","last_change":"2024-04-25T08:50:48.485951+0000","last_active":"2024-04-25T08:50:50.660036+0000","last_peered":"2024-04-25T08:50:50.660036+0000","last_clean":"2024-04-25T08:50:50.660036+0000","last_became_active":"2024-04-25T08:50:48.485529+0000","last_became_peered":"2024-04-25T08:50:48.485529+0000","last_unstale":"2024-04-25T08:50:50.660036+0000","last_undegraded":"2024-04-25T08:50:50.660036+0000","last_fullsized":"2024-04-25T08:50:50.660036+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:46.707610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463505+0000","last_change":"2024-04-25T08:50:48.477441+0000","last_active":"2024-04-25T08:50:49.463505+0000","last_peered":"2024-04-25T08:50:49.463505+0000","last_clean":"2024-04-25T08:50:49.463505+0000","last_became_active":"2024-04-25T08:50:48.477134+0000","last_became_peered":"2024-04-25T08:50:48.477134+0000","last_unstale":"2024-04-25T08:50:49.463505+0000","last_undegraded":"2024-04-25T08:50:49.463505+0000","last_fullsized":"2024-04-25T08:50:49.463505+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:29.410218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462932+0000","last_change":"2024-04-25T08:50:47.479235+0000","last_active":"2024-04-25T08:50:49.462932+0000","last_peered":"2024-04-25T08:50:49.462932+0000","last_clean":"2024-04-25T08:50:49.462932+0000","last_became_active":"2024-04-25T08:50:47.478942+0000","last_became_peered":"2024-04-25T08:50:47.478942+0000","last_unstale":"2024-04-25T08:50:49.462932+0000","last_undegraded":"2024-04-25T08:50:49.462932+0000","last_fullsized":"2024-04-25T08:50:49.462932+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:46.669342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660077+0000","last_change":"2024-04-25T08:50:48.489948+0000","last_active":"2024-04-25T08:50:50.660077+0000","last_peered":"2024-04-25T08:50:50.660077+0000","last_clean":"2024-04-25T08:50:50.660077+0000","last_became_active":"2024-04-25T08:50:48.488512+0000","last_became_peered":"2024-04-25T08:50:48.488512+0000","last_unstale":"2024-04-25T08:50:50.660077+0000","last_undegraded":"2024-04-25T08:50:50.660077+0000","last_fullsized":"2024-04-25T08:50:50.660077+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:02.484051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661226+0000","last_change":"2024-04-25T08:50:47.480237+0000","last_active":"2024-04-25T08:50:50.661226+0000","last_peered":"2024-04-25T08:50:50.661226+0000","last_clean":"2024-04-25T08:50:50.661226+0000","last_became_active":"2024-04-25T08:50:47.479958+0000","last_became_peered":"2024-04-25T08:50:47.479958+0000","last_unstale":"2024-04-25T08:50:50.661226+0000","last_undegraded":"2024-04-25T08:50:50.661226+0000","last_fullsized":"2024-04-25T08:50:50.661226+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:11.578821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466276+0000","last_change":"2024-04-25T08:50:48.471546+0000","last_active":"2024-04-25T08:50:49.466276+0000","last_peered":"2024-04-25T08:50:49.466276+0000","last_clean":"2024-04-25T08:50:49.466276+0000","last_became_active":"2024-04-25T08:50:48.471052+0000","last_became_peered":"2024-04-25T08:50:48.471052+0000","last_unstale":"2024-04-25T08:50:49.466276+0000","last_undegraded":"2024-04-25T08:50:49.466276+0000","last_fullsized":"2024-04-25T08:50:49.466276+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:02.994114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466107+0000","last_change":"2024-04-25T08:50:47.481237+0000","last_active":"2024-04-25T08:50:49.466107+0000","last_peered":"2024-04-25T08:50:49.466107+0000","last_clean":"2024-04-25T08:50:49.466107+0000","last_became_active":"2024-04-25T08:50:47.480947+0000","last_became_peered":"2024-04-25T08:50:47.480947+0000","last_unstale":"2024-04-25T08:50:49.466107+0000","last_undegraded":"2024-04-25T08:50:49.466107+0000","last_fullsized":"2024-04-25T08:50:49.466107+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:46.158215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490657+0000","last_change":"2024-04-25T08:50:48.487849+0000","last_active":"2024-04-25T08:50:49.490657+0000","last_peered":"2024-04-25T08:50:49.490657+0000","last_clean":"2024-04-25T08:50:49.490657+0000","last_became_active":"2024-04-25T08:50:48.487561+0000","last_became_peered":"2024-04-25T08:50:48.487561+0000","last_unstale":"2024-04-25T08:50:49.490657+0000","last_undegraded":"2024-04-25T08:50:49.490657+0000","last_fullsized":"2024-04-25T08:50:49.490657+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.676510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660671+0000","last_change":"2024-04-25T08:50:47.467828+0000","last_active":"2024-04-25T08:50:50.660671+0000","last_peered":"2024-04-25T08:50:50.660671+0000","last_clean":"2024-04-25T08:50:50.660671+0000","last_became_active":"2024-04-25T08:50:47.467530+0000","last_became_peered":"2024-04-25T08:50:47.467530+0000","last_unstale":"2024-04-25T08:50:50.660671+0000","last_undegraded":"2024-04-25T08:50:50.660671+0000","last_fullsized":"2024-04-25T08:50:50.660671+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:10.299319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462782+0000","last_change":"2024-04-25T08:50:48.484439+0000","last_active":"2024-04-25T08:50:49.462782+0000","last_peered":"2024-04-25T08:50:49.462782+0000","last_clean":"2024-04-25T08:50:49.462782+0000","last_became_active":"2024-04-25T08:50:48.484173+0000","last_became_peered":"2024-04-25T08:50:48.484173+0000","last_unstale":"2024-04-25T08:50:49.462782+0000","last_undegraded":"2024-04-25T08:50:49.462782+0000","last_fullsized":"2024-04-25T08:50:49.462782+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:21.446688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662734+0000","last_change":"2024-04-25T08:50:47.484474+0000","last_active":"2024-04-25T08:50:50.662734+0000","last_peered":"2024-04-25T08:50:50.662734+0000","last_clean":"2024-04-25T08:50:50.662734+0000","last_became_active":"2024-04-25T08:50:47.484155+0000","last_became_peered":"2024-04-25T08:50:47.484155+0000","last_unstale":"2024-04-25T08:50:50.662734+0000","last_undegraded":"2024-04-25T08:50:50.662734+0000","last_fullsized":"2024-04-25T08:50:50.662734+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:18.639078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463739+0000","last_change":"2024-04-25T08:50:48.477464+0000","last_active":"2024-04-25T08:50:49.463739+0000","last_peered":"2024-04-25T08:50:49.463739+0000","last_clean":"2024-04-25T08:50:49.463739+0000","last_became_active":"2024-04-25T08:50:48.477139+0000","last_became_peered":"2024-04-25T08:50:48.477139+0000","last_unstale":"2024-04-25T08:50:49.463739+0000","last_undegraded":"2024-04-25T08:50:49.463739+0000","last_fullsized":"2024-04-25T08:50:49.463739+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:13.218720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490310+0000","last_change":"2024-04-25T08:50:47.476837+0000","last_active":"2024-04-25T08:50:49.490310+0000","last_peered":"2024-04-25T08:50:49.490310+0000","last_clean":"2024-04-25T08:50:49.490310+0000","last_became_active":"2024-04-25T08:50:47.476477+0000","last_became_peered":"2024-04-25T08:50:47.476477+0000","last_unstale":"2024-04-25T08:50:49.490310+0000","last_undegraded":"2024-04-25T08:50:49.490310+0000","last_fullsized":"2024-04-25T08:50:49.490310+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:20.854277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663844+0000","last_change":"2024-04-25T08:50:48.490496+0000","last_active":"2024-04-25T08:50:50.663844+0000","last_peered":"2024-04-25T08:50:50.663844+0000","last_clean":"2024-04-25T08:50:50.663844+0000","last_became_active":"2024-04-25T08:50:48.490203+0000","last_became_peered":"2024-04-25T08:50:48.490203+0000","last_unstale":"2024-04-25T08:50:50.663844+0000","last_undegraded":"2024-04-25T08:50:50.663844+0000","last_fullsized":"2024-04-25T08:50:50.663844+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:07.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465546+0000","last_change":"2024-04-25T08:50:47.475588+0000","last_active":"2024-04-25T08:50:49.465546+0000","last_peered":"2024-04-25T08:50:49.465546+0000","last_clean":"2024-04-25T08:50:49.465546+0000","last_became_active":"2024-04-25T08:50:47.475281+0000","last_became_peered":"2024-04-25T08:50:47.475281+0000","last_unstale":"2024-04-25T08:50:49.465546+0000","last_undegraded":"2024-04-25T08:50:49.465546+0000","last_fullsized":"2024-04-25T08:50:49.465546+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:57.813694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463052+0000","last_change":"2024-04-25T08:50:48.479461+0000","last_active":"2024-04-25T08:50:49.463052+0000","last_peered":"2024-04-25T08:50:49.463052+0000","last_clean":"2024-04-25T08:50:49.463052+0000","last_became_active":"2024-04-25T08:50:48.478946+0000","last_became_peered":"2024-04-25T08:50:48.478946+0000","last_unstale":"2024-04-25T08:50:49.463052+0000","last_undegraded":"2024-04-25T08:50:49.463052+0000","last_fullsized":"2024-04-25T08:50:49.463052+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:49.338966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462019+0000","last_change":"2024-04-25T08:50:47.477440+0000","last_active":"2024-04-25T08:50:49.462019+0000","last_peered":"2024-04-25T08:50:49.462019+0000","last_clean":"2024-04-25T08:50:49.462019+0000","last_became_active":"2024-04-25T08:50:47.477161+0000","last_became_peered":"2024-04-25T08:50:47.477161+0000","last_unstale":"2024-04-25T08:50:49.462019+0000","last_undegraded":"2024-04-25T08:50:49.462019+0000","last_fullsized":"2024-04-25T08:50:49.462019+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:18.639078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463245+0000","last_change":"2024-04-25T08:50:48.473937+0000","last_active":"2024-04-25T08:50:49.463245+0000","last_peered":"2024-04-25T08:50:49.463245+0000","last_clean":"2024-04-25T08:50:49.463245+0000","last_became_active":"2024-04-25T08:50:48.473638+0000","last_became_peered":"2024-04-25T08:50:48.473638+0000","last_unstale":"2024-04-25T08:50:49.463245+0000","last_undegraded":"2024-04-25T08:50:49.463245+0000","last_fullsized":"2024-04-25T08:50:49.463245+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:53.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465936+0000","last_change":"2024-04-25T08:50:47.485270+0000","last_active":"2024-04-25T08:50:49.465936+0000","last_peered":"2024-04-25T08:50:49.465936+0000","last_clean":"2024-04-25T08:50:49.465936+0000","last_became_active":"2024-04-25T08:50:47.481349+0000","last_became_peered":"2024-04-25T08:50:47.481349+0000","last_unstale":"2024-04-25T08:50:49.465936+0000","last_undegraded":"2024-04-25T08:50:49.465936+0000","last_fullsized":"2024-04-25T08:50:49.465936+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:45.365914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660497+0000","last_change":"2024-04-25T08:50:48.493623+0000","last_active":"2024-04-25T08:50:50.660497+0000","last_peered":"2024-04-25T08:50:50.660497+0000","last_clean":"2024-04-25T08:50:50.660497+0000","last_became_active":"2024-04-25T08:50:48.493278+0000","last_became_peered":"2024-04-25T08:50:48.493278+0000","last_unstale":"2024-04-25T08:50:50.660497+0000","last_undegraded":"2024-04-25T08:50:50.660497+0000","last_fullsized":"2024-04-25T08:50:50.660497+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:20.656908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463480+0000","last_change":"2024-04-25T08:50:47.476603+0000","last_active":"2024-04-25T08:50:49.463480+0000","last_peered":"2024-04-25T08:50:49.463480+0000","last_clean":"2024-04-25T08:50:49.463480+0000","last_became_active":"2024-04-25T08:50:47.476321+0000","last_became_peered":"2024-04-25T08:50:47.476321+0000","last_unstale":"2024-04-25T08:50:49.463480+0000","last_undegraded":"2024-04-25T08:50:49.463480+0000","last_fullsized":"2024-04-25T08:50:49.463480+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:09.022500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660402+0000","last_change":"2024-04-25T08:50:48.489430+0000","last_active":"2024-04-25T08:50:50.660402+0000","last_peered":"2024-04-25T08:50:50.660402+0000","last_clean":"2024-04-25T08:50:50.660402+0000","last_became_active":"2024-04-25T08:50:48.489153+0000","last_became_peered":"2024-04-25T08:50:48.489153+0000","last_unstale":"2024-04-25T08:50:50.660402+0000","last_undegraded":"2024-04-25T08:50:50.660402+0000","last_fullsized":"2024-04-25T08:50:50.660402+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:26.106123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662913+0000","last_change":"2024-04-25T08:50:47.485045+0000","last_active":"2024-04-25T08:50:50.662913+0000","last_peered":"2024-04-25T08:50:50.662913+0000","last_clean":"2024-04-25T08:50:50.662913+0000","last_became_active":"2024-04-25T08:50:47.484741+0000","last_became_peered":"2024-04-25T08:50:47.484741+0000","last_unstale":"2024-04-25T08:50:50.662913+0000","last_undegraded":"2024-04-25T08:50:50.662913+0000","last_fullsized":"2024-04-25T08:50:50.662913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:46.669342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660803+0000","last_change":"2024-04-25T08:50:48.488504+0000","last_active":"2024-04-25T08:50:50.660803+0000","last_peered":"2024-04-25T08:50:50.660803+0000","last_clean":"2024-04-25T08:50:50.660803+0000","last_became_active":"2024-04-25T08:50:48.488192+0000","last_became_peered":"2024-04-25T08:50:48.488192+0000","last_unstale":"2024-04-25T08:50:50.660803+0000","last_undegraded":"2024-04-25T08:50:50.660803+0000","last_fullsized":"2024-04-25T08:50:50.660803+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:52.126434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663469+0000","last_change":"2024-04-25T08:50:47.484075+0000","last_active":"2024-04-25T08:50:50.663469+0000","last_peered":"2024-04-25T08:50:50.663469+0000","last_clean":"2024-04-25T08:50:50.663469+0000","last_became_active":"2024-04-25T08:50:47.483508+0000","last_became_peered":"2024-04-25T08:50:47.483508+0000","last_unstale":"2024-04-25T08:50:50.663469+0000","last_undegraded":"2024-04-25T08:50:50.663469+0000","last_fullsized":"2024-04-25T08:50:50.663469+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:10.299319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490528+0000","last_change":"2024-04-25T08:50:48.490545+0000","last_active":"2024-04-25T08:50:49.490528+0000","last_peered":"2024-04-25T08:50:49.490528+0000","last_clean":"2024-04-25T08:50:49.490528+0000","last_became_active":"2024-04-25T08:50:48.490120+0000","last_became_peered":"2024-04-25T08:50:48.490120+0000","last_unstale":"2024-04-25T08:50:49.490528+0000","last_undegraded":"2024-04-25T08:50:49.490528+0000","last_fullsized":"2024-04-25T08:50:49.490528+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:52.126434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660740+0000","last_change":"2024-04-25T08:50:47.484670+0000","last_active":"2024-04-25T08:50:50.660740+0000","last_peered":"2024-04-25T08:50:50.660740+0000","last_clean":"2024-04-25T08:50:50.660740+0000","last_became_active":"2024-04-25T08:50:47.483722+0000","last_became_peered":"2024-04-25T08:50:47.483722+0000","last_unstale":"2024-04-25T08:50:50.660740+0000","last_undegraded":"2024-04-25T08:50:50.660740+0000","last_fullsized":"2024-04-25T08:50:50.660740+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:02.515846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663584+0000","last_change":"2024-04-25T08:50:48.478959+0000","last_active":"2024-04-25T08:50:50.663584+0000","last_peered":"2024-04-25T08:50:50.663584+0000","last_clean":"2024-04-25T08:50:50.663584+0000","last_became_active":"2024-04-25T08:50:48.478690+0000","last_became_peered":"2024-04-25T08:50:48.478690+0000","last_unstale":"2024-04-25T08:50:50.663584+0000","last_undegraded":"2024-04-25T08:50:50.663584+0000","last_fullsized":"2024-04-25T08:50:50.663584+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:40.772018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462347+0000","last_change":"2024-04-25T08:50:47.473100+0000","last_active":"2024-04-25T08:50:49.462347+0000","last_peered":"2024-04-25T08:50:49.462347+0000","last_clean":"2024-04-25T08:50:49.462347+0000","last_became_active":"2024-04-25T08:50:47.472754+0000","last_became_peered":"2024-04-25T08:50:47.472754+0000","last_unstale":"2024-04-25T08:50:49.462347+0000","last_undegraded":"2024-04-25T08:50:49.462347+0000","last_fullsized":"2024-04-25T08:50:49.462347+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:39.764851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661248+0000","last_change":"2024-04-25T08:50:48.486298+0000","last_active":"2024-04-25T08:50:50.661248+0000","last_peered":"2024-04-25T08:50:50.661248+0000","last_clean":"2024-04-25T08:50:50.661248+0000","last_became_active":"2024-04-25T08:50:48.486012+0000","last_became_peered":"2024-04-25T08:50:48.486012+0000","last_unstale":"2024-04-25T08:50:50.661248+0000","last_undegraded":"2024-04-25T08:50:50.661248+0000","last_fullsized":"2024-04-25T08:50:50.661248+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:40.000648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660043+0000","last_change":"2024-04-25T08:50:47.473573+0000","last_active":"2024-04-25T08:50:50.660043+0000","last_peered":"2024-04-25T08:50:50.660043+0000","last_clean":"2024-04-25T08:50:50.660043+0000","last_became_active":"2024-04-25T08:50:47.472779+0000","last_became_peered":"2024-04-25T08:50:47.472779+0000","last_unstale":"2024-04-25T08:50:50.660043+0000","last_undegraded":"2024-04-25T08:50:50.660043+0000","last_fullsized":"2024-04-25T08:50:50.660043+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:13.074380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463807+0000","last_change":"2024-04-25T08:50:48.487954+0000","last_active":"2024-04-25T08:50:49.463807+0000","last_peered":"2024-04-25T08:50:49.463807+0000","last_clean":"2024-04-25T08:50:49.463807+0000","last_became_active":"2024-04-25T08:50:48.487582+0000","last_became_peered":"2024-04-25T08:50:48.487582+0000","last_unstale":"2024-04-25T08:50:49.463807+0000","last_undegraded":"2024-04-25T08:50:49.463807+0000","last_fullsized":"2024-04-25T08:50:49.463807+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:10.997023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465522+0000","last_change":"2024-04-25T08:50:47.475954+0000","last_active":"2024-04-25T08:50:49.465522+0000","last_peered":"2024-04-25T08:50:49.465522+0000","last_clean":"2024-04-25T08:50:49.465522+0000","last_became_active":"2024-04-25T08:50:47.475269+0000","last_became_peered":"2024-04-25T08:50:47.475269+0000","last_unstale":"2024-04-25T08:50:49.465522+0000","last_undegraded":"2024-04-25T08:50:49.465522+0000","last_fullsized":"2024-04-25T08:50:49.465522+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:46.961095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660140+0000","last_change":"2024-04-25T08:50:48.480241+0000","last_active":"2024-04-25T08:50:50.660140+0000","last_peered":"2024-04-25T08:50:50.660140+0000","last_clean":"2024-04-25T08:50:50.660140+0000","last_became_active":"2024-04-25T08:50:48.479905+0000","last_became_peered":"2024-04-25T08:50:48.479905+0000","last_unstale":"2024-04-25T08:50:50.660140+0000","last_undegraded":"2024-04-25T08:50:50.660140+0000","last_fullsized":"2024-04-25T08:50:50.660140+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:13.400633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663657+0000","last_change":"2024-04-25T08:50:47.479962+0000","last_active":"2024-04-25T08:50:50.663657+0000","last_peered":"2024-04-25T08:50:50.663657+0000","last_clean":"2024-04-25T08:50:50.663657+0000","last_became_active":"2024-04-25T08:50:47.479009+0000","last_became_peered":"2024-04-25T08:50:47.479009+0000","last_unstale":"2024-04-25T08:50:50.663657+0000","last_undegraded":"2024-04-25T08:50:50.663657+0000","last_fullsized":"2024-04-25T08:50:50.663657+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:21.594237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663183+0000","last_change":"2024-04-25T08:50:48.475915+0000","last_active":"2024-04-25T08:50:50.663183+0000","last_peered":"2024-04-25T08:50:50.663183+0000","last_clean":"2024-04-25T08:50:50.663183+0000","last_became_active":"2024-04-25T08:50:48.475537+0000","last_became_peered":"2024-04-25T08:50:48.475537+0000","last_unstale":"2024-04-25T08:50:50.663183+0000","last_undegraded":"2024-04-25T08:50:50.663183+0000","last_fullsized":"2024-04-25T08:50:50.663183+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:30.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663040+0000","last_change":"2024-04-25T08:50:47.479576+0000","last_active":"2024-04-25T08:50:50.663040+0000","last_peered":"2024-04-25T08:50:50.663040+0000","last_clean":"2024-04-25T08:50:50.663040+0000","last_became_active":"2024-04-25T08:50:47.479308+0000","last_became_peered":"2024-04-25T08:50:47.479308+0000","last_unstale":"2024-04-25T08:50:50.663040+0000","last_undegraded":"2024-04-25T08:50:50.663040+0000","last_fullsized":"2024-04-25T08:50:50.663040+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:39.764851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660683+0000","last_change":"2024-04-25T08:50:48.475110+0000","last_active":"2024-04-25T08:50:50.660683+0000","last_peered":"2024-04-25T08:50:50.660683+0000","last_clean":"2024-04-25T08:50:50.660683+0000","last_became_active":"2024-04-25T08:50:48.474785+0000","last_became_peered":"2024-04-25T08:50:48.474785+0000","last_unstale":"2024-04-25T08:50:50.660683+0000","last_undegraded":"2024-04-25T08:50:50.660683+0000","last_fullsized":"2024-04-25T08:50:50.660683+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:04.379008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490544+0000","last_change":"2024-04-25T08:50:47.483654+0000","last_active":"2024-04-25T08:50:49.490544+0000","last_peered":"2024-04-25T08:50:49.490544+0000","last_clean":"2024-04-25T08:50:49.490544+0000","last_became_active":"2024-04-25T08:50:47.483403+0000","last_became_peered":"2024-04-25T08:50:47.483403+0000","last_unstale":"2024-04-25T08:50:49.490544+0000","last_undegraded":"2024-04-25T08:50:49.490544+0000","last_fullsized":"2024-04-25T08:50:49.490544+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:46.961095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463627+0000","last_change":"2024-04-25T08:50:48.482993+0000","last_active":"2024-04-25T08:50:49.463627+0000","last_peered":"2024-04-25T08:50:49.463627+0000","last_clean":"2024-04-25T08:50:49.463627+0000","last_became_active":"2024-04-25T08:50:48.482713+0000","last_became_peered":"2024-04-25T08:50:48.482713+0000","last_unstale":"2024-04-25T08:50:49.463627+0000","last_undegraded":"2024-04-25T08:50:49.463627+0000","last_fullsized":"2024-04-25T08:50:49.463627+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:13.276757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661152+0000","last_change":"2024-04-25T08:50:47.481872+0000","last_active":"2024-04-25T08:50:50.661152+0000","last_peered":"2024-04-25T08:50:50.661152+0000","last_clean":"2024-04-25T08:50:50.661152+0000","last_became_active":"2024-04-25T08:50:47.481589+0000","last_became_peered":"2024-04-25T08:50:47.481589+0000","last_unstale":"2024-04-25T08:50:50.661152+0000","last_undegraded":"2024-04-25T08:50:50.661152+0000","last_fullsized":"2024-04-25T08:50:50.661152+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:45.722329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463773+0000","last_change":"2024-04-25T08:50:48.468293+0000","last_active":"2024-04-25T08:50:49.463773+0000","last_peered":"2024-04-25T08:50:49.463773+0000","last_clean":"2024-04-25T08:50:49.463773+0000","last_became_active":"2024-04-25T08:50:48.468022+0000","last_became_peered":"2024-04-25T08:50:48.468022+0000","last_unstale":"2024-04-25T08:50:49.463773+0000","last_undegraded":"2024-04-25T08:50:49.463773+0000","last_fullsized":"2024-04-25T08:50:49.463773+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:25.114089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660159+0000","last_change":"2024-04-25T08:50:47.475919+0000","last_active":"2024-04-25T08:50:50.660159+0000","last_peered":"2024-04-25T08:50:50.660159+0000","last_clean":"2024-04-25T08:50:50.660159+0000","last_became_active":"2024-04-25T08:50:47.475589+0000","last_became_peered":"2024-04-25T08:50:47.475589+0000","last_unstale":"2024-04-25T08:50:50.660159+0000","last_undegraded":"2024-04-25T08:50:50.660159+0000","last_fullsized":"2024-04-25T08:50:50.660159+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:51.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463052+0000","last_change":"2024-04-25T08:50:48.480280+0000","last_active":"2024-04-25T08:50:49.463052+0000","last_peered":"2024-04-25T08:50:49.463052+0000","last_clean":"2024-04-25T08:50:49.463052+0000","last_became_active":"2024-04-25T08:50:48.480013+0000","last_became_peered":"2024-04-25T08:50:48.480013+0000","last_unstale":"2024-04-25T08:50:49.463052+0000","last_undegraded":"2024-04-25T08:50:49.463052+0000","last_fullsized":"2024-04-25T08:50:49.463052+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.389533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662882+0000","last_change":"2024-04-25T08:50:47.479727+0000","last_active":"2024-04-25T08:50:50.662882+0000","last_peered":"2024-04-25T08:50:50.662882+0000","last_clean":"2024-04-25T08:50:50.662882+0000","last_became_active":"2024-04-25T08:50:47.479464+0000","last_became_peered":"2024-04-25T08:50:47.479464+0000","last_unstale":"2024-04-25T08:50:50.662882+0000","last_undegraded":"2024-04-25T08:50:50.662882+0000","last_fullsized":"2024-04-25T08:50:50.662882+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:02.515846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661099+0000","last_change":"2024-04-25T08:50:48.490457+0000","last_active":"2024-04-25T08:50:50.661099+0000","last_peered":"2024-04-25T08:50:50.661099+0000","last_clean":"2024-04-25T08:50:50.661099+0000","last_became_active":"2024-04-25T08:50:48.488728+0000","last_became_peered":"2024-04-25T08:50:48.488728+0000","last_unstale":"2024-04-25T08:50:50.661099+0000","last_undegraded":"2024-04-25T08:50:50.661099+0000","last_fullsized":"2024-04-25T08:50:50.661099+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:20.939116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660092+0000","last_change":"2024-04-25T08:50:47.472781+0000","last_active":"2024-04-25T08:50:50.660092+0000","last_peered":"2024-04-25T08:50:50.660092+0000","last_clean":"2024-04-25T08:50:50.660092+0000","last_became_active":"2024-04-25T08:50:47.472447+0000","last_became_peered":"2024-04-25T08:50:47.472447+0000","last_unstale":"2024-04-25T08:50:50.660092+0000","last_undegraded":"2024-04-25T08:50:50.660092+0000","last_fullsized":"2024-04-25T08:50:50.660092+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.945660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463243+0000","last_change":"2024-04-25T08:50:48.478014+0000","last_active":"2024-04-25T08:50:49.463243+0000","last_peered":"2024-04-25T08:50:49.463243+0000","last_clean":"2024-04-25T08:50:49.463243+0000","last_became_active":"2024-04-25T08:50:48.477765+0000","last_became_peered":"2024-04-25T08:50:48.477765+0000","last_unstale":"2024-04-25T08:50:49.463243+0000","last_undegraded":"2024-04-25T08:50:49.463243+0000","last_fullsized":"2024-04-25T08:50:49.463243+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:13.863654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661514+0000","last_change":"2024-04-25T08:50:47.477416+0000","last_active":"2024-04-25T08:50:50.661514+0000","last_peered":"2024-04-25T08:50:50.661514+0000","last_clean":"2024-04-25T08:50:50.661514+0000","last_became_active":"2024-04-25T08:50:47.477157+0000","last_became_peered":"2024-04-25T08:50:47.477157+0000","last_unstale":"2024-04-25T08:50:50.661514+0000","last_undegraded":"2024-04-25T08:50:50.661514+0000","last_fullsized":"2024-04-25T08:50:50.661514+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:21.594237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465624+0000","last_change":"2024-04-25T08:50:48.479655+0000","last_active":"2024-04-25T08:50:49.465624+0000","last_peered":"2024-04-25T08:50:49.465624+0000","last_clean":"2024-04-25T08:50:49.465624+0000","last_became_active":"2024-04-25T08:50:48.479222+0000","last_became_peered":"2024-04-25T08:50:48.479222+0000","last_unstale":"2024-04-25T08:50:49.465624+0000","last_undegraded":"2024-04-25T08:50:49.465624+0000","last_fullsized":"2024-04-25T08:50:49.465624+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.676510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"106'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463647+0000","last_change":"2024-04-25T08:50:47.483125+0000","last_active":"2024-04-25T08:50:49.463647+0000","last_peered":"2024-04-25T08:50:49.463647+0000","last_clean":"2024-04-25T08:50:49.463647+0000","last_became_active":"2024-04-25T08:50:47.481358+0000","last_became_peered":"2024-04-25T08:50:47.481358+0000","last_unstale":"2024-04-25T08:50:49.463647+0000","last_undegraded":"2024-04-25T08:50:49.463647+0000","last_fullsized":"2024-04-25T08:50:49.463647+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:29.086389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462270+0000","last_change":"2024-04-25T08:50:48.479501+0000","last_active":"2024-04-25T08:50:49.462270+0000","last_peered":"2024-04-25T08:50:49.462270+0000","last_clean":"2024-04-25T08:50:49.462270+0000","last_became_active":"2024-04-25T08:50:48.479201+0000","last_became_peered":"2024-04-25T08:50:48.479201+0000","last_unstale":"2024-04-25T08:50:49.462270+0000","last_undegraded":"2024-04-25T08:50:49.462270+0000","last_fullsized":"2024-04-25T08:50:49.462270+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:51.015513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663561+0000","last_change":"2024-04-25T08:50:47.479154+0000","last_active":"2024-04-25T08:50:50.663561+0000","last_peered":"2024-04-25T08:50:50.663561+0000","last_clean":"2024-04-25T08:50:50.663561+0000","last_became_active":"2024-04-25T08:50:47.477739+0000","last_became_peered":"2024-04-25T08:50:47.477739+0000","last_unstale":"2024-04-25T08:50:50.663561+0000","last_undegraded":"2024-04-25T08:50:50.663561+0000","last_fullsized":"2024-04-25T08:50:50.663561+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:58.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490551+0000","last_change":"2024-04-25T08:50:48.488816+0000","last_active":"2024-04-25T08:50:49.490551+0000","last_peered":"2024-04-25T08:50:49.490551+0000","last_clean":"2024-04-25T08:50:49.490551+0000","last_became_active":"2024-04-25T08:50:48.488399+0000","last_became_peered":"2024-04-25T08:50:48.488399+0000","last_unstale":"2024-04-25T08:50:49.490551+0000","last_undegraded":"2024-04-25T08:50:49.490551+0000","last_fullsized":"2024-04-25T08:50:49.490551+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:30.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660751+0000","last_change":"2024-04-25T08:50:47.473632+0000","last_active":"2024-04-25T08:50:50.660751+0000","last_peered":"2024-04-25T08:50:50.660751+0000","last_clean":"2024-04-25T08:50:50.660751+0000","last_became_active":"2024-04-25T08:50:47.473285+0000","last_became_peered":"2024-04-25T08:50:47.473285+0000","last_unstale":"2024-04-25T08:50:50.660751+0000","last_undegraded":"2024-04-25T08:50:50.660751+0000","last_fullsized":"2024-04-25T08:50:50.660751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:58.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661041+0000","last_change":"2024-04-25T08:50:48.489306+0000","last_active":"2024-04-25T08:50:50.661041+0000","last_peered":"2024-04-25T08:50:50.661041+0000","last_clean":"2024-04-25T08:50:50.661041+0000","last_became_active":"2024-04-25T08:50:48.489035+0000","last_became_peered":"2024-04-25T08:50:48.489035+0000","last_unstale":"2024-04-25T08:50:50.661041+0000","last_undegraded":"2024-04-25T08:50:50.661041+0000","last_fullsized":"2024-04-25T08:50:50.661041+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:34.999810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465745+0000","last_change":"2024-04-25T08:50:47.481624+0000","last_active":"2024-04-25T08:50:49.465745+0000","last_peered":"2024-04-25T08:50:49.465745+0000","last_clean":"2024-04-25T08:50:49.465745+0000","last_became_active":"2024-04-25T08:50:47.481051+0000","last_became_peered":"2024-04-25T08:50:47.481051+0000","last_unstale":"2024-04-25T08:50:49.465745+0000","last_undegraded":"2024-04-25T08:50:49.465745+0000","last_fullsized":"2024-04-25T08:50:49.465745+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:59.512829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661303+0000","last_change":"2024-04-25T08:50:48.490140+0000","last_active":"2024-04-25T08:50:50.661303+0000","last_peered":"2024-04-25T08:50:50.661303+0000","last_clean":"2024-04-25T08:50:50.661303+0000","last_became_active":"2024-04-25T08:50:48.489843+0000","last_became_peered":"2024-04-25T08:50:48.489843+0000","last_unstale":"2024-04-25T08:50:50.661303+0000","last_undegraded":"2024-04-25T08:50:50.661303+0000","last_fullsized":"2024-04-25T08:50:50.661303+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:48.209591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660500+0000","last_change":"2024-04-25T08:50:47.479010+0000","last_active":"2024-04-25T08:50:50.660500+0000","last_peered":"2024-04-25T08:50:50.660500+0000","last_clean":"2024-04-25T08:50:50.660500+0000","last_became_active":"2024-04-25T08:50:47.478768+0000","last_became_peered":"2024-04-25T08:50:47.478768+0000","last_unstale":"2024-04-25T08:50:50.660500+0000","last_undegraded":"2024-04-25T08:50:50.660500+0000","last_fullsized":"2024-04-25T08:50:50.660500+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:39.575982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663034+0000","last_change":"2024-04-25T08:50:48.492853+0000","last_active":"2024-04-25T08:50:50.663034+0000","last_peered":"2024-04-25T08:50:50.663034+0000","last_clean":"2024-04-25T08:50:50.663034+0000","last_became_active":"2024-04-25T08:50:48.491331+0000","last_became_peered":"2024-04-25T08:50:48.491331+0000","last_unstale":"2024-04-25T08:50:50.663034+0000","last_undegraded":"2024-04-25T08:50:50.663034+0000","last_fullsized":"2024-04-25T08:50:50.663034+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:51.015513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663122+0000","last_change":"2024-04-25T08:50:47.480096+0000","last_active":"2024-04-25T08:50:50.663122+0000","last_peered":"2024-04-25T08:50:50.663122+0000","last_clean":"2024-04-25T08:50:50.663122+0000","last_became_active":"2024-04-25T08:50:47.477889+0000","last_became_peered":"2024-04-25T08:50:47.477889+0000","last_unstale":"2024-04-25T08:50:50.663122+0000","last_undegraded":"2024-04-25T08:50:50.663122+0000","last_fullsized":"2024-04-25T08:50:50.663122+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:16.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490863+0000","last_change":"2024-04-25T08:50:48.486956+0000","last_active":"2024-04-25T08:50:49.490863+0000","last_peered":"2024-04-25T08:50:49.490863+0000","last_clean":"2024-04-25T08:50:49.490863+0000","last_became_active":"2024-04-25T08:50:48.485562+0000","last_became_peered":"2024-04-25T08:50:48.485562+0000","last_unstale":"2024-04-25T08:50:49.490863+0000","last_undegraded":"2024-04-25T08:50:49.490863+0000","last_fullsized":"2024-04-25T08:50:49.490863+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:00.051922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660171+0000","last_change":"2024-04-25T08:50:47.478107+0000","last_active":"2024-04-25T08:50:50.660171+0000","last_peered":"2024-04-25T08:50:50.660171+0000","last_clean":"2024-04-25T08:50:50.660171+0000","last_became_active":"2024-04-25T08:50:47.477775+0000","last_became_peered":"2024-04-25T08:50:47.477775+0000","last_unstale":"2024-04-25T08:50:50.660171+0000","last_undegraded":"2024-04-25T08:50:50.660171+0000","last_fullsized":"2024-04-25T08:50:50.660171+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:10.520973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490444+0000","last_change":"2024-04-25T08:50:48.488454+0000","last_active":"2024-04-25T08:50:49.490444+0000","last_peered":"2024-04-25T08:50:49.490444+0000","last_clean":"2024-04-25T08:50:49.490444+0000","last_became_active":"2024-04-25T08:50:48.488168+0000","last_became_peered":"2024-04-25T08:50:48.488168+0000","last_unstale":"2024-04-25T08:50:49.490444+0000","last_undegraded":"2024-04-25T08:50:49.490444+0000","last_fullsized":"2024-04-25T08:50:49.490444+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:04.379008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661117+0000","last_change":"2024-04-25T08:50:47.476087+0000","last_active":"2024-04-25T08:50:50.661117+0000","last_peered":"2024-04-25T08:50:50.661117+0000","last_clean":"2024-04-25T08:50:50.661117+0000","last_became_active":"2024-04-25T08:50:47.475441+0000","last_became_peered":"2024-04-25T08:50:47.475441+0000","last_unstale":"2024-04-25T08:50:50.661117+0000","last_undegraded":"2024-04-25T08:50:50.661117+0000","last_fullsized":"2024-04-25T08:50:50.661117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:04.616234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":331,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662354+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:50:50.662354+0000","last_peered":"2024-04-25T08:50:50.662354+0000","last_clean":"2024-04-25T08:50:50.662354+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:50:50.662354+0000","last_undegraded":"2024-04-25T08:50:50.662354+0000","last_fullsized":"2024-04-25T08:50:50.662354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1916928,"data_stored":1202437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":614125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1314816,"data_stored":611909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":612001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":612491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1318912,"data_stored":611943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":611943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":1203979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:50:52.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:50:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cfb200640 1 Processor -- start 2024-04-25T08:50:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cfb200640 1 -- start start 2024-04-25T08:50:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cfb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cf406db40 0x7f8cf406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cfb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 0x7f8cf406b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cfb200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf415e180 con 0x7f8cf40af3f0 2024-04-25T08:50:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cfb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf40b3ca0 con 0x7f8cf406db40 2024-04-25T08:50:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cfb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf415e460 con 0x7f8cf40b3550 2024-04-25T08:50:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cebe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cf406db40 0x7f8cf406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cebe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cf406db40 0x7f8cf406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40148/0 (socket says 172.21.15.46:40148) 2024-04-25T08:50:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cebe00640 1 -- 172.21.15.46:0/3271013415 learned_addr learned my addr 172.21.15.46:0/3271013415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cf9200640 1 --2- 172.21.15.46:0/3271013415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 0x7f8cf406b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cebe00640 1 -- 172.21.15.46:0/3271013415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 msgr2=0x7f8cf406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cebe00640 1 --2- 172.21.15.46:0/3271013415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 0x7f8cf406b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cebe00640 1 -- 172.21.15.46:0/3271013415 >> v1:172.21.15.82:6789/0 conn(0x7f8cf40af3f0 legacy=0x7f8cf40b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cebe00640 1 -- 172.21.15.46:0/3271013415 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf415e740 con 0x7f8cf406db40 2024-04-25T08:50:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.771+0000 7f8cf9200640 1 --2- 172.21.15.46:0/3271013415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 0x7f8cf406b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cebe00640 1 --2- 172.21.15.46:0/3271013415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 0x7f8cf406ac10 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009870 tx=0x7f8cdc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf842b369e78ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cf3400640 1 -- 172.21.15.46:0/3271013415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0b0070 con 0x7f8cf406db40 2024-04-25T08:50:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cf3400640 1 -- 172.21.15.46:0/3271013415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cdc0ab600 con 0x7f8cf406db40 2024-04-25T08:50:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cf3400640 1 -- 172.21.15.46:0/3271013415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc0ab9d0 con 0x7f8cf406db40 2024-04-25T08:50:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cfb200640 1 -- 172.21.15.46:0/3271013415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 msgr2=0x7f8cf406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cfb200640 1 --2- 172.21.15.46:0/3271013415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 0x7f8cf406ac10 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009870 tx=0x7f8cdc01c520 comp rx=0 tx=0).stop 2024-04-25T08:50:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cfb200640 1 -- 172.21.15.46:0/3271013415 shutdown_connections 2024-04-25T08:50:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cfb200640 1 --2- 172.21.15.46:0/3271013415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 0x7f8cf406b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cfb200640 1 --2- 172.21.15.46:0/3271013415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 0x7f8cf406ac10 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cfb200640 1 -- 172.21.15.46:0/3271013415 >> 172.21.15.46:0/3271013415 conn(0x7f8cf40cfd90 msgr2=0x7f8cf40d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cfb200640 1 -- 172.21.15.46:0/3271013415 shutdown_connections 2024-04-25T08:50:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.775+0000 7f8cfb200640 1 -- 172.21.15.46:0/3271013415 wait complete. 2024-04-25T08:50:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 Processor -- start 2024-04-25T08:50:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 -- start start 2024-04-25T08:50:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 0x7f8cf413f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cf40af3f0 0x7f8cf415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 0x7f8cf4152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf416a870 con 0x7f8cf406db40 2024-04-25T08:50:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf416a6f0 con 0x7f8cf40af3f0 2024-04-25T08:50:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf416a9f0 con 0x7f8cf40b3550 2024-04-25T08:50:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 0x7f8cf413f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 0x7f8cf413f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40154/0 (socket says 172.21.15.46:40154) 2024-04-25T08:50:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf3e00640 1 -- 172.21.15.46:0/2792820428 learned_addr learned my addr 172.21.15.46:0/2792820428 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cebe00640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cf40af3f0 0x7f8cf415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf9200640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 0x7f8cf4152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf3e00640 1 -- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 msgr2=0x7f8cf4152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf3e00640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 0x7f8cf4152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf3e00640 1 -- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cf40af3f0 msgr2=0x7f8cf415d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf3e00640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cf40af3f0 0x7f8cf415d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf3e00640 1 -- 172.21.15.46:0/2792820428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf413fcb0 con 0x7f8cf406db40 2024-04-25T08:50:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf3e00640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 0x7f8cf413f770 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400c480 tx=0x7f8ce400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf0c00640 1 -- 172.21.15.46:0/2792820428 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4012830 con 0x7f8cf406db40 2024-04-25T08:50:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf0c00640 1 -- 172.21.15.46:0/2792820428 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce4017070 con 0x7f8cf406db40 2024-04-25T08:50:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cf0c00640 1 -- 172.21.15.46:0/2792820428 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce4012c30 con 0x7f8cf406db40 2024-04-25T08:50:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf415c350 con 0x7f8cf406db40 2024-04-25T08:50:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf415c910 con 0x7f8cf406db40 2024-04-25T08:50:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.779+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf4069080 con 0x7f8cf406db40 2024-04-25T08:50:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.783+0000 7f8cf0c00640 1 -- 172.21.15.46:0/2792820428 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ce401c460 con 0x7f8cf406db40 2024-04-25T08:50:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.783+0000 7f8cf0c00640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cd0066f60 0x7f8cd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.783+0000 7f8cf0c00640 1 -- 172.21.15.46:0/2792820428 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8ce4084fa0 con 0x7f8cf406db40 2024-04-25T08:50:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.783+0000 7f8cebe00640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cd0066f60 0x7f8cd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.783+0000 7f8cebe00640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cd0066f60 0x7f8cd0069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0ad000 tx=0x7f8cdc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.787+0000 7f8cf0c00640 1 -- 172.21.15.46:0/2792820428 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ce4055e10 con 0x7f8cf406db40 2024-04-25T08:50:52.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.943+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8cf4161e10 con 0x7f8cd0066f60 2024-04-25T08:50:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.951+0000 7f8cf0c00640 1 -- 172.21.15.46:0/2792820428 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252485 (secure 0 0 0) 0x7f8cf4161e10 con 0x7f8cd0066f60 2024-04-25T08:50:52.945 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:50:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cd0066f60 msgr2=0x7f8cd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cd0066f60 0x7f8cd0069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc0ad000 tx=0x7f8cdc008040 comp rx=0 tx=0).stop 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 msgr2=0x7f8cf413f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 0x7f8cf413f770 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400c480 tx=0x7f8ce400c940 comp rx=0 tx=0).stop 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 shutdown_connections 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8cd0066f60 0x7f8cd0069420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8cf40b3550 0x7f8cf4152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cf40af3f0 0x7f8cf415d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 --2- 172.21.15.46:0/2792820428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf406db40 0x7f8cf413f770 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 >> 172.21.15.46:0/2792820428 conn(0x7f8cf40cfd90 msgr2=0x7f8cf4160580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 shutdown_connections 2024-04-25T08:50:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:52.955+0000 7f8cfb200640 1 -- 172.21.15.46:0/2792820428 wait complete. 2024-04-25T08:50:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-25T08:50:52.385129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260952,"kb_used_data":11496,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713768,"statfs":{"total":773094113280,"available":772826898432,"internally_reserved":0,"allocated":11771904,"data_stored":6080828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.935186"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463695+0000","last_change":"2024-04-25T08:50:48.480492+0000","last_active":"2024-04-25T08:50:49.463695+0000","last_peered":"2024-04-25T08:50:49.463695+0000","last_clean":"2024-04-25T08:50:49.463695+0000","last_became_active":"2024-04-25T08:50:48.480001+0000","last_became_peered":"2024-04-25T08:50:48.480001+0000","last_unstale":"2024-04-25T08:50:49.463695+0000","last_undegraded":"2024-04-25T08:50:49.463695+0000","last_fullsized":"2024-04-25T08:50:49.463695+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:23.222694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663660+0000","last_change":"2024-04-25T08:50:48.478572+0000","last_active":"2024-04-25T08:50:50.663660+0000","last_peered":"2024-04-25T08:50:50.663660+0000","last_clean":"2024-04-25T08:50:50.663660+0000","last_became_active":"2024-04-25T08:50:48.478286+0000","last_became_peered":"2024-04-25T08:50:48.478286+0000","last_unstale":"2024-04-25T08:50:50.663660+0000","last_undegraded":"2024-04-25T08:50:50.663660+0000","last_fullsized":"2024-04-25T08:50:50.663660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:00.051922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463567+0000","last_change":"2024-04-25T08:50:48.495025+0000","last_active":"2024-04-25T08:50:49.463567+0000","last_peered":"2024-04-25T08:50:49.463567+0000","last_clean":"2024-04-25T08:50:49.463567+0000","last_became_active":"2024-04-25T08:50:48.494746+0000","last_became_peered":"2024-04-25T08:50:48.494746+0000","last_unstale":"2024-04-25T08:50:49.463567+0000","last_undegraded":"2024-04-25T08:50:49.463567+0000","last_fullsized":"2024-04-25T08:50:49.463567+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:45.190214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462565+0000","last_change":"2024-04-25T08:50:48.475449+0000","last_active":"2024-04-25T08:50:49.462565+0000","last_peered":"2024-04-25T08:50:49.462565+0000","last_clean":"2024-04-25T08:50:49.462565+0000","last_became_active":"2024-04-25T08:50:48.475137+0000","last_became_peered":"2024-04-25T08:50:48.475137+0000","last_unstale":"2024-04-25T08:50:49.462565+0000","last_undegraded":"2024-04-25T08:50:49.462565+0000","last_fullsized":"2024-04-25T08:50:49.462565+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:25.480050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660663+0000","last_change":"2024-04-25T08:50:48.467088+0000","last_active":"2024-04-25T08:50:50.660663+0000","last_peered":"2024-04-25T08:50:50.660663+0000","last_clean":"2024-04-25T08:50:50.660663+0000","last_became_active":"2024-04-25T08:50:48.466790+0000","last_became_peered":"2024-04-25T08:50:48.466790+0000","last_unstale":"2024-04-25T08:50:50.660663+0000","last_undegraded":"2024-04-25T08:50:50.660663+0000","last_fullsized":"2024-04-25T08:50:50.660663+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:38.926179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663184+0000","last_change":"2024-04-25T08:50:48.486891+0000","last_active":"2024-04-25T08:50:50.663184+0000","last_peered":"2024-04-25T08:50:50.663184+0000","last_clean":"2024-04-25T08:50:50.663184+0000","last_became_active":"2024-04-25T08:50:48.485703+0000","last_became_peered":"2024-04-25T08:50:48.485703+0000","last_unstale":"2024-04-25T08:50:50.663184+0000","last_undegraded":"2024-04-25T08:50:50.663184+0000","last_fullsized":"2024-04-25T08:50:50.663184+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:52.126434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663317+0000","last_change":"2024-04-25T08:50:48.476906+0000","last_active":"2024-04-25T08:50:50.663317+0000","last_peered":"2024-04-25T08:50:50.663317+0000","last_clean":"2024-04-25T08:50:50.663317+0000","last_became_active":"2024-04-25T08:50:48.476148+0000","last_became_peered":"2024-04-25T08:50:48.476148+0000","last_unstale":"2024-04-25T08:50:50.663317+0000","last_undegraded":"2024-04-25T08:50:50.663317+0000","last_fullsized":"2024-04-25T08:50:50.663317+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:19.646246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.659996+0000","last_change":"2024-04-25T08:50:48.482855+0000","last_active":"2024-04-25T08:50:50.659996+0000","last_peered":"2024-04-25T08:50:50.659996+0000","last_clean":"2024-04-25T08:50:50.659996+0000","last_became_active":"2024-04-25T08:50:48.482682+0000","last_became_peered":"2024-04-25T08:50:48.482682+0000","last_unstale":"2024-04-25T08:50:50.659996+0000","last_undegraded":"2024-04-25T08:50:50.659996+0000","last_fullsized":"2024-04-25T08:50:50.659996+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:21.014431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663763+0000","last_change":"2024-04-25T08:50:48.481048+0000","last_active":"2024-04-25T08:50:50.663763+0000","last_peered":"2024-04-25T08:50:50.663763+0000","last_clean":"2024-04-25T08:50:50.663763+0000","last_became_active":"2024-04-25T08:50:48.480757+0000","last_became_peered":"2024-04-25T08:50:48.480757+0000","last_unstale":"2024-04-25T08:50:50.663763+0000","last_undegraded":"2024-04-25T08:50:50.663763+0000","last_fullsized":"2024-04-25T08:50:50.663763+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:48.132422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661393+0000","last_change":"2024-04-25T08:50:48.476393+0000","last_active":"2024-04-25T08:50:50.661393+0000","last_peered":"2024-04-25T08:50:50.661393+0000","last_clean":"2024-04-25T08:50:50.661393+0000","last_became_active":"2024-04-25T08:50:48.476101+0000","last_became_peered":"2024-04-25T08:50:48.476101+0000","last_unstale":"2024-04-25T08:50:50.661393+0000","last_undegraded":"2024-04-25T08:50:50.661393+0000","last_fullsized":"2024-04-25T08:50:50.661393+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:19.646246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490631+0000","last_change":"2024-04-25T08:50:48.485607+0000","last_active":"2024-04-25T08:50:49.490631+0000","last_peered":"2024-04-25T08:50:49.490631+0000","last_clean":"2024-04-25T08:50:49.490631+0000","last_became_active":"2024-04-25T08:50:48.485245+0000","last_became_peered":"2024-04-25T08:50:48.485245+0000","last_unstale":"2024-04-25T08:50:49.490631+0000","last_undegraded":"2024-04-25T08:50:49.490631+0000","last_fullsized":"2024-04-25T08:50:49.490631+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:19.646246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466107+0000","last_change":"2024-04-25T08:50:48.479907+0000","last_active":"2024-04-25T08:50:49.466107+0000","last_peered":"2024-04-25T08:50:49.466107+0000","last_clean":"2024-04-25T08:50:49.466107+0000","last_became_active":"2024-04-25T08:50:48.479391+0000","last_became_peered":"2024-04-25T08:50:48.479391+0000","last_unstale":"2024-04-25T08:50:49.466107+0000","last_undegraded":"2024-04-25T08:50:49.466107+0000","last_fullsized":"2024-04-25T08:50:49.466107+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:07.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462270+0000","last_change":"2024-04-25T08:50:48.487454+0000","last_active":"2024-04-25T08:50:49.462270+0000","last_peered":"2024-04-25T08:50:49.462270+0000","last_clean":"2024-04-25T08:50:49.462270+0000","last_became_active":"2024-04-25T08:50:48.487185+0000","last_became_peered":"2024-04-25T08:50:48.487185+0000","last_unstale":"2024-04-25T08:50:49.462270+0000","last_undegraded":"2024-04-25T08:50:49.462270+0000","last_fullsized":"2024-04-25T08:50:49.462270+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:31.314109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663186+0000","last_change":"2024-04-25T08:50:48.480452+0000","last_active":"2024-04-25T08:50:50.663186+0000","last_peered":"2024-04-25T08:50:50.663186+0000","last_clean":"2024-04-25T08:50:50.663186+0000","last_became_active":"2024-04-25T08:50:48.480218+0000","last_became_peered":"2024-04-25T08:50:48.480218+0000","last_unstale":"2024-04-25T08:50:50.663186+0000","last_undegraded":"2024-04-25T08:50:50.663186+0000","last_fullsized":"2024-04-25T08:50:50.663186+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:31.314109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662915+0000","last_change":"2024-04-25T08:50:48.491310+0000","last_active":"2024-04-25T08:50:50.662915+0000","last_peered":"2024-04-25T08:50:50.662915+0000","last_clean":"2024-04-25T08:50:50.662915+0000","last_became_active":"2024-04-25T08:50:48.490954+0000","last_became_peered":"2024-04-25T08:50:48.490954+0000","last_unstale":"2024-04-25T08:50:50.662915+0000","last_undegraded":"2024-04-25T08:50:50.662915+0000","last_fullsized":"2024-04-25T08:50:50.662915+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:39.031265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662534+0000","last_change":"2024-04-25T08:50:48.483454+0000","last_active":"2024-04-25T08:50:50.662534+0000","last_peered":"2024-04-25T08:50:50.662534+0000","last_clean":"2024-04-25T08:50:50.662534+0000","last_became_active":"2024-04-25T08:50:48.483119+0000","last_became_peered":"2024-04-25T08:50:48.483119+0000","last_unstale":"2024-04-25T08:50:50.662534+0000","last_undegraded":"2024-04-25T08:50:50.662534+0000","last_fullsized":"2024-04-25T08:50:50.662534+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:02.994114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463703+0000","last_change":"2024-04-25T08:50:48.484656+0000","last_active":"2024-04-25T08:50:49.463703+0000","last_peered":"2024-04-25T08:50:49.463703+0000","last_clean":"2024-04-25T08:50:49.463703+0000","last_became_active":"2024-04-25T08:50:48.484324+0000","last_became_peered":"2024-04-25T08:50:48.484324+0000","last_unstale":"2024-04-25T08:50:49.463703+0000","last_undegraded":"2024-04-25T08:50:49.463703+0000","last_fullsized":"2024-04-25T08:50:49.463703+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:25.144536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663773+0000","last_change":"2024-04-25T08:50:48.486940+0000","last_active":"2024-04-25T08:50:50.663773+0000","last_peered":"2024-04-25T08:50:50.663773+0000","last_clean":"2024-04-25T08:50:50.663773+0000","last_became_active":"2024-04-25T08:50:48.486396+0000","last_became_peered":"2024-04-25T08:50:48.486396+0000","last_unstale":"2024-04-25T08:50:50.663773+0000","last_undegraded":"2024-04-25T08:50:50.663773+0000","last_fullsized":"2024-04-25T08:50:50.663773+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:43.918413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463588+0000","last_change":"2024-04-25T08:50:48.477793+0000","last_active":"2024-04-25T08:50:49.463588+0000","last_peered":"2024-04-25T08:50:49.463588+0000","last_clean":"2024-04-25T08:50:49.463588+0000","last_became_active":"2024-04-25T08:50:48.477502+0000","last_became_peered":"2024-04-25T08:50:48.477502+0000","last_unstale":"2024-04-25T08:50:49.463588+0000","last_undegraded":"2024-04-25T08:50:49.463588+0000","last_fullsized":"2024-04-25T08:50:49.463588+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:51.145418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465924+0000","last_change":"2024-04-25T08:50:48.473067+0000","last_active":"2024-04-25T08:50:49.465924+0000","last_peered":"2024-04-25T08:50:49.465924+0000","last_clean":"2024-04-25T08:50:49.465924+0000","last_became_active":"2024-04-25T08:50:48.472748+0000","last_became_peered":"2024-04-25T08:50:48.472748+0000","last_unstale":"2024-04-25T08:50:49.465924+0000","last_undegraded":"2024-04-25T08:50:49.465924+0000","last_fullsized":"2024-04-25T08:50:49.465924+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:40.772018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661092+0000","last_change":"2024-04-25T08:50:48.488160+0000","last_active":"2024-04-25T08:50:50.661092+0000","last_peered":"2024-04-25T08:50:50.661092+0000","last_clean":"2024-04-25T08:50:50.661092+0000","last_became_active":"2024-04-25T08:50:48.487856+0000","last_became_peered":"2024-04-25T08:50:48.487856+0000","last_unstale":"2024-04-25T08:50:50.661092+0000","last_undegraded":"2024-04-25T08:50:50.661092+0000","last_fullsized":"2024-04-25T08:50:50.661092+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:50.544561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660897+0000","last_change":"2024-04-25T08:50:48.492748+0000","last_active":"2024-04-25T08:50:50.660897+0000","last_peered":"2024-04-25T08:50:50.660897+0000","last_clean":"2024-04-25T08:50:50.660897+0000","last_became_active":"2024-04-25T08:50:48.492330+0000","last_became_peered":"2024-04-25T08:50:48.492330+0000","last_unstale":"2024-04-25T08:50:50.660897+0000","last_undegraded":"2024-04-25T08:50:50.660897+0000","last_fullsized":"2024-04-25T08:50:50.660897+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:30.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463289+0000","last_change":"2024-04-25T08:50:48.488353+0000","last_active":"2024-04-25T08:50:49.463289+0000","last_peered":"2024-04-25T08:50:49.463289+0000","last_clean":"2024-04-25T08:50:49.463289+0000","last_became_active":"2024-04-25T08:50:48.487815+0000","last_became_peered":"2024-04-25T08:50:48.487815+0000","last_unstale":"2024-04-25T08:50:49.463289+0000","last_undegraded":"2024-04-25T08:50:49.463289+0000","last_fullsized":"2024-04-25T08:50:49.463289+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:47.577495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660187+0000","last_change":"2024-04-25T08:50:48.467030+0000","last_active":"2024-04-25T08:50:50.660187+0000","last_peered":"2024-04-25T08:50:50.660187+0000","last_clean":"2024-04-25T08:50:50.660187+0000","last_became_active":"2024-04-25T08:50:48.466258+0000","last_became_peered":"2024-04-25T08:50:48.466258+0000","last_unstale":"2024-04-25T08:50:50.660187+0000","last_undegraded":"2024-04-25T08:50:50.660187+0000","last_fullsized":"2024-04-25T08:50:50.660187+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:03.616967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466080+0000","last_change":"2024-04-25T08:50:48.489451+0000","last_active":"2024-04-25T08:50:49.466080+0000","last_peered":"2024-04-25T08:50:49.466080+0000","last_clean":"2024-04-25T08:50:49.466080+0000","last_became_active":"2024-04-25T08:50:48.489150+0000","last_became_peered":"2024-04-25T08:50:48.489150+0000","last_unstale":"2024-04-25T08:50:49.466080+0000","last_undegraded":"2024-04-25T08:50:49.466080+0000","last_fullsized":"2024-04-25T08:50:49.466080+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:43.918413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465959+0000","last_change":"2024-04-25T08:50:48.481618+0000","last_active":"2024-04-25T08:50:49.465959+0000","last_peered":"2024-04-25T08:50:49.465959+0000","last_clean":"2024-04-25T08:50:49.465959+0000","last_became_active":"2024-04-25T08:50:48.481275+0000","last_became_peered":"2024-04-25T08:50:48.481275+0000","last_unstale":"2024-04-25T08:50:49.465959+0000","last_undegraded":"2024-04-25T08:50:49.465959+0000","last_fullsized":"2024-04-25T08:50:49.465959+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:00.051922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490732+0000","last_change":"2024-04-25T08:50:48.490030+0000","last_active":"2024-04-25T08:50:49.490732+0000","last_peered":"2024-04-25T08:50:49.490732+0000","last_clean":"2024-04-25T08:50:49.490732+0000","last_became_active":"2024-04-25T08:50:48.489676+0000","last_became_peered":"2024-04-25T08:50:48.489676+0000","last_unstale":"2024-04-25T08:50:49.490732+0000","last_undegraded":"2024-04-25T08:50:49.490732+0000","last_fullsized":"2024-04-25T08:50:49.490732+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:40.772018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663434+0000","last_change":"2024-04-25T08:50:48.487672+0000","last_active":"2024-04-25T08:50:50.663434+0000","last_peered":"2024-04-25T08:50:50.663434+0000","last_clean":"2024-04-25T08:50:50.663434+0000","last_became_active":"2024-04-25T08:50:48.487381+0000","last_became_peered":"2024-04-25T08:50:48.487381+0000","last_unstale":"2024-04-25T08:50:50.663434+0000","last_undegraded":"2024-04-25T08:50:50.663434+0000","last_fullsized":"2024-04-25T08:50:50.663434+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.676510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661018+0000","last_change":"2024-04-25T08:50:48.481642+0000","last_active":"2024-04-25T08:50:50.661018+0000","last_peered":"2024-04-25T08:50:50.661018+0000","last_clean":"2024-04-25T08:50:50.661018+0000","last_became_active":"2024-04-25T08:50:48.481347+0000","last_became_peered":"2024-04-25T08:50:48.481347+0000","last_unstale":"2024-04-25T08:50:50.661018+0000","last_undegraded":"2024-04-25T08:50:50.661018+0000","last_fullsized":"2024-04-25T08:50:50.661018+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:46.644691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466093+0000","last_change":"2024-04-25T08:50:48.489807+0000","last_active":"2024-04-25T08:50:49.466093+0000","last_peered":"2024-04-25T08:50:49.466093+0000","last_clean":"2024-04-25T08:50:49.466093+0000","last_became_active":"2024-04-25T08:50:48.489100+0000","last_became_peered":"2024-04-25T08:50:48.489100+0000","last_unstale":"2024-04-25T08:50:49.466093+0000","last_undegraded":"2024-04-25T08:50:49.466093+0000","last_fullsized":"2024-04-25T08:50:49.466093+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:48.132422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662801+0000","last_change":"2024-04-25T08:50:48.476136+0000","last_active":"2024-04-25T08:50:50.662801+0000","last_peered":"2024-04-25T08:50:50.662801+0000","last_clean":"2024-04-25T08:50:50.662801+0000","last_became_active":"2024-04-25T08:50:48.475543+0000","last_became_peered":"2024-04-25T08:50:48.475543+0000","last_unstale":"2024-04-25T08:50:50.662801+0000","last_undegraded":"2024-04-25T08:50:50.662801+0000","last_fullsized":"2024-04-25T08:50:50.662801+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:04.379008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660036+0000","last_change":"2024-04-25T08:50:48.485951+0000","last_active":"2024-04-25T08:50:50.660036+0000","last_peered":"2024-04-25T08:50:50.660036+0000","last_clean":"2024-04-25T08:50:50.660036+0000","last_became_active":"2024-04-25T08:50:48.485529+0000","last_became_peered":"2024-04-25T08:50:48.485529+0000","last_unstale":"2024-04-25T08:50:50.660036+0000","last_undegraded":"2024-04-25T08:50:50.660036+0000","last_fullsized":"2024-04-25T08:50:50.660036+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:46.707610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463505+0000","last_change":"2024-04-25T08:50:48.477441+0000","last_active":"2024-04-25T08:50:49.463505+0000","last_peered":"2024-04-25T08:50:49.463505+0000","last_clean":"2024-04-25T08:50:49.463505+0000","last_became_active":"2024-04-25T08:50:48.477134+0000","last_became_peered":"2024-04-25T08:50:48.477134+0000","last_unstale":"2024-04-25T08:50:49.463505+0000","last_undegraded":"2024-04-25T08:50:49.463505+0000","last_fullsized":"2024-04-25T08:50:49.463505+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:29.410218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"105'4","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462932+0000","last_change":"2024-04-25T08:50:47.479235+0000","last_active":"2024-04-25T08:50:49.462932+0000","last_peered":"2024-04-25T08:50:49.462932+0000","last_clean":"2024-04-25T08:50:49.462932+0000","last_became_active":"2024-04-25T08:50:47.478942+0000","last_became_peered":"2024-04-25T08:50:47.478942+0000","last_unstale":"2024-04-25T08:50:49.462932+0000","last_undegraded":"2024-04-25T08:50:49.462932+0000","last_fullsized":"2024-04-25T08:50:49.462932+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:46.669342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660077+0000","last_change":"2024-04-25T08:50:48.489948+0000","last_active":"2024-04-25T08:50:50.660077+0000","last_peered":"2024-04-25T08:50:50.660077+0000","last_clean":"2024-04-25T08:50:50.660077+0000","last_became_active":"2024-04-25T08:50:48.488512+0000","last_became_peered":"2024-04-25T08:50:48.488512+0000","last_unstale":"2024-04-25T08:50:50.660077+0000","last_undegraded":"2024-04-25T08:50:50.660077+0000","last_fullsized":"2024-04-25T08:50:50.660077+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:02.484051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661226+0000","last_change":"2024-04-25T08:50:47.480237+0000","last_active":"2024-04-25T08:50:50.661226+0000","last_peered":"2024-04-25T08:50:50.661226+0000","last_clean":"2024-04-25T08:50:50.661226+0000","last_became_active":"2024-04-25T08:50:47.479958+0000","last_became_peered":"2024-04-25T08:50:47.479958+0000","last_unstale":"2024-04-25T08:50:50.661226+0000","last_undegraded":"2024-04-25T08:50:50.661226+0000","last_fullsized":"2024-04-25T08:50:50.661226+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:11.578821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466276+0000","last_change":"2024-04-25T08:50:48.471546+0000","last_active":"2024-04-25T08:50:49.466276+0000","last_peered":"2024-04-25T08:50:49.466276+0000","last_clean":"2024-04-25T08:50:49.466276+0000","last_became_active":"2024-04-25T08:50:48.471052+0000","last_became_peered":"2024-04-25T08:50:48.471052+0000","last_unstale":"2024-04-25T08:50:49.466276+0000","last_undegraded":"2024-04-25T08:50:49.466276+0000","last_fullsized":"2024-04-25T08:50:49.466276+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:02.994114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.466107+0000","last_change":"2024-04-25T08:50:47.481237+0000","last_active":"2024-04-25T08:50:49.466107+0000","last_peered":"2024-04-25T08:50:49.466107+0000","last_clean":"2024-04-25T08:50:49.466107+0000","last_became_active":"2024-04-25T08:50:47.480947+0000","last_became_peered":"2024-04-25T08:50:47.480947+0000","last_unstale":"2024-04-25T08:50:49.466107+0000","last_undegraded":"2024-04-25T08:50:49.466107+0000","last_fullsized":"2024-04-25T08:50:49.466107+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:46.158215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490657+0000","last_change":"2024-04-25T08:50:48.487849+0000","last_active":"2024-04-25T08:50:49.490657+0000","last_peered":"2024-04-25T08:50:49.490657+0000","last_clean":"2024-04-25T08:50:49.490657+0000","last_became_active":"2024-04-25T08:50:48.487561+0000","last_became_peered":"2024-04-25T08:50:48.487561+0000","last_unstale":"2024-04-25T08:50:49.490657+0000","last_undegraded":"2024-04-25T08:50:49.490657+0000","last_fullsized":"2024-04-25T08:50:49.490657+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.676510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660671+0000","last_change":"2024-04-25T08:50:47.467828+0000","last_active":"2024-04-25T08:50:50.660671+0000","last_peered":"2024-04-25T08:50:50.660671+0000","last_clean":"2024-04-25T08:50:50.660671+0000","last_became_active":"2024-04-25T08:50:47.467530+0000","last_became_peered":"2024-04-25T08:50:47.467530+0000","last_unstale":"2024-04-25T08:50:50.660671+0000","last_undegraded":"2024-04-25T08:50:50.660671+0000","last_fullsized":"2024-04-25T08:50:50.660671+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:10.299319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462782+0000","last_change":"2024-04-25T08:50:48.484439+0000","last_active":"2024-04-25T08:50:49.462782+0000","last_peered":"2024-04-25T08:50:49.462782+0000","last_clean":"2024-04-25T08:50:49.462782+0000","last_became_active":"2024-04-25T08:50:48.484173+0000","last_became_peered":"2024-04-25T08:50:48.484173+0000","last_unstale":"2024-04-25T08:50:49.462782+0000","last_undegraded":"2024-04-25T08:50:49.462782+0000","last_fullsized":"2024-04-25T08:50:49.462782+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:21.446688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662734+0000","last_change":"2024-04-25T08:50:47.484474+0000","last_active":"2024-04-25T08:50:50.662734+0000","last_peered":"2024-04-25T08:50:50.662734+0000","last_clean":"2024-04-25T08:50:50.662734+0000","last_became_active":"2024-04-25T08:50:47.484155+0000","last_became_peered":"2024-04-25T08:50:47.484155+0000","last_unstale":"2024-04-25T08:50:50.662734+0000","last_undegraded":"2024-04-25T08:50:50.662734+0000","last_fullsized":"2024-04-25T08:50:50.662734+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:18.639078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463739+0000","last_change":"2024-04-25T08:50:48.477464+0000","last_active":"2024-04-25T08:50:49.463739+0000","last_peered":"2024-04-25T08:50:49.463739+0000","last_clean":"2024-04-25T08:50:49.463739+0000","last_became_active":"2024-04-25T08:50:48.477139+0000","last_became_peered":"2024-04-25T08:50:48.477139+0000","last_unstale":"2024-04-25T08:50:49.463739+0000","last_undegraded":"2024-04-25T08:50:49.463739+0000","last_fullsized":"2024-04-25T08:50:49.463739+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:13.218720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490310+0000","last_change":"2024-04-25T08:50:47.476837+0000","last_active":"2024-04-25T08:50:49.490310+0000","last_peered":"2024-04-25T08:50:49.490310+0000","last_clean":"2024-04-25T08:50:49.490310+0000","last_became_active":"2024-04-25T08:50:47.476477+0000","last_became_peered":"2024-04-25T08:50:47.476477+0000","last_unstale":"2024-04-25T08:50:49.490310+0000","last_undegraded":"2024-04-25T08:50:49.490310+0000","last_fullsized":"2024-04-25T08:50:49.490310+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:20.854277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663844+0000","last_change":"2024-04-25T08:50:48.490496+0000","last_active":"2024-04-25T08:50:50.663844+0000","last_peered":"2024-04-25T08:50:50.663844+0000","last_clean":"2024-04-25T08:50:50.663844+0000","last_became_active":"2024-04-25T08:50:48.490203+0000","last_became_peered":"2024-04-25T08:50:48.490203+0000","last_unstale":"2024-04-25T08:50:50.663844+0000","last_undegraded":"2024-04-25T08:50:50.663844+0000","last_fullsized":"2024-04-25T08:50:50.663844+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:07.091576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465546+0000","last_change":"2024-04-25T08:50:47.475588+0000","last_active":"2024-04-25T08:50:49.465546+0000","last_peered":"2024-04-25T08:50:49.465546+0000","last_clean":"2024-04-25T08:50:49.465546+0000","last_became_active":"2024-04-25T08:50:47.475281+0000","last_became_peered":"2024-04-25T08:50:47.475281+0000","last_unstale":"2024-04-25T08:50:49.465546+0000","last_undegraded":"2024-04-25T08:50:49.465546+0000","last_fullsized":"2024-04-25T08:50:49.465546+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:57.813694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463052+0000","last_change":"2024-04-25T08:50:48.479461+0000","last_active":"2024-04-25T08:50:49.463052+0000","last_peered":"2024-04-25T08:50:49.463052+0000","last_clean":"2024-04-25T08:50:49.463052+0000","last_became_active":"2024-04-25T08:50:48.478946+0000","last_became_peered":"2024-04-25T08:50:48.478946+0000","last_unstale":"2024-04-25T08:50:49.463052+0000","last_undegraded":"2024-04-25T08:50:49.463052+0000","last_fullsized":"2024-04-25T08:50:49.463052+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:49.338966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462019+0000","last_change":"2024-04-25T08:50:47.477440+0000","last_active":"2024-04-25T08:50:49.462019+0000","last_peered":"2024-04-25T08:50:49.462019+0000","last_clean":"2024-04-25T08:50:49.462019+0000","last_became_active":"2024-04-25T08:50:47.477161+0000","last_became_peered":"2024-04-25T08:50:47.477161+0000","last_unstale":"2024-04-25T08:50:49.462019+0000","last_undegraded":"2024-04-25T08:50:49.462019+0000","last_fullsized":"2024-04-25T08:50:49.462019+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:18.639078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463245+0000","last_change":"2024-04-25T08:50:48.473937+0000","last_active":"2024-04-25T08:50:49.463245+0000","last_peered":"2024-04-25T08:50:49.463245+0000","last_clean":"2024-04-25T08:50:49.463245+0000","last_became_active":"2024-04-25T08:50:48.473638+0000","last_became_peered":"2024-04-25T08:50:48.473638+0000","last_unstale":"2024-04-25T08:50:49.463245+0000","last_undegraded":"2024-04-25T08:50:49.463245+0000","last_fullsized":"2024-04-25T08:50:49.463245+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:53.606520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465936+0000","last_change":"2024-04-25T08:50:47.485270+0000","last_active":"2024-04-25T08:50:49.465936+0000","last_peered":"2024-04-25T08:50:49.465936+0000","last_clean":"2024-04-25T08:50:49.465936+0000","last_became_active":"2024-04-25T08:50:47.481349+0000","last_became_peered":"2024-04-25T08:50:47.481349+0000","last_unstale":"2024-04-25T08:50:49.465936+0000","last_undegraded":"2024-04-25T08:50:49.465936+0000","last_fullsized":"2024-04-25T08:50:49.465936+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:45.365914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660497+0000","last_change":"2024-04-25T08:50:48.493623+0000","last_active":"2024-04-25T08:50:50.660497+0000","last_peered":"2024-04-25T08:50:50.660497+0000","last_clean":"2024-04-25T08:50:50.660497+0000","last_became_active":"2024-04-25T08:50:48.493278+0000","last_became_peered":"2024-04-25T08:50:48.493278+0000","last_unstale":"2024-04-25T08:50:50.660497+0000","last_undegraded":"2024-04-25T08:50:50.660497+0000","last_fullsized":"2024-04-25T08:50:50.660497+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:20.656908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463480+0000","last_change":"2024-04-25T08:50:47.476603+0000","last_active":"2024-04-25T08:50:49.463480+0000","last_peered":"2024-04-25T08:50:49.463480+0000","last_clean":"2024-04-25T08:50:49.463480+0000","last_became_active":"2024-04-25T08:50:47.476321+0000","last_became_peered":"2024-04-25T08:50:47.476321+0000","last_unstale":"2024-04-25T08:50:49.463480+0000","last_undegraded":"2024-04-25T08:50:49.463480+0000","last_fullsized":"2024-04-25T08:50:49.463480+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:09.022500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660402+0000","last_change":"2024-04-25T08:50:48.489430+0000","last_active":"2024-04-25T08:50:50.660402+0000","last_peered":"2024-04-25T08:50:50.660402+0000","last_clean":"2024-04-25T08:50:50.660402+0000","last_became_active":"2024-04-25T08:50:48.489153+0000","last_became_peered":"2024-04-25T08:50:48.489153+0000","last_unstale":"2024-04-25T08:50:50.660402+0000","last_undegraded":"2024-04-25T08:50:50.660402+0000","last_fullsized":"2024-04-25T08:50:50.660402+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:26.106123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"105'3","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662913+0000","last_change":"2024-04-25T08:50:47.485045+0000","last_active":"2024-04-25T08:50:50.662913+0000","last_peered":"2024-04-25T08:50:50.662913+0000","last_clean":"2024-04-25T08:50:50.662913+0000","last_became_active":"2024-04-25T08:50:47.484741+0000","last_became_peered":"2024-04-25T08:50:47.484741+0000","last_unstale":"2024-04-25T08:50:50.662913+0000","last_undegraded":"2024-04-25T08:50:50.662913+0000","last_fullsized":"2024-04-25T08:50:50.662913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:46.669342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660803+0000","last_change":"2024-04-25T08:50:48.488504+0000","last_active":"2024-04-25T08:50:50.660803+0000","last_peered":"2024-04-25T08:50:50.660803+0000","last_clean":"2024-04-25T08:50:50.660803+0000","last_became_active":"2024-04-25T08:50:48.488192+0000","last_became_peered":"2024-04-25T08:50:48.488192+0000","last_unstale":"2024-04-25T08:50:50.660803+0000","last_undegraded":"2024-04-25T08:50:50.660803+0000","last_fullsized":"2024-04-25T08:50:50.660803+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:52.126434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"105'6","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663469+0000","last_change":"2024-04-25T08:50:47.484075+0000","last_active":"2024-04-25T08:50:50.663469+0000","last_peered":"2024-04-25T08:50:50.663469+0000","last_clean":"2024-04-25T08:50:50.663469+0000","last_became_active":"2024-04-25T08:50:47.483508+0000","last_became_peered":"2024-04-25T08:50:47.483508+0000","last_unstale":"2024-04-25T08:50:50.663469+0000","last_undegraded":"2024-04-25T08:50:50.663469+0000","last_fullsized":"2024-04-25T08:50:50.663469+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:10.299319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490528+0000","last_change":"2024-04-25T08:50:48.490545+0000","last_active":"2024-04-25T08:50:49.490528+0000","last_peered":"2024-04-25T08:50:49.490528+0000","last_clean":"2024-04-25T08:50:49.490528+0000","last_became_active":"2024-04-25T08:50:48.490120+0000","last_became_peered":"2024-04-25T08:50:48.490120+0000","last_unstale":"2024-04-25T08:50:49.490528+0000","last_undegraded":"2024-04-25T08:50:49.490528+0000","last_fullsized":"2024-04-25T08:50:49.490528+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:52.126434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660740+0000","last_change":"2024-04-25T08:50:47.484670+0000","last_active":"2024-04-25T08:50:50.660740+0000","last_peered":"2024-04-25T08:50:50.660740+0000","last_clean":"2024-04-25T08:50:50.660740+0000","last_became_active":"2024-04-25T08:50:47.483722+0000","last_became_peered":"2024-04-25T08:50:47.483722+0000","last_unstale":"2024-04-25T08:50:50.660740+0000","last_undegraded":"2024-04-25T08:50:50.660740+0000","last_fullsized":"2024-04-25T08:50:50.660740+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:02.515846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663584+0000","last_change":"2024-04-25T08:50:48.478959+0000","last_active":"2024-04-25T08:50:50.663584+0000","last_peered":"2024-04-25T08:50:50.663584+0000","last_clean":"2024-04-25T08:50:50.663584+0000","last_became_active":"2024-04-25T08:50:48.478690+0000","last_became_peered":"2024-04-25T08:50:48.478690+0000","last_unstale":"2024-04-25T08:50:50.663584+0000","last_undegraded":"2024-04-25T08:50:50.663584+0000","last_fullsized":"2024-04-25T08:50:50.663584+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:40.772018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462347+0000","last_change":"2024-04-25T08:50:47.473100+0000","last_active":"2024-04-25T08:50:49.462347+0000","last_peered":"2024-04-25T08:50:49.462347+0000","last_clean":"2024-04-25T08:50:49.462347+0000","last_became_active":"2024-04-25T08:50:47.472754+0000","last_became_peered":"2024-04-25T08:50:47.472754+0000","last_unstale":"2024-04-25T08:50:49.462347+0000","last_undegraded":"2024-04-25T08:50:49.462347+0000","last_fullsized":"2024-04-25T08:50:49.462347+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:39.764851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661248+0000","last_change":"2024-04-25T08:50:48.486298+0000","last_active":"2024-04-25T08:50:50.661248+0000","last_peered":"2024-04-25T08:50:50.661248+0000","last_clean":"2024-04-25T08:50:50.661248+0000","last_became_active":"2024-04-25T08:50:48.486012+0000","last_became_peered":"2024-04-25T08:50:48.486012+0000","last_unstale":"2024-04-25T08:50:50.661248+0000","last_undegraded":"2024-04-25T08:50:50.661248+0000","last_fullsized":"2024-04-25T08:50:50.661248+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:40.000648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660043+0000","last_change":"2024-04-25T08:50:47.473573+0000","last_active":"2024-04-25T08:50:50.660043+0000","last_peered":"2024-04-25T08:50:50.660043+0000","last_clean":"2024-04-25T08:50:50.660043+0000","last_became_active":"2024-04-25T08:50:47.472779+0000","last_became_peered":"2024-04-25T08:50:47.472779+0000","last_unstale":"2024-04-25T08:50:50.660043+0000","last_undegraded":"2024-04-25T08:50:50.660043+0000","last_fullsized":"2024-04-25T08:50:50.660043+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:13.074380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463807+0000","last_change":"2024-04-25T08:50:48.487954+0000","last_active":"2024-04-25T08:50:49.463807+0000","last_peered":"2024-04-25T08:50:49.463807+0000","last_clean":"2024-04-25T08:50:49.463807+0000","last_became_active":"2024-04-25T08:50:48.487582+0000","last_became_peered":"2024-04-25T08:50:48.487582+0000","last_unstale":"2024-04-25T08:50:49.463807+0000","last_undegraded":"2024-04-25T08:50:49.463807+0000","last_fullsized":"2024-04-25T08:50:49.463807+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:10.997023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"105'8","reported_seq":22,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465522+0000","last_change":"2024-04-25T08:50:47.475954+0000","last_active":"2024-04-25T08:50:49.465522+0000","last_peered":"2024-04-25T08:50:49.465522+0000","last_clean":"2024-04-25T08:50:49.465522+0000","last_became_active":"2024-04-25T08:50:47.475269+0000","last_became_peered":"2024-04-25T08:50:47.475269+0000","last_unstale":"2024-04-25T08:50:49.465522+0000","last_undegraded":"2024-04-25T08:50:49.465522+0000","last_fullsized":"2024-04-25T08:50:49.465522+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:46.961095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660140+0000","last_change":"2024-04-25T08:50:48.480241+0000","last_active":"2024-04-25T08:50:50.660140+0000","last_peered":"2024-04-25T08:50:50.660140+0000","last_clean":"2024-04-25T08:50:50.660140+0000","last_became_active":"2024-04-25T08:50:48.479905+0000","last_became_peered":"2024-04-25T08:50:48.479905+0000","last_unstale":"2024-04-25T08:50:50.660140+0000","last_undegraded":"2024-04-25T08:50:50.660140+0000","last_fullsized":"2024-04-25T08:50:50.660140+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:13.400633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663657+0000","last_change":"2024-04-25T08:50:47.479962+0000","last_active":"2024-04-25T08:50:50.663657+0000","last_peered":"2024-04-25T08:50:50.663657+0000","last_clean":"2024-04-25T08:50:50.663657+0000","last_became_active":"2024-04-25T08:50:47.479009+0000","last_became_peered":"2024-04-25T08:50:47.479009+0000","last_unstale":"2024-04-25T08:50:50.663657+0000","last_undegraded":"2024-04-25T08:50:50.663657+0000","last_fullsized":"2024-04-25T08:50:50.663657+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:21.594237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663183+0000","last_change":"2024-04-25T08:50:48.475915+0000","last_active":"2024-04-25T08:50:50.663183+0000","last_peered":"2024-04-25T08:50:50.663183+0000","last_clean":"2024-04-25T08:50:50.663183+0000","last_became_active":"2024-04-25T08:50:48.475537+0000","last_became_peered":"2024-04-25T08:50:48.475537+0000","last_unstale":"2024-04-25T08:50:50.663183+0000","last_undegraded":"2024-04-25T08:50:50.663183+0000","last_fullsized":"2024-04-25T08:50:50.663183+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:30.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"105'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663040+0000","last_change":"2024-04-25T08:50:47.479576+0000","last_active":"2024-04-25T08:50:50.663040+0000","last_peered":"2024-04-25T08:50:50.663040+0000","last_clean":"2024-04-25T08:50:50.663040+0000","last_became_active":"2024-04-25T08:50:47.479308+0000","last_became_peered":"2024-04-25T08:50:47.479308+0000","last_unstale":"2024-04-25T08:50:50.663040+0000","last_undegraded":"2024-04-25T08:50:50.663040+0000","last_fullsized":"2024-04-25T08:50:50.663040+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:39.764851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660683+0000","last_change":"2024-04-25T08:50:48.475110+0000","last_active":"2024-04-25T08:50:50.660683+0000","last_peered":"2024-04-25T08:50:50.660683+0000","last_clean":"2024-04-25T08:50:50.660683+0000","last_became_active":"2024-04-25T08:50:48.474785+0000","last_became_peered":"2024-04-25T08:50:48.474785+0000","last_unstale":"2024-04-25T08:50:50.660683+0000","last_undegraded":"2024-04-25T08:50:50.660683+0000","last_fullsized":"2024-04-25T08:50:50.660683+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:04.379008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490544+0000","last_change":"2024-04-25T08:50:47.483654+0000","last_active":"2024-04-25T08:50:49.490544+0000","last_peered":"2024-04-25T08:50:49.490544+0000","last_clean":"2024-04-25T08:50:49.490544+0000","last_became_active":"2024-04-25T08:50:47.483403+0000","last_became_peered":"2024-04-25T08:50:47.483403+0000","last_unstale":"2024-04-25T08:50:49.490544+0000","last_undegraded":"2024-04-25T08:50:49.490544+0000","last_fullsized":"2024-04-25T08:50:49.490544+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:46.961095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463627+0000","last_change":"2024-04-25T08:50:48.482993+0000","last_active":"2024-04-25T08:50:49.463627+0000","last_peered":"2024-04-25T08:50:49.463627+0000","last_clean":"2024-04-25T08:50:49.463627+0000","last_became_active":"2024-04-25T08:50:48.482713+0000","last_became_peered":"2024-04-25T08:50:48.482713+0000","last_unstale":"2024-04-25T08:50:49.463627+0000","last_undegraded":"2024-04-25T08:50:49.463627+0000","last_fullsized":"2024-04-25T08:50:49.463627+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:13.276757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661152+0000","last_change":"2024-04-25T08:50:47.481872+0000","last_active":"2024-04-25T08:50:50.661152+0000","last_peered":"2024-04-25T08:50:50.661152+0000","last_clean":"2024-04-25T08:50:50.661152+0000","last_became_active":"2024-04-25T08:50:47.481589+0000","last_became_peered":"2024-04-25T08:50:47.481589+0000","last_unstale":"2024-04-25T08:50:50.661152+0000","last_undegraded":"2024-04-25T08:50:50.661152+0000","last_fullsized":"2024-04-25T08:50:50.661152+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:45.722329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463773+0000","last_change":"2024-04-25T08:50:48.468293+0000","last_active":"2024-04-25T08:50:49.463773+0000","last_peered":"2024-04-25T08:50:49.463773+0000","last_clean":"2024-04-25T08:50:49.463773+0000","last_became_active":"2024-04-25T08:50:48.468022+0000","last_became_peered":"2024-04-25T08:50:48.468022+0000","last_unstale":"2024-04-25T08:50:49.463773+0000","last_undegraded":"2024-04-25T08:50:49.463773+0000","last_fullsized":"2024-04-25T08:50:49.463773+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:25.114089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660159+0000","last_change":"2024-04-25T08:50:47.475919+0000","last_active":"2024-04-25T08:50:50.660159+0000","last_peered":"2024-04-25T08:50:50.660159+0000","last_clean":"2024-04-25T08:50:50.660159+0000","last_became_active":"2024-04-25T08:50:47.475589+0000","last_became_peered":"2024-04-25T08:50:47.475589+0000","last_unstale":"2024-04-25T08:50:50.660159+0000","last_undegraded":"2024-04-25T08:50:50.660159+0000","last_fullsized":"2024-04-25T08:50:50.660159+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:51.346218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463052+0000","last_change":"2024-04-25T08:50:48.480280+0000","last_active":"2024-04-25T08:50:49.463052+0000","last_peered":"2024-04-25T08:50:49.463052+0000","last_clean":"2024-04-25T08:50:49.463052+0000","last_became_active":"2024-04-25T08:50:48.480013+0000","last_became_peered":"2024-04-25T08:50:48.480013+0000","last_unstale":"2024-04-25T08:50:49.463052+0000","last_undegraded":"2024-04-25T08:50:49.463052+0000","last_fullsized":"2024-04-25T08:50:49.463052+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.389533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662882+0000","last_change":"2024-04-25T08:50:47.479727+0000","last_active":"2024-04-25T08:50:50.662882+0000","last_peered":"2024-04-25T08:50:50.662882+0000","last_clean":"2024-04-25T08:50:50.662882+0000","last_became_active":"2024-04-25T08:50:47.479464+0000","last_became_peered":"2024-04-25T08:50:47.479464+0000","last_unstale":"2024-04-25T08:50:50.662882+0000","last_undegraded":"2024-04-25T08:50:50.662882+0000","last_fullsized":"2024-04-25T08:50:50.662882+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:02.515846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661099+0000","last_change":"2024-04-25T08:50:48.490457+0000","last_active":"2024-04-25T08:50:50.661099+0000","last_peered":"2024-04-25T08:50:50.661099+0000","last_clean":"2024-04-25T08:50:50.661099+0000","last_became_active":"2024-04-25T08:50:48.488728+0000","last_became_peered":"2024-04-25T08:50:48.488728+0000","last_unstale":"2024-04-25T08:50:50.661099+0000","last_undegraded":"2024-04-25T08:50:50.661099+0000","last_fullsized":"2024-04-25T08:50:50.661099+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:20.939116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660092+0000","last_change":"2024-04-25T08:50:47.472781+0000","last_active":"2024-04-25T08:50:50.660092+0000","last_peered":"2024-04-25T08:50:50.660092+0000","last_clean":"2024-04-25T08:50:50.660092+0000","last_became_active":"2024-04-25T08:50:47.472447+0000","last_became_peered":"2024-04-25T08:50:47.472447+0000","last_unstale":"2024-04-25T08:50:50.660092+0000","last_undegraded":"2024-04-25T08:50:50.660092+0000","last_fullsized":"2024-04-25T08:50:50.660092+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.945660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463243+0000","last_change":"2024-04-25T08:50:48.478014+0000","last_active":"2024-04-25T08:50:49.463243+0000","last_peered":"2024-04-25T08:50:49.463243+0000","last_clean":"2024-04-25T08:50:49.463243+0000","last_became_active":"2024-04-25T08:50:48.477765+0000","last_became_peered":"2024-04-25T08:50:48.477765+0000","last_unstale":"2024-04-25T08:50:49.463243+0000","last_undegraded":"2024-04-25T08:50:49.463243+0000","last_fullsized":"2024-04-25T08:50:49.463243+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:13.863654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661514+0000","last_change":"2024-04-25T08:50:47.477416+0000","last_active":"2024-04-25T08:50:50.661514+0000","last_peered":"2024-04-25T08:50:50.661514+0000","last_clean":"2024-04-25T08:50:50.661514+0000","last_became_active":"2024-04-25T08:50:47.477157+0000","last_became_peered":"2024-04-25T08:50:47.477157+0000","last_unstale":"2024-04-25T08:50:50.661514+0000","last_undegraded":"2024-04-25T08:50:50.661514+0000","last_fullsized":"2024-04-25T08:50:50.661514+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:21.594237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465624+0000","last_change":"2024-04-25T08:50:48.479655+0000","last_active":"2024-04-25T08:50:49.465624+0000","last_peered":"2024-04-25T08:50:49.465624+0000","last_clean":"2024-04-25T08:50:49.465624+0000","last_became_active":"2024-04-25T08:50:48.479222+0000","last_became_peered":"2024-04-25T08:50:48.479222+0000","last_unstale":"2024-04-25T08:50:49.465624+0000","last_undegraded":"2024-04-25T08:50:49.465624+0000","last_fullsized":"2024-04-25T08:50:49.465624+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:47.676510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"106'1","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.463647+0000","last_change":"2024-04-25T08:50:47.483125+0000","last_active":"2024-04-25T08:50:49.463647+0000","last_peered":"2024-04-25T08:50:49.463647+0000","last_clean":"2024-04-25T08:50:49.463647+0000","last_became_active":"2024-04-25T08:50:47.481358+0000","last_became_peered":"2024-04-25T08:50:47.481358+0000","last_unstale":"2024-04-25T08:50:49.463647+0000","last_undegraded":"2024-04-25T08:50:49.463647+0000","last_fullsized":"2024-04-25T08:50:49.463647+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:29.086389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.462270+0000","last_change":"2024-04-25T08:50:48.479501+0000","last_active":"2024-04-25T08:50:49.462270+0000","last_peered":"2024-04-25T08:50:49.462270+0000","last_clean":"2024-04-25T08:50:49.462270+0000","last_became_active":"2024-04-25T08:50:48.479201+0000","last_became_peered":"2024-04-25T08:50:48.479201+0000","last_unstale":"2024-04-25T08:50:49.462270+0000","last_undegraded":"2024-04-25T08:50:49.462270+0000","last_fullsized":"2024-04-25T08:50:49.462270+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:51.015513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663561+0000","last_change":"2024-04-25T08:50:47.479154+0000","last_active":"2024-04-25T08:50:50.663561+0000","last_peered":"2024-04-25T08:50:50.663561+0000","last_clean":"2024-04-25T08:50:50.663561+0000","last_became_active":"2024-04-25T08:50:47.477739+0000","last_became_peered":"2024-04-25T08:50:47.477739+0000","last_unstale":"2024-04-25T08:50:50.663561+0000","last_undegraded":"2024-04-25T08:50:50.663561+0000","last_fullsized":"2024-04-25T08:50:50.663561+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:58.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490551+0000","last_change":"2024-04-25T08:50:48.488816+0000","last_active":"2024-04-25T08:50:49.490551+0000","last_peered":"2024-04-25T08:50:49.490551+0000","last_clean":"2024-04-25T08:50:49.490551+0000","last_became_active":"2024-04-25T08:50:48.488399+0000","last_became_peered":"2024-04-25T08:50:48.488399+0000","last_unstale":"2024-04-25T08:50:49.490551+0000","last_undegraded":"2024-04-25T08:50:49.490551+0000","last_fullsized":"2024-04-25T08:50:49.490551+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:30.761163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660751+0000","last_change":"2024-04-25T08:50:47.473632+0000","last_active":"2024-04-25T08:50:50.660751+0000","last_peered":"2024-04-25T08:50:50.660751+0000","last_clean":"2024-04-25T08:50:50.660751+0000","last_became_active":"2024-04-25T08:50:47.473285+0000","last_became_peered":"2024-04-25T08:50:47.473285+0000","last_unstale":"2024-04-25T08:50:50.660751+0000","last_undegraded":"2024-04-25T08:50:50.660751+0000","last_fullsized":"2024-04-25T08:50:50.660751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:58.293771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661041+0000","last_change":"2024-04-25T08:50:48.489306+0000","last_active":"2024-04-25T08:50:50.661041+0000","last_peered":"2024-04-25T08:50:50.661041+0000","last_clean":"2024-04-25T08:50:50.661041+0000","last_became_active":"2024-04-25T08:50:48.489035+0000","last_became_peered":"2024-04-25T08:50:48.489035+0000","last_unstale":"2024-04-25T08:50:50.661041+0000","last_undegraded":"2024-04-25T08:50:50.661041+0000","last_fullsized":"2024-04-25T08:50:50.661041+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:34.999810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.465745+0000","last_change":"2024-04-25T08:50:47.481624+0000","last_active":"2024-04-25T08:50:49.465745+0000","last_peered":"2024-04-25T08:50:49.465745+0000","last_clean":"2024-04-25T08:50:49.465745+0000","last_became_active":"2024-04-25T08:50:47.481051+0000","last_became_peered":"2024-04-25T08:50:47.481051+0000","last_unstale":"2024-04-25T08:50:49.465745+0000","last_undegraded":"2024-04-25T08:50:49.465745+0000","last_fullsized":"2024-04-25T08:50:49.465745+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:59.512829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661303+0000","last_change":"2024-04-25T08:50:48.490140+0000","last_active":"2024-04-25T08:50:50.661303+0000","last_peered":"2024-04-25T08:50:50.661303+0000","last_clean":"2024-04-25T08:50:50.661303+0000","last_became_active":"2024-04-25T08:50:48.489843+0000","last_became_peered":"2024-04-25T08:50:48.489843+0000","last_unstale":"2024-04-25T08:50:50.661303+0000","last_undegraded":"2024-04-25T08:50:50.661303+0000","last_fullsized":"2024-04-25T08:50:50.661303+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:48.209591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660500+0000","last_change":"2024-04-25T08:50:47.479010+0000","last_active":"2024-04-25T08:50:50.660500+0000","last_peered":"2024-04-25T08:50:50.660500+0000","last_clean":"2024-04-25T08:50:50.660500+0000","last_became_active":"2024-04-25T08:50:47.478768+0000","last_became_peered":"2024-04-25T08:50:47.478768+0000","last_unstale":"2024-04-25T08:50:50.660500+0000","last_undegraded":"2024-04-25T08:50:50.660500+0000","last_fullsized":"2024-04-25T08:50:50.660500+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:39.575982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663034+0000","last_change":"2024-04-25T08:50:48.492853+0000","last_active":"2024-04-25T08:50:50.663034+0000","last_peered":"2024-04-25T08:50:50.663034+0000","last_clean":"2024-04-25T08:50:50.663034+0000","last_became_active":"2024-04-25T08:50:48.491331+0000","last_became_peered":"2024-04-25T08:50:48.491331+0000","last_unstale":"2024-04-25T08:50:50.663034+0000","last_undegraded":"2024-04-25T08:50:50.663034+0000","last_fullsized":"2024-04-25T08:50:50.663034+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:51.015513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.663122+0000","last_change":"2024-04-25T08:50:47.480096+0000","last_active":"2024-04-25T08:50:50.663122+0000","last_peered":"2024-04-25T08:50:50.663122+0000","last_clean":"2024-04-25T08:50:50.663122+0000","last_became_active":"2024-04-25T08:50:47.477889+0000","last_became_peered":"2024-04-25T08:50:47.477889+0000","last_unstale":"2024-04-25T08:50:50.663122+0000","last_undegraded":"2024-04-25T08:50:50.663122+0000","last_fullsized":"2024-04-25T08:50:50.663122+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:16.871598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490863+0000","last_change":"2024-04-25T08:50:48.486956+0000","last_active":"2024-04-25T08:50:49.490863+0000","last_peered":"2024-04-25T08:50:49.490863+0000","last_clean":"2024-04-25T08:50:49.490863+0000","last_became_active":"2024-04-25T08:50:48.485562+0000","last_became_peered":"2024-04-25T08:50:48.485562+0000","last_unstale":"2024-04-25T08:50:49.490863+0000","last_undegraded":"2024-04-25T08:50:49.490863+0000","last_fullsized":"2024-04-25T08:50:49.490863+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:00.051922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.660171+0000","last_change":"2024-04-25T08:50:47.478107+0000","last_active":"2024-04-25T08:50:50.660171+0000","last_peered":"2024-04-25T08:50:50.660171+0000","last_clean":"2024-04-25T08:50:50.660171+0000","last_became_active":"2024-04-25T08:50:47.477775+0000","last_became_peered":"2024-04-25T08:50:47.477775+0000","last_unstale":"2024-04-25T08:50:50.660171+0000","last_undegraded":"2024-04-25T08:50:50.660171+0000","last_fullsized":"2024-04-25T08:50:50.660171+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:10.520973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:49.490444+0000","last_change":"2024-04-25T08:50:48.488454+0000","last_active":"2024-04-25T08:50:49.490444+0000","last_peered":"2024-04-25T08:50:49.490444+0000","last_clean":"2024-04-25T08:50:49.490444+0000","last_became_active":"2024-04-25T08:50:48.488168+0000","last_became_peered":"2024-04-25T08:50:48.488168+0000","last_unstale":"2024-04-25T08:50:49.490444+0000","last_undegraded":"2024-04-25T08:50:49.490444+0000","last_fullsized":"2024-04-25T08:50:49.490444+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:47.447570+0000","last_clean_scrub_stamp":"2024-04-25T08:50:47.447570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:04.379008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"105'2","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.661117+0000","last_change":"2024-04-25T08:50:47.476087+0000","last_active":"2024-04-25T08:50:50.661117+0000","last_peered":"2024-04-25T08:50:50.661117+0000","last_clean":"2024-04-25T08:50:50.661117+0000","last_became_active":"2024-04-25T08:50:47.475441+0000","last_became_peered":"2024-04-25T08:50:47.475441+0000","last_unstale":"2024-04-25T08:50:50.661117+0000","last_undegraded":"2024-04-25T08:50:50.661117+0000","last_fullsized":"2024-04-25T08:50:50.661117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:50:46.440403+0000","last_clean_scrub_stamp":"2024-04-25T08:50:46.440403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:04.616234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":331,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:50:50.662354+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:50:50.662354+0000","last_peered":"2024-04-25T08:50:50.662354+0000","last_clean":"2024-04-25T08:50:50.662354+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:50:50.662354+0000","last_undegraded":"2024-04-25T08:50:50.662354+0000","last_fullsized":"2024-04-25T08:50:50.662354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574957,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":1916928,"data_stored":1202437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":614125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1314816,"data_stored":611909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":612001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":612491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1318912,"data_stored":611943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":611943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1925120,"data_stored":1203979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:50:52.968 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:50:52.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:50:53.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.167+0000 7fe40e200640 1 Processor -- start 2024-04-25T08:50:53.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.167+0000 7fe40e200640 1 -- start start 2024-04-25T08:50:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.167+0000 7fe40e200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe408145310 0x7fe40806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.167+0000 7fe40e200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40806dc70 0x7fe4080cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.167+0000 7fe40e200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4080cd930 con 0x7fe4080cf1b0 2024-04-25T08:50:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.167+0000 7fe40e200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4080cdbd0 con 0x7fe408145310 2024-04-25T08:50:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.167+0000 7fe40e200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080cdeb0 con 0x7fe40806dc70 2024-04-25T08:50:53.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.171+0000 7fe407400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe408145310 0x7fe40806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.171+0000 7fe3fea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40806dc70 0x7fe4080cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.171+0000 7fe407400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe408145310 0x7fe40806e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:46976/0 (socket says 172.21.15.46:46976) 2024-04-25T08:50:53.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.171+0000 7fe407400640 1 -- 172.21.15.46:0/1290475367 learned_addr learned my addr 172.21.15.46:0/1290475367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:53.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.171+0000 7fe407400640 1 -- 172.21.15.46:0/1290475367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40806dc70 msgr2=0x7fe4080cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:53.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.171+0000 7fe407400640 1 --2- 172.21.15.46:0/1290475367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40806dc70 0x7fe4080cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.171+0000 7fe407400640 1 -- 172.21.15.46:0/1290475367 >> v1:172.21.15.82:6789/0 conn(0x7fe4080cf1b0 legacy=0x7fe40815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.171+0000 7fe407400640 1 -- 172.21.15.46:0/1290475367 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4080ce190 con 0x7fe408145310 2024-04-25T08:50:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.171+0000 7fe407400640 1 --2- 172.21.15.46:0/1290475367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408145310 0x7fe40806e0d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009870 tx=0x7fe3f801c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78510d8ac15dced7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe406a00640 1 -- 172.21.15.46:0/1290475367 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80a95f0 con 0x7fe408145310 2024-04-25T08:50:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe406a00640 1 -- 172.21.15.46:0/1290475367 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f80a9790 con 0x7fe408145310 2024-04-25T08:50:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe406a00640 1 -- 172.21.15.46:0/1290475367 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80a9b60 con 0x7fe408145310 2024-04-25T08:50:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 -- 172.21.15.46:0/1290475367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408145310 msgr2=0x7fe40806e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 --2- 172.21.15.46:0/1290475367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408145310 0x7fe40806e0d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009870 tx=0x7fe3f801c530 comp rx=0 tx=0).stop 2024-04-25T08:50:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 -- 172.21.15.46:0/1290475367 shutdown_connections 2024-04-25T08:50:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 --2- 172.21.15.46:0/1290475367 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe40806dc70 0x7fe4080cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 --2- 172.21.15.46:0/1290475367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408145310 0x7fe40806e0d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 -- 172.21.15.46:0/1290475367 >> 172.21.15.46:0/1290475367 conn(0x7fe40806ac10 msgr2=0x7fe40806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 -- 172.21.15.46:0/1290475367 shutdown_connections 2024-04-25T08:50:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 -- 172.21.15.46:0/1290475367 wait complete. 2024-04-25T08:50:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 Processor -- start 2024-04-25T08:50:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 -- start start 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4080cf1b0 0x7fe4080ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe408145310 0x7fe4081f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408158f70 0x7fe408144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40815d920 con 0x7fe4080cf1b0 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40815d7a0 con 0x7fe408158f70 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe40e200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe408168510 con 0x7fe408145310 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe407400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4080cf1b0 0x7fe4080ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe407400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4080cf1b0 0x7fe4080ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40164/0 (socket says 172.21.15.46:40164) 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.175+0000 7fe407400640 1 -- 172.21.15.46:0/2432473882 learned_addr learned my addr 172.21.15.46:0/2432473882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe407e00640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408158f70 0x7fe408144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe3fea00640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe408145310 0x7fe4081f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe407400640 1 -- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe408145310 msgr2=0x7fe4081f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe407400640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe408145310 0x7fe4081f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe407400640 1 -- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408158f70 msgr2=0x7fe408144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe407400640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408158f70 0x7fe408144470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe407400640 1 -- 172.21.15.46:0/2432473882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4080ae110 con 0x7fe4080cf1b0 2024-04-25T08:50:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe3fea00640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe408145310 0x7fe4081f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe407400640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4080cf1b0 0x7fe4080ada80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fe3f801a160 tx=0x7fe3f80192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe3ffe00640 1 -- 172.21.15.46:0/2432473882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80a95f0 con 0x7fe4080cf1b0 2024-04-25T08:50:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe3ffe00640 1 -- 172.21.15.46:0/2432473882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f80b1820 con 0x7fe4080cf1b0 2024-04-25T08:50:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe3ffe00640 1 -- 172.21.15.46:0/2432473882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80b1c10 con 0x7fe4080cf1b0 2024-04-25T08:50:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4080ae3f0 con 0x7fe4080cf1b0 2024-04-25T08:50:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe408159780 con 0x7fe4080cf1b0 2024-04-25T08:50:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe408069010 con 0x7fe4080cf1b0 2024-04-25T08:50:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe3ffe00640 1 -- 172.21.15.46:0/2432473882 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3f80b93e0 con 0x7fe4080cf1b0 2024-04-25T08:50:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe3ffe00640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3e4066f90 0x7fe3e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.179+0000 7fe3ffe00640 1 -- 172.21.15.46:0/2432473882 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe3f811f580 con 0x7fe4080cf1b0 2024-04-25T08:50:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.183+0000 7fe3fea00640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3e4066f90 0x7fe3e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.183+0000 7fe3fea00640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3e4066f90 0x7fe3e4069450 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe408159470 tx=0x7fe3f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:53.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.183+0000 7fe3ffe00640 1 -- 172.21.15.46:0/2432473882 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3f80c1030 con 0x7fe4080cf1b0 2024-04-25T08:50:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.403+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe40806d550 con 0x7fe4080cf1b0 2024-04-25T08:50:53.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.403+0000 7fe3ffe00640 1 -- 172.21.15.46:0/2432473882 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe3f80ad020 con 0x7fe4080cf1b0 2024-04-25T08:50:53.395 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-25T08:50:53.395 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q== 2024-04-25T08:50:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-25T08:50:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-25T08:50:53.396 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:50:53.396 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-25T08:50:53.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3e4066f90 msgr2=0x7fe3e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3e4066f90 0x7fe3e4069450 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe408159470 tx=0x7fe3f0008040 comp rx=0 tx=0).stop 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4080cf1b0 msgr2=0x7fe4080ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4080cf1b0 0x7fe4080ada80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fe3f801a160 tx=0x7fe3f80192d0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 shutdown_connections 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe3e4066f90 0x7fe3e4069450 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe408158f70 0x7fe408144470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe408145310 0x7fe4081f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 --2- 172.21.15.46:0/2432473882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4080cf1b0 0x7fe4080ada80 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 >> 172.21.15.46:0/2432473882 conn(0x7fe40806ac10 msgr2=0x7fe40815ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 shutdown_connections 2024-04-25T08:50:53.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.407+0000 7fe40e200640 1 -- 172.21.15.46:0/2432473882 wait complete. 2024-04-25T08:50:53.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:53.429 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:50:53.439+0000 7f465c000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:50:53.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.567+0000 7f2df3600640 1 Processor -- start 2024-04-25T08:50:53.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.567+0000 7f2df3600640 1 -- start start 2024-04-25T08:50:53.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.567+0000 7f2df3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec15a750 0x7f2dec06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.567+0000 7f2df3600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2dec0cc9e0 0x7f2dec06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.567+0000 7f2df3600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dec06d5e0 con 0x7f2dec0aed10 2024-04-25T08:50:53.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.567+0000 7f2df3600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dec15ab30 con 0x7f2dec0cc9e0 2024-04-25T08:50:53.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.567+0000 7f2df3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec0ce4f0 con 0x7f2dec15a750 2024-04-25T08:50:53.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.571+0000 7f2df1600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2dec0cc9e0 0x7f2dec06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.571+0000 7f2df1600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2dec0cc9e0 0x7f2dec06f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:46988/0 (socket says 172.21.15.46:46988) 2024-04-25T08:50:53.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.571+0000 7f2df1600640 1 -- 172.21.15.46:0/1579063273 learned_addr learned my addr 172.21.15.46:0/1579063273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:53.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.571+0000 7f2df0c00640 1 --2- 172.21.15.46:0/1579063273 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec15a750 0x7f2dec06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.571+0000 7f2df1600640 1 -- 172.21.15.46:0/1579063273 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec15a750 msgr2=0x7f2dec06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:53.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.571+0000 7f2df1600640 1 --2- 172.21.15.46:0/1579063273 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec15a750 0x7f2dec06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.571+0000 7f2df1600640 1 -- 172.21.15.46:0/1579063273 >> v1:172.21.15.82:6789/0 conn(0x7f2dec0aed10 legacy=0x7f2dec0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.571+0000 7f2df1600640 1 -- 172.21.15.46:0/1579063273 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec0ce720 con 0x7f2dec0cc9e0 2024-04-25T08:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.571+0000 7f2df1600640 1 --2- 172.21.15.46:0/1579063273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dec0cc9e0 0x7f2dec06f150 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009e50 tx=0x7f2de401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca2eb62358b830a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2deb400640 1 -- 172.21.15.46:0/1579063273 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40a57f0 con 0x7f2dec0cc9e0 2024-04-25T08:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2deb400640 1 -- 172.21.15.46:0/1579063273 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de40a5990 con 0x7f2dec0cc9e0 2024-04-25T08:50:53.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2deb400640 1 -- 172.21.15.46:0/1579063273 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40a5d60 con 0x7f2dec0cc9e0 2024-04-25T08:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 -- 172.21.15.46:0/1579063273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dec0cc9e0 msgr2=0x7f2dec06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 --2- 172.21.15.46:0/1579063273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dec0cc9e0 0x7f2dec06f150 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009e50 tx=0x7f2de401ca30 comp rx=0 tx=0).stop 2024-04-25T08:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 -- 172.21.15.46:0/1579063273 shutdown_connections 2024-04-25T08:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 --2- 172.21.15.46:0/1579063273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dec0cc9e0 0x7f2dec06f150 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 --2- 172.21.15.46:0/1579063273 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec15a750 0x7f2dec06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 -- 172.21.15.46:0/1579063273 >> 172.21.15.46:0/1579063273 conn(0x7f2dec0b12a0 msgr2=0x7f2dec0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 -- 172.21.15.46:0/1579063273 shutdown_connections 2024-04-25T08:50:53.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 -- 172.21.15.46:0/1579063273 wait complete. 2024-04-25T08:50:53.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 Processor -- start 2024-04-25T08:50:53.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 -- start start 2024-04-25T08:50:53.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dec0aed10 0x7f2dec14e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec0cc9e0 0x7f2dec142e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dec15a750 0x7f2dec0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec06db00 con 0x7f2dec15a750 2024-04-25T08:50:53.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec06d270 con 0x7f2dec0aed10 2024-04-25T08:50:53.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec06dc80 con 0x7f2dec0cc9e0 2024-04-25T08:50:53.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dec15a750 0x7f2dec0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dec15a750 0x7f2dec0adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40176/0 (socket says 172.21.15.46:40176) 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df1600640 1 -- 172.21.15.46:0/1348355120 learned_addr learned my addr 172.21.15.46:0/1348355120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2debe00640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec0cc9e0 0x7f2dec142e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.575+0000 7f2df0c00640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dec0aed10 0x7f2dec14e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2df1600640 1 -- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec0cc9e0 msgr2=0x7f2dec142e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2df1600640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec0cc9e0 0x7f2dec142e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2df1600640 1 -- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dec0aed10 msgr2=0x7f2dec14e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2df1600640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dec0aed10 0x7f2dec14e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2df1600640 1 -- 172.21.15.46:0/1348355120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec1433d0 con 0x7f2dec15a750 2024-04-25T08:50:53.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2df1600640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dec15a750 0x7f2dec0adb10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2de401a660 tx=0x7f2de40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:53.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2de8c00640 1 -- 172.21.15.46:0/1348355120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40b8670 con 0x7f2dec15a750 2024-04-25T08:50:53.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2de8c00640 1 -- 172.21.15.46:0/1348355120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de40be070 con 0x7f2dec15a750 2024-04-25T08:50:53.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2de8c00640 1 -- 172.21.15.46:0/1348355120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de40b8a90 con 0x7f2dec15a750 2024-04-25T08:50:53.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dec14e0a0 con 0x7f2dec15a750 2024-04-25T08:50:53.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec14ddc0 con 0x7f2dec15a750 2024-04-25T08:50:53.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dec069080 con 0x7f2dec15a750 2024-04-25T08:50:53.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.579+0000 7f2de8c00640 1 -- 172.21.15.46:0/1348355120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2de40b8c30 con 0x7f2dec15a750 2024-04-25T08:50:53.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.583+0000 7f2de8c00640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dcc066f60 0x7f2dcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:53.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.583+0000 7f2de8c00640 1 -- 172.21.15.46:0/1348355120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2de41202b0 con 0x7f2dec15a750 2024-04-25T08:50:53.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.583+0000 7f2df0c00640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dcc066f60 0x7f2dcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:53.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.583+0000 7f2df0c00640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dcc066f60 0x7f2dcc069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009ac0 tx=0x7f2de0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:53.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.583+0000 7f2de8c00640 1 -- 172.21.15.46:0/1348355120 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2de4016620 con 0x7f2dec15a750 2024-04-25T08:50:53.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dec15da70 con 0x7f2dec15a750 2024-04-25T08:50:53.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2de8c00640 1 -- 172.21.15.46:0/1348355120 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f2de40a5960 con 0x7f2dec15a750 2024-04-25T08:50:53.774 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:53.774 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:50:50.870175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8334":{"gid":8334,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/623970467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":623970467},{"type":"v1","addr":"172.21.15.46:6835","nonce":623970467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:50:53.774 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 83 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dcc066f60 msgr2=0x7f2dcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dcc066f60 0x7f2dcc069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009ac0 tx=0x7f2de0009190 comp rx=0 tx=0).stop 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dec15a750 msgr2=0x7f2dec0adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dec15a750 0x7f2dec0adb10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f2de401a660 tx=0x7f2de40197d0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 shutdown_connections 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2dcc066f60 0x7f2dcc069420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dec15a750 0x7f2dec0adb10 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2dec0cc9e0 0x7f2dec142e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 --2- 172.21.15.46:0/1348355120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2dec0aed10 0x7f2dec14e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 >> 172.21.15.46:0/1348355120 conn(0x7f2dec0b12a0 msgr2=0x7f2dec150330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 shutdown_connections 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:53.783+0000 7f2df3600640 1 -- 172.21.15.46:0/1348355120 wait complete. 2024-04-25T08:50:54.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:50:47.860021+0000', 'modified': '2024-04-25T08:50:50.870175+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8334}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8334': {'gid': 8334, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/623970467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 623970467}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 623970467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:50:54.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:50:54.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:50:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff712400640 1 Processor -- start 2024-04-25T08:50:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff712400640 1 -- start start 2024-04-25T08:50:54.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff712400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 0x7ff70c0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff712400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff70c0cd6d0 0x7ff70c09a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff712400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c15f300 con 0x7ff70c09ab50 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff712400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff70c0cddb0 con 0x7ff70c0cd6d0 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff712400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0cc3d0 con 0x7ff70c15ef20 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff702a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff70c0cd6d0 0x7ff70c09a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff702a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff70c0cd6d0 0x7ff70c09a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40190/0 (socket says 172.21.15.46:40190) 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff702a00640 1 -- 172.21.15.46:0/341181603 learned_addr learned my addr 172.21.15.46:0/341181603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff70b400640 1 --2- 172.21.15.46:0/341181603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 0x7ff70c0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff702a00640 1 -- 172.21.15.46:0/341181603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 msgr2=0x7ff70c0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff702a00640 1 --2- 172.21.15.46:0/341181603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 0x7ff70c0cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff702a00640 1 -- 172.21.15.46:0/341181603 >> v1:172.21.15.82:6789/0 conn(0x7ff70c09ab50 legacy=0x7ff70c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff702a00640 1 -- 172.21.15.46:0/341181603 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c0cc6b0 con 0x7ff70c0cd6d0 2024-04-25T08:50:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.355+0000 7ff70b400640 1 --2- 172.21.15.46:0/341181603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 0x7ff70c0cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff702a00640 1 --2- 172.21.15.46:0/341181603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c0cd6d0 0x7ff70c09a610 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009870 tx=0x7ff6f401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899abd3e75940842 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff70aa00640 1 -- 172.21.15.46:0/341181603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40a95f0 con 0x7ff70c0cd6d0 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff70aa00640 1 -- 172.21.15.46:0/341181603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f40a9790 con 0x7ff70c0cd6d0 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff70aa00640 1 -- 172.21.15.46:0/341181603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40a9b60 con 0x7ff70c0cd6d0 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff712400640 1 -- 172.21.15.46:0/341181603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c0cd6d0 msgr2=0x7ff70c09a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff712400640 1 --2- 172.21.15.46:0/341181603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c0cd6d0 0x7ff70c09a610 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009870 tx=0x7ff6f401c530 comp rx=0 tx=0).stop 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff712400640 1 -- 172.21.15.46:0/341181603 shutdown_connections 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff712400640 1 --2- 172.21.15.46:0/341181603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c0cd6d0 0x7ff70c09a610 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff712400640 1 --2- 172.21.15.46:0/341181603 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 0x7ff70c0cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff712400640 1 -- 172.21.15.46:0/341181603 >> 172.21.15.46:0/341181603 conn(0x7ff70c0b3120 msgr2=0x7ff70c0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff712400640 1 -- 172.21.15.46:0/341181603 shutdown_connections 2024-04-25T08:50:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.359+0000 7ff712400640 1 -- 172.21.15.46:0/341181603 wait complete. 2024-04-25T08:50:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 Processor -- start 2024-04-25T08:50:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 -- start start 2024-04-25T08:50:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c09ab50 0x7ff70c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 0x7ff70c152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff70c1530d0 0x7ff70c153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0c5b10 con 0x7ff70c09ab50 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0c5990 con 0x7ff70c1530d0 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c09a110 con 0x7ff70c15ef20 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c09ab50 0x7ff70c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c09ab50 0x7ff70c0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40196/0 (socket says 172.21.15.46:40196) 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70b400640 1 -- 172.21.15.46:0/3185487927 learned_addr learned my addr 172.21.15.46:0/3185487927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70be00640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff70c1530d0 0x7ff70c153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff702a00640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 0x7ff70c152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70b400640 1 -- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 msgr2=0x7ff70c152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70b400640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 0x7ff70c152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70b400640 1 -- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff70c1530d0 msgr2=0x7ff70c153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70b400640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff70c1530d0 0x7ff70c153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70b400640 1 -- 172.21.15.46:0/3185487927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c157a10 con 0x7ff70c09ab50 2024-04-25T08:50:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff70b400640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c09ab50 0x7ff70c0ce190 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00d800 tx=0x7ff6fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff703e00640 1 -- 172.21.15.46:0/3185487927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc018020 con 0x7ff70c09ab50 2024-04-25T08:50:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff703e00640 1 -- 172.21.15.46:0/3185487927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc0096c0 con 0x7ff70c09ab50 2024-04-25T08:50:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff703e00640 1 -- 172.21.15.46:0/3185487927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc009ab0 con 0x7ff70c09ab50 2024-04-25T08:50:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 -- 172.21.15.46:0/3185487927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c157cf0 con 0x7ff70c09ab50 2024-04-25T08:50:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.363+0000 7ff712400640 1 -- 172.21.15.46:0/3185487927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c153910 con 0x7ff70c09ab50 2024-04-25T08:50:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff703e00640 1 -- 172.21.15.46:0/3185487927 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6fc01b020 con 0x7ff70c09ab50 2024-04-25T08:50:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff703e00640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6e8066f60 0x7ff6e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff703e00640 1 -- 172.21.15.46:0/3185487927 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6fc0857c0 con 0x7ff70c09ab50 2024-04-25T08:50:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff715823000 1 Processor -- start 2024-04-25T08:50:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6e8066f60 0x7ff6e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff715823000 1 -- start start 2024-04-25T08:50:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6e8066f60 0x7ff6e8069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40ae000 tx=0x7ff6f40b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff715823000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad22dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff715823000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad1e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff715823000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad1e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff715823000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ad0cb5f0 con 0x5568ad22e240 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff715823000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ad032da0 con 0x5568ad1e3f30 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff715823000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568acfbb150 con 0x5568ad22d920 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad1e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad1e39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40210/0 (socket says 172.21.15.46:40210) 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 -- 172.21.15.46:0/2318163252 learned_addr learned my addr 172.21.15.46:0/2318163252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff70b400640 1 --2- 172.21.15.46:0/2318163252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad22dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff70be00640 1 --2- 172.21.15.46:0/2318163252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad1e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 -- 172.21.15.46:0/2318163252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 msgr2=0x5568ad22dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 --2- 172.21.15.46:0/2318163252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad22dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 -- 172.21.15.46:0/2318163252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 msgr2=0x5568ad1e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 --2- 172.21.15.46:0/2318163252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad1e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.367+0000 7ff702a00640 1 -- 172.21.15.46:0/2318163252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568acfd4ca0 con 0x5568ad22e240 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 --2- 172.21.15.46:0/2318163252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad1e39f0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009630 tx=0x7ff6f4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb4769b9adeeae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff6fbe00640 1 -- 172.21.15.46:0/2318163252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40cc070 con 0x5568ad22e240 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff6fbe00640 1 -- 172.21.15.46:0/2318163252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f40b1c50 con 0x5568ad22e240 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff6fbe00640 1 -- 172.21.15.46:0/2318163252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40bc8e0 con 0x5568ad22e240 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 -- 172.21.15.46:0/2318163252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 msgr2=0x5568ad1e39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 --2- 172.21.15.46:0/2318163252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad1e39f0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009630 tx=0x7ff6f4004740 comp rx=0 tx=0).stop 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 -- 172.21.15.46:0/2318163252 shutdown_connections 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 --2- 172.21.15.46:0/2318163252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad1e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 --2- 172.21.15.46:0/2318163252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad1e39f0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 --2- 172.21.15.46:0/2318163252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad22dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 -- 172.21.15.46:0/2318163252 >> 172.21.15.46:0/2318163252 conn(0x5568ad1ea2a0 msgr2=0x5568ad1e22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:54.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 -- 172.21.15.46:0/2318163252 shutdown_connections 2024-04-25T08:50:54.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 -- 172.21.15.46:0/2318163252 wait complete. 2024-04-25T08:50:54.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 -- start start 2024-04-25T08:50:54.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 Processor -- start 2024-04-25T08:50:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad234e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22d920 0x5568ad2353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22e240 0x5568ad1e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568acfed990 con 0x5568ad22d920 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ad02ee40 con 0x5568ad1e3f30 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568acfbe900 con 0x5568ad22e240 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff715823000 10 client.0 _ll_register_callbacks cb 0x5568ad0e61c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22d920 0x5568ad2353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22d920 0x5568ad2353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40224/0 (socket says 172.21.15.46:40224) 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 -- 172.21.15.46:0/3519529740 learned_addr learned my addr 172.21.15.46:0/3519529740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff70b400640 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad234e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff70be00640 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22e240 0x5568ad1e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 -- 172.21.15.46:0/3519529740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22e240 msgr2=0x5568ad1e7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22e240 0x5568ad1e7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 -- 172.21.15.46:0/3519529740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 msgr2=0x5568ad234e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad234e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 -- 172.21.15.46:0/3519529740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568acfeb630 con 0x5568ad22d920 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff70be00640 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22e240 0x5568ad1e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff702a00640 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22d920 0x5568ad2353a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009630 tx=0x7ff6f401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2da73ca1ca1162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.371+0000 7ff6f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3519529740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40d5060 con 0x5568ad22d920 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3519529740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6f40d0470 con 0x5568ad22d920 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 -- 172.21.15.46:0/3519529740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6f40b1c50 con 0x5568ad22d920 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3519529740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6f40db050 con 0x5568ad22d920 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 10 client.8460 fetch_fsmap learned FSMap version 83 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 -- 172.21.15.46:0/3519529740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5568acfbb3a0 con 0x5568ad22d920 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3519529740 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f40b1c50 con 0x5568ad22d920 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3519529740 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff6f40de020 con 0x5568ad22d920 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 10 client.8460 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 10 client.8460 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 20 client.8460 populate_metadata read hostname 'smithi046' 2024-04-25T08:50:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x5568ad1e9160 0x5568ad23eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 4 client.8460 mds_command: new command op to 8334 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 -- 172.21.15.46:0/3519529740 --> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568acfcea50 con 0x5568ad1e9160 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff70b400640 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x5568ad1e9160 0x5568ad23eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff6f8c00640 10 client.8460 ms_handle_connect on v2:172.21.15.46:6834/623970467 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff70b400640 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x5568ad1e9160 0x5568ad23eec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7298e60addebddf9 server_cookie=9cab960e176537f1 in_seq=0 out_seq=0 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3519529740 <== mds.0 v2:172.21.15.46:6834/623970467 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5568acfcea50 con 0x5568ad1e9160 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff6f8c00640 10 client.8460 handle_command_reply: tid=0 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 client.8460 shutdown 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 -- 172.21.15.46:0/3519529740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22d920 msgr2=0x5568ad2353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22d920 0x5568ad2353a0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009630 tx=0x7ff6f401ad70 comp rx=0 tx=0).stop 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 -- 172.21.15.46:0/3519529740 shutdown_connections 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22e240 0x5568ad1e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x5568ad1e9160 0x5568ad23eec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22d920 0x5568ad2353a0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 --2- 172.21.15.46:0/3519529740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad234e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff6f8c00640 0 client.8460 ms_handle_reset on v2:172.21.15.46:6834/623970467 2024-04-25T08:50:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 -- 172.21.15.46:0/3519529740 >> 172.21.15.46:0/3519529740 conn(0x5568ad1ea2a0 msgr2=0x5568ad1e29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 -- 172.21.15.46:0/3519529740 shutdown_connections 2024-04-25T08:50:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 1 -- 172.21.15.46:0/3519529740 wait complete. 2024-04-25T08:50:54.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.375+0000 7ff715823000 20 client.8460 trim_cache size 0 max 16384 2024-04-25T08:50:54.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff715823000 1 Processor -- start 2024-04-25T08:50:54.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff715823000 1 -- start start 2024-04-25T08:50:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff715823000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad28d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff715823000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad28dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff715823000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad28e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff715823000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c1fffe0 con 0x5568ad22e240 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff715823000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c151e00 con 0x5568ad1e3f30 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff715823000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c152400 con 0x5568ad22d920 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff70be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad28e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff70be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad28e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40228/0 (socket says 172.21.15.46:40228) 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff70be00640 1 -- 172.21.15.46:0/1934637007 learned_addr learned my addr 172.21.15.46:0/1934637007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff70b400640 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad28d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.391+0000 7ff702a00640 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad28dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 -- 172.21.15.46:0/1934637007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 msgr2=0x5568ad28dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad28dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 -- 172.21.15.46:0/1934637007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 msgr2=0x5568ad28e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad28e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 -- 172.21.15.46:0/1934637007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568ad0b4600 con 0x5568ad1e3f30 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70be00640 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad28e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:54.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad28d710 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc083ac0 tx=0x7ff6fc01f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74aa03f0b92b3694 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff6fb400640 1 -- 172.21.15.46:0/1934637007 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc069e60 con 0x5568ad1e3f30 2024-04-25T08:50:54.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff6fb400640 1 -- 172.21.15.46:0/1934637007 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc053f40 con 0x5568ad1e3f30 2024-04-25T08:50:54.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff6fb400640 1 -- 172.21.15.46:0/1934637007 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0541a0 con 0x5568ad1e3f30 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 -- 172.21.15.46:0/1934637007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 msgr2=0x5568ad28d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad28d710 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc083ac0 tx=0x7ff6fc01f040 comp rx=0 tx=0).stop 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 -- 172.21.15.46:0/1934637007 shutdown_connections 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad22e240 0x5568ad28e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad28dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 --2- 172.21.15.46:0/1934637007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad1e3f30 0x5568ad28d710 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 -- 172.21.15.46:0/1934637007 >> 172.21.15.46:0/1934637007 conn(0x5568ad1ea2a0 msgr2=0x5568ad1e7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 -- 172.21.15.46:0/1934637007 shutdown_connections 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 -- 172.21.15.46:0/1934637007 wait complete. 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 -- start start 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 Processor -- start 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad1e3f30 0x5568ad292a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad292f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad22e240 0x5568ad2765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ad0373d0 con 0x5568ad1e3f30 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568ad0eed50 con 0x5568ad22e240 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568acfcd980 con 0x5568ad22d920 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff715823000 10 client.0 _ll_register_callbacks cb 0x7ff6f40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad1e3f30 0x5568ad292a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad22e240 0x5568ad2765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad1e3f30 0x5568ad292a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40234/0 (socket says 172.21.15.46:40234) 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 -- 172.21.15.46:0/3159029669 learned_addr learned my addr 172.21.15.46:0/3159029669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff702a00640 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad292f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 -- 172.21.15.46:0/3159029669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 msgr2=0x5568ad292f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad292f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 -- 172.21.15.46:0/3159029669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad22e240 msgr2=0x5568ad2765b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad22e240 0x5568ad2765b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70b400640 1 -- 172.21.15.46:0/3159029669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568acfc79b0 con 0x5568ad1e3f30 2024-04-25T08:50:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff70be00640 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad22e240 0x5568ad2765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.395+0000 7ff702a00640 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad292f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff70b400640 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad1e3f30 0x5568ad292a20 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc052300 tx=0x7ff6fc05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd7f706b7d65f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:50:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3159029669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc069e60 con 0x5568ad1e3f30 2024-04-25T08:50:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3159029669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6fc052510 con 0x5568ad1e3f30 2024-04-25T08:50:54.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 -- 172.21.15.46:0/3159029669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff6fc053f40 con 0x5568ad1e3f30 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3159029669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff6fc07d120 con 0x5568ad1e3f30 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 10 client.8466 fetch_fsmap learned FSMap version 83 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 -- 172.21.15.46:0/3159029669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5568acfda0b0 con 0x5568ad1e3f30 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3159029669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6fc053f40 con 0x5568ad1e3f30 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3159029669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff6fc013050 con 0x5568ad1e3f30 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 10 client.8466 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 10 client.8466 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 20 client.8466 populate_metadata read hostname 'smithi046' 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x5568ad276af0 0x5568ad276ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 4 client.8466 mds_command: new command op to 8334 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 -- 172.21.15.46:0/3159029669 --> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5568ad0402e0 con 0x5568ad276af0 2024-04-25T08:50:54.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff702a00640 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x5568ad276af0 0x5568ad276ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff702a00640 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x5568ad276af0 0x5568ad276ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f5a5faf4f4377e server_cookie=1385ceb09d726632 in_seq=0 out_seq=0 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 10 client.8466 ms_handle_connect on v2:172.21.15.46:6834/623970467 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 1 -- 172.21.15.46:0/3159029669 <== mds.0 v2:172.21.15.46:6834/623970467 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5568ad0402e0 con 0x5568ad276af0 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 10 client.8466 handle_command_reply: tid=0 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 client.8466 shutdown 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 -- 172.21.15.46:0/3159029669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad1e3f30 msgr2=0x5568ad292a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad1e3f30 0x5568ad292a20 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc052300 tx=0x7ff6fc05c040 comp rx=0 tx=0).stop 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 -- 172.21.15.46:0/3159029669 shutdown_connections 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568ad22e240 0x5568ad2765b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5568ad22d920 0x5568ad292f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x5568ad276af0 0x5568ad276ed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 --2- 172.21.15.46:0/3159029669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5568ad1e3f30 0x5568ad292a20 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 -- 172.21.15.46:0/3159029669 >> 172.21.15.46:0/3159029669 conn(0x5568ad1ea2a0 msgr2=0x5568ad1e6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff6f8c00640 0 client.8466 ms_handle_reset on v2:172.21.15.46:6834/623970467 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 -- 172.21.15.46:0/3159029669 shutdown_connections 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 1 -- 172.21.15.46:0/3159029669 wait complete. 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff715823000 20 client.8466 trim_cache size 0 max 16384 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8334, 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:50:54.395 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:50:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:50:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.9251295949999996, 2024-04-25T08:50:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 83, 2024-04-25T08:50:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 106, 2024-04-25T08:50:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 106, 2024-04-25T08:50:54.396 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 11.758329836 2024-04-25T08:50:54.396 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 -- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6e8066f60 msgr2=0x7ff6e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6e8066f60 0x7ff6e8069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff6f40ae000 tx=0x7ff6f40b3040 comp rx=0 tx=0).stop 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 -- 172.21.15.46:0/3185487927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c09ab50 msgr2=0x7ff70c0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c09ab50 0x7ff70c0ce190 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00d800 tx=0x7ff6fc00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 -- 172.21.15.46:0/3185487927 shutdown_connections 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff6e8066f60 0x7ff6e8069420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff70c1530d0 0x7ff70c153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff70c15ef20 0x7ff70c152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 --2- 172.21.15.46:0/3185487927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff70c09ab50 0x7ff70c0ce190 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 -- 172.21.15.46:0/3185487927 >> 172.21.15.46:0/3185487927 conn(0x7ff70c0b3120 msgr2=0x7ff70c15f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 -- 172.21.15.46:0/3185487927 shutdown_connections 2024-04-25T08:50:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.399+0000 7ff712400640 1 -- 172.21.15.46:0/3185487927 wait complete. 2024-04-25T08:50:54.411 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:50:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.559+0000 7f0a7bc00640 1 Processor -- start 2024-04-25T08:50:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.559+0000 7f0a7bc00640 1 -- start start 2024-04-25T08:50:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a7bc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a74159230 0x7f0a7415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a7bc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a740cbeb0 0x7f0a740cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a7bc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a740cc700 con 0x7f0a740cda80 2024-04-25T08:50:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a7bc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a740cc9e0 con 0x7f0a740cbeb0 2024-04-25T08:50:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a7bc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a740cccc0 con 0x7f0a74159230 2024-04-25T08:50:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a740cbeb0 0x7f0a740cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a740cbeb0 0x7f0a740cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40236/0 (socket says 172.21.15.46:40236) 2024-04-25T08:50:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a73e00640 1 -- 172.21.15.46:0/802484151 learned_addr learned my addr 172.21.15.46:0/802484151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a79c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0a740cda80 0x7f0a7415f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45036/0 (socket says 172.21.15.46:45036) 2024-04-25T08:50:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a79200640 1 --2- 172.21.15.46:0/802484151 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a74159230 0x7f0a7415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a73e00640 1 -- 172.21.15.46:0/802484151 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a74159230 msgr2=0x7f0a7415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a73e00640 1 --2- 172.21.15.46:0/802484151 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a74159230 0x7f0a7415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a73e00640 1 -- 172.21.15.46:0/802484151 >> v1:172.21.15.46:6789/0 conn(0x7f0a740cda80 legacy=0x7f0a7415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a73e00640 1 -- 172.21.15.46:0/802484151 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a740ccfa0 con 0x7f0a740cbeb0 2024-04-25T08:50:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.563+0000 7f0a73400640 1 -- 172.21.15.46:0/802484151 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062575889 0 0) 0x7f0a740cc700 con 0x7f0a740cda80 2024-04-25T08:50:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a73e00640 1 --2- 172.21.15.46:0/802484151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cbeb0 0x7f0a740cd540 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009870 tx=0x7f0a6001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5775713063243d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a73400640 1 -- 172.21.15.46:0/802484151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a600b1070 con 0x7f0a740cbeb0 2024-04-25T08:50:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a73400640 1 -- 172.21.15.46:0/802484151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a600a8050 con 0x7f0a740cbeb0 2024-04-25T08:50:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a73400640 1 -- 172.21.15.46:0/802484151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a600ac860 con 0x7f0a740cbeb0 2024-04-25T08:50:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 -- 172.21.15.46:0/802484151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cbeb0 msgr2=0x7f0a740cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 --2- 172.21.15.46:0/802484151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cbeb0 0x7f0a740cd540 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009870 tx=0x7f0a6001c4d0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 -- 172.21.15.46:0/802484151 shutdown_connections 2024-04-25T08:50:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 --2- 172.21.15.46:0/802484151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cbeb0 0x7f0a740cd540 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 --2- 172.21.15.46:0/802484151 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a74159230 0x7f0a7415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 -- 172.21.15.46:0/802484151 >> 172.21.15.46:0/802484151 conn(0x7f0a740b12a0 msgr2=0x7f0a740b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 -- 172.21.15.46:0/802484151 shutdown_connections 2024-04-25T08:50:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 -- 172.21.15.46:0/802484151 wait complete. 2024-04-25T08:50:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 Processor -- start 2024-04-25T08:50:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.567+0000 7f0a7bc00640 1 -- start start 2024-04-25T08:50:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a7bc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a740cbeb0 0x7f0a7413cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a7bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cda80 0x7f0a74164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a7bc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74159230 0x7f0a74157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a7bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a74168900 con 0x7f0a740cda80 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a7bc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a74168780 con 0x7f0a74159230 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a7bc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a74168a80 con 0x7f0a740cbeb0 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cda80 0x7f0a74164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cda80 0x7f0a74164900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40252/0 (socket says 172.21.15.46:40252) 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a73e00640 1 -- 172.21.15.46:0/3543119337 learned_addr learned my addr 172.21.15.46:0/3543119337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a73e00640 1 -- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a740cbeb0 msgr2=0x7f0a7413cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a73e00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a740cbeb0 0x7f0a7413cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a73e00640 1 -- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74159230 msgr2=0x7f0a74157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a79c00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74159230 0x7f0a74157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a73e00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74159230 0x7f0a74157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a73e00640 1 -- 172.21.15.46:0/3543119337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74165360 con 0x7f0a740cda80 2024-04-25T08:50:54.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a79c00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74159230 0x7f0a74157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a73e00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cda80 0x7f0a74164900 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f0a6001a100 tx=0x7f0a6001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a70c00640 1 -- 172.21.15.46:0/3543119337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a600b1040 con 0x7f0a740cda80 2024-04-25T08:50:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a70c00640 1 -- 172.21.15.46:0/3543119337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a600acdf0 con 0x7f0a740cda80 2024-04-25T08:50:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a70c00640 1 -- 172.21.15.46:0/3543119337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60004220 con 0x7f0a740cda80 2024-04-25T08:50:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a74165640 con 0x7f0a740cda80 2024-04-25T08:50:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a74165c00 con 0x7f0a740cda80 2024-04-25T08:50:54.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a74069080 con 0x7f0a740cda80 2024-04-25T08:50:54.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.571+0000 7f0a70c00640 1 -- 172.21.15.46:0/3543119337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a600c0060 con 0x7f0a740cda80 2024-04-25T08:50:54.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.575+0000 7f0a70c00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a54066f60 0x7f0a54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:54.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.575+0000 7f0a70c00640 1 -- 172.21.15.46:0/3543119337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a6011ea70 con 0x7f0a740cda80 2024-04-25T08:50:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.575+0000 7f0a79200640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a54066f60 0x7f0a54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:54.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.575+0000 7f0a79200640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a54066f60 0x7f0a54069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0a68006aa0 tx=0x7f0a68006a30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:54.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.579+0000 7f0a70c00640 1 -- 172.21.15.46:0/3543119337 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a60016150 con 0x7f0a740cda80 2024-04-25T08:50:54.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.775+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a7415cc50 con 0x7f0a740cda80 2024-04-25T08:50:54.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.775+0000 7f0a70c00640 1 -- 172.21.15.46:0/3543119337 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0a600b4360 con 0x7f0a740cda80 2024-04-25T08:50:54.767 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:50:54.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a54066f60 msgr2=0x7f0a54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a54066f60 0x7f0a54069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0a68006aa0 tx=0x7f0a68006a30 comp rx=0 tx=0).stop 2024-04-25T08:50:54.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cda80 msgr2=0x7f0a74164900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:54.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cda80 0x7f0a74164900 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f0a6001a100 tx=0x7f0a6001ad10 comp rx=0 tx=0).stop 2024-04-25T08:50:54.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 shutdown_connections 2024-04-25T08:50:54.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0a54066f60 0x7f0a54069420 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a74159230 0x7f0a74157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a740cda80 0x7f0a74164900 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 --2- 172.21.15.46:0/3543119337 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0a740cbeb0 0x7f0a7413cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:54.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 >> 172.21.15.46:0/3543119337 conn(0x7f0a740b12a0 msgr2=0x7f0a740b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:54.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 shutdown_connections 2024-04-25T08:50:54.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:54.779+0000 7f0a7bc00640 1 -- 172.21.15.46:0/3543119337 wait complete. 2024-04-25T08:50:54.789 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:50:54.789 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:50:54.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:50:54.789 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:50:54.790 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:54.790 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:50:54.790 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:50:54.790 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:50:54.790 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:50:54.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:54.790 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:50:54.799 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:54.800 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41948sec preferred_lft 41948sec 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:54.801 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:54.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:54.803 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:50:54.861 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:50:54.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:54.864 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:50:54.917 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:50:54.917 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:50:54.917 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:50:54.917 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:50:54.917 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:50:54.917 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:50:54.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:54.918 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:50:54.918 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:50:54.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:50:54.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:50:54.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:50:54.918 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:50:55.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:55.009 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:50:55.019 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:50:55.019 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:50:55.019 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:50:55.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:55.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:50:55.079 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:50:55.079 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:55.131 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:50:55.132 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:50:55.132 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:55.183 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:50:55.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:55.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:50:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:50:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:50:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:50:55.291 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:50:55.291 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:50:55.292 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:50:55.292 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:50:55.292 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:50:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:50:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:50:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:50:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:50:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:50:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:50:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:50:55.301 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:50:55.305 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:50:55.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:55.305 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:55.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:55.325 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:50:55.384 INFO:teuthology.orchestra.run.smithi046.stdout:client7193 2024-04-25T08:50:55.385 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:55.385 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:55.436 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:55.436 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:55.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:55.492 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T08:50:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records in 2024-04-25T08:50:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records out 2024-04-25T08:50:55.654 INFO:teuthology.orchestra.run.smithi046.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.104763 s, 60.1 MB/s 2024-04-25T08:50:55.655 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:55.655 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:55.664 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:55.664 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:55.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:50:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.907+0000 7f6481e00640 1 Processor -- start 2024-04-25T08:50:55.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.907+0000 7f6481e00640 1 -- start start 2024-04-25T08:50:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f6481e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f647c0cf140 0x7f647c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f6481e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c06ed70 con 0x7f647c0cf520 2024-04-25T08:50:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f6481e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f647c0cde00 con 0x7f647c0afd40 2024-04-25T08:50:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f6481e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c0afb40 con 0x7f647c0cf140 2024-04-25T08:50:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f647b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f647c0cf520 0x7f647c0cd6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45038/0 (socket says 172.21.15.46:45038) 2024-04-25T08:50:55.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f647b400640 1 -- 172.21.15.46:0/2512997352 learned_addr learned my addr 172.21.15.46:0/2512997352 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f647aa00640 1 --2- 172.21.15.46:0/2512997352 >> v2:172.21.15.82:3300/0 conn(0x7f647c0cf140 0x7f647c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f647a000640 1 -- 172.21.15.46:0/2512997352 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372134592 0 0) 0x7f64740014a0 con 0x7f647c0cf520 2024-04-25T08:50:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f647a000640 1 -- 172.21.15.46:0/2512997352 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6460003530 con 0x7f647c0cf520 2024-04-25T08:50:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f647aa00640 1 -- 172.21.15.46:0/2512997352 >> v1:172.21.15.82:6789/0 conn(0x7f647c0afd40 legacy=0x7f647c0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f647aa00640 1 -- 172.21.15.46:0/2512997352 >> v1:172.21.15.46:6789/0 conn(0x7f647c0cf520 legacy=0x7f647c0cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:55.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.911+0000 7f647aa00640 1 -- 172.21.15.46:0/2512997352 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c0d13f0 con 0x7f647c0cf140 2024-04-25T08:50:55.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f647aa00640 1 --2- 172.21.15.46:0/2512997352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f647c0cf140 0x7f647c15d590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f64640097c0 tx=0x7f646401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=584994c4ac67311b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:55.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f647a000640 1 -- 172.21.15.46:0/2512997352 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64640ab610 con 0x7f647c0cf140 2024-04-25T08:50:55.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f647a000640 1 -- 172.21.15.46:0/2512997352 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64640b0070 con 0x7f647c0cf140 2024-04-25T08:50:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f647a000640 1 -- 172.21.15.46:0/2512997352 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64640ab9d0 con 0x7f647c0cf140 2024-04-25T08:50:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f6481e00640 1 -- 172.21.15.46:0/2512997352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f647c0cf140 msgr2=0x7f647c15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f6481e00640 1 --2- 172.21.15.46:0/2512997352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f647c0cf140 0x7f647c15d590 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f64640097c0 tx=0x7f646401c410 comp rx=0 tx=0).stop 2024-04-25T08:50:55.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f6481e00640 1 -- 172.21.15.46:0/2512997352 shutdown_connections 2024-04-25T08:50:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f6481e00640 1 --2- 172.21.15.46:0/2512997352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f647c0cf140 0x7f647c15d590 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f6481e00640 1 -- 172.21.15.46:0/2512997352 >> 172.21.15.46:0/2512997352 conn(0x7f647c0728c0 msgr2=0x7f647c099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f6481e00640 1 -- 172.21.15.46:0/2512997352 shutdown_connections 2024-04-25T08:50:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f6481e00640 1 -- 172.21.15.46:0/2512997352 wait complete. 2024-04-25T08:50:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f6481e00640 1 Processor -- start 2024-04-25T08:50:55.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.915+0000 7f6481e00640 1 -- start start 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6481e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f647c0afd40 0x7f647c09a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6481e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0cf140 0x7f647c0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6481e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f647c0cf520 0x7f647c150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6481e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c162010 con 0x7f647c0cf140 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6481e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c161d30 con 0x7f647c0afd40 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6481e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c15dee0 con 0x7f647c0cf520 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6472000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0cf140 0x7f647c0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6472000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0cf140 0x7f647c0b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40256/0 (socket says 172.21.15.46:40256) 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6472000640 1 -- 172.21.15.46:0/1216354849 learned_addr learned my addr 172.21.15.46:0/1216354849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:55.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f647aa00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f647c0afd40 0x7f647c09a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f647b400640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f647c0cf520 0x7f647c150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6472000640 1 -- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f647c0cf520 msgr2=0x7f647c150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6472000640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f647c0cf520 0x7f647c150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6472000640 1 -- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f647c0afd40 msgr2=0x7f647c09a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6472000640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f647c0afd40 0x7f647c09a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6472000640 1 -- 172.21.15.46:0/1216354849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c09ad20 con 0x7f647c0cf140 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f647b400640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f647c0cf520 0x7f647c150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6472000640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0cf140 0x7f647c0b05a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c340 tx=0x7f646c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6473e00640 1 -- 172.21.15.46:0/1216354849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c0052d0 con 0x7f647c0cf140 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6473e00640 1 -- 172.21.15.46:0/1216354849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c005470 con 0x7f647c0cf140 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6473e00640 1 -- 172.21.15.46:0/1216354849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c007260 con 0x7f647c0cf140 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f647c09d230 con 0x7f647c0cf140 2024-04-25T08:50:55.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c09c3b0 con 0x7f647c0cf140 2024-04-25T08:50:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c06ccb0 con 0x7f647c0cf140 2024-04-25T08:50:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.919+0000 7f6473e00640 1 -- 172.21.15.46:0/1216354849 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f646c007400 con 0x7f647c0cf140 2024-04-25T08:50:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.923+0000 7f6473e00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6458066f60 0x7f6458069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.923+0000 7f6473e00640 1 -- 172.21.15.46:0/1216354849 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f646c084700 con 0x7f647c0cf140 2024-04-25T08:50:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.923+0000 7f647aa00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6458066f60 0x7f6458069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:55.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.923+0000 7f647aa00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6458066f60 0x7f6458069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f64640af000 tx=0x7f64640a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:55.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:55.927+0000 7f6473e00640 1 -- 172.21.15.46:0/1216354849 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f646c0554e0 con 0x7f647c0cf140 2024-04-25T08:50:56.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.099+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f647c069080 con 0x7f647c0cf140 2024-04-25T08:50:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.099+0000 7f6473e00640 1 -- 172.21.15.46:0/1216354849 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16836 (secure 0 0 0) 0x7f646c082e20 con 0x7f647c0cf140 2024-04-25T08:50:56.090 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:56.090 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:50:49.457661+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:50:45.647599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:50:46.857198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:50:56.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6458066f60 msgr2=0x7f6458069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6458066f60 0x7f6458069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f64640af000 tx=0x7f64640a7000 comp rx=0 tx=0).stop 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0cf140 msgr2=0x7f647c0b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0cf140 0x7f647c0b05a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f646c00c340 tx=0x7f646c00c800 comp rx=0 tx=0).stop 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 shutdown_connections 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6458066f60 0x7f6458069420 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f647c0cf520 0x7f647c150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f647c0cf140 0x7f647c0b05a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 --2- 172.21.15.46:0/1216354849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f647c0afd40 0x7f647c09a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 >> 172.21.15.46:0/1216354849 conn(0x7f647c0728c0 msgr2=0x7f647c151c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 shutdown_connections 2024-04-25T08:50:56.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.103+0000 7f6481e00640 1 -- 172.21.15.46:0/1216354849 wait complete. 2024-04-25T08:50:56.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:50:45.647599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:50:46.857198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:50:56.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:50:56.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5897e00640 1 Processor -- start 2024-04-25T08:50:56.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5897e00640 1 -- start start 2024-04-25T08:50:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5897e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f589806c730 0x7f589806bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5897e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589809acb0 con 0x7f58980d0390 2024-04-25T08:50:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5897e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589809a9d0 con 0x7f589809a410 2024-04-25T08:50:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5897e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58980b3dc0 con 0x7f589806c730 2024-04-25T08:50:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5896a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f589806c730 0x7f589806bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5897400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f58980d0390 0x7f58980cbab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45054/0 (socket says 172.21.15.46:45054) 2024-04-25T08:50:56.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5897400640 1 -- 172.21.15.46:0/1423029456 learned_addr learned my addr 172.21.15.46:0/1423029456 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5896a00640 1 -- 172.21.15.46:0/1423029456 >> v1:172.21.15.82:6789/0 conn(0x7f589809a410 legacy=0x7f589806cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5896a00640 1 -- 172.21.15.46:0/1423029456 >> v1:172.21.15.46:6789/0 conn(0x7f58980d0390 legacy=0x7f58980cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.267+0000 7f5896a00640 1 -- 172.21.15.46:0/1423029456 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58980cf8b0 con 0x7f589806c730 2024-04-25T08:50:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5896a00640 1 --2- 172.21.15.46:0/1423029456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589806c730 0x7f589806bf30 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f58880097f0 tx=0x7f588801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad43f70b5d130abe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5896000640 1 -- 172.21.15.46:0/1423029456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58880b0070 con 0x7f589806c730 2024-04-25T08:50:56.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5896000640 1 -- 172.21.15.46:0/1423029456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58880ab600 con 0x7f589806c730 2024-04-25T08:50:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5896000640 1 -- 172.21.15.46:0/1423029456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58880ab9d0 con 0x7f589806c730 2024-04-25T08:50:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5897e00640 1 -- 172.21.15.46:0/1423029456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589806c730 msgr2=0x7f589806bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5897e00640 1 --2- 172.21.15.46:0/1423029456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589806c730 0x7f589806bf30 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f58880097f0 tx=0x7f588801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5897e00640 1 -- 172.21.15.46:0/1423029456 shutdown_connections 2024-04-25T08:50:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5897e00640 1 --2- 172.21.15.46:0/1423029456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589806c730 0x7f589806bf30 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5897e00640 1 -- 172.21.15.46:0/1423029456 >> 172.21.15.46:0/1423029456 conn(0x7f58980b2f50 msgr2=0x7f58980b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5897e00640 1 -- 172.21.15.46:0/1423029456 shutdown_connections 2024-04-25T08:50:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5897e00640 1 -- 172.21.15.46:0/1423029456 wait complete. 2024-04-25T08:50:56.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.271+0000 7f5897e00640 1 Processor -- start 2024-04-25T08:50:56.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 -- start start 2024-04-25T08:50:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f589806c730 0x7f58980ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f589809a410 0x7f589814a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58980d0390 0x7f58980b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980cc340 con 0x7f58980d0390 2024-04-25T08:50:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980cc1c0 con 0x7f589809a410 2024-04-25T08:50:56.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589816c460 con 0x7f589806c730 2024-04-25T08:50:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58980d0390 0x7f58980b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58980d0390 0x7f58980b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40278/0 (socket says 172.21.15.46:40278) 2024-04-25T08:50:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897400640 1 -- 172.21.15.46:0/606115053 learned_addr learned my addr 172.21.15.46:0/606115053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5896a00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f589806c730 0x7f58980ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f588fe00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f589809a410 0x7f589814a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897400640 1 -- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f589806c730 msgr2=0x7f58980ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897400640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f589806c730 0x7f58980ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897400640 1 -- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f589809a410 msgr2=0x7f589814a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897400640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f589809a410 0x7f589814a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897400640 1 -- 172.21.15.46:0/606115053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58980cb320 con 0x7f58980d0390 2024-04-25T08:50:56.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897400640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58980d0390 0x7f58980b4080 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f589000ca80 tx=0x7f589000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f588ea00640 1 -- 172.21.15.46:0/606115053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890017070 con 0x7f58980d0390 2024-04-25T08:50:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f588ea00640 1 -- 172.21.15.46:0/606115053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589000abd0 con 0x7f58980d0390 2024-04-25T08:50:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f588ea00640 1 -- 172.21.15.46:0/606115053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890012400 con 0x7f58980d0390 2024-04-25T08:50:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58980c8b90 con 0x7f58980d0390 2024-04-25T08:50:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58980b7020 con 0x7f58980d0390 2024-04-25T08:50:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.275+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5898069010 con 0x7f58980d0390 2024-04-25T08:50:56.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.279+0000 7f588ea00640 1 -- 172.21.15.46:0/606115053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f589001a070 con 0x7f58980d0390 2024-04-25T08:50:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.279+0000 7f588ea00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5874066f90 0x7f5874069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.279+0000 7f588ea00640 1 -- 172.21.15.46:0/606115053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5890084e40 con 0x7f58980d0390 2024-04-25T08:50:56.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.279+0000 7f5896a00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5874066f90 0x7f5874069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.279+0000 7f5896a00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5874066f90 0x7f5874069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f58880ad000 tx=0x7f58880a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.283+0000 7f588ea00640 1 -- 172.21.15.46:0/606115053 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5890055540 con 0x7f58980d0390 2024-04-25T08:50:56.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.443+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f589806c510 con 0x7f58980d0390 2024-04-25T08:50:56.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.443+0000 7f588ea00640 1 -- 172.21.15.46:0/606115053 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58900595e0 con 0x7f58980d0390 2024-04-25T08:50:56.435 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:50:56.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5874066f90 msgr2=0x7f5874069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5874066f90 0x7f5874069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f58880ad000 tx=0x7f58880a6000 comp rx=0 tx=0).stop 2024-04-25T08:50:56.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58980d0390 msgr2=0x7f58980b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58980d0390 0x7f58980b4080 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f589000ca80 tx=0x7f589000cf40 comp rx=0 tx=0).stop 2024-04-25T08:50:56.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 shutdown_connections 2024-04-25T08:50:56.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5874066f90 0x7f5874069450 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58980d0390 0x7f58980b4080 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f589809a410 0x7f589814a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 --2- 172.21.15.46:0/606115053 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f589806c730 0x7f58980ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 >> 172.21.15.46:0/606115053 conn(0x7f58980b2f50 msgr2=0x7f5898070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 shutdown_connections 2024-04-25T08:50:56.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.447+0000 7f5897e00640 1 -- 172.21.15.46:0/606115053 wait complete. 2024-04-25T08:50:56.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:50:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.611+0000 7fcd3c400640 1 Processor -- start 2024-04-25T08:50:56.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.611+0000 7fcd3c400640 1 -- start start 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd3c400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd34145310 0x7fcd3406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd3c400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd3406dc70 0x7fcd340cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd3c400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd340cd930 con 0x7fcd340cf1b0 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd3c400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd340cdbd0 con 0x7fcd34145310 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd3c400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd340cdeb0 con 0x7fcd3406dc70 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd3a400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcd340cf1b0 0x7fcd3415d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45066/0 (socket says 172.21.15.46:45066) 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd3a400640 1 -- 172.21.15.46:0/2014358141 learned_addr learned my addr 172.21.15.46:0/2014358141 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd39000640 1 --2- 172.21.15.46:0/2014358141 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd3406dc70 0x7fcd340cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd39a00640 1 --2- 172.21.15.46:0/2014358141 >> v2:172.21.15.82:3300/0 conn(0x7fcd34145310 0x7fcd3406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd33e00640 1 -- 172.21.15.46:0/2014358141 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304436302 0 0) 0x7fcd340cd930 con 0x7fcd340cf1b0 2024-04-25T08:50:56.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd33e00640 1 -- 172.21.15.46:0/2014358141 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd1c003530 con 0x7fcd340cf1b0 2024-04-25T08:50:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd33e00640 1 -- 172.21.15.46:0/2014358141 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1004992444 0 0) 0x7fcd1c003530 con 0x7fcd340cf1b0 2024-04-25T08:50:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd39000640 1 -- 172.21.15.46:0/2014358141 >> v2:172.21.15.82:3300/0 conn(0x7fcd34145310 msgr2=0x7fcd3406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd39000640 1 --2- 172.21.15.46:0/2014358141 >> v2:172.21.15.82:3300/0 conn(0x7fcd34145310 0x7fcd3406e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd39000640 1 -- 172.21.15.46:0/2014358141 >> v1:172.21.15.46:6789/0 conn(0x7fcd340cf1b0 legacy=0x7fcd3415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd39000640 1 -- 172.21.15.46:0/2014358141 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd340ce190 con 0x7fcd3406dc70 2024-04-25T08:50:56.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.615+0000 7fcd39a00640 1 --2- 172.21.15.46:0/2014358141 >> v2:172.21.15.82:3300/0 conn(0x7fcd34145310 0x7fcd3406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd39000640 1 --2- 172.21.15.46:0/2014358141 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd3406dc70 0x7fcd340cec70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcd200097c0 tx=0x7fcd2001c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f76aaabb1a36996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd33e00640 1 -- 172.21.15.46:0/2014358141 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200b0070 con 0x7fcd3406dc70 2024-04-25T08:50:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd33e00640 1 -- 172.21.15.46:0/2014358141 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd200ab400 con 0x7fcd3406dc70 2024-04-25T08:50:56.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd33e00640 1 -- 172.21.15.46:0/2014358141 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c001a00 con 0x7fcd3406dc70 2024-04-25T08:50:56.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 -- 172.21.15.46:0/2014358141 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd3406dc70 msgr2=0x7fcd340cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 --2- 172.21.15.46:0/2014358141 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd3406dc70 0x7fcd340cec70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcd200097c0 tx=0x7fcd2001c390 comp rx=0 tx=0).stop 2024-04-25T08:50:56.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 -- 172.21.15.46:0/2014358141 shutdown_connections 2024-04-25T08:50:56.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 --2- 172.21.15.46:0/2014358141 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd3406dc70 0x7fcd340cec70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 --2- 172.21.15.46:0/2014358141 >> v2:172.21.15.82:3300/0 conn(0x7fcd34145310 0x7fcd3406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 -- 172.21.15.46:0/2014358141 >> 172.21.15.46:0/2014358141 conn(0x7fcd3406ac10 msgr2=0x7fcd3406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 -- 172.21.15.46:0/2014358141 shutdown_connections 2024-04-25T08:50:56.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 -- 172.21.15.46:0/2014358141 wait complete. 2024-04-25T08:50:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 Processor -- start 2024-04-25T08:50:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 -- start start 2024-04-25T08:50:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd340cf1b0 0x7fcd340cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34145310 0x7fcd340cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd340cbc50 0x7fcd340b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3415d920 con 0x7fcd340cbc50 2024-04-25T08:50:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3415d7a0 con 0x7fcd34145310 2024-04-25T08:50:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.619+0000 7fcd3c400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd34168510 con 0x7fcd340cf1b0 2024-04-25T08:50:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd340cbc50 0x7fcd340b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd340cbc50 0x7fcd340b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40282/0 (socket says 172.21.15.46:40282) 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3a400640 1 -- 172.21.15.46:0/3463593147 learned_addr learned my addr 172.21.15.46:0/3463593147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd39a00640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd340cf1b0 0x7fcd340cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd39000640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34145310 0x7fcd340cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3a400640 1 -- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd340cf1b0 msgr2=0x7fcd340cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3a400640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd340cf1b0 0x7fcd340cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3a400640 1 -- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34145310 msgr2=0x7fcd340cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3a400640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34145310 0x7fcd340cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3a400640 1 -- 172.21.15.46:0/3463593147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd340ad6e0 con 0x7fcd340cbc50 2024-04-25T08:50:56.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd39000640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34145310 0x7fcd340cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3a400640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd340cbc50 0x7fcd340b37c0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c001570 tx=0x7fcd2c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd31600640 1 -- 172.21.15.46:0/3463593147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c012400 con 0x7fcd340cbc50 2024-04-25T08:50:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd31600640 1 -- 172.21.15.46:0/3463593147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c0125c0 con 0x7fcd340cbc50 2024-04-25T08:50:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd340ad9c0 con 0x7fcd340cbc50 2024-04-25T08:50:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd31600640 1 -- 172.21.15.46:0/3463593147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0129b0 con 0x7fcd340cbc50 2024-04-25T08:50:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd340adf80 con 0x7fcd340cbc50 2024-04-25T08:50:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd34069010 con 0x7fcd340cbc50 2024-04-25T08:50:56.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.623+0000 7fcd31600640 1 -- 172.21.15.46:0/3463593147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd2c012b50 con 0x7fcd340cbc50 2024-04-25T08:50:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.627+0000 7fcd31600640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd14066f90 0x7fcd14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.627+0000 7fcd31600640 1 -- 172.21.15.46:0/3463593147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd2c0859f0 con 0x7fcd340cbc50 2024-04-25T08:50:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.627+0000 7fcd39a00640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd14066f90 0x7fcd14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.627+0000 7fcd39a00640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd14066f90 0x7fcd14069450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009a50 tx=0x7fcd28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.631+0000 7fcd31600640 1 -- 172.21.15.46:0/3463593147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd2c056860 con 0x7fcd340cbc50 2024-04-25T08:50:56.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.791+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd340b2410 con 0x7fcd340cbc50 2024-04-25T08:50:56.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.791+0000 7fcd31600640 1 -- 172.21.15.46:0/3463593147 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd2c056160 con 0x7fcd340cbc50 2024-04-25T08:50:56.783 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:50:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd14066f90 msgr2=0x7fcd14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd14066f90 0x7fcd14069450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009a50 tx=0x7fcd28008040 comp rx=0 tx=0).stop 2024-04-25T08:50:56.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd340cbc50 msgr2=0x7fcd340b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd340cbc50 0x7fcd340b37c0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c001570 tx=0x7fcd2c00a430 comp rx=0 tx=0).stop 2024-04-25T08:50:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 shutdown_connections 2024-04-25T08:50:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd14066f90 0x7fcd14069450 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd340cbc50 0x7fcd340b37c0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34145310 0x7fcd340cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 --2- 172.21.15.46:0/3463593147 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd340cf1b0 0x7fcd340cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 >> 172.21.15.46:0/3463593147 conn(0x7fcd3406ac10 msgr2=0x7fcd340b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 shutdown_connections 2024-04-25T08:50:56.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.795+0000 7fcd3c400640 1 -- 172.21.15.46:0/3463593147 wait complete. 2024-04-25T08:50:56.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:50:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0eaea00640 1 Processor -- start 2024-04-25T08:50:56.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0eaea00640 1 -- start start 2024-04-25T08:50:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0eaea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0ea806caf0 0x7f0ea80d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0eaea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea80cfa40 con 0x7f0ea8160f70 2024-04-25T08:50:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0eaea00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea80cfce0 con 0x7f0ea806bd20 2024-04-25T08:50:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0eaea00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea80cffc0 con 0x7f0ea806caf0 2024-04-25T08:50:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0ea7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0ea8160f70 0x7f0ea806c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45074/0 (socket says 172.21.15.46:45074) 2024-04-25T08:50:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0ea7e00640 1 -- 172.21.15.46:0/1946748918 learned_addr learned my addr 172.21.15.46:0/1946748918 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0e9f400640 1 --2- 172.21.15.46:0/1946748918 >> v2:172.21.15.82:3300/0 conn(0x7f0ea806caf0 0x7f0ea80d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0ea7400640 1 -- 172.21.15.46:0/1946748918 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574652313 0 0) 0x7f0ea80cfa40 con 0x7f0ea8160f70 2024-04-25T08:50:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0ea7400640 1 -- 172.21.15.46:0/1946748918 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e8c003530 con 0x7f0ea8160f70 2024-04-25T08:50:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0e9f400640 1 -- 172.21.15.46:0/1946748918 >> v1:172.21.15.82:6789/0 conn(0x7f0ea806bd20 legacy=0x7f0ea80cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0e9f400640 1 -- 172.21.15.46:0/1946748918 >> v1:172.21.15.46:6789/0 conn(0x7f0ea8160f70 legacy=0x7f0ea806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.959+0000 7f0e9f400640 1 -- 172.21.15.46:0/1946748918 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80d02a0 con 0x7f0ea806caf0 2024-04-25T08:50:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0e9f400640 1 --2- 172.21.15.46:0/1946748918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea806caf0 0x7f0ea80d0d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0e900097c0 tx=0x7f0e9001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d2abf67ce83429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0ea7400640 1 -- 172.21.15.46:0/1946748918 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e900b5070 con 0x7f0ea806caf0 2024-04-25T08:50:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0ea7400640 1 -- 172.21.15.46:0/1946748918 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e900a8990 con 0x7f0ea806caf0 2024-04-25T08:50:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0ea7400640 1 -- 172.21.15.46:0/1946748918 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e900a8d60 con 0x7f0ea806caf0 2024-04-25T08:50:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0eaea00640 1 -- 172.21.15.46:0/1946748918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea806caf0 msgr2=0x7f0ea80d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0eaea00640 1 --2- 172.21.15.46:0/1946748918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea806caf0 0x7f0ea80d0d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0e900097c0 tx=0x7f0e9001ec90 comp rx=0 tx=0).stop 2024-04-25T08:50:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0eaea00640 1 -- 172.21.15.46:0/1946748918 shutdown_connections 2024-04-25T08:50:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0eaea00640 1 --2- 172.21.15.46:0/1946748918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea806caf0 0x7f0ea80d0d80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0eaea00640 1 -- 172.21.15.46:0/1946748918 >> 172.21.15.46:0/1946748918 conn(0x7f0ea80b3130 msgr2=0x7f0ea80b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0eaea00640 1 -- 172.21.15.46:0/1946748918 shutdown_connections 2024-04-25T08:50:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.963+0000 7f0eaea00640 1 -- 172.21.15.46:0/1946748918 wait complete. 2024-04-25T08:50:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 Processor -- start 2024-04-25T08:50:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 -- start start 2024-04-25T08:50:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea806bd20 0x7f0ea80b6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ea806caf0 0x7f0ea8099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea8160f70 0x7f0ea8099f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea8161350 con 0x7f0ea806bd20 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80d0860 con 0x7f0ea8160f70 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81614d0 con 0x7f0ea806caf0 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea806bd20 0x7f0ea80b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea806bd20 0x7f0ea80b6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40296/0 (socket says 172.21.15.46:40296) 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea7e00640 1 -- 172.21.15.46:0/2399920232 learned_addr learned my addr 172.21.15.46:0/2399920232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0e9f400640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ea806caf0 0x7f0ea8099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaca00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea8160f70 0x7f0ea8099f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea7e00640 1 -- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ea806caf0 msgr2=0x7f0ea8099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea7e00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ea806caf0 0x7f0ea8099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea7e00640 1 -- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea8160f70 msgr2=0x7f0ea8099f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:56.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea7e00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea8160f70 0x7f0ea8099f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea7e00640 1 -- 172.21.15.46:0/2399920232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80b4560 con 0x7f0ea806bd20 2024-04-25T08:50:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaca00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea8160f70 0x7f0ea8099f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea7e00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea806bd20 0x7f0ea80b6780 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800c420 tx=0x7f0e9800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea4c00640 1 -- 172.21.15.46:0/2399920232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98017070 con 0x7f0ea806bd20 2024-04-25T08:50:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea4c00640 1 -- 172.21.15.46:0/2399920232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e98005cf0 con 0x7f0ea806bd20 2024-04-25T08:50:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea80b4840 con 0x7f0ea806bd20 2024-04-25T08:50:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0ea4c00640 1 -- 172.21.15.46:0/2399920232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e980072d0 con 0x7f0ea806bd20 2024-04-25T08:50:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.967+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea80b4e00 con 0x7f0ea806bd20 2024-04-25T08:50:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.971+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea8069010 con 0x7f0ea806bd20 2024-04-25T08:50:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.971+0000 7f0ea4c00640 1 -- 172.21.15.46:0/2399920232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e98007470 con 0x7f0ea806bd20 2024-04-25T08:50:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.971+0000 7f0ea4c00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e84066f60 0x7f0e84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.971+0000 7f0ea4c00640 1 -- 172.21.15.46:0/2399920232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e98084490 con 0x7f0ea806bd20 2024-04-25T08:50:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.971+0000 7f0e9f400640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e84066f60 0x7f0e84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.971+0000 7f0e9f400640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e84066f60 0x7f0e84069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e900b2000 tx=0x7f0e90008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:56.975+0000 7f0ea4c00640 1 -- 172.21.15.46:0/2399920232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e98054d00 con 0x7f0ea806bd20 2024-04-25T08:50:57.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ea809ad50 con 0x7f0ea806bd20 2024-04-25T08:50:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0ea4c00640 1 -- 172.21.15.46:0/2399920232 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e98058da0 con 0x7f0ea806bd20 2024-04-25T08:50:57.129 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:50:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e84066f60 msgr2=0x7f0e84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e84066f60 0x7f0e84069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0e900b2000 tx=0x7f0e90008040 comp rx=0 tx=0).stop 2024-04-25T08:50:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea806bd20 msgr2=0x7f0ea80b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea806bd20 0x7f0ea80b6780 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800c420 tx=0x7f0e9800c8e0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 shutdown_connections 2024-04-25T08:50:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0e84066f60 0x7f0e84069420 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ea8160f70 0x7f0ea8099f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0ea806caf0 0x7f0ea8099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 --2- 172.21.15.46:0/2399920232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea806bd20 0x7f0ea80b6780 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 >> 172.21.15.46:0/2399920232 conn(0x7f0ea80b3130 msgr2=0x7f0ea80b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 shutdown_connections 2024-04-25T08:50:57.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.139+0000 7f0eaea00640 1 -- 172.21.15.46:0/2399920232 wait complete. 2024-04-25T08:50:57.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:50:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.299+0000 7fe19e600640 1 Processor -- start 2024-04-25T08:50:57.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.299+0000 7fe19e600640 1 -- start start 2024-04-25T08:50:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe19e600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe198099a00 0x7fe198099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe19e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 0x7fe198161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe19e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980cf5b0 con 0x7fe19806bd20 2024-04-25T08:50:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe19e600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19806d060 con 0x7fe198099a00 2024-04-25T08:50:57.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe19e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19809a320 con 0x7fe19806c190 2024-04-25T08:50:57.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe18fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe19806bd20 0x7fe1981616b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45082/0 (socket says 172.21.15.46:45082) 2024-04-25T08:50:57.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe18fe00640 1 -- 172.21.15.46:0/2990957368 learned_addr learned my addr 172.21.15.46:0/2990957368 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe197e00640 1 --2- 172.21.15.46:0/2990957368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 0x7fe198161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe197400640 1 --2- 172.21.15.46:0/2990957368 >> v2:172.21.15.82:3300/0 conn(0x7fe198099a00 0x7fe198099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe196a00640 1 -- 172.21.15.46:0/2990957368 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489041056 0 0) 0x7fe1980cf5b0 con 0x7fe19806bd20 2024-04-25T08:50:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe196a00640 1 -- 172.21.15.46:0/2990957368 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c003530 con 0x7fe19806bd20 2024-04-25T08:50:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe197e00640 1 -- 172.21.15.46:0/2990957368 >> v2:172.21.15.82:3300/0 conn(0x7fe198099a00 msgr2=0x7fe198099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe197e00640 1 --2- 172.21.15.46:0/2990957368 >> v2:172.21.15.82:3300/0 conn(0x7fe198099a00 0x7fe198099de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe197e00640 1 -- 172.21.15.46:0/2990957368 >> v1:172.21.15.46:6789/0 conn(0x7fe19806bd20 legacy=0x7fe1981616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe197e00640 1 -- 172.21.15.46:0/2990957368 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19806fa90 con 0x7fe19806c190 2024-04-25T08:50:57.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.303+0000 7fe197400640 1 --2- 172.21.15.46:0/2990957368 >> v2:172.21.15.82:3300/0 conn(0x7fe198099a00 0x7fe198099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:50:57.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe197e00640 1 --2- 172.21.15.46:0/2990957368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 0x7fe198161dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe190009d90 tx=0x7fe19001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700bfc5b859b8020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:57.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe196a00640 1 -- 172.21.15.46:0/2990957368 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1900b2070 con 0x7fe19806c190 2024-04-25T08:50:57.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe196a00640 1 -- 172.21.15.46:0/2990957368 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1900a5a00 con 0x7fe19806c190 2024-04-25T08:50:57.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe196a00640 1 -- 172.21.15.46:0/2990957368 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1900a5dd0 con 0x7fe19806c190 2024-04-25T08:50:57.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 -- 172.21.15.46:0/2990957368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 msgr2=0x7fe198161dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 --2- 172.21.15.46:0/2990957368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 0x7fe198161dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe190009d90 tx=0x7fe19001c940 comp rx=0 tx=0).stop 2024-04-25T08:50:57.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 -- 172.21.15.46:0/2990957368 shutdown_connections 2024-04-25T08:50:57.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 --2- 172.21.15.46:0/2990957368 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 0x7fe198161dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 --2- 172.21.15.46:0/2990957368 >> v2:172.21.15.82:3300/0 conn(0x7fe198099a00 0x7fe198099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 -- 172.21.15.46:0/2990957368 >> 172.21.15.46:0/2990957368 conn(0x7fe1980d16b0 msgr2=0x7fe1980d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:57.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 -- 172.21.15.46:0/2990957368 shutdown_connections 2024-04-25T08:50:57.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 -- 172.21.15.46:0/2990957368 wait complete. 2024-04-25T08:50:57.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 Processor -- start 2024-04-25T08:50:57.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.307+0000 7fe19e600640 1 -- start start 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe19e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe19806bd20 0x7fe198157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe19e600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 0x7fe1980b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe19e600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe198099a00 0x7fe1980b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe19e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19809a750 con 0x7fe19806bd20 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe19e600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19806fd50 con 0x7fe198099a00 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe19e600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe198071280 con 0x7fe19806c190 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe19806bd20 0x7fe198157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe19806bd20 0x7fe198157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40310/0 (socket says 172.21.15.46:40310) 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197400640 1 -- 172.21.15.46:0/398608685 learned_addr learned my addr 172.21.15.46:0/398608685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe18fe00640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 0x7fe1980b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197e00640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe198099a00 0x7fe1980b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197400640 1 -- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 msgr2=0x7fe1980b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197400640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 0x7fe1980b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197400640 1 -- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe198099a00 msgr2=0x7fe1980b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197400640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe198099a00 0x7fe1980b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197400640 1 -- 172.21.15.46:0/398608685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1980b7460 con 0x7fe19806bd20 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197e00640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe198099a00 0x7fe1980b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe197400640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe19806bd20 0x7fe198157df0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fe18800ebe0 tx=0x7fe18800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe18f400640 1 -- 172.21.15.46:0/398608685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188017070 con 0x7fe19806bd20 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe18f400640 1 -- 172.21.15.46:0/398608685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18800cc30 con 0x7fe19806bd20 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe18f400640 1 -- 172.21.15.46:0/398608685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188013470 con 0x7fe19806bd20 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe198158ad0 con 0x7fe19806bd20 2024-04-25T08:50:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1980b49f0 con 0x7fe19806bd20 2024-04-25T08:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe198069080 con 0x7fe19806bd20 2024-04-25T08:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.311+0000 7fe18f400640 1 -- 172.21.15.46:0/398608685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe18801a070 con 0x7fe19806bd20 2024-04-25T08:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.315+0000 7fe18f400640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe174066f60 0x7fe174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.315+0000 7fe18f400640 1 -- 172.21.15.46:0/398608685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1880851c0 con 0x7fe19806bd20 2024-04-25T08:50:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.315+0000 7fe18fe00640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe174066f60 0x7fe174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.315+0000 7fe18fe00640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe174066f60 0x7fe174069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe198161d70 tx=0x7fe180009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:57.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.319+0000 7fe18f400640 1 -- 172.21.15.46:0/398608685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe188055fa0 con 0x7fe19806bd20 2024-04-25T08:50:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe1980b7630 con 0x7fe19806bd20 2024-04-25T08:50:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe18f400640 1 -- 172.21.15.46:0/398608685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe1880558a0 con 0x7fe19806bd20 2024-04-25T08:50:57.484 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe174066f60 msgr2=0x7fe174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe174066f60 0x7fe174069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe198161d70 tx=0x7fe180009040 comp rx=0 tx=0).stop 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe19806bd20 msgr2=0x7fe198157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe19806bd20 0x7fe198157df0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fe18800ebe0 tx=0x7fe18800c520 comp rx=0 tx=0).stop 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 shutdown_connections 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe174066f60 0x7fe174069420 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe198099a00 0x7fe1980b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe19806c190 0x7fe1980b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 --2- 172.21.15.46:0/398608685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe19806bd20 0x7fe198157df0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 >> 172.21.15.46:0/398608685 conn(0x7fe1980d16b0 msgr2=0x7fe19815da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:57.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 shutdown_connections 2024-04-25T08:50:57.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.495+0000 7fe19e600640 1 -- 172.21.15.46:0/398608685 wait complete. 2024-04-25T08:50:57.507 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:50:57.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:57.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:57.537 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:50:57.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:57.538 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:50:57.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:57.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.783+0000 7fe1efe00640 1 Processor -- start 2024-04-25T08:50:57.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.783+0000 7fe1efe00640 1 -- start start 2024-04-25T08:50:57.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1efe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1e8159230 0x7fe1e815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1efe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e80cc700 con 0x7fe1e80cdc50 2024-04-25T08:50:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1efe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e80cc9e0 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1efe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e80cccc0 con 0x7fe1e8159230 2024-04-25T08:50:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1ed400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1e8159230 0x7fe1e815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1ed400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe1e8159230 0x7fe1e815bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40316/0 (socket says 172.21.15.46:40316) 2024-04-25T08:50:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1ed400640 1 -- 172.21.15.46:0/887189515 learned_addr learned my addr 172.21.15.46:0/887189515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1ed400640 1 -- 172.21.15.46:0/887189515 >> v1:172.21.15.82:6789/0 conn(0x7fe1e80cbeb0 legacy=0x7fe1e80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1ed400640 1 -- 172.21.15.46:0/887189515 >> v1:172.21.15.46:6789/0 conn(0x7fe1e80cdc50 legacy=0x7fe1e815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.787+0000 7fe1ed400640 1 -- 172.21.15.46:0/887189515 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e80ccfa0 con 0x7fe1e8159230 2024-04-25T08:50:57.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1ed400640 1 --2- 172.21.15.46:0/887189515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e8159230 0x7fe1e815bdb0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009870 tx=0x7fe1dc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279c5689f37a4dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:57.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1e7e00640 1 -- 172.21.15.46:0/887189515 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0aa620 con 0x7fe1e8159230 2024-04-25T08:50:57.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1e7e00640 1 -- 172.21.15.46:0/887189515 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc0af070 con 0x7fe1e8159230 2024-04-25T08:50:57.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1e7e00640 1 -- 172.21.15.46:0/887189515 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0aa9e0 con 0x7fe1e8159230 2024-04-25T08:50:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1efe00640 1 -- 172.21.15.46:0/887189515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e8159230 msgr2=0x7fe1e815bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1efe00640 1 --2- 172.21.15.46:0/887189515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e8159230 0x7fe1e815bdb0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009870 tx=0x7fe1dc01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1efe00640 1 -- 172.21.15.46:0/887189515 shutdown_connections 2024-04-25T08:50:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1efe00640 1 --2- 172.21.15.46:0/887189515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e8159230 0x7fe1e815bdb0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1efe00640 1 -- 172.21.15.46:0/887189515 >> 172.21.15.46:0/887189515 conn(0x7fe1e80b12a0 msgr2=0x7fe1e80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1efe00640 1 -- 172.21.15.46:0/887189515 shutdown_connections 2024-04-25T08:50:57.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1efe00640 1 -- 172.21.15.46:0/887189515 wait complete. 2024-04-25T08:50:57.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1efe00640 1 Processor -- start 2024-04-25T08:50:57.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.791+0000 7fe1efe00640 1 -- start start 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e80cbeb0 0x7fe1e806ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1efe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1e80cdc50 0x7fe1e8155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e8159230 0x7fe1e813a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1efe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e815e120 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1efe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80cc4a0 con 0x7fe1e8159230 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1efe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e815e2a0 con 0x7fe1e80cdc50 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e80cbeb0 0x7fe1e806ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e80cbeb0 0x7fe1e806ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40328/0 (socket says 172.21.15.46:40328) 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ed400640 1 -- 172.21.15.46:0/1661244050 learned_addr learned my addr 172.21.15.46:0/1661244050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1eca00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1e80cdc50 0x7fe1e8155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ede00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e8159230 0x7fe1e813a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ed400640 1 -- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1e80cdc50 msgr2=0x7fe1e8155140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ed400640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1e80cdc50 0x7fe1e8155140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ed400640 1 -- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e8159230 msgr2=0x7fe1e813a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ed400640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e8159230 0x7fe1e813a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ed400640 1 -- 172.21.15.46:0/1661244050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8138d00 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1ed400640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e80cbeb0 0x7fe1e806ed90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0ac040 tx=0x7fe1dc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1e5600640 1 -- 172.21.15.46:0/1661244050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0bd3f0 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1e5600640 1 -- 172.21.15.46:0/1661244050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc0bd590 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1e5600640 1 -- 172.21.15.46:0/1661244050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc0bd980 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e8138fe0 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e8139ac0 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e8069080 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.795+0000 7fe1e5600640 1 -- 172.21.15.46:0/1661244050 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1dc0b33f0 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.799+0000 7fe1e5600640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe1c8066f60 0x7fe1c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:57.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.799+0000 7fe1e5600640 1 -- 172.21.15.46:0/1661244050 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1dc11f520 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.799+0000 7fe1eca00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe1c8066f60 0x7fe1c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:57.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.799+0000 7fe1eca00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe1c8066f60 0x7fe1c8069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe1e806eb50 tx=0x7fe1d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:57.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.803+0000 7fe1e5600640 1 -- 172.21.15.46:0/1661244050 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe1dc0c3060 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.995+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1e8067e70 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1e5600640 1 -- 172.21.15.46:0/1661244050 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fe1dc016150 con 0x7fe1e80cbeb0 2024-04-25T08:50:57.988 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:57.988 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:50:50.870175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8334":{"gid":8334,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/623970467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":623970467},{"type":"v1","addr":"172.21.15.46:6835","nonce":623970467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:50:57.988 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 83 2024-04-25T08:50:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe1c8066f60 msgr2=0x7fe1c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe1c8066f60 0x7fe1c8069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe1e806eb50 tx=0x7fe1d4009040 comp rx=0 tx=0).stop 2024-04-25T08:50:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e80cbeb0 msgr2=0x7fe1e806ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:57.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e80cbeb0 0x7fe1e806ed90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0ac040 tx=0x7fe1dc01ace0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 shutdown_connections 2024-04-25T08:50:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe1c8066f60 0x7fe1c8069420 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1e8159230 0x7fe1e813a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe1e80cdc50 0x7fe1e8155140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 --2- 172.21.15.46:0/1661244050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1e80cbeb0 0x7fe1e806ed90 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 >> 172.21.15.46:0/1661244050 conn(0x7fe1e80b12a0 msgr2=0x7fe1e80b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 shutdown_connections 2024-04-25T08:50:57.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:57.999+0000 7fe1efe00640 1 -- 172.21.15.46:0/1661244050 wait complete. 2024-04-25T08:50:58.009 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:50:58.177 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-25T08:50:58.178 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:50:58.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:50:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.327+0000 7efed8c00640 1 Processor -- start 2024-04-25T08:50:58.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.327+0000 7efed8c00640 1 -- start start 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed8c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efed406f320 0x7efed4162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed8c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed40b35b0 con 0x7efed40aed10 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed8c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed406d5e0 con 0x7efed415a750 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed8c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed415ab30 con 0x7efed406f320 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efed40aed10 0x7efed4164750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39502/0 (socket says 172.21.15.46:39502) 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed2800640 1 -- 172.21.15.46:0/814933288 learned_addr learned my addr 172.21.15.46:0/814933288 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed1400640 1 --2- 172.21.15.46:0/814933288 >> v2:172.21.15.82:3300/0 conn(0x7efed406f320 0x7efed4162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed0a00640 1 -- 172.21.15.46:0/814933288 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199092606 0 0) 0x7efed40b35b0 con 0x7efed40aed10 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed0a00640 1 -- 172.21.15.46:0/814933288 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeb8003530 con 0x7efed40aed10 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed1400640 1 -- 172.21.15.46:0/814933288 >> v1:172.21.15.82:6789/0 conn(0x7efed415a750 legacy=0x7efed406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed0a00640 1 -- 172.21.15.46:0/814933288 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359529745 0 0) 0x7efed406d5e0 con 0x7efed415a750 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed1400640 1 -- 172.21.15.46:0/814933288 >> v1:172.21.15.46:6789/0 conn(0x7efed40aed10 legacy=0x7efed4164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed1400640 1 -- 172.21.15.46:0/814933288 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40ce720 con 0x7efed406f320 2024-04-25T08:50:58.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.331+0000 7efed1400640 1 --2- 172.21.15.46:0/814933288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed406f320 0x7efed4162200 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efec00097c0 tx=0x7efec001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13da0c16d13799aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed0a00640 1 -- 172.21.15.46:0/814933288 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00b0070 con 0x7efed406f320 2024-04-25T08:50:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed0a00640 1 -- 172.21.15.46:0/814933288 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec00ab600 con 0x7efed406f320 2024-04-25T08:50:58.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed0a00640 1 -- 172.21.15.46:0/814933288 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00ab9d0 con 0x7efed406f320 2024-04-25T08:50:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 -- 172.21.15.46:0/814933288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed406f320 msgr2=0x7efed4162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 --2- 172.21.15.46:0/814933288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed406f320 0x7efed4162200 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efec00097c0 tx=0x7efec001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 -- 172.21.15.46:0/814933288 shutdown_connections 2024-04-25T08:50:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 --2- 172.21.15.46:0/814933288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed406f320 0x7efed4162200 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 -- 172.21.15.46:0/814933288 >> 172.21.15.46:0/814933288 conn(0x7efed40b12a0 msgr2=0x7efed40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 -- 172.21.15.46:0/814933288 shutdown_connections 2024-04-25T08:50:58.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 -- 172.21.15.46:0/814933288 wait complete. 2024-04-25T08:50:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 Processor -- start 2024-04-25T08:50:58.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 -- start start 2024-04-25T08:50:58.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efed406f320 0x7efed4154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed40aed10 0x7efed414e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed415a750 0x7efed414ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed4168690 con 0x7efed415a750 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed406d270 con 0x7efed40aed10 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed8c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed4168810 con 0x7efed406f320 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed415a750 0x7efed414ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed415a750 0x7efed414ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45040/0 (socket says 172.21.15.46:45040) 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.335+0000 7efed2800640 1 -- 172.21.15.46:0/194657765 learned_addr learned my addr 172.21.15.46:0/194657765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed1e00640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efed406f320 0x7efed4154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed1400640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed40aed10 0x7efed414e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed2800640 1 -- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efed406f320 msgr2=0x7efed4154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed2800640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efed406f320 0x7efed4154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed2800640 1 -- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed40aed10 msgr2=0x7efed414e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed2800640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed40aed10 0x7efed414e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed2800640 1 -- 172.21.15.46:0/194657765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed4154ca0 con 0x7efed415a750 2024-04-25T08:50:58.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed1400640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed40aed10 0x7efed414e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:58.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed2800640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed415a750 0x7efed414ec20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7efecc00cb70 tx=0x7efecc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efeca000640 1 -- 172.21.15.46:0/194657765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc017070 con 0x7efed415a750 2024-04-25T08:50:58.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efeca000640 1 -- 172.21.15.46:0/194657765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc00ad40 con 0x7efed415a750 2024-04-25T08:50:58.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efeca000640 1 -- 172.21.15.46:0/194657765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc012620 con 0x7efed415a750 2024-04-25T08:50:58.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed4157320 con 0x7efed415a750 2024-04-25T08:50:58.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed41578e0 con 0x7efed415a750 2024-04-25T08:50:58.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed4069080 con 0x7efed415a750 2024-04-25T08:50:58.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efeca000640 1 -- 172.21.15.46:0/194657765 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efecc01a070 con 0x7efed415a750 2024-04-25T08:50:58.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efeca000640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efeb4066f60 0x7efeb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.339+0000 7efeca000640 1 -- 172.21.15.46:0/194657765 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efecc084ff0 con 0x7efed415a750 2024-04-25T08:50:58.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.343+0000 7efed1e00640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efeb4066f60 0x7efeb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.343+0000 7efed1e00640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efeb4066f60 0x7efeb4069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efed40b4690 tx=0x7efec4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.343+0000 7efeca000640 1 -- 172.21.15.46:0/194657765 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efecc055e60 con 0x7efed415a750 2024-04-25T08:50:58.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.539+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efed40b3bf0 con 0x7efed415a750 2024-04-25T08:50:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.539+0000 7efeca000640 1 -- 172.21.15.46:0/194657765 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7efecc055760 con 0x7efed415a750 2024-04-25T08:50:58.531 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:50:58.532 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:50:50.870175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8334":{"gid":8334,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/623970467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":623970467},{"type":"v1","addr":"172.21.15.46:6835","nonce":623970467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:50:58.532 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 83 2024-04-25T08:50:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efeb4066f60 msgr2=0x7efeb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efeb4066f60 0x7efeb4069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efed40b4690 tx=0x7efec4009040 comp rx=0 tx=0).stop 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed415a750 msgr2=0x7efed414ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed415a750 0x7efed414ec20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7efecc00cb70 tx=0x7efecc00a430 comp rx=0 tx=0).stop 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 shutdown_connections 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efeb4066f60 0x7efeb4069420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed415a750 0x7efed414ec20 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed40aed10 0x7efed414e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 --2- 172.21.15.46:0/194657765 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efed406f320 0x7efed4154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 >> 172.21.15.46:0/194657765 conn(0x7efed40b12a0 msgr2=0x7efed40b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 shutdown_connections 2024-04-25T08:50:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.543+0000 7efed8c00640 1 -- 172.21.15.46:0/194657765 wait complete. 2024-04-25T08:50:58.553 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-25T08:50:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.607+0000 7f974fca8080 1 Processor -- start 2024-04-25T08:50:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.607+0000 7f974fca8080 1 -- start start 2024-04-25T08:50:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805bd54c0 0x563805ca7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805c1cf20 0x563805ca89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 0x563805ca8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805c8c7d0 con 0x563805bd54c0 2024-04-25T08:50:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805b799f0 con 0x563805c8b560 2024-04-25T08:50:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563805c8ca70 con 0x563805c1cf20 2024-04-25T08:50:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805bd54c0 0x563805ca7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805bd54c0 0x563805ca7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45070/0 (socket says 172.21.15.46:45070) 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974ce00640 1 -- 172.21.15.46:0/1004210921 learned_addr learned my addr 172.21.15.46:0/1004210921 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974d800640 1 --2- 172.21.15.46:0/1004210921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 0x563805ca8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f9747e00640 1 --2- 172.21.15.46:0/1004210921 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805c1cf20 0x563805ca89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974ce00640 1 -- 172.21.15.46:0/1004210921 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805c1cf20 msgr2=0x563805ca89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974ce00640 1 --2- 172.21.15.46:0/1004210921 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805c1cf20 0x563805ca89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974ce00640 1 -- 172.21.15.46:0/1004210921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 msgr2=0x563805ca8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974ce00640 1 --2- 172.21.15.46:0/1004210921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 0x563805ca8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974ce00640 1 -- 172.21.15.46:0/1004210921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563805caad40 con 0x563805bd54c0 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974ce00640 1 --2- 172.21.15.46:0/1004210921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805bd54c0 0x563805ca7470 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f974000a4a0 tx=0x7f974000a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb86aca79f4bb4b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f9746a00640 1 -- 172.21.15.46:0/1004210921 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974000b8d0 con 0x563805bd54c0 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f9746a00640 1 -- 172.21.15.46:0/1004210921 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974000f070 con 0x563805bd54c0 2024-04-25T08:50:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f9746a00640 1 -- 172.21.15.46:0/1004210921 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974000bcd0 con 0x563805bd54c0 2024-04-25T08:50:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- 172.21.15.46:0/1004210921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805bd54c0 msgr2=0x563805ca7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- 172.21.15.46:0/1004210921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805bd54c0 0x563805ca7470 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f974000a4a0 tx=0x7f974000a960 comp rx=0 tx=0).stop 2024-04-25T08:50:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- 172.21.15.46:0/1004210921 shutdown_connections 2024-04-25T08:50:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- 172.21.15.46:0/1004210921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 0x563805ca8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- 172.21.15.46:0/1004210921 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805c1cf20 0x563805ca89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- 172.21.15.46:0/1004210921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805bd54c0 0x563805ca7470 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- 172.21.15.46:0/1004210921 >> 172.21.15.46:0/1004210921 conn(0x563805bd3230 msgr2=0x563805c1c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- 172.21.15.46:0/1004210921 shutdown_connections 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- 172.21.15.46:0/1004210921 wait complete. 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 Processor -- start 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- start start 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805bd54c0 0x563805ca6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805c1cf20 0x563805c1c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 0x563805c8ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805cab280 con 0x563805c1cf20 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563805caafe0 con 0x563805c8b560 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f974fca8080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563805cab520 con 0x563805bd54c0 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f9747e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805c1cf20 0x563805c1c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f9747e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805c1cf20 0x563805c1c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45074/0 (socket says 172.21.15.46:45074) 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.611+0000 7f9747e00640 1 -- 172.21.15.46:0/2057123634 learned_addr learned my addr 172.21.15.46:0/2057123634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f974d800640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 0x563805c8ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f974ce00640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805bd54c0 0x563805ca6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9747e00640 1 -- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805bd54c0 msgr2=0x563805ca6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9747e00640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805bd54c0 0x563805ca6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9747e00640 1 -- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 msgr2=0x563805c8ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9747e00640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 0x563805c8ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9747e00640 1 -- 172.21.15.46:0/2057123634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563805c8b380 con 0x563805c1cf20 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f974d800640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563805c8b560 0x563805c8ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f974ce00640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563805bd54c0 0x563805ca6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9747e00640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805c1cf20 0x563805c1c510 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f97380ad040 tx=0x7f97380181d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9745600640 1 -- 172.21.15.46:0/2057123634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97380be070 con 0x563805c1cf20 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9745600640 1 -- 172.21.15.46:0/2057123634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97380b5650 con 0x563805c1cf20 2024-04-25T08:50:58.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563805ca5e00 con 0x563805c1cf20 2024-04-25T08:50:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9745600640 1 -- 172.21.15.46:0/2057123634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97380af040 con 0x563805c1cf20 2024-04-25T08:50:58.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563805ca6340 con 0x563805c1cf20 2024-04-25T08:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9745600640 1 -- 172.21.15.46:0/2057123634 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97380c1070 con 0x563805c1cf20 2024-04-25T08:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9745600640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f972c065940 0x7f972c067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f9745600640 1 -- 172.21.15.46:0/2057123634 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f973811e5b0 con 0x563805c1cf20 2024-04-25T08:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f974fca8080 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563805ca42d0 0x563805ca46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.615+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) v9 -- 0x563805ca4bf0 con 0x563805ca42d0 2024-04-25T08:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974ce00640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f972c065940 0x7f972c067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974d800640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563805ca42d0 0x563805ca46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974d800640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563805ca42d0 0x563805ca46b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974ce00640 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f972c065940 0x7f972c067e00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f974000a390 tx=0x7f974000af90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974d800640 1 -- 172.21.15.46:0/2057123634 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f97480098a0 con 0x563805ca42d0 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563805ca42d0 msgr2=0x563805ca46b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563805ca42d0 0x563805ca46b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f972c065940 msgr2=0x7f972c067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f972c065940 0x7f972c067e00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f974000a390 tx=0x7f974000af90 comp rx=0 tx=0).stop 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805c1cf20 msgr2=0x563805c1c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 --2- 172.21.15.46:0/2057123634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563805c1cf20 0x563805c1c510 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f97380ad040 tx=0x7f97380181d0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974d800640 1 -- 172.21.15.46:0/2057123634 reap_dead start 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 shutdown_connections 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 >> 172.21.15.46:0/2057123634 conn(0x563805bd3230 msgr2=0x563805c18dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 shutdown_connections 2024-04-25T08:50:58.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.619+0000 7f974fca8080 1 -- 172.21.15.46:0/2057123634 wait complete. 2024-04-25T08:50:58.613 INFO:teuthology.orchestra.run.smithi046.stdout:����� 2024-04-25T08:50:58.614 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:50:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f8916600640 1 Processor -- start 2024-04-25T08:50:58.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f8916600640 1 -- start start 2024-04-25T08:50:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f8916600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89100cf140 0x7f891015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f8916600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f89100cf520 0x7f891015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f8916600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891006ed70 con 0x7f89100afd40 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f8916600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89100afb40 con 0x7f89100cf140 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f8916600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89100d13f0 con 0x7f89100cf520 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f890f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f89100cf140 0x7f891015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f8906a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f89100afd40 0x7f89100ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39508/0 (socket says 172.21.15.46:39508) 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f8906a00640 1 -- 172.21.15.46:0/1128811695 learned_addr learned my addr 172.21.15.46:0/1128811695 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f890fe00640 1 --2- 172.21.15.46:0/1128811695 >> v2:172.21.15.82:3300/0 conn(0x7f89100cf520 0x7f891015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f890f400640 1 -- 172.21.15.46:0/1128811695 >> v2:172.21.15.82:3300/0 conn(0x7f89100cf520 msgr2=0x7f891015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f890f400640 1 --2- 172.21.15.46:0/1128811695 >> v2:172.21.15.82:3300/0 conn(0x7f89100cf520 0x7f891015dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f890f400640 1 -- 172.21.15.46:0/1128811695 >> v1:172.21.15.46:6789/0 conn(0x7f89100afd40 legacy=0x7f89100ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f890f400640 1 -- 172.21.15.46:0/1128811695 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8910162530 con 0x7f89100cf140 2024-04-25T08:50:58.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.807+0000 7f890ea00640 1 -- 172.21.15.46:0/1128811695 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936497950 0 0) 0x7f88f8001120 con 0x7f89100afd40 2024-04-25T08:50:58.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f890f400640 1 --2- 172.21.15.46:0/1128811695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf140 0x7f891015d590 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8910070df0 tx=0x7f890001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d2fc36e81282e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f890ea00640 1 -- 172.21.15.46:0/1128811695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89000b2070 con 0x7f89100cf140 2024-04-25T08:50:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f890ea00640 1 -- 172.21.15.46:0/1128811695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89000a5790 con 0x7f89100cf140 2024-04-25T08:50:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f890ea00640 1 -- 172.21.15.46:0/1128811695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89000a5b60 con 0x7f89100cf140 2024-04-25T08:50:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f8916600640 1 -- 172.21.15.46:0/1128811695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf140 msgr2=0x7f891015d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f8916600640 1 --2- 172.21.15.46:0/1128811695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf140 0x7f891015d590 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8910070df0 tx=0x7f890001c8a0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f8916600640 1 -- 172.21.15.46:0/1128811695 shutdown_connections 2024-04-25T08:50:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f8916600640 1 --2- 172.21.15.46:0/1128811695 >> v2:172.21.15.82:3300/0 conn(0x7f89100cf520 0x7f891015dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f8916600640 1 --2- 172.21.15.46:0/1128811695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf140 0x7f891015d590 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f8916600640 1 -- 172.21.15.46:0/1128811695 >> 172.21.15.46:0/1128811695 conn(0x7f89100728c0 msgr2=0x7f8910099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f8916600640 1 -- 172.21.15.46:0/1128811695 shutdown_connections 2024-04-25T08:50:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.811+0000 7f8916600640 1 -- 172.21.15.46:0/1128811695 wait complete. 2024-04-25T08:50:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 Processor -- start 2024-04-25T08:50:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 -- start start 2024-04-25T08:50:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89100afd40 0x7f89101552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89100cf140 0x7f891014eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf520 0x7f891014f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89100af7f0 con 0x7f89100cf520 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8910162010 con 0x7f89100cf140 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891006b620 con 0x7f89100afd40 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf520 0x7f891014f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf520 0x7f891014f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45092/0 (socket says 172.21.15.46:45092) 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890fe00640 1 -- 172.21.15.46:0/2022807963 learned_addr learned my addr 172.21.15.46:0/2022807963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8906a00640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89100cf140 0x7f891014eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890f400640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89100afd40 0x7f89101552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890fe00640 1 -- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89100afd40 msgr2=0x7f89101552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890fe00640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89100afd40 0x7f89101552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890fe00640 1 -- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89100cf140 msgr2=0x7f891014eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890fe00640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89100cf140 0x7f891014eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890fe00640 1 -- 172.21.15.46:0/2022807963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100b2420 con 0x7f89100cf520 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8906a00640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89100cf140 0x7f891014eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890f400640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89100afd40 0x7f89101552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f890fe00640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf520 0x7f891014f420 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f890800d560 tx=0x7f890800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8907e00640 1 -- 172.21.15.46:0/2022807963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080125f0 con 0x7f89100cf520 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8907e00640 1 -- 172.21.15.46:0/2022807963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8908019070 con 0x7f89100cf520 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8907e00640 1 -- 172.21.15.46:0/2022807963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080129f0 con 0x7f89100cf520 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8910160c30 con 0x7f89100cf520 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8910160f10 con 0x7f89100cf520 2024-04-25T08:50:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.815+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8910069080 con 0x7f89100cf520 2024-04-25T08:50:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.819+0000 7f8907e00640 1 -- 172.21.15.46:0/2022807963 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8908012b90 con 0x7f89100cf520 2024-04-25T08:50:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.819+0000 7f8907e00640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88ec066f60 0x7f88ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:50:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.819+0000 7f8907e00640 1 -- 172.21.15.46:0/2022807963 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8908085650 con 0x7f89100cf520 2024-04-25T08:50:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.819+0000 7f890f400640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88ec066f60 0x7f88ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:50:58.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.819+0000 7f890f400640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88ec066f60 0x7f88ec069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f89000af000 tx=0x7f89000a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:50:58.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:58.823+0000 7f8907e00640 1 -- 172.21.15.46:0/2022807963 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8908056430 con 0x7f89100cf520 2024-04-25T08:50:59.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.019+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f89100cce60 con 0x7f89100cf520 2024-04-25T08:50:59.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.019+0000 7f8907e00640 1 -- 172.21.15.46:0/2022807963 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8908055d30 con 0x7f89100cf520 2024-04-25T08:50:59.011 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:50:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88ec066f60 msgr2=0x7f88ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88ec066f60 0x7f88ec069420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f89000af000 tx=0x7f89000a7070 comp rx=0 tx=0).stop 2024-04-25T08:50:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf520 msgr2=0x7f891014f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:50:59.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf520 0x7f891014f420 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f890800d560 tx=0x7f890800da20 comp rx=0 tx=0).stop 2024-04-25T08:50:59.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 shutdown_connections 2024-04-25T08:50:59.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f88ec066f60 0x7f88ec069420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:59.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89100cf520 0x7f891014f420 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:59.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89100cf140 0x7f891014eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:59.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 --2- 172.21.15.46:0/2022807963 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f89100afd40 0x7f89101552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:50:59.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 >> 172.21.15.46:0/2022807963 conn(0x7f89100728c0 msgr2=0x7f89100b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:50:59.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 shutdown_connections 2024-04-25T08:50:59.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:50:59.023+0000 7f8916600640 1 -- 172.21.15.46:0/2022807963 wait complete. 2024-04-25T08:50:59.033 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:50:59.033 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:50:59.033 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:50:59.033 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:50:59.033 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:59.033 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:50:59.033 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:50:59.033 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:50:59.034 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:50:59.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.034 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:50:59.046 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41944sec preferred_lft 41944sec 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:50:59.047 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:59.048 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:50:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:50:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:50:59.049 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:50:59.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.049 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:50:59.109 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:50:59.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.112 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:50:59.164 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:50:59.164 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:50:59.164 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:50:59.164 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:50:59.164 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:50:59.164 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:50:59.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.165 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:50:59.165 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:50:59.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:50:59.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:50:59.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:50:59.165 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:50:59.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.258 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:50:59.267 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:50:59.267 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:50:59.267 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:50:59.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.267 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:50:59.331 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:50:59.331 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:59.379 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:50:59.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:50:59.380 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:59.431 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:50:59.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:50:59.522 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:50:59.522 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:50:59.522 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:50:59.537 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:50:59.537 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:50:59.537 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:50:59.537 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:50:59.537 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:50:59.547 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:50:59.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:50:59.547 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:50:59.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:50:59.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:50:59.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:50:59.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:50:59.547 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:50:59.552 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:50:59.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:50:59.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.571 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:50:59.630 INFO:teuthology.orchestra.run.smithi046.stdout:client7226 2024-04-25T08:50:59.631 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:59.631 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:59.681 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:59.681 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:59.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.736 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T08:50:59.884 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records in 2024-04-25T08:50:59.885 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records out 2024-04-25T08:50:59.885 INFO:teuthology.orchestra.run.smithi046.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0932763 s, 67.4 MB/s 2024-04-25T08:50:59.886 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:50:59.886 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:50:59.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:50:59.895 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T08:51:00.037 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records in 2024-04-25T08:51:00.038 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records out 2024-04-25T08:51:00.038 INFO:teuthology.orchestra.run.smithi046.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0880764 s, 71.4 MB/s 2024-04-25T08:51:00.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:51:00.039 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T08:51:00.049 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-04-25T08:51:00.049 INFO:teuthology.orchestra.run.smithi046.stdout:./file2_sixmegs 2024-04-25T08:51:00.049 INFO:teuthology.orchestra.run.smithi046.stdout:./file3_sixmegs 2024-04-25T08:51:00.049 INFO:teuthology.orchestra.run.smithi046.stdout:./file1_sixmegs 2024-04-25T08:51:00.050 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:51:00.050 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:51:00.051 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:51:00.051 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:51:00.051 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T08:51:00.051 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:51:00.157 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:51:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:51:00.166 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:51:00.166 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:51:00.166 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:51:00.166 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:51:00.166 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-25T08:51:00.166 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:51:00.274 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-04-25T08:51:00.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:51:00.283 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:51:00.283 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:51:00.283 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:51:00.283 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:51:00.283 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-25T08:51:00.283 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:51:00.393 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-04-25T08:51:00.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:51:00.403 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:51:00.403 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:51:00.403 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:51:00.403 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:51:00.403 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-25T08:51:00.403 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:51:00.515 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-25T08:51:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:51:00.525 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:51:00.525 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:51:00.576 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:51:00.576 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:51:00.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:51:00.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.827+0000 7f7a47a00640 1 Processor -- start 2024-04-25T08:51:00.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.827+0000 7f7a47a00640 1 -- start start 2024-04-25T08:51:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a47a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a47a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7a400aed10 0x7f7a4006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a47a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4006d5e0 con 0x7f7a400cc9e0 2024-04-25T08:51:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a47a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4015ab30 con 0x7f7a400aed10 2024-04-25T08:51:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a47a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a400ce4f0 con 0x7f7a4015a750 2024-04-25T08:51:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a37e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7a400aed10 0x7f7a4006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a45000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a37e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7a400aed10 0x7f7a4006f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:51154/0 (socket says 172.21.15.46:51154) 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a45000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4006ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:56234/0 (socket says 172.21.15.46:56234) 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a37e00640 1 -- 172.21.15.46:0/4056977925 learned_addr learned my addr 172.21.15.46:0/4056977925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a3fe00640 1 -- 172.21.15.46:0/4056977925 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576324186 0 0) 0x7f7a4006d5e0 con 0x7f7a400cc9e0 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a3fe00640 1 -- 172.21.15.46:0/4056977925 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a24003530 con 0x7f7a400cc9e0 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a45000640 1 -- 172.21.15.46:0/4056977925 >> v2:172.21.15.82:3300/0 conn(0x7f7a400aed10 msgr2=0x7f7a4006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a45000640 1 --2- 172.21.15.46:0/4056977925 >> v2:172.21.15.82:3300/0 conn(0x7f7a400aed10 0x7f7a4006f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a45000640 1 -- 172.21.15.46:0/4056977925 >> v1:172.21.15.82:6789/0 conn(0x7f7a400cc9e0 legacy=0x7f7a400b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a45000640 1 -- 172.21.15.46:0/4056977925 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a400ce720 con 0x7f7a4015a750 2024-04-25T08:51:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.831+0000 7f7a37e00640 1 --2- 172.21.15.46:0/4056977925 >> v2:172.21.15.82:3300/0 conn(0x7f7a400aed10 0x7f7a4006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:51:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a45000640 1 --2- 172.21.15.46:0/4056977925 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4006ec10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a300097c0 tx=0x7f7a3001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1e10bbb7448f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a3fe00640 1 -- 172.21.15.46:0/4056977925 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a300b2070 con 0x7f7a4015a750 2024-04-25T08:51:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a3fe00640 1 -- 172.21.15.46:0/4056977925 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a300ad640 con 0x7f7a4015a750 2024-04-25T08:51:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a3fe00640 1 -- 172.21.15.46:0/4056977925 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a300ada10 con 0x7f7a4015a750 2024-04-25T08:51:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a47a00640 1 -- 172.21.15.46:0/4056977925 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 msgr2=0x7f7a4006ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a47a00640 1 --2- 172.21.15.46:0/4056977925 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4006ec10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7a300097c0 tx=0x7f7a3001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:51:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a47a00640 1 -- 172.21.15.46:0/4056977925 shutdown_connections 2024-04-25T08:51:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a47a00640 1 --2- 172.21.15.46:0/4056977925 >> v2:172.21.15.82:3300/0 conn(0x7f7a400aed10 0x7f7a4006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a47a00640 1 --2- 172.21.15.46:0/4056977925 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4006ec10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a47a00640 1 -- 172.21.15.46:0/4056977925 >> 172.21.15.46:0/4056977925 conn(0x7f7a400b12a0 msgr2=0x7f7a400b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a47a00640 1 -- 172.21.15.46:0/4056977925 shutdown_connections 2024-04-25T08:51:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.835+0000 7f7a47a00640 1 -- 172.21.15.46:0/4056977925 wait complete. 2024-04-25T08:51:00.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 Processor -- start 2024-04-25T08:51:00.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 -- start start 2024-04-25T08:51:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a400aed10 0x7f7a40156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a400cc9e0 0x7f7a4014f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4006db00 con 0x7f7a400cc9e0 2024-04-25T08:51:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4006d270 con 0x7f7a400aed10 2024-04-25T08:51:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4006dc80 con 0x7f7a4015a750 2024-04-25T08:51:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a400cc9e0 0x7f7a4014f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a400cc9e0 0x7f7a4014f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45106/0 (socket says 172.21.15.46:45106) 2024-04-25T08:51:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a37e00640 1 -- 172.21.15.46:0/1768844731 learned_addr learned my addr 172.21.15.46:0/1768844731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a45000640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a400aed10 0x7f7a40156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a45a00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a37e00640 1 -- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 msgr2=0x7f7a4014fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a37e00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4014fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a37e00640 1 -- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a400aed10 msgr2=0x7f7a40156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a37e00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a400aed10 0x7f7a40156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a37e00640 1 -- 172.21.15.46:0/1768844731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a401503d0 con 0x7f7a400cc9e0 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a37e00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a400cc9e0 0x7f7a4014f850 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800ed70 tx=0x7f7a2800d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a3d600640 1 -- 172.21.15.46:0/1768844731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28010850 con 0x7f7a400cc9e0 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a3d600640 1 -- 172.21.15.46:0/1768844731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a280109f0 con 0x7f7a400cc9e0 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a401506b0 con 0x7f7a400cc9e0 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a3d600640 1 -- 172.21.15.46:0/1768844731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28010da0 con 0x7f7a400cc9e0 2024-04-25T08:51:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a40150c70 con 0x7f7a400cc9e0 2024-04-25T08:51:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.839+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a40069080 con 0x7f7a400cc9e0 2024-04-25T08:51:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.843+0000 7f7a3d600640 1 -- 172.21.15.46:0/1768844731 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a280213f0 con 0x7f7a400cc9e0 2024-04-25T08:51:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.843+0000 7f7a3d600640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7a1c066f60 0x7f7a1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.843+0000 7f7a3d600640 1 -- 172.21.15.46:0/1768844731 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7a28085c40 con 0x7f7a400cc9e0 2024-04-25T08:51:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.843+0000 7f7a45000640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7a1c066f60 0x7f7a1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.843+0000 7f7a45000640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7a1c066f60 0x7f7a1c069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7a300af000 tx=0x7f7a300a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:00.847+0000 7f7a3d600640 1 -- 172.21.15.46:0/1768844731 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a28014030 con 0x7f7a400cc9e0 2024-04-25T08:51:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.007+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a400b0600 con 0x7f7a400cc9e0 2024-04-25T08:51:01.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a3d600640 1 -- 172.21.15.46:0/1768844731 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16836 (secure 0 0 0) 0x7f7a2800ee60 con 0x7f7a400cc9e0 2024-04-25T08:51:01.001 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:01.001 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:50:49.457661+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:50:45.647599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:50:46.857198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:51:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7a1c066f60 msgr2=0x7f7a1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7a1c066f60 0x7f7a1c069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7a300af000 tx=0x7f7a300a7070 comp rx=0 tx=0).stop 2024-04-25T08:51:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a400cc9e0 msgr2=0x7f7a4014f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a400cc9e0 0x7f7a4014f850 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800ed70 tx=0x7f7a2800d720 comp rx=0 tx=0).stop 2024-04-25T08:51:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 shutdown_connections 2024-04-25T08:51:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7a1c066f60 0x7f7a1c069420 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7a4015a750 0x7f7a4014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a400cc9e0 0x7f7a4014f850 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 --2- 172.21.15.46:0/1768844731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a400aed10 0x7f7a40156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 >> 172.21.15.46:0/1768844731 conn(0x7f7a400b12a0 msgr2=0x7f7a400b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 shutdown_connections 2024-04-25T08:51:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.011+0000 7f7a47a00640 1 -- 172.21.15.46:0/1768844731 wait complete. 2024-04-25T08:51:01.023 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:50:45.647599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:50:46.857198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:51:01.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:51:01.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.175+0000 7f5e91000640 1 Processor -- start 2024-04-25T08:51:01.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.175+0000 7f5e91000640 1 -- start start 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e91000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e8c15dae0 0x7f5e8c072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e91000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06e620 0x7f5e8c06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e91000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5e8c06ea60 0x7f5e8c06c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e91000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8c06e190 con 0x7f5e8c15dae0 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e91000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8c0b3850 con 0x7f5e8c06ea60 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e91000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c0b3af0 con 0x7f5e8c06e620 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e89e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e8c15dae0 0x7f5e8c072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e89e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e8c15dae0 0x7f5e8c072600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45120/0 (socket says 172.21.15.46:45120) 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e89e00640 1 -- 172.21.15.46:0/532564576 learned_addr learned my addr 172.21.15.46:0/532564576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e8a800640 1 --2- 172.21.15.46:0/532564576 >> v2:172.21.15.82:3300/0 conn(0x7f5e8c06ea60 0x7f5e8c06c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e81400640 1 --2- 172.21.15.46:0/532564576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06e620 0x7f5e8c06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e89e00640 1 -- 172.21.15.46:0/532564576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06e620 msgr2=0x7f5e8c06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e89e00640 1 --2- 172.21.15.46:0/532564576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06e620 0x7f5e8c06bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e89e00640 1 -- 172.21.15.46:0/532564576 >> v2:172.21.15.82:3300/0 conn(0x7f5e8c06ea60 msgr2=0x7f5e8c06c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e89e00640 1 --2- 172.21.15.46:0/532564576 >> v2:172.21.15.82:3300/0 conn(0x7f5e8c06ea60 0x7f5e8c06c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.179+0000 7f5e89e00640 1 -- 172.21.15.46:0/532564576 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c0b3dd0 con 0x7f5e8c15dae0 2024-04-25T08:51:01.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e89e00640 1 --2- 172.21.15.46:0/532564576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 0x7f5e8c072600 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0d0e10 tx=0x7f5e7c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fe93b983326b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e89400640 1 -- 172.21.15.46:0/532564576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0aa610 con 0x7f5e8c15dae0 2024-04-25T08:51:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e89400640 1 -- 172.21.15.46:0/532564576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7c0af070 con 0x7f5e8c15dae0 2024-04-25T08:51:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e89400640 1 -- 172.21.15.46:0/532564576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c0aa9d0 con 0x7f5e8c15dae0 2024-04-25T08:51:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 -- 172.21.15.46:0/532564576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 msgr2=0x7f5e8c072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 --2- 172.21.15.46:0/532564576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 0x7f5e8c072600 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c0d0e10 tx=0x7f5e7c01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 -- 172.21.15.46:0/532564576 shutdown_connections 2024-04-25T08:51:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 --2- 172.21.15.46:0/532564576 >> v2:172.21.15.82:3300/0 conn(0x7f5e8c06ea60 0x7f5e8c06c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 --2- 172.21.15.46:0/532564576 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06e620 0x7f5e8c06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 --2- 172.21.15.46:0/532564576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 0x7f5e8c072600 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 -- 172.21.15.46:0/532564576 >> 172.21.15.46:0/532564576 conn(0x7f5e8c071ae0 msgr2=0x7f5e8c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 -- 172.21.15.46:0/532564576 shutdown_connections 2024-04-25T08:51:01.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 -- 172.21.15.46:0/532564576 wait complete. 2024-04-25T08:51:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 Processor -- start 2024-04-25T08:51:01.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.183+0000 7f5e91000640 1 -- start start 2024-04-25T08:51:01.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e91000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e8c06e620 0x7f5e8c152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e91000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06ea60 0x7f5e8c158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e91000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 0x7f5e8c14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e91000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c06f6d0 con 0x7f5e8c15dae0 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e91000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c15dec0 con 0x7f5e8c06e620 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e91000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c06f850 con 0x7f5e8c06ea60 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e8a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 0x7f5e8c14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e8a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 0x7f5e8c14dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45136/0 (socket says 172.21.15.46:45136) 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e8a800640 1 -- 172.21.15.46:0/2657926253 learned_addr learned my addr 172.21.15.46:0/2657926253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e89e00640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e8c06e620 0x7f5e8c152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e81400640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06ea60 0x7f5e8c158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e8a800640 1 -- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06ea60 msgr2=0x7f5e8c158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e8a800640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06ea60 0x7f5e8c158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e8a800640 1 -- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e8c06e620 msgr2=0x7f5e8c152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e8a800640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e8c06e620 0x7f5e8c152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e8a800640 1 -- 172.21.15.46:0/2657926253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c159110 con 0x7f5e8c15dae0 2024-04-25T08:51:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e8a800640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 0x7f5e8c14dd60 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400efc0 tx=0x7f5e8400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e82a00640 1 -- 172.21.15.46:0/2657926253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84018020 con 0x7f5e8c15dae0 2024-04-25T08:51:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e82a00640 1 -- 172.21.15.46:0/2657926253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e84014480 con 0x7f5e8c15dae0 2024-04-25T08:51:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c1593f0 con 0x7f5e8c15dae0 2024-04-25T08:51:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e82a00640 1 -- 172.21.15.46:0/2657926253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e84014870 con 0x7f5e8c15dae0 2024-04-25T08:51:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c1599b0 con 0x7f5e8c15dae0 2024-04-25T08:51:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c1554b0 con 0x7f5e8c15dae0 2024-04-25T08:51:01.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.187+0000 7f5e82a00640 1 -- 172.21.15.46:0/2657926253 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e8401b020 con 0x7f5e8c15dae0 2024-04-25T08:51:01.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.191+0000 7f5e82a00640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e68066f60 0x7f5e68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.191+0000 7f5e82a00640 1 -- 172.21.15.46:0/2657926253 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e84085b90 con 0x7f5e8c15dae0 2024-04-25T08:51:01.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.191+0000 7f5e89e00640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e68066f60 0x7f5e68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.191+0000 7f5e89e00640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e68066f60 0x7f5e68069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0ae000 tx=0x7f5e7c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:01.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.195+0000 7f5e82a00640 1 -- 172.21.15.46:0/2657926253 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e84022070 con 0x7f5e8c15dae0 2024-04-25T08:51:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.359+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e8c069010 con 0x7f5e8c15dae0 2024-04-25T08:51:01.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.359+0000 7f5e82a00640 1 -- 172.21.15.46:0/2657926253 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e84010040 con 0x7f5e8c15dae0 2024-04-25T08:51:01.350 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:51:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.359+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e68066f60 msgr2=0x7f5e68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.359+0000 7f5e91000640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e68066f60 0x7f5e68069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c0ae000 tx=0x7f5e7c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:51:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.359+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 msgr2=0x7f5e8c14dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.359+0000 7f5e91000640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 0x7f5e8c14dd60 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400efc0 tx=0x7f5e8400c490 comp rx=0 tx=0).stop 2024-04-25T08:51:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.363+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 shutdown_connections 2024-04-25T08:51:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.363+0000 7f5e91000640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5e68066f60 0x7f5e68069420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.363+0000 7f5e91000640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8c15dae0 0x7f5e8c14dd60 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.363+0000 7f5e91000640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5e8c06ea60 0x7f5e8c158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.363+0000 7f5e91000640 1 --2- 172.21.15.46:0/2657926253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e8c06e620 0x7f5e8c152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.363+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 >> 172.21.15.46:0/2657926253 conn(0x7f5e8c071ae0 msgr2=0x7f5e8c0afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.363+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 shutdown_connections 2024-04-25T08:51:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.363+0000 7f5e91000640 1 -- 172.21.15.46:0/2657926253 wait complete. 2024-04-25T08:51:01.371 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:51:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a8600640 1 Processor -- start 2024-04-25T08:51:01.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a8600640 1 -- start start 2024-04-25T08:51:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a0143990 0x7f41a00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a8600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41a01440d0 0x7f41a006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a8600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a015e440 con 0x7f41a006ef80 2024-04-25T08:51:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a8600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a015ed30 con 0x7f41a01440d0 2024-04-25T08:51:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00b3730 con 0x7f41a0143990 2024-04-25T08:51:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a5200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41a01440d0 0x7f41a006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a5200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f41a01440d0 0x7f41a006ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45146/0 (socket says 172.21.15.46:45146) 2024-04-25T08:51:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a5200640 1 -- 172.21.15.46:0/1913412188 learned_addr learned my addr 172.21.15.46:0/1913412188 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a5c00640 1 --2- 172.21.15.46:0/1913412188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a0143990 0x7f41a00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a5200640 1 -- 172.21.15.46:0/1913412188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a0143990 msgr2=0x7f41a00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a5200640 1 --2- 172.21.15.46:0/1913412188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a0143990 0x7f41a00cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a5200640 1 -- 172.21.15.46:0/1913412188 >> v1:172.21.15.82:6789/0 conn(0x7f41a006ef80 legacy=0x7f41a00cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.527+0000 7f41a5200640 1 -- 172.21.15.46:0/1913412188 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a015e6e0 con 0x7f41a01440d0 2024-04-25T08:51:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f41a5200640 1 --2- 172.21.15.46:0/1913412188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a01440d0 0x7f41a006ea40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f418c009870 tx=0x7f418c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c326815d20d0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:01.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f419fe00640 1 -- 172.21.15.46:0/1913412188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0b2070 con 0x7f41a01440d0 2024-04-25T08:51:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f419fe00640 1 -- 172.21.15.46:0/1913412188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c0ad640 con 0x7f41a01440d0 2024-04-25T08:51:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f419fe00640 1 -- 172.21.15.46:0/1913412188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0ada10 con 0x7f41a01440d0 2024-04-25T08:51:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f41a8600640 1 -- 172.21.15.46:0/1913412188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a01440d0 msgr2=0x7f41a006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f41a8600640 1 --2- 172.21.15.46:0/1913412188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a01440d0 0x7f41a006ea40 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f418c009870 tx=0x7f418c01c560 comp rx=0 tx=0).stop 2024-04-25T08:51:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f41a8600640 1 -- 172.21.15.46:0/1913412188 shutdown_connections 2024-04-25T08:51:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f41a8600640 1 --2- 172.21.15.46:0/1913412188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a01440d0 0x7f41a006ea40 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f41a8600640 1 --2- 172.21.15.46:0/1913412188 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a0143990 0x7f41a00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f41a8600640 1 -- 172.21.15.46:0/1913412188 >> 172.21.15.46:0/1913412188 conn(0x7f41a006ac10 msgr2=0x7f41a006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f41a8600640 1 -- 172.21.15.46:0/1913412188 shutdown_connections 2024-04-25T08:51:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.531+0000 7f41a8600640 1 -- 172.21.15.46:0/1913412188 wait complete. 2024-04-25T08:51:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 Processor -- start 2024-04-25T08:51:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 -- start start 2024-04-25T08:51:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41a006ef80 0x7f41a015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a0143990 0x7f41a0159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a01440d0 0x7f41a015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a0143d70 con 0x7f41a0143990 2024-04-25T08:51:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a006f390 con 0x7f41a006ef80 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a0143ef0 con 0x7f41a01440d0 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a0143990 0x7f41a0159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a0143990 0x7f41a0159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45160/0 (socket says 172.21.15.46:45160) 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5200640 1 -- 172.21.15.46:0/1012675255 learned_addr learned my addr 172.21.15.46:0/1012675255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5c00640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41a006ef80 0x7f41a015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a6600640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a01440d0 0x7f41a015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5200640 1 -- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a01440d0 msgr2=0x7f41a015a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5200640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a01440d0 0x7f41a015a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5200640 1 -- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41a006ef80 msgr2=0x7f41a015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5200640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41a006ef80 0x7f41a015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5200640 1 -- 172.21.15.46:0/1012675255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a00b20a0 con 0x7f41a0143990 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a6600640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a01440d0 0x7f41a015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a5200640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a0143990 0x7f41a0159db0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f418c0b0040 tx=0x7f418c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f419d600640 1 -- 172.21.15.46:0/1012675255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0bd070 con 0x7f41a0143990 2024-04-25T08:51:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f419d600640 1 -- 172.21.15.46:0/1012675255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c0b6550 con 0x7f41a0143990 2024-04-25T08:51:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f419d600640 1 -- 172.21.15.46:0/1012675255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c0b2070 con 0x7f41a0143990 2024-04-25T08:51:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a00b2380 con 0x7f41a0143990 2024-04-25T08:51:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a00b2940 con 0x7f41a0143990 2024-04-25T08:51:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.535+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a0069010 con 0x7f41a0143990 2024-04-25T08:51:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.539+0000 7f419d600640 1 -- 172.21.15.46:0/1012675255 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f418c0c0070 con 0x7f41a0143990 2024-04-25T08:51:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.539+0000 7f419d600640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4180066f60 0x7f4180069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.539+0000 7f419d600640 1 -- 172.21.15.46:0/1012675255 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f418c11eb30 con 0x7f41a0143990 2024-04-25T08:51:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.539+0000 7f41a5c00640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4180066f60 0x7f4180069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.539+0000 7f41a5c00640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4180066f60 0x7f4180069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4194009b60 tx=0x7f4194009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.543+0000 7f419d600640 1 -- 172.21.15.46:0/1012675255 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f418c016150 con 0x7f41a0143990 2024-04-25T08:51:01.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41a00b2b80 con 0x7f41a0143990 2024-04-25T08:51:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f419d600640 1 -- 172.21.15.46:0/1012675255 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f418c0ef360 con 0x7f41a0143990 2024-04-25T08:51:01.694 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:51:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4180066f60 msgr2=0x7f4180069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4180066f60 0x7f4180069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4194009b60 tx=0x7f4194009190 comp rx=0 tx=0).stop 2024-04-25T08:51:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a0143990 msgr2=0x7f41a0159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a0143990 0x7f41a0159db0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f418c0b0040 tx=0x7f418c019300 comp rx=0 tx=0).stop 2024-04-25T08:51:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 shutdown_connections 2024-04-25T08:51:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4180066f60 0x7f4180069420 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f41a01440d0 0x7f41a015a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41a0143990 0x7f41a0159db0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 --2- 172.21.15.46:0/1012675255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41a006ef80 0x7f41a015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 >> 172.21.15.46:0/1012675255 conn(0x7f41a006ac10 msgr2=0x7f41a013cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:01.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.703+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 shutdown_connections 2024-04-25T08:51:01.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.707+0000 7f41a8600640 1 -- 172.21.15.46:0/1012675255 wait complete. 2024-04-25T08:51:01.715 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:51:01.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f2856200640 1 Processor -- start 2024-04-25T08:51:01.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f2856200640 1 -- start start 2024-04-25T08:51:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f2856200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f285015dae0 0x7f2850072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f2856200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 0x7f285006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f2856200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28500b3850 con 0x7f285006ea60 2024-04-25T08:51:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f2856200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285006e190 con 0x7f285015dae0 2024-04-25T08:51:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f2856200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28500b3af0 con 0x7f285006e620 2024-04-25T08:51:01.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f284f400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f285015dae0 0x7f2850072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f2847e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 0x7f285006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f284f400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f285015dae0 0x7f2850072600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:51206/0 (socket says 172.21.15.46:51206) 2024-04-25T08:51:01.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.867+0000 7f284f400640 1 -- 172.21.15.46:0/927873403 learned_addr learned my addr 172.21.15.46:0/927873403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:01.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f284f400640 1 -- 172.21.15.46:0/927873403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 msgr2=0x7f285006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:01.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f284f400640 1 --2- 172.21.15.46:0/927873403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 0x7f285006bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f284f400640 1 -- 172.21.15.46:0/927873403 >> v1:172.21.15.82:6789/0 conn(0x7f285006ea60 legacy=0x7f28500acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f284f400640 1 -- 172.21.15.46:0/927873403 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500b3dd0 con 0x7f285015dae0 2024-04-25T08:51:01.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f284ea00640 1 -- 172.21.15.46:0/927873403 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230488429 0 0) 0x7f28500b3850 con 0x7f285006ea60 2024-04-25T08:51:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f284f400640 1 --2- 172.21.15.46:0/927873403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285015dae0 0x7f2850072600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f285006e310 tx=0x7f284001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a049a11b82bc560e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f284ea00640 1 -- 172.21.15.46:0/927873403 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400ab620 con 0x7f285015dae0 2024-04-25T08:51:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f284ea00640 1 -- 172.21.15.46:0/927873403 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28400b0070 con 0x7f285015dae0 2024-04-25T08:51:01.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f284ea00640 1 -- 172.21.15.46:0/927873403 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400ab9e0 con 0x7f285015dae0 2024-04-25T08:51:01.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f2856200640 1 -- 172.21.15.46:0/927873403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285015dae0 msgr2=0x7f2850072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f2856200640 1 --2- 172.21.15.46:0/927873403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285015dae0 0x7f2850072600 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f285006e310 tx=0x7f284001c510 comp rx=0 tx=0).stop 2024-04-25T08:51:01.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f2856200640 1 -- 172.21.15.46:0/927873403 shutdown_connections 2024-04-25T08:51:01.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f2856200640 1 --2- 172.21.15.46:0/927873403 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 0x7f285006bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f2856200640 1 --2- 172.21.15.46:0/927873403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285015dae0 0x7f2850072600 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f2856200640 1 -- 172.21.15.46:0/927873403 >> 172.21.15.46:0/927873403 conn(0x7f2850071ae0 msgr2=0x7f2850071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:01.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.871+0000 7f2856200640 1 -- 172.21.15.46:0/927873403 shutdown_connections 2024-04-25T08:51:01.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2856200640 1 -- 172.21.15.46:0/927873403 wait complete. 2024-04-25T08:51:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2856200640 1 Processor -- start 2024-04-25T08:51:01.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2856200640 1 -- start start 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2856200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 0x7f285014df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2856200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285006ea60 0x7f285014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2856200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285015dae0 0x7f2850150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2856200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285015dec0 con 0x7f285015dae0 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2856200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285006c570 con 0x7f285006ea60 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2856200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285006f6d0 con 0x7f285006e620 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285015dae0 0x7f2850150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285015dae0 0x7f2850150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45164/0 (socket says 172.21.15.46:45164) 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284fe00640 1 -- 172.21.15.46:0/3880820196 learned_addr learned my addr 172.21.15.46:0/3880820196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284f400640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 0x7f285014df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2847e00640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285006ea60 0x7f285014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284fe00640 1 -- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 msgr2=0x7f285014df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284fe00640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 0x7f285014df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284fe00640 1 -- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285006ea60 msgr2=0x7f285014fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284fe00640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285006ea60 0x7f285014fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284fe00640 1 -- 172.21.15.46:0/3880820196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2850151b50 con 0x7f285015dae0 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f284fe00640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285015dae0 0x7f2850150070 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f284800c780 tx=0x7f284800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.875+0000 7f2847400640 1 -- 172.21.15.46:0/3880820196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848003db0 con 0x7f285015dae0 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f2847400640 1 -- 172.21.15.46:0/3880820196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2848005070 con 0x7f285015dae0 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2850151e30 con 0x7f285015dae0 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f2847400640 1 -- 172.21.15.46:0/3880820196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2848005460 con 0x7f285015dae0 2024-04-25T08:51:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28501523f0 con 0x7f285015dae0 2024-04-25T08:51:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28501525f0 con 0x7f285015dae0 2024-04-25T08:51:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f2847400640 1 -- 172.21.15.46:0/3880820196 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2848005620 con 0x7f285015dae0 2024-04-25T08:51:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f2847400640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f282c066f60 0x7f282c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f2847400640 1 -- 172.21.15.46:0/3880820196 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28480850c0 con 0x7f285015dae0 2024-04-25T08:51:01.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f284f400640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f282c066f60 0x7f282c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:01.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.879+0000 7f284f400640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f282c066f60 0x7f282c069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f28400af000 tx=0x7f28400a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:01.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:01.883+0000 7f2847400640 1 -- 172.21.15.46:0/3880820196 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28501525f0 con 0x7f285015dae0 2024-04-25T08:51:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.043+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f285006f490 con 0x7f285015dae0 2024-04-25T08:51:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.043+0000 7f2847400640 1 -- 172.21.15.46:0/3880820196 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2848055850 con 0x7f285015dae0 2024-04-25T08:51:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f282c066f60 msgr2=0x7f282c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f282c066f60 0x7f282c069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f28400af000 tx=0x7f28400a7000 comp rx=0 tx=0).stop 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285015dae0 msgr2=0x7f2850150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285015dae0 0x7f2850150070 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f284800c780 tx=0x7f284800cc40 comp rx=0 tx=0).stop 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 shutdown_connections 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f282c066f60 0x7f282c069420 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285015dae0 0x7f2850150070 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f285006ea60 0x7f285014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 --2- 172.21.15.46:0/3880820196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f285006e620 0x7f285014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 >> 172.21.15.46:0/3880820196 conn(0x7f2850071ae0 msgr2=0x7f28500b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 shutdown_connections 2024-04-25T08:51:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.047+0000 7f2856200640 1 -- 172.21.15.46:0/3880820196 wait complete. 2024-04-25T08:51:02.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:51:02.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.207+0000 7f512dc00640 1 Processor -- start 2024-04-25T08:51:02.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.207+0000 7f512dc00640 1 -- start start 2024-04-25T08:51:02.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.207+0000 7f512dc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f512815bdb0 0x7f51280cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.207+0000 7f512dc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f51280cda80 0x7f512815d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f512dc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51280cc700 con 0x7f5128159230 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f512dc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51280cded0 con 0x7f512815bdb0 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f512dc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51280cc9e0 con 0x7f51280cda80 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f511e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f512815bdb0 0x7f51280cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f511e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f512815bdb0 0x7f51280cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45180/0 (socket says 172.21.15.46:45180) 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f511e000640 1 -- 172.21.15.46:0/2838227413 learned_addr learned my addr 172.21.15.46:0/2838227413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f5127400640 1 --2- 172.21.15.46:0/2838227413 >> v2:172.21.15.82:3300/0 conn(0x7f51280cda80 0x7f512815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f511e000640 1 -- 172.21.15.46:0/2838227413 >> v2:172.21.15.82:3300/0 conn(0x7f51280cda80 msgr2=0x7f512815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f511e000640 1 --2- 172.21.15.46:0/2838227413 >> v2:172.21.15.82:3300/0 conn(0x7f51280cda80 0x7f512815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f511e000640 1 -- 172.21.15.46:0/2838227413 >> v1:172.21.15.82:6789/0 conn(0x7f5128159230 legacy=0x7f51280cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:02.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f511e000640 1 -- 172.21.15.46:0/2838227413 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51280cccc0 con 0x7f512815bdb0 2024-04-25T08:51:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f511e000640 1 --2- 172.21.15.46:0/2838227413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 0x7f51280cd540 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f51280b2340 tx=0x7f511001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316cc552f26b38fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f5126000640 1 -- 172.21.15.46:0/2838227413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51100b0070 con 0x7f512815bdb0 2024-04-25T08:51:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f5126000640 1 -- 172.21.15.46:0/2838227413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51100ab600 con 0x7f512815bdb0 2024-04-25T08:51:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.211+0000 7f5126000640 1 -- 172.21.15.46:0/2838227413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51100ab9d0 con 0x7f512815bdb0 2024-04-25T08:51:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 -- 172.21.15.46:0/2838227413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 msgr2=0x7f51280cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 --2- 172.21.15.46:0/2838227413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 0x7f51280cd540 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f51280b2340 tx=0x7f511001c450 comp rx=0 tx=0).stop 2024-04-25T08:51:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 -- 172.21.15.46:0/2838227413 shutdown_connections 2024-04-25T08:51:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 --2- 172.21.15.46:0/2838227413 >> v2:172.21.15.82:3300/0 conn(0x7f51280cda80 0x7f512815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 --2- 172.21.15.46:0/2838227413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 0x7f51280cd540 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 -- 172.21.15.46:0/2838227413 >> 172.21.15.46:0/2838227413 conn(0x7f51280b12a0 msgr2=0x7f51280b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 -- 172.21.15.46:0/2838227413 shutdown_connections 2024-04-25T08:51:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 -- 172.21.15.46:0/2838227413 wait complete. 2024-04-25T08:51:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 Processor -- start 2024-04-25T08:51:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 -- start start 2024-04-25T08:51:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51280cda80 0x7f5128142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5128159230 0x7f512813f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 0x7f5128164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512815e0d0 con 0x7f512815bdb0 2024-04-25T08:51:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512815df50 con 0x7f5128159230 2024-04-25T08:51:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f512dc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512815e250 con 0x7f51280cda80 2024-04-25T08:51:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f5127400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 0x7f5128164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f5127400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 0x7f5128164900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45194/0 (socket says 172.21.15.46:45194) 2024-04-25T08:51:02.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f5127400640 1 -- 172.21.15.46:0/1804357313 learned_addr learned my addr 172.21.15.46:0/1804357313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f5126a00640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51280cda80 0x7f5128142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.215+0000 7f511e000640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5128159230 0x7f512813f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f5127400640 1 -- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51280cda80 msgr2=0x7f5128142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f5127400640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51280cda80 0x7f5128142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f5127400640 1 -- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5128159230 msgr2=0x7f512813f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f5127400640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5128159230 0x7f512813f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f5127400640 1 -- 172.21.15.46:0/1804357313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512806d3f0 con 0x7f512815bdb0 2024-04-25T08:51:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f511e000640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5128159230 0x7f512813f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:51:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f5127400640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 0x7f5128164900 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f5120007e40 tx=0x7f512000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f511f400640 1 -- 172.21.15.46:0/1804357313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120014620 con 0x7f512815bdb0 2024-04-25T08:51:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f511f400640 1 -- 172.21.15.46:0/1804357313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120019070 con 0x7f512815bdb0 2024-04-25T08:51:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f511f400640 1 -- 172.21.15.46:0/1804357313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120014a20 con 0x7f512815bdb0 2024-04-25T08:51:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512806d6d0 con 0x7f512815bdb0 2024-04-25T08:51:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512806dc90 con 0x7f512815bdb0 2024-04-25T08:51:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5128069080 con 0x7f512815bdb0 2024-04-25T08:51:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f511f400640 1 -- 172.21.15.46:0/1804357313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5120014bc0 con 0x7f512815bdb0 2024-04-25T08:51:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f511f400640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5104066f90 0x7f5104069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.219+0000 7f511f400640 1 -- 172.21.15.46:0/1804357313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5120085990 con 0x7f512815bdb0 2024-04-25T08:51:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.223+0000 7f5126a00640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5104066f90 0x7f5104069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.223+0000 7f5126a00640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5104066f90 0x7f5104069450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5118007fd0 tx=0x7f5118007e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.223+0000 7f511f400640 1 -- 172.21.15.46:0/1804357313 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5120056130 con 0x7f512815bdb0 2024-04-25T08:51:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51280723d0 con 0x7f512815bdb0 2024-04-25T08:51:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f511f400640 1 -- 172.21.15.46:0/1804357313 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7f512005a1d0 con 0x7f512815bdb0 2024-04-25T08:51:02.389 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:51:02.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5104066f90 msgr2=0x7f5104069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5104066f90 0x7f5104069450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5118007fd0 tx=0x7f5118007e40 comp rx=0 tx=0).stop 2024-04-25T08:51:02.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 msgr2=0x7f5128164900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 0x7f5128164900 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f5120007e40 tx=0x7f512000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 shutdown_connections 2024-04-25T08:51:02.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5104066f90 0x7f5104069450 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f512815bdb0 0x7f5128164900 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5128159230 0x7f512813f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 --2- 172.21.15.46:0/1804357313 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f51280cda80 0x7f5128142090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 >> 172.21.15.46:0/1804357313 conn(0x7f51280b12a0 msgr2=0x7f51280af180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:02.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 shutdown_connections 2024-04-25T08:51:02.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.399+0000 7f512dc00640 1 -- 172.21.15.46:0/1804357313 wait complete. 2024-04-25T08:51:02.411 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:51:02.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:51:02.411 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:51:02.469 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:51:02.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:51:02.469 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:51:02.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:51:02.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.679+0000 7f0ddca00640 1 Processor -- start 2024-04-25T08:51:02.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0ddca00640 1 -- start start 2024-04-25T08:51:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0ddca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0dd809a610 0x7f0dd809aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0ddca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd815f300 con 0x7f0dd8099a00 2024-04-25T08:51:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0ddca00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd80cd8a0 con 0x7f0dd815ef20 2024-04-25T08:51:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0ddca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd80cddb0 con 0x7f0dd809a610 2024-04-25T08:51:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0dcfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0dd809a610 0x7f0dd809aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0dd6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0dd8099a00 0x7f0dd80c5180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39518/0 (socket says 172.21.15.46:39518) 2024-04-25T08:51:02.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0dd6800640 1 -- 172.21.15.46:0/782832058 learned_addr learned my addr 172.21.15.46:0/782832058 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:02.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0dcfe00640 1 -- 172.21.15.46:0/782832058 >> v1:172.21.15.82:6789/0 conn(0x7f0dd815ef20 legacy=0x7f0dd80cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0dcfe00640 1 -- 172.21.15.46:0/782832058 >> v1:172.21.15.46:6789/0 conn(0x7f0dd8099a00 legacy=0x7f0dd80c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.683+0000 7f0dcfe00640 1 -- 172.21.15.46:0/782832058 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd80cc3d0 con 0x7f0dd809a610 2024-04-25T08:51:02.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0dcfe00640 1 --2- 172.21.15.46:0/782832058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd809a610 0x7f0dd809aa10 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00097c0 tx=0x7f0dc001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d124615413c74f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:02.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0dd5400640 1 -- 172.21.15.46:0/782832058 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00b1070 con 0x7f0dd809a610 2024-04-25T08:51:02.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0dd5400640 1 -- 172.21.15.46:0/782832058 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc00ac5f0 con 0x7f0dd809a610 2024-04-25T08:51:02.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0dd5400640 1 -- 172.21.15.46:0/782832058 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00ac9c0 con 0x7f0dd809a610 2024-04-25T08:51:02.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0ddca00640 1 -- 172.21.15.46:0/782832058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd809a610 msgr2=0x7f0dd809aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0ddca00640 1 --2- 172.21.15.46:0/782832058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd809a610 0x7f0dd809aa10 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00097c0 tx=0x7f0dc001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0ddca00640 1 -- 172.21.15.46:0/782832058 shutdown_connections 2024-04-25T08:51:02.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0ddca00640 1 --2- 172.21.15.46:0/782832058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd809a610 0x7f0dd809aa10 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0ddca00640 1 -- 172.21.15.46:0/782832058 >> 172.21.15.46:0/782832058 conn(0x7f0dd80b3120 msgr2=0x7f0dd80b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:02.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0ddca00640 1 -- 172.21.15.46:0/782832058 shutdown_connections 2024-04-25T08:51:02.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0ddca00640 1 -- 172.21.15.46:0/782832058 wait complete. 2024-04-25T08:51:02.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0ddca00640 1 Processor -- start 2024-04-25T08:51:02.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.687+0000 7f0ddca00640 1 -- start start 2024-04-25T08:51:02.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0ddca00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0dd8099a00 0x7f0dd80ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0ddca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dd809a610 0x7f0dd815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0ddca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd815ef20 0x7f0dd815e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0ddca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd809a220 con 0x7f0dd815ef20 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0ddca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd80cc010 con 0x7f0dd809a610 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0ddca00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd80c9880 con 0x7f0dd8099a00 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd815ef20 0x7f0dd815e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd815ef20 0x7f0dd815e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45204/0 (socket says 172.21.15.46:45204) 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd6800640 1 -- 172.21.15.46:0/306821672 learned_addr learned my addr 172.21.15.46:0/306821672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dcfe00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dd809a610 0x7f0dd815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd5e00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0dd8099a00 0x7f0dd80ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd6800640 1 -- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0dd8099a00 msgr2=0x7f0dd80ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd6800640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0dd8099a00 0x7f0dd80ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd6800640 1 -- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dd809a610 msgr2=0x7f0dd815dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd6800640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dd809a610 0x7f0dd815dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd6800640 1 -- 172.21.15.46:0/306821672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd80c8c70 con 0x7f0dd815ef20 2024-04-25T08:51:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dcfe00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dd809a610 0x7f0dd815dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:51:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dd6800640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd815ef20 0x7f0dd815e180 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000cab0 tx=0x7f0dd000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dce000640 1 -- 172.21.15.46:0/306821672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0017070 con 0x7f0dd815ef20 2024-04-25T08:51:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dce000640 1 -- 172.21.15.46:0/306821672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd000ac30 con 0x7f0dd815ef20 2024-04-25T08:51:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dce000640 1 -- 172.21.15.46:0/306821672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0012400 con 0x7f0dd815ef20 2024-04-25T08:51:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd80c8f50 con 0x7f0dd815ef20 2024-04-25T08:51:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd80c9510 con 0x7f0dd815ef20 2024-04-25T08:51:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd8069080 con 0x7f0dd815ef20 2024-04-25T08:51:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.691+0000 7f0dce000640 1 -- 172.21.15.46:0/306821672 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0dd001a070 con 0x7f0dd815ef20 2024-04-25T08:51:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.695+0000 7f0dce000640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0db4066f60 0x7f0db4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:02.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.695+0000 7f0dce000640 1 -- 172.21.15.46:0/306821672 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0dd0084e00 con 0x7f0dd815ef20 2024-04-25T08:51:02.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.695+0000 7f0dd5e00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0db4066f60 0x7f0db4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:02.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.695+0000 7f0dd5e00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0db4066f60 0x7f0db4069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8001810 tx=0x7f0dc8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.699+0000 7f0dce000640 1 -- 172.21.15.46:0/306821672 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0dd0055590 con 0x7f0dd815ef20 2024-04-25T08:51:02.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.891+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dd80d0700 con 0x7f0dd815ef20 2024-04-25T08:51:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.891+0000 7f0dce000640 1 -- 172.21.15.46:0/306821672 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f0dd0059630 con 0x7f0dd815ef20 2024-04-25T08:51:02.883 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:02.883 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:50:50.870175+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8334},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8334":{"gid":8334,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/623970467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":623970467},{"type":"v1","addr":"172.21.15.46:6835","nonce":623970467}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:51:02.883 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 83 2024-04-25T08:51:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0db4066f60 msgr2=0x7f0db4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0db4066f60 0x7f0db4069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8001810 tx=0x7f0dc8009040 comp rx=0 tx=0).stop 2024-04-25T08:51:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd815ef20 msgr2=0x7f0dd815e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd815ef20 0x7f0dd815e180 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0dd000cab0 tx=0x7f0dd000cf70 comp rx=0 tx=0).stop 2024-04-25T08:51:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 shutdown_connections 2024-04-25T08:51:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0db4066f60 0x7f0db4069420 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0dd815ef20 0x7f0dd815e180 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dd809a610 0x7f0dd815dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 --2- 172.21.15.46:0/306821672 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0dd8099a00 0x7f0dd80ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 >> 172.21.15.46:0/306821672 conn(0x7f0dd80b3120 msgr2=0x7f0dd80cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 shutdown_connections 2024-04-25T08:51:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:02.895+0000 7f0ddca00640 1 -- 172.21.15.46:0/306821672 wait complete. 2024-04-25T08:51:02.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:51:03.076 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-04-25T08:51:03.077 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:51:03.077 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:51:03.077 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:51:03.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:51:03.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:51:03.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:51:03.087 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:51:03.136 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T08:51:03.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:51:03.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T08:51:03.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:51:03.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T08:51:03.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.335+0000 7fd3d1a00640 1 Processor -- start 2024-04-25T08:51:03.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3d1a00640 1 -- start start 2024-04-25T08:51:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3d1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3cc15b810 0x7fd3cc06f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3d1a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3cc06efd0 0x7fd3cc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3d1a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc15c130 0x7fd3cc06bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3d1a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc0af4e0 con 0x7fd3cc15b810 2024-04-25T08:51:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3d1a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc0ad430 con 0x7fd3cc06efd0 2024-04-25T08:51:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3d1a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0ad710 con 0x7fd3cc15c130 2024-04-25T08:51:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3caa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3cc15b810 0x7fd3cc06f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3caa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3cc15b810 0x7fd3cc06f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45214/0 (socket says 172.21.15.46:45214) 2024-04-25T08:51:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3caa00640 1 -- 172.21.15.46:0/3287137737 learned_addr learned my addr 172.21.15.46:0/3287137737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3c2000640 1 --2- 172.21.15.46:0/3287137737 >> v2:172.21.15.82:3300/0 conn(0x7fd3cc06efd0 0x7fd3cc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3cb400640 1 --2- 172.21.15.46:0/3287137737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc15c130 0x7fd3cc06bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3caa00640 1 -- 172.21.15.46:0/3287137737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc15c130 msgr2=0x7fd3cc06bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3caa00640 1 --2- 172.21.15.46:0/3287137737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc15c130 0x7fd3cc06bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3caa00640 1 -- 172.21.15.46:0/3287137737 >> v2:172.21.15.82:3300/0 conn(0x7fd3cc06efd0 msgr2=0x7fd3cc15bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3caa00640 1 --2- 172.21.15.46:0/3287137737 >> v2:172.21.15.82:3300/0 conn(0x7fd3cc06efd0 0x7fd3cc15bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.339+0000 7fd3caa00640 1 -- 172.21.15.46:0/3287137737 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0726f0 con 0x7fd3cc15b810 2024-04-25T08:51:03.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3caa00640 1 --2- 172.21.15.46:0/3287137737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 0x7fd3cc06f520 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0681c0 tx=0x7fd3bc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d368d79342a2347b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3ca000640 1 -- 172.21.15.46:0/3287137737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0b0070 con 0x7fd3cc15b810 2024-04-25T08:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3ca000640 1 -- 172.21.15.46:0/3287137737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc0ab600 con 0x7fd3cc15b810 2024-04-25T08:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3ca000640 1 -- 172.21.15.46:0/3287137737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0ab9d0 con 0x7fd3cc15b810 2024-04-25T08:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 -- 172.21.15.46:0/3287137737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 msgr2=0x7fd3cc06f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/3287137737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 0x7fd3cc06f520 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0681c0 tx=0x7fd3bc01c490 comp rx=0 tx=0).stop 2024-04-25T08:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 -- 172.21.15.46:0/3287137737 shutdown_connections 2024-04-25T08:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/3287137737 >> v2:172.21.15.82:3300/0 conn(0x7fd3cc06efd0 0x7fd3cc15bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/3287137737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc15c130 0x7fd3cc06bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/3287137737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 0x7fd3cc06f520 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 -- 172.21.15.46:0/3287137737 >> 172.21.15.46:0/3287137737 conn(0x7fd3cc071a40 msgr2=0x7fd3cc071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 -- 172.21.15.46:0/3287137737 shutdown_connections 2024-04-25T08:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 -- 172.21.15.46:0/3287137737 wait complete. 2024-04-25T08:51:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 Processor -- start 2024-04-25T08:51:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.343+0000 7fd3d1a00640 1 -- start start 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3d1a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc06efd0 0x7fd3cc1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3d1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 0x7fd3cc099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3d1a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc15c130 0x7fd3cc09c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3d1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc06c570 con 0x7fd3cc15b810 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3d1a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc06ed30 con 0x7fd3cc15c130 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3d1a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc15f330 con 0x7fd3cc06efd0 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 0x7fd3cc099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 0x7fd3cc099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45218/0 (socket says 172.21.15.46:45218) 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c2000640 1 -- 172.21.15.46:0/1592515512 learned_addr learned my addr 172.21.15.46:0/1592515512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3caa00640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc06efd0 0x7fd3cc1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3cb400640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc15c130 0x7fd3cc09c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c2000640 1 -- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc06efd0 msgr2=0x7fd3cc1511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c2000640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc06efd0 0x7fd3cc1511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c2000640 1 -- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc15c130 msgr2=0x7fd3cc09c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c2000640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc15c130 0x7fd3cc09c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c2000640 1 -- 172.21.15.46:0/1592515512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc09a4c0 con 0x7fd3cc15b810 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c2000640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 0x7fd3cc099a00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400d830 tx=0x7fd3b400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c3400640 1 -- 172.21.15.46:0/1592515512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400b6c0 con 0x7fd3cc15b810 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c3400640 1 -- 172.21.15.46:0/1592515512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b4019070 con 0x7fd3cc15b810 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3d1a00640 1 -- 172.21.15.46:0/1592515512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc09a7a0 con 0x7fd3cc15b810 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3c3400640 1 -- 172.21.15.46:0/1592515512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b400ba60 con 0x7fd3cc15b810 2024-04-25T08:51:03.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.347+0000 7fd3d1a00640 1 -- 172.21.15.46:0/1592515512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc09ad60 con 0x7fd3cc15b810 2024-04-25T08:51:03.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3c3400640 1 -- 172.21.15.46:0/1592515512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3b401e460 con 0x7fd3cc15b810 2024-04-25T08:51:03.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3c3400640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3a8066f60 0x7fd3a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3c3400640 1 -- 172.21.15.46:0/1592515512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3b4085820 con 0x7fd3cc15b810 2024-04-25T08:51:03.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3a8066f60 0x7fd3a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3d4e4a000 1 Processor -- start 2024-04-25T08:51:03.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3d4e4a000 1 -- start start 2024-04-25T08:51:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3a8066f60 0x7fd3a8069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0ad000 tx=0x7fd3bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 0x55ea78079d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea7807a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea77f60f40 con 0x55ea780c3c90 2024-04-25T08:51:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea77ec8070 con 0x55ea780c45b0 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea77e501a0 con 0x55ea7807a290 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780c4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45228/0 (socket says 172.21.15.46:45228) 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 -- 172.21.15.46:0/2424036309 learned_addr learned my addr 172.21.15.46:0/2424036309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3c2000640 1 --2- 172.21.15.46:0/2424036309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 0x55ea78079d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3cb400640 1 --2- 172.21.15.46:0/2424036309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea7807a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 -- 172.21.15.46:0/2424036309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 msgr2=0x55ea7807a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 --2- 172.21.15.46:0/2424036309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea7807a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 -- 172.21.15.46:0/2424036309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 msgr2=0x55ea78079d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 --2- 172.21.15.46:0/2424036309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 0x55ea78079d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 -- 172.21.15.46:0/2424036309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea77e4e510 con 0x55ea780c3c90 2024-04-25T08:51:03.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3caa00640 1 --2- 172.21.15.46:0/2424036309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780c4070 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009630 tx=0x7fd3bc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4463045aefdb200d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3bb400640 1 -- 172.21.15.46:0/2424036309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0c7070 con 0x55ea780c3c90 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3bb400640 1 -- 172.21.15.46:0/2424036309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc0abda0 con 0x55ea780c3c90 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.351+0000 7fd3bb400640 1 -- 172.21.15.46:0/2424036309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0b0040 con 0x55ea780c3c90 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2424036309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 msgr2=0x55ea780c4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2424036309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780c4070 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009630 tx=0x7fd3bc004740 comp rx=0 tx=0).stop 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2424036309 shutdown_connections 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2424036309 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea7807a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2424036309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 0x55ea78079d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2424036309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780c4070 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2424036309 >> 172.21.15.46:0/2424036309 conn(0x55ea78076240 msgr2=0x55ea78078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2424036309 shutdown_connections 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2424036309 wait complete. 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- start start 2024-04-25T08:51:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 Processor -- start 2024-04-25T08:51:03.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea7807a290 0x55ea78084e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 0x55ea78085370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea780c45b0 0x55ea780858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea77e826c0 con 0x55ea7807a290 2024-04-25T08:51:03.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea77ec4110 con 0x55ea780c3c90 2024-04-25T08:51:03.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea78083200 con 0x55ea780c45b0 2024-04-25T08:51:03.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 10 client.0 _ll_register_callbacks cb 0x55ea78065570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:03.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea7807a290 0x55ea78084e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea7807a290 0x55ea78084e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45242/0 (socket says 172.21.15.46:45242) 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3caa00640 1 -- 172.21.15.46:0/16295449 learned_addr learned my addr 172.21.15.46:0/16295449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3cb400640 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea780c45b0 0x55ea780858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3c2000640 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 0x55ea78085370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3caa00640 1 -- 172.21.15.46:0/16295449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea780c45b0 msgr2=0x55ea780858b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3caa00640 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea780c45b0 0x55ea780858b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3caa00640 1 -- 172.21.15.46:0/16295449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 msgr2=0x55ea78085370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3caa00640 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 0x55ea78085370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3caa00640 1 -- 172.21.15.46:0/16295449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea77e503b0 con 0x55ea7807a290 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3caa00640 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea7807a290 0x55ea78084e30 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc097d20 tx=0x7fd3bc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d439ed1d7d180d35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3afe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3afe00640 1 -- 172.21.15.46:0/16295449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0cf060 con 0x55ea7807a290 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3afe00640 1 -- 172.21.15.46:0/16295449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3bc0b6470 con 0x55ea7807a290 2024-04-25T08:51:03.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.355+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/16295449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3bc0abda0 con 0x55ea7807a290 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3afe00640 1 -- 172.21.15.46:0/16295449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3bc0c7070 con 0x55ea7807a290 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 10 client.8586 fetch_fsmap learned FSMap version 83 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/16295449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55ea77f49e00 con 0x55ea7807a290 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3afe00640 1 -- 172.21.15.46:0/16295449 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc0abda0 con 0x55ea7807a290 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3afe00640 1 -- 172.21.15.46:0/16295449 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd3bc0b6670 con 0x55ea7807a290 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 10 client.8586 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 10 client.8586 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 20 client.8586 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55ea78086180 0x55ea780d5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 4 client.8586 mds_command: new command op to 8334 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/16295449 --> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea77e63d60 con 0x55ea78086180 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3c2000640 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55ea78086180 0x55ea780d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3afe00640 10 client.8586 ms_handle_connect on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:03.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3c2000640 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55ea78086180 0x55ea780d5410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1688310e34aab25 server_cookie=22927eb77530eca8 in_seq=0 out_seq=0 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3afe00640 1 -- 172.21.15.46:0/16295449 <== mds.0 v2:172.21.15.46:6834/623970467 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea77e63d60 con 0x55ea78086180 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3afe00640 10 client.8586 handle_command_reply: tid=0 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 client.8586 shutdown 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/16295449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea7807a290 msgr2=0x55ea78084e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea7807a290 0x55ea78084e30 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc097d20 tx=0x7fd3bc01acd0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/16295449 shutdown_connections 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea780c45b0 0x55ea780858b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 0x55ea78085370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55ea78086180 0x55ea780d5410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/16295449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea7807a290 0x55ea78084e30 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3afe00640 0 client.8586 ms_handle_reset on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/16295449 >> 172.21.15.46:0/16295449 conn(0x55ea78076240 msgr2=0x55ea78078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/16295449 shutdown_connections 2024-04-25T08:51:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/16295449 wait complete. 2024-04-25T08:51:03.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.359+0000 7fd3d4e4a000 20 client.8586 trim_cache size 0 max 16384 2024-04-25T08:51:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2ff800640 1 Processor -- start 2024-04-25T08:51:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2ff800640 1 -- start start 2024-04-25T08:51:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2ff800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff2f8142100 0x7ff2f80cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2ff800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2f806e2e0 0x7ff2f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2ff800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f80cd1c0 con 0x7ff2f80b1ed0 2024-04-25T08:51:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2ff800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f80cd4a0 con 0x7ff2f806e2e0 2024-04-25T08:51:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2ff800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f80cd780 con 0x7ff2f8142100 2024-04-25T08:51:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2f7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff2f80b1ed0 0x7ff2f8142730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39522/0 (socket says 172.21.15.46:39522) 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2f7e00640 1 -- 172.21.15.46:0/4201402051 learned_addr learned my addr 172.21.15.46:0/4201402051 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2fd800640 1 --2- 172.21.15.46:0/4201402051 >> v2:172.21.15.46:3300/0 conn(0x7ff2f806e2e0 0x7ff2f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2fce00640 1 --2- 172.21.15.46:0/4201402051 >> v2:172.21.15.82:3300/0 conn(0x7ff2f8142100 0x7ff2f80cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2f7400640 1 -- 172.21.15.46:0/4201402051 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 514708430 0 0) 0x7ff2f80cd1c0 con 0x7ff2f80b1ed0 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2f7400640 1 -- 172.21.15.46:0/4201402051 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e0003530 con 0x7ff2f80b1ed0 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2fce00640 1 -- 172.21.15.46:0/4201402051 >> v2:172.21.15.46:3300/0 conn(0x7ff2f806e2e0 msgr2=0x7ff2f806e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2fce00640 1 --2- 172.21.15.46:0/4201402051 >> v2:172.21.15.46:3300/0 conn(0x7ff2f806e2e0 0x7ff2f806e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2fce00640 1 -- 172.21.15.46:0/4201402051 >> v1:172.21.15.46:6789/0 conn(0x7ff2f80b1ed0 legacy=0x7ff2f8142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2fce00640 1 -- 172.21.15.46:0/4201402051 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80cda60 con 0x7ff2f8142100 2024-04-25T08:51:03.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.363+0000 7ff2fd800640 1 --2- 172.21.15.46:0/4201402051 >> v2:172.21.15.46:3300/0 conn(0x7ff2f806e2e0 0x7ff2f806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:51:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2fce00640 1 --2- 172.21.15.46:0/4201402051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2f8142100 0x7ff2f80cc960 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff2f806cff0 tx=0x7ff2ec01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0859e1c068adad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2f7400640 1 -- 172.21.15.46:0/4201402051 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0b0070 con 0x7ff2f8142100 2024-04-25T08:51:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2f7400640 1 -- 172.21.15.46:0/4201402051 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec0ab600 con 0x7ff2f8142100 2024-04-25T08:51:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2f7400640 1 -- 172.21.15.46:0/4201402051 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec0ab9d0 con 0x7ff2f8142100 2024-04-25T08:51:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2ff800640 1 -- 172.21.15.46:0/4201402051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2f8142100 msgr2=0x7ff2f80cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2ff800640 1 --2- 172.21.15.46:0/4201402051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2f8142100 0x7ff2f80cc960 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff2f806cff0 tx=0x7ff2ec01c480 comp rx=0 tx=0).stop 2024-04-25T08:51:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2ff800640 1 -- 172.21.15.46:0/4201402051 shutdown_connections 2024-04-25T08:51:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2ff800640 1 --2- 172.21.15.46:0/4201402051 >> v2:172.21.15.46:3300/0 conn(0x7ff2f806e2e0 0x7ff2f806e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2ff800640 1 --2- 172.21.15.46:0/4201402051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2f8142100 0x7ff2f80cc960 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2ff800640 1 -- 172.21.15.46:0/4201402051 >> 172.21.15.46:0/4201402051 conn(0x7ff2f80b13b0 msgr2=0x7ff2f80b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2ff800640 1 -- 172.21.15.46:0/4201402051 shutdown_connections 2024-04-25T08:51:03.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.367+0000 7ff2ff800640 1 -- 172.21.15.46:0/4201402051 wait complete. 2024-04-25T08:51:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 Processor -- start 2024-04-25T08:51:03.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 -- start start 2024-04-25T08:51:03.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2f806e2e0 0x7ff2f813b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2f80b1ed0 0x7ff2f81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f8142100 0x7ff2f813ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8168560 con 0x7ff2f8142100 2024-04-25T08:51:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f80ccea0 con 0x7ff2f806e2e0 2024-04-25T08:51:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f81686e0 con 0x7ff2f80b1ed0 2024-04-25T08:51:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f8142100 0x7ff2f813ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f8142100 0x7ff2f813ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45262/0 (socket says 172.21.15.46:45262) 2024-04-25T08:51:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fd800640 1 -- 172.21.15.46:0/273785207 learned_addr learned my addr 172.21.15.46:0/273785207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fce00640 1 --2- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2f806e2e0 0x7ff2f813b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2f7e00640 1 --2- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2f80b1ed0 0x7ff2f81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fd800640 1 -- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2f80b1ed0 msgr2=0x7ff2f81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fd800640 1 --2- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff2f80b1ed0 0x7ff2f81393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fd800640 1 -- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2f806e2e0 msgr2=0x7ff2f813b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fd800640 1 --2- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2f806e2e0 0x7ff2f813b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fd800640 1 -- 172.21.15.46:0/273785207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f813b840 con 0x7ff2f8142100 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fce00640 1 --2- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2f806e2e0 0x7ff2f813b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2fd800640 1 --2- 172.21.15.46:0/273785207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2f8142100 0x7ff2f813ead0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000efc0 tx=0x7ff2f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2f4c00640 1 -- 172.21.15.46:0/273785207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0018030 con 0x7ff2f8142100 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2f4c00640 1 -- 172.21.15.46:0/273785207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f000cdf0 con 0x7ff2f8142100 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2f4c00640 1 -- 172.21.15.46:0/273785207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0003bc0 con 0x7ff2f8142100 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 -- 172.21.15.46:0/273785207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f813f3b0 con 0x7ff2f8142100 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 -- 172.21.15.46:0/273785207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f80acf60 con 0x7ff2f8142100 2024-04-25T08:51:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.371+0000 7ff2ff800640 1 -- 172.21.15.46:0/273785207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff2f8069010 con 0x7ff2f8142100 2024-04-25T08:51:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.375+0000 7ff2f4c00640 1 -- 172.21.15.46:0/273785207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2f001b070 con 0x7ff2f8142100 2024-04-25T08:51:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.375+0000 7ff2f4c00640 1 --2- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff2d8066f60 0x7ff2d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.375+0000 7ff2f4c00640 1 -- 172.21.15.46:0/273785207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff2f0086650 con 0x7ff2f8142100 2024-04-25T08:51:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.375+0000 7ff2f4c00640 1 -- 172.21.15.46:0/273785207 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+511 (secure 0 0 0) 0x7ff2f0022070 con 0x7ff2f8142100 2024-04-25T08:51:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.375+0000 7ff2ff800640 1 -- 172.21.15.46:0/273785207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ff2f80739c0 con 0x7ff2f8142100 2024-04-25T08:51:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.375+0000 7ff2fce00640 1 --2- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff2d8066f60 0x7ff2d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.375+0000 7ff2fce00640 1 --2- 172.21.15.46:0/273785207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff2d8066f60 0x7ff2d8069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec0ad000 tx=0x7ff2ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 Processor -- start 2024-04-25T08:51:03.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- start start 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea780d7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 0x55ea781252d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc1fee70 con 0x55ea780c3c90 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc1fe030 con 0x55ea780c45b0 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc1f30e0 con 0x55ea7807a290 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3c2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3c2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780d7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45264/0 (socket says 172.21.15.46:45264) 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3c2000640 1 -- 172.21.15.46:0/411264239 learned_addr learned my addr 172.21.15.46:0/411264239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3cb400640 1 --2- 172.21.15.46:0/411264239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 0x55ea781252d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3caa00640 1 --2- 172.21.15.46:0/411264239 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea780d7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3c2000640 1 -- 172.21.15.46:0/411264239 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 msgr2=0x55ea780d7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3c2000640 1 --2- 172.21.15.46:0/411264239 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea780d7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3c2000640 1 -- 172.21.15.46:0/411264239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 msgr2=0x55ea781252d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3c2000640 1 --2- 172.21.15.46:0/411264239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 0x55ea781252d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3c2000640 1 -- 172.21.15.46:0/411264239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea77f1fc50 con 0x55ea780c3c90 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3c2000640 1 --2- 172.21.15.46:0/411264239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780d7960 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400b4b0 tx=0x7fd3b4085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b020efedc44ee6a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3baa00640 1 -- 172.21.15.46:0/411264239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40740b0 con 0x55ea780c3c90 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3baa00640 1 -- 172.21.15.46:0/411264239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b4054070 con 0x55ea780c3c90 2024-04-25T08:51:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3baa00640 1 -- 172.21.15.46:0/411264239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4052640 con 0x55ea780c3c90 2024-04-25T08:51:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/411264239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 msgr2=0x55ea780d7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/411264239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780d7960 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400b4b0 tx=0x7fd3b4085040 comp rx=0 tx=0).stop 2024-04-25T08:51:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/411264239 shutdown_connections 2024-04-25T08:51:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/411264239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c45b0 0x55ea781252d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/411264239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c3c90 0x55ea780d7960 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/411264239 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea780d7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/411264239 >> 172.21.15.46:0/411264239 conn(0x55ea78076240 msgr2=0x55ea7807ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/411264239 shutdown_connections 2024-04-25T08:51:03.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/411264239 wait complete. 2024-04-25T08:51:03.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.379+0000 7fd3d4e4a000 1 -- start start 2024-04-25T08:51:03.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 Processor -- start 2024-04-25T08:51:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea780d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 0x55ea780dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c45b0 0x55ea780e0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea77ec4110 con 0x55ea780c45b0 2024-04-25T08:51:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea77e7ce30 con 0x55ea780c3c90 2024-04-25T08:51:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea77e6abd0 con 0x55ea7807a290 2024-04-25T08:51:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 10 client.0 _ll_register_callbacks cb 0x7fd3bc0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:03.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c45b0 0x55ea780e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c45b0 0x55ea780e0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45270/0 (socket says 172.21.15.46:45270) 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3cb400640 1 -- 172.21.15.46:0/2709790818 learned_addr learned my addr 172.21.15.46:0/2709790818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3caa00640 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea780d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3c2000640 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 0x55ea780dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3cb400640 1 -- 172.21.15.46:0/2709790818 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 msgr2=0x55ea780d7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3cb400640 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea780d7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3cb400640 1 -- 172.21.15.46:0/2709790818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 msgr2=0x55ea780dfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3cb400640 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 0x55ea780dfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3cb400640 1 -- 172.21.15.46:0/2709790818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea77e90120 con 0x55ea780c45b0 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3cb400640 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c45b0 0x55ea780e0420 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4004660 tx=0x7fd3c4018790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51314aa63173dfdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3afe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3afe00640 1 -- 172.21.15.46:0/2709790818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4018e20 con 0x55ea780c45b0 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3afe00640 1 -- 172.21.15.46:0/2709790818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd3c4004750 con 0x55ea780c45b0 2024-04-25T08:51:03.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2709790818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3b4054070 con 0x55ea780c45b0 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3afe00640 1 -- 172.21.15.46:0/2709790818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3c4019430 con 0x55ea780c45b0 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 10 client.8601 fetch_fsmap learned FSMap version 83 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2709790818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55ea77e6f170 con 0x55ea780c45b0 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3afe00640 1 -- 172.21.15.46:0/2709790818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b4054070 con 0x55ea780c45b0 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3afe00640 1 -- 172.21.15.46:0/2709790818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd3c402a050 con 0x55ea780c45b0 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 10 client.8601 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 10 client.8601 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 20 client.8601 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55ea78124170 0x55ea78124550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 4 client.8601 mds_command: new command op to 8334 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2709790818 --> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55ea77ed55b0 con 0x55ea78124170 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3caa00640 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55ea78124170 0x55ea78124550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3caa00640 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55ea78124170 0x55ea78124550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9c052b3bc13dac server_cookie=eaf8851b97f74f9b in_seq=0 out_seq=0 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.383+0000 7fd3afe00640 10 client.8601 ms_handle_connect on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:03.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3afe00640 1 -- 172.21.15.46:0/2709790818 <== mds.0 v2:172.21.15.46:6834/623970467 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ea77ed55b0 con 0x55ea78124170 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3afe00640 10 client.8601 handle_command_reply: tid=0 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 client.8601 shutdown 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2709790818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c45b0 msgr2=0x55ea780e0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c45b0 0x55ea780e0420 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd3c4004660 tx=0x7fd3c4018790 comp rx=0 tx=0).stop 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2709790818 shutdown_connections 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55ea78124170 0x55ea78124550 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ea780c45b0 0x55ea780e0420 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea780c3c90 0x55ea780dfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 --2- 172.21.15.46:0/2709790818 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ea7807a290 0x55ea780d7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3afe00640 0 client.8601 ms_handle_reset on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2709790818 >> 172.21.15.46:0/2709790818 conn(0x55ea78076240 msgr2=0x55ea78078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2709790818 shutdown_connections 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 1 -- 172.21.15.46:0/2709790818 wait complete. 2024-04-25T08:51:03.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d4e4a000 20 client.8601 trim_cache size 0 max 16384 2024-04-25T08:51:03.382 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:51:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:51:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "3daa2a67-f559-4319-abf2-72ee8fb83a0e", 2024-04-25T08:51:03.382 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:51:03.382 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 -- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3a8066f60 msgr2=0x7fd3a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3a8066f60 0x7fd3a8069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0ad000 tx=0x7fd3bc008040 comp rx=0 tx=0).stop 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 -- 172.21.15.46:0/1592515512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 msgr2=0x7fd3cc099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 0x7fd3cc099a00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400d830 tx=0x7fd3b400dcf0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 -- 172.21.15.46:0/1592515512 shutdown_connections 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd3a8066f60 0x7fd3a8069420 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc15c130 0x7fd3cc09c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3cc15b810 0x7fd3cc099a00 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 --2- 172.21.15.46:0/1592515512 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd3cc06efd0 0x7fd3cc1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 -- 172.21.15.46:0/1592515512 >> 172.21.15.46:0/1592515512 conn(0x7fd3cc071a40 msgr2=0x7fd3cc06d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 -- 172.21.15.46:0/1592515512 shutdown_connections 2024-04-25T08:51:03.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.387+0000 7fd3d1a00640 1 -- 172.21.15.46:0/1592515512 wait complete. 2024-04-25T08:51:03.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:51:03.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T08:51:03.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.519+0000 7ff2f4c00640 1 -- 172.21.15.46:0/273785207 <== mon.0 v2:172.21.15.46:3300/0 7 ==== log(1 entries from seq 1156 at 2024-04-25T08:51:02.688094+0000) v1 ==== 322+0+0 (secure 0 0 0) 0x7ff2f0054e60 con 0x7ff2f8142100 2024-04-25T08:51:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9172e00640 1 Processor -- start 2024-04-25T08:51:03.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9172e00640 1 -- start start 2024-04-25T08:51:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9172e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 0x7f916c160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9172e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f916c09a040 0x7f916c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9172e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0cc530 con 0x7f916c099c60 2024-04-25T08:51:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9172e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c0cc300 con 0x7f916c09a040 2024-04-25T08:51:03.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9172e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0cc810 con 0x7f916c0cf210 2024-04-25T08:51:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9170e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f916c09a040 0x7f916c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9170e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f916c09a040 0x7f916c15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45282/0 (socket says 172.21.15.46:45282) 2024-04-25T08:51:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9170e00640 1 -- 172.21.15.46:0/3900709511 learned_addr learned my addr 172.21.15.46:0/3900709511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9163e00640 1 --2- 172.21.15.46:0/3900709511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 0x7f916c160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f916be00640 1 --1- 172.21.15.46:0/3900709511 >> v1:172.21.15.82:6789/0 conn(0x7f916c099c60 0x7f916c0cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:40926/0 (socket says 172.21.15.46:40926) 2024-04-25T08:51:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9170e00640 1 -- 172.21.15.46:0/3900709511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 msgr2=0x7f916c160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9170e00640 1 --2- 172.21.15.46:0/3900709511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 0x7f916c160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9170e00640 1 -- 172.21.15.46:0/3900709511 >> v1:172.21.15.82:6789/0 conn(0x7f916c099c60 legacy=0x7f916c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.555+0000 7f9170e00640 1 -- 172.21.15.46:0/3900709511 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0ccaf0 con 0x7f916c09a040 2024-04-25T08:51:03.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f9170e00640 1 --2- 172.21.15.46:0/3900709511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c09a040 0x7f916c15eb60 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9164009e50 tx=0x7f916401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67779dda1e5be164 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f916b400640 1 -- 172.21.15.46:0/3900709511 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640a5810 con 0x7f916c09a040 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f916b400640 1 -- 172.21.15.46:0/3900709511 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91640a59b0 con 0x7f916c09a040 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f916b400640 1 -- 172.21.15.46:0/3900709511 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640a5d80 con 0x7f916c09a040 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f9172e00640 1 -- 172.21.15.46:0/3900709511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c09a040 msgr2=0x7f916c15eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f9172e00640 1 --2- 172.21.15.46:0/3900709511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c09a040 0x7f916c15eb60 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9164009e50 tx=0x7f916401ca30 comp rx=0 tx=0).stop 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f9172e00640 1 -- 172.21.15.46:0/3900709511 shutdown_connections 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f9172e00640 1 --2- 172.21.15.46:0/3900709511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c09a040 0x7f916c15eb60 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f9172e00640 1 --2- 172.21.15.46:0/3900709511 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 0x7f916c160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f9172e00640 1 -- 172.21.15.46:0/3900709511 >> 172.21.15.46:0/3900709511 conn(0x7f916c0b3130 msgr2=0x7f916c0b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f9172e00640 1 -- 172.21.15.46:0/3900709511 shutdown_connections 2024-04-25T08:51:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.559+0000 7f9172e00640 1 -- 172.21.15.46:0/3900709511 wait complete. 2024-04-25T08:51:03.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 Processor -- start 2024-04-25T08:51:03.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 -- start start 2024-04-25T08:51:03.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c099c60 0x7f916c0c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c09a040 0x7f916c1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 0x7f916c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b1930 con 0x7f916c099c60 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b17b0 con 0x7f916c09a040 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b1ab0 con 0x7f916c0cf210 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f916be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c099c60 0x7f916c0c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f916be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c099c60 0x7f916c0c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45288/0 (socket says 172.21.15.46:45288) 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f916be00640 1 -- 172.21.15.46:0/3824929599 learned_addr learned my addr 172.21.15.46:0/3824929599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9163e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c09a040 0x7f916c1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9170e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 0x7f916c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f916be00640 1 -- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 msgr2=0x7f916c1576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f916be00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 0x7f916c1576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f916be00640 1 -- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c09a040 msgr2=0x7f916c1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f916be00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c09a040 0x7f916c1571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f916be00640 1 -- 172.21.15.46:0/3824929599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0b5c50 con 0x7f916c099c60 2024-04-25T08:51:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f916be00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c099c60 0x7f916c0c6e90 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c370 tx=0x7f915c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9168c00640 1 -- 172.21.15.46:0/3824929599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c0074e0 con 0x7f916c099c60 2024-04-25T08:51:03.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9168c00640 1 -- 172.21.15.46:0/3824929599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c007680 con 0x7f916c099c60 2024-04-25T08:51:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9168c00640 1 -- 172.21.15.46:0/3824929599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c004a30 con 0x7f916c099c60 2024-04-25T08:51:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 -- 172.21.15.46:0/3824929599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0b5f30 con 0x7f916c099c60 2024-04-25T08:51:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.563+0000 7f9172e00640 1 -- 172.21.15.46:0/3824929599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0b64f0 con 0x7f916c099c60 2024-04-25T08:51:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.567+0000 7f9168c00640 1 -- 172.21.15.46:0/3824929599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f915c01a020 con 0x7f916c099c60 2024-04-25T08:51:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.567+0000 7f9168c00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9148066f60 0x7f9148069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.567+0000 7f9168c00640 1 -- 172.21.15.46:0/3824929599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f915c084850 con 0x7f916c099c60 2024-04-25T08:51:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.567+0000 7f917626d000 1 Processor -- start 2024-04-25T08:51:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.567+0000 7f9163e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9148066f60 0x7f9148069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.567+0000 7f917626d000 1 -- start start 2024-04-25T08:51:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f9163e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9148066f60 0x7f9148069420 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f916c1f9de0 tx=0x7f9154009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcda99d0 0x55c7fcda9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcd5fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 0x55c7fcd60480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcbacff0 con 0x55c7fcda99d0 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcb35150 con 0x55c7fcdaa2f0 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcba9090 con 0x55c7fcd600a0 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f916be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcda99d0 0x55c7fcda9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f916be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcda99d0 0x55c7fcda9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45296/0 (socket says 172.21.15.46:45296) 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f916be00640 1 -- 172.21.15.46:0/548294174 learned_addr learned my addr 172.21.15.46:0/548294174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f9163e00640 1 --2- 172.21.15.46:0/548294174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcd5fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f9170e00640 1 --2- 172.21.15.46:0/548294174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 0x55c7fcd60480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f916be00640 1 -- 172.21.15.46:0/548294174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 msgr2=0x55c7fcd60480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f916be00640 1 --2- 172.21.15.46:0/548294174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 0x55c7fcd60480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f916be00640 1 -- 172.21.15.46:0/548294174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 msgr2=0x55c7fcd5fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f916be00640 1 --2- 172.21.15.46:0/548294174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcd5fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f916be00640 1 -- 172.21.15.46:0/548294174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7fcd430d0 con 0x55c7fcda99d0 2024-04-25T08:51:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f916be00640 1 --2- 172.21.15.46:0/548294174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcda99d0 0x55c7fcda9db0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f915c054fe0 tx=0x7f915c0551d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15bf41b52826c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f9160c00640 1 -- 172.21.15.46:0/548294174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c0516c0 con 0x55c7fcda99d0 2024-04-25T08:51:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f9160c00640 1 -- 172.21.15.46:0/548294174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c0552b0 con 0x55c7fcda99d0 2024-04-25T08:51:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f9160c00640 1 -- 172.21.15.46:0/548294174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c051860 con 0x55c7fcda99d0 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 -- 172.21.15.46:0/548294174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcda99d0 msgr2=0x55c7fcda9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 --2- 172.21.15.46:0/548294174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcda99d0 0x55c7fcda9db0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f915c054fe0 tx=0x7f915c0551d0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 -- 172.21.15.46:0/548294174 shutdown_connections 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 --2- 172.21.15.46:0/548294174 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 0x55c7fcd60480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 --2- 172.21.15.46:0/548294174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcd5fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 --2- 172.21.15.46:0/548294174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcda99d0 0x55c7fcda9db0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 -- 172.21.15.46:0/548294174 >> 172.21.15.46:0/548294174 conn(0x55c7fcd5c020 msgr2=0x55c7fcd5e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 -- 172.21.15.46:0/548294174 shutdown_connections 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 -- 172.21.15.46:0/548294174 wait complete. 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 -- start start 2024-04-25T08:51:03.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.571+0000 7f917626d000 1 Processor -- start 2024-04-25T08:51:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcd600a0 0x55c7fcd8e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 0x55c7fcd8ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcd8f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcb71250 con 0x55c7fcdaa2f0 2024-04-25T08:51:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcb67690 con 0x55c7fcd600a0 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcb35a40 con 0x55c7fcda99d0 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 10 client.0 _ll_register_callbacks cb 0x55c7fcd4ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9170e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcd8f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f916be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcd600a0 0x55c7fcd8e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f916be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcd600a0 0x55c7fcd8e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:51316/0 (socket says 172.21.15.46:51316) 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f916be00640 1 -- 172.21.15.46:0/3583827618 learned_addr learned my addr 172.21.15.46:0/3583827618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9163e00640 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 0x55c7fcd8ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f916be00640 1 -- 172.21.15.46:0/3583827618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 msgr2=0x55c7fcd8ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f916be00640 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 0x55c7fcd8ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f916be00640 1 -- 172.21.15.46:0/3583827618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 msgr2=0x55c7fcd8f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f916be00640 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcd8f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f916be00640 1 -- 172.21.15.46:0/3583827618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7fcc5a1c0 con 0x55c7fcd600a0 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9170e00640 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcd8f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:51:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f916be00640 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcd600a0 0x55c7fcd8e7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f915c054fe0 tx=0x7f915c000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2759de07dc98b054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9159600640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9159600640 1 -- 172.21.15.46:0/3583827618 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c004bd0 con 0x55c7fcd600a0 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9159600640 1 -- 172.21.15.46:0/3583827618 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f915c004d70 con 0x55c7fcd600a0 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 -- 172.21.15.46:0/3583827618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f915c0552b0 con 0x55c7fcd600a0 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9159600640 1 -- 172.21.15.46:0/3583827618 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f915c05d220 con 0x55c7fcd600a0 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 10 client.8140 fetch_fsmap learned FSMap version 83 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 -- 172.21.15.46:0/3583827618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55c7fcc56170 con 0x55c7fcd600a0 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9159600640 1 -- 172.21.15.46:0/3583827618 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c0552b0 con 0x55c7fcd600a0 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9159600640 1 -- 172.21.15.46:0/3583827618 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f915c0544a0 con 0x55c7fcd600a0 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 10 client.8140 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 10 client.8140 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 20 client.8140 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55c7fcd8ff50 0x55c7fcdbbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 4 client.8140 mds_command: new command op to 8334 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f917626d000 1 -- 172.21.15.46:0/3583827618 --> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7fcb48c60 con 0x55c7fcd8ff50 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9163e00640 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55c7fcd8ff50 0x55c7fcdbbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9163e00640 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55c7fcd8ff50 0x55c7fcdbbc90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b54fd21241a1058 server_cookie=863b7bfebda7684c in_seq=0 out_seq=0 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9159600640 10 client.8140 ms_handle_connect on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9159600640 1 -- 172.21.15.46:0/3583827618 <== mds.0 v2:172.21.15.46:6834/623970467 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7fcb48c60 con 0x55c7fcd8ff50 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.575+0000 7f9159600640 10 client.8140 handle_command_reply: tid=0 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 client.8140 shutdown 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 -- 172.21.15.46:0/3583827618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcd600a0 msgr2=0x55c7fcd8e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcd600a0 0x55c7fcd8e7f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f915c054fe0 tx=0x7f915c000ee0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 -- 172.21.15.46:0/3583827618 shutdown_connections 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55c7fcd8ff50 0x55c7fcdbbc90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcd8f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 0x55c7fcd8ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 --2- 172.21.15.46:0/3583827618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcd600a0 0x55c7fcd8e7f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 -- 172.21.15.46:0/3583827618 >> 172.21.15.46:0/3583827618 conn(0x55c7fcd5c020 msgr2=0x55c7fcd5eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f9159600640 0 client.8140 ms_handle_reset on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 -- 172.21.15.46:0/3583827618 shutdown_connections 2024-04-25T08:51:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 1 -- 172.21.15.46:0/3583827618 wait complete. 2024-04-25T08:51:03.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.579+0000 7f917626d000 20 client.8140 trim_cache size 0 max 16384 2024-04-25T08:51:03.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.591+0000 7f917626d000 1 Processor -- start 2024-04-25T08:51:03.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.591+0000 7f917626d000 1 -- start start 2024-04-25T08:51:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 0x55c7fcd5fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcda99d0 0x55c7fcda8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcda9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c2035f0 con 0x55c7fcdaa2f0 2024-04-25T08:51:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c2027f0 con 0x55c7fcda99d0 2024-04-25T08:51:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c6540 con 0x55c7fcd600a0 2024-04-25T08:51:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9170e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcda9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9170e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcda9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45318/0 (socket says 172.21.15.46:45318) 2024-04-25T08:51:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9170e00640 1 -- 172.21.15.46:0/1982309079 learned_addr learned my addr 172.21.15.46:0/1982309079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f916be00640 1 --2- 172.21.15.46:0/1982309079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 0x55c7fcd5fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9163e00640 1 --2- 172.21.15.46:0/1982309079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcda99d0 0x55c7fcda8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9170e00640 1 -- 172.21.15.46:0/1982309079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 msgr2=0x55c7fcd5fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9170e00640 1 --2- 172.21.15.46:0/1982309079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 0x55c7fcd5fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9170e00640 1 -- 172.21.15.46:0/1982309079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcda99d0 msgr2=0x55c7fcda8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9170e00640 1 --2- 172.21.15.46:0/1982309079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcda99d0 0x55c7fcda8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9170e00640 1 -- 172.21.15.46:0/1982309079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7fcbf3de0 con 0x55c7fcdaa2f0 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f9170e00640 1 --2- 172.21.15.46:0/1982309079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcda9040 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9164009e50 tx=0x7f9164004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcdf4b215c28290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f915be00640 1 -- 172.21.15.46:0/1982309079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640a5950 con 0x55c7fcdaa2f0 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f915be00640 1 -- 172.21.15.46:0/1982309079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91640a5af0 con 0x55c7fcdaa2f0 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f915be00640 1 -- 172.21.15.46:0/1982309079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640c1440 con 0x55c7fcdaa2f0 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 -- 172.21.15.46:0/1982309079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 msgr2=0x55c7fcda9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 --2- 172.21.15.46:0/1982309079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcda9040 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9164009e50 tx=0x7f9164004ac0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 -- 172.21.15.46:0/1982309079 shutdown_connections 2024-04-25T08:51:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 --2- 172.21.15.46:0/1982309079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcda9040 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 --2- 172.21.15.46:0/1982309079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcda99d0 0x55c7fcda8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 --2- 172.21.15.46:0/1982309079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcd600a0 0x55c7fcd5fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 -- 172.21.15.46:0/1982309079 >> 172.21.15.46:0/1982309079 conn(0x55c7fcd5c020 msgr2=0x55c7fcda83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 -- 172.21.15.46:0/1982309079 shutdown_connections 2024-04-25T08:51:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 -- 172.21.15.46:0/1982309079 wait complete. 2024-04-25T08:51:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 -- start start 2024-04-25T08:51:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.595+0000 7f917626d000 1 Processor -- start 2024-04-25T08:51:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcd600a0 0x55c7fce0a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 0x55c7fce0a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcdbc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcb47e00 con 0x55c7fcd600a0 2024-04-25T08:51:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c2021f0 con 0x55c7fcdaa2f0 2024-04-25T08:51:03.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7fcbac300 con 0x55c7fcda99d0 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 10 client.0 _ll_register_callbacks cb 0x55c7fcd2b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f916be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcd600a0 0x55c7fce0a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f916be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcd600a0 0x55c7fce0a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45324/0 (socket says 172.21.15.46:45324) 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f916be00640 1 -- 172.21.15.46:0/3328402014 learned_addr learned my addr 172.21.15.46:0/3328402014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9170e00640 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcdbc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9163e00640 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 0x55c7fce0a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f916be00640 1 -- 172.21.15.46:0/3328402014 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 msgr2=0x55c7fce0a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f916be00640 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 0x55c7fce0a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f916be00640 1 -- 172.21.15.46:0/3328402014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 msgr2=0x55c7fcdbc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f916be00640 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcdbc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f916be00640 1 -- 172.21.15.46:0/3328402014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7fcb4e8e0 con 0x55c7fcd600a0 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9170e00640 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcdbc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f916be00640 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcd600a0 0x55c7fce0a350 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f915c055210 tx=0x7f915c054f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415500a161e86bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:03.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 1 -- 172.21.15.46:0/3328402014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00ce60 con 0x55c7fcd600a0 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 1 -- 172.21.15.46:0/3328402014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f915c004dd0 con 0x55c7fcd600a0 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- 172.21.15.46:0/3328402014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91640a5af0 con 0x55c7fcd600a0 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 1 -- 172.21.15.46:0/3328402014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f915c0552b0 con 0x55c7fcd600a0 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 10 client.8619 fetch_fsmap learned FSMap version 83 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- 172.21.15.46:0/3328402014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x55c7fcc068f0 con 0x55c7fcd600a0 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 1 -- 172.21.15.46:0/3328402014 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91640a5af0 con 0x55c7fcd600a0 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 1 -- 172.21.15.46:0/3328402014 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f915c07b070 con 0x55c7fcd600a0 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 10 client.8619 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 10 client.8619 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 20 client.8619 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55c7fcdbcea0 0x55c7fcdbd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 4 client.8619 mds_command: new command op to 8334 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:51:03.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- 172.21.15.46:0/3328402014 --> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c7fcbba530 con 0x55c7fcdbcea0 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9163e00640 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55c7fcdbcea0 0x55c7fcdbd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9163e00640 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55c7fcdbcea0 0x55c7fcdbd2b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c4e8da8c13cbc7 server_cookie=fbc0241498050171 in_seq=0 out_seq=0 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 10 client.8619 ms_handle_connect on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 1 -- 172.21.15.46:0/3328402014 <== mds.0 v2:172.21.15.46:6834/623970467 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55c7fcbba530 con 0x55c7fcdbcea0 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 10 client.8619 handle_command_reply: tid=0 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 client.8619 shutdown 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- 172.21.15.46:0/3328402014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcd600a0 msgr2=0x55c7fce0a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcd600a0 0x55c7fce0a350 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f915c055210 tx=0x7f915c054f50 comp rx=0 tx=0).stop 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- 172.21.15.46:0/3328402014 shutdown_connections 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7fcdaa2f0 0x55c7fcdbc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55c7fcda99d0 0x55c7fce0a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x55c7fcdbcea0 0x55c7fcdbd2b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 --2- 172.21.15.46:0/3328402014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7fcd600a0 0x55c7fce0a350 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- 172.21.15.46:0/3328402014 >> 172.21.15.46:0/3328402014 conn(0x55c7fcd5c020 msgr2=0x55c7fcda8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f9159600640 0 client.8619 ms_handle_reset on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- 172.21.15.46:0/3328402014 shutdown_connections 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 1 -- 172.21.15.46:0/3328402014 wait complete. 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.599+0000 7f917626d000 20 client.8619 trim_cache size 0 max 16384 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stdout: "3daa2a67-f559-4319-abf2-72ee8fb83a0e": { 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "3daa2a67-f559-4319-abf2-72ee8fb83a0e", 2024-04-25T08:51:03.596 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-04-25T08:51:03.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:51:03.597 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:51:03.597 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:51:03.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 -- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9148066f60 msgr2=0x7f9148069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9148066f60 0x7f9148069420 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f916c1f9de0 tx=0x7f9154009190 comp rx=0 tx=0).stop 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 -- 172.21.15.46:0/3824929599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c099c60 msgr2=0x7f916c0c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c099c60 0x7f916c0c6e90 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c370 tx=0x7f915c00c830 comp rx=0 tx=0).stop 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 -- 172.21.15.46:0/3824929599 shutdown_connections 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9148066f60 0x7f9148069420 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f916c0cf210 0x7f916c1576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f916c09a040 0x7f916c1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 --2- 172.21.15.46:0/3824929599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916c099c60 0x7f916c0c6e90 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 -- 172.21.15.46:0/3824929599 >> 172.21.15.46:0/3824929599 conn(0x7f916c0b3130 msgr2=0x7f916c0b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 -- 172.21.15.46:0/3824929599 shutdown_connections 2024-04-25T08:51:03.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:03.603+0000 7f9172e00640 1 -- 172.21.15.46:0/3824929599 wait complete. 2024-04-25T08:51:03.611 INFO:tasks.cephfs.filesystem:scrub status for tag:3daa2a67-f559-4319-abf2-72ee8fb83a0e - {'path': '/', 'tag': '3daa2a67-f559-4319-abf2-72ee8fb83a0e', 'options': 'recursive,repair'} 2024-04-25T08:51:04.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:04.535+0000 7ff2f4c00640 1 -- 172.21.15.46:0/273785207 <== mon.0 v2:172.21.15.46:3300/0 8 ==== log(5 entries from seq 1 at 2024-04-25T08:51:03.387599+0000) v1 ==== 981+0+0 (secure 0 0 0) 0x7ff2f00550c0 con 0x7ff2f8142100 2024-04-25T08:51:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: cluster: 2024-04-25T08:51:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: id: 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:51:04.524 INFO:teuthology.orchestra.run.smithi046.stdout: health: HEALTH_OK 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: services: 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: mgr: x(active, since 9m), standbys: y 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 9m) 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: data: 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: volumes: 1/1 healthy 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: pools: 3 pools, 97 pgs 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: objects: 30 objects, 19 MiB 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: pgs: 97 active+clean 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: io: 2024-04-25T08:51:04.525 INFO:teuthology.orchestra.run.smithi046.stdout: client: 85 B/s rd, 1.5 MiB/s wr, 0 op/s rd, 6 op/s wr 2024-04-25T08:51:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:04.526 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:04.526 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:03.387599+0000 mds.c [INF] scrub queued for path: / 2024-04-25T08:51:04.526 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:03.387601+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T08:51:04.526 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:03.387605+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T08:51:04.526 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:03.388981+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T08:51:09.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:09.543+0000 7ff2f4c00640 1 -- 172.21.15.46:0/273785207 <== mon.0 v2:172.21.15.46:3300/0 9 ==== log(1 entries from seq 5 at 2024-04-25T08:51:07.646461+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff2f0084a60 con 0x7ff2f8142100 2024-04-25T08:51:09.534 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:07.646461+0000 mds.c [INF] scrub summary: idle 2024-04-25T08:51:21.384 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:51:21.388 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:51:33.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:51:33.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T08:51:33.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd56a00640 1 Processor -- start 2024-04-25T08:51:33.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd56a00640 1 -- start start 2024-04-25T08:51:33.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd56a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd501440d0 0x7fcd500cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd56a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcd5006f150 0x7fcd500b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd56a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5015e440 con 0x7fcd50143990 2024-04-25T08:51:33.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd56a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5015ed30 con 0x7fcd5006f150 2024-04-25T08:51:33.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd56a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5015e6e0 con 0x7fcd501440d0 2024-04-25T08:51:33.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcd50143990 0x7fcd5006ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48614/0 (socket says 172.21.15.46:48614) 2024-04-25T08:51:33.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4fe00640 1 -- 172.21.15.46:0/3646855530 learned_addr learned my addr 172.21.15.46:0/3646855530 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:33.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd47400640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd501440d0 0x7fcd500cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd54a00640 1 --2- 172.21.15.46:0/3646855530 >> v2:172.21.15.82:3300/0 conn(0x7fcd5006f150 0x7fcd500b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925708307 0 0) 0x7fcd5015e440 con 0x7fcd50143990 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd34005740 con 0x7fcd50143990 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1196794487 0 0) 0x7fcd34005740 con 0x7fcd50143990 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd501440d0 msgr2=0x7fcd500cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd501440d0 0x7fcd500cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 >> v2:172.21.15.82:3300/0 conn(0x7fcd5006f150 msgr2=0x7fcd500b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 --2- 172.21.15.46:0/3646855530 >> v2:172.21.15.82:3300/0 conn(0x7fcd5006f150 0x7fcd500b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5015e9c0 con 0x7fcd50143990 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fcd40001680 con 0x7fcd50143990 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 >> v1:172.21.15.46:6789/0 conn(0x7fcd50143990 legacy=0x7fcd5006ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34005c40 0x7fcd34006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd34006a20 0x7fcd34006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd501440d0 0x7fcd340073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5015ed30 con 0x7fcd34006a20 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5015e6e0 con 0x7fcd34005c40 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3400dc90 con 0x7fcd501440d0 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd47400640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd34006a20 0x7fcd34006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.774+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34005c40 0x7fcd34006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd54a00640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd501440d0 0x7fcd340073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd47400640 1 -- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd501440d0 msgr2=0x7fcd340073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd47400640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd501440d0 0x7fcd340073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd47400640 1 -- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34005c40 msgr2=0x7fcd34006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd47400640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34005c40 0x7fcd34006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd47400640 1 -- 172.21.15.46:0/3646855530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34005740 con 0x7fcd34006a20 2024-04-25T08:51:33.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd54a00640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd501440d0 0x7fcd340073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:33.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd47400640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd34006a20 0x7fcd34006ea0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009ab0 tx=0x7fcd3801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e8d5562479fc31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380b2070 con 0x7fcd34006a20 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd380a5520 con 0x7fcd34006a20 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd4f400640 1 -- 172.21.15.46:0/3646855530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380096b0 con 0x7fcd34006a20 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd56a00640 1 -- 172.21.15.46:0/3646855530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd34006a20 msgr2=0x7fcd34006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd56a00640 1 --2- 172.21.15.46:0/3646855530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd34006a20 0x7fcd34006ea0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009ab0 tx=0x7fcd3801c9e0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd54a00640 1 -- 172.21.15.46:0/3646855530 reap_dead start 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd56a00640 1 -- 172.21.15.46:0/3646855530 shutdown_connections 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.778+0000 7fcd56a00640 1 -- 172.21.15.46:0/3646855530 >> 172.21.15.46:0/3646855530 conn(0x7fcd5006ac10 msgr2=0x7fcd5006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 -- 172.21.15.46:0/3646855530 shutdown_connections 2024-04-25T08:51:33.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 -- 172.21.15.46:0/3646855530 wait complete. 2024-04-25T08:51:33.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 Processor -- start 2024-04-25T08:51:33.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 -- start start 2024-04-25T08:51:33.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34006a20 0x7fcd50140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd5013ed00 0x7fcd50138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd500cc310 0x7fcd500cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50143ef0 con 0x7fcd5013ed00 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50143d70 con 0x7fcd34006a20 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd56a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500b3470 con 0x7fcd500cc310 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd47400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd5013ed00 0x7fcd50138fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd47400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd5013ed00 0x7fcd50138fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59122/0 (socket says 172.21.15.46:59122) 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd47400640 1 -- 172.21.15.46:0/977430207 learned_addr learned my addr 172.21.15.46:0/977430207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34006a20 0x7fcd50140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd54a00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd500cc310 0x7fcd500cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd47400640 1 -- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd500cc310 msgr2=0x7fcd500cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd47400640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd500cc310 0x7fcd500cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd47400640 1 -- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34006a20 msgr2=0x7fcd50140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd47400640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34006a20 0x7fcd50140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd47400640 1 -- 172.21.15.46:0/977430207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5013f8e0 con 0x7fcd5013ed00 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd54a00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd500cc310 0x7fcd500cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd47400640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd5013ed00 0x7fcd50138fe0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801b0e0 tx=0x7fcd34007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.782+0000 7fcd4cc00640 1 -- 172.21.15.46:0/977430207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380b2040 con 0x7fcd5013ed00 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd4cc00640 1 -- 172.21.15.46:0/977430207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3800f000 con 0x7fcd5013ed00 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd4cc00640 1 -- 172.21.15.46:0/977430207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38004220 con 0x7fcd5013ed00 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd56a00640 1 -- 172.21.15.46:0/977430207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5013fbc0 con 0x7fcd5013ed00 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd56a00640 1 -- 172.21.15.46:0/977430207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd50140180 con 0x7fcd5013ed00 2024-04-25T08:51:33.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd4cc00640 1 -- 172.21.15.46:0/977430207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd38003620 con 0x7fcd5013ed00 2024-04-25T08:51:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd4cc00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd2c066f60 0x7fcd2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd4cc00640 1 -- 172.21.15.46:0/977430207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcd3811e980 con 0x7fcd5013ed00 2024-04-25T08:51:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd2c066f60 0x7fcd2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd59f2e000 1 Processor -- start 2024-04-25T08:51:33.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.786+0000 7fcd59f2e000 1 -- start start 2024-04-25T08:51:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd2c066f60 0x7fcd2c069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcd40001ec0 tx=0x7fcd40001e10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adbe30 0x561e46adc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adc750 0x561e46a91f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a924b0 0x561e46a92890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e468daff0 con 0x561e46a924b0 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e46863150 con 0x561e46adc750 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e468d7090 con 0x561e46adbe30 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd54a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a924b0 0x561e46a92890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd54a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a924b0 0x561e46a92890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59126/0 (socket says 172.21.15.46:59126) 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd54a00640 1 -- 172.21.15.46:0/2217233734 learned_addr learned my addr 172.21.15.46:0/2217233734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd47400640 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adc750 0x561e46a91f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adbe30 0x561e46adc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd54a00640 1 -- 172.21.15.46:0/2217233734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adbe30 msgr2=0x561e46adc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd54a00640 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adbe30 0x561e46adc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd54a00640 1 -- 172.21.15.46:0/2217233734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adc750 msgr2=0x561e46a91f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd54a00640 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adc750 0x561e46a91f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd54a00640 1 -- 172.21.15.46:0/2217233734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e46a710d0 con 0x561e46a924b0 2024-04-25T08:51:33.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd47400640 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adc750 0x561e46a91f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd54a00640 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a924b0 0x561e46a92890 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fcd5006d140 tx=0x7fcd48009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4fee69201cafc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd44c00640 1 -- 172.21.15.46:0/2217233734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4801a070 con 0x561e46a924b0 2024-04-25T08:51:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd44c00640 1 -- 172.21.15.46:0/2217233734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48016830 con 0x561e46a924b0 2024-04-25T08:51:33.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd44c00640 1 -- 172.21.15.46:0/2217233734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48016c00 con 0x561e46a924b0 2024-04-25T08:51:33.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2217233734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a924b0 msgr2=0x561e46a92890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a924b0 0x561e46a92890 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fcd5006d140 tx=0x7fcd48009750 comp rx=0 tx=0).stop 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2217233734 shutdown_connections 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a924b0 0x561e46a92890 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adc750 0x561e46a91f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2217233734 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adbe30 0x561e46adc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2217233734 >> 172.21.15.46:0/2217233734 conn(0x561e46a8a020 msgr2=0x561e46a8c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2217233734 shutdown_connections 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2217233734 wait complete. 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 -- start start 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.790+0000 7fcd59f2e000 1 Processor -- start 2024-04-25T08:51:33.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 0x561e46adb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46a98360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adc750 0x561e46a988a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e468df620 con 0x561e46adbe30 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e46895690 con 0x561e46a924b0 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e46863a40 con 0x561e46adc750 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 10 client.0 _ll_register_callbacks cb 0x561e46a78e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd47400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46a98360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd4fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 0x561e46adb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd47400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46a98360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59140/0 (socket says 172.21.15.46:59140) 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd47400640 1 -- 172.21.15.46:0/1647209326 learned_addr learned my addr 172.21.15.46:0/1647209326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd54a00640 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adc750 0x561e46a988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd47400640 1 -- 172.21.15.46:0/1647209326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adc750 msgr2=0x561e46a988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd47400640 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adc750 0x561e46a988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd47400640 1 -- 172.21.15.46:0/1647209326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 msgr2=0x561e46adb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd47400640 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 0x561e46adb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd47400640 1 -- 172.21.15.46:0/1647209326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e469881c0 con 0x561e46adbe30 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd47400640 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46a98360 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801cf50 tx=0x7fcd380ef0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ce258e58d126c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 1 -- 172.21.15.46:0/1647209326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380043c0 con 0x561e46adbe30 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 1 -- 172.21.15.46:0/1647209326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcd380f72c0 con 0x561e46adbe30 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1647209326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd48016830 con 0x561e46adbe30 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 1 -- 172.21.15.46:0/1647209326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd380f74c0 con 0x561e46adbe30 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 10 client.8631 fetch_fsmap learned FSMap version 83 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1647209326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x561e46984170 con 0x561e46adbe30 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 1 -- 172.21.15.46:0/1647209326 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48016830 con 0x561e46adbe30 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 1 -- 172.21.15.46:0/1647209326 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd380b5030 con 0x561e46adbe30 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 10 client.8631 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 10 client.8631 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:33.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 20 client.8631 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x561e46a99610 0x561e46aee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 4 client.8631 mds_command: new command op to 8334 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:51:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1647209326 --> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e46876c60 con 0x561e46a99610 2024-04-25T08:51:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x561e46a99610 0x561e46aee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x561e46a99610 0x561e46aee0f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1030e541b3ad939 server_cookie=2a93a29cdae4075b in_seq=0 out_seq=0 2024-04-25T08:51:33.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 10 client.8631 ms_handle_connect on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 1 -- 172.21.15.46:0/1647209326 <== mds.0 v2:172.21.15.46:6834/623970467 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e46876c60 con 0x561e46a99610 2024-04-25T08:51:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 10 client.8631 handle_command_reply: tid=0 2024-04-25T08:51:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 client.8631 shutdown 2024-04-25T08:51:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1647209326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 msgr2=0x561e46a98360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46a98360 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801cf50 tx=0x7fcd380ef0f0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1647209326 shutdown_connections 2024-04-25T08:51:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46adc750 0x561e46a988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x561e46a99610 0x561e46aee0f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46a98360 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1647209326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 0x561e46adb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1647209326 >> 172.21.15.46:0/1647209326 conn(0x561e46a8a020 msgr2=0x561e46a93120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd3d600640 0 client.8631 ms_handle_reset on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.794+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1647209326 shutdown_connections 2024-04-25T08:51:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.798+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1647209326 wait complete. 2024-04-25T08:51:33.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.798+0000 7fcd59f2e000 20 client.8631 trim_cache size 0 max 16384 2024-04-25T08:51:33.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.810+0000 7fcd59f2e000 1 Processor -- start 2024-04-25T08:51:33.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.810+0000 7fcd59f2e000 1 -- start start 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.810+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a8a020 0x561e46ae92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.810+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 0x561e46ae97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.810+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46abb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.810+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501ff8b0 con 0x561e46adbe30 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.810+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501fea30 con 0x561e46a924b0 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.810+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501fdf30 con 0x561e46a8a020 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46abb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46abb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59152/0 (socket says 172.21.15.46:59152) 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 -- 172.21.15.46:0/1967994848 learned_addr learned my addr 172.21.15.46:0/1967994848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a8a020 0x561e46ae92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd47400640 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 0x561e46ae97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 -- 172.21.15.46:0/1967994848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a8a020 msgr2=0x561e46ae92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a8a020 0x561e46ae92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 -- 172.21.15.46:0/1967994848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 msgr2=0x561e46ae97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 0x561e46ae97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 -- 172.21.15.46:0/1967994848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e46921de0 con 0x561e46adbe30 2024-04-25T08:51:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd47400640 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 0x561e46ae97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46abb7a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fcd48019000 tx=0x7fcd48010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2655027794d687a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd3fe00640 1 -- 172.21.15.46:0/1967994848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4801a040 con 0x561e46adbe30 2024-04-25T08:51:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd3fe00640 1 -- 172.21.15.46:0/1967994848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48003bb0 con 0x561e46adbe30 2024-04-25T08:51:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd3fe00640 1 -- 172.21.15.46:0/1967994848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48004040 con 0x561e46adbe30 2024-04-25T08:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1967994848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 msgr2=0x561e46abb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46abb7a0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fcd48019000 tx=0x7fcd48010000 comp rx=0 tx=0).stop 2024-04-25T08:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1967994848 shutdown_connections 2024-04-25T08:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46adbe30 0x561e46abb7a0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46a924b0 0x561e46ae97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/1967994848 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a8a020 0x561e46ae92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1967994848 >> 172.21.15.46:0/1967994848 conn(0x561e46a89b00 msgr2=0x561e46ae8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1967994848 shutdown_connections 2024-04-25T08:51:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 -- 172.21.15.46:0/1967994848 wait complete. 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 -- start start 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 Processor -- start 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a8a020 0x561e46aee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a924b0 0x561e46aee630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adbe30 0x561e46aeeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e46875e00 con 0x561e46a8a020 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd501fe430 con 0x561e46adbe30 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e468da300 con 0x561e46a924b0 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd59f2e000 10 client.0 _ll_register_callbacks cb 0x561e46a59040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd4fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a8a020 0x561e46aee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd4fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a8a020 0x561e46aee0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59154/0 (socket says 172.21.15.46:59154) 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd4fe00640 1 -- 172.21.15.46:0/2606483494 learned_addr learned my addr 172.21.15.46:0/2606483494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd54a00640 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adbe30 0x561e46aeeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.814+0000 7fcd47400640 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a924b0 0x561e46aee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd4fe00640 1 -- 172.21.15.46:0/2606483494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a924b0 msgr2=0x561e46aee630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a924b0 0x561e46aee630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd4fe00640 1 -- 172.21.15.46:0/2606483494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adbe30 msgr2=0x561e46aeeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adbe30 0x561e46aeeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd4fe00640 1 -- 172.21.15.46:0/2606483494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e4687c8e0 con 0x561e46a8a020 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd4fe00640 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a8a020 0x561e46aee0f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fcd40015800 tx=0x7fcd40015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51878f15efa57da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 1 -- 172.21.15.46:0/2606483494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40024630 con 0x561e46a8a020 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 1 -- 172.21.15.46:0/2606483494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcd400247d0 con 0x561e46a8a020 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2606483494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd48003bb0 con 0x561e46a8a020 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 1 -- 172.21.15.46:0/2606483494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd40036050 con 0x561e46a8a020 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 10 client.8637 fetch_fsmap learned FSMap version 83 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2606483494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x561e469348f0 con 0x561e46a8a020 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 1 -- 172.21.15.46:0/2606483494 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48003bb0 con 0x561e46a8a020 2024-04-25T08:51:33.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 1 -- 172.21.15.46:0/2606483494 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd40039020 con 0x561e46a8a020 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 10 client.8637 fetch_fsmap finished waiting for FSMap version 83 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 10 client.8637 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 20 client.8637 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x561e46adc750 0x561e46b43fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 4 client.8637 mds_command: new command op to 8334 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2606483494 --> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561e468e8530 con 0x561e46adc750 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd47400640 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x561e46adc750 0x561e46b43fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd47400640 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x561e46adc750 0x561e46b43fe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88c4573001611bb5 server_cookie=a784d27064171f0b in_seq=0 out_seq=0 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 10 client.8637 ms_handle_connect on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 1 -- 172.21.15.46:0/2606483494 <== mds.0 v2:172.21.15.46:6834/623970467 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561e468e8530 con 0x561e46adc750 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 10 client.8637 handle_command_reply: tid=0 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 client.8637 shutdown 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2606483494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a8a020 msgr2=0x561e46aee0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a8a020 0x561e46aee0f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fcd40015800 tx=0x7fcd40015000 comp rx=0 tx=0).stop 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2606483494 shutdown_connections 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561e46adbe30 0x561e46aeeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561e46a924b0 0x561e46aee630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.46:6834/623970467,v1:172.21.15.46:6835/623970467] conn(0x561e46adc750 0x561e46b43fe0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 --2- 172.21.15.46:0/2606483494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561e46a8a020 0x561e46aee0f0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2606483494 >> 172.21.15.46:0/2606483494 conn(0x561e46a89b00 msgr2=0x561e46ae9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd3d600640 0 client.8637 ms_handle_reset on v2:172.21.15.46:6834/623970467 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2606483494 shutdown_connections 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 1 -- 172.21.15.46:0/2606483494 wait complete. 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd59f2e000 20 client.8637 trim_cache size 0 max 16384 2024-04-25T08:51:33.815 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:51:33.816 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:51:33.816 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:51:33.816 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:51:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 -- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd2c066f60 msgr2=0x7fcd2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd2c066f60 0x7fcd2c069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcd40001ec0 tx=0x7fcd40001e10 comp rx=0 tx=0).stop 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 -- 172.21.15.46:0/977430207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd5013ed00 msgr2=0x7fcd50138fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd5013ed00 0x7fcd50138fe0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801b0e0 tx=0x7fcd34007a50 comp rx=0 tx=0).stop 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 -- 172.21.15.46:0/977430207 shutdown_connections 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcd2c066f60 0x7fcd2c069420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fcd500cc310 0x7fcd500cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd5013ed00 0x7fcd50138fe0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 --2- 172.21.15.46:0/977430207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd34006a20 0x7fcd50140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 -- 172.21.15.46:0/977430207 >> 172.21.15.46:0/977430207 conn(0x7fcd5006ac10 msgr2=0x7fcd5015fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 -- 172.21.15.46:0/977430207 shutdown_connections 2024-04-25T08:51:33.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:33.818+0000 7fcd56a00640 1 -- 172.21.15.46:0/977430207 wait complete. 2024-04-25T08:51:33.831 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:51:33.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:51:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf4400640 1 Processor -- start 2024-04-25T08:51:34.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf4400640 1 -- start start 2024-04-25T08:51:34.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf4400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fabec15a750 0x7fabec06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf4400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fabec0cc9e0 0x7fabec06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf4400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec06d5e0 con 0x7fabec0aed10 2024-04-25T08:51:34.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf4400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec15ab30 con 0x7fabec0cc9e0 2024-04-25T08:51:34.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf4400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec0ce4f0 con 0x7fabec15a750 2024-04-25T08:51:34.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf1000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fabec0aed10 0x7fabec0b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48616/0 (socket says 172.21.15.46:48616) 2024-04-25T08:51:34.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf1000640 1 -- 172.21.15.46:0/3118866001 learned_addr learned my addr 172.21.15.46:0/3118866001 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:34.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf2400640 1 --2- 172.21.15.46:0/3118866001 >> v2:172.21.15.46:3300/0 conn(0x7fabec0cc9e0 0x7fabec06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf1a00640 1 --2- 172.21.15.46:0/3118866001 >> v2:172.21.15.82:3300/0 conn(0x7fabec15a750 0x7fabec06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabebe00640 1 -- 172.21.15.46:0/3118866001 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289824927 0 0) 0x7fabec06d5e0 con 0x7fabec0aed10 2024-04-25T08:51:34.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf2400640 1 -- 172.21.15.46:0/3118866001 >> v2:172.21.15.82:3300/0 conn(0x7fabec15a750 msgr2=0x7fabec06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf2400640 1 --2- 172.21.15.46:0/3118866001 >> v2:172.21.15.82:3300/0 conn(0x7fabec15a750 0x7fabec06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf2400640 1 -- 172.21.15.46:0/3118866001 >> v1:172.21.15.46:6789/0 conn(0x7fabec0aed10 legacy=0x7fabec0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf2400640 1 -- 172.21.15.46:0/3118866001 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0ce720 con 0x7fabec0cc9e0 2024-04-25T08:51:34.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.086+0000 7fabf1a00640 1 --2- 172.21.15.46:0/3118866001 >> v2:172.21.15.82:3300/0 conn(0x7fabec15a750 0x7fabec06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:34.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf2400640 1 --2- 172.21.15.46:0/3118866001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cc9e0 0x7fabec06f150 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fabec04ffd0 tx=0x7fabe401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83594ff62bbe5162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabebe00640 1 -- 172.21.15.46:0/3118866001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40b0070 con 0x7fabec0cc9e0 2024-04-25T08:51:34.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabebe00640 1 -- 172.21.15.46:0/3118866001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40ab600 con 0x7fabec0cc9e0 2024-04-25T08:51:34.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabebe00640 1 -- 172.21.15.46:0/3118866001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40ab9d0 con 0x7fabec0cc9e0 2024-04-25T08:51:34.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 -- 172.21.15.46:0/3118866001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cc9e0 msgr2=0x7fabec06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 --2- 172.21.15.46:0/3118866001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cc9e0 0x7fabec06f150 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fabec04ffd0 tx=0x7fabe401c480 comp rx=0 tx=0).stop 2024-04-25T08:51:34.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 -- 172.21.15.46:0/3118866001 shutdown_connections 2024-04-25T08:51:34.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 --2- 172.21.15.46:0/3118866001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cc9e0 0x7fabec06f150 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 --2- 172.21.15.46:0/3118866001 >> v2:172.21.15.82:3300/0 conn(0x7fabec15a750 0x7fabec06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 -- 172.21.15.46:0/3118866001 >> 172.21.15.46:0/3118866001 conn(0x7fabec0b12a0 msgr2=0x7fabec0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 -- 172.21.15.46:0/3118866001 shutdown_connections 2024-04-25T08:51:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 -- 172.21.15.46:0/3118866001 wait complete. 2024-04-25T08:51:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 Processor -- start 2024-04-25T08:51:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.090+0000 7fabf4400640 1 -- start start 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf4400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabec0aed10 0x7fabec144a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf4400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fabec0cc9e0 0x7fabec1585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf4400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec15a750 0x7fabec156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf4400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec167230 con 0x7fabec15a750 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf4400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec06d270 con 0x7fabec0aed10 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf4400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec1673b0 con 0x7fabec0cc9e0 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec15a750 0x7fabec156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec15a750 0x7fabec156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59170/0 (socket says 172.21.15.46:59170) 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf2400640 1 -- 172.21.15.46:0/2174727926 learned_addr learned my addr 172.21.15.46:0/2174727926 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf1000640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fabec0cc9e0 0x7fabec1585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf1a00640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabec0aed10 0x7fabec144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf2400640 1 -- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fabec0cc9e0 msgr2=0x7fabec1585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf2400640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fabec0cc9e0 0x7fabec1585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf2400640 1 -- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabec0aed10 msgr2=0x7fabec144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf2400640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabec0aed10 0x7fabec144a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf2400640 1 -- 172.21.15.46:0/2174727926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0b3d30 con 0x7fabec15a750 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf2400640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec15a750 0x7fabec156590 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fabe40ae040 tx=0x7fabe401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabe9600640 1 -- 172.21.15.46:0/2174727926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40bc070 con 0x7fabec15a750 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabe9600640 1 -- 172.21.15.46:0/2174727926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40b1040 con 0x7fabec15a750 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec1574c0 con 0x7fabec15a750 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabe9600640 1 -- 172.21.15.46:0/2174727926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40b0050 con 0x7fabec15a750 2024-04-25T08:51:34.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec1571e0 con 0x7fabec15a750 2024-04-25T08:51:34.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.094+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec069080 con 0x7fabec15a750 2024-04-25T08:51:34.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.098+0000 7fabe9600640 1 -- 172.21.15.46:0/2174727926 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fabe40bf070 con 0x7fabec15a750 2024-04-25T08:51:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.098+0000 7fabe9600640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fabcc066f60 0x7fabcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.098+0000 7fabe9600640 1 -- 172.21.15.46:0/2174727926 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fabe411f0c0 con 0x7fabec15a750 2024-04-25T08:51:34.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.098+0000 7fabf1a00640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fabcc066f60 0x7fabcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.098+0000 7fabf1a00640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fabcc066f60 0x7fabcc069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fabec1f9be0 tx=0x7fabe0003a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.102+0000 7fabe9600640 1 -- 172.21.15.46:0/2174727926 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fabe4016070 con 0x7fabec15a750 2024-04-25T08:51:34.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.294+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fabec0b0ac0 con 0x7fabec15a750 2024-04-25T08:51:34.288 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:51:34.298+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:51:34.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.298+0000 7fabe9600640 1 -- 172.21.15.46:0/2174727926 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7fabe40eff90 con 0x7fabec15a750 2024-04-25T08:51:34.290 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:51:34.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fabcc066f60 msgr2=0x7fabcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fabcc066f60 0x7fabcc069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fabec1f9be0 tx=0x7fabe0003a20 comp rx=0 tx=0).stop 2024-04-25T08:51:34.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec15a750 msgr2=0x7fabec156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec15a750 0x7fabec156590 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fabe40ae040 tx=0x7fabe401acc0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 shutdown_connections 2024-04-25T08:51:34.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fabcc066f60 0x7fabcc069420 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec15a750 0x7fabec156590 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fabec0cc9e0 0x7fabec1585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 --2- 172.21.15.46:0/2174727926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabec0aed10 0x7fabec144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 >> 172.21.15.46:0/2174727926 conn(0x7fabec0b12a0 msgr2=0x7fabec155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 shutdown_connections 2024-04-25T08:51:34.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.302+0000 7fabf4400640 1 -- 172.21.15.46:0/2174727926 wait complete. 2024-04-25T08:51:34.304 INFO:tasks.ceph.mds.c.smithi046.stderr: -6881> 2024-04-25T08:50:53.439+0000 7f465c000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:51:34.314 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-25T08:51:34.355 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:51:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f928902d400 1 -- start start 2024-04-25T08:51:34.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f928902d400 1 Processor -- start 2024-04-25T08:51:34.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f928902d400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea79a5800 0x559ea7b5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f928902d400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea79a5400 0x559ea7b5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f928902d400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea79a5000 0x559ea7b5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f928902d400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea876a180 con 0x559ea79a5800 2024-04-25T08:51:34.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f928902d400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea876a300 con 0x559ea79a5400 2024-04-25T08:51:34.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f928902d400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ea876a480 con 0x559ea79a5000 2024-04-25T08:51:34.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9287000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea79a5800 0x559ea7b5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9287000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea79a5800 0x559ea7b5e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59208/0 (socket says 172.21.15.46:59208) 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9287000640 1 -- 172.21.15.46:0/3168300207 learned_addr learned my addr 172.21.15.46:0/3168300207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9287a00640 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea79a5000 0x559ea7b5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9286600640 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea79a5400 0x559ea7b5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9287000640 1 -- 172.21.15.46:0/3168300207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea79a5000 msgr2=0x559ea7b5f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9287000640 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea79a5000 0x559ea7b5f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9287000640 1 -- 172.21.15.46:0/3168300207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea79a5400 msgr2=0x559ea7b5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9287000640 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea79a5400 0x559ea7b5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9287000640 1 -- 172.21.15.46:0/3168300207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ea7a46960 con 0x559ea79a5800 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.370+0000 7f9286600640 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea79a5400 0x559ea7b5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:34.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9287000640 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea79a5800 0x559ea7b5e580 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x559ea880d380 tx=0x559ea7a671a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ff102048ff1d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9283e00640 1 -- 172.21.15.46:0/3168300207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ea79da4e0 con 0x559ea79a5800 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9283e00640 1 -- 172.21.15.46:0/3168300207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ea79ee1c0 con 0x559ea79a5800 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9283e00640 1 -- 172.21.15.46:0/3168300207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ea79dab60 con 0x559ea79a5800 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 -- 172.21.15.46:0/3168300207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ea7a46d20 con 0x559ea79a5800 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 -- 172.21.15.46:0/3168300207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559ea7a46f00 con 0x559ea79a5800 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 4 waiting for MDS map... 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9283e00640 1 -- 172.21.15.46:0/3168300207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x559ea88c2600 con 0x559ea79a5800 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9283e00640 1 -- 172.21.15.46:0/3168300207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 84) v1 ==== 2723+0+0 (secure 0 0 0) 0x559ea79fb180 con 0x559ea79a5800 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 4 Got MDS map 84 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 10 main: JournalTool::main 2024-04-25T08:51:34.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 4 main: JournalTool: connecting to RADOS... 2024-04-25T08:51:34.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 Processor -- start 2024-04-25T08:51:34.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 -- start start 2024-04-25T08:51:34.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea88bcc00 0x559ea7b5f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bd000 0x559ea7b5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea876a600 con 0x559ea88bd400 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea876a780 con 0x559ea88bcc00 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ea876a900 con 0x559ea88bd000 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9286600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9286600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59216/0 (socket says 172.21.15.46:59216) 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9286600640 1 -- 172.21.15.46:0/2961788969 learned_addr learned my addr 172.21.15.46:0/2961788969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9287000640 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bd000 0x559ea7b5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9287a00640 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea88bcc00 0x559ea7b5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9286600640 1 -- 172.21.15.46:0/2961788969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bd000 msgr2=0x559ea7b5fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9286600640 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bd000 0x559ea7b5fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9286600640 1 -- 172.21.15.46:0/2961788969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea88bcc00 msgr2=0x559ea7b5f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9286600640 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea88bcc00 0x559ea7b5f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9286600640 1 -- 172.21.15.46:0/2961788969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ea7a46d20 con 0x559ea88bd400 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9287000640 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bd000 0x559ea7b5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9286600640 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60100 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x559ea880c9f0 tx=0x559ea7a7c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f15a360aedc4efd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9281600640 1 -- 172.21.15.46:0/2961788969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ea79da9c0 con 0x559ea88bd400 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9281600640 1 -- 172.21.15.46:0/2961788969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ea79eec40 con 0x559ea88bd400 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f9281600640 1 -- 172.21.15.46:0/2961788969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ea79da4e0 con 0x559ea88bd400 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 -- 172.21.15.46:0/2961788969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 msgr2=0x559ea7b60100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60100 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x559ea880c9f0 tx=0x559ea7a7c000 comp rx=0 tx=0).stop 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 -- 172.21.15.46:0/2961788969 shutdown_connections 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60100 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bd000 0x559ea7b5fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 --2- 172.21.15.46:0/2961788969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea88bcc00 0x559ea7b5f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.374+0000 7f928902d400 1 -- 172.21.15.46:0/2961788969 >> 172.21.15.46:0/2961788969 conn(0x559ea79a4c00 msgr2=0x559ea79d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 -- 172.21.15.46:0/2961788969 shutdown_connections 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 -- 172.21.15.46:0/2961788969 wait complete. 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 Processor -- start 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 -- start start 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bcc00 0x559ea7b61180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea88bd000 0x559ea7b60c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea876bb00 con 0x559ea88bd400 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ea876b980 con 0x559ea88bd000 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ea876b800 con 0x559ea88bcc00 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9286600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9286600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59230/0 (socket says 172.21.15.46:59230) 2024-04-25T08:51:34.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9286600640 1 -- 172.21.15.46:0/177374085 learned_addr learned my addr 172.21.15.46:0/177374085 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9287000640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea88bd000 0x559ea7b60c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9287a00640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bcc00 0x559ea7b61180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9286600640 1 -- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bcc00 msgr2=0x559ea7b61180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9286600640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea88bcc00 0x559ea7b61180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9286600640 1 -- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea88bd000 msgr2=0x559ea7b60c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9286600640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea88bd000 0x559ea7b60c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9286600640 1 -- 172.21.15.46:0/177374085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ea7a47a40 con 0x559ea88bd400 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f9286600640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60680 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x559ea88ba810 tx=0x559ea7a7c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f927f800640 1 -- 172.21.15.46:0/177374085 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ea79dab60 con 0x559ea88bd400 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f927f800640 1 -- 172.21.15.46:0/177374085 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ea79eea80 con 0x559ea88bd400 2024-04-25T08:51:34.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f927f800640 1 -- 172.21.15.46:0/177374085 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ea79da000 con 0x559ea88bd400 2024-04-25T08:51:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559ea7a47c20 con 0x559ea88bd400 2024-04-25T08:51:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ea7a461e0 con 0x559ea88bd400 2024-04-25T08:51:34.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.378+0000 7f928902d400 4 main: JournalTool: resolving pool 15 2024-04-25T08:51:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f927f800640 1 -- 172.21.15.46:0/177374085 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x559ea892cc00 con 0x559ea88bd400 2024-04-25T08:51:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f927f800640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x559ea7dbe000 0x559ea7b61700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f927f800640 1 -- 172.21.15.46:0/177374085 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x559ea7cde600 con 0x559ea88bd400 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 4 main: JournalTool: creating IoCtx.. 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 4 main: Executing for rank 0 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 4 Filtering by inode '1099511627777' 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x559ea88bd800 0x559ea7b60100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x559ea88bdc00 con 0x559ea88bd800 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f9287a00640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x559ea7dbe000 0x559ea7b61700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f9287000640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x559ea88bd800 0x559ea7b60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f9287000640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x559ea88bd800 0x559ea7b60100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f9287a00640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x559ea7dbe000 0x559ea7b61700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x559ea7ab27e0 tx=0x559ea88ddf20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f9287000640 1 -- 172.21.15.46:0/177374085 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x559ea7b20000 con 0x559ea88bd800 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 4 Pointer 400.00000000 is readable 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x559ea7dbf000 con 0x559ea88bd800 2024-04-25T08:51:34.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f9287000640 1 -- 172.21.15.46:0/177374085 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x559ea7b20000 con 0x559ea88bd800 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 10 Header 0x400000 0x407953 0x407ccf 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 10 Starting journal scan from offset 0x407953 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 31059~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x559ea7dbf400 con 0x559ea88bd800 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f9287000640 1 -- 172.21.15.46:0/177374085 <== osd.6 v2:172.21.15.82:6823/1178952145 3 ==== osd_op_reply(3 200.00000001 [read 31059~892 out=892b] v0'0 uv29 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x559ea7b20000 con 0x559ea88bd800 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 10 Parsing data, 0x37c bytes available 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 10 Attempting decode at 0x407953 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 10 Consumed 0x37c bytes 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 10 Valid entry at 0x407953 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x559ea7dbf800 0x559ea7b5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.382+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x559ea7dbfc00 con 0x559ea7dbf800 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f9287a00640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x559ea7dbf800 0x559ea7b5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f9287a00640 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x559ea7dbf800 0x559ea7b5fb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f9287a00640 1 -- 172.21.15.46:0/177374085 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x559ea7b20500 con 0x559ea7dbf800 2024-04-25T08:51:34.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 4 Reached end of journal objects 2024-04-25T08:51:34.381 INFO:teuthology.orchestra.run.smithi046.stdout:Events by type: 2024-04-25T08:51:34.381 INFO:teuthology.orchestra.run.smithi046.stdout:Errors: 0 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 4 Scanned objects, 0 missing, 1 valid 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 4 Events scanned, 0 gaps 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 4 Found 1 valid events 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 4 Selected 0 events events for processing 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 4 0 events 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x559ea88bd800 msgr2=0x559ea7b60100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x559ea88bd800 0x559ea7b60100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x559ea7dbf800 msgr2=0x559ea7b5fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x559ea7dbf800 0x559ea7b5fb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x559ea7dbe000 msgr2=0x559ea7b61700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x559ea7dbe000 0x559ea7b61700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x559ea7ab27e0 tx=0x559ea88ddf20 comp rx=0 tx=0).stop 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f9286600640 1 -- 172.21.15.46:0/177374085 reap_dead start 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 msgr2=0x559ea7b60680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60680 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x559ea88ba810 tx=0x559ea7a7c3c0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 shutdown_connections 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 --2- 172.21.15.46:0/177374085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea88bd400 0x559ea7b60680 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 >> 172.21.15.46:0/177374085 conn(0x559ea79a4c00 msgr2=0x559ea79d3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 shutdown_connections 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.386+0000 7f928902d400 1 -- 172.21.15.46:0/177374085 wait complete. 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.390+0000 7f928902d400 1 -- 172.21.15.46:0/3168300207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea79a5800 msgr2=0x559ea7b5e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.390+0000 7f928902d400 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea79a5800 0x559ea7b5e580 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x559ea880d380 tx=0x559ea7a671a0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.390+0000 7f928902d400 1 -- 172.21.15.46:0/3168300207 shutdown_connections 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.390+0000 7f928902d400 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559ea79a5000 0x559ea7b5f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.390+0000 7f928902d400 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ea79a5400 0x559ea7b5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.390+0000 7f928902d400 1 --2- 172.21.15.46:0/3168300207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559ea79a5800 0x559ea7b5e580 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.390+0000 7f928902d400 1 -- 172.21.15.46:0/3168300207 >> 172.21.15.46:0/3168300207 conn(0x559ea79a5c00 msgr2=0x559ea79d3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.390+0000 7f928902d400 1 -- 172.21.15.46:0/3168300207 shutdown_connections 2024-04-25T08:51:34.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.390+0000 7f928902d400 1 -- 172.21.15.46:0/3168300207 wait complete. 2024-04-25T08:51:34.386 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.071845, result: Events by type: Errors: 0 2024-04-25T08:51:34.386 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-25T08:51:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 -- start start 2024-04-25T08:51:34.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 Processor -- start 2024-04-25T08:51:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331fb7f800 0x56331fd38580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331fb7f400 0x56331fd38b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56331fb7f000 0x56331fd39080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563320952180 con 0x56331fb7f800 2024-04-25T08:51:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563320952300 con 0x56331fb7f400 2024-04-25T08:51:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563320952480 con 0x56331fb7f000 2024-04-25T08:51:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331fb7f800 0x56331fd38580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331fb7f800 0x56331fd38580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59250/0 (socket says 172.21.15.46:59250) 2024-04-25T08:51:34.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5cc00640 1 -- 172.21.15.46:0/3455734698 learned_addr learned my addr 172.21.15.46:0/3455734698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5d600640 1 --2- 172.21.15.46:0/3455734698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56331fb7f000 0x56331fd39080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5c200640 1 --2- 172.21.15.46:0/3455734698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331fb7f400 0x56331fd38b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5cc00640 1 -- 172.21.15.46:0/3455734698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56331fb7f000 msgr2=0x56331fd39080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5cc00640 1 --2- 172.21.15.46:0/3455734698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56331fb7f000 0x56331fd39080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5cc00640 1 -- 172.21.15.46:0/3455734698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331fb7f400 msgr2=0x56331fd38b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5cc00640 1 --2- 172.21.15.46:0/3455734698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331fb7f400 0x56331fd38b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5cc00640 1 -- 172.21.15.46:0/3455734698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56331fc20960 con 0x56331fb7f800 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5cc00640 1 --2- 172.21.15.46:0/3455734698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331fb7f800 0x56331fd38580 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x5633209e29f0 tx=0x56331fc56000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890519913790ef01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd59a00640 1 -- 172.21.15.46:0/3455734698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56331fbb49c0 con 0x56331fb7f800 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd59a00640 1 -- 172.21.15.46:0/3455734698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56331fbc8e00 con 0x56331fb7f800 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd59a00640 1 -- 172.21.15.46:0/3455734698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56331fbb44e0 con 0x56331fb7f800 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3455734698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56331fc20d20 con 0x56331fb7f800 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3455734698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56331fc20f00 con 0x56331fb7f800 2024-04-25T08:51:34.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.486+0000 7fdd5eb7c400 4 waiting for MDS map... 2024-04-25T08:51:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd59a00640 1 -- 172.21.15.46:0/3455734698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x563320aa4200 con 0x56331fb7f800 2024-04-25T08:51:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd59a00640 1 -- 172.21.15.46:0/3455734698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 84) v1 ==== 2723+0+0 (secure 0 0 0) 0x56331fbd5180 con 0x56331fb7f800 2024-04-25T08:51:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 4 Got MDS map 84 2024-04-25T08:51:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 10 main: JournalTool::main 2024-04-25T08:51:34.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 4 main: JournalTool: connecting to RADOS... 2024-04-25T08:51:34.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 Processor -- start 2024-04-25T08:51:34.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 -- start start 2024-04-25T08:51:34.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563320955800 0x56331fd39600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563320955c00 0x56331fd39b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320afe000 0x56331fd3a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563320952600 con 0x563320afe000 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563320952780 con 0x563320955800 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563320952900 con 0x563320955c00 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320afe000 0x56331fd3a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320afe000 0x56331fd3a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59264/0 (socket says 172.21.15.46:59264) 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5c200640 1 -- 172.21.15.46:0/1890697342 learned_addr learned my addr 172.21.15.46:0/1890697342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5d600640 1 --2- 172.21.15.46:0/1890697342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563320955800 0x56331fd39600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5cc00640 1 --2- 172.21.15.46:0/1890697342 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563320955c00 0x56331fd39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5c200640 1 -- 172.21.15.46:0/1890697342 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563320955c00 msgr2=0x56331fd39b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5c200640 1 --2- 172.21.15.46:0/1890697342 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563320955c00 0x56331fd39b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5c200640 1 -- 172.21.15.46:0/1890697342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563320955800 msgr2=0x56331fd39600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5c200640 1 --2- 172.21.15.46:0/1890697342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563320955800 0x56331fd39600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5c200640 1 -- 172.21.15.46:0/1890697342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56331fc201e0 con 0x563320afe000 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5c200640 1 --2- 172.21.15.46:0/1890697342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320afe000 0x56331fd3a100 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x563320a92900 tx=0x563320946c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd5722baf230824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd57200640 1 -- 172.21.15.46:0/1890697342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56331fbb49c0 con 0x563320afe000 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd57200640 1 -- 172.21.15.46:0/1890697342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56331fbc8540 con 0x563320afe000 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd57200640 1 -- 172.21.15.46:0/1890697342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56331fbb44e0 con 0x563320afe000 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/1890697342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320afe000 msgr2=0x56331fd3a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/1890697342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320afe000 0x56331fd3a100 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x563320a92900 tx=0x563320946c00 comp rx=0 tx=0).stop 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/1890697342 shutdown_connections 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/1890697342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320afe000 0x56331fd3a100 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/1890697342 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563320955c00 0x56331fd39b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/1890697342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563320955800 0x56331fd39600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/1890697342 >> 172.21.15.46:0/1890697342 conn(0x56331fb7ec00 msgr2=0x56331fbad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/1890697342 shutdown_connections 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.490+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/1890697342 wait complete. 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 Processor -- start 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 -- start start 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563320955c00 0x56331fd3b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563320955800 0x56331fd3ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320aff800 0x56331fd3a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563320953c80 con 0x563320aff800 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563320953b00 con 0x563320955c00 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563320953980 con 0x563320955800 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320aff800 0x56331fd3a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320aff800 0x56331fd3a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59274/0 (socket says 172.21.15.46:59274) 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5c200640 1 -- 172.21.15.46:0/3788144594 learned_addr learned my addr 172.21.15.46:0/3788144594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5d600640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563320955c00 0x56331fd3b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5cc00640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563320955800 0x56331fd3ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5c200640 1 -- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563320955800 msgr2=0x56331fd3ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5c200640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563320955800 0x56331fd3ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5c200640 1 -- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563320955c00 msgr2=0x56331fd3b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5c200640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563320955c00 0x56331fd3b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5c200640 1 -- 172.21.15.46:0/3788144594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56331fc21a40 con 0x563320aff800 2024-04-25T08:51:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5c200640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320aff800 0x56331fd3a680 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x563320a935c0 tx=0x56331fc37080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd55400640 1 -- 172.21.15.46:0/3788144594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56331fbb4000 con 0x563320aff800 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd55400640 1 -- 172.21.15.46:0/3788144594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56331fbc8a80 con 0x563320aff800 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd55400640 1 -- 172.21.15.46:0/3788144594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56331fbb49c0 con 0x563320aff800 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56331fc21c20 con 0x563320aff800 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56331fc21a40 con 0x563320aff800 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.494+0000 7fdd5eb7c400 4 main: JournalTool: resolving pool 15 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd55400640 1 -- 172.21.15.46:0/3788144594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x563320af8c00 con 0x563320aff800 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd55400640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x56331feae800 0x56331fd3b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd55400640 1 -- 172.21.15.46:0/3788144594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x56331fec2400 con 0x563320aff800 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 4 main: JournalTool: creating IoCtx.. 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 4 main: Executing for rank 0 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 4 Filtering by inode '1099511627778' 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563320aff400 0x56331fd3a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x563320aff000 con 0x563320aff400 2024-04-25T08:51:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5d600640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x56331feae800 0x56331fd3b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5cc00640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563320aff400 0x56331fd3a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5d600640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x56331feae800 0x56331fd3b700 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x56331fc8c780 tx=0x56331ffb4000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5cc00640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563320aff400 0x56331fd3a100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5cc00640 1 -- 172.21.15.46:0/3788144594 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x56331fcfa000 con 0x563320aff400 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 4 Pointer 400.00000000 is readable 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x563320afec00 con 0x563320aff400 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5cc00640 1 -- 172.21.15.46:0/3788144594 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x56331fcfa000 con 0x563320aff400 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 10 Header 0x400000 0x407953 0x407ccf 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 10 Starting journal scan from offset 0x407953 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 31059~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x563320afe800 con 0x563320aff400 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5cc00640 1 -- 172.21.15.46:0/3788144594 <== osd.6 v2:172.21.15.82:6823/1178952145 3 ==== osd_op_reply(3 200.00000001 [read 31059~892 out=892b] v0'0 uv29 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x56331fcfa000 con 0x563320aff400 2024-04-25T08:51:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T08:51:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 10 Parsing data, 0x37c bytes available 2024-04-25T08:51:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 10 Attempting decode at 0x407953 2024-04-25T08:51:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 10 Consumed 0x37c bytes 2024-04-25T08:51:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 10 Valid entry at 0x407953 2024-04-25T08:51:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563320afe400 0x56331fd39b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v9 -- 0x563320afe000 con 0x563320afe400 2024-04-25T08:51:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.498+0000 7fdd5d600640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563320afe400 0x56331fd39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5d600640 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563320afe400 0x56331fd39b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5d600640 1 -- 172.21.15.46:0/3788144594 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56331fcfa500 con 0x563320afe400 2024-04-25T08:51:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:Events by type: 2024-04-25T08:51:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:Errors: 0 2024-04-25T08:51:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 4 Reached end of journal objects 2024-04-25T08:51:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 4 Scanned objects, 0 missing, 1 valid 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 4 Events scanned, 0 gaps 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 4 Found 1 valid events 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 4 Selected 0 events events for processing 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 4 0 events 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563320aff400 msgr2=0x56331fd3a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563320aff400 0x56331fd3a100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563320afe400 msgr2=0x56331fd39b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563320afe400 0x56331fd39b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x56331feae800 msgr2=0x56331fd3b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x56331feae800 0x56331fd3b700 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x56331fc8c780 tx=0x56331ffb4000 comp rx=0 tx=0).stop 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320aff800 msgr2=0x56331fd3a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5c200640 1 -- 172.21.15.46:0/3788144594 reap_dead start 2024-04-25T08:51:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320aff800 0x56331fd3a680 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x563320a935c0 tx=0x56331fc37080 comp rx=0 tx=0).stop 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 shutdown_connections 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3788144594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563320aff800 0x56331fd3a680 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 >> 172.21.15.46:0/3788144594 conn(0x56331fb7ec00 msgr2=0x56331fbad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 shutdown_connections 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3788144594 wait complete. 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3455734698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331fb7f800 msgr2=0x56331fd38580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3455734698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331fb7f800 0x56331fd38580 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x5633209e29f0 tx=0x56331fc56000 comp rx=0 tx=0).stop 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3455734698 shutdown_connections 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3455734698 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x56331fb7f000 0x56331fd39080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3455734698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331fb7f400 0x56331fd38b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 --2- 172.21.15.46:0/3455734698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56331fb7f800 0x56331fd38580 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3455734698 >> 172.21.15.46:0/3455734698 conn(0x56331fb7fc00 msgr2=0x56331fbad000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3455734698 shutdown_connections 2024-04-25T08:51:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.502+0000 7fdd5eb7c400 1 -- 172.21.15.46:0/3455734698 wait complete. 2024-04-25T08:51:34.503 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.116888, result: Events by type: Errors: 0 2024-04-25T08:51:34.503 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-25T08:51:34.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.598+0000 7f1041559080 1 Processor -- start 2024-04-25T08:51:34.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.598+0000 7f1041559080 1 -- start start 2024-04-25T08:51:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c32ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559865b5ee20 0x559865b8e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559865bacb90 0x559865c2f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559865c37020 con 0x559865b5ea40 2024-04-25T08:51:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559865b059f0 con 0x559865b5ee20 2024-04-25T08:51:34.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x559865c371a0 con 0x559865bacb90 2024-04-25T08:51:34.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c32ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c32ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59298/0 (socket says 172.21.15.46:59298) 2024-04-25T08:51:34.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103e800640 1 -- 172.21.15.46:0/227040666 learned_addr learned my addr 172.21.15.46:0/227040666 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103e800640 1 -- 172.21.15.46:0/227040666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559865bacb90 msgr2=0x559865c2f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103e800640 1 --2- 172.21.15.46:0/227040666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559865bacb90 0x559865c2f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103e800640 1 -- 172.21.15.46:0/227040666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559865b5ee20 msgr2=0x559865b8e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:34.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103e800640 1 --2- 172.21.15.46:0/227040666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559865b5ee20 0x559865b8e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103e800640 1 -- 172.21.15.46:0/227040666 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559865cc83e0 con 0x559865b5ea40 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103e800640 1 --2- 172.21.15.46:0/227040666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c32ea0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f1030018f80 tx=0x7f1030003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d123245171482a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103ca00640 1 -- 172.21.15.46:0/227040666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10300af040 con 0x559865b5ea40 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103ca00640 1 -- 172.21.15.46:0/227040666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10300b9070 con 0x559865b5ea40 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f103ca00640 1 -- 172.21.15.46:0/227040666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10300aacf0 con 0x559865b5ea40 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- 172.21.15.46:0/227040666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 msgr2=0x559865c32ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- 172.21.15.46:0/227040666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c32ea0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f1030018f80 tx=0x7f1030003030 comp rx=0 tx=0).stop 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- 172.21.15.46:0/227040666 shutdown_connections 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- 172.21.15.46:0/227040666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559865bacb90 0x559865c2f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- 172.21.15.46:0/227040666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559865b5ee20 0x559865b8e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- 172.21.15.46:0/227040666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c32ea0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- 172.21.15.46:0/227040666 >> 172.21.15.46:0/227040666 conn(0x559865b5f730 msgr2=0x559865b90030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- 172.21.15.46:0/227040666 shutdown_connections 2024-04-25T08:51:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- 172.21.15.46:0/227040666 wait complete. 2024-04-25T08:51:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 Processor -- start 2024-04-25T08:51:34.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- start start 2024-04-25T08:51:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c2a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559865b5ee20 0x559865b8e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559865bacb90 0x559865b8e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559865cc8920 con 0x559865b5ea40 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559865cc8680 con 0x559865b5ee20 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.602+0000 7f1041559080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x559865cc8bc0 con 0x559865bacb90 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c2a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c2a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59310/0 (socket says 172.21.15.46:59310) 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103e800640 1 -- 172.21.15.46:0/156968775 learned_addr learned my addr 172.21.15.46:0/156968775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103de00640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559865b5ee20 0x559865b8e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103f200640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559865bacb90 0x559865b8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103e800640 1 -- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559865bacb90 msgr2=0x559865b8e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103e800640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559865bacb90 0x559865b8e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103e800640 1 -- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559865b5ee20 msgr2=0x559865b8e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103e800640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559865b5ee20 0x559865b8e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103e800640 1 -- 172.21.15.46:0/156968775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559865c2ed60 con 0x559865b5ea40 2024-04-25T08:51:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103f200640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x559865bacb90 0x559865b8e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f103e800640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c2a0c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f10300b6040 tx=0x7f10300180f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f1037400640 1 -- 172.21.15.46:0/156968775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10300af040 con 0x559865b5ea40 2024-04-25T08:51:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f1037400640 1 -- 172.21.15.46:0/156968775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10300b4980 con 0x559865b5ea40 2024-04-25T08:51:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f1037400640 1 -- 172.21.15.46:0/156968775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10300b4d70 con 0x559865b5ea40 2024-04-25T08:51:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559865c2f000 con 0x559865b5ea40 2024-04-25T08:51:34.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559865c2f540 con 0x559865b5ea40 2024-04-25T08:51:34.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f1037400640 1 -- 172.21.15.46:0/156968775 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10300bd3f0 con 0x559865b5ea40 2024-04-25T08:51:34.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f1037400640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1020065970 0x7f1020067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.606+0000 7f1037400640 1 -- 172.21.15.46:0/156968775 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f1030120a80 con 0x559865b5ea40 2024-04-25T08:51:34.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x559865c2e560 0x559865c2ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v9 -- 0x559865c2b260 con 0x559865c2e560 2024-04-25T08:51:34.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f103de00640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1020065970 0x7f1020067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f103f200640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x559865c2e560 0x559865c2ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f103de00640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1020065970 0x7f1020067e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e3a0 tx=0x7f102c001680 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f103f200640 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x559865c2e560 0x559865c2ad20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f103f200640 1 -- 172.21.15.46:0/156968775 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v109'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f103800c740 con 0x559865c2e560 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x559865c2e560 msgr2=0x559865c2ad20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x559865c2e560 0x559865c2ad20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1020065970 msgr2=0x7f1020067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1020065970 0x7f1020067e30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f102c00e3a0 tx=0x7f102c001680 comp rx=0 tx=0).stop 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 msgr2=0x559865c2a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 --2- 172.21.15.46:0/156968775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559865b5ea40 0x559865c2a0c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f10300b6040 tx=0x7f10300180f0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f103f200640 1 -- 172.21.15.46:0/156968775 reap_dead start 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 shutdown_connections 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 >> 172.21.15.46:0/156968775 conn(0x559865b5f730 msgr2=0x559865baad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 shutdown_connections 2024-04-25T08:51:34.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.610+0000 7f1041559080 1 -- 172.21.15.46:0/156968775 wait complete. 2024-04-25T08:51:34.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T08:51:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.798+0000 7fc6f0e00640 1 Processor -- start 2024-04-25T08:51:34.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6f0e00640 1 -- start start 2024-04-25T08:51:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6f0e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6ec15bbb0 0x7fc6ec0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6f0e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec0b5280 con 0x7fc6ec0b4da0 2024-04-25T08:51:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6f0e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec0b5560 con 0x7fc6ec0b4930 2024-04-25T08:51:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6f0e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ec0b5840 con 0x7fc6ec15bbb0 2024-04-25T08:51:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6eb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6ec15bbb0 0x7fc6ec0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6ebe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6ec0b4da0 0x7fc6ec165160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48656/0 (socket says 172.21.15.46:48656) 2024-04-25T08:51:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6ebe00640 1 -- 172.21.15.46:0/568935412 learned_addr learned my addr 172.21.15.46:0/568935412 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6eb400640 1 -- 172.21.15.46:0/568935412 >> v1:172.21.15.82:6789/0 conn(0x7fc6ec0b4930 legacy=0x7fc6ec15bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6eb400640 1 -- 172.21.15.46:0/568935412 >> v1:172.21.15.46:6789/0 conn(0x7fc6ec0b4da0 legacy=0x7fc6ec165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.802+0000 7fc6eb400640 1 -- 172.21.15.46:0/568935412 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0b5b20 con 0x7fc6ec15bbb0 2024-04-25T08:51:34.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6eb400640 1 --2- 172.21.15.46:0/568935412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec15bbb0 0x7fc6ec0cc000 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0097c0 tx=0x7fc6dc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b705a6a7cbeced7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6eaa00640 1 -- 172.21.15.46:0/568935412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0b0070 con 0x7fc6ec15bbb0 2024-04-25T08:51:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6eaa00640 1 -- 172.21.15.46:0/568935412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc0ab600 con 0x7fc6ec15bbb0 2024-04-25T08:51:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6eaa00640 1 -- 172.21.15.46:0/568935412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0ab9d0 con 0x7fc6ec15bbb0 2024-04-25T08:51:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6f0e00640 1 -- 172.21.15.46:0/568935412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec15bbb0 msgr2=0x7fc6ec0cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6f0e00640 1 --2- 172.21.15.46:0/568935412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec15bbb0 0x7fc6ec0cc000 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0097c0 tx=0x7fc6dc01c410 comp rx=0 tx=0).stop 2024-04-25T08:51:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6f0e00640 1 -- 172.21.15.46:0/568935412 shutdown_connections 2024-04-25T08:51:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6f0e00640 1 --2- 172.21.15.46:0/568935412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec15bbb0 0x7fc6ec0cc000 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6f0e00640 1 -- 172.21.15.46:0/568935412 >> 172.21.15.46:0/568935412 conn(0x7fc6ec0cc5a0 msgr2=0x7fc6ec0cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6f0e00640 1 -- 172.21.15.46:0/568935412 shutdown_connections 2024-04-25T08:51:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6f0e00640 1 -- 172.21.15.46:0/568935412 wait complete. 2024-04-25T08:51:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6f0e00640 1 Processor -- start 2024-04-25T08:51:34.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.806+0000 7fc6f0e00640 1 -- start start 2024-04-25T08:51:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6f0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec0b4930 0x7fc6ec138cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6f0e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6ec0b4da0 0x7fc6ec144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6f0e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6ec15bbb0 0x7fc6ec142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6f0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec169840 con 0x7fc6ec0b4930 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6f0e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1696c0 con 0x7fc6ec15bbb0 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6f0e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1699c0 con 0x7fc6ec0b4da0 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec0b4930 0x7fc6ec138cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec0b4930 0x7fc6ec138cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59334/0 (socket says 172.21.15.46:59334) 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6eb400640 1 -- 172.21.15.46:0/1117960413 learned_addr learned my addr 172.21.15.46:0/1117960413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6ebe00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6ec15bbb0 0x7fc6ec142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6e3e00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6ec0b4da0 0x7fc6ec144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6eb400640 1 -- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6ec0b4da0 msgr2=0x7fc6ec144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6eb400640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6ec0b4da0 0x7fc6ec144050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6eb400640 1 -- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6ec15bbb0 msgr2=0x7fc6ec142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6eb400640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6ec15bbb0 0x7fc6ec142030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6eb400640 1 -- 172.21.15.46:0/1117960413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec13d390 con 0x7fc6ec0b4930 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6eb400640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec0b4930 0x7fc6ec138cb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc01a040 tx=0x7fc6dc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6e3400640 1 -- 172.21.15.46:0/1117960413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0b0050 con 0x7fc6ec0b4930 2024-04-25T08:51:34.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6e3400640 1 -- 172.21.15.46:0/1117960413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc0bc070 con 0x7fc6ec0b4930 2024-04-25T08:51:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6e3400640 1 -- 172.21.15.46:0/1117960413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0aac00 con 0x7fc6ec0b4930 2024-04-25T08:51:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec13d670 con 0x7fc6ec0b4930 2024-04-25T08:51:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec144590 con 0x7fc6ec0b4930 2024-04-25T08:51:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6ec13dc80 con 0x7fc6ec0b4930 2024-04-25T08:51:34.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.810+0000 7fc6e3400640 1 -- 172.21.15.46:0/1117960413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6dc0c1460 con 0x7fc6ec0b4930 2024-04-25T08:51:34.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.814+0000 7fc6e3400640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc6c8066f60 0x7fc6c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:34.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.814+0000 7fc6e3400640 1 -- 172.21.15.46:0/1117960413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fc6dc11fd80 con 0x7fc6ec0b4930 2024-04-25T08:51:34.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.814+0000 7fc6e3e00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc6c8066f60 0x7fc6c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:34.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.814+0000 7fc6e3e00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc6c8066f60 0x7fc6c8069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec13c4e0 tx=0x7fc6d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:34.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:34.818+0000 7fc6e3400640 1 -- 172.21.15.46:0/1117960413 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6dc016090 con 0x7fc6ec0b4930 2024-04-25T08:51:35.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.014+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fc6ec069010 con 0x7fc6ec0b4930 2024-04-25T08:51:35.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6e3400640 1 -- 172.21.15.46:0/1117960413 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v85) v1 ==== 162+0+0 (secure 0 0 0) 0x7fc6dc0f0b80 con 0x7fc6ec0b4930 2024-04-25T08:51:35.297 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T08:51:35.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc6c8066f60 msgr2=0x7fc6c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:35.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc6c8066f60 0x7fc6c8069420 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec13c4e0 tx=0x7fc6d4009040 comp rx=0 tx=0).stop 2024-04-25T08:51:35.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec0b4930 msgr2=0x7fc6ec138cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:35.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec0b4930 0x7fc6ec138cb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc01a040 tx=0x7fc6dc01ac50 comp rx=0 tx=0).stop 2024-04-25T08:51:35.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 shutdown_connections 2024-04-25T08:51:35.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc6c8066f60 0x7fc6c8069420 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6ec15bbb0 0x7fc6ec142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc6ec0b4da0 0x7fc6ec144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 --2- 172.21.15.46:0/1117960413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6ec0b4930 0x7fc6ec138cb0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 >> 172.21.15.46:0/1117960413 conn(0x7fc6ec0cc5a0 msgr2=0x7fc6ec0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:35.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 shutdown_connections 2024-04-25T08:51:35.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.306+0000 7fc6f0e00640 1 -- 172.21.15.46:0/1117960413 wait complete. 2024-04-25T08:51:35.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:51:35.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd0f800640 1 Processor -- start 2024-04-25T08:51:35.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd0f800640 1 -- start start 2024-04-25T08:51:35.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd0f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cc1e0 0x7fbd0815f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:35.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd0f800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0806b7d0 con 0x7fbd080cf3a0 2024-04-25T08:51:35.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd0f800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0815be90 con 0x7fbd080cf8a0 2024-04-25T08:51:35.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd0f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0815c170 con 0x7fbd080cc1e0 2024-04-25T08:51:35.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd07e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cc1e0 0x7fbd0815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:35.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd0d800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbd080cf3a0 0x7fbd0815f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48666/0 (socket says 172.21.15.46:48666) 2024-04-25T08:51:35.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd0d800640 1 -- 172.21.15.46:0/2114984666 learned_addr learned my addr 172.21.15.46:0/2114984666 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:35.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd0ce00640 1 --1- 172.21.15.46:0/2114984666 >> v1:172.21.15.82:6789/0 conn(0x7fbd080cf8a0 0x7fbd080cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:59352/0 (socket says 172.21.15.46:59352) 2024-04-25T08:51:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd07400640 1 -- 172.21.15.46:0/2114984666 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 237765885 0 0) 0x7fbd0806b7d0 con 0x7fbd080cf3a0 2024-04-25T08:51:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.474+0000 7fbd07400640 1 -- 172.21.15.46:0/2114984666 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcf0003530 con 0x7fbd080cf3a0 2024-04-25T08:51:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.478+0000 7fbd07400640 1 -- 172.21.15.46:0/2114984666 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275035004 0 0) 0x7fbd0815be90 con 0x7fbd080cf8a0 2024-04-25T08:51:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.478+0000 7fbd07400640 1 -- 172.21.15.46:0/2114984666 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd0806b7d0 con 0x7fbd080cf8a0 2024-04-25T08:51:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.478+0000 7fbd07e00640 1 -- 172.21.15.46:0/2114984666 >> v1:172.21.15.82:6789/0 conn(0x7fbd080cf8a0 legacy=0x7fbd080cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.478+0000 7fbd07e00640 1 -- 172.21.15.46:0/2114984666 >> v1:172.21.15.46:6789/0 conn(0x7fbd080cf3a0 legacy=0x7fbd0815f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.478+0000 7fbd07e00640 1 -- 172.21.15.46:0/2114984666 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0815c450 con 0x7fbd080cc1e0 2024-04-25T08:51:35.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.478+0000 7fbd07e00640 1 --2- 172.21.15.46:0/2114984666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cc1e0 0x7fbd0815f370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40097f0 tx=0x7fbcf401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ad42e31750cdb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:35.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.478+0000 7fbd07400640 1 -- 172.21.15.46:0/2114984666 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40b2070 con 0x7fbd080cc1e0 2024-04-25T08:51:35.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.478+0000 7fbd07400640 1 -- 172.21.15.46:0/2114984666 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf40a9070 con 0x7fbd080cc1e0 2024-04-25T08:51:35.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.478+0000 7fbd07400640 1 -- 172.21.15.46:0/2114984666 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40ad860 con 0x7fbd080cc1e0 2024-04-25T08:51:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 -- 172.21.15.46:0/2114984666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cc1e0 msgr2=0x7fbd0815f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 --2- 172.21.15.46:0/2114984666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cc1e0 0x7fbd0815f370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40097f0 tx=0x7fbcf401c450 comp rx=0 tx=0).stop 2024-04-25T08:51:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 -- 172.21.15.46:0/2114984666 shutdown_connections 2024-04-25T08:51:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 --2- 172.21.15.46:0/2114984666 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cc1e0 0x7fbd0815f370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 -- 172.21.15.46:0/2114984666 >> 172.21.15.46:0/2114984666 conn(0x7fbd08099a00 msgr2=0x7fbd080b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 -- 172.21.15.46:0/2114984666 shutdown_connections 2024-04-25T08:51:35.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 -- 172.21.15.46:0/2114984666 wait complete. 2024-04-25T08:51:35.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 Processor -- start 2024-04-25T08:51:35.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 -- start start 2024-04-25T08:51:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd080cc1e0 0x7fbd081503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd080cf3a0 0x7fbd080b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:35.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cf8a0 0x7fbd0814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0816a0b0 con 0x7fbd080cf3a0 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08169f30 con 0x7fbd080cc1e0 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0816a230 con 0x7fbd080cf8a0 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd080cf3a0 0x7fbd080b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd080cf3a0 0x7fbd080b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59350/0 (socket says 172.21.15.46:59350) 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd07e00640 1 -- 172.21.15.46:0/4160898219 learned_addr learned my addr 172.21.15.46:0/4160898219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0ce00640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd080cc1e0 0x7fbd081503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd0d800640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cf8a0 0x7fbd0814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd07e00640 1 -- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cf8a0 msgr2=0x7fbd0814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd07e00640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cf8a0 0x7fbd0814bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd07e00640 1 -- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd080cc1e0 msgr2=0x7fbd081503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd07e00640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd080cc1e0 0x7fbd081503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.482+0000 7fbd07e00640 1 -- 172.21.15.46:0/4160898219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0814cb30 con 0x7fbd080cf3a0 2024-04-25T08:51:35.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd07e00640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd080cf3a0 0x7fbd080b1a50 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40b0040 tx=0x7fbcf40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd04c00640 1 -- 172.21.15.46:0/4160898219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40bb020 con 0x7fbd080cf3a0 2024-04-25T08:51:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd04c00640 1 -- 172.21.15.46:0/4160898219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf4003820 con 0x7fbd080cf3a0 2024-04-25T08:51:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd04c00640 1 -- 172.21.15.46:0/4160898219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40b2040 con 0x7fbd080cf3a0 2024-04-25T08:51:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd08152370 con 0x7fbd080cf3a0 2024-04-25T08:51:35.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd08151d20 con 0x7fbd080cf3a0 2024-04-25T08:51:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd08151b10 con 0x7fbd080cf3a0 2024-04-25T08:51:35.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd04c00640 1 -- 172.21.15.46:0/4160898219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbcf40be020 con 0x7fbd080cf3a0 2024-04-25T08:51:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd04c00640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbce8066f60 0x7fbce8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd04c00640 1 -- 172.21.15.46:0/4160898219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fbcf411f0d0 con 0x7fbd080cf3a0 2024-04-25T08:51:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.486+0000 7fbd0ce00640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbce8066f60 0x7fbce8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:35.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.490+0000 7fbd0ce00640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbce8066f60 0x7fbce8069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0011f0 tx=0x7fbcfc007b80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:35.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.490+0000 7fbd04c00640 1 -- 172.21.15.46:0/4160898219 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbcf40c5030 con 0x7fbd080cf3a0 2024-04-25T08:51:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.686+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd08069080 con 0x7fbd080cf3a0 2024-04-25T08:51:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.686+0000 7fbd04c00640 1 -- 172.21.15.46:0/4160898219 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fbd0814cb30 con 0x7fbd080cf3a0 2024-04-25T08:51:35.679 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:35.679 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":86,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:51:35.310580+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8658},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8658":{"gid":8658,"name":"c","rank":0,"incarnation":86,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/1174892156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1174892156},{"type":"v1","addr":"172.21.15.46:6835","nonce":1174892156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:51:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 86 2024-04-25T08:51:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbce8066f60 msgr2=0x7fbce8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbce8066f60 0x7fbce8069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0011f0 tx=0x7fbcfc007b80 comp rx=0 tx=0).stop 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd080cf3a0 msgr2=0x7fbd080b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd080cf3a0 0x7fbd080b1a50 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40b0040 tx=0x7fbcf40191f0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 shutdown_connections 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbce8066f60 0x7fbce8069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd080cf8a0 0x7fbd0814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd080cf3a0 0x7fbd080b1a50 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 --2- 172.21.15.46:0/4160898219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd080cc1e0 0x7fbd081503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 >> 172.21.15.46:0/4160898219 conn(0x7fbd08099a00 msgr2=0x7fbd080ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 shutdown_connections 2024-04-25T08:51:35.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:35.690+0000 7fbd0f800640 1 -- 172.21.15.46:0/4160898219 wait complete. 2024-04-25T08:51:36.103 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:50:47.860021+0000', 'modified': '2024-04-25T08:51:35.310580+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8658}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8658': {'gid': 8658, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/1174892156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1174892156}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1174892156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:51:36.103 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8658:up:replay 2024-04-25T08:51:37.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:51:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc58800640 1 Processor -- start 2024-04-25T08:51:37.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc58800640 1 -- start start 2024-04-25T08:51:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc58800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc500aed10 0x7fdc50165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc58800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc500b35b0 con 0x7fdc5015a750 2024-04-25T08:51:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc58800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5006d5e0 con 0x7fdc5006f320 2024-04-25T08:51:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc58800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc5015ab30 con 0x7fdc500aed10 2024-04-25T08:51:37.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc56800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdc500aed10 0x7fdc50165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc55e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdc5015a750 0x7fdc5006ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48676/0 (socket says 172.21.15.46:48676) 2024-04-25T08:51:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc55e00640 1 -- 172.21.15.46:0/957648364 learned_addr learned my addr 172.21.15.46:0/957648364 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc56800640 1 -- 172.21.15.46:0/957648364 >> v1:172.21.15.82:6789/0 conn(0x7fdc5006f320 legacy=0x7fdc50162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc56800640 1 -- 172.21.15.46:0/957648364 >> v1:172.21.15.46:6789/0 conn(0x7fdc5015a750 legacy=0x7fdc5006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:37.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.266+0000 7fdc56800640 1 -- 172.21.15.46:0/957648364 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc500ce720 con 0x7fdc500aed10 2024-04-25T08:51:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc56800640 1 --2- 172.21.15.46:0/957648364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 0x7fdc50165930 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009d90 tx=0x7fdc4c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8316a0dc7d471 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc54a00640 1 -- 172.21.15.46:0/957648364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0b2070 con 0x7fdc500aed10 2024-04-25T08:51:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc54a00640 1 -- 172.21.15.46:0/957648364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c0a5830 con 0x7fdc500aed10 2024-04-25T08:51:37.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc54a00640 1 -- 172.21.15.46:0/957648364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0a5c00 con 0x7fdc500aed10 2024-04-25T08:51:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc58800640 1 -- 172.21.15.46:0/957648364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 msgr2=0x7fdc50165930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc58800640 1 --2- 172.21.15.46:0/957648364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 0x7fdc50165930 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009d90 tx=0x7fdc4c01c970 comp rx=0 tx=0).stop 2024-04-25T08:51:37.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc58800640 1 -- 172.21.15.46:0/957648364 shutdown_connections 2024-04-25T08:51:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc58800640 1 --2- 172.21.15.46:0/957648364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 0x7fdc50165930 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc58800640 1 -- 172.21.15.46:0/957648364 >> 172.21.15.46:0/957648364 conn(0x7fdc500b12a0 msgr2=0x7fdc500b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc58800640 1 -- 172.21.15.46:0/957648364 shutdown_connections 2024-04-25T08:51:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.270+0000 7fdc58800640 1 -- 172.21.15.46:0/957648364 wait complete. 2024-04-25T08:51:37.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 Processor -- start 2024-04-25T08:51:37.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 -- start start 2024-04-25T08:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc5006f320 0x7fdc501390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 0x7fdc50158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc5015a750 0x7fdc50159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50168690 con 0x7fdc500aed10 2024-04-25T08:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5006d270 con 0x7fdc5015a750 2024-04-25T08:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50168810 con 0x7fdc5006f320 2024-04-25T08:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc55400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 0x7fdc50158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:37.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc55400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 0x7fdc50158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59368/0 (socket says 172.21.15.46:59368) 2024-04-25T08:51:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc55400640 1 -- 172.21.15.46:0/1226627181 learned_addr learned my addr 172.21.15.46:0/1226627181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc55400640 1 -- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc5006f320 msgr2=0x7fdc501390f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc55400640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc5006f320 0x7fdc501390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc55400640 1 -- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc5015a750 msgr2=0x7fdc50159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc55400640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc5015a750 0x7fdc50159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:37.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc55400640 1 -- 172.21.15.46:0/1226627181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc500ade50 con 0x7fdc500aed10 2024-04-25T08:51:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc55400640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 0x7fdc50158e00 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c420 tx=0x7fdc4000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc4a000640 1 -- 172.21.15.46:0/1226627181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc400052d0 con 0x7fdc500aed10 2024-04-25T08:51:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc4a000640 1 -- 172.21.15.46:0/1226627181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc40005470 con 0x7fdc500aed10 2024-04-25T08:51:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc4a000640 1 -- 172.21.15.46:0/1226627181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40007260 con 0x7fdc500aed10 2024-04-25T08:51:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc500ae130 con 0x7fdc500aed10 2024-04-25T08:51:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc500ae6f0 con 0x7fdc500aed10 2024-04-25T08:51:37.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.274+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc50069080 con 0x7fdc500aed10 2024-04-25T08:51:37.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.278+0000 7fdc4a000640 1 -- 172.21.15.46:0/1226627181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc40007400 con 0x7fdc500aed10 2024-04-25T08:51:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.278+0000 7fdc4a000640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc34066f60 0x7fdc34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.278+0000 7fdc4a000640 1 -- 172.21.15.46:0/1226627181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fdc40084800 con 0x7fdc500aed10 2024-04-25T08:51:37.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.278+0000 7fdc55e00640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc34066f60 0x7fdc34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:37.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.278+0000 7fdc55e00640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc34066f60 0x7fdc34069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdc500b4470 tx=0x7fdc44007a50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:37.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.282+0000 7fdc4a000640 1 -- 172.21.15.46:0/1226627181 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc40055570 con 0x7fdc500aed10 2024-04-25T08:51:37.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc50150d60 con 0x7fdc500aed10 2024-04-25T08:51:37.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc4a000640 1 -- 172.21.15.46:0/1226627181 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fdc40054e70 con 0x7fdc500aed10 2024-04-25T08:51:37.470 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:37.470 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:51:36.317127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8658},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8658":{"gid":8658,"name":"c","rank":0,"incarnation":86,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/1174892156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1174892156},{"type":"v1","addr":"172.21.15.46:6835","nonce":1174892156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:51:37.470 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 88 2024-04-25T08:51:37.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc34066f60 msgr2=0x7fdc34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc34066f60 0x7fdc34069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdc500b4470 tx=0x7fdc44007a50 comp rx=0 tx=0).stop 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 msgr2=0x7fdc50158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 0x7fdc50158e00 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c420 tx=0x7fdc4000c8e0 comp rx=0 tx=0).stop 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 shutdown_connections 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fdc34066f60 0x7fdc34069420 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc5015a750 0x7fdc50159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc500aed10 0x7fdc50158e00 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 --2- 172.21.15.46:0/1226627181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdc5006f320 0x7fdc501390f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 >> 172.21.15.46:0/1226627181 conn(0x7fdc500b12a0 msgr2=0x7fdc501656a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 shutdown_connections 2024-04-25T08:51:37.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:37.478+0000 7fdc58800640 1 -- 172.21.15.46:0/1226627181 wait complete. 2024-04-25T08:51:37.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:50:47.860021+0000', 'modified': '2024-04-25T08:51:36.317127+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8658}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8658': {'gid': 8658, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/1174892156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1174892156}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1174892156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:51:37.894 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8658:up:rejoin 2024-04-25T08:51:38.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:51:39.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f12adc00640 1 Processor -- start 2024-04-25T08:51:39.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f12adc00640 1 -- start start 2024-04-25T08:51:39.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f12adc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 0x7f12a815bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f12adc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12a80aeb20 0x7f12a80aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f12adc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a80af600 con 0x7f12a806ac10 2024-04-25T08:51:39.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f12adc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a80b0210 con 0x7f12a80aeb20 2024-04-25T08:51:39.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f12adc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a80af8e0 con 0x7f12a81450b0 2024-04-25T08:51:39.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f129f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12a80aeb20 0x7f12a80aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f129f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12a80aeb20 0x7f12a80aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39232/0 (socket says 172.21.15.46:39232) 2024-04-25T08:51:39.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f129f400640 1 -- 172.21.15.46:0/2295856968 learned_addr learned my addr 172.21.15.46:0/2295856968 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:39.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f12a7e00640 1 --2- 172.21.15.46:0/2295856968 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 0x7f12a815bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f129f400640 1 -- 172.21.15.46:0/2295856968 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 msgr2=0x7f12a815bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f129f400640 1 --2- 172.21.15.46:0/2295856968 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 0x7f12a815bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f129f400640 1 -- 172.21.15.46:0/2295856968 >> v1:172.21.15.82:6789/0 conn(0x7f12a806ac10 legacy=0x7f12a80b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.058+0000 7f129f400640 1 -- 172.21.15.46:0/2295856968 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a80afbc0 con 0x7f12a80aeb20 2024-04-25T08:51:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f129f400640 1 --2- 172.21.15.46:0/2295856968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 0x7f12a80aef80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f1290009870 tx=0x7f129001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb4f1e17816be05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12a7400640 1 -- 172.21.15.46:0/2295856968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12900b0070 con 0x7f12a80aeb20 2024-04-25T08:51:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12a7400640 1 -- 172.21.15.46:0/2295856968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1290004230 con 0x7f12a80aeb20 2024-04-25T08:51:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12a7400640 1 -- 172.21.15.46:0/2295856968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290003620 con 0x7f12a80aeb20 2024-04-25T08:51:39.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 -- 172.21.15.46:0/2295856968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 msgr2=0x7f12a80aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 --2- 172.21.15.46:0/2295856968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 0x7f12a80aef80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f1290009870 tx=0x7f129001c560 comp rx=0 tx=0).stop 2024-04-25T08:51:39.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 -- 172.21.15.46:0/2295856968 shutdown_connections 2024-04-25T08:51:39.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 --2- 172.21.15.46:0/2295856968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 0x7f12a80aef80 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 --2- 172.21.15.46:0/2295856968 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 0x7f12a815bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 -- 172.21.15.46:0/2295856968 >> 172.21.15.46:0/2295856968 conn(0x7f12a8099a00 msgr2=0x7f12a80b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:39.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 -- 172.21.15.46:0/2295856968 shutdown_connections 2024-04-25T08:51:39.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 -- 172.21.15.46:0/2295856968 wait complete. 2024-04-25T08:51:39.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 Processor -- start 2024-04-25T08:51:39.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.062+0000 7f12adc00640 1 -- start start 2024-04-25T08:51:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12adc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12a806ac10 0x7f12a815b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12adc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 0x7f12a8160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12adc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 0x7f12a81610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12adc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81682d0 con 0x7f12a80aeb20 2024-04-25T08:51:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12adc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a806bb30 con 0x7f12a806ac10 2024-04-25T08:51:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12adc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8168450 con 0x7f12a81450b0 2024-04-25T08:51:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f129f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 0x7f12a8160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f129f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 0x7f12a8160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39244/0 (socket says 172.21.15.46:39244) 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f129f400640 1 -- 172.21.15.46:0/4240226896 learned_addr learned my addr 172.21.15.46:0/4240226896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12a7e00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12a806ac10 0x7f12a815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12ad200640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 0x7f12a81610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f129f400640 1 -- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 msgr2=0x7f12a81610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f129f400640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 0x7f12a81610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f129f400640 1 -- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12a806ac10 msgr2=0x7f12a815b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f129f400640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12a806ac10 0x7f12a815b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f129f400640 1 -- 172.21.15.46:0/4240226896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a81421f0 con 0x7f12a80aeb20 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12ad200640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 0x7f12a81610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:39.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f129f400640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 0x7f12a8160ba0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f129001a190 tx=0x7f1290019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12a4c00640 1 -- 172.21.15.46:0/4240226896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12900b0040 con 0x7f12a80aeb20 2024-04-25T08:51:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12a4c00640 1 -- 172.21.15.46:0/4240226896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12900bb070 con 0x7f12a80aeb20 2024-04-25T08:51:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12a4c00640 1 -- 172.21.15.46:0/4240226896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290003e30 con 0x7f12a80aeb20 2024-04-25T08:51:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a80b3bb0 con 0x7f12a80aeb20 2024-04-25T08:51:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a809d380 con 0x7f12a80aeb20 2024-04-25T08:51:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.066+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a8069010 con 0x7f12a80aeb20 2024-04-25T08:51:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.070+0000 7f12a4c00640 1 -- 172.21.15.46:0/4240226896 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12900c0460 con 0x7f12a80aeb20 2024-04-25T08:51:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.070+0000 7f12a4c00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1284066fb0 0x7f1284069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.070+0000 7f12a4c00640 1 -- 172.21.15.46:0/4240226896 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f129011f630 con 0x7f12a80aeb20 2024-04-25T08:51:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.070+0000 7f12a7e00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1284066fb0 0x7f1284069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.070+0000 7f12a7e00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1284066fb0 0x7f1284069470 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1298009ac0 tx=0x7f1298009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.074+0000 7f12a4c00640 1 -- 172.21.15.46:0/4240226896 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1290016150 con 0x7f12a80aeb20 2024-04-25T08:51:39.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.270+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12a806fe40 con 0x7f12a80aeb20 2024-04-25T08:51:39.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.270+0000 7f12a4c00640 1 -- 172.21.15.46:0/4240226896 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f12900f0540 con 0x7f12a80aeb20 2024-04-25T08:51:39.262 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:39.262 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:51:37.321744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8658},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8658":{"gid":8658,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1174892156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1174892156},{"type":"v1","addr":"172.21.15.46:6835","nonce":1174892156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:51:39.263 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 89 2024-04-25T08:51:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1284066fb0 msgr2=0x7f1284069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1284066fb0 0x7f1284069470 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1298009ac0 tx=0x7f1298009190 comp rx=0 tx=0).stop 2024-04-25T08:51:39.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 msgr2=0x7f12a8160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 0x7f12a8160ba0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f129001a190 tx=0x7f1290019300 comp rx=0 tx=0).stop 2024-04-25T08:51:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 shutdown_connections 2024-04-25T08:51:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1284066fb0 0x7f1284069470 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f12a81450b0 0x7f12a81610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12a80aeb20 0x7f12a8160ba0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 --2- 172.21.15.46:0/4240226896 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12a806ac10 0x7f12a815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 >> 172.21.15.46:0/4240226896 conn(0x7f12a8099a00 msgr2=0x7f12a806ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 shutdown_connections 2024-04-25T08:51:39.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.274+0000 7f12adc00640 1 -- 172.21.15.46:0/4240226896 wait complete. 2024-04-25T08:51:39.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:50:47.860021+0000', 'modified': '2024-04-25T08:51:37.321744+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8658}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8658': {'gid': 8658, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/1174892156', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1174892156}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1174892156}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:51:39.688 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:51:39.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:51:39.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.842+0000 7fbd2b000640 1 Processor -- start 2024-04-25T08:51:39.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.842+0000 7fbd2b000640 1 -- start start 2024-04-25T08:51:39.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.842+0000 7fbd2b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd240cdc50 0x7fbd24166770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.842+0000 7fbd2b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd240cc700 con 0x7fbd24159230 2024-04-25T08:51:39.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.842+0000 7fbd2b000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd240cc9e0 con 0x7fbd2415bdb0 2024-04-25T08:51:39.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.842+0000 7fbd2b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd240cccc0 con 0x7fbd240cdc50 2024-04-25T08:51:39.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.842+0000 7fbd23e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbd24159230 0x7fbd240cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43158/0 (socket says 172.21.15.46:43158) 2024-04-25T08:51:39.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.842+0000 7fbd23e00640 1 -- 172.21.15.46:0/1311771314 learned_addr learned my addr 172.21.15.46:0/1311771314 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:39.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.842+0000 7fbd29000640 1 --2- 172.21.15.46:0/1311771314 >> v2:172.21.15.46:3300/0 conn(0x7fbd240cdc50 0x7fbd24166770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd23400640 1 -- 172.21.15.46:0/1311771314 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849300524 0 0) 0x7fbd240cc700 con 0x7fbd24159230 2024-04-25T08:51:39.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd23400640 1 -- 172.21.15.46:0/1311771314 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd08003530 con 0x7fbd24159230 2024-04-25T08:51:39.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd29000640 1 -- 172.21.15.46:0/1311771314 >> v1:172.21.15.82:6789/0 conn(0x7fbd2415bdb0 legacy=0x7fbd240cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd29000640 1 -- 172.21.15.46:0/1311771314 >> v1:172.21.15.46:6789/0 conn(0x7fbd24159230 legacy=0x7fbd240cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd29000640 1 -- 172.21.15.46:0/1311771314 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240ccfa0 con 0x7fbd240cdc50 2024-04-25T08:51:39.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd29000640 1 --2- 172.21.15.46:0/1311771314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd240cdc50 0x7fbd24166770 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009d90 tx=0x7fbd1c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659b8774f547b74a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd23400640 1 -- 172.21.15.46:0/1311771314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0b2070 con 0x7fbd240cdc50 2024-04-25T08:51:39.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd23400640 1 -- 172.21.15.46:0/1311771314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c0a5870 con 0x7fbd240cdc50 2024-04-25T08:51:39.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd23400640 1 -- 172.21.15.46:0/1311771314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0a5c40 con 0x7fbd240cdc50 2024-04-25T08:51:39.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd2b000640 1 -- 172.21.15.46:0/1311771314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd240cdc50 msgr2=0x7fbd24166770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd2b000640 1 --2- 172.21.15.46:0/1311771314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd240cdc50 0x7fbd24166770 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009d90 tx=0x7fbd1c01c8a0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd2b000640 1 -- 172.21.15.46:0/1311771314 shutdown_connections 2024-04-25T08:51:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd2b000640 1 --2- 172.21.15.46:0/1311771314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd240cdc50 0x7fbd24166770 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd2b000640 1 -- 172.21.15.46:0/1311771314 >> 172.21.15.46:0/1311771314 conn(0x7fbd240b12a0 msgr2=0x7fbd240b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd2b000640 1 -- 172.21.15.46:0/1311771314 shutdown_connections 2024-04-25T08:51:39.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.846+0000 7fbd2b000640 1 -- 172.21.15.46:0/1311771314 wait complete. 2024-04-25T08:51:39.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd2b000640 1 Processor -- start 2024-04-25T08:51:39.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd2b000640 1 -- start start 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd2b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd240cdc50 0x7fbd2413f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd2b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd24159230 0x7fbd2406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd2b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd2415bdb0 0x7fbd2406def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd2b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd24169860 con 0x7fbd24159230 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd2b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd241696e0 con 0x7fbd240cdc50 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd2b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd241699e0 con 0x7fbd2415bdb0 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd1b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd24159230 0x7fbd2406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd1b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd24159230 0x7fbd2406e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39258/0 (socket says 172.21.15.46:39258) 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd1b400640 1 -- 172.21.15.46:0/2301962558 learned_addr learned my addr 172.21.15.46:0/2301962558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd23e00640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd240cdc50 0x7fbd2413f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd29000640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd2415bdb0 0x7fbd2406def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd1b400640 1 -- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd2415bdb0 msgr2=0x7fbd2406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd1b400640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd2415bdb0 0x7fbd2406def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd1b400640 1 -- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd240cdc50 msgr2=0x7fbd2413f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd1b400640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd240cdc50 0x7fbd2413f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd1b400640 1 -- 172.21.15.46:0/2301962558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2406f2a0 con 0x7fbd24159230 2024-04-25T08:51:39.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd1b400640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd24159230 0x7fbd2406e9c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b410 tx=0x7fbd0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd20c00640 1 -- 172.21.15.46:0/2301962558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c015030 con 0x7fbd24159230 2024-04-25T08:51:39.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd20c00640 1 -- 172.21.15.46:0/2301962558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c004020 con 0x7fbd24159230 2024-04-25T08:51:39.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd20c00640 1 -- 172.21.15.46:0/2301962558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c004450 con 0x7fbd24159230 2024-04-25T08:51:39.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.850+0000 7fbd2b000640 1 -- 172.21.15.46:0/2301962558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2406d410 con 0x7fbd24159230 2024-04-25T08:51:39.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.854+0000 7fbd2b000640 1 -- 172.21.15.46:0/2301962558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2406d9d0 con 0x7fbd24159230 2024-04-25T08:51:39.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.854+0000 7fbd20c00640 1 -- 172.21.15.46:0/2301962558 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd0c018070 con 0x7fbd24159230 2024-04-25T08:51:39.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.854+0000 7fbd20c00640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd00066f60 0x7fbd00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.854+0000 7fbd20c00640 1 -- 172.21.15.46:0/2301962558 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fbd0c084d10 con 0x7fbd24159230 2024-04-25T08:51:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.854+0000 7fbd2e53c000 1 Processor -- start 2024-04-25T08:51:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.854+0000 7fbd23e00640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd00066f60 0x7fbd00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.854+0000 7fbd2e53c000 1 -- start start 2024-04-25T08:51:39.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd00066f60 0x7fbd00069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007fd0 tx=0x7fbd14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x558392025d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 0x558391fdb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 0x558391fdc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558391ec35f0 con 0x558392025920 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558391e2ada0 con 0x558392026240 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x558391db3150 con 0x558391fdbf30 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x558392025d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x558392025d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39266/0 (socket says 172.21.15.46:39266) 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 -- 172.21.15.46:0/3265932255 learned_addr learned my addr 172.21.15.46:0/3265932255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd29000640 1 --2- 172.21.15.46:0/3265932255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 0x558391fdc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd1b400640 1 --2- 172.21.15.46:0/3265932255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 0x558391fdb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 -- 172.21.15.46:0/3265932255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 msgr2=0x558391fdc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 --2- 172.21.15.46:0/3265932255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 0x558391fdc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 -- 172.21.15.46:0/3265932255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 msgr2=0x558391fdb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 --2- 172.21.15.46:0/3265932255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 0x558391fdb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 -- 172.21.15.46:0/3265932255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558391dccca0 con 0x558392025920 2024-04-25T08:51:39.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd23e00640 1 --2- 172.21.15.46:0/3265932255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x558392025d00 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007a50 tx=0x7fbd14000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a6bc80e8ab49dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd18c00640 1 -- 172.21.15.46:0/3265932255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14025070 con 0x558392025920 2024-04-25T08:51:39.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd18c00640 1 -- 172.21.15.46:0/3265932255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd14015bf0 con 0x558392025920 2024-04-25T08:51:39.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd18c00640 1 -- 172.21.15.46:0/3265932255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14020400 con 0x558392025920 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3265932255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 msgr2=0x558392025d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3265932255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x558392025d00 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007a50 tx=0x7fbd14000ee0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3265932255 shutdown_connections 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3265932255 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 0x558391fdc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3265932255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 0x558391fdb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3265932255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x558392025d00 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3265932255 >> 172.21.15.46:0/3265932255 conn(0x558391fe22a0 msgr2=0x558391fda2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3265932255 shutdown_connections 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3265932255 wait complete. 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 -- start start 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.858+0000 7fbd2e53c000 1 Processor -- start 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 0x55839202ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x55839202d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 0x558391fdfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558391de5990 con 0x558392025920 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558391e26e40 con 0x558392026240 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x558391db6900 con 0x558391fdbf30 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 10 client.0 _ll_register_callbacks cb 0x558391ede1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd1b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x55839202d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd1b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x55839202d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39270/0 (socket says 172.21.15.46:39270) 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd1b400640 1 -- 172.21.15.46:0/1110655270 learned_addr learned my addr 172.21.15.46:0/1110655270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd29000640 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 0x558391fdfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd23e00640 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 0x55839202ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd1b400640 1 -- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 msgr2=0x55839202ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd1b400640 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 0x55839202ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd1b400640 1 -- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 msgr2=0x558391fdfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd1b400640 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 0x558391fdfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd1b400640 1 -- 172.21.15.46:0/1110655270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558391de3630 con 0x558392025920 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd23e00640 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 0x55839202ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd29000640 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 0x558391fdfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd1b400640 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x55839202d3a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c082fa0 tx=0x7fbd0c05da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208530bf0e7278de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 1 -- 172.21.15.46:0/1110655270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0592c0 con 0x558392025920 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 1 -- 172.21.15.46:0/1110655270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fbd0c054880 con 0x558392025920 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1110655270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd14015bf0 con 0x558392025920 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 1 -- 172.21.15.46:0/1110655270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd0c054a80 con 0x558392025920 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 10 client.8736 fetch_fsmap learned FSMap version 89 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1110655270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x558391db33a0 con 0x558392025920 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 1 -- 172.21.15.46:0/1110655270 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd14015bf0 con 0x558392025920 2024-04-25T08:51:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 1 -- 172.21.15.46:0/1110655270 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd0c022060 con 0x558392025920 2024-04-25T08:51:39.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 10 client.8736 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:51:39.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 10 client.8736 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:51:39.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 20 client.8736 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:39.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x558391fe1160 0x558392036ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 4 client.8736 mds_command: new command op to 8658 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:51:39.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1110655270 --> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558391dc6a50 con 0x558391fe1160 2024-04-25T08:51:39.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd23e00640 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x558391fe1160 0x558392036ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd23e00640 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x558391fe1160 0x558392036ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d6be7be18ffaf6 server_cookie=57ef77717558fc1b in_seq=0 out_seq=0 2024-04-25T08:51:39.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 10 client.8736 ms_handle_connect on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 1 -- 172.21.15.46:0/1110655270 <== mds.0 v2:172.21.15.46:6834/1174892156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558391dc6a50 con 0x558391fe1160 2024-04-25T08:51:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 10 client.8736 handle_command_reply: tid=0 2024-04-25T08:51:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 client.8736 shutdown 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1110655270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 msgr2=0x55839202d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x55839202d3a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c082fa0 tx=0x7fbd0c05da90 comp rx=0 tx=0).stop 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1110655270 shutdown_connections 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x558391fe1160 0x558392036ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392026240 0x558391fdfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558392025920 0x55839202d3a0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1110655270 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fdbf30 0x55839202ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd11600640 0 client.8736 ms_handle_reset on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.862+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1110655270 >> 172.21.15.46:0/1110655270 conn(0x558391fe22a0 msgr2=0x558391fda9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.866+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1110655270 shutdown_connections 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.866+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1110655270 wait complete. 2024-04-25T08:51:39.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.866+0000 7fbd2e53c000 20 client.8736 trim_cache size 0 max 16384 2024-04-25T08:51:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.878+0000 7fbd2e53c000 1 Processor -- start 2024-04-25T08:51:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.878+0000 7fbd2e53c000 1 -- start start 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fe22a0 0x558392080d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x558392081280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392025920 0x5583920817c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd24202610 con 0x558391fdbf30 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240b3090 con 0x558392025920 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd241428f0 con 0x558391fe22a0 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd1b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x558392081280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd1b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x558392081280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39272/0 (socket says 172.21.15.46:39272) 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd1b400640 1 -- 172.21.15.46:0/1341642510 learned_addr learned my addr 172.21.15.46:0/1341642510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd23e00640 1 --2- 172.21.15.46:0/1341642510 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fe22a0 0x558392080d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd29000640 1 --2- 172.21.15.46:0/1341642510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392025920 0x5583920817c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd1b400640 1 -- 172.21.15.46:0/1341642510 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fe22a0 msgr2=0x558392080d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd1b400640 1 --2- 172.21.15.46:0/1341642510 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fe22a0 0x558392080d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd1b400640 1 -- 172.21.15.46:0/1341642510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392025920 msgr2=0x5583920817c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd1b400640 1 --2- 172.21.15.46:0/1341642510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392025920 0x5583920817c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd1b400640 1 -- 172.21.15.46:0/1341642510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558391eac600 con 0x558391fdbf30 2024-04-25T08:51:39.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd1b400640 1 --2- 172.21.15.46:0/1341642510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x558392081280 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c051cd0 tx=0x7fbd0c06b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf30767810a4b9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd13e00640 1 -- 172.21.15.46:0/1341642510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c051180 con 0x558391fdbf30 2024-04-25T08:51:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd13e00640 1 -- 172.21.15.46:0/1341642510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c081070 con 0x558391fdbf30 2024-04-25T08:51:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd13e00640 1 -- 172.21.15.46:0/1341642510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c073170 con 0x558391fdbf30 2024-04-25T08:51:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1341642510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 msgr2=0x558392081280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1341642510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x558392081280 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c051cd0 tx=0x7fbd0c06b000 comp rx=0 tx=0).stop 2024-04-25T08:51:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1341642510 shutdown_connections 2024-04-25T08:51:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1341642510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558392025920 0x5583920817c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1341642510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x558392081280 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/1341642510 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558391fe22a0 0x558392080d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1341642510 >> 172.21.15.46:0/1341642510 conn(0x55839203b6b0 msgr2=0x558391fde320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1341642510 shutdown_connections 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 -- 172.21.15.46:0/1341642510 wait complete. 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 -- start start 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.882+0000 7fbd2e53c000 1 Processor -- start 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x5583920315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558391fe22a0 0x558392068fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558392025920 0x558392069520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558391e2f3d0 con 0x558391fdbf30 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558391ee6d50 con 0x558391fe22a0 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x558391dc5980 con 0x558392025920 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 10 client.0 _ll_register_callbacks cb 0x7fbd14001540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x5583920315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x5583920315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39280/0 (socket says 172.21.15.46:39280) 2024-04-25T08:51:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd23e00640 1 -- 172.21.15.46:0/3805957619 learned_addr learned my addr 172.21.15.46:0/3805957619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd23e00640 1 -- 172.21.15.46:0/3805957619 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558392025920 msgr2=0x558392069520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd23e00640 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558392025920 0x558392069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd23e00640 1 -- 172.21.15.46:0/3805957619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558391fe22a0 msgr2=0x558392068fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd23e00640 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558391fe22a0 0x558392068fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd23e00640 1 -- 172.21.15.46:0/3805957619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558391dbf9b0 con 0x558391fdbf30 2024-04-25T08:51:39.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd23e00640 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x5583920315d0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fbd140068a0 tx=0x7fbd140010b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cb26b6298b6006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:39.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:39.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 1 -- 172.21.15.46:0/3805957619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14025040 con 0x558391fdbf30 2024-04-25T08:51:39.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 1 -- 172.21.15.46:0/3805957619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fbd1403ad20 con 0x558391fdbf30 2024-04-25T08:51:39.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3805957619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd0c081070 con 0x558391fdbf30 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 1 -- 172.21.15.46:0/3805957619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd14015bf0 con 0x558391fdbf30 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 10 client.8742 fetch_fsmap learned FSMap version 89 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3805957619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x558391dd20b0 con 0x558391fdbf30 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 1 -- 172.21.15.46:0/3805957619 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c081070 con 0x558391fdbf30 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 1 -- 172.21.15.46:0/3805957619 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd14020800 con 0x558391fdbf30 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 10 client.8742 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 10 client.8742 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 20 client.8742 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x558392026240 0x558392069de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 4 client.8742 mds_command: new command op to 8658 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3805957619 --> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558391e382e0 con 0x558392026240 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd1b400640 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x558392026240 0x558392069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd1b400640 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x558392026240 0x558392069de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=932647d91fd73235 server_cookie=e44023934cc23dc1 in_seq=0 out_seq=0 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 10 client.8742 ms_handle_connect on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:39.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 1 -- 172.21.15.46:0/3805957619 <== mds.0 v2:172.21.15.46:6834/1174892156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x558391e382e0 con 0x558392026240 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 10 client.8742 handle_command_reply: tid=0 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 client.8742 shutdown 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3805957619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 msgr2=0x5583920315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x5583920315d0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fbd140068a0 tx=0x7fbd140010b0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3805957619 shutdown_connections 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x558392026240 0x558392069de0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558392025920 0x558392069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558391fe22a0 0x558392068fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 --2- 172.21.15.46:0/3805957619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558391fdbf30 0x5583920315d0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd11600640 0 client.8742 ms_handle_reset on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3805957619 >> 172.21.15.46:0/3805957619 conn(0x55839203b6b0 msgr2=0x558391fdebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3805957619 shutdown_connections 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 1 -- 172.21.15.46:0/3805957619 wait complete. 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2e53c000 20 client.8742 trim_cache size 0 max 16384 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8658, 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:51:39.883 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:51:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5733533199999998, 2024-04-25T08:51:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 89, 2024-04-25T08:51:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 109, 2024-04-25T08:51:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 109, 2024-04-25T08:51:39.884 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.519527858 2024-04-25T08:51:39.884 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 -- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd00066f60 msgr2=0x7fbd00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd00066f60 0x7fbd00069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbd14007fd0 tx=0x7fbd14009040 comp rx=0 tx=0).stop 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 -- 172.21.15.46:0/2301962558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd24159230 msgr2=0x7fbd2406e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd24159230 0x7fbd2406e9c0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00b410 tx=0x7fbd0c00b8d0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 -- 172.21.15.46:0/2301962558 shutdown_connections 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbd00066f60 0x7fbd00069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbd2415bdb0 0x7fbd2406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd24159230 0x7fbd2406e9c0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 --2- 172.21.15.46:0/2301962558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd240cdc50 0x7fbd2413f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.886+0000 7fbd2b000640 1 -- 172.21.15.46:0/2301962558 >> 172.21.15.46:0/2301962558 conn(0x7fbd240b12a0 msgr2=0x7fbd240cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.890+0000 7fbd2b000640 1 -- 172.21.15.46:0/2301962558 shutdown_connections 2024-04-25T08:51:39.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:39.890+0000 7fbd2b000640 1 -- 172.21.15.46:0/2301962558 wait complete. 2024-04-25T08:51:39.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T08:51:39.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:51:39.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T08:51:40.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc38e400640 1 Processor -- start 2024-04-25T08:51:40.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc38e400640 1 -- start start 2024-04-25T08:51:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc38e400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 0x7fc3880cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc38e400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc3880cfcb0 0x7fc3880d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc38e400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38809af70 con 0x7fc3880cf4b0 2024-04-25T08:51:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc38e400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc388099aa0 con 0x7fc3880cfcb0 2024-04-25T08:51:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc38e400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc388099d80 con 0x7fc38815bb90 2024-04-25T08:51:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc37ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc3880cf4b0 0x7fc38809a860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43170/0 (socket says 172.21.15.46:43170) 2024-04-25T08:51:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc37ea00640 1 -- 172.21.15.46:0/2558945375 learned_addr learned my addr 172.21.15.46:0/2558945375 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc387e00640 1 --2- 172.21.15.46:0/2558945375 >> v2:172.21.15.82:3300/0 conn(0x7fc3880cfcb0 0x7fc3880d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc387400640 1 --2- 172.21.15.46:0/2558945375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 0x7fc3880cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc386a00640 1 -- 172.21.15.46:0/2558945375 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777900655 0 0) 0x7fc38809af70 con 0x7fc3880cf4b0 2024-04-25T08:51:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc386a00640 1 -- 172.21.15.46:0/2558945375 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc36c003530 con 0x7fc3880cf4b0 2024-04-25T08:51:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc387e00640 1 -- 172.21.15.46:0/2558945375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 msgr2=0x7fc3880cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc387e00640 1 --2- 172.21.15.46:0/2558945375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 0x7fc3880cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc387e00640 1 -- 172.21.15.46:0/2558945375 >> v1:172.21.15.46:6789/0 conn(0x7fc3880cf4b0 legacy=0x7fc38809a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.058+0000 7fc387e00640 1 -- 172.21.15.46:0/2558945375 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38809a060 con 0x7fc3880cfcb0 2024-04-25T08:51:40.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc387e00640 1 --2- 172.21.15.46:0/2558945375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3880cfcb0 0x7fc3880d0140 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc380009d90 tx=0x7fc38001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d6e28d5a0be122c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc386a00640 1 -- 172.21.15.46:0/2558945375 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800b2070 con 0x7fc3880cfcb0 2024-04-25T08:51:40.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc386a00640 1 -- 172.21.15.46:0/2558945375 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3800a57a0 con 0x7fc3880cfcb0 2024-04-25T08:51:40.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc386a00640 1 -- 172.21.15.46:0/2558945375 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800a5b70 con 0x7fc3880cfcb0 2024-04-25T08:51:40.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc38e400640 1 -- 172.21.15.46:0/2558945375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3880cfcb0 msgr2=0x7fc3880d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc38e400640 1 --2- 172.21.15.46:0/2558945375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3880cfcb0 0x7fc3880d0140 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc380009d90 tx=0x7fc38001c8e0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc38e400640 1 -- 172.21.15.46:0/2558945375 shutdown_connections 2024-04-25T08:51:40.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc38e400640 1 --2- 172.21.15.46:0/2558945375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3880cfcb0 0x7fc3880d0140 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc38e400640 1 --2- 172.21.15.46:0/2558945375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 0x7fc3880cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc38e400640 1 -- 172.21.15.46:0/2558945375 >> 172.21.15.46:0/2558945375 conn(0x7fc3880b31c0 msgr2=0x7fc3880b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc38e400640 1 -- 172.21.15.46:0/2558945375 shutdown_connections 2024-04-25T08:51:40.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.062+0000 7fc38e400640 1 -- 172.21.15.46:0/2558945375 wait complete. 2024-04-25T08:51:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 Processor -- start 2024-04-25T08:51:40.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 -- start start 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3880cf4b0 0x7fc38815a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3880cfcb0 0x7fc3880af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 0x7fc3880af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc388168a70 con 0x7fc3880cfcb0 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3881688f0 con 0x7fc3880cf4b0 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc388168bf0 con 0x7fc38815bb90 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3880cfcb0 0x7fc3880af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3880cfcb0 0x7fc3880af320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39290/0 (socket says 172.21.15.46:39290) 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37ea00640 1 -- 172.21.15.46:0/2624837680 learned_addr learned my addr 172.21.15.46:0/2624837680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc387400640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3880cf4b0 0x7fc38815a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc387e00640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 0x7fc3880af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37ea00640 1 -- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 msgr2=0x7fc3880af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37ea00640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 0x7fc3880af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37ea00640 1 -- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3880cf4b0 msgr2=0x7fc38815a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37ea00640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3880cf4b0 0x7fc38815a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37ea00640 1 -- 172.21.15.46:0/2624837680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388164b50 con 0x7fc3880cfcb0 2024-04-25T08:51:40.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37ea00640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3880cfcb0 0x7fc3880af320 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc37800c3c0 tx=0x7fc37800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37fe00640 1 -- 172.21.15.46:0/2624837680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378017070 con 0x7fc3880cfcb0 2024-04-25T08:51:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37fe00640 1 -- 172.21.15.46:0/2624837680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378005cf0 con 0x7fc3880cfcb0 2024-04-25T08:51:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc37fe00640 1 -- 172.21.15.46:0/2624837680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3780072f0 con 0x7fc3880cfcb0 2024-04-25T08:51:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 -- 172.21.15.46:0/2624837680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3880caf70 con 0x7fc3880cfcb0 2024-04-25T08:51:40.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.066+0000 7fc38e400640 1 -- 172.21.15.46:0/2624837680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38815c150 con 0x7fc3880cfcb0 2024-04-25T08:51:40.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.070+0000 7fc37fe00640 1 -- 172.21.15.46:0/2624837680 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc37801a070 con 0x7fc3880cfcb0 2024-04-25T08:51:40.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.070+0000 7fc37fe00640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc364066f60 0x7fc364069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.070+0000 7fc37fe00640 1 -- 172.21.15.46:0/2624837680 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fc3780844b0 con 0x7fc3880cfcb0 2024-04-25T08:51:40.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.070+0000 7fc391820000 1 Processor -- start 2024-04-25T08:51:40.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.070+0000 7fc387400640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc364066f60 0x7fc364069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- start start 2024-04-25T08:51:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140ccc90 0x561b140cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b14082d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc387400640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc364066f60 0x7fc364069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc370009b60 tx=0x7fc370009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b14083290 0x561b14083670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b13f69f40 con 0x561b140cd5b0 2024-04-25T08:51:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b13ed1070 con 0x561b140ccc90 2024-04-25T08:51:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b13e591a0 con 0x561b14083290 2024-04-25T08:51:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc37ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b14082d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc37ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b14082d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39298/0 (socket says 172.21.15.46:39298) 2024-04-25T08:51:40.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc37ea00640 1 -- 172.21.15.46:0/3695186738 learned_addr learned my addr 172.21.15.46:0/3695186738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc387400640 1 --2- 172.21.15.46:0/3695186738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140ccc90 0x561b140cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc387e00640 1 --2- 172.21.15.46:0/3695186738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b14083290 0x561b14083670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc37ea00640 1 -- 172.21.15.46:0/3695186738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b14083290 msgr2=0x561b14083670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc37ea00640 1 --2- 172.21.15.46:0/3695186738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b14083290 0x561b14083670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc37ea00640 1 -- 172.21.15.46:0/3695186738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140ccc90 msgr2=0x561b140cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc37ea00640 1 --2- 172.21.15.46:0/3695186738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140ccc90 0x561b140cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc37ea00640 1 -- 172.21.15.46:0/3695186738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b13e57510 con 0x561b140cd5b0 2024-04-25T08:51:40.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc37ea00640 1 --2- 172.21.15.46:0/3695186738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b14082d50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc378000f40 tx=0x7fc378054c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3aa4af6b3a2a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc377e00640 1 -- 172.21.15.46:0/3695186738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37805d360 con 0x561b140cd5b0 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc377e00640 1 -- 172.21.15.46:0/3695186738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378052da0 con 0x561b140cd5b0 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc377e00640 1 -- 172.21.15.46:0/3695186738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378051370 con 0x561b140cd5b0 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- 172.21.15.46:0/3695186738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 msgr2=0x561b14082d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 --2- 172.21.15.46:0/3695186738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b14082d50 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc378000f40 tx=0x7fc378054c00 comp rx=0 tx=0).stop 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- 172.21.15.46:0/3695186738 shutdown_connections 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 --2- 172.21.15.46:0/3695186738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b14083290 0x561b14083670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 --2- 172.21.15.46:0/3695186738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b14082d50 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 --2- 172.21.15.46:0/3695186738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140ccc90 0x561b140cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- 172.21.15.46:0/3695186738 >> 172.21.15.46:0/3695186738 conn(0x561b1407f240 msgr2=0x561b14081630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- 172.21.15.46:0/3695186738 shutdown_connections 2024-04-25T08:51:40.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- 172.21.15.46:0/3695186738 wait complete. 2024-04-25T08:51:40.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.074+0000 7fc391820000 1 -- start start 2024-04-25T08:51:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 Processor -- start 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 0x561b1408de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b1408e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b1408e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b13e8b6c0 con 0x561b140cd5b0 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b13ecd110 con 0x561b14083290 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b13f87f20 con 0x561b140ccc90 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 10 client.0 _ll_register_callbacks cb 0x561b1406e570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b1408e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b1408e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39300/0 (socket says 172.21.15.46:39300) 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387e00640 1 -- 172.21.15.46:0/3157848644 learned_addr learned my addr 172.21.15.46:0/3157848644 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc37ea00640 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b1408e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387400640 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 0x561b1408de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387e00640 1 -- 172.21.15.46:0/3157848644 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 msgr2=0x561b1408e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387e00640 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b1408e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387e00640 1 -- 172.21.15.46:0/3157848644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 msgr2=0x561b1408de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387e00640 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 0x561b1408de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387e00640 1 -- 172.21.15.46:0/3157848644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b13e593b0 con 0x561b140cd5b0 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387e00640 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b1408e8b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fc380009d90 tx=0x7fc38001b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9103aa894b56088a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc374c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc374c00640 1 -- 172.21.15.46:0/3157848644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800b2040 con 0x561b140cd5b0 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 -- 172.21.15.46:0/3157848644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc378052da0 con 0x561b140cd5b0 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc374c00640 1 -- 172.21.15.46:0/3157848644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fc3800c5cc0 con 0x561b140cd5b0 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc374c00640 1 -- 172.21.15.46:0/3157848644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3800a5960 con 0x561b140cd5b0 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 10 client.8754 fetch_fsmap learned FSMap version 89 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 -- 172.21.15.46:0/3157848644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x561b13f52e00 con 0x561b140cd5b0 2024-04-25T08:51:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc374c00640 1 -- 172.21.15.46:0/3157848644 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc378052da0 con 0x561b140cd5b0 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc374c00640 1 -- 172.21.15.46:0/3157848644 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc3800a5b00 con 0x561b140cd5b0 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 10 client.8754 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 10 client.8754 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 20 client.8754 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x561b1408f150 0x561b140de410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 4 client.8754 mds_command: new command op to 8658 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc391820000 1 -- 172.21.15.46:0/3157848644 --> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b13e6cd60 con 0x561b1408f150 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.078+0000 7fc387400640 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x561b1408f150 0x561b140de410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc387400640 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x561b1408f150 0x561b140de410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebae49144fd4a34 server_cookie=60c3c4e12657b78a in_seq=0 out_seq=0 2024-04-25T08:51:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc374c00640 10 client.8754 ms_handle_connect on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc374c00640 1 -- 172.21.15.46:0/3157848644 <== mds.0 v2:172.21.15.46:6834/1174892156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b13e6cd60 con 0x561b1408f150 2024-04-25T08:51:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc374c00640 10 client.8754 handle_command_reply: tid=0 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 client.8754 shutdown 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 -- 172.21.15.46:0/3157848644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 msgr2=0x561b1408e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b1408e8b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fc380009d90 tx=0x7fc38001b120 comp rx=0 tx=0).stop 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 -- 172.21.15.46:0/3157848644 shutdown_connections 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x561b1408f150 0x561b140de410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b1408e8b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b1408e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 --2- 172.21.15.46:0/3157848644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 0x561b1408de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 -- 172.21.15.46:0/3157848644 >> 172.21.15.46:0/3157848644 conn(0x561b1407f240 msgr2=0x561b14081ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc374c00640 0 client.8754 ms_handle_reset on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 -- 172.21.15.46:0/3157848644 shutdown_connections 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 1 -- 172.21.15.46:0/3157848644 wait complete. 2024-04-25T08:51:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.082+0000 7fc391820000 20 client.8754 trim_cache size 0 max 16384 2024-04-25T08:51:40.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd24e00640 1 Processor -- start 2024-04-25T08:51:40.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd24e00640 1 -- start start 2024-04-25T08:51:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd24e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efd2015f4c0 0x7efd2006c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd24e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2006d1e0 con 0x7efd200b2d10 2024-04-25T08:51:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd24e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd200cf9e0 con 0x7efd2009a110 2024-04-25T08:51:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd24e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2006cc20 con 0x7efd2015f4c0 2024-04-25T08:51:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd1de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efd200b2d10 0x7efd20099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43174/0 (socket says 172.21.15.46:43174) 2024-04-25T08:51:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd1de00640 1 -- 172.21.15.46:0/1880707272 learned_addr learned my addr 172.21.15.46:0/1880707272 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:40.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd1e800640 1 --2- 172.21.15.46:0/1880707272 >> v2:172.21.15.82:3300/0 conn(0x7efd2015f4c0 0x7efd2006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495485986 0 0) 0x7efd10001120 con 0x7efd200b2d10 2024-04-25T08:51:40.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd04003530 con 0x7efd200b2d10 2024-04-25T08:51:40.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.086+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2554142258 0 0) 0x7efd04003530 con 0x7efd200b2d10 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 >> v2:172.21.15.82:3300/0 conn(0x7efd2015f4c0 msgr2=0x7efd2006c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 --2- 172.21.15.46:0/1880707272 >> v2:172.21.15.82:3300/0 conn(0x7efd2015f4c0 0x7efd2006c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 >> v1:172.21.15.82:6789/0 conn(0x7efd2009a110 legacy=0x7efd2006bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2006cf00 con 0x7efd200b2d10 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7efd10001850 con 0x7efd200b2d10 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 >> v1:172.21.15.46:6789/0 conn(0x7efd200b2d10 legacy=0x7efd20099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd040039c0 0x7efd04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 0x7efd04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd040051d0 0x7efd0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd2006cc20 con 0x7efd04004810 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0400c070 con 0x7efd040051d0 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0400c270 con 0x7efd040039c0 2024-04-25T08:51:40.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1d400640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 0x7efd04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1de00640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd040039c0 0x7efd04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1e800640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd040051d0 0x7efd0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1d400640 1 -- 172.21.15.46:0/1880707272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd040039c0 msgr2=0x7efd04003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1d400640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd040039c0 0x7efd04003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1d400640 1 -- 172.21.15.46:0/1880707272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd040051d0 msgr2=0x7efd0400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1d400640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd040051d0 0x7efd0400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1d400640 1 -- 172.21.15.46:0/1880707272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd04003530 con 0x7efd04004810 2024-04-25T08:51:40.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1e800640 1 -- 172.21.15.46:0/1880707272 reap_dead start 2024-04-25T08:51:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1d400640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 0x7efd04004c90 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7efd0c001780 tx=0x7efd0c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7217cfef2a75756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c0013a0 con 0x7efd04004810 2024-04-25T08:51:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c0052f0 con 0x7efd04004810 2024-04-25T08:51:40.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.090+0000 7efd1ca00640 1 -- 172.21.15.46:0/1880707272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c003840 con 0x7efd04004810 2024-04-25T08:51:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 -- 172.21.15.46:0/1880707272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 msgr2=0x7efd04004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 0x7efd04004c90 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7efd0c001780 tx=0x7efd0c01c400 comp rx=0 tx=0).stop 2024-04-25T08:51:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 -- 172.21.15.46:0/1880707272 shutdown_connections 2024-04-25T08:51:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 --2- 172.21.15.46:0/1880707272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 0x7efd04004c90 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 -- 172.21.15.46:0/1880707272 >> 172.21.15.46:0/1880707272 conn(0x7efd200b3160 msgr2=0x7efd200b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 -- 172.21.15.46:0/1880707272 shutdown_connections 2024-04-25T08:51:40.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 -- 172.21.15.46:0/1880707272 wait complete. 2024-04-25T08:51:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 Processor -- start 2024-04-25T08:51:40.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 -- start start 2024-04-25T08:51:40.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 0x7efd20158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd201f80c0 0x7efd201f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd201f45c0 0x7efd201f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd200cff60 con 0x7efd04004810 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd200cfc80 con 0x7efd201f80c0 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd24e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd20099a00 con 0x7efd201f45c0 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 0x7efd20158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 0x7efd20158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39320/0 (socket says 172.21.15.46:39320) 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1de00640 1 -- 172.21.15.46:0/766103502 learned_addr learned my addr 172.21.15.46:0/766103502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1de00640 1 -- 172.21.15.46:0/766103502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd201f45c0 msgr2=0x7efd201f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1de00640 1 --2- 172.21.15.46:0/766103502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd201f45c0 0x7efd201f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1de00640 1 -- 172.21.15.46:0/766103502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd201f80c0 msgr2=0x7efd201f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1d400640 1 --2- 172.21.15.46:0/766103502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd201f80c0 0x7efd201f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1de00640 1 --2- 172.21.15.46:0/766103502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd201f80c0 0x7efd201f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1de00640 1 -- 172.21.15.46:0/766103502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd201f4f80 con 0x7efd04004810 2024-04-25T08:51:40.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.094+0000 7efd1d400640 1 --2- 172.21.15.46:0/766103502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd201f80c0 0x7efd201f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:51:40.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd1de00640 1 --2- 172.21.15.46:0/766103502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd04004810 0x7efd20158320 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7efd100019c0 tx=0x7efd100126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd16000640 1 -- 172.21.15.46:0/766103502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1001c070 con 0x7efd04004810 2024-04-25T08:51:40.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd16000640 1 -- 172.21.15.46:0/766103502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd100155e0 con 0x7efd04004810 2024-04-25T08:51:40.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd16000640 1 -- 172.21.15.46:0/766103502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd100159d0 con 0x7efd04004810 2024-04-25T08:51:40.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd24e00640 1 -- 172.21.15.46:0/766103502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd201f8520 con 0x7efd04004810 2024-04-25T08:51:40.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd24e00640 1 -- 172.21.15.46:0/766103502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd20158c90 con 0x7efd04004810 2024-04-25T08:51:40.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd24e00640 1 -- 172.21.15.46:0/766103502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7efd20069010 con 0x7efd04004810 2024-04-25T08:51:40.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd16000640 1 -- 172.21.15.46:0/766103502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd10015b70 con 0x7efd04004810 2024-04-25T08:51:40.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd16000640 1 --2- 172.21.15.46:0/766103502 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd00066f90 0x7efd00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7fc391820000 1 Processor -- start 2024-04-25T08:51:40.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7fc391820000 1 -- start start 2024-04-25T08:51:40.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd16000640 1 -- 172.21.15.46:0/766103502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7efd10087ff0 con 0x7efd04004810 2024-04-25T08:51:40.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd16000640 1 -- 172.21.15.46:0/766103502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+512 (secure 0 0 0) 0x7efd201f4f80 con 0x7efd04004810 2024-04-25T08:51:40.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.098+0000 7efd24e00640 1 -- 172.21.15.46:0/766103502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7efd2009a530 con 0x7efd04004810 2024-04-25T08:51:40.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7efd1d400640 1 --2- 172.21.15.46:0/766103502 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd00066f90 0x7efd00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b14083290 0x561b14082640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b14082b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140cd5b0 0x561b140de410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3882034d0 con 0x561b14083290 2024-04-25T08:51:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3882026b0 con 0x561b140cd5b0 2024-04-25T08:51:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815da20 con 0x561b140ccc90 2024-04-25T08:51:40.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b14083290 0x561b14082640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b14083290 0x561b14082640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39328/0 (socket says 172.21.15.46:39328) 2024-04-25T08:51:40.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387400640 1 -- 172.21.15.46:0/1773720171 learned_addr learned my addr 172.21.15.46:0/1773720171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387e00640 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140cd5b0 0x561b140de410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc37ea00640 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b14082b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7efd1d400640 1 --2- 172.21.15.46:0/766103502 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd00066f90 0x7efd00069450 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd0c001750 tx=0x7efd0c0011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387400640 1 -- 172.21.15.46:0/1773720171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 msgr2=0x561b14082b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387400640 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b14082b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387400640 1 -- 172.21.15.46:0/1773720171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140cd5b0 msgr2=0x561b140de410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387400640 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140cd5b0 0x561b140de410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387400640 1 -- 172.21.15.46:0/1773720171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b13f28c50 con 0x561b14083290 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc37ea00640 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b14082b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc387400640 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b14083290 0x561b14082640 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fc37002ceb0 tx=0x7fc370018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281a40af06b0ec05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc377400640 1 -- 172.21.15.46:0/1773720171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370004030 con 0x561b14083290 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc377400640 1 -- 172.21.15.46:0/1773720171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3700041d0 con 0x561b14083290 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc377400640 1 -- 172.21.15.46:0/1773720171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc370014940 con 0x561b14083290 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 -- 172.21.15.46:0/1773720171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b14083290 msgr2=0x561b14082640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b14083290 0x561b14082640 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fc37002ceb0 tx=0x7fc370018000 comp rx=0 tx=0).stop 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 -- 172.21.15.46:0/1773720171 shutdown_connections 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b140cd5b0 0x561b140de410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b14082b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 --2- 172.21.15.46:0/1773720171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b14083290 0x561b14082640 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 -- 172.21.15.46:0/1773720171 >> 172.21.15.46:0/1773720171 conn(0x561b1407f240 msgr2=0x561b14087d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 -- 172.21.15.46:0/1773720171 shutdown_connections 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 -- 172.21.15.46:0/1773720171 wait complete. 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 -- start start 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.102+0000 7fc391820000 1 Processor -- start 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 0x561b140e0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b140e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b140e9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b14088b70 con 0x561b140cd5b0 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3882020b0 con 0x561b14083290 2024-04-25T08:51:40.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b1408c200 con 0x561b140ccc90 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 10 client.0 _ll_register_callbacks cb 0x7fc3800b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b140e9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b140e9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39332/0 (socket says 172.21.15.46:39332) 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387e00640 1 -- 172.21.15.46:0/4253361710 learned_addr learned my addr 172.21.15.46:0/4253361710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc37ea00640 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b140e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387400640 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 0x561b140e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387e00640 1 -- 172.21.15.46:0/4253361710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 msgr2=0x561b140e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387e00640 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b140e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387e00640 1 -- 172.21.15.46:0/4253361710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 msgr2=0x561b140e0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387e00640 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 0x561b140e0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387e00640 1 -- 172.21.15.46:0/4253361710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b13e99120 con 0x561b140cd5b0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387e00640 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b140e9420 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc38001c920 tx=0x7fc380009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449e7efcf0cd791e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 1 -- 172.21.15.46:0/4253361710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800b2050 con 0x561b140cd5b0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 1 -- 172.21.15.46:0/4253361710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fc3800aecf0 con 0x561b140cd5b0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 -- 172.21.15.46:0/4253361710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3700041d0 con 0x561b140cd5b0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 1 -- 172.21.15.46:0/4253361710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3800dd050 con 0x561b140cd5b0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 10 client.8766 fetch_fsmap learned FSMap version 89 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 -- 172.21.15.46:0/4253361710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x561b13e78170 con 0x561b140cd5b0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 1 -- 172.21.15.46:0/4253361710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3700041d0 con 0x561b140cd5b0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 1 -- 172.21.15.46:0/4253361710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc3800e0020 con 0x561b140cd5b0 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 10 client.8766 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 10 client.8766 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:40.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 20 client.8766 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x561b140e9960 0x561b1412cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 4 client.8766 mds_command: new command op to 8658 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T08:51:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 -- 172.21.15.46:0/4253361710 --> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x561b13ede5b0 con 0x561b140e9960 2024-04-25T08:51:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387400640 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x561b140e9960 0x561b1412cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 10 client.8766 ms_handle_connect on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc387400640 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x561b140e9960 0x561b1412cf70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c804d6924b524f3 server_cookie=65f8ed0e5b4460e3 in_seq=0 out_seq=0 2024-04-25T08:51:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 1 -- 172.21.15.46:0/4253361710 <== mds.0 v2:172.21.15.46:6834/1174892156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561b13ede5b0 con 0x561b140e9960 2024-04-25T08:51:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 10 client.8766 handle_command_reply: tid=0 2024-04-25T08:51:40.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 client.8766 shutdown 2024-04-25T08:51:40.103 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:51:40.106+0000 7f844da00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T08:51:40.103 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:51:40.106+0000 7f844da00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T08:51:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 -- 172.21.15.46:0/4253361710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 msgr2=0x561b140e9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b140e9420 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fc38001c920 tx=0x7fc380009750 comp rx=0 tx=0).stop 2024-04-25T08:51:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 -- 172.21.15.46:0/4253361710 shutdown_connections 2024-04-25T08:51:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x561b140e9960 0x561b1412cf70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561b140cd5b0 0x561b140e9420 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x561b140ccc90 0x561b140e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 --2- 172.21.15.46:0/4253361710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561b14083290 0x561b140e0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc391820000 1 -- 172.21.15.46:0/4253361710 >> 172.21.15.46:0/4253361710 conn(0x561b1407f240 msgr2=0x561b14081630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.106+0000 7fc374c00640 0 client.8766 ms_handle_reset on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:40.104 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:51:40.110+0000 7f844da00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T08:51:40.105 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:51:40.110+0000 7f844da00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T08:51:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc391820000 1 -- 172.21.15.46:0/4253361710 shutdown_connections 2024-04-25T08:51:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc391820000 1 -- 172.21.15.46:0/4253361710 wait complete. 2024-04-25T08:51:40.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc391820000 20 client.8766 trim_cache size 0 max 16384 2024-04-25T08:51:40.105 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:51:40.106 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-04-25T08:51:40.106 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "eeeda6d8-699e-4220-bc59-79837e0d6705", 2024-04-25T08:51:40.106 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-04-25T08:51:40.106 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:51:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 -- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc364066f60 msgr2=0x7fc364069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc364066f60 0x7fc364069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc370009b60 tx=0x7fc370009190 comp rx=0 tx=0).stop 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 -- 172.21.15.46:0/2624837680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3880cfcb0 msgr2=0x7fc3880af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3880cfcb0 0x7fc3880af320 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc37800c3c0 tx=0x7fc37800c880 comp rx=0 tx=0).stop 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 -- 172.21.15.46:0/2624837680 shutdown_connections 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc364066f60 0x7fc364069420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc38815bb90 0x7fc3880af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3880cfcb0 0x7fc3880af320 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 --2- 172.21.15.46:0/2624837680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3880cf4b0 0x7fc38815a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 -- 172.21.15.46:0/2624837680 >> 172.21.15.46:0/2624837680 conn(0x7fc3880b31c0 msgr2=0x7fc388165520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 -- 172.21.15.46:0/2624837680 shutdown_connections 2024-04-25T08:51:40.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.110+0000 7fc38e400640 1 -- 172.21.15.46:0/2624837680 wait complete. 2024-04-25T08:51:40.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:51:40.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T08:51:40.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff8e00640 1 Processor -- start 2024-04-25T08:51:40.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff8e00640 1 -- start start 2024-04-25T08:51:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff4159300 0x7f9ff40cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff8e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff41596e0 con 0x7f9ff406b320 2024-04-25T08:51:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff8e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff40cd3d0 con 0x7f9ff40ce040 2024-04-25T08:51:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff8e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40cfda0 con 0x7f9ff4159300 2024-04-25T08:51:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ff406b320 0x7f9ff406fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43180/0 (socket says 172.21.15.46:43180) 2024-04-25T08:51:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff2800640 1 -- 172.21.15.46:0/1929810562 learned_addr learned my addr 172.21.15.46:0/1929810562 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:51:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/1929810562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff4159300 0x7f9ff40cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff1400640 1 -- 172.21.15.46:0/1929810562 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368517384 0 0) 0x7f9ff41596e0 con 0x7f9ff406b320 2024-04-25T08:51:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff1400640 1 -- 172.21.15.46:0/1929810562 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd8003530 con 0x7f9ff406b320 2024-04-25T08:51:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff1400640 1 -- 172.21.15.46:0/1929810562 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907752222 0 0) 0x7f9ff40cd3d0 con 0x7f9ff40ce040 2024-04-25T08:51:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff1400640 1 -- 172.21.15.46:0/1929810562 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ff41596e0 con 0x7f9ff40ce040 2024-04-25T08:51:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff1e00640 1 -- 172.21.15.46:0/1929810562 >> v1:172.21.15.82:6789/0 conn(0x7f9ff40ce040 legacy=0x7f9ff406ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff1e00640 1 -- 172.21.15.46:0/1929810562 >> v1:172.21.15.46:6789/0 conn(0x7f9ff406b320 legacy=0x7f9ff406fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.274+0000 7f9ff1e00640 1 -- 172.21.15.46:0/1929810562 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40cd6b0 con 0x7f9ff4159300 2024-04-25T08:51:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/1929810562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff4159300 0x7f9ff40cdb00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40097f0 tx=0x7f9fe401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17dfb3e85ff02f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff1400640 1 -- 172.21.15.46:0/1929810562 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40ab650 con 0x7f9ff4159300 2024-04-25T08:51:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff1400640 1 -- 172.21.15.46:0/1929810562 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe40b0070 con 0x7f9ff4159300 2024-04-25T08:51:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff1400640 1 -- 172.21.15.46:0/1929810562 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40aba10 con 0x7f9ff4159300 2024-04-25T08:51:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff8e00640 1 -- 172.21.15.46:0/1929810562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff4159300 msgr2=0x7f9ff40cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff8e00640 1 --2- 172.21.15.46:0/1929810562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff4159300 0x7f9ff40cdb00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40097f0 tx=0x7f9fe401c4b0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff8e00640 1 -- 172.21.15.46:0/1929810562 shutdown_connections 2024-04-25T08:51:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff8e00640 1 --2- 172.21.15.46:0/1929810562 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff4159300 0x7f9ff40cdb00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff8e00640 1 -- 172.21.15.46:0/1929810562 >> 172.21.15.46:0/1929810562 conn(0x7f9ff406e3e0 msgr2=0x7f9ff406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff8e00640 1 -- 172.21.15.46:0/1929810562 shutdown_connections 2024-04-25T08:51:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.278+0000 7f9ff8e00640 1 -- 172.21.15.46:0/1929810562 wait complete. 2024-04-25T08:51:40.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 Processor -- start 2024-04-25T08:51:40.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 -- start start 2024-04-25T08:51:40.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff406b320 0x7f9ff414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff40ce040 0x7f9ff414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff4159300 0x7f9ff40ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40d13c0 con 0x7f9ff40ce040 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40d1240 con 0x7f9ff4159300 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40ccf80 con 0x7f9ff406b320 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9febe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff40ce040 0x7f9ff414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9febe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff40ce040 0x7f9ff414f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39344/0 (socket says 172.21.15.46:39344) 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9febe00640 1 -- 172.21.15.46:0/3645056146 learned_addr learned my addr 172.21.15.46:0/3645056146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff406b320 0x7f9ff414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff2800640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff4159300 0x7f9ff40ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9febe00640 1 -- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff406b320 msgr2=0x7f9ff414fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9febe00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff406b320 0x7f9ff414fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9febe00640 1 -- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff4159300 msgr2=0x7f9ff40ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9febe00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff4159300 0x7f9ff40ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9febe00640 1 -- 172.21.15.46:0/3645056146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40b3b00 con 0x7f9ff40ce040 2024-04-25T08:51:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff2800640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff4159300 0x7f9ff40ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:51:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9febe00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff40ce040 0x7f9ff414f020 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c530 tx=0x7f9fdc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9fea000640 1 -- 172.21.15.46:0/3645056146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0052d0 con 0x7f9ff40ce040 2024-04-25T08:51:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9fea000640 1 -- 172.21.15.46:0/3645056146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc005490 con 0x7f9ff40ce040 2024-04-25T08:51:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 -- 172.21.15.46:0/3645056146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff40af760 con 0x7f9ff40ce040 2024-04-25T08:51:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9fea000640 1 -- 172.21.15.46:0/3645056146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc012850 con 0x7f9ff40ce040 2024-04-25T08:51:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.282+0000 7f9ff8e00640 1 -- 172.21.15.46:0/3645056146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff40ade10 con 0x7f9ff40ce040 2024-04-25T08:51:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9fea000640 1 -- 172.21.15.46:0/3645056146 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9fdc0129f0 con 0x7f9ff40ce040 2024-04-25T08:51:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9fea000640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9fd0066f60 0x7f9fd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9fea000640 1 -- 172.21.15.46:0/3645056146 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f9fdc084fe0 con 0x7f9ff40ce040 2024-04-25T08:51:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9fd0066f60 0x7f9fd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ffc2c7000 1 Processor -- start 2024-04-25T08:51:40.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ffc2c7000 1 -- start start 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9fd0066f60 0x7f9fd0069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40af000 tx=0x7f9fe40a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731ec9d0 0x5569731ecdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569731a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731a30a0 0x5569731a3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556972fefff0 con 0x5569731ed2f0 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556972f78150 con 0x5569731a30a0 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556972fec090 con 0x5569731ec9d0 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9febe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569731a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9febe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569731a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39358/0 (socket says 172.21.15.46:39358) 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9febe00640 1 -- 172.21.15.46:0/317856223 learned_addr learned my addr 172.21.15.46:0/317856223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/317856223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731ec9d0 0x5569731ecdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9ff2800640 1 --2- 172.21.15.46:0/317856223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731a30a0 0x5569731a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9febe00640 1 -- 172.21.15.46:0/317856223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731ec9d0 msgr2=0x5569731ecdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9febe00640 1 --2- 172.21.15.46:0/317856223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731ec9d0 0x5569731ecdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9febe00640 1 -- 172.21.15.46:0/317856223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731a30a0 msgr2=0x5569731a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9febe00640 1 --2- 172.21.15.46:0/317856223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731a30a0 0x5569731a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.286+0000 7f9febe00640 1 -- 172.21.15.46:0/317856223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569731860d0 con 0x5569731ed2f0 2024-04-25T08:51:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 --2- 172.21.15.46:0/317856223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569731a2b60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00cef0 tx=0x7f9fdc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f54bf674c75f7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9fe2a00640 1 -- 172.21.15.46:0/317856223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc05dd90 con 0x5569731ed2f0 2024-04-25T08:51:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9fe2a00640 1 -- 172.21.15.46:0/317856223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc06d170 con 0x5569731ed2f0 2024-04-25T08:51:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9fe2a00640 1 -- 172.21.15.46:0/317856223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc012ce0 con 0x5569731ed2f0 2024-04-25T08:51:40.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/317856223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 msgr2=0x5569731a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/317856223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569731a2b60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00cef0 tx=0x7f9fdc012000 comp rx=0 tx=0).stop 2024-04-25T08:51:40.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/317856223 shutdown_connections 2024-04-25T08:51:40.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/317856223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731a30a0 0x5569731a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/317856223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569731a2b60 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/317856223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731ec9d0 0x5569731ecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/317856223 >> 172.21.15.46:0/317856223 conn(0x55697319f020 msgr2=0x5569731a1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/317856223 shutdown_connections 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/317856223 wait complete. 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- start start 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 Processor -- start 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 0x5569731d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556972fb4250 con 0x5569731ec9d0 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556972faa690 con 0x5569731ed2f0 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556972f78a40 con 0x5569731a30a0 2024-04-25T08:51:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 10 client.0 _ll_register_callbacks cb 0x55697318de90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731d1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39366/0 (socket says 172.21.15.46:39366) 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 -- 172.21.15.46:0/29172970 learned_addr learned my addr 172.21.15.46:0/29172970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ff2800640 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 0x5569731d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 -- 172.21.15.46:0/29172970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 msgr2=0x5569731d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 -- 172.21.15.46:0/29172970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 msgr2=0x5569731d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 0x5569731d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 -- 172.21.15.46:0/29172970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55697309d1c0 con 0x5569731ec9d0 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9febe00640 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731d1d30 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00cef0 tx=0x7f9fdc000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f339b8e4d8533f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9fd7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9fd7400640 1 -- 172.21.15.46:0/29172970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc072170 con 0x5569731ec9d0 2024-04-25T08:51:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9fd7400640 1 -- 172.21.15.46:0/29172970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f9fdc051cf0 con 0x5569731ec9d0 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.290+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/29172970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fdc06d170 con 0x5569731ec9d0 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9fd7400640 1 -- 172.21.15.46:0/29172970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fdc07a050 con 0x5569731ec9d0 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 10 client.8778 fetch_fsmap learned FSMap version 89 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/29172970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x556973099170 con 0x5569731ec9d0 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9fd7400640 1 -- 172.21.15.46:0/29172970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc06d170 con 0x5569731ec9d0 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9fd7400640 1 -- 172.21.15.46:0/29172970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9fdc073b40 con 0x5569731ec9d0 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 10 client.8778 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 10 client.8778 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 20 client.8778 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x5569731fcc80 0x5569731ff070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 4 client.8778 mds_command: new command op to 8658 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/29172970 --> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556972f8bc60 con 0x5569731fcc80 2024-04-25T08:51:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x5569731fcc80 0x5569731ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x5569731fcc80 0x5569731ff070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d78ea7be51aa38 server_cookie=a3208e5ad0442a2d in_seq=0 out_seq=0 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9fd7400640 10 client.8778 ms_handle_connect on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9fd7400640 1 -- 172.21.15.46:0/29172970 <== mds.0 v2:172.21.15.46:6834/1174892156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556972f8bc60 con 0x5569731fcc80 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9fd7400640 10 client.8778 handle_command_reply: tid=0 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 client.8778 shutdown 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/29172970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 msgr2=0x5569731d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731d1d30 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00cef0 tx=0x7f9fdc000bd0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/29172970 shutdown_connections 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x5569731fcc80 0x5569731ff070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 0x5569731d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731d1d30 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9fd7400640 0 client.8778 ms_handle_reset on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/29172970 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/29172970 >> 172.21.15.46:0/29172970 conn(0x55697319f020 msgr2=0x5569731a1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/29172970 shutdown_connections 2024-04-25T08:51:40.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/29172970 wait complete. 2024-04-25T08:51:40.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.294+0000 7f9ffc2c7000 20 client.8778 trim_cache size 0 max 16384 2024-04-25T08:51:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ffc2c7000 1 Processor -- start 2024-04-25T08:51:40.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ffc2c7000 1 -- start start 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731a2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 0x5569731a1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4203800 con 0x5569731ec9d0 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff42029b0 con 0x5569731ed2f0 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40b4ff0 con 0x5569731a30a0 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9febe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9febe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39374/0 (socket says 172.21.15.46:39374) 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9febe00640 1 -- 172.21.15.46:0/1852428220 learned_addr learned my addr 172.21.15.46:0/1852428220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/1852428220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731a2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9ff2800640 1 --2- 172.21.15.46:0/1852428220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 0x5569731a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9febe00640 1 -- 172.21.15.46:0/1852428220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 msgr2=0x5569731a2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9febe00640 1 --2- 172.21.15.46:0/1852428220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731a2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9febe00640 1 -- 172.21.15.46:0/1852428220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 msgr2=0x5569731a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9febe00640 1 --2- 172.21.15.46:0/1852428220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 0x5569731a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9febe00640 1 -- 172.21.15.46:0/1852428220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556973036de0 con 0x5569731ec9d0 2024-04-25T08:51:40.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9febe00640 1 --2- 172.21.15.46:0/1852428220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731a1410 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc05ddb0 tx=0x7f9fdc06f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e93ee75e9e0f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9fe2000640 1 -- 172.21.15.46:0/1852428220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc055450 con 0x5569731ec9d0 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9fe2000640 1 -- 172.21.15.46:0/1852428220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc081070 con 0x5569731ec9d0 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.310+0000 7f9fe2000640 1 -- 172.21.15.46:0/1852428220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc072170 con 0x5569731ec9d0 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/1852428220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 msgr2=0x5569731a1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/1852428220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731a1410 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc05ddb0 tx=0x7f9fdc06f070 comp rx=0 tx=0).stop 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/1852428220 shutdown_connections 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/1852428220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ed2f0 0x5569731a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/1852428220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ec9d0 0x5569731a1410 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/1852428220 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731a2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/1852428220 >> 172.21.15.46:0/1852428220 conn(0x55697319f020 msgr2=0x5569731f9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/1852428220 shutdown_connections 2024-04-25T08:51:40.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/1852428220 wait complete. 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- start start 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 Processor -- start 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731ff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ec9d0 0x5569731ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569732001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556972f8ae00 con 0x5569731ed2f0 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff42023b0 con 0x5569731ec9d0 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556972fef300 con 0x5569731a30a0 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 10 client.0 _ll_register_callbacks cb 0x55697316e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569732001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569732001f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39384/0 (socket says 172.21.15.46:39384) 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff2800640 1 -- 172.21.15.46:0/988312079 learned_addr learned my addr 172.21.15.46:0/988312079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9febe00640 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ec9d0 0x5569731ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff2800640 1 -- 172.21.15.46:0/988312079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 msgr2=0x5569731ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff2800640 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff2800640 1 -- 172.21.15.46:0/988312079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ec9d0 msgr2=0x5569731ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff2800640 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ec9d0 0x5569731ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff2800640 1 -- 172.21.15.46:0/988312079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556972f918e0 con 0x5569731ed2f0 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ff2800640 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569732001f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f9ff41598b0 tx=0x7f9fec007ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f863086079f9ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9fd7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9fd7400640 1 -- 172.21.15.46:0/988312079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec013e20 con 0x5569731ed2f0 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9fd7400640 1 -- 172.21.15.46:0/988312079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f9fec015410 con 0x5569731ed2f0 2024-04-25T08:51:40.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/988312079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fdc081070 con 0x5569731ed2f0 2024-04-25T08:51:40.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9fd7400640 1 -- 172.21.15.46:0/988312079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fec024050 con 0x5569731ed2f0 2024-04-25T08:51:40.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 10 client.8784 fetch_fsmap learned FSMap version 89 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/988312079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5569730498f0 con 0x5569731ed2f0 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9fd7400640 1 -- 172.21.15.46:0/988312079 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc081070 con 0x5569731ed2f0 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9fd7400640 1 -- 172.21.15.46:0/988312079 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9fec027020 con 0x5569731ed2f0 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 10 client.8784 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 10 client.8784 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.314+0000 7f9ffc2c7000 20 client.8784 populate_metadata read hostname 'smithi046' 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x556973252f50 0x556973255340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 4 client.8784 mds_command: new command op to 8658 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/988312079 --> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556972ffd530 con 0x556973252f50 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x556973252f50 0x556973255340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9fd7400640 10 client.8784 ms_handle_connect on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff1e00640 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x556973252f50 0x556973255340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e3a89c1b16a78bd server_cookie=d7102baf710adc52 in_seq=0 out_seq=0 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9fd7400640 1 -- 172.21.15.46:0/988312079 <== mds.0 v2:172.21.15.46:6834/1174892156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x556972ffd530 con 0x556973252f50 2024-04-25T08:51:40.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9fd7400640 10 client.8784 handle_command_reply: tid=0 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 client.8784 shutdown 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/988312079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 msgr2=0x5569732001f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569732001f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f9ff41598b0 tx=0x7f9fec007ba0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/988312079 shutdown_connections 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x556973252f50 0x556973255340 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569731ed2f0 0x5569732001f0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5569731ec9d0 0x5569731ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 --2- 172.21.15.46:0/988312079 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5569731a30a0 0x5569731ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9fd7400640 0 client.8784 ms_handle_reset on v2:172.21.15.46:6834/1174892156 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/988312079 >> 172.21.15.46:0/988312079 conn(0x55697319f020 msgr2=0x5569731ebfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/988312079 shutdown_connections 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 1 -- 172.21.15.46:0/988312079 wait complete. 2024-04-25T08:51:40.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ffc2c7000 20 client.8784 trim_cache size 0 max 16384 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "eeeda6d8-699e-4220-bc59-79837e0d6705": { 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "eeeda6d8-699e-4220-bc59-79837e0d6705", 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:51:40.314 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:51:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 -- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9fd0066f60 msgr2=0x7f9fd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9fd0066f60 0x7f9fd0069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40af000 tx=0x7f9fe40a7000 comp rx=0 tx=0).stop 2024-04-25T08:51:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 -- 172.21.15.46:0/3645056146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff40ce040 msgr2=0x7f9ff414f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:51:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff40ce040 0x7f9ff414f020 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c530 tx=0x7f9fdc00c9f0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 -- 172.21.15.46:0/3645056146 shutdown_connections 2024-04-25T08:51:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9fd0066f60 0x7f9fd0069420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ff4159300 0x7f9ff40ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ff40ce040 0x7f9ff414f020 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 --2- 172.21.15.46:0/3645056146 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9ff406b320 0x7f9ff414fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:51:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 -- 172.21.15.46:0/3645056146 >> 172.21.15.46:0/3645056146 conn(0x7f9ff406e3e0 msgr2=0x7f9ff415d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:51:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 -- 172.21.15.46:0/3645056146 shutdown_connections 2024-04-25T08:51:40.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.318+0000 7f9ff8e00640 1 -- 172.21.15.46:0/3645056146 wait complete. 2024-04-25T08:51:40.329 INFO:tasks.cephfs.filesystem:scrub status for tag:eeeda6d8-699e-4220-bc59-79837e0d6705 - {'path': '/', 'tag': 'eeeda6d8-699e-4220-bc59-79837e0d6705', 'options': 'recursive,repair'} 2024-04-25T08:51:40.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:40.610+0000 7efd16000640 1 -- 172.21.15.46:0/766103502 <== mon.0 v2:172.21.15.46:3300/0 7 ==== log(8 entries from seq 1 at 2024-04-25T08:51:40.110834+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7efd10086430 con 0x7efd04004810 2024-04-25T08:51:40.600 INFO:teuthology.orchestra.run.smithi046.stdout: cluster: 2024-04-25T08:51:40.601 INFO:teuthology.orchestra.run.smithi046.stdout: id: 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:51:40.601 INFO:teuthology.orchestra.run.smithi046.stdout: health: HEALTH_OK 2024-04-25T08:51:40.601 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:40.601 INFO:teuthology.orchestra.run.smithi046.stdout: services: 2024-04-25T08:51:40.601 INFO:teuthology.orchestra.run.smithi046.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-04-25T08:51:40.602 INFO:teuthology.orchestra.run.smithi046.stdout: mgr: x(active, since 9m), standbys: y 2024-04-25T08:51:40.602 INFO:teuthology.orchestra.run.smithi046.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T08:51:40.602 INFO:teuthology.orchestra.run.smithi046.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-25T08:51:40.602 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:40.602 INFO:teuthology.orchestra.run.smithi046.stdout: data: 2024-04-25T08:51:40.602 INFO:teuthology.orchestra.run.smithi046.stdout: volumes: 1/1 healthy 2024-04-25T08:51:40.603 INFO:teuthology.orchestra.run.smithi046.stdout: pools: 3 pools, 97 pgs 2024-04-25T08:51:40.603 INFO:teuthology.orchestra.run.smithi046.stdout: objects: 30 objects, 19 MiB 2024-04-25T08:51:40.603 INFO:teuthology.orchestra.run.smithi046.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-25T08:51:40.603 INFO:teuthology.orchestra.run.smithi046.stdout: pgs: 97 active+clean 2024-04-25T08:51:40.603 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:40.603 INFO:teuthology.orchestra.run.smithi046.stdout: io: 2024-04-25T08:51:40.604 INFO:teuthology.orchestra.run.smithi046.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-25T08:51:40.604 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:40.604 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:51:40.604 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:40.110834+0000 mds.c [INF] scrub queued for path: / 2024-04-25T08:51:40.604 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:40.110837+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T08:51:40.604 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:40.110840+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T08:51:40.605 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:40.111483+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-25T08:51:40.605 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:40.111488+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-04-25T08:51:40.605 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:40.111825+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-25T08:51:40.605 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:40.111829+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-04-25T08:51:40.605 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:40.112238+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T08:51:45.609 INFO:teuthology.orchestra.run.smithi046.stdout:2024-04-25T08:51:44.373824+0000 mds.c [INF] scrub summary: idle 2024-04-25T08:51:45.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:51:45.618+0000 7efd16000640 1 -- 172.21.15.46:0/766103502 <== mon.0 v2:172.21.15.46:3300/0 8 ==== log(1 entries from seq 9 at 2024-04-25T08:51:44.373824+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7efd100567a0 con 0x7efd04004810 2024-04-25T08:51:51.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:51:51.442 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:52:10.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:10.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T08:52:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8804000640 1 Processor -- start 2024-04-25T08:52:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8804000640 1 -- start start 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8804000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 0x7f87fc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8804000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87fc06dc70 0x7f87fc0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8804000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87fc0cd930 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8804000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87fc0cdbd0 con 0x7f87fc06dc70 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8804000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0cdeb0 con 0x7f87fc145310 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8800c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87fc06dc70 0x7f87fc0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8800c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f87fc06dc70 0x7f87fc0cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53308/0 (socket says 172.21.15.46:53308) 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8802000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87fc0cf1b0 0x7f87fc15d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58124/0 (socket says 172.21.15.46:58124) 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8800c00640 1 -- 172.21.15.46:0/305633011 learned_addr learned my addr 172.21.15.46:0/305633011 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.494+0000 7f8801600640 1 --2- 172.21.15.46:0/305633011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 0x7f87fc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8800c00640 1 -- 172.21.15.46:0/305633011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 msgr2=0x7f87fc06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8800c00640 1 --2- 172.21.15.46:0/305633011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 0x7f87fc06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8800c00640 1 -- 172.21.15.46:0/305633011 >> v1:172.21.15.46:6789/0 conn(0x7f87fc0cf1b0 legacy=0x7f87fc15d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8800c00640 1 -- 172.21.15.46:0/305633011 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc0ce190 con 0x7f87fc06dc70 2024-04-25T08:52:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f87fbe00640 1 -- 172.21.15.46:0/305633011 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390833866 0 0) 0x7f87fc0cd930 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8800c00640 1 --2- 172.21.15.46:0/305633011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc06dc70 0x7f87fc0cec70 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009870 tx=0x7f87e801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809376d220701e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f87fbe00640 1 -- 172.21.15.46:0/305633011 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e80a95f0 con 0x7f87fc06dc70 2024-04-25T08:52:10.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f87fbe00640 1 -- 172.21.15.46:0/305633011 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e80a9790 con 0x7f87fc06dc70 2024-04-25T08:52:10.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f87fbe00640 1 -- 172.21.15.46:0/305633011 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e80a9b60 con 0x7f87fc06dc70 2024-04-25T08:52:10.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8804000640 1 -- 172.21.15.46:0/305633011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc06dc70 msgr2=0x7f87fc0cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8804000640 1 --2- 172.21.15.46:0/305633011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc06dc70 0x7f87fc0cec70 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f87e8009870 tx=0x7f87e801c530 comp rx=0 tx=0).stop 2024-04-25T08:52:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8804000640 1 -- 172.21.15.46:0/305633011 shutdown_connections 2024-04-25T08:52:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8804000640 1 --2- 172.21.15.46:0/305633011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc06dc70 0x7f87fc0cec70 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8804000640 1 --2- 172.21.15.46:0/305633011 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 0x7f87fc06e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.498+0000 7f8804000640 1 -- 172.21.15.46:0/305633011 >> 172.21.15.46:0/305633011 conn(0x7f87fc06ac10 msgr2=0x7f87fc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 -- 172.21.15.46:0/305633011 shutdown_connections 2024-04-25T08:52:10.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 -- 172.21.15.46:0/305633011 wait complete. 2024-04-25T08:52:10.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 Processor -- start 2024-04-25T08:52:10.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 -- start start 2024-04-25T08:52:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc0cf1b0 0x7f87fc158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 0x7f87fc0a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc0a9620 0x7f87fc0a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc15d920 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc15d7a0 con 0x7f87fc0a9620 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8804000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc168510 con 0x7f87fc145310 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8801600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc0cf1b0 0x7f87fc158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8801600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc0cf1b0 0x7f87fc158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53324/0 (socket says 172.21.15.46:53324) 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8801600640 1 -- 172.21.15.46:0/4009134341 learned_addr learned my addr 172.21.15.46:0/4009134341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8800c00640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 0x7f87fc0a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8802000640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc0a9620 0x7f87fc0a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8801600640 1 -- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 msgr2=0x7f87fc0a90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8801600640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 0x7f87fc0a90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8801600640 1 -- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc0a9620 msgr2=0x7f87fc0a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8801600640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc0a9620 0x7f87fc0a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8801600640 1 -- 172.21.15.46:0/4009134341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc140820 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f8801600640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc0cf1b0 0x7f87fc158e80 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e870 tx=0x7f87f000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f87f9600640 1 -- 172.21.15.46:0/4009134341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0017070 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f87f9600640 1 -- 172.21.15.46:0/4009134341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f000c940 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.502+0000 7f87f9600640 1 -- 172.21.15.46:0/4009134341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f000ccf0 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.506+0000 7f8804000640 1 -- 172.21.15.46:0/4009134341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc140b00 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.506+0000 7f8804000640 1 -- 172.21.15.46:0/4009134341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc1410c0 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.506+0000 7f87f9600640 1 -- 172.21.15.46:0/4009134341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87f001a070 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.506+0000 7f87f9600640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87dc066f60 0x7f87dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.506+0000 7f87f9600640 1 -- 172.21.15.46:0/4009134341 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f87f0085420 con 0x7f87fc0cf1b0 2024-04-25T08:52:10.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87dc066f60 0x7f87dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 Processor -- start 2024-04-25T08:52:10.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 -- start start 2024-04-25T08:52:10.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87dc066f60 0x7f87dc069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f87e80ae000 tx=0x7f87e80b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 0x560c01ffddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fb3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fb4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c01e00ff0 con 0x560c01ffe2f0 2024-04-25T08:52:10.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c01d89150 con 0x560c01ffd9d0 2024-04-25T08:52:10.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c01dfd090 con 0x560c01fb40a0 2024-04-25T08:52:10.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fb3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fb3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53326/0 (socket says 172.21.15.46:53326) 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 -- 172.21.15.46:0/3329565052 learned_addr learned my addr 172.21.15.46:0/3329565052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8801600640 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 0x560c01ffddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8802000640 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fb4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 -- 172.21.15.46:0/3329565052 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 msgr2=0x560c01fb4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fb4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 -- 172.21.15.46:0/3329565052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 msgr2=0x560c01ffddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 0x560c01ffddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 -- 172.21.15.46:0/3329565052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c01f970d0 con 0x560c01ffe2f0 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8802000640 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fb4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:10.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8800c00640 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fb3b60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f87e801a160 tx=0x7f87e8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966927a0c9af3d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f87ee000640 1 -- 172.21.15.46:0/3329565052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e80a9d10 con 0x560c01ffe2f0 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f87ee000640 1 -- 172.21.15.46:0/3329565052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e80bc940 con 0x560c01ffe2f0 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f87ee000640 1 -- 172.21.15.46:0/3329565052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e80bccf0 con 0x560c01ffe2f0 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 -- 172.21.15.46:0/3329565052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 msgr2=0x560c01fb3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fb3b60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f87e801a160 tx=0x7f87e8004740 comp rx=0 tx=0).stop 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 -- 172.21.15.46:0/3329565052 shutdown_connections 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fb4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fb3b60 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 --2- 172.21.15.46:0/3329565052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 0x560c01ffddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 -- 172.21.15.46:0/3329565052 >> 172.21.15.46:0/3329565052 conn(0x560c01fb0020 msgr2=0x560c01fb2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 -- 172.21.15.46:0/3329565052 shutdown_connections 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.510+0000 7f8807453000 1 -- 172.21.15.46:0/3329565052 wait complete. 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 -- start start 2024-04-25T08:52:10.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 Processor -- start 2024-04-25T08:52:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fe6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 0x560c01fe7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fe7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c01dc5250 con 0x560c01ffe2f0 2024-04-25T08:52:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c01dbb690 con 0x560c01ffd9d0 2024-04-25T08:52:10.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c01d89a40 con 0x560c01fb40a0 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 10 client.0 _ll_register_callbacks cb 0x560c01f9ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8802000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fe7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8802000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fe7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53334/0 (socket says 172.21.15.46:53334) 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8802000640 1 -- 172.21.15.46:0/1232921944 learned_addr learned my addr 172.21.15.46:0/1232921944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8800c00640 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 0x560c01fe7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8801600640 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fe6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8802000640 1 -- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 msgr2=0x560c01fe6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8802000640 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fe6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8802000640 1 -- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 msgr2=0x560c01fe7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8802000640 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 0x560c01fe7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8802000640 1 -- 172.21.15.46:0/1232921944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c01eae1c0 con 0x560c01ffe2f0 2024-04-25T08:52:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8800c00640 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 0x560c01fe7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8801600640 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fe6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8802000640 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fe7a70 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f87f4007310 tx=0x7f87f40077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea2c673a69b1a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f87e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f87e2a00640 1 -- 172.21.15.46:0/1232921944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4011830 con 0x560c01ffe2f0 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f87e2a00640 1 -- 172.21.15.46:0/1232921944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f87f40119d0 con 0x560c01ffe2f0 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 -- 172.21.15.46:0/1232921944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87e80bc940 con 0x560c01ffe2f0 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f87e2a00640 1 -- 172.21.15.46:0/1232921944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f4022050 con 0x560c01ffe2f0 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 10 client.8799 fetch_fsmap learned FSMap version 89 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 -- 172.21.15.46:0/1232921944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x560c01eaa170 con 0x560c01ffe2f0 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f87e2a00640 1 -- 172.21.15.46:0/1232921944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e80bc940 con 0x560c01ffe2f0 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f87e2a00640 1 -- 172.21.15.46:0/1232921944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f87f4025020 con 0x560c01ffe2f0 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 10 client.8799 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 10 client.8799 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 20 client.8799 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x560c01fe8340 0x560c020141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 4 client.8799 mds_command: new command op to 8658 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.514+0000 7f8807453000 1 -- 172.21.15.46:0/1232921944 --> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c01d9cc60 con 0x560c01fe8340 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8801600640 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x560c01fe8340 0x560c020141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f87e2a00640 10 client.8799 ms_handle_connect on v2:172.21.15.46:6834/1174892156 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8801600640 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x560c01fe8340 0x560c020141a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db38833b498bd9f server_cookie=7a9a8e4e87811432 in_seq=0 out_seq=0 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f87e2a00640 1 -- 172.21.15.46:0/1232921944 <== mds.0 v2:172.21.15.46:6834/1174892156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560c01d9cc60 con 0x560c01fe8340 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f87e2a00640 10 client.8799 handle_command_reply: tid=0 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 client.8799 shutdown 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 -- 172.21.15.46:0/1232921944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 msgr2=0x560c01fe7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fe7a70 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f87f4007310 tx=0x7f87f40077d0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 -- 172.21.15.46:0/1232921944 shutdown_connections 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x560c01fe8340 0x560c020141a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c01fe7a70 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffd9d0 0x560c01fe7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 --2- 172.21.15.46:0/1232921944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01fb40a0 0x560c01fe6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f87e2a00640 0 client.8799 ms_handle_reset on v2:172.21.15.46:6834/1174892156 2024-04-25T08:52:10.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 -- 172.21.15.46:0/1232921944 >> 172.21.15.46:0/1232921944 conn(0x560c01fb0020 msgr2=0x560c01fb2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:10.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 -- 172.21.15.46:0/1232921944 shutdown_connections 2024-04-25T08:52:10.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 1 -- 172.21.15.46:0/1232921944 wait complete. 2024-04-25T08:52:10.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.518+0000 7f8807453000 20 client.8799 trim_cache size 0 max 16384 2024-04-25T08:52:10.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 Processor -- start 2024-04-25T08:52:10.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 -- start start 2024-04-25T08:52:10.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01fb40a0 0x560c01ffc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 0x560c01ffcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffe2f0 0x560c01fb3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc1fef40 con 0x560c01fb40a0 2024-04-25T08:52:10.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0b4080 con 0x560c01ffe2f0 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc143610 con 0x560c01ffd9d0 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8801600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01fb40a0 0x560c01ffc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8801600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01fb40a0 0x560c01ffc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53342/0 (socket says 172.21.15.46:53342) 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8801600640 1 -- 172.21.15.46:0/2585172569 learned_addr learned my addr 172.21.15.46:0/2585172569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8802000640 1 --2- 172.21.15.46:0/2585172569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffe2f0 0x560c01fb3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8800c00640 1 --2- 172.21.15.46:0/2585172569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 0x560c01ffcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8801600640 1 -- 172.21.15.46:0/2585172569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 msgr2=0x560c01ffcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8801600640 1 --2- 172.21.15.46:0/2585172569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 0x560c01ffcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8801600640 1 -- 172.21.15.46:0/2585172569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffe2f0 msgr2=0x560c01fb3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8801600640 1 --2- 172.21.15.46:0/2585172569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffe2f0 0x560c01fb3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8801600640 1 -- 172.21.15.46:0/2585172569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c01e47de0 con 0x560c01fb40a0 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8801600640 1 --2- 172.21.15.46:0/2585172569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01fb40a0 0x560c01ffc7c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f87f000f010 tx=0x7f87f0083610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc0c4a66a5cd968 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f87ed600640 1 -- 172.21.15.46:0/2585172569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0021040 con 0x560c01fb40a0 2024-04-25T08:52:10.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f87ed600640 1 -- 172.21.15.46:0/2585172569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f0056e50 con 0x560c01fb40a0 2024-04-25T08:52:10.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f87ed600640 1 -- 172.21.15.46:0/2585172569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f00850b0 con 0x560c01fb40a0 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 -- 172.21.15.46:0/2585172569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01fb40a0 msgr2=0x560c01ffc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 --2- 172.21.15.46:0/2585172569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01fb40a0 0x560c01ffc7c0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f87f000f010 tx=0x7f87f0083610 comp rx=0 tx=0).stop 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 -- 172.21.15.46:0/2585172569 shutdown_connections 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 --2- 172.21.15.46:0/2585172569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01ffe2f0 0x560c01fb3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 --2- 172.21.15.46:0/2585172569 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 0x560c01ffcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 --2- 172.21.15.46:0/2585172569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01fb40a0 0x560c01ffc7c0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 -- 172.21.15.46:0/2585172569 >> 172.21.15.46:0/2585172569 conn(0x560c01fb0020 msgr2=0x560c01fe0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 -- 172.21.15.46:0/2585172569 shutdown_connections 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.534+0000 7f8807453000 1 -- 172.21.15.46:0/2585172569 wait complete. 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 -- start start 2024-04-25T08:52:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 Processor -- start 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01fb40a0 0x560c020141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 0x560c020146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c02014c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c01d9be00 con 0x560c01ffe2f0 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc0b3a80 con 0x560c01fb40a0 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c01e00300 con 0x560c01ffd9d0 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 10 client.0 _ll_register_callbacks cb 0x560c01f7f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8802000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c02014c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8802000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c02014c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53348/0 (socket says 172.21.15.46:53348) 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8802000640 1 -- 172.21.15.46:0/752762289 learned_addr learned my addr 172.21.15.46:0/752762289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8800c00640 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 0x560c020146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8801600640 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01fb40a0 0x560c020141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8802000640 1 -- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 msgr2=0x560c020146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8802000640 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 0x560c020146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8802000640 1 -- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01fb40a0 msgr2=0x560c020141a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8802000640 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01fb40a0 0x560c020141a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8802000640 1 -- 172.21.15.46:0/752762289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c01da28e0 con 0x560c01ffe2f0 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8800c00640 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 0x560c020146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8801600640 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01fb40a0 0x560c020141a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8802000640 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c02014c20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f87f4007310 tx=0x7f87f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79a7f6421dd10ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f87e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f87e2a00640 1 -- 172.21.15.46:0/752762289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4015880 con 0x560c01ffe2f0 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f87e2a00640 1 -- 172.21.15.46:0/752762289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f87f4015a20 con 0x560c01ffe2f0 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 -- 172.21.15.46:0/752762289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87f0056e50 con 0x560c01ffe2f0 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f87e2a00640 1 -- 172.21.15.46:0/752762289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f4011620 con 0x560c01ffe2f0 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 10 client.8805 fetch_fsmap learned FSMap version 89 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 -- 172.21.15.46:0/752762289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x560c01e5a8f0 con 0x560c01ffe2f0 2024-04-25T08:52:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f87e2a00640 1 -- 172.21.15.46:0/752762289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f4039070 con 0x560c01ffe2f0 2024-04-25T08:52:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f87e2a00640 1 -- 172.21.15.46:0/752762289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 89) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f87f4025070 con 0x560c01ffe2f0 2024-04-25T08:52:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 10 client.8805 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T08:52:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 10 client.8805 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T08:52:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 20 client.8805 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x560c02015160 0x560c02015570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 4 client.8805 mds_command: new command op to 8658 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:52:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8807453000 1 -- 172.21.15.46:0/752762289 --> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c01e0e530 con 0x560c02015160 2024-04-25T08:52:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.538+0000 7f8801600640 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x560c02015160 0x560c02015570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8801600640 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x560c02015160 0x560c02015570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e48c1263be05269 server_cookie=3d2be7eba65f9044 in_seq=0 out_seq=0 2024-04-25T08:52:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f87e2a00640 10 client.8805 ms_handle_connect on v2:172.21.15.46:6834/1174892156 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f87e2a00640 1 -- 172.21.15.46:0/752762289 <== mds.0 v2:172.21.15.46:6834/1174892156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560c01e0e530 con 0x560c02015160 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f87e2a00640 10 client.8805 handle_command_reply: tid=0 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 client.8805 shutdown 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 -- 172.21.15.46:0/752762289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 msgr2=0x560c02014c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c02014c20 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f87f4007310 tx=0x7f87f4014000 comp rx=0 tx=0).stop 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 -- 172.21.15.46:0/752762289 shutdown_connections 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.46:6834/1174892156,v1:172.21.15.46:6835/1174892156] conn(0x560c02015160 0x560c02015570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c01ffe2f0 0x560c02014c20 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x560c01ffd9d0 0x560c020146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f87e2a00640 0 client.8805 ms_handle_reset on v2:172.21.15.46:6834/1174892156 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 --2- 172.21.15.46:0/752762289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c01fb40a0 0x560c020141a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 -- 172.21.15.46:0/752762289 >> 172.21.15.46:0/752762289 conn(0x560c01fb0020 msgr2=0x560c01ffc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 -- 172.21.15.46:0/752762289 shutdown_connections 2024-04-25T08:52:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 1 -- 172.21.15.46:0/752762289 wait complete. 2024-04-25T08:52:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8807453000 20 client.8805 trim_cache size 0 max 16384 2024-04-25T08:52:10.537 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:52:10.537 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-04-25T08:52:10.537 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-04-25T08:52:10.537 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:52:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 -- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87dc066f60 msgr2=0x7f87dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87dc066f60 0x7f87dc069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f87e80ae000 tx=0x7f87e80b3040 comp rx=0 tx=0).stop 2024-04-25T08:52:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 -- 172.21.15.46:0/4009134341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc0cf1b0 msgr2=0x7f87fc158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc0cf1b0 0x7f87fc158e80 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f87f000e870 tx=0x7f87f000ec70 comp rx=0 tx=0).stop 2024-04-25T08:52:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 -- 172.21.15.46:0/4009134341 shutdown_connections 2024-04-25T08:52:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f87dc066f60 0x7f87dc069420 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87fc0a9620 0x7f87fc0a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f87fc145310 0x7f87fc0a90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 --2- 172.21.15.46:0/4009134341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87fc0cf1b0 0x7f87fc158e80 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 -- 172.21.15.46:0/4009134341 >> 172.21.15.46:0/4009134341 conn(0x7f87fc06ac10 msgr2=0x7f87fc15be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:10.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 -- 172.21.15.46:0/4009134341 shutdown_connections 2024-04-25T08:52:10.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.542+0000 7f8804000640 1 -- 172.21.15.46:0/4009134341 wait complete. 2024-04-25T08:52:10.552 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:52:10.653 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:52:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8ca800640 1 Processor -- start 2024-04-25T08:52:10.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8ca800640 1 -- start start 2024-04-25T08:52:10.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8ca800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c4099a00 0x7fd8c4099de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8ca800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8c40ccde0 0x7fd8c40cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8ca800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c409a320 con 0x7fd8c40b2240 2024-04-25T08:52:10.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8ca800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c406db90 con 0x7fd8c40ccde0 2024-04-25T08:52:10.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8ca800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c406bdc0 con 0x7fd8c4099a00 2024-04-25T08:52:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8baa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8c40ccde0 0x7fd8c40cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8baa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8c40ccde0 0x7fd8c40cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53356/0 (socket says 172.21.15.46:53356) 2024-04-25T08:52:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8baa00640 1 -- 172.21.15.46:0/2646231470 learned_addr learned my addr 172.21.15.46:0/2646231470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8c3400640 1 --2- 172.21.15.46:0/2646231470 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c4099a00 0x7fd8c4099de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8baa00640 1 -- 172.21.15.46:0/2646231470 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c4099a00 msgr2=0x7fd8c4099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:10.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8baa00640 1 --2- 172.21.15.46:0/2646231470 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c4099a00 0x7fd8c4099de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8baa00640 1 -- 172.21.15.46:0/2646231470 >> v1:172.21.15.82:6789/0 conn(0x7fd8c40b2240 legacy=0x7fd8c40b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.806+0000 7fd8baa00640 1 -- 172.21.15.46:0/2646231470 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c406c0a0 con 0x7fd8c40ccde0 2024-04-25T08:52:10.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8baa00640 1 --2- 172.21.15.46:0/2646231470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c40ccde0 0x7fd8c40cd260 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009e50 tx=0x7fd8ac01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279cc5352c3bcc40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8c2a00640 1 -- 172.21.15.46:0/2646231470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0a5830 con 0x7fd8c40ccde0 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8c2a00640 1 -- 172.21.15.46:0/2646231470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac0a59d0 con 0x7fd8c40ccde0 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8c2a00640 1 -- 172.21.15.46:0/2646231470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0a5da0 con 0x7fd8c40ccde0 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8ca800640 1 -- 172.21.15.46:0/2646231470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c40ccde0 msgr2=0x7fd8c40cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8ca800640 1 --2- 172.21.15.46:0/2646231470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c40ccde0 0x7fd8c40cd260 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac009e50 tx=0x7fd8ac01c970 comp rx=0 tx=0).stop 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8ca800640 1 -- 172.21.15.46:0/2646231470 shutdown_connections 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8ca800640 1 --2- 172.21.15.46:0/2646231470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c40ccde0 0x7fd8c40cd260 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8ca800640 1 --2- 172.21.15.46:0/2646231470 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c4099a00 0x7fd8c4099de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8ca800640 1 -- 172.21.15.46:0/2646231470 >> 172.21.15.46:0/2646231470 conn(0x7fd8c40b3120 msgr2=0x7fd8c40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8ca800640 1 -- 172.21.15.46:0/2646231470 shutdown_connections 2024-04-25T08:52:10.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.810+0000 7fd8ca800640 1 -- 172.21.15.46:0/2646231470 wait complete. 2024-04-25T08:52:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 Processor -- start 2024-04-25T08:52:10.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 -- start start 2024-04-25T08:52:10.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c4099a00 0x7fd8c40c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8c40b2240 0x7fd8c41540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c40ccde0 0x7fd8c40c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40cd7a0 con 0x7fd8c4099a00 2024-04-25T08:52:10.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40cdb80 con 0x7fd8c40b2240 2024-04-25T08:52:10.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c40cd920 con 0x7fd8c40ccde0 2024-04-25T08:52:10.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c4099a00 0x7fd8c40c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c4099a00 0x7fd8c40c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53368/0 (socket says 172.21.15.46:53368) 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3400640 1 -- 172.21.15.46:0/2106199906 learned_addr learned my addr 172.21.15.46:0/2106199906 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8baa00640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8c40b2240 0x7fd8c41540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3400640 1 -- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c40ccde0 msgr2=0x7fd8c40c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3e00640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c40ccde0 0x7fd8c40c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3400640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c40ccde0 0x7fd8c40c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3400640 1 -- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8c40b2240 msgr2=0x7fd8c41540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3400640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8c40b2240 0x7fd8c41540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3400640 1 -- 172.21.15.46:0/2106199906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c40cffb0 con 0x7fd8c4099a00 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8baa00640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8c40b2240 0x7fd8c41540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3e00640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c40ccde0 0x7fd8c40c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8c3400640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c4099a00 0x7fd8c40c6f70 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400e8a0 tx=0x7fd8b400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8bbe00640 1 -- 172.21.15.46:0/2106199906 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4017070 con 0x7fd8c4099a00 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8bbe00640 1 -- 172.21.15.46:0/2106199906 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400c940 con 0x7fd8c4099a00 2024-04-25T08:52:10.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8bbe00640 1 -- 172.21.15.46:0/2106199906 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400ccf0 con 0x7fd8c4099a00 2024-04-25T08:52:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c40ca660 con 0x7fd8c4099a00 2024-04-25T08:52:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.814+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c40cac20 con 0x7fd8c4099a00 2024-04-25T08:52:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.818+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c4069080 con 0x7fd8c4099a00 2024-04-25T08:52:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.818+0000 7fd8bbe00640 1 -- 172.21.15.46:0/2106199906 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8b401a070 con 0x7fd8c4099a00 2024-04-25T08:52:10.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.818+0000 7fd8bbe00640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8a0066f60 0x7fd8a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.818+0000 7fd8bbe00640 1 -- 172.21.15.46:0/2106199906 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fd8b4085140 con 0x7fd8c4099a00 2024-04-25T08:52:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.818+0000 7fd8baa00640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8a0066f60 0x7fd8a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:10.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.818+0000 7fd8baa00640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8a0066f60 0x7fd8a0069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0ae000 tx=0x7fd8ac0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:10.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:10.822+0000 7fd8bbe00640 1 -- 172.21.15.46:0/2106199906 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8b4055f40 con 0x7fd8c4099a00 2024-04-25T08:52:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.014+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd8c4155d40 con 0x7fd8c4099a00 2024-04-25T08:52:11.008 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:52:11.018+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:52:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.018+0000 7fd8bbe00640 1 -- 172.21.15.46:0/2106199906 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd8b4055840 con 0x7fd8c4099a00 2024-04-25T08:52:11.010 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:52:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8a0066f60 msgr2=0x7fd8a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8a0066f60 0x7fd8a0069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0ae000 tx=0x7fd8ac0a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c4099a00 msgr2=0x7fd8c40c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c4099a00 0x7fd8c40c6f70 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400e8a0 tx=0x7fd8b400eca0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 shutdown_connections 2024-04-25T08:52:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8a0066f60 0x7fd8a0069420 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd8c40ccde0 0x7fd8c40c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8c40b2240 0x7fd8c41540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 --2- 172.21.15.46:0/2106199906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8c4099a00 0x7fd8c40c6f70 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 >> 172.21.15.46:0/2106199906 conn(0x7fd8c40b3120 msgr2=0x7fd8c40737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 shutdown_connections 2024-04-25T08:52:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.022+0000 7fd8ca800640 1 -- 172.21.15.46:0/2106199906 wait complete. 2024-04-25T08:52:11.019 INFO:tasks.ceph.mds.c.smithi046.stderr: -1145> 2024-04-25T08:51:40.106+0000 7f844da00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T08:52:11.019 INFO:tasks.ceph.mds.c.smithi046.stderr: -1142> 2024-04-25T08:51:40.106+0000 7f844da00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T08:52:11.019 INFO:tasks.ceph.mds.c.smithi046.stderr: -1103> 2024-04-25T08:51:40.110+0000 7f844da00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T08:52:11.019 INFO:tasks.ceph.mds.c.smithi046.stderr: -1100> 2024-04-25T08:51:40.110+0000 7f844da00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T08:52:11.034 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-25T08:52:11.062 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:52:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.086+0000 7f8892a18400 1 -- start start 2024-04-25T08:52:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.086+0000 7f8892a18400 1 Processor -- start 2024-04-25T08:52:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad66cdb800 0x55ad66e94580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad66cdb400 0x55ad66e94b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad66cdb000 0x55ad66e95080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad67bdc180 con 0x55ad66cdb400 2024-04-25T08:52:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad67bdc300 con 0x55ad66cdb800 2024-04-25T08:52:11.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad67bdc480 con 0x55ad66cdb000 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad66cdb400 0x55ad66e94b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad66cdb400 0x55ad66e94b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53390/0 (socket says 172.21.15.46:53390) 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890000640 1 -- 172.21.15.46:0/782519058 learned_addr learned my addr 172.21.15.46:0/782519058 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890a00640 1 --2- 172.21.15.46:0/782519058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad66cdb800 0x55ad66e94580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8891400640 1 --2- 172.21.15.46:0/782519058 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad66cdb000 0x55ad66e95080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890000640 1 -- 172.21.15.46:0/782519058 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad66cdb000 msgr2=0x55ad66e95080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890000640 1 --2- 172.21.15.46:0/782519058 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad66cdb000 0x55ad66e95080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890000640 1 -- 172.21.15.46:0/782519058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad66cdb800 msgr2=0x55ad66e94580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890000640 1 --2- 172.21.15.46:0/782519058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad66cdb800 0x55ad66e94580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890000640 1 -- 172.21.15.46:0/782519058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad67a92b40 con 0x55ad66cdb400 2024-04-25T08:52:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890000640 1 --2- 172.21.15.46:0/782519058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad66cdb400 0x55ad66e94b00 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x55ad67b47200 tx=0x55ad66d9d1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c22f8306721fea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f888d800640 1 -- 172.21.15.46:0/782519058 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ad66d104e0 con 0x55ad66cdb400 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f888d800640 1 -- 172.21.15.46:0/782519058 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ad66d241c0 con 0x55ad66cdb400 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f888d800640 1 -- 172.21.15.46:0/782519058 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ad66d10b60 con 0x55ad66cdb400 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 -- 172.21.15.46:0/782519058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad67a92d20 con 0x55ad66cdb400 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 -- 172.21.15.46:0/782519058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ad67a92f00 con 0x55ad66cdb400 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 4 waiting for MDS map... 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f888d800640 1 -- 172.21.15.46:0/782519058 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x55ad67bf0600 con 0x55ad66cdb400 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f888d800640 1 -- 172.21.15.46:0/782519058 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 90) v1 ==== 2723+0+0 (secure 0 0 0) 0x55ad66d31180 con 0x55ad66cdb400 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 4 Got MDS map 90 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 10 main: main 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 4 main: connecting to RADOS... 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 Processor -- start 2024-04-25T08:52:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 -- start start 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad67bf2c00 0x55ad66e95600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf3000 0x55ad66e95b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad67bf3400 0x55ad66e96100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad67bdc600 con 0x55ad67bf3000 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad67bdc780 con 0x55ad67bf2c00 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8892a18400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad67bdc900 con 0x55ad67bf3400 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf3000 0x55ad66e95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf3000 0x55ad66e95b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53392/0 (socket says 172.21.15.46:53392) 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.090+0000 7f8890a00640 1 -- 172.21.15.46:0/3076556480 learned_addr learned my addr 172.21.15.46:0/3076556480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 --2- 172.21.15.46:0/3076556480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad67bf2c00 0x55ad66e95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8890000640 1 --2- 172.21.15.46:0/3076556480 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad67bf3400 0x55ad66e96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8890a00640 1 -- 172.21.15.46:0/3076556480 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad67bf3400 msgr2=0x55ad66e96100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8890a00640 1 --2- 172.21.15.46:0/3076556480 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad67bf3400 0x55ad66e96100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8890a00640 1 -- 172.21.15.46:0/3076556480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad67bf2c00 msgr2=0x55ad66e95600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8890a00640 1 --2- 172.21.15.46:0/3076556480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad67bf2c00 0x55ad66e95600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8890a00640 1 -- 172.21.15.46:0/3076556480 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad67a92d20 con 0x55ad67bf3000 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8890a00640 1 --2- 172.21.15.46:0/3076556480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf3000 0x55ad66e95b80 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x55ad67c58090 tx=0x55ad67c06c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fb01fad894774d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f888b000640 1 -- 172.21.15.46:0/3076556480 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ad66d104e0 con 0x55ad67bf3000 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f888b000640 1 -- 172.21.15.46:0/3076556480 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ad66d24e00 con 0x55ad67bf3000 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f888b000640 1 -- 172.21.15.46:0/3076556480 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ad66d10b60 con 0x55ad67bf3000 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 -- 172.21.15.46:0/3076556480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf3000 msgr2=0x55ad66e95b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 --2- 172.21.15.46:0/3076556480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf3000 0x55ad66e95b80 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x55ad67c58090 tx=0x55ad67c06c00 comp rx=0 tx=0).stop 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 -- 172.21.15.46:0/3076556480 shutdown_connections 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 --2- 172.21.15.46:0/3076556480 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad67bf3400 0x55ad66e96100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 --2- 172.21.15.46:0/3076556480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf3000 0x55ad66e95b80 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 --2- 172.21.15.46:0/3076556480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad67bf2c00 0x55ad66e95600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 -- 172.21.15.46:0/3076556480 >> 172.21.15.46:0/3076556480 conn(0x55ad66cdac00 msgr2=0x55ad66d09800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 -- 172.21.15.46:0/3076556480 shutdown_connections 2024-04-25T08:52:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 -- 172.21.15.46:0/3076556480 wait complete. 2024-04-25T08:52:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 Processor -- start 2024-04-25T08:52:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 -- start start 2024-04-25T08:52:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf2c00 0x55ad66e95600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad67bf3400 0x55ad66e96100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad67bf3800 0x55ad66e96680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad67bddc80 con 0x55ad67bf2c00 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad67bddb00 con 0x55ad67bf3800 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8892a18400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad67bdd980 con 0x55ad67bf3400 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf2c00 0x55ad66e95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf2c00 0x55ad66e95600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53408/0 (socket says 172.21.15.46:53408) 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 -- 172.21.15.46:0/1692185690 learned_addr learned my addr 172.21.15.46:0/1692185690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8890a00640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad67bf3400 0x55ad66e96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8890000640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad67bf3800 0x55ad66e96680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 -- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad67bf3400 msgr2=0x55ad66e96100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad67bf3400 0x55ad66e96100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 -- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad67bf3800 msgr2=0x55ad66e96680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad67bf3800 0x55ad66e96680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 -- 172.21.15.46:0/1692185690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad67a93c20 con 0x55ad67bf2c00 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.094+0000 7f8891400640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf2c00 0x55ad66e95600 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x55ad67b46960 tx=0x55ad67aa4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8889200640 1 -- 172.21.15.46:0/1692185690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ad66d109c0 con 0x55ad67bf2c00 2024-04-25T08:52:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8889200640 1 -- 172.21.15.46:0/1692185690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ad66d24c40 con 0x55ad67bf2c00 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8889200640 1 -- 172.21.15.46:0/1692185690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ad66d104e0 con 0x55ad67bf2c00 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ad67a93e00 con 0x55ad67bf2c00 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad67a921e0 con 0x55ad67bf2c00 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8892a18400 4 main: resolving pool 15 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8889200640 1 -- 172.21.15.46:0/1692185690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55ad67c3ac00 con 0x55ad67bf2c00 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8889200640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55ad67bf3000 0x55ad66e97700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8889200640 1 -- 172.21.15.46:0/1692185690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x55ad67c5c600 con 0x55ad67bf2c00 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8892a18400 4 main: creating IoCtx.. 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8892a18400 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55ad67bf3c00 0x55ad66e96c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x55ad67108000 con 0x55ad67bf3c00 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8890a00640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55ad67bf3000 0x55ad66e97700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8890000640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55ad67bf3c00 0x55ad66e96c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8890a00640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55ad67bf3000 0x55ad66e97700 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55ad67b46b70 tx=0x55ad66d92e40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.098+0000 7f8890000640 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55ad67bf3c00 0x55ad66e96c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8890000640 1 -- 172.21.15.46:0/1692185690 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55ad66de6000 con 0x55ad67bf3c00 2024-04-25T08:52:11.093 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: "0": { 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: "data": { 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 5, 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: "inotable": { 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: "projected_free": [ 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: "start": 1099511627779, 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: "len": 1099511627773 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: "free": [ 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:11.094 INFO:teuthology.orchestra.run.smithi046.stdout: "start": 1099511627779, 2024-04-25T08:52:11.095 INFO:teuthology.orchestra.run.smithi046.stdout: "len": 1099511627773 2024-04-25T08:52:11.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:11.095 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-25T08:52:11.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:11.095 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:11.095 INFO:teuthology.orchestra.run.smithi046.stdout: "result": 0 2024-04-25T08:52:11.095 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:11.095 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:52:11.095 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55ad67bf3c00 msgr2=0x55ad66e96c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55ad67bf3c00 0x55ad66e96c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55ad67bf3000 msgr2=0x55ad66e97700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55ad67bf3000 0x55ad66e97700 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55ad67b46b70 tx=0x55ad66d92e40 comp rx=0 tx=0).stop 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf2c00 msgr2=0x55ad66e95600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 --2- 172.21.15.46:0/1692185690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad67bf2c00 0x55ad66e95600 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x55ad67b46960 tx=0x55ad67aa4000 comp rx=0 tx=0).stop 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8890a00640 1 -- 172.21.15.46:0/1692185690 reap_dead start 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 shutdown_connections 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 >> 172.21.15.46:0/1692185690 conn(0x55ad66cdac00 msgr2=0x55ad66d09800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 shutdown_connections 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/1692185690 wait complete. 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/782519058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad66cdb400 msgr2=0x55ad66e94b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 --2- 172.21.15.46:0/782519058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad66cdb400 0x55ad66e94b00 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x55ad67b47200 tx=0x55ad66d9d1a0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/782519058 shutdown_connections 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 --2- 172.21.15.46:0/782519058 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55ad66cdb000 0x55ad66e95080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 --2- 172.21.15.46:0/782519058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad66cdb400 0x55ad66e94b00 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 --2- 172.21.15.46:0/782519058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ad66cdb800 0x55ad66e94580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/782519058 >> 172.21.15.46:0/782519058 conn(0x55ad66cdbc00 msgr2=0x55ad66d09000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:11.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/782519058 shutdown_connections 2024-04-25T08:52:11.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.102+0000 7f8892a18400 1 -- 172.21.15.46:0/782519058 wait complete. 2024-04-25T08:52:11.099 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.065672, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-25T08:52:11.100 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:52:11.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:52:11.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:52:11.310 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:52:11.310 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:11.311 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:11.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:52:11.319 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:52:11.319 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:11.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:52:11.329 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:11.329 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:11.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:52:11.376 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:52:11.376 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:11.384 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:52:11.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5748c00640 1 Processor -- start 2024-04-25T08:52:11.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5748c00640 1 -- start start 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5748c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f574415a550 0x7f574415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5748c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57440b3e90 0x7f57440b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5748c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f574415a930 0x7f574415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5748c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57440d1010 con 0x7f574415a930 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5748c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574415f000 con 0x7f574415a550 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5748c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57440d0400 con 0x7f57440b3e90 2024-04-25T08:52:11.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f574415a930 0x7f574415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f574415a930 0x7f574415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53412/0 (socket says 172.21.15.46:53412) 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743e00640 1 -- 172.21.15.46:0/1537469114 learned_addr learned my addr 172.21.15.46:0/1537469114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f573be00640 1 --2- 172.21.15.46:0/1537469114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57440b3e90 0x7f57440b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743400640 1 --2- 172.21.15.46:0/1537469114 >> v2:172.21.15.82:3300/0 conn(0x7f574415a550 0x7f574415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743e00640 1 -- 172.21.15.46:0/1537469114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57440b3e90 msgr2=0x7f57440b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743e00640 1 --2- 172.21.15.46:0/1537469114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57440b3e90 0x7f57440b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743e00640 1 -- 172.21.15.46:0/1537469114 >> v2:172.21.15.82:3300/0 conn(0x7f574415a550 msgr2=0x7f574415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743e00640 1 --2- 172.21.15.46:0/1537469114 >> v2:172.21.15.82:3300/0 conn(0x7f574415a550 0x7f574415eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743e00640 1 -- 172.21.15.46:0/1537469114 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57440d06e0 con 0x7f574415a930 2024-04-25T08:52:11.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.570+0000 7f5743400640 1 --2- 172.21.15.46:0/1537469114 >> v2:172.21.15.82:3300/0 conn(0x7f574415a550 0x7f574415eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:11.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5743e00640 1 --2- 172.21.15.46:0/1537469114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f574415a930 0x7f574415ad80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f5744071a00 tx=0x7f573c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb5c045c1ef8284 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5742a00640 1 -- 172.21.15.46:0/1537469114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c0a5680 con 0x7f574415a930 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5742a00640 1 -- 172.21.15.46:0/1537469114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573c0a5820 con 0x7f574415a930 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5742a00640 1 -- 172.21.15.46:0/1537469114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c0a5bf0 con 0x7f574415a930 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 -- 172.21.15.46:0/1537469114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f574415a930 msgr2=0x7f574415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 --2- 172.21.15.46:0/1537469114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f574415a930 0x7f574415ad80 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f5744071a00 tx=0x7f573c01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 -- 172.21.15.46:0/1537469114 shutdown_connections 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 --2- 172.21.15.46:0/1537469114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f574415a930 0x7f574415ad80 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 --2- 172.21.15.46:0/1537469114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f57440b3e90 0x7f57440b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 --2- 172.21.15.46:0/1537469114 >> v2:172.21.15.82:3300/0 conn(0x7f574415a550 0x7f574415eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 -- 172.21.15.46:0/1537469114 >> 172.21.15.46:0/1537469114 conn(0x7f5744099a00 msgr2=0x7f57440b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 -- 172.21.15.46:0/1537469114 shutdown_connections 2024-04-25T08:52:11.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 -- 172.21.15.46:0/1537469114 wait complete. 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 Processor -- start 2024-04-25T08:52:11.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.574+0000 7f5748c00640 1 -- start start 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5748c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57440b3e90 0x7f574409c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5748c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574415a550 0x7f57441595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5748c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f574415a930 0x7f57441f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5748c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57440ad920 con 0x7f57440b3e90 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5748c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57440b61d0 con 0x7f574415a550 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5748c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57440adaa0 con 0x7f574415a930 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57440b3e90 0x7f574409c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57440b3e90 0x7f574409c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53416/0 (socket says 172.21.15.46:53416) 2024-04-25T08:52:11.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743400640 1 -- 172.21.15.46:0/4112006935 learned_addr learned my addr 172.21.15.46:0/4112006935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743e00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f574415a930 0x7f57441f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f573be00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574415a550 0x7f57441595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743400640 1 -- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f574415a930 msgr2=0x7f57441f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743400640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f574415a930 0x7f57441f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743400640 1 -- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574415a550 msgr2=0x7f57441595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743400640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574415a550 0x7f57441595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743400640 1 -- 172.21.15.46:0/4112006935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57441f7030 con 0x7f57440b3e90 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f573be00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574415a550 0x7f57441595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5743400640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57440b3e90 0x7f574409c3f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f573400c7c0 tx=0x7f573400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f573b400640 1 -- 172.21.15.46:0/4112006935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734007a30 con 0x7f57440b3e90 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f573b400640 1 -- 172.21.15.46:0/4112006935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5734007bd0 con 0x7f57440b3e90 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f573b400640 1 -- 172.21.15.46:0/4112006935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57340113f0 con 0x7f57440b3e90 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574409ca20 con 0x7f57440b3e90 2024-04-25T08:52:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574409cfe0 con 0x7f57440b3e90 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5744069010 con 0x7f57440b3e90 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.578+0000 7f573b400640 1 -- 172.21.15.46:0/4112006935 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5734011590 con 0x7f57440b3e90 2024-04-25T08:52:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.582+0000 7f573b400640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5720066f90 0x7f5720069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.582+0000 7f573b400640 1 -- 172.21.15.46:0/4112006935 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f5734085390 con 0x7f57440b3e90 2024-04-25T08:52:11.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.582+0000 7f573be00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5720066f90 0x7f5720069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.582+0000 7f573be00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5720066f90 0x7f5720069450 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f57441f95c0 tx=0x7f572c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.586+0000 7f573b400640 1 -- 172.21.15.46:0/4112006935 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57441f7030 con 0x7f57440b3e90 2024-04-25T08:52:11.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5744099ee0 con 0x7f57440b3e90 2024-04-25T08:52:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f573b400640 1 -- 172.21.15.46:0/4112006935 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4140 (secure 0 0 0) 0x7f5734015020 con 0x7f57440b3e90 2024-04-25T08:52:11.777 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:11.777 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:52:11.019700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:52:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 90 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5720066f90 msgr2=0x7f5720069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5720066f90 0x7f5720069450 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f57441f95c0 tx=0x7f572c008040 comp rx=0 tx=0).stop 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57440b3e90 msgr2=0x7f574409c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57440b3e90 0x7f574409c3f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f573400c7c0 tx=0x7f573400cc80 comp rx=0 tx=0).stop 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 shutdown_connections 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5720066f90 0x7f5720069450 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f574415a930 0x7f57441f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f574415a550 0x7f57441595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 --2- 172.21.15.46:0/4112006935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57440b3e90 0x7f574409c3f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 >> 172.21.15.46:0/4112006935 conn(0x7f5744099a00 msgr2=0x7f57441530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 shutdown_connections 2024-04-25T08:52:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.786+0000 7f5748c00640 1 -- 172.21.15.46:0/4112006935 wait complete. 2024-04-25T08:52:11.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dff200640 1 Processor -- start 2024-04-25T08:52:11.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dff200640 1 -- start start 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dff200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e001450b0 0x7f7e0015bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dff200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e000aeb20 0x7f7e000aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dff200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e000af600 con 0x7f7e0006ac10 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dff200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e000b0210 con 0x7f7e001450b0 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dff200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000af8e0 con 0x7f7e000aeb20 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dfde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e001450b0 0x7f7e0015bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dfde00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e001450b0 0x7f7e0015bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53426/0 (socket says 172.21.15.46:53426) 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dfde00640 1 -- 172.21.15.46:0/1018913850 learned_addr learned my addr 172.21.15.46:0/1018913850 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7df7e00640 1 --2- 172.21.15.46:0/1018913850 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e000aeb20 0x7f7e000aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dfde00640 1 -- 172.21.15.46:0/1018913850 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e000aeb20 msgr2=0x7f7e000aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:11.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dfde00640 1 --2- 172.21.15.46:0/1018913850 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e000aeb20 0x7f7e000aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dfde00640 1 -- 172.21.15.46:0/1018913850 >> v1:172.21.15.82:6789/0 conn(0x7f7e0006ac10 legacy=0x7f7e000b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.950+0000 7f7dfde00640 1 -- 172.21.15.46:0/1018913850 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e000afbc0 con 0x7f7e001450b0 2024-04-25T08:52:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dfde00640 1 --2- 172.21.15.46:0/1018913850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 0x7f7e0015bb70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009870 tx=0x7f7df001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7c20dcaab40dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dfd400640 1 -- 172.21.15.46:0/1018913850 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df00b0070 con 0x7f7e001450b0 2024-04-25T08:52:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dfd400640 1 -- 172.21.15.46:0/1018913850 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df0004230 con 0x7f7e001450b0 2024-04-25T08:52:11.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dfd400640 1 -- 172.21.15.46:0/1018913850 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df0003620 con 0x7f7e001450b0 2024-04-25T08:52:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 -- 172.21.15.46:0/1018913850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 msgr2=0x7f7e0015bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 --2- 172.21.15.46:0/1018913850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 0x7f7e0015bb70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009870 tx=0x7f7df001c560 comp rx=0 tx=0).stop 2024-04-25T08:52:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 -- 172.21.15.46:0/1018913850 shutdown_connections 2024-04-25T08:52:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 --2- 172.21.15.46:0/1018913850 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e000aeb20 0x7f7e000aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 --2- 172.21.15.46:0/1018913850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 0x7f7e0015bb70 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 -- 172.21.15.46:0/1018913850 >> 172.21.15.46:0/1018913850 conn(0x7f7e00099a00 msgr2=0x7f7e000b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 -- 172.21.15.46:0/1018913850 shutdown_connections 2024-04-25T08:52:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 -- 172.21.15.46:0/1018913850 wait complete. 2024-04-25T08:52:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 Processor -- start 2024-04-25T08:52:11.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.954+0000 7f7dff200640 1 -- start start 2024-04-25T08:52:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dff200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e0006ac10 0x7f7e000d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dff200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e000aeb20 0x7f7e0013a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dff200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 0x7f7e0013b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dff200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e00168450 con 0x7f7e001450b0 2024-04-25T08:52:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dff200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001682d0 con 0x7f7e000aeb20 2024-04-25T08:52:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dff200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e001685d0 con 0x7f7e0006ac10 2024-04-25T08:52:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 0x7f7e0013b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 0x7f7e0013b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53432/0 (socket says 172.21.15.46:53432) 2024-04-25T08:52:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfe800640 1 -- 172.21.15.46:0/2061504646 learned_addr learned my addr 172.21.15.46:0/2061504646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfde00640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e0006ac10 0x7f7e000d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7df7e00640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e000aeb20 0x7f7e0013a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfe800640 1 -- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e0006ac10 msgr2=0x7f7e000d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfe800640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e0006ac10 0x7f7e000d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfe800640 1 -- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e000aeb20 msgr2=0x7f7e0013a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfe800640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e000aeb20 0x7f7e0013a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfe800640 1 -- 172.21.15.46:0/2061504646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e00159d30 con 0x7f7e001450b0 2024-04-25T08:52:11.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dfe800640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 0x7f7e0013b270 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c770 tx=0x7f7df800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7df6000640 1 -- 172.21.15.46:0/2061504646 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df8003b80 con 0x7f7e001450b0 2024-04-25T08:52:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7df6000640 1 -- 172.21.15.46:0/2061504646 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df8003d20 con 0x7f7e001450b0 2024-04-25T08:52:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7df6000640 1 -- 172.21.15.46:0/2061504646 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df800aa90 con 0x7f7e001450b0 2024-04-25T08:52:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0015a010 con 0x7f7e001450b0 2024-04-25T08:52:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0015a5d0 con 0x7f7e001450b0 2024-04-25T08:52:11.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.958+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e00069010 con 0x7f7e001450b0 2024-04-25T08:52:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.962+0000 7f7df6000640 1 -- 172.21.15.46:0/2061504646 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7df800ac30 con 0x7f7e001450b0 2024-04-25T08:52:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.962+0000 7f7df6000640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7ddc066f90 0x7f7ddc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:11.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.962+0000 7f7df6000640 1 -- 172.21.15.46:0/2061504646 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f7df80850e0 con 0x7f7e001450b0 2024-04-25T08:52:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.962+0000 7f7dfde00640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7ddc066f90 0x7f7ddc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:11.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.962+0000 7f7dfde00640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7ddc066f90 0x7f7ddc069450 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7df00ad000 tx=0x7f7df00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:11.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:11.966+0000 7f7df6000640 1 -- 172.21.15.46:0/2061504646 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7df80557b0 con 0x7f7e001450b0 2024-04-25T08:52:12.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.134+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e000d10a0 con 0x7f7e001450b0 2024-04-25T08:52:12.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.134+0000 7f7df6000640 1 -- 172.21.15.46:0/2061504646 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17143 (secure 0 0 0) 0x7f7df8059850 con 0x7f7e001450b0 2024-04-25T08:52:12.126 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:12.126 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:11.019712+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:50:45.647599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:50:46.857198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1174892156":"2024-04-26T08:52:11.019690+0000","172.21.15.46:6834/1174892156":"2024-04-26T08:52:11.019690+0000","172.21.15.46:6835/623970467":"2024-04-26T08:51:34.299560+0000","172.21.15.46:6834/623970467":"2024-04-26T08:51:34.299560+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7ddc066f90 msgr2=0x7f7ddc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7ddc066f90 0x7f7ddc069450 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7df00ad000 tx=0x7f7df00a6000 comp rx=0 tx=0).stop 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 msgr2=0x7f7e0013b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 0x7f7e0013b270 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f7df800c770 tx=0x7f7df800cc30 comp rx=0 tx=0).stop 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 shutdown_connections 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7ddc066f90 0x7f7ddc069450 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e001450b0 0x7f7e0013b270 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e000aeb20 0x7f7e0013a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 --2- 172.21.15.46:0/2061504646 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7e0006ac10 0x7f7e000d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 >> 172.21.15.46:0/2061504646 conn(0x7f7e00099a00 msgr2=0x7f7e0015cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 shutdown_connections 2024-04-25T08:52:12.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.138+0000 7f7dff200640 1 -- 172.21.15.46:0/2061504646 wait complete. 2024-04-25T08:52:12.148 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:50:45.647599+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:50:46.857198+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:52:12.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:12.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08fe600640 1 Processor -- start 2024-04-25T08:52:12.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08fe600640 1 -- start start 2024-04-25T08:52:12.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08fe600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f806b230 0x7f08f80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08fe600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f806b610 con 0x7f08f8162380 2024-04-25T08:52:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08fe600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f806f980 con 0x7f08f80ce220 2024-04-25T08:52:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08fe600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f806fc60 con 0x7f08f806b230 2024-04-25T08:52:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f08f8162380 0x7f08f80cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58126/0 (socket says 172.21.15.46:58126) 2024-04-25T08:52:12.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f7e00640 1 -- 172.21.15.46:0/2794629333 learned_addr learned my addr 172.21.15.46:0/2794629333 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f7400640 1 --2- 172.21.15.46:0/2794629333 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f806b230 0x7f08f80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f6a00640 1 -- 172.21.15.46:0/2794629333 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890999516 0 0) 0x7f08f806b610 con 0x7f08f8162380 2024-04-25T08:52:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f6a00640 1 -- 172.21.15.46:0/2794629333 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08dc003530 con 0x7f08f8162380 2024-04-25T08:52:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f6a00640 1 -- 172.21.15.46:0/2794629333 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334141909 0 0) 0x7f08f806f980 con 0x7f08f80ce220 2024-04-25T08:52:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f6a00640 1 -- 172.21.15.46:0/2794629333 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f806b610 con 0x7f08f80ce220 2024-04-25T08:52:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f7400640 1 -- 172.21.15.46:0/2794629333 >> v1:172.21.15.82:6789/0 conn(0x7f08f80ce220 legacy=0x7f08f8161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f7400640 1 -- 172.21.15.46:0/2794629333 >> v1:172.21.15.46:6789/0 conn(0x7f08f8162380 legacy=0x7f08f80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.302+0000 7f08f6a00640 1 -- 172.21.15.46:0/2794629333 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2311131884 0 0) 0x7f08dc003530 con 0x7f08f8162380 2024-04-25T08:52:12.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08f7400640 1 -- 172.21.15.46:0/2794629333 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f806ff40 con 0x7f08f806b230 2024-04-25T08:52:12.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08f7400640 1 --2- 172.21.15.46:0/2794629333 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f806b230 0x7f08f80cccf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08e80097f0 tx=0x7f08e801c2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45cc32f22b19de25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08f6a00640 1 -- 172.21.15.46:0/2794629333 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e80b0070 con 0x7f08f806b230 2024-04-25T08:52:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08f6a00640 1 -- 172.21.15.46:0/2794629333 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e80ab400 con 0x7f08f806b230 2024-04-25T08:52:12.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08f6a00640 1 -- 172.21.15.46:0/2794629333 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0001680 con 0x7f08f806b230 2024-04-25T08:52:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08fe600640 1 -- 172.21.15.46:0/2794629333 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f806b230 msgr2=0x7f08f80cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08fe600640 1 --2- 172.21.15.46:0/2794629333 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f806b230 0x7f08f80cccf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08e80097f0 tx=0x7f08e801c2b0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08fe600640 1 -- 172.21.15.46:0/2794629333 shutdown_connections 2024-04-25T08:52:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08fe600640 1 --2- 172.21.15.46:0/2794629333 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f806b230 0x7f08f80cccf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08fe600640 1 -- 172.21.15.46:0/2794629333 >> 172.21.15.46:0/2794629333 conn(0x7f08f8099a00 msgr2=0x7f08f80b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08fe600640 1 -- 172.21.15.46:0/2794629333 shutdown_connections 2024-04-25T08:52:12.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.306+0000 7f08fe600640 1 -- 172.21.15.46:0/2794629333 wait complete. 2024-04-25T08:52:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08fe600640 1 Processor -- start 2024-04-25T08:52:12.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08fe600640 1 -- start start 2024-04-25T08:52:12.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08fe600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08f806b230 0x7f08f815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08fe600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f8162380 0x7f08f80b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08fe600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f8155560 0x7f08f815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08fe600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f80d0120 con 0x7f08f8162380 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08fe600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f80cffa0 con 0x7f08f806b230 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08fe600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f80d02a0 con 0x7f08f8155560 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f8162380 0x7f08f80b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08eea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f8162380 0x7f08f80b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53438/0 (socket says 172.21.15.46:53438) 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08eea00640 1 -- 172.21.15.46:0/45317454 learned_addr learned my addr 172.21.15.46:0/45317454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08f7400640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08f806b230 0x7f08f815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08f7e00640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f8155560 0x7f08f815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08eea00640 1 -- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f8155560 msgr2=0x7f08f815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08eea00640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f8155560 0x7f08f815f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08eea00640 1 -- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08f806b230 msgr2=0x7f08f815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08eea00640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08f806b230 0x7f08f815cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08eea00640 1 -- 172.21.15.46:0/45317454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f806f2c0 con 0x7f08f8162380 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08eea00640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f8162380 0x7f08f80b38b0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c7a0 tx=0x7f08e000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:12.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08efe00640 1 -- 172.21.15.46:0/45317454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0003b80 con 0x7f08f8162380 2024-04-25T08:52:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08efe00640 1 -- 172.21.15.46:0/45317454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e0003d40 con 0x7f08f8162380 2024-04-25T08:52:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08efe00640 1 -- 172.21.15.46:0/45317454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e000aa90 con 0x7f08f8162380 2024-04-25T08:52:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.310+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f815c070 con 0x7f08f8162380 2024-04-25T08:52:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.314+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f815c610 con 0x7f08f8162380 2024-04-25T08:52:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.314+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f8069010 con 0x7f08f8162380 2024-04-25T08:52:12.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.314+0000 7f08efe00640 1 -- 172.21.15.46:0/45317454 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08e000ac30 con 0x7f08f8162380 2024-04-25T08:52:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.314+0000 7f08efe00640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08d4066f60 0x7f08d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.314+0000 7f08efe00640 1 -- 172.21.15.46:0/45317454 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f08e0085060 con 0x7f08f8162380 2024-04-25T08:52:12.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.314+0000 7f08f7400640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08d4066f60 0x7f08d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.318+0000 7f08f7400640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08d4066f60 0x7f08d4069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f08e80ad000 tx=0x7f08e80a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:12.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.318+0000 7f08efe00640 1 -- 172.21.15.46:0/45317454 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08e0055840 con 0x7f08f8162380 2024-04-25T08:52:12.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.482+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08f806b810 con 0x7f08f8162380 2024-04-25T08:52:12.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.482+0000 7f08efe00640 1 -- 172.21.15.46:0/45317454 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08e00598e0 con 0x7f08f8162380 2024-04-25T08:52:12.475 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:52:12.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08d4066f60 msgr2=0x7f08d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08d4066f60 0x7f08d4069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f08e80ad000 tx=0x7f08e80a6000 comp rx=0 tx=0).stop 2024-04-25T08:52:12.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f8162380 msgr2=0x7f08f80b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f8162380 0x7f08f80b38b0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c7a0 tx=0x7f08e000cc60 comp rx=0 tx=0).stop 2024-04-25T08:52:12.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 shutdown_connections 2024-04-25T08:52:12.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f08d4066f60 0x7f08d4069420 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f08f8155560 0x7f08f815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08f8162380 0x7f08f80b38b0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 --2- 172.21.15.46:0/45317454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08f806b230 0x7f08f815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 >> 172.21.15.46:0/45317454 conn(0x7f08f8099a00 msgr2=0x7f08f80b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 shutdown_connections 2024-04-25T08:52:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.486+0000 7f08fe600640 1 -- 172.21.15.46:0/45317454 wait complete. 2024-04-25T08:52:12.495 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:52:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.646+0000 7f9f66400640 1 Processor -- start 2024-04-25T08:52:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f66400640 1 -- start start 2024-04-25T08:52:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f66400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 0x7f9f6006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f66400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f6015d090 0x7f9f6015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f66400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f600cd930 con 0x7f9f6006dc70 2024-04-25T08:52:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f66400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f600cdbd0 con 0x7f9f6015d090 2024-04-25T08:52:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f66400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600cdeb0 con 0x7f9f60145310 2024-04-25T08:52:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f57e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9f6006dc70 0x7f9f600cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58134/0 (socket says 172.21.15.46:58134) 2024-04-25T08:52:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f57e00640 1 -- 172.21.15.46:0/1090564299 learned_addr learned my addr 172.21.15.46:0/1090564299 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:12.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f5fe00640 1 --2- 172.21.15.46:0/1090564299 >> v2:172.21.15.46:3300/0 conn(0x7f9f6015d090 0x7f9f6015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f5f400640 1 --2- 172.21.15.46:0/1090564299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 0x7f9f6006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f5fe00640 1 -- 172.21.15.46:0/1090564299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 msgr2=0x7f9f6006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f5fe00640 1 --2- 172.21.15.46:0/1090564299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 0x7f9f6006e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f5fe00640 1 -- 172.21.15.46:0/1090564299 >> v1:172.21.15.46:6789/0 conn(0x7f9f6006dc70 legacy=0x7f9f600cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f5fe00640 1 -- 172.21.15.46:0/1090564299 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600ce190 con 0x7f9f6015d090 2024-04-25T08:52:12.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.650+0000 7f9f5ea00640 1 -- 172.21.15.46:0/1090564299 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129399101 0 0) 0x7f9f600cd930 con 0x7f9f6006dc70 2024-04-25T08:52:12.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f5fe00640 1 --2- 172.21.15.46:0/1090564299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 0x7f9f6015d540 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009d90 tx=0x7f9f5801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244a3006f4061f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:12.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f5ea00640 1 -- 172.21.15.46:0/1090564299 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f580b2070 con 0x7f9f6015d090 2024-04-25T08:52:12.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f5ea00640 1 -- 172.21.15.46:0/1090564299 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f580a5920 con 0x7f9f6015d090 2024-04-25T08:52:12.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f5ea00640 1 -- 172.21.15.46:0/1090564299 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f580a5cf0 con 0x7f9f6015d090 2024-04-25T08:52:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 -- 172.21.15.46:0/1090564299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 msgr2=0x7f9f6015d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 --2- 172.21.15.46:0/1090564299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 0x7f9f6015d540 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009d90 tx=0x7f9f5801c970 comp rx=0 tx=0).stop 2024-04-25T08:52:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 -- 172.21.15.46:0/1090564299 shutdown_connections 2024-04-25T08:52:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 --2- 172.21.15.46:0/1090564299 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 0x7f9f6015d540 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 --2- 172.21.15.46:0/1090564299 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 0x7f9f6006e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 -- 172.21.15.46:0/1090564299 >> 172.21.15.46:0/1090564299 conn(0x7f9f6006ac10 msgr2=0x7f9f6006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 -- 172.21.15.46:0/1090564299 shutdown_connections 2024-04-25T08:52:12.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 -- 172.21.15.46:0/1090564299 wait complete. 2024-04-25T08:52:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 Processor -- start 2024-04-25T08:52:12.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.654+0000 7f9f66400640 1 -- start start 2024-04-25T08:52:12.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f66400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 0x7f9f60159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f66400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 0x7f9f600a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f66400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f60158690 0x7f9f600ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f66400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60168510 con 0x7f9f6015d090 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f66400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f600cf380 con 0x7f9f60158690 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f66400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f60168690 con 0x7f9f60145310 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 0x7f9f600a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 0x7f9f600a9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53462/0 (socket says 172.21.15.46:53462) 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57e00640 1 -- 172.21.15.46:0/4170028940 learned_addr learned my addr 172.21.15.46:0/4170028940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f5f400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 0x7f9f60159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f5fe00640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f60158690 0x7f9f600ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57e00640 1 -- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 msgr2=0x7f9f60159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57e00640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 0x7f9f60159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57e00640 1 -- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f60158690 msgr2=0x7f9f600ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57e00640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f60158690 0x7f9f600ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57e00640 1 -- 172.21.15.46:0/4170028940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600aa050 con 0x7f9f6015d090 2024-04-25T08:52:12.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57e00640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 0x7f9f600a9800 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9f480011f0 tx=0x7f9f4800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:12.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57400640 1 -- 172.21.15.46:0/4170028940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48018070 con 0x7f9f6015d090 2024-04-25T08:52:12.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57400640 1 -- 172.21.15.46:0/4170028940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f480074e0 con 0x7f9f6015d090 2024-04-25T08:52:12.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f600aa330 con 0x7f9f6015d090 2024-04-25T08:52:12.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57400640 1 -- 172.21.15.46:0/4170028940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48004a10 con 0x7f9f6015d090 2024-04-25T08:52:12.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f600aa8f0 con 0x7f9f6015d090 2024-04-25T08:52:12.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f60069010 con 0x7f9f6015d090 2024-04-25T08:52:12.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.658+0000 7f9f57400640 1 -- 172.21.15.46:0/4170028940 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f4801b070 con 0x7f9f6015d090 2024-04-25T08:52:12.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.662+0000 7f9f57400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f3c066f90 0x7f9f3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.662+0000 7f9f57400640 1 -- 172.21.15.46:0/4170028940 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f9f480845e0 con 0x7f9f6015d090 2024-04-25T08:52:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.662+0000 7f9f5f400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f3c066f90 0x7f9f3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.662+0000 7f9f5f400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f3c066f90 0x7f9f3c069450 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9f50007a60 tx=0x7f9f50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:12.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.666+0000 7f9f57400640 1 -- 172.21.15.46:0/4170028940 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f48054cb0 con 0x7f9f6015d090 2024-04-25T08:52:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9f600a9d40 con 0x7f9f6015d090 2024-04-25T08:52:12.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f57400640 1 -- 172.21.15.46:0/4170028940 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9f48058d50 con 0x7f9f6015d090 2024-04-25T08:52:12.821 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:52:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f3c066f90 msgr2=0x7f9f3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f3c066f90 0x7f9f3c069450 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9f50007a60 tx=0x7f9f50008040 comp rx=0 tx=0).stop 2024-04-25T08:52:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 msgr2=0x7f9f600a9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 0x7f9f600a9800 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9f480011f0 tx=0x7f9f4800c9b0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 shutdown_connections 2024-04-25T08:52:12.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f3c066f90 0x7f9f3c069450 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f60158690 0x7f9f600ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f6015d090 0x7f9f600a9800 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 --2- 172.21.15.46:0/4170028940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f60145310 0x7f9f60159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 >> 172.21.15.46:0/4170028940 conn(0x7f9f6006ac10 msgr2=0x7f9f600b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 shutdown_connections 2024-04-25T08:52:12.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.830+0000 7f9f66400640 1 -- 172.21.15.46:0/4170028940 wait complete. 2024-04-25T08:52:12.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:52:12.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.994+0000 7fce1fc00640 1 Processor -- start 2024-04-25T08:52:12.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.994+0000 7fce1fc00640 1 -- start start 2024-04-25T08:52:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1fc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fce18145310 0x7fce1806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1fc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1815d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1fc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce180cd930 con 0x7fce1806dc70 2024-04-25T08:52:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1fc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce180cdbd0 con 0x7fce18145310 2024-04-25T08:52:12.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1fc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce180cdeb0 con 0x7fce1815d090 2024-04-25T08:52:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1d200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fce18145310 0x7fce1806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1dc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1dc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1815d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:57776/0 (socket says 172.21.15.46:57776) 2024-04-25T08:52:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1dc00640 1 -- 172.21.15.46:0/2113505852 learned_addr learned my addr 172.21.15.46:0/2113505852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1dc00640 1 -- 172.21.15.46:0/2113505852 >> v2:172.21.15.82:3300/0 conn(0x7fce18145310 msgr2=0x7fce1806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1dc00640 1 --2- 172.21.15.46:0/2113505852 >> v2:172.21.15.82:3300/0 conn(0x7fce18145310 0x7fce1806e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1dc00640 1 -- 172.21.15.46:0/2113505852 >> v1:172.21.15.82:6789/0 conn(0x7fce1806dc70 legacy=0x7fce180cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:12.998+0000 7fce1dc00640 1 -- 172.21.15.46:0/2113505852 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce180ce190 con 0x7fce1815d090 2024-04-25T08:52:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1dc00640 1 --2- 172.21.15.46:0/2113505852 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1815d520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fce10009e50 tx=0x7fce1001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59082b79ea9d6130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce17e00640 1 -- 172.21.15.46:0/2113505852 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce100a57f0 con 0x7fce1815d090 2024-04-25T08:52:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce17e00640 1 -- 172.21.15.46:0/2113505852 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce100a5990 con 0x7fce1815d090 2024-04-25T08:52:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce17e00640 1 -- 172.21.15.46:0/2113505852 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce100a5d60 con 0x7fce1815d090 2024-04-25T08:52:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 -- 172.21.15.46:0/2113505852 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 msgr2=0x7fce1815d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 --2- 172.21.15.46:0/2113505852 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1815d520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fce10009e50 tx=0x7fce1001ca30 comp rx=0 tx=0).stop 2024-04-25T08:52:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 -- 172.21.15.46:0/2113505852 shutdown_connections 2024-04-25T08:52:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 --2- 172.21.15.46:0/2113505852 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1815d520 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 --2- 172.21.15.46:0/2113505852 >> v2:172.21.15.82:3300/0 conn(0x7fce18145310 0x7fce1806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 -- 172.21.15.46:0/2113505852 >> 172.21.15.46:0/2113505852 conn(0x7fce1806ac10 msgr2=0x7fce1806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 -- 172.21.15.46:0/2113505852 shutdown_connections 2024-04-25T08:52:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 -- 172.21.15.46:0/2113505852 wait complete. 2024-04-25T08:52:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 Processor -- start 2024-04-25T08:52:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 -- start start 2024-04-25T08:52:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.002+0000 7fce1fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce18145310 0x7fce180b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1fc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1813ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1fc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce18139070 0x7fce1813e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1fc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce18168510 con 0x7fce18145310 2024-04-25T08:52:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1fc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce180cf380 con 0x7fce18139070 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1fc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce18168690 con 0x7fce1815d090 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce18145310 0x7fce180b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce18145310 0x7fce180b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53478/0 (socket says 172.21.15.46:53478) 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1d200640 1 -- 172.21.15.46:0/4139206830 learned_addr learned my addr 172.21.15.46:0/4139206830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1dc00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce18139070 0x7fce1813e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce0fe00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1813ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1d200640 1 -- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 msgr2=0x7fce1813ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1d200640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1813ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1d200640 1 -- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce18139070 msgr2=0x7fce1813e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1d200640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce18139070 0x7fce1813e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1d200640 1 -- 172.21.15.46:0/4139206830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1813d360 con 0x7fce18145310 2024-04-25T08:52:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1d200640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce18145310 0x7fce180b4390 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fce0800e960 tx=0x7fce0800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce15600640 1 -- 172.21.15.46:0/4139206830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce08017070 con 0x7fce18145310 2024-04-25T08:52:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce15600640 1 -- 172.21.15.46:0/4139206830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0800c9a0 con 0x7fce18145310 2024-04-25T08:52:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce15600640 1 -- 172.21.15.46:0/4139206830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0800cd50 con 0x7fce18145310 2024-04-25T08:52:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1813d600 con 0x7fce18145310 2024-04-25T08:52:12.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1813dbc0 con 0x7fce18145310 2024-04-25T08:52:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce18069010 con 0x7fce18145310 2024-04-25T08:52:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.006+0000 7fce15600640 1 -- 172.21.15.46:0/4139206830 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce0801a070 con 0x7fce18145310 2024-04-25T08:52:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.010+0000 7fce15600640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdf4066f60 0x7fcdf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.010+0000 7fce15600640 1 -- 172.21.15.46:0/4139206830 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7fce08085130 con 0x7fce18145310 2024-04-25T08:52:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.010+0000 7fce0fe00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdf4066f60 0x7fcdf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.010+0000 7fce0fe00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdf4066f60 0x7fcdf4069420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fce180ae970 tx=0x7fce00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:13.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.014+0000 7fce15600640 1 -- 172.21.15.46:0/4139206830 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce08055930 con 0x7fce18145310 2024-04-25T08:52:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce180b10b0 con 0x7fce18145310 2024-04-25T08:52:13.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce15600640 1 -- 172.21.15.46:0/4139206830 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce080599d0 con 0x7fce18145310 2024-04-25T08:52:13.165 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:52:13.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdf4066f60 msgr2=0x7fcdf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdf4066f60 0x7fcdf4069420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fce180ae970 tx=0x7fce00009040 comp rx=0 tx=0).stop 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce18145310 msgr2=0x7fce180b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce18145310 0x7fce180b4390 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fce0800e960 tx=0x7fce0800ee20 comp rx=0 tx=0).stop 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 shutdown_connections 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fcdf4066f60 0x7fcdf4069420 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce18139070 0x7fce1813e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fce1815d090 0x7fce1813ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 --2- 172.21.15.46:0/4139206830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce18145310 0x7fce180b4390 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 >> 172.21.15.46:0/4139206830 conn(0x7fce1806ac10 msgr2=0x7fce181443c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 shutdown_connections 2024-04-25T08:52:13.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.174+0000 7fce1fc00640 1 -- 172.21.15.46:0/4139206830 wait complete. 2024-04-25T08:52:13.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.334+0000 7f6aa4200640 1 Processor -- start 2024-04-25T08:52:13.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.334+0000 7f6aa4200640 1 -- start start 2024-04-25T08:52:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa4200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa4200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9c06bdc0 con 0x7f6a9c0b2240 2024-04-25T08:52:13.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa4200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a9c06c0a0 con 0x7f6a9c099a00 2024-04-25T08:52:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa4200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c06db90 con 0x7f6a9c0ccde0 2024-04-25T08:52:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa2200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6a9c0b2240 0x7f6a9c0b26d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58144/0 (socket says 172.21.15.46:58144) 2024-04-25T08:52:13.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa2200640 1 -- 172.21.15.46:0/1273953062 learned_addr learned my addr 172.21.15.46:0/1273953062 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa0e00640 1 --2- 172.21.15.46:0/1273953062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6a9be00640 1 -- 172.21.15.46:0/1273953062 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773150556 0 0) 0x7f6a9c06bdc0 con 0x7f6a9c0b2240 2024-04-25T08:52:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6a9be00640 1 -- 172.21.15.46:0/1273953062 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a84003530 con 0x7f6a9c0b2240 2024-04-25T08:52:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa0e00640 1 -- 172.21.15.46:0/1273953062 >> v1:172.21.15.82:6789/0 conn(0x7f6a9c099a00 legacy=0x7f6a9c099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6a9be00640 1 -- 172.21.15.46:0/1273953062 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199800728 0 0) 0x7f6a9c06c0a0 con 0x7f6a9c099a00 2024-04-25T08:52:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa0e00640 1 -- 172.21.15.46:0/1273953062 >> v1:172.21.15.46:6789/0 conn(0x7f6a9c0b2240 legacy=0x7f6a9c0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.338+0000 7f6aa0e00640 1 -- 172.21.15.46:0/1273953062 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c06c380 con 0x7f6a9c0ccde0 2024-04-25T08:52:13.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa0e00640 1 --2- 172.21.15.46:0/1273953062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c0cd260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a880097f0 tx=0x7f6a8801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb2b14c3a63e561 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6a9be00640 1 -- 172.21.15.46:0/1273953062 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a880b1070 con 0x7f6a9c0ccde0 2024-04-25T08:52:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6a9be00640 1 -- 172.21.15.46:0/1273953062 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a880ac410 con 0x7f6a9c0ccde0 2024-04-25T08:52:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6a9be00640 1 -- 172.21.15.46:0/1273953062 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a880ac7e0 con 0x7f6a9c0ccde0 2024-04-25T08:52:13.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa4200640 1 -- 172.21.15.46:0/1273953062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 msgr2=0x7f6a9c0cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa4200640 1 --2- 172.21.15.46:0/1273953062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c0cd260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a880097f0 tx=0x7f6a8801c420 comp rx=0 tx=0).stop 2024-04-25T08:52:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa4200640 1 -- 172.21.15.46:0/1273953062 shutdown_connections 2024-04-25T08:52:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa4200640 1 --2- 172.21.15.46:0/1273953062 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c0cd260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa4200640 1 -- 172.21.15.46:0/1273953062 >> 172.21.15.46:0/1273953062 conn(0x7f6a9c0b3120 msgr2=0x7f6a9c0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa4200640 1 -- 172.21.15.46:0/1273953062 shutdown_connections 2024-04-25T08:52:13.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa4200640 1 -- 172.21.15.46:0/1273953062 wait complete. 2024-04-25T08:52:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa4200640 1 Processor -- start 2024-04-25T08:52:13.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.342+0000 7f6aa4200640 1 -- start start 2024-04-25T08:52:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa4200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a9c099a00 0x7f6a9c0b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa4200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c0b2240 0x7f6a9c157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa4200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa4200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0cd920 con 0x7f6a9c099a00 2024-04-25T08:52:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa4200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0cd7a0 con 0x7f6a9c0b2240 2024-04-25T08:52:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa4200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9c0b2de0 con 0x7f6a9c0ccde0 2024-04-25T08:52:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a9c099a00 0x7f6a9c0b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a9c099a00 0x7f6a9c0b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53494/0 (socket says 172.21.15.46:53494) 2024-04-25T08:52:13.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa1800640 1 -- 172.21.15.46:0/510908944 learned_addr learned my addr 172.21.15.46:0/510908944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa1800640 1 -- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 msgr2=0x7f6a9c157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa0e00640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c0b2240 0x7f6a9c157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa2200640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa1800640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa1800640 1 -- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c0b2240 msgr2=0x7f6a9c157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa1800640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c0b2240 0x7f6a9c157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa1800640 1 -- 172.21.15.46:0/510908944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9c0b57e0 con 0x7f6a9c099a00 2024-04-25T08:52:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa0e00640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c0b2240 0x7f6a9c157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa2200640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa1800640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a9c099a00 0x7f6a9c0b6a00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f6a90001ec0 tx=0x7f6a9000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6a99600640 1 -- 172.21.15.46:0/510908944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90017070 con 0x7f6a9c099a00 2024-04-25T08:52:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6a99600640 1 -- 172.21.15.46:0/510908944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90003600 con 0x7f6a9c099a00 2024-04-25T08:52:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6a99600640 1 -- 172.21.15.46:0/510908944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900039f0 con 0x7f6a9c099a00 2024-04-25T08:52:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9c0b5ac0 con 0x7f6a9c099a00 2024-04-25T08:52:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9c0b6080 con 0x7f6a9c099a00 2024-04-25T08:52:13.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9c069080 con 0x7f6a9c099a00 2024-04-25T08:52:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.346+0000 7f6a99600640 1 -- 172.21.15.46:0/510908944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a9001a070 con 0x7f6a9c099a00 2024-04-25T08:52:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.350+0000 7f6a99600640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a7c066f60 0x7f6a7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.350+0000 7f6a99600640 1 -- 172.21.15.46:0/510908944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f6a900846c0 con 0x7f6a9c099a00 2024-04-25T08:52:13.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.350+0000 7f6aa0e00640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a7c066f60 0x7f6a7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.350+0000 7f6aa0e00640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a7c066f60 0x7f6a7c069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6a880ae000 tx=0x7f6a880a6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:13.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.354+0000 7f6a99600640 1 -- 172.21.15.46:0/510908944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a900553e0 con 0x7f6a9c099a00 2024-04-25T08:52:13.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.546+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a9c160fb0 con 0x7f6a9c099a00 2024-04-25T08:52:13.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.546+0000 7f6a99600640 1 -- 172.21.15.46:0/510908944 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f6a90054ce0 con 0x7f6a9c099a00 2024-04-25T08:52:13.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:13.538 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8817,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/893392755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":893392755},{"type":"v1","addr":"172.21.15.46:6835","nonce":893392755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:52:11.019700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:52:13.538 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 91 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.546+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a7c066f60 msgr2=0x7f6a7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.546+0000 7f6aa4200640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a7c066f60 0x7f6a7c069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6a880ae000 tx=0x7f6a880a6040 comp rx=0 tx=0).stop 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.546+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a9c099a00 msgr2=0x7f6a9c0b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.546+0000 7f6aa4200640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a9c099a00 0x7f6a9c0b6a00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f6a90001ec0 tx=0x7f6a9000c8e0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.550+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 shutdown_connections 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.550+0000 7f6aa4200640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6a7c066f60 0x7f6a7c069420 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.550+0000 7f6aa4200640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6a9c0ccde0 0x7f6a9c157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.550+0000 7f6aa4200640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a9c0b2240 0x7f6a9c157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.550+0000 7f6aa4200640 1 --2- 172.21.15.46:0/510908944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a9c099a00 0x7f6a9c0b6a00 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.550+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 >> 172.21.15.46:0/510908944 conn(0x7f6a9c0b3120 msgr2=0x7f6a9c070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.550+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 shutdown_connections 2024-04-25T08:52:13.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.550+0000 7f6aa4200640 1 -- 172.21.15.46:0/510908944 wait complete. 2024-04-25T08:52:13.559 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdef9400640 1 Processor -- start 2024-04-25T08:52:13.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdef9400640 1 -- start start 2024-04-25T08:52:13.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdef9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdef409a610 0x7fdef409aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdef9400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdef4099a00 0x7fdef4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdef9400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef415f300 con 0x7fdef415ef20 2024-04-25T08:52:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdef9400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef409a3b0 con 0x7fdef409a610 2024-04-25T08:52:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdef9400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef40cd8a0 con 0x7fdef4099a00 2024-04-25T08:52:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdee9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdef409a610 0x7fdef409aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdee9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdef409a610 0x7fdef409aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53502/0 (socket says 172.21.15.46:53502) 2024-04-25T08:52:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdee9e00640 1 -- 172.21.15.46:0/4273273263 learned_addr learned my addr 172.21.15.46:0/4273273263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdef3200640 1 --2- 172.21.15.46:0/4273273263 >> v2:172.21.15.82:3300/0 conn(0x7fdef4099a00 0x7fdef4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdee9e00640 1 -- 172.21.15.46:0/4273273263 >> v2:172.21.15.82:3300/0 conn(0x7fdef4099a00 msgr2=0x7fdef4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:13.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdee9e00640 1 --2- 172.21.15.46:0/4273273263 >> v2:172.21.15.82:3300/0 conn(0x7fdef4099a00 0x7fdef4099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdee9e00640 1 -- 172.21.15.46:0/4273273263 >> v1:172.21.15.82:6789/0 conn(0x7fdef415ef20 legacy=0x7fdef40cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.710+0000 7fdee9e00640 1 -- 172.21.15.46:0/4273273263 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef40cddb0 con 0x7fdef409a610 2024-04-25T08:52:13.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdee9e00640 1 --2- 172.21.15.46:0/4273273263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef409a610 0x7fdef409aa10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fdef4072be0 tx=0x7fdedc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c669ccad7e6eb1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:13.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef1e00640 1 -- 172.21.15.46:0/4273273263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc0ab610 con 0x7fdef409a610 2024-04-25T08:52:13.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef1e00640 1 -- 172.21.15.46:0/4273273263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc0b0070 con 0x7fdef409a610 2024-04-25T08:52:13.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef1e00640 1 -- 172.21.15.46:0/4273273263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc0ab9d0 con 0x7fdef409a610 2024-04-25T08:52:13.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef9400640 1 -- 172.21.15.46:0/4273273263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef409a610 msgr2=0x7fdef409aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef9400640 1 --2- 172.21.15.46:0/4273273263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef409a610 0x7fdef409aa10 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fdef4072be0 tx=0x7fdedc01c520 comp rx=0 tx=0).stop 2024-04-25T08:52:13.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef9400640 1 -- 172.21.15.46:0/4273273263 shutdown_connections 2024-04-25T08:52:13.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef9400640 1 --2- 172.21.15.46:0/4273273263 >> v2:172.21.15.82:3300/0 conn(0x7fdef4099a00 0x7fdef4099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef9400640 1 --2- 172.21.15.46:0/4273273263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef409a610 0x7fdef409aa10 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef9400640 1 -- 172.21.15.46:0/4273273263 >> 172.21.15.46:0/4273273263 conn(0x7fdef40b3120 msgr2=0x7fdef40b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:13.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef9400640 1 -- 172.21.15.46:0/4273273263 shutdown_connections 2024-04-25T08:52:13.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.714+0000 7fdef9400640 1 -- 172.21.15.46:0/4273273263 wait complete. 2024-04-25T08:52:13.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 Processor -- start 2024-04-25T08:52:13.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 -- start start 2024-04-25T08:52:13.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef4099a00 0x7fdef40b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdef409a610 0x7fdef4159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef415ef20 0x7fdef40cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40cc010 con 0x7fdef415ef20 2024-04-25T08:52:13.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40ccc70 con 0x7fdef4099a00 2024-04-25T08:52:13.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40cc190 con 0x7fdef409a610 2024-04-25T08:52:13.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef415ef20 0x7fdef40cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef415ef20 0x7fdef40cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53516/0 (socket says 172.21.15.46:53516) 2024-04-25T08:52:13.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef3200640 1 -- 172.21.15.46:0/3493635517 learned_addr learned my addr 172.21.15.46:0/3493635517 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:13.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdee9e00640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdef409a610 0x7fdef4159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef2800640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef4099a00 0x7fdef40b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef3200640 1 -- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdef409a610 msgr2=0x7fdef4159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef3200640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdef409a610 0x7fdef4159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef3200640 1 -- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef4099a00 msgr2=0x7fdef40b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef3200640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef4099a00 0x7fdef40b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef3200640 1 -- 172.21.15.46:0/3493635517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef40b5500 con 0x7fdef415ef20 2024-04-25T08:52:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef3200640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef415ef20 0x7fdef40cab40 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009d30 tx=0x7fdeec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdeeb400640 1 -- 172.21.15.46:0/3493635517 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec018020 con 0x7fdef415ef20 2024-04-25T08:52:13.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdeeb400640 1 -- 172.21.15.46:0/3493635517 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeec014480 con 0x7fdef415ef20 2024-04-25T08:52:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdeeb400640 1 -- 172.21.15.46:0/3493635517 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec014870 con 0x7fdef415ef20 2024-04-25T08:52:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef40b4980 con 0x7fdef415ef20 2024-04-25T08:52:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef40b5220 con 0x7fdef415ef20 2024-04-25T08:52:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.718+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef40b4d70 con 0x7fdef415ef20 2024-04-25T08:52:13.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.722+0000 7fdeeb400640 1 -- 172.21.15.46:0/3493635517 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdeec01b020 con 0x7fdef415ef20 2024-04-25T08:52:13.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.722+0000 7fdeeb400640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fded0066f60 0x7fded0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:13.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.722+0000 7fdeeb400640 1 -- 172.21.15.46:0/3493635517 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7fdeec085bf0 con 0x7fdef415ef20 2024-04-25T08:52:13.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.722+0000 7fdef2800640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fded0066f60 0x7fded0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:13.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.722+0000 7fdef2800640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fded0066f60 0x7fded0069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdef40c5090 tx=0x7fdee4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:13.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.726+0000 7fdeeb400640 1 -- 172.21.15.46:0/3493635517 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdeec022070 con 0x7fdef415ef20 2024-04-25T08:52:13.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.890+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdef4069080 con 0x7fdef415ef20 2024-04-25T08:52:13.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.890+0000 7fdeeb400640 1 -- 172.21.15.46:0/3493635517 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17143 (secure 0 0 0) 0x7fdeec010040 con 0x7fdef415ef20 2024-04-25T08:52:13.882 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:13.882 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:11.019712+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:50:45.647599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:50:46.857198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1174892156":"2024-04-26T08:52:11.019690+0000","172.21.15.46:6834/1174892156":"2024-04-26T08:52:11.019690+0000","172.21.15.46:6835/623970467":"2024-04-26T08:51:34.299560+0000","172.21.15.46:6834/623970467":"2024-04-26T08:51:34.299560+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fded0066f60 msgr2=0x7fded0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fded0066f60 0x7fded0069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdef40c5090 tx=0x7fdee4009040 comp rx=0 tx=0).stop 2024-04-25T08:52:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef415ef20 msgr2=0x7fdef40cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef415ef20 0x7fdef40cab40 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009d30 tx=0x7fdeec00c490 comp rx=0 tx=0).stop 2024-04-25T08:52:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 shutdown_connections 2024-04-25T08:52:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fded0066f60 0x7fded0069420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdef415ef20 0x7fdef40cab40 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fdef409a610 0x7fdef4159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 --2- 172.21.15.46:0/3493635517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdef4099a00 0x7fdef40b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 >> 172.21.15.46:0/3493635517 conn(0x7fdef40b3120 msgr2=0x7fdef4073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 shutdown_connections 2024-04-25T08:52:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:13.894+0000 7fdef9400640 1 -- 172.21.15.46:0/3493635517 wait complete. 2024-04-25T08:52:13.906 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:52:13.906 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.054+0000 7fe2e2a00640 1 Processor -- start 2024-04-25T08:52:14.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.054+0000 7fe2e2a00640 1 -- start start 2024-04-25T08:52:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.054+0000 7fe2e2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc159230 0x7fe2dc15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2e2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2dc0cbeb0 0x7fe2dc0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2e2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2dc0cda80 0x7fe2dc15f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2e2a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc0cc700 con 0x7fe2dc0cbeb0 2024-04-25T08:52:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2e2a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc0cc9e0 con 0x7fe2dc0cda80 2024-04-25T08:52:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2e2a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0cccc0 con 0x7fe2dc159230 2024-04-25T08:52:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2d3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2dc0cbeb0 0x7fe2dc0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2d3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2dc0cbeb0 0x7fe2dc0cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53522/0 (socket says 172.21.15.46:53522) 2024-04-25T08:52:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2d3e00640 1 -- 172.21.15.46:0/577245964 learned_addr learned my addr 172.21.15.46:0/577245964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:14.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2dbe00640 1 --2- 172.21.15.46:0/577245964 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc159230 0x7fe2dc15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2e0a00640 1 --2- 172.21.15.46:0/577245964 >> v2:172.21.15.82:3300/0 conn(0x7fe2dc0cda80 0x7fe2dc15f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2d3e00640 1 -- 172.21.15.46:0/577245964 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc159230 msgr2=0x7fe2dc15bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2d3e00640 1 --2- 172.21.15.46:0/577245964 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc159230 0x7fe2dc15bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2d3e00640 1 -- 172.21.15.46:0/577245964 >> v2:172.21.15.82:3300/0 conn(0x7fe2dc0cda80 msgr2=0x7fe2dc15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:14.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2d3e00640 1 --2- 172.21.15.46:0/577245964 >> v2:172.21.15.82:3300/0 conn(0x7fe2dc0cda80 0x7fe2dc15f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2d3e00640 1 -- 172.21.15.46:0/577245964 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0ccfa0 con 0x7fe2dc0cbeb0 2024-04-25T08:52:14.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2e0a00640 1 --2- 172.21.15.46:0/577245964 >> v2:172.21.15.82:3300/0 conn(0x7fe2dc0cda80 0x7fe2dc15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2d3e00640 1 --2- 172.21.15.46:0/577245964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cbeb0 0x7fe2dc0cd540 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc15c2f0 tx=0x7fe2c401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9697b079275320b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2db400640 1 -- 172.21.15.46:0/577245964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40b0070 con 0x7fe2dc0cbeb0 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2db400640 1 -- 172.21.15.46:0/577245964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c40ab600 con 0x7fe2dc0cbeb0 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.058+0000 7fe2db400640 1 -- 172.21.15.46:0/577245964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40ab9d0 con 0x7fe2dc0cbeb0 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 -- 172.21.15.46:0/577245964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cbeb0 msgr2=0x7fe2dc0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/577245964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cbeb0 0x7fe2dc0cd540 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc15c2f0 tx=0x7fe2c401c1d0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 -- 172.21.15.46:0/577245964 shutdown_connections 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/577245964 >> v2:172.21.15.82:3300/0 conn(0x7fe2dc0cda80 0x7fe2dc15f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/577245964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cbeb0 0x7fe2dc0cd540 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/577245964 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc159230 0x7fe2dc15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 -- 172.21.15.46:0/577245964 >> 172.21.15.46:0/577245964 conn(0x7fe2dc0b12a0 msgr2=0x7fe2dc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:14.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 -- 172.21.15.46:0/577245964 shutdown_connections 2024-04-25T08:52:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 -- 172.21.15.46:0/577245964 wait complete. 2024-04-25T08:52:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 Processor -- start 2024-04-25T08:52:14.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 -- start start 2024-04-25T08:52:14.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc0cbeb0 0x7fe2dc159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cda80 0x7fe2dc12eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2dc159230 0x7fe2dc141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc15d380 con 0x7fe2dc0cda80 2024-04-25T08:52:14.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc15d200 con 0x7fe2dc159230 2024-04-25T08:52:14.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e2a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc15d500 con 0x7fe2dc0cbeb0 2024-04-25T08:52:14.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cda80 0x7fe2dc12eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cda80 0x7fe2dc12eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53536/0 (socket says 172.21.15.46:53536) 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2d3e00640 1 -- 172.21.15.46:0/2338315851 learned_addr learned my addr 172.21.15.46:0/2338315851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2dbe00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc0cbeb0 0x7fe2dc159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e0a00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2dc159230 0x7fe2dc141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2d3e00640 1 -- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc0cbeb0 msgr2=0x7fe2dc159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2d3e00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc0cbeb0 0x7fe2dc159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2d3e00640 1 -- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2dc159230 msgr2=0x7fe2dc141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2d3e00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2dc159230 0x7fe2dc141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2d3e00640 1 -- 172.21.15.46:0/2338315851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc12f070 con 0x7fe2dc0cda80 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.062+0000 7fe2e0a00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2dc159230 0x7fe2dc141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2d3e00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cda80 0x7fe2dc12eb30 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40ae040 tx=0x7fe2c401aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2d8c00640 1 -- 172.21.15.46:0/2338315851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40ba020 con 0x7fe2dc0cda80 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2d8c00640 1 -- 172.21.15.46:0/2338315851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c40b6420 con 0x7fe2dc0cda80 2024-04-25T08:52:14.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2d8c00640 1 -- 172.21.15.46:0/2338315851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c40b0040 con 0x7fe2dc0cda80 2024-04-25T08:52:14.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc140760 con 0x7fe2dc0cda80 2024-04-25T08:52:14.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc140d20 con 0x7fe2dc0cda80 2024-04-25T08:52:14.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc0cd4c0 con 0x7fe2dc0cda80 2024-04-25T08:52:14.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2d8c00640 1 -- 172.21.15.46:0/2338315851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2c40bd020 con 0x7fe2dc0cda80 2024-04-25T08:52:14.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2d8c00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2b8066f60 0x7fe2b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2d8c00640 1 -- 172.21.15.46:0/2338315851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7fe2c411f470 con 0x7fe2dc0cda80 2024-04-25T08:52:14.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.066+0000 7fe2dbe00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2b8066f60 0x7fe2b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.070+0000 7fe2dbe00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2b8066f60 0x7fe2b8069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009ac0 tx=0x7fe2cc009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:14.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.070+0000 7fe2d8c00640 1 -- 172.21.15.46:0/2338315851 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2c40c4030 con 0x7fe2dc0cda80 2024-04-25T08:52:14.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2dc069080 con 0x7fe2dc0cda80 2024-04-25T08:52:14.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2d8c00640 1 -- 172.21.15.46:0/2338315851 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4928 (secure 0 0 0) 0x7fe2c4016110 con 0x7fe2dc0cda80 2024-04-25T08:52:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:14.257 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8817,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/893392755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":893392755},{"type":"v1","addr":"172.21.15.46:6835","nonce":893392755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:52:11.019700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:52:14.258 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 91 2024-04-25T08:52:14.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2b8066f60 msgr2=0x7fe2b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2b8066f60 0x7fe2b8069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc009ac0 tx=0x7fe2cc009190 comp rx=0 tx=0).stop 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cda80 msgr2=0x7fe2dc12eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cda80 0x7fe2dc12eb30 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe2c40ae040 tx=0x7fe2c401aa10 comp rx=0 tx=0).stop 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 shutdown_connections 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2b8066f60 0x7fe2b8069420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2dc159230 0x7fe2dc141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2dc0cda80 0x7fe2dc12eb30 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 --2- 172.21.15.46:0/2338315851 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe2dc0cbeb0 0x7fe2dc159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 >> 172.21.15.46:0/2338315851 conn(0x7fe2dc0b12a0 msgr2=0x7fe2dc15e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 shutdown_connections 2024-04-25T08:52:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.266+0000 7fe2e2a00640 1 -- 172.21.15.46:0/2338315851 wait complete. 2024-04-25T08:52:14.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.426+0000 7f1dcb200640 1 Processor -- start 2024-04-25T08:52:14.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.426+0000 7f1dcb200640 1 -- start start 2024-04-25T08:52:14.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dcb200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1dc415a750 0x7f1dc406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dcb200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc406d5e0 con 0x7f1dc40cc9e0 2024-04-25T08:52:14.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dcb200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc40ce720 con 0x7f1dc40aed10 2024-04-25T08:52:14.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dcb200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc415ab30 con 0x7f1dc415a750 2024-04-25T08:52:14.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc9200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1dc40cc9e0 0x7f1dc40b2cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58154/0 (socket says 172.21.15.46:58154) 2024-04-25T08:52:14.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc9200640 1 -- 172.21.15.46:0/1437208415 learned_addr learned my addr 172.21.15.46:0/1437208415 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:14.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc3e00640 1 --2- 172.21.15.46:0/1437208415 >> v2:172.21.15.82:3300/0 conn(0x7f1dc415a750 0x7f1dc406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc3400640 1 -- 172.21.15.46:0/1437208415 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952867574 0 0) 0x7f1dc406d5e0 con 0x7f1dc40cc9e0 2024-04-25T08:52:14.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc3400640 1 -- 172.21.15.46:0/1437208415 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1da8005740 con 0x7f1dc40cc9e0 2024-04-25T08:52:14.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc3400640 1 -- 172.21.15.46:0/1437208415 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332125937 0 0) 0x7f1dc40ce720 con 0x7f1dc40aed10 2024-04-25T08:52:14.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc3400640 1 -- 172.21.15.46:0/1437208415 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dc406d5e0 con 0x7f1dc40aed10 2024-04-25T08:52:14.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc3e00640 1 -- 172.21.15.46:0/1437208415 >> v1:172.21.15.82:6789/0 conn(0x7f1dc40aed10 legacy=0x7f1dc40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc3400640 1 -- 172.21.15.46:0/1437208415 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2520314209 0 0) 0x7f1da8005740 con 0x7f1dc40cc9e0 2024-04-25T08:52:14.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc3e00640 1 -- 172.21.15.46:0/1437208415 >> v1:172.21.15.46:6789/0 conn(0x7f1dc40cc9e0 legacy=0x7f1dc40b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.430+0000 7f1dc3e00640 1 -- 172.21.15.46:0/1437208415 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc40cea00 con 0x7f1dc415a750 2024-04-25T08:52:14.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dc3e00640 1 --2- 172.21.15.46:0/1437208415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc415a750 0x7f1dc406ec10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0097c0 tx=0x7f1dac01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b035c70475c111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:14.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dc3400640 1 -- 172.21.15.46:0/1437208415 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0b0070 con 0x7f1dc415a750 2024-04-25T08:52:14.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dc3400640 1 -- 172.21.15.46:0/1437208415 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac0ab400 con 0x7f1dc415a750 2024-04-25T08:52:14.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dc3400640 1 -- 172.21.15.46:0/1437208415 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc001a00 con 0x7f1dc415a750 2024-04-25T08:52:14.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dcb200640 1 -- 172.21.15.46:0/1437208415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc415a750 msgr2=0x7f1dc406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dcb200640 1 --2- 172.21.15.46:0/1437208415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc415a750 0x7f1dc406ec10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0097c0 tx=0x7f1dac01c3c0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dcb200640 1 -- 172.21.15.46:0/1437208415 shutdown_connections 2024-04-25T08:52:14.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dcb200640 1 --2- 172.21.15.46:0/1437208415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc415a750 0x7f1dc406ec10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dcb200640 1 -- 172.21.15.46:0/1437208415 >> 172.21.15.46:0/1437208415 conn(0x7f1dc40b12a0 msgr2=0x7f1dc40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:14.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dcb200640 1 -- 172.21.15.46:0/1437208415 shutdown_connections 2024-04-25T08:52:14.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dcb200640 1 -- 172.21.15.46:0/1437208415 wait complete. 2024-04-25T08:52:14.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dcb200640 1 Processor -- start 2024-04-25T08:52:14.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.434+0000 7f1dcb200640 1 -- start start 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dcb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc40aed10 0x7f1dc4145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dcb200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1dc40cc9e0 0x7f1dc414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dcb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1dc415a750 0x7f1dc413fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dcb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc406e310 con 0x7f1dc415a750 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dcb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc406f560 con 0x7f1dc40aed10 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dcb200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc406e490 con 0x7f1dc40cc9e0 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1dc415a750 0x7f1dc413fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1dc415a750 0x7f1dc413fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53552/0 (socket says 172.21.15.46:53552) 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc9200640 1 -- 172.21.15.46:0/4174399181 learned_addr learned my addr 172.21.15.46:0/4174399181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dbb400640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1dc40cc9e0 0x7f1dc414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc3e00640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc40aed10 0x7f1dc4145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc9200640 1 -- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1dc40cc9e0 msgr2=0x7f1dc414bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc9200640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1dc40cc9e0 0x7f1dc414bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc9200640 1 -- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc40aed10 msgr2=0x7f1dc4145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc9200640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc40aed10 0x7f1dc4145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc9200640 1 -- 172.21.15.46:0/4174399181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc40ad490 con 0x7f1dc415a750 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc9200640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1dc415a750 0x7f1dc413fd00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc001570 tx=0x7f1dbc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc0c00640 1 -- 172.21.15.46:0/4174399181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc012400 con 0x7f1dc415a750 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc0c00640 1 -- 172.21.15.46:0/4174399181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc0125a0 con 0x7f1dc415a750 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dc0c00640 1 -- 172.21.15.46:0/4174399181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0129d0 con 0x7f1dc415a750 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc40ad770 con 0x7f1dc415a750 2024-04-25T08:52:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc4143240 con 0x7f1dc415a750 2024-04-25T08:52:14.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.438+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc4069080 con 0x7f1dc415a750 2024-04-25T08:52:14.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.442+0000 7f1dc0c00640 1 -- 172.21.15.46:0/4174399181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1dbc012b70 con 0x7f1dc415a750 2024-04-25T08:52:14.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.442+0000 7f1dc0c00640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1da0066f60 0x7f1da0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.442+0000 7f1dc0c00640 1 -- 172.21.15.46:0/4174399181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f1dbc085aa0 con 0x7f1dc415a750 2024-04-25T08:52:14.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.442+0000 7f1dc3e00640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1da0066f60 0x7f1da0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.442+0000 7f1dc3e00640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1da0066f60 0x7f1da0069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0ad000 tx=0x7f1dac0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:14.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.446+0000 7f1dc0c00640 1 -- 172.21.15.46:0/4174399181 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1dbc056170 con 0x7f1dc415a750 2024-04-25T08:52:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.642+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dc4151f30 con 0x7f1dc415a750 2024-04-25T08:52:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.642+0000 7f1dc0c00640 1 -- 172.21.15.46:0/4174399181 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f1dbc05a210 con 0x7f1dc415a750 2024-04-25T08:52:14.634 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:14.634 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8817,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/893392755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":893392755},{"type":"v1","addr":"172.21.15.46:6835","nonce":893392755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:50:47.860021+0000","modified":"2024-04-25T08:52:11.019700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:52:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 91 2024-04-25T08:52:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.642+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1da0066f60 msgr2=0x7f1da0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.642+0000 7f1dcb200640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1da0066f60 0x7f1da0069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0ad000 tx=0x7f1dac0a6000 comp rx=0 tx=0).stop 2024-04-25T08:52:14.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.642+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1dc415a750 msgr2=0x7f1dc413fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.642+0000 7f1dcb200640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1dc415a750 0x7f1dc413fd00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc001570 tx=0x7f1dbc00a430 comp rx=0 tx=0).stop 2024-04-25T08:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.646+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 shutdown_connections 2024-04-25T08:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.646+0000 7f1dcb200640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1da0066f60 0x7f1da0069420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.646+0000 7f1dcb200640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1dc415a750 0x7f1dc413fd00 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.646+0000 7f1dcb200640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1dc40cc9e0 0x7f1dc414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.646+0000 7f1dcb200640 1 --2- 172.21.15.46:0/4174399181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1dc40aed10 0x7f1dc4145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.646+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 >> 172.21.15.46:0/4174399181 conn(0x7f1dc40b12a0 msgr2=0x7f1dc40b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.646+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 shutdown_connections 2024-04-25T08:52:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.646+0000 7f1dcb200640 1 -- 172.21.15.46:0/4174399181 wait complete. 2024-04-25T08:52:14.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.806+0000 7f65b2a00640 1 Processor -- start 2024-04-25T08:52:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.806+0000 7f65b2a00640 1 -- start start 2024-04-25T08:52:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.806+0000 7f65b2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac159230 0x7f65ac15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.806+0000 7f65b2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65ac0cbeb0 0x7f65ac0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.806+0000 7f65b2a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ac0cc700 con 0x7f65ac0cda80 2024-04-25T08:52:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.806+0000 7f65b2a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac0cc9e0 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.806+0000 7f65b2a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0cccc0 con 0x7f65ac159230 2024-04-25T08:52:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65abe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac159230 0x7f65ac15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65abe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac159230 0x7f65ac15bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:57854/0 (socket says 172.21.15.46:57854) 2024-04-25T08:52:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65abe00640 1 -- 172.21.15.46:0/3157587669 learned_addr learned my addr 172.21.15.46:0/3157587669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65b0a00640 1 --1- 172.21.15.46:0/3157587669 >> v1:172.21.15.82:6789/0 conn(0x7f65ac0cda80 0x7f65ac15f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:53040/0 (socket says 172.21.15.46:53040) 2024-04-25T08:52:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65a3e00640 1 --2- 172.21.15.46:0/3157587669 >> v2:172.21.15.82:3300/0 conn(0x7f65ac0cbeb0 0x7f65ac0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65abe00640 1 -- 172.21.15.46:0/3157587669 >> v2:172.21.15.82:3300/0 conn(0x7f65ac0cbeb0 msgr2=0x7f65ac0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65abe00640 1 --2- 172.21.15.46:0/3157587669 >> v2:172.21.15.82:3300/0 conn(0x7f65ac0cbeb0 0x7f65ac0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65abe00640 1 -- 172.21.15.46:0/3157587669 >> v1:172.21.15.82:6789/0 conn(0x7f65ac0cda80 legacy=0x7f65ac15f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65abe00640 1 -- 172.21.15.46:0/3157587669 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac0ccfa0 con 0x7f65ac159230 2024-04-25T08:52:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65ab400640 1 -- 172.21.15.46:0/3157587669 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84281655 0 0) 0x7f65ac0cc700 con 0x7f65ac0cda80 2024-04-25T08:52:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65abe00640 1 --2- 172.21.15.46:0/3157587669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac159230 0x7f65ac15bdb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f659c009870 tx=0x7f659c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4084538fdd03dec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65ab400640 1 -- 172.21.15.46:0/3157587669 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0b1070 con 0x7f65ac159230 2024-04-25T08:52:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65ab400640 1 -- 172.21.15.46:0/3157587669 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c0a8050 con 0x7f65ac159230 2024-04-25T08:52:14.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.810+0000 7f65ab400640 1 -- 172.21.15.46:0/3157587669 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0ac860 con 0x7f65ac159230 2024-04-25T08:52:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 -- 172.21.15.46:0/3157587669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac159230 msgr2=0x7f65ac15bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 --2- 172.21.15.46:0/3157587669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac159230 0x7f65ac15bdb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f659c009870 tx=0x7f659c01c4d0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 -- 172.21.15.46:0/3157587669 shutdown_connections 2024-04-25T08:52:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 --2- 172.21.15.46:0/3157587669 >> v2:172.21.15.82:3300/0 conn(0x7f65ac0cbeb0 0x7f65ac0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 --2- 172.21.15.46:0/3157587669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac159230 0x7f65ac15bdb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 -- 172.21.15.46:0/3157587669 >> 172.21.15.46:0/3157587669 conn(0x7f65ac0b12a0 msgr2=0x7f65ac0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 -- 172.21.15.46:0/3157587669 shutdown_connections 2024-04-25T08:52:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 -- 172.21.15.46:0/3157587669 wait complete. 2024-04-25T08:52:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 Processor -- start 2024-04-25T08:52:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 -- start start 2024-04-25T08:52:14.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac0cbeb0 0x7f65ac13f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac0cda80 0x7f65ac159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65ac159230 0x7f65ac1f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac168780 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0cc4a0 con 0x7f65ac159230 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b2a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac168900 con 0x7f65ac0cda80 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac0cbeb0 0x7f65ac13f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac0cbeb0 0x7f65ac13f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53568/0 (socket says 172.21.15.46:53568) 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65abe00640 1 -- 172.21.15.46:0/1561205510 learned_addr learned my addr 172.21.15.46:0/1561205510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65a3e00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac0cda80 0x7f65ac159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b0a00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65ac159230 0x7f65ac1f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65abe00640 1 -- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac0cda80 msgr2=0x7f65ac159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65abe00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac0cda80 0x7f65ac159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65abe00640 1 -- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65ac159230 msgr2=0x7f65ac1f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65abe00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65ac159230 0x7f65ac1f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65abe00640 1 -- 172.21.15.46:0/1561205510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac165ba0 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.814+0000 7f65b0a00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65ac159230 0x7f65ac1f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65abe00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac0cbeb0 0x7f65ac13f6c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f659c0af040 tx=0x7f659c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65a8c00640 1 -- 172.21.15.46:0/1561205510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0b1040 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65a8c00640 1 -- 172.21.15.46:0/1561205510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c0bd070 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65a8c00640 1 -- 172.21.15.46:0/1561205510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0b7660 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac165e80 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ac166440 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ac069080 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65a8c00640 1 -- 172.21.15.46:0/1561205510 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f659c0b7800 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65a8c00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6588066f60 0x7f6588069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65a8c00640 1 -- 172.21.15.46:0/1561205510 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f659c11f6a0 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.818+0000 7f65a3e00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6588066f60 0x7f6588069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:14.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.822+0000 7f65a3e00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6588066f60 0x7f6588069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6594009a50 tx=0x7f6594008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:14.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.822+0000 7f65a8c00640 1 -- 172.21.15.46:0/1561205510 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f659c016150 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.986+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65ac0723d0 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.986+0000 7f65a8c00640 1 -- 172.21.15.46:0/1561205510 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17143 (secure 0 0 0) 0x7f659c0f03c0 con 0x7f65ac0cbeb0 2024-04-25T08:52:14.978 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:14.979 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:11.019712+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:50:45.647599+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:50:46.857198+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1174892156":"2024-04-26T08:52:11.019690+0000","172.21.15.46:6834/1174892156":"2024-04-26T08:52:11.019690+0000","172.21.15.46:6835/623970467":"2024-04-26T08:51:34.299560+0000","172.21.15.46:6834/623970467":"2024-04-26T08:51:34.299560+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:14.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6588066f60 msgr2=0x7f6588069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6588066f60 0x7f6588069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6594009a50 tx=0x7f6594008040 comp rx=0 tx=0).stop 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac0cbeb0 msgr2=0x7f65ac13f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac0cbeb0 0x7f65ac13f6c0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f659c0af040 tx=0x7f659c01ad10 comp rx=0 tx=0).stop 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 shutdown_connections 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6588066f60 0x7f6588069420 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65ac159230 0x7f65ac1f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65ac0cda80 0x7f65ac159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 --2- 172.21.15.46:0/1561205510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65ac0cbeb0 0x7f65ac13f6c0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 >> 172.21.15.46:0/1561205510 conn(0x7f65ac0b12a0 msgr2=0x7f65ac0af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 shutdown_connections 2024-04-25T08:52:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:14.990+0000 7f65b2a00640 1 -- 172.21.15.46:0/1561205510 wait complete. 2024-04-25T08:52:15.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:52:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.150+0000 7ff1bea00640 1 Processor -- start 2024-04-25T08:52:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.150+0000 7ff1bea00640 1 -- start start 2024-04-25T08:52:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b815dbd0 0x7ff1b80b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1bea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff1b80713a0 0x7ff1b806c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1bea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b80b5250 con 0x7ff1b80b0c30 2024-04-25T08:52:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1bea00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b80b5af0 con 0x7ff1b80713a0 2024-04-25T08:52:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1bea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b80b5530 con 0x7ff1b815dbd0 2024-04-25T08:52:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1af400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff1b80b0c30 0x7ff1b8070c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58156/0 (socket says 172.21.15.46:58156) 2024-04-25T08:52:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1af400640 1 -- 172.21.15.46:0/1652909948 learned_addr learned my addr 172.21.15.46:0/1652909948 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/1652909948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b815dbd0 0x7ff1b80b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1bca00640 1 --2- 172.21.15.46:0/1652909948 >> v2:172.21.15.82:3300/0 conn(0x7ff1b80713a0 0x7ff1b806c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1b7400640 1 -- 172.21.15.46:0/1652909948 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002720116 0 0) 0x7ff1b80b5250 con 0x7ff1b80b0c30 2024-04-25T08:52:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1b7400640 1 -- 172.21.15.46:0/1652909948 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff19c003530 con 0x7ff1b80b0c30 2024-04-25T08:52:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1b7e00640 1 -- 172.21.15.46:0/1652909948 >> v2:172.21.15.82:3300/0 conn(0x7ff1b80713a0 msgr2=0x7ff1b806c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/1652909948 >> v2:172.21.15.82:3300/0 conn(0x7ff1b80713a0 0x7ff1b806c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1b7e00640 1 -- 172.21.15.46:0/1652909948 >> v1:172.21.15.46:6789/0 conn(0x7ff1b80b0c30 legacy=0x7ff1b8070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.154+0000 7ff1b7e00640 1 -- 172.21.15.46:0/1652909948 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b80b5810 con 0x7ff1b815dbd0 2024-04-25T08:52:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/1652909948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b815dbd0 0x7ff1b80b06f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009d90 tx=0x7ff1a801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7adb203a24d706ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1b7400640 1 -- 172.21.15.46:0/1652909948 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80b2070 con 0x7ff1b815dbd0 2024-04-25T08:52:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1b7400640 1 -- 172.21.15.46:0/1652909948 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a80a5830 con 0x7ff1b815dbd0 2024-04-25T08:52:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1b7400640 1 -- 172.21.15.46:0/1652909948 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80a5c00 con 0x7ff1b815dbd0 2024-04-25T08:52:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 -- 172.21.15.46:0/1652909948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b815dbd0 msgr2=0x7ff1b80b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 --2- 172.21.15.46:0/1652909948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b815dbd0 0x7ff1b80b06f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009d90 tx=0x7ff1a801c970 comp rx=0 tx=0).stop 2024-04-25T08:52:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 -- 172.21.15.46:0/1652909948 shutdown_connections 2024-04-25T08:52:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 --2- 172.21.15.46:0/1652909948 >> v2:172.21.15.82:3300/0 conn(0x7ff1b80713a0 0x7ff1b806c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 --2- 172.21.15.46:0/1652909948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b815dbd0 0x7ff1b80b06f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 -- 172.21.15.46:0/1652909948 >> 172.21.15.46:0/1652909948 conn(0x7ff1b806f4f0 msgr2=0x7ff1b806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 -- 172.21.15.46:0/1652909948 shutdown_connections 2024-04-25T08:52:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 -- 172.21.15.46:0/1652909948 wait complete. 2024-04-25T08:52:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 Processor -- start 2024-04-25T08:52:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 -- start start 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b80713a0 0x7ff1b8150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b80b0c30 0x7ff1b8152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b815dbd0 0x7ff1b8152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b806d460 con 0x7ff1b815dbd0 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b806d2e0 con 0x7ff1b80b0c30 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.158+0000 7ff1bea00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b806d5e0 con 0x7ff1b80713a0 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b815dbd0 0x7ff1b8152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b815dbd0 0x7ff1b8152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53576/0 (socket says 172.21.15.46:53576) 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bca00640 1 -- 172.21.15.46:0/562345226 learned_addr learned my addr 172.21.15.46:0/562345226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b80713a0 0x7ff1b8150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1af400640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b80b0c30 0x7ff1b8152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bca00640 1 -- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b80713a0 msgr2=0x7ff1b8150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bca00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b80713a0 0x7ff1b8150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bca00640 1 -- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b80b0c30 msgr2=0x7ff1b8152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bca00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b80b0c30 0x7ff1b8152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bca00640 1 -- 172.21.15.46:0/562345226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b80ce8b0 con 0x7ff1b815dbd0 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b80713a0 0x7ff1b8150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1af400640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b80b0c30 0x7ff1b8152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bca00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b815dbd0 0x7ff1b8152f80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e850 tx=0x7ff1b000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1b4c00640 1 -- 172.21.15.46:0/562345226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0017070 con 0x7ff1b815dbd0 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1b4c00640 1 -- 172.21.15.46:0/562345226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b000c940 con 0x7ff1b815dbd0 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b80ccbd0 con 0x7ff1b815dbd0 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1b4c00640 1 -- 172.21.15.46:0/562345226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000cd10 con 0x7ff1b815dbd0 2024-04-25T08:52:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b80ceb50 con 0x7ff1b815dbd0 2024-04-25T08:52:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b80ced50 con 0x7ff1b815dbd0 2024-04-25T08:52:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1b4c00640 1 -- 172.21.15.46:0/562345226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1b001a070 con 0x7ff1b815dbd0 2024-04-25T08:52:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.162+0000 7ff1b4c00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff194066f60 0x7ff194069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.166+0000 7ff1b4c00640 1 -- 172.21.15.46:0/562345226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7ff1b0085640 con 0x7ff1b815dbd0 2024-04-25T08:52:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.166+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff194066f60 0x7ff194069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.166+0000 7ff1b7e00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff194066f60 0x7ff194069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80af000 tx=0x7ff1a80a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.170+0000 7ff1b4c00640 1 -- 172.21.15.46:0/562345226 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1b0056400 con 0x7ff1b815dbd0 2024-04-25T08:52:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.366+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff1b80cd1f0 con 0x7ff1b815dbd0 2024-04-25T08:52:15.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.686+0000 7ff1b4c00640 1 -- 172.21.15.46:0/562345226 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff1b0055d00 con 0x7ff1b815dbd0 2024-04-25T08:52:15.678 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:52:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff194066f60 msgr2=0x7ff194069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff194066f60 0x7ff194069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80af000 tx=0x7ff1a80a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:15.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b815dbd0 msgr2=0x7ff1b8152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b815dbd0 0x7ff1b8152f80 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e850 tx=0x7ff1b000ed10 comp rx=0 tx=0).stop 2024-04-25T08:52:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 shutdown_connections 2024-04-25T08:52:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff194066f60 0x7ff194069420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1b815dbd0 0x7ff1b8152f80 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1b80b0c30 0x7ff1b8152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 --2- 172.21.15.46:0/562345226 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff1b80713a0 0x7ff1b8150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 >> 172.21.15.46:0/562345226 conn(0x7ff1b806f4f0 msgr2=0x7ff1b815b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 shutdown_connections 2024-04-25T08:52:15.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.690+0000 7ff1bea00640 1 -- 172.21.15.46:0/562345226 wait complete. 2024-04-25T08:52:15.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:52:15.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.850+0000 7fe296e00640 1 Processor -- start 2024-04-25T08:52:15.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.850+0000 7fe296e00640 1 -- start start 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe296e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29006dc70 0x7fe2900cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe296e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2900cd930 con 0x7fe2900cf1b0 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe296e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2900cdbd0 con 0x7fe290145310 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe296e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900cdeb0 con 0x7fe29006dc70 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe294e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe2900cf1b0 0x7fe29015d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58164/0 (socket says 172.21.15.46:58164) 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe294e00640 1 -- 172.21.15.46:0/786900363 learned_addr learned my addr 172.21.15.46:0/786900363 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe287e00640 1 --2- 172.21.15.46:0/786900363 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29006dc70 0x7fe2900cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe28f400640 1 -- 172.21.15.46:0/786900363 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625277372 0 0) 0x7fe2900cd930 con 0x7fe2900cf1b0 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe28f400640 1 -- 172.21.15.46:0/786900363 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe274003530 con 0x7fe2900cf1b0 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe28f400640 1 -- 172.21.15.46:0/786900363 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346550082 0 0) 0x7fe2900cdbd0 con 0x7fe290145310 2024-04-25T08:52:15.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe28f400640 1 -- 172.21.15.46:0/786900363 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2900cd930 con 0x7fe290145310 2024-04-25T08:52:15.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe287e00640 1 -- 172.21.15.46:0/786900363 >> v1:172.21.15.82:6789/0 conn(0x7fe290145310 legacy=0x7fe29006e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe287e00640 1 -- 172.21.15.46:0/786900363 >> v1:172.21.15.46:6789/0 conn(0x7fe2900cf1b0 legacy=0x7fe29015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.854+0000 7fe287e00640 1 -- 172.21.15.46:0/786900363 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900ce190 con 0x7fe29006dc70 2024-04-25T08:52:15.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe287e00640 1 --2- 172.21.15.46:0/786900363 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29006dc70 0x7fe2900cec70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2780097f0 tx=0x7fe27801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ab1e2233e57713 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:15.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe28f400640 1 -- 172.21.15.46:0/786900363 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2780aa610 con 0x7fe29006dc70 2024-04-25T08:52:15.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe28f400640 1 -- 172.21.15.46:0/786900363 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2780af070 con 0x7fe29006dc70 2024-04-25T08:52:15.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe28f400640 1 -- 172.21.15.46:0/786900363 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2780aa9d0 con 0x7fe29006dc70 2024-04-25T08:52:15.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 -- 172.21.15.46:0/786900363 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29006dc70 msgr2=0x7fe2900cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 --2- 172.21.15.46:0/786900363 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29006dc70 0x7fe2900cec70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe2780097f0 tx=0x7fe27801c4b0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 -- 172.21.15.46:0/786900363 shutdown_connections 2024-04-25T08:52:15.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 --2- 172.21.15.46:0/786900363 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29006dc70 0x7fe2900cec70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 -- 172.21.15.46:0/786900363 >> 172.21.15.46:0/786900363 conn(0x7fe29006ac10 msgr2=0x7fe29006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:15.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 -- 172.21.15.46:0/786900363 shutdown_connections 2024-04-25T08:52:15.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 -- 172.21.15.46:0/786900363 wait complete. 2024-04-25T08:52:15.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 Processor -- start 2024-04-25T08:52:15.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 -- start start 2024-04-25T08:52:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2900cf1b0 0x7fe290158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe290145310 0x7fe29015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29013f0c0 0x7fe29015b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe290169520 con 0x7fe2900cf1b0 2024-04-25T08:52:15.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29015d9f0 con 0x7fe290145310 2024-04-25T08:52:15.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe296e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2901696a0 con 0x7fe29013f0c0 2024-04-25T08:52:15.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe28fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2900cf1b0 0x7fe290158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe28fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2900cf1b0 0x7fe290158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53588/0 (socket says 172.21.15.46:53588) 2024-04-25T08:52:15.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.858+0000 7fe28fe00640 1 -- 172.21.15.46:0/1592421602 learned_addr learned my addr 172.21.15.46:0/1592421602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:15.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe294e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29013f0c0 0x7fe29015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe287e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe290145310 0x7fe29015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28fe00640 1 -- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29013f0c0 msgr2=0x7fe29015b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28fe00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29013f0c0 0x7fe29015b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28fe00640 1 -- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe290145310 msgr2=0x7fe29015add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:15.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28fe00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe290145310 0x7fe29015add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:15.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28fe00640 1 -- 172.21.15.46:0/1592421602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29013e300 con 0x7fe2900cf1b0 2024-04-25T08:52:15.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28fe00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2900cf1b0 0x7fe290158e20 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fe28000c460 tx=0x7fe28000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:15.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28cc00640 1 -- 172.21.15.46:0/1592421602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280017070 con 0x7fe2900cf1b0 2024-04-25T08:52:15.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28cc00640 1 -- 172.21.15.46:0/1592421602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2800052f0 con 0x7fe2900cf1b0 2024-04-25T08:52:15.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28cc00640 1 -- 172.21.15.46:0/1592421602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe280003840 con 0x7fe2900cf1b0 2024-04-25T08:52:15.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29013e5e0 con 0x7fe2900cf1b0 2024-04-25T08:52:15.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29013eba0 con 0x7fe2900cf1b0 2024-04-25T08:52:15.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe290069010 con 0x7fe2900cf1b0 2024-04-25T08:52:15.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28cc00640 1 -- 172.21.15.46:0/1592421602 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe28001a070 con 0x7fe2900cf1b0 2024-04-25T08:52:15.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28cc00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe26c066f60 0x7fe26c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:15.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.862+0000 7fe28cc00640 1 -- 172.21.15.46:0/1592421602 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7fe280084650 con 0x7fe2900cf1b0 2024-04-25T08:52:15.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.866+0000 7fe287e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe26c066f60 0x7fe26c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:15.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.866+0000 7fe287e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe26c066f60 0x7fe26c069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe2780ae000 tx=0x7fe2780a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:15.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:15.870+0000 7fe28cc00640 1 -- 172.21.15.46:0/1592421602 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe280055400 con 0x7fe2900cf1b0 2024-04-25T08:52:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.062+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe29013ede0 con 0x7fe2900cf1b0 2024-04-25T08:52:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe28cc00640 1 -- 172.21.15.46:0/1592421602 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe280054d00 con 0x7fe2900cf1b0 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe26c066f60 msgr2=0x7fe26c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe26c066f60 0x7fe26c069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe2780ae000 tx=0x7fe2780a7000 comp rx=0 tx=0).stop 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2900cf1b0 msgr2=0x7fe290158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2900cf1b0 0x7fe290158e20 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fe28000c460 tx=0x7fe28000c920 comp rx=0 tx=0).stop 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 shutdown_connections 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe26c066f60 0x7fe26c069420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe29013f0c0 0x7fe29015b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe290145310 0x7fe29015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 --2- 172.21.15.46:0/1592421602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2900cf1b0 0x7fe290158e20 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 >> 172.21.15.46:0/1592421602 conn(0x7fe29006ac10 msgr2=0x7fe290143e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:16.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 shutdown_connections 2024-04-25T08:52:16.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.694+0000 7fe296e00640 1 -- 172.21.15.46:0/1592421602 wait complete. 2024-04-25T08:52:16.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:52:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa8400640 1 Processor -- start 2024-04-25T08:52:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa8400640 1 -- start start 2024-04-25T08:52:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa8400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fa013e6b0 0x7f9fa00711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa8400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9fa0071720 0x7f9fa015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa8400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fa015c4b0 0x7f9fa00cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa8400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa0071b80 con 0x7f9fa015c4b0 2024-04-25T08:52:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa8400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa015cce0 con 0x7f9fa0071720 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa8400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa006d330 con 0x7f9fa013e6b0 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa6400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fa015c4b0 0x7f9fa00cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa6400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fa015c4b0 0x7f9fa00cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53592/0 (socket says 172.21.15.46:53592) 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa6400640 1 -- 172.21.15.46:0/2112841052 learned_addr learned my addr 172.21.15.46:0/2112841052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa5a00640 1 --2- 172.21.15.46:0/2112841052 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fa013e6b0 0x7f9fa00711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa5000640 1 --2- 172.21.15.46:0/2112841052 >> v2:172.21.15.82:3300/0 conn(0x7f9fa0071720 0x7f9fa015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa6400640 1 -- 172.21.15.46:0/2112841052 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fa013e6b0 msgr2=0x7f9fa00711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa6400640 1 --2- 172.21.15.46:0/2112841052 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fa013e6b0 0x7f9fa00711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa6400640 1 -- 172.21.15.46:0/2112841052 >> v2:172.21.15.82:3300/0 conn(0x7f9fa0071720 msgr2=0x7f9fa015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa6400640 1 --2- 172.21.15.46:0/2112841052 >> v2:172.21.15.82:3300/0 conn(0x7f9fa0071720 0x7f9fa015bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.870+0000 7f9fa6400640 1 -- 172.21.15.46:0/2112841052 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa006d610 con 0x7f9fa015c4b0 2024-04-25T08:52:16.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa6400640 1 --2- 172.21.15.46:0/2112841052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa015c4b0 0x7f9fa00cd180 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00b49b0 tx=0x7f9f9801c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53c357612c69362 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:16.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9f9fe00640 1 -- 172.21.15.46:0/2112841052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980a6040 con 0x7f9fa015c4b0 2024-04-25T08:52:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9f9fe00640 1 -- 172.21.15.46:0/2112841052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f980af070 con 0x7f9fa015c4b0 2024-04-25T08:52:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9f9fe00640 1 -- 172.21.15.46:0/2112841052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980aa660 con 0x7f9fa015c4b0 2024-04-25T08:52:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa8400640 1 -- 172.21.15.46:0/2112841052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa015c4b0 msgr2=0x7f9fa00cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2112841052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa015c4b0 0x7f9fa00cd180 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00b49b0 tx=0x7f9f9801c610 comp rx=0 tx=0).stop 2024-04-25T08:52:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa8400640 1 -- 172.21.15.46:0/2112841052 shutdown_connections 2024-04-25T08:52:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2112841052 >> v2:172.21.15.82:3300/0 conn(0x7f9fa0071720 0x7f9fa015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2112841052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa015c4b0 0x7f9fa00cd180 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2112841052 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fa013e6b0 0x7f9fa00711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa8400640 1 -- 172.21.15.46:0/2112841052 >> 172.21.15.46:0/2112841052 conn(0x7f9fa0067050 msgr2=0x7f9fa006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa8400640 1 -- 172.21.15.46:0/2112841052 shutdown_connections 2024-04-25T08:52:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.874+0000 7f9fa8400640 1 -- 172.21.15.46:0/2112841052 wait complete. 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 Processor -- start 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 -- start start 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa0071720 0x7f9fa01363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fa013e6b0 0x7f9fa015dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fa015c4b0 0x7f9fa015e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa013ef70 con 0x7f9fa0071720 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa013edf0 con 0x7f9fa015c4b0 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa006e0d0 con 0x7f9fa013e6b0 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa0071720 0x7f9fa01363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa0071720 0x7f9fa01363d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53594/0 (socket says 172.21.15.46:53594) 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa5a00640 1 -- 172.21.15.46:0/2509244353 learned_addr learned my addr 172.21.15.46:0/2509244353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa5a00640 1 -- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fa013e6b0 msgr2=0x7f9fa015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa5a00640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fa013e6b0 0x7f9fa015dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa5a00640 1 -- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fa015c4b0 msgr2=0x7f9fa015e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa5a00640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fa015c4b0 0x7f9fa015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa5a00640 1 -- 172.21.15.46:0/2509244353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa0137b50 con 0x7f9fa0071720 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa5a00640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa0071720 0x7f9fa01363d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400d800 tx=0x7f9f9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9f9d600640 1 -- 172.21.15.46:0/2509244353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400b6c0 con 0x7f9fa0071720 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9f9d600640 1 -- 172.21.15.46:0/2509244353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f94019070 con 0x7f9fa0071720 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa0137e30 con 0x7f9fa0071720 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9f9d600640 1 -- 172.21.15.46:0/2509244353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400ba60 con 0x7f9fa0071720 2024-04-25T08:52:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.878+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa01383f0 con 0x7f9fa0071720 2024-04-25T08:52:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.882+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa0069080 con 0x7f9fa0071720 2024-04-25T08:52:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.882+0000 7f9f9d600640 1 -- 172.21.15.46:0/2509244353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f9400bc00 con 0x7f9fa0071720 2024-04-25T08:52:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.882+0000 7f9f9d600640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f80066f60 0x7f9f80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.882+0000 7f9f9d600640 1 -- 172.21.15.46:0/2509244353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f9f94085920 con 0x7f9fa0071720 2024-04-25T08:52:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.882+0000 7f9fa5000640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f80066f60 0x7f9f80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.882+0000 7f9fa5000640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f80066f60 0x7f9f80069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00ceae0 tx=0x7f9f8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:16.886+0000 7f9f9d600640 1 -- 172.21.15.46:0/2509244353 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f940565b0 con 0x7f9fa0071720 2024-04-25T08:52:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.050+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9fa00cf240 con 0x7f9fa0071720 2024-04-25T08:52:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.694+0000 7f9f9d600640 1 -- 172.21.15.46:0/2509244353 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9f94055eb0 con 0x7f9fa0071720 2024-04-25T08:52:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:52:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f80066f60 msgr2=0x7f9f80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f80066f60 0x7f9f80069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9fa00ceae0 tx=0x7f9f8c008040 comp rx=0 tx=0).stop 2024-04-25T08:52:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa0071720 msgr2=0x7f9fa01363d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa0071720 0x7f9fa01363d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f9f9400d800 tx=0x7f9f9400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:52:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 shutdown_connections 2024-04-25T08:52:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f80066f60 0x7f9f80069420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fa015c4b0 0x7f9fa015e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fa013e6b0 0x7f9fa015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 --2- 172.21.15.46:0/2509244353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fa0071720 0x7f9fa01363d0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 >> 172.21.15.46:0/2509244353 conn(0x7f9fa0067050 msgr2=0x7f9fa013fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 shutdown_connections 2024-04-25T08:52:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.698+0000 7f9fa8400640 1 -- 172.21.15.46:0/2509244353 wait complete. 2024-04-25T08:52:17.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:52:17.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.874+0000 7f11f1600640 1 Processor -- start 2024-04-25T08:52:17.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.874+0000 7f11f1600640 1 -- start start 2024-04-25T08:52:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11f1600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11ec15c680 0x7f11ec166370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11f1600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ec06d330 con 0x7f11ec0718f0 2024-04-25T08:52:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11f1600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ec06d610 con 0x7f11ec13e6b0 2024-04-25T08:52:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11f1600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ec15cce0 con 0x7f11ec15c680 2024-04-25T08:52:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11eb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11ec15c680 0x7f11ec166370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11e2000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f11ec0718f0 0x7f11ec15bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58176/0 (socket says 172.21.15.46:58176) 2024-04-25T08:52:17.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11eb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11ec15c680 0x7f11ec166370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53598/0 (socket says 172.21.15.46:53598) 2024-04-25T08:52:17.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11e2000640 1 -- 172.21.15.46:0/231709753 learned_addr learned my addr 172.21.15.46:0/231709753 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:17.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11eb400640 1 -- 172.21.15.46:0/231709753 >> v1:172.21.15.82:6789/0 conn(0x7f11ec13e6b0 legacy=0x7f11ec0711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:17.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11eb400640 1 -- 172.21.15.46:0/231709753 >> v1:172.21.15.46:6789/0 conn(0x7f11ec0718f0 legacy=0x7f11ec15bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:17.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.878+0000 7f11eb400640 1 -- 172.21.15.46:0/231709753 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec06d8f0 con 0x7f11ec15c680 2024-04-25T08:52:17.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11eb400640 1 --2- 172.21.15.46:0/231709753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec15c680 0x7f11ec166370 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009dc0 tx=0x7f11e401c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337672957ce057cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11ea000640 1 -- 172.21.15.46:0/231709753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40a8610 con 0x7f11ec15c680 2024-04-25T08:52:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11ea000640 1 -- 172.21.15.46:0/231709753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e40a87b0 con 0x7f11ec15c680 2024-04-25T08:52:17.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11ea000640 1 -- 172.21.15.46:0/231709753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40a8b80 con 0x7f11ec15c680 2024-04-25T08:52:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 -- 172.21.15.46:0/231709753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec15c680 msgr2=0x7f11ec166370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 --2- 172.21.15.46:0/231709753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec15c680 0x7f11ec166370 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009dc0 tx=0x7f11e401c7f0 comp rx=0 tx=0).stop 2024-04-25T08:52:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 -- 172.21.15.46:0/231709753 shutdown_connections 2024-04-25T08:52:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 --2- 172.21.15.46:0/231709753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec15c680 0x7f11ec166370 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 -- 172.21.15.46:0/231709753 >> 172.21.15.46:0/231709753 conn(0x7f11ec067050 msgr2=0x7f11ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 -- 172.21.15.46:0/231709753 shutdown_connections 2024-04-25T08:52:17.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 -- 172.21.15.46:0/231709753 wait complete. 2024-04-25T08:52:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 Processor -- start 2024-04-25T08:52:17.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 -- start start 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec0718f0 0x7f11ec13c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11ec13e6b0 0x7f11ec1641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11ec15c680 0x7f11ec164720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec1696f0 con 0x7f11ec0718f0 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec169870 con 0x7f11ec13e6b0 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11f1600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec1699f0 con 0x7f11ec15c680 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec0718f0 0x7f11ec13c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec0718f0 0x7f11ec13c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53614/0 (socket says 172.21.15.46:53614) 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11eaa00640 1 -- 172.21.15.46:0/2276517193 learned_addr learned my addr 172.21.15.46:0/2276517193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11eb400640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11ec15c680 0x7f11ec164720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.882+0000 7f11e2000640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11ec13e6b0 0x7f11ec1641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:17.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11eaa00640 1 -- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11ec15c680 msgr2=0x7f11ec164720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11eaa00640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11ec15c680 0x7f11ec164720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11eaa00640 1 -- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11ec13e6b0 msgr2=0x7f11ec1641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11eaa00640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11ec13e6b0 0x7f11ec1641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11eaa00640 1 -- 172.21.15.46:0/2276517193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec163840 con 0x7f11ec0718f0 2024-04-25T08:52:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11eaa00640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec0718f0 0x7f11ec13c830 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00c450 tx=0x7f11dc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:17.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11e3400640 1 -- 172.21.15.46:0/2276517193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc012830 con 0x7f11ec0718f0 2024-04-25T08:52:17.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11e3400640 1 -- 172.21.15.46:0/2276517193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc017070 con 0x7f11ec0718f0 2024-04-25T08:52:17.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11e3400640 1 -- 172.21.15.46:0/2276517193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc012c30 con 0x7f11ec0718f0 2024-04-25T08:52:17.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ec1627a0 con 0x7f11ec0718f0 2024-04-25T08:52:17.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ec162d60 con 0x7f11ec0718f0 2024-04-25T08:52:17.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ec069080 con 0x7f11ec0718f0 2024-04-25T08:52:17.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11e3400640 1 -- 172.21.15.46:0/2276517193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11dc01c460 con 0x7f11ec0718f0 2024-04-25T08:52:17.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11e3400640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c8066f60 0x7f11c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:17.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11e3400640 1 -- 172.21.15.46:0/2276517193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5864+0+0 (secure 0 0 0) 0x7f11dc084fb0 con 0x7f11ec0718f0 2024-04-25T08:52:17.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.886+0000 7f11e2000640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c8066f60 0x7f11c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.890+0000 7f11e2000640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c8066f60 0x7f11c8069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f11ec15d270 tx=0x7f11d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:17.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:17.890+0000 7f11e3400640 1 -- 172.21.15.46:0/2276517193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11dc055e80 con 0x7f11ec0718f0 2024-04-25T08:52:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.058+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11ec0cf240 con 0x7f11ec0718f0 2024-04-25T08:52:18.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.710+0000 7f11e3400640 1 -- 172.21.15.46:0/2276517193 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7f11dc055780 con 0x7f11ec0718f0 2024-04-25T08:52:18.703 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-25T08:52:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c8066f60 msgr2=0x7f11c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c8066f60 0x7f11c8069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f11ec15d270 tx=0x7f11d4009040 comp rx=0 tx=0).stop 2024-04-25T08:52:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec0718f0 msgr2=0x7f11ec13c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec0718f0 0x7f11ec13c830 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00c450 tx=0x7f11dc00c910 comp rx=0 tx=0).stop 2024-04-25T08:52:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 shutdown_connections 2024-04-25T08:52:18.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f11c8066f60 0x7f11c8069420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11ec15c680 0x7f11ec164720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11ec13e6b0 0x7f11ec1641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 --2- 172.21.15.46:0/2276517193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11ec0718f0 0x7f11ec13c830 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 >> 172.21.15.46:0/2276517193 conn(0x7f11ec067050 msgr2=0x7f11ec0cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 shutdown_connections 2024-04-25T08:52:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.714+0000 7f11f1600640 1 -- 172.21.15.46:0/2276517193 wait complete. 2024-04-25T08:52:18.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:52:18.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.886+0000 7f7e00000640 1 Processor -- start 2024-04-25T08:52:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7e00000640 1 -- start start 2024-04-25T08:52:18.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7e00000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 0x7f7df80cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:18.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7e00000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7df806f150 0x7f7df80b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:18.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7e00000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df815e440 con 0x7f7df8143990 2024-04-25T08:52:18.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7e00000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df815ed30 con 0x7f7df806f150 2024-04-25T08:52:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7e00000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df815e6e0 con 0x7f7df81440d0 2024-04-25T08:52:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfe000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7df806f150 0x7f7df80b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfcc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 0x7f7df80cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfe000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7df806f150 0x7f7df80b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:54048/0 (socket says 172.21.15.46:54048) 2024-04-25T08:52:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfe000640 1 -- 172.21.15.46:0/1468627962 learned_addr learned my addr 172.21.15.46:0/1468627962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfd600640 1 --1- 172.21.15.46:0/1468627962 >> v1:172.21.15.82:6789/0 conn(0x7f7df8143990 0x7f7df806ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:48084/0 (socket says 172.21.15.46:48084) 2024-04-25T08:52:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfe000640 1 -- 172.21.15.46:0/1468627962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 msgr2=0x7f7df80cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:18.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfe000640 1 --2- 172.21.15.46:0/1468627962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 0x7f7df80cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfe000640 1 -- 172.21.15.46:0/1468627962 >> v1:172.21.15.82:6789/0 conn(0x7f7df8143990 legacy=0x7f7df806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:18.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfe000640 1 -- 172.21.15.46:0/1468627962 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df815e9c0 con 0x7f7df806f150 2024-04-25T08:52:18.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.890+0000 7f7dfcc00640 1 --2- 172.21.15.46:0/1468627962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 0x7f7df80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:18.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7dfe000640 1 --2- 172.21.15.46:0/1468627962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df806f150 0x7f7df80b2720 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009e50 tx=0x7f7df001c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7633695af9c7c6cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:18.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7df7e00640 1 -- 172.21.15.46:0/1468627962 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df00a5660 con 0x7f7df806f150 2024-04-25T08:52:18.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7df7e00640 1 -- 172.21.15.46:0/1468627962 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df00a5800 con 0x7f7df806f150 2024-04-25T08:52:18.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7df7e00640 1 -- 172.21.15.46:0/1468627962 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df00a5bd0 con 0x7f7df806f150 2024-04-25T08:52:18.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 -- 172.21.15.46:0/1468627962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df806f150 msgr2=0x7f7df80b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:18.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 --2- 172.21.15.46:0/1468627962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df806f150 0x7f7df80b2720 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009e50 tx=0x7f7df001c9a0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 -- 172.21.15.46:0/1468627962 shutdown_connections 2024-04-25T08:52:18.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 --2- 172.21.15.46:0/1468627962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df806f150 0x7f7df80b2720 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 --2- 172.21.15.46:0/1468627962 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 0x7f7df80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 -- 172.21.15.46:0/1468627962 >> 172.21.15.46:0/1468627962 conn(0x7f7df806ac10 msgr2=0x7f7df806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:18.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 -- 172.21.15.46:0/1468627962 shutdown_connections 2024-04-25T08:52:18.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 -- 172.21.15.46:0/1468627962 wait complete. 2024-04-25T08:52:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 Processor -- start 2024-04-25T08:52:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.894+0000 7f7e00000640 1 -- start start 2024-04-25T08:52:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7e00000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df806f150 0x7f7df815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7e00000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df8143990 0x7f7df815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:18.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7e00000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 0x7f7df80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7e00000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df80b35f0 con 0x7f7df806f150 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7e00000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df80b3470 con 0x7f7df8143990 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7e00000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df80b3770 con 0x7f7df81440d0 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df806f150 0x7f7df815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df806f150 0x7f7df815bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40718/0 (socket says 172.21.15.46:40718) 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfd600640 1 -- 172.21.15.46:0/3411870116 learned_addr learned my addr 172.21.15.46:0/3411870116 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfe000640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 0x7f7df80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfcc00640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df8143990 0x7f7df815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfd600640 1 -- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 msgr2=0x7f7df80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfd600640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 0x7f7df80b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfd600640 1 -- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df8143990 msgr2=0x7f7df815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfd600640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df8143990 0x7f7df815c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfd600640 1 -- 172.21.15.46:0/3411870116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df815df70 con 0x7f7df806f150 2024-04-25T08:52:18.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfcc00640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df8143990 0x7f7df815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7dfd600640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df806f150 0x7f7df815bf10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00c4c0 tx=0x7f7dec00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7df5600640 1 -- 172.21.15.46:0/3411870116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec018070 con 0x7f7df806f150 2024-04-25T08:52:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7df5600640 1 -- 172.21.15.46:0/3411870116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec007480 con 0x7f7df806f150 2024-04-25T08:52:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7df5600640 1 -- 172.21.15.46:0/3411870116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec0047f0 con 0x7f7df806f150 2024-04-25T08:52:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df815ced0 con 0x7f7df806f150 2024-04-25T08:52:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df815d490 con 0x7f7df806f150 2024-04-25T08:52:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df8069010 con 0x7f7df806f150 2024-04-25T08:52:18.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.898+0000 7f7df5600640 1 -- 172.21.15.46:0/3411870116 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dec01b070 con 0x7f7df806f150 2024-04-25T08:52:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.902+0000 7f7df5600640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dd8066f60 0x7f7dd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.902+0000 7f7df5600640 1 -- 172.21.15.46:0/3411870116 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7dec0840f0 con 0x7f7df806f150 2024-04-25T08:52:18.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.902+0000 7f7dfcc00640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dd8066f60 0x7f7dd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:18.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.902+0000 7f7dfcc00640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dd8066f60 0x7f7dd8069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b820 tx=0x7f7de400b3a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:18.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:18.906+0000 7f7df5600640 1 -- 172.21.15.46:0/3411870116 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dec054bc0 con 0x7f7df806f150 2024-04-25T08:52:19.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.082+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7df80cdf40 con 0x7f7df806f150 2024-04-25T08:52:19.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7df5600640 1 -- 172.21.15.46:0/3411870116 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7dec058c60 con 0x7f7df806f150 2024-04-25T08:52:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dd8066f60 msgr2=0x7f7dd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dd8066f60 0x7f7dd8069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b820 tx=0x7f7de400b3a0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df806f150 msgr2=0x7f7df815bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df806f150 0x7f7df815bf10 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00c4c0 tx=0x7f7dec00c980 comp rx=0 tx=0).stop 2024-04-25T08:52:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 shutdown_connections 2024-04-25T08:52:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7dd8066f60 0x7f7dd8069420 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7df81440d0 0x7f7df80b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df8143990 0x7f7df815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 --2- 172.21.15.46:0/3411870116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7df806f150 0x7f7df815bf10 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 >> 172.21.15.46:0/3411870116 conn(0x7f7df806ac10 msgr2=0x7f7df8163850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:19.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 shutdown_connections 2024-04-25T08:52:19.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.086+0000 7f7e00000640 1 -- 172.21.15.46:0/3411870116 wait complete. 2024-04-25T08:52:19.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T08:52:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f260dc00640 1 Processor -- start 2024-04-25T08:52:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f260dc00640 1 -- start start 2024-04-25T08:52:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f260dc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f260815dca0 0x7f26080d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f260dc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f260806ed70 con 0x7f26080cf140 2024-04-25T08:52:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f260dc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2608162530 con 0x7f26080afd40 2024-04-25T08:52:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f260dc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26080afb40 con 0x7f260815dca0 2024-04-25T08:52:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2606a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26080cf140 0x7f260815d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39270/0 (socket says 172.21.15.46:39270) 2024-04-25T08:52:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2606a00640 1 -- 172.21.15.46:0/2526206086 learned_addr learned my addr 172.21.15.46:0/2526206086 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2606000640 1 --2- 172.21.15.46:0/2526206086 >> v2:172.21.15.82:3300/0 conn(0x7f260815dca0 0x7f26080d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831857624 0 0) 0x7f25f8001120 con 0x7f26080cf140 2024-04-25T08:52:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25ec003530 con 0x7f26080cf140 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3850166019 0 0) 0x7f25ec003530 con 0x7f26080cf140 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 >> v2:172.21.15.82:3300/0 conn(0x7f260815dca0 msgr2=0x7f26080d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 --2- 172.21.15.46:0/2526206086 >> v2:172.21.15.82:3300/0 conn(0x7f260815dca0 0x7f26080d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 >> v1:172.21.15.82:6789/0 conn(0x7f26080afd40 legacy=0x7f26080d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608161770 con 0x7f26080cf140 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f25f8001850 con 0x7f26080cf140 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 >> v1:172.21.15.46:6789/0 conn(0x7f26080cf140 legacy=0x7f260815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec0039c0 0x7f25ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec004810 0x7f25ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25ec0051d0 0x7f25ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26080afb40 con 0x7f25ec0051d0 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec00c170 con 0x7f25ec004810 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec00c370 con 0x7f25ec0039c0 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2607400640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25ec0051d0 0x7f25ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2606000640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec004810 0x7f25ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2606a00640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec0039c0 0x7f25ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2607400640 1 -- 172.21.15.46:0/2526206086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec0039c0 msgr2=0x7f25ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2607400640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec0039c0 0x7f25ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2607400640 1 -- 172.21.15.46:0/2526206086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec004810 msgr2=0x7f25ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2607400640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25ec004810 0x7f25ec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.250+0000 7f2607400640 1 -- 172.21.15.46:0/2526206086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec003530 con 0x7f25ec0051d0 2024-04-25T08:52:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f2607400640 1 -- 172.21.15.46:0/2526206086 reap_dead start 2024-04-25T08:52:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f2607400640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25ec0051d0 0x7f25ec00b9d0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f2600001e40 tx=0x7f260001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacbf657be07bec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600001a90 con 0x7f25ec0051d0 2024-04-25T08:52:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2600003b80 con 0x7f25ec0051d0 2024-04-25T08:52:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f2605600640 1 -- 172.21.15.46:0/2526206086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2600005020 con 0x7f25ec0051d0 2024-04-25T08:52:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f260dc00640 1 -- 172.21.15.46:0/2526206086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25ec0051d0 msgr2=0x7f25ec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f260dc00640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25ec0051d0 0x7f25ec00b9d0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f2600001e40 tx=0x7f260001c760 comp rx=0 tx=0).stop 2024-04-25T08:52:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f260dc00640 1 -- 172.21.15.46:0/2526206086 shutdown_connections 2024-04-25T08:52:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f260dc00640 1 --2- 172.21.15.46:0/2526206086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25ec0051d0 0x7f25ec00b9d0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f260dc00640 1 -- 172.21.15.46:0/2526206086 >> 172.21.15.46:0/2526206086 conn(0x7f26080728c0 msgr2=0x7f2608099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f260dc00640 1 -- 172.21.15.46:0/2526206086 shutdown_connections 2024-04-25T08:52:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.254+0000 7f260dc00640 1 -- 172.21.15.46:0/2526206086 wait complete. 2024-04-25T08:52:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 Processor -- start 2024-04-25T08:52:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 -- start start 2024-04-25T08:52:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec0051d0 0x7f26081f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080b2a90 0x7f26081f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26081f70e0 0x7f26080d0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608162010 con 0x7f26080b2a90 2024-04-25T08:52:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2608161d30 con 0x7f26081f70e0 2024-04-25T08:52:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26080af7f0 con 0x7f25ec0051d0 2024-04-25T08:52:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080b2a90 0x7f26081f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080b2a90 0x7f26081f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40738/0 (socket says 172.21.15.46:40738) 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606000640 1 -- 172.21.15.46:0/2119632069 learned_addr learned my addr 172.21.15.46:0/2119632069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606a00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec0051d0 0x7f26081f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2607400640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26081f70e0 0x7f26080d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606000640 1 -- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec0051d0 msgr2=0x7f26081f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606000640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec0051d0 0x7f26081f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606000640 1 -- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26081f70e0 msgr2=0x7f26080d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606000640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26081f70e0 0x7f26080d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606000640 1 -- 172.21.15.46:0/2119632069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26081601c0 con 0x7f26080b2a90 2024-04-25T08:52:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f2606000640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080b2a90 0x7f26081f6ba0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f25f4011960 tx=0x7f25f4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f25fea00640 1 -- 172.21.15.46:0/2119632069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f4019070 con 0x7f26080b2a90 2024-04-25T08:52:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f25fea00640 1 -- 172.21.15.46:0/2119632069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f4004240 con 0x7f26080b2a90 2024-04-25T08:52:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f25fea00640 1 -- 172.21.15.46:0/2119632069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f400f880 con 0x7f26080b2a90 2024-04-25T08:52:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26080b2f10 con 0x7f26080b2a90 2024-04-25T08:52:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26081f79f0 con 0x7f26080b2a90 2024-04-25T08:52:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.258+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260806ccb0 con 0x7f26080b2a90 2024-04-25T08:52:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.262+0000 7f25fea00640 1 -- 172.21.15.46:0/2119632069 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25f401c070 con 0x7f26080b2a90 2024-04-25T08:52:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.262+0000 7f25fea00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25e8066f90 0x7f25e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.262+0000 7f25fea00640 1 -- 172.21.15.46:0/2119632069 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f25f4085210 con 0x7f26080b2a90 2024-04-25T08:52:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.262+0000 7f2606a00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25e8066f90 0x7f25e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.262+0000 7f2606a00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25e8066f90 0x7f25e8069450 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f25f8001060 tx=0x7f25f80011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:19.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.266+0000 7f25fea00640 1 -- 172.21.15.46:0/2119632069 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25f4055cf0 con 0x7f26080b2a90 2024-04-25T08:52:19.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.486+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f2608069080 con 0x7f26080b2a90 2024-04-25T08:52:19.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f25fea00640 1 -- 172.21.15.46:0/2119632069 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v473) v1 ==== 146+0+0 (secure 0 0 0) 0x7f25f4059d90 con 0x7f26080b2a90 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25e8066f90 msgr2=0x7f25e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25e8066f90 0x7f25e8069450 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f25f8001060 tx=0x7f25f80011a0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080b2a90 msgr2=0x7f26081f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080b2a90 0x7f26081f6ba0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f25f4011960 tx=0x7f25f4011e20 comp rx=0 tx=0).stop 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 shutdown_connections 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f25e8066f90 0x7f25e8069450 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26081f70e0 0x7f26080d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26080b2a90 0x7f26081f6ba0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 --2- 172.21.15.46:0/2119632069 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f25ec0051d0 0x7f26081f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 >> 172.21.15.46:0/2119632069 conn(0x7f26080728c0 msgr2=0x7f260806ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:19.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 shutdown_connections 2024-04-25T08:52:19.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.714+0000 7f260dc00640 1 -- 172.21.15.46:0/2119632069 wait complete. 2024-04-25T08:52:19.727 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T08:52:19.727 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:52:19.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:19.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9850600640 1 Processor -- start 2024-04-25T08:52:19.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9850600640 1 -- start start 2024-04-25T08:52:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9850600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f98480cc150 0x7f98480b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9850600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98480b25a0 con 0x7f98480cdd10 2024-04-25T08:52:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9850600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98480cc6f0 con 0x7f9848157b00 2024-04-25T08:52:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9850600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984806d980 con 0x7f98480cc150 2024-04-25T08:52:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f984e600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98480cdd10 0x7f98480ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39284/0 (socket says 172.21.15.46:39284) 2024-04-25T08:52:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f984e600640 1 -- 172.21.15.46:0/1785652430 learned_addr learned my addr 172.21.15.46:0/1785652430 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:19.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f984d200640 1 --2- 172.21.15.46:0/1785652430 >> v2:172.21.15.82:3300/0 conn(0x7f98480cc150 0x7f98480b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3484062779 0 0) 0x7f98480b25a0 con 0x7f98480cdd10 2024-04-25T08:52:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9830003530 con 0x7f98480cdd10 2024-04-25T08:52:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550787359 0 0) 0x7f98480cc6f0 con 0x7f9848157b00 2024-04-25T08:52:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98480b25a0 con 0x7f9848157b00 2024-04-25T08:52:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3236629069 0 0) 0x7f9830003530 con 0x7f98480cdd10 2024-04-25T08:52:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 >> v2:172.21.15.82:3300/0 conn(0x7f98480cc150 msgr2=0x7f98480b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 --2- 172.21.15.46:0/1785652430 >> v2:172.21.15.82:3300/0 conn(0x7f98480cc150 0x7f98480b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 >> v1:172.21.15.82:6789/0 conn(0x7f9848157b00 legacy=0x7f984806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98480b2860 con 0x7f98480cdd10 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f9840001a00 con 0x7f98480cdd10 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f984d200640 1 --2- 172.21.15.46:0/1785652430 >> v2:172.21.15.82:3300/0 conn(0x7f98480cc150 0x7f98480b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 >> v1:172.21.15.46:6789/0 conn(0x7f98480cdd10 legacy=0x7f98480ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9830003ba0 0x7f9830003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f9830004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9830005290 0x7f983000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984806d980 con 0x7f9830004810 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983000c1d0 con 0x7f9830005290 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983000c3d0 con 0x7f9830003ba0 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f984d200640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f9830004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f984e600640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9830005290 0x7f983000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.878+0000 7f984dc00640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9830003ba0 0x7f9830003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f984d200640 1 -- 172.21.15.46:0/1785652430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9830003ba0 msgr2=0x7f9830003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:19.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f984d200640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9830003ba0 0x7f9830003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f984d200640 1 -- 172.21.15.46:0/1785652430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9830005290 msgr2=0x7f983000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:19.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f984d200640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9830005290 0x7f983000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f984d200640 1 -- 172.21.15.46:0/1785652430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830003530 con 0x7f9830004810 2024-04-25T08:52:19.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f984e600640 1 -- 172.21.15.46:0/1785652430 reap_dead start 2024-04-25T08:52:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f984d200640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f9830004c90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f9834009cb0 tx=0x7f983401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c28467fffe43a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98340b2070 con 0x7f9830004810 2024-04-25T08:52:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98340a57a0 con 0x7f9830004810 2024-04-25T08:52:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f9847e00640 1 -- 172.21.15.46:0/1785652430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834009880 con 0x7f9830004810 2024-04-25T08:52:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f9850600640 1 -- 172.21.15.46:0/1785652430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 msgr2=0x7f9830004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.882+0000 7f9850600640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f9830004c90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f9834009cb0 tx=0x7f983401cc10 comp rx=0 tx=0).stop 2024-04-25T08:52:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 -- 172.21.15.46:0/1785652430 shutdown_connections 2024-04-25T08:52:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 --2- 172.21.15.46:0/1785652430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f9830004c90 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 -- 172.21.15.46:0/1785652430 >> 172.21.15.46:0/1785652430 conn(0x7f98480b12a0 msgr2=0x7f98480b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:19.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 -- 172.21.15.46:0/1785652430 shutdown_connections 2024-04-25T08:52:19.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 -- 172.21.15.46:0/1785652430 wait complete. 2024-04-25T08:52:19.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 Processor -- start 2024-04-25T08:52:19.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 -- start start 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f98481f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9848156670 0x7f984814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9848156af0 0x7f984814e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984806d800 con 0x7f9830004810 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984806d680 con 0x7f9848156af0 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9850600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98480ce900 con 0x7f9848156670 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f98481f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f98481f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40762/0 (socket says 172.21.15.46:40762) 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984dc00640 1 -- 172.21.15.46:0/3074505211 learned_addr learned my addr 172.21.15.46:0/3074505211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984e600640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9848156af0 0x7f984814e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984d200640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9848156670 0x7f984814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984dc00640 1 -- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9848156670 msgr2=0x7f984814dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984dc00640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9848156670 0x7f984814dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984dc00640 1 -- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9848156af0 msgr2=0x7f984814e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984dc00640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9848156af0 0x7f984814e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984dc00640 1 -- 172.21.15.46:0/3074505211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9848154ea0 con 0x7f9830004810 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984d200640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9848156670 0x7f984814dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984e600640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9848156af0 0x7f984814e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f984dc00640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f98481f7750 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f983c00f770 tx=0x7f983c00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.886+0000 7f9845600640 1 -- 172.21.15.46:0/3074505211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0050d0 con 0x7f9830004810 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.890+0000 7f9845600640 1 -- 172.21.15.46:0/3074505211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c018070 con 0x7f9830004810 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.890+0000 7f9845600640 1 -- 172.21.15.46:0/3074505211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00d730 con 0x7f9830004810 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.890+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98481f7c90 con 0x7f9830004810 2024-04-25T08:52:19.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.890+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9848153fd0 con 0x7f9830004810 2024-04-25T08:52:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.890+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9848069080 con 0x7f9830004810 2024-04-25T08:52:19.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.890+0000 7f9845600640 1 -- 172.21.15.46:0/3074505211 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f983c00dae0 con 0x7f9830004810 2024-04-25T08:52:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.890+0000 7f9845600640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9828066f90 0x7f9828069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.890+0000 7f9845600640 1 -- 172.21.15.46:0/3074505211 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f983c085070 con 0x7f9830004810 2024-04-25T08:52:19.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.890+0000 7f984d200640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9828066f90 0x7f9828069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:19.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.894+0000 7f984d200640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9828066f90 0x7f9828069450 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f98340af000 tx=0x7f9834008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:19.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:19.894+0000 7f9845600640 1 -- 172.21.15.46:0/3074505211 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f983c056100 con 0x7f9830004810 2024-04-25T08:52:20.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.058+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98480724a0 con 0x7f9830004810 2024-04-25T08:52:20.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9845600640 1 -- 172.21.15.46:0/3074505211 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+13383 (secure 0 0 0) 0x7f983c055a00 con 0x7f9830004810 2024-04-25T08:52:20.052 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:20.052 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":112,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:18.707994+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1174892156":"2024-04-26T08:52:11.019690+0000","172.21.15.46:6834/1174892156":"2024-04-26T08:52:11.019690+0000","172.21.15.46:6835/623970467":"2024-04-26T08:51:34.299560+0000","172.21.15.46:6834/623970467":"2024-04-26T08:51:34.299560+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9828066f90 msgr2=0x7f9828069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9828066f90 0x7f9828069450 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f98340af000 tx=0x7f9834008040 comp rx=0 tx=0).stop 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 msgr2=0x7f98481f7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f98481f7750 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f983c00f770 tx=0x7f983c00fc30 comp rx=0 tx=0).stop 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 shutdown_connections 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9828066f90 0x7f9828069450 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9848156af0 0x7f984814e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9848156670 0x7f984814dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 --2- 172.21.15.46:0/3074505211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9830004810 0x7f98481f7750 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 >> 172.21.15.46:0/3074505211 conn(0x7f98480b12a0 msgr2=0x7f984814ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 shutdown_connections 2024-04-25T08:52:20.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.062+0000 7f9850600640 1 -- 172.21.15.46:0/3074505211 wait complete. 2024-04-25T08:52:20.074 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:52:20.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.226+0000 7f5d2f800640 1 Processor -- start 2024-04-25T08:52:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.226+0000 7f5d2f800640 1 -- start start 2024-04-25T08:52:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.226+0000 7f5d2f800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5d281440d0 0x7f5d280cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.226+0000 7f5d2f800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2815e440 con 0x7f5d2806f150 2024-04-25T08:52:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.226+0000 7f5d2f800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2815e6e0 con 0x7f5d28143990 2024-04-25T08:52:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.226+0000 7f5d2f800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2815ed30 con 0x7f5d281440d0 2024-04-25T08:52:20.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.226+0000 7f5d2d800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5d2806f150 0x7f5d280b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39300/0 (socket says 172.21.15.46:39300) 2024-04-25T08:52:20.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.226+0000 7f5d2d800640 1 -- 172.21.15.46:0/3793313721 learned_addr learned my addr 172.21.15.46:0/3793313721 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:20.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d1fe00640 1 --2- 172.21.15.46:0/3793313721 >> v2:172.21.15.82:3300/0 conn(0x7f5d281440d0 0x7f5d280cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d27e00640 1 -- 172.21.15.46:0/3793313721 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837325161 0 0) 0x7f5d2815e440 con 0x7f5d2806f150 2024-04-25T08:52:20.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d27e00640 1 -- 172.21.15.46:0/3793313721 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d0c003530 con 0x7f5d2806f150 2024-04-25T08:52:20.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d1fe00640 1 -- 172.21.15.46:0/3793313721 >> v1:172.21.15.82:6789/0 conn(0x7f5d28143990 legacy=0x7f5d2806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d1fe00640 1 -- 172.21.15.46:0/3793313721 >> v1:172.21.15.46:6789/0 conn(0x7f5d2806f150 legacy=0x7f5d280b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d1fe00640 1 -- 172.21.15.46:0/3793313721 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2815e9c0 con 0x7f5d281440d0 2024-04-25T08:52:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d1fe00640 1 --2- 172.21.15.46:0/3793313721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d281440d0 0x7f5d280cecc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5d2815eeb0 tx=0x7f5d1001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e20cc847e2b812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d27e00640 1 -- 172.21.15.46:0/3793313721 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d100b0070 con 0x7f5d281440d0 2024-04-25T08:52:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d27e00640 1 -- 172.21.15.46:0/3793313721 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d100ab600 con 0x7f5d281440d0 2024-04-25T08:52:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.230+0000 7f5d27e00640 1 -- 172.21.15.46:0/3793313721 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d100ab9d0 con 0x7f5d281440d0 2024-04-25T08:52:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 -- 172.21.15.46:0/3793313721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d281440d0 msgr2=0x7f5d280cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 --2- 172.21.15.46:0/3793313721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d281440d0 0x7f5d280cecc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5d2815eeb0 tx=0x7f5d1001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 -- 172.21.15.46:0/3793313721 shutdown_connections 2024-04-25T08:52:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 --2- 172.21.15.46:0/3793313721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d281440d0 0x7f5d280cecc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 -- 172.21.15.46:0/3793313721 >> 172.21.15.46:0/3793313721 conn(0x7f5d2806ac10 msgr2=0x7f5d2806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 -- 172.21.15.46:0/3793313721 shutdown_connections 2024-04-25T08:52:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 -- 172.21.15.46:0/3793313721 wait complete. 2024-04-25T08:52:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 Processor -- start 2024-04-25T08:52:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 -- start start 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d2806f150 0x7f5d28159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d28143990 0x7f5d2815adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d281440d0 0x7f5d28159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d281686a0 con 0x7f5d281440d0 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28168520 con 0x7f5d2806f150 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2f800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28168820 con 0x7f5d28143990 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d281440d0 0x7f5d28159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d281440d0 0x7f5d28159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40768/0 (socket says 172.21.15.46:40768) 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2d800640 1 -- 172.21.15.46:0/544086940 learned_addr learned my addr 172.21.15.46:0/544086940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2ce00640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d2806f150 0x7f5d28159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d1fe00640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d28143990 0x7f5d2815adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2d800640 1 -- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d28143990 msgr2=0x7f5d2815adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2d800640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d28143990 0x7f5d2815adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2d800640 1 -- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d2806f150 msgr2=0x7f5d28159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2d800640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d2806f150 0x7f5d28159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.234+0000 7f5d2d800640 1 -- 172.21.15.46:0/544086940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d281599b0 con 0x7f5d281440d0 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d2d800640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d281440d0 0x7f5d28159470 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000cb40 tx=0x7f5d2000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d25600640 1 -- 172.21.15.46:0/544086940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20017070 con 0x7f5d281440d0 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d25600640 1 -- 172.21.15.46:0/544086940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2000aca0 con 0x7f5d281440d0 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d25600640 1 -- 172.21.15.46:0/544086940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20012620 con 0x7f5d281440d0 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2815b480 con 0x7f5d281440d0 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2815ba40 con 0x7f5d281440d0 2024-04-25T08:52:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d28069010 con 0x7f5d281440d0 2024-04-25T08:52:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d25600640 1 -- 172.21.15.46:0/544086940 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d2001a070 con 0x7f5d281440d0 2024-04-25T08:52:20.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d25600640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d04066f60 0x7f5d04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d25600640 1 -- 172.21.15.46:0/544086940 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5d20084be0 con 0x7f5d281440d0 2024-04-25T08:52:20.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.238+0000 7f5d2ce00640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d04066f60 0x7f5d04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.242+0000 7f5d2ce00640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d04066f60 0x7f5d04069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5d18007fd0 tx=0x7f5d18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:20.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.242+0000 7f5d25600640 1 -- 172.21.15.46:0/544086940 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d20055750 con 0x7f5d281440d0 2024-04-25T08:52:20.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.406+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d280b2c10 con 0x7f5d281440d0 2024-04-25T08:52:20.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.406+0000 7f5d25600640 1 -- 172.21.15.46:0/544086940 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d200597f0 con 0x7f5d281440d0 2024-04-25T08:52:20.398 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:52:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d04066f60 msgr2=0x7f5d04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d04066f60 0x7f5d04069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5d18007fd0 tx=0x7f5d18009040 comp rx=0 tx=0).stop 2024-04-25T08:52:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d281440d0 msgr2=0x7f5d28159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d281440d0 0x7f5d28159470 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000cb40 tx=0x7f5d2000a060 comp rx=0 tx=0).stop 2024-04-25T08:52:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 shutdown_connections 2024-04-25T08:52:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d04066f60 0x7f5d04069420 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d281440d0 0x7f5d28159470 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d28143990 0x7f5d2815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 --2- 172.21.15.46:0/544086940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d2806f150 0x7f5d28159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 >> 172.21.15.46:0/544086940 conn(0x7f5d2806ac10 msgr2=0x7f5d2813e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 shutdown_connections 2024-04-25T08:52:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.410+0000 7f5d2f800640 1 -- 172.21.15.46:0/544086940 wait complete. 2024-04-25T08:52:20.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T08:52:20.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c48600640 1 Processor -- start 2024-04-25T08:52:20.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c48600640 1 -- start start 2024-04-25T08:52:20.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c48600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 0x7f8c4009aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c48600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4015f300 con 0x7f8c40099a00 2024-04-25T08:52:20.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c48600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c400cd8a0 con 0x7f8c4015ef20 2024-04-25T08:52:20.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c48600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c400cddb0 con 0x7f8c4009a610 2024-04-25T08:52:20.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c46600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8c40099a00 0x7f8c400c5180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39304/0 (socket says 172.21.15.46:39304) 2024-04-25T08:52:20.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c46600640 1 -- 172.21.15.46:0/304544564 learned_addr learned my addr 172.21.15.46:0/304544564 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c45200640 1 --2- 172.21.15.46:0/304544564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 0x7f8c4009aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c3fe00640 1 -- 172.21.15.46:0/304544564 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573214257 0 0) 0x7f8c4015f300 con 0x7f8c40099a00 2024-04-25T08:52:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c3fe00640 1 -- 172.21.15.46:0/304544564 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c28003530 con 0x7f8c40099a00 2024-04-25T08:52:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c3fe00640 1 -- 172.21.15.46:0/304544564 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645126589 0 0) 0x7f8c400cd8a0 con 0x7f8c4015ef20 2024-04-25T08:52:20.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c3fe00640 1 -- 172.21.15.46:0/304544564 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c4015f300 con 0x7f8c4015ef20 2024-04-25T08:52:20.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c45200640 1 -- 172.21.15.46:0/304544564 >> v1:172.21.15.82:6789/0 conn(0x7f8c4015ef20 legacy=0x7f8c400cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c45200640 1 -- 172.21.15.46:0/304544564 >> v1:172.21.15.46:6789/0 conn(0x7f8c40099a00 legacy=0x7f8c400c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.570+0000 7f8c45200640 1 -- 172.21.15.46:0/304544564 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c400cc3d0 con 0x7f8c4009a610 2024-04-25T08:52:20.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c45200640 1 --2- 172.21.15.46:0/304544564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 0x7f8c4009aa10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0097f0 tx=0x7f8c2c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=851d63fc84ab7093 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:20.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c3fe00640 1 -- 172.21.15.46:0/304544564 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c0b1070 con 0x7f8c4009a610 2024-04-25T08:52:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c3fe00640 1 -- 172.21.15.46:0/304544564 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2c0ac630 con 0x7f8c4009a610 2024-04-25T08:52:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c3fe00640 1 -- 172.21.15.46:0/304544564 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c0a8050 con 0x7f8c4009a610 2024-04-25T08:52:20.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c48600640 1 -- 172.21.15.46:0/304544564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 msgr2=0x7f8c4009aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c48600640 1 --2- 172.21.15.46:0/304544564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 0x7f8c4009aa10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c0097f0 tx=0x7f8c2c01c4e0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c48600640 1 -- 172.21.15.46:0/304544564 shutdown_connections 2024-04-25T08:52:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c48600640 1 --2- 172.21.15.46:0/304544564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 0x7f8c4009aa10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c48600640 1 -- 172.21.15.46:0/304544564 >> 172.21.15.46:0/304544564 conn(0x7f8c400b3120 msgr2=0x7f8c400b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c48600640 1 -- 172.21.15.46:0/304544564 shutdown_connections 2024-04-25T08:52:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.574+0000 7f8c48600640 1 -- 172.21.15.46:0/304544564 wait complete. 2024-04-25T08:52:20.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 Processor -- start 2024-04-25T08:52:20.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 -- start start 2024-04-25T08:52:20.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c40099a00 0x7f8c4014f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 0x7f8c40155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c4015ef20 0x7f8c4015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4009a220 con 0x7f8c4015ef20 2024-04-25T08:52:20.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c400cc010 con 0x7f8c40099a00 2024-04-25T08:52:20.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c40151470 con 0x7f8c4009a610 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c46600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c4015ef20 0x7f8c4015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c46600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c4015ef20 0x7f8c4015d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40778/0 (socket says 172.21.15.46:40778) 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c46600640 1 -- 172.21.15.46:0/1247895728 learned_addr learned my addr 172.21.15.46:0/1247895728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c45200640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 0x7f8c40155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c45c00640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c40099a00 0x7f8c4014f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c46600640 1 -- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 msgr2=0x7f8c40155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c46600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 0x7f8c40155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c46600640 1 -- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c40099a00 msgr2=0x7f8c4014f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c46600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c40099a00 0x7f8c4014f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c46600640 1 -- 172.21.15.46:0/1247895728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4006c230 con 0x7f8c4015ef20 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c46600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c4015ef20 0x7f8c4015d570 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800caa0 tx=0x7f8c3800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c3d600640 1 -- 172.21.15.46:0/1247895728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38017070 con 0x7f8c4015ef20 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c3d600640 1 -- 172.21.15.46:0/1247895728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3800ac00 con 0x7f8c4015ef20 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c401516c0 con 0x7f8c4015ef20 2024-04-25T08:52:20.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c3d600640 1 -- 172.21.15.46:0/1247895728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38012400 con 0x7f8c4015ef20 2024-04-25T08:52:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.578+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4006dcb0 con 0x7f8c4015ef20 2024-04-25T08:52:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.582+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c40069080 con 0x7f8c4015ef20 2024-04-25T08:52:20.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.582+0000 7f8c3d600640 1 -- 172.21.15.46:0/1247895728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c3801a070 con 0x7f8c4015ef20 2024-04-25T08:52:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.582+0000 7f8c3d600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c20066f60 0x7f8c20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.582+0000 7f8c3d600640 1 -- 172.21.15.46:0/1247895728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8c38084be0 con 0x7f8c4015ef20 2024-04-25T08:52:20.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.582+0000 7f8c45c00640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c20066f60 0x7f8c20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:20.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.582+0000 7f8c45c00640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c20066f60 0x7f8c20069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8c340017c0 tx=0x7f8c34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:20.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.586+0000 7f8c3d600640 1 -- 172.21.15.46:0/1247895728 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c380556b0 con 0x7f8c4015ef20 2024-04-25T08:52:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:20.806+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f8c400cd800 con 0x7f8c4015ef20 2024-04-25T08:52:21.504 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:52:21.508 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:52:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c3d600640 1 -- 172.21.15.46:0/1247895728 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v475) v1 ==== 147+0+0 (secure 0 0 0) 0x7f8c38059750 con 0x7f8c4015ef20 2024-04-25T08:52:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c20066f60 msgr2=0x7f8c20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c20066f60 0x7f8c20069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8c340017c0 tx=0x7f8c34009040 comp rx=0 tx=0).stop 2024-04-25T08:52:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c4015ef20 msgr2=0x7f8c4015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c4015ef20 0x7f8c4015d570 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800caa0 tx=0x7f8c3800cf60 comp rx=0 tx=0).stop 2024-04-25T08:52:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 shutdown_connections 2024-04-25T08:52:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8c20066f60 0x7f8c20069420 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c4015ef20 0x7f8c4015d570 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8c4009a610 0x7f8c40155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 --2- 172.21.15.46:0/1247895728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c40099a00 0x7f8c4014f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 >> 172.21.15.46:0/1247895728 conn(0x7f8c400b3120 msgr2=0x7f8c400c62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 shutdown_connections 2024-04-25T08:52:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.722+0000 7f8c48600640 1 -- 172.21.15.46:0/1247895728 wait complete. 2024-04-25T08:52:21.739 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:52:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.894+0000 7f2fe3600640 1 Processor -- start 2024-04-25T08:52:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.894+0000 7f2fe3600640 1 -- start start 2024-04-25T08:52:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe3600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2fdc145310 0x7f2fdc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 0x7f2fdc15d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe3600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc0cd930 con 0x7f2fdc06dc70 2024-04-25T08:52:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe3600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc0cdbd0 con 0x7f2fdc145310 2024-04-25T08:52:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0cdeb0 con 0x7f2fdc15d090 2024-04-25T08:52:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fdbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2fdc06dc70 0x7f2fdc0cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39318/0 (socket says 172.21.15.46:39318) 2024-04-25T08:52:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fdbe00640 1 -- 172.21.15.46:0/959507893 learned_addr learned my addr 172.21.15.46:0/959507893 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe0c00640 1 --2- 172.21.15.46:0/959507893 >> v2:172.21.15.82:3300/0 conn(0x7f2fdc145310 0x7f2fdc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe1600640 1 --2- 172.21.15.46:0/959507893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 0x7f2fdc15d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fdb400640 1 -- 172.21.15.46:0/959507893 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 167339345 0 0) 0x7f2fdc0cd930 con 0x7f2fdc06dc70 2024-04-25T08:52:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fdb400640 1 -- 172.21.15.46:0/959507893 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc4003530 con 0x7f2fdc06dc70 2024-04-25T08:52:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe0c00640 1 -- 172.21.15.46:0/959507893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 msgr2=0x7f2fdc15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe0c00640 1 --2- 172.21.15.46:0/959507893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 0x7f2fdc15d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe0c00640 1 -- 172.21.15.46:0/959507893 >> v1:172.21.15.46:6789/0 conn(0x7f2fdc06dc70 legacy=0x7f2fdc0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.898+0000 7f2fe0c00640 1 -- 172.21.15.46:0/959507893 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc0ce190 con 0x7f2fdc145310 2024-04-25T08:52:21.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe0c00640 1 --2- 172.21.15.46:0/959507893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 0x7f2fdc06e0d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00097c0 tx=0x7f2fd001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398d4c04f0bf46c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fdb400640 1 -- 172.21.15.46:0/959507893 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00b2070 con 0x7f2fdc145310 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fdb400640 1 -- 172.21.15.46:0/959507893 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd00ad640 con 0x7f2fdc145310 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fdb400640 1 -- 172.21.15.46:0/959507893 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00ada10 con 0x7f2fdc145310 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 -- 172.21.15.46:0/959507893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 msgr2=0x7f2fdc06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 --2- 172.21.15.46:0/959507893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 0x7f2fdc06e0d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00097c0 tx=0x7f2fd001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 -- 172.21.15.46:0/959507893 shutdown_connections 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 --2- 172.21.15.46:0/959507893 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 0x7f2fdc15d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 --2- 172.21.15.46:0/959507893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 0x7f2fdc06e0d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 -- 172.21.15.46:0/959507893 >> 172.21.15.46:0/959507893 conn(0x7f2fdc06ac10 msgr2=0x7f2fdc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 -- 172.21.15.46:0/959507893 shutdown_connections 2024-04-25T08:52:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 -- 172.21.15.46:0/959507893 wait complete. 2024-04-25T08:52:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 Processor -- start 2024-04-25T08:52:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.902+0000 7f2fe3600640 1 -- start start 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe3600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 0x7f2fdc0b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 0x7f2fdc13d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe3600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc158e90 0x7f2fdc1f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe3600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc168510 con 0x7f2fdc158e90 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe3600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc0cf380 con 0x7f2fdc145310 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc168690 con 0x7f2fdc15d090 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc158e90 0x7f2fdc1f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc158e90 0x7f2fdc1f69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40794/0 (socket says 172.21.15.46:40794) 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe1600640 1 -- 172.21.15.46:0/129853948 learned_addr learned my addr 172.21.15.46:0/129853948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe0c00640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 0x7f2fdc0b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe1600640 1 -- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 msgr2=0x7f2fdc13d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fdbe00640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 0x7f2fdc13d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe1600640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 0x7f2fdc13d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe1600640 1 -- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 msgr2=0x7f2fdc0b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe1600640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 0x7f2fdc0b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe1600640 1 -- 172.21.15.46:0/129853948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc13dd70 con 0x7f2fdc158e90 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe0c00640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 0x7f2fdc0b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fdbe00640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 0x7f2fdc13d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe1600640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc158e90 0x7f2fdc1f69e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400efc0 tx=0x7f2fd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fd8c00640 1 -- 172.21.15.46:0/129853948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4018020 con 0x7f2fdc158e90 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fd8c00640 1 -- 172.21.15.46:0/129853948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd4014480 con 0x7f2fdc158e90 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fd8c00640 1 -- 172.21.15.46:0/129853948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4014870 con 0x7f2fdc158e90 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc0b2e00 con 0x7f2fdc158e90 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc0b33c0 con 0x7f2fdc158e90 2024-04-25T08:52:21.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc069010 con 0x7f2fdc158e90 2024-04-25T08:52:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.906+0000 7f2fd8c00640 1 -- 172.21.15.46:0/129853948 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2fd401b020 con 0x7f2fdc158e90 2024-04-25T08:52:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.910+0000 7f2fd8c00640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2fbc066f60 0x7f2fbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.910+0000 7f2fd8c00640 1 -- 172.21.15.46:0/129853948 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2fd4085a30 con 0x7f2fdc158e90 2024-04-25T08:52:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.910+0000 7f2fe0c00640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2fbc066f60 0x7f2fbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.910+0000 7f2fe0c00640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2fbc066f60 0x7f2fbc069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00af000 tx=0x7f2fd00a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:21.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:21.914+0000 7f2fd8c00640 1 -- 172.21.15.46:0/129853948 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2fd4022070 con 0x7f2fdc158e90 2024-04-25T08:52:22.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.074+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2fdc15e910 con 0x7f2fdc158e90 2024-04-25T08:52:22.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.078+0000 7f2fd8c00640 1 -- 172.21.15.46:0/129853948 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2fd4010040 con 0x7f2fdc158e90 2024-04-25T08:52:22.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2fbc066f60 msgr2=0x7f2fbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2fbc066f60 0x7f2fbc069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00af000 tx=0x7f2fd00a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc158e90 msgr2=0x7f2fdc1f69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc158e90 0x7f2fdc1f69e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400efc0 tx=0x7f2fd400c490 comp rx=0 tx=0).stop 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 shutdown_connections 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2fbc066f60 0x7f2fbc069420 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fdc158e90 0x7f2fdc1f69e0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2fdc15d090 0x7f2fdc13d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 --2- 172.21.15.46:0/129853948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc145310 0x7f2fdc0b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 >> 172.21.15.46:0/129853948 conn(0x7f2fdc06ac10 msgr2=0x7f2fdc159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 shutdown_connections 2024-04-25T08:52:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.082+0000 7f2fe3600640 1 -- 172.21.15.46:0/129853948 wait complete. 2024-04-25T08:52:22.092 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:52:22.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:52:22.198 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:52:22.266 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:52:22.267 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:22.267 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:22.275 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:52:22.276 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:52:22.276 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:22.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:52:22.284 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:22.285 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:22.331 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:52:22.332 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:52:22.332 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:22.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:52:22.340 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:52:22.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.526+0000 7ff38ae00640 1 Processor -- start 2024-04-25T08:52:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.526+0000 7ff38ae00640 1 -- start start 2024-04-25T08:52:22.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff38ae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3840ad4c0 0x7ff3840af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff38ae00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff38406f570 0x7ff38415c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff38ae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38415fed0 con 0x7ff38415b9a0 2024-04-25T08:52:22.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff38ae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3841601b0 con 0x7ff3840ad4c0 2024-04-25T08:52:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff38ae00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38415c0f0 con 0x7ff38406f570 2024-04-25T08:52:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff37b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3840ad4c0 0x7ff3840af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff383e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff38415b9a0 0x7ff3840acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39328/0 (socket says 172.21.15.46:39328) 2024-04-25T08:52:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff383e00640 1 -- 172.21.15.46:0/1661708014 learned_addr learned my addr 172.21.15.46:0/1661708014 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff388e00640 1 --2- 172.21.15.46:0/1661708014 >> v2:172.21.15.82:3300/0 conn(0x7ff38406f570 0x7ff38415c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff37b400640 1 -- 172.21.15.46:0/1661708014 >> v2:172.21.15.82:3300/0 conn(0x7ff38406f570 msgr2=0x7ff38415c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:22.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff37b400640 1 --2- 172.21.15.46:0/1661708014 >> v2:172.21.15.82:3300/0 conn(0x7ff38406f570 0x7ff38415c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff37b400640 1 -- 172.21.15.46:0/1661708014 >> v1:172.21.15.46:6789/0 conn(0x7ff38415b9a0 legacy=0x7ff3840acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff37b400640 1 -- 172.21.15.46:0/1661708014 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38415c3d0 con 0x7ff3840ad4c0 2024-04-25T08:52:22.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.530+0000 7ff388e00640 1 --2- 172.21.15.46:0/1661708014 >> v2:172.21.15.82:3300/0 conn(0x7ff38406f570 0x7ff38415c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:22.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff37b400640 1 --2- 172.21.15.46:0/1661708014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3840ad4c0 0x7ff3840af030 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff38406da10 tx=0x7ff36c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82068974ee792f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:22.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff383400640 1 -- 172.21.15.46:0/1661708014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0b0070 con 0x7ff3840ad4c0 2024-04-25T08:52:22.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff383400640 1 -- 172.21.15.46:0/1661708014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c004230 con 0x7ff3840ad4c0 2024-04-25T08:52:22.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff383400640 1 -- 172.21.15.46:0/1661708014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c003620 con 0x7ff3840ad4c0 2024-04-25T08:52:22.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 -- 172.21.15.46:0/1661708014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3840ad4c0 msgr2=0x7ff3840af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 --2- 172.21.15.46:0/1661708014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3840ad4c0 0x7ff3840af030 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7ff38406da10 tx=0x7ff36c01c330 comp rx=0 tx=0).stop 2024-04-25T08:52:22.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 -- 172.21.15.46:0/1661708014 shutdown_connections 2024-04-25T08:52:22.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 --2- 172.21.15.46:0/1661708014 >> v2:172.21.15.82:3300/0 conn(0x7ff38406f570 0x7ff38415c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 --2- 172.21.15.46:0/1661708014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3840ad4c0 0x7ff3840af030 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 -- 172.21.15.46:0/1661708014 >> 172.21.15.46:0/1661708014 conn(0x7ff3840cfd90 msgr2=0x7ff3840d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:22.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 -- 172.21.15.46:0/1661708014 shutdown_connections 2024-04-25T08:52:22.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 -- 172.21.15.46:0/1661708014 wait complete. 2024-04-25T08:52:22.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 Processor -- start 2024-04-25T08:52:22.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 -- start start 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.534+0000 7ff38ae00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38406f570 0x7ff38415d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff38ae00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3840ad4c0 0x7ff384158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff38ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38415b9a0 0x7ff3841463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff38ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38406b7b0 con 0x7ff38415b9a0 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff38ae00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38406b630 con 0x7ff38406f570 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff38ae00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38415d7e0 con 0x7ff3840ad4c0 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff388e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38415b9a0 0x7ff3841463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff388e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38415b9a0 0x7ff3841463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40820/0 (socket says 172.21.15.46:40820) 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff388e00640 1 -- 172.21.15.46:0/3199041099 learned_addr learned my addr 172.21.15.46:0/3199041099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff37b400640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3840ad4c0 0x7ff384158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff383e00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38406f570 0x7ff38415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff388e00640 1 -- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3840ad4c0 msgr2=0x7ff384158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff388e00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3840ad4c0 0x7ff384158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff388e00640 1 -- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38406f570 msgr2=0x7ff38415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff388e00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38406f570 0x7ff38415d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff388e00640 1 -- 172.21.15.46:0/3199041099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38415dac0 con 0x7ff38415b9a0 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff37b400640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3840ad4c0 0x7ff384158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff388e00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38415b9a0 0x7ff3841463e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00ce40 tx=0x7ff37c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff380c00640 1 -- 172.21.15.46:0/3199041099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c01a070 con 0x7ff38415b9a0 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff380c00640 1 -- 172.21.15.46:0/3199041099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c00d840 con 0x7ff38415b9a0 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38415dda0 con 0x7ff38415b9a0 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff380c00640 1 -- 172.21.15.46:0/3199041099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c00dc30 con 0x7ff38415b9a0 2024-04-25T08:52:22.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38415e360 con 0x7ff38415b9a0 2024-04-25T08:52:22.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3840cf850 con 0x7ff38415b9a0 2024-04-25T08:52:22.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.538+0000 7ff380c00640 1 -- 172.21.15.46:0/3199041099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff37c01d070 con 0x7ff38415b9a0 2024-04-25T08:52:22.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.542+0000 7ff380c00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff360066f90 0x7ff360069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.542+0000 7ff380c00640 1 -- 172.21.15.46:0/3199041099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff37c0857a0 con 0x7ff38415b9a0 2024-04-25T08:52:22.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.542+0000 7ff383e00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff360066f90 0x7ff360069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.542+0000 7ff383e00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff360066f90 0x7ff360069450 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff374000f70 tx=0x7ff374009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:22.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.546+0000 7ff380c00640 1 -- 172.21.15.46:0/3199041099 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3840cf850 con 0x7ff38415b9a0 2024-04-25T08:52:22.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.746+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff384069080 con 0x7ff38415b9a0 2024-04-25T08:52:22.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.746+0000 7ff380c00640 1 -- 172.21.15.46:0/3199041099 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3693 (secure 0 0 0) 0x7ff37c056830 con 0x7ff38415b9a0 2024-04-25T08:52:22.739 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:22.739 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7885,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3434993948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3434993948},{"type":"v1","addr":"172.21.15.82:6836","nonce":3434993948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7891,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1583051059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":1583051059},{"type":"v1","addr":"172.21.15.82:6837","nonce":1583051059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8328,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/918260204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":918260204},{"type":"v1","addr":"172.21.15.46:6833","nonce":918260204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":8817,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/893392755","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":893392755},{"type":"v1","addr":"172.21.15.46:6835","nonce":893392755}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[]} 2024-04-25T08:52:22.739 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 93 2024-04-25T08:52:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff360066f90 msgr2=0x7ff360069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff360066f90 0x7ff360069450 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff374000f70 tx=0x7ff374009040 comp rx=0 tx=0).stop 2024-04-25T08:52:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38415b9a0 msgr2=0x7ff3841463e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38415b9a0 0x7ff3841463e0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00ce40 tx=0x7ff37c00a710 comp rx=0 tx=0).stop 2024-04-25T08:52:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 shutdown_connections 2024-04-25T08:52:22.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff360066f90 0x7ff360069450 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff38415b9a0 0x7ff3841463e0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff3840ad4c0 0x7ff384158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 --2- 172.21.15.46:0/3199041099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff38406f570 0x7ff38415d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 >> 172.21.15.46:0/3199041099 conn(0x7ff3840cfd90 msgr2=0x7ff3840b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:22.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 shutdown_connections 2024-04-25T08:52:22.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.750+0000 7ff38ae00640 1 -- 172.21.15.46:0/3199041099 wait complete. 2024-04-25T08:52:22.760 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:52:22.760 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:52:22.760 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:52:22.760 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:52:22.761 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:52:22.761 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:52:22.761 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:52:22.761 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:52:22.761 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:52:22.761 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:52:22.761 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:52:22.761 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:52:22.762 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:52:22.762 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:52:22.762 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:52:22.762 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:52:22.774 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:52:22.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:52:22.775 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:52:22.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:52:22.777 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:52:22.777 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:52:22.778 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:52:22.778 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:52:22.818 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:52:22.818 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:22.819 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:52:22.820 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:22.822 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:52:22.823 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:22.824 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:52:22.824 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:22.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:52:22.904 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:52:22.905 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:52:22.912 INFO:tasks.ceph.mds.b.smithi082.stdout:starting mds.b at 2024-04-25T08:52:22.912 INFO:tasks.ceph.mds.d.smithi082.stdout:starting mds.d at 2024-04-25T08:52:22.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0be00640 1 Processor -- start 2024-04-25T08:52:22.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0be00640 1 -- start start 2024-04-25T08:52:22.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a0c15a710 0x7f9a0c0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a0c15aaf0 0x7f9a0c0b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0c0b1800 con 0x7f9a0c0c5d00 2024-04-25T08:52:22.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cd7c0 con 0x7f9a0c15aaf0 2024-04-25T08:52:22.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cdaa0 con 0x7f9a0c15a710 2024-04-25T08:52:22.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a0c15aaf0 0x7f9a0c0b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9a0c0c5d00 0x7f9a0c0ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39344/0 (socket says 172.21.15.46:39344) 2024-04-25T08:52:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0b400640 1 -- 172.21.15.46:0/807365092 learned_addr learned my addr 172.21.15.46:0/807365092 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0aa00640 1 --2- 172.21.15.46:0/807365092 >> v2:172.21.15.82:3300/0 conn(0x7f9a0c15a710 0x7f9a0c0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a03e00640 1 -- 172.21.15.46:0/807365092 >> v2:172.21.15.82:3300/0 conn(0x7f9a0c15a710 msgr2=0x7f9a0c0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a03e00640 1 --2- 172.21.15.46:0/807365092 >> v2:172.21.15.82:3300/0 conn(0x7f9a0c15a710 0x7f9a0c0c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a03e00640 1 -- 172.21.15.46:0/807365092 >> v1:172.21.15.46:6789/0 conn(0x7f9a0c0c5d00 legacy=0x7f9a0c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a03e00640 1 -- 172.21.15.46:0/807365092 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0cf6b0 con 0x7f9a0c15aaf0 2024-04-25T08:52:22.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.982+0000 7f9a0a000640 1 -- 172.21.15.46:0/807365092 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118668191 0 0) 0x7f9a0c0b1800 con 0x7f9a0c0c5d00 2024-04-25T08:52:22.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a03e00640 1 --2- 172.21.15.46:0/807365092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15aaf0 0x7f9a0c0b2aa0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0d1470 tx=0x7f99f401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e7e8dcad35f18f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:22.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0a000640 1 -- 172.21.15.46:0/807365092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f40b0070 con 0x7f9a0c15aaf0 2024-04-25T08:52:22.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0a000640 1 -- 172.21.15.46:0/807365092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f40ab600 con 0x7f9a0c15aaf0 2024-04-25T08:52:22.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0a000640 1 -- 172.21.15.46:0/807365092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f40ab9d0 con 0x7f9a0c15aaf0 2024-04-25T08:52:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0be00640 1 -- 172.21.15.46:0/807365092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15aaf0 msgr2=0x7f9a0c0b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0be00640 1 --2- 172.21.15.46:0/807365092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15aaf0 0x7f9a0c0b2aa0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0d1470 tx=0x7f99f401c3f0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0be00640 1 -- 172.21.15.46:0/807365092 shutdown_connections 2024-04-25T08:52:22.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0be00640 1 --2- 172.21.15.46:0/807365092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15aaf0 0x7f9a0c0b2aa0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0be00640 1 --2- 172.21.15.46:0/807365092 >> v2:172.21.15.82:3300/0 conn(0x7f9a0c15a710 0x7f9a0c0c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0be00640 1 -- 172.21.15.46:0/807365092 >> 172.21.15.46:0/807365092 conn(0x7f9a0c0b1f80 msgr2=0x7f9a0c0b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0be00640 1 -- 172.21.15.46:0/807365092 shutdown_connections 2024-04-25T08:52:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0be00640 1 -- 172.21.15.46:0/807365092 wait complete. 2024-04-25T08:52:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.986+0000 7f9a0be00640 1 Processor -- start 2024-04-25T08:52:22.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 -- start start 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0c0c5d00 0x7f9a0c153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15a710 0x7f9a0c0b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a0c15aaf0 0x7f9a0c157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cd480 con 0x7f9a0c15a710 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0cd300 con 0x7f9a0c0c5d00 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0c0c6110 con 0x7f9a0c15aaf0 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15a710 0x7f9a0c0b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15a710 0x7f9a0c0b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40854/0 (socket says 172.21.15.46:40854) 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03e00640 1 -- 172.21.15.46:0/4061338542 learned_addr learned my addr 172.21.15.46:0/4061338542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0b400640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a0c15aaf0 0x7f9a0c157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0aa00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0c0c5d00 0x7f9a0c153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03e00640 1 -- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a0c15aaf0 msgr2=0x7f9a0c157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03e00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a0c15aaf0 0x7f9a0c157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03e00640 1 -- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0c0c5d00 msgr2=0x7f9a0c153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03e00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0c0c5d00 0x7f9a0c153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03e00640 1 -- 172.21.15.46:0/4061338542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c0c7e80 con 0x7f9a0c15a710 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0b400640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a0c15aaf0 0x7f9a0c157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03e00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15a710 0x7f9a0c0b50b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f99f4019000 tx=0x7f99f4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03400640 1 -- 172.21.15.46:0/4061338542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f40ba070 con 0x7f9a0c15a710 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03400640 1 -- 172.21.15.46:0/4061338542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f40b5870 con 0x7f9a0c15a710 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a03400640 1 -- 172.21.15.46:0/4061338542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f40b0040 con 0x7f9a0c15a710 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0c0b3630 con 0x7f9a0c15a710 2024-04-25T08:52:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0c0b3bf0 con 0x7f9a0c15a710 2024-04-25T08:52:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.990+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0c069010 con 0x7f9a0c15a710 2024-04-25T08:52:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.994+0000 7f9a03400640 1 -- 172.21.15.46:0/4061338542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99f40bd070 con 0x7f9a0c15a710 2024-04-25T08:52:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.994+0000 7f9a03400640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99e8066f60 0x7f99e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.994+0000 7f9a03400640 1 -- 172.21.15.46:0/4061338542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f99f411f720 con 0x7f9a0c15a710 2024-04-25T08:52:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.994+0000 7f9a0aa00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99e8066f60 0x7f99e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.994+0000 7f9a0aa00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99e8066f60 0x7f99e8069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009af0 tx=0x7f99fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:22.998+0000 7f9a03400640 1 -- 172.21.15.46:0/4061338542 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99f4016070 con 0x7f9a0c15a710 2024-04-25T08:52:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:23.178+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9a0c0c6780 con 0x7f9a0c15a710 2024-04-25T08:52:24.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a03400640 1 -- 172.21.15.46:0/4061338542 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v117) v1 ==== 97+0+0 (secure 0 0 0) 0x7f99f40f6070 con 0x7f9a0c15a710 2024-04-25T08:52:24.092 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-04-25T08:52:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99e8066f60 msgr2=0x7f99e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99e8066f60 0x7f99e8069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009af0 tx=0x7f99fc008040 comp rx=0 tx=0).stop 2024-04-25T08:52:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15a710 msgr2=0x7f9a0c0b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15a710 0x7f9a0c0b50b0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f99f4019000 tx=0x7f99f4019190 comp rx=0 tx=0).stop 2024-04-25T08:52:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 shutdown_connections 2024-04-25T08:52:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f99e8066f60 0x7f99e8069420 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a0c15aaf0 0x7f9a0c157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a0c15a710 0x7f9a0c0b50b0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 --2- 172.21.15.46:0/4061338542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0c0c5d00 0x7f9a0c153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 >> 172.21.15.46:0/4061338542 conn(0x7f9a0c0b1f80 msgr2=0x7f9a0c1599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 shutdown_connections 2024-04-25T08:52:24.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.102+0000 7f9a0be00640 1 -- 172.21.15.46:0/4061338542 wait complete. 2024-04-25T08:52:24.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:52:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7f000640 1 Processor -- start 2024-04-25T08:52:24.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7f000640 1 -- start start 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7f000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e7815a750 0x7f8e7806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7f000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8e780cc9e0 0x7f8e7806f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7f000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7806d5e0 con 0x7f8e780aed10 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7f000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e7815ab30 con 0x7f8e780cc9e0 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7f000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e780ce4f0 con 0x7f8e7815a750 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8e780cc9e0 0x7f8e7806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e77400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8e780aed10 0x7f8e780b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39346/0 (socket says 172.21.15.46:39346) 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e77400640 1 -- 172.21.15.46:0/1139895753 learned_addr learned my addr 172.21.15.46:0/1139895753 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e77e00640 1 --2- 172.21.15.46:0/1139895753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e7815a750 0x7f8e7806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e76a00640 1 -- 172.21.15.46:0/1139895753 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496229704 0 0) 0x7f8e7806d5e0 con 0x7f8e780aed10 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e76a00640 1 -- 172.21.15.46:0/1139895753 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e5c003530 con 0x7f8e780aed10 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7d000640 1 -- 172.21.15.46:0/1139895753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e7815a750 msgr2=0x7f8e7806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7d000640 1 --2- 172.21.15.46:0/1139895753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e7815a750 0x7f8e7806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7d000640 1 -- 172.21.15.46:0/1139895753 >> v1:172.21.15.46:6789/0 conn(0x7f8e780aed10 legacy=0x7f8e780b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.274+0000 7f8e7d000640 1 -- 172.21.15.46:0/1139895753 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e780ce720 con 0x7f8e780cc9e0 2024-04-25T08:52:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7d000640 1 --2- 172.21.15.46:0/1139895753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780cc9e0 0x7f8e7806f150 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009d90 tx=0x7f8e7001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2816a2e6abe7b68c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e76a00640 1 -- 172.21.15.46:0/1139895753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700b2070 con 0x7f8e780cc9e0 2024-04-25T08:52:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e76a00640 1 -- 172.21.15.46:0/1139895753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e700a5920 con 0x7f8e780cc9e0 2024-04-25T08:52:24.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e76a00640 1 -- 172.21.15.46:0/1139895753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e700a5cd0 con 0x7f8e780cc9e0 2024-04-25T08:52:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 -- 172.21.15.46:0/1139895753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780cc9e0 msgr2=0x7f8e7806f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 --2- 172.21.15.46:0/1139895753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780cc9e0 0x7f8e7806f150 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009d90 tx=0x7f8e7001c970 comp rx=0 tx=0).stop 2024-04-25T08:52:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 -- 172.21.15.46:0/1139895753 shutdown_connections 2024-04-25T08:52:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 --2- 172.21.15.46:0/1139895753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780cc9e0 0x7f8e7806f150 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 --2- 172.21.15.46:0/1139895753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e7815a750 0x7f8e7806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 -- 172.21.15.46:0/1139895753 >> 172.21.15.46:0/1139895753 conn(0x7f8e780b12a0 msgr2=0x7f8e780b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 -- 172.21.15.46:0/1139895753 shutdown_connections 2024-04-25T08:52:24.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 -- 172.21.15.46:0/1139895753 wait complete. 2024-04-25T08:52:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 Processor -- start 2024-04-25T08:52:24.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.278+0000 7f8e7f000640 1 -- start start 2024-04-25T08:52:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780aed10 0x7f8e781f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7f000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e780cc9e0 0x7f8e78156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7f000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7815a750 0x7f8e7814dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e78165220 con 0x7f8e780aed10 2024-04-25T08:52:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7f000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7806d270 con 0x7f8e7815a750 2024-04-25T08:52:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7f000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e781653a0 con 0x7f8e780cc9e0 2024-04-25T08:52:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780aed10 0x7f8e781f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780aed10 0x7f8e781f74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40880/0 (socket says 172.21.15.46:40880) 2024-04-25T08:52:24.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77e00640 1 -- 172.21.15.46:0/1147408556 learned_addr learned my addr 172.21.15.46:0/1147408556 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77400640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e780cc9e0 0x7f8e78156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7d000640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7815a750 0x7f8e7814dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77e00640 1 -- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e780cc9e0 msgr2=0x7f8e78156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77e00640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e780cc9e0 0x7f8e78156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77e00640 1 -- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7815a750 msgr2=0x7f8e7814dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77e00640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7815a750 0x7f8e7814dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77e00640 1 -- 172.21.15.46:0/1147408556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78156c70 con 0x7f8e780aed10 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77400640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e780cc9e0 0x7f8e78156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7d000640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7815a750 0x7f8e7814dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e77e00640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780aed10 0x7f8e781f74c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800d5f0 tx=0x7f8e6800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e6fe00640 1 -- 172.21.15.46:0/1147408556 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e680125f0 con 0x7f8e780aed10 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e6fe00640 1 -- 172.21.15.46:0/1147408556 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e68019070 con 0x7f8e780aed10 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e6fe00640 1 -- 172.21.15.46:0/1147408556 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e680129b0 con 0x7f8e780aed10 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7815ae40 con 0x7f8e780aed10 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7815b400 con 0x7f8e780aed10 2024-04-25T08:52:24.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.282+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e78069080 con 0x7f8e780aed10 2024-04-25T08:52:24.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.286+0000 7f8e6fe00640 1 -- 172.21.15.46:0/1147408556 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e68012b50 con 0x7f8e780aed10 2024-04-25T08:52:24.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.286+0000 7f8e6fe00640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e58066f60 0x7f8e58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.286+0000 7f8e6fe00640 1 -- 172.21.15.46:0/1147408556 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8e68085220 con 0x7f8e780aed10 2024-04-25T08:52:24.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.286+0000 7f8e77400640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e58066f60 0x7f8e58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.286+0000 7f8e77400640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e58066f60 0x7f8e58069420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8e78157d00 tx=0x7f8e64009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.290+0000 7f8e6fe00640 1 -- 172.21.15.46:0/1147408556 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8e68056480 con 0x7f8e780aed10 2024-04-25T08:52:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.506+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8e7814b490 con 0x7f8e780aed10 2024-04-25T08:52:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e6fe00640 1 -- 172.21.15.46:0/1147408556 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f8e68055d80 con 0x7f8e780aed10 2024-04-25T08:52:24.500 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:24.500 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:52:24.500 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-04-25T08:52:24.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.500 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-04-25T08:52:24.500 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmq+gDGRAAtB5ObM/zelMmCZ4cJPghdQ==", 2024-04-25T08:52:24.500 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.500 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:52:24.500 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmyIaAJRAA9zodtgftP7faNth4UPyCTA==", 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:52:24.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmFahWIBAAORHmzDLXPjuANF+/W2wKpQ==", 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.502 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmRhrPLBAATXOHl1jGJYmIlnYk/JPcOw==", 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA0FypmaHiqEhAAYzasCl8t+pls5IHe9tCP4g==", 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:52:24.503 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA3FypmOGTuGhAA37F+qRI/+pj5dkLc3TBIUQ==", 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-04-25T08:52:24.504 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA6Fypm/YbNJRAAP1QyYDLY/xFvRz63R+H/Pw==", 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQA9FypmiT8ALhAAX/2CJJeJ+2SGNhwah8NTBQ==", 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBDFypmyG88JBAAq1065gzLIIR/6fYaw0bBTg==", 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBGFypmNir3KxAAszCXJJ1KFtCusDZhBHAZSQ==", 2024-04-25T08:52:24.506 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBJFypm4r7LNxAA42A07ntI6OOgblk+4i1Yiw==", 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.507 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQBNFypmG1fOBhAAOKRgw04jIo8H3u2hWIZe2g==", 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q==", 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:52:24.508 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmxPyHOhAADBuigZlq3GHURsJkB0nnyA==", 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypm2PP0NhAAixV/kkY7eFPXUW/nJDOCOA==", 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAzFypmaVDtBBAAKnWk9CxZzg2r6T/IycuFEg==", 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.510 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAxFypm2W2MOBAAN7TDQiZXFV6+pWMTar95sg==", 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.511 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmzhptFBAAlFQZQn1GSseEguy5zfw2CA==", 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAyFypmAY6CEBAA2qHNALNTRKmCJRZatDqAvw==", 2024-04-25T08:52:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-04-25T08:52:24.513 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-04-25T08:52:24.513 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-04-25T08:52:24.513 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-04-25T08:52:24.513 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.513 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:24.513 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-04-25T08:52:24.513 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:52:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e58066f60 msgr2=0x7f8e58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e58066f60 0x7f8e58069420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8e78157d00 tx=0x7f8e64009040 comp rx=0 tx=0).stop 2024-04-25T08:52:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780aed10 msgr2=0x7f8e781f74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780aed10 0x7f8e781f74c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800d5f0 tx=0x7f8e6800dab0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 shutdown_connections 2024-04-25T08:52:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8e58066f60 0x7f8e58069420 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e7815a750 0x7f8e7814dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8e780cc9e0 0x7f8e78156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 --2- 172.21.15.46:0/1147408556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e780aed10 0x7f8e781f74c0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 >> 172.21.15.46:0/1147408556 conn(0x7f8e780b12a0 msgr2=0x7f8e7806db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 shutdown_connections 2024-04-25T08:52:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.510+0000 7f8e7f000640 1 -- 172.21.15.46:0/1147408556 wait complete. 2024-04-25T08:52:24.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb031200640 1 Processor -- start 2024-04-25T08:52:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb031200640 1 -- start start 2024-04-25T08:52:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb031200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb02c15a750 0x7fb02c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb031200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb02c0aed10 0x7fb02c06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb031200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02c06d5e0 con 0x7fb02c0cc9e0 2024-04-25T08:52:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb031200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02c15ab30 con 0x7fb02c0aed10 2024-04-25T08:52:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb031200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02c0ce4f0 con 0x7fb02c15a750 2024-04-25T08:52:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb021400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb02c0aed10 0x7fb02c06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb021400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb02c0aed10 0x7fb02c06f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40886/0 (socket says 172.21.15.46:40886) 2024-04-25T08:52:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb02a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb02c0cc9e0 0x7fb02c0b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39352/0 (socket says 172.21.15.46:39352) 2024-04-25T08:52:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb021400640 1 -- 172.21.15.46:0/3219047978 learned_addr learned my addr 172.21.15.46:0/3219047978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb029e00640 1 --2- 172.21.15.46:0/3219047978 >> v2:172.21.15.82:3300/0 conn(0x7fb02c15a750 0x7fb02c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb021400640 1 -- 172.21.15.46:0/3219047978 >> v2:172.21.15.82:3300/0 conn(0x7fb02c15a750 msgr2=0x7fb02c06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb021400640 1 --2- 172.21.15.46:0/3219047978 >> v2:172.21.15.82:3300/0 conn(0x7fb02c15a750 0x7fb02c06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb021400640 1 -- 172.21.15.46:0/3219047978 >> v1:172.21.15.46:6789/0 conn(0x7fb02c0cc9e0 legacy=0x7fb02c0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb021400640 1 -- 172.21.15.46:0/3219047978 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0ce720 con 0x7fb02c0aed10 2024-04-25T08:52:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb029400640 1 -- 172.21.15.46:0/3219047978 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159263177 0 0) 0x7fb02c06d5e0 con 0x7fb02c0cc9e0 2024-04-25T08:52:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.670+0000 7fb029e00640 1 --2- 172.21.15.46:0/3219047978 >> v2:172.21.15.82:3300/0 conn(0x7fb02c15a750 0x7fb02c06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:24.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb021400640 1 --2- 172.21.15.46:0/3219047978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c0aed10 0x7fb02c06f150 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fb02c04ffd0 tx=0x7fb01401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c7ed320e2c76ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb029400640 1 -- 172.21.15.46:0/3219047978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140b4070 con 0x7fb02c0aed10 2024-04-25T08:52:24.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb029400640 1 -- 172.21.15.46:0/3219047978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0140a7bb0 con 0x7fb02c0aed10 2024-04-25T08:52:24.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb029400640 1 -- 172.21.15.46:0/3219047978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140af440 con 0x7fb02c0aed10 2024-04-25T08:52:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb031200640 1 -- 172.21.15.46:0/3219047978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c0aed10 msgr2=0x7fb02c06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb031200640 1 --2- 172.21.15.46:0/3219047978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c0aed10 0x7fb02c06f150 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fb02c04ffd0 tx=0x7fb01401ebf0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb031200640 1 -- 172.21.15.46:0/3219047978 shutdown_connections 2024-04-25T08:52:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb031200640 1 --2- 172.21.15.46:0/3219047978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c0aed10 0x7fb02c06f150 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb031200640 1 --2- 172.21.15.46:0/3219047978 >> v2:172.21.15.82:3300/0 conn(0x7fb02c15a750 0x7fb02c06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb031200640 1 -- 172.21.15.46:0/3219047978 >> 172.21.15.46:0/3219047978 conn(0x7fb02c0b12a0 msgr2=0x7fb02c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb031200640 1 -- 172.21.15.46:0/3219047978 shutdown_connections 2024-04-25T08:52:24.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.674+0000 7fb031200640 1 -- 172.21.15.46:0/3219047978 wait complete. 2024-04-25T08:52:24.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 Processor -- start 2024-04-25T08:52:24.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 -- start start 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02c0aed10 0x7fb02c142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb02c0cc9e0 0x7fb02c14e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c15a750 0x7fb02c14ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c06db00 con 0x7fb02c15a750 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c06d270 con 0x7fb02c0aed10 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c06dc80 con 0x7fb02c0cc9e0 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb02a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c15a750 0x7fb02c14ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb02a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c15a750 0x7fb02c14ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40890/0 (socket says 172.21.15.46:40890) 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb02a800640 1 -- 172.21.15.46:0/2146379252 learned_addr learned my addr 172.21.15.46:0/2146379252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb029e00640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02c0aed10 0x7fb02c142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb021400640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb02c0cc9e0 0x7fb02c14e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb02a800640 1 -- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb02c0cc9e0 msgr2=0x7fb02c14e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb02a800640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb02c0cc9e0 0x7fb02c14e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb02a800640 1 -- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02c0aed10 msgr2=0x7fb02c142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb02a800640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02c0aed10 0x7fb02c142f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb02a800640 1 -- 172.21.15.46:0/2146379252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c15cf20 con 0x7fb02c15a750 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb029e00640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02c0aed10 0x7fb02c142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb02a800640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c15a750 0x7fb02c14ee40 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fb02400c7d0 tx=0x7fb02400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb022a00640 1 -- 172.21.15.46:0/2146379252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024018070 con 0x7fb02c15a750 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb022a00640 1 -- 172.21.15.46:0/2146379252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02400a640 con 0x7fb02c15a750 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb022a00640 1 -- 172.21.15.46:0/2146379252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02400aa70 con 0x7fb02c15a750 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c15d200 con 0x7fb02c15a750 2024-04-25T08:52:24.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.678+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c15d7c0 con 0x7fb02c15a750 2024-04-25T08:52:24.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.682+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c069080 con 0x7fb02c15a750 2024-04-25T08:52:24.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.682+0000 7fb022a00640 1 -- 172.21.15.46:0/2146379252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb02401b070 con 0x7fb02c15a750 2024-04-25T08:52:24.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.682+0000 7fb022a00640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb008066f60 0x7fb008069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.682+0000 7fb022a00640 1 -- 172.21.15.46:0/2146379252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb0240843f0 con 0x7fb02c15a750 2024-04-25T08:52:24.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.682+0000 7fb029e00640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb008066f60 0x7fb008069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.682+0000 7fb029e00640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb008066f60 0x7fb008069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb02c14f640 tx=0x7fb01c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.686+0000 7fb022a00640 1 -- 172.21.15.46:0/2146379252 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb024054f60 con 0x7fb02c15a750 2024-04-25T08:52:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.850+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb02c143510 con 0x7fb02c15a750 2024-04-25T08:52:24.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb022a00640 1 -- 172.21.15.46:0/2146379252 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+13126 (secure 0 0 0) 0x7fb024059000 con 0x7fb02c15a750 2024-04-25T08:52:24.844 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:24.844 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":117,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:24.101296+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:24.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb008066f60 msgr2=0x7fb008069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb008066f60 0x7fb008069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb02c14f640 tx=0x7fb01c008040 comp rx=0 tx=0).stop 2024-04-25T08:52:24.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c15a750 msgr2=0x7fb02c14ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c15a750 0x7fb02c14ee40 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fb02400c7d0 tx=0x7fb02400cc90 comp rx=0 tx=0).stop 2024-04-25T08:52:24.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 shutdown_connections 2024-04-25T08:52:24.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb008066f60 0x7fb008069420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c15a750 0x7fb02c14ee40 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb02c0cc9e0 0x7fb02c14e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 --2- 172.21.15.46:0/2146379252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb02c0aed10 0x7fb02c142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 >> 172.21.15.46:0/2146379252 conn(0x7fb02c0b12a0 msgr2=0x7fb02c1505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 shutdown_connections 2024-04-25T08:52:24.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:24.854+0000 7fb031200640 1 -- 172.21.15.46:0/2146379252 wait complete. 2024-04-25T08:52:24.864 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:52:24.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:25.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e46400640 1 Processor -- start 2024-04-25T08:52:25.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e46400640 1 -- start start 2024-04-25T08:52:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e46400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e4006ed90 0x7f3e40147070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e46400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3e401475b0 0x7f3e40071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e46400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4015e460 con 0x7f3e4015df60 2024-04-25T08:52:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e46400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4015e740 con 0x7f3e4006ed90 2024-04-25T08:52:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e46400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e4015ea20 con 0x7f3e401475b0 2024-04-25T08:52:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e3f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e4006ed90 0x7f3e40147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e3f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e4006ed90 0x7f3e40147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40896/0 (socket says 172.21.15.46:40896) 2024-04-25T08:52:25.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e3f400640 1 -- 172.21.15.46:0/2048897940 learned_addr learned my addr 172.21.15.46:0/2048897940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e36a00640 1 --2- 172.21.15.46:0/2048897940 >> v2:172.21.15.82:3300/0 conn(0x7f3e401475b0 0x7f3e40071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e3f400640 1 -- 172.21.15.46:0/2048897940 >> v2:172.21.15.82:3300/0 conn(0x7f3e401475b0 msgr2=0x7f3e40071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e3f400640 1 --2- 172.21.15.46:0/2048897940 >> v2:172.21.15.82:3300/0 conn(0x7f3e401475b0 0x7f3e40071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e3f400640 1 -- 172.21.15.46:0/2048897940 >> v1:172.21.15.82:6789/0 conn(0x7f3e4015df60 legacy=0x7f3e40161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e3f400640 1 -- 172.21.15.46:0/2048897940 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4015ed00 con 0x7f3e4006ed90 2024-04-25T08:52:25.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.018+0000 7f3e36a00640 1 --2- 172.21.15.46:0/2048897940 >> v2:172.21.15.82:3300/0 conn(0x7f3e401475b0 0x7f3e40071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:25.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e3f400640 1 --2- 172.21.15.46:0/2048897940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 0x7f3e40147070 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f3e40068750 tx=0x7f3e3001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a823b994e3f4be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e3ea00640 1 -- 172.21.15.46:0/2048897940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300a7070 con 0x7f3e4006ed90 2024-04-25T08:52:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e3ea00640 1 -- 172.21.15.46:0/2048897940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e300b0070 con 0x7f3e4006ed90 2024-04-25T08:52:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e3ea00640 1 -- 172.21.15.46:0/2048897940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300ab890 con 0x7f3e4006ed90 2024-04-25T08:52:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e46400640 1 -- 172.21.15.46:0/2048897940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 msgr2=0x7f3e40147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e46400640 1 --2- 172.21.15.46:0/2048897940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 0x7f3e40147070 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f3e40068750 tx=0x7f3e3001c450 comp rx=0 tx=0).stop 2024-04-25T08:52:25.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e46400640 1 -- 172.21.15.46:0/2048897940 shutdown_connections 2024-04-25T08:52:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e46400640 1 --2- 172.21.15.46:0/2048897940 >> v2:172.21.15.82:3300/0 conn(0x7f3e401475b0 0x7f3e40071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e46400640 1 --2- 172.21.15.46:0/2048897940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 0x7f3e40147070 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e46400640 1 -- 172.21.15.46:0/2048897940 >> 172.21.15.46:0/2048897940 conn(0x7f3e400b1670 msgr2=0x7f3e400b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e46400640 1 -- 172.21.15.46:0/2048897940 shutdown_connections 2024-04-25T08:52:25.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.022+0000 7f3e46400640 1 -- 172.21.15.46:0/2048897940 wait complete. 2024-04-25T08:52:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 Processor -- start 2024-04-25T08:52:25.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 -- start start 2024-04-25T08:52:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 0x7f3e401f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e4015df60 0x7f3e401381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e401f75c0 0x7f3e40138730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4016ac80 con 0x7f3e4006ed90 2024-04-25T08:52:25.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4016ab00 con 0x7f3e401f75c0 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4016ae00 con 0x7f3e4015df60 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 0x7f3e401f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 0x7f3e401f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40906/0 (socket says 172.21.15.46:40906) 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3f400640 1 -- 172.21.15.46:0/736231564 learned_addr learned my addr 172.21.15.46:0/736231564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3fe00640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e401f75c0 0x7f3e40138730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e36a00640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e4015df60 0x7f3e401381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3f400640 1 -- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e4015df60 msgr2=0x7f3e401381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3f400640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e4015df60 0x7f3e401381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3f400640 1 -- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e401f75c0 msgr2=0x7f3e40138730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3f400640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e401f75c0 0x7f3e40138730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3f400640 1 -- 172.21.15.46:0/736231564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40137150 con 0x7f3e4006ed90 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3fe00640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e401f75c0 0x7f3e40138730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e3f400640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 0x7f3e401f7080 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f3e300ad040 tx=0x7f3e3001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:25.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e37e00640 1 -- 172.21.15.46:0/736231564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300b4800 con 0x7f3e4006ed90 2024-04-25T08:52:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e37e00640 1 -- 172.21.15.46:0/736231564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e300b49c0 con 0x7f3e4006ed90 2024-04-25T08:52:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e37e00640 1 -- 172.21.15.46:0/736231564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300b4db0 con 0x7f3e4006ed90 2024-04-25T08:52:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e40137430 con 0x7f3e4006ed90 2024-04-25T08:52:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e401379f0 con 0x7f3e4006ed90 2024-04-25T08:52:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.026+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e40069010 con 0x7f3e4006ed90 2024-04-25T08:52:25.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.030+0000 7f3e37e00640 1 -- 172.21.15.46:0/736231564 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e300b3380 con 0x7f3e4006ed90 2024-04-25T08:52:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.030+0000 7f3e37e00640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e1c066f60 0x7f3e1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.030+0000 7f3e37e00640 1 -- 172.21.15.46:0/736231564 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3e3011f470 con 0x7f3e4006ed90 2024-04-25T08:52:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.030+0000 7f3e36a00640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e1c066f60 0x7f3e1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.030+0000 7f3e36a00640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e1c066f60 0x7f3e1c069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3e40158ab0 tx=0x7f3e28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:25.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.034+0000 7f3e37e00640 1 -- 172.21.15.46:0/736231564 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e300160d0 con 0x7f3e4006ed90 2024-04-25T08:52:25.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e40072710 con 0x7f3e4006ed90 2024-04-25T08:52:25.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e37e00640 1 -- 172.21.15.46:0/736231564 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e300f0110 con 0x7f3e4006ed90 2024-04-25T08:52:25.192 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:52:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e1c066f60 msgr2=0x7f3e1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e1c066f60 0x7f3e1c069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3e40158ab0 tx=0x7f3e28008040 comp rx=0 tx=0).stop 2024-04-25T08:52:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 msgr2=0x7f3e401f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 0x7f3e401f7080 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f3e300ad040 tx=0x7f3e3001ac90 comp rx=0 tx=0).stop 2024-04-25T08:52:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 shutdown_connections 2024-04-25T08:52:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3e1c066f60 0x7f3e1c069420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e401f75c0 0x7f3e40138730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3e4015df60 0x7f3e401381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 --2- 172.21.15.46:0/736231564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e4006ed90 0x7f3e401f7080 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 >> 172.21.15.46:0/736231564 conn(0x7f3e400b1670 msgr2=0x7f3e400b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 shutdown_connections 2024-04-25T08:52:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.202+0000 7f3e46400640 1 -- 172.21.15.46:0/736231564 wait complete. 2024-04-25T08:52:25.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:52:25.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd19be00640 1 Processor -- start 2024-04-25T08:52:25.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd19be00640 1 -- start start 2024-04-25T08:52:25.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd19be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 0x7fd1940b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd19be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd1940b0c30 0x7fd194070c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd19be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1940b5250 con 0x7fd1940711d0 2024-04-25T08:52:25.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd19be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1940b5af0 con 0x7fd1940b0c30 2024-04-25T08:52:25.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd19be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940b5530 con 0x7fd19415dbd0 2024-04-25T08:52:25.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd199e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd1940711d0 0x7fd19415dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39356/0 (socket says 172.21.15.46:39356) 2024-04-25T08:52:25.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd199e00640 1 -- 172.21.15.46:0/346408338 learned_addr learned my addr 172.21.15.46:0/346408338 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:25.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd199400640 1 --2- 172.21.15.46:0/346408338 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 0x7fd1940b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.362+0000 7fd198a00640 1 --2- 172.21.15.46:0/346408338 >> v2:172.21.15.82:3300/0 conn(0x7fd1940b0c30 0x7fd194070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd193e00640 1 -- 172.21.15.46:0/346408338 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474737020 0 0) 0x7fd1940b5250 con 0x7fd1940711d0 2024-04-25T08:52:25.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd193e00640 1 -- 172.21.15.46:0/346408338 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd17c003530 con 0x7fd1940711d0 2024-04-25T08:52:25.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd199400640 1 -- 172.21.15.46:0/346408338 >> v2:172.21.15.82:3300/0 conn(0x7fd1940b0c30 msgr2=0x7fd194070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:25.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd199400640 1 --2- 172.21.15.46:0/346408338 >> v2:172.21.15.82:3300/0 conn(0x7fd1940b0c30 0x7fd194070c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd199400640 1 -- 172.21.15.46:0/346408338 >> v1:172.21.15.46:6789/0 conn(0x7fd1940711d0 legacy=0x7fd19415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd199400640 1 -- 172.21.15.46:0/346408338 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940b5810 con 0x7fd19415dbd0 2024-04-25T08:52:25.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd199400640 1 --2- 172.21.15.46:0/346408338 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 0x7fd1940b06f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd1880097c0 tx=0x7fd18801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d877c6dd5d66430f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:25.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd193e00640 1 -- 172.21.15.46:0/346408338 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880b0070 con 0x7fd19415dbd0 2024-04-25T08:52:25.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd193e00640 1 -- 172.21.15.46:0/346408338 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1880ab600 con 0x7fd19415dbd0 2024-04-25T08:52:25.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd193e00640 1 -- 172.21.15.46:0/346408338 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880ab9d0 con 0x7fd19415dbd0 2024-04-25T08:52:25.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd19be00640 1 -- 172.21.15.46:0/346408338 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 msgr2=0x7fd1940b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.366+0000 7fd19be00640 1 --2- 172.21.15.46:0/346408338 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 0x7fd1940b06f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd1880097c0 tx=0x7fd18801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 -- 172.21.15.46:0/346408338 shutdown_connections 2024-04-25T08:52:25.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 --2- 172.21.15.46:0/346408338 >> v2:172.21.15.82:3300/0 conn(0x7fd1940b0c30 0x7fd194070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 --2- 172.21.15.46:0/346408338 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 0x7fd1940b06f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 -- 172.21.15.46:0/346408338 >> 172.21.15.46:0/346408338 conn(0x7fd19406f4f0 msgr2=0x7fd19406f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:25.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 -- 172.21.15.46:0/346408338 shutdown_connections 2024-04-25T08:52:25.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 -- 172.21.15.46:0/346408338 wait complete. 2024-04-25T08:52:25.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 Processor -- start 2024-04-25T08:52:25.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 -- start start 2024-04-25T08:52:25.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1940711d0 0x7fd1940cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1940b0c30 0x7fd1941521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 0x7fd19413a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940715e0 con 0x7fd1940711d0 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940b1010 con 0x7fd1940b0c30 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd19be00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19416a850 con 0x7fd19415dbd0 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1940711d0 0x7fd1940cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1940711d0 0x7fd1940cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40918/0 (socket says 172.21.15.46:40918) 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199400640 1 -- 172.21.15.46:0/573510123 learned_addr learned my addr 172.21.15.46:0/573510123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199e00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 0x7fd19413a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd198a00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1940b0c30 0x7fd1941521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199400640 1 -- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 msgr2=0x7fd19413a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199400640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 0x7fd19413a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199400640 1 -- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1940b0c30 msgr2=0x7fd1941521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199400640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1940b0c30 0x7fd1941521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199400640 1 -- 172.21.15.46:0/573510123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd194155510 con 0x7fd1940711d0 2024-04-25T08:52:25.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd199400640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1940711d0 0x7fd1940cd200 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fd1880ae040 tx=0x7fd18801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd191600640 1 -- 172.21.15.46:0/573510123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880b0050 con 0x7fd1940711d0 2024-04-25T08:52:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.370+0000 7fd191600640 1 -- 172.21.15.46:0/573510123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1880bc070 con 0x7fd1940711d0 2024-04-25T08:52:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.374+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19414e580 con 0x7fd1940711d0 2024-04-25T08:52:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.374+0000 7fd191600640 1 -- 172.21.15.46:0/573510123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880b5a50 con 0x7fd1940711d0 2024-04-25T08:52:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.374+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd194154d10 con 0x7fd1940711d0 2024-04-25T08:52:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.374+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd194069080 con 0x7fd1940711d0 2024-04-25T08:52:25.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.374+0000 7fd191600640 1 -- 172.21.15.46:0/573510123 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd1880ab4c0 con 0x7fd1940711d0 2024-04-25T08:52:25.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.374+0000 7fd191600640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd174066f60 0x7fd174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.374+0000 7fd191600640 1 -- 172.21.15.46:0/573510123 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd18811f750 con 0x7fd1940711d0 2024-04-25T08:52:25.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.374+0000 7fd198a00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd174066f60 0x7fd174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.374+0000 7fd198a00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd174066f60 0x7fd174069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd19414ea10 tx=0x7fd180008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:25.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.378+0000 7fd191600640 1 -- 172.21.15.46:0/573510123 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd188016090 con 0x7fd1940711d0 2024-04-25T08:52:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.542+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd1941547e0 con 0x7fd1940711d0 2024-04-25T08:52:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.542+0000 7fd191600640 1 -- 172.21.15.46:0/573510123 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd1880f0920 con 0x7fd1940711d0 2024-04-25T08:52:25.534 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:25.534 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-04-25T08:52:25.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-04-25T08:52:25.534 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-04-25T08:52:25.534 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-04-25T08:52:25.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-04-25T08:52:25.534 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd174066f60 msgr2=0x7fd174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd174066f60 0x7fd174069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd19414ea10 tx=0x7fd180008040 comp rx=0 tx=0).stop 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1940711d0 msgr2=0x7fd1940cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1940711d0 0x7fd1940cd200 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fd1880ae040 tx=0x7fd18801ace0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 shutdown_connections 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd174066f60 0x7fd174069420 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd19415dbd0 0x7fd19413a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1940b0c30 0x7fd1941521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 --2- 172.21.15.46:0/573510123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1940711d0 0x7fd1940cd200 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 >> 172.21.15.46:0/573510123 conn(0x7fd19406f4f0 msgr2=0x7fd1940b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 shutdown_connections 2024-04-25T08:52:25.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.546+0000 7fd19be00640 1 -- 172.21.15.46:0/573510123 wait complete. 2024-04-25T08:52:25.555 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:52:25.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:52:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f8600640 1 Processor -- start 2024-04-25T08:52:25.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f8600640 1 -- start start 2024-04-25T08:52:25.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f00cfcb0 0x7f30f00d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f8600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f009af70 con 0x7f30f00cf4b0 2024-04-25T08:52:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f8600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30f0099aa0 con 0x7f30f015bb90 2024-04-25T08:52:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f0099d80 con 0x7f30f00cfcb0 2024-04-25T08:52:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f5200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f30f00cf4b0 0x7f30f009a860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39362/0 (socket says 172.21.15.46:39362) 2024-04-25T08:52:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f5200640 1 -- 172.21.15.46:0/2505590364 learned_addr learned my addr 172.21.15.46:0/2505590364 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:25.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f6600640 1 --2- 172.21.15.46:0/2505590364 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f00cfcb0 0x7f30f00d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30efe00640 1 -- 172.21.15.46:0/2505590364 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641105750 0 0) 0x7f30f009af70 con 0x7f30f00cf4b0 2024-04-25T08:52:25.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30efe00640 1 -- 172.21.15.46:0/2505590364 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30d8003530 con 0x7f30f00cf4b0 2024-04-25T08:52:25.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f6600640 1 -- 172.21.15.46:0/2505590364 >> v1:172.21.15.82:6789/0 conn(0x7f30f015bb90 legacy=0x7f30f00cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f6600640 1 -- 172.21.15.46:0/2505590364 >> v1:172.21.15.46:6789/0 conn(0x7f30f00cf4b0 legacy=0x7f30f009a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30efe00640 1 -- 172.21.15.46:0/2505590364 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988605459 0 0) 0x7f30f0099aa0 con 0x7f30f015bb90 2024-04-25T08:52:25.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.710+0000 7f30f6600640 1 -- 172.21.15.46:0/2505590364 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f009a060 con 0x7f30f00cfcb0 2024-04-25T08:52:25.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30f6600640 1 --2- 172.21.15.46:0/2505590364 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f00cfcb0 0x7f30f00d0140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009dc0 tx=0x7f30e801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48ef30377ad4fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30efe00640 1 -- 172.21.15.46:0/2505590364 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e80b2070 con 0x7f30f00cfcb0 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30efe00640 1 -- 172.21.15.46:0/2505590364 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e80a58d0 con 0x7f30f00cfcb0 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30efe00640 1 -- 172.21.15.46:0/2505590364 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e80a5ca0 con 0x7f30f00cfcb0 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30f8600640 1 -- 172.21.15.46:0/2505590364 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f00cfcb0 msgr2=0x7f30f00d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30f8600640 1 --2- 172.21.15.46:0/2505590364 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f00cfcb0 0x7f30f00d0140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009dc0 tx=0x7f30e801c900 comp rx=0 tx=0).stop 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30f8600640 1 -- 172.21.15.46:0/2505590364 shutdown_connections 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30f8600640 1 --2- 172.21.15.46:0/2505590364 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f00cfcb0 0x7f30f00d0140 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30f8600640 1 -- 172.21.15.46:0/2505590364 >> 172.21.15.46:0/2505590364 conn(0x7f30f00b31c0 msgr2=0x7f30f00b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30f8600640 1 -- 172.21.15.46:0/2505590364 shutdown_connections 2024-04-25T08:52:25.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.714+0000 7f30f8600640 1 -- 172.21.15.46:0/2505590364 wait complete. 2024-04-25T08:52:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 Processor -- start 2024-04-25T08:52:25.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 -- start start 2024-04-25T08:52:25.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f00cf4b0 0x7f30f00ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f00cfcb0 0x7f30f015cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f015bb90 0x7f30f015d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f0169c90 con 0x7f30f00cfcb0 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f0169b10 con 0x7f30f00cf4b0 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30f0169e10 con 0x7f30f015bb90 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f00cfcb0 0x7f30f015cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f00cfcb0 0x7f30f015cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40926/0 (socket says 172.21.15.46:40926) 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5200640 1 -- 172.21.15.46:0/3651830467 learned_addr learned my addr 172.21.15.46:0/3651830467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5c00640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f00cf4b0 0x7f30f00ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f6600640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f015bb90 0x7f30f015d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5200640 1 -- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f015bb90 msgr2=0x7f30f015d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5200640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f015bb90 0x7f30f015d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5200640 1 -- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f00cf4b0 msgr2=0x7f30f00ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5200640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f00cf4b0 0x7f30f00ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5200640 1 -- 172.21.15.46:0/3651830467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f0167720 con 0x7f30f00cfcb0 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f5200640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f00cfcb0 0x7f30f015cac0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c3c0 tx=0x7f30dc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:25.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30ed600640 1 -- 172.21.15.46:0/3651830467 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc017070 con 0x7f30f00cfcb0 2024-04-25T08:52:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30ed600640 1 -- 172.21.15.46:0/3651830467 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc005cf0 con 0x7f30f00cfcb0 2024-04-25T08:52:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30ed600640 1 -- 172.21.15.46:0/3651830467 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0072f0 con 0x7f30f00cfcb0 2024-04-25T08:52:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f0165f60 con 0x7f30f00cfcb0 2024-04-25T08:52:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.718+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f015dc60 con 0x7f30f00cfcb0 2024-04-25T08:52:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.722+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f0069080 con 0x7f30f00cfcb0 2024-04-25T08:52:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.722+0000 7f30ed600640 1 -- 172.21.15.46:0/3651830467 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30dc01a070 con 0x7f30f00cfcb0 2024-04-25T08:52:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.722+0000 7f30ed600640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30d0066f60 0x7f30d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.722+0000 7f30ed600640 1 -- 172.21.15.46:0/3651830467 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f30dc084310 con 0x7f30f00cfcb0 2024-04-25T08:52:25.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.722+0000 7f30f5c00640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30d0066f60 0x7f30d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:25.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.722+0000 7f30f5c00640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30d0066f60 0x7f30d0069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f30e4000f70 tx=0x7f30e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:25.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.726+0000 7f30ed600640 1 -- 172.21.15.46:0/3651830467 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30f0165f60 con 0x7f30f00cfcb0 2024-04-25T08:52:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:25.890+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f30f00737e0 con 0x7f30f00cfcb0 2024-04-25T08:52:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.738+0000 7f30ed600640 1 -- 172.21.15.46:0/3651830467 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v118) v1 ==== 138+0+0 (secure 0 0 0) 0x7f30dc055440 con 0x7f30f00cfcb0 2024-04-25T08:52:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-04-25T08:52:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30d0066f60 msgr2=0x7f30d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30d0066f60 0x7f30d0069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f30e4000f70 tx=0x7f30e4009040 comp rx=0 tx=0).stop 2024-04-25T08:52:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f00cfcb0 msgr2=0x7f30f015cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f00cfcb0 0x7f30f015cac0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c3c0 tx=0x7f30dc00c880 comp rx=0 tx=0).stop 2024-04-25T08:52:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 shutdown_connections 2024-04-25T08:52:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f30d0066f60 0x7f30d0069420 secure :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f30e4000f70 tx=0x7f30e4009040 comp rx=0 tx=0).stop 2024-04-25T08:52:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f30f015bb90 0x7f30f015d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30f00cfcb0 0x7f30f015cac0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 --2- 172.21.15.46:0/3651830467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f00cf4b0 0x7f30f00ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 >> 172.21.15.46:0/3651830467 conn(0x7f30f00b31c0 msgr2=0x7f30f015ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 shutdown_connections 2024-04-25T08:52:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.742+0000 7f30f8600640 1 -- 172.21.15.46:0/3651830467 wait complete. 2024-04-25T08:52:26.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:52:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f1d000640 1 Processor -- start 2024-04-25T08:52:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f1d000640 1 -- start start 2024-04-25T08:52:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f1d000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1f18143990 0x7f1f180cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f1d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f181440d0 0x7f1f1806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f1d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f1815e440 con 0x7f1f1806ef80 2024-04-25T08:52:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f1d000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f1815ed30 con 0x7f1f18143990 2024-04-25T08:52:26.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f1d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f180b3730 con 0x7f1f181440d0 2024-04-25T08:52:26.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f16800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1f1806ef80 0x7f1f180cd570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39378/0 (socket says 172.21.15.46:39378) 2024-04-25T08:52:26.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f16800640 1 -- 172.21.15.46:0/407208310 learned_addr learned my addr 172.21.15.46:0/407208310 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f0d400640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f181440d0 0x7f1f1806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15e00640 1 --2- 172.21.15.46:0/407208310 >> v2:172.21.15.82:3300/0 conn(0x7f1f18143990 0x7f1f180cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325667242 0 0) 0x7f1f1815e440 con 0x7f1f1806ef80 2024-04-25T08:52:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1efc003530 con 0x7f1f1806ef80 2024-04-25T08:52:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1981941270 0 0) 0x7f1efc003530 con 0x7f1f1806ef80 2024-04-25T08:52:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f181440d0 msgr2=0x7f1f1806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f181440d0 0x7f1f1806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 >> v2:172.21.15.82:3300/0 conn(0x7f1f18143990 msgr2=0x7f1f180cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 --2- 172.21.15.46:0/407208310 >> v2:172.21.15.82:3300/0 conn(0x7f1f18143990 0x7f1f180cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1815e6e0 con 0x7f1f1806ef80 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f1f10001a00 con 0x7f1f1806ef80 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 >> v1:172.21.15.46:6789/0 conn(0x7f1f1806ef80 legacy=0x7f1f180cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f181440d0 0x7f1efc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1efc004000 0x7f1efc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1efc004d50 0x7f1efc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1815ed30 con 0x7f1efc004000 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f180b3730 con 0x7f1efc004d50 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc00dc60 con 0x7f1f181440d0 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f0d400640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1efc004000 0x7f1efc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f16800640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1efc004d50 0x7f1efc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f15e00640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f181440d0 0x7f1efc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f0d400640 1 -- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f181440d0 msgr2=0x7f1efc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f0d400640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f181440d0 0x7f1efc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f0d400640 1 -- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1efc004d50 msgr2=0x7f1efc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f0d400640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1efc004d50 0x7f1efc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.926+0000 7f1f0d400640 1 -- 172.21.15.46:0/407208310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc003530 con 0x7f1efc004000 2024-04-25T08:52:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f0d400640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1efc004000 0x7f1efc004810 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009ae0 tx=0x7f1f0001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2235491727f586a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000b2070 con 0x7f1efc004000 2024-04-25T08:52:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f000a5660 con 0x7f1efc004000 2024-04-25T08:52:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f15400640 1 -- 172.21.15.46:0/407208310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000096b0 con 0x7f1efc004000 2024-04-25T08:52:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f1d000640 1 -- 172.21.15.46:0/407208310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1efc004000 msgr2=0x7f1efc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f1d000640 1 --2- 172.21.15.46:0/407208310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1efc004000 0x7f1efc004810 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009ae0 tx=0x7f1f0001ca70 comp rx=0 tx=0).stop 2024-04-25T08:52:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f16800640 1 -- 172.21.15.46:0/407208310 reap_dead start 2024-04-25T08:52:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f1d000640 1 -- 172.21.15.46:0/407208310 shutdown_connections 2024-04-25T08:52:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f1d000640 1 -- 172.21.15.46:0/407208310 >> 172.21.15.46:0/407208310 conn(0x7f1f1806ac10 msgr2=0x7f1f1806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f1d000640 1 -- 172.21.15.46:0/407208310 shutdown_connections 2024-04-25T08:52:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.930+0000 7f1f1d000640 1 -- 172.21.15.46:0/407208310 wait complete. 2024-04-25T08:52:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 Processor -- start 2024-04-25T08:52:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 -- start start 2024-04-25T08:52:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1efc004000 0x7f1f18141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f1813b280 0x7f1f1813b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f1814ed40 0x7f1f1814f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1806f410 con 0x7f1f1813b280 2024-04-25T08:52:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f180cf410 con 0x7f1efc004000 2024-04-25T08:52:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1815f1c0 con 0x7f1f1814ed40 2024-04-25T08:52:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f1813b280 0x7f1f1813b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:26.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f1813b280 0x7f1f1813b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40946/0 (socket says 172.21.15.46:40946) 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0d400640 1 -- 172.21.15.46:0/3945380509 learned_addr learned my addr 172.21.15.46:0/3945380509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f15e00640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1efc004000 0x7f1f18141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f16800640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f1814ed40 0x7f1f1814f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0d400640 1 -- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f1814ed40 msgr2=0x7f1f1814f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0d400640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f1814ed40 0x7f1f1814f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0d400640 1 -- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1efc004000 msgr2=0x7f1f18141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0d400640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1efc004000 0x7f1f18141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0d400640 1 -- 172.21.15.46:0/3945380509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1813e630 con 0x7f1f1813b280 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0d400640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f1813b280 0x7f1f1813b730 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009ae0 tx=0x7f1efc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0ea00640 1 -- 172.21.15.46:0/3945380509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000bc030 con 0x7f1f1813b280 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0ea00640 1 -- 172.21.15.46:0/3945380509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0000f000 con 0x7f1f1813b280 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f0ea00640 1 -- 172.21.15.46:0/3945380509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f000b2040 con 0x7f1f1813b280 2024-04-25T08:52:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1813e910 con 0x7f1f1813b280 2024-04-25T08:52:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.934+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f18141530 con 0x7f1f1813b280 2024-04-25T08:52:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.938+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f18069010 con 0x7f1f1813b280 2024-04-25T08:52:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.938+0000 7f1f0ea00640 1 -- 172.21.15.46:0/3945380509 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f000bf070 con 0x7f1f1813b280 2024-04-25T08:52:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.938+0000 7f1f0ea00640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ef4066f90 0x7f1ef4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.938+0000 7f1f0ea00640 1 -- 172.21.15.46:0/3945380509 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f1f0011f0b0 con 0x7f1f1813b280 2024-04-25T08:52:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.938+0000 7f1f15e00640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ef4066f90 0x7f1ef4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.938+0000 7f1f15e00640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ef4066f90 0x7f1ef4069450 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1f1814cb80 tx=0x7f1f08009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:26.942+0000 7f1f0ea00640 1 -- 172.21.15.46:0/3945380509 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f00016780 con 0x7f1f1813b280 2024-04-25T08:52:27.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.106+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1f180cc6c0 con 0x7f1f1813b280 2024-04-25T08:52:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.742+0000 7f1f0ea00640 1 -- 172.21.15.46:0/3945380509 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v119) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1f000efab0 con 0x7f1f1813b280 2024-04-25T08:52:27.733 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ef4066f90 msgr2=0x7f1ef4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ef4066f90 0x7f1ef4069450 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1f1814cb80 tx=0x7f1f08009190 comp rx=0 tx=0).stop 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f1813b280 msgr2=0x7f1f1813b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f1813b280 0x7f1f1813b730 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009ae0 tx=0x7f1efc005840 comp rx=0 tx=0).stop 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 shutdown_connections 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1ef4066f90 0x7f1ef4069450 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1f1814ed40 0x7f1f1814f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f1813b280 0x7f1f1813b730 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 --2- 172.21.15.46:0/3945380509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1efc004000 0x7f1f18141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 >> 172.21.15.46:0/3945380509 conn(0x7f1f1806ac10 msgr2=0x7f1f180cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:27.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 shutdown_connections 2024-04-25T08:52:27.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.746+0000 7f1f1d000640 1 -- 172.21.15.46:0/3945380509 wait complete. 2024-04-25T08:52:27.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:52:27.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.930+0000 7f1eaa200640 1 Processor -- start 2024-04-25T08:52:27.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.930+0000 7f1eaa200640 1 -- start start 2024-04-25T08:52:27.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1eaa200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406e620 0x7f1ea406bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1eaa200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ea406ea60 0x7f1ea406c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1eaa200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea40b3850 con 0x7f1ea415dae0 2024-04-25T08:52:27.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1eaa200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea406e190 con 0x7f1ea406ea60 2024-04-25T08:52:27.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1eaa200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40b3af0 con 0x7f1ea406e620 2024-04-25T08:52:27.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1ea3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ea406ea60 0x7f1ea406c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1ea3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ea406ea60 0x7f1ea406c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40958/0 (socket says 172.21.15.46:40958) 2024-04-25T08:52:27.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1ea3e00640 1 -- 172.21.15.46:0/788790737 learned_addr learned my addr 172.21.15.46:0/788790737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:27.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1e9aa00640 1 --2- 172.21.15.46:0/788790737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406e620 0x7f1ea406bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1ea3e00640 1 -- 172.21.15.46:0/788790737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406e620 msgr2=0x7f1ea406bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:27.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1ea3e00640 1 --2- 172.21.15.46:0/788790737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406e620 0x7f1ea406bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1ea3e00640 1 -- 172.21.15.46:0/788790737 >> v1:172.21.15.82:6789/0 conn(0x7f1ea415dae0 legacy=0x7f1ea4072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1ea3e00640 1 -- 172.21.15.46:0/788790737 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40b3dd0 con 0x7f1ea406ea60 2024-04-25T08:52:27.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.934+0000 7f1ea3e00640 1 --2- 172.21.15.46:0/788790737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea406ea60 0x7f1ea406c160 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f1ea406e310 tx=0x7f1e9c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdd01538d933047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1ea2a00640 1 -- 172.21.15.46:0/788790737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0aa670 con 0x7f1ea406ea60 2024-04-25T08:52:27.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1ea2a00640 1 -- 172.21.15.46:0/788790737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c0af070 con 0x7f1ea406ea60 2024-04-25T08:52:27.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1ea2a00640 1 -- 172.21.15.46:0/788790737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0aaa30 con 0x7f1ea406ea60 2024-04-25T08:52:27.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 -- 172.21.15.46:0/788790737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea406ea60 msgr2=0x7f1ea406c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 --2- 172.21.15.46:0/788790737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea406ea60 0x7f1ea406c160 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f1ea406e310 tx=0x7f1e9c01c510 comp rx=0 tx=0).stop 2024-04-25T08:52:27.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 -- 172.21.15.46:0/788790737 shutdown_connections 2024-04-25T08:52:27.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 --2- 172.21.15.46:0/788790737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea406ea60 0x7f1ea406c160 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 --2- 172.21.15.46:0/788790737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406e620 0x7f1ea406bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 -- 172.21.15.46:0/788790737 >> 172.21.15.46:0/788790737 conn(0x7f1ea4071ae0 msgr2=0x7f1ea4071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:27.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 -- 172.21.15.46:0/788790737 shutdown_connections 2024-04-25T08:52:27.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 -- 172.21.15.46:0/788790737 wait complete. 2024-04-25T08:52:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 Processor -- start 2024-04-25T08:52:27.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 -- start start 2024-04-25T08:52:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ea406e620 0x7f1ea414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406ea60 0x7f1ea4160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea415dae0 0x7f1ea413d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea406f850 con 0x7f1ea415dae0 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea406f6d0 con 0x7f1ea406e620 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1eaa200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40ad5c0 con 0x7f1ea406ea60 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1ea3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea415dae0 0x7f1ea413d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1ea3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea415dae0 0x7f1ea413d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40966/0 (socket says 172.21.15.46:40966) 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1ea3e00640 1 -- 172.21.15.46:0/2605840336 learned_addr learned my addr 172.21.15.46:0/2605840336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1ea3400640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ea406e620 0x7f1ea414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.938+0000 7f1e9aa00640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406ea60 0x7f1ea4160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1ea3e00640 1 -- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406ea60 msgr2=0x7f1ea4160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1ea3e00640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406ea60 0x7f1ea4160960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1ea3e00640 1 -- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ea406e620 msgr2=0x7f1ea414d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1ea3e00640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ea406e620 0x7f1ea414d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1ea3e00640 1 -- 172.21.15.46:0/2605840336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea4160ea0 con 0x7f1ea415dae0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1ea3e00640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea415dae0 0x7f1ea413d770 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c01abc0 tx=0x7f1e9c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1e9be00640 1 -- 172.21.15.46:0/2605840336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0bd5f0 con 0x7f1ea415dae0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1e9be00640 1 -- 172.21.15.46:0/2605840336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c0bd790 con 0x7f1ea415dae0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea415f8c0 con 0x7f1ea415dae0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1e9be00640 1 -- 172.21.15.46:0/2605840336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0bdb80 con 0x7f1ea415dae0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea415fe80 con 0x7f1ea415dae0 2024-04-25T08:52:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea4069010 con 0x7f1ea415dae0 2024-04-25T08:52:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.942+0000 7f1e9be00640 1 -- 172.21.15.46:0/2605840336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e9c0b33f0 con 0x7f1ea415dae0 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.946+0000 7f1e9be00640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e80066f90 0x7f1e80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.946+0000 7f1e9be00640 1 -- 172.21.15.46:0/2605840336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f1e9c11fc90 con 0x7f1ea415dae0 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.946+0000 7f1ea3400640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e80066f90 0x7f1e80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.946+0000 7f1ea3400640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e80066f90 0x7f1e80069450 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1e94001e00 tx=0x7f1e94009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:27.950+0000 7f1e9be00640 1 -- 172.21.15.46:0/2605840336 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e9c0c3060 con 0x7f1ea415dae0 2024-04-25T08:52:28.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.142+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1ea4153b60 con 0x7f1ea415dae0 2024-04-25T08:52:28.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.766+0000 7f1e9be00640 1 -- 172.21.15.46:0/2605840336 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v95) v1 ==== 174+0+0 (secure 0 0 0) 0x7f1e9c016130 con 0x7f1ea415dae0 2024-04-25T08:52:28.763 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T08:52:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e80066f90 msgr2=0x7f1e80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e80066f90 0x7f1e80069450 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1e94001e00 tx=0x7f1e94009040 comp rx=0 tx=0).stop 2024-04-25T08:52:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea415dae0 msgr2=0x7f1ea413d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea415dae0 0x7f1ea413d770 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c01abc0 tx=0x7f1e9c01ad50 comp rx=0 tx=0).stop 2024-04-25T08:52:28.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 shutdown_connections 2024-04-25T08:52:28.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e80066f90 0x7f1e80069450 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ea415dae0 0x7f1ea413d770 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1ea406ea60 0x7f1ea4160960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 --2- 172.21.15.46:0/2605840336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ea406e620 0x7f1ea414d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.774+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 >> 172.21.15.46:0/2605840336 conn(0x7f1ea4071ae0 msgr2=0x7f1ea40723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:28.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.778+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 shutdown_connections 2024-04-25T08:52:28.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.778+0000 7f1eaa200640 1 -- 172.21.15.46:0/2605840336 wait complete. 2024-04-25T08:52:28.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:28.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa067e00640 1 Processor -- start 2024-04-25T08:52:28.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa067e00640 1 -- start start 2024-04-25T08:52:28.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa067e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa06015bdb0 0x7fa0600cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa067e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa0600cda80 0x7fa06015d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa067e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0600cc700 con 0x7fa060159230 2024-04-25T08:52:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa067e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0600cded0 con 0x7fa0600cda80 2024-04-25T08:52:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa067e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0600cc9e0 con 0x7fa06015bdb0 2024-04-25T08:52:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa065e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa0600cda80 0x7fa06015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa065e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa0600cda80 0x7fa06015d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60574/0 (socket says 172.21.15.46:60574) 2024-04-25T08:52:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa065e00640 1 -- 172.21.15.46:0/798100165 learned_addr learned my addr 172.21.15.46:0/798100165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:28.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa065400640 1 --1- 172.21.15.46:0/798100165 >> v1:172.21.15.46:6789/0 conn(0x7fa060159230 0x7fa0600cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38220/0 (socket says 172.21.15.46:38220) 2024-04-25T08:52:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa064a00640 1 --2- 172.21.15.46:0/798100165 >> v2:172.21.15.82:3300/0 conn(0x7fa06015bdb0 0x7fa0600cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa065e00640 1 -- 172.21.15.46:0/798100165 >> v2:172.21.15.82:3300/0 conn(0x7fa06015bdb0 msgr2=0x7fa0600cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa065e00640 1 --2- 172.21.15.46:0/798100165 >> v2:172.21.15.82:3300/0 conn(0x7fa06015bdb0 0x7fa0600cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa065e00640 1 -- 172.21.15.46:0/798100165 >> v1:172.21.15.46:6789/0 conn(0x7fa060159230 legacy=0x7fa0600cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa065e00640 1 -- 172.21.15.46:0/798100165 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0600cccc0 con 0x7fa0600cda80 2024-04-25T08:52:28.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.966+0000 7fa05fe00640 1 -- 172.21.15.46:0/798100165 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025500737 0 0) 0x7fa0600cc700 con 0x7fa060159230 2024-04-25T08:52:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa065e00640 1 --2- 172.21.15.46:0/798100165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 0x7fa06015d200 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa060050000 tx=0x7fa05801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720123e689b51150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa05fe00640 1 -- 172.21.15.46:0/798100165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0580a5710 con 0x7fa0600cda80 2024-04-25T08:52:28.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa05fe00640 1 -- 172.21.15.46:0/798100165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0580a58b0 con 0x7fa0600cda80 2024-04-25T08:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa05fe00640 1 -- 172.21.15.46:0/798100165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0580a5c80 con 0x7fa0600cda80 2024-04-25T08:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa067e00640 1 -- 172.21.15.46:0/798100165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 msgr2=0x7fa06015d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa067e00640 1 --2- 172.21.15.46:0/798100165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 0x7fa06015d200 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa060050000 tx=0x7fa05801c960 comp rx=0 tx=0).stop 2024-04-25T08:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa067e00640 1 -- 172.21.15.46:0/798100165 shutdown_connections 2024-04-25T08:52:28.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa067e00640 1 --2- 172.21.15.46:0/798100165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 0x7fa06015d200 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa067e00640 1 --2- 172.21.15.46:0/798100165 >> v2:172.21.15.82:3300/0 conn(0x7fa06015bdb0 0x7fa0600cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa067e00640 1 -- 172.21.15.46:0/798100165 >> 172.21.15.46:0/798100165 conn(0x7fa0600b12a0 msgr2=0x7fa0600b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa067e00640 1 -- 172.21.15.46:0/798100165 shutdown_connections 2024-04-25T08:52:28.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.970+0000 7fa067e00640 1 -- 172.21.15.46:0/798100165 wait complete. 2024-04-25T08:52:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 Processor -- start 2024-04-25T08:52:28.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 -- start start 2024-04-25T08:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 0x7fa060157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa060159230 0x7fa0600b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa06015bdb0 0x7fa060140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06015e0d0 con 0x7fa0600cda80 2024-04-25T08:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06015df50 con 0x7fa06015bdb0 2024-04-25T08:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06015e250 con 0x7fa060159230 2024-04-25T08:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa065400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 0x7fa060157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa065400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 0x7fa060157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60576/0 (socket says 172.21.15.46:60576) 2024-04-25T08:52:28.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa065400640 1 -- 172.21.15.46:0/2162678468 learned_addr learned my addr 172.21.15.46:0/2162678468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa065400640 1 -- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa060159230 msgr2=0x7fa0600b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa065400640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa060159230 0x7fa0600b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa065400640 1 -- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa06015bdb0 msgr2=0x7fa060140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa065400640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa06015bdb0 0x7fa060140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa065400640 1 -- 172.21.15.46:0/2162678468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0601f6c90 con 0x7fa0600cda80 2024-04-25T08:52:28.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa065400640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 0x7fa060157ce0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fa0540011f0 tx=0x7fa05400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa05d600640 1 -- 172.21.15.46:0/2162678468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054018070 con 0x7fa0600cda80 2024-04-25T08:52:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa05d600640 1 -- 172.21.15.46:0/2162678468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0540074e0 con 0x7fa0600cda80 2024-04-25T08:52:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa05d600640 1 -- 172.21.15.46:0/2162678468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054004a10 con 0x7fa0600cda80 2024-04-25T08:52:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0601f6f70 con 0x7fa0600cda80 2024-04-25T08:52:28.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0601f7530 con 0x7fa0600cda80 2024-04-25T08:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.974+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa060069080 con 0x7fa0600cda80 2024-04-25T08:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.978+0000 7fa05d600640 1 -- 172.21.15.46:0/2162678468 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa05401b070 con 0x7fa0600cda80 2024-04-25T08:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.978+0000 7fa05d600640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa040066f60 0x7fa040069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.978+0000 7fa05d600640 1 -- 172.21.15.46:0/2162678468 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa054084590 con 0x7fa0600cda80 2024-04-25T08:52:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.978+0000 7fa064a00640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa040066f60 0x7fa040069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.978+0000 7fa064a00640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa040066f60 0x7fa040069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa04c006aa0 tx=0x7fa04c006a30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:28.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:28.982+0000 7fa05d600640 1 -- 172.21.15.46:0/2162678468 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa054055400 con 0x7fa0600cda80 2024-04-25T08:52:29.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.146+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0600aec50 con 0x7fa0600cda80 2024-04-25T08:52:29.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.146+0000 7fa05d600640 1 -- 172.21.15.46:0/2162678468 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16917 (secure 0 0 0) 0x7fa054054d00 con 0x7fa0600cda80 2024-04-25T08:52:29.142 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:29.142 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":120,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:28.147293+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa040066f60 msgr2=0x7fa040069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa040066f60 0x7fa040069420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa04c006aa0 tx=0x7fa04c006a30 comp rx=0 tx=0).stop 2024-04-25T08:52:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 msgr2=0x7fa060157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 0x7fa060157ce0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fa0540011f0 tx=0x7fa05400c9b0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 shutdown_connections 2024-04-25T08:52:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa040066f60 0x7fa040069420 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa06015bdb0 0x7fa060140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa060159230 0x7fa0600b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 --2- 172.21.15.46:0/2162678468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0600cda80 0x7fa060157ce0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 >> 172.21.15.46:0/2162678468 conn(0x7fa0600b12a0 msgr2=0x7fa0600cbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 shutdown_connections 2024-04-25T08:52:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.150+0000 7fa067e00640 1 -- 172.21.15.46:0/2162678468 wait complete. 2024-04-25T08:52:29.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:29.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.310+0000 7fb79ec00640 1 Processor -- start 2024-04-25T08:52:29.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.310+0000 7fb79ec00640 1 -- start start 2024-04-25T08:52:29.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb79ec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb798157b00 0x7fb7980b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb79ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7980cdd10 0x7fb7980ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb79ec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7980b25a0 con 0x7fb7980cc150 2024-04-25T08:52:29.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb79ec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79806d980 con 0x7fb798157b00 2024-04-25T08:52:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb79ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980b2eb0 con 0x7fb7980cdd10 2024-04-25T08:52:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb797e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb798157b00 0x7fb7980b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb797e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb798157b00 0x7fb7980b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60588/0 (socket says 172.21.15.46:60588) 2024-04-25T08:52:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb797e00640 1 -- 172.21.15.46:0/1368815474 learned_addr learned my addr 172.21.15.46:0/1368815474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb79cc00640 1 --2- 172.21.15.46:0/1368815474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7980cdd10 0x7fb7980ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb797e00640 1 -- 172.21.15.46:0/1368815474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7980cdd10 msgr2=0x7fb7980ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb797e00640 1 --2- 172.21.15.46:0/1368815474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7980cdd10 0x7fb7980ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb797e00640 1 -- 172.21.15.46:0/1368815474 >> v1:172.21.15.82:6789/0 conn(0x7fb7980cc150 legacy=0x7fb79806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.314+0000 7fb797e00640 1 -- 172.21.15.46:0/1368815474 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980cc6f0 con 0x7fb798157b00 2024-04-25T08:52:29.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb797e00640 1 --2- 172.21.15.46:0/1368815474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb798157b00 0x7fb7980b3340 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb788009e50 tx=0x7fb78801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa1545c40cf1bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb797400640 1 -- 172.21.15.46:0/1368815474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880a56b0 con 0x7fb798157b00 2024-04-25T08:52:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb797400640 1 -- 172.21.15.46:0/1368815474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7880a5850 con 0x7fb798157b00 2024-04-25T08:52:29.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb797400640 1 -- 172.21.15.46:0/1368815474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7880a5c20 con 0x7fb798157b00 2024-04-25T08:52:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 -- 172.21.15.46:0/1368815474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb798157b00 msgr2=0x7fb7980b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 --2- 172.21.15.46:0/1368815474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb798157b00 0x7fb7980b3340 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb788009e50 tx=0x7fb78801c8e0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 -- 172.21.15.46:0/1368815474 shutdown_connections 2024-04-25T08:52:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 --2- 172.21.15.46:0/1368815474 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb7980cdd10 0x7fb7980ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 --2- 172.21.15.46:0/1368815474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb798157b00 0x7fb7980b3340 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 -- 172.21.15.46:0/1368815474 >> 172.21.15.46:0/1368815474 conn(0x7fb7980b12a0 msgr2=0x7fb7980b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 -- 172.21.15.46:0/1368815474 shutdown_connections 2024-04-25T08:52:29.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 -- 172.21.15.46:0/1368815474 wait complete. 2024-04-25T08:52:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 Processor -- start 2024-04-25T08:52:29.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 -- start start 2024-04-25T08:52:29.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7980cc150 0x7fb798151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7980cdd10 0x7fb7980ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb798157b00 0x7fb79806db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79815e3d0 con 0x7fb7980cdd10 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79815e250 con 0x7fb7980cc150 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.318+0000 7fb79ec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79815e550 con 0x7fb798157b00 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb78f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7980cdd10 0x7fb7980ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb78f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7980cdd10 0x7fb7980ae420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60604/0 (socket says 172.21.15.46:60604) 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb78f400640 1 -- 172.21.15.46:0/3653436427 learned_addr learned my addr 172.21.15.46:0/3653436427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb79cc00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb798157b00 0x7fb79806db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb797e00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7980cc150 0x7fb798151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb78f400640 1 -- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb798157b00 msgr2=0x7fb79806db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb78f400640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb798157b00 0x7fb79806db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb78f400640 1 -- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7980cc150 msgr2=0x7fb798151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb78f400640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7980cc150 0x7fb798151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb78f400640 1 -- 172.21.15.46:0/3653436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980ce470 con 0x7fb7980cdd10 2024-04-25T08:52:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb78f400640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7980cdd10 0x7fb7980ae420 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c450 tx=0x7fb78000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb794c00640 1 -- 172.21.15.46:0/3653436427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780016020 con 0x7fb7980cdd10 2024-04-25T08:52:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb794c00640 1 -- 172.21.15.46:0/3653436427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7800052d0 con 0x7fb7980cdd10 2024-04-25T08:52:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb794c00640 1 -- 172.21.15.46:0/3653436427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780007260 con 0x7fb7980cdd10 2024-04-25T08:52:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79815aa40 con 0x7fb7980cdd10 2024-04-25T08:52:29.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb798156200 con 0x7fb7980cdd10 2024-04-25T08:52:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb798069080 con 0x7fb7980cdd10 2024-04-25T08:52:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.322+0000 7fb794c00640 1 -- 172.21.15.46:0/3653436427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb780007400 con 0x7fb7980cdd10 2024-04-25T08:52:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.326+0000 7fb794c00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb774066f90 0x7fb774069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.326+0000 7fb794c00640 1 -- 172.21.15.46:0/3653436427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb780084a10 con 0x7fb7980cdd10 2024-04-25T08:52:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.326+0000 7fb797e00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb774066f90 0x7fb774069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.326+0000 7fb797e00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb774066f90 0x7fb774069450 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb7880ae000 tx=0x7fb7880a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:29.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.330+0000 7fb794c00640 1 -- 172.21.15.46:0/3653436427 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb780020020 con 0x7fb7980cdd10 2024-04-25T08:52:29.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.490+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb798155d00 con 0x7fb7980cdd10 2024-04-25T08:52:29.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb794c00640 1 -- 172.21.15.46:0/3653436427 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16917 (secure 0 0 0) 0x7fb780005480 con 0x7fb7980cdd10 2024-04-25T08:52:29.484 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:29.485 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":120,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:28.147293+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb774066f90 msgr2=0x7fb774069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb774066f90 0x7fb774069450 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb7880ae000 tx=0x7fb7880a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7980cdd10 msgr2=0x7fb7980ae420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7980cdd10 0x7fb7980ae420 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c450 tx=0x7fb78000c910 comp rx=0 tx=0).stop 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 shutdown_connections 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb774066f90 0x7fb774069450 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb798157b00 0x7fb79806db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7980cdd10 0x7fb7980ae420 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 --2- 172.21.15.46:0/3653436427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7980cc150 0x7fb798151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 >> 172.21.15.46:0/3653436427 conn(0x7fb7980b12a0 msgr2=0x7fb798073880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 shutdown_connections 2024-04-25T08:52:29.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.494+0000 7fb79ec00640 1 -- 172.21.15.46:0/3653436427 wait complete. 2024-04-25T08:52:29.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:52:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.658+0000 7f2d82e00640 1 Processor -- start 2024-04-25T08:52:29.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.658+0000 7f2d82e00640 1 -- start start 2024-04-25T08:52:29.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.658+0000 7f2d82e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2d7c15dd50 0x7f2d7c0cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.658+0000 7f2d82e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d7c0d0450 0x7f2d7c0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.658+0000 7f2d82e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7c09a780 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.658+0000 7f2d82e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c0708c0 con 0x7f2d7c0d0450 2024-04-25T08:52:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.658+0000 7f2d82e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c09aa60 con 0x7f2d7c15dd50 2024-04-25T08:52:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.658+0000 7f2d80e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d7c0d0450 0x7f2d7c0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d7be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2d7c06f4d0 0x7f2d7c0cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38222/0 (socket says 172.21.15.46:38222) 2024-04-25T08:52:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d7be00640 1 -- 172.21.15.46:0/1824419306 learned_addr learned my addr 172.21.15.46:0/1824419306 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d80e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d7c0d0450 0x7f2d7c0cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60620/0 (socket says 172.21.15.46:60620) 2024-04-25T08:52:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d73e00640 1 --2- 172.21.15.46:0/1824419306 >> v2:172.21.15.82:3300/0 conn(0x7f2d7c15dd50 0x7f2d7c0cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d80e00640 1 -- 172.21.15.46:0/1824419306 >> v2:172.21.15.82:3300/0 conn(0x7f2d7c15dd50 msgr2=0x7f2d7c0cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d80e00640 1 --2- 172.21.15.46:0/1824419306 >> v2:172.21.15.82:3300/0 conn(0x7f2d7c15dd50 0x7f2d7c0cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d80e00640 1 -- 172.21.15.46:0/1824419306 >> v1:172.21.15.46:6789/0 conn(0x7f2d7c06f4d0 legacy=0x7f2d7c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d80e00640 1 -- 172.21.15.46:0/1824419306 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c09ad40 con 0x7f2d7c0d0450 2024-04-25T08:52:29.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d80e00640 1 --2- 172.21.15.46:0/1824419306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c0d0450 0x7f2d7c0cc4c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c071720 tx=0x7f2d7401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec06adcb24e0e1eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d7b400640 1 -- 172.21.15.46:0/1824419306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d740b2070 con 0x7f2d7c0d0450 2024-04-25T08:52:29.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d7b400640 1 -- 172.21.15.46:0/1824419306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d740a59c0 con 0x7f2d7c0d0450 2024-04-25T08:52:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d7b400640 1 -- 172.21.15.46:0/1824419306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d740a5d90 con 0x7f2d7c0d0450 2024-04-25T08:52:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d82e00640 1 -- 172.21.15.46:0/1824419306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c0d0450 msgr2=0x7f2d7c0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.662+0000 7f2d82e00640 1 --2- 172.21.15.46:0/1824419306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c0d0450 0x7f2d7c0cc4c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c071720 tx=0x7f2d7401c900 comp rx=0 tx=0).stop 2024-04-25T08:52:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 -- 172.21.15.46:0/1824419306 shutdown_connections 2024-04-25T08:52:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 --2- 172.21.15.46:0/1824419306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c0d0450 0x7f2d7c0cc4c0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 --2- 172.21.15.46:0/1824419306 >> v2:172.21.15.82:3300/0 conn(0x7f2d7c15dd50 0x7f2d7c0cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 -- 172.21.15.46:0/1824419306 >> 172.21.15.46:0/1824419306 conn(0x7f2d7c0b3160 msgr2=0x7f2d7c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 -- 172.21.15.46:0/1824419306 shutdown_connections 2024-04-25T08:52:29.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 -- 172.21.15.46:0/1824419306 wait complete. 2024-04-25T08:52:29.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 Processor -- start 2024-04-25T08:52:29.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 -- start start 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c06f4d0 0x7f2d7c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d7c0d0450 0x7f2d7c158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d7c15dd50 0x7f2d7c154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0cd390 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0cd210 con 0x7f2d7c0d0450 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0cd510 con 0x7f2d7c15dd50 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c06f4d0 0x7f2d7c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d7be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c06f4d0 0x7f2d7c0c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60632/0 (socket says 172.21.15.46:60632) 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d7be00640 1 -- 172.21.15.46:0/38135606 learned_addr learned my addr 172.21.15.46:0/38135606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d7be00640 1 -- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d7c15dd50 msgr2=0x7f2d7c154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d7be00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d7c15dd50 0x7f2d7c154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d7be00640 1 -- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d7c0d0450 msgr2=0x7f2d7c158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d7be00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d7c0d0450 0x7f2d7c158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d7be00640 1 -- 172.21.15.46:0/38135606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c06bec0 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d7be00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c06f4d0 0x7f2d7c0c8740 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c450 tx=0x7f2d6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:29.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d78c00640 1 -- 172.21.15.46:0/38135606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c017070 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d78c00640 1 -- 172.21.15.46:0/38135606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c005cf0 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c06c1a0 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.666+0000 7f2d78c00640 1 -- 172.21.15.46:0/38135606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c007290 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.670+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c06c760 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.670+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c069010 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.670+0000 7f2d78c00640 1 -- 172.21.15.46:0/38135606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d6c01a070 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.670+0000 7f2d78c00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d58066f60 0x7f2d58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.670+0000 7f2d78c00640 1 -- 172.21.15.46:0/38135606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d6c0844e0 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.670+0000 7f2d73e00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d58066f60 0x7f2d58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.670+0000 7f2d73e00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d58066f60 0x7f2d58069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2d64006aa0 tx=0x7f2d64006a30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:29.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.674+0000 7f2d78c00640 1 -- 172.21.15.46:0/38135606 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d6c055360 con 0x7f2d7c06f4d0 2024-04-25T08:52:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:29.878+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2d7c070e50 con 0x7f2d7c06f4d0 2024-04-25T08:52:30.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.778+0000 7f2d78c00640 1 -- 172.21.15.46:0/38135606 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2d6c054c60 con 0x7f2d7c06f4d0 2024-04-25T08:52:30.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d58066f60 msgr2=0x7f2d58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d58066f60 0x7f2d58069420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2d64006aa0 tx=0x7f2d64006a30 comp rx=0 tx=0).stop 2024-04-25T08:52:30.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c06f4d0 msgr2=0x7f2d7c0c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c06f4d0 0x7f2d7c0c8740 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c450 tx=0x7f2d6c00c910 comp rx=0 tx=0).stop 2024-04-25T08:52:30.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 shutdown_connections 2024-04-25T08:52:30.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2d58066f60 0x7f2d58069420 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2d7c15dd50 0x7f2d7c154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d7c0d0450 0x7f2d7c158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 --2- 172.21.15.46:0/38135606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d7c06f4d0 0x7f2d7c0c8740 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 >> 172.21.15.46:0/38135606 conn(0x7f2d7c0b3160 msgr2=0x7f2d7c0cf7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:30.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 shutdown_connections 2024-04-25T08:52:30.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.782+0000 7f2d82e00640 1 -- 172.21.15.46:0/38135606 wait complete. 2024-04-25T08:52:30.795 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:52:30.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:52:30.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.954+0000 7f4ce5200640 1 Processor -- start 2024-04-25T08:52:30.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.954+0000 7f4ce5200640 1 -- start start 2024-04-25T08:52:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.954+0000 7f4ce5200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce0072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.954+0000 7f4ce5200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4ce006e620 0x7f4ce006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.954+0000 7f4ce5200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce00b3850 con 0x7f4ce006ea60 2024-04-25T08:52:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.954+0000 7f4ce5200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce006e190 con 0x7f4ce006e620 2024-04-25T08:52:30.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.954+0000 7f4ce5200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce00b3af0 con 0x7f4ce015dae0 2024-04-25T08:52:30.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce0072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cde800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4ce006ea60 0x7f4ce00acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:58442/0 (socket says 172.21.15.46:58442) 2024-04-25T08:52:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cde800640 1 -- 172.21.15.46:0/2193195446 learned_addr learned my addr 172.21.15.46:0/2193195446 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cd5400640 1 --2- 172.21.15.46:0/2193195446 >> v2:172.21.15.82:3300/0 conn(0x7f4ce006e620 0x7f4ce006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cdd400640 1 -- 172.21.15.46:0/2193195446 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765885696 0 0) 0x7f4ce00b3850 con 0x7f4ce006ea60 2024-04-25T08:52:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cdd400640 1 -- 172.21.15.46:0/2193195446 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cc4003530 con 0x7f4ce006ea60 2024-04-25T08:52:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cdde00640 1 -- 172.21.15.46:0/2193195446 >> v2:172.21.15.82:3300/0 conn(0x7f4ce006e620 msgr2=0x7f4ce006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cdde00640 1 --2- 172.21.15.46:0/2193195446 >> v2:172.21.15.82:3300/0 conn(0x7f4ce006e620 0x7f4ce006bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cdde00640 1 -- 172.21.15.46:0/2193195446 >> v1:172.21.15.82:6789/0 conn(0x7f4ce006ea60 legacy=0x7f4ce00acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cdde00640 1 -- 172.21.15.46:0/2193195446 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce00b3dd0 con 0x7f4ce015dae0 2024-04-25T08:52:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.958+0000 7f4cdde00640 1 --2- 172.21.15.46:0/2193195446 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce0072600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4ce006e310 tx=0x7f4cd001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3cf507e502ae6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:30.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4cdd400640 1 -- 172.21.15.46:0/2193195446 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00b2070 con 0x7f4ce015dae0 2024-04-25T08:52:30.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4cdd400640 1 -- 172.21.15.46:0/2193195446 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd00a58f0 con 0x7f4ce015dae0 2024-04-25T08:52:30.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4cdd400640 1 -- 172.21.15.46:0/2193195446 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00a5ca0 con 0x7f4ce015dae0 2024-04-25T08:52:30.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 -- 172.21.15.46:0/2193195446 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 msgr2=0x7f4ce0072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 --2- 172.21.15.46:0/2193195446 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce0072600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4ce006e310 tx=0x7f4cd001c920 comp rx=0 tx=0).stop 2024-04-25T08:52:30.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 -- 172.21.15.46:0/2193195446 shutdown_connections 2024-04-25T08:52:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 --2- 172.21.15.46:0/2193195446 >> v2:172.21.15.82:3300/0 conn(0x7f4ce006e620 0x7f4ce006bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 --2- 172.21.15.46:0/2193195446 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce0072600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 -- 172.21.15.46:0/2193195446 >> 172.21.15.46:0/2193195446 conn(0x7f4ce0071ae0 msgr2=0x7f4ce0071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 -- 172.21.15.46:0/2193195446 shutdown_connections 2024-04-25T08:52:30.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 -- 172.21.15.46:0/2193195446 wait complete. 2024-04-25T08:52:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 Processor -- start 2024-04-25T08:52:30.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 -- start start 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce006e620 0x7f4ce015f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce006ea60 0x7f4ce014aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce014afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce015dec0 con 0x7f4ce006e620 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce006c570 con 0x7f4ce006ea60 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4ce5200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce006f6d0 con 0x7f4ce015dae0 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce006e620 0x7f4ce015f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce006e620 0x7f4ce015f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60640/0 (socket says 172.21.15.46:60640) 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.962+0000 7f4cdde00640 1 -- 172.21.15.46:0/740223941 learned_addr learned my addr 172.21.15.46:0/740223941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cde800640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce014afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cd5400640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce006ea60 0x7f4ce014aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cdde00640 1 -- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 msgr2=0x7f4ce014afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cdde00640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce014afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cdde00640 1 -- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce006ea60 msgr2=0x7f4ce014aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cdde00640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce006ea60 0x7f4ce014aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cdde00640 1 -- 172.21.15.46:0/740223941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce01fa580 con 0x7f4ce006e620 2024-04-25T08:52:30.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cd5400640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce006ea60 0x7f4ce014aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:30.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cde800640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce014afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:30.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cdde00640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce006e620 0x7f4ce015f2f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b0040 tx=0x7f4cd00196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:30.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cd6a00640 1 -- 172.21.15.46:0/740223941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00b2040 con 0x7f4ce006e620 2024-04-25T08:52:30.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce01fa7f0 con 0x7f4ce006e620 2024-04-25T08:52:30.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cd6a00640 1 -- 172.21.15.46:0/740223941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd00b8470 con 0x7f4ce006e620 2024-04-25T08:52:30.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cd6a00640 1 -- 172.21.15.46:0/740223941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00b8880 con 0x7f4ce006e620 2024-04-25T08:52:30.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce01fadb0 con 0x7f4ce006e620 2024-04-25T08:52:30.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce0069010 con 0x7f4ce006e620 2024-04-25T08:52:30.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cd6a00640 1 -- 172.21.15.46:0/740223941 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cd00c1020 con 0x7f4ce006e620 2024-04-25T08:52:30.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cd6a00640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cbc066f90 0x7f4cbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.966+0000 7f4cd6a00640 1 -- 172.21.15.46:0/740223941 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4cd011f790 con 0x7f4ce006e620 2024-04-25T08:52:30.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.970+0000 7f4cd5400640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cbc066f90 0x7f4cbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.970+0000 7f4cd5400640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cbc066f90 0x7f4cbc069450 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00af5a0 tx=0x7f4cc8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:30.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:30.974+0000 7f4cd6a00640 1 -- 172.21.15.46:0/740223941 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cd0016540 con 0x7f4ce006e620 2024-04-25T08:52:31.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.170+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4ce0067c50 con 0x7f4ce006e620 2024-04-25T08:52:31.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.778+0000 7f4cd6a00640 1 -- 172.21.15.46:0/740223941 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4cd00f06a0 con 0x7f4ce006e620 2024-04-25T08:52:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cbc066f90 msgr2=0x7f4cbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cbc066f90 0x7f4cbc069450 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00af5a0 tx=0x7f4cc8008040 comp rx=0 tx=0).stop 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce006e620 msgr2=0x7f4ce015f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce006e620 0x7f4ce015f2f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00b0040 tx=0x7f4cd00196c0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 shutdown_connections 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4cbc066f90 0x7f4cbc069450 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4ce015dae0 0x7f4ce014afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ce006ea60 0x7f4ce014aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 --2- 172.21.15.46:0/740223941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ce006e620 0x7f4ce015f2f0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 >> 172.21.15.46:0/740223941 conn(0x7f4ce0071ae0 msgr2=0x7f4ce00acdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 shutdown_connections 2024-04-25T08:52:31.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.782+0000 7f4ce5200640 1 -- 172.21.15.46:0/740223941 wait complete. 2024-04-25T08:52:31.797 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:52:31.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.950+0000 7fd9c3e00640 1 Processor -- start 2024-04-25T08:52:31.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.950+0000 7fd9c3e00640 1 -- start start 2024-04-25T08:52:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd9bc15a750 0x7fd9bc06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9bc0cc9e0 0x7fd9bc06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc06d5e0 con 0x7fd9bc0aed10 2024-04-25T08:52:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9bc15ab30 con 0x7fd9bc0cc9e0 2024-04-25T08:52:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c3e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9bc0ce4f0 con 0x7fd9bc15a750 2024-04-25T08:52:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9bc0cc9e0 0x7fd9bc06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9bc0cc9e0 0x7fd9bc06f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60650/0 (socket says 172.21.15.46:60650) 2024-04-25T08:52:31.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c1e00640 1 -- 172.21.15.46:0/2027483311 learned_addr learned my addr 172.21.15.46:0/2027483311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c1e00640 1 -- 172.21.15.46:0/2027483311 >> v2:172.21.15.82:3300/0 conn(0x7fd9bc15a750 msgr2=0x7fd9bc06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c1e00640 1 --2- 172.21.15.46:0/2027483311 >> v2:172.21.15.82:3300/0 conn(0x7fd9bc15a750 0x7fd9bc06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c1e00640 1 -- 172.21.15.46:0/2027483311 >> v1:172.21.15.82:6789/0 conn(0x7fd9bc0aed10 legacy=0x7fd9bc0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:31.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c1e00640 1 -- 172.21.15.46:0/2027483311 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc0ce720 con 0x7fd9bc0cc9e0 2024-04-25T08:52:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.954+0000 7fd9c1e00640 1 --2- 172.21.15.46:0/2027483311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc0cc9e0 0x7fd9bc06f150 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc04ffd0 tx=0x7fd9b401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8169418248b41d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:31.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9bbe00640 1 -- 172.21.15.46:0/2027483311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40a5760 con 0x7fd9bc0cc9e0 2024-04-25T08:52:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9bbe00640 1 -- 172.21.15.46:0/2027483311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b40a5900 con 0x7fd9bc0cc9e0 2024-04-25T08:52:31.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9bbe00640 1 -- 172.21.15.46:0/2027483311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40a5cd0 con 0x7fd9bc0cc9e0 2024-04-25T08:52:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 -- 172.21.15.46:0/2027483311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc0cc9e0 msgr2=0x7fd9bc06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 --2- 172.21.15.46:0/2027483311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc0cc9e0 0x7fd9bc06f150 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc04ffd0 tx=0x7fd9b401c9b0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 -- 172.21.15.46:0/2027483311 shutdown_connections 2024-04-25T08:52:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 --2- 172.21.15.46:0/2027483311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc0cc9e0 0x7fd9bc06f150 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 --2- 172.21.15.46:0/2027483311 >> v2:172.21.15.82:3300/0 conn(0x7fd9bc15a750 0x7fd9bc06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 -- 172.21.15.46:0/2027483311 >> 172.21.15.46:0/2027483311 conn(0x7fd9bc0b12a0 msgr2=0x7fd9bc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 -- 172.21.15.46:0/2027483311 shutdown_connections 2024-04-25T08:52:31.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 -- 172.21.15.46:0/2027483311 wait complete. 2024-04-25T08:52:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 Processor -- start 2024-04-25T08:52:31.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 -- start start 2024-04-25T08:52:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9bc0aed10 0x7fd9bc155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc0cc9e0 0x7fd9bc14a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc15a750 0x7fd9bc155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc165220 con 0x7fd9bc15a750 2024-04-25T08:52:31.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc06d270 con 0x7fd9bc0cc9e0 2024-04-25T08:52:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1653a0 con 0x7fd9bc0aed10 2024-04-25T08:52:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc15a750 0x7fd9bc155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc15a750 0x7fd9bc155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60660/0 (socket says 172.21.15.46:60660) 2024-04-25T08:52:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c1e00640 1 -- 172.21.15.46:0/352123981 learned_addr learned my addr 172.21.15.46:0/352123981 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c1400640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9bc0aed10 0x7fd9bc155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.958+0000 7fd9c0a00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc0cc9e0 0x7fd9bc14a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9c1e00640 1 -- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9bc0aed10 msgr2=0x7fd9bc155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9c1e00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9bc0aed10 0x7fd9bc155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9c1e00640 1 -- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc0cc9e0 msgr2=0x7fd9bc14a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9c1e00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc0cc9e0 0x7fd9bc14a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9c1e00640 1 -- 172.21.15.46:0/352123981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc156510 con 0x7fd9bc15a750 2024-04-25T08:52:31.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9c1e00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc15a750 0x7fd9bc155390 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40af040 tx=0x7fd9b4019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9b9600640 1 -- 172.21.15.46:0/352123981 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40bc020 con 0x7fd9bc15a750 2024-04-25T08:52:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9b9600640 1 -- 172.21.15.46:0/352123981 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b40b8670 con 0x7fd9bc15a750 2024-04-25T08:52:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9b9600640 1 -- 172.21.15.46:0/352123981 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40b8a60 con 0x7fd9bc15a750 2024-04-25T08:52:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc14cfb0 con 0x7fd9bc15a750 2024-04-25T08:52:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc14aad0 con 0x7fd9bc15a750 2024-04-25T08:52:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc069080 con 0x7fd9bc15a750 2024-04-25T08:52:31.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9b9600640 1 -- 172.21.15.46:0/352123981 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9b40bf020 con 0x7fd9bc15a750 2024-04-25T08:52:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9b9600640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd99c066f60 0x7fd99c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.962+0000 7fd9b9600640 1 -- 172.21.15.46:0/352123981 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9b41202e0 con 0x7fd9bc15a750 2024-04-25T08:52:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.966+0000 7fd9c1400640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd99c066f60 0x7fd99c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.966+0000 7fd9c1400640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd99c066f60 0x7fd99c069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098f0 tx=0x7fd9b0006f90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:31.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:31.966+0000 7fd9b9600640 1 -- 172.21.15.46:0/352123981 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9b40c6030 con 0x7fd9bc15a750 2024-04-25T08:52:32.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.162+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9bc0ae880 con 0x7fd9bc15a750 2024-04-25T08:52:32.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.162+0000 7fd9b9600640 1 -- 172.21.15.46:0/352123981 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4931 (secure 0 0 0) 0x7fd9b40165a0 con 0x7fd9bc15a750 2024-04-25T08:52:32.155 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:32.155 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1279640422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1279640422},{"type":"v1","addr":"172.21.15.46:6834","nonce":1279640422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:31.172481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/51204523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":51204523},{"type":"v1","addr":"172.21.15.46:6835","nonce":51204523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:52:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 99 2024-04-25T08:52:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd99c066f60 msgr2=0x7fd99c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd99c066f60 0x7fd99c069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098f0 tx=0x7fd9b0006f90 comp rx=0 tx=0).stop 2024-04-25T08:52:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc15a750 msgr2=0x7fd9bc155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc15a750 0x7fd9bc155390 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40af040 tx=0x7fd9b4019750 comp rx=0 tx=0).stop 2024-04-25T08:52:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 shutdown_connections 2024-04-25T08:52:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd99c066f60 0x7fd99c069420 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9bc15a750 0x7fd9bc155390 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9bc0cc9e0 0x7fd9bc14a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 --2- 172.21.15.46:0/352123981 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd9bc0aed10 0x7fd9bc155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 >> 172.21.15.46:0/352123981 conn(0x7fd9bc0b12a0 msgr2=0x7fd9bc156b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 shutdown_connections 2024-04-25T08:52:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.166+0000 7fd9c3e00640 1 -- 172.21.15.46:0/352123981 wait complete. 2024-04-25T08:52:32.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:32.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.326+0000 7f5da4800640 1 Processor -- start 2024-04-25T08:52:32.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da4800640 1 -- start start 2024-04-25T08:52:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da4800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5d9c15ef20 0x7f5d9c0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da4800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d9c0cd6d0 0x7f5d9c09a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da4800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09ab50 0x7f5d9c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da4800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c15f300 con 0x7f5d9c0cd6d0 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da4800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c0cddb0 con 0x7f5d9c15ef20 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da4800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0cc3d0 con 0x7f5d9c09ab50 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d9c0cd6d0 0x7f5d9c09a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d9c0cd6d0 0x7f5d9c09a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60674/0 (socket says 172.21.15.46:60674) 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da1400640 1 -- 172.21.15.46:0/855644446 learned_addr learned my addr 172.21.15.46:0/855644446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da2800640 1 --2- 172.21.15.46:0/855644446 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09ab50 0x7f5d9c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da1e00640 1 --2- 172.21.15.46:0/855644446 >> v2:172.21.15.82:3300/0 conn(0x7f5d9c15ef20 0x7f5d9c0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da1400640 1 -- 172.21.15.46:0/855644446 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09ab50 msgr2=0x7f5d9c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da1400640 1 --2- 172.21.15.46:0/855644446 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09ab50 0x7f5d9c099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da1400640 1 -- 172.21.15.46:0/855644446 >> v2:172.21.15.82:3300/0 conn(0x7f5d9c15ef20 msgr2=0x7f5d9c0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da1400640 1 --2- 172.21.15.46:0/855644446 >> v2:172.21.15.82:3300/0 conn(0x7f5d9c15ef20 0x7f5d9c0cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.330+0000 7f5da1400640 1 -- 172.21.15.46:0/855644446 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0cc6b0 con 0x7f5d9c0cd6d0 2024-04-25T08:52:32.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da1400640 1 --2- 172.21.15.46:0/855644446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c0cd6d0 0x7f5d9c09a610 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c09a5e0 tx=0x7f5d8c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881ca74a81e5d232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:32.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da0a00640 1 -- 172.21.15.46:0/855644446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0aa430 con 0x7f5d9c0cd6d0 2024-04-25T08:52:32.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da0a00640 1 -- 172.21.15.46:0/855644446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c0af070 con 0x7f5d9c0cd6d0 2024-04-25T08:52:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da0a00640 1 -- 172.21.15.46:0/855644446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0aa7f0 con 0x7f5d9c0cd6d0 2024-04-25T08:52:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 -- 172.21.15.46:0/855644446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c0cd6d0 msgr2=0x7f5d9c09a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 --2- 172.21.15.46:0/855644446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c0cd6d0 0x7f5d9c09a610 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c09a5e0 tx=0x7f5d8c01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 -- 172.21.15.46:0/855644446 shutdown_connections 2024-04-25T08:52:32.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 --2- 172.21.15.46:0/855644446 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09ab50 0x7f5d9c099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 --2- 172.21.15.46:0/855644446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c0cd6d0 0x7f5d9c09a610 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 --2- 172.21.15.46:0/855644446 >> v2:172.21.15.82:3300/0 conn(0x7f5d9c15ef20 0x7f5d9c0cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 -- 172.21.15.46:0/855644446 >> 172.21.15.46:0/855644446 conn(0x7f5d9c0b3120 msgr2=0x7f5d9c0b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:32.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 -- 172.21.15.46:0/855644446 shutdown_connections 2024-04-25T08:52:32.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 -- 172.21.15.46:0/855644446 wait complete. 2024-04-25T08:52:32.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 Processor -- start 2024-04-25T08:52:32.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.334+0000 7f5da4800640 1 -- start start 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da4800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c09ab50 0x7f5d9c06bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da4800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c15ef20 0x7f5d9c09c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da4800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09cbf0 0x7f5d9c0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da4800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c15f540 con 0x7f5d9c15ef20 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da4800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c09a350 con 0x7f5d9c09ab50 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da4800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c15f6c0 con 0x7f5d9c09cbf0 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c15ef20 0x7f5d9c09c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c15ef20 0x7f5d9c09c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60680/0 (socket says 172.21.15.46:60680) 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1400640 1 -- 172.21.15.46:0/1129993224 learned_addr learned my addr 172.21.15.46:0/1129993224 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da2800640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09cbf0 0x7f5d9c0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1e00640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c09ab50 0x7f5d9c06bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1400640 1 -- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09cbf0 msgr2=0x7f5d9c0ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1400640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09cbf0 0x7f5d9c0ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1400640 1 -- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c09ab50 msgr2=0x7f5d9c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1400640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c09ab50 0x7f5d9c06bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1400640 1 -- 172.21.15.46:0/1129993224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0c9640 con 0x7f5d9c15ef20 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da1400640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c15ef20 0x7f5d9c09c6b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c01aba0 tx=0x7f5d8c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5d96000640 1 -- 172.21.15.46:0/1129993224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0bd5f0 con 0x7f5d9c15ef20 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5d96000640 1 -- 172.21.15.46:0/1129993224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c0bd790 con 0x7f5d9c15ef20 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c0c9920 con 0x7f5d9c15ef20 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5d96000640 1 -- 172.21.15.46:0/1129993224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0bdb80 con 0x7f5d9c15ef20 2024-04-25T08:52:32.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c06ca10 con 0x7f5d9c15ef20 2024-04-25T08:52:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c0c9ec0 con 0x7f5d9c15ef20 2024-04-25T08:52:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.338+0000 7f5d96000640 1 -- 172.21.15.46:0/1129993224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d8c0b33f0 con 0x7f5d9c15ef20 2024-04-25T08:52:32.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.342+0000 7f5d96000640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d80066f60 0x7f5d80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.342+0000 7f5d96000640 1 -- 172.21.15.46:0/1129993224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d8c11fc50 con 0x7f5d9c15ef20 2024-04-25T08:52:32.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.342+0000 7f5da1e00640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d80066f60 0x7f5d80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.342+0000 7f5da1e00640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d80066f60 0x7f5d80069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b930 tx=0x7f5d90008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:32.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.346+0000 7f5d96000640 1 -- 172.21.15.46:0/1129993224 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d8c0c3060 con 0x7f5d9c15ef20 2024-04-25T08:52:32.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.506+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d9c0d0100 con 0x7f5d9c15ef20 2024-04-25T08:52:32.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5d96000640 1 -- 172.21.15.46:0/1129993224 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16904 (secure 0 0 0) 0x7f5d8c016110 con 0x7f5d9c15ef20 2024-04-25T08:52:32.500 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:32.500 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:29.760504+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:32.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d80066f60 msgr2=0x7f5d80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d80066f60 0x7f5d80069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000b930 tx=0x7f5d90008040 comp rx=0 tx=0).stop 2024-04-25T08:52:32.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c15ef20 msgr2=0x7f5d9c09c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c15ef20 0x7f5d9c09c6b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c01aba0 tx=0x7f5d8c01ad30 comp rx=0 tx=0).stop 2024-04-25T08:52:32.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 shutdown_connections 2024-04-25T08:52:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d80066f60 0x7f5d80069420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d9c09cbf0 0x7f5d9c0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d9c15ef20 0x7f5d9c09c6b0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 --2- 172.21.15.46:0/1129993224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d9c09ab50 0x7f5d9c06bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 >> 172.21.15.46:0/1129993224 conn(0x7f5d9c0b3120 msgr2=0x7f5d9c0c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 shutdown_connections 2024-04-25T08:52:32.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.510+0000 7f5da4800640 1 -- 172.21.15.46:0/1129993224 wait complete. 2024-04-25T08:52:32.522 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:52:32.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:52:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.670+0000 7f1e3c000640 1 Processor -- start 2024-04-25T08:52:32.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.670+0000 7f1e3c000640 1 -- start start 2024-04-25T08:52:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.670+0000 7f1e3c000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1e3406db40 0x7f1e3406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.670+0000 7f1e3c000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e340b3550 0x7f1e3406b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.670+0000 7f1e3c000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3415e180 con 0x7f1e340af3f0 2024-04-25T08:52:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.670+0000 7f1e3c000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e340b3ca0 con 0x7f1e3406db40 2024-04-25T08:52:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.670+0000 7f1e3c000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3415e460 con 0x7f1e340b3550 2024-04-25T08:52:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e38c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1e3406db40 0x7f1e3406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e38c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1e3406db40 0x7f1e3406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60692/0 (socket says 172.21.15.46:60692) 2024-04-25T08:52:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e38c00640 1 -- 172.21.15.46:0/2668598047 learned_addr learned my addr 172.21.15.46:0/2668598047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e3a000640 1 --2- 172.21.15.46:0/2668598047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e340b3550 0x7f1e3406b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e38c00640 1 -- 172.21.15.46:0/2668598047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e340b3550 msgr2=0x7f1e3406b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e38c00640 1 --2- 172.21.15.46:0/2668598047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e340b3550 0x7f1e3406b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e38c00640 1 -- 172.21.15.46:0/2668598047 >> v1:172.21.15.82:6789/0 conn(0x7f1e340af3f0 legacy=0x7f1e340b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e38c00640 1 -- 172.21.15.46:0/2668598047 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3415e740 con 0x7f1e3406db40 2024-04-25T08:52:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e38c00640 1 --2- 172.21.15.46:0/2668598047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e3406db40 0x7f1e3406ac10 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009870 tx=0x7f1e2001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e966623cde7199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e33e00640 1 -- 172.21.15.46:0/2668598047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200b2070 con 0x7f1e3406db40 2024-04-25T08:52:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e33e00640 1 -- 172.21.15.46:0/2668598047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e200a9070 con 0x7f1e3406db40 2024-04-25T08:52:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.674+0000 7f1e33e00640 1 -- 172.21.15.46:0/2668598047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200ad880 con 0x7f1e3406db40 2024-04-25T08:52:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 -- 172.21.15.46:0/2668598047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e3406db40 msgr2=0x7f1e3406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 --2- 172.21.15.46:0/2668598047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e3406db40 0x7f1e3406ac10 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009870 tx=0x7f1e2001c560 comp rx=0 tx=0).stop 2024-04-25T08:52:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 -- 172.21.15.46:0/2668598047 shutdown_connections 2024-04-25T08:52:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 --2- 172.21.15.46:0/2668598047 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e340b3550 0x7f1e3406b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 --2- 172.21.15.46:0/2668598047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e3406db40 0x7f1e3406ac10 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 -- 172.21.15.46:0/2668598047 >> 172.21.15.46:0/2668598047 conn(0x7f1e340cfd90 msgr2=0x7f1e340d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 -- 172.21.15.46:0/2668598047 shutdown_connections 2024-04-25T08:52:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 -- 172.21.15.46:0/2668598047 wait complete. 2024-04-25T08:52:32.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 Processor -- start 2024-04-25T08:52:32.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 -- start start 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e3406db40 0x7f1e340cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e340af3f0 0x7f1e34154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e340b3550 0x7f1e340b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3416a870 con 0x7f1e340af3f0 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3416a6f0 con 0x7f1e340b3550 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3c000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3416a9f0 con 0x7f1e3406db40 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e38c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e340af3f0 0x7f1e34154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e38c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e340af3f0 0x7f1e34154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60700/0 (socket says 172.21.15.46:60700) 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e38c00640 1 -- 172.21.15.46:0/1194898990 learned_addr learned my addr 172.21.15.46:0/1194898990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e39600640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e3406db40 0x7f1e340cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e3a000640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e340b3550 0x7f1e340b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e38c00640 1 -- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e3406db40 msgr2=0x7f1e340cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e38c00640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e3406db40 0x7f1e340cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e38c00640 1 -- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e340b3550 msgr2=0x7f1e340b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e38c00640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e340b3550 0x7f1e340b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e38c00640 1 -- 172.21.15.46:0/1194898990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e340cee50 con 0x7f1e340af3f0 2024-04-25T08:52:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.678+0000 7f1e38c00640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e340af3f0 0x7f1e34154550 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f1e200b0040 tx=0x7f1e20019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e31600640 1 -- 172.21.15.46:0/1194898990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200bb070 con 0x7f1e340af3f0 2024-04-25T08:52:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e31600640 1 -- 172.21.15.46:0/1194898990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e200add40 con 0x7f1e340af3f0 2024-04-25T08:52:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e31600640 1 -- 172.21.15.46:0/1194898990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200b2040 con 0x7f1e340af3f0 2024-04-25T08:52:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e340cf0f0 con 0x7f1e340af3f0 2024-04-25T08:52:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e340ccd00 con 0x7f1e340af3f0 2024-04-25T08:52:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e340cf5a0 con 0x7f1e340af3f0 2024-04-25T08:52:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e31600640 1 -- 172.21.15.46:0/1194898990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e200ab070 con 0x7f1e340af3f0 2024-04-25T08:52:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e31600640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e14066f90 0x7f1e14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e31600640 1 -- 172.21.15.46:0/1194898990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e2011dfb0 con 0x7f1e340af3f0 2024-04-25T08:52:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.682+0000 7f1e39600640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e14066f90 0x7f1e14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.686+0000 7f1e39600640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e14066f90 0x7f1e14069450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000f70 tx=0x7f1e28009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:32.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.686+0000 7f1e31600640 1 -- 172.21.15.46:0/1194898990 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e20016150 con 0x7f1e340af3f0 2024-04-25T08:52:32.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.846+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e34069080 con 0x7f1e14066f90 2024-04-25T08:52:32.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.854+0000 7f1e31600640 1 -- 172.21.15.46:0/1194898990 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7f1e34069080 con 0x7f1e14066f90 2024-04-25T08:52:32.848 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:32.850 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:52:32.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e14066f90 msgr2=0x7f1e14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e14066f90 0x7f1e14069450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000f70 tx=0x7f1e28009040 comp rx=0 tx=0).stop 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e340af3f0 msgr2=0x7f1e34154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e340af3f0 0x7f1e34154550 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f1e200b0040 tx=0x7f1e20019300 comp rx=0 tx=0).stop 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 shutdown_connections 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1e14066f90 0x7f1e14069450 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e340b3550 0x7f1e340b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e340af3f0 0x7f1e34154550 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 --2- 172.21.15.46:0/1194898990 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e3406db40 0x7f1e340cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 >> 172.21.15.46:0/1194898990 conn(0x7f1e340cfd90 msgr2=0x7f1e34155540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 shutdown_connections 2024-04-25T08:52:32.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:32.858+0000 7f1e3c000640 1 -- 172.21.15.46:0/1194898990 wait complete. 2024-04-25T08:52:32.868 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-04-25T08:52:32.411358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262468,"kb_used_data":13000,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712252,"statfs":{"total":773094113280,"available":772825346048,"internally_reserved":0,"allocated":13312000,"data_stored":6826148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.665630"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772771+0000","last_change":"2024-04-25T08:52:28.796407+0000","last_active":"2024-04-25T08:52:29.772771+0000","last_peered":"2024-04-25T08:52:29.772771+0000","last_clean":"2024-04-25T08:52:29.772771+0000","last_became_active":"2024-04-25T08:52:28.796160+0000","last_became_peered":"2024-04-25T08:52:28.796160+0000","last_unstale":"2024-04-25T08:52:29.772771+0000","last_undegraded":"2024-04-25T08:52:29.772771+0000","last_fullsized":"2024-04-25T08:52:29.772771+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:46.781517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687548+0000","last_change":"2024-04-25T08:52:28.783204+0000","last_active":"2024-04-25T08:52:30.687548+0000","last_peered":"2024-04-25T08:52:30.687548+0000","last_clean":"2024-04-25T08:52:30.687548+0000","last_became_active":"2024-04-25T08:52:28.782923+0000","last_became_peered":"2024-04-25T08:52:28.782923+0000","last_unstale":"2024-04-25T08:52:30.687548+0000","last_undegraded":"2024-04-25T08:52:30.687548+0000","last_fullsized":"2024-04-25T08:52:30.687548+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:25.560105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687562+0000","last_change":"2024-04-25T08:52:28.777910+0000","last_active":"2024-04-25T08:52:30.687562+0000","last_peered":"2024-04-25T08:52:30.687562+0000","last_clean":"2024-04-25T08:52:30.687562+0000","last_became_active":"2024-04-25T08:52:28.777616+0000","last_became_peered":"2024-04-25T08:52:28.777616+0000","last_unstale":"2024-04-25T08:52:30.687562+0000","last_undegraded":"2024-04-25T08:52:30.687562+0000","last_fullsized":"2024-04-25T08:52:30.687562+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:40.330466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771563+0000","last_change":"2024-04-25T08:52:28.794781+0000","last_active":"2024-04-25T08:52:29.771563+0000","last_peered":"2024-04-25T08:52:29.771563+0000","last_clean":"2024-04-25T08:52:29.771563+0000","last_became_active":"2024-04-25T08:52:28.793692+0000","last_became_peered":"2024-04-25T08:52:28.793692+0000","last_unstale":"2024-04-25T08:52:29.771563+0000","last_undegraded":"2024-04-25T08:52:29.771563+0000","last_fullsized":"2024-04-25T08:52:29.771563+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:40.351417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771551+0000","last_change":"2024-04-25T08:52:28.792340+0000","last_active":"2024-04-25T08:52:29.771551+0000","last_peered":"2024-04-25T08:52:29.771551+0000","last_clean":"2024-04-25T08:52:29.771551+0000","last_became_active":"2024-04-25T08:52:28.792042+0000","last_became_peered":"2024-04-25T08:52:28.792042+0000","last_unstale":"2024-04-25T08:52:29.771551+0000","last_undegraded":"2024-04-25T08:52:29.771551+0000","last_fullsized":"2024-04-25T08:52:29.771551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:38.118504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767736+0000","last_change":"2024-04-25T08:52:28.805339+0000","last_active":"2024-04-25T08:52:29.767736+0000","last_peered":"2024-04-25T08:52:29.767736+0000","last_clean":"2024-04-25T08:52:29.767736+0000","last_became_active":"2024-04-25T08:52:28.805001+0000","last_became_peered":"2024-04-25T08:52:28.805001+0000","last_unstale":"2024-04-25T08:52:29.767736+0000","last_undegraded":"2024-04-25T08:52:29.767736+0000","last_fullsized":"2024-04-25T08:52:29.767736+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:07.435476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767452+0000","last_change":"2024-04-25T08:52:28.781489+0000","last_active":"2024-04-25T08:52:29.767452+0000","last_peered":"2024-04-25T08:52:29.767452+0000","last_clean":"2024-04-25T08:52:29.767452+0000","last_became_active":"2024-04-25T08:52:28.781106+0000","last_became_peered":"2024-04-25T08:52:28.781106+0000","last_unstale":"2024-04-25T08:52:29.767452+0000","last_undegraded":"2024-04-25T08:52:29.767452+0000","last_fullsized":"2024-04-25T08:52:29.767452+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:49.454655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686736+0000","last_change":"2024-04-25T08:52:28.781819+0000","last_active":"2024-04-25T08:52:30.686736+0000","last_peered":"2024-04-25T08:52:30.686736+0000","last_clean":"2024-04-25T08:52:30.686736+0000","last_became_active":"2024-04-25T08:52:28.778319+0000","last_became_peered":"2024-04-25T08:52:28.778319+0000","last_unstale":"2024-04-25T08:52:30.686736+0000","last_undegraded":"2024-04-25T08:52:30.686736+0000","last_fullsized":"2024-04-25T08:52:30.686736+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:47.330267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772055+0000","last_change":"2024-04-25T08:52:28.811092+0000","last_active":"2024-04-25T08:52:29.772055+0000","last_peered":"2024-04-25T08:52:29.772055+0000","last_clean":"2024-04-25T08:52:29.772055+0000","last_became_active":"2024-04-25T08:52:28.810786+0000","last_became_peered":"2024-04-25T08:52:28.810786+0000","last_unstale":"2024-04-25T08:52:29.772055+0000","last_undegraded":"2024-04-25T08:52:29.772055+0000","last_fullsized":"2024-04-25T08:52:29.772055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:52.042338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765506+0000","last_change":"2024-04-25T08:52:28.793586+0000","last_active":"2024-04-25T08:52:29.765506+0000","last_peered":"2024-04-25T08:52:29.765506+0000","last_clean":"2024-04-25T08:52:29.765506+0000","last_became_active":"2024-04-25T08:52:28.793253+0000","last_became_peered":"2024-04-25T08:52:28.793253+0000","last_unstale":"2024-04-25T08:52:29.765506+0000","last_undegraded":"2024-04-25T08:52:29.765506+0000","last_fullsized":"2024-04-25T08:52:29.765506+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:46.781517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772529+0000","last_change":"2024-04-25T08:52:28.792673+0000","last_active":"2024-04-25T08:52:29.772529+0000","last_peered":"2024-04-25T08:52:29.772529+0000","last_clean":"2024-04-25T08:52:29.772529+0000","last_became_active":"2024-04-25T08:52:28.792335+0000","last_became_peered":"2024-04-25T08:52:28.792335+0000","last_unstale":"2024-04-25T08:52:29.772529+0000","last_undegraded":"2024-04-25T08:52:29.772529+0000","last_fullsized":"2024-04-25T08:52:29.772529+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:00.700601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767785+0000","last_change":"2024-04-25T08:52:28.790192+0000","last_active":"2024-04-25T08:52:29.767785+0000","last_peered":"2024-04-25T08:52:29.767785+0000","last_clean":"2024-04-25T08:52:29.767785+0000","last_became_active":"2024-04-25T08:52:28.789857+0000","last_became_peered":"2024-04-25T08:52:28.789857+0000","last_unstale":"2024-04-25T08:52:29.767785+0000","last_undegraded":"2024-04-25T08:52:29.767785+0000","last_fullsized":"2024-04-25T08:52:29.767785+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:28.270255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765196+0000","last_change":"2024-04-25T08:52:28.789117+0000","last_active":"2024-04-25T08:52:29.765196+0000","last_peered":"2024-04-25T08:52:29.765196+0000","last_clean":"2024-04-25T08:52:29.765196+0000","last_became_active":"2024-04-25T08:52:28.788796+0000","last_became_peered":"2024-04-25T08:52:28.788796+0000","last_unstale":"2024-04-25T08:52:29.765196+0000","last_undegraded":"2024-04-25T08:52:29.765196+0000","last_fullsized":"2024-04-25T08:52:29.765196+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:38.118504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687830+0000","last_change":"2024-04-25T08:52:28.784288+0000","last_active":"2024-04-25T08:52:30.687830+0000","last_peered":"2024-04-25T08:52:30.687830+0000","last_clean":"2024-04-25T08:52:30.687830+0000","last_became_active":"2024-04-25T08:52:28.777415+0000","last_became_peered":"2024-04-25T08:52:28.777415+0000","last_unstale":"2024-04-25T08:52:30.687830+0000","last_undegraded":"2024-04-25T08:52:30.687830+0000","last_fullsized":"2024-04-25T08:52:30.687830+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:28.268710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771540+0000","last_change":"2024-04-25T08:52:28.793355+0000","last_active":"2024-04-25T08:52:29.771540+0000","last_peered":"2024-04-25T08:52:29.771540+0000","last_clean":"2024-04-25T08:52:29.771540+0000","last_became_active":"2024-04-25T08:52:28.792994+0000","last_became_peered":"2024-04-25T08:52:28.792994+0000","last_unstale":"2024-04-25T08:52:29.771540+0000","last_undegraded":"2024-04-25T08:52:29.771540+0000","last_fullsized":"2024-04-25T08:52:29.771540+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:22.564053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767365+0000","last_change":"2024-04-25T08:52:28.793383+0000","last_active":"2024-04-25T08:52:29.767365+0000","last_peered":"2024-04-25T08:52:29.767365+0000","last_clean":"2024-04-25T08:52:29.767365+0000","last_became_active":"2024-04-25T08:52:28.793037+0000","last_became_peered":"2024-04-25T08:52:28.793037+0000","last_unstale":"2024-04-25T08:52:29.767365+0000","last_undegraded":"2024-04-25T08:52:29.767365+0000","last_fullsized":"2024-04-25T08:52:29.767365+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:44.934583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765564+0000","last_change":"2024-04-25T08:52:28.787029+0000","last_active":"2024-04-25T08:52:29.765564+0000","last_peered":"2024-04-25T08:52:29.765564+0000","last_clean":"2024-04-25T08:52:29.765564+0000","last_became_active":"2024-04-25T08:52:28.786288+0000","last_became_peered":"2024-04-25T08:52:28.786288+0000","last_unstale":"2024-04-25T08:52:29.765564+0000","last_undegraded":"2024-04-25T08:52:29.765564+0000","last_fullsized":"2024-04-25T08:52:29.765564+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:39.452095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687550+0000","last_change":"2024-04-25T08:52:28.782977+0000","last_active":"2024-04-25T08:52:30.687550+0000","last_peered":"2024-04-25T08:52:30.687550+0000","last_clean":"2024-04-25T08:52:30.687550+0000","last_became_active":"2024-04-25T08:52:28.782697+0000","last_became_peered":"2024-04-25T08:52:28.782697+0000","last_unstale":"2024-04-25T08:52:30.687550+0000","last_undegraded":"2024-04-25T08:52:30.687550+0000","last_fullsized":"2024-04-25T08:52:30.687550+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:25.027253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791315+0000","last_change":"2024-04-25T08:52:28.787742+0000","last_active":"2024-04-25T08:52:29.791315+0000","last_peered":"2024-04-25T08:52:29.791315+0000","last_clean":"2024-04-25T08:52:29.791315+0000","last_became_active":"2024-04-25T08:52:28.787455+0000","last_became_peered":"2024-04-25T08:52:28.787455+0000","last_unstale":"2024-04-25T08:52:29.791315+0000","last_undegraded":"2024-04-25T08:52:29.791315+0000","last_fullsized":"2024-04-25T08:52:29.791315+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:22.564053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686747+0000","last_change":"2024-04-25T08:52:28.788421+0000","last_active":"2024-04-25T08:52:30.686747+0000","last_peered":"2024-04-25T08:52:30.686747+0000","last_clean":"2024-04-25T08:52:30.686747+0000","last_became_active":"2024-04-25T08:52:28.788126+0000","last_became_peered":"2024-04-25T08:52:28.788126+0000","last_unstale":"2024-04-25T08:52:30.686747+0000","last_undegraded":"2024-04-25T08:52:30.686747+0000","last_fullsized":"2024-04-25T08:52:30.686747+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:19.325601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771568+0000","last_change":"2024-04-25T08:52:28.801404+0000","last_active":"2024-04-25T08:52:29.771568+0000","last_peered":"2024-04-25T08:52:29.771568+0000","last_clean":"2024-04-25T08:52:29.771568+0000","last_became_active":"2024-04-25T08:52:28.801117+0000","last_became_peered":"2024-04-25T08:52:28.801117+0000","last_unstale":"2024-04-25T08:52:29.771568+0000","last_undegraded":"2024-04-25T08:52:29.771568+0000","last_fullsized":"2024-04-25T08:52:29.771568+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:45.883389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772012+0000","last_change":"2024-04-25T08:52:28.807114+0000","last_active":"2024-04-25T08:52:29.772012+0000","last_peered":"2024-04-25T08:52:29.772012+0000","last_clean":"2024-04-25T08:52:29.772012+0000","last_became_active":"2024-04-25T08:52:28.806796+0000","last_became_peered":"2024-04-25T08:52:28.806796+0000","last_unstale":"2024-04-25T08:52:29.772012+0000","last_undegraded":"2024-04-25T08:52:29.772012+0000","last_fullsized":"2024-04-25T08:52:29.772012+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:05.210040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771912+0000","last_change":"2024-04-25T08:52:28.807125+0000","last_active":"2024-04-25T08:52:29.771912+0000","last_peered":"2024-04-25T08:52:29.771912+0000","last_clean":"2024-04-25T08:52:29.771912+0000","last_became_active":"2024-04-25T08:52:28.806802+0000","last_became_peered":"2024-04-25T08:52:28.806802+0000","last_unstale":"2024-04-25T08:52:29.771912+0000","last_undegraded":"2024-04-25T08:52:29.771912+0000","last_fullsized":"2024-04-25T08:52:29.771912+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:06.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689395+0000","last_change":"2024-04-25T08:52:28.780541+0000","last_active":"2024-04-25T08:52:30.689395+0000","last_peered":"2024-04-25T08:52:30.689395+0000","last_clean":"2024-04-25T08:52:30.689395+0000","last_became_active":"2024-04-25T08:52:28.780240+0000","last_became_peered":"2024-04-25T08:52:28.780240+0000","last_unstale":"2024-04-25T08:52:30.689395+0000","last_undegraded":"2024-04-25T08:52:30.689395+0000","last_fullsized":"2024-04-25T08:52:30.689395+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:40.680311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689797+0000","last_change":"2024-04-25T08:52:28.790369+0000","last_active":"2024-04-25T08:52:30.689797+0000","last_peered":"2024-04-25T08:52:30.689797+0000","last_clean":"2024-04-25T08:52:30.689797+0000","last_became_active":"2024-04-25T08:52:28.790106+0000","last_became_peered":"2024-04-25T08:52:28.790106+0000","last_unstale":"2024-04-25T08:52:30.689797+0000","last_undegraded":"2024-04-25T08:52:30.689797+0000","last_fullsized":"2024-04-25T08:52:30.689797+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:19.831604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686984+0000","last_change":"2024-04-25T08:52:28.789897+0000","last_active":"2024-04-25T08:52:30.686984+0000","last_peered":"2024-04-25T08:52:30.686984+0000","last_clean":"2024-04-25T08:52:30.686984+0000","last_became_active":"2024-04-25T08:52:28.789587+0000","last_became_peered":"2024-04-25T08:52:28.789587+0000","last_unstale":"2024-04-25T08:52:30.686984+0000","last_undegraded":"2024-04-25T08:52:30.686984+0000","last_fullsized":"2024-04-25T08:52:30.686984+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:11.608445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791423+0000","last_change":"2024-04-25T08:52:28.781138+0000","last_active":"2024-04-25T08:52:29.791423+0000","last_peered":"2024-04-25T08:52:29.791423+0000","last_clean":"2024-04-25T08:52:29.791423+0000","last_became_active":"2024-04-25T08:52:28.780610+0000","last_became_peered":"2024-04-25T08:52:28.780610+0000","last_unstale":"2024-04-25T08:52:29.791423+0000","last_undegraded":"2024-04-25T08:52:29.791423+0000","last_fullsized":"2024-04-25T08:52:29.791423+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:38.118504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767774+0000","last_change":"2024-04-25T08:52:28.790741+0000","last_active":"2024-04-25T08:52:29.767774+0000","last_peered":"2024-04-25T08:52:29.767774+0000","last_clean":"2024-04-25T08:52:29.767774+0000","last_became_active":"2024-04-25T08:52:28.790407+0000","last_became_peered":"2024-04-25T08:52:28.790407+0000","last_unstale":"2024-04-25T08:52:29.767774+0000","last_undegraded":"2024-04-25T08:52:29.767774+0000","last_fullsized":"2024-04-25T08:52:29.767774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:18.237461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687004+0000","last_change":"2024-04-25T08:52:28.801910+0000","last_active":"2024-04-25T08:52:30.687004+0000","last_peered":"2024-04-25T08:52:30.687004+0000","last_clean":"2024-04-25T08:52:30.687004+0000","last_became_active":"2024-04-25T08:52:28.801369+0000","last_became_peered":"2024-04-25T08:52:28.801369+0000","last_unstale":"2024-04-25T08:52:30.687004+0000","last_undegraded":"2024-04-25T08:52:30.687004+0000","last_fullsized":"2024-04-25T08:52:30.687004+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:05.774385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687774+0000","last_change":"2024-04-25T08:52:28.787169+0000","last_active":"2024-04-25T08:52:30.687774+0000","last_peered":"2024-04-25T08:52:30.687774+0000","last_clean":"2024-04-25T08:52:30.687774+0000","last_became_active":"2024-04-25T08:52:28.786818+0000","last_became_peered":"2024-04-25T08:52:28.786818+0000","last_unstale":"2024-04-25T08:52:30.687774+0000","last_undegraded":"2024-04-25T08:52:30.687774+0000","last_fullsized":"2024-04-25T08:52:30.687774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:48.576685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689476+0000","last_change":"2024-04-25T08:52:28.789637+0000","last_active":"2024-04-25T08:52:30.689476+0000","last_peered":"2024-04-25T08:52:30.689476+0000","last_clean":"2024-04-25T08:52:30.689476+0000","last_became_active":"2024-04-25T08:52:28.788725+0000","last_became_peered":"2024-04-25T08:52:28.788725+0000","last_unstale":"2024-04-25T08:52:30.689476+0000","last_undegraded":"2024-04-25T08:52:30.689476+0000","last_fullsized":"2024-04-25T08:52:30.689476+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:28.294422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689404+0000","last_change":"2024-04-25T08:52:28.779128+0000","last_active":"2024-04-25T08:52:30.689404+0000","last_peered":"2024-04-25T08:52:30.689404+0000","last_clean":"2024-04-25T08:52:30.689404+0000","last_became_active":"2024-04-25T08:52:28.778735+0000","last_became_peered":"2024-04-25T08:52:28.778735+0000","last_unstale":"2024-04-25T08:52:30.689404+0000","last_undegraded":"2024-04-25T08:52:30.689404+0000","last_fullsized":"2024-04-25T08:52:30.689404+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:05.210040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765312+0000","last_change":"2024-04-25T08:52:28.800939+0000","last_active":"2024-04-25T08:52:29.765312+0000","last_peered":"2024-04-25T08:52:29.765312+0000","last_clean":"2024-04-25T08:52:29.765312+0000","last_became_active":"2024-04-25T08:52:28.796551+0000","last_became_peered":"2024-04-25T08:52:28.796551+0000","last_unstale":"2024-04-25T08:52:29.765312+0000","last_undegraded":"2024-04-25T08:52:29.765312+0000","last_fullsized":"2024-04-25T08:52:29.765312+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:28.294422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791288+0000","last_change":"2024-04-25T08:52:27.759922+0000","last_active":"2024-04-25T08:52:29.791288+0000","last_peered":"2024-04-25T08:52:29.791288+0000","last_clean":"2024-04-25T08:52:29.791288+0000","last_became_active":"2024-04-25T08:52:27.759618+0000","last_became_peered":"2024-04-25T08:52:27.759618+0000","last_unstale":"2024-04-25T08:52:29.791288+0000","last_undegraded":"2024-04-25T08:52:29.791288+0000","last_fullsized":"2024-04-25T08:52:29.791288+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:30.432898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767768+0000","last_change":"2024-04-25T08:52:28.804623+0000","last_active":"2024-04-25T08:52:29.767768+0000","last_peered":"2024-04-25T08:52:29.767768+0000","last_clean":"2024-04-25T08:52:29.767768+0000","last_became_active":"2024-04-25T08:52:28.804195+0000","last_became_peered":"2024-04-25T08:52:28.804195+0000","last_unstale":"2024-04-25T08:52:29.767768+0000","last_undegraded":"2024-04-25T08:52:29.767768+0000","last_fullsized":"2024-04-25T08:52:29.767768+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:02.043215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765432+0000","last_change":"2024-04-25T08:52:27.771247+0000","last_active":"2024-04-25T08:52:29.765432+0000","last_peered":"2024-04-25T08:52:29.765432+0000","last_clean":"2024-04-25T08:52:29.765432+0000","last_became_active":"2024-04-25T08:52:27.770975+0000","last_became_peered":"2024-04-25T08:52:27.770975+0000","last_unstale":"2024-04-25T08:52:29.765432+0000","last_undegraded":"2024-04-25T08:52:29.765432+0000","last_fullsized":"2024-04-25T08:52:29.765432+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:08.697698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689599+0000","last_change":"2024-04-25T08:52:28.787762+0000","last_active":"2024-04-25T08:52:30.689599+0000","last_peered":"2024-04-25T08:52:30.689599+0000","last_clean":"2024-04-25T08:52:30.689599+0000","last_became_active":"2024-04-25T08:52:28.787213+0000","last_became_peered":"2024-04-25T08:52:28.787213+0000","last_unstale":"2024-04-25T08:52:30.689599+0000","last_undegraded":"2024-04-25T08:52:30.689599+0000","last_fullsized":"2024-04-25T08:52:30.689599+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:46.781517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772236+0000","last_change":"2024-04-25T08:52:27.769244+0000","last_active":"2024-04-25T08:52:29.772236+0000","last_peered":"2024-04-25T08:52:29.772236+0000","last_clean":"2024-04-25T08:52:29.772236+0000","last_became_active":"2024-04-25T08:52:27.768944+0000","last_became_peered":"2024-04-25T08:52:27.768944+0000","last_unstale":"2024-04-25T08:52:29.772236+0000","last_undegraded":"2024-04-25T08:52:29.772236+0000","last_fullsized":"2024-04-25T08:52:29.772236+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:39.673455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772202+0000","last_change":"2024-04-25T08:52:28.801317+0000","last_active":"2024-04-25T08:52:29.772202+0000","last_peered":"2024-04-25T08:52:29.772202+0000","last_clean":"2024-04-25T08:52:29.772202+0000","last_became_active":"2024-04-25T08:52:28.800998+0000","last_became_peered":"2024-04-25T08:52:28.800998+0000","last_unstale":"2024-04-25T08:52:29.772202+0000","last_undegraded":"2024-04-25T08:52:29.772202+0000","last_fullsized":"2024-04-25T08:52:29.772202+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:43.632419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765197+0000","last_change":"2024-04-25T08:52:27.776310+0000","last_active":"2024-04-25T08:52:29.765197+0000","last_peered":"2024-04-25T08:52:29.765197+0000","last_clean":"2024-04-25T08:52:29.765197+0000","last_became_active":"2024-04-25T08:52:27.775485+0000","last_became_peered":"2024-04-25T08:52:27.775485+0000","last_unstale":"2024-04-25T08:52:29.765197+0000","last_undegraded":"2024-04-25T08:52:29.765197+0000","last_fullsized":"2024-04-25T08:52:29.765197+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:32.893999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.764830+0000","last_change":"2024-04-25T08:52:28.769312+0000","last_active":"2024-04-25T08:52:29.764830+0000","last_peered":"2024-04-25T08:52:29.764830+0000","last_clean":"2024-04-25T08:52:29.764830+0000","last_became_active":"2024-04-25T08:52:28.768639+0000","last_became_peered":"2024-04-25T08:52:28.768639+0000","last_unstale":"2024-04-25T08:52:29.764830+0000","last_undegraded":"2024-04-25T08:52:29.764830+0000","last_fullsized":"2024-04-25T08:52:29.764830+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:52.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771619+0000","last_change":"2024-04-25T08:52:27.780766+0000","last_active":"2024-04-25T08:52:29.771619+0000","last_peered":"2024-04-25T08:52:29.771619+0000","last_clean":"2024-04-25T08:52:29.771619+0000","last_became_active":"2024-04-25T08:52:27.780475+0000","last_became_peered":"2024-04-25T08:52:27.780475+0000","last_unstale":"2024-04-25T08:52:29.771619+0000","last_undegraded":"2024-04-25T08:52:29.771619+0000","last_fullsized":"2024-04-25T08:52:29.771619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:24.477694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771710+0000","last_change":"2024-04-25T08:52:28.787847+0000","last_active":"2024-04-25T08:52:29.771710+0000","last_peered":"2024-04-25T08:52:29.771710+0000","last_clean":"2024-04-25T08:52:29.771710+0000","last_became_active":"2024-04-25T08:52:28.787584+0000","last_became_peered":"2024-04-25T08:52:28.787584+0000","last_unstale":"2024-04-25T08:52:29.771710+0000","last_undegraded":"2024-04-25T08:52:29.771710+0000","last_fullsized":"2024-04-25T08:52:29.771710+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:40.680311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767808+0000","last_change":"2024-04-25T08:52:27.774079+0000","last_active":"2024-04-25T08:52:29.767808+0000","last_peered":"2024-04-25T08:52:29.767808+0000","last_clean":"2024-04-25T08:52:29.767808+0000","last_became_active":"2024-04-25T08:52:27.773822+0000","last_became_peered":"2024-04-25T08:52:27.773822+0000","last_unstale":"2024-04-25T08:52:29.767808+0000","last_undegraded":"2024-04-25T08:52:29.767808+0000","last_fullsized":"2024-04-25T08:52:29.767808+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.989380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687882+0000","last_change":"2024-04-25T08:52:28.773260+0000","last_active":"2024-04-25T08:52:30.687882+0000","last_peered":"2024-04-25T08:52:30.687882+0000","last_clean":"2024-04-25T08:52:30.687882+0000","last_became_active":"2024-04-25T08:52:28.771289+0000","last_became_peered":"2024-04-25T08:52:28.771289+0000","last_unstale":"2024-04-25T08:52:30.687882+0000","last_undegraded":"2024-04-25T08:52:30.687882+0000","last_fullsized":"2024-04-25T08:52:30.687882+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:43.398832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689552+0000","last_change":"2024-04-25T08:52:27.774534+0000","last_active":"2024-04-25T08:52:30.689552+0000","last_peered":"2024-04-25T08:52:30.689552+0000","last_clean":"2024-04-25T08:52:30.689552+0000","last_became_active":"2024-04-25T08:52:27.774263+0000","last_became_peered":"2024-04-25T08:52:27.774263+0000","last_unstale":"2024-04-25T08:52:30.689552+0000","last_undegraded":"2024-04-25T08:52:30.689552+0000","last_fullsized":"2024-04-25T08:52:30.689552+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:08.697698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765206+0000","last_change":"2024-04-25T08:52:28.772399+0000","last_active":"2024-04-25T08:52:29.765206+0000","last_peered":"2024-04-25T08:52:29.765206+0000","last_clean":"2024-04-25T08:52:29.765206+0000","last_became_active":"2024-04-25T08:52:28.772116+0000","last_became_peered":"2024-04-25T08:52:28.772116+0000","last_unstale":"2024-04-25T08:52:29.765206+0000","last_undegraded":"2024-04-25T08:52:29.765206+0000","last_fullsized":"2024-04-25T08:52:29.765206+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:40.680311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767449+0000","last_change":"2024-04-25T08:52:27.778982+0000","last_active":"2024-04-25T08:52:29.767449+0000","last_peered":"2024-04-25T08:52:29.767449+0000","last_clean":"2024-04-25T08:52:29.767449+0000","last_became_active":"2024-04-25T08:52:27.778686+0000","last_became_peered":"2024-04-25T08:52:27.778686+0000","last_unstale":"2024-04-25T08:52:29.767449+0000","last_undegraded":"2024-04-25T08:52:29.767449+0000","last_fullsized":"2024-04-25T08:52:29.767449+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:40.752776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791128+0000","last_change":"2024-04-25T08:52:28.781774+0000","last_active":"2024-04-25T08:52:29.791128+0000","last_peered":"2024-04-25T08:52:29.791128+0000","last_clean":"2024-04-25T08:52:29.791128+0000","last_became_active":"2024-04-25T08:52:28.781512+0000","last_became_peered":"2024-04-25T08:52:28.781512+0000","last_unstale":"2024-04-25T08:52:29.791128+0000","last_undegraded":"2024-04-25T08:52:29.791128+0000","last_fullsized":"2024-04-25T08:52:29.791128+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:28.608853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791293+0000","last_change":"2024-04-25T08:52:27.771765+0000","last_active":"2024-04-25T08:52:29.791293+0000","last_peered":"2024-04-25T08:52:29.791293+0000","last_clean":"2024-04-25T08:52:29.791293+0000","last_became_active":"2024-04-25T08:52:27.769985+0000","last_became_peered":"2024-04-25T08:52:27.769985+0000","last_unstale":"2024-04-25T08:52:29.791293+0000","last_undegraded":"2024-04-25T08:52:29.791293+0000","last_fullsized":"2024-04-25T08:52:29.791293+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:52.564237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689710+0000","last_change":"2024-04-25T08:52:28.801178+0000","last_active":"2024-04-25T08:52:30.689710+0000","last_peered":"2024-04-25T08:52:30.689710+0000","last_clean":"2024-04-25T08:52:30.689710+0000","last_became_active":"2024-04-25T08:52:28.800840+0000","last_became_peered":"2024-04-25T08:52:28.800840+0000","last_unstale":"2024-04-25T08:52:30.689710+0000","last_undegraded":"2024-04-25T08:52:30.689710+0000","last_fullsized":"2024-04-25T08:52:30.689710+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:49.117872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689612+0000","last_change":"2024-04-25T08:52:27.774894+0000","last_active":"2024-04-25T08:52:30.689612+0000","last_peered":"2024-04-25T08:52:30.689612+0000","last_clean":"2024-04-25T08:52:30.689612+0000","last_became_active":"2024-04-25T08:52:27.774551+0000","last_became_peered":"2024-04-25T08:52:27.774551+0000","last_unstale":"2024-04-25T08:52:30.689612+0000","last_undegraded":"2024-04-25T08:52:30.689612+0000","last_fullsized":"2024-04-25T08:52:30.689612+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:30.432898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687782+0000","last_change":"2024-04-25T08:52:28.777489+0000","last_active":"2024-04-25T08:52:30.687782+0000","last_peered":"2024-04-25T08:52:30.687782+0000","last_clean":"2024-04-25T08:52:30.687782+0000","last_became_active":"2024-04-25T08:52:28.777181+0000","last_became_peered":"2024-04-25T08:52:28.777181+0000","last_unstale":"2024-04-25T08:52:30.687782+0000","last_undegraded":"2024-04-25T08:52:30.687782+0000","last_fullsized":"2024-04-25T08:52:30.687782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:41.197433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689802+0000","last_change":"2024-04-25T08:52:27.774154+0000","last_active":"2024-04-25T08:52:30.689802+0000","last_peered":"2024-04-25T08:52:30.689802+0000","last_clean":"2024-04-25T08:52:30.689802+0000","last_became_active":"2024-04-25T08:52:27.773331+0000","last_became_peered":"2024-04-25T08:52:27.773331+0000","last_unstale":"2024-04-25T08:52:30.689802+0000","last_undegraded":"2024-04-25T08:52:30.689802+0000","last_fullsized":"2024-04-25T08:52:30.689802+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:24.477694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689899+0000","last_change":"2024-04-25T08:52:28.797964+0000","last_active":"2024-04-25T08:52:30.689899+0000","last_peered":"2024-04-25T08:52:30.689899+0000","last_clean":"2024-04-25T08:52:30.689899+0000","last_became_active":"2024-04-25T08:52:28.797707+0000","last_became_peered":"2024-04-25T08:52:28.797707+0000","last_unstale":"2024-04-25T08:52:30.689899+0000","last_undegraded":"2024-04-25T08:52:30.689899+0000","last_fullsized":"2024-04-25T08:52:30.689899+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.134987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687282+0000","last_change":"2024-04-25T08:52:27.766037+0000","last_active":"2024-04-25T08:52:30.687282+0000","last_peered":"2024-04-25T08:52:30.687282+0000","last_clean":"2024-04-25T08:52:30.687282+0000","last_became_active":"2024-04-25T08:52:27.765712+0000","last_became_peered":"2024-04-25T08:52:27.765712+0000","last_unstale":"2024-04-25T08:52:30.687282+0000","last_undegraded":"2024-04-25T08:52:30.687282+0000","last_fullsized":"2024-04-25T08:52:30.687282+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:45.774662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791213+0000","last_change":"2024-04-25T08:52:28.779728+0000","last_active":"2024-04-25T08:52:29.791213+0000","last_peered":"2024-04-25T08:52:29.791213+0000","last_clean":"2024-04-25T08:52:29.791213+0000","last_became_active":"2024-04-25T08:52:28.779352+0000","last_became_peered":"2024-04-25T08:52:28.779352+0000","last_unstale":"2024-04-25T08:52:29.791213+0000","last_undegraded":"2024-04-25T08:52:29.791213+0000","last_fullsized":"2024-04-25T08:52:29.791213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:52.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687280+0000","last_change":"2024-04-25T08:52:27.755434+0000","last_active":"2024-04-25T08:52:30.687280+0000","last_peered":"2024-04-25T08:52:30.687280+0000","last_clean":"2024-04-25T08:52:30.687280+0000","last_became_active":"2024-04-25T08:52:27.754991+0000","last_became_peered":"2024-04-25T08:52:27.754991+0000","last_unstale":"2024-04-25T08:52:30.687280+0000","last_undegraded":"2024-04-25T08:52:30.687280+0000","last_fullsized":"2024-04-25T08:52:30.687280+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:27.287567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.764959+0000","last_change":"2024-04-25T08:52:28.782221+0000","last_active":"2024-04-25T08:52:29.764959+0000","last_peered":"2024-04-25T08:52:29.764959+0000","last_clean":"2024-04-25T08:52:29.764959+0000","last_became_active":"2024-04-25T08:52:28.781893+0000","last_became_peered":"2024-04-25T08:52:28.781893+0000","last_unstale":"2024-04-25T08:52:29.764959+0000","last_undegraded":"2024-04-25T08:52:29.764959+0000","last_fullsized":"2024-04-25T08:52:29.764959+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:22.564053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687945+0000","last_change":"2024-04-25T08:52:27.769364+0000","last_active":"2024-04-25T08:52:30.687945+0000","last_peered":"2024-04-25T08:52:30.687945+0000","last_clean":"2024-04-25T08:52:30.687945+0000","last_became_active":"2024-04-25T08:52:27.769082+0000","last_became_peered":"2024-04-25T08:52:27.769082+0000","last_unstale":"2024-04-25T08:52:30.687945+0000","last_undegraded":"2024-04-25T08:52:30.687945+0000","last_fullsized":"2024-04-25T08:52:30.687945+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:38.445240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686774+0000","last_change":"2024-04-25T08:52:28.790993+0000","last_active":"2024-04-25T08:52:30.686774+0000","last_peered":"2024-04-25T08:52:30.686774+0000","last_clean":"2024-04-25T08:52:30.686774+0000","last_became_active":"2024-04-25T08:52:28.790700+0000","last_became_peered":"2024-04-25T08:52:28.790700+0000","last_unstale":"2024-04-25T08:52:30.686774+0000","last_undegraded":"2024-04-25T08:52:30.686774+0000","last_fullsized":"2024-04-25T08:52:30.686774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:31.309849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687518+0000","last_change":"2024-04-25T08:52:27.765019+0000","last_active":"2024-04-25T08:52:30.687518+0000","last_peered":"2024-04-25T08:52:30.687518+0000","last_clean":"2024-04-25T08:52:30.687518+0000","last_became_active":"2024-04-25T08:52:27.764761+0000","last_became_peered":"2024-04-25T08:52:27.764761+0000","last_unstale":"2024-04-25T08:52:30.687518+0000","last_undegraded":"2024-04-25T08:52:30.687518+0000","last_fullsized":"2024-04-25T08:52:30.687518+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:18.824748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689612+0000","last_change":"2024-04-25T08:52:28.787436+0000","last_active":"2024-04-25T08:52:30.689612+0000","last_peered":"2024-04-25T08:52:30.689612+0000","last_clean":"2024-04-25T08:52:30.689612+0000","last_became_active":"2024-04-25T08:52:28.787143+0000","last_became_peered":"2024-04-25T08:52:28.787143+0000","last_unstale":"2024-04-25T08:52:30.689612+0000","last_undegraded":"2024-04-25T08:52:30.689612+0000","last_fullsized":"2024-04-25T08:52:30.689612+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:39.452095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765526+0000","last_change":"2024-04-25T08:52:27.776624+0000","last_active":"2024-04-25T08:52:29.765526+0000","last_peered":"2024-04-25T08:52:29.765526+0000","last_clean":"2024-04-25T08:52:29.765526+0000","last_became_active":"2024-04-25T08:52:27.775698+0000","last_became_peered":"2024-04-25T08:52:27.775698+0000","last_unstale":"2024-04-25T08:52:29.765526+0000","last_undegraded":"2024-04-25T08:52:29.765526+0000","last_fullsized":"2024-04-25T08:52:29.765526+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:26.864975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772648+0000","last_change":"2024-04-25T08:52:28.806469+0000","last_active":"2024-04-25T08:52:29.772648+0000","last_peered":"2024-04-25T08:52:29.772648+0000","last_clean":"2024-04-25T08:52:29.772648+0000","last_became_active":"2024-04-25T08:52:28.806170+0000","last_became_peered":"2024-04-25T08:52:28.806170+0000","last_unstale":"2024-04-25T08:52:29.772648+0000","last_undegraded":"2024-04-25T08:52:29.772648+0000","last_fullsized":"2024-04-25T08:52:29.772648+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:28.294422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686969+0000","last_change":"2024-04-25T08:52:27.768179+0000","last_active":"2024-04-25T08:52:30.686969+0000","last_peered":"2024-04-25T08:52:30.686969+0000","last_clean":"2024-04-25T08:52:30.686969+0000","last_became_active":"2024-04-25T08:52:27.767851+0000","last_became_peered":"2024-04-25T08:52:27.767851+0000","last_unstale":"2024-04-25T08:52:30.686969+0000","last_undegraded":"2024-04-25T08:52:30.686969+0000","last_fullsized":"2024-04-25T08:52:30.686969+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:20.059498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767972+0000","last_change":"2024-04-25T08:52:28.789158+0000","last_active":"2024-04-25T08:52:29.767972+0000","last_peered":"2024-04-25T08:52:29.767972+0000","last_clean":"2024-04-25T08:52:29.767972+0000","last_became_active":"2024-04-25T08:52:28.788874+0000","last_became_peered":"2024-04-25T08:52:28.788874+0000","last_unstale":"2024-04-25T08:52:29.767972+0000","last_undegraded":"2024-04-25T08:52:29.767972+0000","last_fullsized":"2024-04-25T08:52:29.767972+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:18.764244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772426+0000","last_change":"2024-04-25T08:52:27.767777+0000","last_active":"2024-04-25T08:52:29.772426+0000","last_peered":"2024-04-25T08:52:29.772426+0000","last_clean":"2024-04-25T08:52:29.772426+0000","last_became_active":"2024-04-25T08:52:27.767493+0000","last_became_peered":"2024-04-25T08:52:27.767493+0000","last_unstale":"2024-04-25T08:52:29.772426+0000","last_undegraded":"2024-04-25T08:52:29.772426+0000","last_fullsized":"2024-04-25T08:52:29.772426+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:04.203184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689906+0000","last_change":"2024-04-25T08:52:28.783290+0000","last_active":"2024-04-25T08:52:30.689906+0000","last_peered":"2024-04-25T08:52:30.689906+0000","last_clean":"2024-04-25T08:52:30.689906+0000","last_became_active":"2024-04-25T08:52:28.782977+0000","last_became_peered":"2024-04-25T08:52:28.782977+0000","last_unstale":"2024-04-25T08:52:30.689906+0000","last_undegraded":"2024-04-25T08:52:30.689906+0000","last_fullsized":"2024-04-25T08:52:30.689906+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:14.620184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771622+0000","last_change":"2024-04-25T08:52:27.775280+0000","last_active":"2024-04-25T08:52:29.771622+0000","last_peered":"2024-04-25T08:52:29.771622+0000","last_clean":"2024-04-25T08:52:29.771622+0000","last_became_active":"2024-04-25T08:52:27.774648+0000","last_became_peered":"2024-04-25T08:52:27.774648+0000","last_unstale":"2024-04-25T08:52:29.771622+0000","last_undegraded":"2024-04-25T08:52:29.771622+0000","last_fullsized":"2024-04-25T08:52:29.771622+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:37.111648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767396+0000","last_change":"2024-04-25T08:52:28.780067+0000","last_active":"2024-04-25T08:52:29.767396+0000","last_peered":"2024-04-25T08:52:29.767396+0000","last_clean":"2024-04-25T08:52:29.767396+0000","last_became_active":"2024-04-25T08:52:28.779644+0000","last_became_peered":"2024-04-25T08:52:28.779644+0000","last_unstale":"2024-04-25T08:52:29.767396+0000","last_undegraded":"2024-04-25T08:52:29.767396+0000","last_fullsized":"2024-04-25T08:52:29.767396+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:10.150149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767429+0000","last_change":"2024-04-25T08:52:27.774918+0000","last_active":"2024-04-25T08:52:29.767429+0000","last_peered":"2024-04-25T08:52:29.767429+0000","last_clean":"2024-04-25T08:52:29.767429+0000","last_became_active":"2024-04-25T08:52:27.774640+0000","last_became_peered":"2024-04-25T08:52:27.774640+0000","last_unstale":"2024-04-25T08:52:29.767429+0000","last_undegraded":"2024-04-25T08:52:29.767429+0000","last_fullsized":"2024-04-25T08:52:29.767429+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:02.128741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686829+0000","last_change":"2024-04-25T08:52:28.779294+0000","last_active":"2024-04-25T08:52:30.686829+0000","last_peered":"2024-04-25T08:52:30.686829+0000","last_clean":"2024-04-25T08:52:30.686829+0000","last_became_active":"2024-04-25T08:52:28.778552+0000","last_became_peered":"2024-04-25T08:52:28.778552+0000","last_unstale":"2024-04-25T08:52:30.686829+0000","last_undegraded":"2024-04-25T08:52:30.686829+0000","last_fullsized":"2024-04-25T08:52:30.686829+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:14.559336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771363+0000","last_change":"2024-04-25T08:52:27.770519+0000","last_active":"2024-04-25T08:52:29.771363+0000","last_peered":"2024-04-25T08:52:29.771363+0000","last_clean":"2024-04-25T08:52:29.771363+0000","last_became_active":"2024-04-25T08:52:27.770130+0000","last_became_peered":"2024-04-25T08:52:27.770130+0000","last_unstale":"2024-04-25T08:52:29.771363+0000","last_undegraded":"2024-04-25T08:52:29.771363+0000","last_fullsized":"2024-04-25T08:52:29.771363+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.557197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.688005+0000","last_change":"2024-04-25T08:52:28.777003+0000","last_active":"2024-04-25T08:52:30.688005+0000","last_peered":"2024-04-25T08:52:30.688005+0000","last_clean":"2024-04-25T08:52:30.688005+0000","last_became_active":"2024-04-25T08:52:28.776718+0000","last_became_peered":"2024-04-25T08:52:28.776718+0000","last_unstale":"2024-04-25T08:52:30.688005+0000","last_undegraded":"2024-04-25T08:52:30.688005+0000","last_fullsized":"2024-04-25T08:52:30.688005+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:51.574030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687667+0000","last_change":"2024-04-25T08:52:27.771106+0000","last_active":"2024-04-25T08:52:30.687667+0000","last_peered":"2024-04-25T08:52:30.687667+0000","last_clean":"2024-04-25T08:52:30.687667+0000","last_became_active":"2024-04-25T08:52:27.770812+0000","last_became_peered":"2024-04-25T08:52:27.770812+0000","last_unstale":"2024-04-25T08:52:30.687667+0000","last_undegraded":"2024-04-25T08:52:30.687667+0000","last_fullsized":"2024-04-25T08:52:30.687667+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:48.111016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689473+0000","last_change":"2024-04-25T08:52:28.791202+0000","last_active":"2024-04-25T08:52:30.689473+0000","last_peered":"2024-04-25T08:52:30.689473+0000","last_clean":"2024-04-25T08:52:30.689473+0000","last_became_active":"2024-04-25T08:52:28.790928+0000","last_became_peered":"2024-04-25T08:52:28.790928+0000","last_unstale":"2024-04-25T08:52:30.689473+0000","last_undegraded":"2024-04-25T08:52:30.689473+0000","last_fullsized":"2024-04-25T08:52:30.689473+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:00.700601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771073+0000","last_change":"2024-04-25T08:52:27.769806+0000","last_active":"2024-04-25T08:52:29.771073+0000","last_peered":"2024-04-25T08:52:29.771073+0000","last_clean":"2024-04-25T08:52:29.771073+0000","last_became_active":"2024-04-25T08:52:27.769480+0000","last_became_peered":"2024-04-25T08:52:27.769480+0000","last_unstale":"2024-04-25T08:52:29.771073+0000","last_undegraded":"2024-04-25T08:52:29.771073+0000","last_fullsized":"2024-04-25T08:52:29.771073+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.601997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767961+0000","last_change":"2024-04-25T08:52:28.808090+0000","last_active":"2024-04-25T08:52:29.767961+0000","last_peered":"2024-04-25T08:52:29.767961+0000","last_clean":"2024-04-25T08:52:29.767961+0000","last_became_active":"2024-04-25T08:52:28.804783+0000","last_became_peered":"2024-04-25T08:52:28.804783+0000","last_unstale":"2024-04-25T08:52:29.767961+0000","last_undegraded":"2024-04-25T08:52:29.767961+0000","last_fullsized":"2024-04-25T08:52:29.767961+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:15.498721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767921+0000","last_change":"2024-04-25T08:52:27.774011+0000","last_active":"2024-04-25T08:52:29.767921+0000","last_peered":"2024-04-25T08:52:29.767921+0000","last_clean":"2024-04-25T08:52:29.767921+0000","last_became_active":"2024-04-25T08:52:27.773740+0000","last_became_peered":"2024-04-25T08:52:27.773740+0000","last_unstale":"2024-04-25T08:52:29.767921+0000","last_undegraded":"2024-04-25T08:52:29.767921+0000","last_fullsized":"2024-04-25T08:52:29.767921+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:35.236040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765301+0000","last_change":"2024-04-25T08:52:28.769857+0000","last_active":"2024-04-25T08:52:29.765301+0000","last_peered":"2024-04-25T08:52:29.765301+0000","last_clean":"2024-04-25T08:52:29.765301+0000","last_became_active":"2024-04-25T08:52:28.769473+0000","last_became_peered":"2024-04-25T08:52:28.769473+0000","last_unstale":"2024-04-25T08:52:29.765301+0000","last_undegraded":"2024-04-25T08:52:29.765301+0000","last_fullsized":"2024-04-25T08:52:29.765301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:00.700601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791416+0000","last_change":"2024-04-25T08:52:27.773247+0000","last_active":"2024-04-25T08:52:29.791416+0000","last_peered":"2024-04-25T08:52:29.791416+0000","last_clean":"2024-04-25T08:52:29.791416+0000","last_became_active":"2024-04-25T08:52:27.772735+0000","last_became_peered":"2024-04-25T08:52:27.772735+0000","last_unstale":"2024-04-25T08:52:29.791416+0000","last_undegraded":"2024-04-25T08:52:29.791416+0000","last_fullsized":"2024-04-25T08:52:29.791416+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:24.477694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772527+0000","last_change":"2024-04-25T08:52:28.795161+0000","last_active":"2024-04-25T08:52:29.772527+0000","last_peered":"2024-04-25T08:52:29.772527+0000","last_clean":"2024-04-25T08:52:29.772527+0000","last_became_active":"2024-04-25T08:52:28.794852+0000","last_became_peered":"2024-04-25T08:52:28.794852+0000","last_unstale":"2024-04-25T08:52:29.772527+0000","last_undegraded":"2024-04-25T08:52:29.772527+0000","last_fullsized":"2024-04-25T08:52:29.772527+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:48.730169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791376+0000","last_change":"2024-04-25T08:52:27.772330+0000","last_active":"2024-04-25T08:52:29.791376+0000","last_peered":"2024-04-25T08:52:29.791376+0000","last_clean":"2024-04-25T08:52:29.791376+0000","last_became_active":"2024-04-25T08:52:27.771976+0000","last_became_peered":"2024-04-25T08:52:27.771976+0000","last_unstale":"2024-04-25T08:52:29.791376+0000","last_undegraded":"2024-04-25T08:52:29.791376+0000","last_fullsized":"2024-04-25T08:52:29.791376+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:02.510174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687140+0000","last_change":"2024-04-25T08:52:28.790503+0000","last_active":"2024-04-25T08:52:30.687140+0000","last_peered":"2024-04-25T08:52:30.687140+0000","last_clean":"2024-04-25T08:52:30.687140+0000","last_became_active":"2024-04-25T08:52:28.790178+0000","last_became_peered":"2024-04-25T08:52:28.790178+0000","last_unstale":"2024-04-25T08:52:30.687140+0000","last_undegraded":"2024-04-25T08:52:30.687140+0000","last_fullsized":"2024-04-25T08:52:30.687140+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.741024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767852+0000","last_change":"2024-04-25T08:52:27.779165+0000","last_active":"2024-04-25T08:52:29.767852+0000","last_peered":"2024-04-25T08:52:29.767852+0000","last_clean":"2024-04-25T08:52:29.767852+0000","last_became_active":"2024-04-25T08:52:27.778673+0000","last_became_peered":"2024-04-25T08:52:27.778673+0000","last_unstale":"2024-04-25T08:52:29.767852+0000","last_undegraded":"2024-04-25T08:52:29.767852+0000","last_fullsized":"2024-04-25T08:52:29.767852+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:49.007000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767571+0000","last_change":"2024-04-25T08:52:28.784199+0000","last_active":"2024-04-25T08:52:29.767571+0000","last_peered":"2024-04-25T08:52:29.767571+0000","last_clean":"2024-04-25T08:52:29.767571+0000","last_became_active":"2024-04-25T08:52:28.783910+0000","last_became_peered":"2024-04-25T08:52:28.783910+0000","last_unstale":"2024-04-25T08:52:29.767571+0000","last_undegraded":"2024-04-25T08:52:29.767571+0000","last_fullsized":"2024-04-25T08:52:29.767571+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:42.752762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772286+0000","last_change":"2024-04-25T08:52:27.769647+0000","last_active":"2024-04-25T08:52:29.772286+0000","last_peered":"2024-04-25T08:52:29.772286+0000","last_clean":"2024-04-25T08:52:29.772286+0000","last_became_active":"2024-04-25T08:52:27.769327+0000","last_became_peered":"2024-04-25T08:52:27.769327+0000","last_unstale":"2024-04-25T08:52:29.772286+0000","last_undegraded":"2024-04-25T08:52:29.772286+0000","last_fullsized":"2024-04-25T08:52:29.772286+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:30.432898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687683+0000","last_change":"2024-04-25T08:52:28.780586+0000","last_active":"2024-04-25T08:52:30.687683+0000","last_peered":"2024-04-25T08:52:30.687683+0000","last_clean":"2024-04-25T08:52:30.687683+0000","last_became_active":"2024-04-25T08:52:28.778484+0000","last_became_peered":"2024-04-25T08:52:28.778484+0000","last_unstale":"2024-04-25T08:52:30.687683+0000","last_undegraded":"2024-04-25T08:52:30.687683+0000","last_fullsized":"2024-04-25T08:52:30.687683+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:03.226604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772044+0000","last_change":"2024-04-25T08:52:27.772684+0000","last_active":"2024-04-25T08:52:29.772044+0000","last_peered":"2024-04-25T08:52:29.772044+0000","last_clean":"2024-04-25T08:52:29.772044+0000","last_became_active":"2024-04-25T08:52:27.772418+0000","last_became_peered":"2024-04-25T08:52:27.772418+0000","last_unstale":"2024-04-25T08:52:29.772044+0000","last_undegraded":"2024-04-25T08:52:29.772044+0000","last_fullsized":"2024-04-25T08:52:29.772044+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:52.564237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771301+0000","last_change":"2024-04-25T08:52:28.789740+0000","last_active":"2024-04-25T08:52:29.771301+0000","last_peered":"2024-04-25T08:52:29.771301+0000","last_clean":"2024-04-25T08:52:29.771301+0000","last_became_active":"2024-04-25T08:52:28.789225+0000","last_became_peered":"2024-04-25T08:52:28.789225+0000","last_unstale":"2024-04-25T08:52:29.771301+0000","last_undegraded":"2024-04-25T08:52:29.771301+0000","last_fullsized":"2024-04-25T08:52:29.771301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:03.016575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767418+0000","last_change":"2024-04-25T08:52:27.778931+0000","last_active":"2024-04-25T08:52:29.767418+0000","last_peered":"2024-04-25T08:52:29.767418+0000","last_clean":"2024-04-25T08:52:29.767418+0000","last_became_active":"2024-04-25T08:52:27.778576+0000","last_became_peered":"2024-04-25T08:52:27.778576+0000","last_unstale":"2024-04-25T08:52:29.767418+0000","last_undegraded":"2024-04-25T08:52:29.767418+0000","last_fullsized":"2024-04-25T08:52:29.767418+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:38.899682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765292+0000","last_change":"2024-04-25T08:52:28.777359+0000","last_active":"2024-04-25T08:52:29.765292+0000","last_peered":"2024-04-25T08:52:29.765292+0000","last_clean":"2024-04-25T08:52:29.765292+0000","last_became_active":"2024-04-25T08:52:28.777070+0000","last_became_peered":"2024-04-25T08:52:28.777070+0000","last_unstale":"2024-04-25T08:52:29.765292+0000","last_undegraded":"2024-04-25T08:52:29.765292+0000","last_fullsized":"2024-04-25T08:52:29.765292+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:05.210040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771211+0000","last_change":"2024-04-25T08:52:27.764732+0000","last_active":"2024-04-25T08:52:29.771211+0000","last_peered":"2024-04-25T08:52:29.771211+0000","last_clean":"2024-04-25T08:52:29.771211+0000","last_became_active":"2024-04-25T08:52:27.764463+0000","last_became_peered":"2024-04-25T08:52:27.764463+0000","last_unstale":"2024-04-25T08:52:29.771211+0000","last_undegraded":"2024-04-25T08:52:29.771211+0000","last_fullsized":"2024-04-25T08:52:29.771211+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:51.777156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771735+0000","last_change":"2024-04-25T08:52:28.801621+0000","last_active":"2024-04-25T08:52:29.771735+0000","last_peered":"2024-04-25T08:52:29.771735+0000","last_clean":"2024-04-25T08:52:29.771735+0000","last_became_active":"2024-04-25T08:52:28.801326+0000","last_became_peered":"2024-04-25T08:52:28.801326+0000","last_unstale":"2024-04-25T08:52:29.771735+0000","last_undegraded":"2024-04-25T08:52:29.771735+0000","last_fullsized":"2024-04-25T08:52:29.771735+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:43.502514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686873+0000","last_change":"2024-04-25T08:52:27.752950+0000","last_active":"2024-04-25T08:52:30.686873+0000","last_peered":"2024-04-25T08:52:30.686873+0000","last_clean":"2024-04-25T08:52:30.686873+0000","last_became_active":"2024-04-25T08:52:27.752624+0000","last_became_peered":"2024-04-25T08:52:27.752624+0000","last_unstale":"2024-04-25T08:52:30.686873+0000","last_undegraded":"2024-04-25T08:52:30.686873+0000","last_fullsized":"2024-04-25T08:52:30.686873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:59.693746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":359,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771414+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:52:29.771414+0000","last_peered":"2024-04-25T08:52:29.771414+0000","last_clean":"2024-04-25T08:52:29.771414+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:52:29.771414+0000","last_undegraded":"2024-04-25T08:52:29.771414+0000","last_fullsized":"2024-04-25T08:52:29.771414+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2117632,"data_stored":1296218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":705656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":705234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1519616,"data_stored":706686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":706720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":705074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":705096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2105344,"data_stored":1295464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:52:32.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:52:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.058+0000 7fa5c9200640 1 Processor -- start 2024-04-25T08:52:33.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.058+0000 7fa5c9200640 1 -- start start 2024-04-25T08:52:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c9200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c415a750 0x7fa5c406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c9200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa5c40aed10 0x7fa5c406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c9200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c406d5e0 con 0x7fa5c40cc9e0 2024-04-25T08:52:33.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c9200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c415ab30 con 0x7fa5c40aed10 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c9200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c40ce4f0 con 0x7fa5c415a750 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa5c40cc9e0 0x7fa5c40b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38236/0 (socket says 172.21.15.46:38236) 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c2800640 1 -- 172.21.15.46:0/1594152822 learned_addr learned my addr 172.21.15.46:0/1594152822 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c1e00640 1 --2- 172.21.15.46:0/1594152822 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c415a750 0x7fa5c406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5b9400640 1 --2- 172.21.15.46:0/1594152822 >> v2:172.21.15.82:3300/0 conn(0x7fa5c40aed10 0x7fa5c406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c1400640 1 -- 172.21.15.46:0/1594152822 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046879730 0 0) 0x7fa5c406d5e0 con 0x7fa5c40cc9e0 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c1400640 1 -- 172.21.15.46:0/1594152822 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a8003530 con 0x7fa5c40cc9e0 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c1e00640 1 -- 172.21.15.46:0/1594152822 >> v2:172.21.15.82:3300/0 conn(0x7fa5c40aed10 msgr2=0x7fa5c406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c1e00640 1 --2- 172.21.15.46:0/1594152822 >> v2:172.21.15.82:3300/0 conn(0x7fa5c40aed10 0x7fa5c406f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c1e00640 1 -- 172.21.15.46:0/1594152822 >> v1:172.21.15.46:6789/0 conn(0x7fa5c40cc9e0 legacy=0x7fa5c40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.062+0000 7fa5c1e00640 1 -- 172.21.15.46:0/1594152822 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c40ce720 con 0x7fa5c415a750 2024-04-25T08:52:33.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c1e00640 1 --2- 172.21.15.46:0/1594152822 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c415a750 0x7fa5c406ec10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400bfa0 tx=0x7fa5b401eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7bbbae22ce8b6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c1400640 1 -- 172.21.15.46:0/1594152822 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400b2d0 con 0x7fa5c415a750 2024-04-25T08:52:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c1400640 1 -- 172.21.15.46:0/1594152822 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b40a7930 con 0x7fa5c415a750 2024-04-25T08:52:33.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c1400640 1 -- 172.21.15.46:0/1594152822 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40a7ce0 con 0x7fa5c415a750 2024-04-25T08:52:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c9200640 1 -- 172.21.15.46:0/1594152822 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c415a750 msgr2=0x7fa5c406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c9200640 1 --2- 172.21.15.46:0/1594152822 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c415a750 0x7fa5c406ec10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400bfa0 tx=0x7fa5b401eb70 comp rx=0 tx=0).stop 2024-04-25T08:52:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c9200640 1 -- 172.21.15.46:0/1594152822 shutdown_connections 2024-04-25T08:52:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c9200640 1 --2- 172.21.15.46:0/1594152822 >> v2:172.21.15.82:3300/0 conn(0x7fa5c40aed10 0x7fa5c406f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c9200640 1 --2- 172.21.15.46:0/1594152822 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c415a750 0x7fa5c406ec10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c9200640 1 -- 172.21.15.46:0/1594152822 >> 172.21.15.46:0/1594152822 conn(0x7fa5c40b12a0 msgr2=0x7fa5c40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c9200640 1 -- 172.21.15.46:0/1594152822 shutdown_connections 2024-04-25T08:52:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c9200640 1 -- 172.21.15.46:0/1594152822 wait complete. 2024-04-25T08:52:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.066+0000 7fa5c9200640 1 Processor -- start 2024-04-25T08:52:33.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 -- start start 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c40aed10 0x7fa5c415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c40cc9e0 0x7fa5c41531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5c415a750 0x7fa5c413fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c406db00 con 0x7fa5c40aed10 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c406d270 con 0x7fa5c415a750 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c406dc80 con 0x7fa5c40cc9e0 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c40aed10 0x7fa5c415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c40aed10 0x7fa5c415d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60714/0 (socket says 172.21.15.46:60714) 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c1e00640 1 -- 172.21.15.46:0/395611181 learned_addr learned my addr 172.21.15.46:0/395611181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:33.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5b9400640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c40cc9e0 0x7fa5c41531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c2800640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5c415a750 0x7fa5c413fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c1e00640 1 -- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c40cc9e0 msgr2=0x7fa5c41531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c1e00640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c40cc9e0 0x7fa5c41531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c1e00640 1 -- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5c415a750 msgr2=0x7fa5c413fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c1e00640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5c415a750 0x7fa5c413fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c1e00640 1 -- 172.21.15.46:0/395611181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c4153870 con 0x7fa5c40aed10 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c2800640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5c415a750 0x7fa5c413fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c1e00640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c40aed10 0x7fa5c415d6d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa5b401c7a0 tx=0x7fa5b401b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5baa00640 1 -- 172.21.15.46:0/395611181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40a7db0 con 0x7fa5c40aed10 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5baa00640 1 -- 172.21.15.46:0/395611181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b40039a0 con 0x7fa5c40aed10 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c40ad420 con 0x7fa5c40aed10 2024-04-25T08:52:33.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5baa00640 1 -- 172.21.15.46:0/395611181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4003e10 con 0x7fa5c40aed10 2024-04-25T08:52:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c40ad9e0 con 0x7fa5c40aed10 2024-04-25T08:52:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.070+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c4069080 con 0x7fa5c40aed10 2024-04-25T08:52:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.074+0000 7fa5baa00640 1 -- 172.21.15.46:0/395611181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5b40c0020 con 0x7fa5c40aed10 2024-04-25T08:52:33.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.074+0000 7fa5baa00640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5a0066f60 0x7fa5a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.074+0000 7fa5baa00640 1 -- 172.21.15.46:0/395611181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa5b411fa20 con 0x7fa5c40aed10 2024-04-25T08:52:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.074+0000 7fa5b9400640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5a0066f60 0x7fa5a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.074+0000 7fa5b9400640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5a0066f60 0x7fa5a0069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa5c415b1f0 tx=0x7fa5ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:33.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.078+0000 7fa5baa00640 1 -- 172.21.15.46:0/395611181 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5b40c7020 con 0x7fa5c40aed10 2024-04-25T08:52:33.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.234+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5c40b2a70 con 0x7fa5a0066f60 2024-04-25T08:52:33.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.242+0000 7fa5baa00640 1 -- 172.21.15.46:0/395611181 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7fa5c40b2a70 con 0x7fa5a0066f60 2024-04-25T08:52:33.236 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:52:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5a0066f60 msgr2=0x7fa5a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5a0066f60 0x7fa5a0069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa5c415b1f0 tx=0x7fa5ac008040 comp rx=0 tx=0).stop 2024-04-25T08:52:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c40aed10 msgr2=0x7fa5c415d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c40aed10 0x7fa5c415d6d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa5b401c7a0 tx=0x7fa5b401b910 comp rx=0 tx=0).stop 2024-04-25T08:52:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 shutdown_connections 2024-04-25T08:52:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5a0066f60 0x7fa5a0069420 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5c415a750 0x7fa5c413fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa5c40cc9e0 0x7fa5c41531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 --2- 172.21.15.46:0/395611181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c40aed10 0x7fa5c415d6d0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 >> 172.21.15.46:0/395611181 conn(0x7fa5c40b12a0 msgr2=0x7fa5c4071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 shutdown_connections 2024-04-25T08:52:33.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.246+0000 7fa5c9200640 1 -- 172.21.15.46:0/395611181 wait complete. 2024-04-25T08:52:33.256 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-04-25T08:52:32.411358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262468,"kb_used_data":13000,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712252,"statfs":{"total":773094113280,"available":772825346048,"internally_reserved":0,"allocated":13312000,"data_stored":6826148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.665630"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772771+0000","last_change":"2024-04-25T08:52:28.796407+0000","last_active":"2024-04-25T08:52:29.772771+0000","last_peered":"2024-04-25T08:52:29.772771+0000","last_clean":"2024-04-25T08:52:29.772771+0000","last_became_active":"2024-04-25T08:52:28.796160+0000","last_became_peered":"2024-04-25T08:52:28.796160+0000","last_unstale":"2024-04-25T08:52:29.772771+0000","last_undegraded":"2024-04-25T08:52:29.772771+0000","last_fullsized":"2024-04-25T08:52:29.772771+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:46.781517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687548+0000","last_change":"2024-04-25T08:52:28.783204+0000","last_active":"2024-04-25T08:52:30.687548+0000","last_peered":"2024-04-25T08:52:30.687548+0000","last_clean":"2024-04-25T08:52:30.687548+0000","last_became_active":"2024-04-25T08:52:28.782923+0000","last_became_peered":"2024-04-25T08:52:28.782923+0000","last_unstale":"2024-04-25T08:52:30.687548+0000","last_undegraded":"2024-04-25T08:52:30.687548+0000","last_fullsized":"2024-04-25T08:52:30.687548+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:25.560105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687562+0000","last_change":"2024-04-25T08:52:28.777910+0000","last_active":"2024-04-25T08:52:30.687562+0000","last_peered":"2024-04-25T08:52:30.687562+0000","last_clean":"2024-04-25T08:52:30.687562+0000","last_became_active":"2024-04-25T08:52:28.777616+0000","last_became_peered":"2024-04-25T08:52:28.777616+0000","last_unstale":"2024-04-25T08:52:30.687562+0000","last_undegraded":"2024-04-25T08:52:30.687562+0000","last_fullsized":"2024-04-25T08:52:30.687562+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:40.330466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771563+0000","last_change":"2024-04-25T08:52:28.794781+0000","last_active":"2024-04-25T08:52:29.771563+0000","last_peered":"2024-04-25T08:52:29.771563+0000","last_clean":"2024-04-25T08:52:29.771563+0000","last_became_active":"2024-04-25T08:52:28.793692+0000","last_became_peered":"2024-04-25T08:52:28.793692+0000","last_unstale":"2024-04-25T08:52:29.771563+0000","last_undegraded":"2024-04-25T08:52:29.771563+0000","last_fullsized":"2024-04-25T08:52:29.771563+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:40.351417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771551+0000","last_change":"2024-04-25T08:52:28.792340+0000","last_active":"2024-04-25T08:52:29.771551+0000","last_peered":"2024-04-25T08:52:29.771551+0000","last_clean":"2024-04-25T08:52:29.771551+0000","last_became_active":"2024-04-25T08:52:28.792042+0000","last_became_peered":"2024-04-25T08:52:28.792042+0000","last_unstale":"2024-04-25T08:52:29.771551+0000","last_undegraded":"2024-04-25T08:52:29.771551+0000","last_fullsized":"2024-04-25T08:52:29.771551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:38.118504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767736+0000","last_change":"2024-04-25T08:52:28.805339+0000","last_active":"2024-04-25T08:52:29.767736+0000","last_peered":"2024-04-25T08:52:29.767736+0000","last_clean":"2024-04-25T08:52:29.767736+0000","last_became_active":"2024-04-25T08:52:28.805001+0000","last_became_peered":"2024-04-25T08:52:28.805001+0000","last_unstale":"2024-04-25T08:52:29.767736+0000","last_undegraded":"2024-04-25T08:52:29.767736+0000","last_fullsized":"2024-04-25T08:52:29.767736+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:07.435476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767452+0000","last_change":"2024-04-25T08:52:28.781489+0000","last_active":"2024-04-25T08:52:29.767452+0000","last_peered":"2024-04-25T08:52:29.767452+0000","last_clean":"2024-04-25T08:52:29.767452+0000","last_became_active":"2024-04-25T08:52:28.781106+0000","last_became_peered":"2024-04-25T08:52:28.781106+0000","last_unstale":"2024-04-25T08:52:29.767452+0000","last_undegraded":"2024-04-25T08:52:29.767452+0000","last_fullsized":"2024-04-25T08:52:29.767452+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:49.454655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686736+0000","last_change":"2024-04-25T08:52:28.781819+0000","last_active":"2024-04-25T08:52:30.686736+0000","last_peered":"2024-04-25T08:52:30.686736+0000","last_clean":"2024-04-25T08:52:30.686736+0000","last_became_active":"2024-04-25T08:52:28.778319+0000","last_became_peered":"2024-04-25T08:52:28.778319+0000","last_unstale":"2024-04-25T08:52:30.686736+0000","last_undegraded":"2024-04-25T08:52:30.686736+0000","last_fullsized":"2024-04-25T08:52:30.686736+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:47.330267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772055+0000","last_change":"2024-04-25T08:52:28.811092+0000","last_active":"2024-04-25T08:52:29.772055+0000","last_peered":"2024-04-25T08:52:29.772055+0000","last_clean":"2024-04-25T08:52:29.772055+0000","last_became_active":"2024-04-25T08:52:28.810786+0000","last_became_peered":"2024-04-25T08:52:28.810786+0000","last_unstale":"2024-04-25T08:52:29.772055+0000","last_undegraded":"2024-04-25T08:52:29.772055+0000","last_fullsized":"2024-04-25T08:52:29.772055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:52.042338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765506+0000","last_change":"2024-04-25T08:52:28.793586+0000","last_active":"2024-04-25T08:52:29.765506+0000","last_peered":"2024-04-25T08:52:29.765506+0000","last_clean":"2024-04-25T08:52:29.765506+0000","last_became_active":"2024-04-25T08:52:28.793253+0000","last_became_peered":"2024-04-25T08:52:28.793253+0000","last_unstale":"2024-04-25T08:52:29.765506+0000","last_undegraded":"2024-04-25T08:52:29.765506+0000","last_fullsized":"2024-04-25T08:52:29.765506+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:46.781517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772529+0000","last_change":"2024-04-25T08:52:28.792673+0000","last_active":"2024-04-25T08:52:29.772529+0000","last_peered":"2024-04-25T08:52:29.772529+0000","last_clean":"2024-04-25T08:52:29.772529+0000","last_became_active":"2024-04-25T08:52:28.792335+0000","last_became_peered":"2024-04-25T08:52:28.792335+0000","last_unstale":"2024-04-25T08:52:29.772529+0000","last_undegraded":"2024-04-25T08:52:29.772529+0000","last_fullsized":"2024-04-25T08:52:29.772529+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:00.700601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767785+0000","last_change":"2024-04-25T08:52:28.790192+0000","last_active":"2024-04-25T08:52:29.767785+0000","last_peered":"2024-04-25T08:52:29.767785+0000","last_clean":"2024-04-25T08:52:29.767785+0000","last_became_active":"2024-04-25T08:52:28.789857+0000","last_became_peered":"2024-04-25T08:52:28.789857+0000","last_unstale":"2024-04-25T08:52:29.767785+0000","last_undegraded":"2024-04-25T08:52:29.767785+0000","last_fullsized":"2024-04-25T08:52:29.767785+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:28.270255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765196+0000","last_change":"2024-04-25T08:52:28.789117+0000","last_active":"2024-04-25T08:52:29.765196+0000","last_peered":"2024-04-25T08:52:29.765196+0000","last_clean":"2024-04-25T08:52:29.765196+0000","last_became_active":"2024-04-25T08:52:28.788796+0000","last_became_peered":"2024-04-25T08:52:28.788796+0000","last_unstale":"2024-04-25T08:52:29.765196+0000","last_undegraded":"2024-04-25T08:52:29.765196+0000","last_fullsized":"2024-04-25T08:52:29.765196+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:38.118504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687830+0000","last_change":"2024-04-25T08:52:28.784288+0000","last_active":"2024-04-25T08:52:30.687830+0000","last_peered":"2024-04-25T08:52:30.687830+0000","last_clean":"2024-04-25T08:52:30.687830+0000","last_became_active":"2024-04-25T08:52:28.777415+0000","last_became_peered":"2024-04-25T08:52:28.777415+0000","last_unstale":"2024-04-25T08:52:30.687830+0000","last_undegraded":"2024-04-25T08:52:30.687830+0000","last_fullsized":"2024-04-25T08:52:30.687830+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:28.268710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771540+0000","last_change":"2024-04-25T08:52:28.793355+0000","last_active":"2024-04-25T08:52:29.771540+0000","last_peered":"2024-04-25T08:52:29.771540+0000","last_clean":"2024-04-25T08:52:29.771540+0000","last_became_active":"2024-04-25T08:52:28.792994+0000","last_became_peered":"2024-04-25T08:52:28.792994+0000","last_unstale":"2024-04-25T08:52:29.771540+0000","last_undegraded":"2024-04-25T08:52:29.771540+0000","last_fullsized":"2024-04-25T08:52:29.771540+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:22.564053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767365+0000","last_change":"2024-04-25T08:52:28.793383+0000","last_active":"2024-04-25T08:52:29.767365+0000","last_peered":"2024-04-25T08:52:29.767365+0000","last_clean":"2024-04-25T08:52:29.767365+0000","last_became_active":"2024-04-25T08:52:28.793037+0000","last_became_peered":"2024-04-25T08:52:28.793037+0000","last_unstale":"2024-04-25T08:52:29.767365+0000","last_undegraded":"2024-04-25T08:52:29.767365+0000","last_fullsized":"2024-04-25T08:52:29.767365+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:44.934583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765564+0000","last_change":"2024-04-25T08:52:28.787029+0000","last_active":"2024-04-25T08:52:29.765564+0000","last_peered":"2024-04-25T08:52:29.765564+0000","last_clean":"2024-04-25T08:52:29.765564+0000","last_became_active":"2024-04-25T08:52:28.786288+0000","last_became_peered":"2024-04-25T08:52:28.786288+0000","last_unstale":"2024-04-25T08:52:29.765564+0000","last_undegraded":"2024-04-25T08:52:29.765564+0000","last_fullsized":"2024-04-25T08:52:29.765564+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:39.452095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687550+0000","last_change":"2024-04-25T08:52:28.782977+0000","last_active":"2024-04-25T08:52:30.687550+0000","last_peered":"2024-04-25T08:52:30.687550+0000","last_clean":"2024-04-25T08:52:30.687550+0000","last_became_active":"2024-04-25T08:52:28.782697+0000","last_became_peered":"2024-04-25T08:52:28.782697+0000","last_unstale":"2024-04-25T08:52:30.687550+0000","last_undegraded":"2024-04-25T08:52:30.687550+0000","last_fullsized":"2024-04-25T08:52:30.687550+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:25.027253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791315+0000","last_change":"2024-04-25T08:52:28.787742+0000","last_active":"2024-04-25T08:52:29.791315+0000","last_peered":"2024-04-25T08:52:29.791315+0000","last_clean":"2024-04-25T08:52:29.791315+0000","last_became_active":"2024-04-25T08:52:28.787455+0000","last_became_peered":"2024-04-25T08:52:28.787455+0000","last_unstale":"2024-04-25T08:52:29.791315+0000","last_undegraded":"2024-04-25T08:52:29.791315+0000","last_fullsized":"2024-04-25T08:52:29.791315+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:22.564053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686747+0000","last_change":"2024-04-25T08:52:28.788421+0000","last_active":"2024-04-25T08:52:30.686747+0000","last_peered":"2024-04-25T08:52:30.686747+0000","last_clean":"2024-04-25T08:52:30.686747+0000","last_became_active":"2024-04-25T08:52:28.788126+0000","last_became_peered":"2024-04-25T08:52:28.788126+0000","last_unstale":"2024-04-25T08:52:30.686747+0000","last_undegraded":"2024-04-25T08:52:30.686747+0000","last_fullsized":"2024-04-25T08:52:30.686747+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:19.325601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771568+0000","last_change":"2024-04-25T08:52:28.801404+0000","last_active":"2024-04-25T08:52:29.771568+0000","last_peered":"2024-04-25T08:52:29.771568+0000","last_clean":"2024-04-25T08:52:29.771568+0000","last_became_active":"2024-04-25T08:52:28.801117+0000","last_became_peered":"2024-04-25T08:52:28.801117+0000","last_unstale":"2024-04-25T08:52:29.771568+0000","last_undegraded":"2024-04-25T08:52:29.771568+0000","last_fullsized":"2024-04-25T08:52:29.771568+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:45.883389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772012+0000","last_change":"2024-04-25T08:52:28.807114+0000","last_active":"2024-04-25T08:52:29.772012+0000","last_peered":"2024-04-25T08:52:29.772012+0000","last_clean":"2024-04-25T08:52:29.772012+0000","last_became_active":"2024-04-25T08:52:28.806796+0000","last_became_peered":"2024-04-25T08:52:28.806796+0000","last_unstale":"2024-04-25T08:52:29.772012+0000","last_undegraded":"2024-04-25T08:52:29.772012+0000","last_fullsized":"2024-04-25T08:52:29.772012+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:05.210040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771912+0000","last_change":"2024-04-25T08:52:28.807125+0000","last_active":"2024-04-25T08:52:29.771912+0000","last_peered":"2024-04-25T08:52:29.771912+0000","last_clean":"2024-04-25T08:52:29.771912+0000","last_became_active":"2024-04-25T08:52:28.806802+0000","last_became_peered":"2024-04-25T08:52:28.806802+0000","last_unstale":"2024-04-25T08:52:29.771912+0000","last_undegraded":"2024-04-25T08:52:29.771912+0000","last_fullsized":"2024-04-25T08:52:29.771912+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:06.767521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689395+0000","last_change":"2024-04-25T08:52:28.780541+0000","last_active":"2024-04-25T08:52:30.689395+0000","last_peered":"2024-04-25T08:52:30.689395+0000","last_clean":"2024-04-25T08:52:30.689395+0000","last_became_active":"2024-04-25T08:52:28.780240+0000","last_became_peered":"2024-04-25T08:52:28.780240+0000","last_unstale":"2024-04-25T08:52:30.689395+0000","last_undegraded":"2024-04-25T08:52:30.689395+0000","last_fullsized":"2024-04-25T08:52:30.689395+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:40.680311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689797+0000","last_change":"2024-04-25T08:52:28.790369+0000","last_active":"2024-04-25T08:52:30.689797+0000","last_peered":"2024-04-25T08:52:30.689797+0000","last_clean":"2024-04-25T08:52:30.689797+0000","last_became_active":"2024-04-25T08:52:28.790106+0000","last_became_peered":"2024-04-25T08:52:28.790106+0000","last_unstale":"2024-04-25T08:52:30.689797+0000","last_undegraded":"2024-04-25T08:52:30.689797+0000","last_fullsized":"2024-04-25T08:52:30.689797+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:19.831604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686984+0000","last_change":"2024-04-25T08:52:28.789897+0000","last_active":"2024-04-25T08:52:30.686984+0000","last_peered":"2024-04-25T08:52:30.686984+0000","last_clean":"2024-04-25T08:52:30.686984+0000","last_became_active":"2024-04-25T08:52:28.789587+0000","last_became_peered":"2024-04-25T08:52:28.789587+0000","last_unstale":"2024-04-25T08:52:30.686984+0000","last_undegraded":"2024-04-25T08:52:30.686984+0000","last_fullsized":"2024-04-25T08:52:30.686984+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:11.608445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791423+0000","last_change":"2024-04-25T08:52:28.781138+0000","last_active":"2024-04-25T08:52:29.791423+0000","last_peered":"2024-04-25T08:52:29.791423+0000","last_clean":"2024-04-25T08:52:29.791423+0000","last_became_active":"2024-04-25T08:52:28.780610+0000","last_became_peered":"2024-04-25T08:52:28.780610+0000","last_unstale":"2024-04-25T08:52:29.791423+0000","last_undegraded":"2024-04-25T08:52:29.791423+0000","last_fullsized":"2024-04-25T08:52:29.791423+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:38.118504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767774+0000","last_change":"2024-04-25T08:52:28.790741+0000","last_active":"2024-04-25T08:52:29.767774+0000","last_peered":"2024-04-25T08:52:29.767774+0000","last_clean":"2024-04-25T08:52:29.767774+0000","last_became_active":"2024-04-25T08:52:28.790407+0000","last_became_peered":"2024-04-25T08:52:28.790407+0000","last_unstale":"2024-04-25T08:52:29.767774+0000","last_undegraded":"2024-04-25T08:52:29.767774+0000","last_fullsized":"2024-04-25T08:52:29.767774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:18.237461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687004+0000","last_change":"2024-04-25T08:52:28.801910+0000","last_active":"2024-04-25T08:52:30.687004+0000","last_peered":"2024-04-25T08:52:30.687004+0000","last_clean":"2024-04-25T08:52:30.687004+0000","last_became_active":"2024-04-25T08:52:28.801369+0000","last_became_peered":"2024-04-25T08:52:28.801369+0000","last_unstale":"2024-04-25T08:52:30.687004+0000","last_undegraded":"2024-04-25T08:52:30.687004+0000","last_fullsized":"2024-04-25T08:52:30.687004+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:05.774385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687774+0000","last_change":"2024-04-25T08:52:28.787169+0000","last_active":"2024-04-25T08:52:30.687774+0000","last_peered":"2024-04-25T08:52:30.687774+0000","last_clean":"2024-04-25T08:52:30.687774+0000","last_became_active":"2024-04-25T08:52:28.786818+0000","last_became_peered":"2024-04-25T08:52:28.786818+0000","last_unstale":"2024-04-25T08:52:30.687774+0000","last_undegraded":"2024-04-25T08:52:30.687774+0000","last_fullsized":"2024-04-25T08:52:30.687774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:48.576685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689476+0000","last_change":"2024-04-25T08:52:28.789637+0000","last_active":"2024-04-25T08:52:30.689476+0000","last_peered":"2024-04-25T08:52:30.689476+0000","last_clean":"2024-04-25T08:52:30.689476+0000","last_became_active":"2024-04-25T08:52:28.788725+0000","last_became_peered":"2024-04-25T08:52:28.788725+0000","last_unstale":"2024-04-25T08:52:30.689476+0000","last_undegraded":"2024-04-25T08:52:30.689476+0000","last_fullsized":"2024-04-25T08:52:30.689476+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:28.294422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689404+0000","last_change":"2024-04-25T08:52:28.779128+0000","last_active":"2024-04-25T08:52:30.689404+0000","last_peered":"2024-04-25T08:52:30.689404+0000","last_clean":"2024-04-25T08:52:30.689404+0000","last_became_active":"2024-04-25T08:52:28.778735+0000","last_became_peered":"2024-04-25T08:52:28.778735+0000","last_unstale":"2024-04-25T08:52:30.689404+0000","last_undegraded":"2024-04-25T08:52:30.689404+0000","last_fullsized":"2024-04-25T08:52:30.689404+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:05.210040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765312+0000","last_change":"2024-04-25T08:52:28.800939+0000","last_active":"2024-04-25T08:52:29.765312+0000","last_peered":"2024-04-25T08:52:29.765312+0000","last_clean":"2024-04-25T08:52:29.765312+0000","last_became_active":"2024-04-25T08:52:28.796551+0000","last_became_peered":"2024-04-25T08:52:28.796551+0000","last_unstale":"2024-04-25T08:52:29.765312+0000","last_undegraded":"2024-04-25T08:52:29.765312+0000","last_fullsized":"2024-04-25T08:52:29.765312+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:28.294422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791288+0000","last_change":"2024-04-25T08:52:27.759922+0000","last_active":"2024-04-25T08:52:29.791288+0000","last_peered":"2024-04-25T08:52:29.791288+0000","last_clean":"2024-04-25T08:52:29.791288+0000","last_became_active":"2024-04-25T08:52:27.759618+0000","last_became_peered":"2024-04-25T08:52:27.759618+0000","last_unstale":"2024-04-25T08:52:29.791288+0000","last_undegraded":"2024-04-25T08:52:29.791288+0000","last_fullsized":"2024-04-25T08:52:29.791288+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:30.432898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767768+0000","last_change":"2024-04-25T08:52:28.804623+0000","last_active":"2024-04-25T08:52:29.767768+0000","last_peered":"2024-04-25T08:52:29.767768+0000","last_clean":"2024-04-25T08:52:29.767768+0000","last_became_active":"2024-04-25T08:52:28.804195+0000","last_became_peered":"2024-04-25T08:52:28.804195+0000","last_unstale":"2024-04-25T08:52:29.767768+0000","last_undegraded":"2024-04-25T08:52:29.767768+0000","last_fullsized":"2024-04-25T08:52:29.767768+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:02.043215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765432+0000","last_change":"2024-04-25T08:52:27.771247+0000","last_active":"2024-04-25T08:52:29.765432+0000","last_peered":"2024-04-25T08:52:29.765432+0000","last_clean":"2024-04-25T08:52:29.765432+0000","last_became_active":"2024-04-25T08:52:27.770975+0000","last_became_peered":"2024-04-25T08:52:27.770975+0000","last_unstale":"2024-04-25T08:52:29.765432+0000","last_undegraded":"2024-04-25T08:52:29.765432+0000","last_fullsized":"2024-04-25T08:52:29.765432+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:08.697698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689599+0000","last_change":"2024-04-25T08:52:28.787762+0000","last_active":"2024-04-25T08:52:30.689599+0000","last_peered":"2024-04-25T08:52:30.689599+0000","last_clean":"2024-04-25T08:52:30.689599+0000","last_became_active":"2024-04-25T08:52:28.787213+0000","last_became_peered":"2024-04-25T08:52:28.787213+0000","last_unstale":"2024-04-25T08:52:30.689599+0000","last_undegraded":"2024-04-25T08:52:30.689599+0000","last_fullsized":"2024-04-25T08:52:30.689599+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:46.781517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772236+0000","last_change":"2024-04-25T08:52:27.769244+0000","last_active":"2024-04-25T08:52:29.772236+0000","last_peered":"2024-04-25T08:52:29.772236+0000","last_clean":"2024-04-25T08:52:29.772236+0000","last_became_active":"2024-04-25T08:52:27.768944+0000","last_became_peered":"2024-04-25T08:52:27.768944+0000","last_unstale":"2024-04-25T08:52:29.772236+0000","last_undegraded":"2024-04-25T08:52:29.772236+0000","last_fullsized":"2024-04-25T08:52:29.772236+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:39.673455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772202+0000","last_change":"2024-04-25T08:52:28.801317+0000","last_active":"2024-04-25T08:52:29.772202+0000","last_peered":"2024-04-25T08:52:29.772202+0000","last_clean":"2024-04-25T08:52:29.772202+0000","last_became_active":"2024-04-25T08:52:28.800998+0000","last_became_peered":"2024-04-25T08:52:28.800998+0000","last_unstale":"2024-04-25T08:52:29.772202+0000","last_undegraded":"2024-04-25T08:52:29.772202+0000","last_fullsized":"2024-04-25T08:52:29.772202+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:43.632419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765197+0000","last_change":"2024-04-25T08:52:27.776310+0000","last_active":"2024-04-25T08:52:29.765197+0000","last_peered":"2024-04-25T08:52:29.765197+0000","last_clean":"2024-04-25T08:52:29.765197+0000","last_became_active":"2024-04-25T08:52:27.775485+0000","last_became_peered":"2024-04-25T08:52:27.775485+0000","last_unstale":"2024-04-25T08:52:29.765197+0000","last_undegraded":"2024-04-25T08:52:29.765197+0000","last_fullsized":"2024-04-25T08:52:29.765197+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:32.893999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.764830+0000","last_change":"2024-04-25T08:52:28.769312+0000","last_active":"2024-04-25T08:52:29.764830+0000","last_peered":"2024-04-25T08:52:29.764830+0000","last_clean":"2024-04-25T08:52:29.764830+0000","last_became_active":"2024-04-25T08:52:28.768639+0000","last_became_peered":"2024-04-25T08:52:28.768639+0000","last_unstale":"2024-04-25T08:52:29.764830+0000","last_undegraded":"2024-04-25T08:52:29.764830+0000","last_fullsized":"2024-04-25T08:52:29.764830+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:52.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771619+0000","last_change":"2024-04-25T08:52:27.780766+0000","last_active":"2024-04-25T08:52:29.771619+0000","last_peered":"2024-04-25T08:52:29.771619+0000","last_clean":"2024-04-25T08:52:29.771619+0000","last_became_active":"2024-04-25T08:52:27.780475+0000","last_became_peered":"2024-04-25T08:52:27.780475+0000","last_unstale":"2024-04-25T08:52:29.771619+0000","last_undegraded":"2024-04-25T08:52:29.771619+0000","last_fullsized":"2024-04-25T08:52:29.771619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:24.477694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771710+0000","last_change":"2024-04-25T08:52:28.787847+0000","last_active":"2024-04-25T08:52:29.771710+0000","last_peered":"2024-04-25T08:52:29.771710+0000","last_clean":"2024-04-25T08:52:29.771710+0000","last_became_active":"2024-04-25T08:52:28.787584+0000","last_became_peered":"2024-04-25T08:52:28.787584+0000","last_unstale":"2024-04-25T08:52:29.771710+0000","last_undegraded":"2024-04-25T08:52:29.771710+0000","last_fullsized":"2024-04-25T08:52:29.771710+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:40.680311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767808+0000","last_change":"2024-04-25T08:52:27.774079+0000","last_active":"2024-04-25T08:52:29.767808+0000","last_peered":"2024-04-25T08:52:29.767808+0000","last_clean":"2024-04-25T08:52:29.767808+0000","last_became_active":"2024-04-25T08:52:27.773822+0000","last_became_peered":"2024-04-25T08:52:27.773822+0000","last_unstale":"2024-04-25T08:52:29.767808+0000","last_undegraded":"2024-04-25T08:52:29.767808+0000","last_fullsized":"2024-04-25T08:52:29.767808+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:30.989380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687882+0000","last_change":"2024-04-25T08:52:28.773260+0000","last_active":"2024-04-25T08:52:30.687882+0000","last_peered":"2024-04-25T08:52:30.687882+0000","last_clean":"2024-04-25T08:52:30.687882+0000","last_became_active":"2024-04-25T08:52:28.771289+0000","last_became_peered":"2024-04-25T08:52:28.771289+0000","last_unstale":"2024-04-25T08:52:30.687882+0000","last_undegraded":"2024-04-25T08:52:30.687882+0000","last_fullsized":"2024-04-25T08:52:30.687882+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:43.398832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689552+0000","last_change":"2024-04-25T08:52:27.774534+0000","last_active":"2024-04-25T08:52:30.689552+0000","last_peered":"2024-04-25T08:52:30.689552+0000","last_clean":"2024-04-25T08:52:30.689552+0000","last_became_active":"2024-04-25T08:52:27.774263+0000","last_became_peered":"2024-04-25T08:52:27.774263+0000","last_unstale":"2024-04-25T08:52:30.689552+0000","last_undegraded":"2024-04-25T08:52:30.689552+0000","last_fullsized":"2024-04-25T08:52:30.689552+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:08.697698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765206+0000","last_change":"2024-04-25T08:52:28.772399+0000","last_active":"2024-04-25T08:52:29.765206+0000","last_peered":"2024-04-25T08:52:29.765206+0000","last_clean":"2024-04-25T08:52:29.765206+0000","last_became_active":"2024-04-25T08:52:28.772116+0000","last_became_peered":"2024-04-25T08:52:28.772116+0000","last_unstale":"2024-04-25T08:52:29.765206+0000","last_undegraded":"2024-04-25T08:52:29.765206+0000","last_fullsized":"2024-04-25T08:52:29.765206+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:40.680311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767449+0000","last_change":"2024-04-25T08:52:27.778982+0000","last_active":"2024-04-25T08:52:29.767449+0000","last_peered":"2024-04-25T08:52:29.767449+0000","last_clean":"2024-04-25T08:52:29.767449+0000","last_became_active":"2024-04-25T08:52:27.778686+0000","last_became_peered":"2024-04-25T08:52:27.778686+0000","last_unstale":"2024-04-25T08:52:29.767449+0000","last_undegraded":"2024-04-25T08:52:29.767449+0000","last_fullsized":"2024-04-25T08:52:29.767449+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:40.752776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791128+0000","last_change":"2024-04-25T08:52:28.781774+0000","last_active":"2024-04-25T08:52:29.791128+0000","last_peered":"2024-04-25T08:52:29.791128+0000","last_clean":"2024-04-25T08:52:29.791128+0000","last_became_active":"2024-04-25T08:52:28.781512+0000","last_became_peered":"2024-04-25T08:52:28.781512+0000","last_unstale":"2024-04-25T08:52:29.791128+0000","last_undegraded":"2024-04-25T08:52:29.791128+0000","last_fullsized":"2024-04-25T08:52:29.791128+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:28.608853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791293+0000","last_change":"2024-04-25T08:52:27.771765+0000","last_active":"2024-04-25T08:52:29.791293+0000","last_peered":"2024-04-25T08:52:29.791293+0000","last_clean":"2024-04-25T08:52:29.791293+0000","last_became_active":"2024-04-25T08:52:27.769985+0000","last_became_peered":"2024-04-25T08:52:27.769985+0000","last_unstale":"2024-04-25T08:52:29.791293+0000","last_undegraded":"2024-04-25T08:52:29.791293+0000","last_fullsized":"2024-04-25T08:52:29.791293+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:52.564237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689710+0000","last_change":"2024-04-25T08:52:28.801178+0000","last_active":"2024-04-25T08:52:30.689710+0000","last_peered":"2024-04-25T08:52:30.689710+0000","last_clean":"2024-04-25T08:52:30.689710+0000","last_became_active":"2024-04-25T08:52:28.800840+0000","last_became_peered":"2024-04-25T08:52:28.800840+0000","last_unstale":"2024-04-25T08:52:30.689710+0000","last_undegraded":"2024-04-25T08:52:30.689710+0000","last_fullsized":"2024-04-25T08:52:30.689710+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:49.117872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689612+0000","last_change":"2024-04-25T08:52:27.774894+0000","last_active":"2024-04-25T08:52:30.689612+0000","last_peered":"2024-04-25T08:52:30.689612+0000","last_clean":"2024-04-25T08:52:30.689612+0000","last_became_active":"2024-04-25T08:52:27.774551+0000","last_became_peered":"2024-04-25T08:52:27.774551+0000","last_unstale":"2024-04-25T08:52:30.689612+0000","last_undegraded":"2024-04-25T08:52:30.689612+0000","last_fullsized":"2024-04-25T08:52:30.689612+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:30.432898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687782+0000","last_change":"2024-04-25T08:52:28.777489+0000","last_active":"2024-04-25T08:52:30.687782+0000","last_peered":"2024-04-25T08:52:30.687782+0000","last_clean":"2024-04-25T08:52:30.687782+0000","last_became_active":"2024-04-25T08:52:28.777181+0000","last_became_peered":"2024-04-25T08:52:28.777181+0000","last_unstale":"2024-04-25T08:52:30.687782+0000","last_undegraded":"2024-04-25T08:52:30.687782+0000","last_fullsized":"2024-04-25T08:52:30.687782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:41.197433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689802+0000","last_change":"2024-04-25T08:52:27.774154+0000","last_active":"2024-04-25T08:52:30.689802+0000","last_peered":"2024-04-25T08:52:30.689802+0000","last_clean":"2024-04-25T08:52:30.689802+0000","last_became_active":"2024-04-25T08:52:27.773331+0000","last_became_peered":"2024-04-25T08:52:27.773331+0000","last_unstale":"2024-04-25T08:52:30.689802+0000","last_undegraded":"2024-04-25T08:52:30.689802+0000","last_fullsized":"2024-04-25T08:52:30.689802+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:24.477694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689899+0000","last_change":"2024-04-25T08:52:28.797964+0000","last_active":"2024-04-25T08:52:30.689899+0000","last_peered":"2024-04-25T08:52:30.689899+0000","last_clean":"2024-04-25T08:52:30.689899+0000","last_became_active":"2024-04-25T08:52:28.797707+0000","last_became_peered":"2024-04-25T08:52:28.797707+0000","last_unstale":"2024-04-25T08:52:30.689899+0000","last_undegraded":"2024-04-25T08:52:30.689899+0000","last_fullsized":"2024-04-25T08:52:30.689899+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.134987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687282+0000","last_change":"2024-04-25T08:52:27.766037+0000","last_active":"2024-04-25T08:52:30.687282+0000","last_peered":"2024-04-25T08:52:30.687282+0000","last_clean":"2024-04-25T08:52:30.687282+0000","last_became_active":"2024-04-25T08:52:27.765712+0000","last_became_peered":"2024-04-25T08:52:27.765712+0000","last_unstale":"2024-04-25T08:52:30.687282+0000","last_undegraded":"2024-04-25T08:52:30.687282+0000","last_fullsized":"2024-04-25T08:52:30.687282+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:45.774662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791213+0000","last_change":"2024-04-25T08:52:28.779728+0000","last_active":"2024-04-25T08:52:29.791213+0000","last_peered":"2024-04-25T08:52:29.791213+0000","last_clean":"2024-04-25T08:52:29.791213+0000","last_became_active":"2024-04-25T08:52:28.779352+0000","last_became_peered":"2024-04-25T08:52:28.779352+0000","last_unstale":"2024-04-25T08:52:29.791213+0000","last_undegraded":"2024-04-25T08:52:29.791213+0000","last_fullsized":"2024-04-25T08:52:29.791213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:52.784012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687280+0000","last_change":"2024-04-25T08:52:27.755434+0000","last_active":"2024-04-25T08:52:30.687280+0000","last_peered":"2024-04-25T08:52:30.687280+0000","last_clean":"2024-04-25T08:52:30.687280+0000","last_became_active":"2024-04-25T08:52:27.754991+0000","last_became_peered":"2024-04-25T08:52:27.754991+0000","last_unstale":"2024-04-25T08:52:30.687280+0000","last_undegraded":"2024-04-25T08:52:30.687280+0000","last_fullsized":"2024-04-25T08:52:30.687280+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:27.287567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.764959+0000","last_change":"2024-04-25T08:52:28.782221+0000","last_active":"2024-04-25T08:52:29.764959+0000","last_peered":"2024-04-25T08:52:29.764959+0000","last_clean":"2024-04-25T08:52:29.764959+0000","last_became_active":"2024-04-25T08:52:28.781893+0000","last_became_peered":"2024-04-25T08:52:28.781893+0000","last_unstale":"2024-04-25T08:52:29.764959+0000","last_undegraded":"2024-04-25T08:52:29.764959+0000","last_fullsized":"2024-04-25T08:52:29.764959+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:22.564053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687945+0000","last_change":"2024-04-25T08:52:27.769364+0000","last_active":"2024-04-25T08:52:30.687945+0000","last_peered":"2024-04-25T08:52:30.687945+0000","last_clean":"2024-04-25T08:52:30.687945+0000","last_became_active":"2024-04-25T08:52:27.769082+0000","last_became_peered":"2024-04-25T08:52:27.769082+0000","last_unstale":"2024-04-25T08:52:30.687945+0000","last_undegraded":"2024-04-25T08:52:30.687945+0000","last_fullsized":"2024-04-25T08:52:30.687945+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:38.445240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686774+0000","last_change":"2024-04-25T08:52:28.790993+0000","last_active":"2024-04-25T08:52:30.686774+0000","last_peered":"2024-04-25T08:52:30.686774+0000","last_clean":"2024-04-25T08:52:30.686774+0000","last_became_active":"2024-04-25T08:52:28.790700+0000","last_became_peered":"2024-04-25T08:52:28.790700+0000","last_unstale":"2024-04-25T08:52:30.686774+0000","last_undegraded":"2024-04-25T08:52:30.686774+0000","last_fullsized":"2024-04-25T08:52:30.686774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:31.309849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687518+0000","last_change":"2024-04-25T08:52:27.765019+0000","last_active":"2024-04-25T08:52:30.687518+0000","last_peered":"2024-04-25T08:52:30.687518+0000","last_clean":"2024-04-25T08:52:30.687518+0000","last_became_active":"2024-04-25T08:52:27.764761+0000","last_became_peered":"2024-04-25T08:52:27.764761+0000","last_unstale":"2024-04-25T08:52:30.687518+0000","last_undegraded":"2024-04-25T08:52:30.687518+0000","last_fullsized":"2024-04-25T08:52:30.687518+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:18.824748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689612+0000","last_change":"2024-04-25T08:52:28.787436+0000","last_active":"2024-04-25T08:52:30.689612+0000","last_peered":"2024-04-25T08:52:30.689612+0000","last_clean":"2024-04-25T08:52:30.689612+0000","last_became_active":"2024-04-25T08:52:28.787143+0000","last_became_peered":"2024-04-25T08:52:28.787143+0000","last_unstale":"2024-04-25T08:52:30.689612+0000","last_undegraded":"2024-04-25T08:52:30.689612+0000","last_fullsized":"2024-04-25T08:52:30.689612+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:39.452095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765526+0000","last_change":"2024-04-25T08:52:27.776624+0000","last_active":"2024-04-25T08:52:29.765526+0000","last_peered":"2024-04-25T08:52:29.765526+0000","last_clean":"2024-04-25T08:52:29.765526+0000","last_became_active":"2024-04-25T08:52:27.775698+0000","last_became_peered":"2024-04-25T08:52:27.775698+0000","last_unstale":"2024-04-25T08:52:29.765526+0000","last_undegraded":"2024-04-25T08:52:29.765526+0000","last_fullsized":"2024-04-25T08:52:29.765526+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:26.864975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772648+0000","last_change":"2024-04-25T08:52:28.806469+0000","last_active":"2024-04-25T08:52:29.772648+0000","last_peered":"2024-04-25T08:52:29.772648+0000","last_clean":"2024-04-25T08:52:29.772648+0000","last_became_active":"2024-04-25T08:52:28.806170+0000","last_became_peered":"2024-04-25T08:52:28.806170+0000","last_unstale":"2024-04-25T08:52:29.772648+0000","last_undegraded":"2024-04-25T08:52:29.772648+0000","last_fullsized":"2024-04-25T08:52:29.772648+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:28.294422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686969+0000","last_change":"2024-04-25T08:52:27.768179+0000","last_active":"2024-04-25T08:52:30.686969+0000","last_peered":"2024-04-25T08:52:30.686969+0000","last_clean":"2024-04-25T08:52:30.686969+0000","last_became_active":"2024-04-25T08:52:27.767851+0000","last_became_peered":"2024-04-25T08:52:27.767851+0000","last_unstale":"2024-04-25T08:52:30.686969+0000","last_undegraded":"2024-04-25T08:52:30.686969+0000","last_fullsized":"2024-04-25T08:52:30.686969+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:20.059498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767972+0000","last_change":"2024-04-25T08:52:28.789158+0000","last_active":"2024-04-25T08:52:29.767972+0000","last_peered":"2024-04-25T08:52:29.767972+0000","last_clean":"2024-04-25T08:52:29.767972+0000","last_became_active":"2024-04-25T08:52:28.788874+0000","last_became_peered":"2024-04-25T08:52:28.788874+0000","last_unstale":"2024-04-25T08:52:29.767972+0000","last_undegraded":"2024-04-25T08:52:29.767972+0000","last_fullsized":"2024-04-25T08:52:29.767972+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:18.764244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772426+0000","last_change":"2024-04-25T08:52:27.767777+0000","last_active":"2024-04-25T08:52:29.772426+0000","last_peered":"2024-04-25T08:52:29.772426+0000","last_clean":"2024-04-25T08:52:29.772426+0000","last_became_active":"2024-04-25T08:52:27.767493+0000","last_became_peered":"2024-04-25T08:52:27.767493+0000","last_unstale":"2024-04-25T08:52:29.772426+0000","last_undegraded":"2024-04-25T08:52:29.772426+0000","last_fullsized":"2024-04-25T08:52:29.772426+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:04.203184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689906+0000","last_change":"2024-04-25T08:52:28.783290+0000","last_active":"2024-04-25T08:52:30.689906+0000","last_peered":"2024-04-25T08:52:30.689906+0000","last_clean":"2024-04-25T08:52:30.689906+0000","last_became_active":"2024-04-25T08:52:28.782977+0000","last_became_peered":"2024-04-25T08:52:28.782977+0000","last_unstale":"2024-04-25T08:52:30.689906+0000","last_undegraded":"2024-04-25T08:52:30.689906+0000","last_fullsized":"2024-04-25T08:52:30.689906+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:14.620184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771622+0000","last_change":"2024-04-25T08:52:27.775280+0000","last_active":"2024-04-25T08:52:29.771622+0000","last_peered":"2024-04-25T08:52:29.771622+0000","last_clean":"2024-04-25T08:52:29.771622+0000","last_became_active":"2024-04-25T08:52:27.774648+0000","last_became_peered":"2024-04-25T08:52:27.774648+0000","last_unstale":"2024-04-25T08:52:29.771622+0000","last_undegraded":"2024-04-25T08:52:29.771622+0000","last_fullsized":"2024-04-25T08:52:29.771622+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:37.111648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767396+0000","last_change":"2024-04-25T08:52:28.780067+0000","last_active":"2024-04-25T08:52:29.767396+0000","last_peered":"2024-04-25T08:52:29.767396+0000","last_clean":"2024-04-25T08:52:29.767396+0000","last_became_active":"2024-04-25T08:52:28.779644+0000","last_became_peered":"2024-04-25T08:52:28.779644+0000","last_unstale":"2024-04-25T08:52:29.767396+0000","last_undegraded":"2024-04-25T08:52:29.767396+0000","last_fullsized":"2024-04-25T08:52:29.767396+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:10.150149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767429+0000","last_change":"2024-04-25T08:52:27.774918+0000","last_active":"2024-04-25T08:52:29.767429+0000","last_peered":"2024-04-25T08:52:29.767429+0000","last_clean":"2024-04-25T08:52:29.767429+0000","last_became_active":"2024-04-25T08:52:27.774640+0000","last_became_peered":"2024-04-25T08:52:27.774640+0000","last_unstale":"2024-04-25T08:52:29.767429+0000","last_undegraded":"2024-04-25T08:52:29.767429+0000","last_fullsized":"2024-04-25T08:52:29.767429+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:02.128741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686829+0000","last_change":"2024-04-25T08:52:28.779294+0000","last_active":"2024-04-25T08:52:30.686829+0000","last_peered":"2024-04-25T08:52:30.686829+0000","last_clean":"2024-04-25T08:52:30.686829+0000","last_became_active":"2024-04-25T08:52:28.778552+0000","last_became_peered":"2024-04-25T08:52:28.778552+0000","last_unstale":"2024-04-25T08:52:30.686829+0000","last_undegraded":"2024-04-25T08:52:30.686829+0000","last_fullsized":"2024-04-25T08:52:30.686829+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:14.559336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771363+0000","last_change":"2024-04-25T08:52:27.770519+0000","last_active":"2024-04-25T08:52:29.771363+0000","last_peered":"2024-04-25T08:52:29.771363+0000","last_clean":"2024-04-25T08:52:29.771363+0000","last_became_active":"2024-04-25T08:52:27.770130+0000","last_became_peered":"2024-04-25T08:52:27.770130+0000","last_unstale":"2024-04-25T08:52:29.771363+0000","last_undegraded":"2024-04-25T08:52:29.771363+0000","last_fullsized":"2024-04-25T08:52:29.771363+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.557197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.688005+0000","last_change":"2024-04-25T08:52:28.777003+0000","last_active":"2024-04-25T08:52:30.688005+0000","last_peered":"2024-04-25T08:52:30.688005+0000","last_clean":"2024-04-25T08:52:30.688005+0000","last_became_active":"2024-04-25T08:52:28.776718+0000","last_became_peered":"2024-04-25T08:52:28.776718+0000","last_unstale":"2024-04-25T08:52:30.688005+0000","last_undegraded":"2024-04-25T08:52:30.688005+0000","last_fullsized":"2024-04-25T08:52:30.688005+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:51.574030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687667+0000","last_change":"2024-04-25T08:52:27.771106+0000","last_active":"2024-04-25T08:52:30.687667+0000","last_peered":"2024-04-25T08:52:30.687667+0000","last_clean":"2024-04-25T08:52:30.687667+0000","last_became_active":"2024-04-25T08:52:27.770812+0000","last_became_peered":"2024-04-25T08:52:27.770812+0000","last_unstale":"2024-04-25T08:52:30.687667+0000","last_undegraded":"2024-04-25T08:52:30.687667+0000","last_fullsized":"2024-04-25T08:52:30.687667+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:48.111016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.689473+0000","last_change":"2024-04-25T08:52:28.791202+0000","last_active":"2024-04-25T08:52:30.689473+0000","last_peered":"2024-04-25T08:52:30.689473+0000","last_clean":"2024-04-25T08:52:30.689473+0000","last_became_active":"2024-04-25T08:52:28.790928+0000","last_became_peered":"2024-04-25T08:52:28.790928+0000","last_unstale":"2024-04-25T08:52:30.689473+0000","last_undegraded":"2024-04-25T08:52:30.689473+0000","last_fullsized":"2024-04-25T08:52:30.689473+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:00.700601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771073+0000","last_change":"2024-04-25T08:52:27.769806+0000","last_active":"2024-04-25T08:52:29.771073+0000","last_peered":"2024-04-25T08:52:29.771073+0000","last_clean":"2024-04-25T08:52:29.771073+0000","last_became_active":"2024-04-25T08:52:27.769480+0000","last_became_peered":"2024-04-25T08:52:27.769480+0000","last_unstale":"2024-04-25T08:52:29.771073+0000","last_undegraded":"2024-04-25T08:52:29.771073+0000","last_fullsized":"2024-04-25T08:52:29.771073+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:27.601997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767961+0000","last_change":"2024-04-25T08:52:28.808090+0000","last_active":"2024-04-25T08:52:29.767961+0000","last_peered":"2024-04-25T08:52:29.767961+0000","last_clean":"2024-04-25T08:52:29.767961+0000","last_became_active":"2024-04-25T08:52:28.804783+0000","last_became_peered":"2024-04-25T08:52:28.804783+0000","last_unstale":"2024-04-25T08:52:29.767961+0000","last_undegraded":"2024-04-25T08:52:29.767961+0000","last_fullsized":"2024-04-25T08:52:29.767961+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:15.498721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767921+0000","last_change":"2024-04-25T08:52:27.774011+0000","last_active":"2024-04-25T08:52:29.767921+0000","last_peered":"2024-04-25T08:52:29.767921+0000","last_clean":"2024-04-25T08:52:29.767921+0000","last_became_active":"2024-04-25T08:52:27.773740+0000","last_became_peered":"2024-04-25T08:52:27.773740+0000","last_unstale":"2024-04-25T08:52:29.767921+0000","last_undegraded":"2024-04-25T08:52:29.767921+0000","last_fullsized":"2024-04-25T08:52:29.767921+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:35.236040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765301+0000","last_change":"2024-04-25T08:52:28.769857+0000","last_active":"2024-04-25T08:52:29.765301+0000","last_peered":"2024-04-25T08:52:29.765301+0000","last_clean":"2024-04-25T08:52:29.765301+0000","last_became_active":"2024-04-25T08:52:28.769473+0000","last_became_peered":"2024-04-25T08:52:28.769473+0000","last_unstale":"2024-04-25T08:52:29.765301+0000","last_undegraded":"2024-04-25T08:52:29.765301+0000","last_fullsized":"2024-04-25T08:52:29.765301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:00.700601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791416+0000","last_change":"2024-04-25T08:52:27.773247+0000","last_active":"2024-04-25T08:52:29.791416+0000","last_peered":"2024-04-25T08:52:29.791416+0000","last_clean":"2024-04-25T08:52:29.791416+0000","last_became_active":"2024-04-25T08:52:27.772735+0000","last_became_peered":"2024-04-25T08:52:27.772735+0000","last_unstale":"2024-04-25T08:52:29.791416+0000","last_undegraded":"2024-04-25T08:52:29.791416+0000","last_fullsized":"2024-04-25T08:52:29.791416+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:24.477694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772527+0000","last_change":"2024-04-25T08:52:28.795161+0000","last_active":"2024-04-25T08:52:29.772527+0000","last_peered":"2024-04-25T08:52:29.772527+0000","last_clean":"2024-04-25T08:52:29.772527+0000","last_became_active":"2024-04-25T08:52:28.794852+0000","last_became_peered":"2024-04-25T08:52:28.794852+0000","last_unstale":"2024-04-25T08:52:29.772527+0000","last_undegraded":"2024-04-25T08:52:29.772527+0000","last_fullsized":"2024-04-25T08:52:29.772527+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:48.730169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.791376+0000","last_change":"2024-04-25T08:52:27.772330+0000","last_active":"2024-04-25T08:52:29.791376+0000","last_peered":"2024-04-25T08:52:29.791376+0000","last_clean":"2024-04-25T08:52:29.791376+0000","last_became_active":"2024-04-25T08:52:27.771976+0000","last_became_peered":"2024-04-25T08:52:27.771976+0000","last_unstale":"2024-04-25T08:52:29.791376+0000","last_undegraded":"2024-04-25T08:52:29.791376+0000","last_fullsized":"2024-04-25T08:52:29.791376+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:02.510174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687140+0000","last_change":"2024-04-25T08:52:28.790503+0000","last_active":"2024-04-25T08:52:30.687140+0000","last_peered":"2024-04-25T08:52:30.687140+0000","last_clean":"2024-04-25T08:52:30.687140+0000","last_became_active":"2024-04-25T08:52:28.790178+0000","last_became_peered":"2024-04-25T08:52:28.790178+0000","last_unstale":"2024-04-25T08:52:30.687140+0000","last_undegraded":"2024-04-25T08:52:30.687140+0000","last_fullsized":"2024-04-25T08:52:30.687140+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:01.741024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767852+0000","last_change":"2024-04-25T08:52:27.779165+0000","last_active":"2024-04-25T08:52:29.767852+0000","last_peered":"2024-04-25T08:52:29.767852+0000","last_clean":"2024-04-25T08:52:29.767852+0000","last_became_active":"2024-04-25T08:52:27.778673+0000","last_became_peered":"2024-04-25T08:52:27.778673+0000","last_unstale":"2024-04-25T08:52:29.767852+0000","last_undegraded":"2024-04-25T08:52:29.767852+0000","last_fullsized":"2024-04-25T08:52:29.767852+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:49.007000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767571+0000","last_change":"2024-04-25T08:52:28.784199+0000","last_active":"2024-04-25T08:52:29.767571+0000","last_peered":"2024-04-25T08:52:29.767571+0000","last_clean":"2024-04-25T08:52:29.767571+0000","last_became_active":"2024-04-25T08:52:28.783910+0000","last_became_peered":"2024-04-25T08:52:28.783910+0000","last_unstale":"2024-04-25T08:52:29.767571+0000","last_undegraded":"2024-04-25T08:52:29.767571+0000","last_fullsized":"2024-04-25T08:52:29.767571+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:42.752762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772286+0000","last_change":"2024-04-25T08:52:27.769647+0000","last_active":"2024-04-25T08:52:29.772286+0000","last_peered":"2024-04-25T08:52:29.772286+0000","last_clean":"2024-04-25T08:52:29.772286+0000","last_became_active":"2024-04-25T08:52:27.769327+0000","last_became_peered":"2024-04-25T08:52:27.769327+0000","last_unstale":"2024-04-25T08:52:29.772286+0000","last_undegraded":"2024-04-25T08:52:29.772286+0000","last_fullsized":"2024-04-25T08:52:29.772286+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:30.432898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.687683+0000","last_change":"2024-04-25T08:52:28.780586+0000","last_active":"2024-04-25T08:52:30.687683+0000","last_peered":"2024-04-25T08:52:30.687683+0000","last_clean":"2024-04-25T08:52:30.687683+0000","last_became_active":"2024-04-25T08:52:28.778484+0000","last_became_peered":"2024-04-25T08:52:28.778484+0000","last_unstale":"2024-04-25T08:52:30.687683+0000","last_undegraded":"2024-04-25T08:52:30.687683+0000","last_fullsized":"2024-04-25T08:52:30.687683+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:03.226604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.772044+0000","last_change":"2024-04-25T08:52:27.772684+0000","last_active":"2024-04-25T08:52:29.772044+0000","last_peered":"2024-04-25T08:52:29.772044+0000","last_clean":"2024-04-25T08:52:29.772044+0000","last_became_active":"2024-04-25T08:52:27.772418+0000","last_became_peered":"2024-04-25T08:52:27.772418+0000","last_unstale":"2024-04-25T08:52:29.772044+0000","last_undegraded":"2024-04-25T08:52:29.772044+0000","last_fullsized":"2024-04-25T08:52:29.772044+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:52.564237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771301+0000","last_change":"2024-04-25T08:52:28.789740+0000","last_active":"2024-04-25T08:52:29.771301+0000","last_peered":"2024-04-25T08:52:29.771301+0000","last_clean":"2024-04-25T08:52:29.771301+0000","last_became_active":"2024-04-25T08:52:28.789225+0000","last_became_peered":"2024-04-25T08:52:28.789225+0000","last_unstale":"2024-04-25T08:52:29.771301+0000","last_undegraded":"2024-04-25T08:52:29.771301+0000","last_fullsized":"2024-04-25T08:52:29.771301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:03.016575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.767418+0000","last_change":"2024-04-25T08:52:27.778931+0000","last_active":"2024-04-25T08:52:29.767418+0000","last_peered":"2024-04-25T08:52:29.767418+0000","last_clean":"2024-04-25T08:52:29.767418+0000","last_became_active":"2024-04-25T08:52:27.778576+0000","last_became_peered":"2024-04-25T08:52:27.778576+0000","last_unstale":"2024-04-25T08:52:29.767418+0000","last_undegraded":"2024-04-25T08:52:29.767418+0000","last_fullsized":"2024-04-25T08:52:29.767418+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:38.899682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.765292+0000","last_change":"2024-04-25T08:52:28.777359+0000","last_active":"2024-04-25T08:52:29.765292+0000","last_peered":"2024-04-25T08:52:29.765292+0000","last_clean":"2024-04-25T08:52:29.765292+0000","last_became_active":"2024-04-25T08:52:28.777070+0000","last_became_peered":"2024-04-25T08:52:28.777070+0000","last_unstale":"2024-04-25T08:52:29.765292+0000","last_undegraded":"2024-04-25T08:52:29.765292+0000","last_fullsized":"2024-04-25T08:52:29.765292+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:05.210040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771211+0000","last_change":"2024-04-25T08:52:27.764732+0000","last_active":"2024-04-25T08:52:29.771211+0000","last_peered":"2024-04-25T08:52:29.771211+0000","last_clean":"2024-04-25T08:52:29.771211+0000","last_became_active":"2024-04-25T08:52:27.764463+0000","last_became_peered":"2024-04-25T08:52:27.764463+0000","last_unstale":"2024-04-25T08:52:29.771211+0000","last_undegraded":"2024-04-25T08:52:29.771211+0000","last_fullsized":"2024-04-25T08:52:29.771211+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:51.777156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771735+0000","last_change":"2024-04-25T08:52:28.801621+0000","last_active":"2024-04-25T08:52:29.771735+0000","last_peered":"2024-04-25T08:52:29.771735+0000","last_clean":"2024-04-25T08:52:29.771735+0000","last_became_active":"2024-04-25T08:52:28.801326+0000","last_became_peered":"2024-04-25T08:52:28.801326+0000","last_unstale":"2024-04-25T08:52:29.771735+0000","last_undegraded":"2024-04-25T08:52:29.771735+0000","last_fullsized":"2024-04-25T08:52:29.771735+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:27.741906+0000","last_clean_scrub_stamp":"2024-04-25T08:52:27.741906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:43.502514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:30.686873+0000","last_change":"2024-04-25T08:52:27.752950+0000","last_active":"2024-04-25T08:52:30.686873+0000","last_peered":"2024-04-25T08:52:30.686873+0000","last_clean":"2024-04-25T08:52:30.686873+0000","last_became_active":"2024-04-25T08:52:27.752624+0000","last_became_peered":"2024-04-25T08:52:27.752624+0000","last_unstale":"2024-04-25T08:52:30.686873+0000","last_undegraded":"2024-04-25T08:52:30.686873+0000","last_fullsized":"2024-04-25T08:52:30.686873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:26.735050+0000","last_clean_scrub_stamp":"2024-04-25T08:52:26.735050+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:59.693746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":359,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:52:29.771414+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:52:29.771414+0000","last_peered":"2024-04-25T08:52:29.771414+0000","last_clean":"2024-04-25T08:52:29.771414+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:52:29.771414+0000","last_undegraded":"2024-04-25T08:52:29.771414+0000","last_fullsized":"2024-04-25T08:52:29.771414+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574977,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2117632,"data_stored":1296218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574977,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":705656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":705234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1519616,"data_stored":706686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1523712,"data_stored":706720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":705074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574977,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":705096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2105344,"data_stored":1295464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:52:33.261 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:52:33.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:52:33.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.454+0000 7fa2e4400640 1 Processor -- start 2024-04-25T08:52:33.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.454+0000 7fa2e4400640 1 -- start start 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e4400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2dc06f150 0x7fa2dc0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e4400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc15e440 con 0x7fa2dc1440d0 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e4400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc15e6e0 con 0x7fa2dc143990 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e4400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2dc15ed30 con 0x7fa2dc06f150 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e2400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa2dc06f150 0x7fa2dc0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e1000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa2dc1440d0 0x7fa2dc0b2720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38244/0 (socket says 172.21.15.46:38244) 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e1000640 1 -- 172.21.15.46:0/400363921 learned_addr learned my addr 172.21.15.46:0/400363921 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e2400640 1 -- 172.21.15.46:0/400363921 >> v1:172.21.15.82:6789/0 conn(0x7fa2dc143990 legacy=0x7fa2dc06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e2400640 1 -- 172.21.15.46:0/400363921 >> v1:172.21.15.46:6789/0 conn(0x7fa2dc1440d0 legacy=0x7fa2dc0b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e2400640 1 -- 172.21.15.46:0/400363921 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc15e9c0 con 0x7fa2dc06f150 2024-04-25T08:52:33.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2dbe00640 1 -- 172.21.15.46:0/400363921 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942216958 0 0) 0x7fa2dc15e440 con 0x7fa2dc1440d0 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.458+0000 7fa2e2400640 1 --2- 172.21.15.46:0/400363921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc06f150 0x7fa2dc0cecc0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc15eeb0 tx=0x7fa2d401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44fd9d6db90713c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2dbe00640 1 -- 172.21.15.46:0/400363921 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40b2070 con 0x7fa2dc06f150 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2dbe00640 1 -- 172.21.15.46:0/400363921 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d40a5920 con 0x7fa2dc06f150 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2dbe00640 1 -- 172.21.15.46:0/400363921 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d40a5cf0 con 0x7fa2dc06f150 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 -- 172.21.15.46:0/400363921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc06f150 msgr2=0x7fa2dc0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 --2- 172.21.15.46:0/400363921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc06f150 0x7fa2dc0cecc0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc15eeb0 tx=0x7fa2d401c950 comp rx=0 tx=0).stop 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 -- 172.21.15.46:0/400363921 shutdown_connections 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 --2- 172.21.15.46:0/400363921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc06f150 0x7fa2dc0cecc0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 -- 172.21.15.46:0/400363921 >> 172.21.15.46:0/400363921 conn(0x7fa2dc06ac10 msgr2=0x7fa2dc06b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 -- 172.21.15.46:0/400363921 shutdown_connections 2024-04-25T08:52:33.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 -- 172.21.15.46:0/400363921 wait complete. 2024-04-25T08:52:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 Processor -- start 2024-04-25T08:52:33.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 -- start start 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2dc06f150 0x7fa2dc15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc143990 0x7fa2dc15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2dc1440d0 0x7fa2dc139840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc1686a0 con 0x7fa2dc143990 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc168520 con 0x7fa2dc1440d0 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e4400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc168820 con 0x7fa2dc06f150 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.462+0000 7fa2e1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc143990 0x7fa2dc15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc143990 0x7fa2dc15cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60742/0 (socket says 172.21.15.46:60742) 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e1000640 1 -- 172.21.15.46:0/1045476172 learned_addr learned my addr 172.21.15.46:0/1045476172 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e1a00640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2dc06f150 0x7fa2dc15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e2400640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2dc1440d0 0x7fa2dc139840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e1000640 1 -- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2dc06f150 msgr2=0x7fa2dc15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e1000640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2dc06f150 0x7fa2dc15dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e1000640 1 -- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2dc1440d0 msgr2=0x7fa2dc139840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e1000640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2dc1440d0 0x7fa2dc139840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e1000640 1 -- 172.21.15.46:0/1045476172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc141aa0 con 0x7fa2dc143990 2024-04-25T08:52:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e1000640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc143990 0x7fa2dc15cee0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c410 tx=0x7fa2c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:33.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2d9600640 1 -- 172.21.15.46:0/1045476172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80074e0 con 0x7fa2dc143990 2024-04-25T08:52:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2d9600640 1 -- 172.21.15.46:0/1045476172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c80047f0 con 0x7fa2dc143990 2024-04-25T08:52:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc141d80 con 0x7fa2dc143990 2024-04-25T08:52:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2d9600640 1 -- 172.21.15.46:0/1045476172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8007680 con 0x7fa2dc143990 2024-04-25T08:52:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc0afd70 con 0x7fa2dc143990 2024-04-25T08:52:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc069010 con 0x7fa2dc143990 2024-04-25T08:52:33.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2d9600640 1 -- 172.21.15.46:0/1045476172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2c8011070 con 0x7fa2dc143990 2024-04-25T08:52:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2d9600640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2bc066f60 0x7fa2bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.466+0000 7fa2d9600640 1 -- 172.21.15.46:0/1045476172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2c8084d60 con 0x7fa2dc143990 2024-04-25T08:52:33.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.470+0000 7fa2e1a00640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2bc066f60 0x7fa2bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.470+0000 7fa2e1a00640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2bc066f60 0x7fa2bc069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0b4c10 tx=0x7fa2d0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:33.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.470+0000 7fa2d9600640 1 -- 172.21.15.46:0/1045476172 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2c80554f0 con 0x7fa2dc143990 2024-04-25T08:52:33.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.690+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa2dc06ece0 con 0x7fa2dc143990 2024-04-25T08:52:33.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2d9600640 1 -- 172.21.15.46:0/1045476172 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa2c8059590 con 0x7fa2dc143990 2024-04-25T08:52:33.685 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-04-25T08:52:33.685 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAyFypmV3fBMBAAY3+akscyKt7qMdpBrhIw7Q== 2024-04-25T08:52:33.686 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2024-04-25T08:52:33.686 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2024-04-25T08:52:33.686 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:52:33.686 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-04-25T08:52:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2bc066f60 msgr2=0x7fa2bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2bc066f60 0x7fa2bc069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc0b4c10 tx=0x7fa2d0009040 comp rx=0 tx=0).stop 2024-04-25T08:52:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc143990 msgr2=0x7fa2dc15cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc143990 0x7fa2dc15cee0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c410 tx=0x7fa2c800c8d0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 shutdown_connections 2024-04-25T08:52:33.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa2bc066f60 0x7fa2bc069420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2dc1440d0 0x7fa2dc139840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2dc143990 0x7fa2dc15cee0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 --2- 172.21.15.46:0/1045476172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2dc06f150 0x7fa2dc15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 >> 172.21.15.46:0/1045476172 conn(0x7fa2dc06ac10 msgr2=0x7fa2dc0cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 shutdown_connections 2024-04-25T08:52:33.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.694+0000 7fa2e4400640 1 -- 172.21.15.46:0/1045476172 wait complete. 2024-04-25T08:52:33.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:33.727 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-04-25T08:52:33.734+0000 7f6fcd800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:52:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f1832800640 1 Processor -- start 2024-04-25T08:52:33.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f1832800640 1 -- start start 2024-04-25T08:52:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f1832800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c15bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f1832800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f182c0cdc50 0x7f182c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f1832800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0cc700 con 0x7f182c0cbeb0 2024-04-25T08:52:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f1832800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c0cc9e0 con 0x7f182c0cdc50 2024-04-25T08:52:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f1832800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0cccc0 con 0x7f182c159230 2024-04-25T08:52:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f182c0cdc50 0x7f182c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f182c0cdc50 0x7f182c15d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60756/0 (socket says 172.21.15.46:60756) 2024-04-25T08:52:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182be00640 1 -- 172.21.15.46:0/3827178543 learned_addr learned my addr 172.21.15.46:0/3827178543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f1822a00640 1 --1- 172.21.15.46:0/3827178543 >> v1:172.21.15.46:6789/0 conn(0x7f182c0cbeb0 0x7f182c0cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38250/0 (socket says 172.21.15.46:38250) 2024-04-25T08:52:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182be00640 1 -- 172.21.15.46:0/3827178543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 msgr2=0x7f182c15bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182b400640 1 --2- 172.21.15.46:0/3827178543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c15bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182be00640 1 --2- 172.21.15.46:0/3827178543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c15bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182be00640 1 -- 172.21.15.46:0/3827178543 >> v1:172.21.15.46:6789/0 conn(0x7f182c0cbeb0 legacy=0x7f182c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182be00640 1 -- 172.21.15.46:0/3827178543 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0ccfa0 con 0x7f182c0cdc50 2024-04-25T08:52:33.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182aa00640 1 -- 172.21.15.46:0/3827178543 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352257512 0 0) 0x7f182c0cc700 con 0x7f182c0cbeb0 2024-04-25T08:52:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.858+0000 7f182b400640 1 --2- 172.21.15.46:0/3827178543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f182be00640 1 --2- 172.21.15.46:0/3827178543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 0x7f182c15d200 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f1824009e50 tx=0x7f182401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1648f6b1c941e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f182aa00640 1 -- 172.21.15.46:0/3827178543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240a5850 con 0x7f182c0cdc50 2024-04-25T08:52:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f182aa00640 1 -- 172.21.15.46:0/3827178543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240a59f0 con 0x7f182c0cdc50 2024-04-25T08:52:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f182aa00640 1 -- 172.21.15.46:0/3827178543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240a5dc0 con 0x7f182c0cdc50 2024-04-25T08:52:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f1832800640 1 -- 172.21.15.46:0/3827178543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 msgr2=0x7f182c15d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f1832800640 1 --2- 172.21.15.46:0/3827178543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 0x7f182c15d200 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f1824009e50 tx=0x7f182401c970 comp rx=0 tx=0).stop 2024-04-25T08:52:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f1832800640 1 -- 172.21.15.46:0/3827178543 shutdown_connections 2024-04-25T08:52:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f1832800640 1 --2- 172.21.15.46:0/3827178543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 0x7f182c15d200 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f1832800640 1 --2- 172.21.15.46:0/3827178543 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c15bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f1832800640 1 -- 172.21.15.46:0/3827178543 >> 172.21.15.46:0/3827178543 conn(0x7f182c0b12a0 msgr2=0x7f182c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f1832800640 1 -- 172.21.15.46:0/3827178543 shutdown_connections 2024-04-25T08:52:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.862+0000 7f1832800640 1 -- 172.21.15.46:0/3827178543 wait complete. 2024-04-25T08:52:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 Processor -- start 2024-04-25T08:52:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 -- start start 2024-04-25T08:52:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f182c0cbeb0 0x7f182c15a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 0x7f182c15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c1f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c15df50 con 0x7f182c0cdc50 2024-04-25T08:52:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0cc4e0 con 0x7f182c0cbeb0 2024-04-25T08:52:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c15e0d0 con 0x7f182c159230 2024-04-25T08:52:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1822a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 0x7f182c15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1822a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 0x7f182c15ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60768/0 (socket says 172.21.15.46:60768) 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1822a00640 1 -- 172.21.15.46:0/1653077694 learned_addr learned my addr 172.21.15.46:0/1653077694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f182b400640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f182c0cbeb0 0x7f182c15a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f182be00640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c1f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1822a00640 1 -- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 msgr2=0x7f182c1f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1822a00640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c1f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1822a00640 1 -- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f182c0cbeb0 msgr2=0x7f182c15a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1822a00640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f182c0cbeb0 0x7f182c15a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1822a00640 1 -- 172.21.15.46:0/1653077694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c1f7140 con 0x7f182c0cdc50 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f182be00640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c1f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1822a00640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 0x7f182c15ad80 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f18140011f0 tx=0x7f181400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1823e00640 1 -- 172.21.15.46:0/1653077694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814018070 con 0x7f182c0cdc50 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1823e00640 1 -- 172.21.15.46:0/1653077694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18140074e0 con 0x7f182c0cdc50 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1823e00640 1 -- 172.21.15.46:0/1653077694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814004a10 con 0x7f182c0cdc50 2024-04-25T08:52:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c1f6e60 con 0x7f182c0cdc50 2024-04-25T08:52:33.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c159d90 con 0x7f182c0cdc50 2024-04-25T08:52:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.866+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c069080 con 0x7f182c0cdc50 2024-04-25T08:52:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.870+0000 7f1823e00640 1 -- 172.21.15.46:0/1653077694 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1814004bb0 con 0x7f182c0cdc50 2024-04-25T08:52:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.870+0000 7f1823e00640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1808066f60 0x7f1808069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.870+0000 7f1823e00640 1 -- 172.21.15.46:0/1653077694 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18140844f0 con 0x7f182c0cdc50 2024-04-25T08:52:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.870+0000 7f182b400640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1808066f60 0x7f1808069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:33.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.870+0000 7f182b400640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1808066f60 0x7f1808069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f181c006fd0 tx=0x7f181c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:33.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:33.874+0000 7f1823e00640 1 -- 172.21.15.46:0/1653077694 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1814054d10 con 0x7f182c0cdc50 2024-04-25T08:52:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f182c0723d0 con 0x7f182c0cdc50 2024-04-25T08:52:34.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1823e00640 1 -- 172.21.15.46:0/1653077694 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4931 (secure 0 0 0) 0x7f1814058db0 con 0x7f182c0cdc50 2024-04-25T08:52:34.067 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:34.067 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1279640422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1279640422},{"type":"v1","addr":"172.21.15.46:6834","nonce":1279640422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:31.172481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8973},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8973":{"gid":8973,"name":"c","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/51204523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":51204523},{"type":"v1","addr":"172.21.15.46:6835","nonce":51204523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:52:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 99 2024-04-25T08:52:34.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1808066f60 msgr2=0x7f1808069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1808066f60 0x7f1808069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f181c006fd0 tx=0x7f181c008040 comp rx=0 tx=0).stop 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 msgr2=0x7f182c15ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 0x7f182c15ad80 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f18140011f0 tx=0x7f181400c9b0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 shutdown_connections 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1808066f60 0x7f1808069420 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f182c159230 0x7f182c1f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c0cdc50 0x7f182c15ad80 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 --2- 172.21.15.46:0/1653077694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f182c0cbeb0 0x7f182c15a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 >> 172.21.15.46:0/1653077694 conn(0x7f182c0b12a0 msgr2=0x7f182c165ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 shutdown_connections 2024-04-25T08:52:34.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.074+0000 7f1832800640 1 -- 172.21.15.46:0/1653077694 wait complete. 2024-04-25T08:52:34.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:52:28.147634+0000', 'modified': '2024-04-25T08:52:31.172481+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8973}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8973': {'gid': 8973, 'name': 'c', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/51204523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 51204523}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 51204523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:52:34.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:52:34.487 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:52:34.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bcec00640 1 Processor -- start 2024-04-25T08:52:34.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bcec00640 1 -- start start 2024-04-25T08:52:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bcec00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5bc8159230 0x7f5bc815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bcec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cbeb0 0x7f5bc80cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bcec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5bc80cda80 0x7f5bc815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bcec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80cc700 con 0x7f5bc80cda80 2024-04-25T08:52:34.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bcec00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80cc9e0 con 0x7f5bc8159230 2024-04-25T08:52:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bcec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80cccc0 con 0x7f5bc80cbeb0 2024-04-25T08:52:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bccc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5bc80cda80 0x7f5bc815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bccc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5bc80cda80 0x7f5bc815f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60770/0 (socket says 172.21.15.46:60770) 2024-04-25T08:52:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bccc00640 1 -- 172.21.15.46:0/2286816267 learned_addr learned my addr 172.21.15.46:0/2286816267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/2286816267 >> v2:172.21.15.82:3300/0 conn(0x7f5bc8159230 0x7f5bc815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bbf400640 1 --2- 172.21.15.46:0/2286816267 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cbeb0 0x7f5bc80cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bccc00640 1 -- 172.21.15.46:0/2286816267 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cbeb0 msgr2=0x7f5bc80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bccc00640 1 --2- 172.21.15.46:0/2286816267 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cbeb0 0x7f5bc80cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bccc00640 1 -- 172.21.15.46:0/2286816267 >> v2:172.21.15.82:3300/0 conn(0x7f5bc8159230 msgr2=0x7f5bc815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bccc00640 1 --2- 172.21.15.46:0/2286816267 >> v2:172.21.15.82:3300/0 conn(0x7f5bc8159230 0x7f5bc815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.642+0000 7f5bccc00640 1 -- 172.21.15.46:0/2286816267 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80ccfa0 con 0x7f5bc80cda80 2024-04-25T08:52:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bccc00640 1 --2- 172.21.15.46:0/2286816267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc80cda80 0x7f5bc815f3e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8072a90 tx=0x7f5bc001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3ab931f3c18382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bc7400640 1 -- 172.21.15.46:0/2286816267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00b0070 con 0x7f5bc80cda80 2024-04-25T08:52:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bc7400640 1 -- 172.21.15.46:0/2286816267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc00ab600 con 0x7f5bc80cda80 2024-04-25T08:52:34.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bc7400640 1 -- 172.21.15.46:0/2286816267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00ab9d0 con 0x7f5bc80cda80 2024-04-25T08:52:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 -- 172.21.15.46:0/2286816267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc80cda80 msgr2=0x7f5bc815f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2286816267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc80cda80 0x7f5bc815f3e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8072a90 tx=0x7f5bc001c1d0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 -- 172.21.15.46:0/2286816267 shutdown_connections 2024-04-25T08:52:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2286816267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc80cda80 0x7f5bc815f3e0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2286816267 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cbeb0 0x7f5bc80cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2286816267 >> v2:172.21.15.82:3300/0 conn(0x7f5bc8159230 0x7f5bc815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 -- 172.21.15.46:0/2286816267 >> 172.21.15.46:0/2286816267 conn(0x7f5bc80b12a0 msgr2=0x7f5bc80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 -- 172.21.15.46:0/2286816267 shutdown_connections 2024-04-25T08:52:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 -- 172.21.15.46:0/2286816267 wait complete. 2024-04-25T08:52:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.646+0000 7f5bcec00640 1 Processor -- start 2024-04-25T08:52:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bcec00640 1 -- start start 2024-04-25T08:52:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bcec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bc80cbeb0 0x7f5bc81419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bcec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cda80 0x7f5bc8141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bcec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8159230 0x7f5bc81fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bcec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc815d200 con 0x7f5bc8159230 2024-04-25T08:52:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bcec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80cc4a0 con 0x7f5bc80cbeb0 2024-04-25T08:52:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bcec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc815d380 con 0x7f5bc80cda80 2024-04-25T08:52:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8159230 0x7f5bc81fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8159230 0x7f5bc81fa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60784/0 (socket says 172.21.15.46:60784) 2024-04-25T08:52:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bccc00640 1 -- 172.21.15.46:0/2902949375 learned_addr learned my addr 172.21.15.46:0/2902949375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bbf400640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cda80 0x7f5bc8141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bc80cbeb0 0x7f5bc81419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bccc00640 1 -- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cda80 msgr2=0x7f5bc8141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bccc00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cda80 0x7f5bc8141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bccc00640 1 -- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bc80cbeb0 msgr2=0x7f5bc81419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bccc00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bc80cbeb0 0x7f5bc81419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bccc00640 1 -- 172.21.15.46:0/2902949375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc8157870 con 0x7f5bc8159230 2024-04-25T08:52:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bc80cbeb0 0x7f5bc81419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:34.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bccc00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8159230 0x7f5bc81fa1c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5bc001a880 tx=0x7f5bc001aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bc4c00640 1 -- 172.21.15.46:0/2902949375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00ba020 con 0x7f5bc8159230 2024-04-25T08:52:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bc4c00640 1 -- 172.21.15.46:0/2902949375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc00b6420 con 0x7f5bc8159230 2024-04-25T08:52:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bc4c00640 1 -- 172.21.15.46:0/2902949375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00b0040 con 0x7f5bc8159230 2024-04-25T08:52:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bcec00640 1 -- 172.21.15.46:0/2902949375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc8157b50 con 0x7f5bc8159230 2024-04-25T08:52:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.650+0000 7f5bcec00640 1 -- 172.21.15.46:0/2902949375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc8153210 con 0x7f5bc8159230 2024-04-25T08:52:34.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bc4c00640 1 -- 172.21.15.46:0/2902949375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bc00bd020 con 0x7f5bc8159230 2024-04-25T08:52:34.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bc4c00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ba4066f60 0x7f5ba4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bc4c00640 1 -- 172.21.15.46:0/2902949375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5bc011f490 con 0x7f5bc8159230 2024-04-25T08:52:34.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ba4066f60 0x7f5ba4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bd1fbc000 1 Processor -- start 2024-04-25T08:52:34.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bd1fbc000 1 -- start start 2024-04-25T08:52:34.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ba4066f60 0x7f5ba4069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800b9f0 tx=0x7f5bb8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 0x55e57e932d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e8e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e8e8f30 0x55e57e8e9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57e7d05f0 con 0x55e57e933240 2024-04-25T08:52:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57e737da0 con 0x55e57e932920 2024-04-25T08:52:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e57e6c0150 con 0x55e57e8e8f30 2024-04-25T08:52:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e8e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e8e89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60798/0 (socket says 172.21.15.46:60798) 2024-04-25T08:52:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bbf400640 1 -- 172.21.15.46:0/1004057497 learned_addr learned my addr 172.21.15.46:0/1004057497 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/1004057497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 0x55e57e932d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bccc00640 1 --2- 172.21.15.46:0/1004057497 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e8e8f30 0x55e57e8e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bbf400640 1 -- 172.21.15.46:0/1004057497 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e8e8f30 msgr2=0x55e57e8e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bbf400640 1 --2- 172.21.15.46:0/1004057497 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e8e8f30 0x55e57e8e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bbf400640 1 -- 172.21.15.46:0/1004057497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 msgr2=0x55e57e932d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bbf400640 1 --2- 172.21.15.46:0/1004057497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 0x55e57e932d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bbf400640 1 -- 172.21.15.46:0/1004057497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e57e6d9ca0 con 0x55e57e933240 2024-04-25T08:52:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.654+0000 7f5bbf400640 1 --2- 172.21.15.46:0/1004057497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e8e89f0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009f30 tx=0x7f5bb0003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73bea202022d664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bbcc00640 1 -- 172.21.15.46:0/1004057497 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0015420 con 0x55e57e933240 2024-04-25T08:52:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bbcc00640 1 -- 172.21.15.46:0/1004057497 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb001a070 con 0x55e57e933240 2024-04-25T08:52:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bbcc00640 1 -- 172.21.15.46:0/1004057497 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00157c0 con 0x55e57e933240 2024-04-25T08:52:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1004057497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 msgr2=0x55e57e8e89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1004057497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e8e89f0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009f30 tx=0x7f5bb0003e30 comp rx=0 tx=0).stop 2024-04-25T08:52:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1004057497 shutdown_connections 2024-04-25T08:52:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1004057497 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e8e8f30 0x55e57e8e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1004057497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e8e89f0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1004057497 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 0x55e57e932d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1004057497 >> 172.21.15.46:0/1004057497 conn(0x55e57e8ef2a0 msgr2=0x55e57e8e72d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1004057497 shutdown_connections 2024-04-25T08:52:34.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1004057497 wait complete. 2024-04-25T08:52:34.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 -- start start 2024-04-25T08:52:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 Processor -- start 2024-04-25T08:52:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e8e8f30 0x55e57e939c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 0x55e57e93a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 0x55e57e8ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57e6f2990 con 0x55e57e8e8f30 2024-04-25T08:52:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57e733e40 con 0x55e57e932920 2024-04-25T08:52:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e57e6c3900 con 0x55e57e933240 2024-04-25T08:52:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bd1fbc000 10 client.0 _ll_register_callbacks cb 0x55e57e7eb1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bc7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e8e8f30 0x55e57e939c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bc7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e8e8f30 0x55e57e939c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60814/0 (socket says 172.21.15.46:60814) 2024-04-25T08:52:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bc7e00640 1 -- 172.21.15.46:0/3051415487 learned_addr learned my addr 172.21.15.46:0/3051415487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bbf400640 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 0x55e57e93a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bccc00640 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 0x55e57e8ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bc7e00640 1 -- 172.21.15.46:0/3051415487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 msgr2=0x55e57e8ecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 0x55e57e8ecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bc7e00640 1 -- 172.21.15.46:0/3051415487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 msgr2=0x55e57e93a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 0x55e57e93a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bc7e00640 1 -- 172.21.15.46:0/3051415487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e57e6f0630 con 0x55e57e8e8f30 2024-04-25T08:52:34.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e8e8f30 0x55e57e939c80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8014690 tx=0x7f5bb80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f1032c184ac4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.658+0000 7f5bb5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bb5600640 1 -- 172.21.15.46:0/3051415487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8025070 con 0x55e57e8e8f30 2024-04-25T08:52:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bb5600640 1 -- 172.21.15.46:0/3051415487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5bb80159e0 con 0x55e57e8e8f30 2024-04-25T08:52:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/3051415487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bb001a070 con 0x55e57e8e8f30 2024-04-25T08:52:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bb5600640 1 -- 172.21.15.46:0/3051415487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb8030050 con 0x55e57e8e8f30 2024-04-25T08:52:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 10 client.9117 fetch_fsmap learned FSMap version 99 2024-04-25T08:52:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/3051415487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55e57e6c03a0 con 0x55e57e8e8f30 2024-04-25T08:52:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bb5600640 1 -- 172.21.15.46:0/3051415487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb001a070 con 0x55e57e8e8f30 2024-04-25T08:52:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bb5600640 1 -- 172.21.15.46:0/3051415487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5bb8033020 con 0x55e57e8e8f30 2024-04-25T08:52:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 10 client.9117 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T08:52:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 10 client.9117 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:52:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 20 client.9117 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x55e57e8ee160 0x55e57e943ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 4 client.9117 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:52:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/3051415487 --> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e57e6d3a50 con 0x55e57e8ee160 2024-04-25T08:52:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bbf400640 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x55e57e8ee160 0x55e57e943ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bbf400640 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x55e57e8ee160 0x55e57e943ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81287f530ede9705 server_cookie=621de53028ed7cc3 in_seq=0 out_seq=0 2024-04-25T08:52:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bb5600640 10 client.9117 ms_handle_connect on v2:172.21.15.46:6833/51204523 2024-04-25T08:52:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bb5600640 1 -- 172.21.15.46:0/3051415487 <== mds.0 v2:172.21.15.46:6833/51204523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e57e6d3a50 con 0x55e57e8ee160 2024-04-25T08:52:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bb5600640 10 client.9117 handle_command_reply: tid=0 2024-04-25T08:52:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 client.9117 shutdown 2024-04-25T08:52:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/3051415487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e8e8f30 msgr2=0x55e57e939c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e8e8f30 0x55e57e939c80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8014690 tx=0x7f5bb80046d0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/3051415487 shutdown_connections 2024-04-25T08:52:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x55e57e8ee160 0x55e57e943ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 0x55e57e8ecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e932920 0x55e57e93a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bb5600640 0 client.9117 ms_handle_reset on v2:172.21.15.46:6833/51204523 2024-04-25T08:52:34.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/3051415487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e8e8f30 0x55e57e939c80 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/3051415487 >> 172.21.15.46:0/3051415487 conn(0x55e57e8ef2a0 msgr2=0x55e57e8e79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/3051415487 shutdown_connections 2024-04-25T08:52:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/3051415487 wait complete. 2024-04-25T08:52:34.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.662+0000 7f5bd1fbc000 20 client.9117 trim_cache size 0 max 16384 2024-04-25T08:52:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.678+0000 7f5bd1fbc000 1 Processor -- start 2024-04-25T08:52:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.678+0000 7f5bd1fbc000 1 -- start start 2024-04-25T08:52:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.678+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 0x55e57e98e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.678+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e932920 0x55e57e98e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.678+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e98ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.678+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8202500 con 0x55e57e933240 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.678+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8156e80 con 0x55e57e8e8f30 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.678+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8157480 con 0x55e57e932920 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e98ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e98ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60826/0 (socket says 172.21.15.46:60826) 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 -- 172.21.15.46:0/4115248618 learned_addr learned my addr 172.21.15.46:0/4115248618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bbf400640 1 --2- 172.21.15.46:0/4115248618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e932920 0x55e57e98e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/4115248618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 0x55e57e98e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 -- 172.21.15.46:0/4115248618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e932920 msgr2=0x55e57e98e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 --2- 172.21.15.46:0/4115248618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e932920 0x55e57e98e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 -- 172.21.15.46:0/4115248618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 msgr2=0x55e57e98e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 --2- 172.21.15.46:0/4115248618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 0x55e57e98e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 -- 172.21.15.46:0/4115248618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e57e933070 con 0x55e57e933240 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 --2- 172.21.15.46:0/4115248618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e98ee20 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0097b70 tx=0x7f5bc00c1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa3d1f107c95fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bb7e00640 1 -- 172.21.15.46:0/4115248618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00f3cc0 con 0x55e57e933240 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bb7e00640 1 -- 172.21.15.46:0/4115248618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc00f8280 con 0x55e57e933240 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bb7e00640 1 -- 172.21.15.46:0/4115248618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00ec1e0 con 0x55e57e933240 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/4115248618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 msgr2=0x55e57e98ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/4115248618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e98ee20 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0097b70 tx=0x7f5bc00c1040 comp rx=0 tx=0).stop 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/4115248618 shutdown_connections 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/4115248618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e933240 0x55e57e98ee20 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/4115248618 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e932920 0x55e57e98e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/4115248618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 0x55e57e98e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/4115248618 >> 172.21.15.46:0/4115248618 conn(0x55e57e8ef2a0 msgr2=0x55e57e9301a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/4115248618 shutdown_connections 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/4115248618 wait complete. 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 -- start start 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 Processor -- start 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 0x55e57e9937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e932920 0x55e57e993d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 0x55e57e97a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57e73c3d0 con 0x55e57e932920 2024-04-25T08:52:34.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57e7f3d50 con 0x55e57e8e8f30 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e57e6d2980 con 0x55e57e933240 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bd1fbc000 10 client.0 _ll_register_callbacks cb 0x7f5bb0009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e932920 0x55e57e993d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bbf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e932920 0x55e57e993d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60840/0 (socket says 172.21.15.46:60840) 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bbf400640 1 -- 172.21.15.46:0/1885073114 learned_addr learned my addr 172.21.15.46:0/1885073114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 0x55e57e9937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.682+0000 7f5bccc00640 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 0x55e57e97a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bbf400640 1 -- 172.21.15.46:0/1885073114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 msgr2=0x55e57e97a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bbf400640 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 0x55e57e97a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bbf400640 1 -- 172.21.15.46:0/1885073114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 msgr2=0x55e57e9937d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bbf400640 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 0x55e57e9937d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bbf400640 1 -- 172.21.15.46:0/1885073114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e57e7b9600 con 0x55e57e932920 2024-04-25T08:52:34.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bbf400640 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e932920 0x55e57e993d10 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0016000 tx=0x7f5bb0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d24d406e54663b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 1 -- 172.21.15.46:0/1885073114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0029520 con 0x55e57e932920 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 1 -- 172.21.15.46:0/1885073114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5bb00296c0 con 0x55e57e932920 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1885073114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bc00f8280 con 0x55e57e932920 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 1 -- 172.21.15.46:0/1885073114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb0038050 con 0x55e57e932920 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 10 client.9123 fetch_fsmap learned FSMap version 99 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1885073114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55e57e6cc9b0 con 0x55e57e932920 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 1 -- 172.21.15.46:0/1885073114 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc00f8280 con 0x55e57e932920 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 1 -- 172.21.15.46:0/1885073114 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5bb003b020 con 0x55e57e932920 2024-04-25T08:52:34.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 10 client.9123 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 10 client.9123 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 20 client.9123 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x55e57e97aca0 0x55e57e97b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 4 client.9123 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1885073114 --> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e57e7452e0 con 0x55e57e97aca0 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x55e57e97aca0 0x55e57e97b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 10 client.9123 ms_handle_connect on v2:172.21.15.46:6833/51204523 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bc7e00640 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x55e57e97aca0 0x55e57e97b080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866729d3243b6566 server_cookie=d291ecea8a6de3c4 in_seq=0 out_seq=0 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 1 -- 172.21.15.46:0/1885073114 <== mds.0 v2:172.21.15.46:6833/51204523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55e57e7452e0 con 0x55e57e97aca0 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 10 client.9123 handle_command_reply: tid=0 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 client.9123 shutdown 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1885073114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e932920 msgr2=0x55e57e993d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e932920 0x55e57e993d10 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0016000 tx=0x7f5bb0004ac0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1885073114 shutdown_connections 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x55e57e97aca0 0x55e57e97b080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e57e933240 0x55e57e97a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e57e932920 0x55e57e993d10 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 --2- 172.21.15.46:0/1885073114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e57e8e8f30 0x55e57e9937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bb5600640 0 client.9123 ms_handle_reset on v2:172.21.15.46:6833/51204523 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1885073114 >> 172.21.15.46:0/1885073114 conn(0x55e57e8ef2a0 msgr2=0x55e57e931140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1885073114 shutdown_connections 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 1 -- 172.21.15.46:0/1885073114 wait complete. 2024-04-25T08:52:34.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bd1fbc000 20 client.9123 trim_cache size 0 max 16384 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8973, 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.9184551550000002, 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 99, 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 121, 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 121, 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 11.768005455999999 2024-04-25T08:52:34.696 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:52:34.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 -- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ba4066f60 msgr2=0x7f5ba4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ba4066f60 0x7f5ba4069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800b9f0 tx=0x7f5bb8008040 comp rx=0 tx=0).stop 2024-04-25T08:52:34.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 -- 172.21.15.46:0/2902949375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8159230 msgr2=0x7f5bc81fa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8159230 0x7f5bc81fa1c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5bc001a880 tx=0x7f5bc001aa10 comp rx=0 tx=0).stop 2024-04-25T08:52:34.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 -- 172.21.15.46:0/2902949375 shutdown_connections 2024-04-25T08:52:34.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5ba4066f60 0x7f5ba4069420 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bc8159230 0x7f5bc81fa1c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bc80cda80 0x7f5bc8141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 --2- 172.21.15.46:0/2902949375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bc80cbeb0 0x7f5bc81419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 -- 172.21.15.46:0/2902949375 >> 172.21.15.46:0/2902949375 conn(0x7f5bc80b12a0 msgr2=0x7f5bc80b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 -- 172.21.15.46:0/2902949375 shutdown_connections 2024-04-25T08:52:34.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.686+0000 7f5bcec00640 1 -- 172.21.15.46:0/2902949375 wait complete. 2024-04-25T08:52:34.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:52:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.850+0000 7f4b37600640 1 Processor -- start 2024-04-25T08:52:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.850+0000 7f4b37600640 1 -- start start 2024-04-25T08:52:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b37600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b300af3f0 0x7f4b300b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b37600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b3006f630 0x7f4b300b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b37600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3015e180 con 0x7f4b3006db40 2024-04-25T08:52:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b37600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b300b3ca0 con 0x7f4b300af3f0 2024-04-25T08:52:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b37600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3015e460 con 0x7f4b3006f630 2024-04-25T08:52:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b34c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b300af3f0 0x7f4b300b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b34c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b300af3f0 0x7f4b300b2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60846/0 (socket says 172.21.15.46:60846) 2024-04-25T08:52:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b34c00640 1 -- 172.21.15.46:0/3320585894 learned_addr learned my addr 172.21.15.46:0/3320585894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b35600640 1 --2- 172.21.15.46:0/3320585894 >> v2:172.21.15.82:3300/0 conn(0x7f4b3006f630 0x7f4b300b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b34c00640 1 -- 172.21.15.46:0/3320585894 >> v2:172.21.15.82:3300/0 conn(0x7f4b3006f630 msgr2=0x7f4b300b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b34c00640 1 --2- 172.21.15.46:0/3320585894 >> v2:172.21.15.82:3300/0 conn(0x7f4b3006f630 0x7f4b300b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b34c00640 1 -- 172.21.15.46:0/3320585894 >> v1:172.21.15.82:6789/0 conn(0x7f4b3006db40 legacy=0x7f4b3006e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b2fe00640 1 -- 172.21.15.46:0/3320585894 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939983570 0 0) 0x7f4b3015e180 con 0x7f4b3006db40 2024-04-25T08:52:34.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b34c00640 1 -- 172.21.15.46:0/3320585894 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3015e740 con 0x7f4b300af3f0 2024-04-25T08:52:34.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.854+0000 7f4b34c00640 1 --2- 172.21.15.46:0/3320585894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b300af3f0 0x7f4b300b2e40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f4b300b4660 tx=0x7f4b2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38790ac3c38025e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b2fe00640 1 -- 172.21.15.46:0/3320585894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b200b0070 con 0x7f4b300af3f0 2024-04-25T08:52:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b2fe00640 1 -- 172.21.15.46:0/3320585894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b200ab600 con 0x7f4b300af3f0 2024-04-25T08:52:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b2fe00640 1 -- 172.21.15.46:0/3320585894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b200ab9d0 con 0x7f4b300af3f0 2024-04-25T08:52:34.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 -- 172.21.15.46:0/3320585894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b300af3f0 msgr2=0x7f4b300b2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 --2- 172.21.15.46:0/3320585894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b300af3f0 0x7f4b300b2e40 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f4b300b4660 tx=0x7f4b2001c520 comp rx=0 tx=0).stop 2024-04-25T08:52:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 -- 172.21.15.46:0/3320585894 shutdown_connections 2024-04-25T08:52:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 --2- 172.21.15.46:0/3320585894 >> v2:172.21.15.82:3300/0 conn(0x7f4b3006f630 0x7f4b300b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 --2- 172.21.15.46:0/3320585894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b300af3f0 0x7f4b300b2e40 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 -- 172.21.15.46:0/3320585894 >> 172.21.15.46:0/3320585894 conn(0x7f4b300cfd90 msgr2=0x7f4b300d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 -- 172.21.15.46:0/3320585894 shutdown_connections 2024-04-25T08:52:34.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 -- 172.21.15.46:0/3320585894 wait complete. 2024-04-25T08:52:34.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 Processor -- start 2024-04-25T08:52:34.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 -- start start 2024-04-25T08:52:34.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3006db40 0x7f4b3014e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b3006f630 0x7f4b3015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b300af3f0 0x7f4b301394a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3006bda0 con 0x7f4b3006db40 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3006bc20 con 0x7f4b300af3f0 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b37600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3016a6f0 con 0x7f4b3006f630 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b34c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3006db40 0x7f4b3014e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b34c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3006db40 0x7f4b3014e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60862/0 (socket says 172.21.15.46:60862) 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.858+0000 7f4b34c00640 1 -- 172.21.15.46:0/1671860095 learned_addr learned my addr 172.21.15.46:0/1671860095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b27e00640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b3006f630 0x7f4b3015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b35600640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b300af3f0 0x7f4b301394a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b34c00640 1 -- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b3006f630 msgr2=0x7f4b3015cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b34c00640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b3006f630 0x7f4b3015cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b34c00640 1 -- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b300af3f0 msgr2=0x7f4b301394a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b34c00640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b300af3f0 0x7f4b301394a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b34c00640 1 -- 172.21.15.46:0/1671860095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b30156f10 con 0x7f4b3006db40 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b27e00640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b3006f630 0x7f4b3015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:34.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b35600640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b300af3f0 0x7f4b301394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:34.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b34c00640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3006db40 0x7f4b3014e4a0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f4b200ae040 tx=0x7f4b2001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b2d600640 1 -- 172.21.15.46:0/1671860095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b200b0040 con 0x7f4b3006db40 2024-04-25T08:52:34.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b2d600640 1 -- 172.21.15.46:0/1671860095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b200bc070 con 0x7f4b3006db40 2024-04-25T08:52:34.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b2d600640 1 -- 172.21.15.46:0/1671860095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b200b6640 con 0x7f4b3006db40 2024-04-25T08:52:34.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b301571f0 con 0x7f4b3006db40 2024-04-25T08:52:34.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b301577b0 con 0x7f4b3006db40 2024-04-25T08:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b30069080 con 0x7f4b3006db40 2024-04-25T08:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b2d600640 1 -- 172.21.15.46:0/1671860095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b200b67e0 con 0x7f4b3006db40 2024-04-25T08:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b2d600640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b0c066f60 0x7f4b0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.862+0000 7f4b2d600640 1 -- 172.21.15.46:0/1671860095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b2011f510 con 0x7f4b3006db40 2024-04-25T08:52:34.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.866+0000 7f4b27e00640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b0c066f60 0x7f4b0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.866+0000 7f4b27e00640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b0c066f60 0x7f4b0c069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4b3015d820 tx=0x7f4b18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:34.866+0000 7f4b2d600640 1 -- 172.21.15.46:0/1671860095 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b20016110 con 0x7f4b3006db40 2024-04-25T08:52:35.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.066+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4b30158070 con 0x7f4b3006db40 2024-04-25T08:52:35.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.066+0000 7f4b2d600640 1 -- 172.21.15.46:0/1671860095 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4b200f02f0 con 0x7f4b3006db40 2024-04-25T08:52:35.058 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:52:35.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b0c066f60 msgr2=0x7f4b0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:35.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b0c066f60 0x7f4b0c069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4b3015d820 tx=0x7f4b18009040 comp rx=0 tx=0).stop 2024-04-25T08:52:35.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3006db40 msgr2=0x7f4b3014e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:35.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3006db40 0x7f4b3014e4a0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f4b200ae040 tx=0x7f4b2001ad60 comp rx=0 tx=0).stop 2024-04-25T08:52:35.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 shutdown_connections 2024-04-25T08:52:35.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4b0c066f60 0x7f4b0c069420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b300af3f0 0x7f4b301394a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4b3006f630 0x7f4b3015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 --2- 172.21.15.46:0/1671860095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3006db40 0x7f4b3014e4a0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 >> 172.21.15.46:0/1671860095 conn(0x7f4b300cfd90 msgr2=0x7f4b300b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:35.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 shutdown_connections 2024-04-25T08:52:35.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35.070+0000 7f4b37600640 1 -- 172.21.15.46:0/1671860095 wait complete. 2024-04-25T08:52:35.080 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:52:35.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:52:35.081 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:52:35.081 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:52:35.081 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:35.081 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:52:35.081 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:52:35.081 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:52:35.081 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:52:35.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.081 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:52:35.090 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:52:35.091 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41848sec preferred_lft 41848sec 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:52:35.092 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:52:35.093 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:35.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.095 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:52:35.157 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:52:35.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.159 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:52:35.216 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:52:35.217 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:52:35.217 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:52:35.217 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:52:35.217 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:52:35.217 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:52:35.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.218 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:52:35.218 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:52:35.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:52:35.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:52:35.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:52:35.218 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:52:35.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.314 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:52:35.323 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:52:35.323 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:52:35.324 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:52:35.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:52:35.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:52:35.392 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:35.447 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:52:35.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:52:35.448 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:35.499 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:52:35.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:35.589 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:35.589 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:52:35.589 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:52:35.604 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:35.604 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:52:35.604 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:52:35.604 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:52:35.604 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:52:35.614 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:35.614 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:52:35.614 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:52:35.614 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:52:35.614 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:52:35.615 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:52:35.615 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:52:35.615 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:52:35.620 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:52:35.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:35.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.641 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:52:35.701 INFO:teuthology.orchestra.run.smithi046.stdout:client9132 2024-04-25T08:52:35.702 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:35.702 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> parent = os.path.join(mount_point, "parent") 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> os.mkdir(parent) 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> flushed = os.path.join(parent, "flushed") 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> os.mkdir(flushed) 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> for f in ["alpha", "bravo", "charlie"]: 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-25T08:52:35.756 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:35.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:35.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:35.885 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T08:52:35.946 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-04-25T08:52:35.946 INFO:teuthology.orchestra.run.smithi046.stdout:./parent 2024-04-25T08:52:35.946 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed 2024-04-25T08:52:35.946 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/charlie 2024-04-25T08:52:35.946 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/alpha 2024-04-25T08:52:35.946 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/bravo 2024-04-25T08:52:35.947 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:35.947 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:35.947 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:35.947 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:35.947 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T08:52:35.947 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:36.061 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:52:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:36.070 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:36.070 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:36.070 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:36.070 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:36.070 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T08:52:36.070 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:36.181 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-25T08:52:36.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:36.190 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:36.190 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:36.191 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:36.191 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:36.191 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T08:52:36.191 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:36.300 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-04-25T08:52:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:36.311 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:36.311 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:36.311 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:36.311 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:36.311 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T08:52:36.312 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:36.425 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627780 2024-04-25T08:52:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:36.434 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:36.434 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:36.435 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:36.435 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:36.435 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T08:52:36.435 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:36.545 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-04-25T08:52:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:36.554 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:36.554 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:36.554 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:36.554 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:36.554 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T08:52:36.554 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:36.665 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627779 2024-04-25T08:52:36.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:36.674 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:36.674 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:36.730 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:36.731 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:36.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:36.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.974+0000 7f54bec00640 1 Processor -- start 2024-04-25T08:52:36.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.974+0000 7f54bec00640 1 -- start start 2024-04-25T08:52:36.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b815df60 0x7f54b80716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b80b3f20 con 0x7f54b80b3140 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b8071e10 con 0x7f54b806da90 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b8072990 con 0x7f54b815df60 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f54b80b3140 0x7f54b80b3520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38252/0 (socket says 172.21.15.46:38252) 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7e00640 1 -- 172.21.15.46:0/2061357888 learned_addr learned my addr 172.21.15.46:0/2061357888 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54afe00640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b815df60 0x7f54b80716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129009480 0 0) 0x7f54b80b3f20 con 0x7f54b80b3140 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f549c003530 con 0x7f54b80b3140 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733497517 0 0) 0x7f54b8071e10 con 0x7f54b806da90 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54b80b3f20 con 0x7f54b806da90 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2969187794 0 0) 0x7f549c003530 con 0x7f54b80b3140 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b815df60 msgr2=0x7f54b80716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b815df60 0x7f54b80716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 >> v1:172.21.15.82:6789/0 conn(0x7f54b806da90 legacy=0x7f54b806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b80720f0 con 0x7f54b80b3140 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f54a8001680 con 0x7f54b80b3140 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54afe00640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b815df60 0x7f54b80716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 >> v1:172.21.15.46:6789/0 conn(0x7f54b80b3140 legacy=0x7f54b80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549c003b80 0x7f549c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b815df60 0x7f549c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c004d50 0x7f549c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b8072990 con 0x7f549c004d50 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c00bb90 con 0x7f549c003b80 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c00bd90 con 0x7f54b815df60 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bcc00640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c004d50 0x7f549c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54b7e00640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549c003b80 0x7f549c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54afe00640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b815df60 0x7f549c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bcc00640 1 -- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b815df60 msgr2=0x7f549c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bcc00640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b815df60 0x7f549c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bcc00640 1 -- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549c003b80 msgr2=0x7f549c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bcc00640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549c003b80 0x7f549c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.978+0000 7f54bcc00640 1 -- 172.21.15.46:0/2061357888 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c003530 con 0x7f549c004d50 2024-04-25T08:52:36.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54bcc00640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c004d50 0x7f549c0051d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f54b0007fd0 tx=0x7f54b001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516327c84406738a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:36.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0001c20 con 0x7f549c004d50 2024-04-25T08:52:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b0003ba0 con 0x7f549c004d50 2024-04-25T08:52:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54b7400640 1 -- 172.21.15.46:0/2061357888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b00050d0 con 0x7f549c004d50 2024-04-25T08:52:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54bec00640 1 -- 172.21.15.46:0/2061357888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c004d50 msgr2=0x7f549c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54bec00640 1 --2- 172.21.15.46:0/2061357888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f549c004d50 0x7f549c0051d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f54b0007fd0 tx=0x7f54b001c8b0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54bcc00640 1 -- 172.21.15.46:0/2061357888 reap_dead start 2024-04-25T08:52:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54bec00640 1 -- 172.21.15.46:0/2061357888 shutdown_connections 2024-04-25T08:52:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54bec00640 1 -- 172.21.15.46:0/2061357888 >> 172.21.15.46:0/2061357888 conn(0x7f54b80b1750 msgr2=0x7f54b80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54bec00640 1 -- 172.21.15.46:0/2061357888 shutdown_connections 2024-04-25T08:52:36.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.982+0000 7f54bec00640 1 -- 172.21.15.46:0/2061357888 wait complete. 2024-04-25T08:52:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 Processor -- start 2024-04-25T08:52:36.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 -- start start 2024-04-25T08:52:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549c004d50 0x7f54b80cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b80cd3a0 0x7f54b815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b8167820 0x7f54b8167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b8147940 con 0x7f54b8167820 2024-04-25T08:52:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b81477c0 con 0x7f549c004d50 2024-04-25T08:52:36.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b81696c0 con 0x7f54b80cd3a0 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b8167820 0x7f54b8167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b8167820 0x7f54b8167ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60880/0 (socket says 172.21.15.46:60880) 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bcc00640 1 -- 172.21.15.46:0/1739149678 learned_addr learned my addr 172.21.15.46:0/1739149678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54b7e00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549c004d50 0x7f54b80cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54afe00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b80cd3a0 0x7f54b815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bcc00640 1 -- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b80cd3a0 msgr2=0x7f54b815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bcc00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b80cd3a0 0x7f54b815d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bcc00640 1 -- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549c004d50 msgr2=0x7f54b80cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bcc00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549c004d50 0x7f54b80cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bcc00640 1 -- 172.21.15.46:0/1739149678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b80cf190 con 0x7f54b8167820 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bcc00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b8167820 0x7f54b8167ca0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f54b001cd90 tx=0x7f549c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54b4c00640 1 -- 172.21.15.46:0/1739149678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0001c00 con 0x7f54b8167820 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54b4c00640 1 -- 172.21.15.46:0/1739149678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b00b6070 con 0x7f54b8167820 2024-04-25T08:52:36.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54b4c00640 1 -- 172.21.15.46:0/1739149678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b0005630 con 0x7f54b8167820 2024-04-25T08:52:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b81614c0 con 0x7f54b8167820 2024-04-25T08:52:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b80cdc40 con 0x7f54b8167820 2024-04-25T08:52:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.986+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b81681e0 con 0x7f54b8167820 2024-04-25T08:52:36.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.990+0000 7f54b4c00640 1 -- 172.21.15.46:0/1739149678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f54b00a64a0 con 0x7f54b8167820 2024-04-25T08:52:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.990+0000 7f54b4c00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5494066f90 0x7f5494069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.990+0000 7f54b4c00640 1 -- 172.21.15.46:0/1739149678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f54b011eb30 con 0x7f54b8167820 2024-04-25T08:52:36.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.990+0000 7f54b7e00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5494066f90 0x7f5494069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.990+0000 7f54b7e00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5494066f90 0x7f5494069450 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f54a8001ec0 tx=0x7f54a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:36.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:36.994+0000 7f54b4c00640 1 -- 172.21.15.46:0/1739149678 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f54b00165f0 con 0x7f54b8167820 2024-04-25T08:52:37.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.158+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54b8161690 con 0x7f54b8167820 2024-04-25T08:52:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54b4c00640 1 -- 172.21.15.46:0/1739149678 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16904 (secure 0 0 0) 0x7f54b00ef910 con 0x7f54b8167820 2024-04-25T08:52:37.152 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:37.152 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:29.760504+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:37.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5494066f90 msgr2=0x7f5494069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5494066f90 0x7f5494069450 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f54a8001ec0 tx=0x7f54a8009040 comp rx=0 tx=0).stop 2024-04-25T08:52:37.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b8167820 msgr2=0x7f54b8167ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b8167820 0x7f54b8167ca0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f54b001cd90 tx=0x7f549c005840 comp rx=0 tx=0).stop 2024-04-25T08:52:37.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 shutdown_connections 2024-04-25T08:52:37.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5494066f90 0x7f5494069450 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b8167820 0x7f54b8167ca0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f54b80cd3a0 0x7f54b815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 --2- 172.21.15.46:0/1739149678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549c004d50 0x7f54b80cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 >> 172.21.15.46:0/1739149678 conn(0x7f54b80b1750 msgr2=0x7f54b80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 shutdown_connections 2024-04-25T08:52:37.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.162+0000 7f54bec00640 1 -- 172.21.15.46:0/1739149678 wait complete. 2024-04-25T08:52:37.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:25.941254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:27.155001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:52:37.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:37.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65cee00640 1 Processor -- start 2024-04-25T08:52:37.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65cee00640 1 -- start start 2024-04-25T08:52:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65cee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65c806dc70 0x7f65c80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65cee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 0x7f65c815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65cee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80cd930 con 0x7f65c8145310 2024-04-25T08:52:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65cee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80cdbd0 con 0x7f65c806dc70 2024-04-25T08:52:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65cee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80cdeb0 con 0x7f65c80cf1b0 2024-04-25T08:52:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65c7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65c806dc70 0x7f65c80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65c7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65c806dc70 0x7f65c80cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60894/0 (socket says 172.21.15.46:60894) 2024-04-25T08:52:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65c7400640 1 -- 172.21.15.46:0/4167946339 learned_addr learned my addr 172.21.15.46:0/4167946339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.326+0000 7f65cce00640 1 --2- 172.21.15.46:0/4167946339 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 0x7f65c815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65c7400640 1 -- 172.21.15.46:0/4167946339 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 msgr2=0x7f65c815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65c7400640 1 --2- 172.21.15.46:0/4167946339 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 0x7f65c815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65c7400640 1 -- 172.21.15.46:0/4167946339 >> v1:172.21.15.46:6789/0 conn(0x7f65c8145310 legacy=0x7f65c806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65c7400640 1 -- 172.21.15.46:0/4167946339 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80ce190 con 0x7f65c806dc70 2024-04-25T08:52:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65c7400640 1 --2- 172.21.15.46:0/4167946339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c806dc70 0x7f65c80cec70 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009870 tx=0x7f65b401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12757ffd76b3980 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65c6a00640 1 -- 172.21.15.46:0/4167946339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40ab640 con 0x7f65c806dc70 2024-04-25T08:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65c6a00640 1 -- 172.21.15.46:0/4167946339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b40b0070 con 0x7f65c806dc70 2024-04-25T08:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65c6a00640 1 -- 172.21.15.46:0/4167946339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40aba00 con 0x7f65c806dc70 2024-04-25T08:52:37.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65cee00640 1 -- 172.21.15.46:0/4167946339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c806dc70 msgr2=0x7f65c80cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.330+0000 7f65cee00640 1 --2- 172.21.15.46:0/4167946339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c806dc70 0x7f65c80cec70 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f65b4009870 tx=0x7f65b401c530 comp rx=0 tx=0).stop 2024-04-25T08:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 -- 172.21.15.46:0/4167946339 shutdown_connections 2024-04-25T08:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 --2- 172.21.15.46:0/4167946339 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 0x7f65c815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 --2- 172.21.15.46:0/4167946339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c806dc70 0x7f65c80cec70 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 -- 172.21.15.46:0/4167946339 >> 172.21.15.46:0/4167946339 conn(0x7f65c806ac10 msgr2=0x7f65c806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 -- 172.21.15.46:0/4167946339 shutdown_connections 2024-04-25T08:52:37.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 -- 172.21.15.46:0/4167946339 wait complete. 2024-04-25T08:52:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 Processor -- start 2024-04-25T08:52:37.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 -- start start 2024-04-25T08:52:37.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 0x7f65c813eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c8145310 0x7f65c8158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80b1400 0x7f65c81413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c81688a0 con 0x7f65c8145310 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8168720 con 0x7f65c80b1400 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cee00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8168a20 con 0x7f65c80cf1b0 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c8145310 0x7f65c8158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c8145310 0x7f65c8158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60908/0 (socket says 172.21.15.46:60908) 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7400640 1 -- 172.21.15.46:0/225245743 learned_addr learned my addr 172.21.15.46:0/225245743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65cce00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80b1400 0x7f65c81413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7e00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 0x7f65c813eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7400640 1 -- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 msgr2=0x7f65c813eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7400640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 0x7f65c813eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7400640 1 -- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80b1400 msgr2=0x7f65c81413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7400640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80b1400 0x7f65c81413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7400640 1 -- 172.21.15.46:0/225245743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c813da80 con 0x7f65c8145310 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7e00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 0x7f65c813eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.334+0000 7f65c7400640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c8145310 0x7f65c8158340 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f65b40ad040 tx=0x7f65b401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65bfe00640 1 -- 172.21.15.46:0/225245743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40be3f0 con 0x7f65c8145310 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65bfe00640 1 -- 172.21.15.46:0/225245743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b40be590 con 0x7f65c8145310 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c813dd60 con 0x7f65c8145310 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65bfe00640 1 -- 172.21.15.46:0/225245743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b40be980 con 0x7f65c8145310 2024-04-25T08:52:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c813e320 con 0x7f65c8145310 2024-04-25T08:52:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c8069010 con 0x7f65c8145310 2024-04-25T08:52:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65bfe00640 1 -- 172.21.15.46:0/225245743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65b40b43f0 con 0x7f65c8145310 2024-04-25T08:52:37.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65bfe00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65a8066f90 0x7f65a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65bfe00640 1 -- 172.21.15.46:0/225245743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65b411f460 con 0x7f65c8145310 2024-04-25T08:52:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.338+0000 7f65c7e00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65a8066f90 0x7f65a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.342+0000 7f65c7e00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65a8066f90 0x7f65a8069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f65b8001d70 tx=0x7f65b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.342+0000 7f65bfe00640 1 -- 172.21.15.46:0/225245743 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65b40c4060 con 0x7f65c8145310 2024-04-25T08:52:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65c8142370 con 0x7f65c8145310 2024-04-25T08:52:37.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65bfe00640 1 -- 172.21.15.46:0/225245743 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65b4016150 con 0x7f65c8145310 2024-04-25T08:52:37.498 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:52:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65a8066f90 msgr2=0x7f65a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65a8066f90 0x7f65a8069450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f65b8001d70 tx=0x7f65b8009040 comp rx=0 tx=0).stop 2024-04-25T08:52:37.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c8145310 msgr2=0x7f65c8158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c8145310 0x7f65c8158340 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f65b40ad040 tx=0x7f65b401ad70 comp rx=0 tx=0).stop 2024-04-25T08:52:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 shutdown_connections 2024-04-25T08:52:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f65a8066f90 0x7f65a8069450 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80b1400 0x7f65c81413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65c8145310 0x7f65c8158340 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 --2- 172.21.15.46:0/225245743 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f65c80cf1b0 0x7f65c813eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 >> 172.21.15.46:0/225245743 conn(0x7f65c806ac10 msgr2=0x7f65c8158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 shutdown_connections 2024-04-25T08:52:37.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.506+0000 7f65cee00640 1 -- 172.21.15.46:0/225245743 wait complete. 2024-04-25T08:52:37.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:52:37.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.670+0000 7f97c4a00640 1 Processor -- start 2024-04-25T08:52:37.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.670+0000 7f97c4a00640 1 -- start start 2024-04-25T08:52:37.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97c4a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f97c015dae0 0x7f97c0072600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97c4a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c006ea60 0x7f97c00acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97c4a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00b3850 con 0x7f97c006e620 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97c4a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c006e190 con 0x7f97c015dae0 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97c4a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00b3af0 con 0x7f97c006ea60 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97b5400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f97c006e620 0x7f97c006bc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38266/0 (socket says 172.21.15.46:38266) 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97b5400640 1 -- 172.21.15.46:0/774639024 learned_addr learned my addr 172.21.15.46:0/774639024 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97bde00640 1 --2- 172.21.15.46:0/774639024 >> v2:172.21.15.82:3300/0 conn(0x7f97c015dae0 0x7f97c0072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97be800640 1 --2- 172.21.15.46:0/774639024 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c006ea60 0x7f97c00acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97bd400640 1 -- 172.21.15.46:0/774639024 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495434536 0 0) 0x7f97c00b3850 con 0x7f97c006e620 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97bd400640 1 -- 172.21.15.46:0/774639024 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a4003530 con 0x7f97c006e620 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97bde00640 1 -- 172.21.15.46:0/774639024 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c006ea60 msgr2=0x7f97c00acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97bde00640 1 --2- 172.21.15.46:0/774639024 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c006ea60 0x7f97c00acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97bde00640 1 -- 172.21.15.46:0/774639024 >> v1:172.21.15.46:6789/0 conn(0x7f97c006e620 legacy=0x7f97c006bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.674+0000 7f97bde00640 1 -- 172.21.15.46:0/774639024 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00b3dd0 con 0x7f97c015dae0 2024-04-25T08:52:37.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97bde00640 1 --2- 172.21.15.46:0/774639024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c015dae0 0x7f97c0072600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f97c006e310 tx=0x7f97b001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2da7576dd8de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97bd400640 1 -- 172.21.15.46:0/774639024 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b00b2070 con 0x7f97c015dae0 2024-04-25T08:52:37.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97bd400640 1 -- 172.21.15.46:0/774639024 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b00a59e0 con 0x7f97c015dae0 2024-04-25T08:52:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97bd400640 1 -- 172.21.15.46:0/774639024 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b00a5db0 con 0x7f97c015dae0 2024-04-25T08:52:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 -- 172.21.15.46:0/774639024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c015dae0 msgr2=0x7f97c0072600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 --2- 172.21.15.46:0/774639024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c015dae0 0x7f97c0072600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f97c006e310 tx=0x7f97b001c920 comp rx=0 tx=0).stop 2024-04-25T08:52:37.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 -- 172.21.15.46:0/774639024 shutdown_connections 2024-04-25T08:52:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 --2- 172.21.15.46:0/774639024 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c006ea60 0x7f97c00acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 --2- 172.21.15.46:0/774639024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c015dae0 0x7f97c0072600 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 -- 172.21.15.46:0/774639024 >> 172.21.15.46:0/774639024 conn(0x7f97c0071ae0 msgr2=0x7f97c0071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 -- 172.21.15.46:0/774639024 shutdown_connections 2024-04-25T08:52:37.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 -- 172.21.15.46:0/774639024 wait complete. 2024-04-25T08:52:37.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 Processor -- start 2024-04-25T08:52:37.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.678+0000 7f97c4a00640 1 -- start start 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97c4a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c006e620 0x7f97c0160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c006ea60 0x7f97c01f9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97c4a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c015dae0 0x7f97c01fa320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97c4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c006f6d0 con 0x7f97c006ea60 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97c4a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c015dec0 con 0x7f97c006e620 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97c4a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c006f850 con 0x7f97c015dae0 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c006ea60 0x7f97c01f9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c006ea60 0x7f97c01f9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60924/0 (socket says 172.21.15.46:60924) 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b5400640 1 -- 172.21.15.46:0/3727798753 learned_addr learned my addr 172.21.15.46:0/3727798753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97bde00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c006e620 0x7f97c0160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97be800640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c015dae0 0x7f97c01fa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b5400640 1 -- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c015dae0 msgr2=0x7f97c01fa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b5400640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c015dae0 0x7f97c01fa320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b5400640 1 -- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c006e620 msgr2=0x7f97c0160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b5400640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c006e620 0x7f97c0160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b5400640 1 -- 172.21.15.46:0/3727798753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c014ca30 con 0x7f97c006ea60 2024-04-25T08:52:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b5400640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c006ea60 0x7f97c01f9de0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f97a80010f0 tx=0x7f97a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b6a00640 1 -- 172.21.15.46:0/3727798753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8017070 con 0x7f97c006ea60 2024-04-25T08:52:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c0156390 con 0x7f97c006ea60 2024-04-25T08:52:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b6a00640 1 -- 172.21.15.46:0/3727798753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a8005d10 con 0x7f97c006ea60 2024-04-25T08:52:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b6a00640 1 -- 172.21.15.46:0/3727798753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8007310 con 0x7f97c006ea60 2024-04-25T08:52:37.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c0156670 con 0x7f97c006ea60 2024-04-25T08:52:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c0069010 con 0x7f97c006ea60 2024-04-25T08:52:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.682+0000 7f97b6a00640 1 -- 172.21.15.46:0/3727798753 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97a801a070 con 0x7f97c006ea60 2024-04-25T08:52:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.686+0000 7f97b6a00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f979c066f60 0x7f979c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.686+0000 7f97b6a00640 1 -- 172.21.15.46:0/3727798753 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97a80846a0 con 0x7f97c006ea60 2024-04-25T08:52:37.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.686+0000 7f97bde00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f979c066f60 0x7f979c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.686+0000 7f97bde00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f979c066f60 0x7f979c069420 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f97b00af000 tx=0x7f97b00a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.690+0000 7f97b6a00640 1 -- 172.21.15.46:0/3727798753 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97a8055490 con 0x7f97c006ea60 2024-04-25T08:52:37.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.850+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97c0067e90 con 0x7f97c006ea60 2024-04-25T08:52:37.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.850+0000 7f97b6a00640 1 -- 172.21.15.46:0/3727798753 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97a8054d90 con 0x7f97c006ea60 2024-04-25T08:52:37.843 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:52:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f979c066f60 msgr2=0x7f979c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f979c066f60 0x7f979c069420 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f97b00af000 tx=0x7f97b00a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c006ea60 msgr2=0x7f97c01f9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c006ea60 0x7f97c01f9de0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f97a80010f0 tx=0x7f97a800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 shutdown_connections 2024-04-25T08:52:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f979c066f60 0x7f979c069420 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f97c015dae0 0x7f97c01fa320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97c006ea60 0x7f97c01f9de0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 --2- 172.21.15.46:0/3727798753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97c006e620 0x7f97c0160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 >> 172.21.15.46:0/3727798753 conn(0x7f97c0071ae0 msgr2=0x7f97c0071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 shutdown_connections 2024-04-25T08:52:37.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:37.854+0000 7f97c4a00640 1 -- 172.21.15.46:0/3727798753 wait complete. 2024-04-25T08:52:37.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:52:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b3b800640 1 Processor -- start 2024-04-25T08:52:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b3b800640 1 -- start start 2024-04-25T08:52:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b3b800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0b340cf210 0x7f0b34160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b3b800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b3409a040 0x7f0b3415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b3b800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b340cc530 con 0x7f0b34099c60 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b3b800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b340cc300 con 0x7f0b3409a040 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b3b800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b340cc810 con 0x7f0b340cf210 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b39800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b3409a040 0x7f0b3415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b39800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b3409a040 0x7f0b3415eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60934/0 (socket says 172.21.15.46:60934) 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b39800640 1 -- 172.21.15.46:0/396551156 learned_addr learned my addr 172.21.15.46:0/396551156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b2be00640 1 --2- 172.21.15.46:0/396551156 >> v2:172.21.15.82:3300/0 conn(0x7f0b340cf210 0x7f0b34160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b39800640 1 -- 172.21.15.46:0/396551156 >> v2:172.21.15.82:3300/0 conn(0x7f0b340cf210 msgr2=0x7f0b34160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b39800640 1 --2- 172.21.15.46:0/396551156 >> v2:172.21.15.82:3300/0 conn(0x7f0b340cf210 0x7f0b34160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b39800640 1 -- 172.21.15.46:0/396551156 >> v1:172.21.15.82:6789/0 conn(0x7f0b34099c60 legacy=0x7f0b340cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.018+0000 7f0b39800640 1 -- 172.21.15.46:0/396551156 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b340ccaf0 con 0x7f0b3409a040 2024-04-25T08:52:38.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b39800640 1 --2- 172.21.15.46:0/396551156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3409a040 0x7f0b3415eb60 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f0b340730c0 tx=0x7f0b2c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f189fe33708b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b33e00640 1 -- 172.21.15.46:0/396551156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0a57d0 con 0x7f0b3409a040 2024-04-25T08:52:38.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b33e00640 1 -- 172.21.15.46:0/396551156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c0a5970 con 0x7f0b3409a040 2024-04-25T08:52:38.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b33e00640 1 -- 172.21.15.46:0/396551156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0a5d40 con 0x7f0b3409a040 2024-04-25T08:52:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 -- 172.21.15.46:0/396551156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3409a040 msgr2=0x7f0b3415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 --2- 172.21.15.46:0/396551156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3409a040 0x7f0b3415eb60 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f0b340730c0 tx=0x7f0b2c01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 -- 172.21.15.46:0/396551156 shutdown_connections 2024-04-25T08:52:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 --2- 172.21.15.46:0/396551156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b3409a040 0x7f0b3415eb60 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 --2- 172.21.15.46:0/396551156 >> v2:172.21.15.82:3300/0 conn(0x7f0b340cf210 0x7f0b34160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 -- 172.21.15.46:0/396551156 >> 172.21.15.46:0/396551156 conn(0x7f0b340b3130 msgr2=0x7f0b340b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 -- 172.21.15.46:0/396551156 shutdown_connections 2024-04-25T08:52:38.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 -- 172.21.15.46:0/396551156 wait complete. 2024-04-25T08:52:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 Processor -- start 2024-04-25T08:52:38.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.022+0000 7f0b3b800640 1 -- start start 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b3b800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0b34099c60 0x7f0b3406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b3b800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3409a040 0x7f0b3415c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b3b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b340cf210 0x7f0b34155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b3b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340b1930 con 0x7f0b340cf210 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b3b800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340b17b0 con 0x7f0b3409a040 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b3b800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b340b1ab0 con 0x7f0b34099c60 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b39800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b340cf210 0x7f0b34155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b39800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b340cf210 0x7f0b34155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60946/0 (socket says 172.21.15.46:60946) 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b39800640 1 -- 172.21.15.46:0/945192731 learned_addr learned my addr 172.21.15.46:0/945192731 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:38.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b39800640 1 -- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0b34099c60 msgr2=0x7f0b3406d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b38e00640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0b34099c60 0x7f0b3406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b2be00640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3409a040 0x7f0b3415c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b39800640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0b34099c60 0x7f0b3406d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b39800640 1 -- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3409a040 msgr2=0x7f0b3415c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b39800640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3409a040 0x7f0b3415c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b39800640 1 -- 172.21.15.46:0/945192731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3415dd70 con 0x7f0b340cf210 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b38e00640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0b34099c60 0x7f0b3406d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b2be00640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3409a040 0x7f0b3415c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b39800640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b340cf210 0x7f0b34155d30 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0af040 tx=0x7f0b2c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b31600640 1 -- 172.21.15.46:0/945192731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0b8670 con 0x7f0b340cf210 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b31600640 1 -- 172.21.15.46:0/945192731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c0be070 con 0x7f0b340cf210 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b31600640 1 -- 172.21.15.46:0/945192731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0b8a70 con 0x7f0b340cf210 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3406f3a0 con 0x7f0b340cf210 2024-04-25T08:52:38.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3415e2f0 con 0x7f0b340cf210 2024-04-25T08:52:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.026+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b34069010 con 0x7f0b340cf210 2024-04-25T08:52:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.030+0000 7f0b31600640 1 -- 172.21.15.46:0/945192731 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b2c0b8c10 con 0x7f0b340cf210 2024-04-25T08:52:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.030+0000 7f0b31600640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b10066f60 0x7f0b10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.030+0000 7f0b31600640 1 -- 172.21.15.46:0/945192731 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b2c120370 con 0x7f0b340cf210 2024-04-25T08:52:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.030+0000 7f0b38e00640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b10066f60 0x7f0b10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.030+0000 7f0b38e00640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b10066f60 0x7f0b10069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b34164ea0 tx=0x7f0b24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.034+0000 7f0b31600640 1 -- 172.21.15.46:0/945192731 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b2c0165e0 con 0x7f0b340cf210 2024-04-25T08:52:38.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b340b35b0 con 0x7f0b340cf210 2024-04-25T08:52:38.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b31600640 1 -- 172.21.15.46:0/945192731 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b2c0f1150 con 0x7f0b340cf210 2024-04-25T08:52:38.189 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b10066f60 msgr2=0x7f0b10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b10066f60 0x7f0b10069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b34164ea0 tx=0x7f0b24009040 comp rx=0 tx=0).stop 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b340cf210 msgr2=0x7f0b34155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b340cf210 0x7f0b34155d30 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0af040 tx=0x7f0b2c019790 comp rx=0 tx=0).stop 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 shutdown_connections 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0b10066f60 0x7f0b10069420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b340cf210 0x7f0b34155d30 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b3409a040 0x7f0b3415c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 --2- 172.21.15.46:0/945192731 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0b34099c60 0x7f0b3406d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 >> 172.21.15.46:0/945192731 conn(0x7f0b340b3130 msgr2=0x7f0b340ce050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 shutdown_connections 2024-04-25T08:52:38.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.198+0000 7f0b3b800640 1 -- 172.21.15.46:0/945192731 wait complete. 2024-04-25T08:52:38.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:52:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d62c00640 1 Processor -- start 2024-04-25T08:52:38.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d62c00640 1 -- start start 2024-04-25T08:52:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d62c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3d5c15ec80 0x7f3d5c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d62c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 0x7f3d5c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d62c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d5c0d0530 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d62c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d5c15f060 con 0x7f3d5c15ec80 2024-04-25T08:52:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d62c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0cc020 con 0x7f3d5c09a110 2024-04-25T08:52:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d62200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 0x7f3d5c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d61800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3d5c15ec80 0x7f3d5c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d60e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3d5c0cf4d0 0x7f3d5c099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:55316/0 (socket says 172.21.15.46:55316) 2024-04-25T08:52:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d60e00640 1 -- 172.21.15.46:0/1390604498 learned_addr learned my addr 172.21.15.46:0/1390604498 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:38.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d61800640 1 -- 172.21.15.46:0/1390604498 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 msgr2=0x7f3d5c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d61800640 1 --2- 172.21.15.46:0/1390604498 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 0x7f3d5c15b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d61800640 1 -- 172.21.15.46:0/1390604498 >> v1:172.21.15.82:6789/0 conn(0x7f3d5c0cf4d0 legacy=0x7f3d5c099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d61800640 1 -- 172.21.15.46:0/1390604498 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c1601d0 con 0x7f3d5c15ec80 2024-04-25T08:52:38.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.362+0000 7f3d62200640 1 --2- 172.21.15.46:0/1390604498 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 0x7f3d5c15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:38.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d61800640 1 --2- 172.21.15.46:0/1390604498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d5c15ec80 0x7f3d5c15fc90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3d500097c0 tx=0x7f3d5001c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=825d1a02d33b1d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d5be00640 1 -- 172.21.15.46:0/1390604498 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500b1070 con 0x7f3d5c15ec80 2024-04-25T08:52:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d5be00640 1 -- 172.21.15.46:0/1390604498 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d500a8050 con 0x7f3d5c15ec80 2024-04-25T08:52:38.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d5be00640 1 -- 172.21.15.46:0/1390604498 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500ac860 con 0x7f3d5c15ec80 2024-04-25T08:52:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d62c00640 1 -- 172.21.15.46:0/1390604498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d5c15ec80 msgr2=0x7f3d5c15fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d62c00640 1 --2- 172.21.15.46:0/1390604498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d5c15ec80 0x7f3d5c15fc90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3d500097c0 tx=0x7f3d5001c3e0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d62c00640 1 -- 172.21.15.46:0/1390604498 shutdown_connections 2024-04-25T08:52:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d62c00640 1 --2- 172.21.15.46:0/1390604498 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 0x7f3d5c15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d62c00640 1 --2- 172.21.15.46:0/1390604498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d5c15ec80 0x7f3d5c15fc90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d62c00640 1 -- 172.21.15.46:0/1390604498 >> 172.21.15.46:0/1390604498 conn(0x7f3d5c0b2f90 msgr2=0x7f3d5c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d62c00640 1 -- 172.21.15.46:0/1390604498 shutdown_connections 2024-04-25T08:52:38.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.366+0000 7f3d62c00640 1 -- 172.21.15.46:0/1390604498 wait complete. 2024-04-25T08:52:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 Processor -- start 2024-04-25T08:52:38.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 -- start start 2024-04-25T08:52:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 0x7f3d5c0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d5c0cf4d0 0x7f3d5c1647e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d5c15ec80 0x7f3d5c164d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0d0250 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c0cff70 con 0x7f3d5c15ec80 2024-04-25T08:52:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c15d080 con 0x7f3d5c09a110 2024-04-25T08:52:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d60e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d5c0cf4d0 0x7f3d5c1647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d60e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d5c0cf4d0 0x7f3d5c1647e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47878/0 (socket says 172.21.15.46:47878) 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d60e00640 1 -- 172.21.15.46:0/431857282 learned_addr learned my addr 172.21.15.46:0/431857282 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d61800640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 0x7f3d5c0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62200640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d5c15ec80 0x7f3d5c164d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d60e00640 1 -- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 msgr2=0x7f3d5c0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d60e00640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 0x7f3d5c0c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d60e00640 1 -- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d5c15ec80 msgr2=0x7f3d5c164d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d60e00640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d5c15ec80 0x7f3d5c164d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d60e00640 1 -- 172.21.15.46:0/431857282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c0c82c0 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d60e00640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d5c0cf4d0 0x7f3d5c1647e0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800c430 tx=0x7f3d4800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d59600640 1 -- 172.21.15.46:0/431857282 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d48017070 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d59600640 1 -- 172.21.15.46:0/431857282 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d48005cf0 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d59600640 1 -- 172.21.15.46:0/431857282 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d480072f0 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c165350 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.370+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c165910 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.374+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c069010 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.374+0000 7f3d59600640 1 -- 172.21.15.46:0/431857282 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d4801a070 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.374+0000 7f3d59600640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d3c066f60 0x7f3d3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.374+0000 7f3d59600640 1 -- 172.21.15.46:0/431857282 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d480844c0 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.374+0000 7f3d61800640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d3c066f60 0x7f3d3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.374+0000 7f3d61800640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d3c066f60 0x7f3d3c069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3d500ae000 tx=0x7f3d50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.378+0000 7f3d59600640 1 -- 172.21.15.46:0/431857282 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d5c0d0530 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3d5c0b3ea0 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d59600640 1 -- 172.21.15.46:0/431857282 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3d48054c50 con 0x7f3d5c0cf4d0 2024-04-25T08:52:38.545 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:52:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d3c066f60 msgr2=0x7f3d3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d3c066f60 0x7f3d3c069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3d500ae000 tx=0x7f3d50008040 comp rx=0 tx=0).stop 2024-04-25T08:52:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d5c0cf4d0 msgr2=0x7f3d5c1647e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d5c0cf4d0 0x7f3d5c1647e0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800c430 tx=0x7f3d4800c830 comp rx=0 tx=0).stop 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 shutdown_connections 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3d3c066f60 0x7f3d3c069420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d5c15ec80 0x7f3d5c164d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d5c0cf4d0 0x7f3d5c1647e0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 --2- 172.21.15.46:0/431857282 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3d5c09a110 0x7f3d5c0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 >> 172.21.15.46:0/431857282 conn(0x7f3d5c0b2f90 msgr2=0x7f3d5c157090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 shutdown_connections 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.554+0000 7f3d62c00640 1 -- 172.21.15.46:0/431857282 wait complete. 2024-04-25T08:52:38.566 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:52:38.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:38.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:38.627 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:52:38.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:38.627 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:52:38.643 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:52:38.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T08:52:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc31c00640 1 Processor -- start 2024-04-25T08:52:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc31c00640 1 -- start start 2024-04-25T08:52:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc31c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc2c0afdf0 0x7fbc2c06ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc31c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c06b2a0 0x7fbc2c0d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc31c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c06b700 con 0x7fbc2c15d900 2024-04-25T08:52:38.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc31c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c06fca0 con 0x7fbc2c0afdf0 2024-04-25T08:52:38.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc31c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c06ff80 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc2b400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c06b2a0 0x7fbc2c0d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc22000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc2c0afdf0 0x7fbc2c06ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc2aa00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbc2c15d900 0x7fbc2c0af6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:55328/0 (socket says 172.21.15.46:55328) 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.842+0000 7fbc2aa00640 1 -- 172.21.15.46:0/2245587575 learned_addr learned my addr 172.21.15.46:0/2245587575 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2a000640 1 -- 172.21.15.46:0/2245587575 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142726598 0 0) 0x7fbc2c06b700 con 0x7fbc2c15d900 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2a000640 1 -- 172.21.15.46:0/2245587575 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc10003530 con 0x7fbc2c15d900 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2b400640 1 -- 172.21.15.46:0/2245587575 >> v2:172.21.15.82:3300/0 conn(0x7fbc2c0afdf0 msgr2=0x7fbc2c06ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2b400640 1 --2- 172.21.15.46:0/2245587575 >> v2:172.21.15.82:3300/0 conn(0x7fbc2c0afdf0 0x7fbc2c06ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2b400640 1 -- 172.21.15.46:0/2245587575 >> v1:172.21.15.82:6789/0 conn(0x7fbc2c15d900 legacy=0x7fbc2c0af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2b400640 1 -- 172.21.15.46:0/2245587575 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c070260 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc22000640 1 --2- 172.21.15.46:0/2245587575 >> v2:172.21.15.82:3300/0 conn(0x7fbc2c0afdf0 0x7fbc2c06ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:38.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2b400640 1 --2- 172.21.15.46:0/2245587575 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c06b2a0 0x7fbc2c0d0210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009d90 tx=0x7fbc2401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24651ae7c1fee60c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2a000640 1 -- 172.21.15.46:0/2245587575 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240b2070 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2a000640 1 -- 172.21.15.46:0/2245587575 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc240a5890 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.846+0000 7fbc2a000640 1 -- 172.21.15.46:0/2245587575 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240a5c60 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 -- 172.21.15.46:0/2245587575 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c06b2a0 msgr2=0x7fbc2c0d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 --2- 172.21.15.46:0/2245587575 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c06b2a0 0x7fbc2c0d0210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009d90 tx=0x7fbc2401c8e0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 -- 172.21.15.46:0/2245587575 shutdown_connections 2024-04-25T08:52:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 --2- 172.21.15.46:0/2245587575 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c06b2a0 0x7fbc2c0d0210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 --2- 172.21.15.46:0/2245587575 >> v2:172.21.15.82:3300/0 conn(0x7fbc2c0afdf0 0x7fbc2c06ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 -- 172.21.15.46:0/2245587575 >> 172.21.15.46:0/2245587575 conn(0x7fbc2c06e3e0 msgr2=0x7fbc2c06e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 -- 172.21.15.46:0/2245587575 shutdown_connections 2024-04-25T08:52:38.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 -- 172.21.15.46:0/2245587575 wait complete. 2024-04-25T08:52:38.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 Processor -- start 2024-04-25T08:52:38.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 -- start start 2024-04-25T08:52:38.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c06b2a0 0x7fbc2c143740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c0afdf0 0x7fbc2c151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c15d900 0x7fbc2c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0d10e0 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0d0f60 con 0x7fbc2c0afdf0 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc31c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0d1260 con 0x7fbc2c15d900 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c06b2a0 0x7fbc2c143740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c06b2a0 0x7fbc2c143740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47882/0 (socket says 172.21.15.46:47882) 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2aa00640 1 -- 172.21.15.46:0/1746719821 learned_addr learned my addr 172.21.15.46:0/1746719821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc22000640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c0afdf0 0x7fbc2c151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2b400640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c15d900 0x7fbc2c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2aa00640 1 -- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c15d900 msgr2=0x7fbc2c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c15d900 0x7fbc2c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2aa00640 1 -- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c0afdf0 msgr2=0x7fbc2c151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c0afdf0 0x7fbc2c151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2aa00640 1 -- 172.21.15.46:0/1746719821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c1522e0 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.850+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c06b2a0 0x7fbc2c143740 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0010f0 tx=0x7fbc1c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc23400640 1 -- 172.21.15.46:0/1746719821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c017070 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc23400640 1 -- 172.21.15.46:0/1746719821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c005d10 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc23400640 1 -- 172.21.15.46:0/1746719821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c007310 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc31c00640 1 -- 172.21.15.46:0/1746719821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c1525c0 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc31c00640 1 -- 172.21.15.46:0/1746719821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c152b80 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc23400640 1 -- 172.21.15.46:0/1746719821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc1c01a070 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc23400640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbc08066f60 0x7fbc08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc23400640 1 -- 172.21.15.46:0/1746719821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc1c084610 con 0x7fbc2c06b2a0 2024-04-25T08:52:38.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc350e8000 1 Processor -- start 2024-04-25T08:52:38.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc22000640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbc08066f60 0x7fbc08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.854+0000 7fbc350e8000 1 -- start start 2024-04-25T08:52:38.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc22000640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbc08066f60 0x7fbc08069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009580 tx=0x7fbc14009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d19d0 0x5564459d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d22f0 0x556445987b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x556445988480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564457d4ff0 con 0x5564459880a0 2024-04-25T08:52:38.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644575d150 con 0x5564459d22f0 2024-04-25T08:52:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564457d1090 con 0x5564459d19d0 2024-04-25T08:52:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x556445988480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x556445988480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47884/0 (socket says 172.21.15.46:47884) 2024-04-25T08:52:38.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2b400640 1 -- 172.21.15.46:0/167829979 learned_addr learned my addr 172.21.15.46:0/167829979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d19d0 0x5564459d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc22000640 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d22f0 0x556445987b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2b400640 1 -- 172.21.15.46:0/167829979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d19d0 msgr2=0x5564459d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2b400640 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d19d0 0x5564459d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2b400640 1 -- 172.21.15.46:0/167829979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d22f0 msgr2=0x556445987b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2b400640 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d22f0 0x556445987b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2b400640 1 -- 172.21.15.46:0/167829979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55644596b0d0 con 0x5564459880a0 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc22000640 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d22f0 0x556445987b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc2b400640 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x556445988480 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fbc240b0040 tx=0x7fbc24004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce3c12705ae8236 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc1b400640 1 -- 172.21.15.46:0/167829979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240be070 con 0x5564459880a0 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc1b400640 1 -- 172.21.15.46:0/167829979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc240b3040 con 0x5564459880a0 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc1b400640 1 -- 172.21.15.46:0/167829979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240b2040 con 0x5564459880a0 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 -- 172.21.15.46:0/167829979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 msgr2=0x556445988480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x556445988480 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fbc240b0040 tx=0x7fbc24004ac0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 -- 172.21.15.46:0/167829979 shutdown_connections 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x556445988480 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d22f0 0x556445987b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 --2- 172.21.15.46:0/167829979 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d19d0 0x5564459d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 -- 172.21.15.46:0/167829979 >> 172.21.15.46:0/167829979 conn(0x556445984020 msgr2=0x556445986440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 -- 172.21.15.46:0/167829979 shutdown_connections 2024-04-25T08:52:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 -- 172.21.15.46:0/167829979 wait complete. 2024-04-25T08:52:38.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 -- start start 2024-04-25T08:52:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.858+0000 7fbc350e8000 1 Processor -- start 2024-04-25T08:52:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x5564459b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d19d0 0x5564459b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x5564459b7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556445799250 con 0x5564459880a0 2024-04-25T08:52:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644578f690 con 0x5564459d19d0 2024-04-25T08:52:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55644575da40 con 0x5564459d22f0 2024-04-25T08:52:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 10 client.0 _ll_register_callbacks cb 0x556445972e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:38.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x5564459b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x5564459b67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47892/0 (socket says 172.21.15.46:47892) 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2aa00640 1 -- 172.21.15.46:0/2243308923 learned_addr learned my addr 172.21.15.46:0/2243308923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc22000640 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d19d0 0x5564459b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2b400640 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x5564459b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2aa00640 1 -- 172.21.15.46:0/2243308923 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 msgr2=0x5564459b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x5564459b7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2aa00640 1 -- 172.21.15.46:0/2243308923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d19d0 msgr2=0x5564459b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d19d0 0x5564459b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2aa00640 1 -- 172.21.15.46:0/2243308923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564458821c0 con 0x5564459880a0 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x5564459b67f0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c054ca0 tx=0x7fbc1c054e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39126fca6e89ade0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc0fe00640 1 -- 172.21.15.46:0/2243308923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0530d0 con 0x5564459880a0 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc0fe00640 1 -- 172.21.15.46:0/2243308923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc1c054160 con 0x5564459880a0 2024-04-25T08:52:38.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 -- 172.21.15.46:0/2243308923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc240b3040 con 0x5564459880a0 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc0fe00640 1 -- 172.21.15.46:0/2243308923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc1c054360 con 0x5564459880a0 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 10 client.9171 fetch_fsmap learned FSMap version 99 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 -- 172.21.15.46:0/2243308923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55644587e170 con 0x5564459880a0 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc0fe00640 1 -- 172.21.15.46:0/2243308923 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc240b3040 con 0x5564459880a0 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc0fe00640 1 -- 172.21.15.46:0/2243308923 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc1c0840c0 con 0x5564459880a0 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 10 client.9171 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 10 client.9171 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 20 client.9171 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x5564459e1c80 0x5564459e4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 4 client.9171 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 -- 172.21.15.46:0/2243308923 --> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556445770c60 con 0x5564459e1c80 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc22000640 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x5564459e1c80 0x5564459e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc22000640 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x5564459e1c80 0x5564459e4070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ffab3e70893641 server_cookie=d732dbab3b1faf4e in_seq=0 out_seq=0 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc0fe00640 10 client.9171 ms_handle_connect on v2:172.21.15.46:6833/51204523 2024-04-25T08:52:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc0fe00640 1 -- 172.21.15.46:0/2243308923 <== mds.0 v2:172.21.15.46:6833/51204523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556445770c60 con 0x5564459e1c80 2024-04-25T08:52:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc0fe00640 10 client.9171 handle_command_reply: tid=0 2024-04-25T08:52:38.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.862+0000 7fbc350e8000 1 client.9171 shutdown 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 -- 172.21.15.46:0/2243308923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 msgr2=0x5564459b67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x5564459b67f0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c054ca0 tx=0x7fbc1c054e50 comp rx=0 tx=0).stop 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 -- 172.21.15.46:0/2243308923 shutdown_connections 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x5564459e1c80 0x5564459e4070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x5564459b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459d19d0 0x5564459b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc0fe00640 0 client.9171 ms_handle_reset on v2:172.21.15.46:6833/51204523 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 --2- 172.21.15.46:0/2243308923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459880a0 0x5564459b67f0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 -- 172.21.15.46:0/2243308923 >> 172.21.15.46:0/2243308923 conn(0x556445984020 msgr2=0x556445986ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 -- 172.21.15.46:0/2243308923 shutdown_connections 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 1 -- 172.21.15.46:0/2243308923 wait complete. 2024-04-25T08:52:38.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.866+0000 7fbc350e8000 20 client.9171 trim_cache size 0 max 16384 2024-04-25T08:52:38.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc350e8000 1 Processor -- start 2024-04-25T08:52:38.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc350e8000 1 -- start start 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 0x556445a2e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x556445a2e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x556445a2eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc350e8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c202e10 con 0x5564459d19d0 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc350e8000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c14a670 con 0x5564459880a0 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc350e8000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c14a070 con 0x5564459d22f0 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc22000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x556445a2e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc22000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x556445a2e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47898/0 (socket says 172.21.15.46:47898) 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc22000640 1 -- 172.21.15.46:0/504120594 learned_addr learned my addr 172.21.15.46:0/504120594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:38.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 0x556445a2e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc2b400640 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x556445a2eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc22000640 1 -- 172.21.15.46:0/504120594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 msgr2=0x556445a2eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc22000640 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x556445a2eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc22000640 1 -- 172.21.15.46:0/504120594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 msgr2=0x556445a2e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc22000640 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 0x556445a2e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc22000640 1 -- 172.21.15.46:0/504120594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55644581bde0 con 0x5564459d19d0 2024-04-25T08:52:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 0x556445a2e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:38.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.878+0000 7fbc22000640 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x556445a2e630 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fbc14029be0 tx=0x7fbc14018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e1ae52cd745c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc1aa00640 1 -- 172.21.15.46:0/504120594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14029e50 con 0x5564459d19d0 2024-04-25T08:52:38.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc1aa00640 1 -- 172.21.15.46:0/504120594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1402f070 con 0x5564459d19d0 2024-04-25T08:52:38.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc1aa00640 1 -- 172.21.15.46:0/504120594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14024b00 con 0x5564459d19d0 2024-04-25T08:52:38.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- 172.21.15.46:0/504120594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 msgr2=0x556445a2e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x556445a2e630 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fbc14029be0 tx=0x7fbc14018000 comp rx=0 tx=0).stop 2024-04-25T08:52:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- 172.21.15.46:0/504120594 shutdown_connections 2024-04-25T08:52:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x556445a2eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x556445a2e630 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 --2- 172.21.15.46:0/504120594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 0x556445a2e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- 172.21.15.46:0/504120594 >> 172.21.15.46:0/504120594 conn(0x556445984020 msgr2=0x5564459defc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- 172.21.15.46:0/504120594 shutdown_connections 2024-04-25T08:52:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- 172.21.15.46:0/504120594 wait complete. 2024-04-25T08:52:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- start start 2024-04-25T08:52:38.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 Processor -- start 2024-04-25T08:52:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 0x5564459e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x5564459e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x5564459e4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564457d4300 con 0x5564459d19d0 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55644576fe00 con 0x5564459880a0 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564457d9620 con 0x5564459d22f0 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 10 client.0 _ll_register_callbacks cb 0x556445953040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc22000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x5564459e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc22000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x5564459e48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47904/0 (socket says 172.21.15.46:47904) 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc22000640 1 -- 172.21.15.46:0/3218541866 learned_addr learned my addr 172.21.15.46:0/3218541866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc2b400640 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x5564459e4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 0x5564459e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc22000640 1 -- 172.21.15.46:0/3218541866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 msgr2=0x5564459e4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc22000640 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x5564459e4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc22000640 1 -- 172.21.15.46:0/3218541866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 msgr2=0x5564459e4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc22000640 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 0x5564459e4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc22000640 1 -- 172.21.15.46:0/3218541866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564457768e0 con 0x5564459d19d0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc22000640 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x5564459e48c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fbc14024830 tx=0x7fbc14004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dc6a5d14280fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc0fe00640 1 -- 172.21.15.46:0/3218541866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14039070 con 0x5564459d19d0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc0fe00640 1 -- 172.21.15.46:0/3218541866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc14024cf0 con 0x5564459d19d0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.882+0000 7fbc350e8000 1 -- 172.21.15.46:0/3218541866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc1402f070 con 0x5564459d19d0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc0fe00640 1 -- 172.21.15.46:0/3218541866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc14018050 con 0x5564459d19d0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc350e8000 10 client.9177 fetch_fsmap learned FSMap version 99 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc350e8000 1 -- 172.21.15.46:0/3218541866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55644582e8f0 con 0x5564459d19d0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc0fe00640 1 -- 172.21.15.46:0/3218541866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1402f070 con 0x5564459d19d0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc0fe00640 1 -- 172.21.15.46:0/3218541866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 99) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc14020040 con 0x5564459d19d0 2024-04-25T08:52:38.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc350e8000 10 client.9177 fetch_fsmap finished waiting for FSMap version 99 2024-04-25T08:52:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc350e8000 10 client.9177 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T08:52:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc350e8000 20 client.9177 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc350e8000 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x5564459e5790 0x556445a39f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc350e8000 4 client.9177 mds_command: new command op to 8973 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T08:52:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc350e8000 1 -- 172.21.15.46:0/3218541866 --> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5564457e2530 con 0x5564459e5790 2024-04-25T08:52:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x5564459e5790 0x556445a39f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc2aa00640 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x5564459e5790 0x556445a39f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a955b6c0cf4e1007 server_cookie=f37d40a9fa4f7b09 in_seq=0 out_seq=0 2024-04-25T08:52:38.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.886+0000 7fbc0fe00640 10 client.9177 ms_handle_connect on v2:172.21.15.46:6833/51204523 2024-04-25T08:52:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc0fe00640 1 -- 172.21.15.46:0/3218541866 <== mds.0 v2:172.21.15.46:6833/51204523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5564457e2530 con 0x5564459e5790 2024-04-25T08:52:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc0fe00640 10 client.9177 handle_command_reply: tid=0 2024-04-25T08:52:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc350e8000 1 client.9177 shutdown 2024-04-25T08:52:38.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc350e8000 1 -- 172.21.15.46:0/3218541866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 msgr2=0x5564459e48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc350e8000 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x5564459e48c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fbc14024830 tx=0x7fbc14004740 comp rx=0 tx=0).stop 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc350e8000 1 -- 172.21.15.46:0/3218541866 shutdown_connections 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc350e8000 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.46:6833/51204523,v1:172.21.15.46:6835/51204523] conn(0x5564459e5790 0x556445a39f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc350e8000 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5564459d22f0 0x5564459e4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc350e8000 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564459d19d0 0x5564459e48c0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc350e8000 1 --2- 172.21.15.46:0/3218541866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564459880a0 0x5564459e4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc0fe00640 0 client.9177 ms_handle_reset on v2:172.21.15.46:6833/51204523 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.894+0000 7fbc350e8000 1 -- 172.21.15.46:0/3218541866 >> 172.21.15.46:0/3218541866 conn(0x556445984020 msgr2=0x5564459d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc350e8000 1 -- 172.21.15.46:0/3218541866 shutdown_connections 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc350e8000 1 -- 172.21.15.46:0/3218541866 wait complete. 2024-04-25T08:52:38.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc350e8000 20 client.9177 trim_cache size 0 max 16384 2024-04-25T08:52:38.889 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:52:38.889 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "", 2024-04-25T08:52:38.889 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-25T08:52:38.889 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 -- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbc08066f60 msgr2=0x7fbc08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbc08066f60 0x7fbc08069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009580 tx=0x7fbc14009190 comp rx=0 tx=0).stop 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 -- 172.21.15.46:0/1746719821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c06b2a0 msgr2=0x7fbc2c143740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c06b2a0 0x7fbc2c143740 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0010f0 tx=0x7fbc1c00c890 comp rx=0 tx=0).stop 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 -- 172.21.15.46:0/1746719821 shutdown_connections 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbc08066f60 0x7fbc08069420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbc2c15d900 0x7fbc2c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc2c0afdf0 0x7fbc2c151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 --2- 172.21.15.46:0/1746719821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c06b2a0 0x7fbc2c143740 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 -- 172.21.15.46:0/1746719821 >> 172.21.15.46:0/1746719821 conn(0x7fbc2c06e3e0 msgr2=0x7fbc2c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 -- 172.21.15.46:0/1746719821 shutdown_connections 2024-04-25T08:52:38.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:38.898+0000 7fbc31c00640 1 -- 172.21.15.46:0/1746719821 wait complete. 2024-04-25T08:52:38.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:52:39.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.058+0000 7f3cdb600640 1 Processor -- start 2024-04-25T08:52:39.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.058+0000 7f3cdb600640 1 -- start start 2024-04-25T08:52:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.058+0000 7f3cdb600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 0x7f3cd406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.058+0000 7f3cdb600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd406d5e0 con 0x7f3cd40aed10 2024-04-25T08:52:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.058+0000 7f3cdb600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40ce720 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.058+0000 7f3cdb600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd415ab30 con 0x7f3cd415a750 2024-04-25T08:52:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.058+0000 7f3ccbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3cd40aed10 0x7f3cd40b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48462/0 (socket says 172.21.15.46:48462) 2024-04-25T08:52:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.058+0000 7f3ccbe00640 1 -- 172.21.15.46:0/652830336 learned_addr learned my addr 172.21.15.46:0/652830336 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.058+0000 7f3cd8c00640 1 --2- 172.21.15.46:0/652830336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 0x7f3cd406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd3e00640 1 -- 172.21.15.46:0/652830336 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168446536 0 0) 0x7f3cd406d5e0 con 0x7f3cd40aed10 2024-04-25T08:52:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd3e00640 1 -- 172.21.15.46:0/652830336 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb8005740 con 0x7f3cd40aed10 2024-04-25T08:52:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd3e00640 1 -- 172.21.15.46:0/652830336 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150809201 0 0) 0x7f3cd40ce720 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd3e00640 1 -- 172.21.15.46:0/652830336 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd406d5e0 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd8c00640 1 -- 172.21.15.46:0/652830336 >> v1:172.21.15.82:6789/0 conn(0x7f3cd40cc9e0 legacy=0x7f3cd40b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd8c00640 1 -- 172.21.15.46:0/652830336 >> v1:172.21.15.46:6789/0 conn(0x7f3cd40aed10 legacy=0x7f3cd40b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd8c00640 1 -- 172.21.15.46:0/652830336 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd40cea00 con 0x7f3cd415a750 2024-04-25T08:52:39.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd8c00640 1 --2- 172.21.15.46:0/652830336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 0x7f3cd406ec10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40097f0 tx=0x7f3cc401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b7acc7fc826016f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:39.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd3e00640 1 -- 172.21.15.46:0/652830336 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40b0070 con 0x7f3cd415a750 2024-04-25T08:52:39.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd3e00640 1 -- 172.21.15.46:0/652830336 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc40ab600 con 0x7f3cd415a750 2024-04-25T08:52:39.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.062+0000 7f3cd3e00640 1 -- 172.21.15.46:0/652830336 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc40ab9d0 con 0x7f3cd415a750 2024-04-25T08:52:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 -- 172.21.15.46:0/652830336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 msgr2=0x7f3cd406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 --2- 172.21.15.46:0/652830336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 0x7f3cd406ec10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40097f0 tx=0x7f3cc401c4e0 comp rx=0 tx=0).stop 2024-04-25T08:52:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 -- 172.21.15.46:0/652830336 shutdown_connections 2024-04-25T08:52:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 --2- 172.21.15.46:0/652830336 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 0x7f3cd406ec10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 -- 172.21.15.46:0/652830336 >> 172.21.15.46:0/652830336 conn(0x7f3cd40b12a0 msgr2=0x7f3cd40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 -- 172.21.15.46:0/652830336 shutdown_connections 2024-04-25T08:52:39.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 -- 172.21.15.46:0/652830336 wait complete. 2024-04-25T08:52:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 Processor -- start 2024-04-25T08:52:39.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 -- start start 2024-04-25T08:52:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd40aed10 0x7f3cd41f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd40cc9e0 0x7f3cd415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 0x7f3cd415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd406e310 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd406f560 con 0x7f3cd40aed10 2024-04-25T08:52:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cdb600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd406e490 con 0x7f3cd415a750 2024-04-25T08:52:39.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3ccbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd40cc9e0 0x7f3cd415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3ccbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd40cc9e0 0x7f3cd415cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47920/0 (socket says 172.21.15.46:47920) 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3ccbe00640 1 -- 172.21.15.46:0/2325154738 learned_addr learned my addr 172.21.15.46:0/2325154738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cd9600640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 0x7f3cd415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3cd8c00640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd40aed10 0x7f3cd41f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3ccbe00640 1 -- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 msgr2=0x7f3cd415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3ccbe00640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 0x7f3cd415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3ccbe00640 1 -- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd40aed10 msgr2=0x7f3cd41f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3ccbe00640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd40aed10 0x7f3cd41f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.066+0000 7f3ccbe00640 1 -- 172.21.15.46:0/2325154738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd415d6d0 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3ccbe00640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd40cc9e0 0x7f3cd415cb10 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c500 tx=0x7f3cbc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cd1600640 1 -- 172.21.15.46:0/2325154738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc017070 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cd1600640 1 -- 172.21.15.46:0/2325154738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc005cf0 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd415d9b0 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cd1600640 1 -- 172.21.15.46:0/2325154738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc007330 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd4153c10 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd4069080 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cd1600640 1 -- 172.21.15.46:0/2325154738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cbc01a070 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cd1600640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3cb0066fe0 0x7f3cb00694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:39.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cd1600640 1 -- 172.21.15.46:0/2325154738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cbc084650 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.070+0000 7f3cd8c00640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3cb0066fe0 0x7f3cb00694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:39.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.074+0000 7f3cd8c00640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3cb0066fe0 0x7f3cb00694a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40ad000 tx=0x7f3cc40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:39.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.074+0000 7f3cd1600640 1 -- 172.21.15.46:0/2325154738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3cbc055540 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.274+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3cd40b00a0 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cd1600640 1 -- 172.21.15.46:0/2325154738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3cbc054e40 con 0x7f3cd40cc9e0 2024-04-25T08:52:39.268 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:52:39.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3cb0066fe0 msgr2=0x7f3cb00694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:39.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3cb0066fe0 0x7f3cb00694a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40ad000 tx=0x7f3cc40a6000 comp rx=0 tx=0).stop 2024-04-25T08:52:39.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd40cc9e0 msgr2=0x7f3cd415cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:39.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd40cc9e0 0x7f3cd415cb10 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00c500 tx=0x7f3cbc00c9c0 comp rx=0 tx=0).stop 2024-04-25T08:52:39.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 shutdown_connections 2024-04-25T08:52:39.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3cb0066fe0 0x7f3cb00694a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:39.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3cd415a750 0x7f3cd415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:39.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cd40cc9e0 0x7f3cd415cb10 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:39.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 --2- 172.21.15.46:0/2325154738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd40aed10 0x7f3cd41f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:39.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 >> 172.21.15.46:0/2325154738 conn(0x7f3cd40b12a0 msgr2=0x7f3cd40b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:39.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 shutdown_connections 2024-04-25T08:52:39.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:39.278+0000 7f3cdb600640 1 -- 172.21.15.46:0/2325154738 wait complete. 2024-04-25T08:52:39.289 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:52:39.289 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:52:39.289 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:52:39.289 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:52:39.289 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:39.290 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:52:39.290 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:52:39.290 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:52:39.290 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:52:39.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.290 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:52:39.299 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:52:39.299 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:52:39.299 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:52:39.299 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41843sec preferred_lft 41843sec 2024-04-25T08:52:39.300 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:39.301 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:52:39.302 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:52:39.302 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:52:39.302 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:39.302 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:52:39.302 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:52:39.302 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:52:39.302 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:39.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.303 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:52:39.365 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:52:39.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.367 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:52:39.425 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:52:39.425 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:52:39.425 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:52:39.425 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:52:39.425 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:52:39.425 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:52:39.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.426 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:52:39.426 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:52:39.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:52:39.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:52:39.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:52:39.426 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:52:39.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.522 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:52:39.530 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:52:39.530 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:52:39.531 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:52:39.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.531 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:52:39.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:52:39.599 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:39.655 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:52:39.655 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:52:39.655 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:39.711 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:39.805 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:39.805 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:52:39.805 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:52:39.821 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:39.821 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:52:39.821 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:52:39.821 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:52:39.821 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:52:39.829 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:39.829 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:52:39.829 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:52:39.829 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:52:39.829 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:52:39.830 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:52:39.830 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:52:39.830 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:52:39.833 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:52:39.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.833 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:39.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.853 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:52:39.910 INFO:teuthology.orchestra.run.smithi046.stdout:client7703 2024-04-25T08:52:39.910 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:39.910 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:39.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.963 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-25T08:52:40.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:40.021 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-25T08:52:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:8+0 records in 2024-04-25T08:52:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:8+0 records out 2024-04-25T08:52:40.119 INFO:teuthology.orchestra.run.smithi046.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0467272 s, 180 MB/s 2024-04-25T08:52:40.120 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:40.120 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:40.120 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:40.120 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:40.120 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T08:52:40.120 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:40.186 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627781 2024-04-25T08:52:40.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:40.195 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:40.195 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:40.195 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:40.195 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:40.195 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T08:52:40.195 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:40.309 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627782 2024-04-25T08:52:40.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:40.318 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:40.318 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:40.368 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:40.368 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:40.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.610+0000 7f14f2000640 1 Processor -- start 2024-04-25T08:52:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.610+0000 7f14f2000640 1 -- start start 2024-04-25T08:52:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14f2000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f14ec06db40 0x7f14ec06ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14f2000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec15e180 con 0x7f14ec0b3550 2024-04-25T08:52:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14f2000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec15e460 con 0x7f14ec0af3f0 2024-04-25T08:52:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14f2000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0b3ca0 con 0x7f14ec06db40 2024-04-25T08:52:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14ebe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f14ec0b3550 0x7f14ec06b150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48468/0 (socket says 172.21.15.46:48468) 2024-04-25T08:52:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14ebe00640 1 -- 172.21.15.46:0/2400625101 learned_addr learned my addr 172.21.15.46:0/2400625101 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14e3e00640 1 --2- 172.21.15.46:0/2400625101 >> v2:172.21.15.82:3300/0 conn(0x7f14ec06db40 0x7f14ec06ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14eaa00640 1 -- 172.21.15.46:0/2400625101 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304093624 0 0) 0x7f14ec15e180 con 0x7f14ec0b3550 2024-04-25T08:52:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14eaa00640 1 -- 172.21.15.46:0/2400625101 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14d0003530 con 0x7f14ec0b3550 2024-04-25T08:52:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14eaa00640 1 -- 172.21.15.46:0/2400625101 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002643076 0 0) 0x7f14ec15e460 con 0x7f14ec0af3f0 2024-04-25T08:52:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14eaa00640 1 -- 172.21.15.46:0/2400625101 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14ec15e180 con 0x7f14ec0af3f0 2024-04-25T08:52:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14e3e00640 1 -- 172.21.15.46:0/2400625101 >> v1:172.21.15.82:6789/0 conn(0x7f14ec0af3f0 legacy=0x7f14ec0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14e3e00640 1 -- 172.21.15.46:0/2400625101 >> v1:172.21.15.46:6789/0 conn(0x7f14ec0b3550 legacy=0x7f14ec06b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.614+0000 7f14e3e00640 1 -- 172.21.15.46:0/2400625101 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec15e740 con 0x7f14ec06db40 2024-04-25T08:52:40.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14e3e00640 1 --2- 172.21.15.46:0/2400625101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14ec06db40 0x7f14ec06ac10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f14d40097c0 tx=0x7f14d401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147eecd35fe10650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:40.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14eaa00640 1 -- 172.21.15.46:0/2400625101 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40b0070 con 0x7f14ec06db40 2024-04-25T08:52:40.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14eaa00640 1 -- 172.21.15.46:0/2400625101 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d40ab600 con 0x7f14ec06db40 2024-04-25T08:52:40.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14eaa00640 1 -- 172.21.15.46:0/2400625101 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d40ab9d0 con 0x7f14ec06db40 2024-04-25T08:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14f2000640 1 -- 172.21.15.46:0/2400625101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14ec06db40 msgr2=0x7f14ec06ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14f2000640 1 --2- 172.21.15.46:0/2400625101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14ec06db40 0x7f14ec06ac10 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f14d40097c0 tx=0x7f14d401c410 comp rx=0 tx=0).stop 2024-04-25T08:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14f2000640 1 -- 172.21.15.46:0/2400625101 shutdown_connections 2024-04-25T08:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14f2000640 1 --2- 172.21.15.46:0/2400625101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14ec06db40 0x7f14ec06ac10 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14f2000640 1 -- 172.21.15.46:0/2400625101 >> 172.21.15.46:0/2400625101 conn(0x7f14ec0cfd90 msgr2=0x7f14ec0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14f2000640 1 -- 172.21.15.46:0/2400625101 shutdown_connections 2024-04-25T08:52:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14f2000640 1 -- 172.21.15.46:0/2400625101 wait complete. 2024-04-25T08:52:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14f2000640 1 Processor -- start 2024-04-25T08:52:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.618+0000 7f14f2000640 1 -- start start 2024-04-25T08:52:40.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14f2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec06db40 0x7f14ec14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14f2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14ec0af3f0 0x7f14ec14f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14f2000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14ec0b3550 0x7f14ec0b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14f2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec06bdf0 con 0x7f14ec06db40 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14f2000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec06bc70 con 0x7f14ec0af3f0 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14f2000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec16bf10 con 0x7f14ec0b3550 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec06db40 0x7f14ec14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec06db40 0x7f14ec14d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47922/0 (socket says 172.21.15.46:47922) 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14eb400640 1 -- 172.21.15.46:0/280399420 learned_addr learned my addr 172.21.15.46:0/280399420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14e3e00640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14ec0af3f0 0x7f14ec14f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14ebe00640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14ec0b3550 0x7f14ec0b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14eb400640 1 -- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14ec0b3550 msgr2=0x7f14ec0b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14eb400640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14ec0b3550 0x7f14ec0b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14eb400640 1 -- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14ec0af3f0 msgr2=0x7f14ec14f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14eb400640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14ec0af3f0 0x7f14ec14f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14eb400640 1 -- 172.21.15.46:0/280399420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec14f840 con 0x7f14ec06db40 2024-04-25T08:52:40.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14eb400640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec06db40 0x7f14ec14d4c0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c500 tx=0x7f14dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:40.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14e3400640 1 -- 172.21.15.46:0/280399420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0052d0 con 0x7f14ec06db40 2024-04-25T08:52:40.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14e3400640 1 -- 172.21.15.46:0/280399420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc005490 con 0x7f14ec06db40 2024-04-25T08:52:40.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec153af0 con 0x7f14ec06db40 2024-04-25T08:52:40.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14e3400640 1 -- 172.21.15.46:0/280399420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc012850 con 0x7f14ec06db40 2024-04-25T08:52:40.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec153d90 con 0x7f14ec06db40 2024-04-25T08:52:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec069080 con 0x7f14ec06db40 2024-04-25T08:52:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.622+0000 7f14e3400640 1 -- 172.21.15.46:0/280399420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14dc0129f0 con 0x7f14ec06db40 2024-04-25T08:52:40.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.626+0000 7f14e3400640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f14c8066f60 0x7f14c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.626+0000 7f14e3400640 1 -- 172.21.15.46:0/280399420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14dc084e90 con 0x7f14ec06db40 2024-04-25T08:52:40.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.626+0000 7f14e3e00640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f14c8066f60 0x7f14c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.626+0000 7f14e3e00640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f14c8066f60 0x7f14c8069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f14d40ad000 tx=0x7f14d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:40.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.630+0000 7f14e3400640 1 -- 172.21.15.46:0/280399420 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14dc055c80 con 0x7f14ec06db40 2024-04-25T08:52:40.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.790+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14ec1579c0 con 0x7f14ec06db40 2024-04-25T08:52:40.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14e3400640 1 -- 172.21.15.46:0/280399420 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16904 (secure 0 0 0) 0x7f14dc086090 con 0x7f14ec06db40 2024-04-25T08:52:40.784 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:40.784 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:29.760504+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f14c8066f60 msgr2=0x7f14c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f14c8066f60 0x7f14c8069420 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f14d40ad000 tx=0x7f14d4008040 comp rx=0 tx=0).stop 2024-04-25T08:52:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec06db40 msgr2=0x7f14ec14d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec06db40 0x7f14ec14d4c0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c500 tx=0x7f14dc00c9c0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 shutdown_connections 2024-04-25T08:52:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f14c8066f60 0x7f14c8069420 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f14ec0b3550 0x7f14ec0b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14ec0af3f0 0x7f14ec14f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 --2- 172.21.15.46:0/280399420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec06db40 0x7f14ec14d4c0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 >> 172.21.15.46:0/280399420 conn(0x7f14ec0cfd90 msgr2=0x7f14ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 shutdown_connections 2024-04-25T08:52:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.794+0000 7f14f2000640 1 -- 172.21.15.46:0/280399420 wait complete. 2024-04-25T08:52:40.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:25.941254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:27.155001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:52:40.807 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:40.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.958+0000 7f9fc5400640 1 Processor -- start 2024-04-25T08:52:40.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.958+0000 7f9fc5400640 1 -- start start 2024-04-25T08:52:40.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fc5400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9fc015d900 0x7f9fc00af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fc5400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc006ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fc5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fc006b2a0 0x7f9fc015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fc5400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc00b0040 con 0x7f9fc006b2a0 2024-04-25T08:52:40.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fc5400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc006b6c0 con 0x7f9fc015d900 2024-04-25T08:52:40.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fc5400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc006fca0 con 0x7f9fc00afc20 2024-04-25T08:52:40.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fbf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fc006b2a0 0x7f9fc015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fbf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fc006b2a0 0x7f9fc015dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47932/0 (socket says 172.21.15.46:47932) 2024-04-25T08:52:40.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fbf200640 1 -- 172.21.15.46:0/2327146196 learned_addr learned my addr 172.21.15.46:0/2327146196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:40.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fbe800640 1 --2- 172.21.15.46:0/2327146196 >> v2:172.21.15.82:3300/0 conn(0x7f9fc015d900 0x7f9fc00af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fb7e00640 1 --2- 172.21.15.46:0/2327146196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc006ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fbf200640 1 -- 172.21.15.46:0/2327146196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 msgr2=0x7f9fc006ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:40.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fbf200640 1 --2- 172.21.15.46:0/2327146196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc006ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fbf200640 1 -- 172.21.15.46:0/2327146196 >> v2:172.21.15.82:3300/0 conn(0x7f9fc015d900 msgr2=0x7f9fc00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:40.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fbf200640 1 --2- 172.21.15.46:0/2327146196 >> v2:172.21.15.82:3300/0 conn(0x7f9fc015d900 0x7f9fc00af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fbf200640 1 -- 172.21.15.46:0/2327146196 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc006ff80 con 0x7f9fc006b2a0 2024-04-25T08:52:40.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.962+0000 7f9fb7e00640 1 --2- 172.21.15.46:0/2327146196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc006ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:40.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fbf200640 1 --2- 172.21.15.46:0/2327146196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc006b2a0 0x7f9fc015dce0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00d1680 tx=0x7f9fb801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e952b8bb4674d885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:40.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fbde00640 1 -- 172.21.15.46:0/2327146196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80a56d0 con 0x7f9fc006b2a0 2024-04-25T08:52:40.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fbde00640 1 -- 172.21.15.46:0/2327146196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb80a5870 con 0x7f9fc006b2a0 2024-04-25T08:52:40.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fbde00640 1 -- 172.21.15.46:0/2327146196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80a5c40 con 0x7f9fc006b2a0 2024-04-25T08:52:40.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 -- 172.21.15.46:0/2327146196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc006b2a0 msgr2=0x7f9fc015dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 --2- 172.21.15.46:0/2327146196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc006b2a0 0x7f9fc015dce0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00d1680 tx=0x7f9fb801c960 comp rx=0 tx=0).stop 2024-04-25T08:52:40.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 -- 172.21.15.46:0/2327146196 shutdown_connections 2024-04-25T08:52:40.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 --2- 172.21.15.46:0/2327146196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc006b2a0 0x7f9fc015dce0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 --2- 172.21.15.46:0/2327146196 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc006ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 --2- 172.21.15.46:0/2327146196 >> v2:172.21.15.82:3300/0 conn(0x7f9fc015d900 0x7f9fc00af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 -- 172.21.15.46:0/2327146196 >> 172.21.15.46:0/2327146196 conn(0x7f9fc006e3e0 msgr2=0x7f9fc006e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:40.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 -- 172.21.15.46:0/2327146196 shutdown_connections 2024-04-25T08:52:40.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 -- 172.21.15.46:0/2327146196 wait complete. 2024-04-25T08:52:40.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 Processor -- start 2024-04-25T08:52:40.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.966+0000 7f9fc5400640 1 -- start start 2024-04-25T08:52:40.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fc5400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc006b2a0 0x7f9fc0145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fc5400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fc5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc015d900 0x7f9fc014e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fc5400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc015ebb0 con 0x7f9fc015d900 2024-04-25T08:52:40.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fc5400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc015ea30 con 0x7f9fc006b2a0 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fc5400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc015ed30 con 0x7f9fc00afc20 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc015d900 0x7f9fc014e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc015d900 0x7f9fc014e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47938/0 (socket says 172.21.15.46:47938) 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbf200640 1 -- 172.21.15.46:0/1385237416 learned_addr learned my addr 172.21.15.46:0/1385237416 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fb7e00640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbf200640 1 -- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 msgr2=0x7f9fc00b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbe800640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc006b2a0 0x7f9fc0145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbf200640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc00b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbf200640 1 -- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc006b2a0 msgr2=0x7f9fc0145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbf200640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc006b2a0 0x7f9fc0145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbf200640 1 -- 172.21.15.46:0/1385237416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc014ef10 con 0x7f9fc015d900 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fb7e00640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc00b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:40.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbe800640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc006b2a0 0x7f9fc0145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:40.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fbf200640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc015d900 0x7f9fc014e3a0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9fb801d200 tx=0x7f9fb8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:40.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fb6a00640 1 -- 172.21.15.46:0/1385237416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80bc020 con 0x7f9fc015d900 2024-04-25T08:52:40.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fb6a00640 1 -- 172.21.15.46:0/1385237416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb80b8670 con 0x7f9fc015d900 2024-04-25T08:52:40.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fb6a00640 1 -- 172.21.15.46:0/1385237416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8009510 con 0x7f9fc015d900 2024-04-25T08:52:40.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc014f1f0 con 0x7f9fc015d900 2024-04-25T08:52:40.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc0145670 con 0x7f9fc015d900 2024-04-25T08:52:40.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc0069010 con 0x7f9fc015d900 2024-04-25T08:52:40.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.970+0000 7f9fb6a00640 1 -- 172.21.15.46:0/1385237416 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9fb80bf020 con 0x7f9fc015d900 2024-04-25T08:52:40.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.974+0000 7f9fb6a00640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f9c066f60 0x7f9f9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:40.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.974+0000 7f9fb6a00640 1 -- 172.21.15.46:0/1385237416 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9fb8120150 con 0x7f9fc015d900 2024-04-25T08:52:40.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.974+0000 7f9fbe800640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f9c066f60 0x7f9f9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:40.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.974+0000 7f9fbe800640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f9c066f60 0x7f9f9c069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0160fc0 tx=0x7f9fb0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:40.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:40.978+0000 7f9fb6a00640 1 -- 172.21.15.46:0/1385237416 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9fb80c6030 con 0x7f9fc015d900 2024-04-25T08:52:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9fc0072890 con 0x7f9fc015d900 2024-04-25T08:52:41.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fb6a00640 1 -- 172.21.15.46:0/1385237416 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9fb80165e0 con 0x7f9fc015d900 2024-04-25T08:52:41.130 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:52:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f9c066f60 msgr2=0x7f9f9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f9c066f60 0x7f9f9c069420 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0160fc0 tx=0x7f9fb0008040 comp rx=0 tx=0).stop 2024-04-25T08:52:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc015d900 msgr2=0x7f9fc014e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc015d900 0x7f9fc014e3a0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9fb801d200 tx=0x7f9fb8019700 comp rx=0 tx=0).stop 2024-04-25T08:52:41.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 shutdown_connections 2024-04-25T08:52:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f9c066f60 0x7f9f9c069420 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fc015d900 0x7f9fc014e3a0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9fc00afc20 0x7f9fc00b21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 --2- 172.21.15.46:0/1385237416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9fc006b2a0 0x7f9fc0145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 >> 172.21.15.46:0/1385237416 conn(0x7f9fc006e3e0 msgr2=0x7f9fc015f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:41.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 shutdown_connections 2024-04-25T08:52:41.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.138+0000 7f9fc5400640 1 -- 172.21.15.46:0/1385237416 wait complete. 2024-04-25T08:52:41.151 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:52:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.302+0000 7f11a9e00640 1 Processor -- start 2024-04-25T08:52:41.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.302+0000 7f11a9e00640 1 -- start start 2024-04-25T08:52:41.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a9e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a415d900 0x7f11a40af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11a406b470 0x7f11a40d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a9e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a40b0040 con 0x7f11a40afc20 2024-04-25T08:52:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a9e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a406fca0 con 0x7f11a406b470 2024-04-25T08:52:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a9e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a406ff80 con 0x7f11a415d900 2024-04-25T08:52:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11a406b470 0x7f11a40d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f119be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f11a40afc20 0x7f11a406ad60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48480/0 (socket says 172.21.15.46:48480) 2024-04-25T08:52:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f119be00640 1 -- 172.21.15.46:0/3656175448 learned_addr learned my addr 172.21.15.46:0/3656175448 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:41.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a2a00640 1 --2- 172.21.15.46:0/3656175448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a415d900 0x7f11a40af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a3400640 1 -- 172.21.15.46:0/3656175448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a415d900 msgr2=0x7f11a40af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a3400640 1 --2- 172.21.15.46:0/3656175448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a415d900 0x7f11a40af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a3400640 1 -- 172.21.15.46:0/3656175448 >> v1:172.21.15.46:6789/0 conn(0x7f11a40afc20 legacy=0x7f11a406ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a3400640 1 -- 172.21.15.46:0/3656175448 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4070260 con 0x7f11a406b470 2024-04-25T08:52:41.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.306+0000 7f11a2a00640 1 --2- 172.21.15.46:0/3656175448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a415d900 0x7f11a40af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:41.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a3400640 1 --2- 172.21.15.46:0/3656175448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a406b470 0x7f11a40d0210 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d90 tx=0x7f119c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9497aed49251aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:41.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a2000640 1 -- 172.21.15.46:0/3656175448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c0b2070 con 0x7f11a406b470 2024-04-25T08:52:41.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a2000640 1 -- 172.21.15.46:0/3656175448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c0a5890 con 0x7f11a406b470 2024-04-25T08:52:41.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a2000640 1 -- 172.21.15.46:0/3656175448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c0a5c60 con 0x7f11a406b470 2024-04-25T08:52:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 -- 172.21.15.46:0/3656175448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a406b470 msgr2=0x7f11a40d0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 --2- 172.21.15.46:0/3656175448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a406b470 0x7f11a40d0210 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d90 tx=0x7f119c01c8e0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 -- 172.21.15.46:0/3656175448 shutdown_connections 2024-04-25T08:52:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 --2- 172.21.15.46:0/3656175448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a406b470 0x7f11a40d0210 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 --2- 172.21.15.46:0/3656175448 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a415d900 0x7f11a40af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 -- 172.21.15.46:0/3656175448 >> 172.21.15.46:0/3656175448 conn(0x7f11a406e3e0 msgr2=0x7f11a406e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 -- 172.21.15.46:0/3656175448 shutdown_connections 2024-04-25T08:52:41.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 -- 172.21.15.46:0/3656175448 wait complete. 2024-04-25T08:52:41.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 Processor -- start 2024-04-25T08:52:41.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 -- start start 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a406b470 0x7f11a40b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a40afc20 0x7f11a414cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a415d900 0x7f11a414d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40d10e0 con 0x7f11a415d900 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40d0f60 con 0x7f11a406b470 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.310+0000 7f11a9e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40d1260 con 0x7f11a40afc20 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a415d900 0x7f11a414d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a415d900 0x7f11a414d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47968/0 (socket says 172.21.15.46:47968) 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a3400640 1 -- 172.21.15.46:0/3140061054 learned_addr learned my addr 172.21.15.46:0/3140061054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a2a00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a406b470 0x7f11a40b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f119be00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a40afc20 0x7f11a414cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a3400640 1 -- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a40afc20 msgr2=0x7f11a414cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a3400640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a40afc20 0x7f11a414cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a3400640 1 -- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a406b470 msgr2=0x7f11a40b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a3400640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a406b470 0x7f11a40b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a3400640 1 -- 172.21.15.46:0/3140061054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a40b23a0 con 0x7f11a415d900 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a3400640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a415d900 0x7f11a414d4d0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f119c0b0040 tx=0x7f119c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f119b400640 1 -- 172.21.15.46:0/3140061054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c0b2050 con 0x7f11a415d900 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f119b400640 1 -- 172.21.15.46:0/3140061054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c0be070 con 0x7f11a415d900 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a40b2680 con 0x7f11a415d900 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f119b400640 1 -- 172.21.15.46:0/3140061054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c0ada90 con 0x7f11a415d900 2024-04-25T08:52:41.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a40b2c40 con 0x7f11a415d900 2024-04-25T08:52:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a4069010 con 0x7f11a415d900 2024-04-25T08:52:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.314+0000 7f119b400640 1 -- 172.21.15.46:0/3140061054 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f119c0adc30 con 0x7f11a415d900 2024-04-25T08:52:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.318+0000 7f119b400640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1180066f60 0x7f1180069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.318+0000 7f119b400640 1 -- 172.21.15.46:0/3140061054 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f119c120340 con 0x7f11a415d900 2024-04-25T08:52:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.318+0000 7f11a2a00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1180066f60 0x7f1180069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.318+0000 7f11a2a00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1180066f60 0x7f1180069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f11940045f0 tx=0x7f1194008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:41.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.318+0000 7f119b400640 1 -- 172.21.15.46:0/3140061054 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f119c016560 con 0x7f11a415d900 2024-04-25T08:52:41.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11a41686d0 con 0x7f11a415d900 2024-04-25T08:52:41.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f119b400640 1 -- 172.21.15.46:0/3140061054 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f119c0f11c0 con 0x7f11a415d900 2024-04-25T08:52:41.473 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:52:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1180066f60 msgr2=0x7f1180069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1180066f60 0x7f1180069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f11940045f0 tx=0x7f1194008040 comp rx=0 tx=0).stop 2024-04-25T08:52:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a415d900 msgr2=0x7f11a414d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a415d900 0x7f11a414d4d0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f119c0b0040 tx=0x7f119c019680 comp rx=0 tx=0).stop 2024-04-25T08:52:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 shutdown_connections 2024-04-25T08:52:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1180066f60 0x7f1180069420 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a415d900 0x7f11a414d4d0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f11a40afc20 0x7f11a414cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 --2- 172.21.15.46:0/3140061054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11a406b470 0x7f11a40b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 >> 172.21.15.46:0/3140061054 conn(0x7f11a406e3e0 msgr2=0x7f11a415f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 shutdown_connections 2024-04-25T08:52:41.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.482+0000 7f11a9e00640 1 -- 172.21.15.46:0/3140061054 wait complete. 2024-04-25T08:52:41.494 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:52:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.646+0000 7fc0ddc00640 1 Processor -- start 2024-04-25T08:52:41.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.646+0000 7fc0ddc00640 1 -- start start 2024-04-25T08:52:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.646+0000 7fc0ddc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc0d8145310 0x7fc0d806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.646+0000 7fc0ddc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0d806dc70 0x7fc0d80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.646+0000 7fc0ddc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d80cd930 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.646+0000 7fc0ddc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d80cdbd0 con 0x7fc0d806dc70 2024-04-25T08:52:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.646+0000 7fc0ddc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d80cdeb0 con 0x7fc0d8145310 2024-04-25T08:52:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0ce000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0d806dc70 0x7fc0d80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0ce000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0d806dc70 0x7fc0d80cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47984/0 (socket says 172.21.15.46:47984) 2024-04-25T08:52:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0ce000640 1 -- 172.21.15.46:0/3556426358 learned_addr learned my addr 172.21.15.46:0/3556426358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0d7400640 1 --1- 172.21.15.46:0/3556426358 >> v1:172.21.15.46:6789/0 conn(0x7fc0d80cf1b0 0x7fc0d815d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48492/0 (socket says 172.21.15.46:48492) 2024-04-25T08:52:41.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0d6a00640 1 --2- 172.21.15.46:0/3556426358 >> v2:172.21.15.82:3300/0 conn(0x7fc0d8145310 0x7fc0d806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0ce000640 1 -- 172.21.15.46:0/3556426358 >> v2:172.21.15.82:3300/0 conn(0x7fc0d8145310 msgr2=0x7fc0d806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0ce000640 1 --2- 172.21.15.46:0/3556426358 >> v2:172.21.15.82:3300/0 conn(0x7fc0d8145310 0x7fc0d806e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0ce000640 1 -- 172.21.15.46:0/3556426358 >> v1:172.21.15.46:6789/0 conn(0x7fc0d80cf1b0 legacy=0x7fc0d815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0ce000640 1 -- 172.21.15.46:0/3556426358 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d80ce190 con 0x7fc0d806dc70 2024-04-25T08:52:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0d6000640 1 -- 172.21.15.46:0/3556426358 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318467956 0 0) 0x7fc0d80cd930 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0ce000640 1 --2- 172.21.15.46:0/3556426358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d806dc70 0x7fc0d80cec70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8072530 tx=0x7fc0c001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4798d535e78618b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:41.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0d6000640 1 -- 172.21.15.46:0/3556426358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c00aa650 con 0x7fc0d806dc70 2024-04-25T08:52:41.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0d6000640 1 -- 172.21.15.46:0/3556426358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c00af070 con 0x7fc0d806dc70 2024-04-25T08:52:41.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.650+0000 7fc0d6000640 1 -- 172.21.15.46:0/3556426358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c00aaa10 con 0x7fc0d806dc70 2024-04-25T08:52:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3556426358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d806dc70 msgr2=0x7fc0d80cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 --2- 172.21.15.46:0/3556426358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d806dc70 0x7fc0d80cec70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8072530 tx=0x7fc0c001c4f0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3556426358 shutdown_connections 2024-04-25T08:52:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 --2- 172.21.15.46:0/3556426358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d806dc70 0x7fc0d80cec70 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 --2- 172.21.15.46:0/3556426358 >> v2:172.21.15.82:3300/0 conn(0x7fc0d8145310 0x7fc0d806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3556426358 >> 172.21.15.46:0/3556426358 conn(0x7fc0d806ac10 msgr2=0x7fc0d806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3556426358 shutdown_connections 2024-04-25T08:52:41.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3556426358 wait complete. 2024-04-25T08:52:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 Processor -- start 2024-04-25T08:52:41.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 -- start start 2024-04-25T08:52:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d80cf1b0 0x7fc0d8155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8145310 0x7fc0d8138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0d81551d0 0x7fc0d8143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d815d920 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d815d7a0 con 0x7fc0d8145310 2024-04-25T08:52:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0ddc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8168510 con 0x7fc0d81551d0 2024-04-25T08:52:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d80cf1b0 0x7fc0d8155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0d6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d80cf1b0 0x7fc0d8155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47992/0 (socket says 172.21.15.46:47992) 2024-04-25T08:52:41.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0d6a00640 1 -- 172.21.15.46:0/3410822395 learned_addr learned my addr 172.21.15.46:0/3410822395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0d6a00640 1 -- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0d81551d0 msgr2=0x7fc0d8143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0d6a00640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0d81551d0 0x7fc0d8143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0d6a00640 1 -- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8145310 msgr2=0x7fc0d8138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0d6a00640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8145310 0x7fc0d8138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0d6a00640 1 -- 172.21.15.46:0/3410822395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d8156050 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.654+0000 7fc0d6a00640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d80cf1b0 0x7fc0d8155b10 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8007e70 tx=0x7fc0c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0cf400640 1 -- 172.21.15.46:0/3410822395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8014620 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0cf400640 1 -- 172.21.15.46:0/3410822395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c8019070 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d8154130 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0cf400640 1 -- 172.21.15.46:0/3410822395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8014a00 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d81546f0 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d8069010 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0cf400640 1 -- 172.21.15.46:0/3410822395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0c8014ba0 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0cf400640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc0b4066f60 0x7fc0b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0cf400640 1 -- 172.21.15.46:0/3410822395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0c8085a80 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.658+0000 7fc0ce000640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc0b4066f60 0x7fc0b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.662+0000 7fc0ce000640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc0b4066f60 0x7fc0b4069420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00ae000 tx=0x7fc0c0004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:41.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.662+0000 7fc0cf400640 1 -- 172.21.15.46:0/3410822395 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0c8087e20 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.826+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0d80acca0 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.826+0000 7fc0cf400640 1 -- 172.21.15.46:0/3410822395 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0c80568f0 con 0x7fc0d80cf1b0 2024-04-25T08:52:41.817 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:52:41.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.826+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc0b4066f60 msgr2=0x7fc0b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.826+0000 7fc0ddc00640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc0b4066f60 0x7fc0b4069420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00ae000 tx=0x7fc0c0004740 comp rx=0 tx=0).stop 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.826+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d80cf1b0 msgr2=0x7fc0d8155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.826+0000 7fc0ddc00640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d80cf1b0 0x7fc0d8155b10 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8007e70 tx=0x7fc0c800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.830+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 shutdown_connections 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.830+0000 7fc0ddc00640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc0b4066f60 0x7fc0b4069420 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.830+0000 7fc0ddc00640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc0d81551d0 0x7fc0d8143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.830+0000 7fc0ddc00640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8145310 0x7fc0d8138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.830+0000 7fc0ddc00640 1 --2- 172.21.15.46:0/3410822395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0d80cf1b0 0x7fc0d8155b10 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.830+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 >> 172.21.15.46:0/3410822395 conn(0x7fc0d806ac10 msgr2=0x7fc0d806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.830+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 shutdown_connections 2024-04-25T08:52:41.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.830+0000 7fc0ddc00640 1 -- 172.21.15.46:0/3410822395 wait complete. 2024-04-25T08:52:41.840 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:52:41.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0366a00640 1 Processor -- start 2024-04-25T08:52:41.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0366a00640 1 -- start start 2024-04-25T08:52:41.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0366a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0360160f50 0x7f036015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0366a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03600adc50 0x7f03600ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0366a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f036006ea00 0x7f036006ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0366a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03600ae5f0 con 0x7f0360160f50 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0366a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036006d680 con 0x7f036006ea00 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0366a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0360161330 con 0x7f03600adc50 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f035fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0360160f50 0x7f036015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f035fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0360160f50 0x7f036015f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47996/0 (socket says 172.21.15.46:47996) 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f035fe00640 1 -- 172.21.15.46:0/1317116602 learned_addr learned my addr 172.21.15.46:0/1317116602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0357e00640 1 --2- 172.21.15.46:0/1317116602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03600adc50 0x7f03600ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.990+0000 7f0364a00640 1 --2- 172.21.15.46:0/1317116602 >> v2:172.21.15.82:3300/0 conn(0x7f036006ea00 0x7f036006ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f035fe00640 1 -- 172.21.15.46:0/1317116602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03600adc50 msgr2=0x7f03600ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f035fe00640 1 --2- 172.21.15.46:0/1317116602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03600adc50 0x7f03600ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f035fe00640 1 -- 172.21.15.46:0/1317116602 >> v2:172.21.15.82:3300/0 conn(0x7f036006ea00 msgr2=0x7f036006ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f035fe00640 1 --2- 172.21.15.46:0/1317116602 >> v2:172.21.15.82:3300/0 conn(0x7f036006ea00 0x7f036006ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f035fe00640 1 -- 172.21.15.46:0/1317116602 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03600ae810 con 0x7f0360160f50 2024-04-25T08:52:41.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0364a00640 1 --2- 172.21.15.46:0/1317116602 >> v2:172.21.15.82:3300/0 conn(0x7f036006ea00 0x7f036006ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:41.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f035fe00640 1 --2- 172.21.15.46:0/1317116602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160f50 0x7f036015f1a0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f0350009870 tx=0x7f035001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47098c816baa3166 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:41.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f035f400640 1 -- 172.21.15.46:0/1317116602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03500b1070 con 0x7f0360160f50 2024-04-25T08:52:41.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f035f400640 1 -- 172.21.15.46:0/1317116602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03500ac630 con 0x7f0360160f50 2024-04-25T08:52:41.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f035f400640 1 -- 172.21.15.46:0/1317116602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03500a8050 con 0x7f0360160f50 2024-04-25T08:52:41.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0366a00640 1 -- 172.21.15.46:0/1317116602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160f50 msgr2=0x7f036015f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0366a00640 1 --2- 172.21.15.46:0/1317116602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160f50 0x7f036015f1a0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f0350009870 tx=0x7f035001c520 comp rx=0 tx=0).stop 2024-04-25T08:52:41.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0366a00640 1 -- 172.21.15.46:0/1317116602 shutdown_connections 2024-04-25T08:52:41.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0366a00640 1 --2- 172.21.15.46:0/1317116602 >> v2:172.21.15.82:3300/0 conn(0x7f036006ea00 0x7f036006ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0366a00640 1 --2- 172.21.15.46:0/1317116602 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f03600adc50 0x7f03600ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0366a00640 1 --2- 172.21.15.46:0/1317116602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0360160f50 0x7f036015f1a0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0366a00640 1 -- 172.21.15.46:0/1317116602 >> 172.21.15.46:0/1317116602 conn(0x7f03600b12a0 msgr2=0x7f03600b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:41.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0366a00640 1 -- 172.21.15.46:0/1317116602 shutdown_connections 2024-04-25T08:52:41.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.994+0000 7f0366a00640 1 -- 172.21.15.46:0/1317116602 wait complete. 2024-04-25T08:52:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 Processor -- start 2024-04-25T08:52:41.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 -- start start 2024-04-25T08:52:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f036006ea00 0x7f036013eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600adc50 0x7f03600cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0360160f50 0x7f03601586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036006e320 con 0x7f036006ea00 2024-04-25T08:52:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03600aedd0 con 0x7f03600adc50 2024-04-25T08:52:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036006e4a0 con 0x7f0360160f50 2024-04-25T08:52:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f036006ea00 0x7f036013eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f036006ea00 0x7f036013eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48006/0 (socket says 172.21.15.46:48006) 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035fe00640 1 -- 172.21.15.46:0/2296886725 learned_addr learned my addr 172.21.15.46:0/2296886725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0357e00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600adc50 0x7f03600cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0364a00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0360160f50 0x7f03601586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035fe00640 1 -- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0360160f50 msgr2=0x7f03601586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035fe00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0360160f50 0x7f03601586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035fe00640 1 -- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600adc50 msgr2=0x7f03600cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035fe00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600adc50 0x7f03600cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035fe00640 1 -- 172.21.15.46:0/2296886725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360158be0 con 0x7f036006ea00 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035fe00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f036006ea00 0x7f036013eff0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f03500af040 tx=0x7f03500192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035cc00640 1 -- 172.21.15.46:0/2296886725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03500bc070 con 0x7f036006ea00 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035cc00640 1 -- 172.21.15.46:0/2296886725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03500b7430 con 0x7f036006ea00 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036013c4f0 con 0x7f036006ea00 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f035cc00640 1 -- 172.21.15.46:0/2296886725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03500b1040 con 0x7f036006ea00 2024-04-25T08:52:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:41.998+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036013cab0 con 0x7f036006ea00 2024-04-25T08:52:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.002+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0360069080 con 0x7f036006ea00 2024-04-25T08:52:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.002+0000 7f035cc00640 1 -- 172.21.15.46:0/2296886725 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03500bf070 con 0x7f036006ea00 2024-04-25T08:52:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.002+0000 7f035cc00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f033c066f60 0x7f033c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.002+0000 7f035cc00640 1 -- 172.21.15.46:0/2296886725 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f035011e950 con 0x7f036006ea00 2024-04-25T08:52:41.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.002+0000 7f0357e00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f033c066f60 0x7f033c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.006+0000 7f0357e00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f033c066f60 0x7f033c069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f036015d8f0 tx=0x7f0348008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:41.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.006+0000 7f035cc00640 1 -- 172.21.15.46:0/2296886725 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0350016110 con 0x7f036006ea00 2024-04-25T08:52:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.178+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f03600cb710 con 0x7f036006ea00 2024-04-25T08:52:42.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.178+0000 7f035cc00640 1 -- 172.21.15.46:0/2296886725 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f03500ef7c0 con 0x7f036006ea00 2024-04-25T08:52:42.171 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-04-25T08:52:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f033c066f60 msgr2=0x7f033c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f033c066f60 0x7f033c069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f036015d8f0 tx=0x7f0348008040 comp rx=0 tx=0).stop 2024-04-25T08:52:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f036006ea00 msgr2=0x7f036013eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f036006ea00 0x7f036013eff0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f03500af040 tx=0x7f03500192c0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 shutdown_connections 2024-04-25T08:52:42.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f033c066f60 0x7f033c069420 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0360160f50 0x7f03601586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03600adc50 0x7f03600cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 --2- 172.21.15.46:0/2296886725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f036006ea00 0x7f036013eff0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 >> 172.21.15.46:0/2296886725 conn(0x7f03600b12a0 msgr2=0x7f03600b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 shutdown_connections 2024-04-25T08:52:42.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.182+0000 7f0366a00640 1 -- 172.21.15.46:0/2296886725 wait complete. 2024-04-25T08:52:42.192 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:52:42.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:42.192 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:42.321 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:52:42.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:42.321 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:52:42.338 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:42.338 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:42.389 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:52:42.389 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:52:42.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:42.389 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:52:42.439 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T08:52:42.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:52:42.440 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:52:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f6d000640 1 Processor -- start 2024-04-25T08:52:42.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f6d000640 1 -- start start 2024-04-25T08:52:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f6d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f680ccde0 0x7f9f680cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f6d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680b2240 0x7f9f680b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f6d000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f6806bdc0 con 0x7f9f68099a00 2024-04-25T08:52:42.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f6d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f6806db90 con 0x7f9f680ccde0 2024-04-25T08:52:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f6d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6806c0a0 con 0x7f9f680b2240 2024-04-25T08:52:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f5d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f680ccde0 0x7f9f680cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f5d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f680ccde0 0x7f9f680cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48022/0 (socket says 172.21.15.46:48022) 2024-04-25T08:52:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f5d400640 1 -- 172.21.15.46:0/530955254 learned_addr learned my addr 172.21.15.46:0/530955254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f66800640 1 --2- 172.21.15.46:0/530955254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680b2240 0x7f9f680b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f5d400640 1 -- 172.21.15.46:0/530955254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680b2240 msgr2=0x7f9f680b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f5d400640 1 --2- 172.21.15.46:0/530955254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680b2240 0x7f9f680b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f5d400640 1 -- 172.21.15.46:0/530955254 >> v1:172.21.15.82:6789/0 conn(0x7f9f68099a00 legacy=0x7f9f68099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.638+0000 7f9f5d400640 1 -- 172.21.15.46:0/530955254 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6806c380 con 0x7f9f680ccde0 2024-04-25T08:52:42.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f5d400640 1 --2- 172.21.15.46:0/530955254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f680ccde0 0x7f9f680cd290 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009e80 tx=0x7f9f5001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0be9adde08f91d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f65400640 1 -- 172.21.15.46:0/530955254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500a57a0 con 0x7f9f680ccde0 2024-04-25T08:52:42.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f65400640 1 -- 172.21.15.46:0/530955254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f500a5940 con 0x7f9f680ccde0 2024-04-25T08:52:42.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f65400640 1 -- 172.21.15.46:0/530955254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500a5d10 con 0x7f9f680ccde0 2024-04-25T08:52:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f6d000640 1 -- 172.21.15.46:0/530955254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f680ccde0 msgr2=0x7f9f680cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f6d000640 1 --2- 172.21.15.46:0/530955254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f680ccde0 0x7f9f680cd290 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009e80 tx=0x7f9f5001c950 comp rx=0 tx=0).stop 2024-04-25T08:52:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f6d000640 1 -- 172.21.15.46:0/530955254 shutdown_connections 2024-04-25T08:52:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f6d000640 1 --2- 172.21.15.46:0/530955254 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680b2240 0x7f9f680b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f6d000640 1 --2- 172.21.15.46:0/530955254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f680ccde0 0x7f9f680cd290 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f6d000640 1 -- 172.21.15.46:0/530955254 >> 172.21.15.46:0/530955254 conn(0x7f9f680b3120 msgr2=0x7f9f680b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f6d000640 1 -- 172.21.15.46:0/530955254 shutdown_connections 2024-04-25T08:52:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.642+0000 7f9f6d000640 1 -- 172.21.15.46:0/530955254 wait complete. 2024-04-25T08:52:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 Processor -- start 2024-04-25T08:52:42.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 -- start start 2024-04-25T08:52:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68099a00 0x7f9f6815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f680b2240 0x7f9f6814f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680ccde0 0x7f9f6814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f680cd7d0 con 0x7f9f68099a00 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6815dd10 con 0x7f9f680b2240 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f681509b0 con 0x7f9f680ccde0 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68099a00 0x7f9f6815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f65e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68099a00 0x7f9f6815b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48036/0 (socket says 172.21.15.46:48036) 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f65e00640 1 -- 172.21.15.46:0/2033552172 learned_addr learned my addr 172.21.15.46:0/2033552172 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f5d400640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f680b2240 0x7f9f6814f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f66800640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680ccde0 0x7f9f6814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f65e00640 1 -- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680ccde0 msgr2=0x7f9f6814fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f65e00640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680ccde0 0x7f9f6814fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f65e00640 1 -- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f680b2240 msgr2=0x7f9f6814f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f65e00640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f680b2240 0x7f9f6814f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f65e00640 1 -- 172.21.15.46:0/2033552172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f68150c90 con 0x7f9f68099a00 2024-04-25T08:52:42.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f65e00640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68099a00 0x7f9f6815b0e0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800c370 tx=0x7f9f5800c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f5ea00640 1 -- 172.21.15.46:0/2033552172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f580052d0 con 0x7f9f68099a00 2024-04-25T08:52:42.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f5ea00640 1 -- 172.21.15.46:0/2033552172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f58005470 con 0x7f9f68099a00 2024-04-25T08:52:42.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f68150f70 con 0x7f9f68099a00 2024-04-25T08:52:42.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f5ea00640 1 -- 172.21.15.46:0/2033552172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f58007260 con 0x7f9f68099a00 2024-04-25T08:52:42.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.646+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f68151530 con 0x7f9f68099a00 2024-04-25T08:52:42.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.650+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f68069080 con 0x7f9f68099a00 2024-04-25T08:52:42.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.650+0000 7f9f5ea00640 1 -- 172.21.15.46:0/2033552172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f58007400 con 0x7f9f68099a00 2024-04-25T08:52:42.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.650+0000 7f9f5ea00640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f44066f60 0x7f9f44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.650+0000 7f9f5ea00640 1 -- 172.21.15.46:0/2033552172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9f58084570 con 0x7f9f68099a00 2024-04-25T08:52:42.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.650+0000 7f9f5d400640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f44066f60 0x7f9f44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.650+0000 7f9f5d400640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f44066f60 0x7f9f44069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9f500ae000 tx=0x7f9f500a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.654+0000 7f9f5ea00640 1 -- 172.21.15.46:0/2033552172 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f58054d00 con 0x7f9f68099a00 2024-04-25T08:52:42.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.854+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9f68073850 con 0x7f9f68099a00 2024-04-25T08:52:42.844 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:52:42.854+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:52:42.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.854+0000 7f9f5ea00640 1 -- 172.21.15.46:0/2033552172 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9f58058da0 con 0x7f9f68099a00 2024-04-25T08:52:42.846 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:52:42.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f44066f60 msgr2=0x7f9f44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f44066f60 0x7f9f44069420 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9f500ae000 tx=0x7f9f500a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:42.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68099a00 msgr2=0x7f9f6815b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68099a00 0x7f9f6815b0e0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800c370 tx=0x7f9f5800c770 comp rx=0 tx=0).stop 2024-04-25T08:52:42.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 shutdown_connections 2024-04-25T08:52:42.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9f44066f60 0x7f9f44069420 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9f680ccde0 0x7f9f6814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f680b2240 0x7f9f6814f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 --2- 172.21.15.46:0/2033552172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f68099a00 0x7f9f6815b0e0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 >> 172.21.15.46:0/2033552172 conn(0x7f9f680b3120 msgr2=0x7f9f680cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 shutdown_connections 2024-04-25T08:52:42.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:42.858+0000 7f9f6d000640 1 -- 172.21.15.46:0/2033552172 wait complete. 2024-04-25T08:52:42.862 INFO:tasks.ceph.mds.c.smithi046.stderr: -8150> 2024-04-25T08:52:33.734+0000 7f6fcd800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:52:42.874 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:52:42.880 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:52:42.914 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-04-25T08:52:42.922 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-25T08:52:42.923 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-25T08:52:42.924 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-25T08:52:42.925 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.926 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.927 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-25T08:52:42.928 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.929 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:52:42.930 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-25T08:52:42.931 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-25T08:52:42.932 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = False 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:52:42.933 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T08:52:42.934 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T08:52:42.935 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T08:52:42.936 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on skipped message = False 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.937 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.938 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:52:42.939 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:42.940 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:52:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:42.942 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T08:52:42.943 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = True 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = False 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:52:42.944 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:52:42.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:52:42.990 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:52:43.030 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-25T08:52:43.031 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-25T08:52:43.032 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.033 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-25T08:52:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-25T08:52:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-25T08:52:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:52:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-25T08:52:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:52:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.034 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-25T08:52:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-25T08:52:43.034 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-25T08:52:43.035 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-25T08:52:43.036 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:52:43.037 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-25T08:52:43.038 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:52:43.039 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-25T08:52:43.040 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-25T08:52:43.041 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = False 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = False 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:52:43.042 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.043 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.044 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:52:43.045 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on skipped message = False 2024-04-25T08:52:43.046 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:52:43.047 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.047 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T08:52:43.047 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T08:52:43.047 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.047 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T08:52:43.047 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T08:52:43.047 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:52:43.047 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.047 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T08:52:43.048 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T08:52:43.049 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T08:52:43.050 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:52:43.051 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:52:43.051 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:43.051 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:43.051 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T08:52:43.051 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.051 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T08:52:43.051 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:43.052 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:52:43.052 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T08:52:43.052 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:52:43.052 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T08:52:43.052 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T08:52:43.052 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:43.053 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T08:52:43.053 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T08:52:43.053 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:43.053 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.053 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:52:43.053 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:52:43.053 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:52:43.054 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.054 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:52:43.054 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:52:43.054 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:52:43.054 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.054 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T08:52:43.055 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:52:43.055 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.055 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:52:43.055 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T08:52:43.055 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:43.055 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T08:52:43.056 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T08:52:43.056 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T08:52:43.056 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T08:52:43.056 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:52:43.056 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:52:43.056 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T08:52:43.057 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T08:52:43.057 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:43.057 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T08:52:43.057 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:52:43.057 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:52:43.057 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T08:52:43.057 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T08:52:43.058 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T08:52:43.058 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T08:52:43.058 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T08:52:43.058 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T08:52:43.058 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T08:52:43.058 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T08:52:43.059 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T08:52:43.059 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T08:52:43.059 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:43.059 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T08:52:43.059 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = False 2024-04-25T08:52:43.059 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T08:52:43.060 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = False 2024-04-25T08:52:43.060 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:52:43.060 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:52:43.060 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:52:43.063 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-25T08:52:43.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2076c8e080 1 Processor -- start 2024-04-25T08:52:43.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2076c8e080 1 -- start start 2024-04-25T08:52:43.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2076c8e080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56264715a8a0 0x5626471524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2076c8e080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5626471e64e0 0x5626471e8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2076c8e080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626471e68c0 0x5626471e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2076c8e080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626471e7a30 con 0x5626471e68c0 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2076c8e080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626470b59f0 con 0x56264715a8a0 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2076c8e080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626471e7bb0 con 0x5626471e64e0 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2074800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626471e68c0 0x5626471e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2074800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626471e68c0 0x5626471e0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48052/0 (socket says 172.21.15.46:48052) 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2074800640 1 -- 172.21.15.46:0/77620828 learned_addr learned my addr 172.21.15.46:0/77620828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2073e00640 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56264715a8a0 0x5626471524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2073400640 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5626471e64e0 0x5626471e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2074800640 1 -- 172.21.15.46:0/77620828 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5626471e64e0 msgr2=0x5626471e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2074800640 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5626471e64e0 0x5626471e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2074800640 1 -- 172.21.15.46:0/77620828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56264715a8a0 msgr2=0x5626471524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2074800640 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56264715a8a0 0x5626471524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2074800640 1 -- 172.21.15.46:0/77620828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562647140af0 con 0x5626471e68c0 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2073e00640 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56264715a8a0 0x5626471524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2074800640 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626471e68c0 0x5626471e0b80 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f206c00a3f0 tx=0x7f206c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320609bd0cbb3e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2072000640 1 -- 172.21.15.46:0/77620828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c0af070 con 0x5626471e68c0 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2072000640 1 -- 172.21.15.46:0/77620828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206c0b9070 con 0x5626471e68c0 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.162+0000 7f2072000640 1 -- 172.21.15.46:0/77620828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c0aacf0 con 0x5626471e68c0 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- 172.21.15.46:0/77620828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626471e68c0 msgr2=0x5626471e0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626471e68c0 0x5626471e0b80 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f206c00a3f0 tx=0x7f206c0033b0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- 172.21.15.46:0/77620828 shutdown_connections 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5626471e68c0 0x5626471e0b80 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5626471e64e0 0x5626471e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 --2- 172.21.15.46:0/77620828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56264715a8a0 0x5626471524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- 172.21.15.46:0/77620828 >> 172.21.15.46:0/77620828 conn(0x56264710f480 msgr2=0x5626471e8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- 172.21.15.46:0/77620828 shutdown_connections 2024-04-25T08:52:43.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- 172.21.15.46:0/77620828 wait complete. 2024-04-25T08:52:43.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 Processor -- start 2024-04-25T08:52:43.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- start start 2024-04-25T08:52:43.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56264715a8a0 0x562647158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626471e64e0 0x5626471c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5626471e68c0 0x5626471c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626471e3ab0 con 0x56264715a8a0 2024-04-25T08:52:43.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626471e3dc0 con 0x5626471e64e0 2024-04-25T08:52:43.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626471e0640 con 0x5626471e68c0 2024-04-25T08:52:43.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56264715a8a0 0x562647158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56264715a8a0 0x562647158610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48060/0 (socket says 172.21.15.46:48060) 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073e00640 1 -- 172.21.15.46:0/512764581 learned_addr learned my addr 172.21.15.46:0/512764581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2074800640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5626471e68c0 0x5626471c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073400640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626471e64e0 0x5626471c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073e00640 1 -- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5626471e68c0 msgr2=0x5626471c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073e00640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5626471e68c0 0x5626471c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073e00640 1 -- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626471e64e0 msgr2=0x5626471c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073e00640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626471e64e0 0x5626471c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073e00640 1 -- 172.21.15.46:0/512764581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626471c9710 con 0x56264715a8a0 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2073e00640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56264715a8a0 0x562647158610 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f206400dfc0 tx=0x7f206400b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2070c00640 1 -- 172.21.15.46:0/512764581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064014020 con 0x56264715a8a0 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2070c00640 1 -- 172.21.15.46:0/512764581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2064008030 con 0x56264715a8a0 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2070c00640 1 -- 172.21.15.46:0/512764581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064010820 con 0x56264715a8a0 2024-04-25T08:52:43.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5626471c99b0 con 0x56264715a8a0 2024-04-25T08:52:43.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.166+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626471c9ef0 con 0x56264715a8a0 2024-04-25T08:52:43.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.170+0000 7f2070c00640 1 -- 172.21.15.46:0/512764581 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2064017020 con 0x56264715a8a0 2024-04-25T08:52:43.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.170+0000 7f2070c00640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2054065970 0x7f2054067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.170+0000 7f2070c00640 1 -- 172.21.15.46:0/512764581 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2064083de0 con 0x56264715a8a0 2024-04-25T08:52:43.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.170+0000 7f2076c8e080 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5626471e1a10 0x5626471e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.170+0000 7f2073400640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2054065970 0x7f2054067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.170+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x5626471e2330 con 0x5626471e1a10 2024-04-25T08:52:43.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.170+0000 7f2074800640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5626471e1a10 0x5626471e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.170+0000 7f2073400640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2054065970 0x7f2054067e30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f20600098b0 tx=0x7f206000c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.170+0000 7f2074800640 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5626471e1a10 0x5626471e1df0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2074800640 1 -- 172.21.15.46:0/512764581 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v122'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f206c0b2070 con 0x5626471e1a10 2024-04-25T08:52:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5626471e1a10 msgr2=0x5626471e1df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5626471e1a10 0x5626471e1df0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2054065970 msgr2=0x7f2054067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2054065970 0x7f2054067e30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f20600098b0 tx=0x7f206000c040 comp rx=0 tx=0).stop 2024-04-25T08:52:43.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56264715a8a0 msgr2=0x562647158610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 --2- 172.21.15.46:0/512764581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56264715a8a0 0x562647158610 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f206400dfc0 tx=0x7f206400b5f0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2074800640 1 -- 172.21.15.46:0/512764581 reap_dead start 2024-04-25T08:52:43.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 shutdown_connections 2024-04-25T08:52:43.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 >> 172.21.15.46:0/512764581 conn(0x56264710f480 msgr2=0x5626471e4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 shutdown_connections 2024-04-25T08:52:43.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.174+0000 7f2076c8e080 1 -- 172.21.15.46:0/512764581 wait complete. 2024-04-25T08:52:43.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T08:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.366+0000 7f27b3000640 1 Processor -- start 2024-04-25T08:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.366+0000 7f27b3000640 1 -- start start 2024-04-25T08:52:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27b3000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f27ac144300 0x7f27ac0b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27b3000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac0b4dc0 0x7f27ac15b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27b3000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac0cf250 con 0x7f27ac15b540 2024-04-25T08:52:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27b3000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac0ce490 con 0x7f27ac144300 2024-04-25T08:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27b3000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0ce770 con 0x7f27ac0b4dc0 2024-04-25T08:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27b1000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f27ac15b540 0x7f27ac06d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48522/0 (socket says 172.21.15.46:48522) 2024-04-25T08:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27b1000640 1 -- 172.21.15.46:0/3634795449 learned_addr learned my addr 172.21.15.46:0/3634795449 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27abe00640 1 --2- 172.21.15.46:0/3634795449 >> v2:172.21.15.82:3300/0 conn(0x7f27ac144300 0x7f27ac0b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27a3400640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac0b4dc0 0x7f27ac15b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456150956 0 0) 0x7f27ac0cf250 con 0x7f27ac15b540 2024-04-25T08:52:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2790005740 con 0x7f27ac15b540 2024-04-25T08:52:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1697543288 0 0) 0x7f2790005740 con 0x7f27ac15b540 2024-04-25T08:52:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac0b4dc0 msgr2=0x7f27ac15b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac0b4dc0 0x7f27ac15b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 >> v2:172.21.15.82:3300/0 conn(0x7f27ac144300 msgr2=0x7f27ac0b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 --2- 172.21.15.46:0/3634795449 >> v2:172.21.15.82:3300/0 conn(0x7f27ac144300 0x7f27ac0b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0cea50 con 0x7f27ac15b540 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f27a4001a00 con 0x7f27ac15b540 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 >> v1:172.21.15.46:6789/0 conn(0x7f27ac15b540 legacy=0x7f27ac06d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac0b4dc0 0x7f2790005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f2790006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2790006f60 0x7f27900073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0ce490 con 0x7f2790006150 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0ce770 con 0x7f2790006f60 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279000dbe0 con 0x7f27ac0b4dc0 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27abe00640 1 --2- 172.21.15.46:0/3634795449 >> v2:172.21.15.82:3300/0 conn(0x7f27ac144300 0x7f27ac0b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27a3400640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f2790006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27b1000640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2790006f60 0x7f27900073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27abe00640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac0b4dc0 0x7f2790005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27a3400640 1 -- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac0b4dc0 msgr2=0x7f2790005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27a3400640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac0b4dc0 0x7f2790005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27a3400640 1 -- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2790006f60 msgr2=0x7f27900073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27a3400640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2790006f60 0x7f27900073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27a3400640 1 -- 172.21.15.46:0/3634795449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790005740 con 0x7f2790006150 2024-04-25T08:52:43.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.370+0000 7f27abe00640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac0b4dc0 0x7f2790005bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27a3400640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f2790006a20 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f2794009ae0 tx=0x7f279401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760d110acf814f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27940b2070 con 0x7f2790006150 2024-04-25T08:52:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27940a5550 con 0x7f2790006150 2024-04-25T08:52:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27ab400640 1 -- 172.21.15.46:0/3634795449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27940096b0 con 0x7f2790006150 2024-04-25T08:52:43.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27b3000640 1 -- 172.21.15.46:0/3634795449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 msgr2=0x7f2790006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27b3000640 1 --2- 172.21.15.46:0/3634795449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f2790006a20 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f2794009ae0 tx=0x7f279401ca10 comp rx=0 tx=0).stop 2024-04-25T08:52:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27b1000640 1 -- 172.21.15.46:0/3634795449 reap_dead start 2024-04-25T08:52:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27b3000640 1 -- 172.21.15.46:0/3634795449 shutdown_connections 2024-04-25T08:52:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27b3000640 1 -- 172.21.15.46:0/3634795449 >> 172.21.15.46:0/3634795449 conn(0x7f27ac06ac60 msgr2=0x7f27ac06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27b3000640 1 -- 172.21.15.46:0/3634795449 shutdown_connections 2024-04-25T08:52:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.374+0000 7f27b3000640 1 -- 172.21.15.46:0/3634795449 wait complete. 2024-04-25T08:52:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 Processor -- start 2024-04-25T08:52:43.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 -- start start 2024-04-25T08:52:43.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f27ac153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac1415e0 0x7f27ac1f05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac0b38f0 0x7f27ac0b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac15b9a0 con 0x7f2790006150 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0cd5b0 con 0x7f27ac0b38f0 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac15bb20 con 0x7f27ac1415e0 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f27ac153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f27ac153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48090/0 (socket says 172.21.15.46:48090) 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27abe00640 1 -- 172.21.15.46:0/3142442669 learned_addr learned my addr 172.21.15.46:0/3142442669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27a3400640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac1415e0 0x7f27ac1f05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b1000640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac0b38f0 0x7f27ac0b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27abe00640 1 -- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac1415e0 msgr2=0x7f27ac1f05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27abe00640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac1415e0 0x7f27ac1f05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27abe00640 1 -- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac0b38f0 msgr2=0x7f27ac0b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27abe00640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac0b38f0 0x7f27ac0b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27abe00640 1 -- 172.21.15.46:0/3142442669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac1f0ae0 con 0x7f2790006150 2024-04-25T08:52:43.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27abe00640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f27ac153730 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f279c0118d0 tx=0x7f279c011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27a8c00640 1 -- 172.21.15.46:0/3142442669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c00fbd0 con 0x7f2790006150 2024-04-25T08:52:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27a8c00640 1 -- 172.21.15.46:0/3142442669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c00fd70 con 0x7f2790006150 2024-04-25T08:52:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac153dd0 con 0x7f2790006150 2024-04-25T08:52:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27a8c00640 1 -- 172.21.15.46:0/3142442669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c015680 con 0x7f2790006150 2024-04-25T08:52:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac154390 con 0x7f2790006150 2024-04-25T08:52:43.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.378+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac04ef30 con 0x7f2790006150 2024-04-25T08:52:43.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.382+0000 7f27a8c00640 1 -- 172.21.15.46:0/3142442669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f279c015820 con 0x7f2790006150 2024-04-25T08:52:43.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.382+0000 7f27a8c00640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2788066f90 0x7f2788069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.382+0000 7f27a8c00640 1 -- 172.21.15.46:0/3142442669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f279c085d10 con 0x7f2790006150 2024-04-25T08:52:43.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.382+0000 7f27a3400640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2788066f90 0x7f2788069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.382+0000 7f27a3400640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2788066f90 0x7f2788069450 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f27940af000 tx=0x7f27940046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.386+0000 7f27a8c00640 1 -- 172.21.15.46:0/3142442669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f279c08b020 con 0x7f2790006150 2024-04-25T08:52:43.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.582+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f27ac1436e0 con 0x7f2790006150 2024-04-25T08:52:43.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27a8c00640 1 -- 172.21.15.46:0/3142442669 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v101) v1 ==== 162+0+0 (secure 0 0 0) 0x7f27ac1f0ae0 con 0x7f2790006150 2024-04-25T08:52:43.853 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T08:52:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2788066f90 msgr2=0x7f2788069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2788066f90 0x7f2788069450 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f27940af000 tx=0x7f27940046d0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 msgr2=0x7f27ac153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f27ac153730 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f279c0118d0 tx=0x7f279c011d90 comp rx=0 tx=0).stop 2024-04-25T08:52:43.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 shutdown_connections 2024-04-25T08:52:43.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2788066f90 0x7f2788069450 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac0b38f0 0x7f27ac0b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f27ac1415e0 0x7f27ac1f05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 --2- 172.21.15.46:0/3142442669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2790006150 0x7f27ac153730 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 >> 172.21.15.46:0/3142442669 conn(0x7f27ac06ac60 msgr2=0x7f27ac071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 shutdown_connections 2024-04-25T08:52:43.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:43.862+0000 7f27b3000640 1 -- 172.21.15.46:0/3142442669 wait complete. 2024-04-25T08:52:43.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.026+0000 7fca37600640 1 Processor -- start 2024-04-25T08:52:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.026+0000 7fca37600640 1 -- start start 2024-04-25T08:52:44.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca37600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 0x7fca300ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca37600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca300cea30 0x7fca3015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca37600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca300b3110 con 0x7fca30143fa0 2024-04-25T08:52:44.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca37600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca300b3a00 con 0x7fca300cea30 2024-04-25T08:52:44.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca37600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca30144380 con 0x7fca3006eb40 2024-04-25T08:52:44.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca34c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fca30143fa0 0x7fca3006e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48524/0 (socket says 172.21.15.46:48524) 2024-04-25T08:52:44.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca34c00640 1 -- 172.21.15.46:0/4128352505 learned_addr learned my addr 172.21.15.46:0/4128352505 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca35600640 1 --2- 172.21.15.46:0/4128352505 >> v2:172.21.15.82:3300/0 conn(0x7fca300cea30 0x7fca3015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca2fe00640 1 --2- 172.21.15.46:0/4128352505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 0x7fca300ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca2f400640 1 -- 172.21.15.46:0/4128352505 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392547481 0 0) 0x7fca300b3110 con 0x7fca30143fa0 2024-04-25T08:52:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca2f400640 1 -- 172.21.15.46:0/4128352505 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca18003530 con 0x7fca30143fa0 2024-04-25T08:52:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca2fe00640 1 -- 172.21.15.46:0/4128352505 >> v2:172.21.15.82:3300/0 conn(0x7fca300cea30 msgr2=0x7fca3015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca2fe00640 1 --2- 172.21.15.46:0/4128352505 >> v2:172.21.15.82:3300/0 conn(0x7fca300cea30 0x7fca3015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca2fe00640 1 -- 172.21.15.46:0/4128352505 >> v1:172.21.15.46:6789/0 conn(0x7fca30143fa0 legacy=0x7fca3006e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca2fe00640 1 -- 172.21.15.46:0/4128352505 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca300b3e90 con 0x7fca3006eb40 2024-04-25T08:52:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.030+0000 7fca35600640 1 --2- 172.21.15.46:0/4128352505 >> v2:172.21.15.82:3300/0 conn(0x7fca300cea30 0x7fca3015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca2fe00640 1 --2- 172.21.15.46:0/4128352505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 0x7fca300ce4f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54c44788f18cf33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca2f400640 1 -- 172.21.15.46:0/4128352505 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0b2070 con 0x7fca3006eb40 2024-04-25T08:52:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca2f400640 1 -- 172.21.15.46:0/4128352505 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0ad640 con 0x7fca3006eb40 2024-04-25T08:52:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca2f400640 1 -- 172.21.15.46:0/4128352505 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0ada10 con 0x7fca3006eb40 2024-04-25T08:52:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 -- 172.21.15.46:0/4128352505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 msgr2=0x7fca300ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 --2- 172.21.15.46:0/4128352505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 0x7fca300ce4f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0097c0 tx=0x7fca1c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 -- 172.21.15.46:0/4128352505 shutdown_connections 2024-04-25T08:52:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 --2- 172.21.15.46:0/4128352505 >> v2:172.21.15.82:3300/0 conn(0x7fca300cea30 0x7fca3015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 --2- 172.21.15.46:0/4128352505 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 0x7fca300ce4f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 -- 172.21.15.46:0/4128352505 >> 172.21.15.46:0/4128352505 conn(0x7fca30066fd0 msgr2=0x7fca3006ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 -- 172.21.15.46:0/4128352505 shutdown_connections 2024-04-25T08:52:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 -- 172.21.15.46:0/4128352505 wait complete. 2024-04-25T08:52:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 Processor -- start 2024-04-25T08:52:44.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.034+0000 7fca37600640 1 -- start start 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca37600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 0x7fca30158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca37600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300cea30 0x7fca3013df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca37600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30143fa0 0x7fca3013a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca37600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300cefc0 con 0x7fca300cea30 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca37600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca300cee40 con 0x7fca30143fa0 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca37600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3015ba50 con 0x7fca3006eb40 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300cea30 0x7fca3013df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300cea30 0x7fca3013df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48094/0 (socket says 172.21.15.46:48094) 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2fe00640 1 -- 172.21.15.46:0/3404861895 learned_addr learned my addr 172.21.15.46:0/3404861895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca34c00640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 0x7fca30158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca35600640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30143fa0 0x7fca3013a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2fe00640 1 -- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 msgr2=0x7fca30158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2fe00640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 0x7fca30158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2fe00640 1 -- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30143fa0 msgr2=0x7fca3013a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2fe00640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30143fa0 0x7fca3013a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2fe00640 1 -- 172.21.15.46:0/3404861895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca301592c0 con 0x7fca300cea30 2024-04-25T08:52:44.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2fe00640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300cea30 0x7fca3013df50 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0b0040 tx=0x7fca1c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2cc00640 1 -- 172.21.15.46:0/3404861895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0b2040 con 0x7fca300cea30 2024-04-25T08:52:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2cc00640 1 -- 172.21.15.46:0/3404861895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0bc070 con 0x7fca300cea30 2024-04-25T08:52:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3013e9a0 con 0x7fca300cea30 2024-04-25T08:52:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2cc00640 1 -- 172.21.15.46:0/3404861895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c006e20 con 0x7fca300cea30 2024-04-25T08:52:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca301f6ff0 con 0x7fca300cea30 2024-04-25T08:52:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca301f6b00 con 0x7fca300cea30 2024-04-25T08:52:44.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.038+0000 7fca2cc00640 1 -- 172.21.15.46:0/3404861895 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca1c0b6670 con 0x7fca300cea30 2024-04-25T08:52:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.042+0000 7fca2cc00640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f90 0x7fca10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.042+0000 7fca2cc00640 1 -- 172.21.15.46:0/3404861895 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fca1c11eff0 con 0x7fca300cea30 2024-04-25T08:52:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.042+0000 7fca34c00640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f90 0x7fca10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.042+0000 7fca34c00640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f90 0x7fca10069450 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fca24007fd0 tx=0x7fca24000bd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.046+0000 7fca2cc00640 1 -- 172.21.15.46:0/3404861895 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca1c016090 con 0x7fca300cea30 2024-04-25T08:52:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca30072c70 con 0x7fca300cea30 2024-04-25T08:52:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca2cc00640 1 -- 172.21.15.46:0/3404861895 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4940 (secure 0 0 0) 0x7fca1c0efe70 con 0x7fca300cea30 2024-04-25T08:52:44.242 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:44.242 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:43.866884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6834/1279640422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1279640422},{"type":"v1","addr":"172.21.15.46:6834","nonce":1279640422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:52:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 102 2024-04-25T08:52:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f90 msgr2=0x7fca10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f90 0x7fca10069450 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fca24007fd0 tx=0x7fca24000bd0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300cea30 msgr2=0x7fca3013df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300cea30 0x7fca3013df50 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0b0040 tx=0x7fca1c019240 comp rx=0 tx=0).stop 2024-04-25T08:52:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 shutdown_connections 2024-04-25T08:52:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fca10066f90 0x7fca10069450 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca30143fa0 0x7fca3013a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca300cea30 0x7fca3013df50 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 --2- 172.21.15.46:0/3404861895 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fca3006eb40 0x7fca30158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 >> 172.21.15.46:0/3404861895 conn(0x7fca30066fd0 msgr2=0x7fca300b02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 shutdown_connections 2024-04-25T08:52:44.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:44.250+0000 7fca37600640 1 -- 172.21.15.46:0/3404861895 wait complete. 2024-04-25T08:52:44.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:52:28.147634+0000', 'modified': '2024-04-25T08:52:43.866884+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8970': {'gid': 8970, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6834/1279640422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 1279640422}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 1279640422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:52:44.664 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8970:up:replay 2024-04-25T08:52:45.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.826+0000 7f44e8e00640 1 Processor -- start 2024-04-25T08:52:45.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.826+0000 7f44e8e00640 1 -- start start 2024-04-25T08:52:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.826+0000 7f44e8e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44e4143fa0 0x7f44e406e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.826+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e40cec00 0x7f44e415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.826+0000 7f44e8e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e40b3110 con 0x7f44e406e970 2024-04-25T08:52:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.826+0000 7f44e8e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e406edd0 con 0x7f44e4143fa0 2024-04-25T08:52:45.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.826+0000 7f44e8e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40b3a00 con 0x7f44e40cec00 2024-04-25T08:52:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44e1e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44e4143fa0 0x7f44e406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44e2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e40cec00 0x7f44e415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44d9400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f44e406e970 0x7f44e40ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:55352/0 (socket says 172.21.15.46:55352) 2024-04-25T08:52:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44d9400640 1 -- 172.21.15.46:0/1914872659 learned_addr learned my addr 172.21.15.46:0/1914872659 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:45.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44e1e00640 1 -- 172.21.15.46:0/1914872659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e40cec00 msgr2=0x7f44e415ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44e1e00640 1 --2- 172.21.15.46:0/1914872659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e40cec00 0x7f44e415ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44e1e00640 1 -- 172.21.15.46:0/1914872659 >> v1:172.21.15.82:6789/0 conn(0x7f44e406e970 legacy=0x7f44e40ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44e1e00640 1 -- 172.21.15.46:0/1914872659 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40b3e90 con 0x7f44e4143fa0 2024-04-25T08:52:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44e2800640 1 --2- 172.21.15.46:0/1914872659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e40cec00 0x7f44e415ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:45.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.830+0000 7f44e1400640 1 -- 172.21.15.46:0/1914872659 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858558613 0 0) 0x7f44e40b3110 con 0x7f44e406e970 2024-04-25T08:52:45.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e1e00640 1 --2- 172.21.15.46:0/1914872659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e4143fa0 0x7f44e406e430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f44e40b3b80 tx=0x7f44d401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc87af15edd3acef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:45.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e1400640 1 -- 172.21.15.46:0/1914872659 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40b0070 con 0x7f44e4143fa0 2024-04-25T08:52:45.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e1400640 1 -- 172.21.15.46:0/1914872659 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d40ab600 con 0x7f44e4143fa0 2024-04-25T08:52:45.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e1400640 1 -- 172.21.15.46:0/1914872659 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40ab9d0 con 0x7f44e4143fa0 2024-04-25T08:52:45.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 -- 172.21.15.46:0/1914872659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e4143fa0 msgr2=0x7f44e406e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 --2- 172.21.15.46:0/1914872659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e4143fa0 0x7f44e406e430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f44e40b3b80 tx=0x7f44d401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 -- 172.21.15.46:0/1914872659 shutdown_connections 2024-04-25T08:52:45.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 --2- 172.21.15.46:0/1914872659 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e40cec00 0x7f44e415ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 --2- 172.21.15.46:0/1914872659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e4143fa0 0x7f44e406e430 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 -- 172.21.15.46:0/1914872659 >> 172.21.15.46:0/1914872659 conn(0x7f44e4066fd0 msgr2=0x7f44e406ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:45.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 -- 172.21.15.46:0/1914872659 shutdown_connections 2024-04-25T08:52:45.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 -- 172.21.15.46:0/1914872659 wait complete. 2024-04-25T08:52:45.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 Processor -- start 2024-04-25T08:52:45.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 -- start start 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e406e970 0x7f44e413b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e40cec00 0x7f44e4141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e4143fa0 0x7f44e4141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e415ba50 con 0x7f44e406e970 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40cf010 con 0x7f44e40cec00 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e8e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e415bbd0 con 0x7f44e4143fa0 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e406e970 0x7f44e413b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e406e970 0x7f44e413b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48104/0 (socket says 172.21.15.46:48104) 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.834+0000 7f44e1e00640 1 -- 172.21.15.46:0/331778611 learned_addr learned my addr 172.21.15.46:0/331778611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44d9400640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e40cec00 0x7f44e4141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e2800640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e4143fa0 0x7f44e4141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e1e00640 1 -- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e4143fa0 msgr2=0x7f44e4141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e1e00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e4143fa0 0x7f44e4141f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e1e00640 1 -- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e40cec00 msgr2=0x7f44e4141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e1e00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e40cec00 0x7f44e4141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e1e00640 1 -- 172.21.15.46:0/331778611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e413e150 con 0x7f44e406e970 2024-04-25T08:52:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e1e00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e406e970 0x7f44e413b500 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f44d40ae040 tx=0x7f44d401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44daa00640 1 -- 172.21.15.46:0/331778611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40b0050 con 0x7f44e406e970 2024-04-25T08:52:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44daa00640 1 -- 172.21.15.46:0/331778611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d40bc070 con 0x7f44e406e970 2024-04-25T08:52:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44daa00640 1 -- 172.21.15.46:0/331778611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4006e20 con 0x7f44e406e970 2024-04-25T08:52:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e413c890 con 0x7f44e406e970 2024-04-25T08:52:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e413ce50 con 0x7f44e406e970 2024-04-25T08:52:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e4072c70 con 0x7f44e406e970 2024-04-25T08:52:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.838+0000 7f44daa00640 1 -- 172.21.15.46:0/331778611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44d40b5870 con 0x7f44e406e970 2024-04-25T08:52:45.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.842+0000 7f44daa00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c0066f60 0x7f44c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.842+0000 7f44daa00640 1 -- 172.21.15.46:0/331778611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f44d411f920 con 0x7f44e406e970 2024-04-25T08:52:45.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.842+0000 7f44d9400640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c0066f60 0x7f44c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.842+0000 7f44d9400640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c0066f60 0x7f44c0069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f44e40cbcd0 tx=0x7f44cc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:45.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:45.846+0000 7f44daa00640 1 -- 172.21.15.46:0/331778611 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44d4016090 con 0x7f44e406e970 2024-04-25T08:52:46.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44e415aae0 con 0x7f44e406e970 2024-04-25T08:52:46.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44daa00640 1 -- 172.21.15.46:0/331778611 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f44d40f06b0 con 0x7f44e406e970 2024-04-25T08:52:46.033 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:46.033 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:44.875511+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":102,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.46:6834/1279640422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1279640422},{"type":"v1","addr":"172.21.15.46:6834","nonce":1279640422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:52:46.033 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 104 2024-04-25T08:52:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c0066f60 msgr2=0x7f44c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c0066f60 0x7f44c0069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f44e40cbcd0 tx=0x7f44cc009040 comp rx=0 tx=0).stop 2024-04-25T08:52:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e406e970 msgr2=0x7f44e413b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e406e970 0x7f44e413b500 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f44d40ae040 tx=0x7f44d401ace0 comp rx=0 tx=0).stop 2024-04-25T08:52:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 shutdown_connections 2024-04-25T08:52:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c0066f60 0x7f44c0069420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e4143fa0 0x7f44e4141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e40cec00 0x7f44e4141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 --2- 172.21.15.46:0/331778611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e406e970 0x7f44e413b500 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 >> 172.21.15.46:0/331778611 conn(0x7f44e4066fd0 msgr2=0x7f44e415c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 shutdown_connections 2024-04-25T08:52:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:46.042+0000 7f44e8e00640 1 -- 172.21.15.46:0/331778611 wait complete. 2024-04-25T08:52:46.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:52:28.147634+0000', 'modified': '2024-04-25T08:52:44.875511+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8970': {'gid': 8970, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.46:6834/1279640422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 1279640422}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 1279640422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:52:46.458 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8970:up:rejoin 2024-04-25T08:52:47.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f473de00640 1 Processor -- start 2024-04-25T08:52:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f473de00640 1 -- start start 2024-04-25T08:52:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f473de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f473815f060 0x7f47380ae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f473de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f473809a110 0x7f47380cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f473de00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47380cc580 con 0x7f47380b0580 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f473de00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47380cd190 con 0x7f473809a110 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f473de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47380cc860 con 0x7f473815f060 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f4737400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f473809a110 0x7f47380cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f472fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f47380b0580 0x7f4738099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48532/0 (socket says 172.21.15.46:48532) 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f472fe00640 1 -- 172.21.15.46:0/359133935 learned_addr learned my addr 172.21.15.46:0/359133935 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f4736a00640 1 --2- 172.21.15.46:0/359133935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f473815f060 0x7f47380ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f4737400640 1 -- 172.21.15.46:0/359133935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f473815f060 msgr2=0x7f47380ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f4737400640 1 --2- 172.21.15.46:0/359133935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f473815f060 0x7f47380ae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f4737400640 1 -- 172.21.15.46:0/359133935 >> v1:172.21.15.46:6789/0 conn(0x7f47380b0580 legacy=0x7f4738099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f4737400640 1 -- 172.21.15.46:0/359133935 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47380ccb40 con 0x7f473809a110 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f4736000640 1 -- 172.21.15.46:0/359133935 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053395244 0 0) 0x7f47380cc580 con 0x7f47380b0580 2024-04-25T08:52:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.622+0000 7f4736a00640 1 --2- 172.21.15.46:0/359133935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f473815f060 0x7f47380ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f4737400640 1 --2- 172.21.15.46:0/359133935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473809a110 0x7f47380cd620 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4730009d90 tx=0x7f473001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09dc87fcb38375b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f4736000640 1 -- 172.21.15.46:0/359133935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300b2070 con 0x7f473809a110 2024-04-25T08:52:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f4736000640 1 -- 172.21.15.46:0/359133935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47300a5a20 con 0x7f473809a110 2024-04-25T08:52:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f4736000640 1 -- 172.21.15.46:0/359133935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300a5df0 con 0x7f473809a110 2024-04-25T08:52:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f473de00640 1 -- 172.21.15.46:0/359133935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473809a110 msgr2=0x7f47380cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f473de00640 1 --2- 172.21.15.46:0/359133935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473809a110 0x7f47380cd620 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4730009d90 tx=0x7f473001c940 comp rx=0 tx=0).stop 2024-04-25T08:52:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f473de00640 1 -- 172.21.15.46:0/359133935 shutdown_connections 2024-04-25T08:52:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f473de00640 1 --2- 172.21.15.46:0/359133935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473809a110 0x7f47380cd620 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f473de00640 1 --2- 172.21.15.46:0/359133935 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f473815f060 0x7f47380ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f473de00640 1 -- 172.21.15.46:0/359133935 >> 172.21.15.46:0/359133935 conn(0x7f47380b32d0 msgr2=0x7f47380b36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f473de00640 1 -- 172.21.15.46:0/359133935 shutdown_connections 2024-04-25T08:52:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.626+0000 7f473de00640 1 -- 172.21.15.46:0/359133935 wait complete. 2024-04-25T08:52:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 Processor -- start 2024-04-25T08:52:47.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 -- start start 2024-04-25T08:52:47.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f473809a110 0x7f4738155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f47380b0580 0x7f47380ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473815f060 0x7f4738165d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380cc1a0 con 0x7f473815f060 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47380cdf70 con 0x7f473809a110 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47380cc320 con 0x7f47380b0580 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4737400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473815f060 0x7f4738165d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4737400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473815f060 0x7f4738165d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48122/0 (socket says 172.21.15.46:48122) 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4737400640 1 -- 172.21.15.46:0/2337258100 learned_addr learned my addr 172.21.15.46:0/2337258100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f472fe00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f47380b0580 0x7f47380ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4736a00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f473809a110 0x7f4738155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4737400640 1 -- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f47380b0580 msgr2=0x7f47380ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4737400640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f47380b0580 0x7f47380ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4737400640 1 -- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f473809a110 msgr2=0x7f4738155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4737400640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f473809a110 0x7f4738155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4737400640 1 -- 172.21.15.46:0/2337258100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4738099a60 con 0x7f473815f060 2024-04-25T08:52:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f4737400640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473815f060 0x7f4738165d80 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4730009d90 tx=0x7f47300196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f472f400640 1 -- 172.21.15.46:0/2337258100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300b2050 con 0x7f473815f060 2024-04-25T08:52:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f472f400640 1 -- 172.21.15.46:0/2337258100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47300be070 con 0x7f473815f060 2024-04-25T08:52:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f472f400640 1 -- 172.21.15.46:0/2337258100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47300adb70 con 0x7f473815f060 2024-04-25T08:52:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47380c9d30 con 0x7f473815f060 2024-04-25T08:52:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.630+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47380ca2f0 con 0x7f473815f060 2024-04-25T08:52:47.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.634+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473809f7c0 con 0x7f473815f060 2024-04-25T08:52:47.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.634+0000 7f472f400640 1 -- 172.21.15.46:0/2337258100 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47300add10 con 0x7f473815f060 2024-04-25T08:52:47.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.634+0000 7f472f400640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4714066f60 0x7f4714069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:47.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.634+0000 7f472f400640 1 -- 172.21.15.46:0/2337258100 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4730120650 con 0x7f473815f060 2024-04-25T08:52:47.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.634+0000 7f4736a00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4714066f60 0x7f4714069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:47.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.634+0000 7f4736a00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4714066f60 0x7f4714069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f472800c900 tx=0x7f4728008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:47.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.638+0000 7f472f400640 1 -- 172.21.15.46:0/2337258100 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4730016560 con 0x7f473815f060 2024-04-25T08:52:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f473804f010 con 0x7f473815f060 2024-04-25T08:52:47.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f472f400640 1 -- 172.21.15.46:0/2337258100 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f47300ab340 con 0x7f473815f060 2024-04-25T08:52:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:47.826 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:45.880336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":9,"addr":"172.21.15.46:6834/1279640422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1279640422},{"type":"v1","addr":"172.21.15.46:6834","nonce":1279640422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:52:47.826 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 105 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4714066f60 msgr2=0x7f4714069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4714066f60 0x7f4714069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f472800c900 tx=0x7f4728008040 comp rx=0 tx=0).stop 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473815f060 msgr2=0x7f4738165d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473815f060 0x7f4738165d80 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4730009d90 tx=0x7f47300196e0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 shutdown_connections 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4714066f60 0x7f4714069420 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f473815f060 0x7f4738165d80 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f47380b0580 0x7f47380ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 --2- 172.21.15.46:0/2337258100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f473809a110 0x7f4738155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.834+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 >> 172.21.15.46:0/2337258100 conn(0x7f47380b32d0 msgr2=0x7f47380b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:47.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.838+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 shutdown_connections 2024-04-25T08:52:47.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:47.838+0000 7f473de00640 1 -- 172.21.15.46:0/2337258100 wait complete. 2024-04-25T08:52:48.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:52:28.147634+0000', 'modified': '2024-04-25T08:52:45.880336+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8970': {'gid': 8970, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.46:6834/1279640422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 1279640422}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 1279640422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:52:48.250 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:52:48.250 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:52:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f192d400640 1 Processor -- start 2024-04-25T08:52:48.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f192d400640 1 -- start start 2024-04-25T08:52:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f192d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f192815b8d0 0x7f19280cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f192d400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f19280cf340 0x7f19280cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f192d400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280b2fc0 con 0x7f192806ac60 2024-04-25T08:52:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f192d400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192815f460 con 0x7f192815b8d0 2024-04-25T08:52:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f192d400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19281602c0 con 0x7f19280cf340 2024-04-25T08:52:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f1926800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f192815b8d0 0x7f19280cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f1926800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f192815b8d0 0x7f19280cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45894/0 (socket says 172.21.15.46:45894) 2024-04-25T08:52:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f1926800640 1 -- 172.21.15.46:0/177963060 learned_addr learned my addr 172.21.15.46:0/177963060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.406+0000 7f191fe00640 1 --2- 172.21.15.46:0/177963060 >> v2:172.21.15.82:3300/0 conn(0x7f19280cf340 0x7f19280cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f1926800640 1 -- 172.21.15.46:0/177963060 >> v2:172.21.15.82:3300/0 conn(0x7f19280cf340 msgr2=0x7f19280cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f1926800640 1 --2- 172.21.15.46:0/177963060 >> v2:172.21.15.82:3300/0 conn(0x7f19280cf340 0x7f19280cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f1926800640 1 -- 172.21.15.46:0/177963060 >> v1:172.21.15.82:6789/0 conn(0x7f192806ac60 legacy=0x7f19280ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f1926800640 1 -- 172.21.15.46:0/177963060 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192815f6b0 con 0x7f192815b8d0 2024-04-25T08:52:48.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f1926800640 1 --2- 172.21.15.46:0/177963060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192815b8d0 0x7f19280cf800 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f19280ce0a0 tx=0x7f191801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273e7d1dd216ee94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f1925e00640 1 -- 172.21.15.46:0/177963060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180aa650 con 0x7f192815b8d0 2024-04-25T08:52:48.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f1925e00640 1 -- 172.21.15.46:0/177963060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19180af070 con 0x7f192815b8d0 2024-04-25T08:52:48.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f1925e00640 1 -- 172.21.15.46:0/177963060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180aaa10 con 0x7f192815b8d0 2024-04-25T08:52:48.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f192d400640 1 -- 172.21.15.46:0/177963060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192815b8d0 msgr2=0x7f19280cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f192d400640 1 --2- 172.21.15.46:0/177963060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192815b8d0 0x7f19280cf800 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f19280ce0a0 tx=0x7f191801c4f0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f192d400640 1 -- 172.21.15.46:0/177963060 shutdown_connections 2024-04-25T08:52:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f192d400640 1 --2- 172.21.15.46:0/177963060 >> v2:172.21.15.82:3300/0 conn(0x7f19280cf340 0x7f19280cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f192d400640 1 --2- 172.21.15.46:0/177963060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f192815b8d0 0x7f19280cf800 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f192d400640 1 -- 172.21.15.46:0/177963060 >> 172.21.15.46:0/177963060 conn(0x7f19280b2100 msgr2=0x7f19280b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f192d400640 1 -- 172.21.15.46:0/177963060 shutdown_connections 2024-04-25T08:52:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.410+0000 7f192d400640 1 -- 172.21.15.46:0/177963060 wait complete. 2024-04-25T08:52:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 Processor -- start 2024-04-25T08:52:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 -- start start 2024-04-25T08:52:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f192806ac60 0x7f19281527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19280cf340 0x7f19280b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f192815b8d0 0x7f19280b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19281685b0 con 0x7f19280cf340 2024-04-25T08:52:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192815bcb0 con 0x7f192815b8d0 2024-04-25T08:52:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1928168730 con 0x7f192806ac60 2024-04-25T08:52:48.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19280cf340 0x7f19280b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19280cf340 0x7f19280b31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45900/0 (socket says 172.21.15.46:45900) 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191fe00640 1 -- 172.21.15.46:0/1672682293 learned_addr learned my addr 172.21.15.46:0/1672682293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f1926800640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f192806ac60 0x7f19281527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f1927200640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f192815b8d0 0x7f19280b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191fe00640 1 -- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f192806ac60 msgr2=0x7f19281527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191fe00640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f192806ac60 0x7f19281527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191fe00640 1 -- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f192815b8d0 msgr2=0x7f19280b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191fe00640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f192815b8d0 0x7f19280b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191fe00640 1 -- 172.21.15.46:0/1672682293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280c7620 con 0x7f19280cf340 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f1927200640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f192815b8d0 0x7f19280b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191fe00640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19280cf340 0x7f19280b31a0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f191000d7d0 tx=0x7f191000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191ea00640 1 -- 172.21.15.46:0/1672682293 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191000b6c0 con 0x7f19280cf340 2024-04-25T08:52:48.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191ea00640 1 -- 172.21.15.46:0/1672682293 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1910019070 con 0x7f19280cf340 2024-04-25T08:52:48.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 -- 172.21.15.46:0/1672682293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19280b4470 con 0x7f19280cf340 2024-04-25T08:52:48.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f191ea00640 1 -- 172.21.15.46:0/1672682293 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191000baa0 con 0x7f19280cf340 2024-04-25T08:52:48.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.414+0000 7f192d400640 1 -- 172.21.15.46:0/1672682293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19280b4a30 con 0x7f19280cf340 2024-04-25T08:52:48.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f191ea00640 1 -- 172.21.15.46:0/1672682293 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f191000bc40 con 0x7f19280cf340 2024-04-25T08:52:48.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f191ea00640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1904066f90 0x7f1904069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f191ea00640 1 -- 172.21.15.46:0/1672682293 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f19100856a0 con 0x7f19280cf340 2024-04-25T08:52:48.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f193093b000 1 Processor -- start 2024-04-25T08:52:48.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f1926800640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1904066f90 0x7f1904069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f193093b000 1 -- start start 2024-04-25T08:52:48.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f1926800640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1904066f90 0x7f1904069450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f19180ae000 tx=0x7f19180a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f193093b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed6dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f193093b000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed692e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f193093b000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed693720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f193093b000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bed5765f0 con 0x555bed693340 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f193093b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bed4ddda0 con 0x555bed6dcb70 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.418+0000 7f193093b000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bed466150 con 0x555bed6dd490 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed693720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed693720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45908/0 (socket says 172.21.15.46:45908) 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 -- 172.21.15.46:0/1718675655 learned_addr learned my addr 172.21.15.46:0/1718675655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f191fe00640 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed692e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1926800640 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed6dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 -- 172.21.15.46:0/1718675655 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 msgr2=0x555bed692e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed692e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 -- 172.21.15.46:0/1718675655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 msgr2=0x555bed6dcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:48.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed6dcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 -- 172.21.15.46:0/1718675655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bed47fca0 con 0x555bed693340 2024-04-25T08:52:48.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f191fe00640 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed692e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:48.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed693720 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1920001d30 tx=0x7f19200021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8b90f1df857597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1917400640 1 -- 172.21.15.46:0/1718675655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920019070 con 0x555bed693340 2024-04-25T08:52:48.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1917400640 1 -- 172.21.15.46:0/1718675655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1920003b90 con 0x555bed693340 2024-04-25T08:52:48.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1917400640 1 -- 172.21.15.46:0/1718675655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192000e920 con 0x555bed693340 2024-04-25T08:52:48.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 -- 172.21.15.46:0/1718675655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 msgr2=0x555bed693720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed693720 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1920001d30 tx=0x7f19200021f0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 -- 172.21.15.46:0/1718675655 shutdown_connections 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed693720 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed692e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 --2- 172.21.15.46:0/1718675655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed6dcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 -- 172.21.15.46:0/1718675655 >> 172.21.15.46:0/1718675655 conn(0x555bed6952a0 msgr2=0x555bed68d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 -- 172.21.15.46:0/1718675655 shutdown_connections 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 -- 172.21.15.46:0/1718675655 wait complete. 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 -- start start 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 Processor -- start 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 0x555bed6e41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed6e4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dd490 0x555bed6e9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bed498990 con 0x555bed6dd490 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bed4d9e40 con 0x555bed6dcb70 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bed469900 con 0x555bed693340 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f193093b000 10 client.0 _ll_register_callbacks cb 0x555bed5911c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dd490 0x555bed6e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dd490 0x555bed6e9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45910/0 (socket says 172.21.15.46:45910) 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.422+0000 7f1927200640 1 -- 172.21.15.46:0/1001930536 learned_addr learned my addr 172.21.15.46:0/1001930536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1926800640 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 0x555bed6e41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f191fe00640 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed6e4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1927200640 1 -- 172.21.15.46:0/1001930536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 msgr2=0x555bed6e41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1927200640 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 0x555bed6e41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1927200640 1 -- 172.21.15.46:0/1001930536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 msgr2=0x555bed6e4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1927200640 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed6e4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1927200640 1 -- 172.21.15.46:0/1001930536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bed496630 con 0x555bed6dd490 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1926800640 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 0x555bed6e41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1927200640 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dd490 0x555bed6e9080 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1920007f00 tx=0x7f1920001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7268e73340a3ecfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 1 -- 172.21.15.46:0/1001930536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920023020 con 0x555bed6dd490 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 1 -- 172.21.15.46:0/1001930536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1920006be0 con 0x555bed6dd490 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 -- 172.21.15.46:0/1001930536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1920003b90 con 0x555bed6dd490 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 1 -- 172.21.15.46:0/1001930536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f192002d050 con 0x555bed6dd490 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 10 client.9276 fetch_fsmap learned FSMap version 105 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 -- 172.21.15.46:0/1001930536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x555bed4663a0 con 0x555bed6dd490 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 1 -- 172.21.15.46:0/1001930536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1920003b90 con 0x555bed6dd490 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 1 -- 172.21.15.46:0/1001930536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1920030020 con 0x555bed6dd490 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 10 client.9276 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 10 client.9276 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 20 client.9276 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:48.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x555bed6dc790 0x555bed6edda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 4 client.9276 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:52:48.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 -- 172.21.15.46:0/1001930536 --> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bed479a50 con 0x555bed6dc790 2024-04-25T08:52:48.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1926800640 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x555bed6dc790 0x555bed6edda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f1926800640 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x555bed6dc790 0x555bed6edda0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e852eba8969318a6 server_cookie=a2ae758acc968d98 in_seq=0 out_seq=0 2024-04-25T08:52:48.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 10 client.9276 ms_handle_connect on v2:172.21.15.46:6832/1279640422 2024-04-25T08:52:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 1 -- 172.21.15.46:0/1001930536 <== mds.0 v2:172.21.15.46:6832/1279640422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555bed479a50 con 0x555bed6dc790 2024-04-25T08:52:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 10 client.9276 handle_command_reply: tid=0 2024-04-25T08:52:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 client.9276 shutdown 2024-04-25T08:52:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 -- 172.21.15.46:0/1001930536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dd490 msgr2=0x555bed6e9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dd490 0x555bed6e9080 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1920007f00 tx=0x7f1920001260 comp rx=0 tx=0).stop 2024-04-25T08:52:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 -- 172.21.15.46:0/1001930536 shutdown_connections 2024-04-25T08:52:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x555bed6dc790 0x555bed6edda0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dd490 0x555bed6e9080 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed6e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 --2- 172.21.15.46:0/1001930536 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 0x555bed6e41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f190be00640 0 client.9276 ms_handle_reset on v2:172.21.15.46:6832/1279640422 2024-04-25T08:52:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 -- 172.21.15.46:0/1001930536 >> 172.21.15.46:0/1001930536 conn(0x555bed6952a0 msgr2=0x555bed6946c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 -- 172.21.15.46:0/1001930536 shutdown_connections 2024-04-25T08:52:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 1 -- 172.21.15.46:0/1001930536 wait complete. 2024-04-25T08:52:48.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.426+0000 7f193093b000 20 client.9276 trim_cache size 0 max 16384 2024-04-25T08:52:48.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.442+0000 7f193093b000 1 Processor -- start 2024-04-25T08:52:48.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.442+0000 7f193093b000 1 -- start start 2024-04-25T08:52:48.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.442+0000 7f193093b000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed68d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed68df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed6ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192806fdd0 con 0x555bed693340 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19280c8e30 con 0x555bed6dcb70 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192806f7d0 con 0x555bed6dd490 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1926800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed68d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1926800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed68d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45914/0 (socket says 172.21.15.46:45914) 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1926800640 1 -- 172.21.15.46:0/893227764 learned_addr learned my addr 172.21.15.46:0/893227764 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f191fe00640 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed68df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1927200640 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed6ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1926800640 1 -- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 msgr2=0x555bed6ecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1926800640 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed6ecda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1926800640 1 -- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 msgr2=0x555bed68df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1926800640 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed68df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1926800640 1 -- 172.21.15.46:0/893227764 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bed693170 con 0x555bed693340 2024-04-25T08:52:48.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f191fe00640 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed68df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:48.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1927200640 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed6ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:48.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1926800640 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed68d9e0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f191801c530 tx=0x7f1918009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6ce07702572eda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1916a00640 1 -- 172.21.15.46:0/893227764 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180cb550 con 0x555bed693340 2024-04-25T08:52:48.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1916a00640 1 -- 172.21.15.46:0/893227764 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19180cf030 con 0x555bed693340 2024-04-25T08:52:48.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f1916a00640 1 -- 172.21.15.46:0/893227764 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19180cb8f0 con 0x555bed693340 2024-04-25T08:52:48.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- 172.21.15.46:0/893227764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 msgr2=0x555bed68d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed68d9e0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f191801c530 tx=0x7f1918009240 comp rx=0 tx=0).stop 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- 172.21.15.46:0/893227764 shutdown_connections 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed6dd490 0x555bed6ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dcb70 0x555bed68df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 --2- 172.21.15.46:0/893227764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed693340 0x555bed68d9e0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- 172.21.15.46:0/893227764 >> 172.21.15.46:0/893227764 conn(0x555bed6952a0 msgr2=0x555bed68d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- 172.21.15.46:0/893227764 shutdown_connections 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- 172.21.15.46:0/893227764 wait complete. 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- start start 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 Processor -- start 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 0x555bed73de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dcb70 0x555bed73e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dd490 0x555bed7217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bed4e23d0 con 0x555bed6dcb70 2024-04-25T08:52:48.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bed599d50 con 0x555bed6dd490 2024-04-25T08:52:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bed478980 con 0x555bed693340 2024-04-25T08:52:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f193093b000 10 client.0 _ll_register_callbacks cb 0x7f19200017d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.446+0000 7f191fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dcb70 0x555bed73e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f191fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dcb70 0x555bed73e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45924/0 (socket says 172.21.15.46:45924) 2024-04-25T08:52:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f191fe00640 1 -- 172.21.15.46:0/3982972031 learned_addr learned my addr 172.21.15.46:0/3982972031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f1926800640 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 0x555bed73de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f1927200640 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dd490 0x555bed7217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f191fe00640 1 -- 172.21.15.46:0/3982972031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 msgr2=0x555bed73de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:48.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f191fe00640 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 0x555bed73de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f191fe00640 1 -- 172.21.15.46:0/3982972031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dd490 msgr2=0x555bed7217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f191fe00640 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dd490 0x555bed7217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f191fe00640 1 -- 172.21.15.46:0/3982972031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bed55f600 con 0x555bed6dcb70 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f191fe00640 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dcb70 0x555bed73e350 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f1910052190 tx=0x7f1910055780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9711b6a53b97e2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 1 -- 172.21.15.46:0/3982972031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191005e440 con 0x555bed6dcb70 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 1 -- 172.21.15.46:0/3982972031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f19100523a0 con 0x555bed6dcb70 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 -- 172.21.15.46:0/3982972031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19180cf030 con 0x555bed6dcb70 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 1 -- 172.21.15.46:0/3982972031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1910055180 con 0x555bed6dcb70 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 10 client.9282 fetch_fsmap learned FSMap version 105 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 -- 172.21.15.46:0/3982972031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x555bed4729b0 con 0x555bed6dcb70 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 1 -- 172.21.15.46:0/3982972031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19180cf030 con 0x555bed6dcb70 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 1 -- 172.21.15.46:0/3982972031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1910013070 con 0x555bed6dcb70 2024-04-25T08:52:48.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 10 client.9282 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 10 client.9282 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 20 client.9282 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x555bed721d30 0x555bed722110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 4 client.9282 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 -- 172.21.15.46:0/3982972031 --> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555bed4eb2e0 con 0x555bed721d30 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f1926800640 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x555bed721d30 0x555bed722110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 10 client.9282 ms_handle_connect on v2:172.21.15.46:6832/1279640422 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f1926800640 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x555bed721d30 0x555bed722110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb9481f5b784ffb server_cookie=d0a72d09dc628968 in_seq=0 out_seq=0 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 1 -- 172.21.15.46:0/3982972031 <== mds.0 v2:172.21.15.46:6832/1279640422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x555bed4eb2e0 con 0x555bed721d30 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 10 client.9282 handle_command_reply: tid=0 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 client.9282 shutdown 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 -- 172.21.15.46:0/3982972031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dcb70 msgr2=0x555bed73e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dcb70 0x555bed73e350 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f1910052190 tx=0x7f1910055780 comp rx=0 tx=0).stop 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 -- 172.21.15.46:0/3982972031 shutdown_connections 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x555bed721d30 0x555bed722110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555bed6dd490 0x555bed7217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555bed6dcb70 0x555bed73e350 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f190be00640 0 client.9282 ms_handle_reset on v2:172.21.15.46:6832/1279640422 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 --2- 172.21.15.46:0/3982972031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x555bed693340 0x555bed73de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 -- 172.21.15.46:0/3982972031 >> 172.21.15.46:0/3982972031 conn(0x555bed6952a0 msgr2=0x555bed68d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 -- 172.21.15.46:0/3982972031 shutdown_connections 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 1 -- 172.21.15.46:0/3982972031 wait complete. 2024-04-25T08:52:48.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f193093b000 20 client.9282 trim_cache size 0 max 16384 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8970, 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5852004820000003, 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 105, 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 122, 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 122, 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 25.534508545000001 2024-04-25T08:52:48.448 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:52:48.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f192d400640 1 -- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1904066f90 msgr2=0x7f1904069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f192d400640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1904066f90 0x7f1904069450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f19180ae000 tx=0x7f19180a7000 comp rx=0 tx=0).stop 2024-04-25T08:52:48.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f192d400640 1 -- 172.21.15.46:0/1672682293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19280cf340 msgr2=0x7f19280b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.450+0000 7f192d400640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19280cf340 0x7f19280b31a0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f191000d7d0 tx=0x7f191000dc90 comp rx=0 tx=0).stop 2024-04-25T08:52:48.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.454+0000 7f192d400640 1 -- 172.21.15.46:0/1672682293 shutdown_connections 2024-04-25T08:52:48.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.454+0000 7f192d400640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1904066f90 0x7f1904069450 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.454+0000 7f192d400640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f192815b8d0 0x7f19280b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.454+0000 7f192d400640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19280cf340 0x7f19280b31a0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.454+0000 7f192d400640 1 --2- 172.21.15.46:0/1672682293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f192806ac60 0x7f19281527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.454+0000 7f192d400640 1 -- 172.21.15.46:0/1672682293 >> 172.21.15.46:0/1672682293 conn(0x7f19280b2100 msgr2=0x7f19280adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.454+0000 7f192d400640 1 -- 172.21.15.46:0/1672682293 shutdown_connections 2024-04-25T08:52:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.454+0000 7f192d400640 1 -- 172.21.15.46:0/1672682293 wait complete. 2024-04-25T08:52:48.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:52:48.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.610+0000 7fb1cc000640 1 Processor -- start 2024-04-25T08:52:48.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.610+0000 7fb1cc000640 1 -- start start 2024-04-25T08:52:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1cc000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1c40719d0 0x7fb1c41401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1cc000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c413f110 con 0x7fb1c415cee0 2024-04-25T08:52:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1cc000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c413f3f0 con 0x7fb1c415ed80 2024-04-25T08:52:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1cc000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c40a8570 con 0x7fb1c40719d0 2024-04-25T08:52:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1ca000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1c40719d0 0x7fb1c41401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1c8c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb1c415cee0 0x7fb1c40712c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45118/0 (socket says 172.21.15.46:45118) 2024-04-25T08:52:48.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1c8c00640 1 -- 172.21.15.46:0/124528355 learned_addr learned my addr 172.21.15.46:0/124528355 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1ca000640 1 -- 172.21.15.46:0/124528355 >> v1:172.21.15.82:6789/0 conn(0x7fb1c415ed80 legacy=0x7fb1c415c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1ca000640 1 -- 172.21.15.46:0/124528355 >> v1:172.21.15.46:6789/0 conn(0x7fb1c415cee0 legacy=0x7fb1c40712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1c3e00640 1 -- 172.21.15.46:0/124528355 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507446931 0 0) 0x7fb1c413f110 con 0x7fb1c415cee0 2024-04-25T08:52:48.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.614+0000 7fb1ca000640 1 -- 172.21.15.46:0/124528355 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c413f6d0 con 0x7fb1c40719d0 2024-04-25T08:52:48.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1ca000640 1 --2- 172.21.15.46:0/124528355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c40719d0 0x7fb1c41401b0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009d90 tx=0x7fb1bc01f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1f257ea731a42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1c3e00640 1 -- 172.21.15.46:0/124528355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0b4070 con 0x7fb1c40719d0 2024-04-25T08:52:48.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1c3e00640 1 -- 172.21.15.46:0/124528355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc0af600 con 0x7fb1c40719d0 2024-04-25T08:52:48.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1c3e00640 1 -- 172.21.15.46:0/124528355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0af9d0 con 0x7fb1c40719d0 2024-04-25T08:52:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1cc000640 1 -- 172.21.15.46:0/124528355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c40719d0 msgr2=0x7fb1c41401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1cc000640 1 --2- 172.21.15.46:0/124528355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c40719d0 0x7fb1c41401b0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009d90 tx=0x7fb1bc01f150 comp rx=0 tx=0).stop 2024-04-25T08:52:48.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1cc000640 1 -- 172.21.15.46:0/124528355 shutdown_connections 2024-04-25T08:52:48.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1cc000640 1 --2- 172.21.15.46:0/124528355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c40719d0 0x7fb1c41401b0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1cc000640 1 -- 172.21.15.46:0/124528355 >> 172.21.15.46:0/124528355 conn(0x7fb1c4066fd0 msgr2=0x7fb1c406ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:48.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1cc000640 1 -- 172.21.15.46:0/124528355 shutdown_connections 2024-04-25T08:52:48.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1cc000640 1 -- 172.21.15.46:0/124528355 wait complete. 2024-04-25T08:52:48.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1cc000640 1 Processor -- start 2024-04-25T08:52:48.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.618+0000 7fb1cc000640 1 -- start start 2024-04-25T08:52:48.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1cc000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1c40719d0 0x7fb1c415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1cc000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c415cee0 0x7fb1c40adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1cc000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1c415ed80 0x7fb1c40abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1cc000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c413fd20 con 0x7fb1c415cee0 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1cc000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c413eac0 con 0x7fb1c415ed80 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1cc000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40cd710 con 0x7fb1c40719d0 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c415cee0 0x7fb1c40adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c415cee0 0x7fb1c40adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45930/0 (socket says 172.21.15.46:45930) 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c8c00640 1 -- 172.21.15.46:0/2724213031 learned_addr learned my addr 172.21.15.46:0/2724213031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c9600640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1c40719d0 0x7fb1c415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1ca000640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1c415ed80 0x7fb1c40abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c8c00640 1 -- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1c40719d0 msgr2=0x7fb1c415abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c8c00640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1c40719d0 0x7fb1c415abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c8c00640 1 -- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1c415ed80 msgr2=0x7fb1c40abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c8c00640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1c415ed80 0x7fb1c40abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c8c00640 1 -- 172.21.15.46:0/2724213031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c40aa240 con 0x7fb1c415cee0 2024-04-25T08:52:48.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1ca000640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1c415ed80 0x7fb1c40abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:48.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c8c00640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c415cee0 0x7fb1c40adec0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c480 tx=0x7fb1b000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c1600640 1 -- 172.21.15.46:0/2724213031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0018070 con 0x7fb1c415cee0 2024-04-25T08:52:48.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c1600640 1 -- 172.21.15.46:0/2724213031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b00074e0 con 0x7fb1c415cee0 2024-04-25T08:52:48.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c1600640 1 -- 172.21.15.46:0/2724213031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0004a30 con 0x7fb1c415cee0 2024-04-25T08:52:48.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c40aa520 con 0x7fb1c415cee0 2024-04-25T08:52:48.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c40aaae0 con 0x7fb1c415cee0 2024-04-25T08:52:48.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c406bbf0 con 0x7fb1c415cee0 2024-04-25T08:52:48.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.622+0000 7fb1c1600640 1 -- 172.21.15.46:0/2724213031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1b001b070 con 0x7fb1c415cee0 2024-04-25T08:52:48.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.626+0000 7fb1c1600640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb1a4066f60 0x7fb1a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:48.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.626+0000 7fb1c1600640 1 -- 172.21.15.46:0/2724213031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb1b0084570 con 0x7fb1c415cee0 2024-04-25T08:52:48.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.626+0000 7fb1c9600640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb1a4066f60 0x7fb1a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:48.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.626+0000 7fb1c9600640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb1a4066f60 0x7fb1a4069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8007fd0 tx=0x7fb1b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:48.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.630+0000 7fb1c1600640 1 -- 172.21.15.46:0/2724213031 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb1b0054d40 con 0x7fb1c415cee0 2024-04-25T08:52:48.727 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-04-25T08:52:48.734+0000 7f661e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:52:48.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.826+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb1c40ccde0 con 0x7fb1c415cee0 2024-04-25T08:52:48.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.826+0000 7fb1c1600640 1 -- 172.21.15.46:0/2724213031 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb1b0058de0 con 0x7fb1c415cee0 2024-04-25T08:52:48.819 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb1a4066f60 msgr2=0x7fb1a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb1a4066f60 0x7fb1a4069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8007fd0 tx=0x7fb1b8009040 comp rx=0 tx=0).stop 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c415cee0 msgr2=0x7fb1c40adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c415cee0 0x7fb1c40adec0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000c480 tx=0x7fb1b000c940 comp rx=0 tx=0).stop 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 shutdown_connections 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb1a4066f60 0x7fb1a4069420 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1c415ed80 0x7fb1c40abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1c415cee0 0x7fb1c40adec0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 --2- 172.21.15.46:0/2724213031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fb1c40719d0 0x7fb1c415abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 >> 172.21.15.46:0/2724213031 conn(0x7fb1c4066fd0 msgr2=0x7fb1c41401b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:48.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 shutdown_connections 2024-04-25T08:52:48.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:48.830+0000 7fb1cc000640 1 -- 172.21.15.46:0/2724213031 wait complete. 2024-04-25T08:52:48.840 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:52:48.840 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:52:48.840 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:52:48.840 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:52:48.840 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:48.840 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:52:48.840 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:52:48.840 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:52:48.840 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:52:48.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:48.841 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:52:48.850 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:52:48.850 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:52:48.850 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:52:48.850 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:48.850 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:52:48.850 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:48.850 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41834sec preferred_lft 41834sec 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:48.851 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:52:48.852 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:52:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:48.853 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:52:48.853 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:52:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:52:48.854 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:48.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:48.856 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:52:48.921 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:52:48.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:48.924 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:52:48.976 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:52:48.977 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:52:48.977 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:52:48.977 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:52:48.977 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:52:48.977 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:52:48.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:48.977 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:52:48.977 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:52:48.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:52:48.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:52:48.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:52:48.978 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:52:49.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:49.070 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:52:49.078 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:52:49.079 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:52:49.079 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:52:49.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:49.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:52:49.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:52:49.143 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:49.196 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:52:49.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:52:49.196 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:49.247 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:52:49.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:49.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:49.340 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:49.340 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:52:49.340 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:52:49.355 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:49.355 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:52:49.355 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:52:49.355 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:52:49.355 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:52:49.364 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:52:49.364 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:52:49.364 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:52:49.364 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:52:49.364 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:52:49.365 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:52:49.365 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:52:49.365 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:52:49.368 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:52:49.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:49.369 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:49.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:49.389 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:52:49.443 INFO:teuthology.orchestra.run.smithi046.stdout:client7781 2024-04-25T08:52:49.443 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:49.443 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:49.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:49.500 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T08:52:49.560 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-04-25T08:52:49.561 INFO:teuthology.orchestra.run.smithi046.stdout:./parent 2024-04-25T08:52:49.561 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed 2024-04-25T08:52:49.561 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/charlie 2024-04-25T08:52:49.561 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/alpha 2024-04-25T08:52:49.561 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/unflushed 2024-04-25T08:52:49.561 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/unflushed/jfile 2024-04-25T08:52:49.562 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:49.562 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:49.562 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:49.562 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:49.563 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T08:52:49.563 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:49.669 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:52:49.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:49.678 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:49.679 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:49.679 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:49.679 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:49.679 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T08:52:49.679 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:49.785 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-25T08:52:49.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:49.793 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:49.794 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:49.794 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:49.794 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:49.794 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T08:52:49.794 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:49.903 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-04-25T08:52:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:49.912 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:49.912 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:49.913 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:49.913 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:49.913 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T08:52:49.913 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:50.021 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627780 2024-04-25T08:52:50.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:50.030 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:50.030 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:50.030 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:50.030 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:50.030 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T08:52:50.030 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:50.141 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-04-25T08:52:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:50.150 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:50.150 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:50.150 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:50.150 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:50.150 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T08:52:50.150 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:50.261 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627781 2024-04-25T08:52:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:50.269 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:52:50.269 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:52:50.270 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:52:50.270 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:52:50.270 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T08:52:50.270 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:52:50.376 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627782 2024-04-25T08:52:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:50.386 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:50.386 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:50.436 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:52:50.436 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:50.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bbe400640 1 Processor -- start 2024-04-25T08:52:50.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bbe400640 1 -- start start 2024-04-25T08:52:50.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bbe400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceac0 0x7f2bb80b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:50.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bbe400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2bb80724a0 0x7f2bb8072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bbe400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb815ad00 con 0x7f2bb80ceea0 2024-04-25T08:52:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bbe400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb80b4eb0 con 0x7f2bb80724a0 2024-04-25T08:52:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bbe400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb815afc0 con 0x7f2bb80ceac0 2024-04-25T08:52:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bafe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2bb80ceea0 0x7f2bb80ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45122/0 (socket says 172.21.15.46:45122) 2024-04-25T08:52:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bafe00640 1 -- 172.21.15.46:0/2816909721 learned_addr learned my addr 172.21.15.46:0/2816909721 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bb7e00640 1 --2- 172.21.15.46:0/2816909721 >> v2:172.21.15.82:3300/0 conn(0x7f2bb80724a0 0x7f2bb8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bb7400640 1 --2- 172.21.15.46:0/2816909721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceac0 0x7f2bb80b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bb6a00640 1 -- 172.21.15.46:0/2816909721 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709061858 0 0) 0x7f2bb815ad00 con 0x7f2bb80ceea0 2024-04-25T08:52:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bb6a00640 1 -- 172.21.15.46:0/2816909721 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b9c003530 con 0x7f2bb80ceea0 2024-04-25T08:52:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bb7e00640 1 -- 172.21.15.46:0/2816909721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceac0 msgr2=0x7f2bb80b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bb7e00640 1 --2- 172.21.15.46:0/2816909721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceac0 0x7f2bb80b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bb7e00640 1 -- 172.21.15.46:0/2816909721 >> v1:172.21.15.46:6789/0 conn(0x7f2bb80ceea0 legacy=0x7f2bb80ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bb7e00640 1 -- 172.21.15.46:0/2816909721 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb815b2a0 con 0x7f2bb80724a0 2024-04-25T08:52:50.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.682+0000 7f2bb7400640 1 --2- 172.21.15.46:0/2816909721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceac0 0x7f2bb80b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bb7e00640 1 --2- 172.21.15.46:0/2816909721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb80724a0 0x7f2bb8072900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009d90 tx=0x7f2bb001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fffb7d79aff0eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bb6a00640 1 -- 172.21.15.46:0/2816909721 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00b2070 con 0x7f2bb80724a0 2024-04-25T08:52:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bb6a00640 1 -- 172.21.15.46:0/2816909721 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb00a5a30 con 0x7f2bb80724a0 2024-04-25T08:52:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bb6a00640 1 -- 172.21.15.46:0/2816909721 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00a5de0 con 0x7f2bb80724a0 2024-04-25T08:52:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bbe400640 1 -- 172.21.15.46:0/2816909721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb80724a0 msgr2=0x7f2bb8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bbe400640 1 --2- 172.21.15.46:0/2816909721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb80724a0 0x7f2bb8072900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009d90 tx=0x7f2bb001c970 comp rx=0 tx=0).stop 2024-04-25T08:52:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bbe400640 1 -- 172.21.15.46:0/2816909721 shutdown_connections 2024-04-25T08:52:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bbe400640 1 --2- 172.21.15.46:0/2816909721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb80724a0 0x7f2bb8072900 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bbe400640 1 --2- 172.21.15.46:0/2816909721 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceac0 0x7f2bb80b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bbe400640 1 -- 172.21.15.46:0/2816909721 >> 172.21.15.46:0/2816909721 conn(0x7f2bb80b2330 msgr2=0x7f2bb80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bbe400640 1 -- 172.21.15.46:0/2816909721 shutdown_connections 2024-04-25T08:52:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.686+0000 7f2bbe400640 1 -- 172.21.15.46:0/2816909721 wait complete. 2024-04-25T08:52:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 Processor -- start 2024-04-25T08:52:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 -- start start 2024-04-25T08:52:50.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb80724a0 0x7f2bb81580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:50.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb80ceac0 0x7f2bb8153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:50.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceea0 0x7f2bb814fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:50.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80cf2a0 con 0x7f2bb80ceac0 2024-04-25T08:52:50.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80d1320 con 0x7f2bb80724a0 2024-04-25T08:52:50.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80cf420 con 0x7f2bb80ceea0 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb80ceac0 0x7f2bb8153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bafe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb80ceac0 0x7f2bb8153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45944/0 (socket says 172.21.15.46:45944) 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bafe00640 1 -- 172.21.15.46:0/2214313109 learned_addr learned my addr 172.21.15.46:0/2214313109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bb7400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb80724a0 0x7f2bb81580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bb7e00640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceea0 0x7f2bb814fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bafe00640 1 -- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceea0 msgr2=0x7f2bb814fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bafe00640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceea0 0x7f2bb814fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bafe00640 1 -- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb80724a0 msgr2=0x7f2bb81580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bafe00640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb80724a0 0x7f2bb81580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bafe00640 1 -- 172.21.15.46:0/2214313109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8156650 con 0x7f2bb80ceac0 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bafe00640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb80ceac0 0x7f2bb8153750 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c3c0 tx=0x7f2ba000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2baf400640 1 -- 172.21.15.46:0/2214313109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0017070 con 0x7f2bb80ceac0 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2baf400640 1 -- 172.21.15.46:0/2214313109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0005cf0 con 0x7f2bb80ceac0 2024-04-25T08:52:50.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2baf400640 1 -- 172.21.15.46:0/2214313109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00072f0 con 0x7f2bb80ceac0 2024-04-25T08:52:50.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb8156930 con 0x7f2bb80ceac0 2024-04-25T08:52:50.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.690+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb8156ef0 con 0x7f2bb80ceac0 2024-04-25T08:52:50.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.694+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb806f910 con 0x7f2bb80ceac0 2024-04-25T08:52:50.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.694+0000 7f2baf400640 1 -- 172.21.15.46:0/2214313109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ba001a070 con 0x7f2bb80ceac0 2024-04-25T08:52:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.694+0000 7f2baf400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2b94066f60 0x7f2b94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.694+0000 7f2baf400640 1 -- 172.21.15.46:0/2214313109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2ba0084530 con 0x7f2bb80ceac0 2024-04-25T08:52:50.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.694+0000 7f2bb7400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2b94066f60 0x7f2b94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:50.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.694+0000 7f2bb7400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2b94066f60 0x7f2b94069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80ccde0 tx=0x7f2ba8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:50.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.698+0000 7f2baf400640 1 -- 172.21.15.46:0/2214313109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ba0054d00 con 0x7f2bb80ceac0 2024-04-25T08:52:50.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.862+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bb81570c0 con 0x7f2bb80ceac0 2024-04-25T08:52:50.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.862+0000 7f2baf400640 1 -- 172.21.15.46:0/2214313109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17029 (secure 0 0 0) 0x7f2ba0058da0 con 0x7f2bb80ceac0 2024-04-25T08:52:50.854 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:50.854 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":122,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:42.856658+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/51204523":"2024-04-26T08:52:42.856639+0000","172.21.15.46:6833/51204523":"2024-04-26T08:52:42.856639+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2b94066f60 msgr2=0x7f2b94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2b94066f60 0x7f2b94069420 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80ccde0 tx=0x7f2ba8009190 comp rx=0 tx=0).stop 2024-04-25T08:52:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb80ceac0 msgr2=0x7f2bb8153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb80ceac0 0x7f2bb8153750 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c3c0 tx=0x7f2ba000c880 comp rx=0 tx=0).stop 2024-04-25T08:52:50.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 shutdown_connections 2024-04-25T08:52:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2b94066f60 0x7f2b94069420 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2bb80ceea0 0x7f2bb814fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bb80ceac0 0x7f2bb8153750 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 --2- 172.21.15.46:0/2214313109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bb80724a0 0x7f2bb81580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 >> 172.21.15.46:0/2214313109 conn(0x7f2bb80b2330 msgr2=0x7f2bb80b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 shutdown_connections 2024-04-25T08:52:50.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:50.866+0000 7f2bbe400640 1 -- 172.21.15.46:0/2214313109 wait complete. 2024-04-25T08:52:50.876 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:25.941254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:27.155001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:52:50.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:51.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a61400640 1 Processor -- start 2024-04-25T08:52:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a61400640 1 -- start start 2024-04-25T08:52:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a61400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a5c15b510 0x7f9a5c15df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a61400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5c1619d0 con 0x7f9a5c161510 2024-04-25T08:52:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a61400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5c161cb0 con 0x7f9a5c0b3710 2024-04-25T08:52:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a61400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5c06ac60 con 0x7f9a5c15b510 2024-04-25T08:52:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a5a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9a5c15b510 0x7f9a5c15df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a51e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9a5c161510 0x7f9a5c0b3000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45132/0 (socket says 172.21.15.46:45132) 2024-04-25T08:52:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a51e00640 1 -- 172.21.15.46:0/1837196102 learned_addr learned my addr 172.21.15.46:0/1837196102 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a5a800640 1 -- 172.21.15.46:0/1837196102 >> v1:172.21.15.82:6789/0 conn(0x7f9a5c0b3710 legacy=0x7f9a5c06b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a5a800640 1 -- 172.21.15.46:0/1837196102 >> v1:172.21.15.46:6789/0 conn(0x7f9a5c161510 legacy=0x7f9a5c0b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a5a800640 1 -- 172.21.15.46:0/1837196102 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c161f90 con 0x7f9a5c15b510 2024-04-25T08:52:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.030+0000 7f9a59e00640 1 -- 172.21.15.46:0/1837196102 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774848221 0 0) 0x7f9a5c1619d0 con 0x7f9a5c161510 2024-04-25T08:52:51.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a5a800640 1 --2- 172.21.15.46:0/1837196102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c15b510 0x7f9a5c15df20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0097c0 tx=0x7f9a4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd835d03a0a017b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:51.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a59e00640 1 -- 172.21.15.46:0/1837196102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0b0070 con 0x7f9a5c15b510 2024-04-25T08:52:51.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a59e00640 1 -- 172.21.15.46:0/1837196102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c0ab600 con 0x7f9a5c15b510 2024-04-25T08:52:51.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a59e00640 1 -- 172.21.15.46:0/1837196102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c0ab9d0 con 0x7f9a5c15b510 2024-04-25T08:52:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a61400640 1 -- 172.21.15.46:0/1837196102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c15b510 msgr2=0x7f9a5c15df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a61400640 1 --2- 172.21.15.46:0/1837196102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c15b510 0x7f9a5c15df20 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0097c0 tx=0x7f9a4c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a61400640 1 -- 172.21.15.46:0/1837196102 shutdown_connections 2024-04-25T08:52:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a61400640 1 --2- 172.21.15.46:0/1837196102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c15b510 0x7f9a5c15df20 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a61400640 1 -- 172.21.15.46:0/1837196102 >> 172.21.15.46:0/1837196102 conn(0x7f9a5c0b2270 msgr2=0x7f9a5c0cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:51.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a61400640 1 -- 172.21.15.46:0/1837196102 shutdown_connections 2024-04-25T08:52:51.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.034+0000 7f9a61400640 1 -- 172.21.15.46:0/1837196102 wait complete. 2024-04-25T08:52:51.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 Processor -- start 2024-04-25T08:52:51.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 -- start start 2024-04-25T08:52:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a5c0b3710 0x7f9a5c149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c15b510 0x7f9a5c0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c161510 0x7f9a5c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c06c380 con 0x7f9a5c161510 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c06c200 con 0x7f9a5c15b510 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c16b8c0 con 0x7f9a5c0b3710 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c161510 0x7f9a5c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5a800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a5c0b3710 0x7f9a5c149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c161510 0x7f9a5c1553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45966/0 (socket says 172.21.15.46:45966) 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5b200640 1 -- 172.21.15.46:0/18671563 learned_addr learned my addr 172.21.15.46:0/18671563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a51e00640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c15b510 0x7f9a5c0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5b200640 1 -- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a5c0b3710 msgr2=0x7f9a5c149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5b200640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a5c0b3710 0x7f9a5c149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5b200640 1 -- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c15b510 msgr2=0x7f9a5c0acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5b200640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c15b510 0x7f9a5c0acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5b200640 1 -- 172.21.15.46:0/18671563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c0ad620 con 0x7f9a5c161510 2024-04-25T08:52:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a5b200640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c161510 0x7f9a5c1553a0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c770 tx=0x7f9a5400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a53400640 1 -- 172.21.15.46:0/18671563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a54003b80 con 0x7f9a5c161510 2024-04-25T08:52:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a53400640 1 -- 172.21.15.46:0/18671563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a54003d20 con 0x7f9a5c161510 2024-04-25T08:52:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a53400640 1 -- 172.21.15.46:0/18671563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5400aa90 con 0x7f9a5c161510 2024-04-25T08:52:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c0ad900 con 0x7f9a5c161510 2024-04-25T08:52:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c158cf0 con 0x7f9a5c161510 2024-04-25T08:52:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.038+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5c04ef30 con 0x7f9a5c161510 2024-04-25T08:52:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.042+0000 7f9a53400640 1 -- 172.21.15.46:0/18671563 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a5400ac30 con 0x7f9a5c161510 2024-04-25T08:52:51.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.042+0000 7f9a53400640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a38066f60 0x7f9a38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.042+0000 7f9a53400640 1 -- 172.21.15.46:0/18671563 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9a54084fe0 con 0x7f9a5c161510 2024-04-25T08:52:51.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.042+0000 7f9a5a800640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a38066f60 0x7f9a38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.042+0000 7f9a5a800640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a38066f60 0x7f9a38069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0ad000 tx=0x7f9a4c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.046+0000 7f9a53400640 1 -- 172.21.15.46:0/18671563 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a540557b0 con 0x7f9a5c161510 2024-04-25T08:52:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a5c158ec0 con 0x7f9a5c161510 2024-04-25T08:52:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a53400640 1 -- 172.21.15.46:0/18671563 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a54059850 con 0x7f9a5c161510 2024-04-25T08:52:51.200 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:52:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a38066f60 msgr2=0x7f9a38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a38066f60 0x7f9a38069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0ad000 tx=0x7f9a4c0a6000 comp rx=0 tx=0).stop 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c161510 msgr2=0x7f9a5c1553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c161510 0x7f9a5c1553a0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400c770 tx=0x7f9a5400cc30 comp rx=0 tx=0).stop 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 shutdown_connections 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9a38066f60 0x7f9a38069420 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a5c161510 0x7f9a5c1553a0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a5c15b510 0x7f9a5c0acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 --2- 172.21.15.46:0/18671563 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9a5c0b3710 0x7f9a5c149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 >> 172.21.15.46:0/18671563 conn(0x7f9a5c0b2270 msgr2=0x7f9a5c1536b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 shutdown_connections 2024-04-25T08:52:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.210+0000 7f9a61400640 1 -- 172.21.15.46:0/18671563 wait complete. 2024-04-25T08:52:51.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:52:51.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fceb1000640 1 Processor -- start 2024-04-25T08:52:51.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fceb1000640 1 -- start start 2024-04-25T08:52:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fceb1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fceac0b4f90 0x7fceac15b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fceb1000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac15b540 0x7fceac0cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fceb1000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceac0cf250 con 0x7fceac144300 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fceb1000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceac0ce490 con 0x7fceac0b4f90 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fceb1000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceac0ce770 con 0x7fceac15b540 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fcea9e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fceac144300 0x7fceac0b4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45148/0 (socket says 172.21.15.46:45148) 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fcea1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fceac0b4f90 0x7fceac15b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fcea9e00640 1 -- 172.21.15.46:0/666992879 learned_addr learned my addr 172.21.15.46:0/666992879 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.374+0000 7fceaa800640 1 --2- 172.21.15.46:0/666992879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac15b540 0x7fceac0cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fcea1400640 1 -- 172.21.15.46:0/666992879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac15b540 msgr2=0x7fceac0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fcea1400640 1 --2- 172.21.15.46:0/666992879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac15b540 0x7fceac0cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fcea1400640 1 -- 172.21.15.46:0/666992879 >> v1:172.21.15.46:6789/0 conn(0x7fceac144300 legacy=0x7fceac0b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fcea1400640 1 -- 172.21.15.46:0/666992879 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac0cea50 con 0x7fceac0b4f90 2024-04-25T08:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fcea9400640 1 -- 172.21.15.46:0/666992879 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842737962 0 0) 0x7fceac0cf250 con 0x7fceac144300 2024-04-25T08:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fcea1400640 1 --2- 172.21.15.46:0/666992879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac0b4f90 0x7fceac15b000 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fce940097c0 tx=0x7fce9401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55071fc8b95468eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fcea9400640 1 -- 172.21.15.46:0/666992879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940b1070 con 0x7fceac0b4f90 2024-04-25T08:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fcea9400640 1 -- 172.21.15.46:0/666992879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce940ac410 con 0x7fceac0b4f90 2024-04-25T08:52:51.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fcea9400640 1 -- 172.21.15.46:0/666992879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940ac7e0 con 0x7fceac0b4f90 2024-04-25T08:52:51.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fceb1000640 1 -- 172.21.15.46:0/666992879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac0b4f90 msgr2=0x7fceac15b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fceb1000640 1 --2- 172.21.15.46:0/666992879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac0b4f90 0x7fceac15b000 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fce940097c0 tx=0x7fce9401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fceb1000640 1 -- 172.21.15.46:0/666992879 shutdown_connections 2024-04-25T08:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fceb1000640 1 --2- 172.21.15.46:0/666992879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac15b540 0x7fceac0cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fceb1000640 1 --2- 172.21.15.46:0/666992879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac0b4f90 0x7fceac15b000 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.378+0000 7fceb1000640 1 -- 172.21.15.46:0/666992879 >> 172.21.15.46:0/666992879 conn(0x7fceac06ac60 msgr2=0x7fceac06b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 -- 172.21.15.46:0/666992879 shutdown_connections 2024-04-25T08:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 -- 172.21.15.46:0/666992879 wait complete. 2024-04-25T08:52:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 Processor -- start 2024-04-25T08:52:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 -- start start 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac0b4f90 0x7fceac133fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac144300 0x7fceac151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceac15b540 0x7fceac15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac167610 con 0x7fceac144300 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceac0cd580 con 0x7fceac15b540 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceac167790 con 0x7fceac0b4f90 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac144300 0x7fceac151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac144300 0x7fceac151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45982/0 (socket says 172.21.15.46:45982) 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea1400640 1 -- 172.21.15.46:0/3419749980 learned_addr learned my addr 172.21.15.46:0/3419749980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea9e00640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac0b4f90 0x7fceac133fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceaa800640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceac15b540 0x7fceac15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea1400640 1 -- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac0b4f90 msgr2=0x7fceac133fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea1400640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac0b4f90 0x7fceac133fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea1400640 1 -- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceac15b540 msgr2=0x7fceac15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea1400640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceac15b540 0x7fceac15ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea1400640 1 -- 172.21.15.46:0/3419749980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac152820 con 0x7fceac144300 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea9e00640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac0b4f90 0x7fceac133fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea1400640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac144300 0x7fceac151da0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fce940097c0 tx=0x7fce9401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea2a00640 1 -- 172.21.15.46:0/3419749980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940b1050 con 0x7fceac144300 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea2a00640 1 -- 172.21.15.46:0/3419749980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce940b2040 con 0x7fceac144300 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fcea2a00640 1 -- 172.21.15.46:0/3419749980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94006e20 con 0x7fceac144300 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.382+0000 7fceb1000640 1 -- 172.21.15.46:0/3419749980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceac152b00 con 0x7fceac144300 2024-04-25T08:52:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.386+0000 7fceb1000640 1 -- 172.21.15.46:0/3419749980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceac1530c0 con 0x7fceac144300 2024-04-25T08:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.386+0000 7fce9b400640 1 -- 172.21.15.46:0/3419749980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceac04ef30 con 0x7fceac144300 2024-04-25T08:52:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.386+0000 7fcea2a00640 1 -- 172.21.15.46:0/3419749980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce940bf060 con 0x7fceac144300 2024-04-25T08:52:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.386+0000 7fcea2a00640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce88066f60 0x7fce88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.386+0000 7fcea2a00640 1 -- 172.21.15.46:0/3419749980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fce9411ebf0 con 0x7fceac144300 2024-04-25T08:52:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.386+0000 7fcea9e00640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce88066f60 0x7fce88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.390+0000 7fcea9e00640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce88066f60 0x7fce88069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fce9c007fd0 tx=0x7fce9c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.390+0000 7fcea2a00640 1 -- 172.21.15.46:0/3419749980 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce94016090 con 0x7fceac144300 2024-04-25T08:52:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.554+0000 7fce9b400640 1 -- 172.21.15.46:0/3419749980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fceac0b1250 con 0x7fceac144300 2024-04-25T08:52:51.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.554+0000 7fcea2a00640 1 -- 172.21.15.46:0/3419749980 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce940ef8f0 con 0x7fceac144300 2024-04-25T08:52:51.547 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 -- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce88066f60 msgr2=0x7fce88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce88066f60 0x7fce88069420 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fce9c007fd0 tx=0x7fce9c009040 comp rx=0 tx=0).stop 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 -- 172.21.15.46:0/3419749980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac144300 msgr2=0x7fceac151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac144300 0x7fceac151da0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fce940097c0 tx=0x7fce9401ace0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 -- 172.21.15.46:0/3419749980 shutdown_connections 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fce88066f60 0x7fce88069420 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceac15b540 0x7fceac15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fceac144300 0x7fceac151da0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 --2- 172.21.15.46:0/3419749980 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fceac0b4f90 0x7fceac133fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 -- 172.21.15.46:0/3419749980 >> 172.21.15.46:0/3419749980 conn(0x7fceac06ac60 msgr2=0x7fceac15acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 -- 172.21.15.46:0/3419749980 shutdown_connections 2024-04-25T08:52:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.558+0000 7fce9b400640 1 -- 172.21.15.46:0/3419749980 wait complete. 2024-04-25T08:52:51.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:52:51.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:52:51.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:52:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.722+0000 7efd85000640 1 Processor -- start 2024-04-25T08:52:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.722+0000 7efd85000640 1 -- start start 2024-04-25T08:52:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.722+0000 7efd85000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efd8015b510 0x7efd8015df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.722+0000 7efd85000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd801619d0 con 0x7efd800b3710 2024-04-25T08:52:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.722+0000 7efd85000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd80161cb0 con 0x7efd80161510 2024-04-25T08:52:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.722+0000 7efd85000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8006ac60 con 0x7efd8015b510 2024-04-25T08:52:51.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efd800b3710 0x7efd8006b0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45158/0 (socket says 172.21.15.46:45158) 2024-04-25T08:52:51.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7e800640 1 -- 172.21.15.46:0/3283688071 learned_addr learned my addr 172.21.15.46:0/3283688071 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:51.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7de00640 1 --2- 172.21.15.46:0/3283688071 >> v2:172.21.15.82:3300/0 conn(0x7efd8015b510 0x7efd8015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7d400640 1 -- 172.21.15.46:0/3283688071 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627010262 0 0) 0x7efd801619d0 con 0x7efd800b3710 2024-04-25T08:52:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7d400640 1 -- 172.21.15.46:0/3283688071 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd64003530 con 0x7efd800b3710 2024-04-25T08:52:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7d400640 1 -- 172.21.15.46:0/3283688071 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548389683 0 0) 0x7efd80161cb0 con 0x7efd80161510 2024-04-25T08:52:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7d400640 1 -- 172.21.15.46:0/3283688071 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd801619d0 con 0x7efd80161510 2024-04-25T08:52:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7de00640 1 -- 172.21.15.46:0/3283688071 >> v1:172.21.15.82:6789/0 conn(0x7efd80161510 legacy=0x7efd800b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7de00640 1 -- 172.21.15.46:0/3283688071 >> v1:172.21.15.46:6789/0 conn(0x7efd800b3710 legacy=0x7efd8006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7de00640 1 -- 172.21.15.46:0/3283688071 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80161f90 con 0x7efd8015b510 2024-04-25T08:52:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7de00640 1 --2- 172.21.15.46:0/3283688071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8015b510 0x7efd8015df20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efd70009d90 tx=0x7efd7001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4efaa6ac17bf8954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.726+0000 7efd7d400640 1 -- 172.21.15.46:0/3283688071 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd700b2070 con 0x7efd8015b510 2024-04-25T08:52:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd7d400640 1 -- 172.21.15.46:0/3283688071 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd700a5900 con 0x7efd8015b510 2024-04-25T08:52:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd7d400640 1 -- 172.21.15.46:0/3283688071 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd700a5cb0 con 0x7efd8015b510 2024-04-25T08:52:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 -- 172.21.15.46:0/3283688071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8015b510 msgr2=0x7efd8015df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 --2- 172.21.15.46:0/3283688071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8015b510 0x7efd8015df20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efd70009d90 tx=0x7efd7001c970 comp rx=0 tx=0).stop 2024-04-25T08:52:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 -- 172.21.15.46:0/3283688071 shutdown_connections 2024-04-25T08:52:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 --2- 172.21.15.46:0/3283688071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd8015b510 0x7efd8015df20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 -- 172.21.15.46:0/3283688071 >> 172.21.15.46:0/3283688071 conn(0x7efd800b2270 msgr2=0x7efd800cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 -- 172.21.15.46:0/3283688071 shutdown_connections 2024-04-25T08:52:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 -- 172.21.15.46:0/3283688071 wait complete. 2024-04-25T08:52:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 Processor -- start 2024-04-25T08:52:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 -- start start 2024-04-25T08:52:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd800b3710 0x7efd80149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8015b510 0x7efd80152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80161510 0x7efd80153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8006c380 con 0x7efd8015b510 2024-04-25T08:52:51.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8006c200 con 0x7efd800b3710 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd85000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8016b8c0 con 0x7efd80161510 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd75400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8015b510 0x7efd80152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd75400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8015b510 0x7efd80152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45994/0 (socket says 172.21.15.46:45994) 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd75400640 1 -- 172.21.15.46:0/1832922080 learned_addr learned my addr 172.21.15.46:0/1832922080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd7de00640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd800b3710 0x7efd80149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd7e800640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80161510 0x7efd80153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd75400640 1 -- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80161510 msgr2=0x7efd80153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd75400640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80161510 0x7efd80153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd75400640 1 -- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd800b3710 msgr2=0x7efd80149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd75400640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd800b3710 0x7efd80149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd75400640 1 -- 172.21.15.46:0/1832922080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd801498d0 con 0x7efd8015b510 2024-04-25T08:52:51.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.730+0000 7efd7e800640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80161510 0x7efd80153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:51.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd75400640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8015b510 0x7efd80152ed0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c390 tx=0x7efd6800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:51.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd76a00640 1 -- 172.21.15.46:0/1832922080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68017070 con 0x7efd8015b510 2024-04-25T08:52:51.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd76a00640 1 -- 172.21.15.46:0/1832922080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd680052d0 con 0x7efd8015b510 2024-04-25T08:52:51.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd76a00640 1 -- 172.21.15.46:0/1832922080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68003840 con 0x7efd8015b510 2024-04-25T08:52:51.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd801545c0 con 0x7efd8015b510 2024-04-25T08:52:51.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd80154b80 con 0x7efd8015b510 2024-04-25T08:52:51.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8004ef30 con 0x7efd8015b510 2024-04-25T08:52:51.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd76a00640 1 -- 172.21.15.46:0/1832922080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd6801a070 con 0x7efd8015b510 2024-04-25T08:52:51.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd76a00640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd5c066f60 0x7efd5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:51.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd76a00640 1 -- 172.21.15.46:0/1832922080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efd68084450 con 0x7efd8015b510 2024-04-25T08:52:51.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.734+0000 7efd7de00640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd5c066f60 0x7efd5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:51.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.738+0000 7efd7de00640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd5c066f60 0x7efd5c069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efd700af000 tx=0x7efd700a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:51.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.738+0000 7efd76a00640 1 -- 172.21.15.46:0/1832922080 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd68005480 con 0x7efd8015b510 2024-04-25T08:52:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd800d0ad0 con 0x7efd8015b510 2024-04-25T08:52:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd76a00640 1 -- 172.21.15.46:0/1832922080 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd68054cc0 con 0x7efd8015b510 2024-04-25T08:52:51.897 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:52:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd5c066f60 msgr2=0x7efd5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd5c066f60 0x7efd5c069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7efd700af000 tx=0x7efd700a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8015b510 msgr2=0x7efd80152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8015b510 0x7efd80152ed0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7efd6800c390 tx=0x7efd6800c850 comp rx=0 tx=0).stop 2024-04-25T08:52:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 shutdown_connections 2024-04-25T08:52:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd5c066f60 0x7efd5c069420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efd80161510 0x7efd80153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd8015b510 0x7efd80152ed0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 --2- 172.21.15.46:0/1832922080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd800b3710 0x7efd80149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 >> 172.21.15.46:0/1832922080 conn(0x7efd800b2270 msgr2=0x7efd80156aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 shutdown_connections 2024-04-25T08:52:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:51.906+0000 7efd85000640 1 -- 172.21.15.46:0/1832922080 wait complete. 2024-04-25T08:52:51.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:52:52.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba59400640 1 Processor -- start 2024-04-25T08:52:52.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba59400640 1 -- start start 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba59400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba54143fa0 0x7fba5406e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba59400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba540cec00 0x7fba5415ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba59400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba540b3110 con 0x7fba5406e970 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba59400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5406edd0 con 0x7fba540cec00 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba59400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba540b3a00 con 0x7fba54143fa0 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba53200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba540cec00 0x7fba5415ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba53200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fba540cec00 0x7fba5415ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46010/0 (socket says 172.21.15.46:46010) 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba53200640 1 -- 172.21.15.46:0/1699846353 learned_addr learned my addr 172.21.15.46:0/1699846353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba52800640 1 --2- 172.21.15.46:0/1699846353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba54143fa0 0x7fba5406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba53200640 1 -- 172.21.15.46:0/1699846353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba54143fa0 msgr2=0x7fba5406e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:52.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba53200640 1 --2- 172.21.15.46:0/1699846353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba54143fa0 0x7fba5406e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba53200640 1 -- 172.21.15.46:0/1699846353 >> v1:172.21.15.82:6789/0 conn(0x7fba5406e970 legacy=0x7fba540ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.070+0000 7fba53200640 1 -- 172.21.15.46:0/1699846353 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540b3e90 con 0x7fba540cec00 2024-04-25T08:52:52.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba53200640 1 --2- 172.21.15.46:0/1699846353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba540cec00 0x7fba5415ad00 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fba540b3b80 tx=0x7fba4c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7957d8898734b8f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba51e00640 1 -- 172.21.15.46:0/1699846353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0a56e0 con 0x7fba540cec00 2024-04-25T08:52:52.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba51e00640 1 -- 172.21.15.46:0/1699846353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c0a5880 con 0x7fba540cec00 2024-04-25T08:52:52.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba51e00640 1 -- 172.21.15.46:0/1699846353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0a5c50 con 0x7fba540cec00 2024-04-25T08:52:52.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba59400640 1 -- 172.21.15.46:0/1699846353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba540cec00 msgr2=0x7fba5415ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba59400640 1 --2- 172.21.15.46:0/1699846353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba540cec00 0x7fba5415ad00 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fba540b3b80 tx=0x7fba4c01ca10 comp rx=0 tx=0).stop 2024-04-25T08:52:52.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba59400640 1 -- 172.21.15.46:0/1699846353 shutdown_connections 2024-04-25T08:52:52.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba59400640 1 --2- 172.21.15.46:0/1699846353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba540cec00 0x7fba5415ad00 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba59400640 1 --2- 172.21.15.46:0/1699846353 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba54143fa0 0x7fba5406e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba59400640 1 -- 172.21.15.46:0/1699846353 >> 172.21.15.46:0/1699846353 conn(0x7fba54066fd0 msgr2=0x7fba5406ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:52.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba59400640 1 -- 172.21.15.46:0/1699846353 shutdown_connections 2024-04-25T08:52:52.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.074+0000 7fba59400640 1 -- 172.21.15.46:0/1699846353 wait complete. 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 Processor -- start 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 -- start start 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba5406e970 0x7fba5413b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba540cec00 0x7fba54149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba54143fa0 0x7fba541496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5415ba50 con 0x7fba54143fa0 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba54144380 con 0x7fba540cec00 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5415bbd0 con 0x7fba5406e970 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba53200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba54143fa0 0x7fba541496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba53200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba54143fa0 0x7fba541496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46018/0 (socket says 172.21.15.46:46018) 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba53200640 1 -- 172.21.15.46:0/960841176 learned_addr learned my addr 172.21.15.46:0/960841176 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba4be00640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba540cec00 0x7fba54149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba52800640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba5406e970 0x7fba5413b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba53200640 1 -- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba5406e970 msgr2=0x7fba5413b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba53200640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba5406e970 0x7fba5413b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba53200640 1 -- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba540cec00 msgr2=0x7fba54149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba53200640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba540cec00 0x7fba54149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba53200640 1 -- 172.21.15.46:0/960841176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540b0fd0 con 0x7fba54143fa0 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba53200640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba54143fa0 0x7fba541496a0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fba4c01a640 tx=0x7fba4c0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba4aa00640 1 -- 172.21.15.46:0/960841176 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0b8670 con 0x7fba54143fa0 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba4aa00640 1 -- 172.21.15.46:0/960841176 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c0be070 con 0x7fba54143fa0 2024-04-25T08:52:52.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba4aa00640 1 -- 172.21.15.46:0/960841176 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0b8a70 con 0x7fba54143fa0 2024-04-25T08:52:52.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba5414b650 con 0x7fba54143fa0 2024-04-25T08:52:52.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.078+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba5414a090 con 0x7fba54143fa0 2024-04-25T08:52:52.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.082+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba54072c70 con 0x7fba54143fa0 2024-04-25T08:52:52.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.082+0000 7fba4aa00640 1 -- 172.21.15.46:0/960841176 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba4c0b8c10 con 0x7fba54143fa0 2024-04-25T08:52:52.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.082+0000 7fba4aa00640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba30066f60 0x7fba30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.082+0000 7fba4aa00640 1 -- 172.21.15.46:0/960841176 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fba4c120590 con 0x7fba54143fa0 2024-04-25T08:52:52.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.082+0000 7fba52800640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba30066f60 0x7fba30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.082+0000 7fba52800640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba30066f60 0x7fba30069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fba440079c0 tx=0x7fba44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.086+0000 7fba4aa00640 1 -- 172.21.15.46:0/960841176 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba5414b650 con 0x7fba54143fa0 2024-04-25T08:52:52.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.258+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fba5415aae0 con 0x7fba54143fa0 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba4aa00640 1 -- 172.21.15.46:0/960841176 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v122) v1 ==== 80+0+118 (secure 0 0 0) 0x7fba4c016600 con 0x7fba54143fa0 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/51204523 2024-04-26T08:52:42.856639+0000 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6833/51204523 2024-04-26T08:52:42.856639+0000 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-04-25T08:52:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba30066f60 msgr2=0x7fba30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba30066f60 0x7fba30069420 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fba440079c0 tx=0x7fba44008040 comp rx=0 tx=0).stop 2024-04-25T08:52:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba54143fa0 msgr2=0x7fba541496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba54143fa0 0x7fba541496a0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fba4c01a640 tx=0x7fba4c0197b0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 shutdown_connections 2024-04-25T08:52:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fba30066f60 0x7fba30069420 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fba54143fa0 0x7fba541496a0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba540cec00 0x7fba54149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 --2- 172.21.15.46:0/960841176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fba5406e970 0x7fba5413b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 >> 172.21.15.46:0/960841176 conn(0x7fba54066fd0 msgr2=0x7fba540b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 shutdown_connections 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.262+0000 7fba59400640 1 -- 172.21.15.46:0/960841176 wait complete. 2024-04-25T08:52:52.273 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:52:52.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:52.274 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:52:52.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:52:52.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:52.300 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:52:52.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:52.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa617e00640 1 Processor -- start 2024-04-25T08:52:52.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa617e00640 1 -- start start 2024-04-25T08:52:52.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa617e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6100afc50 0x7fa6100b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa617e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61006fb90 0x7fa610070010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa617e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa610161640 con 0x7fa61015de90 2024-04-25T08:52:52.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa617e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61015e620 con 0x7fa6100afc50 2024-04-25T08:52:52.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa617e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61015e900 con 0x7fa61006fb90 2024-04-25T08:52:52.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa615e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61006fb90 0x7fa610070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa615400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6100afc50 0x7fa6100b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa614a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa61015de90 0x7fa610161b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:37944/0 (socket says 172.21.15.46:37944) 2024-04-25T08:52:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa614a00640 1 -- 172.21.15.46:0/2377886436 learned_addr learned my addr 172.21.15.46:0/2377886436 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa615e00640 1 --2- 172.21.15.46:0/2377886436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61006fb90 0x7fa610070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:36818/0 (socket says 172.21.15.46:36818) 2024-04-25T08:52:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa60fe00640 1 -- 172.21.15.46:0/2377886436 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926403890 0 0) 0x7fa610161640 con 0x7fa61015de90 2024-04-25T08:52:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa60fe00640 1 -- 172.21.15.46:0/2377886436 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5f8003530 con 0x7fa61015de90 2024-04-25T08:52:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa615e00640 1 -- 172.21.15.46:0/2377886436 >> v2:172.21.15.82:3300/0 conn(0x7fa6100afc50 msgr2=0x7fa6100b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa615e00640 1 --2- 172.21.15.46:0/2377886436 >> v2:172.21.15.82:3300/0 conn(0x7fa6100afc50 0x7fa6100b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa615e00640 1 -- 172.21.15.46:0/2377886436 >> v1:172.21.15.82:6789/0 conn(0x7fa61015de90 legacy=0x7fa610161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa615e00640 1 -- 172.21.15.46:0/2377886436 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61015ebe0 con 0x7fa61006fb90 2024-04-25T08:52:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.554+0000 7fa615400640 1 --2- 172.21.15.46:0/2377886436 >> v2:172.21.15.82:3300/0 conn(0x7fa6100afc50 0x7fa6100b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:52.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa615e00640 1 --2- 172.21.15.46:0/2377886436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61006fb90 0x7fa610070010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa608009d90 tx=0x7fa60801f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e002f881c609d1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa60fe00640 1 -- 172.21.15.46:0/2377886436 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080b4070 con 0x7fa61006fb90 2024-04-25T08:52:52.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa60fe00640 1 -- 172.21.15.46:0/2377886436 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6080af600 con 0x7fa61006fb90 2024-04-25T08:52:52.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa60fe00640 1 -- 172.21.15.46:0/2377886436 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080af9d0 con 0x7fa61006fb90 2024-04-25T08:52:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa617e00640 1 -- 172.21.15.46:0/2377886436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61006fb90 msgr2=0x7fa610070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa617e00640 1 --2- 172.21.15.46:0/2377886436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61006fb90 0x7fa610070010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa608009d90 tx=0x7fa60801f150 comp rx=0 tx=0).stop 2024-04-25T08:52:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa617e00640 1 -- 172.21.15.46:0/2377886436 shutdown_connections 2024-04-25T08:52:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa617e00640 1 --2- 172.21.15.46:0/2377886436 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61006fb90 0x7fa610070010 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa617e00640 1 --2- 172.21.15.46:0/2377886436 >> v2:172.21.15.82:3300/0 conn(0x7fa6100afc50 0x7fa6100b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa617e00640 1 -- 172.21.15.46:0/2377886436 >> 172.21.15.46:0/2377886436 conn(0x7fa61006e3f0 msgr2=0x7fa61006e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa617e00640 1 -- 172.21.15.46:0/2377886436 shutdown_connections 2024-04-25T08:52:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.558+0000 7fa617e00640 1 -- 172.21.15.46:0/2377886436 wait complete. 2024-04-25T08:52:52.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 Processor -- start 2024-04-25T08:52:52.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 -- start start 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61006fb90 0x7fa6101430a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6100afc50 0x7fa6100cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61015de90 0x7fa6101e72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa610162250 con 0x7fa61006fb90 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100b0030 con 0x7fa6100afc50 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6101623d0 con 0x7fa61015de90 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61006fb90 0x7fa6101430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61006fb90 0x7fa6101430a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46028/0 (socket says 172.21.15.46:46028) 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615400640 1 -- 172.21.15.46:0/82688323 learned_addr learned my addr 172.21.15.46:0/82688323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615e00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61015de90 0x7fa6101e72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa614a00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6100afc50 0x7fa6100cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615400640 1 -- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61015de90 msgr2=0x7fa6101e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615400640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61015de90 0x7fa6101e72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615400640 1 -- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6100afc50 msgr2=0x7fa6100cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615400640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6100afc50 0x7fa6100cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615400640 1 -- 172.21.15.46:0/82688323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100ca850 con 0x7fa61006fb90 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa614a00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6100afc50 0x7fa6100cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa615400640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61006fb90 0x7fa6101430a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa60400ebe0 tx=0x7fa60400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa60d600640 1 -- 172.21.15.46:0/82688323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604017070 con 0x7fa61006fb90 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa60d600640 1 -- 172.21.15.46:0/82688323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60400cbd0 con 0x7fa61006fb90 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa60d600640 1 -- 172.21.15.46:0/82688323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604013470 con 0x7fa61006fb90 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6100cab30 con 0x7fa61006fb90 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6100cb0f0 con 0x7fa61006fb90 2024-04-25T08:52:52.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.562+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61006f980 con 0x7fa61006fb90 2024-04-25T08:52:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.566+0000 7fa60d600640 1 -- 172.21.15.46:0/82688323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa60401a070 con 0x7fa61006fb90 2024-04-25T08:52:52.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.566+0000 7fa60d600640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5f0066f90 0x7fa5f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.566+0000 7fa60d600640 1 -- 172.21.15.46:0/82688323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa604085460 con 0x7fa61006fb90 2024-04-25T08:52:52.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.566+0000 7fa614a00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5f0066f90 0x7fa5f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.566+0000 7fa614a00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5f0066f90 0x7fa5f0069450 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa6100d10e0 tx=0x7fa5fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.570+0000 7fa60d600640 1 -- 172.21.15.46:0/82688323 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa604055c30 con 0x7fa61006fb90 2024-04-25T08:52:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa61004f010 con 0x7fa61006fb90 2024-04-25T08:52:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa60d600640 1 -- 172.21.15.46:0/82688323 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4940 (secure 0 0 0) 0x7fa604059cd0 con 0x7fa61006fb90 2024-04-25T08:52:52.765 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:52.765 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:45.880336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":9,"addr":"172.21.15.46:6834/1279640422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1279640422},{"type":"v1","addr":"172.21.15.46:6834","nonce":1279640422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:52:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 105 2024-04-25T08:52:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5f0066f90 msgr2=0x7fa5f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5f0066f90 0x7fa5f0069450 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa6100d10e0 tx=0x7fa5fc009040 comp rx=0 tx=0).stop 2024-04-25T08:52:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61006fb90 msgr2=0x7fa6101430a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61006fb90 0x7fa6101430a0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa60400ebe0 tx=0x7fa60400c520 comp rx=0 tx=0).stop 2024-04-25T08:52:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 shutdown_connections 2024-04-25T08:52:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa5f0066f90 0x7fa5f0069450 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa61015de90 0x7fa6101e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6100afc50 0x7fa6100cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 --2- 172.21.15.46:0/82688323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa61006fb90 0x7fa6101430a0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 >> 172.21.15.46:0/82688323 conn(0x7fa61006e3f0 msgr2=0x7fa61006ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 shutdown_connections 2024-04-25T08:52:52.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:52.774+0000 7fa617e00640 1 -- 172.21.15.46:0/82688323 wait complete. 2024-04-25T08:52:52.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-04-25T08:52:52.955 INFO:teuthology.orchestra.run.smithi046.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-25T08:52:52.956 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-25T08:52:52.956 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000003.00000000 scrub_tag 2024-04-25T08:52:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 Processor -- start 2024-04-25T08:52:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 -- start start 2024-04-25T08:52:53.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc003100 0x558bb4c03ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558bb4c00910 0x558bb4c00cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bb4be6100 0x558bb4be64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb4c029f0 con 0x558bb4be6100 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb4ad09f0 con 0x558bb4c00910 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bb4c02b70 con 0x7fe9bc003100 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bb4be6100 0x558bb4be64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bb4be6100 0x558bb4be64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46060/0 (socket says 172.21.15.46:46060) 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d8800640 1 -- 172.21.15.46:0/2481993387 learned_addr learned my addr 172.21.15.46:0/2481993387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d7e00640 1 --2- 172.21.15.46:0/2481993387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc003100 0x558bb4c03ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d7400640 1 --2- 172.21.15.46:0/2481993387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558bb4c00910 0x558bb4c00cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d8800640 1 -- 172.21.15.46:0/2481993387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc003100 msgr2=0x558bb4c03ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d8800640 1 --2- 172.21.15.46:0/2481993387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc003100 0x558bb4c03ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d8800640 1 -- 172.21.15.46:0/2481993387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558bb4c00910 msgr2=0x558bb4c00cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d8800640 1 --2- 172.21.15.46:0/2481993387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558bb4c00910 0x558bb4c00cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d8800640 1 -- 172.21.15.46:0/2481993387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bb4b5b140 con 0x558bb4be6100 2024-04-25T08:52:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d8800640 1 --2- 172.21.15.46:0/2481993387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bb4be6100 0x558bb4be64e0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x558bb4b31c90 tx=0x7fe9d0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160e894fd09057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d6000640 1 -- 172.21.15.46:0/2481993387 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0018070 con 0x558bb4be6100 2024-04-25T08:52:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d6000640 1 -- 172.21.15.46:0/2481993387 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0003460 con 0x558bb4be6100 2024-04-25T08:52:53.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9d6000640 1 -- 172.21.15.46:0/2481993387 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0003810 con 0x558bb4be6100 2024-04-25T08:52:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 -- 172.21.15.46:0/2481993387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bb4be6100 msgr2=0x558bb4be64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 --2- 172.21.15.46:0/2481993387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bb4be6100 0x558bb4be64e0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x558bb4b31c90 tx=0x7fe9d0006430 comp rx=0 tx=0).stop 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 -- 172.21.15.46:0/2481993387 shutdown_connections 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 --2- 172.21.15.46:0/2481993387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558bb4be6100 0x558bb4be64e0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 --2- 172.21.15.46:0/2481993387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558bb4c00910 0x558bb4c00cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 --2- 172.21.15.46:0/2481993387 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe9bc003100 0x558bb4c03ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 -- 172.21.15.46:0/2481993387 >> 172.21.15.46:0/2481993387 conn(0x558bb4b2a480 msgr2=0x558bb4c03140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 -- 172.21.15.46:0/2481993387 shutdown_connections 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 -- 172.21.15.46:0/2481993387 wait complete. 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.014+0000 7fe9dac67080 1 Processor -- start 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9dac67080 1 -- start start 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9dac67080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558bb4be6100 0x558bb4b59080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9dac67080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558bb4c00910 0x558bb4bfe880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9dac67080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9bc003100 0x558bb4bfedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9dac67080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb4b5b680 con 0x7fe9bc003100 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9dac67080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bb4b5b3e0 con 0x558bb4c00910 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9dac67080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bb4b5b920 con 0x558bb4be6100 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9bc003100 0x558bb4bfedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9bc003100 0x558bb4bfedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46072/0 (socket says 172.21.15.46:46072) 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d8800640 1 -- 172.21.15.46:0/2918142842 learned_addr learned my addr 172.21.15.46:0/2918142842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d7400640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558bb4c00910 0x558bb4bfe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d7e00640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558bb4be6100 0x558bb4b59080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d8800640 1 -- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558bb4be6100 msgr2=0x558bb4b59080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d8800640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x558bb4be6100 0x558bb4b59080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d8800640 1 -- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558bb4c00910 msgr2=0x558bb4bfe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d8800640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558bb4c00910 0x558bb4bfe880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d8800640 1 -- 172.21.15.46:0/2918142842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bb4bff300 con 0x7fe9bc003100 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d8800640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9bc003100 0x558bb4bfedc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0001b60 tx=0x7fe9d0001dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d4c00640 1 -- 172.21.15.46:0/2918142842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0018040 con 0x7fe9bc003100 2024-04-25T08:52:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d4c00640 1 -- 172.21.15.46:0/2918142842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0022070 con 0x7fe9bc003100 2024-04-25T08:52:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9d4c00640 1 -- 172.21.15.46:0/2918142842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d001d810 con 0x7fe9bc003100 2024-04-25T08:52:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558bb4bfb470 con 0x7fe9bc003100 2024-04-25T08:52:53.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.018+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558bb4bfb9b0 con 0x7fe9bc003100 2024-04-25T08:52:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9d4c00640 1 -- 172.21.15.46:0/2918142842 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9d001d9b0 con 0x7fe9bc003100 2024-04-25T08:52:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9d4c00640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b8065940 0x7fe9b8067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9d4c00640 1 -- 172.21.15.46:0/2918142842 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe9d0084e80 con 0x7fe9bc003100 2024-04-25T08:52:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x558bb4b595c0 0x558bb4c9ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:1 18.32 18:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x558bb4bfc0f0 con 0x558bb4b595c0 2024-04-25T08:52:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9d7e00640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b8065940 0x7fe9b8067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9d7400640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x558bb4b595c0 0x558bb4c9ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9d7400640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x558bb4b595c0 0x558bb4c9ec70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9d7e00640 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b8065940 0x7fe9b8067e00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005410 tx=0x7fe9c800c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9d7400640 1 -- 172.21.15.46:0/2918142842 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe9c40a9480 con 0x558bb4b595c0 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:error getting xattr cephfs_data/10000000003.00000000/scrub_tag: (61) No data available 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x558bb4b595c0 msgr2=0x558bb4c9ec70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x558bb4b595c0 0x558bb4c9ec70 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b8065940 msgr2=0x7fe9b8067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe9b8065940 0x7fe9b8067e00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8005410 tx=0x7fe9c800c040 comp rx=0 tx=0).stop 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9bc003100 msgr2=0x558bb4bfedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 --2- 172.21.15.46:0/2918142842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9bc003100 0x558bb4bfedc0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0001b60 tx=0x7fe9d0001dd0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9d8800640 1 -- 172.21.15.46:0/2918142842 reap_dead start 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 shutdown_connections 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 >> 172.21.15.46:0/2918142842 conn(0x558bb4b2a480 msgr2=0x558bb4c03ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 shutdown_connections 2024-04-25T08:52:53.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.022+0000 7fe9dac67080 1 -- 172.21.15.46:0/2918142842 wait complete. 2024-04-25T08:52:53.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:52:53.019 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T08:52:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62ebada080 1 Processor -- start 2024-04-25T08:52:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62ebada080 1 -- start start 2024-04-25T08:52:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62ebada080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564abc149cc0 0x564abc192130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62ebada080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc17b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62ebada080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564abc221450 0x564abc179430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62ebada080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564abc222250 con 0x564abc194450 2024-04-25T08:52:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62ebada080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564abc0f09f0 con 0x564abc221450 2024-04-25T08:52:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62ebada080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564abc2223d0 con 0x564abc149cc0 2024-04-25T08:52:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc17b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc17b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46090/0 (socket says 172.21.15.46:46090) 2024-04-25T08:52:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62e3e00640 1 -- 172.21.15.46:0/3543582204 learned_addr learned my addr 172.21.15.46:0/3543582204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62e3e00640 1 -- 172.21.15.46:0/3543582204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564abc149cc0 msgr2=0x564abc192130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62e3e00640 1 --2- 172.21.15.46:0/3543582204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564abc149cc0 0x564abc192130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62e3e00640 1 -- 172.21.15.46:0/3543582204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564abc221450 msgr2=0x564abc179430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62e3e00640 1 --2- 172.21.15.46:0/3543582204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564abc221450 0x564abc179430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.122+0000 7f62e3e00640 1 -- 172.21.15.46:0/3543582204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564abc1787b0 con 0x564abc194450 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 --2- 172.21.15.46:0/3543582204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc17b8d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f62d400a750 tx=0x7f62d400ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133d6e40ff2d772d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e2a00640 1 -- 172.21.15.46:0/3543582204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d400ae60 con 0x564abc194450 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e2a00640 1 -- 172.21.15.46:0/3543582204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d4008c00 con 0x564abc194450 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e2a00640 1 -- 172.21.15.46:0/3543582204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40133f0 con 0x564abc194450 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 -- 172.21.15.46:0/3543582204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 msgr2=0x564abc17b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 --2- 172.21.15.46:0/3543582204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc17b8d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f62d400a750 tx=0x7f62d400ac10 comp rx=0 tx=0).stop 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 -- 172.21.15.46:0/3543582204 shutdown_connections 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 --2- 172.21.15.46:0/3543582204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564abc221450 0x564abc179430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 --2- 172.21.15.46:0/3543582204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc17b8d0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 --2- 172.21.15.46:0/3543582204 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564abc149cc0 0x564abc192130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 -- 172.21.15.46:0/3543582204 >> 172.21.15.46:0/3543582204 conn(0x564abc14a4c0 msgr2=0x564abc192cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 -- 172.21.15.46:0/3543582204 shutdown_connections 2024-04-25T08:52:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 -- 172.21.15.46:0/3543582204 wait complete. 2024-04-25T08:52:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 Processor -- start 2024-04-25T08:52:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 -- start start 2024-04-25T08:52:53.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564abc149cc0 0x564abc21f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc177e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564abc221450 0x564abc2b5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564abc178cf0 con 0x564abc194450 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564abc178a50 con 0x564abc149cc0 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62ebada080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x564abc192670 con 0x564abc221450 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc177e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc177e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46102/0 (socket says 172.21.15.46:46102) 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 -- 172.21.15.46:0/1527268886 learned_addr learned my addr 172.21.15.46:0/1527268886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e9600640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564abc221450 0x564abc2b5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e8c00640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564abc149cc0 0x564abc21f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 -- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564abc221450 msgr2=0x564abc2b5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x564abc221450 0x564abc2b5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 -- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564abc149cc0 msgr2=0x564abc21f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564abc149cc0 0x564abc21f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 -- 172.21.15.46:0/1527268886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564abc21f590 con 0x564abc194450 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e8c00640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564abc149cc0 0x564abc21f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.126+0000 7f62e3e00640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc177e00 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f62d4013f20 tx=0x7f62d4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e1600640 1 -- 172.21.15.46:0/1527268886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4022070 con 0x564abc194450 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e1600640 1 -- 172.21.15.46:0/1527268886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d401c630 con 0x564abc194450 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e1600640 1 -- 172.21.15.46:0/1527268886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d401ca20 con 0x564abc194450 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564abc21c750 con 0x564abc194450 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564abc205510 con 0x564abc194450 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e1600640 1 -- 172.21.15.46:0/1527268886 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62d4025070 con 0x564abc194450 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e1600640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f62c8065970 0x7f62c8067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e1600640 1 -- 172.21.15.46:0/1527268886 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f62d4010020 con 0x564abc194450 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62ebada080 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x564abc178340 0x564abc205ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x564abc2b5b70 con 0x564abc178340 2024-04-25T08:52:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e8c00640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f62c8065970 0x7f62c8067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e9600640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x564abc178340 0x564abc205ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e8c00640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f62c8065970 0x7f62c8067e30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00d330 tx=0x7f62dc001520 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.130+0000 7f62e9600640 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x564abc178340 0x564abc205ec0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62e9600640 1 -- 172.21.15.46:0/1527268886 <== osd.2 v2:172.21.15.46:6808/3102391403 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f62e40aa490 con 0x564abc178340 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x564abc178340 msgr2=0x564abc205ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x564abc178340 0x564abc205ec0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f62c8065970 msgr2=0x7f62c8067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f62c8065970 0x7f62c8067e30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00d330 tx=0x7f62dc001520 comp rx=0 tx=0).stop 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 msgr2=0x564abc177e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 --2- 172.21.15.46:0/1527268886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564abc194450 0x564abc177e00 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f62d4013f20 tx=0x7f62d4003030 comp rx=0 tx=0).stop 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62e9600640 1 -- 172.21.15.46:0/1527268886 reap_dead start 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 shutdown_connections 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 >> 172.21.15.46:0/1527268886 conn(0x564abc14a4c0 msgr2=0x564abc1957a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 shutdown_connections 2024-04-25T08:52:53.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.134+0000 7f62ebada080 1 -- 172.21.15.46:0/1527268886 wait complete. 2024-04-25T08:52:53.130 INFO:teuthology.orchestra.run.smithi046.stdout:mytag123 2024-04-25T08:52:53.130 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-25T08:52:53.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb944c48080 1 Processor -- start 2024-04-25T08:52:53.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb944c48080 1 -- start start 2024-04-25T08:52:53.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb944c48080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb29f00 0x556e9bb26e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb944c48080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e9bb2a980 0x556e9bbca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb944c48080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2ad60 0x556e9bb3d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb944c48080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9bbcf6f0 con 0x556e9bb2ad60 2024-04-25T08:52:53.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb944c48080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9baa19f0 con 0x556e9bb2a980 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb944c48080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e9bbb3680 con 0x556e9bb29f00 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2ad60 0x556e9bb3d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2ad60 0x556e9bb3d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46106/0 (socket says 172.21.15.46:46106) 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941c00640 1 -- 172.21.15.46:0/326517350 learned_addr learned my addr 172.21.15.46:0/326517350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941200640 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb29f00 0x556e9bb26e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb93be00640 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e9bb2a980 0x556e9bbca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941c00640 1 -- 172.21.15.46:0/326517350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb29f00 msgr2=0x556e9bb26e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941c00640 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb29f00 0x556e9bb26e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941c00640 1 -- 172.21.15.46:0/326517350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e9bb2a980 msgr2=0x556e9bbca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941c00640 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e9bb2a980 0x556e9bbca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941c00640 1 -- 172.21.15.46:0/326517350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e9bbca4e0 con 0x556e9bb2ad60 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941200640 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb29f00 0x556e9bb26e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.226+0000 7fb941c00640 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2ad60 0x556e9bb3d850 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0ae040 tx=0x7fb93c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9466260b2b33057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93aa00640 1 -- 172.21.15.46:0/326517350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0b9070 con 0x556e9bb2ad60 2024-04-25T08:52:53.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93aa00640 1 -- 172.21.15.46:0/326517350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c0ab5f0 con 0x556e9bb2ad60 2024-04-25T08:52:53.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93aa00640 1 -- 172.21.15.46:0/326517350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0b0040 con 0x556e9bb2ad60 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- 172.21.15.46:0/326517350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2ad60 msgr2=0x556e9bb3d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2ad60 0x556e9bb3d850 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0ae040 tx=0x7fb93c0033b0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- 172.21.15.46:0/326517350 shutdown_connections 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2ad60 0x556e9bb3d850 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e9bb2a980 0x556e9bbca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 --2- 172.21.15.46:0/326517350 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb29f00 0x556e9bb26e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- 172.21.15.46:0/326517350 >> 172.21.15.46:0/326517350 conn(0x556e9bafafa0 msgr2=0x556e9bafd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- 172.21.15.46:0/326517350 shutdown_connections 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- 172.21.15.46:0/326517350 wait complete. 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 Processor -- start 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- start start 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e9bb29f00 0x556e9bb42c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2a980 0x556e9bb24670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb2ad60 0x556e9bb24bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9bbce590 con 0x556e9bb2a980 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9bbb2340 con 0x556e9bb29f00 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x556e9bbcdd30 con 0x556e9bb2ad60 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2a980 0x556e9bb24670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2a980 0x556e9bb24670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46120/0 (socket says 172.21.15.46:46120) 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93be00640 1 -- 172.21.15.46:0/4016962498 learned_addr learned my addr 172.21.15.46:0/4016962498 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb941c00640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb2ad60 0x556e9bb24bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb941200640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e9bb29f00 0x556e9bb42c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93be00640 1 -- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb2ad60 msgr2=0x556e9bb24bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93be00640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x556e9bb2ad60 0x556e9bb24bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93be00640 1 -- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e9bb29f00 msgr2=0x556e9bb42c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93be00640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556e9bb29f00 0x556e9bb42c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93be00640 1 -- 172.21.15.46:0/4016962498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e9bb3fea0 con 0x556e9bb2a980 2024-04-25T08:52:53.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb93be00640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2a980 0x556e9bb24670 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fb92c008ab0 tx=0x7fb92c008f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb939600640 1 -- 172.21.15.46:0/4016962498 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c019070 con 0x556e9bb2a980 2024-04-25T08:52:53.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb939600640 1 -- 172.21.15.46:0/4016962498 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c010c60 con 0x556e9bb2a980 2024-04-25T08:52:53.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556e9bb40140 con 0x556e9bb2a980 2024-04-25T08:52:53.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.230+0000 7fb939600640 1 -- 172.21.15.46:0/4016962498 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c015450 con 0x556e9bb2a980 2024-04-25T08:52:53.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e9bb40680 con 0x556e9bb2a980 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb939600640 1 -- 172.21.15.46:0/4016962498 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb92c01c070 con 0x556e9bb2a980 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb939600640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb920065970 0x7fb920067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb939600640 1 -- 172.21.15.46:0/4016962498 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb92c084ba0 con 0x556e9bb2a980 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb944c48080 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556e9bb43150 0x556e9bb3ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x556e9bc6a750 con 0x556e9bb43150 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb941200640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb920065970 0x7fb920067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb941c00640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556e9bb43150 0x556e9bb3ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb941c00640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556e9bb43150 0x556e9bb3ec90 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.234+0000 7fb941200640 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb920065970 0x7fb920067e30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb934000f70 tx=0x7fb934001600 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb941c00640 1 -- 172.21.15.46:0/4016962498 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7fb93c0ab820 con 0x556e9bb43150 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556e9bb43150 msgr2=0x556e9bb3ec90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x556e9bb43150 0x556e9bb3ec90 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb920065970 msgr2=0x7fb920067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fb920065970 0x7fb920067e30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb934000f70 tx=0x7fb934001600 comp rx=0 tx=0).stop 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2a980 msgr2=0x556e9bb24670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 --2- 172.21.15.46:0/4016962498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e9bb2a980 0x556e9bb24670 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fb92c008ab0 tx=0x7fb92c008f70 comp rx=0 tx=0).stop 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb941c00640 1 -- 172.21.15.46:0/4016962498 reap_dead start 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 shutdown_connections 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 >> 172.21.15.46:0/4016962498 conn(0x556e9bafafa0 msgr2=0x556e9bbb24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 shutdown_connections 2024-04-25T08:52:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.238+0000 7fb944c48080 1 -- 172.21.15.46:0/4016962498 wait complete. 2024-04-25T08:52:53.232 INFO:teuthology.orchestra.run.smithi046.stdout:mytag123 2024-04-25T08:52:53.232 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000006.00000000 scrub_tag 2024-04-25T08:52:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.286+0000 7fe47404d080 1 Processor -- start 2024-04-25T08:52:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.286+0000 7fe47404d080 1 -- start start 2024-04-25T08:52:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.286+0000 7fe47404d080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f7ba3a0 0x55877f847570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.286+0000 7fe47404d080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f8483d0 0x55877f8469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.286+0000 7fe47404d080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8487b0 0x55877f79c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.286+0000 7fe47404d080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877f7b6130 con 0x55877f8487b0 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.286+0000 7fe47404d080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877f7179f0 con 0x55877f7ba3a0 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.286+0000 7fe47404d080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55877f7b62b0 con 0x55877f8483d0 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe471200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8487b0 0x55877f79c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe470800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f7ba3a0 0x55877f847570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe471200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8487b0 0x55877f79c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46142/0 (socket says 172.21.15.46:46142) 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe471200640 1 -- 172.21.15.46:0/3522614089 learned_addr learned my addr 172.21.15.46:0/3522614089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe46fe00640 1 --2- 172.21.15.46:0/3522614089 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f8483d0 0x55877f8469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe470800640 1 -- 172.21.15.46:0/3522614089 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f8483d0 msgr2=0x55877f8469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe470800640 1 --2- 172.21.15.46:0/3522614089 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f8483d0 0x55877f8469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe470800640 1 -- 172.21.15.46:0/3522614089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8487b0 msgr2=0x55877f79c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe470800640 1 --2- 172.21.15.46:0/3522614089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8487b0 0x55877f79c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe470800640 1 -- 172.21.15.46:0/3522614089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55877f79f6c0 con 0x55877f7ba3a0 2024-04-25T08:52:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe471200640 1 --2- 172.21.15.46:0/3522614089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8487b0 0x55877f79c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe470800640 1 --2- 172.21.15.46:0/3522614089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f7ba3a0 0x55877f847570 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe464009890 tx=0x7fe464005a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=295aa1fd5518fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe46ea00640 1 -- 172.21.15.46:0/3522614089 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464019070 con 0x55877f7ba3a0 2024-04-25T08:52:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe46ea00640 1 -- 172.21.15.46:0/3522614089 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe464008da0 con 0x55877f7ba3a0 2024-04-25T08:52:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe46ea00640 1 -- 172.21.15.46:0/3522614089 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4640146a0 con 0x55877f7ba3a0 2024-04-25T08:52:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 -- 172.21.15.46:0/3522614089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f7ba3a0 msgr2=0x55877f847570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 --2- 172.21.15.46:0/3522614089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f7ba3a0 0x55877f847570 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe464009890 tx=0x7fe464005a10 comp rx=0 tx=0).stop 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 -- 172.21.15.46:0/3522614089 shutdown_connections 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 --2- 172.21.15.46:0/3522614089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8487b0 0x55877f79c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 --2- 172.21.15.46:0/3522614089 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f8483d0 0x55877f8469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 --2- 172.21.15.46:0/3522614089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f7ba3a0 0x55877f847570 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 -- 172.21.15.46:0/3522614089 >> 172.21.15.46:0/3522614089 conn(0x55877f770fa0 msgr2=0x55877f76f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 -- 172.21.15.46:0/3522614089 shutdown_connections 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 -- 172.21.15.46:0/3522614089 wait complete. 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 Processor -- start 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 -- start start 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f7ba3a0 0x55877f8288d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8483d0 0x55877f843300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f8487b0 0x55877f843840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877f79fc00 con 0x55877f8483d0 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55877f79f960 con 0x55877f8487b0 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.290+0000 7fe47404d080 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55877f79fea0 con 0x55877f7ba3a0 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe46fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8483d0 0x55877f843300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe471200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f8487b0 0x55877f843840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe470800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f7ba3a0 0x55877f8288d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe46fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8483d0 0x55877f843300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46158/0 (socket says 172.21.15.46:46158) 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe46fe00640 1 -- 172.21.15.46:0/3479663503 learned_addr learned my addr 172.21.15.46:0/3479663503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe470800640 1 -- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f8487b0 msgr2=0x55877f843840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe470800640 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55877f8487b0 0x55877f843840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe470800640 1 -- 172.21.15.46:0/3479663503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8483d0 msgr2=0x55877f843300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe470800640 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8483d0 0x55877f843300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe470800640 1 -- 172.21.15.46:0/3479663503 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55877f82bd50 con 0x55877f7ba3a0 2024-04-25T08:52:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe46fe00640 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55877f8483d0 0x55877f843300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe470800640 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f7ba3a0 0x55877f8288d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe464016000 tx=0x7fe464003030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe46d600640 1 -- 172.21.15.46:0/3479663503 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464024020 con 0x55877f7ba3a0 2024-04-25T08:52:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe46d600640 1 -- 172.21.15.46:0/3479663503 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46401a040 con 0x55877f7ba3a0 2024-04-25T08:52:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe46d600640 1 -- 172.21.15.46:0/3479663503 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464019070 con 0x55877f7ba3a0 2024-04-25T08:52:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55877f829170 con 0x55877f7ba3a0 2024-04-25T08:52:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55877f8296b0 con 0x55877f7ba3a0 2024-04-25T08:52:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.294+0000 7fe46d600640 1 -- 172.21.15.46:0/3479663503 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe464027020 con 0x55877f7ba3a0 2024-04-25T08:52:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe46d600640 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe450065970 0x7fe450067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe46d600640 1 -- 172.21.15.46:0/3479663503 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe464085c60 con 0x55877f7ba3a0 2024-04-25T08:52:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55877f829b50 0x55877f829f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:1 18.25 18:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55877f82a470 con 0x55877f829b50 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe46fe00640 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe450065970 0x7fe450067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe471200640 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55877f829b50 0x55877f829f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe46fe00640 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe450065970 0x7fe450067e30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe45c019010 tx=0x7fe45c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe471200640 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55877f829b50 0x55877f829f30 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe471200640 1 -- 172.21.15.46:0/3479663503 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (12) out=12b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7fe468006040 con 0x55877f829b50 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55877f829b50 msgr2=0x55877f829f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55877f829b50 0x55877f829f30 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe450065970 msgr2=0x7fe450067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe450065970 0x7fe450067e30 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe45c019010 tx=0x7fe45c0a6000 comp rx=0 tx=0).stop 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f7ba3a0 msgr2=0x55877f8288d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 --2- 172.21.15.46:0/3479663503 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55877f7ba3a0 0x55877f8288d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe464016000 tx=0x7fe464003030 comp rx=0 tx=0).stop 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe471200640 1 -- 172.21.15.46:0/3479663503 reap_dead start 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 shutdown_connections 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 >> 172.21.15.46:0/3479663503 conn(0x55877f770fa0 msgr2=0x55877f773220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 shutdown_connections 2024-04-25T08:52:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.298+0000 7fe47404d080 1 -- 172.21.15.46:0/3479663503 wait complete. 2024-04-25T08:52:53.294 INFO:teuthology.orchestra.run.smithi046.stdout:mytag123 2024-04-25T08:52:53.295 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:52:53.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T08:52:53.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.486+0000 7f26e9000640 1 Processor -- start 2024-04-25T08:52:53.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.486+0000 7f26e9000640 1 -- start start 2024-04-25T08:52:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.486+0000 7f26e9000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f26e40b1340 0x7f26e406b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.486+0000 7f26e9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e415c080 con 0x7f26e40b2b10 2024-04-25T08:52:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.486+0000 7f26e9000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e415c360 con 0x7f26e40b1720 2024-04-25T08:52:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.486+0000 7f26e9000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e40cbee0 con 0x7f26e40b1340 2024-04-25T08:52:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.486+0000 7f26daa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26e40b2b10 0x7f26e415b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45174/0 (socket says 172.21.15.46:45174) 2024-04-25T08:52:53.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.486+0000 7f26daa00640 1 -- 172.21.15.46:0/765485016 learned_addr learned my addr 172.21.15.46:0/765485016 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e3400640 1 --2- 172.21.15.46:0/765485016 >> v2:172.21.15.82:3300/0 conn(0x7f26e40b1340 0x7f26e406b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e2a00640 1 -- 172.21.15.46:0/765485016 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664095421 0 0) 0x7f26e415c080 con 0x7f26e40b2b10 2024-04-25T08:52:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e2a00640 1 -- 172.21.15.46:0/765485016 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c8003530 con 0x7f26e40b2b10 2024-04-25T08:52:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e2a00640 1 -- 172.21.15.46:0/765485016 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 796743101 0 0) 0x7f26c8003530 con 0x7f26e40b2b10 2024-04-25T08:52:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e3400640 1 -- 172.21.15.46:0/765485016 >> v1:172.21.15.82:6789/0 conn(0x7f26e40b1720 legacy=0x7f26e40cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e3400640 1 -- 172.21.15.46:0/765485016 >> v1:172.21.15.46:6789/0 conn(0x7f26e40b2b10 legacy=0x7f26e415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e3400640 1 -- 172.21.15.46:0/765485016 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e415c640 con 0x7f26e40b1340 2024-04-25T08:52:53.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e3400640 1 --2- 172.21.15.46:0/765485016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e40b1340 0x7f26e406b670 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009d90 tx=0x7f26d401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a6eb5a84b2bf33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e2a00640 1 -- 172.21.15.46:0/765485016 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc001680 con 0x7f26e40b1340 2024-04-25T08:52:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e2a00640 1 -- 172.21.15.46:0/765485016 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d40a64e0 con 0x7f26e40b1340 2024-04-25T08:52:53.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.490+0000 7f26e2a00640 1 -- 172.21.15.46:0/765485016 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d40a6890 con 0x7f26e40b1340 2024-04-25T08:52:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 -- 172.21.15.46:0/765485016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e40b1340 msgr2=0x7f26e406b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 --2- 172.21.15.46:0/765485016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e40b1340 0x7f26e406b670 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009d90 tx=0x7f26d401c890 comp rx=0 tx=0).stop 2024-04-25T08:52:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 -- 172.21.15.46:0/765485016 shutdown_connections 2024-04-25T08:52:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 --2- 172.21.15.46:0/765485016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e40b1340 0x7f26e406b670 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 -- 172.21.15.46:0/765485016 >> 172.21.15.46:0/765485016 conn(0x7f26e4099a00 msgr2=0x7f26e40b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 -- 172.21.15.46:0/765485016 shutdown_connections 2024-04-25T08:52:53.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 -- 172.21.15.46:0/765485016 wait complete. 2024-04-25T08:52:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 Processor -- start 2024-04-25T08:52:53.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 -- start start 2024-04-25T08:52:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40b1340 0x7f26e40c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e40b1720 0x7f26e4150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e40b2b10 0x7f26e40c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40cd600 con 0x7f26e40b1340 2024-04-25T08:52:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40cd480 con 0x7f26e40b1720 2024-04-25T08:52:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e9000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e41692c0 con 0x7f26e40b2b10 2024-04-25T08:52:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40b1340 0x7f26e40c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40b1340 0x7f26e40c61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46164/0 (socket says 172.21.15.46:46164) 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e3400640 1 -- 172.21.15.46:0/3519595835 learned_addr learned my addr 172.21.15.46:0/3519595835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26daa00640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e40b1720 0x7f26e4150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e3e00640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e40b2b10 0x7f26e40c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e3400640 1 -- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e40b2b10 msgr2=0x7f26e40c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e3400640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e40b2b10 0x7f26e40c3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e3400640 1 -- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e40b1720 msgr2=0x7f26e4150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e3400640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e40b1720 0x7f26e4150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.494+0000 7f26e3400640 1 -- 172.21.15.46:0/3519595835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40c6840 con 0x7f26e40b1340 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26e3400640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40b1340 0x7f26e40c61b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f26d40194a0 tx=0x7f26d4019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26dbe00640 1 -- 172.21.15.46:0/3519595835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4009450 con 0x7f26e40b1340 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26dbe00640 1 -- 172.21.15.46:0/3519595835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d40c1070 con 0x7f26e40b1340 2024-04-25T08:52:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26dbe00640 1 -- 172.21.15.46:0/3519595835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d40b8a40 con 0x7f26e40b1340 2024-04-25T08:52:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26e9000640 1 -- 172.21.15.46:0/3519595835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e40c7600 con 0x7f26e40b1340 2024-04-25T08:52:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26e9000640 1 -- 172.21.15.46:0/3519595835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e40c7bc0 con 0x7f26e40b1340 2024-04-25T08:52:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26dbe00640 1 -- 172.21.15.46:0/3519595835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26d40b8be0 con 0x7f26e40b1340 2024-04-25T08:52:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26dbe00640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c0066f60 0x7f26c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26dbe00640 1 -- 172.21.15.46:0/3519595835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f26d4120420 con 0x7f26e40b1340 2024-04-25T08:52:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.498+0000 7f26daa00640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c0066f60 0x7f26c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 Processor -- start 2024-04-25T08:52:53.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- start start 2024-04-25T08:52:53.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c0066f60 0x7f26c0069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f26e40c6aa0 tx=0x7f26cc001d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 0x55d79e0fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79defeff0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79de87150 con 0x55d79e0b20a0 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d79defb090 con 0x55d79e0fb9d0 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e0b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46174/0 (socket says 172.21.15.46:46174) 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 -- 172.21.15.46:0/1167242424 learned_addr learned my addr 172.21.15.46:0/1167242424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26e3400640 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 0x55d79e0fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26e3e00640 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 -- 172.21.15.46:0/1167242424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 msgr2=0x55d79e0fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 0x55d79e0fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 -- 172.21.15.46:0/1167242424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 msgr2=0x55d79e0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e0b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 -- 172.21.15.46:0/1167242424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d79e0950d0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26e3e00640 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e0b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26daa00640 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e0b1b60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f26e4071570 tx=0x7f26cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a5b38b74ca1b09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26d3e00640 1 -- 172.21.15.46:0/1167242424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc025070 con 0x55d79e0fc2f0 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26d3e00640 1 -- 172.21.15.46:0/1167242424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc01b050 con 0x55d79e0fc2f0 2024-04-25T08:52:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26d3e00640 1 -- 172.21.15.46:0/1167242424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc014de0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- 172.21.15.46:0/1167242424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 msgr2=0x55d79e0b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e0b1b60 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f26e4071570 tx=0x7f26cc012000 comp rx=0 tx=0).stop 2024-04-25T08:52:53.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- 172.21.15.46:0/1167242424 shutdown_connections 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e0b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e0b1b60 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 --2- 172.21.15.46:0/1167242424 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 0x55d79e0fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- 172.21.15.46:0/1167242424 >> 172.21.15.46:0/1167242424 conn(0x55d79e0ae020 msgr2=0x55d79e0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- 172.21.15.46:0/1167242424 shutdown_connections 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- 172.21.15.46:0/1167242424 wait complete. 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.502+0000 7f26eda4a000 1 -- start start 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 Processor -- start 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e0e07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fb9d0 0x55d79e0e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fc2f0 0x55d79e0e1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79dec3250 con 0x55d79e0fb9d0 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79deb9690 con 0x55d79e0b20a0 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d79de87a40 con 0x55d79e0fc2f0 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 10 client.0 _ll_register_callbacks cb 0x55d79e09ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fb9d0 0x55d79e0e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fb9d0 0x55d79e0e0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46184/0 (socket says 172.21.15.46:46184) 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26daa00640 1 -- 172.21.15.46:0/2385051740 learned_addr learned my addr 172.21.15.46:0/2385051740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26e3e00640 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fc2f0 0x55d79e0e1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26e3400640 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e0e07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26daa00640 1 -- 172.21.15.46:0/2385051740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fc2f0 msgr2=0x55d79e0e1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26daa00640 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fc2f0 0x55d79e0e1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26daa00640 1 -- 172.21.15.46:0/2385051740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 msgr2=0x55d79e0e07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26daa00640 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e0e07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26daa00640 1 -- 172.21.15.46:0/2385051740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d79dfac1c0 con 0x55d79e0fb9d0 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26daa00640 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fb9d0 0x55d79e0e0d30 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x55d79debb3d0 tx=0x7f26cc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6d610350459392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26d0c00640 1 -- 172.21.15.46:0/2385051740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc030020 con 0x55d79e0fb9d0 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26d0c00640 1 -- 172.21.15.46:0/2385051740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f26cc020b10 con 0x55d79e0fb9d0 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 -- 172.21.15.46:0/2385051740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26cc01b050 con 0x55d79e0fb9d0 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26d0c00640 1 -- 172.21.15.46:0/2385051740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26cc025030 con 0x55d79e0fb9d0 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 10 client.9375 fetch_fsmap learned FSMap version 105 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 -- 172.21.15.46:0/2385051740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55d79dfa8170 con 0x55d79e0fb9d0 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26d0c00640 1 -- 172.21.15.46:0/2385051740 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc01b050 con 0x55d79e0fb9d0 2024-04-25T08:52:53.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26d0c00640 1 -- 172.21.15.46:0/2385051740 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f26cc03d020 con 0x55d79e0fb9d0 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 10 client.9375 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 10 client.9375 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 20 client.9375 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x55d79e10bc80 0x55d79e10e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 4 client.9375 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26eda4a000 1 -- 172.21.15.46:0/2385051740 --> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d79de9ac60 con 0x55d79e10bc80 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26e3400640 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x55d79e10bc80 0x55d79e10e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26e3400640 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x55d79e10bc80 0x55d79e10e070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9be16929682dd46 server_cookie=235d1047583bb118 in_seq=0 out_seq=0 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.506+0000 7f26d0c00640 10 client.9375 ms_handle_connect on v2:172.21.15.46:6832/1279640422 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26d0c00640 1 -- 172.21.15.46:0/2385051740 <== mds.0 v2:172.21.15.46:6832/1279640422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d79de9ac60 con 0x55d79e10bc80 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26d0c00640 10 client.9375 handle_command_reply: tid=0 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 client.9375 shutdown 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 -- 172.21.15.46:0/2385051740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fb9d0 msgr2=0x55d79e0e0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fb9d0 0x55d79e0e0d30 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x55d79debb3d0 tx=0x7f26cc005800 comp rx=0 tx=0).stop 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 -- 172.21.15.46:0/2385051740 shutdown_connections 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x55d79e10bc80 0x55d79e10e070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fc2f0 0x55d79e0e1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fb9d0 0x55d79e0e0d30 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2385051740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e0e07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26d0c00640 0 client.9375 ms_handle_reset on v2:172.21.15.46:6832/1279640422 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 -- 172.21.15.46:0/2385051740 >> 172.21.15.46:0/2385051740 conn(0x55d79e0ae020 msgr2=0x55d79e0b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 -- 172.21.15.46:0/2385051740 shutdown_connections 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 1 -- 172.21.15.46:0/2385051740 wait complete. 2024-04-25T08:52:53.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.510+0000 7f26eda4a000 20 client.9375 trim_cache size 0 max 16384 2024-04-25T08:52:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26eda4a000 1 Processor -- start 2024-04-25T08:52:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26eda4a000 1 -- start start 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0b20a0 0x55d79e157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0fb9d0 0x55d79e158420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e158960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26eda4a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4202de0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26eda4a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4202010 con 0x55d79e0fb9d0 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26eda4a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40c9200 con 0x55d79e0b20a0 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46196/0 (socket says 172.21.15.46:46196) 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26e3e00640 1 -- 172.21.15.46:0/2550584916 learned_addr learned my addr 172.21.15.46:0/2550584916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26e3400640 1 --2- 172.21.15.46:0/2550584916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0b20a0 0x55d79e157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26daa00640 1 --2- 172.21.15.46:0/2550584916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0fb9d0 0x55d79e158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26e3e00640 1 -- 172.21.15.46:0/2550584916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0b20a0 msgr2=0x55d79e157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26e3e00640 1 --2- 172.21.15.46:0/2550584916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0b20a0 0x55d79e157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26e3e00640 1 -- 172.21.15.46:0/2550584916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0fb9d0 msgr2=0x55d79e158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26e3e00640 1 --2- 172.21.15.46:0/2550584916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0fb9d0 0x55d79e158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.522+0000 7f26e3e00640 1 -- 172.21.15.46:0/2550584916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d79df45de0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3e00640 1 --2- 172.21.15.46:0/2550584916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e158960 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f26dc005430 tx=0x7f26dc0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7756c1d7e481895 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26d3400640 1 -- 172.21.15.46:0/2550584916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc007030 con 0x55d79e0fc2f0 2024-04-25T08:52:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26d3400640 1 -- 172.21.15.46:0/2550584916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc0071d0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26d3400640 1 -- 172.21.15.46:0/2550584916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc007580 con 0x55d79e0fc2f0 2024-04-25T08:52:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 -- 172.21.15.46:0/2550584916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 msgr2=0x55d79e158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2550584916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e158960 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f26dc005430 tx=0x7f26dc0039d0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 -- 172.21.15.46:0/2550584916 shutdown_connections 2024-04-25T08:52:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2550584916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e158960 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2550584916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0fb9d0 0x55d79e158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2550584916 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0b20a0 0x55d79e157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 -- 172.21.15.46:0/2550584916 >> 172.21.15.46:0/2550584916 conn(0x55d79e0ae020 msgr2=0x55d79e0fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 -- 172.21.15.46:0/2550584916 shutdown_connections 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 -- 172.21.15.46:0/2550584916 wait complete. 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 -- start start 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 Processor -- start 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e10e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 0x55d79e10e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e10ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d79de99e00 con 0x55d79e0fc2f0 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4201a90 con 0x55d79e0b20a0 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d79defe300 con 0x55d79e0fb9d0 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26eda4a000 10 client.0 _ll_register_callbacks cb 0x55d79e07d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e10ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e10ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46202/0 (socket says 172.21.15.46:46202) 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3e00640 1 -- 172.21.15.46:0/2479908355 learned_addr learned my addr 172.21.15.46:0/2479908355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e10e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26daa00640 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 0x55d79e10e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3e00640 1 -- 172.21.15.46:0/2479908355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 msgr2=0x55d79e10e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3e00640 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 0x55d79e10e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3e00640 1 -- 172.21.15.46:0/2479908355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 msgr2=0x55d79e10e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3e00640 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e10e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.526+0000 7f26e3e00640 1 -- 172.21.15.46:0/2479908355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d79dea08e0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26e3e00640 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e10ee00 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f26dc003ed0 tx=0x7f26dc000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1be9480c7186856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:52:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26d0c00640 1 -- 172.21.15.46:0/2479908355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc006e40 con 0x55d79e0fc2f0 2024-04-25T08:52:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26d0c00640 1 -- 172.21.15.46:0/2479908355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f26dc015470 con 0x55d79e0fc2f0 2024-04-25T08:52:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26eda4a000 1 -- 172.21.15.46:0/2479908355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26dc0071d0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26d0c00640 1 -- 172.21.15.46:0/2479908355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26dc033050 con 0x55d79e0fc2f0 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26eda4a000 10 client.9381 fetch_fsmap learned FSMap version 105 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26eda4a000 1 -- 172.21.15.46:0/2479908355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x55d79df588f0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26d0c00640 1 -- 172.21.15.46:0/2479908355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc0071d0 con 0x55d79e0fc2f0 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26d0c00640 1 -- 172.21.15.46:0/2479908355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 105) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f26dc036020 con 0x55d79e0fc2f0 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26eda4a000 10 client.9381 fetch_fsmap finished waiting for FSMap version 105 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26eda4a000 10 client.9381 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26eda4a000 20 client.9381 populate_metadata read hostname 'smithi046' 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x55d79e10f760 0x55d79e163f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26eda4a000 4 client.9381 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T08:52:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26eda4a000 1 -- 172.21.15.46:0/2479908355 --> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d79df0c530 con 0x55d79e10f760 2024-04-25T08:52:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26e3400640 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x55d79e10f760 0x55d79e163f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26e3400640 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x55d79e10f760 0x55d79e163f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242abfc6d47eab46 server_cookie=3f03b80a9e41981e in_seq=0 out_seq=0 2024-04-25T08:52:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.530+0000 7f26d0c00640 10 client.9381 ms_handle_connect on v2:172.21.15.46:6832/1279640422 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.542+0000 7f26d0c00640 1 -- 172.21.15.46:0/2479908355 <== mds.0 v2:172.21.15.46:6832/1279640422 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d79df0c530 con 0x55d79e10f760 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.542+0000 7f26d0c00640 10 client.9381 handle_command_reply: tid=0 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.542+0000 7f26eda4a000 1 client.9381 shutdown 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 -- 172.21.15.46:0/2479908355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 msgr2=0x55d79e10ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e10ee00 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f26dc003ed0 tx=0x7f26dc000ee0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 -- 172.21.15.46:0/2479908355 shutdown_connections 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.46:6832/1279640422,v1:172.21.15.46:6834/1279640422] conn(0x55d79e10f760 0x55d79e163f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d79e0fc2f0 0x55d79e10ee00 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d79e0fb9d0 0x55d79e10e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26d0c00640 0 client.9381 ms_handle_reset on v2:172.21.15.46:6832/1279640422 2024-04-25T08:52:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 --2- 172.21.15.46:0/2479908355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d79e0b20a0 0x55d79e10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 -- 172.21.15.46:0/2479908355 >> 172.21.15.46:0/2479908355 conn(0x55d79e0ae020 msgr2=0x55d79e0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 -- 172.21.15.46:0/2479908355 shutdown_connections 2024-04-25T08:52:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 1 -- 172.21.15.46:0/2479908355 wait complete. 2024-04-25T08:52:53.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26eda4a000 20 client.9381 trim_cache size 0 max 16384 2024-04-25T08:52:53.537 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:52:53.537 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "", 2024-04-25T08:52:53.537 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-04-25T08:52:53.537 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:52:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 -- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c0066f60 msgr2=0x7f26c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c0066f60 0x7f26c0069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f26e40c6aa0 tx=0x7f26cc001d70 comp rx=0 tx=0).stop 2024-04-25T08:52:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 -- 172.21.15.46:0/3519595835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40b1340 msgr2=0x7f26e40c61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40b1340 0x7f26e40c61b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f26d40194a0 tx=0x7f26d4019630 comp rx=0 tx=0).stop 2024-04-25T08:52:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 -- 172.21.15.46:0/3519595835 shutdown_connections 2024-04-25T08:52:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f26c0066f60 0x7f26c0069420 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f26e40b2b10 0x7f26e40c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26e40b1720 0x7f26e4150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 --2- 172.21.15.46:0/3519595835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26e40b1340 0x7f26e40c61b0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 -- 172.21.15.46:0/3519595835 >> 172.21.15.46:0/3519595835 conn(0x7f26e4099a00 msgr2=0x7f26e406f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 -- 172.21.15.46:0/3519595835 shutdown_connections 2024-04-25T08:52:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.546+0000 7f26e9000640 1 -- 172.21.15.46:0/3519595835 wait complete. 2024-04-25T08:52:53.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:52:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af7800640 1 Processor -- start 2024-04-25T08:52:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af7800640 1 -- start start 2024-04-25T08:52:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af7800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6af01612c0 0x7f6af00caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af7800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6af00af8c0 0x7f6af00cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af7800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af006f9f0 0x7f6af006fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af7800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af00ccf60 con 0x7f6af01612c0 2024-04-25T08:52:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af7800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af00cd1b0 con 0x7f6af00af8c0 2024-04-25T08:52:53.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af7800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af00cd490 con 0x7f6af006f9f0 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6af01612c0 0x7f6af00caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6af01612c0 0x7f6af00caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46212/0 (socket says 172.21.15.46:46212) 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af4e00640 1 -- 172.21.15.46:0/3840773710 learned_addr learned my addr 172.21.15.46:0/3840773710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af5800640 1 --2- 172.21.15.46:0/3840773710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af006f9f0 0x7f6af006fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6ae7e00640 1 --2- 172.21.15.46:0/3840773710 >> v2:172.21.15.82:3300/0 conn(0x7f6af00af8c0 0x7f6af00cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af4e00640 1 -- 172.21.15.46:0/3840773710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af006f9f0 msgr2=0x7f6af006fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af4e00640 1 --2- 172.21.15.46:0/3840773710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af006f9f0 0x7f6af006fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af4e00640 1 -- 172.21.15.46:0/3840773710 >> v2:172.21.15.82:3300/0 conn(0x7f6af00af8c0 msgr2=0x7f6af00cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af4e00640 1 --2- 172.21.15.46:0/3840773710 >> v2:172.21.15.82:3300/0 conn(0x7f6af00af8c0 0x7f6af00cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.706+0000 7f6af4e00640 1 -- 172.21.15.46:0/3840773710 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00cd770 con 0x7f6af01612c0 2024-04-25T08:52:53.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af4e00640 1 --2- 172.21.15.46:0/3840773710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af01612c0 0x7f6af00caba0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f6af006dbf0 tx=0x7f6ae001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b674dd9a8de715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6aefe00640 1 -- 172.21.15.46:0/3840773710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00ab5f0 con 0x7f6af01612c0 2024-04-25T08:52:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6aefe00640 1 -- 172.21.15.46:0/3840773710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae00b0070 con 0x7f6af01612c0 2024-04-25T08:52:53.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6aefe00640 1 -- 172.21.15.46:0/3840773710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00ab9b0 con 0x7f6af01612c0 2024-04-25T08:52:53.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af7800640 1 -- 172.21.15.46:0/3840773710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af01612c0 msgr2=0x7f6af00caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af7800640 1 --2- 172.21.15.46:0/3840773710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af01612c0 0x7f6af00caba0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f6af006dbf0 tx=0x7f6ae001c520 comp rx=0 tx=0).stop 2024-04-25T08:52:53.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af7800640 1 -- 172.21.15.46:0/3840773710 shutdown_connections 2024-04-25T08:52:53.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af7800640 1 --2- 172.21.15.46:0/3840773710 >> v2:172.21.15.82:3300/0 conn(0x7f6af00af8c0 0x7f6af00cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af7800640 1 --2- 172.21.15.46:0/3840773710 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af006f9f0 0x7f6af006fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af7800640 1 --2- 172.21.15.46:0/3840773710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af01612c0 0x7f6af00caba0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af7800640 1 -- 172.21.15.46:0/3840773710 >> 172.21.15.46:0/3840773710 conn(0x7f6af00b2100 msgr2=0x7f6af00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af7800640 1 -- 172.21.15.46:0/3840773710 shutdown_connections 2024-04-25T08:52:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.710+0000 7f6af7800640 1 -- 172.21.15.46:0/3840773710 wait complete. 2024-04-25T08:52:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 Processor -- start 2024-04-25T08:52:53.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 -- start start 2024-04-25T08:52:53.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af006f9f0 0x7f6af00adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af00af8c0 0x7f6af0158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af01612c0 0x7f6af0158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af006bab0 con 0x7f6af006f9f0 2024-04-25T08:52:53.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af006b930 con 0x7f6af01612c0 2024-04-25T08:52:53.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af006e420 con 0x7f6af00af8c0 2024-04-25T08:52:53.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af006f9f0 0x7f6af00adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af006f9f0 0x7f6af00adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46220/0 (socket says 172.21.15.46:46220) 2024-04-25T08:52:53.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af4e00640 1 -- 172.21.15.46:0/764390687 learned_addr learned my addr 172.21.15.46:0/764390687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6ae7e00640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af00af8c0 0x7f6af0158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af5800640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af01612c0 0x7f6af0158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af4e00640 1 -- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af00af8c0 msgr2=0x7f6af0158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af4e00640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af00af8c0 0x7f6af0158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af4e00640 1 -- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af01612c0 msgr2=0x7f6af0158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af4e00640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af01612c0 0x7f6af0158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af4e00640 1 -- 172.21.15.46:0/764390687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af0159520 con 0x7f6af006f9f0 2024-04-25T08:52:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af5800640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af01612c0 0x7f6af0158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af4e00640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af006f9f0 0x7f6af00adc00 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f6ae001abd0 tx=0x7f6ae001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6aed600640 1 -- 172.21.15.46:0/764390687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00be5f0 con 0x7f6af006f9f0 2024-04-25T08:52:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6aed600640 1 -- 172.21.15.46:0/764390687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae00be790 con 0x7f6af006f9f0 2024-04-25T08:52:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6aed600640 1 -- 172.21.15.46:0/764390687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00beb80 con 0x7f6af006f9f0 2024-04-25T08:52:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af0159800 con 0x7f6af006f9f0 2024-04-25T08:52:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.714+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af0159dc0 con 0x7f6af006f9f0 2024-04-25T08:52:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.718+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af009e760 con 0x7f6af006f9f0 2024-04-25T08:52:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.718+0000 7f6aed600640 1 -- 172.21.15.46:0/764390687 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ae00b43f0 con 0x7f6af006f9f0 2024-04-25T08:52:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.718+0000 7f6aed600640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6acc066f60 0x7f6acc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.718+0000 7f6aed600640 1 -- 172.21.15.46:0/764390687 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6ae011fc80 con 0x7f6af006f9f0 2024-04-25T08:52:53.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.718+0000 7f6ae7e00640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6acc066f60 0x7f6acc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.718+0000 7f6ae7e00640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6acc066f60 0x7f6acc069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6af00ca740 tx=0x7f6ad8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.722+0000 7f6aed600640 1 -- 172.21.15.46:0/764390687 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ae00c4060 con 0x7f6af006f9f0 2024-04-25T08:52:53.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.918+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6af004f010 con 0x7f6af006f9f0 2024-04-25T08:52:53.910 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:52:53.918+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:52:53.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6aed600640 1 -- 172.21.15.46:0/764390687 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6ae0016110 con 0x7f6af006f9f0 2024-04-25T08:52:53.912 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:52:53.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6acc066f60 msgr2=0x7f6acc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6acc066f60 0x7f6acc069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6af00ca740 tx=0x7f6ad8008040 comp rx=0 tx=0).stop 2024-04-25T08:52:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af006f9f0 msgr2=0x7f6af00adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af006f9f0 0x7f6af00adc00 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f6ae001abd0 tx=0x7f6ae001ad60 comp rx=0 tx=0).stop 2024-04-25T08:52:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 shutdown_connections 2024-04-25T08:52:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6acc066f60 0x7f6acc069420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6af01612c0 0x7f6af0158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6af00af8c0 0x7f6af0158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 --2- 172.21.15.46:0/764390687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6af006f9f0 0x7f6af00adc00 secure :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f6ae001abd0 tx=0x7f6ae001ad60 comp rx=0 tx=0).stop 2024-04-25T08:52:53.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 >> 172.21.15.46:0/764390687 conn(0x7f6af00b2100 msgr2=0x7f6af00b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 shutdown_connections 2024-04-25T08:52:53.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.922+0000 7f6af7800640 1 -- 172.21.15.46:0/764390687 wait complete. 2024-04-25T08:52:53.922 INFO:tasks.ceph.mds.a.smithi046.stderr: -1765> 2024-04-25T08:52:48.734+0000 7f661e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:52:53.936 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-25T08:52:53.967 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:52:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42dac32400 1 -- start start 2024-04-25T08:52:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42dac32400 1 Processor -- start 2024-04-25T08:52:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42dac32400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645eee31800 0x5645eefebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42dac32400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645eee31400 0x5645eefeb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42dac32400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645eee31000 0x5645eefeb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42dac32400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645efc0c180 con 0x5645eee31800 2024-04-25T08:52:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42dac32400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645eee90900 con 0x5645eee31000 2024-04-25T08:52:53.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42dac32400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645eee90480 con 0x5645eee31400 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645eee31800 0x5645eefebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645eee31800 0x5645eefebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46252/0 (socket says 172.21.15.46:46252) 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7c00640 1 -- 172.21.15.46:0/1806215557 learned_addr learned my addr 172.21.15.46:0/1806215557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d8600640 1 --2- 172.21.15.46:0/1806215557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645eee31000 0x5645eefeb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7200640 1 --2- 172.21.15.46:0/1806215557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645eee31400 0x5645eefeb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7c00640 1 -- 172.21.15.46:0/1806215557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645eee31400 msgr2=0x5645eefeb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7c00640 1 --2- 172.21.15.46:0/1806215557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645eee31400 0x5645eefeb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7c00640 1 -- 172.21.15.46:0/1806215557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645eee31000 msgr2=0x5645eefeb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7c00640 1 --2- 172.21.15.46:0/1806215557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645eee31000 0x5645eefeb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7c00640 1 -- 172.21.15.46:0/1806215557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645eeed2b40 con 0x5645eee31800 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d7c00640 1 --2- 172.21.15.46:0/1806215557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645eee31800 0x5645eefebb80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x5645efc94a20 tx=0x5645eef08000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3863f3f9758dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d4a00640 1 -- 172.21.15.46:0/1806215557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645eee669c0 con 0x5645eee31800 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d4a00640 1 -- 172.21.15.46:0/1806215557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645eee7a1c0 con 0x5645eee31800 2024-04-25T08:52:53.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.994+0000 7f42d4a00640 1 -- 172.21.15.46:0/1806215557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645eee664e0 con 0x5645eee31800 2024-04-25T08:52:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 -- 172.21.15.46:0/1806215557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645eeed2d20 con 0x5645eee31800 2024-04-25T08:52:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 -- 172.21.15.46:0/1806215557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5645eeed2f00 con 0x5645eee31800 2024-04-25T08:52:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 4 waiting for MDS map... 2024-04-25T08:52:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d4a00640 1 -- 172.21.15.46:0/1806215557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x5645efd56200 con 0x5645eee31800 2024-04-25T08:52:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d4a00640 1 -- 172.21.15.46:0/1806215557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x5645eee87180 con 0x5645eee31800 2024-04-25T08:52:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 4 Got MDS map 106 2024-04-25T08:52:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 10 main: JournalTool::main 2024-04-25T08:52:53.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 4 main: JournalTool: connecting to RADOS... 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 Processor -- start 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 -- start start 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 0x5645eefeab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efc0fc00 0x5645eefea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2000 0x5645eefec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645eee90180 con 0x5645efdb2000 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645efc0cd80 con 0x5645efc0f800 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645efc0cf00 con 0x5645efc0fc00 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2000 0x5645eefec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2000 0x5645eefec100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46266/0 (socket says 172.21.15.46:46266) 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efc0fc00 0x5645eefea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7200640 1 -- 172.21.15.46:0/1313034101 learned_addr learned my addr 172.21.15.46:0/1313034101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d8600640 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 0x5645eefeab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7200640 1 -- 172.21.15.46:0/1313034101 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efc0fc00 msgr2=0x5645eefea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7200640 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efc0fc00 0x5645eefea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7200640 1 -- 172.21.15.46:0/1313034101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 msgr2=0x5645eefeab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7200640 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 0x5645eefeab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7200640 1 -- 172.21.15.46:0/1313034101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645eeed21e0 con 0x5645efdb2000 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7c00640 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efc0fc00 0x5645eefea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d8600640 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 0x5645eefeab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d7200640 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2000 0x5645eefec100 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x5645efd628d0 tx=0x5645efd70c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cf46cf3d28082d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d2200640 1 -- 172.21.15.46:0/1313034101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645eee669c0 con 0x5645efdb2000 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d2200640 1 -- 172.21.15.46:0/1313034101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645eee7ac40 con 0x5645efdb2000 2024-04-25T08:52:53.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42d2200640 1 -- 172.21.15.46:0/1313034101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645eee664e0 con 0x5645efdb2000 2024-04-25T08:52:53.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 -- 172.21.15.46:0/1313034101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2000 msgr2=0x5645eefec100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2000 0x5645eefec100 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x5645efd628d0 tx=0x5645efd70c00 comp rx=0 tx=0).stop 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 -- 172.21.15.46:0/1313034101 shutdown_connections 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2000 0x5645eefec100 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efc0fc00 0x5645eefea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 --2- 172.21.15.46:0/1313034101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 0x5645eefeab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:53.998+0000 7f42dac32400 1 -- 172.21.15.46:0/1313034101 >> 172.21.15.46:0/1313034101 conn(0x5645eee30c00 msgr2=0x5645eee5f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 -- 172.21.15.46:0/1313034101 shutdown_connections 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 -- 172.21.15.46:0/1313034101 wait complete. 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 Processor -- start 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 -- start start 2024-04-25T08:52:53.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efc0fc00 0x5645eefeab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 0x5645eefea580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efdb3800 0x5645eefec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645efc0d800 con 0x5645efc0fc00 2024-04-25T08:52:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645efc0d680 con 0x5645efc0f800 2024-04-25T08:52:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645efc0d200 con 0x5645efdb3800 2024-04-25T08:52:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efc0fc00 0x5645eefeab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efc0fc00 0x5645eefeab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46268/0 (socket says 172.21.15.46:46268) 2024-04-25T08:52:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d8600640 1 -- 172.21.15.46:0/2493538172 learned_addr learned my addr 172.21.15.46:0/2493538172 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:53.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d7c00640 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 0x5645eefea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d7200640 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efdb3800 0x5645eefec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d8600640 1 -- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efdb3800 msgr2=0x5645eefec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d8600640 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efdb3800 0x5645eefec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d8600640 1 -- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 msgr2=0x5645eefea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d8600640 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 0x5645eefea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d8600640 1 -- 172.21.15.46:0/2493538172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645eeed3a40 con 0x5645efc0fc00 2024-04-25T08:52:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d8600640 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efc0fc00 0x5645eefeab00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x5645efd635c0 tx=0x5645efd70d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d0400640 1 -- 172.21.15.46:0/2493538172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645eee669c0 con 0x5645efc0fc00 2024-04-25T08:52:54.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d0400640 1 -- 172.21.15.46:0/2493538172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645eee7a540 con 0x5645efc0fc00 2024-04-25T08:52:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42d0400640 1 -- 172.21.15.46:0/2493538172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645eee664e0 con 0x5645efc0fc00 2024-04-25T08:52:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 -- 172.21.15.46:0/2493538172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5645eeed3c20 con 0x5645efc0fc00 2024-04-25T08:52:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 1 -- 172.21.15.46:0/2493538172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645eeed3a40 con 0x5645efc0fc00 2024-04-25T08:52:54.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.002+0000 7f42dac32400 4 main: JournalTool: resolving pool 17 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d0400640 1 -- 172.21.15.46:0/2493538172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5645efdacc00 con 0x5645efc0fc00 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d0400640 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x5645ef16b400 0x5645ef25a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d0400640 1 -- 172.21.15.46:0/2493538172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x5645ef170a00 con 0x5645efc0fc00 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 4 main: JournalTool: creating IoCtx.. 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 4 main: Executing for rank 0 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x5645ef16b400 0x5645ef25a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x5645ef16b400 0x5645ef25a000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x5645efc95410 tx=0x5645eef084e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 1 -- start start 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 1 Processor -- start 2024-04-25T08:52:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efdb3000 0x5645eefec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2c00 0x5645eefecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efdb2800 0x5645eefed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645efc0d380 con 0x5645efdb2c00 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645efc0d500 con 0x5645efdb3000 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42dac32400 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645efc0d800 con 0x5645efdb2800 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2c00 0x5645eefecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2c00 0x5645eefecc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46272/0 (socket says 172.21.15.46:46272) 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 -- 172.21.15.46:0/1759839509 learned_addr learned my addr 172.21.15.46:0/1759839509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 -- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efdb2800 msgr2=0x5645eefed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efdb2800 0x5645eefed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 -- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efdb3000 msgr2=0x5645eefec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d8600640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efdb3000 0x5645eefec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efdb3000 0x5645eefec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d7c00640 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645ef2b0780 con 0x5645efdb2c00 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.006+0000 7f42d8600640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efdb3000 0x5645eefec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42d7c00640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2c00 0x5645eefecc00 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x5645efc956e0 tx=0x5645eef080c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e1af3f91ede1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42cc800640 1 -- 172.21.15.46:0/1759839509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645eee66b60 con 0x5645efdb2c00 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42cc800640 1 -- 172.21.15.46:0/1759839509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645eee7ae00 con 0x5645efdb2c00 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5645ef2b0960 con 0x5645efdb2c00 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5645ef2b0b40 con 0x5645efdb2c00 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42dac32400 4 waiting for MDS map... 2024-04-25T08:52:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42cc800640 1 -- 172.21.15.46:0/1759839509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645eee66340 con 0x5645efdb2c00 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42cc800640 1 -- 172.21.15.46:0/1759839509 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x5645efdb4800 con 0x5645efdb2c00 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42cc800640 1 -- 172.21.15.46:0/1759839509 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x5645eee87880 con 0x5645efdb2c00 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42dac32400 4 Got MDS map 106 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5645efdb2400 0x5645eefed700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5645efdb2000 con 0x5645efdb2400 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42d8600640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5645efdb2400 0x5645eefed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42d8600640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5645efdb2400 0x5645eefed700 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=aa96d546febfb7b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42d8600640 1 -- 172.21.15.46:0/1759839509 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v123'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5645eefac000 con 0x5645efdb2400 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x5645ef2c1800 0x5645ef25a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5645ef2c1c00 con 0x5645ef2c1800 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.010+0000 7f42d7200640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x5645ef2c1800 0x5645ef25a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.014+0000 7f42d7200640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x5645ef2c1800 0x5645ef25a580 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=ca98d3a6c47ec1ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d7200640 1 -- 172.21.15.46:0/1759839509 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v123'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5645eefac500 con 0x5645ef2c1800 2024-04-25T08:52:54.010 INFO:teuthology.orchestra.run.smithi046.stdout:writing EResetJournal entry 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5645ef2c1c00 con 0x5645ef2c1800 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x5645ef16a400 0x5645ef25ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5645ef16a000 con 0x5645ef16a400 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5645efdb3c00 0x5645ef25b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5645ef2ea000 con 0x5645efdb3c00 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5645ef2ea400 0x5645ef25b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5645ef2ea800 con 0x5645ef2ea400 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5645ef2eac00 con 0x5645efdb3c00 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5645ef2eb000 con 0x5645efdb2400 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d7c00640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5645efdb3c00 0x5645ef25b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d8600640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x5645ef16a400 0x5645ef25ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d7200640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5645ef2ea400 0x5645ef25b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d7c00640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5645efdb3c00 0x5645ef25b080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=c8d1c1f3e2e54b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d8600640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x5645ef16a400 0x5645ef25ab00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=4aac1a8aa755deba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d7200640 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5645ef2ea400 0x5645ef25b600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=a2707d2f8496320b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d7200640 1 -- 172.21.15.46:0/1759839509 <== osd.3 v2:172.21.15.46:6824/1647449001 2 ==== osd_op_reply(3 200.00000001 [delete] v123'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5645eefac500 con 0x5645ef2c1800 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d8600640 1 -- 172.21.15.46:0/1759839509 <== osd.1 v2:172.21.15.46:6809/2552657765 2 ==== osd_op_reply(8 200.00000006 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5645eefac000 con 0x5645efdb2400 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d7c00640 1 -- 172.21.15.46:0/1759839509 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(7 200.00000005 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5645eefac780 con 0x5645efdb3c00 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d7c00640 1 -- 172.21.15.46:0/1759839509 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== osd_op_reply(5 200.00000003 [delete] v123'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5645eefac780 con 0x5645efdb3c00 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42d8600640 1 -- 172.21.15.46:0/1759839509 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== osd_op_reply(4 200.00000002 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5645eefac000 con 0x5645ef16a400 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.018+0000 7f42caa00640 1 -- 172.21.15.46:0/1759839509 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5645ef2f0000 con 0x5645ef2c1800 2024-04-25T08:52:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42d7200640 1 -- 172.21.15.46:0/1759839509 <== osd.3 v2:172.21.15.46:6824/1647449001 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v123'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5645eefac500 con 0x5645ef2c1800 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42d7200640 1 -- 172.21.15.46:0/1759839509 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(6 200.00000004 [delete] v123'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5645eefac500 con 0x5645ef2ea400 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5645efdb3c00 msgr2=0x5645ef25b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x5645efdb3c00 0x5645ef25b080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5645efdb2400 msgr2=0x5645eefed700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x5645efdb2400 0x5645eefed700 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x5645ef2c1800 msgr2=0x5645ef25a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x5645ef2c1800 0x5645ef25a580 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x5645ef16a400 msgr2=0x5645ef25ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x5645ef16a400 0x5645ef25ab00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5645ef2ea400 msgr2=0x5645ef25b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x5645ef2ea400 0x5645ef25b600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42d7200640 1 -- 172.21.15.46:0/1759839509 reap_dead start 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2c00 msgr2=0x5645eefecc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1759839509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efdb2c00 0x5645eefecc00 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x5645efc956e0 tx=0x5645eef080c0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42d7200640 1 -- 172.21.15.46:0/1759839509 reap_dead start 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42d7200640 1 -- 172.21.15.46:0/1759839509 reap_dead start 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 shutdown_connections 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 >> 172.21.15.46:0/1759839509 conn(0x5645efdb3400 msgr2=0x5645ef2aa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 shutdown_connections 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1759839509 wait complete. 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x5645ef16b400 msgr2=0x5645ef25a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x5645ef16b400 0x5645ef25a000 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x5645efc95410 tx=0x5645eef084e0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/2493538172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efc0fc00 msgr2=0x5645eefeab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efc0fc00 0x5645eefeab00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x5645efd635c0 tx=0x5645efd70d20 comp rx=0 tx=0).stop 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/2493538172 shutdown_connections 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x5645ef16b400 0x5645ef25a000 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645efdb3800 0x5645eefec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645efc0f800 0x5645eefea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/2493538172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645efc0fc00 0x5645eefeab00 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/2493538172 >> 172.21.15.46:0/2493538172 conn(0x5645eee30c00 msgr2=0x5645eee5f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/2493538172 shutdown_connections 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/2493538172 wait complete. 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1806215557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645eee31800 msgr2=0x5645eefebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1806215557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645eee31800 0x5645eefebb80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x5645efc94a20 tx=0x5645eef08000 comp rx=0 tx=0).stop 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1806215557 shutdown_connections 2024-04-25T08:52:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1806215557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645eee31000 0x5645eefeb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1806215557 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x5645eee31400 0x5645eefeb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 --2- 172.21.15.46:0/1806215557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5645eee31800 0x5645eefebb80 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.022+0000 7f42dac32400 1 -- 172.21.15.46:0/1806215557 >> 172.21.15.46:0/1806215557 conn(0x5645eee31c00 msgr2=0x5645eee5f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.026+0000 7f42dac32400 1 -- 172.21.15.46:0/1806215557 shutdown_connections 2024-04-25T08:52:54.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.026+0000 7f42dac32400 1 -- 172.21.15.46:0/1806215557 wait complete. 2024-04-25T08:52:54.020 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.084530, result: writing EResetJournal entry 2024-04-25T08:52:54.020 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T08:52:54.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.114+0000 7fb1fc82b8c0 1 -- start start 2024-04-25T08:52:54.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.114+0000 7fb1fc82b8c0 1 Processor -- start 2024-04-25T08:52:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d16584d800 0x55d165a06580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d16584d400 0x55d165a06b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d16584d000 0x55d165a07080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16674c180 con 0x55d16584d800 2024-04-25T08:52:54.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16674c300 con 0x55d16584d000 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d16674c480 con 0x55d16584d400 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d16584d800 0x55d165a06580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d16584d800 0x55d165a06580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46288/0 (socket says 172.21.15.46:46288) 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f9800640 1 -- 172.21.15.46:0/4291119752 learned_addr learned my addr 172.21.15.46:0/4291119752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/4291119752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d16584d400 0x55d165a06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fa200640 1 --2- 172.21.15.46:0/4291119752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d16584d000 0x55d165a07080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f9800640 1 -- 172.21.15.46:0/4291119752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d16584d400 msgr2=0x55d165a06b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f9800640 1 --2- 172.21.15.46:0/4291119752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d16584d400 0x55d165a06b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f9800640 1 -- 172.21.15.46:0/4291119752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d16584d000 msgr2=0x55d165a07080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f9800640 1 --2- 172.21.15.46:0/4291119752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d16584d000 0x55d165a07080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f9800640 1 -- 172.21.15.46:0/4291119752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1658721e0 con 0x55d16584d800 2024-04-25T08:52:54.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f9800640 1 --2- 172.21.15.46:0/4291119752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d16584d800 0x55d165a06580 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x55d1666958c0 tx=0x55d165927440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c3fe3661be69b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f6600640 1 -- 172.21.15.46:0/4291119752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d1658824e0 con 0x55d16584d800 2024-04-25T08:52:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f6600640 1 -- 172.21.15.46:0/4291119752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d1658961c0 con 0x55d16584d800 2024-04-25T08:52:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f6600640 1 -- 172.21.15.46:0/4291119752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d165882b60 con 0x55d16584d800 2024-04-25T08:52:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/4291119752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1658732c0 con 0x55d16584d800 2024-04-25T08:52:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/4291119752 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55d165873a40 con 0x55d16584d800 2024-04-25T08:52:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 4 waiting for MDS map... 2024-04-25T08:52:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f6600640 1 -- 172.21.15.46:0/4291119752 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x55d166774600 con 0x55d16584d800 2024-04-25T08:52:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f6600640 1 -- 172.21.15.46:0/4291119752 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x55d1658a3180 con 0x55d16584d800 2024-04-25T08:52:54.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 4 Got MDS map 106 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 4 datascan.main: Using metadata pool output 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 4 datascan.main: connecting to RADOS... 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 Processor -- start 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 -- start start 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166776c00 0x55d165a07600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166777000 0x55d165a07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16674c600 con 0x55d166777400 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16674c780 con 0x55d166777000 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fc82b8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d16674c900 con 0x55d166776c00 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fa200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166776c00 0x55d165a07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1f8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46300/0 (socket says 172.21.15.46:46300) 2024-04-25T08:52:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 -- 172.21.15.46:0/1021713737 learned_addr learned my addr 172.21.15.46:0/1021713737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.118+0000 7fb1fa200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166776c00 0x55d165a07600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:37088/0 (socket says 172.21.15.46:37088) 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f9800640 1 --2- 172.21.15.46:0/1021713737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166777000 0x55d165a07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 -- 172.21.15.46:0/1021713737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166776c00 msgr2=0x55d165a07600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/1021713737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166776c00 0x55d165a07600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 -- 172.21.15.46:0/1021713737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166777000 msgr2=0x55d165a07b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/1021713737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166777000 0x55d165a07b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 -- 172.21.15.46:0/1021713737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d165873c20 con 0x55d166777400 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/1021713737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08100 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x55d166770810 tx=0x55d166778c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1f1600bd3f5040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f3e00640 1 -- 172.21.15.46:0/1021713737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d1658824e0 con 0x55d166777400 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f3e00640 1 -- 172.21.15.46:0/1021713737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d165896e00 con 0x55d166777400 2024-04-25T08:52:54.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f3e00640 1 -- 172.21.15.46:0/1021713737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d165882b60 con 0x55d166777400 2024-04-25T08:52:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/1021713737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 msgr2=0x55d165a08100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/1021713737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08100 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x55d166770810 tx=0x55d166778c00 comp rx=0 tx=0).stop 2024-04-25T08:52:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/1021713737 shutdown_connections 2024-04-25T08:52:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/1021713737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08100 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/1021713737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166777000 0x55d165a07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/1021713737 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166776c00 0x55d165a07600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/1021713737 >> 172.21.15.46:0/1021713737 conn(0x55d16584cc00 msgr2=0x55d16587b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/1021713737 shutdown_connections 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/1021713737 wait complete. 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 Processor -- start 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- start start 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166776c00 0x55d165a09180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166777000 0x55d165a08c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16674dc80 con 0x55d166777400 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d16674db00 con 0x55d166776c00 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d16674d980 con 0x55d166777000 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fa200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166776c00 0x55d165a09180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46314/0 (socket says 172.21.15.46:46314) 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f9800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166777000 0x55d165a08c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 learned_addr learned my addr 172.21.15.46:0/2194769415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166777000 msgr2=0x55d165a08c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d166777000 0x55d165a08c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166776c00 msgr2=0x55d165a09180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166776c00 0x55d165a09180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1658721e0 con 0x55d166777400 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fa200640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d166776c00 0x55d165a09180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08680 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x55d166771ad0 tx=0x55d1658ee120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f2000640 1 -- 172.21.15.46:0/2194769415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d165882000 con 0x55d166777400 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1f2000640 1 -- 172.21.15.46:0/2194769415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d165896c40 con 0x55d166777400 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.122+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d1658732c0 con 0x55d166777400 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1f2000640 1 -- 172.21.15.46:0/2194769415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d1658824e0 con 0x55d166777400 2024-04-25T08:52:54.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d165873e00 con 0x55d166777400 2024-04-25T08:52:54.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1fc82b8c0 4 datascan.init: resolving metadata pool 17 2024-04-25T08:52:54.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1f2000640 1 -- 172.21.15.46:0/2194769415 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55d165b8ac00 con 0x55d166777400 2024-04-25T08:52:54.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1f2000640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55d165b7cc00 0x55d165a09700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1f2000640 1 -- 172.21.15.46:0/2194769415 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x55d165b82a00 con 0x55d166777400 2024-04-25T08:52:54.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1fc82b8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T08:52:54.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1fc82b8c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T08:52:54.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1fc82b8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T08:52:54.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55d166777800 0x55d165a08100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166777c00 con 0x55d166777800 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1fa200640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55d165b7cc00 0x55d165a09700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1f9800640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55d166777800 0x55d165a08100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1f9800640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55d166777800 0x55d165a08100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1fa200640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55d165b7cc00 0x55d165a09700 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x55d165928150 tx=0x55d1658ee000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1f3400640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55d165b7d000 0x55d165c84000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.126+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165b7d400 con 0x55d165b7d000 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.130+0000 7fb1fa200640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55d165b7d000 0x55d165c84000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.130+0000 7fb1fa200640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55d165b7d000 0x55d165c84000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.130+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.130+0000 7fb1f3e00640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55d165b7dc00 0x55d165c84580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.130+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88000 con 0x55d165b7dc00 2024-04-25T08:52:54.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.130+0000 7fb1f9800640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55d165b7dc00 0x55d165c84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f9800640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55d165b7dc00 0x55d165c84580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d165b7dc00 2024-04-25T08:52:54.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165b7d800 con 0x55d166777800 2024-04-25T08:52:54.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88400 con 0x55d166777800 2024-04-25T08:52:54.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 3 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166776800 con 0x55d165b7d000 2024-04-25T08:52:54.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f3e00640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55d165c88000 0x55d165c84b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88800 con 0x55d165c88000 2024-04-25T08:52:54.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55d165c88000 0x55d165c84b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55d165c88000 0x55d165c84b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(7 [pgnls start_epoch 123 out=81b] v121'17 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:8 18.18 18:1ad8d55b:::10000000006.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88c00 con 0x55d165c88000 2024-04-25T08:52:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 2 ==== osd_op_reply(8 10000000006.00000001 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1fc82b8c0 10 datascan.operator(): handling object 1099511627782.1 2024-04-25T08:52:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.134+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:9 18.25 18:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c89000 con 0x55d165b7dc00 2024-04-25T08:52:54.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== osd_op_reply(9 10000000006.00000000 [call] v123'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d16595c000 con 0x55d165b7dc00 2024-04-25T08:52:54.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x55d165c89400 0x55d165a07b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c89800 con 0x55d165c89400 2024-04-25T08:52:54.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1fa200640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x55d165c89400 0x55d165a07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1fa200640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x55d165c89400 0x55d165a07b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== osd_op_reply(10 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165b7d800 con 0x55d166777800 2024-04-25T08:52:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 4 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166776400 con 0x55d166777800 2024-04-25T08:52:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 5 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6000 con 0x55d165b7dc00 2024-04-25T08:52:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.6 v2:172.21.15.82:6823/1178952145 3 ==== osd_op_reply(13 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d165b7dc00 2024-04-25T08:52:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166777c00 con 0x55d166777800 2024-04-25T08:52:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 6 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6400 con 0x55d165b7d000 2024-04-25T08:52:54.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 3 ==== osd_op_reply(15 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f3400640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x55d165c89800 0x55d165c85600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88400 con 0x55d165c89800 2024-04-25T08:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.138+0000 7fb1f9800640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x55d165c89800 0x55d165c85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f9800640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x55d165c89800 0x55d165c85600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.2 v2:172.21.15.46:6808/3102391403 1 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d165c89800 2024-04-25T08:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6800 con 0x55d165c88000 2024-04-25T08:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 3 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f3400640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55d165c89000 0x55d165c85b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166625400 con 0x55d165c89000 2024-04-25T08:52:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55d165c89000 0x55d165c85b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f8e00640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55d165c89000 0x55d165c85b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c89000 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6c00 con 0x55d165c89000 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.7 v2:172.21.15.82:6808/1420997799 2 ==== osd_op_reply(19 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c89000 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165b7d400 con 0x55d165c89400 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 2 ==== osd_op_reply(20 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6800 con 0x55d165c89800 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.2 v2:172.21.15.46:6808/3102391403 2 ==== osd_op_reply(21 [pgnls start_epoch 123 out=81b] v122'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d16595c000 con 0x55d165c89800 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:22 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c89c00 con 0x55d165c89800 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.2 v2:172.21.15.46:6808/3102391403 3 ==== osd_op_reply(22 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d16595c000 con 0x55d165c89800 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1fc82b8c0 10 datascan.operator(): handling object 1099511627778.0 2024-04-25T08:52:54.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.142+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:23 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6000 con 0x55d165c89800 2024-04-25T08:52:54.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.2 v2:172.21.15.46:6808/3102391403 4 ==== osd_op_reply(23 10000000002.00000000 [call] v123'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d16595c000 con 0x55d165c89800 2024-04-25T08:52:54.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:24 18.32 18:4c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7000 con 0x55d165c89400 2024-04-25T08:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 3 ==== osd_op_reply(24 [pgnls start_epoch 123 out=81b] v121'16 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:25 18.32 18:4def72d4:::10000000003.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7400 con 0x55d165c89400 2024-04-25T08:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 4 ==== osd_op_reply(25 10000000003.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fc82b8c0 10 datascan.operator(): handling object 1099511627779.0 2024-04-25T08:52:54.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:26 18.32 18:4def72d4:::10000000003.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7800 con 0x55d165c89400 2024-04-25T08:52:54.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 5 ==== osd_op_reply(26 10000000003.00000000 [call] v123'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:27 18.a 18:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7c00 con 0x55d165c89400 2024-04-25T08:52:54.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 6 ==== osd_op_reply(27 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1f3400640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x55d165b7d800 0x55d165c86100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:28 18.2a 18:54000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166625400 con 0x55d165b7d800 2024-04-25T08:52:54.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.146+0000 7fb1fa200640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x55d165b7d800 0x55d165c86100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1fa200640 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x55d165b7d800 0x55d165c86100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d800 2024-04-25T08:52:54.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:29 18.1a 18:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6400 con 0x55d165c89400 2024-04-25T08:52:54.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 7 ==== osd_op_reply(29 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:30 18.3a 18:5c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88800 con 0x55d166777800 2024-04-25T08:52:54.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 7 ==== osd_op_reply(30 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:31 18.6 18:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166776800 con 0x55d165c89000 2024-04-25T08:52:54.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.7 v2:172.21.15.82:6808/1420997799 3 ==== osd_op_reply(31 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c89000 2024-04-25T08:52:54.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:32 18.26 18:64000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88c00 con 0x55d165c89800 2024-04-25T08:52:54.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.2 v2:172.21.15.46:6808/3102391403 5 ==== osd_op_reply(32 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d165c89800 2024-04-25T08:52:54.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:33 18.16 18:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166776400 con 0x55d165c89400 2024-04-25T08:52:54.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 8 ==== osd_op_reply(33 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:34 18.36 18:6c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166625400 con 0x55d165c88000 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 4 ==== osd_op_reply(34 [pgnls start_epoch 123 out=49b] v121'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:35 18.e 18:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166777c00 con 0x55d166777800 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 8 ==== osd_op_reply(35 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:36 18.2e 18:74000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7000 con 0x55d165c89400 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 9 ==== osd_op_reply(36 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.150+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:37 18.1e 18:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88400 con 0x55d165b7d000 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 4 ==== osd_op_reply(37 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:38 18.3e 18:7c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7400 con 0x55d165c89400 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 10 ==== osd_op_reply(38 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:39 18.1 18:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c89c00 con 0x55d165c89400 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 11 ==== osd_op_reply(39 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:40 18.21 18:84000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7800 con 0x55d165b7d000 2024-04-25T08:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 5 ==== osd_op_reply(40 [pgnls start_epoch 123 out=49b] v121'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:41 18.11 18:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88800 con 0x55d165c88000 2024-04-25T08:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 5 ==== osd_op_reply(41 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:42 18.31 18:8c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7c00 con 0x55d165b7d800 2024-04-25T08:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.4 v2:172.21.15.82:6800/2293006647 2 ==== osd_op_reply(42 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d800 2024-04-25T08:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88c00 con 0x55d165c89400 2024-04-25T08:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 12 ==== osd_op_reply(43 [pgnls start_epoch 123 out=49b] v121'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165b7d400 con 0x55d165c89000 2024-04-25T08:52:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.7 v2:172.21.15.82:6808/1420997799 4 ==== osd_op_reply(44 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c89000 2024-04-25T08:52:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6800 con 0x55d165b7d800 2024-04-25T08:52:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.4 v2:172.21.15.82:6800/2293006647 3 ==== osd_op_reply(45 [pgnls start_epoch 123 out=49b] v121'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d800 2024-04-25T08:52:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:46 18.39 18:9c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6c00 con 0x55d166777800 2024-04-25T08:52:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 9 ==== osd_op_reply(46 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:47 18.5 18:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6000 con 0x55d165c88000 2024-04-25T08:52:54.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 6 ==== osd_op_reply(47 [pgnls start_epoch 123 out=81b] v122'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:48 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166776800 con 0x55d165c88000 2024-04-25T08:52:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 7 ==== osd_op_reply(48 10000000004.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fc82b8c0 10 datascan.operator(): handling object 1099511627780.0 2024-04-25T08:52:54.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.154+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:49 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166776400 con 0x55d165c88000 2024-04-25T08:52:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 8 ==== osd_op_reply(49 10000000004.00000000 [call] v123'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:50 18.25 18:a4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166777c00 con 0x55d165b7dc00 2024-04-25T08:52:54.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.6 v2:172.21.15.82:6823/1178952145 4 ==== osd_op_reply(50 [pgnls start_epoch 123 out=81b] v123'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d16595c000 con 0x55d165b7dc00 2024-04-25T08:52:54.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:51 18.25 18:a5feb6f3:::10000000006.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7000 con 0x55d165b7dc00 2024-04-25T08:52:54.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.6 v2:172.21.15.82:6823/1178952145 5 ==== osd_op_reply(51 10000000006.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d16595c000 con 0x55d165b7dc00 2024-04-25T08:52:54.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1fc82b8c0 10 datascan.operator(): handling object 1099511627782.0 2024-04-25T08:52:54.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:52 18.25 18:a5feb6f3:::10000000006.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7400 con 0x55d165b7dc00 2024-04-25T08:52:54.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.6 v2:172.21.15.82:6823/1178952145 6 ==== osd_op_reply(52 10000000006.00000000 [call] v123'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d16595c000 con 0x55d165b7dc00 2024-04-25T08:52:54.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7800 con 0x55d165b7d000 2024-04-25T08:52:54.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 6 ==== osd_op_reply(53 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.158+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6400 con 0x55d165b7d800 2024-04-25T08:52:54.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.4 v2:172.21.15.82:6800/2293006647 4 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d800 2024-04-25T08:52:54.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165cc2000 con 0x55d165b7d800 2024-04-25T08:52:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.4 v2:172.21.15.82:6800/2293006647 5 ==== osd_op_reply(55 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d800 2024-04-25T08:52:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88800 con 0x55d165b7dc00 2024-04-25T08:52:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.6 v2:172.21.15.82:6823/1178952145 7 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d165b7dc00 2024-04-25T08:52:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165cc2400 con 0x55d166777800 2024-04-25T08:52:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.0 v2:172.21.15.46:6800/3448226631 10 ==== osd_op_reply(57 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d166777800 2024-04-25T08:52:54.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c88400 con 0x55d165c89400 2024-04-25T08:52:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.3 v2:172.21.15.46:6824/1647449001 13 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165c89400 2024-04-25T08:52:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165cc2800 con 0x55d165c88000 2024-04-25T08:52:54.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 9 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165c89c00 con 0x55d165c89800 2024-04-25T08:52:54.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.2 v2:172.21.15.46:6808/3102391403 6 ==== osd_op_reply(60 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d165c89800 2024-04-25T08:52:54.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165cc2c00 con 0x55d165c89000 2024-04-25T08:52:54.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.7 v2:172.21.15.82:6808/1420997799 5 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c89000 2024-04-25T08:52:54.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165b7d400 con 0x55d165c88000 2024-04-25T08:52:54.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 10 ==== osd_op_reply(62 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165cc3000 con 0x55d165b7d000 2024-04-25T08:52:54.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 7 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166625400 con 0x55d165c89000 2024-04-25T08:52:54.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.162+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.7 v2:172.21.15.82:6808/1420997799 6 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c89000 2024-04-25T08:52:54.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165cc3400 con 0x55d165c88000 2024-04-25T08:52:54.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 11 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166776800 con 0x55d165b7d800 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.4 v2:172.21.15.82:6800/2293006647 6 ==== osd_op_reply(66 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d800 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165cc3800 con 0x55d165c89800 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f9800640 1 -- 172.21.15.46:0/2194769415 <== osd.2 v2:172.21.15.46:6808/3102391403 7 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c000 con 0x55d165c89800 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7c00 con 0x55d165b7d000 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 8 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165b7d400 con 0x55d165b7d000 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 9 ==== osd_op_reply(69 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6800 con 0x55d165c89000 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.7 v2:172.21.15.82:6808/1420997799 7 ==== osd_op_reply(70 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c89000 2024-04-25T08:52:54.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166777c00 con 0x55d165b7d000 2024-04-25T08:52:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.5 v2:172.21.15.82:6816/541310136 10 ==== osd_op_reply(71 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d000 2024-04-25T08:52:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca7800 con 0x55d165c88000 2024-04-25T08:52:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.1 v2:172.21.15.46:6809/2552657765 12 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c88000 2024-04-25T08:52:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3e00640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d166776400 con 0x55d165c89000 2024-04-25T08:52:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 <== osd.7 v2:172.21.15.82:6808/1420997799 8 ==== osd_op_reply(73 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c780 con 0x55d165c89000 2024-04-25T08:52:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f3400640 1 -- 172.21.15.46:0/2194769415 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d165ca6400 con 0x55d165b7d800 2024-04-25T08:52:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fa200640 1 -- 172.21.15.46:0/2194769415 <== osd.4 v2:172.21.15.82:6800/2293006647 7 ==== osd_op_reply(74 [pgnls start_epoch 123 out=49b] v121'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d16595c500 con 0x55d165b7d800 2024-04-25T08:52:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55d166777800 msgr2=0x55d165a08100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x55d166777800 0x55d165a08100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55d165c88000 msgr2=0x55d165c84b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x55d165c88000 0x55d165c84b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x55d165c89800 msgr2=0x55d165c85600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x55d165c89800 0x55d165c85600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x55d165c89400 msgr2=0x55d165a07b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x55d165c89400 0x55d165a07b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x55d165b7d800 msgr2=0x55d165c86100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x55d165b7d800 0x55d165c86100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55d165b7d000 msgr2=0x55d165c84000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x55d165b7d000 0x55d165c84000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55d165b7dc00 msgr2=0x55d165c84580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x55d165b7dc00 0x55d165c84580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55d165c89000 msgr2=0x55d165c85b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x55d165c89000 0x55d165c85b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 reap_dead start 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 reap_dead start 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1f8e00640 1 -- 172.21.15.46:0/2194769415 reap_dead start 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55d165b7cc00 msgr2=0x55d165a09700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55d165b7cc00 0x55d165a09700 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x55d165928150 tx=0x55d1658ee000 comp rx=0 tx=0).stop 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 msgr2=0x55d165a08680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.166+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08680 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x55d166771ad0 tx=0x55d1658ee120 comp rx=0 tx=0).stop 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 shutdown_connections 2024-04-25T08:52:54.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x55d165b7cc00 0x55d165a09700 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/2194769415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d166777400 0x55d165a08680 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 >> 172.21.15.46:0/2194769415 conn(0x55d16584cc00 msgr2=0x55d16587b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 shutdown_connections 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/2194769415 wait complete. 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/4291119752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d16584d800 msgr2=0x55d165a06580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/4291119752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d16584d800 0x55d165a06580 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x55d1666958c0 tx=0x55d165927440 comp rx=0 tx=0).stop 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/4291119752 shutdown_connections 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/4291119752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d16584d000 0x55d165a07080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/4291119752 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55d16584d400 0x55d165a06b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 --2- 172.21.15.46:0/4291119752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d16584d800 0x55d165a06580 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/4291119752 >> 172.21.15.46:0/4291119752 conn(0x55d16584dc00 msgr2=0x55d16587b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/4291119752 shutdown_connections 2024-04-25T08:52:54.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.170+0000 7fb1fc82b8c0 1 -- 172.21.15.46:0/4291119752 wait complete. 2024-04-25T08:52:54.165 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.144909, result: 2024-04-25T08:52:54.166 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-25T08:52:54.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 -- start start 2024-04-25T08:52:54.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 Processor -- start 2024-04-25T08:52:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a2f4000 0x563f096eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f09531c00 0x563f096eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f09531800 0x563f096ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f09590900 con 0x563f09531800 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a440000 con 0x563f09531c00 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f0a440180 con 0x563f0a2f4000 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6759400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f09531800 0x563f096ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6759400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f09531800 0x563f096ebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46330/0 (socket says 172.21.15.46:46330) 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6759400640 1 -- 172.21.15.46:0/3715334007 learned_addr learned my addr 172.21.15.46:0/3715334007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6758000640 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f09531c00 0x563f096eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6758a00640 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a2f4000 0x563f096eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6759400640 1 -- 172.21.15.46:0/3715334007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a2f4000 msgr2=0x563f096eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6759400640 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a2f4000 0x563f096eb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6759400640 1 -- 172.21.15.46:0/3715334007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f09531c00 msgr2=0x563f096eab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6759400640 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f09531c00 0x563f096eab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6759400640 1 -- 172.21.15.46:0/3715334007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f095561e0 con 0x563f09531800 2024-04-25T08:52:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6758000640 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f09531c00 0x563f096eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6759400640 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f09531800 0x563f096ebb80 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x563f0a37b8c0 tx=0x563f0960ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e418e9142e7c57e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6755800640 1 -- 172.21.15.46:0/3715334007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563f095669c0 con 0x563f09531800 2024-04-25T08:52:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6755800640 1 -- 172.21.15.46:0/3715334007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563f0957ac40 con 0x563f09531800 2024-04-25T08:52:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f6755800640 1 -- 172.21.15.46:0/3715334007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563f09566b60 con 0x563f09531800 2024-04-25T08:52:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/3715334007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f09557c20 con 0x563f09531800 2024-04-25T08:52:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/3715334007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563f09557e00 con 0x563f09531800 2024-04-25T08:52:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.266+0000 7f675a8ed8c0 4 waiting for MDS map... 2024-04-25T08:52:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6755800640 1 -- 172.21.15.46:0/3715334007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x563f0a45a400 con 0x563f09531800 2024-04-25T08:52:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6755800640 1 -- 172.21.15.46:0/3715334007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x563f09587180 con 0x563f09531800 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 4 Got MDS map 106 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 4 datascan.main: Using metadata pool output 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 4 datascan.main: connecting to RADOS... 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 Processor -- start 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 -- start start 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4400 0x563f096eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a45ec00 0x563f096ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f0a45f000 0x563f096ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a440300 con 0x563f0a2f4400 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a440480 con 0x563f0a45f000 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f0a440600 con 0x563f0a45ec00 2024-04-25T08:52:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6758000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4400 0x563f096eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6758000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4400 0x563f096eb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46342/0 (socket says 172.21.15.46:46342) 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6758000640 1 -- 172.21.15.46:0/2956132056 learned_addr learned my addr 172.21.15.46:0/2956132056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6758000640 1 -- 172.21.15.46:0/2956132056 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a45ec00 msgr2=0x563f096ea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6758000640 1 --2- 172.21.15.46:0/2956132056 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a45ec00 0x563f096ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6758000640 1 -- 172.21.15.46:0/2956132056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f0a45f000 msgr2=0x563f096ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6758000640 1 --2- 172.21.15.46:0/2956132056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f0a45f000 0x563f096ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6758000640 1 -- 172.21.15.46:0/2956132056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f095572c0 con 0x563f0a2f4400 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6758000640 1 --2- 172.21.15.46:0/2956132056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4400 0x563f096eb600 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x563f0a4606c0 tx=0x563f0a46cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69f09152ba1693b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6753000640 1 -- 172.21.15.46:0/2956132056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563f095669c0 con 0x563f0a2f4400 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6753000640 1 -- 172.21.15.46:0/2956132056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563f0957aa80 con 0x563f0a2f4400 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f6753000640 1 -- 172.21.15.46:0/2956132056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563f09566b60 con 0x563f0a2f4400 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2956132056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4400 msgr2=0x563f096eb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2956132056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4400 0x563f096eb600 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x563f0a4606c0 tx=0x563f0a46cc00 comp rx=0 tx=0).stop 2024-04-25T08:52:54.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2956132056 shutdown_connections 2024-04-25T08:52:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2956132056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f0a45f000 0x563f096ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2956132056 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a45ec00 0x563f096ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2956132056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4400 0x563f096eb600 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2956132056 >> 172.21.15.46:0/2956132056 conn(0x563f0a2f4800 msgr2=0x563f0955d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2956132056 shutdown_connections 2024-04-25T08:52:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.270+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2956132056 wait complete. 2024-04-25T08:52:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 Processor -- start 2024-04-25T08:52:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 -- start start 2024-04-25T08:52:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4800 0x563f096ed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a45fc00 0x563f096ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f0a45f800 0x563f096ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a440900 con 0x563f0a2f4800 2024-04-25T08:52:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f0a440780 con 0x563f0a45f800 2024-04-25T08:52:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x563f0a440a80 con 0x563f0a45fc00 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4800 0x563f096ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4800 0x563f096ed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46358/0 (socket says 172.21.15.46:46358) 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 learned_addr learned my addr 172.21.15.46:0/2916562059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758a00640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a45fc00 0x563f096ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f0a45f800 0x563f096ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a45fc00 msgr2=0x563f096ecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a45fc00 0x563f096ecc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f0a45f800 msgr2=0x563f096ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f0a45f800 0x563f096ec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f095563c0 con 0x563f0a2f4800 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4800 0x563f096ed180 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x563f0a461980 tx=0x563f0a46cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6751200640 1 -- 172.21.15.46:0/2916562059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563f09566340 con 0x563f0a2f4800 2024-04-25T08:52:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6751200640 1 -- 172.21.15.46:0/2916562059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563f0957b340 con 0x563f0a2f4800 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563f095561e0 con 0x563f0a2f4800 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6751200640 1 -- 172.21.15.46:0/2916562059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563f095669c0 con 0x563f0a2f4800 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f09557c20 con 0x563f0a2f4800 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f675a8ed8c0 4 datascan.init: resolving metadata pool 17 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.274+0000 7f6751200640 1 -- 172.21.15.46:0/2916562059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x563f09860c00 con 0x563f0a2f4800 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6751200640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563f0985d400 0x563f096ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6751200640 1 -- 172.21.15.46:0/2916562059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x563f0985ea00 con 0x563f0a2f4800 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f675a8ed8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f675a8ed8c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f675a8ed8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563f0a45f400 0x563f096ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a45f000 con 0x563f0a45f400 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6758a00640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563f0985d400 0x563f096ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563f0a45f400 0x563f096ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6758a00640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563f0985d400 0x563f096ed700 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x563f0960d200 tx=0x563f095d20c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563f0a45f400 0x563f096ec100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563f0a45ec00 0x563f096ea580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0985d800 con 0x563f0a45ec00 2024-04-25T08:52:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6759400640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563f0a45ec00 0x563f096ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6759400640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563f0a45ec00 0x563f096ea580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x563f09640500 con 0x563f0a45ec00 2024-04-25T08:52:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563f0985dc00 0x563f096eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09962000 con 0x563f0985dc00 2024-04-25T08:52:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6758a00640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563f0985dc00 0x563f096eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.278+0000 7f6758a00640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563f0985dc00 0x563f096eb600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6752600640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563f0996a000 0x563f09970000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996a400 con 0x563f0996a000 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563f0996a000 0x563f09970000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563f0996a000 0x563f09970000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996ac00 con 0x563f0a45ec00 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0a45ec00 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996a800 con 0x563f0985dc00 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996b000 con 0x563f0985dc00 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996b800 con 0x563f0996a000 2024-04-25T08:52:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6753000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563f0996b400 0x563f09970580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a45f000 con 0x563f0996b400 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.282+0000 7f6759400640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563f0996b400 0x563f09970580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6759400640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563f0996b400 0x563f09970580 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 123 out=49b] v121'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996bc00 con 0x563f0a45f400 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0985d000 con 0x563f0985dc00 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0985d000 con 0x563f0985dc00 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0985cc00 con 0x563f0a45ec00 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.6 v2:172.21.15.82:6823/1178952145 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0a45ec00 2024-04-25T08:52:54.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0985d800 con 0x563f0985dc00 2024-04-25T08:52:54.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996a800 con 0x563f0996a000 2024-04-25T08:52:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6752600640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563f09962000 0x563f09970b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a2f5c00 con 0x563f09962000 2024-04-25T08:52:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6758a00640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563f09962000 0x563f09970b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6758a00640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563f09962000 0x563f09970b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.2 v2:172.21.15.46:6808/3102391403 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f09962000 2024-04-25T08:52:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.286+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996b000 con 0x563f0996b400 2024-04-25T08:52:54.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6752600640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563f0996ac00 0x563f09971080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996a400 con 0x563f0996ac00 2024-04-25T08:52:54.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563f0996ac00 0x563f09971080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6758000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563f0996ac00 0x563f09971080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a45e800 con 0x563f0996ac00 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996b800 con 0x563f0a45f400 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a45f000 con 0x563f09962000 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.2 v2:172.21.15.46:6808/3102391403 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 123 out=81b] v123'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563f09640780 con 0x563f09962000 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09962400 con 0x563f0985dc00 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a000 con 0x563f0996a000 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a000 con 0x563f0a45ec00 2024-04-25T08:52:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.6 v2:172.21.15.82:6823/1178952145 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0a45ec00 2024-04-25T08:52:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.290+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a400 con 0x563f0985dc00 2024-04-25T08:52:54.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996bc00 con 0x563f0985dc00 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a800 con 0x563f0996a000 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996a800 con 0x563f0996b400 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 123 out=49b] v121'17 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998ac00 con 0x563f0a45f400 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996a400 con 0x563f0985dc00 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998b000 con 0x563f0985dc00 2024-04-25T08:52:54.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996b800 con 0x563f0a45ec00 2024-04-25T08:52:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.6 v2:172.21.15.82:6823/1178952145 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0a45ec00 2024-04-25T08:52:54.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0985cc00 con 0x563f0985dc00 2024-04-25T08:52:54.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a45e800 con 0x563f0996a000 2024-04-25T08:52:54.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0985d000 con 0x563f09962000 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.2 v2:172.21.15.46:6808/3102391403 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f09962000 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a000 con 0x563f0996b400 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.294+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0985d800 con 0x563f0996ac00 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a400 con 0x563f0996ac00 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996b000 con 0x563f0a45f400 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998b400 con 0x563f09962000 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.2 v2:172.21.15.46:6808/3102391403 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 123 out=49b] v123'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f09962000 2024-04-25T08:52:54.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996a800 con 0x563f0a45f400 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 123 out=81b] v123'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 20 datascan.forall_objects: OSD matched oid 10000000003.00000000 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 10 datascan.operator(): handling object 10000000003.0 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:42 18.32 18:4def72d4:::10000000003.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09962c00 con 0x563f0a45f400 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 7 ==== osd_op_reply(42 10000000003.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000003 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:43 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09963000 con 0x563f0985dc00 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 13 ==== osd_op_reply(43 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:44 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09963400 con 0x563f0a45f400 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 8 ==== osd_op_reply(44 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+524 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 20 datascan.get_frag_of: frag is * 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:45 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09963800 con 0x563f0985dc00 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 14 ==== osd_op_reply(45 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09963c00 con 0x563f0985dc00 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 15 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 20 datascan.read_dentry: bravo_head not found in result 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000003 at 0x0x10000000001/bravo with size=5 bytes 2024-04-25T08:52:54.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.298+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:47 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998b000 con 0x563f0985dc00 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 16 ==== osd_op_reply(47 10000000001.00000000 [omap-set-vals] v123'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f675a8ed8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:48 18.a 18:50000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996bc00 con 0x563f0a45f400 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6753000640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563f0996b800 0x563f09971600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:49 18.2a 18:54000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998b400 con 0x563f0996b800 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6759400640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563f0996b800 0x563f09971600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6759400640 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563f0996b800 0x563f09971600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== osd_op_reply(49 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b800 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:50 18.1a 18:58000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998b800 con 0x563f0a45f400 2024-04-25T08:52:54.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 10 ==== osd_op_reply(50 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:51 18.3a 18:5c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a800 con 0x563f0985dc00 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 17 ==== osd_op_reply(51 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:52 18.6 18:60000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09962400 con 0x563f0996ac00 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 5 ==== osd_op_reply(52 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.302+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:53 18.26 18:64000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998ac00 con 0x563f09962000 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.2 v2:172.21.15.46:6808/3102391403 5 ==== osd_op_reply(53 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f09962000 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:54 18.16 18:68000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a45f000 con 0x563f0a45f400 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 11 ==== osd_op_reply(54 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:55 18.36 18:6c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a400 con 0x563f0996b400 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 5 ==== osd_op_reply(55 [pgnls-filter start_epoch 123 out=49b] v121'32 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a2f5c00 con 0x563f0985dc00 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998bc00 con 0x563f0a45f400 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 12 ==== osd_op_reply(57 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998b800 con 0x563f0996a000 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a6000 con 0x563f0a45f400 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 13 ==== osd_op_reply(59 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998bc00 con 0x563f0a45f400 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 14 ==== osd_op_reply(60 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a6400 con 0x563f0996a000 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 123 out=49b] v121'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a45f000 con 0x563f0996b400 2024-04-25T08:52:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 6 ==== osd_op_reply(62 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a6800 con 0x563f0996b800 2024-04-25T08:52:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.4 v2:172.21.15.82:6800/2293006647 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b800 2024-04-25T08:52:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.306+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09962c00 con 0x563f0a45f400 2024-04-25T08:52:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 15 ==== osd_op_reply(64 [pgnls-filter start_epoch 123 out=49b] v121'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a6c00 con 0x563f0996ac00 2024-04-25T08:52:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09962800 con 0x563f0996b800 2024-04-25T08:52:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.4 v2:172.21.15.82:6800/2293006647 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 123 out=49b] v121'21 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b800 2024-04-25T08:52:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a7000 con 0x563f0985dc00 2024-04-25T08:52:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09963400 con 0x563f0996b400 2024-04-25T08:52:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 123 out=49b] v123'12 uv12 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a7400 con 0x563f0a45ec00 2024-04-25T08:52:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.6 v2:172.21.15.82:6823/1178952145 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 123 out=49b] v123'17 uv17 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0a45ec00 2024-04-25T08:52:54.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09962400 con 0x563f0996a000 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a7800 con 0x563f0996b800 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.4 v2:172.21.15.82:6800/2293006647 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b800 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0996bc00 con 0x563f0996b800 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.4 v2:172.21.15.82:6800/2293006647 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b800 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a7c00 con 0x563f0a45ec00 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.6 v2:172.21.15.82:6823/1178952145 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0a45ec00 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09962800 con 0x563f0985dc00 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.0 v2:172.21.15.46:6800/3448226631 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f0985dc00 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.310+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09963800 con 0x563f0a45f400 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.3 v2:172.21.15.46:6824/1647449001 16 ==== osd_op_reply(75 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0a45f400 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a6800 con 0x563f0996b400 2024-04-25T08:52:54.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09963c00 con 0x563f09962000 2024-04-25T08:52:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.2 v2:172.21.15.46:6808/3102391403 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f09962000 2024-04-25T08:52:54.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a45f000 con 0x563f0996ac00 2024-04-25T08:52:54.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f09962800 con 0x563f0996b400 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 9 ==== osd_op_reply(79 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a000 con 0x563f0996a000 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a7000 con 0x563f0996ac00 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998b400 con 0x563f0996b400 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a2f5c00 con 0x563f0996b800 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.4 v2:172.21.15.82:6800/2293006647 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b800 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a400 con 0x563f09962000 2024-04-25T08:52:54.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 <== osd.2 v2:172.21.15.46:6808/3102391403 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640780 con 0x563f09962000 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998b000 con 0x563f0996a000 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a7400 con 0x563f0996a000 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.314+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998a800 con 0x563f0996ac00 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a7800 con 0x563f0996a000 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.5 v2:172.21.15.82:6816/541310136 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996a000 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0998ac00 con 0x563f0996b400 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.1 v2:172.21.15.46:6809/2552657765 11 ==== osd_op_reply(89 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b400 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6752600640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f099a7c00 con 0x563f0996ac00 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6758000640 1 -- 172.21.15.46:0/2916562059 <== osd.7 v2:172.21.15.82:6808/1420997799 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640000 con 0x563f0996ac00 2024-04-25T08:52:54.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6753000640 1 -- 172.21.15.46:0/2916562059 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x563f0a45f000 con 0x563f0996b800 2024-04-25T08:52:54.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6759400640 1 -- 172.21.15.46:0/2916562059 <== osd.4 v2:172.21.15.82:6800/2293006647 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 123 out=49b] v121'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563f09640500 con 0x563f0996b800 2024-04-25T08:52:54.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563f0985dc00 msgr2=0x563f096eb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x563f0985dc00 0x563f096eb600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563f0996b400 msgr2=0x563f09970580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x563f0996b400 0x563f09970580 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563f09962000 msgr2=0x563f09970b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x563f09962000 0x563f09970b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563f0a45f400 msgr2=0x563f096ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x563f0a45f400 0x563f096ec100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563f0996b800 msgr2=0x563f09971600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x563f0996b800 0x563f09971600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563f0996a000 msgr2=0x563f09970000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x563f0996a000 0x563f09970000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563f0a45ec00 msgr2=0x563f096ea580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x563f0a45ec00 0x563f096ea580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563f0996ac00 msgr2=0x563f09971080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x563f0996ac00 0x563f09971080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 reap_dead start 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f6758a00640 1 -- 172.21.15.46:0/2916562059 reap_dead start 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563f0985d400 msgr2=0x563f096ed700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563f0985d400 0x563f096ed700 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x563f0960d200 tx=0x563f095d20c0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4800 msgr2=0x563f096ed180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4800 0x563f096ed180 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x563f0a461980 tx=0x563f0a46cc60 comp rx=0 tx=0).stop 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 shutdown_connections 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x563f0985d400 0x563f096ed700 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/2916562059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f0a2f4800 0x563f096ed180 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 >> 172.21.15.46:0/2916562059 conn(0x563f0a2f4400 msgr2=0x563f0955d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 shutdown_connections 2024-04-25T08:52:54.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/2916562059 wait complete. 2024-04-25T08:52:54.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/3715334007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f09531800 msgr2=0x563f096ebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f09531800 0x563f096ebb80 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x563f0a37b8c0 tx=0x563f0960ad80 comp rx=0 tx=0).stop 2024-04-25T08:52:54.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/3715334007 shutdown_connections 2024-04-25T08:52:54.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563f09531800 0x563f096ebb80 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f09531c00 0x563f096eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 --2- 172.21.15.46:0/3715334007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x563f0a2f4000 0x563f096eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/3715334007 >> 172.21.15.46:0/3715334007 conn(0x563f09530c00 msgr2=0x563f0955e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/3715334007 shutdown_connections 2024-04-25T08:52:54.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.318+0000 7f675a8ed8c0 1 -- 172.21.15.46:0/3715334007 wait complete. 2024-04-25T08:52:54.316 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.150406, result: 2024-04-25T08:52:54.316 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T08:52:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.410+0000 7fa40031a8c0 1 -- start start 2024-04-25T08:52:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.410+0000 7fa40031a8c0 1 Processor -- start 2024-04-25T08:52:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.410+0000 7fa40031a8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0cf53800 0x562e0d10db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.410+0000 7fa40031a8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0cf53400 0x562e0d10d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.410+0000 7fa40031a8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0cf53000 0x562e0d10d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.410+0000 7fa40031a8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0cff4180 con 0x562e0cf53000 2024-04-25T08:52:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.410+0000 7fa40031a8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0cfb2900 con 0x562e0cf53800 2024-04-25T08:52:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.410+0000 7fa40031a8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e0cfb2480 con 0x562e0cf53400 2024-04-25T08:52:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0cf53800 0x562e0d10db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0cf53000 0x562e0d10d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0cf53800 0x562e0d10db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:53936/0 (socket says 172.21.15.46:53936) 2024-04-25T08:52:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fda00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0cf53400 0x562e0d10d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 -- 172.21.15.46:0/258764096 learned_addr learned my addr 172.21.15.46:0/258764096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 -- 172.21.15.46:0/258764096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0cf53400 msgr2=0x562e0d10d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 --2- 172.21.15.46:0/258764096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0cf53400 0x562e0d10d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 -- 172.21.15.46:0/258764096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0cf53000 msgr2=0x562e0d10d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 --2- 172.21.15.46:0/258764096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0cf53000 0x562e0d10d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 -- 172.21.15.46:0/258764096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e0cf781e0 con 0x562e0cf53800 2024-04-25T08:52:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fee00640 1 --2- 172.21.15.46:0/258764096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0cf53000 0x562e0d10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 --2- 172.21.15.46:0/258764096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0cf53800 0x562e0d10db80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x562e0dda3860 tx=0x562e0d0163c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb515045658c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fb200640 1 -- 172.21.15.46:0/258764096 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562e0cf884e0 con 0x562e0cf53800 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fb200640 1 -- 172.21.15.46:0/258764096 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e0cf9c1c0 con 0x562e0cf53800 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fb200640 1 -- 172.21.15.46:0/258764096 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562e0cf88b60 con 0x562e0cf53800 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 -- 172.21.15.46:0/258764096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e0cf792c0 con 0x562e0cf53800 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 -- 172.21.15.46:0/258764096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562e0cf79a40 con 0x562e0cf53800 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 4 waiting for MDS map... 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fb200640 1 -- 172.21.15.46:0/258764096 <== mon.1 v2:172.21.15.82:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x562e0de6c600 con 0x562e0cf53800 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fb200640 1 -- 172.21.15.46:0/258764096 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 106) v1 ==== 2723+0+0 (secure 0 0 0) 0x562e0cfa9180 con 0x562e0cf53800 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 4 Got MDS map 106 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 4 datascan.main: Using metadata pool output 2024-04-25T08:52:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 4 datascan.main: connecting to RADOS... 2024-04-25T08:52:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 Processor -- start 2024-04-25T08:52:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 -- start start 2024-04-25T08:52:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6e800 0x562e0d10cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6ec00 0x562e0d10c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6f000 0x562e0d10e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0cfb2180 con 0x562e0de6ec00 2024-04-25T08:52:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0cff4d80 con 0x562e0de6e800 2024-04-25T08:52:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa40031a8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e0cff4f00 con 0x562e0de6f000 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6ec00 0x562e0d10c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6e800 0x562e0d10cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fda00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6f000 0x562e0d10e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6e800 0x562e0d10cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:53948/0 (socket says 172.21.15.46:53948) 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fee00640 1 -- 172.21.15.46:0/1281950191 learned_addr learned my addr 172.21.15.46:0/1281950191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.414+0000 7fa3fe400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6ec00 0x562e0d10c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46392/0 (socket says 172.21.15.46:46392) 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 -- 172.21.15.46:0/1281950191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6f000 msgr2=0x562e0d10e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 --2- 172.21.15.46:0/1281950191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6f000 0x562e0d10e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 -- 172.21.15.46:0/1281950191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6e800 msgr2=0x562e0d10cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 --2- 172.21.15.46:0/1281950191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6e800 0x562e0d10cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 -- 172.21.15.46:0/1281950191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e0cf79c20 con 0x562e0de6ec00 2024-04-25T08:52:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fda00640 1 --2- 172.21.15.46:0/1281950191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6f000 0x562e0d10e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 --2- 172.21.15.46:0/1281950191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6ec00 0x562e0d10c580 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x562e0de70630 tx=0x562e0d00cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43d8d371910bcf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3f8a00640 1 -- 172.21.15.46:0/1281950191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562e0cf88340 con 0x562e0de6ec00 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3f8a00640 1 -- 172.21.15.46:0/1281950191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e0cf9ce00 con 0x562e0de6ec00 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3f8a00640 1 -- 172.21.15.46:0/1281950191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562e0cf884e0 con 0x562e0de6ec00 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- 172.21.15.46:0/1281950191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6ec00 msgr2=0x562e0d10c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/1281950191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6ec00 0x562e0d10c580 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x562e0de70630 tx=0x562e0d00cfc0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- 172.21.15.46:0/1281950191 shutdown_connections 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/1281950191 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6f000 0x562e0d10e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/1281950191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6ec00 0x562e0d10c580 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/1281950191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6e800 0x562e0d10cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- 172.21.15.46:0/1281950191 >> 172.21.15.46:0/1281950191 conn(0x562e0cf52c00 msgr2=0x562e0cf81800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- 172.21.15.46:0/1281950191 shutdown_connections 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- 172.21.15.46:0/1281950191 wait complete. 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 Processor -- start 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- start start 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6e800 0x562e0d10cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6f000 0x562e0d10e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6f400 0x562e0d10e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0cff5680 con 0x562e0de6f000 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e0cff5200 con 0x562e0de6f400 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e0cff5080 con 0x562e0de6e800 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6f000 0x562e0d10e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6f000 0x562e0d10e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46400/0 (socket says 172.21.15.46:46400) 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 learned_addr learned my addr 172.21.15.46:0/3430237355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6f400 0x562e0d10e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fee00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6e800 0x562e0d10cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6e800 msgr2=0x562e0d10cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0de6e800 0x562e0d10cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6f400 msgr2=0x562e0d10e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0de6f400 0x562e0d10e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e0cf781e0 con 0x562e0de6f000 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3fe400640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6f000 0x562e0d10e100 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x562e0de71860 tx=0x562e0de84c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3f6c00640 1 -- 172.21.15.46:0/3430237355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562e0cf88b60 con 0x562e0de6f000 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3f6c00640 1 -- 172.21.15.46:0/3430237355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e0cf9cc40 con 0x562e0de6f000 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa3f6c00640 1 -- 172.21.15.46:0/3430237355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562e0cf88000 con 0x562e0de6f000 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562e0cf792c0 con 0x562e0de6f000 2024-04-25T08:52:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e0cf79e00 con 0x562e0de6f000 2024-04-25T08:52:54.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.418+0000 7fa40031a8c0 4 datascan.init: resolving metadata pool 17 2024-04-25T08:52:54.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3f6c00640 1 -- 172.21.15.46:0/3430237355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x562e0d292c00 con 0x562e0de6f000 2024-04-25T08:52:54.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3f6c00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x562e0dedac00 0x562e0d37a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3f6c00640 1 -- 172.21.15.46:0/3430237355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x562e0d296600 con 0x562e0de6f000 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa40031a8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa40031a8c0 4 datascan.main: resolving metadata pool 17 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x562e0de6f800 0x562e0d10ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0de6fc00 con 0x562e0de6f800 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3fee00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x562e0dedac00 0x562e0d37a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x562e0de6f800 0x562e0d10ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3fee00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x562e0dedac00 0x562e0d37a000 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x562e0de70600 tx=0x562e0de85e60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x562e0de6f800 0x562e0d10ec00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x562e0de6fc00 0x562e0d37a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd33400 con 0x562e0de6fc00 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.422+0000 7fa3fee00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x562e0de6fc00 0x562e0d37a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fee00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x562e0de6fc00 0x562e0d37a580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd33400 con 0x562e0de6f800 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 2 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x562e0dd33400 0x562e0d37ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd33000 con 0x562e0dd33400 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fe400640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x562e0dd33400 0x562e0d37ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fe400640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x562e0dd33400 0x562e0d37ab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd33000 con 0x562e0dd33400 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd33000 con 0x562e0de6fc00 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=149b] v123'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd33000 con 0x562e0de6f800 2024-04-25T08:52:54.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 3 ==== osd_op_reply(7 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x562e0dd33000 0x562e0d37b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd32800 con 0x562e0dd33000 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.426+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x562e0dd33000 0x562e0d37b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x562e0dd33000 0x562e0d37b080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== osd_op_reply(8 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562e0d006000 con 0x562e0dd33000 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x562e0dd32800 0x562e0d37b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd32c00 con 0x562e0dd32800 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fee00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x562e0dd32800 0x562e0d37b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fee00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x562e0dd32800 0x562e0d37b600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== osd_op_reply(9 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0dd32800 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd32c00 con 0x562e0de6fc00 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 3 ==== osd_op_reply(10 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd32c00 con 0x562e0de6f800 2024-04-25T08:52:54.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 4 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dd32c00 con 0x562e0dd33400 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 3 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fe400640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x562e0dd32c00 0x562e0d10c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0deda800 con 0x562e0dd32c00 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fe400640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x562e0dd32c00 0x562e0d10c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fe400640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x562e0dd32c00 0x562e0d10c580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== osd_op_reply(13 [pgnls start_epoch 123 out=77b] v122'10 uv10 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fe400640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x562e0deda800 0x562e0d37c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.430+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0dedb800 con 0x562e0deda800 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x562e0deda800 0x562e0d37c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x562e0deda800 0x562e0d37c680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0deda800 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fda00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x562e0dedb800 0x562e0d37bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dedb800 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fee00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x562e0dedb800 0x562e0d37bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fee00640 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x562e0dedb800 0x562e0d37bb80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.2 v2:172.21.15.46:6808/3102391403 1 ==== osd_op_reply(15 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562e0d006500 con 0x562e0dedb800 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33000 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.1 v2:172.21.15.46:6809/2552657765 2 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0dd33000 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33400 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 4 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6fc00 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 4 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6fc00 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 5 ==== osd_op_reply(19 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0deda800 2024-04-25T08:52:54.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.5 v2:172.21.15.82:6816/541310136 2 ==== osd_op_reply(20 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0deda800 2024-04-25T08:52:54.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.434+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32c00 2024-04-25T08:52:54.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== osd_op_reply(21 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0deda800 2024-04-25T08:52:54.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.5 v2:172.21.15.82:6816/541310136 3 ==== osd_op_reply(22 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x562e0d006000 con 0x562e0deda800 2024-04-25T08:52:54.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6f800 2024-04-25T08:52:54.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 5 ==== osd_op_reply(23 [pgnls start_epoch 123 out=74b] v120'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6f800 2024-04-25T08:52:54.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 6 ==== osd_op_reply(24 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32800 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.4 v2:172.21.15.82:6800/2293006647 2 ==== osd_op_reply(25 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0dd32800 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6fc00 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 6 ==== osd_op_reply(26 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33400 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 5 ==== osd_op_reply(27 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32800 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.4 v2:172.21.15.82:6800/2293006647 3 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0dd32800 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33400 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 6 ==== osd_op_reply(29 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32c00 2024-04-25T08:52:54.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 3 ==== osd_op_reply(30 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33000 2024-04-25T08:52:54.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.1 v2:172.21.15.46:6809/2552657765 3 ==== osd_op_reply(31 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562e0d006000 con 0x562e0dd33000 2024-04-25T08:52:54.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.438+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32c00 2024-04-25T08:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 4 ==== osd_op_reply(32 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T08:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38f800 con 0x562e0de6fc00 2024-04-25T08:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T08:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38fc00 con 0x562e0de6fc00 2024-04-25T08:52:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a2000 con 0x562e0de6fc00 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a2400 con 0x562e0de6fc00 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a2800 con 0x562e0de6fc00 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T08:52:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a2c00 con 0x562e0de6fc00 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a3000 con 0x562e0dd33000 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.1 v2:172.21.15.46:6809/2552657765 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006000 con 0x562e0dd33000 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:40 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a3400 con 0x562e0dedb800 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.2 v2:172.21.15.46:6808/3102391403 2 ==== osd_op_reply(40 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0dedb800 2024-04-25T08:52:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 10000000005.00000000 2024-04-25T08:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:41 17.9 17:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a3800 con 0x562e0de6fc00 2024-04-25T08:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 13 ==== osd_op_reply(41 10000000005.00000000 [omap-get-vals out=531b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+531 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T08:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a3c00 con 0x562e0deda800 2024-04-25T08:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.5 v2:172.21.15.82:6816/541310136 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006000 con 0x562e0deda800 2024-04-25T08:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T08:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T08:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T08:52:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.442+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a2000 con 0x562e0de6f800 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a2400 con 0x562e0de6fc00 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38f800 con 0x562e0dd33400 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-25T08:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38fc00 con 0x562e0dd33400 2024-04-25T08:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1561b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1561 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 20 datascan.scan_links: injected ino detected 2024-04-25T08:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T08:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T08:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T08:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T08:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T08:52:54.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38f400 con 0x562e0dd32c00 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38f000 con 0x562e0de6f800 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 8 ==== osd_op_reply(48 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38f000 con 0x562e0de6fc00 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 15 ==== osd_op_reply(49 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38f000 con 0x562e0de6f800 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 9 ==== osd_op_reply(50 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38f000 con 0x562e0dd33400 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 9 ==== osd_op_reply(51 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33400 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 10 ==== osd_op_reply(52 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.446+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6fc00 2024-04-25T08:52:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 16 ==== osd_op_reply(53 [pgnls start_epoch 123 out=149b] v123'41 uv41 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6f800 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 10 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33000 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.1 v2:172.21.15.46:6809/2552657765 5 ==== osd_op_reply(55 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562e0d006000 con 0x562e0dd33000 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32800 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.4 v2:172.21.15.82:6800/2293006647 4 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0dd32800 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6fc00 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 17 ==== osd_op_reply(57 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6f800 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 11 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33400 2024-04-25T08:52:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 11 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32c00 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 6 ==== osd_op_reply(60 [pgnls start_epoch 123 out=77b] v122'10 uv10 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0deda800 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.5 v2:172.21.15.82:6816/541310136 5 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0deda800 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dedb800 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.2 v2:172.21.15.46:6808/3102391403 3 ==== osd_op_reply(62 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562e0d006500 con 0x562e0dedb800 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33000 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.1 v2:172.21.15.46:6809/2552657765 6 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0dd33000 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33400 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 12 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.450+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6fc00 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 18 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6fc00 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 19 ==== osd_op_reply(66 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0deda800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.5 v2:172.21.15.82:6816/541310136 6 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006000 con 0x562e0deda800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32c00 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 7 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0deda800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.5 v2:172.21.15.82:6816/541310136 7 ==== osd_op_reply(69 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x562e0d006000 con 0x562e0deda800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6f800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 12 ==== osd_op_reply(70 [pgnls start_epoch 123 out=74b] v120'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6f800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 13 ==== osd_op_reply(71 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.4 v2:172.21.15.82:6800/2293006647 5 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0dd32800 2024-04-25T08:52:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0de6fc00 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 20 ==== osd_op_reply(73 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33400 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 13 ==== osd_op_reply(74 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32800 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.4 v2:172.21.15.82:6800/2293006647 6 ==== osd_op_reply(75 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006500 con 0x562e0dd32800 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.454+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33400 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 14 ==== osd_op_reply(76 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32c00 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 8 ==== osd_op_reply(77 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd33000 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.1 v2:172.21.15.46:6809/2552657765 7 ==== osd_op_reply(78 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562e0d006000 con 0x562e0dd33000 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e000 con 0x562e0dd32c00 2024-04-25T08:52:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 9 ==== osd_op_reply(79 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38ec00 con 0x562e0de6fc00 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e800 con 0x562e0de6fc00 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e400 con 0x562e0de6fc00 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a6000 con 0x562e0de6fc00 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T08:52:54.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a6400 con 0x562e0de6fc00 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a6800 con 0x562e0de6fc00 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a6c00 con 0x562e0dd33000 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.1 v2:172.21.15.46:6809/2552657765 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006000 con 0x562e0dd33000 2024-04-25T08:52:54.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.458+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- osd_op(unknown.0.0:87 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a7000 con 0x562e0dedb800 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.2 v2:172.21.15.46:6808/3102391403 4 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006500 con 0x562e0dedb800 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 10000000005.00000000 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:88 17.9 17:9258b67d:::10000000005.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a7400 con 0x562e0de6fc00 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 27 ==== osd_op_reply(88 10000000005.00000000 [omap-get-vals out=531b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+531 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a7800 con 0x562e0deda800 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.5 v2:172.21.15.82:6816/541310136 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006000 con 0x562e0deda800 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T08:52:54.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a7c00 con 0x562e0de6f800 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a6000 con 0x562e0de6fc00 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a6400 con 0x562e0dd33400 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a6800 con 0x562e0dd33400 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1561b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1561 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 20 datascan.scan_links: injected ino detected 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 20 datascan.scan_links: adding 0x10000000003 for future processing to fix dnfirst 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38ec00 con 0x562e0dd32c00 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.6 v2:172.21.15.82:6823/1178952145 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd32c00 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T08:52:54.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-25T08:52:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T08:52:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 10 datascan.scan_links: handling injected_ino 0x10000000003 2024-04-25T08:52:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-25T08:52:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e800 con 0x562e0dd33400 2024-04-25T08:52:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T08:52:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 20 datascan.scan_links: first is now 2 2024-04-25T08:52:54.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.462+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d38e400 con 0x562e0dd33400 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa3fe400640 1 -- 172.21.15.46:0/3430237355 <== osd.0 v2:172.21.15.46:6800/3448226631 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v123'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562e0d006780 con 0x562e0dd33400 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x10000000003 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 10 datascan.scan_links: updating inotable 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a6400 con 0x562e0de6fc00 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa3fee00640 1 -- 172.21.15.46:0/3430237355 <== osd.3 v2:172.21.15.46:6824/1647449001 29 ==== osd_op_reply(97 mds0_inotable [read 0~34 out=34b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x562e0d006500 con 0x562e0de6fc00 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 10 datascan.scan_links: updating snaptable 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- osd_op(unknown.0.0:98 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x562e0d3a6800 con 0x562e0de6f800 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 <== osd.7 v2:172.21.15.82:6808/1420997799 15 ==== osd_op_reply(98 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x562e0d006000 con 0x562e0de6f800 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x562e0dd33400 msgr2=0x562e0d37ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x562e0dd33400 0x562e0d37ab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x562e0dd33000 msgr2=0x562e0d37b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x562e0dd33000 0x562e0d37b080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x562e0dedb800 msgr2=0x562e0d37bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x562e0dedb800 0x562e0d37bb80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x562e0de6fc00 msgr2=0x562e0d37a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x562e0de6fc00 0x562e0d37a580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x562e0dd32800 msgr2=0x562e0d37b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x562e0dd32800 0x562e0d37b600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x562e0deda800 msgr2=0x562e0d37c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x562e0deda800 0x562e0d37c680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x562e0dd32c00 msgr2=0x562e0d10c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x562e0dd32c00 0x562e0d10c580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x562e0de6f800 msgr2=0x562e0d10ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x562e0de6f800 0x562e0d10ec00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 reap_dead start 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 reap_dead start 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 reap_dead start 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa3fda00640 1 -- 172.21.15.46:0/3430237355 reap_dead start 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x562e0dedac00 msgr2=0x562e0d37a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x562e0dedac00 0x562e0d37a000 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x562e0de70600 tx=0x562e0de85e60 comp rx=0 tx=0).stop 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6f000 msgr2=0x562e0d10e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6f000 0x562e0d10e100 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x562e0de71860 tx=0x562e0de84c00 comp rx=0 tx=0).stop 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 shutdown_connections 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x562e0dedac00 0x562e0d37a000 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/3430237355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0de6f000 0x562e0d10e100 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 >> 172.21.15.46:0/3430237355 conn(0x562e0cf52c00 msgr2=0x562e0cf81800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 shutdown_connections 2024-04-25T08:52:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/3430237355 wait complete. 2024-04-25T08:52:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/258764096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0cf53800 msgr2=0x562e0d10db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/258764096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0cf53800 0x562e0d10db80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x562e0dda3860 tx=0x562e0d0163c0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/258764096 shutdown_connections 2024-04-25T08:52:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/258764096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562e0cf53000 0x562e0d10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/258764096 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x562e0cf53400 0x562e0d10d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 --2- 172.21.15.46:0/258764096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562e0cf53800 0x562e0d10db80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/258764096 >> 172.21.15.46:0/258764096 conn(0x562e0cf53c00 msgr2=0x562e0cf81000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/258764096 shutdown_connections 2024-04-25T08:52:54.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.466+0000 7fa40031a8c0 1 -- 172.21.15.46:0/258764096 wait complete. 2024-04-25T08:52:54.469 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.153120, result: 2024-04-25T08:52:54.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:52:54.514 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:52:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-25T08:52:54.555 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:52:54.556 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:52:54.557 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.558 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:52:54.559 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:52:54.560 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.561 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.562 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-25T08:52:54.563 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-25T08:52:54.564 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = False 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-25T08:52:54.565 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2024-04-25T08:52:54.566 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:52:54.566 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:52:54.566 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:52:54.566 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T08:52:54.566 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T08:52:54.566 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:52:54.566 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:52:54.567 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:52:54.568 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on skipped message = False 2024-04-25T08:52:54.569 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T08:52:54.570 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.571 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:52:54.572 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:52:54.573 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.574 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.575 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = False 2024-04-25T08:52:54.576 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T08:52:54.577 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = True 2024-04-25T08:52:54.577 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:52:54.577 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:52:54.577 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:52:54.579 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:52:54.626 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:52:54.666 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-04-25T08:52:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-04-25T08:52:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:52:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-04-25T08:52:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-04-25T08:52:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-04-25T08:52:54.666 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-04-25T08:52:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on skipped message = False 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.670 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:52:54.671 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 20 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 20 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 10 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:52:54.672 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-04-25T08:52:54.673 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:52:54.674 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:52:54.675 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-04-25T08:52:54.676 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-04-25T08:52:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-04-25T08:52:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-04-25T08:52:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2024-04-25T08:52:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-04-25T08:52:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2024-04-25T08:52:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-04-25T08:52:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-04-25T08:52:54.677 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-04-25T08:52:54.677 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T08:52:54.678 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T08:52:54.679 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T08:52:54.680 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.46,172.21.15.82,[v2:172.21.15.82:3301,v1:172.21.15.82:6790]" 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on skipped message = False 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 9ca516a0-1b6e-4031-bb83-1b30eee026c8 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.681 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.682 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:52:54.683 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.684 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:52:54.685 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:52:54.686 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T08:52:54.687 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T08:52:54.688 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T08:52:54.688 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T08:52:54.688 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = True 2024-04-25T08:52:54.688 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T08:52:54.688 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = True 2024-04-25T08:52:54.688 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T08:52:54.688 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T08:52:54.688 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T08:52:54.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T08:52:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd9400640 1 Processor -- start 2024-04-25T08:52:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd9400640 1 -- start start 2024-04-25T08:52:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5bd4145310 0x7f5bd406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd9400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5bd415d090 0x7f5bd415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd9400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd40cd930 con 0x7f5bd406dc70 2024-04-25T08:52:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd9400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40cdbd0 con 0x7f5bd4145310 2024-04-25T08:52:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd9400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40cdeb0 con 0x7f5bd415d090 2024-04-25T08:52:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5bd4145310 0x7f5bd406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5bd4145310 0x7f5bd406e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46402/0 (socket says 172.21.15.46:46402) 2024-04-25T08:52:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd2800640 1 -- 172.21.15.46:0/4246310651 learned_addr learned my addr 172.21.15.46:0/4246310651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd2800640 1 -- 172.21.15.46:0/4246310651 >> v2:172.21.15.82:3300/0 conn(0x7f5bd415d090 msgr2=0x7f5bd415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd2800640 1 --2- 172.21.15.46:0/4246310651 >> v2:172.21.15.82:3300/0 conn(0x7f5bd415d090 0x7f5bd415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd2800640 1 -- 172.21.15.46:0/4246310651 >> v1:172.21.15.82:6789/0 conn(0x7f5bd406dc70 legacy=0x7f5bd40cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.886+0000 7f5bd2800640 1 -- 172.21.15.46:0/4246310651 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd40ce190 con 0x7f5bd4145310 2024-04-25T08:52:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd2800640 1 --2- 172.21.15.46:0/4246310651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd4145310 0x7f5bd406e0d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5bd406d060 tx=0x7f5bc401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32add8e39c6bbaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd1e00640 1 -- 172.21.15.46:0/4246310651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40a5760 con 0x7f5bd4145310 2024-04-25T08:52:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd1e00640 1 -- 172.21.15.46:0/4246310651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc40a5900 con 0x7f5bd4145310 2024-04-25T08:52:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd1e00640 1 -- 172.21.15.46:0/4246310651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40a5cd0 con 0x7f5bd4145310 2024-04-25T08:52:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd9400640 1 -- 172.21.15.46:0/4246310651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd4145310 msgr2=0x7f5bd406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd9400640 1 --2- 172.21.15.46:0/4246310651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd4145310 0x7f5bd406e0d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5bd406d060 tx=0x7f5bc401c9b0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd9400640 1 -- 172.21.15.46:0/4246310651 shutdown_connections 2024-04-25T08:52:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd9400640 1 --2- 172.21.15.46:0/4246310651 >> v2:172.21.15.82:3300/0 conn(0x7f5bd415d090 0x7f5bd415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd9400640 1 --2- 172.21.15.46:0/4246310651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd4145310 0x7f5bd406e0d0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd9400640 1 -- 172.21.15.46:0/4246310651 >> 172.21.15.46:0/4246310651 conn(0x7f5bd406ac10 msgr2=0x7f5bd406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd9400640 1 -- 172.21.15.46:0/4246310651 shutdown_connections 2024-04-25T08:52:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.890+0000 7f5bd9400640 1 -- 172.21.15.46:0/4246310651 wait complete. 2024-04-25T08:52:54.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 Processor -- start 2024-04-25T08:52:54.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 -- start start 2024-04-25T08:52:54.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bd4145310 0x7f5bd4141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bd415d090 0x7f5bd413ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd413b2b0 0x7f5bd4154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4168510 con 0x7f5bd413b2b0 2024-04-25T08:52:54.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40cf380 con 0x7f5bd415d090 2024-04-25T08:52:54.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4168690 con 0x7f5bd4145310 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd413b2b0 0x7f5bd4154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd2800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bd4145310 0x7f5bd4141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd413b2b0 0x7f5bd4154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46408/0 (socket says 172.21.15.46:46408) 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd3200640 1 -- 172.21.15.46:0/643856484 learned_addr learned my addr 172.21.15.46:0/643856484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bc9e00640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bd415d090 0x7f5bd413ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd3200640 1 -- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bd4145310 msgr2=0x7f5bd4141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd3200640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bd4145310 0x7f5bd4141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd3200640 1 -- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bd415d090 msgr2=0x7f5bd413ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd3200640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bd415d090 0x7f5bd413ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd3200640 1 -- 172.21.15.46:0/643856484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd413b040 con 0x7f5bd413b2b0 2024-04-25T08:52:54.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd3200640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd413b2b0 0x7f5bd4154d40 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00b390 tx=0x7f5bcc00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bcb400640 1 -- 172.21.15.46:0/643856484 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc015020 con 0x7f5bd413b2b0 2024-04-25T08:52:54.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bcb400640 1 -- 172.21.15.46:0/643856484 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc011830 con 0x7f5bd413b2b0 2024-04-25T08:52:54.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bcb400640 1 -- 172.21.15.46:0/643856484 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc011c20 con 0x7f5bd413b2b0 2024-04-25T08:52:54.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd40b4040 con 0x7f5bd413b2b0 2024-04-25T08:52:54.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.894+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd4143e00 con 0x7f5bd413b2b0 2024-04-25T08:52:54.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.898+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd4069010 con 0x7f5bd413b2b0 2024-04-25T08:52:54.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.898+0000 7f5bcb400640 1 -- 172.21.15.46:0/643856484 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bcc018020 con 0x7f5bd413b2b0 2024-04-25T08:52:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.898+0000 7f5bcb400640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5bb0066f90 0x7f5bb0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.898+0000 7f5bcb400640 1 -- 172.21.15.46:0/643856484 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f5bcc084f50 con 0x7f5bd413b2b0 2024-04-25T08:52:54.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.898+0000 7f5bd2800640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5bb0066f90 0x7f5bb0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.898+0000 7f5bd2800640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5bb0066f90 0x7f5bb0069450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40ae000 tx=0x7f5bc40a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:54.902+0000 7f5bcb400640 1 -- 172.21.15.46:0/643856484 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bcc01f020 con 0x7f5bd413b2b0 2024-04-25T08:52:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.102+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f5bd40cbe80 con 0x7f5bd413b2b0 2024-04-25T08:52:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.930+0000 7f5bcb400640 1 -- 172.21.15.46:0/643856484 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5bd413b040 con 0x7f5bd413b2b0 2024-04-25T08:52:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T08:52:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5bb0066f90 msgr2=0x7f5bb0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5bb0066f90 0x7f5bb0069450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5bc40ae000 tx=0x7f5bc40a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd413b2b0 msgr2=0x7f5bd4154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd413b2b0 0x7f5bd4154d40 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00b390 tx=0x7f5bcc00b850 comp rx=0 tx=0).stop 2024-04-25T08:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 shutdown_connections 2024-04-25T08:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5bb0066f90 0x7f5bb0069450 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bd413b2b0 0x7f5bd4154d40 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bd415d090 0x7f5bd413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 --2- 172.21.15.46:0/643856484 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5bd4145310 0x7f5bd4141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 >> 172.21.15.46:0/643856484 conn(0x7f5bd406ac10 msgr2=0x7f5bd415e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 shutdown_connections 2024-04-25T08:52:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:55.934+0000 7f5bd9400640 1 -- 172.21.15.46:0/643856484 wait complete. 2024-04-25T08:52:55.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.106+0000 7f0d97200640 1 Processor -- start 2024-04-25T08:52:56.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.106+0000 7f0d97200640 1 -- start start 2024-04-25T08:52:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d97200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0d90160f50 0x7f0d9015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d97200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d900adc50 0x7f0d900ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d97200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d90161330 con 0x7f0d9006ea00 2024-04-25T08:52:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d97200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9006d680 con 0x7f0d900adc50 2024-04-25T08:52:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d97200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d900ae810 con 0x7f0d90160f50 2024-04-25T08:52:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d87e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d900adc50 0x7f0d900ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d87e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0d900adc50 0x7f0d900ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46414/0 (socket says 172.21.15.46:46414) 2024-04-25T08:52:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d87e00640 1 -- 172.21.15.46:0/1786286340 learned_addr learned my addr 172.21.15.46:0/1786286340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d8fe00640 1 --2- 172.21.15.46:0/1786286340 >> v2:172.21.15.82:3300/0 conn(0x7f0d90160f50 0x7f0d9015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d87e00640 1 -- 172.21.15.46:0/1786286340 >> v2:172.21.15.82:3300/0 conn(0x7f0d90160f50 msgr2=0x7f0d9015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d87e00640 1 --2- 172.21.15.46:0/1786286340 >> v2:172.21.15.82:3300/0 conn(0x7f0d90160f50 0x7f0d9015f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d87e00640 1 -- 172.21.15.46:0/1786286340 >> v1:172.21.15.82:6789/0 conn(0x7f0d9006ea00 legacy=0x7f0d9006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d87e00640 1 -- 172.21.15.46:0/1786286340 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900aeaf0 con 0x7f0d900adc50 2024-04-25T08:52:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.110+0000 7f0d8fe00640 1 --2- 172.21.15.46:0/1786286340 >> v2:172.21.15.82:3300/0 conn(0x7f0d90160f50 0x7f0d9015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d87e00640 1 --2- 172.21.15.46:0/1786286340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 0x7f0d900ae0e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f0d90070b00 tx=0x7f0d7801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0da447b5026c764 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d8f400640 1 -- 172.21.15.46:0/1786286340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780ab5f0 con 0x7f0d900adc50 2024-04-25T08:52:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d8f400640 1 -- 172.21.15.46:0/1786286340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d780b0070 con 0x7f0d900adc50 2024-04-25T08:52:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d8f400640 1 -- 172.21.15.46:0/1786286340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780ab9b0 con 0x7f0d900adc50 2024-04-25T08:52:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 -- 172.21.15.46:0/1786286340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 msgr2=0x7f0d900ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 --2- 172.21.15.46:0/1786286340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 0x7f0d900ae0e0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f0d90070b00 tx=0x7f0d7801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 -- 172.21.15.46:0/1786286340 shutdown_connections 2024-04-25T08:52:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 --2- 172.21.15.46:0/1786286340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 0x7f0d900ae0e0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 --2- 172.21.15.46:0/1786286340 >> v2:172.21.15.82:3300/0 conn(0x7f0d90160f50 0x7f0d9015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 -- 172.21.15.46:0/1786286340 >> 172.21.15.46:0/1786286340 conn(0x7f0d900b12a0 msgr2=0x7f0d900b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 -- 172.21.15.46:0/1786286340 shutdown_connections 2024-04-25T08:52:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 -- 172.21.15.46:0/1786286340 wait complete. 2024-04-25T08:52:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 Processor -- start 2024-04-25T08:52:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.114+0000 7f0d97200640 1 -- start start 2024-04-25T08:52:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d97200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9006ea00 0x7f0d90146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d97200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 0x7f0d901fae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d97200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d90160f50 0x7f0d90147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d97200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9006e630 con 0x7f0d900adc50 2024-04-25T08:52:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d97200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9006f260 con 0x7f0d9006ea00 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d97200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9006e7b0 con 0x7f0d90160f50 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 0x7f0d901fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 0x7f0d901fae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46418/0 (socket says 172.21.15.46:46418) 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d87e00640 1 -- 172.21.15.46:0/1509464261 learned_addr learned my addr 172.21.15.46:0/1509464261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d8fe00640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9006ea00 0x7f0d90146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d95200640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d90160f50 0x7f0d90147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d87e00640 1 -- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d90160f50 msgr2=0x7f0d90147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d87e00640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d90160f50 0x7f0d90147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d87e00640 1 -- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9006ea00 msgr2=0x7f0d90146630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d87e00640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9006ea00 0x7f0d90146630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d87e00640 1 -- 172.21.15.46:0/1509464261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d90146d70 con 0x7f0d900adc50 2024-04-25T08:52:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d87e00640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 0x7f0d901fae40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f0d7801a110 tx=0x7f0d7801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d8cc00640 1 -- 172.21.15.46:0/1509464261 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780be5f0 con 0x7f0d900adc50 2024-04-25T08:52:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d8cc00640 1 -- 172.21.15.46:0/1509464261 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d780be790 con 0x7f0d900adc50 2024-04-25T08:52:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d901574c0 con 0x7f0d900adc50 2024-04-25T08:52:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d8cc00640 1 -- 172.21.15.46:0/1509464261 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780beb80 con 0x7f0d900adc50 2024-04-25T08:52:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d90147d50 con 0x7f0d900adc50 2024-04-25T08:52:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d90069080 con 0x7f0d900adc50 2024-04-25T08:52:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.118+0000 7f0d8cc00640 1 -- 172.21.15.46:0/1509464261 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d780b43f0 con 0x7f0d900adc50 2024-04-25T08:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.122+0000 7f0d8cc00640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d6c066f60 0x7f0d6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.122+0000 7f0d8cc00640 1 -- 172.21.15.46:0/1509464261 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f0d7811fb30 con 0x7f0d900adc50 2024-04-25T08:52:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.122+0000 7f0d8fe00640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d6c066f60 0x7f0d6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.122+0000 7f0d8fe00640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d6c066f60 0x7f0d6c069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0d9014aa30 tx=0x7f0d80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.126+0000 7f0d8cc00640 1 -- 172.21.15.46:0/1509464261 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d780c4060 con 0x7f0d900adc50 2024-04-25T08:52:56.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.318+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d90151d50 con 0x7f0d900adc50 2024-04-25T08:52:56.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d8cc00640 1 -- 172.21.15.46:0/1509464261 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f0d780160d0 con 0x7f0d900adc50 2024-04-25T08:52:56.312 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:56.312 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:55.936612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"a","rank":0,"incarnation":109,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6834/1724167790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1724167790},{"type":"v1","addr":"172.21.15.46:6834","nonce":1724167790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:52:56.312 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-04-25T08:52:56.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d6c066f60 msgr2=0x7f0d6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d6c066f60 0x7f0d6c069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0d9014aa30 tx=0x7f0d80008040 comp rx=0 tx=0).stop 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 msgr2=0x7f0d901fae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 0x7f0d901fae40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f0d7801a110 tx=0x7f0d7801ad20 comp rx=0 tx=0).stop 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 shutdown_connections 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0d6c066f60 0x7f0d6c069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f0d90160f50 0x7f0d90147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d900adc50 0x7f0d901fae40 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 --2- 172.21.15.46:0/1509464261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9006ea00 0x7f0d90146630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 >> 172.21.15.46:0/1509464261 conn(0x7f0d900b12a0 msgr2=0x7f0d900b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 shutdown_connections 2024-04-25T08:52:56.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:56.322+0000 7f0d97200640 1 -- 172.21.15.46:0/1509464261 wait complete. 2024-04-25T08:52:56.737 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:52:28.147634+0000', 'modified': '2024-04-25T08:52:55.936612+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9393}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9393': {'gid': 9393, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6834/1724167790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 1724167790}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 1724167790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:52:56.738 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9393:up:replay 2024-04-25T08:52:57.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.898+0000 7f0473000640 1 Processor -- start 2024-04-25T08:52:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.898+0000 7f0473000640 1 -- start start 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f0473000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f046c0ae360 0x7f046c15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f0473000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046c161330 con 0x7f046c160f50 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f0473000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f046c0ae810 con 0x7f046c06ea00 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f0473000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f046c06d680 con 0x7f046c0ae360 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f046be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f046c160f50 0x7f046c0adc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45192/0 (socket says 172.21.15.46:45192) 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f046be00640 1 -- 172.21.15.46:0/3486116809 learned_addr learned my addr 172.21.15.46:0/3486116809 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f0463400640 1 --2- 172.21.15.46:0/3486116809 >> v2:172.21.15.82:3300/0 conn(0x7f046c0ae360 0x7f046c15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f046b400640 1 -- 172.21.15.46:0/3486116809 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678051326 0 0) 0x7f046c161330 con 0x7f046c160f50 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f046b400640 1 -- 172.21.15.46:0/3486116809 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0450003530 con 0x7f046c160f50 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f0463400640 1 -- 172.21.15.46:0/3486116809 >> v1:172.21.15.82:6789/0 conn(0x7f046c06ea00 legacy=0x7f046c06db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f0463400640 1 -- 172.21.15.46:0/3486116809 >> v1:172.21.15.46:6789/0 conn(0x7f046c160f50 legacy=0x7f046c0adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.902+0000 7f0463400640 1 -- 172.21.15.46:0/3486116809 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c0aeaf0 con 0x7f046c0ae360 2024-04-25T08:52:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0463400640 1 --2- 172.21.15.46:0/3486116809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c0ae360 0x7f046c15f1a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f04540097f0 tx=0x7f045401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cef7372854ba7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f046b400640 1 -- 172.21.15.46:0/3486116809 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04540aa5f0 con 0x7f046c0ae360 2024-04-25T08:52:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f046b400640 1 -- 172.21.15.46:0/3486116809 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04540af070 con 0x7f046c0ae360 2024-04-25T08:52:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f046b400640 1 -- 172.21.15.46:0/3486116809 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04540aa990 con 0x7f046c0ae360 2024-04-25T08:52:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0473000640 1 -- 172.21.15.46:0/3486116809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c0ae360 msgr2=0x7f046c15f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0473000640 1 --2- 172.21.15.46:0/3486116809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c0ae360 0x7f046c15f1a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f04540097f0 tx=0x7f045401c470 comp rx=0 tx=0).stop 2024-04-25T08:52:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0473000640 1 -- 172.21.15.46:0/3486116809 shutdown_connections 2024-04-25T08:52:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0473000640 1 --2- 172.21.15.46:0/3486116809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c0ae360 0x7f046c15f1a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0473000640 1 -- 172.21.15.46:0/3486116809 >> 172.21.15.46:0/3486116809 conn(0x7f046c0b12a0 msgr2=0x7f046c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0473000640 1 -- 172.21.15.46:0/3486116809 shutdown_connections 2024-04-25T08:52:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0473000640 1 -- 172.21.15.46:0/3486116809 wait complete. 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0473000640 1 Processor -- start 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.906+0000 7f0473000640 1 -- start start 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0473000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c06ea00 0x7f046c13c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0473000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f046c0ae360 0x7f046c0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0473000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f046c160f50 0x7f046c0ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0473000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c06f220 con 0x7f046c0ae360 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0473000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f046c06e630 con 0x7f046c06ea00 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0473000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f046c142e40 con 0x7f046c160f50 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0463400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f046c0ae360 0x7f046c0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0463400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f046c0ae360 0x7f046c0cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46424/0 (socket says 172.21.15.46:46424) 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0463400640 1 -- 172.21.15.46:0/301365358 learned_addr learned my addr 172.21.15.46:0/301365358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f046be00640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c06ea00 0x7f046c13c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0471000640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f046c160f50 0x7f046c0ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0463400640 1 -- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f046c160f50 msgr2=0x7f046c0ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0463400640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f046c160f50 0x7f046c0ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0463400640 1 -- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c06ea00 msgr2=0x7f046c13c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0463400640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c06ea00 0x7f046c13c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0463400640 1 -- 172.21.15.46:0/301365358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f046c143090 con 0x7f046c0ae360 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f046be00640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c06ea00 0x7f046c13c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0463400640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f046c0ae360 0x7f046c0cd6c0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f045401a0a0 tx=0x7f045401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0468c00640 1 -- 172.21.15.46:0/301365358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0454006e20 con 0x7f046c0ae360 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0468c00640 1 -- 172.21.15.46:0/301365358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0454004230 con 0x7f046c0ae360 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0468c00640 1 -- 172.21.15.46:0/301365358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0454003620 con 0x7f046c0ae360 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f046c143370 con 0x7f046c0ae360 2024-04-25T08:52:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046c13bcf0 con 0x7f046c0ae360 2024-04-25T08:52:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f046c069080 con 0x7f046c0ae360 2024-04-25T08:52:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.910+0000 7f0468c00640 1 -- 172.21.15.46:0/301365358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04540037c0 con 0x7f046c0ae360 2024-04-25T08:52:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.914+0000 7f0468c00640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0448066f60 0x7f0448069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.914+0000 7f0468c00640 1 -- 172.21.15.46:0/301365358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f045411f940 con 0x7f046c0ae360 2024-04-25T08:52:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.914+0000 7f046be00640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0448066f60 0x7f0448069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.914+0000 7f046be00640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0448066f60 0x7f0448069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f045c001570 tx=0x7f045c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:57.918+0000 7f0468c00640 1 -- 172.21.15.46:0/301365358 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04540c5030 con 0x7f046c0ae360 2024-04-25T08:52:58.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.114+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f046c0b3a00 con 0x7f046c0ae360 2024-04-25T08:52:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.114+0000 7f0468c00640 1 -- 172.21.15.46:0/301365358 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f0454016090 con 0x7f046c0ae360 2024-04-25T08:52:58.106 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:58.106 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:56.947862+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"a","rank":0,"incarnation":109,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6834/1724167790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1724167790},{"type":"v1","addr":"172.21.15.46:6834","nonce":1724167790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:52:58.107 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 111 2024-04-25T08:52:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0448066f60 msgr2=0x7f0448069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0448066f60 0x7f0448069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f045c001570 tx=0x7f045c009040 comp rx=0 tx=0).stop 2024-04-25T08:52:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f046c0ae360 msgr2=0x7f046c0cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f046c0ae360 0x7f046c0cd6c0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f045401a0a0 tx=0x7f045401acb0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 shutdown_connections 2024-04-25T08:52:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f0448066f60 0x7f0448069420 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f046c160f50 0x7f046c0ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f046c0ae360 0x7f046c0cd6c0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 --2- 172.21.15.46:0/301365358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f046c06ea00 0x7f046c13c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 >> 172.21.15.46:0/301365358 conn(0x7f046c0b12a0 msgr2=0x7f046c073850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 shutdown_connections 2024-04-25T08:52:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:58.118+0000 7f0473000640 1 -- 172.21.15.46:0/301365358 wait complete. 2024-04-25T08:52:58.531 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:52:28.147634+0000', 'modified': '2024-04-25T08:52:56.947862+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9393}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9393': {'gid': 9393, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6834/1724167790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 1724167790}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 1724167790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:52:58.531 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9393:up:rejoin 2024-04-25T08:52:59.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:59.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4dd400640 1 Processor -- start 2024-04-25T08:52:59.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4dd400640 1 -- start start 2024-04-25T08:52:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4dd400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff4d8159230 0x7ff4d815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4dd400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cbeb0 0x7ff4d80cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4dd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4d80cda80 0x7ff4d815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4dd400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d80cc700 con 0x7ff4d80cda80 2024-04-25T08:52:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4dd400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d80cc9e0 con 0x7ff4d8159230 2024-04-25T08:52:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4dd400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d80cccc0 con 0x7ff4d80cbeb0 2024-04-25T08:52:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4d7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4d80cda80 0x7ff4d815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4d7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4d80cda80 0x7ff4d815f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59938/0 (socket says 172.21.15.46:59938) 2024-04-25T08:52:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4d7200640 1 -- 172.21.15.46:0/1118148425 learned_addr learned my addr 172.21.15.46:0/1118148425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4cfe00640 1 --2- 172.21.15.46:0/1118148425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cbeb0 0x7ff4d80cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4d6800640 1 --2- 172.21.15.46:0/1118148425 >> v2:172.21.15.82:3300/0 conn(0x7ff4d8159230 0x7ff4d815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4d7200640 1 -- 172.21.15.46:0/1118148425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cbeb0 msgr2=0x7ff4d80cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4d7200640 1 --2- 172.21.15.46:0/1118148425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cbeb0 0x7ff4d80cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4d7200640 1 -- 172.21.15.46:0/1118148425 >> v2:172.21.15.82:3300/0 conn(0x7ff4d8159230 msgr2=0x7ff4d815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4d7200640 1 --2- 172.21.15.46:0/1118148425 >> v2:172.21.15.82:3300/0 conn(0x7ff4d8159230 0x7ff4d815bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.694+0000 7ff4d7200640 1 -- 172.21.15.46:0/1118148425 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d80ccfa0 con 0x7ff4d80cda80 2024-04-25T08:52:59.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4d7200640 1 --2- 172.21.15.46:0/1118148425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d80cda80 0x7ff4d815f3e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8072a90 tx=0x7ff4d001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d14023d04f038e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4d5e00640 1 -- 172.21.15.46:0/1118148425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00ac020 con 0x7ff4d80cda80 2024-04-25T08:52:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4d5e00640 1 -- 172.21.15.46:0/1118148425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d00a6440 con 0x7ff4d80cda80 2024-04-25T08:52:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4d5e00640 1 -- 172.21.15.46:0/1118148425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00a6810 con 0x7ff4d80cda80 2024-04-25T08:52:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4dd400640 1 -- 172.21.15.46:0/1118148425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d80cda80 msgr2=0x7ff4d815f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4dd400640 1 --2- 172.21.15.46:0/1118148425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d80cda80 0x7ff4d815f3e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8072a90 tx=0x7ff4d001c6a0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4dd400640 1 -- 172.21.15.46:0/1118148425 shutdown_connections 2024-04-25T08:52:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4dd400640 1 --2- 172.21.15.46:0/1118148425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d80cda80 0x7ff4d815f3e0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4dd400640 1 --2- 172.21.15.46:0/1118148425 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cbeb0 0x7ff4d80cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4dd400640 1 --2- 172.21.15.46:0/1118148425 >> v2:172.21.15.82:3300/0 conn(0x7ff4d8159230 0x7ff4d815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4dd400640 1 -- 172.21.15.46:0/1118148425 >> 172.21.15.46:0/1118148425 conn(0x7ff4d80b12a0 msgr2=0x7ff4d80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4dd400640 1 -- 172.21.15.46:0/1118148425 shutdown_connections 2024-04-25T08:52:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.698+0000 7ff4dd400640 1 -- 172.21.15.46:0/1118148425 wait complete. 2024-04-25T08:52:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 Processor -- start 2024-04-25T08:52:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 -- start start 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80cbeb0 0x7ff4d815b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cda80 0x7ff4d813ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8159230 0x7ff4d80a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d815d200 con 0x7ff4d8159230 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d80cc4a0 con 0x7ff4d80cbeb0 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d815d380 con 0x7ff4d80cda80 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8159230 0x7ff4d80a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d6800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80cbeb0 0x7ff4d815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8159230 0x7ff4d80a8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59952/0 (socket says 172.21.15.46:59952) 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d7200640 1 -- 172.21.15.46:0/4018480616 learned_addr learned my addr 172.21.15.46:0/4018480616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:52:59.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4cfe00640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cda80 0x7ff4d813ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d7200640 1 -- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cda80 msgr2=0x7ff4d813ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d7200640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cda80 0x7ff4d813ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d7200640 1 -- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80cbeb0 msgr2=0x7ff4d815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d7200640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80cbeb0 0x7ff4d815b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d7200640 1 -- 172.21.15.46:0/4018480616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d813d350 con 0x7ff4d8159230 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4d7200640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8159230 0x7ff4d80a8600 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff4d001cf40 tx=0x7ff4d0019440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4cea00640 1 -- 172.21.15.46:0/4018480616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00a6a80 con 0x7ff4d8159230 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4cea00640 1 -- 172.21.15.46:0/4018480616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d00a6c20 con 0x7ff4d8159230 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d80a8b40 con 0x7ff4d8159230 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4cea00640 1 -- 172.21.15.46:0/4018480616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00bb440 con 0x7ff4d8159230 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.702+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d815a990 con 0x7ff4d8159230 2024-04-25T08:52:59.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.706+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d8069080 con 0x7ff4d8159230 2024-04-25T08:52:59.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.706+0000 7ff4cea00640 1 -- 172.21.15.46:0/4018480616 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4d00af020 con 0x7ff4d8159230 2024-04-25T08:52:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.706+0000 7ff4cea00640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff4b4066f60 0x7ff4b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.706+0000 7ff4cea00640 1 -- 172.21.15.46:0/4018480616 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff4d011f170 con 0x7ff4d8159230 2024-04-25T08:52:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.706+0000 7ff4d6800640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff4b4066f60 0x7ff4b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.706+0000 7ff4d6800640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff4b4066f60 0x7ff4b4069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff4d806f580 tx=0x7ff4c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.710+0000 7ff4cea00640 1 -- 172.21.15.46:0/4018480616 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4d00c1060 con 0x7ff4d8159230 2024-04-25T08:52:59.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4d80723d0 con 0x7ff4d8159230 2024-04-25T08:52:59.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4cea00640 1 -- 172.21.15.46:0/4018480616 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7ff4d00165e0 con 0x7ff4d8159230 2024-04-25T08:52:59.901 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:52:59.901 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:57.953462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6834/1724167790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1724167790},{"type":"v1","addr":"172.21.15.46:6834","nonce":1724167790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:52:59.902 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 112 2024-04-25T08:52:59.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff4b4066f60 msgr2=0x7ff4b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff4b4066f60 0x7ff4b4069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ff4d806f580 tx=0x7ff4c8008040 comp rx=0 tx=0).stop 2024-04-25T08:52:59.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8159230 msgr2=0x7ff4d80a8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8159230 0x7ff4d80a8600 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff4d001cf40 tx=0x7ff4d0019440 comp rx=0 tx=0).stop 2024-04-25T08:52:59.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 shutdown_connections 2024-04-25T08:52:59.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ff4b4066f60 0x7ff4b4069420 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4d8159230 0x7ff4d80a8600 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ff4d80cda80 0x7ff4d813ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 --2- 172.21.15.46:0/4018480616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d80cbeb0 0x7ff4d815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 >> 172.21.15.46:0/4018480616 conn(0x7ff4d80b12a0 msgr2=0x7ff4d8141cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:59.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 shutdown_connections 2024-04-25T08:52:59.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:52:59.910+0000 7ff4dd400640 1 -- 172.21.15.46:0/4018480616 wait complete. 2024-04-25T08:53:00.326 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:52:28.147634+0000', 'modified': '2024-04-25T08:52:57.953462+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9393}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9393': {'gid': 9393, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6834/1724167790', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 1724167790}, {'type': 'v1', 'addr': '172.21.15.46:6834', 'nonce': 1724167790}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:53:00.326 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:53:00.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:53:00.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf58800640 1 Processor -- start 2024-04-25T08:53:00.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf58800640 1 -- start start 2024-04-25T08:53:00.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf58800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf5015a750 0x7fbf5006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf58800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbf500cc9e0 0x7fbf5006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf58800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5006d5e0 con 0x7fbf500aed10 2024-04-25T08:53:00.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf58800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5015ab30 con 0x7fbf500cc9e0 2024-04-25T08:53:00.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf58800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf500ce4f0 con 0x7fbf5015a750 2024-04-25T08:53:00.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf55400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbf500aed10 0x7fbf500b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60198/0 (socket says 172.21.15.46:60198) 2024-04-25T08:53:00.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf55400640 1 -- 172.21.15.46:0/617938397 learned_addr learned my addr 172.21.15.46:0/617938397 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf56800640 1 --2- 172.21.15.46:0/617938397 >> v2:172.21.15.82:3300/0 conn(0x7fbf500cc9e0 0x7fbf5006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf55e00640 1 --2- 172.21.15.46:0/617938397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf5015a750 0x7fbf5006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf54a00640 1 -- 172.21.15.46:0/617938397 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790977769 0 0) 0x7fbf5006d5e0 con 0x7fbf500aed10 2024-04-25T08:53:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf54a00640 1 -- 172.21.15.46:0/617938397 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf38003530 con 0x7fbf500aed10 2024-04-25T08:53:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf56800640 1 -- 172.21.15.46:0/617938397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf5015a750 msgr2=0x7fbf5006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf56800640 1 --2- 172.21.15.46:0/617938397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf5015a750 0x7fbf5006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf56800640 1 -- 172.21.15.46:0/617938397 >> v1:172.21.15.46:6789/0 conn(0x7fbf500aed10 legacy=0x7fbf500b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf56800640 1 -- 172.21.15.46:0/617938397 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500ce720 con 0x7fbf500cc9e0 2024-04-25T08:53:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.482+0000 7fbf55e00640 1 --2- 172.21.15.46:0/617938397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf5015a750 0x7fbf5006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:00.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf56800640 1 --2- 172.21.15.46:0/617938397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf500cc9e0 0x7fbf5006f150 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009d90 tx=0x7fbf4c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29b22908b87fe406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf54a00640 1 -- 172.21.15.46:0/617938397 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0b2070 con 0x7fbf500cc9e0 2024-04-25T08:53:00.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf54a00640 1 -- 172.21.15.46:0/617938397 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4c0a5920 con 0x7fbf500cc9e0 2024-04-25T08:53:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf54a00640 1 -- 172.21.15.46:0/617938397 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0a5cf0 con 0x7fbf500cc9e0 2024-04-25T08:53:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf58800640 1 -- 172.21.15.46:0/617938397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf500cc9e0 msgr2=0x7fbf5006f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf58800640 1 --2- 172.21.15.46:0/617938397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf500cc9e0 0x7fbf5006f150 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009d90 tx=0x7fbf4c01c970 comp rx=0 tx=0).stop 2024-04-25T08:53:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf58800640 1 -- 172.21.15.46:0/617938397 shutdown_connections 2024-04-25T08:53:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf58800640 1 --2- 172.21.15.46:0/617938397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf500cc9e0 0x7fbf5006f150 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf58800640 1 --2- 172.21.15.46:0/617938397 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf5015a750 0x7fbf5006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf58800640 1 -- 172.21.15.46:0/617938397 >> 172.21.15.46:0/617938397 conn(0x7fbf500b12a0 msgr2=0x7fbf500b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf58800640 1 -- 172.21.15.46:0/617938397 shutdown_connections 2024-04-25T08:53:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.486+0000 7fbf58800640 1 -- 172.21.15.46:0/617938397 wait complete. 2024-04-25T08:53:00.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 Processor -- start 2024-04-25T08:53:00.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 -- start start 2024-04-25T08:53:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf500aed10 0x7fbf501592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf500cc9e0 0x7fbf50167230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf5015a750 0x7fbf5015acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5006db00 con 0x7fbf500aed10 2024-04-25T08:53:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5006d270 con 0x7fbf5015a750 2024-04-25T08:53:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5006dc80 con 0x7fbf500cc9e0 2024-04-25T08:53:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf500aed10 0x7fbf501592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf500aed10 0x7fbf501592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59960/0 (socket says 172.21.15.46:59960) 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf55e00640 1 -- 172.21.15.46:0/2208118753 learned_addr learned my addr 172.21.15.46:0/2208118753 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf55e00640 1 -- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf500cc9e0 msgr2=0x7fbf50167230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf55e00640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf500cc9e0 0x7fbf50167230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf55e00640 1 -- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf5015a750 msgr2=0x7fbf5015acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf55e00640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf5015a750 0x7fbf5015acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf55e00640 1 -- 172.21.15.46:0/2208118753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf50169130 con 0x7fbf500aed10 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf55e00640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf500aed10 0x7fbf501592d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9b0 tx=0x7fbf4400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf4a000640 1 -- 172.21.15.46:0/2208118753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44017070 con 0x7fbf500aed10 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf4a000640 1 -- 172.21.15.46:0/2208118753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4400ca00 con 0x7fbf500aed10 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf4a000640 1 -- 172.21.15.46:0/2208118753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400cdd0 con 0x7fbf500aed10 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 -- 172.21.15.46:0/2208118753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf500acc30 con 0x7fbf500aed10 2024-04-25T08:53:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.490+0000 7fbf58800640 1 -- 172.21.15.46:0/2208118753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf500ad1f0 con 0x7fbf500aed10 2024-04-25T08:53:00.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf4a000640 1 -- 172.21.15.46:0/2208118753 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf4401a070 con 0x7fbf500aed10 2024-04-25T08:53:00.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf4a000640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf34066f60 0x7fbf34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf4a000640 1 -- 172.21.15.46:0/2208118753 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fbf44085200 con 0x7fbf500aed10 2024-04-25T08:53:00.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf5bc1e000 1 Processor -- start 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf55400640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf34066f60 0x7fbf34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf5bc1e000 1 -- start start 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf55400640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf34066f60 0x7fbf34069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbf40007fd0 tx=0x7fbf40007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5ae920 0x55e06b5aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5af240 0x55e06b5649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 0x55e06b565310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06b44c5f0 con 0x55e06b5ae920 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06b3b3da0 con 0x55e06b5af240 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e06b33c150 con 0x55e06b564f30 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.494+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5ae920 0x55e06b5aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5ae920 0x55e06b5aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59964/0 (socket says 172.21.15.46:59964) 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55e00640 1 -- 172.21.15.46:0/774228455 learned_addr learned my addr 172.21.15.46:0/774228455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:00.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55400640 1 --2- 172.21.15.46:0/774228455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5af240 0x55e06b5649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf56800640 1 --2- 172.21.15.46:0/774228455 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 0x55e06b565310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55e00640 1 -- 172.21.15.46:0/774228455 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 msgr2=0x55e06b565310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55e00640 1 --2- 172.21.15.46:0/774228455 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 0x55e06b565310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55e00640 1 -- 172.21.15.46:0/774228455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5af240 msgr2=0x55e06b5649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55e00640 1 --2- 172.21.15.46:0/774228455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5af240 0x55e06b5649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55e00640 1 -- 172.21.15.46:0/774228455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e06b355ca0 con 0x55e06b5ae920 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55e00640 1 --2- 172.21.15.46:0/774228455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5ae920 0x55e06b5aed00 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbf4405dab0 tx=0x7fbf44055980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73dc1a691f5fd5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf3ea00640 1 -- 172.21.15.46:0/774228455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4406d020 con 0x55e06b5ae920 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf3ea00640 1 -- 172.21.15.46:0/774228455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4407ce50 con 0x55e06b5ae920 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf3ea00640 1 -- 172.21.15.46:0/774228455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44021040 con 0x55e06b5ae920 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/774228455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5ae920 msgr2=0x55e06b5aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/774228455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5ae920 0x55e06b5aed00 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbf4405dab0 tx=0x7fbf44055980 comp rx=0 tx=0).stop 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/774228455 shutdown_connections 2024-04-25T08:53:00.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/774228455 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 0x55e06b565310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/774228455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5af240 0x55e06b5649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/774228455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5ae920 0x55e06b5aed00 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/774228455 >> 172.21.15.46:0/774228455 conn(0x55e06b56b2a0 msgr2=0x55e06b5632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/774228455 shutdown_connections 2024-04-25T08:53:00.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/774228455 wait complete. 2024-04-25T08:53:00.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 -- start start 2024-04-25T08:53:00.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 Processor -- start 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b564f30 0x55e06b5b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5ae920 0x55e06b5b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b568aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06b36e990 con 0x55e06b5af240 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06b3afe40 con 0x55e06b564f30 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e06b33f900 con 0x55e06b5ae920 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf5bc1e000 10 client.0 _ll_register_callbacks cb 0x55e06b4671c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b568aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b568aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59970/0 (socket says 172.21.15.46:59970) 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf56800640 1 -- 172.21.15.46:0/2653865376 learned_addr learned my addr 172.21.15.46:0/2653865376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55400640 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5ae920 0x55e06b5b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf55e00640 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b564f30 0x55e06b5b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.498+0000 7fbf56800640 1 -- 172.21.15.46:0/2653865376 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5ae920 msgr2=0x55e06b5b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf56800640 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5ae920 0x55e06b5b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf56800640 1 -- 172.21.15.46:0/2653865376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b564f30 msgr2=0x55e06b5b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf56800640 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b564f30 0x55e06b5b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf56800640 1 -- 172.21.15.46:0/2653865376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e06b36c630 con 0x55e06b5af240 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf56800640 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b568aa0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c01c9b0 tx=0x7fbf4c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27491a63c3e1877 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 1 -- 172.21.15.46:0/2653865376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0b2040 con 0x55e06b5af240 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 1 -- 172.21.15.46:0/2653865376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fbf4c0a5720 con 0x55e06b5af240 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/2653865376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf4407ce50 con 0x55e06b5af240 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 1 -- 172.21.15.46:0/2653865376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf4c0c8050 con 0x55e06b5af240 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 10 client.9480 fetch_fsmap learned FSMap version 112 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/2653865376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55e06b33c3a0 con 0x55e06b5af240 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 1 -- 172.21.15.46:0/2653865376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4407ce50 con 0x55e06b5af240 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 1 -- 172.21.15.46:0/2653865376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbf4c0cb020 con 0x55e06b5af240 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 10 client.9480 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 10 client.9480 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 20 client.9480 populate_metadata read hostname 'smithi046' 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] conn(0x55e06b56a160 0x55e06b5bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 4 client.9480 mds_command: new command op to 9393 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/2653865376 --> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e06b34fa50 con 0x55e06b56a160 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf55e00640 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] conn(0x55e06b56a160 0x55e06b5bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf55e00640 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] conn(0x55e06b56a160 0x55e06b5bfce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e22a3846fc265d server_cookie=268fcfbebce1336b in_seq=0 out_seq=0 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 10 client.9480 ms_handle_connect on v2:172.21.15.46:6832/1724167790 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 1 -- 172.21.15.46:0/2653865376 <== mds.0 v2:172.21.15.46:6832/1724167790 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e06b34fa50 con 0x55e06b56a160 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 10 client.9480 handle_command_reply: tid=0 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 client.9480 shutdown 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/2653865376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 msgr2=0x55e06b568aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b568aa0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c01c9b0 tx=0x7fbf4c019710 comp rx=0 tx=0).stop 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/2653865376 shutdown_connections 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] conn(0x55e06b56a160 0x55e06b5bfce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b568aa0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5ae920 0x55e06b5b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/2653865376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b564f30 0x55e06b5b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf33400640 0 client.9480 ms_handle_reset on v2:172.21.15.46:6832/1724167790 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/2653865376 >> 172.21.15.46:0/2653865376 conn(0x55e06b56b2a0 msgr2=0x55e06b5639e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/2653865376 shutdown_connections 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/2653865376 wait complete. 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.502+0000 7fbf5bc1e000 20 client.9480 trim_cache size 0 max 16384 2024-04-25T08:53:00.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.518+0000 7fbf5bc1e000 1 Processor -- start 2024-04-25T08:53:00.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.518+0000 7fbf5bc1e000 1 -- start start 2024-04-25T08:53:00.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.518+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 0x55e06b609bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.518+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 0x55e06b60a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.518+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b60a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.518+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf50202550 con 0x55e06b5af240 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.518+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf501586e0 con 0x55e06b5ae920 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.518+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf50201fd0 con 0x55e06b564f30 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.518+0000 7fbf56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b60a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 0x55e06b609bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 0x55e06b60a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf56800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b60a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59974/0 (socket says 172.21.15.46:59974) 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf56800640 1 -- 172.21.15.46:0/917961503 learned_addr learned my addr 172.21.15.46:0/917961503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 -- 172.21.15.46:0/917961503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 msgr2=0x55e06b60a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 --2- 172.21.15.46:0/917961503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 0x55e06b60a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 -- 172.21.15.46:0/917961503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 msgr2=0x55e06b60a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 --2- 172.21.15.46:0/917961503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b60a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 -- 172.21.15.46:0/917961503 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e06b5af070 con 0x55e06b564f30 2024-04-25T08:53:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 --2- 172.21.15.46:0/917961503 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 0x55e06b609bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf44055540 tx=0x7fbf44058000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a6c4ab08d864546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf3e000640 1 -- 172.21.15.46:0/917961503 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4406d050 con 0x55e06b564f30 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf3e000640 1 -- 172.21.15.46:0/917961503 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4407ce50 con 0x55e06b564f30 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf3e000640 1 -- 172.21.15.46:0/917961503 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44021040 con 0x55e06b564f30 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/917961503 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 msgr2=0x55e06b609bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/917961503 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 0x55e06b609bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbf44055540 tx=0x7fbf44058000 comp rx=0 tx=0).stop 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/917961503 shutdown_connections 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/917961503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b5af240 0x55e06b60a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/917961503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 0x55e06b60a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/917961503 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b564f30 0x55e06b609bd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/917961503 >> 172.21.15.46:0/917961503 conn(0x55e06b56b2a0 msgr2=0x55e06b5aca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/917961503 shutdown_connections 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/917961503 wait complete. 2024-04-25T08:53:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 -- start start 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 Processor -- start 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b564f30 0x55e06b5c4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 0x55e06b5f3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5af240 0x55e06b5f3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06b3b83d0 con 0x55e06b564f30 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06b46fd50 con 0x55e06b5ae920 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e06b34e980 con 0x55e06b5af240 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf5bc1e000 10 client.0 _ll_register_callbacks cb 0x7fbf44055610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b564f30 0x55e06b5c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b564f30 0x55e06b5c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59984/0 (socket says 172.21.15.46:59984) 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 -- 172.21.15.46:0/3706001266 learned_addr learned my addr 172.21.15.46:0/3706001266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf56800640 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5af240 0x55e06b5f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55400640 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 0x55e06b5f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 -- 172.21.15.46:0/3706001266 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5af240 msgr2=0x55e06b5f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5af240 0x55e06b5f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 -- 172.21.15.46:0/3706001266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 msgr2=0x55e06b5f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 0x55e06b5f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf55e00640 1 -- 172.21.15.46:0/3706001266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e06b435600 con 0x55e06b564f30 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.522+0000 7fbf56800640 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5af240 0x55e06b5f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf55e00640 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b564f30 0x55e06b5c4310 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fbf44055540 tx=0x7fbf44000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe242ed47a701dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 1 -- 172.21.15.46:0/3706001266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44021040 con 0x55e06b564f30 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 1 -- 172.21.15.46:0/3706001266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fbf44053b40 con 0x55e06b564f30 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/3706001266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf4407ce50 con 0x55e06b564f30 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 1 -- 172.21.15.46:0/3706001266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf4406d020 con 0x55e06b564f30 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 10 client.9486 fetch_fsmap learned FSMap version 112 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/3706001266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55e06b3489b0 con 0x55e06b564f30 2024-04-25T08:53:00.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 1 -- 172.21.15.46:0/3706001266 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4407ce50 con 0x55e06b564f30 2024-04-25T08:53:00.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 1 -- 172.21.15.46:0/3706001266 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 112) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbf44024060 con 0x55e06b564f30 2024-04-25T08:53:00.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 10 client.9486 fetch_fsmap finished waiting for FSMap version 112 2024-04-25T08:53:00.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 10 client.9486 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:53:00.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 20 client.9486 populate_metadata read hostname 'smithi046' 2024-04-25T08:53:00.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] conn(0x55e06b5c4850 0x55e06b5f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 4 client.9486 mds_command: new command op to 9393 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:53:00.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/3706001266 --> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e06b3c12e0 con 0x55e06b5c4850 2024-04-25T08:53:00.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf55400640 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] conn(0x55e06b5c4850 0x55e06b5f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf55400640 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] conn(0x55e06b5c4850 0x55e06b5f3d90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80d57be67c1ed22d server_cookie=3e8374cf56c6b5e7 in_seq=0 out_seq=0 2024-04-25T08:53:00.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 10 client.9486 ms_handle_connect on v2:172.21.15.46:6832/1724167790 2024-04-25T08:53:00.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 1 -- 172.21.15.46:0/3706001266 <== mds.0 v2:172.21.15.46:6832/1724167790 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55e06b3c12e0 con 0x55e06b5c4850 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 10 client.9486 handle_command_reply: tid=0 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 client.9486 shutdown 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/3706001266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b564f30 msgr2=0x55e06b5c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b564f30 0x55e06b5c4310 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fbf44055540 tx=0x7fbf44000bd0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/3706001266 shutdown_connections 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.46:6832/1724167790,v1:172.21.15.46:6834/1724167790] conn(0x55e06b5c4850 0x55e06b5f3d90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x55e06b5af240 0x55e06b5f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e06b5ae920 0x55e06b5f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 --2- 172.21.15.46:0/3706001266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e06b564f30 0x55e06b5c4310 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf33400640 0 client.9486 ms_handle_reset on v2:172.21.15.46:6832/1724167790 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/3706001266 >> 172.21.15.46:0/3706001266 conn(0x55e06b56b2a0 msgr2=0x55e06b5ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/3706001266 shutdown_connections 2024-04-25T08:53:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 1 -- 172.21.15.46:0/3706001266 wait complete. 2024-04-25T08:53:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf5bc1e000 20 client.9486 trim_cache size 0 max 16384 2024-04-25T08:53:00.523 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:53:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "9ca516a0-1b6e-4031-bb83-1b30eee026c8", 2024-04-25T08:53:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-04-25T08:53:00.523 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9393, 2024-04-25T08:53:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-04-25T08:53:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-04-25T08:53:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-04-25T08:53:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5907079130000001, 2024-04-25T08:53:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 112, 2024-04-25T08:53:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 123, 2024-04-25T08:53:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 123, 2024-04-25T08:53:00.524 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 6.545489527 2024-04-25T08:53:00.524 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:53:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 -- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf34066f60 msgr2=0x7fbf34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf34066f60 0x7fbf34069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbf40007fd0 tx=0x7fbf40007900 comp rx=0 tx=0).stop 2024-04-25T08:53:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 -- 172.21.15.46:0/2208118753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf500aed10 msgr2=0x7fbf501592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf500aed10 0x7fbf501592d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9b0 tx=0x7fbf4400ee70 comp rx=0 tx=0).stop 2024-04-25T08:53:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 -- 172.21.15.46:0/2208118753 shutdown_connections 2024-04-25T08:53:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fbf34066f60 0x7fbf34069420 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf5015a750 0x7fbf5015acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fbf500cc9e0 0x7fbf50167230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 --2- 172.21.15.46:0/2208118753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf500aed10 0x7fbf501592d0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 -- 172.21.15.46:0/2208118753 >> 172.21.15.46:0/2208118753 conn(0x7fbf500b12a0 msgr2=0x7fbf50151e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 -- 172.21.15.46:0/2208118753 shutdown_connections 2024-04-25T08:53:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.526+0000 7fbf58800640 1 -- 172.21.15.46:0/2208118753 wait complete. 2024-04-25T08:53:00.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:53:00.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f2a200640 1 Processor -- start 2024-04-25T08:53:00.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f2a200640 1 -- start start 2024-04-25T08:53:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f2a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f2415bdb0 0x7f4f240cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f2a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f240cda80 0x7f4f2415d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f2a200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f240cc700 con 0x7f4f24159230 2024-04-25T08:53:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f2a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f240cc9e0 con 0x7f4f240cda80 2024-04-25T08:53:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f2a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f240cccc0 con 0x7f4f2415bdb0 2024-04-25T08:53:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f23e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f240cda80 0x7f4f2415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f23e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4f240cda80 0x7f4f2415d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59994/0 (socket says 172.21.15.46:59994) 2024-04-25T08:53:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f23e00640 1 -- 172.21.15.46:0/632059392 learned_addr learned my addr 172.21.15.46:0/632059392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:00.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.686+0000 7f4f1be00640 1 --2- 172.21.15.46:0/632059392 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f2415bdb0 0x7f4f240cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f23e00640 1 -- 172.21.15.46:0/632059392 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f2415bdb0 msgr2=0x7f4f240cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f23e00640 1 --2- 172.21.15.46:0/632059392 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f2415bdb0 0x7f4f240cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f23e00640 1 -- 172.21.15.46:0/632059392 >> v1:172.21.15.82:6789/0 conn(0x7f4f24159230 legacy=0x7f4f240cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f23e00640 1 -- 172.21.15.46:0/632059392 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f240ccfa0 con 0x7f4f240cda80 2024-04-25T08:53:00.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f23e00640 1 --2- 172.21.15.46:0/632059392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f240cda80 0x7f4f2415d200 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009e50 tx=0x7f4f1c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d36d2d6dfcdf19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f22a00640 1 -- 172.21.15.46:0/632059392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0a5660 con 0x7f4f240cda80 2024-04-25T08:53:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f22a00640 1 -- 172.21.15.46:0/632059392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c0a5800 con 0x7f4f240cda80 2024-04-25T08:53:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f22a00640 1 -- 172.21.15.46:0/632059392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0a5bd0 con 0x7f4f240cda80 2024-04-25T08:53:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f2a200640 1 -- 172.21.15.46:0/632059392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f240cda80 msgr2=0x7f4f2415d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f2a200640 1 --2- 172.21.15.46:0/632059392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f240cda80 0x7f4f2415d200 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009e50 tx=0x7f4f1c01c9a0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f2a200640 1 -- 172.21.15.46:0/632059392 shutdown_connections 2024-04-25T08:53:00.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f2a200640 1 --2- 172.21.15.46:0/632059392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f240cda80 0x7f4f2415d200 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f2a200640 1 --2- 172.21.15.46:0/632059392 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f2415bdb0 0x7f4f240cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f2a200640 1 -- 172.21.15.46:0/632059392 >> 172.21.15.46:0/632059392 conn(0x7f4f240b12a0 msgr2=0x7f4f240b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f2a200640 1 -- 172.21.15.46:0/632059392 shutdown_connections 2024-04-25T08:53:00.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.690+0000 7f4f2a200640 1 -- 172.21.15.46:0/632059392 wait complete. 2024-04-25T08:53:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f2a200640 1 Processor -- start 2024-04-25T08:53:00.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f2a200640 1 -- start start 2024-04-25T08:53:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f2a200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f240cda80 0x7f4f2406e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f2a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f24159230 0x7f4f24159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f2a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2415bdb0 0x7f4f2413b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f2a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2415e2e0 con 0x7f4f2415bdb0 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f2a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2415e160 con 0x7f4f24159230 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f2a200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2415e460 con 0x7f4f240cda80 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2415bdb0 0x7f4f2413b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2415bdb0 0x7f4f2413b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60010/0 (socket says 172.21.15.46:60010) 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23e00640 1 -- 172.21.15.46:0/1633226160 learned_addr learned my addr 172.21.15.46:0/1633226160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f1be00640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f24159230 0x7f4f24159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23400640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f240cda80 0x7f4f2406e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23e00640 1 -- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f240cda80 msgr2=0x7f4f2406e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23e00640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f240cda80 0x7f4f2406e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23e00640 1 -- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f24159230 msgr2=0x7f4f24159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23e00640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f24159230 0x7f4f24159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23e00640 1 -- 172.21.15.46:0/1633226160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f24067630 con 0x7f4f2415bdb0 2024-04-25T08:53:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f1be00640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f24159230 0x7f4f24159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f23e00640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2415bdb0 0x7f4f2413b290 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c01a5d0 tx=0x7f4f1c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f1b400640 1 -- 172.21.15.46:0/1633226160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0bc030 con 0x7f4f2415bdb0 2024-04-25T08:53:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f1b400640 1 -- 172.21.15.46:0/1633226160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c0b8470 con 0x7f4f2415bdb0 2024-04-25T08:53:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f1b400640 1 -- 172.21.15.46:0/1633226160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c009510 con 0x7f4f2415bdb0 2024-04-25T08:53:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.694+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2415bb00 con 0x7f4f2415bdb0 2024-04-25T08:53:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.698+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2415ad40 con 0x7f4f2415bdb0 2024-04-25T08:53:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.698+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f24069080 con 0x7f4f2415bdb0 2024-04-25T08:53:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.698+0000 7f4f1b400640 1 -- 172.21.15.46:0/1633226160 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f1c0bf070 con 0x7f4f2415bdb0 2024-04-25T08:53:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.698+0000 7f4f1b400640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f00066f60 0x7f4f00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.698+0000 7f4f1b400640 1 -- 172.21.15.46:0/1633226160 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f4f1c11fd20 con 0x7f4f2415bdb0 2024-04-25T08:53:00.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.698+0000 7f4f23400640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f00066f60 0x7f4f00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.702+0000 7f4f23400640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f00066f60 0x7f4f00069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4f240b1200 tx=0x7f4f14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.702+0000 7f4f1b400640 1 -- 172.21.15.46:0/1633226160 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f1c0c6070 con 0x7f4f2415bdb0 2024-04-25T08:53:00.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.902+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f240723d0 con 0x7f4f2415bdb0 2024-04-25T08:53:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f1b400640 1 -- 172.21.15.46:0/1633226160 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4f1c016620 con 0x7f4f2415bdb0 2024-04-25T08:53:00.896 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:53:00.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f00066f60 msgr2=0x7f4f00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f00066f60 0x7f4f00069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4f240b1200 tx=0x7f4f14009040 comp rx=0 tx=0).stop 2024-04-25T08:53:00.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2415bdb0 msgr2=0x7f4f2413b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2415bdb0 0x7f4f2413b290 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c01a5d0 tx=0x7f4f1c019740 comp rx=0 tx=0).stop 2024-04-25T08:53:00.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 shutdown_connections 2024-04-25T08:53:00.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4f00066f60 0x7f4f00069420 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f2415bdb0 0x7f4f2413b290 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f24159230 0x7f4f24159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 --2- 172.21.15.46:0/1633226160 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f4f240cda80 0x7f4f2406e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 >> 172.21.15.46:0/1633226160 conn(0x7f4f240b12a0 msgr2=0x7f4f24142720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 shutdown_connections 2024-04-25T08:53:00.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:00.906+0000 7f4f2a200640 1 -- 172.21.15.46:0/1633226160 wait complete. 2024-04-25T08:53:00.917 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:53:00.917 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:53:00.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-04-25T08:53:00.918 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:53:00.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:00.918 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:53:00.918 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T08:53:00.918 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:53:00.918 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:53:00.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:00.918 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-04-25T08:53:00.928 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:53:00.928 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:53:00.928 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:53:00.928 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:00.928 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-04-25T08:53:00.928 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:00.929 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:00.929 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:00.929 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-04-25T08:53:00.929 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:00.929 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:00.929 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-04-25T08:53:00.929 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:53:00.929 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:00.929 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-04-25T08:53:00.930 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:53:00.930 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41822sec preferred_lft 41822sec 2024-04-25T08:53:00.930 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-04-25T08:53:00.930 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:00.930 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:00.930 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:00.932 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-04-25T08:53:00.932 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:53:00.932 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:5d:f3:4f:4e brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:00.932 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:53:00.932 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:53:00.933 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:00.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:00.934 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:53:00.993 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:53:00.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:00.995 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-04-25T08:53:01.052 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-04-25T08:53:01.052 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:53:01.052 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:53:01.052 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:53:01.052 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:53:01.052 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:53:01.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:01.053 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-04-25T08:53:01.053 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-04-25T08:53:01.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:53:01.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:53:01.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:53:01.053 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-04-25T08:53:01.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:01.145 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:53:01.154 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:53:01.154 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:53:01.155 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:53:01.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:01.155 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-04-25T08:53:01.216 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:01.216 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:01.268 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:53:01.269 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:01.269 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:01.319 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T08:53:01.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:01.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:53:01.408 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:53:01.408 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:53:01.408 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:53:01.423 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:53:01.423 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:53:01.423 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:53:01.423 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:53:01.423 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:53:01.432 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T08:53:01.432 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T08:53:01.432 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-04-25T08:53:01.433 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.82:3300,172.21.15.82:3301" 2024-04-25T08:53:01.433 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T08:53:01.433 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790" 2024-04-25T08:53:01.433 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:6789,172.21.15.82:6789,172.21.15.82:6790:/ 2024-04-25T08:53:01.433 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=9ca516a0-1b6e-4031-bb83-1b30eee026c8" will pass to kernel 2024-04-25T08:53:01.437 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T08:53:01.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:01.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:01.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:01.457 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T08:53:01.510 INFO:teuthology.orchestra.run.smithi046.stdout:client7943 2024-04-25T08:53:01.510 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:53:01.510 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:01.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:01.560 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T08:53:01.620 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-04-25T08:53:01.620 INFO:teuthology.orchestra.run.smithi046.stdout:./parent 2024-04-25T08:53:01.620 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed 2024-04-25T08:53:01.620 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/charlie 2024-04-25T08:53:01.620 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/alpha 2024-04-25T08:53:01.620 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/bravo 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/unflushed 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/unflushed/jfile 2024-04-25T08:53:01.622 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:53:01.622 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:53:01.622 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:53:01.622 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:53:01.622 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T08:53:01.622 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:53:01.729 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-04-25T08:53:01.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:01.739 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:53:01.739 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:53:01.739 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:53:01.739 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:53:01.739 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T08:53:01.739 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:53:01.849 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-04-25T08:53:01.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:01.858 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:53:01.858 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:53:01.858 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:53:01.858 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:53:01.858 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T08:53:01.859 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:53:01.969 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-04-25T08:53:01.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:01.978 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:53:01.978 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:53:01.978 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:53:01.978 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:53:01.978 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T08:53:01.978 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:53:02.089 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627780 2024-04-25T08:53:02.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:02.098 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:53:02.098 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:53:02.098 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:53:02.098 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:53:02.098 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T08:53:02.098 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:53:02.209 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-04-25T08:53:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:02.218 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:53:02.218 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:53:02.219 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:53:02.219 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:53:02.219 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T08:53:02.219 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:53:02.329 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627779 2024-04-25T08:53:02.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:02.339 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:53:02.339 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:53:02.339 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:53:02.339 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:53:02.339 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T08:53:02.339 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:53:02.445 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627781 2024-04-25T08:53:02.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:02.454 DEBUG:teuthology.orchestra.run.smithi046:> stdin-killer -- python3 -c ' 2024-04-25T08:53:02.454 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-04-25T08:53:02.454 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-04-25T08:53:02.454 DEBUG:teuthology.orchestra.run.smithi046:> 2024-04-25T08:53:02.454 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T08:53:02.454 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-04-25T08:53:02.565 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627782 2024-04-25T08:53:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:02.574 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:53:02.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:53:02.702 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:53:02.802 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:53:02.802 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:53:02.802 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:02.811 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:53:02.811 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:02.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:03.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f3265800640 1 Processor -- start 2024-04-25T08:53:03.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f3265800640 1 -- start start 2024-04-25T08:53:03.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f3265800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3260159230 0x7f326015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f3265800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cbeb0 0x7f32600cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f3265800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32600cc700 con 0x7f32600cda80 2024-04-25T08:53:03.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f3265800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32600cc9e0 con 0x7f3260159230 2024-04-25T08:53:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f3265800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32600cccc0 con 0x7f32600cbeb0 2024-04-25T08:53:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f3257e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cbeb0 0x7f32600cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f325f400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f32600cda80 0x7f326015f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:59506/0 (socket says 172.21.15.46:59506) 2024-04-25T08:53:03.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f325f400640 1 -- 172.21.15.46:0/1040964631 learned_addr learned my addr 172.21.15.46:0/1040964631 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:03.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f325ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3260159230 0x7f326015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f325ea00640 1 -- 172.21.15.46:0/1040964631 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cbeb0 msgr2=0x7f32600cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f325ea00640 1 --2- 172.21.15.46:0/1040964631 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cbeb0 0x7f32600cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f325ea00640 1 -- 172.21.15.46:0/1040964631 >> v1:172.21.15.82:6789/0 conn(0x7f32600cda80 legacy=0x7f326015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.057+0000 7f325ea00640 1 -- 172.21.15.46:0/1040964631 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32600ccfa0 con 0x7f3260159230 2024-04-25T08:53:03.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f325ea00640 1 --2- 172.21.15.46:0/1040964631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3260159230 0x7f326015bdb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32500097c0 tx=0x7f325001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d8557a124989fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f325e000640 1 -- 172.21.15.46:0/1040964631 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500b0070 con 0x7f3260159230 2024-04-25T08:53:03.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f325e000640 1 -- 172.21.15.46:0/1040964631 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32500ab600 con 0x7f3260159230 2024-04-25T08:53:03.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f325e000640 1 -- 172.21.15.46:0/1040964631 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32500ab9d0 con 0x7f3260159230 2024-04-25T08:53:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f3265800640 1 -- 172.21.15.46:0/1040964631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3260159230 msgr2=0x7f326015bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f3265800640 1 --2- 172.21.15.46:0/1040964631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3260159230 0x7f326015bdb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32500097c0 tx=0x7f325001c410 comp rx=0 tx=0).stop 2024-04-25T08:53:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f3265800640 1 -- 172.21.15.46:0/1040964631 shutdown_connections 2024-04-25T08:53:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f3265800640 1 --2- 172.21.15.46:0/1040964631 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cbeb0 0x7f32600cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f3265800640 1 --2- 172.21.15.46:0/1040964631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3260159230 0x7f326015bdb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f3265800640 1 -- 172.21.15.46:0/1040964631 >> 172.21.15.46:0/1040964631 conn(0x7f32600b12a0 msgr2=0x7f32600b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f3265800640 1 -- 172.21.15.46:0/1040964631 shutdown_connections 2024-04-25T08:53:03.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.061+0000 7f3265800640 1 -- 172.21.15.46:0/1040964631 wait complete. 2024-04-25T08:53:03.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 Processor -- start 2024-04-25T08:53:03.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 -- start start 2024-04-25T08:53:03.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32600cbeb0 0x7f32601581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cda80 0x7f3260144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260159230 0x7f32600b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3260168780 con 0x7f3260159230 2024-04-25T08:53:03.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32600cc4a0 con 0x7f32600cbeb0 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3260168900 con 0x7f32600cda80 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260159230 0x7f32600b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260159230 0x7f32600b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60018/0 (socket says 172.21.15.46:60018) 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325f400640 1 -- 172.21.15.46:0/2841176890 learned_addr learned my addr 172.21.15.46:0/2841176890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325ea00640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32600cbeb0 0x7f32601581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3257e00640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cda80 0x7f3260144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325f400640 1 -- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cda80 msgr2=0x7f3260144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325f400640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cda80 0x7f3260144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325f400640 1 -- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32600cbeb0 msgr2=0x7f32601581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325f400640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32600cbeb0 0x7f32601581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325f400640 1 -- 172.21.15.46:0/2841176890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326013d2f0 con 0x7f3260159230 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325ea00640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32600cbeb0 0x7f32601581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:03.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f325f400640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260159230 0x7f32600b38c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f325800cbf0 tx=0x7f325800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3256a00640 1 -- 172.21.15.46:0/2841176890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258017070 con 0x7f3260159230 2024-04-25T08:53:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3256a00640 1 -- 172.21.15.46:0/2841176890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325800ad40 con 0x7f3260159230 2024-04-25T08:53:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326013d5d0 con 0x7f3260159230 2024-04-25T08:53:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3256a00640 1 -- 172.21.15.46:0/2841176890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258012600 con 0x7f3260159230 2024-04-25T08:53:03.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.065+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32600acdc0 con 0x7f3260159230 2024-04-25T08:53:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.069+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3260069080 con 0x7f3260159230 2024-04-25T08:53:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.069+0000 7f3256a00640 1 -- 172.21.15.46:0/2841176890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f325801a070 con 0x7f3260159230 2024-04-25T08:53:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.069+0000 7f3256a00640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f323c066f60 0x7f323c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.069+0000 7f3256a00640 1 -- 172.21.15.46:0/2841176890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f3258084f70 con 0x7f3260159230 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.069+0000 7f325ea00640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f323c066f60 0x7f323c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.069+0000 7f325ea00640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f323c066f60 0x7f323c069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f32500ad000 tx=0x7f32500a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.073+0000 7f3256a00640 1 -- 172.21.15.46:0/2841176890 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3258055650 con 0x7f3260159230 2024-04-25T08:53:03.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.237+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f326015c0d0 con 0x7f3260159230 2024-04-25T08:53:03.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.237+0000 7f3256a00640 1 -- 172.21.15.46:0/2841176890 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17159 (secure 0 0 0) 0x7f32580596f0 con 0x7f3260159230 2024-04-25T08:53:03.231 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:03.231 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":123,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:53.922654+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6832/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6835/51204523":"2024-04-26T08:52:42.856639+0000","172.21.15.46:6833/51204523":"2024-04-26T08:52:42.856639+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:03.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f323c066f60 msgr2=0x7f323c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f323c066f60 0x7f323c069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f32500ad000 tx=0x7f32500a6000 comp rx=0 tx=0).stop 2024-04-25T08:53:03.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260159230 msgr2=0x7f32600b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260159230 0x7f32600b38c0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f325800cbf0 tx=0x7f325800a430 comp rx=0 tx=0).stop 2024-04-25T08:53:03.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 shutdown_connections 2024-04-25T08:53:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f323c066f60 0x7f323c069420 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260159230 0x7f32600b38c0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f32600cda80 0x7f3260144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 --2- 172.21.15.46:0/2841176890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32600cbeb0 0x7f32601581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 >> 172.21.15.46:0/2841176890 conn(0x7f32600b12a0 msgr2=0x7f326015c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 shutdown_connections 2024-04-25T08:53:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.241+0000 7f3265800640 1 -- 172.21.15.46:0/2841176890 wait complete. 2024-04-25T08:53:03.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:25.941254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:27.155001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:53:03.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20f6000640 1 Processor -- start 2024-04-25T08:53:03.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20f6000640 1 -- start start 2024-04-25T08:53:03.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20f6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20f0145310 0x7f20f006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20f6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f006dc70 0x7f20f00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20f6000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20f00cf1b0 0x7f20f015d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20f6000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00cd930 con 0x7f20f0145310 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20f6000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00cdbd0 con 0x7f20f00cf1b0 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20f6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00cdeb0 con 0x7f20f006dc70 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20ef400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20f0145310 0x7f20f006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20ef400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20f0145310 0x7f20f006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60022/0 (socket says 172.21.15.46:60022) 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20ef400640 1 -- 172.21.15.46:0/149965845 learned_addr learned my addr 172.21.15.46:0/149965845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20e6a00640 1 --2- 172.21.15.46:0/149965845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f006dc70 0x7f20f00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20efe00640 1 --2- 172.21.15.46:0/149965845 >> v2:172.21.15.82:3300/0 conn(0x7f20f00cf1b0 0x7f20f015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20ef400640 1 -- 172.21.15.46:0/149965845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f006dc70 msgr2=0x7f20f00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20ef400640 1 --2- 172.21.15.46:0/149965845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f006dc70 0x7f20f00cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20ef400640 1 -- 172.21.15.46:0/149965845 >> v2:172.21.15.82:3300/0 conn(0x7f20f00cf1b0 msgr2=0x7f20f015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20ef400640 1 --2- 172.21.15.46:0/149965845 >> v2:172.21.15.82:3300/0 conn(0x7f20f00cf1b0 0x7f20f015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.405+0000 7f20ef400640 1 -- 172.21.15.46:0/149965845 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00ce190 con 0x7f20f0145310 2024-04-25T08:53:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20ef400640 1 --2- 172.21.15.46:0/149965845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f0145310 0x7f20f006e0d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f20f00cd3f0 tx=0x7f20e001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b892db9c80edaab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20eea00640 1 -- 172.21.15.46:0/149965845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e00a5750 con 0x7f20f0145310 2024-04-25T08:53:03.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20eea00640 1 -- 172.21.15.46:0/149965845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e00a58f0 con 0x7f20f0145310 2024-04-25T08:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20eea00640 1 -- 172.21.15.46:0/149965845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e00a5cc0 con 0x7f20f0145310 2024-04-25T08:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20f6000640 1 -- 172.21.15.46:0/149965845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f0145310 msgr2=0x7f20f006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20f6000640 1 --2- 172.21.15.46:0/149965845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f0145310 0x7f20f006e0d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f20f00cd3f0 tx=0x7f20e001c9c0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20f6000640 1 -- 172.21.15.46:0/149965845 shutdown_connections 2024-04-25T08:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20f6000640 1 --2- 172.21.15.46:0/149965845 >> v2:172.21.15.82:3300/0 conn(0x7f20f00cf1b0 0x7f20f015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20f6000640 1 --2- 172.21.15.46:0/149965845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f006dc70 0x7f20f00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20f6000640 1 --2- 172.21.15.46:0/149965845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f0145310 0x7f20f006e0d0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20f6000640 1 -- 172.21.15.46:0/149965845 >> 172.21.15.46:0/149965845 conn(0x7f20f006ac10 msgr2=0x7f20f006b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20f6000640 1 -- 172.21.15.46:0/149965845 shutdown_connections 2024-04-25T08:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.409+0000 7f20f6000640 1 -- 172.21.15.46:0/149965845 wait complete. 2024-04-25T08:53:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 Processor -- start 2024-04-25T08:53:03.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 -- start start 2024-04-25T08:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f00cf1b0 0x7f20f01388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f0145310 0x7f20f01f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f01f8490 0x7f20f0131a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00713f0 con 0x7f20f00cf1b0 2024-04-25T08:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0071270 con 0x7f20f01f8490 2024-04-25T08:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0071570 con 0x7f20f0145310 2024-04-25T08:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f00cf1b0 0x7f20f01388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f00cf1b0 0x7f20f01388c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60024/0 (socket says 172.21.15.46:60024) 2024-04-25T08:53:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20ef400640 1 -- 172.21.15.46:0/157672968 learned_addr learned my addr 172.21.15.46:0/157672968 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20efe00640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f01f8490 0x7f20f0131a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20e6a00640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f0145310 0x7f20f01f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20ef400640 1 -- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f0145310 msgr2=0x7f20f01f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20ef400640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f0145310 0x7f20f01f7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20ef400640 1 -- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f01f8490 msgr2=0x7f20f0131a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20ef400640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f01f8490 0x7f20f0131a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20ef400640 1 -- 172.21.15.46:0/157672968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f0137820 con 0x7f20f00cf1b0 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20e6a00640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f0145310 0x7f20f01f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20ef400640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f00cf1b0 0x7f20f01388c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f20e00af040 tx=0x7f20e0019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20e7e00640 1 -- 172.21.15.46:0/157672968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e00b8670 con 0x7f20f00cf1b0 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20e7e00640 1 -- 172.21.15.46:0/157672968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e00be070 con 0x7f20f00cf1b0 2024-04-25T08:53:03.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f0137b00 con 0x7f20f00cf1b0 2024-04-25T08:53:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20e7e00640 1 -- 172.21.15.46:0/157672968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e00b8a90 con 0x7f20f00cf1b0 2024-04-25T08:53:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.413+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f01380c0 con 0x7f20f00cf1b0 2024-04-25T08:53:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.417+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f01f88a0 con 0x7f20f00cf1b0 2024-04-25T08:53:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.417+0000 7f20e7e00640 1 -- 172.21.15.46:0/157672968 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20e00b8c30 con 0x7f20f00cf1b0 2024-04-25T08:53:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.417+0000 7f20e7e00640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f20cc066f60 0x7f20cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.417+0000 7f20e7e00640 1 -- 172.21.15.46:0/157672968 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f20e0120460 con 0x7f20f00cf1b0 2024-04-25T08:53:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.417+0000 7f20e6a00640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f20cc066f60 0x7f20cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.417+0000 7f20e6a00640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f20cc066f60 0x7f20cc069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ae9e0 tx=0x7f20d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.421+0000 7f20e7e00640 1 -- 172.21.15.46:0/157672968 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20f01f88a0 con 0x7f20f00cf1b0 2024-04-25T08:53:03.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20f0069010 con 0x7f20f00cf1b0 2024-04-25T08:53:03.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20e7e00640 1 -- 172.21.15.46:0/157672968 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20e00165e0 con 0x7f20f00cf1b0 2024-04-25T08:53:03.576 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:53:03.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f20cc066f60 msgr2=0x7f20cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f20cc066f60 0x7f20cc069420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f20f00ae9e0 tx=0x7f20d8008040 comp rx=0 tx=0).stop 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f00cf1b0 msgr2=0x7f20f01388c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f00cf1b0 0x7f20f01388c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f20e00af040 tx=0x7f20e0019760 comp rx=0 tx=0).stop 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 shutdown_connections 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f20cc066f60 0x7f20cc069420 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20f01f8490 0x7f20f0131a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f20f0145310 0x7f20f01f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 --2- 172.21.15.46:0/157672968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f00cf1b0 0x7f20f01388c0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 >> 172.21.15.46:0/157672968 conn(0x7f20f006ac10 msgr2=0x7f20f01442c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 shutdown_connections 2024-04-25T08:53:03.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.585+0000 7f20f6000640 1 -- 172.21.15.46:0/157672968 wait complete. 2024-04-25T08:53:03.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffadf200640 1 Processor -- start 2024-04-25T08:53:03.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffadf200640 1 -- start start 2024-04-25T08:53:03.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffadf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffad8143990 0x7ffad80cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffadf200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffad806f150 0x7ffad80b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffadf200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad815e440 con 0x7ffad81440d0 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffadf200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad815ed30 con 0x7ffad8143990 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffadf200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad815e6e0 con 0x7ffad806f150 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffad7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffad8143990 0x7ffad80cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffad7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffad8143990 0x7ffad80cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60030/0 (socket says 172.21.15.46:60030) 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffad7e00640 1 -- 172.21.15.46:0/1072384729 learned_addr learned my addr 172.21.15.46:0/1072384729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffad7e00640 1 -- 172.21.15.46:0/1072384729 >> v2:172.21.15.82:3300/0 conn(0x7ffad806f150 msgr2=0x7ffad80b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffadd200640 1 --2- 172.21.15.46:0/1072384729 >> v2:172.21.15.82:3300/0 conn(0x7ffad806f150 0x7ffad80b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffad7e00640 1 --2- 172.21.15.46:0/1072384729 >> v2:172.21.15.82:3300/0 conn(0x7ffad806f150 0x7ffad80b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffad7e00640 1 -- 172.21.15.46:0/1072384729 >> v1:172.21.15.82:6789/0 conn(0x7ffad81440d0 legacy=0x7ffad806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffad7e00640 1 -- 172.21.15.46:0/1072384729 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad815e9c0 con 0x7ffad8143990 2024-04-25T08:53:03.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.749+0000 7ffadd200640 1 --2- 172.21.15.46:0/1072384729 >> v2:172.21.15.82:3300/0 conn(0x7ffad806f150 0x7ffad80b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffad7e00640 1 --2- 172.21.15.46:0/1072384729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad8143990 0x7ffad80cecc0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ffad80af320 tx=0x7ffac801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2788466da96f1000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffad7400640 1 -- 172.21.15.46:0/1072384729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80b0070 con 0x7ffad8143990 2024-04-25T08:53:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffad7400640 1 -- 172.21.15.46:0/1072384729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac80ab600 con 0x7ffad8143990 2024-04-25T08:53:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffad7400640 1 -- 172.21.15.46:0/1072384729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80ab9d0 con 0x7ffad8143990 2024-04-25T08:53:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffadf200640 1 -- 172.21.15.46:0/1072384729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad8143990 msgr2=0x7ffad80cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffadf200640 1 --2- 172.21.15.46:0/1072384729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad8143990 0x7ffad80cecc0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7ffad80af320 tx=0x7ffac801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffadf200640 1 -- 172.21.15.46:0/1072384729 shutdown_connections 2024-04-25T08:53:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffadf200640 1 --2- 172.21.15.46:0/1072384729 >> v2:172.21.15.82:3300/0 conn(0x7ffad806f150 0x7ffad80b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffadf200640 1 --2- 172.21.15.46:0/1072384729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad8143990 0x7ffad80cecc0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffadf200640 1 -- 172.21.15.46:0/1072384729 >> 172.21.15.46:0/1072384729 conn(0x7ffad806ac10 msgr2=0x7ffad806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffadf200640 1 -- 172.21.15.46:0/1072384729 shutdown_connections 2024-04-25T08:53:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.753+0000 7ffadf200640 1 -- 172.21.15.46:0/1072384729 wait complete. 2024-04-25T08:53:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 Processor -- start 2024-04-25T08:53:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 -- start start 2024-04-25T08:53:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffad806f150 0x7ffad8142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffad8143990 0x7ffad8141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad81440d0 0x7ffad8159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80b35f0 con 0x7ffad81440d0 2024-04-25T08:53:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80b3470 con 0x7ffad806f150 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80b3770 con 0x7ffad8143990 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad81440d0 0x7ffad8159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad81440d0 0x7ffad8159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60034/0 (socket says 172.21.15.46:60034) 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadd200640 1 -- 172.21.15.46:0/299459004 learned_addr learned my addr 172.21.15.46:0/299459004 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffad7e00640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffad806f150 0x7ffad8142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffacfe00640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffad8143990 0x7ffad8141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadd200640 1 -- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffad8143990 msgr2=0x7ffad8141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadd200640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffad8143990 0x7ffad8141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadd200640 1 -- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffad806f150 msgr2=0x7ffad8142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadd200640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffad806f150 0x7ffad8142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadd200640 1 -- 172.21.15.46:0/299459004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad81424c0 con 0x7ffad81440d0 2024-04-25T08:53:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffacfe00640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffad8143990 0x7ffad8141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadd200640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad81440d0 0x7ffad8159ba0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ffad000cef0 tx=0x7ffad000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffad4c00640 1 -- 172.21.15.46:0/299459004 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad001a070 con 0x7ffad81440d0 2024-04-25T08:53:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffad4c00640 1 -- 172.21.15.46:0/299459004 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad000d860 con 0x7ffad81440d0 2024-04-25T08:53:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad8142f60 con 0x7ffad81440d0 2024-04-25T08:53:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffad4c00640 1 -- 172.21.15.46:0/299459004 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad000dc50 con 0x7ffad81440d0 2024-04-25T08:53:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.757+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad81fa6c0 con 0x7ffad81440d0 2024-04-25T08:53:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.761+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad8069010 con 0x7ffad81440d0 2024-04-25T08:53:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.761+0000 7ffad4c00640 1 -- 172.21.15.46:0/299459004 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffad001d070 con 0x7ffad81440d0 2024-04-25T08:53:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.761+0000 7ffad4c00640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffab4066f60 0x7ffab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.761+0000 7ffad4c00640 1 -- 172.21.15.46:0/299459004 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ffad0085b00 con 0x7ffad81440d0 2024-04-25T08:53:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.761+0000 7ffad7e00640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffab4066f60 0x7ffab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.761+0000 7ffad7e00640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffab4066f60 0x7ffab4069420 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffac80ad000 tx=0x7ffac8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.765+0000 7ffad4c00640 1 -- 172.21.15.46:0/299459004 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffad00568c0 con 0x7ffad81440d0 2024-04-25T08:53:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffad806b200 con 0x7ffad81440d0 2024-04-25T08:53:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffad4c00640 1 -- 172.21.15.46:0/299459004 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffad0085e20 con 0x7ffad81440d0 2024-04-25T08:53:03.918 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:53:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffab4066f60 msgr2=0x7ffab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffab4066f60 0x7ffab4069420 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ffac80ad000 tx=0x7ffac8008040 comp rx=0 tx=0).stop 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad81440d0 msgr2=0x7ffad8159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad81440d0 0x7ffad8159ba0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7ffad000cef0 tx=0x7ffad000a710 comp rx=0 tx=0).stop 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 shutdown_connections 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7ffab4066f60 0x7ffab4069420 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad81440d0 0x7ffad8159ba0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7ffad8143990 0x7ffad8141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 --2- 172.21.15.46:0/299459004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffad806f150 0x7ffad8142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.925+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 >> 172.21.15.46:0/299459004 conn(0x7ffad806ac10 msgr2=0x7ffad80b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.929+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 shutdown_connections 2024-04-25T08:53:03.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:03.929+0000 7ffadf200640 1 -- 172.21.15.46:0/299459004 wait complete. 2024-04-25T08:53:03.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.089+0000 7fc858e00640 1 Processor -- start 2024-04-25T08:53:04.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.089+0000 7fc858e00640 1 -- start start 2024-04-25T08:53:04.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc858e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc85406c730 0x7fc85406bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc858e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8540d0390 0x7fc8540cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc858e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85409acb0 con 0x7fc85409a410 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc858e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8540b3dc0 con 0x7fc85406c730 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc858e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8540cf680 con 0x7fc8540d0390 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc853400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc85406c730 0x7fc85406bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc853400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc85406c730 0x7fc85406bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60050/0 (socket says 172.21.15.46:60050) 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc853400640 1 -- 172.21.15.46:0/1676463663 learned_addr learned my addr 172.21.15.46:0/1676463663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc853400640 1 -- 172.21.15.46:0/1676463663 >> v2:172.21.15.82:3300/0 conn(0x7fc8540d0390 msgr2=0x7fc8540cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc853e00640 1 --2- 172.21.15.46:0/1676463663 >> v2:172.21.15.82:3300/0 conn(0x7fc8540d0390 0x7fc8540cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc853400640 1 --2- 172.21.15.46:0/1676463663 >> v2:172.21.15.82:3300/0 conn(0x7fc8540d0390 0x7fc8540cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc853400640 1 -- 172.21.15.46:0/1676463663 >> v1:172.21.15.82:6789/0 conn(0x7fc85409a410 legacy=0x7fc85406cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:04.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc853400640 1 -- 172.21.15.46:0/1676463663 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85409a9d0 con 0x7fc85406c730 2024-04-25T08:53:04.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.093+0000 7fc853e00640 1 --2- 172.21.15.46:0/1676463663 >> v2:172.21.15.82:3300/0 conn(0x7fc8540d0390 0x7fc8540cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:04.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc853400640 1 --2- 172.21.15.46:0/1676463663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85406c730 0x7fc85406bf30 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fc844009870 tx=0x7fc84401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46eed3b6b79d290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc852a00640 1 -- 172.21.15.46:0/1676463663 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440b1070 con 0x7fc85406c730 2024-04-25T08:53:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc852a00640 1 -- 172.21.15.46:0/1676463663 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8440ac620 con 0x7fc85406c730 2024-04-25T08:53:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc852a00640 1 -- 172.21.15.46:0/1676463663 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8440ac9f0 con 0x7fc85406c730 2024-04-25T08:53:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 -- 172.21.15.46:0/1676463663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85406c730 msgr2=0x7fc85406bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 --2- 172.21.15.46:0/1676463663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85406c730 0x7fc85406bf30 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fc844009870 tx=0x7fc84401c4b0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 -- 172.21.15.46:0/1676463663 shutdown_connections 2024-04-25T08:53:04.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 --2- 172.21.15.46:0/1676463663 >> v2:172.21.15.82:3300/0 conn(0x7fc8540d0390 0x7fc8540cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 --2- 172.21.15.46:0/1676463663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85406c730 0x7fc85406bf30 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 -- 172.21.15.46:0/1676463663 >> 172.21.15.46:0/1676463663 conn(0x7fc8540b2f50 msgr2=0x7fc8540b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 -- 172.21.15.46:0/1676463663 shutdown_connections 2024-04-25T08:53:04.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 -- 172.21.15.46:0/1676463663 wait complete. 2024-04-25T08:53:04.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 Processor -- start 2024-04-25T08:53:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 -- start start 2024-04-25T08:53:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc85406c730 0x7fc8541514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85409a410 0x7fc8540af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc8540d0390 0x7fc8540ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85415dd60 con 0x7fc85409a410 2024-04-25T08:53:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540cc400 con 0x7fc85406c730 2024-04-25T08:53:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc858e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85416b040 con 0x7fc8540d0390 2024-04-25T08:53:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.097+0000 7fc84aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85409a410 0x7fc8540af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc853400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc85406c730 0x7fc8541514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85409a410 0x7fc8540af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60052/0 (socket says 172.21.15.46:60052) 2024-04-25T08:53:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84aa00640 1 -- 172.21.15.46:0/3545293855 learned_addr learned my addr 172.21.15.46:0/3545293855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc853e00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc8540d0390 0x7fc8540ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84aa00640 1 -- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc8540d0390 msgr2=0x7fc8540ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84aa00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc8540d0390 0x7fc8540ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84aa00640 1 -- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc85406c730 msgr2=0x7fc8541514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84aa00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc85406c730 0x7fc8541514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84aa00640 1 -- 172.21.15.46:0/3545293855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540af890 con 0x7fc85409a410 2024-04-25T08:53:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84aa00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85409a410 0x7fc8540af350 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00b440 tx=0x7fc83c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84be00640 1 -- 172.21.15.46:0/3545293855 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c015020 con 0x7fc85409a410 2024-04-25T08:53:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84be00640 1 -- 172.21.15.46:0/3545293855 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83c011890 con 0x7fc85409a410 2024-04-25T08:53:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84be00640 1 -- 172.21.15.46:0/3545293855 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83c011c80 con 0x7fc85409a410 2024-04-25T08:53:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8540b4b30 con 0x7fc85409a410 2024-04-25T08:53:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8540c5410 con 0x7fc85409a410 2024-04-25T08:53:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc854069010 con 0x7fc85409a410 2024-04-25T08:53:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.101+0000 7fc84be00640 1 -- 172.21.15.46:0/3545293855 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc83c018020 con 0x7fc85409a410 2024-04-25T08:53:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.105+0000 7fc84be00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc830066f60 0x7fc830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.105+0000 7fc84be00640 1 -- 172.21.15.46:0/3545293855 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc83c084f60 con 0x7fc85409a410 2024-04-25T08:53:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.105+0000 7fc853400640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc830066f60 0x7fc830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.105+0000 7fc853400640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc830066f60 0x7fc830069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc8440ae000 tx=0x7fc844008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.109+0000 7fc84be00640 1 -- 172.21.15.46:0/3545293855 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc83c01f030 con 0x7fc85409a410 2024-04-25T08:53:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc85406c510 con 0x7fc85409a410 2024-04-25T08:53:04.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc84be00640 1 -- 172.21.15.46:0/3545293855 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc83c00d040 con 0x7fc85409a410 2024-04-25T08:53:04.261 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:53:04.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc830066f60 msgr2=0x7fc830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc830066f60 0x7fc830069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc8440ae000 tx=0x7fc844008040 comp rx=0 tx=0).stop 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85409a410 msgr2=0x7fc8540af350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85409a410 0x7fc8540af350 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00b440 tx=0x7fc83c00b900 comp rx=0 tx=0).stop 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 shutdown_connections 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc830066f60 0x7fc830069420 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc8540d0390 0x7fc8540ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc85409a410 0x7fc8540af350 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 --2- 172.21.15.46:0/3545293855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc85406c730 0x7fc8541514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 >> 172.21.15.46:0/3545293855 conn(0x7fc8540b2f50 msgr2=0x7fc8540b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 shutdown_connections 2024-04-25T08:53:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.269+0000 7fc858e00640 1 -- 172.21.15.46:0/3545293855 wait complete. 2024-04-25T08:53:04.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:53:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef54e00640 1 Processor -- start 2024-04-25T08:53:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef54e00640 1 -- start start 2024-04-25T08:53:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef54e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef5015a750 0x7fef5006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef54e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fef500aed10 0x7fef5006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef54e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef500cc9e0 0x7fef500b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef54e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef5015ab30 con 0x7fef500cc9e0 2024-04-25T08:53:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef54e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef500ce4f0 con 0x7fef500aed10 2024-04-25T08:53:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef54e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5006d8c0 con 0x7fef5015a750 2024-04-25T08:53:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef4e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef500cc9e0 0x7fef500b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef4e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef500cc9e0 0x7fef500b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60060/0 (socket says 172.21.15.46:60060) 2024-04-25T08:53:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef4e800640 1 -- 172.21.15.46:0/2496132048 learned_addr learned my addr 172.21.15.46:0/2496132048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef45400640 1 --2- 172.21.15.46:0/2496132048 >> v2:172.21.15.82:3300/0 conn(0x7fef500aed10 0x7fef5006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.433+0000 7fef4de00640 1 --2- 172.21.15.46:0/2496132048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef5015a750 0x7fef5006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef4e800640 1 -- 172.21.15.46:0/2496132048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef5015a750 msgr2=0x7fef5006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef4e800640 1 --2- 172.21.15.46:0/2496132048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef5015a750 0x7fef5006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef4e800640 1 -- 172.21.15.46:0/2496132048 >> v2:172.21.15.82:3300/0 conn(0x7fef500aed10 msgr2=0x7fef5006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef4e800640 1 --2- 172.21.15.46:0/2496132048 >> v2:172.21.15.82:3300/0 conn(0x7fef500aed10 0x7fef5006f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef4e800640 1 -- 172.21.15.46:0/2496132048 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5006d5e0 con 0x7fef500cc9e0 2024-04-25T08:53:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef45400640 1 --2- 172.21.15.46:0/2496132048 >> v2:172.21.15.82:3300/0 conn(0x7fef500aed10 0x7fef5006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:04.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef4e800640 1 --2- 172.21.15.46:0/2496132048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500cc9e0 0x7fef500b25a0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fef50050000 tx=0x7fef4801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6a005114cf41dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef4d400640 1 -- 172.21.15.46:0/2496132048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef480a5840 con 0x7fef500cc9e0 2024-04-25T08:53:04.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef4d400640 1 -- 172.21.15.46:0/2496132048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef480a59e0 con 0x7fef500cc9e0 2024-04-25T08:53:04.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef4d400640 1 -- 172.21.15.46:0/2496132048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef480a5db0 con 0x7fef500cc9e0 2024-04-25T08:53:04.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef54e00640 1 -- 172.21.15.46:0/2496132048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500cc9e0 msgr2=0x7fef500b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef54e00640 1 --2- 172.21.15.46:0/2496132048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500cc9e0 0x7fef500b25a0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fef50050000 tx=0x7fef4801c9c0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef54e00640 1 -- 172.21.15.46:0/2496132048 shutdown_connections 2024-04-25T08:53:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef54e00640 1 --2- 172.21.15.46:0/2496132048 >> v2:172.21.15.82:3300/0 conn(0x7fef500aed10 0x7fef5006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef54e00640 1 --2- 172.21.15.46:0/2496132048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500cc9e0 0x7fef500b25a0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef54e00640 1 --2- 172.21.15.46:0/2496132048 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef5015a750 0x7fef5006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef54e00640 1 -- 172.21.15.46:0/2496132048 >> 172.21.15.46:0/2496132048 conn(0x7fef500b12a0 msgr2=0x7fef500b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef54e00640 1 -- 172.21.15.46:0/2496132048 shutdown_connections 2024-04-25T08:53:04.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.437+0000 7fef54e00640 1 -- 172.21.15.46:0/2496132048 wait complete. 2024-04-25T08:53:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef54e00640 1 Processor -- start 2024-04-25T08:53:04.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef54e00640 1 -- start start 2024-04-25T08:53:04.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef54e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500aed10 0x7fef501f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef54e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef500cc9e0 0x7fef50150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef54e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef5015a750 0x7fef5015bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef54e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5006d270 con 0x7fef500aed10 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef54e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef500cf200 con 0x7fef5015a750 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef54e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef500b32f0 con 0x7fef500cc9e0 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500aed10 0x7fef501f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef4de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500aed10 0x7fef501f69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60074/0 (socket says 172.21.15.46:60074) 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef4de00640 1 -- 172.21.15.46:0/222419560 learned_addr learned my addr 172.21.15.46:0/222419560 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef45400640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef500cc9e0 0x7fef50150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef4e800640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef5015a750 0x7fef5015bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef4de00640 1 -- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef500cc9e0 msgr2=0x7fef50150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef4de00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef500cc9e0 0x7fef50150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef4de00640 1 -- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef5015a750 msgr2=0x7fef5015bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef4de00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef5015a750 0x7fef5015bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.441+0000 7fef4de00640 1 -- 172.21.15.46:0/222419560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef501514b0 con 0x7fef500aed10 2024-04-25T08:53:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef4de00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500aed10 0x7fef501f69c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d5f0 tx=0x7fef4000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef46a00640 1 -- 172.21.15.46:0/222419560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400125f0 con 0x7fef500aed10 2024-04-25T08:53:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef46a00640 1 -- 172.21.15.46:0/222419560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef40019070 con 0x7fef500aed10 2024-04-25T08:53:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef46a00640 1 -- 172.21.15.46:0/222419560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400129b0 con 0x7fef500aed10 2024-04-25T08:53:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef50156520 con 0x7fef500aed10 2024-04-25T08:53:04.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef50156ac0 con 0x7fef500aed10 2024-04-25T08:53:04.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef50069080 con 0x7fef500aed10 2024-04-25T08:53:04.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef46a00640 1 -- 172.21.15.46:0/222419560 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef40012b50 con 0x7fef500aed10 2024-04-25T08:53:04.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef46a00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fef2c066f90 0x7fef2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef46a00640 1 -- 172.21.15.46:0/222419560 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fef400854c0 con 0x7fef500aed10 2024-04-25T08:53:04.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.445+0000 7fef45400640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fef2c066f90 0x7fef2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.449+0000 7fef45400640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fef2c066f90 0x7fef2c069450 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fef500addd0 tx=0x7fef38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.449+0000 7fef46a00640 1 -- 172.21.15.46:0/222419560 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef40055c30 con 0x7fef500aed10 2024-04-25T08:53:04.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef50154aa0 con 0x7fef500aed10 2024-04-25T08:53:04.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef46a00640 1 -- 172.21.15.46:0/222419560 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v123) v1 ==== 80+0+240 (secure 0 0 0) 0x7fef40059cd0 con 0x7fef500aed10 2024-04-25T08:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/1279640422 2024-04-26T08:52:53.922632+0000 2024-04-25T08:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6832/1279640422 2024-04-26T08:52:53.922632+0000 2024-04-25T08:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/51204523 2024-04-26T08:52:42.856639+0000 2024-04-25T08:53:04.618 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6833/51204523 2024-04-26T08:52:42.856639+0000 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:listed 4 entries 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fef2c066f90 msgr2=0x7fef2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fef2c066f90 0x7fef2c069450 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fef500addd0 tx=0x7fef38008040 comp rx=0 tx=0).stop 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500aed10 msgr2=0x7fef501f69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500aed10 0x7fef501f69c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d5f0 tx=0x7fef4000dab0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 shutdown_connections 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fef2c066f90 0x7fef2c069450 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef5015a750 0x7fef5015bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fef500cc9e0 0x7fef50150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 --2- 172.21.15.46:0/222419560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef500aed10 0x7fef501f69c0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 >> 172.21.15.46:0/222419560 conn(0x7fef500b12a0 msgr2=0x7fef50153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:04.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 shutdown_connections 2024-04-25T08:53:04.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.625+0000 7fef54e00640 1 -- 172.21.15.46:0/222419560 wait complete. 2024-04-25T08:53:04.638 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T08:53:04.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:04.664 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:53:04.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.664 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:53:04.716 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:53:04.716 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:04.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:53:04.726 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:53:04.726 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:04.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:53:04.768 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:53:04.768 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:04.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:53:04.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa707200640 1 Processor -- start 2024-04-25T08:53:04.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa707200640 1 -- start start 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa707200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa708099c60 0x7fa7080d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa707200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa70815d630 0x7fa70815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa707200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7080cff90 con 0x7fa7080cbb30 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa707200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7080d0510 con 0x7fa708099c60 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa707200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70806c830 con 0x7fa70815d630 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa705400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa7080cbb30 0x7fa70809a040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60208/0 (socket says 172.21.15.46:60208) 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa705400640 1 -- 172.21.15.46:0/290904907 learned_addr learned my addr 172.21.15.46:0/290904907 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa706800640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa70815d630 0x7fa70815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa705e00640 1 --2- 172.21.15.46:0/290904907 >> v2:172.21.15.82:3300/0 conn(0x7fa708099c60 0x7fa7080d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107532794 0 0) 0x7fa7080cff90 con 0x7fa7080cbb30 2024-04-25T08:53:04.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6ec003530 con 0x7fa7080cbb30 2024-04-25T08:53:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2971756117 0 0) 0x7fa6ec003530 con 0x7fa7080cbb30 2024-04-25T08:53:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa70815d630 msgr2=0x7fa70815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa70815d630 0x7fa70815daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 >> v2:172.21.15.82:3300/0 conn(0x7fa708099c60 msgr2=0x7fa7080d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 --2- 172.21.15.46:0/290904907 >> v2:172.21.15.82:3300/0 conn(0x7fa708099c60 0x7fa7080d2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7080d0230 con 0x7fa7080cbb30 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fa6f40016a0 con 0x7fa7080cbb30 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 >> v1:172.21.15.46:6789/0 conn(0x7fa7080cbb30 legacy=0x7fa70809a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6ec0039c0 0x7fa6ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa70815d630 0x7fa6ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa6ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7080d0510 con 0x7fa6ec004d50 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70806c830 con 0x7fa6ec0039c0 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec00b9d0 con 0x7fa70815d630 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa706800640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa6ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa705400640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa70815d630 0x7fa6ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.965+0000 7fa705e00640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6ec0039c0 0x7fa6ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa706800640 1 -- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa70815d630 msgr2=0x7fa6ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:04.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa706800640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa70815d630 0x7fa6ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa706800640 1 -- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6ec0039c0 msgr2=0x7fa6ec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:04.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa706800640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6ec0039c0 0x7fa6ec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa706800640 1 -- 172.21.15.46:0/290904907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec003530 con 0x7fa6ec004d50 2024-04-25T08:53:04.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa705e00640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6ec0039c0 0x7fa6ec003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa706800640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa6ec0051d0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa700009fc0 tx=0x7fa70001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89afd589471b2d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7000b2070 con 0x7fa6ec004d50 2024-04-25T08:53:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7000a5c90 con 0x7fa6ec004d50 2024-04-25T08:53:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa704a00640 1 -- 172.21.15.46:0/290904907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700009c10 con 0x7fa6ec004d50 2024-04-25T08:53:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa707200640 1 -- 172.21.15.46:0/290904907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 msgr2=0x7fa6ec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa707200640 1 --2- 172.21.15.46:0/290904907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa6ec0051d0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa700009fc0 tx=0x7fa70001cc80 comp rx=0 tx=0).stop 2024-04-25T08:53:04.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa706800640 1 -- 172.21.15.46:0/290904907 reap_dead start 2024-04-25T08:53:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa707200640 1 -- 172.21.15.46:0/290904907 shutdown_connections 2024-04-25T08:53:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.969+0000 7fa707200640 1 -- 172.21.15.46:0/290904907 >> 172.21.15.46:0/290904907 conn(0x7fa7080d14e0 msgr2=0x7fa7080d18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 -- 172.21.15.46:0/290904907 shutdown_connections 2024-04-25T08:53:04.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 -- 172.21.15.46:0/290904907 wait complete. 2024-04-25T08:53:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 Processor -- start 2024-04-25T08:53:04.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 -- start start 2024-04-25T08:53:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa7080c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7080c89d0 0x7fa7080b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa70814cce0 0x7fa70814d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70809a750 con 0x7fa6ec004d50 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7080cbfc0 con 0x7fa70814cce0 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70809a8d0 con 0x7fa7080c89d0 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa7080c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa7080c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60096/0 (socket says 172.21.15.46:60096) 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705e00640 1 -- 172.21.15.46:0/2585193575 learned_addr learned my addr 172.21.15.46:0/2585193575 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa706800640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa70814cce0 0x7fa70814d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705400640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7080c89d0 0x7fa7080b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705e00640 1 -- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7080c89d0 msgr2=0x7fa7080b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705e00640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7080c89d0 0x7fa7080b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705e00640 1 -- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa70814cce0 msgr2=0x7fa70814d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705e00640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa70814cce0 0x7fa70814d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705e00640 1 -- 172.21.15.46:0/2585193575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7080c8ed0 con 0x7fa6ec004d50 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705400640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7080c89d0 0x7fa7080b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa705e00640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa7080c9f70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8011600 tx=0x7fa6f8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa6fe000640 1 -- 172.21.15.46:0/2585193575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8004220 con 0x7fa6ec004d50 2024-04-25T08:53:04.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa6fe000640 1 -- 172.21.15.46:0/2585193575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f80043c0 con 0x7fa6ec004d50 2024-04-25T08:53:04.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7080c91b0 con 0x7fa6ec004d50 2024-04-25T08:53:04.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.973+0000 7fa6fe000640 1 -- 172.21.15.46:0/2585193575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f800f8e0 con 0x7fa6ec004d50 2024-04-25T08:53:04.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.977+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7080c9770 con 0x7fa6ec004d50 2024-04-25T08:53:04.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.977+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa708069010 con 0x7fa6ec004d50 2024-04-25T08:53:04.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.977+0000 7fa6fe000640 1 -- 172.21.15.46:0/2585193575 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6f800fa80 con 0x7fa6ec004d50 2024-04-25T08:53:04.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.977+0000 7fa6fe000640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6e8066f90 0x7fa6e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.977+0000 7fa6fe000640 1 -- 172.21.15.46:0/2585193575 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa6f8085840 con 0x7fa6ec004d50 2024-04-25T08:53:04.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.977+0000 7fa705400640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6e8066f90 0x7fa6e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.981+0000 7fa705400640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6e8066f90 0x7fa6e8069450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa70809ac20 tx=0x7fa6f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:04.981+0000 7fa6fe000640 1 -- 172.21.15.46:0/2585193575 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6f8019030 con 0x7fa6ec004d50 2024-04-25T08:53:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.177+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7080b7460 con 0x7fa6ec004d50 2024-04-25T08:53:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.177+0000 7fa6fe000640 1 -- 172.21.15.46:0/2585193575 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7fa6f8055f20 con 0x7fa6ec004d50 2024-04-25T08:53:05.171 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:05.171 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:57.953462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6834/1724167790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1724167790},{"type":"v1","addr":"172.21.15.46:6834","nonce":1724167790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:53:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 112 2024-04-25T08:53:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6e8066f90 msgr2=0x7fa6e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6e8066f90 0x7fa6e8069450 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa70809ac20 tx=0x7fa6f4009040 comp rx=0 tx=0).stop 2024-04-25T08:53:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 msgr2=0x7fa7080c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa7080c9f70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8011600 tx=0x7fa6f8011ac0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 shutdown_connections 2024-04-25T08:53:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa6e8066f90 0x7fa6e8069450 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa70814cce0 0x7fa70814d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa7080c89d0 0x7fa7080b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 --2- 172.21.15.46:0/2585193575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6ec004d50 0x7fa7080c9f70 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 >> 172.21.15.46:0/2585193575 conn(0x7fa7080d14e0 msgr2=0x7fa70809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 shutdown_connections 2024-04-25T08:53:05.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.181+0000 7fa707200640 1 -- 172.21.15.46:0/2585193575 wait complete. 2024-04-25T08:53:05.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:05.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe77f200640 1 Processor -- start 2024-04-25T08:53:05.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe77f200640 1 -- start start 2024-04-25T08:53:05.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe77f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7780b3140 0x7fe7780716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe77f200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe77815df60 0x7fe7780b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe77f200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7780b3a60 con 0x7fe778161530 2024-04-25T08:53:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe77f200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe778072990 con 0x7fe77815df60 2024-04-25T08:53:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe77f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780b3f20 con 0x7fe7780b3140 2024-04-25T08:53:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe777e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7780b3140 0x7fe7780716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe777e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7780b3140 0x7fe7780716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:58056/0 (socket says 172.21.15.46:58056) 2024-04-25T08:53:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe77d200640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe778161530 0x7fe7781619c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:59536/0 (socket says 172.21.15.46:59536) 2024-04-25T08:53:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe777e00640 1 -- 172.21.15.46:0/567599907 learned_addr learned my addr 172.21.15.46:0/567599907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.341+0000 7fe76fe00640 1 --2- 172.21.15.46:0/567599907 >> v2:172.21.15.82:3300/0 conn(0x7fe77815df60 0x7fe7780b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe777e00640 1 -- 172.21.15.46:0/567599907 >> v2:172.21.15.82:3300/0 conn(0x7fe77815df60 msgr2=0x7fe7780b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe777e00640 1 --2- 172.21.15.46:0/567599907 >> v2:172.21.15.82:3300/0 conn(0x7fe77815df60 0x7fe7780b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe777e00640 1 -- 172.21.15.46:0/567599907 >> v1:172.21.15.82:6789/0 conn(0x7fe778161530 legacy=0x7fe7781619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe777e00640 1 -- 172.21.15.46:0/567599907 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778071e10 con 0x7fe7780b3140 2024-04-25T08:53:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe777e00640 1 --2- 172.21.15.46:0/567599907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7780b3140 0x7fe7780716c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe768009870 tx=0x7fe76801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11b489a3809ec793 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe777400640 1 -- 172.21.15.46:0/567599907 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680b0070 con 0x7fe7780b3140 2024-04-25T08:53:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe777400640 1 -- 172.21.15.46:0/567599907 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7680ab600 con 0x7fe7780b3140 2024-04-25T08:53:05.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe777400640 1 -- 172.21.15.46:0/567599907 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680ab9d0 con 0x7fe7780b3140 2024-04-25T08:53:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe77f200640 1 -- 172.21.15.46:0/567599907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7780b3140 msgr2=0x7fe7780716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe77f200640 1 --2- 172.21.15.46:0/567599907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7780b3140 0x7fe7780716c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe768009870 tx=0x7fe76801c440 comp rx=0 tx=0).stop 2024-04-25T08:53:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe77f200640 1 -- 172.21.15.46:0/567599907 shutdown_connections 2024-04-25T08:53:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe77f200640 1 --2- 172.21.15.46:0/567599907 >> v2:172.21.15.82:3300/0 conn(0x7fe77815df60 0x7fe7780b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe77f200640 1 --2- 172.21.15.46:0/567599907 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe7780b3140 0x7fe7780716c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe77f200640 1 -- 172.21.15.46:0/567599907 >> 172.21.15.46:0/567599907 conn(0x7fe7780b1750 msgr2=0x7fe7780b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe77f200640 1 -- 172.21.15.46:0/567599907 shutdown_connections 2024-04-25T08:53:05.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.345+0000 7fe77f200640 1 -- 172.21.15.46:0/567599907 wait complete. 2024-04-25T08:53:05.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77f200640 1 Processor -- start 2024-04-25T08:53:05.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77f200640 1 -- start start 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7780b3140 0x7fe778158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe77815df60 0x7fe7780ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77f200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe778161530 0x7fe7780cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe778147940 con 0x7fe77815df60 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781477c0 con 0x7fe7780b3140 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77f200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77816aed0 con 0x7fe778161530 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe76fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe77815df60 0x7fe7780ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe777e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7780b3140 0x7fe778158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe777e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7780b3140 0x7fe778158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:47652/0 (socket says 172.21.15.46:47652) 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe777e00640 1 -- 172.21.15.46:0/2352343399 learned_addr learned my addr 172.21.15.46:0/2352343399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe76fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe77815df60 0x7fe7780ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60100/0 (socket says 172.21.15.46:60100) 2024-04-25T08:53:05.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77d200640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe778161530 0x7fe7780cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe76fe00640 1 -- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe778161530 msgr2=0x7fe7780cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe76fe00640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe778161530 0x7fe7780cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe76fe00640 1 -- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7780b3140 msgr2=0x7fe778158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe76fe00640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7780b3140 0x7fe778158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe76fe00640 1 -- 172.21.15.46:0/2352343399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77815c4c0 con 0x7fe77815df60 2024-04-25T08:53:05.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe777e00640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7780b3140 0x7fe778158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:05.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe76fe00640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe77815df60 0x7fe7780ce750 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe76000e870 tx=0x7fe76000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:05.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe774c00640 1 -- 172.21.15.46:0/2352343399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760017070 con 0x7fe77815df60 2024-04-25T08:53:05.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe774c00640 1 -- 172.21.15.46:0/2352343399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76000ee60 con 0x7fe77815df60 2024-04-25T08:53:05.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77815c7a0 con 0x7fe77815df60 2024-04-25T08:53:05.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.349+0000 7fe774c00640 1 -- 172.21.15.46:0/2352343399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000cb60 con 0x7fe77815df60 2024-04-25T08:53:05.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.353+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77815d9d0 con 0x7fe77815df60 2024-04-25T08:53:05.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.353+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe778069080 con 0x7fe77815df60 2024-04-25T08:53:05.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.353+0000 7fe774c00640 1 -- 172.21.15.46:0/2352343399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe76001a070 con 0x7fe77815df60 2024-04-25T08:53:05.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.353+0000 7fe774c00640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe754066f60 0x7fe754069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.353+0000 7fe774c00640 1 -- 172.21.15.46:0/2352343399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fe7600852e0 con 0x7fe77815df60 2024-04-25T08:53:05.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.353+0000 7fe777e00640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe754066f60 0x7fe754069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.353+0000 7fe777e00640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe754066f60 0x7fe754069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe768009870 tx=0x7fe7680a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:05.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.357+0000 7fe774c00640 1 -- 172.21.15.46:0/2352343399 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe760055af0 con 0x7fe77815df60 2024-04-25T08:53:05.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.521+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7780cc130 con 0x7fe77815df60 2024-04-25T08:53:05.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.521+0000 7fe774c00640 1 -- 172.21.15.46:0/2352343399 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17159 (secure 0 0 0) 0x7fe760059b90 con 0x7fe77815df60 2024-04-25T08:53:05.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:05.515 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":123,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:53.922654+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6832/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6835/51204523":"2024-04-26T08:52:42.856639+0000","172.21.15.46:6833/51204523":"2024-04-26T08:52:42.856639+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:05.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe754066f60 msgr2=0x7fe754069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe754066f60 0x7fe754069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe768009870 tx=0x7fe7680a6000 comp rx=0 tx=0).stop 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe77815df60 msgr2=0x7fe7780ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe77815df60 0x7fe7780ce750 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe76000e870 tx=0x7fe76000ed30 comp rx=0 tx=0).stop 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 shutdown_connections 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe754066f60 0x7fe754069420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe778161530 0x7fe7780cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe77815df60 0x7fe7780ce750 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 --2- 172.21.15.46:0/2352343399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7780b3140 0x7fe778158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 >> 172.21.15.46:0/2352343399 conn(0x7fe7780b1750 msgr2=0x7fe778073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 shutdown_connections 2024-04-25T08:53:05.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.525+0000 7fe77f200640 1 -- 172.21.15.46:0/2352343399 wait complete. 2024-04-25T08:53:05.536 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:42:08.317228+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:25.941254+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:27.155001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:53:05.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:05.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f2380400640 1 Processor -- start 2024-04-25T08:53:05.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f2380400640 1 -- start start 2024-04-25T08:53:05.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f2380400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2378143990 0x7f23780cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f2380400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23781440d0 0x7f237806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f2380400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f237806ef80 0x7f23780cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f2380400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237815ed30 con 0x7f2378143990 2024-04-25T08:53:05.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f2380400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237815e440 con 0x7f237806ef80 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f2380400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237815e6e0 con 0x7f23781440d0 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2378143990 0x7f23780cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2378143990 0x7f23780cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60116/0 (socket says 172.21.15.46:60116) 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237da00640 1 -- 172.21.15.46:0/4274722625 learned_addr learned my addr 172.21.15.46:0/4274722625 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237e400640 1 --2- 172.21.15.46:0/4274722625 >> v2:172.21.15.82:3300/0 conn(0x7f237806ef80 0x7f23780cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237d000640 1 --2- 172.21.15.46:0/4274722625 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23781440d0 0x7f237806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237da00640 1 -- 172.21.15.46:0/4274722625 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23781440d0 msgr2=0x7f237806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237da00640 1 --2- 172.21.15.46:0/4274722625 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23781440d0 0x7f237806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237da00640 1 -- 172.21.15.46:0/4274722625 >> v2:172.21.15.82:3300/0 conn(0x7f237806ef80 msgr2=0x7f23780cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237da00640 1 --2- 172.21.15.46:0/4274722625 >> v2:172.21.15.82:3300/0 conn(0x7f237806ef80 0x7f23780cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.689+0000 7f237da00640 1 -- 172.21.15.46:0/4274722625 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237815e9c0 con 0x7f2378143990 2024-04-25T08:53:05.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f237da00640 1 --2- 172.21.15.46:0/4274722625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378143990 0x7f23780cecc0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f237806f700 tx=0x7f236c01e470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8a739f0d3226b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:05.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2377e00640 1 -- 172.21.15.46:0/4274722625 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0b2070 con 0x7f2378143990 2024-04-25T08:53:05.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2377e00640 1 -- 172.21.15.46:0/4274722625 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c0ad600 con 0x7f2378143990 2024-04-25T08:53:05.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2377e00640 1 -- 172.21.15.46:0/4274722625 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0ad9d0 con 0x7f2378143990 2024-04-25T08:53:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2380400640 1 -- 172.21.15.46:0/4274722625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378143990 msgr2=0x7f23780cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2380400640 1 --2- 172.21.15.46:0/4274722625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378143990 0x7f23780cecc0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f237806f700 tx=0x7f236c01e470 comp rx=0 tx=0).stop 2024-04-25T08:53:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2380400640 1 -- 172.21.15.46:0/4274722625 shutdown_connections 2024-04-25T08:53:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2380400640 1 --2- 172.21.15.46:0/4274722625 >> v2:172.21.15.82:3300/0 conn(0x7f237806ef80 0x7f23780cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2380400640 1 --2- 172.21.15.46:0/4274722625 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f23781440d0 0x7f237806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2380400640 1 --2- 172.21.15.46:0/4274722625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2378143990 0x7f23780cecc0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2380400640 1 -- 172.21.15.46:0/4274722625 >> 172.21.15.46:0/4274722625 conn(0x7f237806ac10 msgr2=0x7f237806b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2380400640 1 -- 172.21.15.46:0/4274722625 shutdown_connections 2024-04-25T08:53:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.693+0000 7f2380400640 1 -- 172.21.15.46:0/4274722625 wait complete. 2024-04-25T08:53:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 Processor -- start 2024-04-25T08:53:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 -- start start 2024-04-25T08:53:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f237806ef80 0x7f23781378e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2378143990 0x7f237813cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23781440d0 0x7f2378159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378143ef0 con 0x7f23781440d0 2024-04-25T08:53:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378143d70 con 0x7f237806ef80 2024-04-25T08:53:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2378071270 con 0x7f2378143990 2024-04-25T08:53:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23781440d0 0x7f2378159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23781440d0 0x7f2378159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60120/0 (socket says 172.21.15.46:60120) 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237e400640 1 -- 172.21.15.46:0/1359438016 learned_addr learned my addr 172.21.15.46:0/1359438016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237da00640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f237806ef80 0x7f23781378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237d000640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2378143990 0x7f237813cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237e400640 1 -- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2378143990 msgr2=0x7f237813cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237e400640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2378143990 0x7f237813cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237e400640 1 -- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f237806ef80 msgr2=0x7f23781378e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237e400640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f237806ef80 0x7f23781378e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237e400640 1 -- 172.21.15.46:0/1359438016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237813d740 con 0x7f23781440d0 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f237e400640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23781440d0 0x7f2378159b80 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f237000efc0 tx=0x7f237000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2375600640 1 -- 172.21.15.46:0/1359438016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370018020 con 0x7f23781440d0 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2375600640 1 -- 172.21.15.46:0/1359438016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2370014480 con 0x7f23781440d0 2024-04-25T08:53:05.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2375600640 1 -- 172.21.15.46:0/1359438016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370014870 con 0x7f23781440d0 2024-04-25T08:53:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237813da20 con 0x7f23781440d0 2024-04-25T08:53:05.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.697+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237813dfe0 con 0x7f23781440d0 2024-04-25T08:53:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.701+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2378069010 con 0x7f23781440d0 2024-04-25T08:53:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.701+0000 7f2375600640 1 -- 172.21.15.46:0/1359438016 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f237001b020 con 0x7f23781440d0 2024-04-25T08:53:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.701+0000 7f2375600640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2358066f60 0x7f2358069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.701+0000 7f2375600640 1 -- 172.21.15.46:0/1359438016 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f2370085f70 con 0x7f23781440d0 2024-04-25T08:53:05.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.701+0000 7f237da00640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2358066f60 0x7f2358069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:05.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.701+0000 7f237da00640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2358066f60 0x7f2358069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f236c0af000 tx=0x7f236c0a8000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.705+0000 7f2375600640 1 -- 172.21.15.46:0/1359438016 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2370022030 con 0x7f23781440d0 2024-04-25T08:53:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.869+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2378138a90 con 0x7f23781440d0 2024-04-25T08:53:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.869+0000 7f2375600640 1 -- 172.21.15.46:0/1359438016 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2370010040 con 0x7f23781440d0 2024-04-25T08:53:05.863 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-04-25T08:53:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2358066f60 msgr2=0x7f2358069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2358066f60 0x7f2358069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f236c0af000 tx=0x7f236c0a8000 comp rx=0 tx=0).stop 2024-04-25T08:53:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23781440d0 msgr2=0x7f2378159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23781440d0 0x7f2378159b80 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f237000efc0 tx=0x7f237000c490 comp rx=0 tx=0).stop 2024-04-25T08:53:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 shutdown_connections 2024-04-25T08:53:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2358066f60 0x7f2358069420 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23781440d0 0x7f2378159b80 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2378143990 0x7f237813cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 --2- 172.21.15.46:0/1359438016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f237806ef80 0x7f23781378e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 >> 172.21.15.46:0/1359438016 conn(0x7f237806ac10 msgr2=0x7f237815b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 shutdown_connections 2024-04-25T08:53:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:05.873+0000 7f2380400640 1 -- 172.21.15.46:0/1359438016 wait complete. 2024-04-25T08:53:05.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:06.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.033+0000 7f44ede00640 1 Processor -- start 2024-04-25T08:53:06.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44ede00640 1 -- start start 2024-04-25T08:53:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44ede00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44e815dd50 0x7f44e80cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44ede00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80d0450 0x7f44e80cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44ede00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e809a780 con 0x7f44e806f4d0 2024-04-25T08:53:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44ede00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e80708c0 con 0x7f44e815dd50 2024-04-25T08:53:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44ede00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e809aa60 con 0x7f44e80d0450 2024-04-25T08:53:06.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44e6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44e815dd50 0x7f44e80cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44e6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44e815dd50 0x7f44e80cff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60128/0 (socket says 172.21.15.46:60128) 2024-04-25T08:53:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44e6000640 1 -- 172.21.15.46:0/481333502 learned_addr learned my addr 172.21.15.46:0/481333502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44e6a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f44e806f4d0 0x7f44e80cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:59550/0 (socket says 172.21.15.46:59550) 2024-04-25T08:53:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44e7400640 1 --2- 172.21.15.46:0/481333502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80d0450 0x7f44e80cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44e6000640 1 -- 172.21.15.46:0/481333502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80d0450 msgr2=0x7f44e80cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44e6000640 1 --2- 172.21.15.46:0/481333502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80d0450 0x7f44e80cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44e6000640 1 -- 172.21.15.46:0/481333502 >> v1:172.21.15.82:6789/0 conn(0x7f44e806f4d0 legacy=0x7f44e80cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.037+0000 7f44e6000640 1 -- 172.21.15.46:0/481333502 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e809ad40 con 0x7f44e815dd50 2024-04-25T08:53:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44e6000640 1 --2- 172.21.15.46:0/481333502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e815dd50 0x7f44e80cff10 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009870 tx=0x7f44d401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90b820f40e3ffec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44e5600640 1 -- 172.21.15.46:0/481333502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40a95f0 con 0x7f44e815dd50 2024-04-25T08:53:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44e5600640 1 -- 172.21.15.46:0/481333502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d40a9790 con 0x7f44e815dd50 2024-04-25T08:53:06.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44e5600640 1 -- 172.21.15.46:0/481333502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40a9b60 con 0x7f44e815dd50 2024-04-25T08:53:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 -- 172.21.15.46:0/481333502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e815dd50 msgr2=0x7f44e80cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 --2- 172.21.15.46:0/481333502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e815dd50 0x7f44e80cff10 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f44d4009870 tx=0x7f44d401c530 comp rx=0 tx=0).stop 2024-04-25T08:53:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 -- 172.21.15.46:0/481333502 shutdown_connections 2024-04-25T08:53:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 --2- 172.21.15.46:0/481333502 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e80d0450 0x7f44e80cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 --2- 172.21.15.46:0/481333502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e815dd50 0x7f44e80cff10 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 -- 172.21.15.46:0/481333502 >> 172.21.15.46:0/481333502 conn(0x7f44e80b3160 msgr2=0x7f44e80b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 -- 172.21.15.46:0/481333502 shutdown_connections 2024-04-25T08:53:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 -- 172.21.15.46:0/481333502 wait complete. 2024-04-25T08:53:06.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 Processor -- start 2024-04-25T08:53:06.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.041+0000 7f44ede00640 1 -- start start 2024-04-25T08:53:06.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e806f4d0 0x7f44e81602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44ede00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e80d0450 0x7f44e815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44ede00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e815dd50 0x7f44e8157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44ede00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80cd390 con 0x7f44e806f4d0 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44ede00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80cd210 con 0x7f44e80d0450 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44ede00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80cd510 con 0x7f44e815dd50 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e806f4d0 0x7f44e81602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e806f4d0 0x7f44e81602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60144/0 (socket says 172.21.15.46:60144) 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6a00640 1 -- 172.21.15.46:0/705108518 learned_addr learned my addr 172.21.15.46:0/705108518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6000640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e80d0450 0x7f44e815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6a00640 1 -- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e815dd50 msgr2=0x7f44e8157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6a00640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e815dd50 0x7f44e8157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6a00640 1 -- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e80d0450 msgr2=0x7f44e815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6a00640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e80d0450 0x7f44e815bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6a00640 1 -- 172.21.15.46:0/705108518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e8160fb0 con 0x7f44e806f4d0 2024-04-25T08:53:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6000640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e80d0450 0x7f44e815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44e6a00640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e806f4d0 0x7f44e81602d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c8a0 tx=0x7f44d800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44df400640 1 -- 172.21.15.46:0/705108518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8017070 con 0x7f44e806f4d0 2024-04-25T08:53:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44df400640 1 -- 172.21.15.46:0/705108518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d800a640 con 0x7f44e806f4d0 2024-04-25T08:53:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e815c120 con 0x7f44e806f4d0 2024-04-25T08:53:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44df400640 1 -- 172.21.15.46:0/705108518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d800a9f0 con 0x7f44e806f4d0 2024-04-25T08:53:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e81609f0 con 0x7f44e806f4d0 2024-04-25T08:53:06.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e8069010 con 0x7f44e806f4d0 2024-04-25T08:53:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.045+0000 7f44df400640 1 -- 172.21.15.46:0/705108518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44d801a070 con 0x7f44e806f4d0 2024-04-25T08:53:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.049+0000 7f44df400640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c8066f90 0x7f44c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.049+0000 7f44df400640 1 -- 172.21.15.46:0/705108518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f44d80848d0 con 0x7f44e806f4d0 2024-04-25T08:53:06.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.049+0000 7f44e6000640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c8066f90 0x7f44c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.049+0000 7f44e6000640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c8066f90 0x7f44c8069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f44d40ae000 tx=0x7f44d40b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:06.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.053+0000 7f44df400640 1 -- 172.21.15.46:0/705108518 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44d8055040 con 0x7f44e806f4d0 2024-04-25T08:53:06.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44e8099d80 con 0x7f44e806f4d0 2024-04-25T08:53:06.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44df400640 1 -- 172.21.15.46:0/705108518 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44d80590e0 con 0x7f44e806f4d0 2024-04-25T08:53:06.206 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c8066f90 msgr2=0x7f44c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c8066f90 0x7f44c8069450 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f44d40ae000 tx=0x7f44d40b3040 comp rx=0 tx=0).stop 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e806f4d0 msgr2=0x7f44e81602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e806f4d0 0x7f44e81602d0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c8a0 tx=0x7f44d800cd60 comp rx=0 tx=0).stop 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 shutdown_connections 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f44c8066f90 0x7f44c8069450 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44e815dd50 0x7f44e8157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44e80d0450 0x7f44e815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 --2- 172.21.15.46:0/705108518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44e806f4d0 0x7f44e81602d0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 >> 172.21.15.46:0/705108518 conn(0x7f44e80b3160 msgr2=0x7f44e815cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:06.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 shutdown_connections 2024-04-25T08:53:06.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.213+0000 7f44ede00640 1 -- 172.21.15.46:0/705108518 wait complete. 2024-04-25T08:53:06.225 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8272600640 1 Processor -- start 2024-04-25T08:53:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8272600640 1 -- start start 2024-04-25T08:53:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8272600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f826c15b5e0 0x7f826c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8272600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f826c0cddd0 0x7f826c06b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8272600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c06bc70 con 0x7f826c06b860 2024-04-25T08:53:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8272600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c0b3c40 con 0x7f826c15b5e0 2024-04-25T08:53:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8272600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c0b3960 con 0x7f826c0cddd0 2024-04-25T08:53:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8271c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f826c06b860 0x7f826c0cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60224/0 (socket says 172.21.15.46:60224) 2024-04-25T08:53:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8271200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f826c15b5e0 0x7f826c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8271c00640 1 -- 172.21.15.46:0/2696072291 learned_addr learned my addr 172.21.15.46:0/2696072291 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:06.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8263e00640 1 --2- 172.21.15.46:0/2696072291 >> v2:172.21.15.82:3300/0 conn(0x7f826c0cddd0 0x7f826c06b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f826be00640 1 -- 172.21.15.46:0/2696072291 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398862958 0 0) 0x7f82640014a0 con 0x7f826c06b860 2024-04-25T08:53:06.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f826be00640 1 -- 172.21.15.46:0/2696072291 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8250003530 con 0x7f826c06b860 2024-04-25T08:53:06.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8271200640 1 -- 172.21.15.46:0/2696072291 >> v2:172.21.15.82:3300/0 conn(0x7f826c0cddd0 msgr2=0x7f826c06b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:06.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8271200640 1 --2- 172.21.15.46:0/2696072291 >> v2:172.21.15.82:3300/0 conn(0x7f826c0cddd0 0x7f826c06b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8271200640 1 -- 172.21.15.46:0/2696072291 >> v1:172.21.15.46:6789/0 conn(0x7f826c06b860 legacy=0x7f826c0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.377+0000 7f8271200640 1 -- 172.21.15.46:0/2696072291 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c15bde0 con 0x7f826c15b5e0 2024-04-25T08:53:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f8271200640 1 --2- 172.21.15.46:0/2696072291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c15b5e0 0x7f826c0b3230 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f826c0af2c0 tx=0x7f825c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c677da6ab393340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f826be00640 1 -- 172.21.15.46:0/2696072291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c0b0070 con 0x7f826c15b5e0 2024-04-25T08:53:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f826be00640 1 -- 172.21.15.46:0/2696072291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c0ab600 con 0x7f826c15b5e0 2024-04-25T08:53:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f826be00640 1 -- 172.21.15.46:0/2696072291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c0ab9d0 con 0x7f826c15b5e0 2024-04-25T08:53:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f8272600640 1 -- 172.21.15.46:0/2696072291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c15b5e0 msgr2=0x7f826c0b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f8272600640 1 --2- 172.21.15.46:0/2696072291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c15b5e0 0x7f826c0b3230 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f826c0af2c0 tx=0x7f825c01c3f0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f8272600640 1 -- 172.21.15.46:0/2696072291 shutdown_connections 2024-04-25T08:53:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f8272600640 1 --2- 172.21.15.46:0/2696072291 >> v2:172.21.15.82:3300/0 conn(0x7f826c0cddd0 0x7f826c06b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f8272600640 1 --2- 172.21.15.46:0/2696072291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c15b5e0 0x7f826c0b3230 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f8272600640 1 -- 172.21.15.46:0/2696072291 >> 172.21.15.46:0/2696072291 conn(0x7f826c0b1b50 msgr2=0x7f826c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f8272600640 1 -- 172.21.15.46:0/2696072291 shutdown_connections 2024-04-25T08:53:06.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.381+0000 7f8272600640 1 -- 172.21.15.46:0/2696072291 wait complete. 2024-04-25T08:53:06.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 Processor -- start 2024-04-25T08:53:06.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 -- start start 2024-04-25T08:53:06.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f826c06b860 0x7f826c13fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c0cddd0 0x7f826c06df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c15b5e0 0x7f826c157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c15bb40 con 0x7f826c0cddd0 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c15b9c0 con 0x7f826c15b5e0 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c168280 con 0x7f826c06b860 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8263e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c0cddd0 0x7f826c06df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8263e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c0cddd0 0x7f826c06df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60162/0 (socket says 172.21.15.46:60162) 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8263e00640 1 -- 172.21.15.46:0/1269046312 learned_addr learned my addr 172.21.15.46:0/1269046312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8271200640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f826c06b860 0x7f826c13fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8271c00640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c15b5e0 0x7f826c157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8263e00640 1 -- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f826c06b860 msgr2=0x7f826c13fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8263e00640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f826c06b860 0x7f826c13fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8263e00640 1 -- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c15b5e0 msgr2=0x7f826c157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8263e00640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c15b5e0 0x7f826c157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8263e00640 1 -- 172.21.15.46:0/1269046312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c06e7a0 con 0x7f826c0cddd0 2024-04-25T08:53:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8263e00640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c0cddd0 0x7f826c06df50 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f825400e930 tx=0x7f825400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8269600640 1 -- 172.21.15.46:0/1269046312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8254017070 con 0x7f826c0cddd0 2024-04-25T08:53:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8269600640 1 -- 172.21.15.46:0/1269046312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825400ca00 con 0x7f826c0cddd0 2024-04-25T08:53:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c06ea80 con 0x7f826c0cddd0 2024-04-25T08:53:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8269600640 1 -- 172.21.15.46:0/1269046312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825400cdb0 con 0x7f826c0cddd0 2024-04-25T08:53:06.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.385+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c154920 con 0x7f826c0cddd0 2024-04-25T08:53:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.389+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c069010 con 0x7f826c0cddd0 2024-04-25T08:53:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.389+0000 7f8269600640 1 -- 172.21.15.46:0/1269046312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f825401a070 con 0x7f826c0cddd0 2024-04-25T08:53:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.389+0000 7f8269600640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8248066f90 0x7f8248069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.389+0000 7f8269600640 1 -- 172.21.15.46:0/1269046312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f8254085660 con 0x7f826c0cddd0 2024-04-25T08:53:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.389+0000 7f8271200640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8248066f90 0x7f8248069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.389+0000 7f8271200640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8248066f90 0x7f8248069450 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f825c0ad000 tx=0x7f825c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:06.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.393+0000 7f8269600640 1 -- 172.21.15.46:0/1269046312 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8254055d70 con 0x7f826c0cddd0 2024-04-25T08:53:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.553+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f826c0b06f0 con 0x7f826c0cddd0 2024-04-25T08:53:06.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.553+0000 7f8269600640 1 -- 172.21.15.46:0/1269046312 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8254059e10 con 0x7f826c0cddd0 2024-04-25T08:53:06.546 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-04-25T08:53:06.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8248066f90 msgr2=0x7f8248069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8248066f90 0x7f8248069450 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f825c0ad000 tx=0x7f825c008040 comp rx=0 tx=0).stop 2024-04-25T08:53:06.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c0cddd0 msgr2=0x7f826c06df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c0cddd0 0x7f826c06df50 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f825400e930 tx=0x7f825400edf0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 shutdown_connections 2024-04-25T08:53:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8248066f90 0x7f8248069450 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f826c15b5e0 0x7f826c157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f826c0cddd0 0x7f826c06df50 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 --2- 172.21.15.46:0/1269046312 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f826c06b860 0x7f826c13fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 >> 172.21.15.46:0/1269046312 conn(0x7f826c0b1b50 msgr2=0x7f826c0b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 shutdown_connections 2024-04-25T08:53:06.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.557+0000 7f8272600640 1 -- 172.21.15.46:0/1269046312 wait complete. 2024-04-25T08:53:06.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a3600640 1 Processor -- start 2024-04-25T08:53:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a3600640 1 -- start start 2024-04-25T08:53:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a3600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f649c15bdb0 0x7f649c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c0cda80 0x7f649c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a3600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649c0cc700 con 0x7f649c159230 2024-04-25T08:53:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a3600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f649c0cc9e0 con 0x7f649c15bdb0 2024-04-25T08:53:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0cccc0 con 0x7f649c0cda80 2024-04-25T08:53:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a1600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c0cda80 0x7f649c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f6493e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f649c15bdb0 0x7f649c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a0c00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f649c159230 0x7f649c0cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.46:59554/0 (socket says 172.21.15.46:59554) 2024-04-25T08:53:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a1600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c0cda80 0x7f649c15d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3301/0 says I am v2:172.21.15.46:58134/0 (socket says 172.21.15.46:58134) 2024-04-25T08:53:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f64a0c00640 1 -- 172.21.15.46:0/197642088 learned_addr learned my addr 172.21.15.46:0/197642088 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f6493e00640 1 -- 172.21.15.46:0/197642088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c0cda80 msgr2=0x7f649c15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f6493e00640 1 --2- 172.21.15.46:0/197642088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c0cda80 0x7f649c15d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f6493e00640 1 -- 172.21.15.46:0/197642088 >> v1:172.21.15.82:6789/0 conn(0x7f649c159230 legacy=0x7f649c0cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.717+0000 7f6493e00640 1 -- 172.21.15.46:0/197642088 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c0ccfa0 con 0x7f649c15bdb0 2024-04-25T08:53:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f6493e00640 1 --2- 172.21.15.46:0/197642088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c15bdb0 0x7f649c0cd540 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f64840097c0 tx=0x7f648401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19881f458959f66e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f649be00640 1 -- 172.21.15.46:0/197642088 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840b0070 con 0x7f649c15bdb0 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f649be00640 1 -- 172.21.15.46:0/197642088 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64840ab600 con 0x7f649c15bdb0 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f649be00640 1 -- 172.21.15.46:0/197642088 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840ab9d0 con 0x7f649c15bdb0 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f64a3600640 1 -- 172.21.15.46:0/197642088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c15bdb0 msgr2=0x7f649c0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f64a3600640 1 --2- 172.21.15.46:0/197642088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c15bdb0 0x7f649c0cd540 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f64840097c0 tx=0x7f648401c410 comp rx=0 tx=0).stop 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f64a3600640 1 -- 172.21.15.46:0/197642088 shutdown_connections 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f64a3600640 1 --2- 172.21.15.46:0/197642088 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c0cda80 0x7f649c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f64a3600640 1 --2- 172.21.15.46:0/197642088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c15bdb0 0x7f649c0cd540 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f64a3600640 1 -- 172.21.15.46:0/197642088 >> 172.21.15.46:0/197642088 conn(0x7f649c0b12a0 msgr2=0x7f649c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f64a3600640 1 -- 172.21.15.46:0/197642088 shutdown_connections 2024-04-25T08:53:06.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.721+0000 7f64a3600640 1 -- 172.21.15.46:0/197642088 wait complete. 2024-04-25T08:53:06.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 Processor -- start 2024-04-25T08:53:06.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 -- start start 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0cda80 0x7f649c13fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c159230 0x7f649c06e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c15bdb0 0x7f649c0af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c15e320 con 0x7f649c159230 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c15e1a0 con 0x7f649c0cda80 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c15e4a0 con 0x7f649c15bdb0 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6493e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c159230 0x7f649c06e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6493e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c159230 0x7f649c06e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60174/0 (socket says 172.21.15.46:60174) 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6493e00640 1 -- 172.21.15.46:0/2300660371 learned_addr learned my addr 172.21.15.46:0/2300660371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:06.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a0c00640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0cda80 0x7f649c13fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a1600640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c15bdb0 0x7f649c0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6493e00640 1 -- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c15bdb0 msgr2=0x7f649c0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6493e00640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c15bdb0 0x7f649c0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6493e00640 1 -- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0cda80 msgr2=0x7f649c13fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6493e00640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0cda80 0x7f649c13fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6493e00640 1 -- 172.21.15.46:0/2300660371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c06eee0 con 0x7f649c159230 2024-04-25T08:53:06.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6493e00640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c159230 0x7f649c06e7e0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f648401a040 tx=0x7f64840191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:06.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6499600640 1 -- 172.21.15.46:0/2300660371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840ba070 con 0x7f649c159230 2024-04-25T08:53:06.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6499600640 1 -- 172.21.15.46:0/2300660371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64840b5460 con 0x7f649c159230 2024-04-25T08:53:06.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649c06f1c0 con 0x7f649c159230 2024-04-25T08:53:06.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f6499600640 1 -- 172.21.15.46:0/2300660371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64840b0040 con 0x7f649c159230 2024-04-25T08:53:06.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.725+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c0b2390 con 0x7f649c159230 2024-04-25T08:53:06.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.729+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c069080 con 0x7f649c159230 2024-04-25T08:53:06.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.729+0000 7f6499600640 1 -- 172.21.15.46:0/2300660371 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64840bd070 con 0x7f649c159230 2024-04-25T08:53:06.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.729+0000 7f6499600640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6478066f60 0x7f6478069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:06.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.729+0000 7f6499600640 1 -- 172.21.15.46:0/2300660371 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f648411f6f0 con 0x7f649c159230 2024-04-25T08:53:06.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.729+0000 7f64a0c00640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6478066f60 0x7f6478069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:06.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.729+0000 7f64a0c00640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6478066f60 0x7f6478069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f648c000f70 tx=0x7f648c007e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.733+0000 7f6499600640 1 -- 172.21.15.46:0/2300660371 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6484016090 con 0x7f649c159230 2024-04-25T08:53:06.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f649c072160 con 0x7f649c159230 2024-04-25T08:53:06.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f6499600640 1 -- 172.21.15.46:0/2300660371 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7f64840f4e20 con 0x7f649c159230 2024-04-25T08:53:06.925 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:06.925 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:57.953462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6834/1724167790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1724167790},{"type":"v1","addr":"172.21.15.46:6834","nonce":1724167790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:53:06.925 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 112 2024-04-25T08:53:06.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6478066f60 msgr2=0x7f6478069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6478066f60 0x7f6478069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f648c000f70 tx=0x7f648c007e90 comp rx=0 tx=0).stop 2024-04-25T08:53:06.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c159230 msgr2=0x7f649c06e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:06.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c159230 0x7f649c06e7e0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f648401a040 tx=0x7f64840191b0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 shutdown_connections 2024-04-25T08:53:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6478066f60 0x7f6478069420 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f649c15bdb0 0x7f649c0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f649c159230 0x7f649c06e7e0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 --2- 172.21.15.46:0/2300660371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f649c0cda80 0x7f649c13fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 >> 172.21.15.46:0/2300660371 conn(0x7f649c0b12a0 msgr2=0x7f649c0b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 shutdown_connections 2024-04-25T08:53:06.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:06.933+0000 7f64a3600640 1 -- 172.21.15.46:0/2300660371 wait complete. 2024-04-25T08:53:06.946 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.093+0000 7f6c20a00640 1 Processor -- start 2024-04-25T08:53:07.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.093+0000 7f6c20a00640 1 -- start start 2024-04-25T08:53:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c20a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c0b29c0 0x7f6c1c15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c20a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c1c099a00 0x7f6c1c099e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c20a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c0b2da0 con 0x7f6c1c06c2b0 2024-04-25T08:53:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c20a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1c161b80 con 0x7f6c1c099a00 2024-04-25T08:53:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c20a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c09a370 con 0x7f6c1c0b29c0 2024-04-25T08:53:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c12a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6c1c06c2b0 0x7f6c1c161470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60232/0 (socket says 172.21.15.46:60232) 2024-04-25T08:53:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c12a00640 1 -- 172.21.15.46:0/691276277 learned_addr learned my addr 172.21.15.46:0/691276277 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c1be00640 1 --2- 172.21.15.46:0/691276277 >> v2:172.21.15.82:3300/0 conn(0x7f6c1c099a00 0x7f6c1c099e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c1b400640 1 --2- 172.21.15.46:0/691276277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c0b29c0 0x7f6c1c15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c1aa00640 1 -- 172.21.15.46:0/691276277 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117486842 0 0) 0x7f6c1c0b2da0 con 0x7f6c1c06c2b0 2024-04-25T08:53:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c1aa00640 1 -- 172.21.15.46:0/691276277 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c00003530 con 0x7f6c1c06c2b0 2024-04-25T08:53:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c1be00640 1 -- 172.21.15.46:0/691276277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c0b29c0 msgr2=0x7f6c1c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c1be00640 1 --2- 172.21.15.46:0/691276277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c0b29c0 0x7f6c1c15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c1be00640 1 -- 172.21.15.46:0/691276277 >> v1:172.21.15.46:6789/0 conn(0x7f6c1c06c2b0 legacy=0x7f6c1c161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.097+0000 7f6c1be00640 1 -- 172.21.15.46:0/691276277 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c1623a0 con 0x7f6c1c099a00 2024-04-25T08:53:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c1be00640 1 --2- 172.21.15.46:0/691276277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1c099a00 0x7f6c1c099e30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6c140097c0 tx=0x7f6c1401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ac9f0cf149f48a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c1aa00640 1 -- 172.21.15.46:0/691276277 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c140a7070 con 0x7f6c1c099a00 2024-04-25T08:53:07.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c1aa00640 1 -- 172.21.15.46:0/691276277 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c140b0070 con 0x7f6c1c099a00 2024-04-25T08:53:07.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c1aa00640 1 -- 172.21.15.46:0/691276277 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c140ab890 con 0x7f6c1c099a00 2024-04-25T08:53:07.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 -- 172.21.15.46:0/691276277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1c099a00 msgr2=0x7f6c1c099e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 --2- 172.21.15.46:0/691276277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1c099a00 0x7f6c1c099e30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6c140097c0 tx=0x7f6c1401c470 comp rx=0 tx=0).stop 2024-04-25T08:53:07.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 -- 172.21.15.46:0/691276277 shutdown_connections 2024-04-25T08:53:07.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 --2- 172.21.15.46:0/691276277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1c099a00 0x7f6c1c099e30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 --2- 172.21.15.46:0/691276277 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c0b29c0 0x7f6c1c15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 -- 172.21.15.46:0/691276277 >> 172.21.15.46:0/691276277 conn(0x7f6c1c0b2f90 msgr2=0x7f6c1c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:07.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 -- 172.21.15.46:0/691276277 shutdown_connections 2024-04-25T08:53:07.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 -- 172.21.15.46:0/691276277 wait complete. 2024-04-25T08:53:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 Processor -- start 2024-04-25T08:53:07.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.101+0000 7f6c20a00640 1 -- start start 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c20a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c06c2b0 0x7f6c1c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c20a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c099a00 0x7f6c1c0b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c20a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1c0b29c0 0x7f6c1c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c20a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c15e930 con 0x7f6c1c099a00 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c20a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0cfc90 con 0x7f6c1c0b29c0 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c20a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c15eab0 con 0x7f6c1c06c2b0 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c12a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c099a00 0x7f6c1c0b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c12a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c099a00 0x7f6c1c0b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60178/0 (socket says 172.21.15.46:60178) 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c12a00640 1 -- 172.21.15.46:0/398355745 learned_addr learned my addr 172.21.15.46:0/398355745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c1be00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1c0b29c0 0x7f6c1c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c1b400640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c06c2b0 0x7f6c1c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c12a00640 1 -- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c06c2b0 msgr2=0x7f6c1c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c12a00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c06c2b0 0x7f6c1c15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c12a00640 1 -- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1c0b29c0 msgr2=0x7f6c1c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c12a00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1c0b29c0 0x7f6c1c0b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c12a00640 1 -- 172.21.15.46:0/398355745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c15df20 con 0x7f6c1c099a00 2024-04-25T08:53:07.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c12a00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c099a00 0x7f6c1c0b40c0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400cb40 tx=0x7f6c0400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c13e00640 1 -- 172.21.15.46:0/398355745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04017070 con 0x7f6c1c099a00 2024-04-25T08:53:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c13e00640 1 -- 172.21.15.46:0/398355745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0400aca0 con 0x7f6c1c099a00 2024-04-25T08:53:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c0b6a80 con 0x7f6c1c099a00 2024-04-25T08:53:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c13e00640 1 -- 172.21.15.46:0/398355745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04012620 con 0x7f6c1c099a00 2024-04-25T08:53:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c154500 con 0x7f6c1c099a00 2024-04-25T08:53:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c069010 con 0x7f6c1c099a00 2024-04-25T08:53:07.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.105+0000 7f6c13e00640 1 -- 172.21.15.46:0/398355745 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c0401a070 con 0x7f6c1c099a00 2024-04-25T08:53:07.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.109+0000 7f6c13e00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6bf8066f90 0x7f6bf8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.109+0000 7f6c13e00640 1 -- 172.21.15.46:0/398355745 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f6c040850d0 con 0x7f6c1c099a00 2024-04-25T08:53:07.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.109+0000 7f6c1b400640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6bf8066f90 0x7f6bf8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.109+0000 7f6c1b400640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6bf8066f90 0x7f6bf8069450 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009a50 tx=0x7f6c0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.113+0000 7f6c13e00640 1 -- 172.21.15.46:0/398355745 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c04055730 con 0x7f6c1c099a00 2024-04-25T08:53:07.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.277+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c1c14cf60 con 0x7f6c1c099a00 2024-04-25T08:53:07.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.277+0000 7f6c13e00640 1 -- 172.21.15.46:0/398355745 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17159 (secure 0 0 0) 0x7f6c040597d0 con 0x7f6c1c099a00 2024-04-25T08:53:07.270 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:07.270 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":123,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:53.922654+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6832/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6835/51204523":"2024-04-26T08:52:42.856639+0000","172.21.15.46:6833/51204523":"2024-04-26T08:52:42.856639+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6bf8066f90 msgr2=0x7f6bf8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6bf8066f90 0x7f6bf8069450 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009a50 tx=0x7f6c0c008040 comp rx=0 tx=0).stop 2024-04-25T08:53:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c099a00 msgr2=0x7f6c1c0b40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c099a00 0x7f6c1c0b40c0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400cb40 tx=0x7f6c0400a060 comp rx=0 tx=0).stop 2024-04-25T08:53:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 shutdown_connections 2024-04-25T08:53:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6bf8066f90 0x7f6bf8069450 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1c0b29c0 0x7f6c1c0b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c1c099a00 0x7f6c1c0b40c0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 --2- 172.21.15.46:0/398355745 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6c1c06c2b0 0x7f6c1c15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 >> 172.21.15.46:0/398355745 conn(0x7f6c1c0b2f90 msgr2=0x7f6c1c0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 shutdown_connections 2024-04-25T08:53:07.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.281+0000 7f6c20a00640 1 -- 172.21.15.46:0/398355745 wait complete. 2024-04-25T08:53:07.292 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:53:07.292 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:07.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.441+0000 7fa292e00640 1 Processor -- start 2024-04-25T08:53:07.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.441+0000 7fa292e00640 1 -- start start 2024-04-25T08:53:07.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.441+0000 7fa292e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa28c06d980 0x7fa28c162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.441+0000 7fa292e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c06ec70 con 0x7fa28c163400 2024-04-25T08:53:07.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.441+0000 7fa292e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c06ef50 con 0x7fa28c1654c0 2024-04-25T08:53:07.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.441+0000 7fa292e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28c06f230 con 0x7fa28c06d980 2024-04-25T08:53:07.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.441+0000 7fa283400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa28c163400 0x7fa28c06d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60238/0 (socket says 172.21.15.46:60238) 2024-04-25T08:53:07.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.441+0000 7fa283400640 1 -- 172.21.15.46:0/2803835813 learned_addr learned my addr 172.21.15.46:0/2803835813 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:07.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa290e00640 1 --2- 172.21.15.46:0/2803835813 >> v2:172.21.15.82:3300/0 conn(0x7fa28c06d980 0x7fa28c162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa28b400640 1 -- 172.21.15.46:0/2803835813 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293416531 0 0) 0x7fa28c06ec70 con 0x7fa28c163400 2024-04-25T08:53:07.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa28b400640 1 -- 172.21.15.46:0/2803835813 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa270003530 con 0x7fa28c163400 2024-04-25T08:53:07.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa28b400640 1 -- 172.21.15.46:0/2803835813 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806032498 0 0) 0x7fa28c06ef50 con 0x7fa28c1654c0 2024-04-25T08:53:07.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa28b400640 1 -- 172.21.15.46:0/2803835813 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa28c06ec70 con 0x7fa28c1654c0 2024-04-25T08:53:07.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa28b400640 1 -- 172.21.15.46:0/2803835813 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2750617107 0 0) 0x7fa270003530 con 0x7fa28c163400 2024-04-25T08:53:07.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa290e00640 1 -- 172.21.15.46:0/2803835813 >> v1:172.21.15.82:6789/0 conn(0x7fa28c1654c0 legacy=0x7fa28c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa290e00640 1 -- 172.21.15.46:0/2803835813 >> v1:172.21.15.46:6789/0 conn(0x7fa28c163400 legacy=0x7fa28c06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa290e00640 1 -- 172.21.15.46:0/2803835813 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c06f510 con 0x7fa28c06d980 2024-04-25T08:53:07.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa290e00640 1 --2- 172.21.15.46:0/2803835813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c06d980 0x7fa28c162330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa2840097c0 tx=0x7fa28401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d03797e8a2d2911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa28b400640 1 -- 172.21.15.46:0/2803835813 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840b0070 con 0x7fa28c06d980 2024-04-25T08:53:07.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa28b400640 1 -- 172.21.15.46:0/2803835813 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2840ab400 con 0x7fa28c06d980 2024-04-25T08:53:07.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.445+0000 7fa28b400640 1 -- 172.21.15.46:0/2803835813 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274001a00 con 0x7fa28c06d980 2024-04-25T08:53:07.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 -- 172.21.15.46:0/2803835813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c06d980 msgr2=0x7fa28c162330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 --2- 172.21.15.46:0/2803835813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c06d980 0x7fa28c162330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa2840097c0 tx=0x7fa28401c3c0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 -- 172.21.15.46:0/2803835813 shutdown_connections 2024-04-25T08:53:07.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 --2- 172.21.15.46:0/2803835813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c06d980 0x7fa28c162330 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 -- 172.21.15.46:0/2803835813 >> 172.21.15.46:0/2803835813 conn(0x7fa28c0b12a0 msgr2=0x7fa28c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:07.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 -- 172.21.15.46:0/2803835813 shutdown_connections 2024-04-25T08:53:07.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 -- 172.21.15.46:0/2803835813 wait complete. 2024-04-25T08:53:07.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 Processor -- start 2024-04-25T08:53:07.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 -- start start 2024-04-25T08:53:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa28c06d980 0x7fa28c15d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c163400 0x7fa28c16c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c16c690 0x7fa28c16a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0ce630 con 0x7fa28c16c690 2024-04-25T08:53:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0ce4b0 con 0x7fa28c163400 2024-04-25T08:53:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa292e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0ce7b0 con 0x7fa28c06d980 2024-04-25T08:53:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa290e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c16c690 0x7fa28c16a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa290e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c16c690 0x7fa28c16a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60190/0 (socket says 172.21.15.46:60190) 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa290e00640 1 -- 172.21.15.46:0/1233945568 learned_addr learned my addr 172.21.15.46:0/1233945568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa283400640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c163400 0x7fa28c16c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa28be00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa28c06d980 0x7fa28c15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa290e00640 1 -- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa28c06d980 msgr2=0x7fa28c15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa290e00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa28c06d980 0x7fa28c15d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa290e00640 1 -- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c163400 msgr2=0x7fa28c16c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa290e00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c163400 0x7fa28c16c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa290e00640 1 -- 172.21.15.46:0/1233945568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c0b3720 con 0x7fa28c16c690 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.449+0000 7fa283400640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c163400 0x7fa28c16c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:07.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa290e00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c16c690 0x7fa28c16a100 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa2840097c0 tx=0x7fa28401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa288c00640 1 -- 172.21.15.46:0/1233945568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840bc070 con 0x7fa28c16c690 2024-04-25T08:53:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa288c00640 1 -- 172.21.15.46:0/1233945568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2840b1040 con 0x7fa28c16c690 2024-04-25T08:53:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c0b39c0 con 0x7fa28c16c690 2024-04-25T08:53:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa288c00640 1 -- 172.21.15.46:0/1233945568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840b0050 con 0x7fa28c16c690 2024-04-25T08:53:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c0b3f80 con 0x7fa28c16c690 2024-04-25T08:53:07.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c16cb40 con 0x7fa28c16c690 2024-04-25T08:53:07.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa288c00640 1 -- 172.21.15.46:0/1233945568 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2840bf070 con 0x7fa28c16c690 2024-04-25T08:53:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa288c00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa268066f60 0x7fa268069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa288c00640 1 -- 172.21.15.46:0/1233945568 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa28411eef0 con 0x7fa28c16c690 2024-04-25T08:53:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.453+0000 7fa28be00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa268066f60 0x7fa268069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.457+0000 7fa28be00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa268066f60 0x7fa268069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0011f0 tx=0x7fa27c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.461+0000 7fa288c00640 1 -- 172.21.15.46:0/1233945568 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa284015fb0 con 0x7fa28c16c690 2024-04-25T08:53:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.657+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa28c069080 con 0x7fa28c16c690 2024-04-25T08:53:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.657+0000 7fa288c00640 1 -- 172.21.15.46:0/1233945568 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7fa2840efcb0 con 0x7fa28c16c690 2024-04-25T08:53:07.651 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:07.651 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:57.953462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6834/1724167790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1724167790},{"type":"v1","addr":"172.21.15.46:6834","nonce":1724167790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:53:07.651 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 112 2024-04-25T08:53:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa268066f60 msgr2=0x7fa268069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa268066f60 0x7fa268069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0011f0 tx=0x7fa27c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c16c690 msgr2=0x7fa28c16a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c16c690 0x7fa28c16a100 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa2840097c0 tx=0x7fa28401ac00 comp rx=0 tx=0).stop 2024-04-25T08:53:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 shutdown_connections 2024-04-25T08:53:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa268066f60 0x7fa268069420 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c16c690 0x7fa28c16a100 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c163400 0x7fa28c16c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 --2- 172.21.15.46:0/1233945568 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa28c06d980 0x7fa28c15d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 >> 172.21.15.46:0/1233945568 conn(0x7fa28c0b12a0 msgr2=0x7fa28c161820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 shutdown_connections 2024-04-25T08:53:07.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.661+0000 7fa292e00640 1 -- 172.21.15.46:0/1233945568 wait complete. 2024-04-25T08:53:07.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:07.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.821+0000 7fc302a00640 1 Processor -- start 2024-04-25T08:53:07.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.821+0000 7fc302a00640 1 -- start start 2024-04-25T08:53:07.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.821+0000 7fc302a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc2fc15a750 0x7fc2fc06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.821+0000 7fc302a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc0cc9e0 0x7fc2fc06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.821+0000 7fc302a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc06d5e0 con 0x7fc2fc0aed10 2024-04-25T08:53:07.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.821+0000 7fc302a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc15ab30 con 0x7fc2fc15a750 2024-04-25T08:53:07.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.821+0000 7fc302a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0ce4f0 con 0x7fc2fc0cc9e0 2024-04-25T08:53:07.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc2fb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc2fc0aed10 0x7fc2fc0b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60250/0 (socket says 172.21.15.46:60250) 2024-04-25T08:53:07.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc2fb400640 1 -- 172.21.15.46:0/1743952004 learned_addr learned my addr 172.21.15.46:0/1743952004 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:07.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc300a00640 1 --2- 172.21.15.46:0/1743952004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc0cc9e0 0x7fc2fc06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1743952004 >> v2:172.21.15.82:3300/0 conn(0x7fc2fc15a750 0x7fc2fc06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc2faa00640 1 -- 172.21.15.46:0/1743952004 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878766888 0 0) 0x7fc2fc06d5e0 con 0x7fc2fc0aed10 2024-04-25T08:53:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc2faa00640 1 -- 172.21.15.46:0/1743952004 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2e0003530 con 0x7fc2fc0aed10 2024-04-25T08:53:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1743952004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc0cc9e0 msgr2=0x7fc2fc06f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1743952004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc0cc9e0 0x7fc2fc06f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1743952004 >> v1:172.21.15.46:6789/0 conn(0x7fc2fc0aed10 legacy=0x7fc2fc0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc2fbe00640 1 -- 172.21.15.46:0/1743952004 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0ce720 con 0x7fc2fc15a750 2024-04-25T08:53:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.825+0000 7fc300a00640 1 --2- 172.21.15.46:0/1743952004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc0cc9e0 0x7fc2fc06f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:07.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/1743952004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2fc15a750 0x7fc2fc06ec10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0097c0 tx=0x7fc2ec01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=607f1c577040615f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc2faa00640 1 -- 172.21.15.46:0/1743952004 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0b2070 con 0x7fc2fc15a750 2024-04-25T08:53:07.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc2faa00640 1 -- 172.21.15.46:0/1743952004 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec0ad640 con 0x7fc2fc15a750 2024-04-25T08:53:07.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc2faa00640 1 -- 172.21.15.46:0/1743952004 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0ada10 con 0x7fc2fc15a750 2024-04-25T08:53:07.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 -- 172.21.15.46:0/1743952004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2fc15a750 msgr2=0x7fc2fc06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 --2- 172.21.15.46:0/1743952004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2fc15a750 0x7fc2fc06ec10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0097c0 tx=0x7fc2ec01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 -- 172.21.15.46:0/1743952004 shutdown_connections 2024-04-25T08:53:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 --2- 172.21.15.46:0/1743952004 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc0cc9e0 0x7fc2fc06f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 --2- 172.21.15.46:0/1743952004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2fc15a750 0x7fc2fc06ec10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 -- 172.21.15.46:0/1743952004 >> 172.21.15.46:0/1743952004 conn(0x7fc2fc0b12a0 msgr2=0x7fc2fc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 -- 172.21.15.46:0/1743952004 shutdown_connections 2024-04-25T08:53:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 -- 172.21.15.46:0/1743952004 wait complete. 2024-04-25T08:53:07.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 Processor -- start 2024-04-25T08:53:07.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 -- start start 2024-04-25T08:53:07.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc0aed10 0x7fc2fc164660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2fc0cc9e0 0x7fc2fc14f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc15a750 0x7fc2fc151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc167230 con 0x7fc2fc0aed10 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc06d270 con 0x7fc2fc0cc9e0 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc302a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc1673b0 con 0x7fc2fc15a750 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc0aed10 0x7fc2fc164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc0aed10 0x7fc2fc164660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60200/0 (socket says 172.21.15.46:60200) 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.829+0000 7fc2fbe00640 1 -- 172.21.15.46:0/3357451814 learned_addr learned my addr 172.21.15.46:0/3357451814 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc300a00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc15a750 0x7fc2fc151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2fb400640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2fc0cc9e0 0x7fc2fc14f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2fbe00640 1 -- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc15a750 msgr2=0x7fc2fc151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc15a750 0x7fc2fc151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2fbe00640 1 -- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2fc0cc9e0 msgr2=0x7fc2fc14f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2fc0cc9e0 0x7fc2fc14f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2fbe00640 1 -- 172.21.15.46:0/3357451814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc073e00 con 0x7fc2fc0aed10 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc300a00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc15a750 0x7fc2fc151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:07.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2fbe00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc0aed10 0x7fc2fc164660 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec01a0d0 tx=0x7fc2ec019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2f3e00640 1 -- 172.21.15.46:0/3357451814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0b2040 con 0x7fc2fc0aed10 2024-04-25T08:53:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2f3e00640 1 -- 172.21.15.46:0/3357451814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec0bc070 con 0x7fc2fc0aed10 2024-04-25T08:53:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc164ba0 con 0x7fc2fc0aed10 2024-04-25T08:53:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2f3e00640 1 -- 172.21.15.46:0/3357451814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec006e20 con 0x7fc2fc0aed10 2024-04-25T08:53:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc1570a0 con 0x7fc2fc0aed10 2024-04-25T08:53:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc069080 con 0x7fc2fc0aed10 2024-04-25T08:53:07.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2f3e00640 1 -- 172.21.15.46:0/3357451814 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2ec0b6670 con 0x7fc2fc0aed10 2024-04-25T08:53:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2f3e00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc2dc066f60 0x7fc2dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.833+0000 7fc2f3e00640 1 -- 172.21.15.46:0/3357451814 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc2ec11f1f0 con 0x7fc2fc0aed10 2024-04-25T08:53:07.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.837+0000 7fc2fb400640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc2dc066f60 0x7fc2dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.837+0000 7fc2fb400640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc2dc066f60 0x7fc2dc069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc06e420 tx=0x7fc2e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:07.837+0000 7fc2f3e00640 1 -- 172.21.15.46:0/3357451814 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2ec016090 con 0x7fc2fc0aed10 2024-04-25T08:53:08.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.033+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2fc06e8d0 con 0x7fc2fc0aed10 2024-04-25T08:53:08.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.033+0000 7fc2f3e00640 1 -- 172.21.15.46:0/3357451814 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4940 (secure 0 0 0) 0x7fc2ec0efe90 con 0x7fc2fc0aed10 2024-04-25T08:53:08.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:08.027 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:28.147634+0000","modified":"2024-04-25T08:52:57.953462+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9393":{"gid":9393,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6834/1724167790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1724167790},{"type":"v1","addr":"172.21.15.46:6834","nonce":1724167790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:53:08.027 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 112 2024-04-25T08:53:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc2dc066f60 msgr2=0x7fc2dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc2dc066f60 0x7fc2dc069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc06e420 tx=0x7fc2e8009040 comp rx=0 tx=0).stop 2024-04-25T08:53:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc0aed10 msgr2=0x7fc2fc164660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc0aed10 0x7fc2fc164660 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec01a0d0 tx=0x7fc2ec019240 comp rx=0 tx=0).stop 2024-04-25T08:53:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 shutdown_connections 2024-04-25T08:53:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc2dc066f60 0x7fc2dc069420 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc2fc15a750 0x7fc2fc151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2fc0cc9e0 0x7fc2fc14f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 --2- 172.21.15.46:0/3357451814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2fc0aed10 0x7fc2fc164660 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 >> 172.21.15.46:0/3357451814 conn(0x7fc2fc0b12a0 msgr2=0x7fc2fc0b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 shutdown_connections 2024-04-25T08:53:08.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.037+0000 7fc302a00640 1 -- 172.21.15.46:0/3357451814 wait complete. 2024-04-25T08:53:08.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:08.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.197+0000 7f5d7cc00640 1 Processor -- start 2024-04-25T08:53:08.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.197+0000 7f5d7cc00640 1 -- start start 2024-04-25T08:53:08.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.197+0000 7f5d7cc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5d7815b980 0x7f5d780cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.197+0000 7f5d7cc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7815bd60 con 0x7f5d780ceee0 2024-04-25T08:53:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.197+0000 7f5d7cc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780cd420 con 0x7f5d780ce580 2024-04-25T08:53:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.197+0000 7f5d7cc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d780cd700 con 0x7f5d7815b980 2024-04-25T08:53:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.197+0000 7f5d76800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5d780ceee0 0x7f5d780d0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51742/0 (socket says 172.21.15.46:51742) 2024-04-25T08:53:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.197+0000 7f5d76800640 1 -- 172.21.15.46:0/3224255267 learned_addr learned my addr 172.21.15.46:0/3224255267 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:08.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.197+0000 7f5d75e00640 1 --2- 172.21.15.46:0/3224255267 >> v2:172.21.15.82:3300/0 conn(0x7f5d7815b980 0x7f5d780cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75400640 1 -- 172.21.15.46:0/3224255267 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591961516 0 0) 0x7f5d7815bd60 con 0x7f5d780ceee0 2024-04-25T08:53:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75400640 1 -- 172.21.15.46:0/3224255267 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d5c003530 con 0x7f5d780ceee0 2024-04-25T08:53:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75400640 1 -- 172.21.15.46:0/3224255267 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353121333 0 0) 0x7f5d780cd420 con 0x7f5d780ce580 2024-04-25T08:53:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75400640 1 -- 172.21.15.46:0/3224255267 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7815bd60 con 0x7f5d780ce580 2024-04-25T08:53:08.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75400640 1 -- 172.21.15.46:0/3224255267 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1706469649 0 0) 0x7f5d5c003530 con 0x7f5d780ceee0 2024-04-25T08:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75e00640 1 -- 172.21.15.46:0/3224255267 >> v1:172.21.15.82:6789/0 conn(0x7f5d780ce580 legacy=0x7f5d780b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75e00640 1 -- 172.21.15.46:0/3224255267 >> v1:172.21.15.46:6789/0 conn(0x7f5d780ceee0 legacy=0x7f5d780d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75e00640 1 -- 172.21.15.46:0/3224255267 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d780cd9e0 con 0x7f5d7815b980 2024-04-25T08:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75e00640 1 --2- 172.21.15.46:0/3224255267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d7815b980 0x7f5d780cce40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5d680097c0 tx=0x7f5d6801c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50cc4eda6769f423 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75400640 1 -- 172.21.15.46:0/3224255267 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d680b0070 con 0x7f5d7815b980 2024-04-25T08:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75400640 1 -- 172.21.15.46:0/3224255267 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d680ab400 con 0x7f5d7815b980 2024-04-25T08:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.201+0000 7f5d75400640 1 -- 172.21.15.46:0/3224255267 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70001680 con 0x7f5d7815b980 2024-04-25T08:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 -- 172.21.15.46:0/3224255267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d7815b980 msgr2=0x7f5d780cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 --2- 172.21.15.46:0/3224255267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d7815b980 0x7f5d780cce40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5d680097c0 tx=0x7f5d6801c390 comp rx=0 tx=0).stop 2024-04-25T08:53:08.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 -- 172.21.15.46:0/3224255267 shutdown_connections 2024-04-25T08:53:08.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 --2- 172.21.15.46:0/3224255267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d7815b980 0x7f5d780cce40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 -- 172.21.15.46:0/3224255267 >> 172.21.15.46:0/3224255267 conn(0x7f5d780728c0 msgr2=0x7f5d78099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:08.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 -- 172.21.15.46:0/3224255267 shutdown_connections 2024-04-25T08:53:08.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 -- 172.21.15.46:0/3224255267 wait complete. 2024-04-25T08:53:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 Processor -- start 2024-04-25T08:53:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 -- start start 2024-04-25T08:53:08.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d780ce580 0x7f5d7814e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d780ceee0 0x7f5d780ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d7815b980 0x7f5d780ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780ce980 con 0x7f5d780ceee0 2024-04-25T08:53:08.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780d1320 con 0x7f5d7815b980 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d7cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780ceb00 con 0x7f5d780ce580 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d6d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d780ceee0 0x7f5d780ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d6d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d780ceee0 0x7f5d780ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42418/0 (socket says 172.21.15.46:42418) 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d6d400640 1 -- 172.21.15.46:0/1178773842 learned_addr learned my addr 172.21.15.46:0/1178773842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d75e00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d780ce580 0x7f5d7814e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d76800640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d7815b980 0x7f5d780ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d6d400640 1 -- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d780ce580 msgr2=0x7f5d7814e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d6d400640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d780ce580 0x7f5d7814e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d6d400640 1 -- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d7815b980 msgr2=0x7f5d780ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d6d400640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d7815b980 0x7f5d780ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d6d400640 1 -- 172.21.15.46:0/1178773842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7809b380 con 0x7f5d780ceee0 2024-04-25T08:53:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.205+0000 7f5d6d400640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d780ceee0 0x7f5d780ae110 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c870 tx=0x7f5d6000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d6ea00640 1 -- 172.21.15.46:0/1178773842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60003b80 con 0x7f5d780ceee0 2024-04-25T08:53:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d6ea00640 1 -- 172.21.15.46:0/1178773842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60003d20 con 0x7f5d780ceee0 2024-04-25T08:53:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d780b0fc0 con 0x7f5d780ceee0 2024-04-25T08:53:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d6ea00640 1 -- 172.21.15.46:0/1178773842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6000aa90 con 0x7f5d780ceee0 2024-04-25T08:53:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d780b1280 con 0x7f5d780ceee0 2024-04-25T08:53:08.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d78069080 con 0x7f5d780ceee0 2024-04-25T08:53:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d6ea00640 1 -- 172.21.15.46:0/1178773842 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d6000ac30 con 0x7f5d780ceee0 2024-04-25T08:53:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d6ea00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d54066f60 0x7f5d54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d6ea00640 1 -- 172.21.15.46:0/1178773842 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f5d600851f0 con 0x7f5d780ceee0 2024-04-25T08:53:08.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d75e00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d54066f60 0x7f5d54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.209+0000 7f5d75e00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d54066f60 0x7f5d54069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5d680ad000 tx=0x7f5d680a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:08.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.213+0000 7f5d6ea00640 1 -- 172.21.15.46:0/1178773842 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d7809b380 con 0x7f5d780ceee0 2024-04-25T08:53:08.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d7815e690 con 0x7f5d780ceee0 2024-04-25T08:53:08.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d6ea00640 1 -- 172.21.15.46:0/1178773842 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17159 (secure 0 0 0) 0x7f5d60055fb0 con 0x7f5d780ceee0 2024-04-25T08:53:08.369 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:08.369 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":123,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:52:53.922654+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:25.941254+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:27.155001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6832/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6835/51204523":"2024-04-26T08:52:42.856639+0000","172.21.15.46:6833/51204523":"2024-04-26T08:52:42.856639+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d54066f60 msgr2=0x7f5d54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d54066f60 0x7f5d54069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5d680ad000 tx=0x7f5d680a6000 comp rx=0 tx=0).stop 2024-04-25T08:53:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d780ceee0 msgr2=0x7f5d780ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d780ceee0 0x7f5d780ae110 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f5d6000c870 tx=0x7f5d6000cd30 comp rx=0 tx=0).stop 2024-04-25T08:53:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 shutdown_connections 2024-04-25T08:53:08.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d54066f60 0x7f5d54069420 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5d7815b980 0x7f5d780ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d780ceee0 0x7f5d780ae110 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 --2- 172.21.15.46:0/1178773842 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5d780ce580 0x7f5d7814e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 >> 172.21.15.46:0/1178773842 conn(0x7f5d780728c0 msgr2=0x7f5d7815d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 shutdown_connections 2024-04-25T08:53:08.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.377+0000 7f5d7cc00640 1 -- 172.21.15.46:0/1178773842 wait complete. 2024-04-25T08:53:08.391 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:53:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f1d400640 1 Processor -- start 2024-04-25T08:53:08.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f1d400640 1 -- start start 2024-04-25T08:53:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f1d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 0x7f2f1815d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f1d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f180afd40 0x7f2f1815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f1d400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1806ed70 con 0x7f2f180cf520 2024-04-25T08:53:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f1d400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f180afb40 con 0x7f2f180afd40 2024-04-25T08:53:08.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f1d400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f180d13f0 con 0x7f2f180cf140 2024-04-25T08:53:08.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f0de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f180afd40 0x7f2f1815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f0de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f180afd40 0x7f2f1815dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42428/0 (socket says 172.21.15.46:42428) 2024-04-25T08:53:08.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f0de00640 1 -- 172.21.15.46:0/850058997 learned_addr learned my addr 172.21.15.46:0/850058997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:08.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.541+0000 7f2f16800640 1 --2- 172.21.15.46:0/850058997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 0x7f2f1815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f0de00640 1 -- 172.21.15.46:0/850058997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 msgr2=0x7f2f1815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:08.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f0de00640 1 --2- 172.21.15.46:0/850058997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 0x7f2f1815d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f0de00640 1 -- 172.21.15.46:0/850058997 >> v1:172.21.15.82:6789/0 conn(0x7f2f180cf520 legacy=0x7f2f180ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f0de00640 1 -- 172.21.15.46:0/850058997 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18162530 con 0x7f2f180afd40 2024-04-25T08:53:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f0de00640 1 --2- 172.21.15.46:0/850058997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 0x7f2f1815dad0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009870 tx=0x7f2f0801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a3a064d063dc59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f15e00640 1 -- 172.21.15.46:0/850058997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f080b0070 con 0x7f2f180afd40 2024-04-25T08:53:08.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f15e00640 1 -- 172.21.15.46:0/850058997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f080ab600 con 0x7f2f180afd40 2024-04-25T08:53:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f15e00640 1 -- 172.21.15.46:0/850058997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f080ab9d0 con 0x7f2f180afd40 2024-04-25T08:53:08.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f1d400640 1 -- 172.21.15.46:0/850058997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 msgr2=0x7f2f1815dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f1d400640 1 --2- 172.21.15.46:0/850058997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 0x7f2f1815dad0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009870 tx=0x7f2f0801c440 comp rx=0 tx=0).stop 2024-04-25T08:53:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f1d400640 1 -- 172.21.15.46:0/850058997 shutdown_connections 2024-04-25T08:53:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f1d400640 1 --2- 172.21.15.46:0/850058997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 0x7f2f1815dad0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f1d400640 1 --2- 172.21.15.46:0/850058997 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 0x7f2f1815d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f1d400640 1 -- 172.21.15.46:0/850058997 >> 172.21.15.46:0/850058997 conn(0x7f2f180728c0 msgr2=0x7f2f18099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f1d400640 1 -- 172.21.15.46:0/850058997 shutdown_connections 2024-04-25T08:53:08.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.545+0000 7f2f1d400640 1 -- 172.21.15.46:0/850058997 wait complete. 2024-04-25T08:53:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 Processor -- start 2024-04-25T08:53:08.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 -- start start 2024-04-25T08:53:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 0x7f2f180cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 0x7f2f18158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f180cf520 0x7f2f181591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f180af7f0 con 0x7f2f180afd40 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f18162010 con 0x7f2f180cf520 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1806b600 con 0x7f2f180cf140 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f16800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 0x7f2f180cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f16800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 0x7f2f180cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42432/0 (socket says 172.21.15.46:42432) 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f16800640 1 -- 172.21.15.46:0/2169620176 learned_addr learned my addr 172.21.15.46:0/2169620176 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f17200640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f180cf520 0x7f2f181591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f0de00640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 0x7f2f18158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f16800640 1 -- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 msgr2=0x7f2f18158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f16800640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 0x7f2f18158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f16800640 1 -- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f180cf520 msgr2=0x7f2f181591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f16800640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f180cf520 0x7f2f181591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f16800640 1 -- 172.21.15.46:0/2169620176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f180b4be0 con 0x7f2f180afd40 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f0de00640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 0x7f2f18158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f16800640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 0x7f2f180cc260 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d5f0 tx=0x7f2f0000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f0f400640 1 -- 172.21.15.46:0/2169620176 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f000125f0 con 0x7f2f180afd40 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f0f400640 1 -- 172.21.15.46:0/2169620176 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00019070 con 0x7f2f180afd40 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f0f400640 1 -- 172.21.15.46:0/2169620176 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f000129b0 con 0x7f2f180afd40 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f180b4ec0 con 0x7f2f180afd40 2024-04-25T08:53:08.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.549+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f180b5480 con 0x7f2f180afd40 2024-04-25T08:53:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.553+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1806ccb0 con 0x7f2f180afd40 2024-04-25T08:53:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.553+0000 7f2f0f400640 1 -- 172.21.15.46:0/2169620176 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f00012b50 con 0x7f2f180afd40 2024-04-25T08:53:08.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.553+0000 7f2f0f400640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ef4066f60 0x7f2ef4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.553+0000 7f2f0f400640 1 -- 172.21.15.46:0/2169620176 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f2f00085620 con 0x7f2f180afd40 2024-04-25T08:53:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.553+0000 7f2f0de00640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ef4066f60 0x7f2ef4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.553+0000 7f2f0de00640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ef4066f60 0x7f2ef4069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2f080ad000 tx=0x7f2f08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:08.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.557+0000 7f2f0f400640 1 -- 172.21.15.46:0/2169620176 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f00056350 con 0x7f2f180afd40 2024-04-25T08:53:08.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.757+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2f18069080 con 0x7f2f180afd40 2024-04-25T08:53:08.750 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-04-25T08:53:08.757+0000 7fccb1800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:53:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.761+0000 7f2f0f400640 1 -- 172.21.15.46:0/2169620176 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2f00055c50 con 0x7f2f180afd40 2024-04-25T08:53:08.752 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.761+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ef4066f60 msgr2=0x7f2ef4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.761+0000 7f2f1d400640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ef4066f60 0x7f2ef4069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2f080ad000 tx=0x7f2f08008040 comp rx=0 tx=0).stop 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.761+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 msgr2=0x7f2f180cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.761+0000 7f2f1d400640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 0x7f2f180cc260 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000d5f0 tx=0x7f2f0000dab0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.765+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 shutdown_connections 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.765+0000 7f2f1d400640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f2ef4066f60 0x7f2ef4069420 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.765+0000 7f2f1d400640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f180cf520 0x7f2f181591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.765+0000 7f2f1d400640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f2f180cf140 0x7f2f18158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.765+0000 7f2f1d400640 1 --2- 172.21.15.46:0/2169620176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f180afd40 0x7f2f180cc260 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.765+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 >> 172.21.15.46:0/2169620176 conn(0x7f2f180728c0 msgr2=0x7f2f180ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.765+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 shutdown_connections 2024-04-25T08:53:08.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.765+0000 7f2f1d400640 1 -- 172.21.15.46:0/2169620176 wait complete. 2024-04-25T08:53:08.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:53:08.810 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-04-25T08:53:08.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.933+0000 7f7868e00640 1 Processor -- start 2024-04-25T08:53:08.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f7868e00640 1 -- start start 2024-04-25T08:53:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f7868e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7864099c60 0x7f7864160140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f7868e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78640cf210 0x7f78640cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f7868e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78640cdb10 con 0x7f786409a040 2024-04-25T08:53:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f7868e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78640cc300 con 0x7f78640cf210 2024-04-25T08:53:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f7868e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78640cc530 con 0x7f7864099c60 2024-04-25T08:53:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f785be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78640cf210 0x7f78640cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f785be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78640cf210 0x7f78640cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42458/0 (socket says 172.21.15.46:42458) 2024-04-25T08:53:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f785be00640 1 -- 172.21.15.46:0/1461526382 learned_addr learned my addr 172.21.15.46:0/1461526382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:08.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f7861e00640 1 --2- 172.21.15.46:0/1461526382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7864099c60 0x7f7864160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f785be00640 1 -- 172.21.15.46:0/1461526382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7864099c60 msgr2=0x7f7864160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f785be00640 1 --2- 172.21.15.46:0/1461526382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7864099c60 0x7f7864160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f785be00640 1 -- 172.21.15.46:0/1461526382 >> v1:172.21.15.82:6789/0 conn(0x7f786409a040 legacy=0x7f786415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.937+0000 7f785be00640 1 -- 172.21.15.46:0/1461526382 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78640cc810 con 0x7f78640cf210 2024-04-25T08:53:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f785be00640 1 --2- 172.21.15.46:0/1461526382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 0x7f78640cd5d0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f784c009870 tx=0x7f784c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b152ae9a9685f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7861400640 1 -- 172.21.15.46:0/1461526382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0ab650 con 0x7f78640cf210 2024-04-25T08:53:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7861400640 1 -- 172.21.15.46:0/1461526382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784c0b0070 con 0x7f78640cf210 2024-04-25T08:53:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7861400640 1 -- 172.21.15.46:0/1461526382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c0aba10 con 0x7f78640cf210 2024-04-25T08:53:08.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 -- 172.21.15.46:0/1461526382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 msgr2=0x7f78640cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 --2- 172.21.15.46:0/1461526382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 0x7f78640cd5d0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f784c009870 tx=0x7f784c01c560 comp rx=0 tx=0).stop 2024-04-25T08:53:08.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 -- 172.21.15.46:0/1461526382 shutdown_connections 2024-04-25T08:53:08.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 --2- 172.21.15.46:0/1461526382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 0x7f78640cd5d0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 --2- 172.21.15.46:0/1461526382 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f7864099c60 0x7f7864160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 -- 172.21.15.46:0/1461526382 >> 172.21.15.46:0/1461526382 conn(0x7f78640b3130 msgr2=0x7f78640b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 -- 172.21.15.46:0/1461526382 shutdown_connections 2024-04-25T08:53:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 -- 172.21.15.46:0/1461526382 wait complete. 2024-04-25T08:53:08.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 Processor -- start 2024-04-25T08:53:08.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.941+0000 7f7868e00640 1 -- start start 2024-04-25T08:53:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7868e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864099c60 0x7f7864155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7868e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786409a040 0x7f78640c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7868e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 0x7f78640c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7868e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78640b1930 con 0x7f78640cf210 2024-04-25T08:53:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7868e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78640b17b0 con 0x7f7864099c60 2024-04-25T08:53:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7868e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78640b1ab0 con 0x7f786409a040 2024-04-25T08:53:08.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7862800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 0x7f78640c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7862800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 0x7f78640c74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42472/0 (socket says 172.21.15.46:42472) 2024-04-25T08:53:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7862800640 1 -- 172.21.15.46:0/2698185586 learned_addr learned my addr 172.21.15.46:0/2698185586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7861e00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864099c60 0x7f7864155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f785be00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786409a040 0x7f78640c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7862800640 1 -- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786409a040 msgr2=0x7f78640c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7862800640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786409a040 0x7f78640c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7862800640 1 -- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864099c60 msgr2=0x7f7864155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7862800640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864099c60 0x7f7864155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7862800640 1 -- 172.21.15.46:0/2698185586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78640c5a50 con 0x7f78640cf210 2024-04-25T08:53:08.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7862800640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 0x7f78640c74a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c770 tx=0x7f785c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f785a000640 1 -- 172.21.15.46:0/2698185586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c003b80 con 0x7f78640cf210 2024-04-25T08:53:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f785a000640 1 -- 172.21.15.46:0/2698185586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c003d20 con 0x7f78640cf210 2024-04-25T08:53:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78640c5d30 con 0x7f78640cf210 2024-04-25T08:53:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f785a000640 1 -- 172.21.15.46:0/2698185586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00aa90 con 0x7f78640cf210 2024-04-25T08:53:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78640c62f0 con 0x7f78640cf210 2024-04-25T08:53:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7864069010 con 0x7f78640cf210 2024-04-25T08:53:08.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.945+0000 7f785a000640 1 -- 172.21.15.46:0/2698185586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f785c00ac30 con 0x7f78640cf210 2024-04-25T08:53:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.949+0000 7f785a000640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7840066f60 0x7f7840069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.949+0000 7f785a000640 1 -- 172.21.15.46:0/2698185586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f785c0850b0 con 0x7f78640cf210 2024-04-25T08:53:08.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.949+0000 7f7861e00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7840066f60 0x7f7840069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:08.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.949+0000 7f7861e00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7840066f60 0x7f7840069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7864073f50 tx=0x7f7854008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:08.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:08.953+0000 7f785a000640 1 -- 172.21.15.46:0/2698185586 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f785c0557e0 con 0x7f78640cf210 2024-04-25T08:53:09.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.153+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7864160320 con 0x7f78640cf210 2024-04-25T08:53:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.761+0000 7f785a000640 1 -- 172.21.15.46:0/2698185586 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 104+0+0 (secure 0 0 0) 0x7f785c059880 con 0x7f78640cf210 2024-04-25T08:53:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7840066f60 msgr2=0x7f7840069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7840066f60 0x7f7840069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7864073f50 tx=0x7f7854008040 comp rx=0 tx=0).stop 2024-04-25T08:53:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 msgr2=0x7f78640c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 0x7f78640c74a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c770 tx=0x7f785c00cc30 comp rx=0 tx=0).stop 2024-04-25T08:53:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 shutdown_connections 2024-04-25T08:53:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f7840066f60 0x7f7840069420 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78640cf210 0x7f78640c74a0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f786409a040 0x7f78640c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 --2- 172.21.15.46:0/2698185586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864099c60 0x7f7864155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 >> 172.21.15.46:0/2698185586 conn(0x7f78640b3130 msgr2=0x7f78640b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 shutdown_connections 2024-04-25T08:53:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.765+0000 7f7868e00640 1 -- 172.21.15.46:0/2698185586 wait complete. 2024-04-25T08:53:09.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:53:09.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.981+0000 7f92a3200640 1 Processor -- start 2024-04-25T08:53:09.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.981+0000 7f92a3200640 1 -- start start 2024-04-25T08:53:09.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.981+0000 7f92a3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f929c15c680 0x7f929c166370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:09.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.981+0000 7f92a3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c06d330 con 0x7f929c0718f0 2024-04-25T08:53:09.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.981+0000 7f92a3200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c06d610 con 0x7f929c13e6b0 2024-04-25T08:53:09.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.981+0000 7f92a3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929c15cce0 con 0x7f929c15c680 2024-04-25T08:53:09.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.981+0000 7f9293400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f929c0718f0 0x7f929c15bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51764/0 (socket says 172.21.15.46:51764) 2024-04-25T08:53:09.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.981+0000 7f9293400640 1 -- 172.21.15.46:0/768179524 learned_addr learned my addr 172.21.15.46:0/768179524 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:09.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.981+0000 7f92a1200640 1 --2- 172.21.15.46:0/768179524 >> v2:172.21.15.46:3300/0 conn(0x7f929c15c680 0x7f929c166370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:09.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.985+0000 7f92a1200640 1 -- 172.21.15.46:0/768179524 >> v1:172.21.15.82:6789/0 conn(0x7f929c13e6b0 legacy=0x7f929c0711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:09.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.985+0000 7f92a1200640 1 -- 172.21.15.46:0/768179524 >> v1:172.21.15.46:6789/0 conn(0x7f929c0718f0 legacy=0x7f929c15bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:09.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.985+0000 7f92a1200640 1 -- 172.21.15.46:0/768179524 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c06d8f0 con 0x7f929c15c680 2024-04-25T08:53:09.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.985+0000 7f929b400640 1 -- 172.21.15.46:0/768179524 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078123303 0 0) 0x7f929c06d330 con 0x7f929c0718f0 2024-04-25T08:53:09.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.985+0000 7f92a1200640 1 --2- 172.21.15.46:0/768179524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15c680 0x7f929c166370 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9294009dc0 tx=0x7f929401c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83a58051beaca9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:09.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.985+0000 7f929b400640 1 -- 172.21.15.46:0/768179524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92940a8650 con 0x7f929c15c680 2024-04-25T08:53:09.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.985+0000 7f929b400640 1 -- 172.21.15.46:0/768179524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92940a87f0 con 0x7f929c15c680 2024-04-25T08:53:09.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.985+0000 7f929b400640 1 -- 172.21.15.46:0/768179524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92940a8bc0 con 0x7f929c15c680 2024-04-25T08:53:09.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 -- 172.21.15.46:0/768179524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15c680 msgr2=0x7f929c166370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:09.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 --2- 172.21.15.46:0/768179524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15c680 0x7f929c166370 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f9294009dc0 tx=0x7f929401c830 comp rx=0 tx=0).stop 2024-04-25T08:53:09.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 -- 172.21.15.46:0/768179524 shutdown_connections 2024-04-25T08:53:09.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 --2- 172.21.15.46:0/768179524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c15c680 0x7f929c166370 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:09.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 -- 172.21.15.46:0/768179524 >> 172.21.15.46:0/768179524 conn(0x7f929c067050 msgr2=0x7f929c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:09.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 -- 172.21.15.46:0/768179524 shutdown_connections 2024-04-25T08:53:09.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 -- 172.21.15.46:0/768179524 wait complete. 2024-04-25T08:53:09.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 Processor -- start 2024-04-25T08:53:09.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 -- start start 2024-04-25T08:53:09.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f929c0718f0 0x7f929c1392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:09.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c13e6b0 0x7f929c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:09.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c15c680 0x7f929c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:09.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1696f0 con 0x7f929c13e6b0 2024-04-25T08:53:09.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c169870 con 0x7f929c15c680 2024-04-25T08:53:09.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a3200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1699f0 con 0x7f929c0718f0 2024-04-25T08:53:09.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f9293400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c13e6b0 0x7f929c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f9293400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c13e6b0 0x7f929c15a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42490/0 (socket says 172.21.15.46:42490) 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f9293400640 1 -- 172.21.15.46:0/655643739 learned_addr learned my addr 172.21.15.46:0/655643739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a1200640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c15c680 0x7f929c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f929be00640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f929c0718f0 0x7f929c1392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f9293400640 1 -- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f929c0718f0 msgr2=0x7f929c1392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f9293400640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f929c0718f0 0x7f929c1392b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f9293400640 1 -- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c15c680 msgr2=0x7f929c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f9293400640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c15c680 0x7f929c15ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f9293400640 1 -- 172.21.15.46:0/655643739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c1397f0 con 0x7f929c13e6b0 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f92a1200640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c15c680 0x7f929c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:09.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.989+0000 7f929be00640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f929c0718f0 0x7f929c1392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f9293400640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c13e6b0 0x7f929c15a810 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f928400c440 tx=0x7f928400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f9298c00640 1 -- 172.21.15.46:0/655643739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284018070 con 0x7f929c13e6b0 2024-04-25T08:53:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f9298c00640 1 -- 172.21.15.46:0/655643739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92840074e0 con 0x7f929c13e6b0 2024-04-25T08:53:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929c0b3520 con 0x7f929c13e6b0 2024-04-25T08:53:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f9298c00640 1 -- 172.21.15.46:0/655643739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9284004a30 con 0x7f929c13e6b0 2024-04-25T08:53:09.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c154730 con 0x7f929c13e6b0 2024-04-25T08:53:09.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929c069080 con 0x7f929c13e6b0 2024-04-25T08:53:09.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f9298c00640 1 -- 172.21.15.46:0/655643739 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f928401b070 con 0x7f929c13e6b0 2024-04-25T08:53:09.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f9298c00640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9278066f60 0x7f9278069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:09.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f9298c00640 1 -- 172.21.15.46:0/655643739 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f92840845d0 con 0x7f929c13e6b0 2024-04-25T08:53:09.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.993+0000 7f929be00640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9278066f60 0x7f9278069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:09.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.997+0000 7f929be00640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9278066f60 0x7f9278069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f929c1fa6b0 tx=0x7f928c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:09.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:09.997+0000 7f9298c00640 1 -- 172.21.15.46:0/655643739 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92840552b0 con 0x7f929c13e6b0 2024-04-25T08:53:10.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:10.161+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f929c0cf240 con 0x7f929c13e6b0 2024-04-25T08:53:10.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:10.997+0000 7f9298c00640 1 -- 172.21.15.46:0/655643739 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9284054bb0 con 0x7f929c13e6b0 2024-04-25T08:53:10.988 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:53:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:10.997+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9278066f60 msgr2=0x7f9278069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9278066f60 0x7f9278069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f929c1fa6b0 tx=0x7f928c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c13e6b0 msgr2=0x7f929c15a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c13e6b0 0x7f929c15a810 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f928400c440 tx=0x7f928400c900 comp rx=0 tx=0).stop 2024-04-25T08:53:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 shutdown_connections 2024-04-25T08:53:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9278066f60 0x7f9278069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f929c15c680 0x7f929c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929c13e6b0 0x7f929c15a810 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 --2- 172.21.15.46:0/655643739 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f929c0718f0 0x7f929c1392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 >> 172.21.15.46:0/655643739 conn(0x7f929c067050 msgr2=0x7f929c06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 shutdown_connections 2024-04-25T08:53:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.001+0000 7f92a3200640 1 -- 172.21.15.46:0/655643739 wait complete. 2024-04-25T08:53:11.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:53:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd8000640 1 Processor -- start 2024-04-25T08:53:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd8000640 1 -- start start 2024-04-25T08:53:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd8000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 0x7f9bd006c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd8000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9bd00702a0 0x7f9bd0070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd8000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd015e910 con 0x7f9bd00d11e0 2024-04-25T08:53:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd8000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd006fe10 con 0x7f9bd00702a0 2024-04-25T08:53:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd8000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd015ebf0 con 0x7f9bd009a110 2024-04-25T08:53:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9bd00702a0 0x7f9bd0070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd5600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9bd00d11e0 0x7f9bd0099a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51776/0 (socket says 172.21.15.46:51776) 2024-04-25T08:53:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd5600640 1 -- 172.21.15.46:0/976978889 learned_addr learned my addr 172.21.15.46:0/976978889 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd4c00640 1 --2- 172.21.15.46:0/976978889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 0x7f9bd006c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd6000640 1 -- 172.21.15.46:0/976978889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 msgr2=0x7f9bd006c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd6000640 1 --2- 172.21.15.46:0/976978889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 0x7f9bd006c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd6000640 1 -- 172.21.15.46:0/976978889 >> v1:172.21.15.46:6789/0 conn(0x7f9bd00d11e0 legacy=0x7f9bd0099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bd6000640 1 -- 172.21.15.46:0/976978889 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd015eed0 con 0x7f9bd00702a0 2024-04-25T08:53:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.177+0000 7f9bcfe00640 1 -- 172.21.15.46:0/976978889 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677236573 0 0) 0x7f9bd015e910 con 0x7f9bd00d11e0 2024-04-25T08:53:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd6000640 1 --2- 172.21.15.46:0/976978889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 0x7f9bd0070750 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d90 tx=0x7f9bc801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a639c64e9d3035e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bcfe00640 1 -- 172.21.15.46:0/976978889 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80b2070 con 0x7f9bd00702a0 2024-04-25T08:53:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bcfe00640 1 -- 172.21.15.46:0/976978889 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc80a59d0 con 0x7f9bd00702a0 2024-04-25T08:53:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bcfe00640 1 -- 172.21.15.46:0/976978889 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80a5da0 con 0x7f9bd00702a0 2024-04-25T08:53:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 -- 172.21.15.46:0/976978889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 msgr2=0x7f9bd0070750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 --2- 172.21.15.46:0/976978889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 0x7f9bd0070750 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d90 tx=0x7f9bc801c970 comp rx=0 tx=0).stop 2024-04-25T08:53:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 -- 172.21.15.46:0/976978889 shutdown_connections 2024-04-25T08:53:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 --2- 172.21.15.46:0/976978889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 0x7f9bd0070750 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 --2- 172.21.15.46:0/976978889 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 0x7f9bd006c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 -- 172.21.15.46:0/976978889 >> 172.21.15.46:0/976978889 conn(0x7f9bd00d16c0 msgr2=0x7f9bd00d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 -- 172.21.15.46:0/976978889 shutdown_connections 2024-04-25T08:53:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 -- 172.21.15.46:0/976978889 wait complete. 2024-04-25T08:53:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 Processor -- start 2024-04-25T08:53:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.181+0000 7f9bd8000640 1 -- start start 2024-04-25T08:53:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd8000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 0x7f9bd014bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd8000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 0x7f9bd014dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd8000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bd00d11e0 0x7f9bd00c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd8000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd015db40 con 0x7f9bd00702a0 2024-04-25T08:53:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd8000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd015dcc0 con 0x7f9bd00d11e0 2024-04-25T08:53:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd8000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd015de40 con 0x7f9bd009a110 2024-04-25T08:53:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 0x7f9bd014bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 0x7f9bd014bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42510/0 (socket says 172.21.15.46:42510) 2024-04-25T08:53:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd5600640 1 -- 172.21.15.46:0/2958634217 learned_addr learned my addr 172.21.15.46:0/2958634217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd4c00640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 0x7f9bd014dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd6000640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bd00d11e0 0x7f9bd00c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd5600640 1 -- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 msgr2=0x7f9bd014dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd5600640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 0x7f9bd014dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd5600640 1 -- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bd00d11e0 msgr2=0x7f9bd00c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd5600640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bd00d11e0 0x7f9bd00c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd5600640 1 -- 172.21.15.46:0/2958634217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd0157ac0 con 0x7f9bd00702a0 2024-04-25T08:53:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd5600640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 0x7f9bd014bed0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400c480 tx=0x7f9bc400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bcd600640 1 -- 172.21.15.46:0/2958634217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4018070 con 0x7f9bd00702a0 2024-04-25T08:53:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bcd600640 1 -- 172.21.15.46:0/2958634217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc4007500 con 0x7f9bd00702a0 2024-04-25T08:53:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd0155100 con 0x7f9bd00702a0 2024-04-25T08:53:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bcd600640 1 -- 172.21.15.46:0/2958634217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4004a30 con 0x7f9bd00702a0 2024-04-25T08:53:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd014e0e0 con 0x7f9bd00702a0 2024-04-25T08:53:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.185+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd0069080 con 0x7f9bd00702a0 2024-04-25T08:53:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.189+0000 7f9bcd600640 1 -- 172.21.15.46:0/2958634217 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bc401b070 con 0x7f9bd00702a0 2024-04-25T08:53:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.189+0000 7f9bcd600640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bb0066f60 0x7f9bb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.189+0000 7f9bcd600640 1 -- 172.21.15.46:0/2958634217 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5921+0+0 (secure 0 0 0) 0x7f9bc40843a0 con 0x7f9bd00702a0 2024-04-25T08:53:11.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.189+0000 7f9bd4c00640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bb0066f60 0x7f9bb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:11.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.189+0000 7f9bd4c00640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bb0066f60 0x7f9bb0069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9bd01fa4f0 tx=0x7f9bbc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.193+0000 7f9bcd600640 1 -- 172.21.15.46:0/2958634217 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bc4054c70 con 0x7f9bd00702a0 2024-04-25T08:53:11.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.353+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9bd01698a0 con 0x7f9bd00702a0 2024-04-25T08:53:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:11.997+0000 7f9bcd600640 1 -- 172.21.15.46:0/2958634217 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9bc4058d10 con 0x7f9bd00702a0 2024-04-25T08:53:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-04-25T08:53:11.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bb0066f60 msgr2=0x7f9bb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:11.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bb0066f60 0x7f9bb0069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9bd01fa4f0 tx=0x7f9bbc008040 comp rx=0 tx=0).stop 2024-04-25T08:53:11.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 msgr2=0x7f9bd014bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:11.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 0x7f9bd014bed0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400c480 tx=0x7f9bc400c940 comp rx=0 tx=0).stop 2024-04-25T08:53:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 shutdown_connections 2024-04-25T08:53:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f9bb0066f60 0x7f9bb0069420 secure :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9bd01fa4f0 tx=0x7f9bbc008040 comp rx=0 tx=0).stop 2024-04-25T08:53:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9bd00d11e0 0x7f9bd00c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9bd009a110 0x7f9bd014dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:11.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 --2- 172.21.15.46:0/2958634217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9bd00702a0 0x7f9bd014bed0 secure :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400c480 tx=0x7f9bc400c940 comp rx=0 tx=0).stop 2024-04-25T08:53:11.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 >> 172.21.15.46:0/2958634217 conn(0x7f9bd00d16c0 msgr2=0x7f9bd0099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:11.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 shutdown_connections 2024-04-25T08:53:11.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.001+0000 7f9bd8000640 1 -- 172.21.15.46:0/2958634217 wait complete. 2024-04-25T08:53:12.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:53:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d9c00640 1 Processor -- start 2024-04-25T08:53:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d9c00640 1 -- start start 2024-04-25T08:53:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d9c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 0x7f38d406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d9c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f38d406dc70 0x7f38d40cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d9c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38d40cf1b0 0x7f38d415d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d9c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d40cd930 con 0x7f38d40cf1b0 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d9c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d40cdbd0 con 0x7f38d406dc70 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d9c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cdeb0 con 0x7f38d4145310 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38d40cf1b0 0x7f38d415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38d40cf1b0 0x7f38d415d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42516/0 (socket says 172.21.15.46:42516) 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d3400640 1 -- 172.21.15.46:0/2332965082 learned_addr learned my addr 172.21.15.46:0/2332965082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d2a00640 1 --2- 172.21.15.46:0/2332965082 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 0x7f38d406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38ca000640 1 --2- 172.21.15.46:0/2332965082 >> v2:172.21.15.82:3300/0 conn(0x7f38d406dc70 0x7f38d40cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d3400640 1 -- 172.21.15.46:0/2332965082 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 msgr2=0x7f38d406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d3400640 1 --2- 172.21.15.46:0/2332965082 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 0x7f38d406e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d3400640 1 -- 172.21.15.46:0/2332965082 >> v2:172.21.15.82:3300/0 conn(0x7f38d406dc70 msgr2=0x7f38d40cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d3400640 1 --2- 172.21.15.46:0/2332965082 >> v2:172.21.15.82:3300/0 conn(0x7f38d406dc70 0x7f38d40cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.177+0000 7f38d3400640 1 -- 172.21.15.46:0/2332965082 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40ce190 con 0x7f38d40cf1b0 2024-04-25T08:53:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d3400640 1 --2- 172.21.15.46:0/2332965082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40cf1b0 0x7f38d415d090 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f38d40cd440 tx=0x7f38cc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8899237bd9abd43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d2000640 1 -- 172.21.15.46:0/2332965082 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0b1070 con 0x7f38d40cf1b0 2024-04-25T08:53:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d2000640 1 -- 172.21.15.46:0/2332965082 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc0ac620 con 0x7f38d40cf1b0 2024-04-25T08:53:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d2000640 1 -- 172.21.15.46:0/2332965082 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0ac9f0 con 0x7f38d40cf1b0 2024-04-25T08:53:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 -- 172.21.15.46:0/2332965082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40cf1b0 msgr2=0x7f38d415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 --2- 172.21.15.46:0/2332965082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40cf1b0 0x7f38d415d090 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f38d40cd440 tx=0x7f38cc01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 -- 172.21.15.46:0/2332965082 shutdown_connections 2024-04-25T08:53:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 --2- 172.21.15.46:0/2332965082 >> v2:172.21.15.82:3300/0 conn(0x7f38d406dc70 0x7f38d40cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 --2- 172.21.15.46:0/2332965082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40cf1b0 0x7f38d415d090 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 --2- 172.21.15.46:0/2332965082 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 0x7f38d406e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 -- 172.21.15.46:0/2332965082 >> 172.21.15.46:0/2332965082 conn(0x7f38d406ac10 msgr2=0x7f38d406b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 -- 172.21.15.46:0/2332965082 shutdown_connections 2024-04-25T08:53:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 -- 172.21.15.46:0/2332965082 wait complete. 2024-04-25T08:53:12.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 Processor -- start 2024-04-25T08:53:12.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.181+0000 7f38d9c00640 1 -- start start 2024-04-25T08:53:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d9c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40cf1b0 0x7f38d413cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d9c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 0x7f38d413edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40b2620 0x7f38d4158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40713f0 con 0x7f38d40b2620 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d9c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4071270 con 0x7f38d40cf1b0 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d9c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4071570 con 0x7f38d4145310 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40cf1b0 0x7f38d413cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40cf1b0 0x7f38d413cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:33384/0 (socket says 172.21.15.46:33384) 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d2a00640 1 -- 172.21.15.46:0/402773758 learned_addr learned my addr 172.21.15.46:0/402773758 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40b2620 0x7f38d4158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38ca000640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 0x7f38d413edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d3400640 1 -- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 msgr2=0x7f38d413edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d3400640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 0x7f38d413edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d3400640 1 -- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40cf1b0 msgr2=0x7f38d413cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d3400640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40cf1b0 0x7f38d413cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d3400640 1 -- 172.21.15.46:0/402773758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40b2b20 con 0x7f38d40b2620 2024-04-25T08:53:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d3400640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40b2620 0x7f38d4158ca0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0ac000 tx=0x7f38cc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38cb400640 1 -- 172.21.15.46:0/402773758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0b1040 con 0x7f38d40b2620 2024-04-25T08:53:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38cb400640 1 -- 172.21.15.46:0/402773758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc0bd070 con 0x7f38d40b2620 2024-04-25T08:53:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38cb400640 1 -- 172.21.15.46:0/402773758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0b7860 con 0x7f38d40b2620 2024-04-25T08:53:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d40b2e00 con 0x7f38d40b2620 2024-04-25T08:53:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d40b33c0 con 0x7f38d40b2620 2024-04-25T08:53:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.185+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d4069010 con 0x7f38d40b2620 2024-04-25T08:53:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.189+0000 7f38cb400640 1 -- 172.21.15.46:0/402773758 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38cc0ac4e0 con 0x7f38d40b2620 2024-04-25T08:53:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.189+0000 7f38cb400640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38b0066f60 0x7f38b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.189+0000 7f38cb400640 1 -- 172.21.15.46:0/402773758 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f38cc11f300 con 0x7f38d40b2620 2024-04-25T08:53:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.189+0000 7f38d2a00640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38b0066f60 0x7f38b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.189+0000 7f38d2a00640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38b0066f60 0x7f38b0069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f38c4003ab0 tx=0x7f38c40098d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:12.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.193+0000 7f38cb400640 1 -- 172.21.15.46:0/402773758 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38cc016110 con 0x7f38d40b2620 2024-04-25T08:53:12.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.357+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f38d40ae5f0 con 0x7f38d40b2620 2024-04-25T08:53:12.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.357+0000 7f38cb400640 1 -- 172.21.15.46:0/402773758 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f38cc0efce0 con 0x7f38d40b2620 2024-04-25T08:53:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38b0066f60 msgr2=0x7f38b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38b0066f60 0x7f38b0069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f38c4003ab0 tx=0x7f38c40098d0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40b2620 msgr2=0x7f38d4158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40b2620 0x7f38d4158ca0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0ac000 tx=0x7f38cc01ad30 comp rx=0 tx=0).stop 2024-04-25T08:53:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 shutdown_connections 2024-04-25T08:53:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f38b0066f60 0x7f38b0069420 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d40b2620 0x7f38d4158ca0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f38d4145310 0x7f38d413edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 --2- 172.21.15.46:0/402773758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38d40cf1b0 0x7f38d413cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 >> 172.21.15.46:0/402773758 conn(0x7f38d406ac10 msgr2=0x7f38d40af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 shutdown_connections 2024-04-25T08:53:12.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.361+0000 7f38d9c00640 1 -- 172.21.15.46:0/402773758 wait complete. 2024-04-25T08:53:12.373 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T08:53:12.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.521+0000 7f8f91c00640 1 Processor -- start 2024-04-25T08:53:12.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.521+0000 7f8f91c00640 1 -- start start 2024-04-25T08:53:12.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.521+0000 7f8f91c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f8c1654c0 0x7f8f8c163400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.521+0000 7f8f91c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c162330 0x7f8f8c06e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.521+0000 7f8f91c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f8c06e9d0 con 0x7f8f8c06d270 2024-04-25T08:53:12.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.521+0000 7f8f91c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8c06ec70 con 0x7f8f8c1654c0 2024-04-25T08:53:12.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.521+0000 7f8f91c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c06ef50 con 0x7f8f8c162330 2024-04-25T08:53:12.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f8c1654c0 0x7f8f8c163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8f8c06d270 0x7f8f8c0cda30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51788/0 (socket says 172.21.15.46:51788) 2024-04-25T08:53:12.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8b400640 1 -- 172.21.15.46:0/3261108879 learned_addr learned my addr 172.21.15.46:0/3261108879 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:12.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f83e00640 1 --2- 172.21.15.46:0/3261108879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c162330 0x7f8f8c06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8aa00640 1 -- 172.21.15.46:0/3261108879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c162330 msgr2=0x7f8f8c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:12.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8aa00640 1 --2- 172.21.15.46:0/3261108879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c162330 0x7f8f8c06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8aa00640 1 -- 172.21.15.46:0/3261108879 >> v1:172.21.15.46:6789/0 conn(0x7f8f8c06d270 legacy=0x7f8f8c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8aa00640 1 -- 172.21.15.46:0/3261108879 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c06f230 con 0x7f8f8c1654c0 2024-04-25T08:53:12.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8a000640 1 -- 172.21.15.46:0/3261108879 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868184342 0 0) 0x7f8f8c06e9d0 con 0x7f8f8c06d270 2024-04-25T08:53:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8aa00640 1 --2- 172.21.15.46:0/3261108879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c1654c0 0x7f8f8c163400 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0097c0 tx=0x7f8f7c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54b0b1bce6457d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8a000640 1 -- 172.21.15.46:0/3261108879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0b0070 con 0x7f8f8c1654c0 2024-04-25T08:53:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8a000640 1 -- 172.21.15.46:0/3261108879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c0ab600 con 0x7f8f8c1654c0 2024-04-25T08:53:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.525+0000 7f8f8a000640 1 -- 172.21.15.46:0/3261108879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0ab9d0 con 0x7f8f8c1654c0 2024-04-25T08:53:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 -- 172.21.15.46:0/3261108879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c1654c0 msgr2=0x7f8f8c163400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 --2- 172.21.15.46:0/3261108879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c1654c0 0x7f8f8c163400 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0097c0 tx=0x7f8f7c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 -- 172.21.15.46:0/3261108879 shutdown_connections 2024-04-25T08:53:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 --2- 172.21.15.46:0/3261108879 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c162330 0x7f8f8c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 --2- 172.21.15.46:0/3261108879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c1654c0 0x7f8f8c163400 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 -- 172.21.15.46:0/3261108879 >> 172.21.15.46:0/3261108879 conn(0x7f8f8c0b12a0 msgr2=0x7f8f8c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 -- 172.21.15.46:0/3261108879 shutdown_connections 2024-04-25T08:53:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 -- 172.21.15.46:0/3261108879 wait complete. 2024-04-25T08:53:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 Processor -- start 2024-04-25T08:53:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 -- start start 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f8c06d270 0x7f8f8c15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c162330 0x7f8f8c167e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c168380 0x7f8f8c0b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c16ee20 con 0x7f8f8c162330 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c16eca0 con 0x7f8f8c06d270 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f91c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c16efa0 con 0x7f8f8c168380 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c162330 0x7f8f8c167e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c162330 0x7f8f8c167e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42544/0 (socket says 172.21.15.46:42544) 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f83e00640 1 -- 172.21.15.46:0/397361892 learned_addr learned my addr 172.21.15.46:0/397361892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f8aa00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f8c06d270 0x7f8f8c15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f8b400640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c168380 0x7f8f8c0b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f83e00640 1 -- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c168380 msgr2=0x7f8f8c0b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f83e00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c168380 0x7f8f8c0b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f83e00640 1 -- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f8c06d270 msgr2=0x7f8f8c15cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f83e00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f8c06d270 0x7f8f8c15cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f83e00640 1 -- 172.21.15.46:0/397361892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c168900 con 0x7f8f8c162330 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.529+0000 7f8f8b400640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c168380 0x7f8f8c0b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f83e00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c162330 0x7f8f8c167e40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400d620 tx=0x7f8f7400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f82a00640 1 -- 172.21.15.46:0/397361892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f740125f0 con 0x7f8f8c162330 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f82a00640 1 -- 172.21.15.46:0/397361892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f74019070 con 0x7f8f8c162330 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f82a00640 1 -- 172.21.15.46:0/397361892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f740129b0 con 0x7f8f8c162330 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c168be0 con 0x7f8f8c162330 2024-04-25T08:53:12.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c1691a0 con 0x7f8f8c162330 2024-04-25T08:53:12.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c069080 con 0x7f8f8c162330 2024-04-25T08:53:12.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f82a00640 1 -- 172.21.15.46:0/397361892 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f74012b50 con 0x7f8f8c162330 2024-04-25T08:53:12.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f82a00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f68066f60 0x7f8f68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f82a00640 1 -- 172.21.15.46:0/397361892 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8f740852f0 con 0x7f8f8c162330 2024-04-25T08:53:12.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.533+0000 7f8f8aa00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f68066f60 0x7f8f68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.537+0000 7f8f8aa00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f68066f60 0x7f8f68069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0ad000 tx=0x7f8f7c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.537+0000 7f8f82a00640 1 -- 172.21.15.46:0/397361892 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f74056320 con 0x7f8f8c162330 2024-04-25T08:53:12.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:12.761+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f8f8c163630 con 0x7f8f8c162330 2024-04-25T08:53:12.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.005+0000 7f8f82a00640 1 -- 172.21.15.46:0/397361892 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v524) v1 ==== 144+0+0 (secure 0 0 0) 0x7f8f74055c20 con 0x7f8f8c162330 2024-04-25T08:53:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f68066f60 msgr2=0x7f8f68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f68066f60 0x7f8f68069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0ad000 tx=0x7f8f7c0a6000 comp rx=0 tx=0).stop 2024-04-25T08:53:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c162330 msgr2=0x7f8f8c167e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c162330 0x7f8f8c167e40 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f8f7400d620 tx=0x7f8f7400dae0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 shutdown_connections 2024-04-25T08:53:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f68066f60 0x7f8f68069420 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f8c168380 0x7f8f8c0b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f8c162330 0x7f8f8c167e40 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 --2- 172.21.15.46:0/397361892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f8c06d270 0x7f8f8c15cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 >> 172.21.15.46:0/397361892 conn(0x7f8f8c0b12a0 msgr2=0x7f8f8c0b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 shutdown_connections 2024-04-25T08:53:13.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:13.009+0000 7f8f91c00640 1 -- 172.21.15.46:0/397361892 wait complete. 2024-04-25T08:53:13.021 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T08:53:13.021 INFO:tasks.cephfs_test_runner: 2024-04-25T08:53:13.021 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T08:53:13.021 INFO:tasks.cephfs_test_runner:Ran 7 tests in 606.138s 2024-04-25T08:53:13.021 INFO:tasks.cephfs_test_runner: 2024-04-25T08:53:13.021 INFO:tasks.cephfs_test_runner:OK 2024-04-25T08:53:13.022 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T08:53:13.032 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T08:53:13.085 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T08:53:13.086 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:53:13.086 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:13.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:53:13.095 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-04-25T08:53:13.095 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:13.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16353404k,nr_inodes=4088351,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=5553 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275608k,nr_inodes=818902,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275612k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:53:13.143 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:53:13.143 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:13.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:53:13.152 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T08:53:13.152 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:13.203 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350240k,nr_inodes=4087560,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=9406 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274976k,nr_inodes=818744,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274980k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:53:13.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.204 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:53:13.212 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:53:13.213 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:53:13.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:13.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.277 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:53:13.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-25T08:53:13.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.506 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-04-25T08:53:13.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.516 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-04-25T08:53:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-04-25T08:53:13.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:53:13.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.588 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T08:53:13.598 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T08:53:13.598 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T08:53:13.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.599 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:13.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.666 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:53:13.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.733 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-25T08:53:13.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.886 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T08:53:13.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.895 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-25T08:53:13.960 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-04-25T08:53:13.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:53:13.963 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T08:53:13.972 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T08:53:13.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:53:14.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e4600640 1 Processor -- start 2024-04-25T08:53:14.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e4600640 1 -- start start 2024-04-25T08:53:14.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e4600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 0x7f06dc06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e4600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f06dc0aed10 0x7f06dc06f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e4600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f06dc0cc9e0 0x7f06dc0b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e4600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06dc15ab30 con 0x7f06dc0aed10 2024-04-25T08:53:14.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e4600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06dc0ce4f0 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e4600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc06d8c0 con 0x7f06dc15a750 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f06dc0aed10 0x7f06dc06f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f06dc0aed10 0x7f06dc06f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42558/0 (socket says 172.21.15.46:42558) 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.125+0000 7f06e1200640 1 -- 172.21.15.46:0/1176866857 learned_addr learned my addr 172.21.15.46:0/1176866857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e2600640 1 --2- 172.21.15.46:0/1176866857 >> v2:172.21.15.82:3300/0 conn(0x7f06dc0cc9e0 0x7f06dc0b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e1c00640 1 --2- 172.21.15.46:0/1176866857 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 0x7f06dc06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e1200640 1 -- 172.21.15.46:0/1176866857 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 msgr2=0x7f06dc06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e1200640 1 --2- 172.21.15.46:0/1176866857 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 0x7f06dc06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e1200640 1 -- 172.21.15.46:0/1176866857 >> v2:172.21.15.82:3300/0 conn(0x7f06dc0cc9e0 msgr2=0x7f06dc0b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e1200640 1 --2- 172.21.15.46:0/1176866857 >> v2:172.21.15.82:3300/0 conn(0x7f06dc0cc9e0 0x7f06dc0b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e1200640 1 -- 172.21.15.46:0/1176866857 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc06d5e0 con 0x7f06dc0aed10 2024-04-25T08:53:14.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e1200640 1 --2- 172.21.15.46:0/1176866857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0aed10 0x7f06dc06f150 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f06dc072930 tx=0x7f06c801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4b99ba5d874b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06dbe00640 1 -- 172.21.15.46:0/1176866857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80aa610 con 0x7f06dc0aed10 2024-04-25T08:53:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06dbe00640 1 -- 172.21.15.46:0/1176866857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c80af070 con 0x7f06dc0aed10 2024-04-25T08:53:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06dbe00640 1 -- 172.21.15.46:0/1176866857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80aa9d0 con 0x7f06dc0aed10 2024-04-25T08:53:14.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e4600640 1 -- 172.21.15.46:0/1176866857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0aed10 msgr2=0x7f06dc06f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.129+0000 7f06e4600640 1 --2- 172.21.15.46:0/1176866857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0aed10 0x7f06dc06f150 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f06dc072930 tx=0x7f06c801c4f0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 -- 172.21.15.46:0/1176866857 shutdown_connections 2024-04-25T08:53:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 --2- 172.21.15.46:0/1176866857 >> v2:172.21.15.82:3300/0 conn(0x7f06dc0cc9e0 0x7f06dc0b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 --2- 172.21.15.46:0/1176866857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0aed10 0x7f06dc06f150 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 --2- 172.21.15.46:0/1176866857 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 0x7f06dc06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 -- 172.21.15.46:0/1176866857 >> 172.21.15.46:0/1176866857 conn(0x7f06dc0b12a0 msgr2=0x7f06dc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 -- 172.21.15.46:0/1176866857 shutdown_connections 2024-04-25T08:53:14.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 -- 172.21.15.46:0/1176866857 wait complete. 2024-04-25T08:53:14.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 Processor -- start 2024-04-25T08:53:14.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 -- start start 2024-04-25T08:53:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06dc0aed10 0x7f06dc157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0cc9e0 0x7f06dc142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 0x7f06dc15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc06d270 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0cf200 con 0x7f06dc0aed10 2024-04-25T08:53:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e4600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc0b32f0 con 0x7f06dc15a750 2024-04-25T08:53:14.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0cc9e0 0x7f06dc142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06dc0aed10 0x7f06dc157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06dc0aed10 0x7f06dc157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.46:33418/0 (socket says 172.21.15.46:33418) 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0cc9e0 0x7f06dc142d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42564/0 (socket says 172.21.15.46:42564) 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1c00640 1 -- 172.21.15.46:0/2475334322 learned_addr learned my addr 172.21.15.46:0/2475334322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e2600640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 0x7f06dc15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1200640 1 -- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 msgr2=0x7f06dc15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1200640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 0x7f06dc15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1200640 1 -- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06dc0aed10 msgr2=0x7f06dc157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1200640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06dc0aed10 0x7f06dc157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1200640 1 -- 172.21.15.46:0/2475334322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06dc156890 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.133+0000 7f06e1c00640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06dc0aed10 0x7f06dc157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:14.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06e1200640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0cc9e0 0x7f06dc142d70 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f06c80ab000 tx=0x7f06c801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06d9600640 1 -- 172.21.15.46:0/2475334322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80bd5f0 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06d9600640 1 -- 172.21.15.46:0/2475334322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c80bd7b0 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc156b70 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06d9600640 1 -- 172.21.15.46:0/2475334322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80bdba0 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc157130 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc069080 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06d9600640 1 -- 172.21.15.46:0/2475334322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f06c80b33f0 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06d9600640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06bc066f60 0x7f06bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06d9600640 1 -- 172.21.15.46:0/2475334322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f06c811fa70 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.137+0000 7f06e1c00640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06bc066f60 0x7f06bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.141+0000 7f06e1c00640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06bc066f60 0x7f06bc069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009f70 tx=0x7f06d0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.141+0000 7f06d9600640 1 -- 172.21.15.46:0/2475334322 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f06c80c3060 con 0x7f06dc0cc9e0 2024-04-25T08:53:14.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.301+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f06dc0afaf0 con 0x7f06bc066f60 2024-04-25T08:53:14.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.301+0000 7f06d9600640 1 -- 172.21.15.46:0/2475334322 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13481 (secure 0 0 0) 0x7f06dc0afaf0 con 0x7f06bc066f60 2024-04-25T08:53:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:14.295 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:53:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06bc066f60 msgr2=0x7f06bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06bc066f60 0x7f06bc069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009f70 tx=0x7f06d0009190 comp rx=0 tx=0).stop 2024-04-25T08:53:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0cc9e0 msgr2=0x7f06dc142d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0cc9e0 0x7f06dc142d70 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f06c80ab000 tx=0x7f06c801ad30 comp rx=0 tx=0).stop 2024-04-25T08:53:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 shutdown_connections 2024-04-25T08:53:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f06bc066f60 0x7f06bc069420 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f06dc15a750 0x7f06dc15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06dc0cc9e0 0x7f06dc142d70 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 --2- 172.21.15.46:0/2475334322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06dc0aed10 0x7f06dc157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 >> 172.21.15.46:0/2475334322 conn(0x7f06dc0b12a0 msgr2=0x7f06dc0af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 shutdown_connections 2024-04-25T08:53:14.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.305+0000 7f06e4600640 1 -- 172.21.15.46:0/2475334322 wait complete. 2024-04-25T08:53:14.315 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-04-25T08:53:12.421789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":271020,"kb_used_data":21576,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754703700,"statfs":{"total":773094113280,"available":772816588800,"internally_reserved":0,"allocated":22093824,"data_stored":15426039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390401,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-172,"num_write_kb":-8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001571"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":369,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T08:53:12.008144+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:53:12.008144+0000","last_peered":"2024-04-25T08:53:12.008144+0000","last_clean":"2024-04-25T08:53:12.008144+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:53:12.008144+0000","last_undegraded":"2024-04-25T08:53:12.008144+0000","last_fullsized":"2024-04-25T08:53:12.008144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37380,"kb_used_data":6200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334460,"statfs":{"total":96636764160,"available":96598487040,"internally_reserved":0,"allocated":6348800,"data_stored":5515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":731318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":731323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":731323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574984,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":731323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":5758976,"data_stored":4925629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":1321686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:53:14.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:53:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.465+0000 7fe7abc00640 1 Processor -- start 2024-04-25T08:53:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.465+0000 7fe7abc00640 1 -- start start 2024-04-25T08:53:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.465+0000 7fe7abc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe7a415bdb0 0x7fe7a40cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.465+0000 7fe7abc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a40cc700 con 0x7fe7a4159230 2024-04-25T08:53:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.465+0000 7fe7abc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a40cc9e0 con 0x7fe7a40cda80 2024-04-25T08:53:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.465+0000 7fe7abc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a40cded0 con 0x7fe7a415bdb0 2024-04-25T08:53:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.465+0000 7fe7a9200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe7a4159230 0x7fe7a40cbeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51804/0 (socket says 172.21.15.46:51804) 2024-04-25T08:53:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.465+0000 7fe7a9200640 1 -- 172.21.15.46:0/419088927 learned_addr learned my addr 172.21.15.46:0/419088927 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe79be00640 1 --2- 172.21.15.46:0/419088927 >> v2:172.21.15.82:3300/0 conn(0x7fe7a415bdb0 0x7fe7a40cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663111989 0 0) 0x7fe7a40cc700 con 0x7fe7a4159230 2024-04-25T08:53:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe788003530 con 0x7fe7a4159230 2024-04-25T08:53:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666421889 0 0) 0x7fe7a40cc9e0 con 0x7fe7a40cda80 2024-04-25T08:53:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a40cc700 con 0x7fe7a40cda80 2024-04-25T08:53:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2222440783 0 0) 0x7fe788003530 con 0x7fe7a4159230 2024-04-25T08:53:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 >> v2:172.21.15.82:3300/0 conn(0x7fe7a415bdb0 msgr2=0x7fe7a40cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 --2- 172.21.15.46:0/419088927 >> v2:172.21.15.82:3300/0 conn(0x7fe7a415bdb0 0x7fe7a40cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 >> v1:172.21.15.82:6789/0 conn(0x7fe7a40cda80 legacy=0x7fe7a415d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a40cccc0 con 0x7fe7a4159230 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fe794001600 con 0x7fe7a4159230 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 >> v1:172.21.15.46:6789/0 conn(0x7fe7a4159230 legacy=0x7fe7a40cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe788003ba0 0x7fe788003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe788004810 0x7fe788004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe788005220 0x7fe78800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40cded0 con 0x7fe788003ba0 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78800c150 con 0x7fe788005220 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78800c350 con 0x7fe788004810 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a9200640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe788003ba0 0x7fe788003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe79be00640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe788004810 0x7fe788004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a9c00640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe788005220 0x7fe78800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a9200640 1 -- 172.21.15.46:0/419088927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe788004810 msgr2=0x7fe788004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a9200640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe788004810 0x7fe788004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a9200640 1 -- 172.21.15.46:0/419088927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe788005220 msgr2=0x7fe78800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a9200640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe788005220 0x7fe78800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a9200640 1 -- 172.21.15.46:0/419088927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788003530 con 0x7fe788003ba0 2024-04-25T08:53:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.469+0000 7fe7a9c00640 1 -- 172.21.15.46:0/419088927 reap_dead start 2024-04-25T08:53:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7a9200640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe788003ba0 0x7fe788003fa0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe794007f30 tx=0x7fe79401cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70698e348249b43a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794007b50 con 0x7fe788003ba0 2024-04-25T08:53:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7940a9020 con 0x7fe788003ba0 2024-04-25T08:53:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7a3e00640 1 -- 172.21.15.46:0/419088927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794005be0 con 0x7fe788003ba0 2024-04-25T08:53:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 -- 172.21.15.46:0/419088927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe788003ba0 msgr2=0x7fe788003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe788003ba0 0x7fe788003fa0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fe794007f30 tx=0x7fe79401cac0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 -- 172.21.15.46:0/419088927 shutdown_connections 2024-04-25T08:53:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 --2- 172.21.15.46:0/419088927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe788003ba0 0x7fe788003fa0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 -- 172.21.15.46:0/419088927 >> 172.21.15.46:0/419088927 conn(0x7fe7a40b12a0 msgr2=0x7fe7a40b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 -- 172.21.15.46:0/419088927 shutdown_connections 2024-04-25T08:53:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 -- 172.21.15.46:0/419088927 wait complete. 2024-04-25T08:53:14.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 Processor -- start 2024-04-25T08:53:14.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 -- start start 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe788003ba0 0x7fe7a41facf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a40b3b60 0x7fe7a41400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.473+0000 7fe7abc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a4158030 0x7fe7a41fa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7abc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40cc440 con 0x7fe7a40b3b60 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7abc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40cc2c0 con 0x7fe7a4158030 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7abc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a415fbf0 con 0x7fe788003ba0 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe79be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a40b3b60 0x7fe7a41400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe79be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a40b3b60 0x7fe7a41400b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42584/0 (socket says 172.21.15.46:42584) 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe79be00640 1 -- 172.21.15.46:0/816314214 learned_addr learned my addr 172.21.15.46:0/816314214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7a9200640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe788003ba0 0x7fe7a41facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7a9c00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a4158030 0x7fe7a41fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe79be00640 1 -- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe788003ba0 msgr2=0x7fe7a41facf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe79be00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe788003ba0 0x7fe7a41facf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe79be00640 1 -- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a4158030 msgr2=0x7fe7a41fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe79be00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a4158030 0x7fe7a41fa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe79be00640 1 -- 172.21.15.46:0/816314214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a40acbd0 con 0x7fe7a40b3b60 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7a9c00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a4158030 0x7fe7a41fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe79be00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a40b3b60 0x7fe7a41400b0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fe78c011900 tx=0x7fe78c011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7a1600640 1 -- 172.21.15.46:0/816314214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c019070 con 0x7fe7a40b3b60 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7a1600640 1 -- 172.21.15.46:0/816314214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c004240 con 0x7fe7a40b3b60 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7a1600640 1 -- 172.21.15.46:0/816314214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0095b0 con 0x7fe7a40b3b60 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a40aceb0 con 0x7fe7a40b3b60 2024-04-25T08:53:14.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a41405f0 con 0x7fe7a40b3b60 2024-04-25T08:53:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a4069080 con 0x7fe7a40b3b60 2024-04-25T08:53:14.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.477+0000 7fe7a1600640 1 -- 172.21.15.46:0/816314214 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe78c01c070 con 0x7fe7a40b3b60 2024-04-25T08:53:14.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.481+0000 7fe7a1600640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe780066f90 0x7fe780069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.481+0000 7fe7a1600640 1 -- 172.21.15.46:0/816314214 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe78c085450 con 0x7fe7a40b3b60 2024-04-25T08:53:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.481+0000 7fe7a9200640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe780066f90 0x7fe780069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.481+0000 7fe7a9200640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe780066f90 0x7fe780069450 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe7940aa000 tx=0x7fe7a40ccbb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:14.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.485+0000 7fe7a1600640 1 -- 172.21.15.46:0/816314214 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe78c056480 con 0x7fe7a40b3b60 2024-04-25T08:53:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.641+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe7a40723d0 con 0x7fe780066f90 2024-04-25T08:53:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.641+0000 7fe7a1600640 1 -- 172.21.15.46:0/816314214 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13481 (secure 0 0 0) 0x7fe7a40723d0 con 0x7fe780066f90 2024-04-25T08:53:14.635 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:53:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe780066f90 msgr2=0x7fe780069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe780066f90 0x7fe780069450 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe7940aa000 tx=0x7fe7a40ccbb0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a40b3b60 msgr2=0x7fe7a41400b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a40b3b60 0x7fe7a41400b0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fe78c011900 tx=0x7fe78c011dc0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 shutdown_connections 2024-04-25T08:53:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe780066f90 0x7fe780069450 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a4158030 0x7fe7a41fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7a40b3b60 0x7fe7a41400b0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 --2- 172.21.15.46:0/816314214 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe788003ba0 0x7fe7a41facf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 >> 172.21.15.46:0/816314214 conn(0x7fe7a40b12a0 msgr2=0x7fe7a406d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 shutdown_connections 2024-04-25T08:53:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.645+0000 7fe7abc00640 1 -- 172.21.15.46:0/816314214 wait complete. 2024-04-25T08:53:14.657 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-04-25T08:53:14.422126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":271020,"kb_used_data":21576,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754703700,"statfs":{"total":773094113280,"available":772816588800,"internally_reserved":0,"allocated":22093824,"data_stored":15426039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392193,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-176,"num_write_kb":-8301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001411"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":369,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T08:53:12.008144+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:53:12.008144+0000","last_peered":"2024-04-25T08:53:12.008144+0000","last_clean":"2024-04-25T08:53:12.008144+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:53:12.008144+0000","last_undegraded":"2024-04-25T08:53:12.008144+0000","last_fullsized":"2024-04-25T08:53:12.008144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37380,"kb_used_data":6200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334460,"statfs":{"total":96636764160,"available":96598487040,"internally_reserved":0,"allocated":6348800,"data_stored":5515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":731318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":731323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":731323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574984,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":731323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":5758976,"data_stored":4925629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":1321686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:53:14.657 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T08:53:14.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:53:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.805+0000 7f3c3b000640 1 Processor -- start 2024-04-25T08:53:14.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.805+0000 7f3c3b000640 1 -- start start 2024-04-25T08:53:14.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c3b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c3b000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c340aed10 0x7f3c34164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c3b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c340b35b0 con 0x7f3c3415a750 2024-04-25T08:53:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c3b000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3415ab30 con 0x7f3c340aed10 2024-04-25T08:53:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c3b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c340ce4f0 con 0x7f3c3406f320 2024-04-25T08:53:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c33e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3c3415a750 0x7f3c3406ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51812/0 (socket says 172.21.15.46:51812) 2024-04-25T08:53:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c33e00640 1 -- 172.21.15.46:0/2893617570 learned_addr learned my addr 172.21.15.46:0/2893617570 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c2b400640 1 --2- 172.21.15.46:0/2893617570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c39000640 1 --2- 172.21.15.46:0/2893617570 >> v2:172.21.15.82:3300/0 conn(0x7f3c340aed10 0x7f3c34164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c33400640 1 -- 172.21.15.46:0/2893617570 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703071374 0 0) 0x7f3c340b35b0 con 0x7f3c3415a750 2024-04-25T08:53:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c33400640 1 -- 172.21.15.46:0/2893617570 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c18003530 con 0x7f3c3415a750 2024-04-25T08:53:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c2b400640 1 -- 172.21.15.46:0/2893617570 >> v2:172.21.15.82:3300/0 conn(0x7f3c340aed10 msgr2=0x7f3c34164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c2b400640 1 --2- 172.21.15.46:0/2893617570 >> v2:172.21.15.82:3300/0 conn(0x7f3c340aed10 0x7f3c34164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c2b400640 1 -- 172.21.15.46:0/2893617570 >> v1:172.21.15.46:6789/0 conn(0x7f3c3415a750 legacy=0x7f3c3406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.809+0000 7f3c2b400640 1 -- 172.21.15.46:0/2893617570 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3406d5e0 con 0x7f3c3406f320 2024-04-25T08:53:14.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c2b400640 1 --2- 172.21.15.46:0/2893617570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34162200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009d90 tx=0x7f3c1c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6789ff4f8e73e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c33400640 1 -- 172.21.15.46:0/2893617570 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0b2070 con 0x7f3c3406f320 2024-04-25T08:53:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c33400640 1 -- 172.21.15.46:0/2893617570 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1c0a5920 con 0x7f3c3406f320 2024-04-25T08:53:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c33400640 1 -- 172.21.15.46:0/2893617570 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0a5cf0 con 0x7f3c3406f320 2024-04-25T08:53:14.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 -- 172.21.15.46:0/2893617570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 msgr2=0x7f3c34162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 --2- 172.21.15.46:0/2893617570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34162200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009d90 tx=0x7f3c1c01c970 comp rx=0 tx=0).stop 2024-04-25T08:53:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 -- 172.21.15.46:0/2893617570 shutdown_connections 2024-04-25T08:53:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 --2- 172.21.15.46:0/2893617570 >> v2:172.21.15.82:3300/0 conn(0x7f3c340aed10 0x7f3c34164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 --2- 172.21.15.46:0/2893617570 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34162200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 -- 172.21.15.46:0/2893617570 >> 172.21.15.46:0/2893617570 conn(0x7f3c340b12a0 msgr2=0x7f3c340b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 -- 172.21.15.46:0/2893617570 shutdown_connections 2024-04-25T08:53:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 -- 172.21.15.46:0/2893617570 wait complete. 2024-04-25T08:53:14.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 Processor -- start 2024-04-25T08:53:14.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.813+0000 7f3c3b000640 1 -- start start 2024-04-25T08:53:14.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c3b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c3b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c340aed10 0x7f3c3414d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c3b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3415a750 0x7f3c3415d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c3b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3406d270 con 0x7f3c3415a750 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c3b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c340cf200 con 0x7f3c340aed10 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c3b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c341674b0 con 0x7f3c3406f320 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c39000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3415a750 0x7f3c3415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c39000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3415a750 0x7f3c3415d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42598/0 (socket says 172.21.15.46:42598) 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c39000640 1 -- 172.21.15.46:0/2787245829 learned_addr learned my addr 172.21.15.46:0/2787245829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c2b400640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c340aed10 0x7f3c3414d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c33e00640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c39000640 1 -- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 msgr2=0x7f3c34154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c39000640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c39000640 1 -- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c340aed10 msgr2=0x7f3c3414d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c39000640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c340aed10 0x7f3c3414d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c39000640 1 -- 172.21.15.46:0/2787245829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3415d840 con 0x7f3c3415a750 2024-04-25T08:53:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c33e00640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c39000640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3415a750 0x7f3c3415d300 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e960 tx=0x7f3c2c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c30c00640 1 -- 172.21.15.46:0/2787245829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c017070 con 0x7f3c3415a750 2024-04-25T08:53:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c30c00640 1 -- 172.21.15.46:0/2787245829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c00c9a0 con 0x7f3c3415a750 2024-04-25T08:53:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3415dae0 con 0x7f3c3415a750 2024-04-25T08:53:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c30c00640 1 -- 172.21.15.46:0/2787245829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00cd70 con 0x7f3c3415a750 2024-04-25T08:53:14.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3415bc90 con 0x7f3c3415a750 2024-04-25T08:53:14.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.817+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c34069080 con 0x7f3c3415a750 2024-04-25T08:53:14.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.821+0000 7f3c30c00640 1 -- 172.21.15.46:0/2787245829 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c2c01a070 con 0x7f3c3415a750 2024-04-25T08:53:14.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.821+0000 7f3c30c00640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c10066f90 0x7f3c10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:14.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.821+0000 7f3c30c00640 1 -- 172.21.15.46:0/2787245829 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f3c2c084f50 con 0x7f3c3415a750 2024-04-25T08:53:14.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.821+0000 7f3c33e00640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c10066f90 0x7f3c10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:14.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.821+0000 7f3c33e00640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c10066f90 0x7f3c10069450 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005890 tx=0x7f3c24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:14.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.825+0000 7f3c30c00640 1 -- 172.21.15.46:0/2787245829 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c2c055930 con 0x7f3c3415a750 2024-04-25T08:53:14.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.985+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c34154270 con 0x7f3c10066f90 2024-04-25T08:53:14.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.985+0000 7f3c30c00640 1 -- 172.21.15.46:0/2787245829 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13481 (secure 0 0 0) 0x7f3c34154270 con 0x7f3c10066f90 2024-04-25T08:53:14.979 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:14.979 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:53:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c10066f90 msgr2=0x7f3c10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c10066f90 0x7f3c10069450 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005890 tx=0x7f3c24009040 comp rx=0 tx=0).stop 2024-04-25T08:53:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3415a750 msgr2=0x7f3c3415d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3415a750 0x7f3c3415d300 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e960 tx=0x7f3c2c00ee20 comp rx=0 tx=0).stop 2024-04-25T08:53:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 shutdown_connections 2024-04-25T08:53:14.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f3c10066f90 0x7f3c10069450 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3415a750 0x7f3c3415d300 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c340aed10 0x7f3c3414d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 --2- 172.21.15.46:0/2787245829 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f3c3406f320 0x7f3c34154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 >> 172.21.15.46:0/2787245829 conn(0x7f3c340b12a0 msgr2=0x7f3c340b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 shutdown_connections 2024-04-25T08:53:14.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:14.989+0000 7f3c3b000640 1 -- 172.21.15.46:0/2787245829 wait complete. 2024-04-25T08:53:15.001 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-04-25T08:53:14.422126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":271020,"kb_used_data":21576,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754703700,"statfs":{"total":773094113280,"available":772816588800,"internally_reserved":0,"allocated":22093824,"data_stored":15426039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392193,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-176,"num_write_kb":-8301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001411"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":369,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T08:53:12.008144+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:53:12.008144+0000","last_peered":"2024-04-25T08:53:12.008144+0000","last_clean":"2024-04-25T08:53:12.008144+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:53:12.008144+0000","last_undegraded":"2024-04-25T08:53:12.008144+0000","last_fullsized":"2024-04-25T08:53:12.008144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37380,"kb_used_data":6200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334460,"statfs":{"total":96636764160,"available":96598487040,"internally_reserved":0,"allocated":6348800,"data_stored":5515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574985,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":731318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574983,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":731323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574983,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":731323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574984,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":731323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574985,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":5758976,"data_stored":4925629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33284,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2154496,"data_stored":1321686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:53:15.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.149+0000 7f5567c00640 1 Processor -- start 2024-04-25T08:53:15.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.149+0000 7f5567c00640 1 -- start start 2024-04-25T08:53:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f5567c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5560159230 0x7f556015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f5567c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f55600cbeb0 0x7f55600cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f5567c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55600cc700 con 0x7f55600cda80 2024-04-25T08:53:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f5567c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55600cc9e0 con 0x7f55600cbeb0 2024-04-25T08:53:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f5567c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55600cccc0 con 0x7f5560159230 2024-04-25T08:53:15.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f5565c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f55600cda80 0x7f556015f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51820/0 (socket says 172.21.15.46:51820) 2024-04-25T08:53:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f5565c00640 1 -- 172.21.15.46:0/2181889950 learned_addr learned my addr 172.21.15.46:0/2181889950 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f555fe00640 1 --2- 172.21.15.46:0/2181889950 >> v2:172.21.15.46:3300/0 conn(0x7f55600cbeb0 0x7f55600cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f5565200640 1 --2- 172.21.15.46:0/2181889950 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5560159230 0x7f556015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f555f400640 1 -- 172.21.15.46:0/2181889950 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451049285 0 0) 0x7f55600cc700 con 0x7f55600cda80 2024-04-25T08:53:15.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f555f400640 1 -- 172.21.15.46:0/2181889950 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5548003530 con 0x7f55600cda80 2024-04-25T08:53:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f555fe00640 1 -- 172.21.15.46:0/2181889950 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5560159230 msgr2=0x7f556015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f555fe00640 1 --2- 172.21.15.46:0/2181889950 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5560159230 0x7f556015bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f555fe00640 1 -- 172.21.15.46:0/2181889950 >> v1:172.21.15.46:6789/0 conn(0x7f55600cda80 legacy=0x7f556015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:15.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.153+0000 7f555fe00640 1 -- 172.21.15.46:0/2181889950 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55600ccfa0 con 0x7f55600cbeb0 2024-04-25T08:53:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f555fe00640 1 --2- 172.21.15.46:0/2181889950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55600cbeb0 0x7f55600cd540 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f554c0097c0 tx=0x7f554c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb9c98989d901c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f555f400640 1 -- 172.21.15.46:0/2181889950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0b0070 con 0x7f55600cbeb0 2024-04-25T08:53:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f555f400640 1 -- 172.21.15.46:0/2181889950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c0ab600 con 0x7f55600cbeb0 2024-04-25T08:53:15.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f555f400640 1 -- 172.21.15.46:0/2181889950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c0ab9d0 con 0x7f55600cbeb0 2024-04-25T08:53:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 -- 172.21.15.46:0/2181889950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55600cbeb0 msgr2=0x7f55600cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 --2- 172.21.15.46:0/2181889950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55600cbeb0 0x7f55600cd540 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f554c0097c0 tx=0x7f554c01c410 comp rx=0 tx=0).stop 2024-04-25T08:53:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 -- 172.21.15.46:0/2181889950 shutdown_connections 2024-04-25T08:53:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 --2- 172.21.15.46:0/2181889950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55600cbeb0 0x7f55600cd540 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 --2- 172.21.15.46:0/2181889950 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5560159230 0x7f556015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 -- 172.21.15.46:0/2181889950 >> 172.21.15.46:0/2181889950 conn(0x7f55600b12a0 msgr2=0x7f55600b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:15.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 -- 172.21.15.46:0/2181889950 shutdown_connections 2024-04-25T08:53:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 -- 172.21.15.46:0/2181889950 wait complete. 2024-04-25T08:53:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 Processor -- start 2024-04-25T08:53:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.157+0000 7f5567c00640 1 -- start start 2024-04-25T08:53:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5567c00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55600cbeb0 0x7f5560159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5567c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55600cda80 0x7f5560166080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5567c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5560159230 0x7f55600b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5567c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5560168900 con 0x7f5560159230 2024-04-25T08:53:15.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5567c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5560168780 con 0x7f55600cda80 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5567c00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5560168a80 con 0x7f55600cbeb0 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5560159230 0x7f55600b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5560159230 0x7f55600b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42626/0 (socket says 172.21.15.46:42626) 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565c00640 1 -- 172.21.15.46:0/2475284775 learned_addr learned my addr 172.21.15.46:0/2475284775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f555fe00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55600cda80 0x7f5560166080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565200640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55600cbeb0 0x7f5560159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565c00640 1 -- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55600cbeb0 msgr2=0x7f5560159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565c00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55600cbeb0 0x7f5560159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565c00640 1 -- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55600cda80 msgr2=0x7f5560166080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565c00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55600cda80 0x7f5560166080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565c00640 1 -- 172.21.15.46:0/2475284775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5560159e50 con 0x7f5560159230 2024-04-25T08:53:15.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5565c00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5560159230 0x7f55600b0780 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f555800cb70 tx=0x7f555800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:15.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f555cc00640 1 -- 172.21.15.46:0/2475284775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558017070 con 0x7f5560159230 2024-04-25T08:53:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f555cc00640 1 -- 172.21.15.46:0/2475284775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555800ad40 con 0x7f5560159230 2024-04-25T08:53:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556006d4a0 con 0x7f5560159230 2024-04-25T08:53:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f555cc00640 1 -- 172.21.15.46:0/2475284775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558012620 con 0x7f5560159230 2024-04-25T08:53:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55600b0cc0 con 0x7f5560159230 2024-04-25T08:53:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5560069080 con 0x7f5560159230 2024-04-25T08:53:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.161+0000 7f555cc00640 1 -- 172.21.15.46:0/2475284775 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f555801a070 con 0x7f5560159230 2024-04-25T08:53:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.165+0000 7f555cc00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5540066f90 0x7f5540069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.165+0000 7f555cc00640 1 -- 172.21.15.46:0/2475284775 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f5558084d60 con 0x7f5560159230 2024-04-25T08:53:15.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.165+0000 7f5565200640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5540066f90 0x7f5540069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:15.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.165+0000 7f5565200640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5540066f90 0x7f5540069450 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5554009b60 tx=0x7f5554009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:15.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.169+0000 7f555cc00640 1 -- 172.21.15.46:0/2475284775 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5558055750 con 0x7f5560159230 2024-04-25T08:53:15.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.333+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55600723d0 con 0x7f5560159230 2024-04-25T08:53:15.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.333+0000 7f555cc00640 1 -- 172.21.15.46:0/2475284775 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+13511 (secure 0 0 0) 0x7f55580597f0 con 0x7f5560159230 2024-04-25T08:53:15.327 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:15.327 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":126,"fsid":"9ca516a0-1b6e-4031-bb83-1b30eee026c8","created":"2024-04-25T08:41:55.097998+0000","modified":"2024-04-25T08:53:11.999837+0000","last_up_change":"2024-04-25T08:42:07.071293+0000","last_in_change":"2024-04-25T08:41:57.803830+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:42:08.317228+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d4c1a6f5-a079-4d33-8583-52aa28158fd8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6801","nonce":3448226631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6803","nonce":3448226631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6807","nonce":3448226631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3448226631},{"type":"v1","addr":"172.21.15.46:6805","nonce":3448226631}]},"public_addr":"172.21.15.46:6801/3448226631","cluster_addr":"172.21.15.46:6803/3448226631","heartbeat_back_addr":"172.21.15.46:6807/3448226631","heartbeat_front_addr":"172.21.15.46:6805/3448226631","state":["exists","up"]},{"osd":1,"uuid":"6a7c621a-0fc4-40b3-aa8e-86048c60c3ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6811","nonce":2552657765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6816","nonce":2552657765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6823","nonce":2552657765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2552657765},{"type":"v1","addr":"172.21.15.46:6820","nonce":2552657765}]},"public_addr":"172.21.15.46:6811/2552657765","cluster_addr":"172.21.15.46:6816/2552657765","heartbeat_back_addr":"172.21.15.46:6823/2552657765","heartbeat_front_addr":"172.21.15.46:6820/2552657765","state":["exists","up"]},{"osd":2,"uuid":"f4a9a449-1363-4a3c-8da8-271902210ab4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6810","nonce":3102391403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6814","nonce":3102391403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6819","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6821","nonce":3102391403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6815","nonce":3102391403},{"type":"v1","addr":"172.21.15.46:6817","nonce":3102391403}]},"public_addr":"172.21.15.46:6810/3102391403","cluster_addr":"172.21.15.46:6814/3102391403","heartbeat_back_addr":"172.21.15.46:6821/3102391403","heartbeat_front_addr":"172.21.15.46:6817/3102391403","state":["exists","up"]},{"osd":3,"uuid":"377bd68c-61e8-4a06-b508-18e213d1f7e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6825","nonce":1647449001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6827","nonce":1647449001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6831","nonce":1647449001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1647449001},{"type":"v1","addr":"172.21.15.46:6829","nonce":1647449001}]},"public_addr":"172.21.15.46:6825/1647449001","cluster_addr":"172.21.15.46:6827/1647449001","heartbeat_back_addr":"172.21.15.46:6831/1647449001","heartbeat_front_addr":"172.21.15.46:6829/1647449001","state":["exists","up"]},{"osd":4,"uuid":"709e6dbb-3635-4856-888e-15eb84401717","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6801","nonce":2293006647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6803","nonce":2293006647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6807","nonce":2293006647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2293006647},{"type":"v1","addr":"172.21.15.82:6805","nonce":2293006647}]},"public_addr":"172.21.15.82:6801/2293006647","cluster_addr":"172.21.15.82:6803/2293006647","heartbeat_back_addr":"172.21.15.82:6807/2293006647","heartbeat_front_addr":"172.21.15.82:6805/2293006647","state":["exists","up"]},{"osd":5,"uuid":"5954690d-8a26-42e5-9d4d-eddd58943333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6817","nonce":541310136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6819","nonce":541310136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6825","nonce":541310136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":541310136},{"type":"v1","addr":"172.21.15.82:6821","nonce":541310136}]},"public_addr":"172.21.15.82:6817/541310136","cluster_addr":"172.21.15.82:6819/541310136","heartbeat_back_addr":"172.21.15.82:6825/541310136","heartbeat_front_addr":"172.21.15.82:6821/541310136","state":["exists","up"]},{"osd":6,"uuid":"77529356-a4a8-4d24-8332-b370296b2b73","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6823","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6824","nonce":1178952145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6827","nonce":1178952145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6831","nonce":1178952145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1178952145},{"type":"v1","addr":"172.21.15.82:6829","nonce":1178952145}]},"public_addr":"172.21.15.82:6824/1178952145","cluster_addr":"172.21.15.82:6827/1178952145","heartbeat_back_addr":"172.21.15.82:6831/1178952145","heartbeat_front_addr":"172.21.15.82:6829/1178952145","state":["exists","up"]},{"osd":7,"uuid":"a4e72542-3f54-4a06-a6a9-044ec460fa4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6809","nonce":1420997799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6811","nonce":1420997799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6815","nonce":1420997799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1420997799},{"type":"v1","addr":"172.21.15.82:6813","nonce":1420997799}]},"public_addr":"172.21.15.82:6809/1420997799","cluster_addr":"172.21.15.82:6811/1420997799","heartbeat_back_addr":"172.21.15.82:6815/1420997799","heartbeat_front_addr":"172.21.15.82:6813/1420997799","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.310856+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.503715+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.401173+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.518445+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.369955+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.447426+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.453450+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:42:03.325442+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/1724167790":"2024-04-26T08:53:08.762427+0000","172.21.15.46:6834/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6832/1279640422":"2024-04-26T08:52:53.922632+0000","172.21.15.46:6835/51204523":"2024-04-26T08:52:42.856639+0000","172.21.15.46:6832/1724167790":"2024-04-26T08:53:08.762427+0000","172.21.15.46:6833/51204523":"2024-04-26T08:52:42.856639+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:15.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5540066f90 msgr2=0x7f5540069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:15.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5540066f90 0x7f5540069450 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5554009b60 tx=0x7f5554009190 comp rx=0 tx=0).stop 2024-04-25T08:53:15.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5560159230 msgr2=0x7f55600b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:15.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5560159230 0x7f55600b0780 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f555800cb70 tx=0x7f555800a430 comp rx=0 tx=0).stop 2024-04-25T08:53:15.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 shutdown_connections 2024-04-25T08:53:15.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5540066f90 0x7f5540069450 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:15.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5560159230 0x7f55600b0780 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:15.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55600cda80 0x7f5560166080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:15.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 --2- 172.21.15.46:0/2475284775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f55600cbeb0 0x7f5560159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:15.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 >> 172.21.15.46:0/2475284775 conn(0x7f55600b12a0 msgr2=0x7f55600af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:15.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 shutdown_connections 2024-04-25T08:53:15.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:15.337+0000 7f5567c00640 1 -- 172.21.15.46:0/2475284775 wait complete. 2024-04-25T08:53:16.350 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T08:53:16.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:53:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe22800640 1 Processor -- start 2024-04-25T08:53:16.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe22800640 1 -- start start 2024-04-25T08:53:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe22800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 0x7efe1c06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe22800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1c0b2e60 con 0x7efe1c141580 2024-04-25T08:53:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe22800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1c0b30d0 con 0x7efe1c0cebe0 2024-04-25T08:53:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe22800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c1412c0 con 0x7efe1c140ee0 2024-04-25T08:53:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe12a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efe1c141580 0x7efe1c0ce4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51822/0 (socket says 172.21.15.46:51822) 2024-04-25T08:53:16.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe12a00640 1 -- 172.21.15.46:0/447795221 learned_addr learned my addr 172.21.15.46:0/447795221 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe1b400640 1 --2- 172.21.15.46:0/447795221 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 0x7efe1c06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe1aa00640 1 -- 172.21.15.46:0/447795221 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270490120 0 0) 0x7efe1c0b2e60 con 0x7efe1c141580 2024-04-25T08:53:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe1aa00640 1 -- 172.21.15.46:0/447795221 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe00003530 con 0x7efe1c141580 2024-04-25T08:53:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe1aa00640 1 -- 172.21.15.46:0/447795221 <== mon.1 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883193967 0 0) 0x7efe1c0b30d0 con 0x7efe1c0cebe0 2024-04-25T08:53:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe1aa00640 1 -- 172.21.15.46:0/447795221 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe1c0b2e60 con 0x7efe1c0cebe0 2024-04-25T08:53:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe1b400640 1 -- 172.21.15.46:0/447795221 >> v1:172.21.15.82:6789/0 conn(0x7efe1c0cebe0 legacy=0x7efe1c0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe1aa00640 1 -- 172.21.15.46:0/447795221 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1329015543 0 0) 0x7efe00003530 con 0x7efe1c141580 2024-04-25T08:53:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe1b400640 1 -- 172.21.15.46:0/447795221 >> v1:172.21.15.46:6789/0 conn(0x7efe1c141580 legacy=0x7efe1c0ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.517+0000 7efe1b400640 1 -- 172.21.15.46:0/447795221 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c0b33b0 con 0x7efe1c140ee0 2024-04-25T08:53:16.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe1b400640 1 --2- 172.21.15.46:0/447795221 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 0x7efe1c06e310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009dc0 tx=0x7efe0c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80d42778cd886205 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe1aa00640 1 -- 172.21.15.46:0/447795221 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0b2070 con 0x7efe1c140ee0 2024-04-25T08:53:16.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe1aa00640 1 -- 172.21.15.46:0/447795221 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c0a5710 con 0x7efe1c140ee0 2024-04-25T08:53:16.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe1aa00640 1 -- 172.21.15.46:0/447795221 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04001680 con 0x7efe1c140ee0 2024-04-25T08:53:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe22800640 1 -- 172.21.15.46:0/447795221 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 msgr2=0x7efe1c06e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe22800640 1 --2- 172.21.15.46:0/447795221 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 0x7efe1c06e310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009dc0 tx=0x7efe0c01c8b0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe22800640 1 -- 172.21.15.46:0/447795221 shutdown_connections 2024-04-25T08:53:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe22800640 1 --2- 172.21.15.46:0/447795221 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 0x7efe1c06e310 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe22800640 1 -- 172.21.15.46:0/447795221 >> 172.21.15.46:0/447795221 conn(0x7efe1c067050 msgr2=0x7efe1c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe22800640 1 -- 172.21.15.46:0/447795221 shutdown_connections 2024-04-25T08:53:16.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.521+0000 7efe22800640 1 -- 172.21.15.46:0/447795221 wait complete. 2024-04-25T08:53:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 Processor -- start 2024-04-25T08:53:16.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 -- start start 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe1c0cebe0 0x7efe1c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 0x7efe1c134590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c141580 0x7efe1c139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c06d5d0 con 0x7efe1c141580 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c06d450 con 0x7efe1c0cebe0 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c06d750 con 0x7efe1c140ee0 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c141580 0x7efe1c139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c141580 0x7efe1c139180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42628/0 (socket says 172.21.15.46:42628) 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1be00640 1 -- 172.21.15.46:0/3190077914 learned_addr learned my addr 172.21.15.46:0/3190077914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1b400640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe1c0cebe0 0x7efe1c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe12a00640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 0x7efe1c134590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1be00640 1 -- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 msgr2=0x7efe1c134590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1be00640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efe1c140ee0 0x7efe1c134590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1be00640 1 -- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe1c0cebe0 msgr2=0x7efe1c158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1be00640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe1c0cebe0 0x7efe1c158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1be00640 1 -- 172.21.15.46:0/3190077914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c1425f0 con 0x7efe1c141580 2024-04-25T08:53:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe1be00640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c141580 0x7efe1c139180 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c410 tx=0x7efe1400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe13e00640 1 -- 172.21.15.46:0/3190077914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14017070 con 0x7efe1c141580 2024-04-25T08:53:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe13e00640 1 -- 172.21.15.46:0/3190077914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe140052d0 con 0x7efe1c141580 2024-04-25T08:53:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe13e00640 1 -- 172.21.15.46:0/3190077914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140054c0 con 0x7efe1c141580 2024-04-25T08:53:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1c1428d0 con 0x7efe1c141580 2024-04-25T08:53:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1c138980 con 0x7efe1c141580 2024-04-25T08:53:16.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.525+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe1c069080 con 0x7efe1c141580 2024-04-25T08:53:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe13e00640 1 -- 172.21.15.46:0/3190077914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe1401a070 con 0x7efe1c141580 2024-04-25T08:53:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe13e00640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdf8066f60 0x7efdf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe13e00640 1 -- 172.21.15.46:0/3190077914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7efe14084300 con 0x7efe1c141580 2024-04-25T08:53:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe13e00640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7efdf806d540 0x7efdf806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe13e00640 1 -- 172.21.15.46:0/3190077914 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efdf8070050 con 0x7efdf806d540 2024-04-25T08:53:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe13e00640 1 -- 172.21.15.46:0/3190077914 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1408b050 con 0x7efe1c141580 2024-04-25T08:53:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe1b400640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdf8066f60 0x7efdf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe12a00640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7efdf806d540 0x7efdf806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe12a00640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7efdf806d540 0x7efdf806f980 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe1b400640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdf8066f60 0x7efdf8069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0af000 tx=0x7efe0c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.529+0000 7efe13e00640 1 -- 172.21.15.46:0/3190077914 <== osd.0 v2:172.21.15.46:6800/3448226631 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efdf8070050 con 0x7efdf806d540 2024-04-25T08:53:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 --> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efe1c0af0e0 con 0x7efdf806d540 2024-04-25T08:53:16.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe13e00640 1 -- 172.21.15.46:0/3190077914 <== osd.0 v2:172.21.15.46:6800/3448226631 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7efe1c0af0e0 con 0x7efdf806d540 2024-04-25T08:53:16.537 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:53:16.537 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:53:16.537 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:53:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7efdf806d540 msgr2=0x7efdf806f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.46:6800/3448226631,v1:172.21.15.46:6801/3448226631] conn(0x7efdf806d540 0x7efdf806f980 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdf8066f60 msgr2=0x7efdf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efdf8066f60 0x7efdf8069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0af000 tx=0x7efe0c0a7070 comp rx=0 tx=0).stop 2024-04-25T08:53:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c141580 msgr2=0x7efe1c139180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 --2- 172.21.15.46:0/3190077914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c141580 0x7efe1c139180 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c410 tx=0x7efe1400c8d0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe1be00640 1 -- 172.21.15.46:0/3190077914 reap_dead start 2024-04-25T08:53:16.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 shutdown_connections 2024-04-25T08:53:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 >> 172.21.15.46:0/3190077914 conn(0x7efe1c067050 msgr2=0x7efe1c140150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 shutdown_connections 2024-04-25T08:53:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.545+0000 7efe22800640 1 -- 172.21.15.46:0/3190077914 wait complete. 2024-04-25T08:53:16.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T08:53:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.705+0000 7fc54b000640 1 Processor -- start 2024-04-25T08:53:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.705+0000 7fc54b000640 1 -- start start 2024-04-25T08:53:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.705+0000 7fc54b000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5440cf140 0x7fc54415d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.705+0000 7fc54b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54406ed70 con 0x7fc5440afd40 2024-04-25T08:53:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.705+0000 7fc54b000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5440cde00 con 0x7fc5440cf520 2024-04-25T08:53:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.705+0000 7fc54b000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5440afb40 con 0x7fc5440cf140 2024-04-25T08:53:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.705+0000 7fc53b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc5440afd40 0x7fc5440ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51834/0 (socket says 172.21.15.46:51834) 2024-04-25T08:53:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.705+0000 7fc53b400640 1 -- 172.21.15.46:0/3336820944 learned_addr learned my addr 172.21.15.46:0/3336820944 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543e00640 1 --2- 172.21.15.46:0/3336820944 >> v2:172.21.15.82:3300/0 conn(0x7fc5440cf140 0x7fc54415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153492683 0 0) 0x7fc534001120 con 0x7fc5440afd40 2024-04-25T08:53:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc528003530 con 0x7fc5440afd40 2024-04-25T08:53:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131054014 0 0) 0x7fc528003530 con 0x7fc5440afd40 2024-04-25T08:53:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 >> v2:172.21.15.82:3300/0 conn(0x7fc5440cf140 msgr2=0x7fc54415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 --2- 172.21.15.46:0/3336820944 >> v2:172.21.15.82:3300/0 conn(0x7fc5440cf140 0x7fc54415d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 >> v1:172.21.15.82:6789/0 conn(0x7fc5440cf520 legacy=0x7fc5440cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440d13f0 con 0x7fc5440afd40 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fc534001850 con 0x7fc5440afd40 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 >> v1:172.21.15.46:6789/0 conn(0x7fc5440afd40 legacy=0x7fc5440ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5280039c0 0x7fc528003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc528004810 0x7fc528004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5280051d0 0x7fc52800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5440afb40 con 0x7fc528004810 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52800c0d0 con 0x7fc5280051d0 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52800c2d0 con 0x7fc5280039c0 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc53b400640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc528004810 0x7fc528004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc543e00640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5280039c0 0x7fc528003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc549000640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5280051d0 0x7fc52800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc53b400640 1 -- 172.21.15.46:0/3336820944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5280039c0 msgr2=0x7fc528003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc53b400640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc5280039c0 0x7fc528003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc53b400640 1 -- 172.21.15.46:0/3336820944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5280051d0 msgr2=0x7fc52800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc53b400640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5280051d0 0x7fc52800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc53b400640 1 -- 172.21.15.46:0/3336820944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc528003530 con 0x7fc528004810 2024-04-25T08:53:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.709+0000 7fc549000640 1 -- 172.21.15.46:0/3336820944 reap_dead start 2024-04-25T08:53:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc53b400640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc528004810 0x7fc528004c90 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c2f0 tx=0x7fc53401cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7634a5821beb1f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534007e40 con 0x7fc528004810 2024-04-25T08:53:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5340a9070 con 0x7fc528004810 2024-04-25T08:53:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc543400640 1 -- 172.21.15.46:0/3336820944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc534005d60 con 0x7fc528004810 2024-04-25T08:53:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 -- 172.21.15.46:0/3336820944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc528004810 msgr2=0x7fc528004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc528004810 0x7fc528004c90 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fc53400c2f0 tx=0x7fc53401cb80 comp rx=0 tx=0).stop 2024-04-25T08:53:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 -- 172.21.15.46:0/3336820944 shutdown_connections 2024-04-25T08:53:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 --2- 172.21.15.46:0/3336820944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc528004810 0x7fc528004c90 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 -- 172.21.15.46:0/3336820944 >> 172.21.15.46:0/3336820944 conn(0x7fc5440728c0 msgr2=0x7fc544099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 -- 172.21.15.46:0/3336820944 shutdown_connections 2024-04-25T08:53:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 -- 172.21.15.46:0/3336820944 wait complete. 2024-04-25T08:53:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 Processor -- start 2024-04-25T08:53:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 -- start start 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc528004810 0x7fc5440cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc544154a30 0x7fc54409def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc54409a5d0 0x7fc54409aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc544161a50 con 0x7fc544154a30 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc544161770 con 0x7fc54409a5d0 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc54b000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc544161d30 con 0x7fc528004810 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc53b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc544154a30 0x7fc54409def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc53b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc544154a30 0x7fc54409def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42644/0 (socket says 172.21.15.46:42644) 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc53b400640 1 -- 172.21.15.46:0/2324518031 learned_addr learned my addr 172.21.15.46:0/2324518031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc543e00640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc528004810 0x7fc5440cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.713+0000 7fc549000640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc54409a5d0 0x7fc54409aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc543e00640 1 -- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc54409a5d0 msgr2=0x7fc54409aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc543e00640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc54409a5d0 0x7fc54409aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc543e00640 1 -- 172.21.15.46:0/2324518031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc544154a30 msgr2=0x7fc54409def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc543e00640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc544154a30 0x7fc54409def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc543e00640 1 -- 172.21.15.46:0/2324518031 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440cc750 con 0x7fc528004810 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc53b400640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc544154a30 0x7fc54409def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc543e00640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc528004810 0x7fc5440cd7d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc5440b3330 tx=0x7fc52c011cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc540c00640 1 -- 172.21.15.46:0/2324518031 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c019070 con 0x7fc528004810 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc540c00640 1 -- 172.21.15.46:0/2324518031 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c004240 con 0x7fc528004810 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc540c00640 1 -- 172.21.15.46:0/2324518031 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c00f7c0 con 0x7fc528004810 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5440cca30 con 0x7fc528004810 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5440cd2d0 con 0x7fc528004810 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54406ccb0 con 0x7fc528004810 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc540c00640 1 -- 172.21.15.46:0/2324518031 <== mon.2 v2:172.21.15.82:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc52c01c070 con 0x7fc528004810 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.717+0000 7fc540c00640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc520066f60 0x7fc520069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.721+0000 7fc540c00640 1 -- 172.21.15.46:0/2324518031 <== mon.2 v2:172.21.15.82:3301/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc52c085470 con 0x7fc528004810 2024-04-25T08:53:16.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.721+0000 7fc53b400640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc520066f60 0x7fc520069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.721+0000 7fc53b400640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc520066f60 0x7fc520069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc53401cbc0 tx=0x7fc5340a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.725+0000 7fc540c00640 1 -- 172.21.15.46:0/2324518031 <== mon.2 v2:172.21.15.82:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc52c055f60 con 0x7fc528004810 2024-04-25T08:53:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc544069060 con 0x7fc520066f60 2024-04-25T08:53:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc540c00640 1 -- 172.21.15.46:0/2324518031 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc544069060 con 0x7fc520066f60 2024-04-25T08:53:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T08:53:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc520066f60 msgr2=0x7fc520069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc520066f60 0x7fc520069420 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fc53401cbc0 tx=0x7fc5340a7070 comp rx=0 tx=0).stop 2024-04-25T08:53:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc528004810 msgr2=0x7fc5440cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc528004810 0x7fc5440cd7d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc5440b3330 tx=0x7fc52c011cf0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 shutdown_connections 2024-04-25T08:53:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fc520066f60 0x7fc520069420 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc54409a5d0 0x7fc54409aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc544154a30 0x7fc54409def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 --2- 172.21.15.46:0/2324518031 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fc528004810 0x7fc5440cd7d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 >> 172.21.15.46:0/2324518031 conn(0x7fc5440728c0 msgr2=0x7fc5440b4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 shutdown_connections 2024-04-25T08:53:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:16.885+0000 7fc54b000640 1 -- 172.21.15.46:0/2324518031 wait complete. 2024-04-25T08:53:16.897 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T08:53:16.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:53:17.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.049+0000 7f42e0600640 1 Processor -- start 2024-04-25T08:53:17.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.049+0000 7f42e0600640 1 -- start start 2024-04-25T08:53:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42e0600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42d813e6b0 0x7f42d80711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42e0600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d806d330 con 0x7f42d8071720 2024-04-25T08:53:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42e0600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d806d610 con 0x7f42d815c680 2024-04-25T08:53:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42e0600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d815cce0 con 0x7f42d813e6b0 2024-04-25T08:53:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42dd200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f42d8071720 0x7f42d815bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51842/0 (socket says 172.21.15.46:51842) 2024-04-25T08:53:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42dd200640 1 -- 172.21.15.46:0/1366953956 learned_addr learned my addr 172.21.15.46:0/1366953956 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:17.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42ddc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42d813e6b0 0x7f42d80711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42ddc00640 1 -- 172.21.15.46:0/1366953956 >> v1:172.21.15.82:6789/0 conn(0x7f42d815c680 legacy=0x7f42d806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42ddc00640 1 -- 172.21.15.46:0/1366953956 >> v1:172.21.15.46:6789/0 conn(0x7f42d8071720 legacy=0x7f42d815bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.053+0000 7f42ddc00640 1 -- 172.21.15.46:0/1366953956 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d806d8f0 con 0x7f42d813e6b0 2024-04-25T08:53:17.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42ddc00640 1 --2- 172.21.15.46:0/1366953956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d813e6b0 0x7f42d80711e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0097c0 tx=0x7f42cc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dab51607e5f3bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42d7e00640 1 -- 172.21.15.46:0/1366953956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0b0070 con 0x7f42d813e6b0 2024-04-25T08:53:17.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42d7e00640 1 -- 172.21.15.46:0/1366953956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc0ab600 con 0x7f42d813e6b0 2024-04-25T08:53:17.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42d7e00640 1 -- 172.21.15.46:0/1366953956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0ab9d0 con 0x7f42d813e6b0 2024-04-25T08:53:17.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 -- 172.21.15.46:0/1366953956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d813e6b0 msgr2=0x7f42d80711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 --2- 172.21.15.46:0/1366953956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d813e6b0 0x7f42d80711e0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0097c0 tx=0x7f42cc01c380 comp rx=0 tx=0).stop 2024-04-25T08:53:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 -- 172.21.15.46:0/1366953956 shutdown_connections 2024-04-25T08:53:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 --2- 172.21.15.46:0/1366953956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d813e6b0 0x7f42d80711e0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 -- 172.21.15.46:0/1366953956 >> 172.21.15.46:0/1366953956 conn(0x7f42d8067050 msgr2=0x7f42d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 -- 172.21.15.46:0/1366953956 shutdown_connections 2024-04-25T08:53:17.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 -- 172.21.15.46:0/1366953956 wait complete. 2024-04-25T08:53:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 Processor -- start 2024-04-25T08:53:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 -- start start 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d8071720 0x7f42d80b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d813e6b0 0x7f42d8153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d815c680 0x7f42d80b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d815ca90 con 0x7f42d8071720 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8071b30 con 0x7f42d815c680 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.057+0000 7f42e0600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d81695a0 con 0x7f42d813e6b0 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d8071720 0x7f42d80b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42ddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d8071720 0x7f42d80b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42668/0 (socket says 172.21.15.46:42668) 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42ddc00640 1 -- 172.21.15.46:0/1799472398 learned_addr learned my addr 172.21.15.46:0/1799472398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42dd200640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d813e6b0 0x7f42d8153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42de600640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d815c680 0x7f42d80b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42ddc00640 1 -- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d813e6b0 msgr2=0x7f42d8153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42ddc00640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d813e6b0 0x7f42d8153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42ddc00640 1 -- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d815c680 msgr2=0x7f42d80b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42ddc00640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42d815c680 0x7f42d80b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42ddc00640 1 -- 172.21.15.46:0/1799472398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8136d80 con 0x7f42d8071720 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42dd200640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f42d813e6b0 0x7f42d8153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42ddc00640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d8071720 0x7f42d80b3f00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0ae040 tx=0x7f42cc01abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42d5600640 1 -- 172.21.15.46:0/1799472398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0b0050 con 0x7f42d8071720 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42d5600640 1 -- 172.21.15.46:0/1799472398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc0bc070 con 0x7f42d8071720 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42d5600640 1 -- 172.21.15.46:0/1799472398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc0b5a50 con 0x7f42d8071720 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d8134670 con 0x7f42d8071720 2024-04-25T08:53:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d8137020 con 0x7f42d8071720 2024-04-25T08:53:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42d8069080 con 0x7f42d8071720 2024-04-25T08:53:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.061+0000 7f42d5600640 1 -- 172.21.15.46:0/1799472398 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42cc0ab4a0 con 0x7f42d8071720 2024-04-25T08:53:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42d5600640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42b8066f60 0x7f42b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42d5600640 1 -- 172.21.15.46:0/1799472398 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f42cc11f8e0 con 0x7f42d8071720 2024-04-25T08:53:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42d5600640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f42b806d540 0x7f42b806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42d5600640 1 -- 172.21.15.46:0/1799472398 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42cc001d40 con 0x7f42b806d540 2024-04-25T08:53:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42d5600640 1 -- 172.21.15.46:0/1799472398 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42cc1242a0 con 0x7f42d8071720 2024-04-25T08:53:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42dd200640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42b8066f60 0x7f42b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42de600640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f42b806d540 0x7f42b806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42de600640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f42b806d540 0x7f42b806f980 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42dd200640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42b8066f60 0x7f42b8069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f42d8133d60 tx=0x7f42c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.065+0000 7f42d5600640 1 -- 172.21.15.46:0/1799472398 <== osd.1 v2:172.21.15.46:6809/2552657765 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f42cc001d40 con 0x7f42b806d540 2024-04-25T08:53:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.077+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 --> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f42d80cf240 con 0x7f42b806d540 2024-04-25T08:53:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.077+0000 7f42d5600640 1 -- 172.21.15.46:0/1799472398 <== osd.1 v2:172.21.15.46:6809/2552657765 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f42d80cf240 con 0x7f42b806d540 2024-04-25T08:53:17.071 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:53:17.071 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:53:17.071 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:53:17.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.077+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f42b806d540 msgr2=0x7f42b806f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.077+0000 7f42e0600640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.46:6809/2552657765,v1:172.21.15.46:6811/2552657765] conn(0x7f42b806d540 0x7f42b806f980 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.081+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42b8066f60 msgr2=0x7f42b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.081+0000 7f42e0600640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f42b8066f60 0x7f42b8069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f42d8133d60 tx=0x7f42c4009040 comp rx=0 tx=0).stop 2024-04-25T08:53:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.081+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d8071720 msgr2=0x7f42d80b3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.081+0000 7f42e0600640 1 --2- 172.21.15.46:0/1799472398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42d8071720 0x7f42d80b3f00 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0ae040 tx=0x7f42cc01abc0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.081+0000 7f42de600640 1 -- 172.21.15.46:0/1799472398 reap_dead start 2024-04-25T08:53:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.081+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 shutdown_connections 2024-04-25T08:53:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.081+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 >> 172.21.15.46:0/1799472398 conn(0x7f42d8067050 msgr2=0x7f42d815bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.081+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 shutdown_connections 2024-04-25T08:53:17.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.081+0000 7f42e0600640 1 -- 172.21.15.46:0/1799472398 wait complete. 2024-04-25T08:53:17.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T08:53:17.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3de00640 1 Processor -- start 2024-04-25T08:53:17.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3de00640 1 -- start start 2024-04-25T08:53:17.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f3813e540 0x7f8f380cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f380cd750 0x7f8f38141560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3de00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f380710e0 con 0x7f8f38071ea0 2024-04-25T08:53:17.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3de00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f380713c0 con 0x7f8f3813e540 2024-04-25T08:53:17.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f380716a0 con 0x7f8f380cd750 2024-04-25T08:53:17.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f3813e540 0x7f8f380cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f3813e540 0x7f8f380cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42674/0 (socket says 172.21.15.46:42674) 2024-04-25T08:53:17.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3ca00640 1 -- 172.21.15.46:0/3432877859 learned_addr learned my addr 172.21.15.46:0/3432877859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:17.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3d400640 1 --2- 172.21.15.46:0/3432877859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f380cd750 0x7f8f38141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3ca00640 1 -- 172.21.15.46:0/3432877859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f380cd750 msgr2=0x7f8f38141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3ca00640 1 --2- 172.21.15.46:0/3432877859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f380cd750 0x7f8f38141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3ca00640 1 -- 172.21.15.46:0/3432877859 >> v1:172.21.15.82:6789/0 conn(0x7f8f38071ea0 legacy=0x7f8f38140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.241+0000 7f8f3ca00640 1 -- 172.21.15.46:0/3432877859 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f38071980 con 0x7f8f3813e540 2024-04-25T08:53:17.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f3ca00640 1 --2- 172.21.15.46:0/3432877859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f3813e540 0x7f8f380cd210 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009870 tx=0x7f8f2c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228845e5b7294b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f37400640 1 -- 172.21.15.46:0/3432877859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0b2070 con 0x7f8f3813e540 2024-04-25T08:53:17.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f37400640 1 -- 172.21.15.46:0/3432877859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c0a9070 con 0x7f8f3813e540 2024-04-25T08:53:17.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f37400640 1 -- 172.21.15.46:0/3432877859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0ad880 con 0x7f8f3813e540 2024-04-25T08:53:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f3de00640 1 -- 172.21.15.46:0/3432877859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f3813e540 msgr2=0x7f8f380cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f3de00640 1 --2- 172.21.15.46:0/3432877859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f3813e540 0x7f8f380cd210 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c009870 tx=0x7f8f2c01c560 comp rx=0 tx=0).stop 2024-04-25T08:53:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f3de00640 1 -- 172.21.15.46:0/3432877859 shutdown_connections 2024-04-25T08:53:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f3de00640 1 --2- 172.21.15.46:0/3432877859 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f380cd750 0x7f8f38141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f3de00640 1 --2- 172.21.15.46:0/3432877859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f3813e540 0x7f8f380cd210 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f3de00640 1 -- 172.21.15.46:0/3432877859 >> 172.21.15.46:0/3432877859 conn(0x7f8f38066fa0 msgr2=0x7f8f3806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f3de00640 1 -- 172.21.15.46:0/3432877859 shutdown_connections 2024-04-25T08:53:17.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.245+0000 7f8f3de00640 1 -- 172.21.15.46:0/3432877859 wait complete. 2024-04-25T08:53:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 Processor -- start 2024-04-25T08:53:17.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 -- start start 2024-04-25T08:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f38071ea0 0x7f8f38135820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380cd750 0x7f8f38141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3813e540 0x7f8f3815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3813f8b0 con 0x7f8f380cd750 2024-04-25T08:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3813f730 con 0x7f8f3813e540 2024-04-25T08:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3813fa30 con 0x7f8f38071ea0 2024-04-25T08:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380cd750 0x7f8f38141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380cd750 0x7f8f38141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42684/0 (socket says 172.21.15.46:42684) 2024-04-25T08:53:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f37e00640 1 -- 172.21.15.46:0/1419590347 learned_addr learned my addr 172.21.15.46:0/1419590347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3ca00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f38071ea0 0x7f8f38135820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3d400640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3813e540 0x7f8f3815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f37e00640 1 -- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f38071ea0 msgr2=0x7f8f38135820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f37e00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f38071ea0 0x7f8f38135820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f37e00640 1 -- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3813e540 msgr2=0x7f8f3815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f37e00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3813e540 0x7f8f3815ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f37e00640 1 -- 172.21.15.46:0/1419590347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f38144b10 con 0x7f8f380cd750 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f37e00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380cd750 0x7f8f38141fd0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400c3a0 tx=0x7f8f2400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f34c00640 1 -- 172.21.15.46:0/1419590347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f240052d0 con 0x7f8f380cd750 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f34c00640 1 -- 172.21.15.46:0/1419590347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f24005470 con 0x7f8f380cd750 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f34c00640 1 -- 172.21.15.46:0/1419590347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24007260 con 0x7f8f380cd750 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3815a430 con 0x7f8f380cd750 2024-04-25T08:53:17.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.249+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f38136050 con 0x7f8f380cd750 2024-04-25T08:53:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.253+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f38144ce0 con 0x7f8f380cd750 2024-04-25T08:53:17.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.253+0000 7f8f34c00640 1 -- 172.21.15.46:0/1419590347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f24007400 con 0x7f8f380cd750 2024-04-25T08:53:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.253+0000 7f8f34c00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f18066f60 0x7f8f18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.253+0000 7f8f34c00640 1 -- 172.21.15.46:0/1419590347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8f240843a0 con 0x7f8f380cd750 2024-04-25T08:53:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.253+0000 7f8f3ca00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f18066f60 0x7f8f18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.253+0000 7f8f3ca00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f18066f60 0x7f8f18069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0af000 tx=0x7f8f2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.257+0000 7f8f34c00640 1 -- 172.21.15.46:0/1419590347 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f38144ce0 con 0x7f8f380cd750 2024-04-25T08:53:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f381448c0 con 0x7f8f18066f60 2024-04-25T08:53:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f34c00640 1 -- 172.21.15.46:0/1419590347 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8f381448c0 con 0x7f8f18066f60 2024-04-25T08:53:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T08:53:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f18066f60 msgr2=0x7f8f18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f18066f60 0x7f8f18069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0af000 tx=0x7f8f2c008040 comp rx=0 tx=0).stop 2024-04-25T08:53:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380cd750 msgr2=0x7f8f38141fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380cd750 0x7f8f38141fd0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f8f2400c3a0 tx=0x7f8f2400c860 comp rx=0 tx=0).stop 2024-04-25T08:53:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 shutdown_connections 2024-04-25T08:53:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8f18066f60 0x7f8f18069420 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f3813e540 0x7f8f3815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f380cd750 0x7f8f38141fd0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 --2- 172.21.15.46:0/1419590347 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8f38071ea0 0x7f8f38135820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 >> 172.21.15.46:0/1419590347 conn(0x7f8f38066fa0 msgr2=0x7f8f3815cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 shutdown_connections 2024-04-25T08:53:17.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.421+0000 7f8f3de00640 1 -- 172.21.15.46:0/1419590347 wait complete. 2024-04-25T08:53:17.434 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T08:53:17.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:53:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.585+0000 7f5dba600640 1 Processor -- start 2024-04-25T08:53:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.585+0000 7f5dba600640 1 -- start start 2024-04-25T08:53:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5dba600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5db406ea20 0x7f5db40ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5dba600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5db4141580 0x7f5db40cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5dba600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40b2e60 con 0x7f5db4140ee0 2024-04-25T08:53:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5dba600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db41412c0 con 0x7f5db4141580 2024-04-25T08:53:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5dba600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40cef50 con 0x7f5db406ea20 2024-04-25T08:53:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5db4140ee0 0x7f5db406e310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51852/0 (socket says 172.21.15.46:51852) 2024-04-25T08:53:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db3400640 1 -- 172.21.15.46:0/4265805685 learned_addr learned my addr 172.21.15.46:0/4265805685 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db3e00640 1 --2- 172.21.15.46:0/4265805685 >> v2:172.21.15.46:3300/0 conn(0x7f5db4141580 0x7f5db40cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db2a00640 1 --2- 172.21.15.46:0/4265805685 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5db406ea20 0x7f5db40ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db2000640 1 -- 172.21.15.46:0/4265805685 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115198960 0 0) 0x7f5db40b2e60 con 0x7f5db4140ee0 2024-04-25T08:53:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db2000640 1 -- 172.21.15.46:0/4265805685 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d98003530 con 0x7f5db4140ee0 2024-04-25T08:53:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db3e00640 1 -- 172.21.15.46:0/4265805685 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5db406ea20 msgr2=0x7f5db40ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db3e00640 1 --2- 172.21.15.46:0/4265805685 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5db406ea20 0x7f5db40ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db3e00640 1 -- 172.21.15.46:0/4265805685 >> v1:172.21.15.46:6789/0 conn(0x7f5db4140ee0 legacy=0x7f5db406e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.589+0000 7f5db3e00640 1 -- 172.21.15.46:0/4265805685 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40b30d0 con 0x7f5db4141580 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5db3e00640 1 --2- 172.21.15.46:0/4265805685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 0x7f5db40cea10 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009dc0 tx=0x7f5dac01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13d990c0122c896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5db2000640 1 -- 172.21.15.46:0/4265805685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0b2070 con 0x7f5db4141580 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5db2000640 1 -- 172.21.15.46:0/4265805685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac0a5a10 con 0x7f5db4141580 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5db2000640 1 -- 172.21.15.46:0/4265805685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0a5de0 con 0x7f5db4141580 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 -- 172.21.15.46:0/4265805685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 msgr2=0x7f5db40cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 --2- 172.21.15.46:0/4265805685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 0x7f5db40cea10 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009dc0 tx=0x7f5dac01c950 comp rx=0 tx=0).stop 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 -- 172.21.15.46:0/4265805685 shutdown_connections 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 --2- 172.21.15.46:0/4265805685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 0x7f5db40cea10 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 --2- 172.21.15.46:0/4265805685 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5db406ea20 0x7f5db40ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 -- 172.21.15.46:0/4265805685 >> 172.21.15.46:0/4265805685 conn(0x7f5db4067050 msgr2=0x7f5db406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 -- 172.21.15.46:0/4265805685 shutdown_connections 2024-04-25T08:53:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 -- 172.21.15.46:0/4265805685 wait complete. 2024-04-25T08:53:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 Processor -- start 2024-04-25T08:53:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 -- start start 2024-04-25T08:53:17.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5db406ea20 0x7f5db415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db4140ee0 0x7f5db413a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.593+0000 7f5dba600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 0x7f5db413a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dba600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b4540 con 0x7f5db4141580 2024-04-25T08:53:17.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dba600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b43c0 con 0x7f5db4140ee0 2024-04-25T08:53:17.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dba600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b46c0 con 0x7f5db406ea20 2024-04-25T08:53:17.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 0x7f5db413a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 0x7f5db413a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42704/0 (socket says 172.21.15.46:42704) 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3e00640 1 -- 172.21.15.46:0/2176277398 learned_addr learned my addr 172.21.15.46:0/2176277398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3400640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5db406ea20 0x7f5db415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db2a00640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db4140ee0 0x7f5db413a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3e00640 1 -- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5db406ea20 msgr2=0x7f5db415a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3e00640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f5db406ea20 0x7f5db415a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3e00640 1 -- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db4140ee0 msgr2=0x7f5db413a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3e00640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db4140ee0 0x7f5db413a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3e00640 1 -- 172.21.15.46:0/2176277398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4141a30 con 0x7f5db4141580 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db2a00640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5db4140ee0 0x7f5db413a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5db3e00640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 0x7f5db413a700 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5dac01b000 tx=0x7f5dac0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dab400640 1 -- 172.21.15.46:0/2176277398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0b2050 con 0x7f5db4141580 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dab400640 1 -- 172.21.15.46:0/2176277398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac0be070 con 0x7f5db4141580 2024-04-25T08:53:17.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dab400640 1 -- 172.21.15.46:0/2176277398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0ada50 con 0x7f5db4141580 2024-04-25T08:53:17.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db4142a90 con 0x7f5db4141580 2024-04-25T08:53:17.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db4141fb0 con 0x7f5db4141580 2024-04-25T08:53:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5db4069080 con 0x7f5db4141580 2024-04-25T08:53:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.597+0000 7f5dab400640 1 -- 172.21.15.46:0/2176277398 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dac0a58d0 con 0x7f5db4141580 2024-04-25T08:53:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5dab400640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d94066f60 0x7f5d94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5dab400640 1 -- 172.21.15.46:0/2176277398 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f5dac1208e0 con 0x7f5db4141580 2024-04-25T08:53:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5dab400640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f5d9406d540 0x7f5d9406f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5dab400640 1 -- 172.21.15.46:0/2176277398 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5dac0c5e70 con 0x7f5d9406d540 2024-04-25T08:53:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5dab400640 1 -- 172.21.15.46:0/2176277398 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dac120d70 con 0x7f5db4141580 2024-04-25T08:53:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5db3400640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d94066f60 0x7f5d94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5db2a00640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f5d9406d540 0x7f5d9406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5db2a00640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f5d9406d540 0x7f5d9406f980 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5db3400640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d94066f60 0x7f5d94069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5da4007fd0 tx=0x7f5da4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.601+0000 7f5dab400640 1 -- 172.21.15.46:0/2176277398 <== osd.2 v2:172.21.15.46:6808/3102391403 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5dac0c5e70 con 0x7f5d9406d540 2024-04-25T08:53:17.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 --> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5db406e800 con 0x7f5d9406d540 2024-04-25T08:53:17.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dab400640 1 -- 172.21.15.46:0/2176277398 <== osd.2 v2:172.21.15.46:6808/3102391403 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5db406e800 con 0x7f5d9406d540 2024-04-25T08:53:17.607 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:53:17.607 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:53:17.607 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:53:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f5d9406d540 msgr2=0x7f5d9406f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.46:6808/3102391403,v1:172.21.15.46:6810/3102391403] conn(0x7f5d9406d540 0x7f5d9406f980 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d94066f60 msgr2=0x7f5d94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5d94066f60 0x7f5d94069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5da4007fd0 tx=0x7f5da4009040 comp rx=0 tx=0).stop 2024-04-25T08:53:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 msgr2=0x7f5db413a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 --2- 172.21.15.46:0/2176277398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4141580 0x7f5db413a700 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5dac01b000 tx=0x7f5dac0196f0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5db3e00640 1 -- 172.21.15.46:0/2176277398 reap_dead start 2024-04-25T08:53:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 shutdown_connections 2024-04-25T08:53:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 >> 172.21.15.46:0/2176277398 conn(0x7f5db4067050 msgr2=0x7f5db413ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.613+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 shutdown_connections 2024-04-25T08:53:17.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.617+0000 7f5dba600640 1 -- 172.21.15.46:0/2176277398 wait complete. 2024-04-25T08:53:17.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T08:53:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.773+0000 7f8f15600640 1 Processor -- start 2024-04-25T08:53:17.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.773+0000 7f8f15600640 1 -- start start 2024-04-25T08:53:17.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f15600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8f1015a750 0x7f8f1006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f15600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f1006d5e0 con 0x7f8f100aed10 2024-04-25T08:53:17.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f15600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f100ce720 con 0x7f8f100cc9e0 2024-04-25T08:53:17.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f15600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1015ab30 con 0x7f8f1015a750 2024-04-25T08:53:17.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0e000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8f100aed10 0x7f8f100b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51864/0 (socket says 172.21.15.46:51864) 2024-04-25T08:53:17.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0e000640 1 -- 172.21.15.46:0/255421836 learned_addr learned my addr 172.21.15.46:0/255421836 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:17.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0ea00640 1 --2- 172.21.15.46:0/255421836 >> v2:172.21.15.82:3300/0 conn(0x7f8f1015a750 0x7f8f1006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3399200215 0 0) 0x7f8f1006d5e0 con 0x7f8f100aed10 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef4003530 con 0x7f8f100aed10 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2010691632 0 0) 0x7f8ef4003530 con 0x7f8f100aed10 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 >> v2:172.21.15.82:3300/0 conn(0x7f8f1015a750 msgr2=0x7f8f1006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 --2- 172.21.15.46:0/255421836 >> v2:172.21.15.82:3300/0 conn(0x7f8f1015a750 0x7f8f1006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 >> v1:172.21.15.82:6789/0 conn(0x7f8f100cc9e0 legacy=0x7f8f100b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100cea00 con 0x7f8f100aed10 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f8efc001680 con 0x7f8f100aed10 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 >> v1:172.21.15.46:6789/0 conn(0x7f8f100aed10 legacy=0x7f8f100b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40039c0 0x7f8ef4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef4004810 0x7f8ef4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef40051d0 0x7f8ef400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef400c040 con 0x7f8ef40051d0 2024-04-25T08:53:17.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef400c1f0 con 0x7f8ef40039c0 2024-04-25T08:53:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef400c3f0 con 0x7f8ef4004810 2024-04-25T08:53:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0f400640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef40051d0 0x7f8ef400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0e000640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef4004810 0x7f8ef4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0ea00640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40039c0 0x7f8ef4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0f400640 1 -- 172.21.15.46:0/255421836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef4004810 msgr2=0x7f8ef4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0f400640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef4004810 0x7f8ef4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0f400640 1 -- 172.21.15.46:0/255421836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40039c0 msgr2=0x7f8ef4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0f400640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40039c0 0x7f8ef4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.777+0000 7f8f0f400640 1 -- 172.21.15.46:0/255421836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4003530 con 0x7f8ef40051d0 2024-04-25T08:53:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f0f400640 1 -- 172.21.15.46:0/255421836 reap_dead start 2024-04-25T08:53:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f0f400640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef40051d0 0x7f8ef400b9d0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f8f08001e70 tx=0x7f8f0801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c798176c8616b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08001a90 con 0x7f8ef40051d0 2024-04-25T08:53:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f08003bc0 con 0x7f8ef40051d0 2024-04-25T08:53:17.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f0d600640 1 -- 172.21.15.46:0/255421836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08005020 con 0x7f8ef40051d0 2024-04-25T08:53:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f15600640 1 -- 172.21.15.46:0/255421836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef40051d0 msgr2=0x7f8ef400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f15600640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef40051d0 0x7f8ef400b9d0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f8f08001e70 tx=0x7f8f0801c780 comp rx=0 tx=0).stop 2024-04-25T08:53:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f15600640 1 -- 172.21.15.46:0/255421836 shutdown_connections 2024-04-25T08:53:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f15600640 1 --2- 172.21.15.46:0/255421836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ef40051d0 0x7f8ef400b9d0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f15600640 1 -- 172.21.15.46:0/255421836 >> 172.21.15.46:0/255421836 conn(0x7f8f100b12a0 msgr2=0x7f8f100b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f15600640 1 -- 172.21.15.46:0/255421836 shutdown_connections 2024-04-25T08:53:17.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.781+0000 7f8f15600640 1 -- 172.21.15.46:0/255421836 wait complete. 2024-04-25T08:53:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 Processor -- start 2024-04-25T08:53:17.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 -- start start 2024-04-25T08:53:17.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef40051d0 0x7f8f10147570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f100c4510 0x7f8f100c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1015b970 0x7f8f1015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1006d8c0 con 0x7f8f1015b970 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100cf200 con 0x7f8f100c4510 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1006d270 con 0x7f8ef40051d0 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1015b970 0x7f8f1015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1015b970 0x7f8f1015bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42710/0 (socket says 172.21.15.46:42710) 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0f400640 1 -- 172.21.15.46:0/4001959794 learned_addr learned my addr 172.21.15.46:0/4001959794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0ea00640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef40051d0 0x7f8f10147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0e000640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f100c4510 0x7f8f100c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0f400640 1 -- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef40051d0 msgr2=0x7f8f10147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0f400640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef40051d0 0x7f8f10147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0f400640 1 -- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f100c4510 msgr2=0x7f8f100c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0f400640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f100c4510 0x7f8f100c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0f400640 1 -- 172.21.15.46:0/4001959794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1015c970 con 0x7f8f1015b970 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0e000640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f100c4510 0x7f8f100c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0ea00640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef40051d0 0x7f8f10147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f0f400640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1015b970 0x7f8f1015bdf0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f8f0801cc60 tx=0x7f8ef4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f06a00640 1 -- 172.21.15.46:0/4001959794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08007dc0 con 0x7f8f1015b970 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f06a00640 1 -- 172.21.15.46:0/4001959794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f080057f0 con 0x7f8f1015b970 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f06a00640 1 -- 172.21.15.46:0/4001959794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08006d70 con 0x7f8f1015b970 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1015c690 con 0x7f8f1015b970 2024-04-25T08:53:17.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f10148200 con 0x7f8f1015b970 2024-04-25T08:53:17.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.785+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f10069080 con 0x7f8f1015b970 2024-04-25T08:53:17.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.789+0000 7f8f06a00640 1 -- 172.21.15.46:0/4001959794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f080a6480 con 0x7f8f1015b970 2024-04-25T08:53:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.789+0000 7f8f06a00640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.789+0000 7f8f06a00640 1 -- 172.21.15.46:0/4001959794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8f0811e650 con 0x7f8f1015b970 2024-04-25T08:53:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.789+0000 7f8f0ea00640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.789+0000 7f8f0ea00640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8ef0066f60 0x7f8ef0069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009a60 tx=0x7f8f00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.793+0000 7f8f06a00640 1 -- 172.21.15.46:0/4001959794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8f080b8060 con 0x7f8f1015b970 2024-04-25T08:53:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.957+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f100b27c0 con 0x7f8ef0066f60 2024-04-25T08:53:17.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.957+0000 7f8f06a00640 1 -- 172.21.15.46:0/4001959794 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8f100b27c0 con 0x7f8ef0066f60 2024-04-25T08:53:17.951 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8ef0066f60 msgr2=0x7f8ef0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8ef0066f60 0x7f8ef0069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009a60 tx=0x7f8f00008040 comp rx=0 tx=0).stop 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1015b970 msgr2=0x7f8f1015bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1015b970 0x7f8f1015bdf0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f8f0801cc60 tx=0x7f8ef4005840 comp rx=0 tx=0).stop 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 shutdown_connections 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f8ef0066f60 0x7f8ef0069420 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f1015b970 0x7f8f1015bdf0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f100c4510 0x7f8f100c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 --2- 172.21.15.46:0/4001959794 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f8ef40051d0 0x7f8f10147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 >> 172.21.15.46:0/4001959794 conn(0x7f8f100b12a0 msgr2=0x7f8f100b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 shutdown_connections 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:17.961+0000 7f8f15600640 1 -- 172.21.15.46:0/4001959794 wait complete. 2024-04-25T08:53:17.972 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T08:53:17.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:53:18.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.125+0000 7f7696200640 1 Processor -- start 2024-04-25T08:53:18.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.125+0000 7f7696200640 1 -- start start 2024-04-25T08:53:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f7696200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f769006bc20 0x7f769006c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f7696200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769015dce0 0x7f769006db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f7696200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769006c540 con 0x7f7690072600 2024-04-25T08:53:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f7696200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769006ee30 con 0x7f769006bc20 2024-04-25T08:53:18.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f7696200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769006f110 con 0x7f769015dce0 2024-04-25T08:53:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f768f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f769006bc20 0x7f769006c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f768f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f769006bc20 0x7f769006c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41108/0 (socket says 172.21.15.46:41108) 2024-04-25T08:53:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f768f400640 1 -- 172.21.15.46:0/1298346345 learned_addr learned my addr 172.21.15.46:0/1298346345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f7686a00640 1 --2- 172.21.15.46:0/1298346345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769015dce0 0x7f769006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f768f400640 1 -- 172.21.15.46:0/1298346345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769015dce0 msgr2=0x7f769006db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f768f400640 1 --2- 172.21.15.46:0/1298346345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769015dce0 0x7f769006db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f768f400640 1 -- 172.21.15.46:0/1298346345 >> v1:172.21.15.82:6789/0 conn(0x7f7690072600 legacy=0x7f769006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f768f400640 1 -- 172.21.15.46:0/1298346345 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769006f3f0 con 0x7f769006bc20 2024-04-25T08:53:18.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.129+0000 7f768f400640 1 --2- 172.21.15.46:0/1298346345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769006bc20 0x7f769006c000 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7680009e50 tx=0x7f768001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60732e2cd1edc98a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f768ea00640 1 -- 172.21.15.46:0/1298346345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800a5890 con 0x7f769006bc20 2024-04-25T08:53:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f768ea00640 1 -- 172.21.15.46:0/1298346345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76800a5a30 con 0x7f769006bc20 2024-04-25T08:53:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f768ea00640 1 -- 172.21.15.46:0/1298346345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76800a5e00 con 0x7f769006bc20 2024-04-25T08:53:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 -- 172.21.15.46:0/1298346345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769006bc20 msgr2=0x7f769006c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 --2- 172.21.15.46:0/1298346345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769006bc20 0x7f769006c000 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7680009e50 tx=0x7f768001ca00 comp rx=0 tx=0).stop 2024-04-25T08:53:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 -- 172.21.15.46:0/1298346345 shutdown_connections 2024-04-25T08:53:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 --2- 172.21.15.46:0/1298346345 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769015dce0 0x7f769006db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 --2- 172.21.15.46:0/1298346345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769006bc20 0x7f769006c000 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 -- 172.21.15.46:0/1298346345 >> 172.21.15.46:0/1298346345 conn(0x7f7690071ae0 msgr2=0x7f7690071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 -- 172.21.15.46:0/1298346345 shutdown_connections 2024-04-25T08:53:18.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 -- 172.21.15.46:0/1298346345 wait complete. 2024-04-25T08:53:18.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 Processor -- start 2024-04-25T08:53:18.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 -- start start 2024-04-25T08:53:18.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769006bc20 0x7f7690140bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7690072600 0x7f76901475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769015dce0 0x7f7690141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769006eba0 con 0x7f769015dce0 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76900b3d20 con 0x7f7690072600 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7696200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769016ae40 con 0x7f769006bc20 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f768fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769015dce0 0x7f7690141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f768fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769015dce0 0x7f7690141df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41122/0 (socket says 172.21.15.46:41122) 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f768fe00640 1 -- 172.21.15.46:0/3915417007 learned_addr learned my addr 172.21.15.46:0/3915417007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f768f400640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769006bc20 0x7f7690140bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.133+0000 7f7686a00640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7690072600 0x7f76901475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f768fe00640 1 -- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769006bc20 msgr2=0x7f7690140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f768fe00640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f769006bc20 0x7f7690140bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f768fe00640 1 -- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7690072600 msgr2=0x7f76901475a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f768fe00640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7690072600 0x7f76901475a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f768fe00640 1 -- 172.21.15.46:0/3915417007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76901467e0 con 0x7f769015dce0 2024-04-25T08:53:18.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f768fe00640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769015dce0 0x7f7690141df0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f768800c430 tx=0x7f768800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7687e00640 1 -- 172.21.15.46:0/3915417007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688017070 con 0x7f769015dce0 2024-04-25T08:53:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7687e00640 1 -- 172.21.15.46:0/3915417007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76880052d0 con 0x7f769015dce0 2024-04-25T08:53:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7687e00640 1 -- 172.21.15.46:0/3915417007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688003800 con 0x7f769015dce0 2024-04-25T08:53:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7690146ac0 con 0x7f769015dce0 2024-04-25T08:53:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7690147080 con 0x7f769015dce0 2024-04-25T08:53:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7690146590 con 0x7f769015dce0 2024-04-25T08:53:18.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7687e00640 1 -- 172.21.15.46:0/3915417007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f768801a070 con 0x7f769015dce0 2024-04-25T08:53:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7687e00640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f766c066f60 0x7f766c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7687e00640 1 -- 172.21.15.46:0/3915417007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f76880849c0 con 0x7f769015dce0 2024-04-25T08:53:18.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7687e00640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f766c06d540 0x7f766c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7687e00640 1 -- 172.21.15.46:0/3915417007 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f766c070050 con 0x7f766c06d540 2024-04-25T08:53:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.137+0000 7f7687e00640 1 -- 172.21.15.46:0/3915417007 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7688084d70 con 0x7f769015dce0 2024-04-25T08:53:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.141+0000 7f768f400640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f766c066f60 0x7f766c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.141+0000 7f7686a00640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f766c06d540 0x7f766c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.141+0000 7f7686a00640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f766c06d540 0x7f766c06f980 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.141+0000 7f768f400640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f766c066f60 0x7f766c069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f76800ae000 tx=0x7f76800a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.141+0000 7f7687e00640 1 -- 172.21.15.46:0/3915417007 <== osd.3 v2:172.21.15.46:6824/1647449001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f766c070050 con 0x7f766c06d540 2024-04-25T08:53:18.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 --> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7690069080 con 0x7f766c06d540 2024-04-25T08:53:18.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7687e00640 1 -- 172.21.15.46:0/3915417007 <== osd.3 v2:172.21.15.46:6824/1647449001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7690069080 con 0x7f766c06d540 2024-04-25T08:53:18.146 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:53:18.147 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:53:18.147 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:53:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f766c06d540 msgr2=0x7f766c06f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.46:6824/1647449001,v1:172.21.15.46:6825/1647449001] conn(0x7f766c06d540 0x7f766c06f980 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f766c066f60 msgr2=0x7f766c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f766c066f60 0x7f766c069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f76800ae000 tx=0x7f76800a7070 comp rx=0 tx=0).stop 2024-04-25T08:53:18.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769015dce0 msgr2=0x7f7690141df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 --2- 172.21.15.46:0/3915417007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f769015dce0 0x7f7690141df0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f768800c430 tx=0x7f768800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f768fe00640 1 -- 172.21.15.46:0/3915417007 reap_dead start 2024-04-25T08:53:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 shutdown_connections 2024-04-25T08:53:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 >> 172.21.15.46:0/3915417007 conn(0x7f7690071ae0 msgr2=0x7f76900b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 shutdown_connections 2024-04-25T08:53:18.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.153+0000 7f7696200640 1 -- 172.21.15.46:0/3915417007 wait complete. 2024-04-25T08:53:18.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T08:53:18.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2ace00640 1 Processor -- start 2024-04-25T08:53:18.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2ace00640 1 -- start start 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2ace00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa2a80aed10 0x7fa2a8165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2ace00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a80b35b0 con 0x7fa2a806f320 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2ace00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a806d5e0 con 0x7fa2a815a750 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2ace00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a815ab30 con 0x7fa2a80aed10 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa29d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa2a806f320 0x7fa2a8162200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53392/0 (socket says 172.21.15.46:53392) 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa29d400640 1 -- 172.21.15.46:0/2470969771 learned_addr learned my addr 172.21.15.46:0/2470969771 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a6800640 1 --2- 172.21.15.46:0/2470969771 >> v2:172.21.15.82:3300/0 conn(0x7fa2a80aed10 0x7fa2a8165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747061057 0 0) 0x7fa2a80b35b0 con 0x7fa2a806f320 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa28c003530 con 0x7fa2a806f320 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 795568831 0 0) 0x7fa28c003530 con 0x7fa2a806f320 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 >> v2:172.21.15.82:3300/0 conn(0x7fa2a80aed10 msgr2=0x7fa2a8165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 --2- 172.21.15.46:0/2470969771 >> v2:172.21.15.82:3300/0 conn(0x7fa2a80aed10 0x7fa2a8165930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 >> v1:172.21.15.82:6789/0 conn(0x7fa2a815a750 legacy=0x7fa2a806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80ce720 con 0x7fa2a806f320 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7fa290001a00 con 0x7fa2a806f320 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 >> v1:172.21.15.46:6789/0 conn(0x7fa2a806f320 legacy=0x7fa2a8162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa28c0039c0 0x7fa28c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa28c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c0051d0 0x7fa28c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c00bff0 con 0x7fa28c004810 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c00c1a0 con 0x7fa28c0051d0 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c00c3a0 con 0x7fa28c0039c0 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa29d400640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa28c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a5e00640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa28c0039c0 0x7fa28c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a6800640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c0051d0 0x7fa28c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa29d400640 1 -- 172.21.15.46:0/2470969771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa28c0039c0 msgr2=0x7fa28c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa29d400640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa28c0039c0 0x7fa28c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa29d400640 1 -- 172.21.15.46:0/2470969771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c0051d0 msgr2=0x7fa28c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa29d400640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa28c0051d0 0x7fa28c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa29d400640 1 -- 172.21.15.46:0/2470969771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c003530 con 0x7fa28c004810 2024-04-25T08:53:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.317+0000 7fa2a6800640 1 -- 172.21.15.46:0/2470969771 reap_dead start 2024-04-25T08:53:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa29d400640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa28c004c90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fa29000a7b0 tx=0x7fa29001d240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920473127c2f22d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29000a3d0 con 0x7fa28c004810 2024-04-25T08:53:18.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2900ad630 con 0x7fa28c004810 2024-04-25T08:53:18.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2a5400640 1 -- 172.21.15.46:0/2470969771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2900ada60 con 0x7fa28c004810 2024-04-25T08:53:18.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2ace00640 1 -- 172.21.15.46:0/2470969771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 msgr2=0x7fa28c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2ace00640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa28c004c90 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fa29000a7b0 tx=0x7fa29001d240 comp rx=0 tx=0).stop 2024-04-25T08:53:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2ace00640 1 -- 172.21.15.46:0/2470969771 shutdown_connections 2024-04-25T08:53:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2ace00640 1 --2- 172.21.15.46:0/2470969771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa28c004c90 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2ace00640 1 -- 172.21.15.46:0/2470969771 >> 172.21.15.46:0/2470969771 conn(0x7fa2a80b12a0 msgr2=0x7fa2a80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2ace00640 1 -- 172.21.15.46:0/2470969771 shutdown_connections 2024-04-25T08:53:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.321+0000 7fa2ace00640 1 -- 172.21.15.46:0/2470969771 wait complete. 2024-04-25T08:53:18.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 Processor -- start 2024-04-25T08:53:18.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 -- start start 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa2a8141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2a81514f0 0x7fa2a8152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2a815ab30 0x7fa2a815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a806d8c0 con 0x7fa28c004810 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80cece0 con 0x7fa2a815ab30 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80cf200 con 0x7fa2a81514f0 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa2a8141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa2a8141ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41140/0 (socket says 172.21.15.46:41140) 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a5e00640 1 -- 172.21.15.46:0/497021276 learned_addr learned my addr 172.21.15.46:0/497021276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a6800640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2a815ab30 0x7fa2a815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa29d400640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2a81514f0 0x7fa2a8152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a5e00640 1 -- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2a81514f0 msgr2=0x7fa2a8152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a5e00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2a81514f0 0x7fa2a8152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a5e00640 1 -- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2a815ab30 msgr2=0x7fa2a815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a5e00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2a815ab30 0x7fa2a815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a5e00640 1 -- 172.21.15.46:0/497021276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a81519d0 con 0x7fa28c004810 2024-04-25T08:53:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2a5e00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa2a8141ef0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fa298011470 tx=0x7fa298011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa29ea00640 1 -- 172.21.15.46:0/497021276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298019020 con 0x7fa28c004810 2024-04-25T08:53:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa29ea00640 1 -- 172.21.15.46:0/497021276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa298015820 con 0x7fa28c004810 2024-04-25T08:53:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a8151cb0 con 0x7fa28c004810 2024-04-25T08:53:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa29ea00640 1 -- 172.21.15.46:0/497021276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298010040 con 0x7fa28c004810 2024-04-25T08:53:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a8152270 con 0x7fa28c004810 2024-04-25T08:53:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.325+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a8069080 con 0x7fa28c004810 2024-04-25T08:53:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.329+0000 7fa29ea00640 1 -- 172.21.15.46:0/497021276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa29801c020 con 0x7fa28c004810 2024-04-25T08:53:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.329+0000 7fa29ea00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa284066f60 0x7fa284069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.329+0000 7fa29ea00640 1 -- 172.21.15.46:0/497021276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa298087e00 con 0x7fa28c004810 2024-04-25T08:53:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.329+0000 7fa29d400640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa284066f60 0x7fa284069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.329+0000 7fa29d400640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa284066f60 0x7fa284069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa2900ad000 tx=0x7fa2900a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.333+0000 7fa29ea00640 1 -- 172.21.15.46:0/497021276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa298023020 con 0x7fa28c004810 2024-04-25T08:53:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.497+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2a8155310 con 0x7fa284066f60 2024-04-25T08:53:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.497+0000 7fa29ea00640 1 -- 172.21.15.46:0/497021276 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa2a8155310 con 0x7fa284066f60 2024-04-25T08:53:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T08:53:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa284066f60 msgr2=0x7fa284069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa284066f60 0x7fa284069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa2900ad000 tx=0x7fa2900a7000 comp rx=0 tx=0).stop 2024-04-25T08:53:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 msgr2=0x7fa2a8141ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa2a8141ef0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fa298011470 tx=0x7fa298011930 comp rx=0 tx=0).stop 2024-04-25T08:53:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 shutdown_connections 2024-04-25T08:53:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fa284066f60 0x7fa284069420 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2a815ab30 0x7fa2a815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fa2a81514f0 0x7fa2a8152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 --2- 172.21.15.46:0/497021276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa28c004810 0x7fa2a8141ef0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 >> 172.21.15.46:0/497021276 conn(0x7fa2a80b12a0 msgr2=0x7fa2a80b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 shutdown_connections 2024-04-25T08:53:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.501+0000 7fa2ace00640 1 -- 172.21.15.46:0/497021276 wait complete. 2024-04-25T08:53:18.512 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T08:53:18.512 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:53:18.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b8cc00640 1 Processor -- start 2024-04-25T08:53:18.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b8cc00640 1 -- start start 2024-04-25T08:53:18.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b8cc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6b880cd5b0 0x7f6b8806c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b8cc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b880d0160 con 0x7f6b88099a00 2024-04-25T08:53:18.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b8cc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b880d0440 con 0x7f6b880d1200 2024-04-25T08:53:18.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b8cc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b880d0720 con 0x7f6b880cd5b0 2024-04-25T08:53:18.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b86800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6b88099a00 0x7f6b8815d340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53404/0 (socket says 172.21.15.46:53404) 2024-04-25T08:53:18.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b86800640 1 -- 172.21.15.46:0/2318467407 learned_addr learned my addr 172.21.15.46:0/2318467407 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:18.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85e00640 1 --2- 172.21.15.46:0/2318467407 >> v2:172.21.15.82:3300/0 conn(0x7f6b880cd5b0 0x7f6b8806c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476641740 0 0) 0x7f6b880d0160 con 0x7f6b88099a00 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b6c003530 con 0x7f6b88099a00 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3083593172 0 0) 0x7f6b6c003530 con 0x7f6b88099a00 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 >> v2:172.21.15.82:3300/0 conn(0x7f6b880cd5b0 msgr2=0x7f6b8806c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 --2- 172.21.15.46:0/2318467407 >> v2:172.21.15.82:3300/0 conn(0x7f6b880cd5b0 0x7f6b8806c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 >> v1:172.21.15.82:6789/0 conn(0x7f6b880d1200 legacy=0x7f6b880cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b880d0a00 con 0x7f6b88099a00 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f6b80001a00 con 0x7f6b88099a00 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 >> v1:172.21.15.46:6789/0 conn(0x7f6b88099a00 legacy=0x7f6b8815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c003a10 0x7f6b6c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c004810 0x7f6b6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b6c0051d0 0x7f6b6c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b880d0720 con 0x7f6b6c004810 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c00c0b0 con 0x7f6b6c003a10 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c00c2b0 con 0x7f6b6c0051d0 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b7d400640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c004810 0x7f6b6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b85e00640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c003a10 0x7f6b6c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b86800640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b6c0051d0 0x7f6b6c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b7d400640 1 -- 172.21.15.46:0/2318467407 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b6c0051d0 msgr2=0x7f6b6c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b7d400640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b6c0051d0 0x7f6b6c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b7d400640 1 -- 172.21.15.46:0/2318467407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c003a10 msgr2=0x7f6b6c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b7d400640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c003a10 0x7f6b6c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b7d400640 1 -- 172.21.15.46:0/2318467407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c003530 con 0x7f6b6c004810 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.665+0000 7f6b86800640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b6c0051d0 0x7f6b6c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b86800640 1 -- 172.21.15.46:0/2318467407 reap_dead start 2024-04-25T08:53:18.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b7d400640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c004810 0x7f6b6c004c90 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f6b70001780 tx=0x7f6b7001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5385ea2dac7677 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b700013a0 con 0x7f6b6c004810 2024-04-25T08:53:18.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b700052f0 con 0x7f6b6c004810 2024-04-25T08:53:18.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b85400640 1 -- 172.21.15.46:0/2318467407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70003840 con 0x7f6b6c004810 2024-04-25T08:53:18.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b8cc00640 1 -- 172.21.15.46:0/2318467407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c004810 msgr2=0x7f6b6c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b8cc00640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c004810 0x7f6b6c004c90 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f6b70001780 tx=0x7f6b7001c400 comp rx=0 tx=0).stop 2024-04-25T08:53:18.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b8cc00640 1 -- 172.21.15.46:0/2318467407 shutdown_connections 2024-04-25T08:53:18.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b8cc00640 1 --2- 172.21.15.46:0/2318467407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b6c004810 0x7f6b6c004c90 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b8cc00640 1 -- 172.21.15.46:0/2318467407 >> 172.21.15.46:0/2318467407 conn(0x7f6b880b31c0 msgr2=0x7f6b880b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b8cc00640 1 -- 172.21.15.46:0/2318467407 shutdown_connections 2024-04-25T08:53:18.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.669+0000 7f6b8cc00640 1 -- 172.21.15.46:0/2318467407 wait complete. 2024-04-25T08:53:18.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 Processor -- start 2024-04-25T08:53:18.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 -- start start 2024-04-25T08:53:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b6c004810 0x7f6b8814ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b880b5950 0x7f6b880b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b8815ba90 0x7f6b8815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b880cdb10 con 0x7f6b8815ba90 2024-04-25T08:53:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b880cd990 con 0x7f6b880b5950 2024-04-25T08:53:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8806c0f0 con 0x7f6b6c004810 2024-04-25T08:53:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b8815ba90 0x7f6b8815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b86800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b8815ba90 0x7f6b8815bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41158/0 (socket says 172.21.15.46:41158) 2024-04-25T08:53:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b86800640 1 -- 172.21.15.46:0/3813245877 learned_addr learned my addr 172.21.15.46:0/3813245877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b85e00640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b6c004810 0x7f6b8814ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b7d400640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b880b5950 0x7f6b880b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b86800640 1 -- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b6c004810 msgr2=0x7f6b8814ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b86800640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f6b6c004810 0x7f6b8814ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b86800640 1 -- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b880b5950 msgr2=0x7f6b880b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b86800640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b880b5950 0x7f6b880b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b86800640 1 -- 172.21.15.46:0/3813245877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b880b64d0 con 0x7f6b8815ba90 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b86800640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b8815ba90 0x7f6b8815bf10 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f6b80001f90 tx=0x7f6b8000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b7ea00640 1 -- 172.21.15.46:0/3813245877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000eb90 con 0x7f6b8815ba90 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b7ea00640 1 -- 172.21.15.46:0/3813245877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b80018070 con 0x7f6b8815ba90 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b7ea00640 1 -- 172.21.15.46:0/3813245877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80013420 con 0x7f6b8815ba90 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b880b67b0 con 0x7f6b8815ba90 2024-04-25T08:53:18.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.673+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8814f230 con 0x7f6b8815ba90 2024-04-25T08:53:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6b880b69f0 con 0x7f6b8815ba90 2024-04-25T08:53:18.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b7ea00640 1 -- 172.21.15.46:0/3813245877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b800135c0 con 0x7f6b8815ba90 2024-04-25T08:53:18.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b7ea00640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b64066f60 0x7f6b64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b7ea00640 1 -- 172.21.15.46:0/3813245877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f6b800851d0 con 0x7f6b8815ba90 2024-04-25T08:53:18.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b7ea00640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7f6b6406d540 0x7f6b6406f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b7ea00640 1 -- 172.21.15.46:0/3813245877 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b80021ec0 con 0x7f6b6406d540 2024-04-25T08:53:18.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b7ea00640 1 -- 172.21.15.46:0/3813245877 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b8008c050 con 0x7f6b8815ba90 2024-04-25T08:53:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b85e00640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b64066f60 0x7f6b64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b7d400640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7f6b6406d540 0x7f6b6406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b85e00640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b64066f60 0x7f6b64069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009c60 tx=0x7f6b78009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.677+0000 7f6b7d400640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7f6b6406d540 0x7f6b6406f980 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.681+0000 7f6b7ea00640 1 -- 172.21.15.46:0/3813245877 <== osd.4 v2:172.21.15.82:6800/2293006647 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6b80021ec0 con 0x7f6b6406d540 2024-04-25T08:53:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 --> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6b88069080 con 0x7f6b6406d540 2024-04-25T08:53:18.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b7ea00640 1 -- 172.21.15.46:0/3813245877 <== osd.4 v2:172.21.15.82:6800/2293006647 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6b88069080 con 0x7f6b6406d540 2024-04-25T08:53:18.686 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:53:18.686 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:53:18.686 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7f6b6406d540 msgr2=0x7f6b6406f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6800/2293006647,v1:172.21.15.82:6801/2293006647] conn(0x7f6b6406d540 0x7f6b6406f980 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b64066f60 msgr2=0x7f6b64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f6b64066f60 0x7f6b64069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6b78009c60 tx=0x7f6b78009190 comp rx=0 tx=0).stop 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b8815ba90 msgr2=0x7f6b8815bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 --2- 172.21.15.46:0/3813245877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b8815ba90 0x7f6b8815bf10 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f6b80001f90 tx=0x7f6b8000e530 comp rx=0 tx=0).stop 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b86800640 1 -- 172.21.15.46:0/3813245877 reap_dead start 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 shutdown_connections 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 >> 172.21.15.46:0/3813245877 conn(0x7f6b880b31c0 msgr2=0x7f6b880b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 shutdown_connections 2024-04-25T08:53:18.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.693+0000 7f6b8cc00640 1 -- 172.21.15.46:0/3813245877 wait complete. 2024-04-25T08:53:18.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T08:53:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.853+0000 7fe068200640 1 Processor -- start 2024-04-25T08:53:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.853+0000 7fe068200640 1 -- start start 2024-04-25T08:53:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe068200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 0x7fe06015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe068200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0600cf520 0x7fe06015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe068200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06006ed70 con 0x7fe0600afd40 2024-04-25T08:53:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe068200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0600afb40 con 0x7fe0600cf520 2024-04-25T08:53:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe068200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600d13f0 con 0x7fe0600cf140 2024-04-25T08:53:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe064e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe0600afd40 0x7fe0600ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53412/0 (socket says 172.21.15.46:53412) 2024-04-25T08:53:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe064e00640 1 -- 172.21.15.46:0/1003650123 learned_addr learned my addr 172.21.15.46:0/1003650123 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe066200640 1 --2- 172.21.15.46:0/1003650123 >> v2:172.21.15.46:3300/0 conn(0x7fe0600cf520 0x7fe06015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe065800640 1 --2- 172.21.15.46:0/1003650123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 0x7fe06015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe05fe00640 1 -- 172.21.15.46:0/1003650123 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762207516 0 0) 0x7fe04c001120 con 0x7fe0600afd40 2024-04-25T08:53:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe05fe00640 1 -- 172.21.15.46:0/1003650123 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe048003530 con 0x7fe0600afd40 2024-04-25T08:53:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe066200640 1 -- 172.21.15.46:0/1003650123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 msgr2=0x7fe06015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe066200640 1 --2- 172.21.15.46:0/1003650123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 0x7fe06015d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe066200640 1 -- 172.21.15.46:0/1003650123 >> v1:172.21.15.46:6789/0 conn(0x7fe0600afd40 legacy=0x7fe0600ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.857+0000 7fe066200640 1 -- 172.21.15.46:0/1003650123 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe060162530 con 0x7fe0600cf520 2024-04-25T08:53:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe066200640 1 --2- 172.21.15.46:0/1003650123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 0x7fe06015dad0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fe058009d90 tx=0x7fe05801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79e632380868be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe05fe00640 1 -- 172.21.15.46:0/1003650123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0580b2070 con 0x7fe0600cf520 2024-04-25T08:53:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe05fe00640 1 -- 172.21.15.46:0/1003650123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0580a5740 con 0x7fe0600cf520 2024-04-25T08:53:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe05fe00640 1 -- 172.21.15.46:0/1003650123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0580a5af0 con 0x7fe0600cf520 2024-04-25T08:53:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 -- 172.21.15.46:0/1003650123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 msgr2=0x7fe06015dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 --2- 172.21.15.46:0/1003650123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 0x7fe06015dad0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fe058009d90 tx=0x7fe05801c850 comp rx=0 tx=0).stop 2024-04-25T08:53:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 -- 172.21.15.46:0/1003650123 shutdown_connections 2024-04-25T08:53:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 --2- 172.21.15.46:0/1003650123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 0x7fe06015dad0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 --2- 172.21.15.46:0/1003650123 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 0x7fe06015d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 -- 172.21.15.46:0/1003650123 >> 172.21.15.46:0/1003650123 conn(0x7fe0600728c0 msgr2=0x7fe060099a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 -- 172.21.15.46:0/1003650123 shutdown_connections 2024-04-25T08:53:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 -- 172.21.15.46:0/1003650123 wait complete. 2024-04-25T08:53:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 Processor -- start 2024-04-25T08:53:18.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 -- start start 2024-04-25T08:53:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0600afd40 0x7fe06015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 0x7fe0600b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 0x7fe06015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600af7f0 con 0x7fe0600cf520 2024-04-25T08:53:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe060162010 con 0x7fe0600afd40 2024-04-25T08:53:18.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.861+0000 7fe068200640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06006b600 con 0x7fe0600cf140 2024-04-25T08:53:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe066200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 0x7fe06015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe066200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 0x7fe06015e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41166/0 (socket says 172.21.15.46:41166) 2024-04-25T08:53:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe066200640 1 -- 172.21.15.46:0/789383377 learned_addr learned my addr 172.21.15.46:0/789383377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe064e00640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 0x7fe0600b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe065800640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0600afd40 0x7fe06015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe066200640 1 -- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 msgr2=0x7fe0600b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe066200640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 0x7fe0600b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe066200640 1 -- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0600afd40 msgr2=0x7fe06015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe066200640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0600afd40 0x7fe06015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe066200640 1 -- 172.21.15.46:0/789383377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe060158d00 con 0x7fe0600cf520 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe066200640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 0x7fe06015e9c0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fe05801af00 tx=0x7fe0580195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe05d600640 1 -- 172.21.15.46:0/789383377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0580b2050 con 0x7fe0600cf520 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe05d600640 1 -- 172.21.15.46:0/789383377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0580be070 con 0x7fe0600cf520 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe060157f40 con 0x7fe0600cf520 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe05d600640 1 -- 172.21.15.46:0/789383377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe058009450 con 0x7fe0600cf520 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe060158500 con 0x7fe0600cf520 2024-04-25T08:53:18.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06006ccb0 con 0x7fe0600cf520 2024-04-25T08:53:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.865+0000 7fe05d600640 1 -- 172.21.15.46:0/789383377 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0580ad870 con 0x7fe0600cf520 2024-04-25T08:53:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.869+0000 7fe05d600640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe040066f60 0x7fe040069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.869+0000 7fe05d600640 1 -- 172.21.15.46:0/789383377 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe05811fe50 con 0x7fe0600cf520 2024-04-25T08:53:18.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.869+0000 7fe065800640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe040066f60 0x7fe040069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.869+0000 7fe065800640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe040066f60 0x7fe040069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe060153e30 tx=0x7fe054008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:18.873+0000 7fe05d600640 1 -- 172.21.15.46:0/789383377 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe058016560 con 0x7fe0600cf520 2024-04-25T08:53:19.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.033+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe060072f90 con 0x7fe040066f60 2024-04-25T08:53:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe05d600640 1 -- 172.21.15.46:0/789383377 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe060072f90 con 0x7fe040066f60 2024-04-25T08:53:19.028 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T08:53:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe040066f60 msgr2=0x7fe040069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe040066f60 0x7fe040069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe060153e30 tx=0x7fe054008040 comp rx=0 tx=0).stop 2024-04-25T08:53:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 msgr2=0x7fe06015e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 0x7fe06015e9c0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fe05801af00 tx=0x7fe0580195f0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 shutdown_connections 2024-04-25T08:53:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe040066f60 0x7fe040069420 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0600cf520 0x7fe06015e9c0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe0600cf140 0x7fe0600b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 --2- 172.21.15.46:0/789383377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0600afd40 0x7fe06015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 >> 172.21.15.46:0/789383377 conn(0x7fe0600728c0 msgr2=0x7fe0600d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 shutdown_connections 2024-04-25T08:53:19.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.037+0000 7fe068200640 1 -- 172.21.15.46:0/789383377 wait complete. 2024-04-25T08:53:19.050 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T08:53:19.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:53:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.201+0000 7f1174400640 1 Processor -- start 2024-04-25T08:53:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.201+0000 7f1174400640 1 -- start start 2024-04-25T08:53:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1174400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f116c0d11e0 0x7f116c06c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1174400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f116c099a00 0x7f116c099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1174400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c15e910 con 0x7f116c161720 2024-04-25T08:53:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1174400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c0712b0 con 0x7f116c099a00 2024-04-25T08:53:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1174400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c06fe10 con 0x7f116c0d11e0 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f116c099a00 0x7f116c099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f116c099a00 0x7f116c099e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41182/0 (socket says 172.21.15.46:41182) 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171000640 1 -- 172.21.15.46:0/1038184948 learned_addr learned my addr 172.21.15.46:0/1038184948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171a00640 1 --2- 172.21.15.46:0/1038184948 >> v2:172.21.15.82:3300/0 conn(0x7f116c0d11e0 0x7f116c06c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171000640 1 -- 172.21.15.46:0/1038184948 >> v2:172.21.15.82:3300/0 conn(0x7f116c0d11e0 msgr2=0x7f116c06c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171000640 1 --2- 172.21.15.46:0/1038184948 >> v2:172.21.15.82:3300/0 conn(0x7f116c0d11e0 0x7f116c06c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171000640 1 -- 172.21.15.46:0/1038184948 >> v1:172.21.15.82:6789/0 conn(0x7f116c161720 legacy=0x7f116c161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171000640 1 -- 172.21.15.46:0/1038184948 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c15ebf0 con 0x7f116c099a00 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171a00640 1 --2- 172.21.15.46:0/1038184948 >> v2:172.21.15.82:3300/0 conn(0x7f116c0d11e0 0x7f116c06c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f1171000640 1 --2- 172.21.15.46:0/1038184948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c099a00 0x7f116c099e80 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f116c072ab0 tx=0x7f115801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d901ca4d250d91fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f116be00640 1 -- 172.21.15.46:0/1038184948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580b0070 con 0x7f116c099a00 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f116be00640 1 -- 172.21.15.46:0/1038184948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11580ab600 con 0x7f116c099a00 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.205+0000 7f116be00640 1 -- 172.21.15.46:0/1038184948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580ab9d0 con 0x7f116c099a00 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 -- 172.21.15.46:0/1038184948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c099a00 msgr2=0x7f116c099e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 --2- 172.21.15.46:0/1038184948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c099a00 0x7f116c099e80 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f116c072ab0 tx=0x7f115801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 -- 172.21.15.46:0/1038184948 shutdown_connections 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 --2- 172.21.15.46:0/1038184948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c099a00 0x7f116c099e80 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 --2- 172.21.15.46:0/1038184948 >> v2:172.21.15.82:3300/0 conn(0x7f116c0d11e0 0x7f116c06c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 -- 172.21.15.46:0/1038184948 >> 172.21.15.46:0/1038184948 conn(0x7f116c0d16c0 msgr2=0x7f116c0d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 -- 172.21.15.46:0/1038184948 shutdown_connections 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 -- 172.21.15.46:0/1038184948 wait complete. 2024-04-25T08:53:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 Processor -- start 2024-04-25T08:53:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 -- start start 2024-04-25T08:53:19.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f116c099a00 0x7f116c0c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f116c0d11e0 0x7f116c0c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c161720 0x7f116c1f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c15dcc0 con 0x7f116c161720 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c15db40 con 0x7f116c099a00 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1174400640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c15de40 con 0x7f116c0d11e0 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1172400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c161720 0x7f116c1f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1171a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f116c099a00 0x7f116c0c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1172400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c161720 0x7f116c1f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41188/0 (socket says 172.21.15.46:41188) 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1172400640 1 -- 172.21.15.46:0/1284070428 learned_addr learned my addr 172.21.15.46:0/1284070428 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.209+0000 7f1171000640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f116c0d11e0 0x7f116c0c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1172400640 1 -- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f116c0d11e0 msgr2=0x7f116c0c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1172400640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f116c0d11e0 0x7f116c0c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1172400640 1 -- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f116c099a00 msgr2=0x7f116c0c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1172400640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f116c099a00 0x7f116c0c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1172400640 1 -- 172.21.15.46:0/1284070428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c1fa9b0 con 0x7f116c161720 2024-04-25T08:53:19.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1172400640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c161720 0x7f116c1f7430 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f116400b970 tx=0x7f116400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1169600640 1 -- 172.21.15.46:0/1284070428 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164019070 con 0x7f116c161720 2024-04-25T08:53:19.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1169600640 1 -- 172.21.15.46:0/1284070428 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116400cb40 con 0x7f116c161720 2024-04-25T08:53:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1169600640 1 -- 172.21.15.46:0/1284070428 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164014440 con 0x7f116c161720 2024-04-25T08:53:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116c1fac90 con 0x7f116c161720 2024-04-25T08:53:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c1f6f10 con 0x7f116c161720 2024-04-25T08:53:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f116c0c8060 con 0x7f116c161720 2024-04-25T08:53:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1169600640 1 -- 172.21.15.46:0/1284070428 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f116401c070 con 0x7f116c161720 2024-04-25T08:53:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1169600640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f114c066f90 0x7f114c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1169600640 1 -- 172.21.15.46:0/1284070428 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f1164084b20 con 0x7f116c161720 2024-04-25T08:53:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1169600640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f114c06d570 0x7f114c06f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1171a00640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f114c066f90 0x7f114c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1169600640 1 -- 172.21.15.46:0/1284070428 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1164016ec0 con 0x7f114c06d570 2024-04-25T08:53:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.213+0000 7f1169600640 1 -- 172.21.15.46:0/1284070428 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11640892a0 con 0x7f116c161720 2024-04-25T08:53:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.217+0000 7f1171000640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f114c06d570 0x7f114c06f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.217+0000 7f1171a00640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f114c066f90 0x7f114c069450 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f116c06ccd0 tx=0x7f1160008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.217+0000 7f1171000640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f114c06d570 0x7f114c06f9b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.217+0000 7f1169600640 1 -- 172.21.15.46:0/1284070428 <== osd.5 v2:172.21.15.82:6816/541310136 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1164016ec0 con 0x7f114c06d570 2024-04-25T08:53:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.229+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 --> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f116c069080 con 0x7f114c06d570 2024-04-25T08:53:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.229+0000 7f1169600640 1 -- 172.21.15.46:0/1284070428 <== osd.5 v2:172.21.15.82:6816/541310136 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f116c069080 con 0x7f114c06d570 2024-04-25T08:53:19.223 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:53:19.223 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:53:19.223 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:53:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f114c06d570 msgr2=0x7f114c06f9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6816/541310136,v1:172.21.15.82:6817/541310136] conn(0x7f114c06d570 0x7f114c06f9b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f114c066f90 msgr2=0x7f114c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f114c066f90 0x7f114c069450 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f116c06ccd0 tx=0x7f1160008040 comp rx=0 tx=0).stop 2024-04-25T08:53:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c161720 msgr2=0x7f116c1f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 --2- 172.21.15.46:0/1284070428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f116c161720 0x7f116c1f7430 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f116400b970 tx=0x7f116400be30 comp rx=0 tx=0).stop 2024-04-25T08:53:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1172400640 1 -- 172.21.15.46:0/1284070428 reap_dead start 2024-04-25T08:53:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 shutdown_connections 2024-04-25T08:53:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 >> 172.21.15.46:0/1284070428 conn(0x7f116c0d16c0 msgr2=0x7f116c15ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 shutdown_connections 2024-04-25T08:53:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.233+0000 7f1174400640 1 -- 172.21.15.46:0/1284070428 wait complete. 2024-04-25T08:53:19.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T08:53:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.389+0000 7f1e20800640 1 Processor -- start 2024-04-25T08:53:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.389+0000 7f1e20800640 1 -- start start 2024-04-25T08:53:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e20800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e1815a750 0x7f1e1806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e20800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e180cc9e0 0x7f1e1806f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e20800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e1806d5e0 con 0x7f1e180aed10 2024-04-25T08:53:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e20800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e1815ab30 con 0x7f1e180cc9e0 2024-04-25T08:53:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e20800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e180ce4f0 con 0x7f1e1815a750 2024-04-25T08:53:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1e180aed10 0x7f1e180b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53418/0 (socket says 172.21.15.46:53418) 2024-04-25T08:53:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1d400640 1 -- 172.21.15.46:0/3006594244 learned_addr learned my addr 172.21.15.46:0/3006594244 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1e800640 1 --2- 172.21.15.46:0/3006594244 >> v2:172.21.15.82:3300/0 conn(0x7f1e180cc9e0 0x7f1e1806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1de00640 1 --2- 172.21.15.46:0/3006594244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e1815a750 0x7f1e1806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1ca00640 1 -- 172.21.15.46:0/3006594244 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771188292 0 0) 0x7f1e1806d5e0 con 0x7f1e180aed10 2024-04-25T08:53:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1ca00640 1 -- 172.21.15.46:0/3006594244 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e00003530 con 0x7f1e180aed10 2024-04-25T08:53:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1de00640 1 -- 172.21.15.46:0/3006594244 >> v2:172.21.15.82:3300/0 conn(0x7f1e180cc9e0 msgr2=0x7f1e1806f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1de00640 1 --2- 172.21.15.46:0/3006594244 >> v2:172.21.15.82:3300/0 conn(0x7f1e180cc9e0 0x7f1e1806f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1de00640 1 -- 172.21.15.46:0/3006594244 >> v1:172.21.15.46:6789/0 conn(0x7f1e180aed10 legacy=0x7f1e180b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1de00640 1 -- 172.21.15.46:0/3006594244 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e180ce720 con 0x7f1e1815a750 2024-04-25T08:53:19.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.393+0000 7f1e1e800640 1 --2- 172.21.15.46:0/3006594244 >> v2:172.21.15.82:3300/0 conn(0x7f1e180cc9e0 0x7f1e1806f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:19.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e1de00640 1 --2- 172.21.15.46:0/3006594244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e1815a750 0x7f1e1806ec10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0097c0 tx=0x7f1e0c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0caad23513dc52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e1ca00640 1 -- 172.21.15.46:0/3006594244 <== mon.2 v2:172.21.15.82:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0b2070 con 0x7f1e1815a750 2024-04-25T08:53:19.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e1ca00640 1 -- 172.21.15.46:0/3006594244 <== mon.2 v2:172.21.15.82:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c0ad640 con 0x7f1e1815a750 2024-04-25T08:53:19.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e1ca00640 1 -- 172.21.15.46:0/3006594244 <== mon.2 v2:172.21.15.82:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0ada10 con 0x7f1e1815a750 2024-04-25T08:53:19.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 -- 172.21.15.46:0/3006594244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e1815a750 msgr2=0x7f1e1806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 --2- 172.21.15.46:0/3006594244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e1815a750 0x7f1e1806ec10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0097c0 tx=0x7f1e0c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 -- 172.21.15.46:0/3006594244 shutdown_connections 2024-04-25T08:53:19.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 --2- 172.21.15.46:0/3006594244 >> v2:172.21.15.82:3300/0 conn(0x7f1e180cc9e0 0x7f1e1806f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 --2- 172.21.15.46:0/3006594244 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e1815a750 0x7f1e1806ec10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 -- 172.21.15.46:0/3006594244 >> 172.21.15.46:0/3006594244 conn(0x7f1e180b12a0 msgr2=0x7f1e180b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 -- 172.21.15.46:0/3006594244 shutdown_connections 2024-04-25T08:53:19.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 -- 172.21.15.46:0/3006594244 wait complete. 2024-04-25T08:53:19.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 Processor -- start 2024-04-25T08:53:19.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.397+0000 7f1e20800640 1 -- start start 2024-04-25T08:53:19.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e20800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e180aed10 0x7f1e1814c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e20800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e180cc9e0 0x7f1e181f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e20800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1815a750 0x7f1e181f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e20800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1806db00 con 0x7f1e180cc9e0 2024-04-25T08:53:19.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e20800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1806d270 con 0x7f1e1815a750 2024-04-25T08:53:19.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e20800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1806dc80 con 0x7f1e180aed10 2024-04-25T08:53:19.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e180cc9e0 0x7f1e181f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e180cc9e0 0x7f1e181f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41194/0 (socket says 172.21.15.46:41194) 2024-04-25T08:53:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1d400640 1 -- 172.21.15.46:0/768084449 learned_addr learned my addr 172.21.15.46:0/768084449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1e800640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1815a750 0x7f1e181f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1de00640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e180aed10 0x7f1e1814c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1d400640 1 -- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e180aed10 msgr2=0x7f1e1814c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1d400640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e180aed10 0x7f1e1814c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1d400640 1 -- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1815a750 msgr2=0x7f1e181f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1d400640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1815a750 0x7f1e181f5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1d400640 1 -- 172.21.15.46:0/768084449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1814af50 con 0x7f1e180cc9e0 2024-04-25T08:53:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1de00640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e180aed10 0x7f1e1814c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e1d400640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e180cc9e0 0x7f1e181f5870 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c450 tx=0x7f1e0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e12000640 1 -- 172.21.15.46:0/768084449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08017070 con 0x7f1e180cc9e0 2024-04-25T08:53:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e12000640 1 -- 172.21.15.46:0/768084449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e08005cf0 con 0x7f1e180cc9e0 2024-04-25T08:53:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e12000640 1 -- 172.21.15.46:0/768084449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08007290 con 0x7f1e180cc9e0 2024-04-25T08:53:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1814a710 con 0x7f1e180cc9e0 2024-04-25T08:53:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1814edd0 con 0x7f1e180cc9e0 2024-04-25T08:53:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e18069080 con 0x7f1e180cc9e0 2024-04-25T08:53:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.401+0000 7f1e12000640 1 -- 172.21.15.46:0/768084449 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e0801a070 con 0x7f1e180cc9e0 2024-04-25T08:53:19.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.405+0000 7f1e12000640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dfc066f60 0x7f1dfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.405+0000 7f1e12000640 1 -- 172.21.15.46:0/768084449 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f1e080843b0 con 0x7f1e180cc9e0 2024-04-25T08:53:19.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.405+0000 7f1e1de00640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dfc066f60 0x7f1dfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.405+0000 7f1e1de00640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dfc066f60 0x7f1dfc069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0af000 tx=0x7f1e0c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.409+0000 7f1e12000640 1 -- 172.21.15.46:0/768084449 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e1814a710 con 0x7f1e180cc9e0 2024-04-25T08:53:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.573+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e180af8d0 con 0x7f1dfc066f60 2024-04-25T08:53:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.573+0000 7f1e12000640 1 -- 172.21.15.46:0/768084449 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1e180af8d0 con 0x7f1dfc066f60 2024-04-25T08:53:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T08:53:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dfc066f60 msgr2=0x7f1dfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dfc066f60 0x7f1dfc069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0af000 tx=0x7f1e0c0a7070 comp rx=0 tx=0).stop 2024-04-25T08:53:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e180cc9e0 msgr2=0x7f1e181f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e180cc9e0 0x7f1e181f5870 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f1e0800c450 tx=0x7f1e0800c910 comp rx=0 tx=0).stop 2024-04-25T08:53:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 shutdown_connections 2024-04-25T08:53:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f1dfc066f60 0x7f1dfc069420 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e1815a750 0x7f1e181f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e180cc9e0 0x7f1e181f5870 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 --2- 172.21.15.46:0/768084449 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f1e180aed10 0x7f1e1814c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 >> 172.21.15.46:0/768084449 conn(0x7f1e180b12a0 msgr2=0x7f1e18071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 shutdown_connections 2024-04-25T08:53:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.577+0000 7f1e20800640 1 -- 172.21.15.46:0/768084449 wait complete. 2024-04-25T08:53:19.590 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T08:53:19.590 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:53:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.741+0000 7fd91fe00640 1 Processor -- start 2024-04-25T08:53:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.741+0000 7fd91fe00640 1 -- start start 2024-04-25T08:53:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918140ee0 0x7fd91806e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9180cebe0 0x7fd91806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9180b2e60 con 0x7fd918141580 2024-04-25T08:53:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9181412c0 con 0x7fd9180cebe0 2024-04-25T08:53:19.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180b30d0 con 0x7fd918140ee0 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91ca00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd918141580 0x7fd9180ce4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53424/0 (socket says 172.21.15.46:53424) 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91ca00640 1 -- 172.21.15.46:0/3092613293 learned_addr learned my addr 172.21.15.46:0/3092613293 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91de00640 1 --2- 172.21.15.46:0/3092613293 >> v2:172.21.15.46:3300/0 conn(0x7fd9180cebe0 0x7fd91806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91d400640 1 --2- 172.21.15.46:0/3092613293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918140ee0 0x7fd91806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91de00640 1 -- 172.21.15.46:0/3092613293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918140ee0 msgr2=0x7fd91806e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91de00640 1 --2- 172.21.15.46:0/3092613293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918140ee0 0x7fd91806e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91de00640 1 -- 172.21.15.46:0/3092613293 >> v1:172.21.15.46:6789/0 conn(0x7fd918141580 legacy=0x7fd9180ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd917e00640 1 -- 172.21.15.46:0/3092613293 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671949565 0 0) 0x7fd9180b2e60 con 0x7fd918141580 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91de00640 1 -- 172.21.15.46:0/3092613293 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180b33b0 con 0x7fd9180cebe0 2024-04-25T08:53:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.745+0000 7fd91d400640 1 --2- 172.21.15.46:0/3092613293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918140ee0 0x7fd91806e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:19.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91de00640 1 --2- 172.21.15.46:0/3092613293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9180cebe0 0x7fd91806e850 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd910009d90 tx=0x7fd91001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c79c428ca94702f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd917e00640 1 -- 172.21.15.46:0/3092613293 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100b2070 con 0x7fd9180cebe0 2024-04-25T08:53:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd917e00640 1 -- 172.21.15.46:0/3092613293 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100a5a50 con 0x7fd9180cebe0 2024-04-25T08:53:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd917e00640 1 -- 172.21.15.46:0/3092613293 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100a5e20 con 0x7fd9180cebe0 2024-04-25T08:53:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 -- 172.21.15.46:0/3092613293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9180cebe0 msgr2=0x7fd91806e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 --2- 172.21.15.46:0/3092613293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9180cebe0 0x7fd91806e850 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd910009d90 tx=0x7fd91001c970 comp rx=0 tx=0).stop 2024-04-25T08:53:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 -- 172.21.15.46:0/3092613293 shutdown_connections 2024-04-25T08:53:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 --2- 172.21.15.46:0/3092613293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9180cebe0 0x7fd91806e850 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 --2- 172.21.15.46:0/3092613293 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918140ee0 0x7fd91806e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 -- 172.21.15.46:0/3092613293 >> 172.21.15.46:0/3092613293 conn(0x7fd918067050 msgr2=0x7fd91806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 -- 172.21.15.46:0/3092613293 shutdown_connections 2024-04-25T08:53:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 -- 172.21.15.46:0/3092613293 wait complete. 2024-04-25T08:53:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 Processor -- start 2024-04-25T08:53:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 -- start start 2024-04-25T08:53:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9180cebe0 0x7fd9181f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd918140ee0 0x7fd91813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918141580 0x7fd91813d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180b4790 con 0x7fd918140ee0 2024-04-25T08:53:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180b4610 con 0x7fd9180cebe0 2024-04-25T08:53:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.749+0000 7fd91fe00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9180b4910 con 0x7fd918141580 2024-04-25T08:53:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd918140ee0 0x7fd91813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd918140ee0 0x7fd91813cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41210/0 (socket says 172.21.15.46:41210) 2024-04-25T08:53:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91ca00640 1 -- 172.21.15.46:0/1319937581 learned_addr learned my addr 172.21.15.46:0/1319937581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91d400640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9180cebe0 0x7fd9181f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91de00640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918141580 0x7fd91813d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91ca00640 1 -- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918141580 msgr2=0x7fd91813d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91ca00640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fd918141580 0x7fd91813d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91ca00640 1 -- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9180cebe0 msgr2=0x7fd9181f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91ca00640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9180cebe0 0x7fd9181f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91ca00640 1 -- 172.21.15.46:0/1319937581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9181f6d80 con 0x7fd918140ee0 2024-04-25T08:53:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91ca00640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd918140ee0 0x7fd91813cb50 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd90400c480 tx=0x7fd90400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd915600640 1 -- 172.21.15.46:0/1319937581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904018070 con 0x7fd918140ee0 2024-04-25T08:53:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd915600640 1 -- 172.21.15.46:0/1319937581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9040074e0 con 0x7fd918140ee0 2024-04-25T08:53:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91813baf0 con 0x7fd918140ee0 2024-04-25T08:53:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd915600640 1 -- 172.21.15.46:0/1319937581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9040049f0 con 0x7fd918140ee0 2024-04-25T08:53:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91813c0b0 con 0x7fd918140ee0 2024-04-25T08:53:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd91813c700 con 0x7fd918140ee0 2024-04-25T08:53:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.753+0000 7fd915600640 1 -- 172.21.15.46:0/1319937581 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd90401b070 con 0x7fd918140ee0 2024-04-25T08:53:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd915600640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f8066f60 0x7fd8f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd915600640 1 -- 172.21.15.46:0/1319937581 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fd9040843f0 con 0x7fd918140ee0 2024-04-25T08:53:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd915600640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7fd8f806d540 0x7fd8f806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd915600640 1 -- 172.21.15.46:0/1319937581 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd8f8070050 con 0x7fd8f806d540 2024-04-25T08:53:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd915600640 1 -- 172.21.15.46:0/1319937581 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd904084880 con 0x7fd918140ee0 2024-04-25T08:53:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd91d400640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f8066f60 0x7fd8f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd91de00640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7fd8f806d540 0x7fd8f806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd91d400640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f8066f60 0x7fd8f8069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd91815d1f0 tx=0x7fd90c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd91de00640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7fd8f806d540 0x7fd8f806f980 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.757+0000 7fd915600640 1 -- 172.21.15.46:0/1319937581 <== osd.6 v2:172.21.15.82:6823/1178952145 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd8f8070050 con 0x7fd8f806d540 2024-04-25T08:53:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.769+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 --> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd918069080 con 0x7fd8f806d540 2024-04-25T08:53:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd915600640 1 -- 172.21.15.46:0/1319937581 <== osd.6 v2:172.21.15.82:6823/1178952145 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd918069080 con 0x7fd8f806d540 2024-04-25T08:53:19.763 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:53:19.763 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:53:19.764 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:53:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7fd8f806d540 msgr2=0x7fd8f806f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6823/1178952145,v1:172.21.15.82:6824/1178952145] conn(0x7fd8f806d540 0x7fd8f806f980 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f8066f60 msgr2=0x7fd8f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fd8f8066f60 0x7fd8f8069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd91815d1f0 tx=0x7fd90c008040 comp rx=0 tx=0).stop 2024-04-25T08:53:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd918140ee0 msgr2=0x7fd91813cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 --2- 172.21.15.46:0/1319937581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd918140ee0 0x7fd91813cb50 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fd90400c480 tx=0x7fd90400c940 comp rx=0 tx=0).stop 2024-04-25T08:53:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91de00640 1 -- 172.21.15.46:0/1319937581 reap_dead start 2024-04-25T08:53:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 shutdown_connections 2024-04-25T08:53:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 >> 172.21.15.46:0/1319937581 conn(0x7fd918067050 msgr2=0x7fd9180711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 shutdown_connections 2024-04-25T08:53:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.773+0000 7fd91fe00640 1 -- 172.21.15.46:0/1319937581 wait complete. 2024-04-25T08:53:19.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T08:53:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3870a00640 1 Processor -- start 2024-04-25T08:53:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3870a00640 1 -- start start 2024-04-25T08:53:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3870a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c157b00 0x7f386c0b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3870a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f386c0cc150 0x7f386c0cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3870a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f386c0ce250 0x7f386c0ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3870a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0b2eb0 con 0x7f386c0cc150 2024-04-25T08:53:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3870a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f386c0b25a0 con 0x7f386c0ce250 2024-04-25T08:53:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3870a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c0cc6f0 con 0x7f386c157b00 2024-04-25T08:53:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3869400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f386c0cc150 0x7f386c0cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3869400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f386c0cc150 0x7f386c0cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41214/0 (socket says 172.21.15.46:41214) 2024-04-25T08:53:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3869400640 1 -- 172.21.15.46:0/2574689296 learned_addr learned my addr 172.21.15.46:0/2574689296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3869e00640 1 --2- 172.21.15.46:0/2574689296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c157b00 0x7f386c0b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f386a800640 1 --2- 172.21.15.46:0/2574689296 >> v2:172.21.15.82:3300/0 conn(0x7f386c0ce250 0x7f386c0ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3869400640 1 -- 172.21.15.46:0/2574689296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c157b00 msgr2=0x7f386c0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3869400640 1 --2- 172.21.15.46:0/2574689296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c157b00 0x7f386c0b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3869400640 1 -- 172.21.15.46:0/2574689296 >> v2:172.21.15.82:3300/0 conn(0x7f386c0ce250 msgr2=0x7f386c0ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3869400640 1 --2- 172.21.15.46:0/2574689296 >> v2:172.21.15.82:3300/0 conn(0x7f386c0ce250 0x7f386c0ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.933+0000 7f3869400640 1 -- 172.21.15.46:0/2574689296 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c0b2860 con 0x7f386c0cc150 2024-04-25T08:53:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3869400640 1 --2- 172.21.15.46:0/2574689296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c0cc150 0x7f386c0cdd10 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f386c0af1c0 tx=0x7f385801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626f032942b69210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3868a00640 1 -- 172.21.15.46:0/2574689296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38580ab650 con 0x7f386c0cc150 2024-04-25T08:53:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3868a00640 1 -- 172.21.15.46:0/2574689296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38580b0070 con 0x7f386c0cc150 2024-04-25T08:53:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3868a00640 1 -- 172.21.15.46:0/2574689296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38580aba10 con 0x7f386c0cc150 2024-04-25T08:53:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 -- 172.21.15.46:0/2574689296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c0cc150 msgr2=0x7f386c0cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 --2- 172.21.15.46:0/2574689296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c0cc150 0x7f386c0cdd10 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f386c0af1c0 tx=0x7f385801c4f0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 -- 172.21.15.46:0/2574689296 shutdown_connections 2024-04-25T08:53:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 --2- 172.21.15.46:0/2574689296 >> v2:172.21.15.82:3300/0 conn(0x7f386c0ce250 0x7f386c0ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 --2- 172.21.15.46:0/2574689296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c0cc150 0x7f386c0cdd10 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 --2- 172.21.15.46:0/2574689296 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c157b00 0x7f386c0b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 -- 172.21.15.46:0/2574689296 >> 172.21.15.46:0/2574689296 conn(0x7f386c0b12a0 msgr2=0x7f386c0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 -- 172.21.15.46:0/2574689296 shutdown_connections 2024-04-25T08:53:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 -- 172.21.15.46:0/2574689296 wait complete. 2024-04-25T08:53:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.937+0000 7f3870a00640 1 Processor -- start 2024-04-25T08:53:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 -- start start 2024-04-25T08:53:19.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0cc150 0x7f386c14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c0ce250 0x7f386c0ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c157b00 0x7f386c0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c15a7e0 con 0x7f386c157b00 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386c15a660 con 0x7f386c0cc150 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386c15a960 con 0x7f386c0ce250 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f386a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c157b00 0x7f386c0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f386a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c157b00 0x7f386c0ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41222/0 (socket says 172.21.15.46:41222) 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f386a800640 1 -- 172.21.15.46:0/2219986974 learned_addr learned my addr 172.21.15.46:0/2219986974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3869e00640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0cc150 0x7f386c14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3869400640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c0ce250 0x7f386c0ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f386a800640 1 -- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c0ce250 msgr2=0x7f386c0ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f386a800640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c0ce250 0x7f386c0ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f386a800640 1 -- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0cc150 msgr2=0x7f386c14ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f386a800640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0cc150 0x7f386c14ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f386a800640 1 -- 172.21.15.46:0/2219986974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f386c154100 con 0x7f386c157b00 2024-04-25T08:53:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f386a800640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c157b00 0x7f386c0ae780 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f386400eed0 tx=0x7f386400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3862000640 1 -- 172.21.15.46:0/2219986974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864018020 con 0x7f386c157b00 2024-04-25T08:53:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3862000640 1 -- 172.21.15.46:0/2219986974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864014400 con 0x7f386c157b00 2024-04-25T08:53:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3862000640 1 -- 172.21.15.46:0/2219986974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38640147f0 con 0x7f386c157b00 2024-04-25T08:53:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386c14b270 con 0x7f386c157b00 2024-04-25T08:53:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386c1f6b40 con 0x7f386c157b00 2024-04-25T08:53:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.941+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f386c069080 con 0x7f386c157b00 2024-04-25T08:53:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.945+0000 7f3862000640 1 -- 172.21.15.46:0/2219986974 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f386401b020 con 0x7f386c157b00 2024-04-25T08:53:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.945+0000 7f3862000640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f384c066f60 0x7f384c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.945+0000 7f3862000640 1 -- 172.21.15.46:0/2219986974 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f38640854c0 con 0x7f386c157b00 2024-04-25T08:53:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.945+0000 7f3869e00640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f384c066f60 0x7f384c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.945+0000 7f3869e00640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f384c066f60 0x7f384c069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f386c0cc120 tx=0x7f385c009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:19.949+0000 7f3862000640 1 -- 172.21.15.46:0/2219986974 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3864022070 con 0x7f386c157b00 2024-04-25T08:53:20.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f386c0724a0 con 0x7f384c066f60 2024-04-25T08:53:20.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3862000640 1 -- 172.21.15.46:0/2219986974 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f386c0724a0 con 0x7f384c066f60 2024-04-25T08:53:20.104 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f384c066f60 msgr2=0x7f384c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f384c066f60 0x7f384c069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f386c0cc120 tx=0x7f385c009190 comp rx=0 tx=0).stop 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c157b00 msgr2=0x7f386c0ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c157b00 0x7f386c0ae780 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f386400eed0 tx=0x7f386400b480 comp rx=0 tx=0).stop 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 shutdown_connections 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f384c066f60 0x7f384c069420 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f386c157b00 0x7f386c0ae780 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f386c0ce250 0x7f386c0ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 --2- 172.21.15.46:0/2219986974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f386c0cc150 0x7f386c14ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 >> 172.21.15.46:0/2219986974 conn(0x7f386c0b12a0 msgr2=0x7f386c0b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 shutdown_connections 2024-04-25T08:53:20.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.113+0000 7f3870a00640 1 -- 172.21.15.46:0/2219986974 wait complete. 2024-04-25T08:53:20.125 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T08:53:20.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:53:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.277+0000 7f9060800640 1 Processor -- start 2024-04-25T08:53:20.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.277+0000 7f9060800640 1 -- start start 2024-04-25T08:53:20.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.277+0000 7f9060800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f905813e6b0 0x7f90580711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.277+0000 7f9060800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f905815c680 0x7f905806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.277+0000 7f9060800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905806d330 con 0x7f9058071720 2024-04-25T08:53:20.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.277+0000 7f9060800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905815cce0 con 0x7f905813e6b0 2024-04-25T08:53:20.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.277+0000 7f9060800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905806d610 con 0x7f905815c680 2024-04-25T08:53:20.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.277+0000 7f905de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f905813e6b0 0x7f90580711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f905813e6b0 0x7f90580711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41234/0 (socket says 172.21.15.46:41234) 2024-04-25T08:53:20.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905de00640 1 -- 172.21.15.46:0/1397556845 learned_addr learned my addr 172.21.15.46:0/1397556845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:20.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905de00640 1 -- 172.21.15.46:0/1397556845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f905815c680 msgr2=0x7f905806e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905de00640 1 --2- 172.21.15.46:0/1397556845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f905815c680 0x7f905806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905de00640 1 -- 172.21.15.46:0/1397556845 >> v1:172.21.15.82:6789/0 conn(0x7f9058071720 legacy=0x7f905815bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905de00640 1 -- 172.21.15.46:0/1397556845 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905806d8f0 con 0x7f905813e6b0 2024-04-25T08:53:20.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905de00640 1 --2- 172.21.15.46:0/1397556845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905813e6b0 0x7f90580711e0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f904c009870 tx=0x7f904c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27778a0d94657db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905ca00640 1 -- 172.21.15.46:0/1397556845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0b0070 con 0x7f905813e6b0 2024-04-25T08:53:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905ca00640 1 -- 172.21.15.46:0/1397556845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c0a4e50 con 0x7f905813e6b0 2024-04-25T08:53:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f905ca00640 1 -- 172.21.15.46:0/1397556845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0ab820 con 0x7f905813e6b0 2024-04-25T08:53:20.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f9060800640 1 -- 172.21.15.46:0/1397556845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905813e6b0 msgr2=0x7f90580711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.281+0000 7f9060800640 1 --2- 172.21.15.46:0/1397556845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905813e6b0 0x7f90580711e0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f904c009870 tx=0x7f904c01c400 comp rx=0 tx=0).stop 2024-04-25T08:53:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 -- 172.21.15.46:0/1397556845 shutdown_connections 2024-04-25T08:53:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 --2- 172.21.15.46:0/1397556845 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f905815c680 0x7f905806e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 --2- 172.21.15.46:0/1397556845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905813e6b0 0x7f90580711e0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 -- 172.21.15.46:0/1397556845 >> 172.21.15.46:0/1397556845 conn(0x7f9058067050 msgr2=0x7f905806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 -- 172.21.15.46:0/1397556845 shutdown_connections 2024-04-25T08:53:20.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 -- 172.21.15.46:0/1397556845 wait complete. 2024-04-25T08:53:20.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 Processor -- start 2024-04-25T08:53:20.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 -- start start 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9058071720 0x7f9058161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f905813e6b0 0x7f9058163e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905815c680 0x7f90581f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9058168590 con 0x7f905815c680 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9058168710 con 0x7f905813e6b0 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f9060800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9058168890 con 0x7f9058071720 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905815c680 0x7f90581f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905815c680 0x7f90581f70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41240/0 (socket says 172.21.15.46:41240) 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905e800640 1 -- 172.21.15.46:0/1756683039 learned_addr learned my addr 172.21.15.46:0/1756683039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905de00640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9058071720 0x7f9058161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905d400640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f905813e6b0 0x7f9058163e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905e800640 1 -- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9058071720 msgr2=0x7f9058161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905e800640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f9058071720 0x7f9058161790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905e800640 1 -- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f905813e6b0 msgr2=0x7f9058163e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905e800640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f905813e6b0 0x7f9058163e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905e800640 1 -- 172.21.15.46:0/1756683039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905806e340 con 0x7f905815c680 2024-04-25T08:53:20.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.285+0000 7f905e800640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905815c680 0x7f90581f70b0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f905400b970 tx=0x7f905400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9052000640 1 -- 172.21.15.46:0/1756683039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054019070 con 0x7f905815c680 2024-04-25T08:53:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9052000640 1 -- 172.21.15.46:0/1756683039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905400cb60 con 0x7f905815c680 2024-04-25T08:53:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9052000640 1 -- 172.21.15.46:0/1756683039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054014440 con 0x7f905815c680 2024-04-25T08:53:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90581606f0 con 0x7f905815c680 2024-04-25T08:53:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9058160cb0 con 0x7f905815c680 2024-04-25T08:53:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9058069080 con 0x7f905815c680 2024-04-25T08:53:20.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9052000640 1 -- 172.21.15.46:0/1756683039 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f905401c070 con 0x7f905815c680 2024-04-25T08:53:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9052000640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f903c066f60 0x7f903c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9052000640 1 -- 172.21.15.46:0/1756683039 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f90540852c0 con 0x7f905815c680 2024-04-25T08:53:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9052000640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f903c06d540 0x7f903c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f905de00640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f903c066f60 0x7f903c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9052000640 1 -- 172.21.15.46:0/1756683039 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9054009220 con 0x7f903c06d540 2024-04-25T08:53:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f9052000640 1 -- 172.21.15.46:0/1756683039 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9054085750 con 0x7f905815c680 2024-04-25T08:53:20.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.289+0000 7f905d400640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f903c06d540 0x7f903c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.293+0000 7f905de00640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f903c066f60 0x7f903c069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f904c0ad000 tx=0x7f904c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.293+0000 7f905d400640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f903c06d540 0x7f903c06f980 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.293+0000 7f9052000640 1 -- 172.21.15.46:0/1756683039 <== osd.7 v2:172.21.15.82:6808/1420997799 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9054009220 con 0x7f903c06d540 2024-04-25T08:53:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.305+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 --> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9058161cd0 con 0x7f903c06d540 2024-04-25T08:53:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.305+0000 7f9052000640 1 -- 172.21.15.46:0/1756683039 <== osd.7 v2:172.21.15.82:6808/1420997799 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9058161cd0 con 0x7f903c06d540 2024-04-25T08:53:20.299 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-04-25T08:53:20.299 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:53:20.299 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-04-25T08:53:20.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.305+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f903c06d540 msgr2=0x7f903c06f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.305+0000 7f9060800640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6808/1420997799,v1:172.21.15.82:6809/1420997799] conn(0x7f903c06d540 0x7f903c06f980 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.305+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f903c066f60 msgr2=0x7f903c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.305+0000 7f9060800640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f903c066f60 0x7f903c069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f904c0ad000 tx=0x7f904c0a6000 comp rx=0 tx=0).stop 2024-04-25T08:53:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.305+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905815c680 msgr2=0x7f90581f70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.305+0000 7f9060800640 1 --2- 172.21.15.46:0/1756683039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f905815c680 0x7f90581f70b0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f905400b970 tx=0x7f905400be30 comp rx=0 tx=0).stop 2024-04-25T08:53:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.305+0000 7f905e800640 1 -- 172.21.15.46:0/1756683039 reap_dead start 2024-04-25T08:53:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.309+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 shutdown_connections 2024-04-25T08:53:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.309+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 >> 172.21.15.46:0/1756683039 conn(0x7f9058067050 msgr2=0x7f905813da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.309+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 shutdown_connections 2024-04-25T08:53:20.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.309+0000 7f9060800640 1 -- 172.21.15.46:0/1756683039 wait complete. 2024-04-25T08:53:20.317 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T08:53:20.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58bd600640 1 Processor -- start 2024-04-25T08:53:20.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58bd600640 1 -- start start 2024-04-25T08:53:20.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58bd600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b80b29c0 0x7f58b815f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58bd600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b80b2da0 con 0x7f58b8099a00 2024-04-25T08:53:20.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58bd600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b81623a0 con 0x7f58b806c2b0 2024-04-25T08:53:20.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58bd600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8161b80 con 0x7f58b80b29c0 2024-04-25T08:53:20.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58bcc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f58b8099a00 0x7f58b815e120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53434/0 (socket says 172.21.15.46:53434) 2024-04-25T08:53:20.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58bcc00640 1 -- 172.21.15.46:0/3694032223 learned_addr learned my addr 172.21.15.46:0/3694032223 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:20.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b7e00640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b80b29c0 0x7f58b815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646388375 0 0) 0x7f58b80b2da0 con 0x7f58b8099a00 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f589c003530 con 0x7f58b8099a00 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 410484751 0 0) 0x7f589c003530 con 0x7f58b8099a00 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b80b29c0 msgr2=0x7f58b815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b80b29c0 0x7f58b815f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 >> v1:172.21.15.82:6789/0 conn(0x7f58b806c2b0 legacy=0x7f58b8161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b8161de0 con 0x7f58b8099a00 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1575119793 0 0) 0x7f58b0001a00 con 0x7f58b8099a00 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 >> v1:172.21.15.46:6789/0 conn(0x7f58b8099a00 legacy=0x7f58b815e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f589c0039f0 0x7f589c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f589c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b80b29c0 0x7f589c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8161b80 con 0x7f589c004810 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c00bab0 con 0x7f589c0039f0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c00bcb0 con 0x7f58b80b29c0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b7400640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f589c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b7e00640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f589c0039f0 0x7f589c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58bcc00640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b80b29c0 0x7f589c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b7400640 1 -- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b80b29c0 msgr2=0x7f589c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b7400640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b80b29c0 0x7f589c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b7400640 1 -- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f589c0039f0 msgr2=0x7f589c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b7400640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f589c0039f0 0x7f589c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.469+0000 7f58b7400640 1 -- 172.21.15.46:0/3694032223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c003530 con 0x7f589c004810 2024-04-25T08:53:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58b7400640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f589c004c90 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f58a4001750 tx=0x7f58a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2477e8cf66cbe18a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40013a0 con 0x7f589c004810 2024-04-25T08:53:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a40052f0 con 0x7f589c004810 2024-04-25T08:53:20.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58b6a00640 1 -- 172.21.15.46:0/3694032223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4003840 con 0x7f589c004810 2024-04-25T08:53:20.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58bd600640 1 -- 172.21.15.46:0/3694032223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 msgr2=0x7f589c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58bd600640 1 --2- 172.21.15.46:0/3694032223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f589c004c90 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f58a4001750 tx=0x7f58a401c3c0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58bcc00640 1 -- 172.21.15.46:0/3694032223 reap_dead start 2024-04-25T08:53:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58bd600640 1 -- 172.21.15.46:0/3694032223 shutdown_connections 2024-04-25T08:53:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58bd600640 1 -- 172.21.15.46:0/3694032223 >> 172.21.15.46:0/3694032223 conn(0x7f58b80b2f90 msgr2=0x7f58b80b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58bd600640 1 -- 172.21.15.46:0/3694032223 shutdown_connections 2024-04-25T08:53:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.473+0000 7f58bd600640 1 -- 172.21.15.46:0/3694032223 wait complete. 2024-04-25T08:53:20.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 Processor -- start 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 -- start start 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f58b809d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b814f880 0x7f58b80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58b809ea10 0x7f58b809b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80cfc90 con 0x7f589c004810 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80cf9b0 con 0x7f58b809ea10 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b815f250 con 0x7f58b814f880 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f58b809d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f58b809d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41262/0 (socket says 172.21.15.46:41262) 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7e00640 1 -- 172.21.15.46:0/3645330409 learned_addr learned my addr 172.21.15.46:0/3645330409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7400640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b814f880 0x7f58b80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7e00640 1 -- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b814f880 msgr2=0x7f58b80b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7e00640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b814f880 0x7f58b80b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7e00640 1 -- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58b809ea10 msgr2=0x7f58b809b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bcc00640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58b809ea10 0x7f58b809b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7e00640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58b809ea10 0x7f58b809b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7e00640 1 -- 172.21.15.46:0/3645330409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b814fd00 con 0x7f589c004810 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7400640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b814f880 0x7f58b80b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bcc00640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58b809ea10 0x7f58b809b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58b7e00640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f58b809d2b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f58a80119c0 tx=0x7f58a8011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58afe00640 1 -- 172.21.15.46:0/3645330409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a800fa90 con 0x7f589c004810 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58afe00640 1 -- 172.21.15.46:0/3645330409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a800fc30 con 0x7f589c004810 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b809d970 con 0x7f589c004810 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58afe00640 1 -- 172.21.15.46:0/3645330409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8014440 con 0x7f589c004810 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.477+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b809df30 con 0x7f589c004810 2024-04-25T08:53:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.481+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b8069010 con 0x7f589c004810 2024-04-25T08:53:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.481+0000 7f58afe00640 1 -- 172.21.15.46:0/3645330409 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58a8012040 con 0x7f589c004810 2024-04-25T08:53:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.481+0000 7f58afe00640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5898066f60 0x7f5898069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.481+0000 7f58afe00640 1 -- 172.21.15.46:0/3645330409 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f58a8085480 con 0x7f589c004810 2024-04-25T08:53:20.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.481+0000 7f58b7400640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5898066f60 0x7f5898069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.481+0000 7f58b7400640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5898066f60 0x7f5898069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f589c0057f0 tx=0x7f58a40011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.485+0000 7f58afe00640 1 -- 172.21.15.46:0/3645330409 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58a8019030 con 0x7f589c004810 2024-04-25T08:53:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.645+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f58b809a530 con 0x7f5898066f60 2024-04-25T08:53:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58afe00640 1 -- 172.21.15.46:0/3645330409 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f58b809a530 con 0x7f5898066f60 2024-04-25T08:53:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T08:53:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5898066f60 msgr2=0x7f5898069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5898066f60 0x7f5898069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f589c0057f0 tx=0x7f58a40011a0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 msgr2=0x7f58b809d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f58b809d2b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f58a80119c0 tx=0x7f58a8011e80 comp rx=0 tx=0).stop 2024-04-25T08:53:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 shutdown_connections 2024-04-25T08:53:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f5898066f60 0x7f5898069420 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58b809ea10 0x7f58b809b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f58b814f880 0x7f58b80b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 --2- 172.21.15.46:0/3645330409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f589c004810 0x7f58b809d2b0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 >> 172.21.15.46:0/3645330409 conn(0x7f58b80b2f90 msgr2=0x7f58b806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 shutdown_connections 2024-04-25T08:53:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.649+0000 7f58bd600640 1 -- 172.21.15.46:0/3645330409 wait complete. 2024-04-25T08:53:20.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:53:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4349e00640 1 Processor -- start 2024-04-25T08:53:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4349e00640 1 -- start start 2024-04-25T08:53:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4349e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f434415ef20 0x7f43440cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4349e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43440cd6d0 0x7f434409a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4349e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434415f300 con 0x7f434409ab50 2024-04-25T08:53:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4349e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43440cddb0 con 0x7f434415ef20 2024-04-25T08:53:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4349e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43440cc3d0 con 0x7f43440cd6d0 2024-04-25T08:53:20.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4342a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f434415ef20 0x7f43440cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4342a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f434415ef20 0x7f43440cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41276/0 (socket says 172.21.15.46:41276) 2024-04-25T08:53:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4342a00640 1 -- 172.21.15.46:0/2468736098 learned_addr learned my addr 172.21.15.46:0/2468736098 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f433a000640 1 --2- 172.21.15.46:0/2468736098 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43440cd6d0 0x7f434409a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4342a00640 1 -- 172.21.15.46:0/2468736098 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43440cd6d0 msgr2=0x7f434409a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4342a00640 1 --2- 172.21.15.46:0/2468736098 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43440cd6d0 0x7f434409a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4342a00640 1 -- 172.21.15.46:0/2468736098 >> v1:172.21.15.82:6789/0 conn(0x7f434409ab50 legacy=0x7f4344099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f4342a00640 1 -- 172.21.15.46:0/2468736098 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43440cc6b0 con 0x7f434415ef20 2024-04-25T08:53:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.813+0000 7f433a000640 1 --2- 172.21.15.46:0/2468736098 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43440cd6d0 0x7f434409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4342a00640 1 --2- 172.21.15.46:0/2468736098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f434415ef20 0x7f43440cd190 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f4334009870 tx=0x7f433401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ddd820bfba1b32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4342000640 1 -- 172.21.15.46:0/2468736098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43340aa610 con 0x7f434415ef20 2024-04-25T08:53:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4342000640 1 -- 172.21.15.46:0/2468736098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43340af070 con 0x7f434415ef20 2024-04-25T08:53:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4342000640 1 -- 172.21.15.46:0/2468736098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43340aa9d0 con 0x7f434415ef20 2024-04-25T08:53:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4349e00640 1 -- 172.21.15.46:0/2468736098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f434415ef20 msgr2=0x7f43440cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4349e00640 1 --2- 172.21.15.46:0/2468736098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f434415ef20 0x7f43440cd190 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f4334009870 tx=0x7f433401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4349e00640 1 -- 172.21.15.46:0/2468736098 shutdown_connections 2024-04-25T08:53:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4349e00640 1 --2- 172.21.15.46:0/2468736098 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f43440cd6d0 0x7f434409a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4349e00640 1 --2- 172.21.15.46:0/2468736098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f434415ef20 0x7f43440cd190 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4349e00640 1 -- 172.21.15.46:0/2468736098 >> 172.21.15.46:0/2468736098 conn(0x7f43440b3120 msgr2=0x7f43440b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4349e00640 1 -- 172.21.15.46:0/2468736098 shutdown_connections 2024-04-25T08:53:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.817+0000 7f4349e00640 1 -- 172.21.15.46:0/2468736098 wait complete. 2024-04-25T08:53:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 Processor -- start 2024-04-25T08:53:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 -- start start 2024-04-25T08:53:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f434409ab50 0x7f43440b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f434415ef20 0x7f434409e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4344153f00 0x7f43440b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43440c5b10 con 0x7f4344153f00 2024-04-25T08:53:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43440c5990 con 0x7f434409ab50 2024-04-25T08:53:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434409a110 con 0x7f434415ef20 2024-04-25T08:53:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4343400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4344153f00 0x7f43440b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4343400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4344153f00 0x7f43440b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41282/0 (socket says 172.21.15.46:41282) 2024-04-25T08:53:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4343400640 1 -- 172.21.15.46:0/2598814260 learned_addr learned my addr 172.21.15.46:0/2598814260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4342a00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f434409ab50 0x7f43440b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f433a000640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f434415ef20 0x7f434409e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4343400640 1 -- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f434415ef20 msgr2=0x7f434409e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4343400640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f434415ef20 0x7f434409e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4343400640 1 -- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f434409ab50 msgr2=0x7f43440b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4343400640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f434409ab50 0x7f43440b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4343400640 1 -- 172.21.15.46:0/2598814260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43440b64e0 con 0x7f4344153f00 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f433a000640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f434415ef20 0x7f434409e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4343400640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4344153f00 0x7f43440b1480 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f433c00c3c0 tx=0x7f433c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f433be00640 1 -- 172.21.15.46:0/2598814260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011790 con 0x7f4344153f00 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f433be00640 1 -- 172.21.15.46:0/2598814260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c017070 con 0x7f4344153f00 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43440b67c0 con 0x7f4344153f00 2024-04-25T08:53:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f433be00640 1 -- 172.21.15.46:0/2598814260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011b70 con 0x7f4344153f00 2024-04-25T08:53:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43440b61f0 con 0x7f4344153f00 2024-04-25T08:53:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.821+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4344069080 con 0x7f4344153f00 2024-04-25T08:53:20.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.825+0000 7f433be00640 1 -- 172.21.15.46:0/2598814260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f433c011d10 con 0x7f4344153f00 2024-04-25T08:53:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.825+0000 7f433be00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4320066f60 0x7f4320069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.825+0000 7f433be00640 1 -- 172.21.15.46:0/2598814260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f433c084ce0 con 0x7f4344153f00 2024-04-25T08:53:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.825+0000 7f4342a00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4320066f60 0x7f4320069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.825+0000 7f4342a00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4320066f60 0x7f4320069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f43340ae000 tx=0x7f43340a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.829+0000 7f433be00640 1 -- 172.21.15.46:0/2598814260 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f433c055d10 con 0x7f4344153f00 2024-04-25T08:53:20.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.985+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43440d0700 con 0x7f4320066f60 2024-04-25T08:53:20.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f433be00640 1 -- 172.21.15.46:0/2598814260 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13466 (secure 0 0 0) 0x7f43440d0700 con 0x7f4320066f60 2024-04-25T08:53:20.980 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:20.980 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:53:20.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4320066f60 msgr2=0x7f4320069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4320066f60 0x7f4320069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f43340ae000 tx=0x7f43340a7000 comp rx=0 tx=0).stop 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4344153f00 msgr2=0x7f43440b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4344153f00 0x7f43440b1480 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f433c00c3c0 tx=0x7f433c00c880 comp rx=0 tx=0).stop 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 shutdown_connections 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4320066f60 0x7f4320069420 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4344153f00 0x7f43440b1480 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f434415ef20 0x7f434409e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 --2- 172.21.15.46:0/2598814260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f434409ab50 0x7f43440b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 >> 172.21.15.46:0/2598814260 conn(0x7f43440b3120 msgr2=0x7f43440ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 shutdown_connections 2024-04-25T08:53:20.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:20.989+0000 7f4349e00640 1 -- 172.21.15.46:0/2598814260 wait complete. 2024-04-25T08:53:21.001 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-04-25T08:53:20.423326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262948,"kb_used_data":13480,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711772,"statfs":{"total":773094113280,"available":772824854528,"internally_reserved":0,"allocated":13803520,"data_stored":7080312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.419224"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":369,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T08:53:12.008144+0000","last_change":"2024-04-25T08:42:10.439189+0000","last_active":"2024-04-25T08:53:12.008144+0000","last_peered":"2024-04-25T08:53:12.008144+0000","last_clean":"2024-04-25T08:53:12.008144+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:53:12.008144+0000","last_undegraded":"2024-04-25T08:53:12.008144+0000","last_fullsized":"2024-04-25T08:53:12.008144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:09.076376+0000","last_clean_scrub_stamp":"2024-04-25T08:42:09.076376+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:06.504363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574986,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2170880,"data_stored":1327815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574985,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574984,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2170880,"data_stored":1327815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:53:21.004 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T08:42:09.076376+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=8, tm_min=42, tm_sec=9, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=8, tm_min=53, tm_sec=15, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T08:53:21.004 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T08:53:21.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:21.626 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:41.006 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:53:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe30c000640 1 Processor -- start 2024-04-25T08:53:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe30c000640 1 -- start start 2024-04-25T08:53:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe30c000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe30415a750 0x7fe30406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe30c000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3040aed10 0x7fe30406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe30c000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30406d5e0 con 0x7fe3040cc9e0 2024-04-25T08:53:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe30c000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30415ab30 con 0x7fe30415a750 2024-04-25T08:53:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe30c000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3040ce4f0 con 0x7fe3040aed10 2024-04-25T08:53:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe30a000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe3040cc9e0 0x7fe3040b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41232/0 (socket says 172.21.15.46:41232) 2024-04-25T08:53:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe30a000640 1 -- 172.21.15.46:0/1378236667 learned_addr learned my addr 172.21.15.46:0/1378236667 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe309600640 1 --2- 172.21.15.46:0/1378236667 >> v2:172.21.15.46:3300/0 conn(0x7fe30415a750 0x7fe30406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.165+0000 7fe308c00640 1 --2- 172.21.15.46:0/1378236667 >> v2:172.21.15.82:3300/0 conn(0x7fe3040aed10 0x7fe30406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.169+0000 7fe309600640 1 -- 172.21.15.46:0/1378236667 >> v2:172.21.15.82:3300/0 conn(0x7fe3040aed10 msgr2=0x7fe30406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.169+0000 7fe309600640 1 --2- 172.21.15.46:0/1378236667 >> v2:172.21.15.82:3300/0 conn(0x7fe3040aed10 0x7fe30406f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.169+0000 7fe309600640 1 -- 172.21.15.46:0/1378236667 >> v1:172.21.15.46:6789/0 conn(0x7fe3040cc9e0 legacy=0x7fe3040b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.169+0000 7fe309600640 1 -- 172.21.15.46:0/1378236667 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3040ce720 con 0x7fe30415a750 2024-04-25T08:53:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.169+0000 7fe309600640 1 --2- 172.21.15.46:0/1378236667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30415a750 0x7fe30406ec10 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fe3040cdca0 tx=0x7fe2f801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb2aac2b9088b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.169+0000 7fe303e00640 1 -- 172.21.15.46:0/1378236667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80b0070 con 0x7fe30415a750 2024-04-25T08:53:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.169+0000 7fe303e00640 1 -- 172.21.15.46:0/1378236667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f80ab600 con 0x7fe30415a750 2024-04-25T08:53:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.169+0000 7fe303e00640 1 -- 172.21.15.46:0/1378236667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80ab9d0 con 0x7fe30415a750 2024-04-25T08:53:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 -- 172.21.15.46:0/1378236667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30415a750 msgr2=0x7fe30406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 --2- 172.21.15.46:0/1378236667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30415a750 0x7fe30406ec10 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fe3040cdca0 tx=0x7fe2f801c480 comp rx=0 tx=0).stop 2024-04-25T08:53:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 -- 172.21.15.46:0/1378236667 shutdown_connections 2024-04-25T08:53:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 --2- 172.21.15.46:0/1378236667 >> v2:172.21.15.82:3300/0 conn(0x7fe3040aed10 0x7fe30406f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 --2- 172.21.15.46:0/1378236667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe30415a750 0x7fe30406ec10 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 -- 172.21.15.46:0/1378236667 >> 172.21.15.46:0/1378236667 conn(0x7fe3040b12a0 msgr2=0x7fe3040b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 -- 172.21.15.46:0/1378236667 shutdown_connections 2024-04-25T08:53:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 -- 172.21.15.46:0/1378236667 wait complete. 2024-04-25T08:53:41.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 Processor -- start 2024-04-25T08:53:41.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 -- start start 2024-04-25T08:53:41.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3040aed10 0x7fe3040a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3040cc9e0 0x7fe3041fe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30415a750 0x7fe3041fe700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304167230 con 0x7fe3040cc9e0 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30406d270 con 0x7fe30415a750 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30c000640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041673b0 con 0x7fe3040aed10 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe308c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3040cc9e0 0x7fe3041fe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe308c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3040cc9e0 0x7fe3041fe1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52310/0 (socket says 172.21.15.46:52310) 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe308c00640 1 -- 172.21.15.46:0/1112149951 learned_addr learned my addr 172.21.15.46:0/1112149951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe309600640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3040aed10 0x7fe3040a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30a000640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30415a750 0x7fe3041fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe308c00640 1 -- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3040aed10 msgr2=0x7fe3040a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe308c00640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3040aed10 0x7fe3040a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe308c00640 1 -- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30415a750 msgr2=0x7fe3041fe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe308c00640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30415a750 0x7fe3041fe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe308c00640 1 -- 172.21.15.46:0/1112149951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3041582f0 con 0x7fe3040cc9e0 2024-04-25T08:53:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe30a000640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30415a750 0x7fe3041fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.173+0000 7fe308c00640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3040cc9e0 0x7fe3041fe1c0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000e900 tx=0x7fe2f000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe301600640 1 -- 172.21.15.46:0/1112149951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0017070 con 0x7fe3040cc9e0 2024-04-25T08:53:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe301600640 1 -- 172.21.15.46:0/1112149951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f000ca00 con 0x7fe3040cc9e0 2024-04-25T08:53:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe301600640 1 -- 172.21.15.46:0/1112149951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000cdf0 con 0x7fe3040cc9e0 2024-04-25T08:53:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3041585d0 con 0x7fe3040cc9e0 2024-04-25T08:53:41.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe304158b90 con 0x7fe3040cc9e0 2024-04-25T08:53:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe304069080 con 0x7fe3040cc9e0 2024-04-25T08:53:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe301600640 1 -- 172.21.15.46:0/1112149951 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2f001a070 con 0x7fe3040cc9e0 2024-04-25T08:53:41.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe301600640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e4066f60 0x7fe2e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe301600640 1 -- 172.21.15.46:0/1112149951 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe2f0085270 con 0x7fe3040cc9e0 2024-04-25T08:53:41.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.177+0000 7fe309600640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e4066f60 0x7fe2e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.181+0000 7fe309600640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e4066f60 0x7fe2e4069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80ad000 tx=0x7fe2f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.181+0000 7fe301600640 1 -- 172.21.15.46:0/1112149951 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2f0055ce0 con 0x7fe3040cc9e0 2024-04-25T08:53:41.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.341+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 --> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3040ad6b0 con 0x7fe2e4066f60 2024-04-25T08:53:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.341+0000 7fe301600640 1 -- 172.21.15.46:0/1112149951 <== mgr.4110 v2:172.21.15.82:6832/2153338604 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13379 (secure 0 0 0) 0x7fe3040ad6b0 con 0x7fe2e4066f60 2024-04-25T08:53:41.335 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-04-25T08:53:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e4066f60 msgr2=0x7fe2e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e4066f60 0x7fe2e4069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80ad000 tx=0x7fe2f8008040 comp rx=0 tx=0).stop 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3040cc9e0 msgr2=0x7fe3041fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3040cc9e0 0x7fe3041fe1c0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000e900 tx=0x7fe2f000edc0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 shutdown_connections 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7fe2e4066f60 0x7fe2e4069420 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe30415a750 0x7fe3041fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3040cc9e0 0x7fe3041fe1c0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 --2- 172.21.15.46:0/1112149951 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7fe3040aed10 0x7fe3040a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 >> 172.21.15.46:0/1112149951 conn(0x7fe3040b12a0 msgr2=0x7fe3040b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 shutdown_connections 2024-04-25T08:53:41.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.345+0000 7fe30c000640 1 -- 172.21.15.46:0/1112149951 wait complete. 2024-04-25T08:53:41.357 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":466,"stamp":"2024-04-25T08:53:40.428025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262932,"kb_used_data":13480,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711788,"statfs":{"total":773094113280,"available":772824870912,"internally_reserved":0,"allocated":13803520,"data_stored":7080312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":255315506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002723"},"pg_stats":[{"pgid":"1.0","version":"126'119","reported_seq":380,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T08:53:20.746498+0000","last_change":"2024-04-25T08:53:20.746549+0000","last_active":"2024-04-25T08:53:20.746498+0000","last_peered":"2024-04-25T08:53:20.746498+0000","last_clean":"2024-04-25T08:53:20.746498+0000","last_became_active":"2024-04-25T08:42:10.436519+0000","last_became_peered":"2024-04-25T08:42:10.436519+0000","last_unstale":"2024-04-25T08:53:20.746498+0000","last_undegraded":"2024-04-25T08:53:20.746498+0000","last_fullsized":"2024-04-25T08:53:20.746498+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'119","last_scrub_stamp":"2024-04-25T08:53:20.746409+0000","last_deep_scrub":"126'119","last_deep_scrub_stamp":"2024-04-25T08:53:20.746409+0000","last_clean_scrub_stamp":"2024-04-25T08:53:20.746409+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:17.537229+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2170880,"data_stored":1327815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1576960,"data_stored":737447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2170880,"data_stored":1327815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:53:41.358 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T08:53:41.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb3e00640 1 Processor -- start 2024-04-25T08:53:41.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb3e00640 1 -- start start 2024-04-25T08:53:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efdac06caf0 0x7efdac0d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 0x7efdac162130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdac0cfa40 con 0x7efdac160f70 2024-04-25T08:53:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdac162670 con 0x7efdac06caf0 2024-04-25T08:53:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdac0cfce0 con 0x7efdac06bd20 2024-04-25T08:53:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efdac160f70 0x7efdac06c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41248/0 (socket says 172.21.15.46:41248) 2024-04-25T08:53:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb1400640 1 -- 172.21.15.46:0/2405608775 learned_addr learned my addr 172.21.15.46:0/2405608775 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb0a00640 1 --2- 172.21.15.46:0/2405608775 >> v2:172.21.15.46:3300/0 conn(0x7efdac06caf0 0x7efdac0d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdb1e00640 1 --2- 172.21.15.46:0/2405608775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 0x7efdac162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdabe00640 1 -- 172.21.15.46:0/2405608775 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002459887 0 0) 0x7efdac0cfa40 con 0x7efdac160f70 2024-04-25T08:53:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.505+0000 7efdabe00640 1 -- 172.21.15.46:0/2405608775 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd94003530 con 0x7efdac160f70 2024-04-25T08:53:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb0a00640 1 -- 172.21.15.46:0/2405608775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 msgr2=0x7efdac162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb0a00640 1 --2- 172.21.15.46:0/2405608775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 0x7efdac162130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb0a00640 1 -- 172.21.15.46:0/2405608775 >> v1:172.21.15.46:6789/0 conn(0x7efdac160f70 legacy=0x7efdac06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb0a00640 1 -- 172.21.15.46:0/2405608775 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac0cffc0 con 0x7efdac06caf0 2024-04-25T08:53:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb0a00640 1 --2- 172.21.15.46:0/2405608775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac06caf0 0x7efdac0d0d80 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7efd980097c0 tx=0x7efd9801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529ab4660b09b2d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdabe00640 1 -- 172.21.15.46:0/2405608775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980b5070 con 0x7efdac06caf0 2024-04-25T08:53:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdabe00640 1 -- 172.21.15.46:0/2405608775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd980a8990 con 0x7efdac06caf0 2024-04-25T08:53:41.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdabe00640 1 -- 172.21.15.46:0/2405608775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980a8d60 con 0x7efdac06caf0 2024-04-25T08:53:41.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb3e00640 1 -- 172.21.15.46:0/2405608775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac06caf0 msgr2=0x7efdac0d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb3e00640 1 --2- 172.21.15.46:0/2405608775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac06caf0 0x7efdac0d0d80 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7efd980097c0 tx=0x7efd9801ec90 comp rx=0 tx=0).stop 2024-04-25T08:53:41.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb3e00640 1 -- 172.21.15.46:0/2405608775 shutdown_connections 2024-04-25T08:53:41.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb3e00640 1 --2- 172.21.15.46:0/2405608775 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 0x7efdac162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb3e00640 1 --2- 172.21.15.46:0/2405608775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac06caf0 0x7efdac0d0d80 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb3e00640 1 -- 172.21.15.46:0/2405608775 >> 172.21.15.46:0/2405608775 conn(0x7efdac0b3130 msgr2=0x7efdac0b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb3e00640 1 -- 172.21.15.46:0/2405608775 shutdown_connections 2024-04-25T08:53:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.509+0000 7efdb3e00640 1 -- 172.21.15.46:0/2405608775 wait complete. 2024-04-25T08:53:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 Processor -- start 2024-04-25T08:53:41.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 -- start start 2024-04-25T08:53:41.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 0x7efdac15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac06caf0 0x7efdac0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac160f70 0x7efdac0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac0d0860 con 0x7efdac160f70 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac0d0580 con 0x7efdac06caf0 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdac161350 con 0x7efdac06bd20 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac160f70 0x7efdac0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac160f70 0x7efdac0ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52328/0 (socket says 172.21.15.46:52328) 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1e00640 1 -- 172.21.15.46:0/2571430539 learned_addr learned my addr 172.21.15.46:0/2571430539 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb0a00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac06caf0 0x7efdac0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1400640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 0x7efdac15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1e00640 1 -- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 msgr2=0x7efdac15ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1e00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 0x7efdac15ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1e00640 1 -- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac06caf0 msgr2=0x7efdac0c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1e00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac06caf0 0x7efdac0c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1e00640 1 -- 172.21.15.46:0/2571430539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac0caee0 con 0x7efdac160f70 2024-04-25T08:53:41.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb1e00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac160f70 0x7efdac0ca9a0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7efda400efc0 tx=0x7efda400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efda9600640 1 -- 172.21.15.46:0/2571430539 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4018020 con 0x7efdac160f70 2024-04-25T08:53:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efda9600640 1 -- 172.21.15.46:0/2571430539 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda4014480 con 0x7efdac160f70 2024-04-25T08:53:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efda9600640 1 -- 172.21.15.46:0/2571430539 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4014870 con 0x7efdac160f70 2024-04-25T08:53:41.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac0cb180 con 0x7efdac160f70 2024-04-25T08:53:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.513+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac0c8710 con 0x7efdac160f70 2024-04-25T08:53:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.517+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac069010 con 0x7efdac160f70 2024-04-25T08:53:41.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.517+0000 7efda9600640 1 -- 172.21.15.46:0/2571430539 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efda401b020 con 0x7efdac160f70 2024-04-25T08:53:41.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.517+0000 7efda9600640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd8c066f60 0x7efd8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.517+0000 7efda9600640 1 -- 172.21.15.46:0/2571430539 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7efda4085a50 con 0x7efdac160f70 2024-04-25T08:53:41.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.517+0000 7efdb1400640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd8c066f60 0x7efd8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.517+0000 7efdb1400640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd8c066f60 0x7efd8c069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7efda0005890 tx=0x7efda0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.521+0000 7efda9600640 1 -- 172.21.15.46:0/2571430539 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efda4022070 con 0x7efdac160f70 2024-04-25T08:53:41.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.685+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7efdac0b3f60 con 0x7efdac160f70 2024-04-25T08:53:41.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efda9600640 1 -- 172.21.15.46:0/2571430539 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7efda4010040 con 0x7efdac160f70 2024-04-25T08:53:41.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efda9600640 1 -- 172.21.15.46:0/2571430539 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7efda408b080 con 0x7efdac160f70 2024-04-25T08:53:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd8c066f60 msgr2=0x7efd8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd8c066f60 0x7efd8c069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7efda0005890 tx=0x7efda0009040 comp rx=0 tx=0).stop 2024-04-25T08:53:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac160f70 msgr2=0x7efdac0ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac160f70 0x7efdac0ca9a0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7efda400efc0 tx=0x7efda400c490 comp rx=0 tx=0).stop 2024-04-25T08:53:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 shutdown_connections 2024-04-25T08:53:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7efd8c066f60 0x7efd8c069420 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdac160f70 0x7efdac0ca9a0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdac06caf0 0x7efdac0c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 --2- 172.21.15.46:0/2571430539 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7efdac06bd20 0x7efdac15ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 >> 172.21.15.46:0/2571430539 conn(0x7efdac0b3130 msgr2=0x7efdac073640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 shutdown_connections 2024-04-25T08:53:41.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.689+0000 7efdb3e00640 1 -- 172.21.15.46:0/2571430539 wait complete. 2024-04-25T08:53:41.703 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:53:41.704 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f449c800640 1 Processor -- start 2024-04-25T08:53:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f449c800640 1 -- start start 2024-04-25T08:53:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f449c800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f449415a750 0x7f449406ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f449c800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44940aed10 0x7f449406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f449c800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449406d5e0 con 0x7f44940cc9e0 2024-04-25T08:53:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f449c800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449415ab30 con 0x7f449415a750 2024-04-25T08:53:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f449c800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44940ce4f0 con 0x7f44940aed10 2024-04-25T08:53:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f4499e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f449415a750 0x7f449406ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f449a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f44940cc9e0 0x7f44940b25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41254/0 (socket says 172.21.15.46:41254) 2024-04-25T08:53:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f449a800640 1 -- 172.21.15.46:0/2946881378 learned_addr learned my addr 172.21.15.46:0/2946881378 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-04-25T08:53:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f4499400640 1 --2- 172.21.15.46:0/2946881378 >> v2:172.21.15.82:3300/0 conn(0x7f44940aed10 0x7f449406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f4499e00640 1 -- 172.21.15.46:0/2946881378 >> v2:172.21.15.82:3300/0 conn(0x7f44940aed10 msgr2=0x7f449406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f4499e00640 1 --2- 172.21.15.46:0/2946881378 >> v2:172.21.15.82:3300/0 conn(0x7f44940aed10 0x7f449406f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f4499e00640 1 -- 172.21.15.46:0/2946881378 >> v1:172.21.15.46:6789/0 conn(0x7f44940cc9e0 legacy=0x7f44940b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f4499e00640 1 -- 172.21.15.46:0/2946881378 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44940ce720 con 0x7f449415a750 2024-04-25T08:53:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.853+0000 7f4498a00640 1 -- 172.21.15.46:0/2946881378 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969375808 0 0) 0x7f449406d5e0 con 0x7f44940cc9e0 2024-04-25T08:53:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f4499e00640 1 --2- 172.21.15.46:0/2946881378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449415a750 0x7f449406ec10 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f44940cdca0 tx=0x7f448801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bae1c8698f3566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f4498a00640 1 -- 172.21.15.46:0/2946881378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44880b0070 con 0x7f449415a750 2024-04-25T08:53:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f4498a00640 1 -- 172.21.15.46:0/2946881378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f44880ab600 con 0x7f449415a750 2024-04-25T08:53:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f4498a00640 1 -- 172.21.15.46:0/2946881378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44880ab9f0 con 0x7f449415a750 2024-04-25T08:53:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f449c800640 1 -- 172.21.15.46:0/2946881378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449415a750 msgr2=0x7f449406ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f449c800640 1 --2- 172.21.15.46:0/2946881378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449415a750 0x7f449406ec10 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f44940cdca0 tx=0x7f448801c480 comp rx=0 tx=0).stop 2024-04-25T08:53:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f449c800640 1 -- 172.21.15.46:0/2946881378 shutdown_connections 2024-04-25T08:53:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f449c800640 1 --2- 172.21.15.46:0/2946881378 >> v2:172.21.15.82:3300/0 conn(0x7f44940aed10 0x7f449406f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f449c800640 1 --2- 172.21.15.46:0/2946881378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449415a750 0x7f449406ec10 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f449c800640 1 -- 172.21.15.46:0/2946881378 >> 172.21.15.46:0/2946881378 conn(0x7f44940b12a0 msgr2=0x7f44940b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f449c800640 1 -- 172.21.15.46:0/2946881378 shutdown_connections 2024-04-25T08:53:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.857+0000 7f449c800640 1 -- 172.21.15.46:0/2946881378 wait complete. 2024-04-25T08:53:41.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 Processor -- start 2024-04-25T08:53:41.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 -- start start 2024-04-25T08:53:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 --2- >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44940aed10 0x7f4494149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44940cc9e0 0x7f44940af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f449415a750 0x7f44940af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449406db00 con 0x7f44940cc9e0 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449406d270 con 0x7f449415a750 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 -- --> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449406dc80 con 0x7f44940aed10 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44940cc9e0 0x7f44940af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44940cc9e0 0x7f44940af420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52338/0 (socket says 172.21.15.46:52338) 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499400640 1 -- 172.21.15.46:0/3029571183 learned_addr learned my addr 172.21.15.46:0/3029571183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499e00640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44940aed10 0x7f4494149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449a800640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f449415a750 0x7f44940af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499400640 1 -- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44940aed10 msgr2=0x7f4494149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499400640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44940aed10 0x7f4494149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499400640 1 -- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f449415a750 msgr2=0x7f44940af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499400640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f449415a750 0x7f44940af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499400640 1 -- 172.21.15.46:0/3029571183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449414a3f0 con 0x7f44940cc9e0 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449a800640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f449415a750 0x7f44940af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:41.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f4499400640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44940cc9e0 0x7f44940af420 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f448400d800 tx=0x7f448400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f448e000640 1 -- 172.21.15.46:0/3029571183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448400b6c0 con 0x7f44940cc9e0 2024-04-25T08:53:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f448e000640 1 -- 172.21.15.46:0/3029571183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4484019070 con 0x7f44940cc9e0 2024-04-25T08:53:41.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f448e000640 1 -- 172.21.15.46:0/3029571183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448400bac0 con 0x7f44940cc9e0 2024-04-25T08:53:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449414a6d0 con 0x7f44940cc9e0 2024-04-25T08:53:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449414ac90 con 0x7f44940cc9e0 2024-04-25T08:53:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.861+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4494069080 con 0x7f44940cc9e0 2024-04-25T08:53:41.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.865+0000 7f448e000640 1 -- 172.21.15.46:0/3029571183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f448400bc60 con 0x7f44940cc9e0 2024-04-25T08:53:41.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.865+0000 7f448e000640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4478066f60 0x7f4478069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.865+0000 7f448e000640 1 -- 172.21.15.46:0/3029571183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f4484085410 con 0x7f44940cc9e0 2024-04-25T08:53:41.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.865+0000 7f4499e00640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4478066f60 0x7f4478069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.865+0000 7f4499e00640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4478066f60 0x7f4478069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f44880ad000 tx=0x7f44880a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:41.869+0000 7f448e000640 1 -- 172.21.15.46:0/3029571183 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4484056440 con 0x7f44940cc9e0 2024-04-25T08:53:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.061+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4494153d20 con 0x7f44940cc9e0 2024-04-25T08:53:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.061+0000 7f448e000640 1 -- 172.21.15.46:0/3029571183 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+3696 (secure 0 0 0) 0x7f4484055d40 con 0x7f44940cc9e0 2024-04-25T08:53:42.055 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-04-25T08:53:42.055 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7559,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/193328250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":193328250},{"type":"v1","addr":"172.21.15.82:6836","nonce":193328250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":7730,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/443359018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":443359018},{"type":"v1","addr":"172.21.15.46:6835","nonce":443359018}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101},{"gid":8461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2058573719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2058573719},{"type":"v1","addr":"172.21.15.82:6837","nonce":2058573719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":9591,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/1268454752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1268454752},{"type":"v1","addr":"172.21.15.46:6834","nonce":1268454752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":114}],"filesystems":[]} 2024-04-25T08:53:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 114 2024-04-25T08:53:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4478066f60 msgr2=0x7f4478069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4478066f60 0x7f4478069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f44880ad000 tx=0x7f44880a6000 comp rx=0 tx=0).stop 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44940cc9e0 msgr2=0x7f44940af420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44940cc9e0 0x7f44940af420 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f448400d800 tx=0x7f448400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 shutdown_connections 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:6832/2153338604,v1:172.21.15.82:6833/2153338604] conn(0x7f4478066f60 0x7f4478069420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f449415a750 0x7f44940af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44940cc9e0 0x7f44940af420 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 --2- 172.21.15.46:0/3029571183 >> [v2:172.21.15.82:3301/0,v1:172.21.15.82:6790/0] conn(0x7f44940aed10 0x7f4494149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 >> 172.21.15.46:0/3029571183 conn(0x7f44940b12a0 msgr2=0x7f44940ac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 shutdown_connections 2024-04-25T08:53:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-04-25T08:53:42.065+0000 7f449c800640 1 -- 172.21.15.46:0/3029571183 wait complete. 2024-04-25T08:53:42.078 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T08:53:42.079 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T08:53:42.079 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:53:42.079 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.092 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:53:42.092 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:53:42.092 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.147 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:53:42.147 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:53:42.147 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.162 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:53:42.163 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:53:42.163 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.220 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:53:42.221 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T08:53:42.221 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T08:53:42.221 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.280 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T08:53:42.280 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T08:53:42.280 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.372 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T08:53:42.372 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T08:53:42.372 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.469 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T08:53:42.470 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T08:53:42.470 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.564 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T08:53:42.564 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T08:53:42.564 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.644 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T08:53:42.645 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T08:53:42.645 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:42.736 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T08:53:42.737 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T08:53:42.737 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:46.152 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-25T08:53:48.838 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T08:53:48.839 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T08:53:48.839 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:48.916 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T08:53:48.916 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T08:53:48.917 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T08:53:48.917 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:48.973 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T08:53:48.974 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T08:53:48.974 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:49.034 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T08:53:49.034 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T08:53:49.034 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T08:53:49.034 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:49.054 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T08:53:49.054 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T08:53:49.054 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:49.070 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T08:53:49.070 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T08:53:49.071 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:49.131 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T08:53:49.131 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T08:53:49.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-25T08:53:49.166 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:53:49.166 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T08:53:49.457 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:53:49.458 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T08:53:49.734 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:53:49.734 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T08:53:49.996 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:53:49.996 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T08:53:50.159 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:53:50.159 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T08:53:50.404 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:53:50.404 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T08:53:50.590 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:53:50.590 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T08:53:50.774 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:53:50.774 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T08:53:51.020 INFO:tasks.ceph:Archiving mon data... 2024-04-25T08:53:51.021 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/data/mon.a.tgz 2024-04-25T08:53:51.021 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-04-25T08:53:51.031 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.F60kYAzCrf 2024-04-25T08:53:51.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.F60kYAzCrf 2024-04-25T08:53:51.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-25T08:53:51.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.F60kYAzCrf 2024-04-25T08:53:51.264 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.F60kYAzCrf is 415KB 2024-04-25T08:53:51.334 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.F60kYAzCrf 2024-04-25T08:53:51.343 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/data/mon.b.tgz 2024-04-25T08:53:51.343 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-04-25T08:53:51.352 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.BrW4DUUwYf 2024-04-25T08:53:51.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.BrW4DUUwYf 2024-04-25T08:53:51.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.BrW4DUUwYf 2024-04-25T08:53:51.578 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.BrW4DUUwYf is 423KB 2024-04-25T08:53:51.645 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.BrW4DUUwYf 2024-04-25T08:53:51.654 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/data/mon.c.tgz 2024-04-25T08:53:51.655 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-04-25T08:53:51.701 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.JqNr3U6Nkw 2024-04-25T08:53:51.702 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.JqNr3U6Nkw 2024-04-25T08:53:51.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:51.717 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:51.842 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.JqNr3U6Nkw 2024-04-25T08:53:51.918 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.JqNr3U6Nkw is 423KB 2024-04-25T08:53:51.981 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.JqNr3U6Nkw 2024-04-25T08:53:51.989 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T08:53:51.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T08:53:51.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T08:53:52.064 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T08:53:52.097 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T08:53:52.098 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T08:53:52.099 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/remote/smithi046/crash 2024-04-25T08:53:52.100 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T08:53:52.128 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/remote/smithi082/crash 2024-04-25T08:53:52.129 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T08:53:52.156 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T08:53:52.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:52.174 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:52.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:52.284 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:52.304 INFO:tasks.ceph:Compressing logs... 2024-04-25T08:53:52.304 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T08:53:52.330 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T08:53:52.360 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41753.log 2024-04-25T08:53:52.361 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39009.log 2024-04-25T08:53:52.363 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45503.log 2024-04-25T08:53:52.363 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41753.log.gz 2024-04-25T08:53:52.363 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35466.log 2024-04-25T08:53:52.363 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39009.log.gz 2024-04-25T08:53:52.366 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32840.log 2024-04-25T08:53:52.366 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45503.log.gz 2024-04-25T08:53:52.366 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49495.log 2024-04-25T08:53:52.366 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35466.log.gz 2024-04-25T08:53:52.366 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52474.log 2024-04-25T08:53:52.366 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32840.log.gz 2024-04-25T08:53:52.367 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50319.log 2024-04-25T08:53:52.367 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49495.log.gz 2024-04-25T08:53:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33417.log 2024-04-25T08:53:52.368 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52474.log.gz 2024-04-25T08:53:52.368 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34553.log 2024-04-25T08:53:52.368 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50319.log.gz 2024-04-25T08:53:52.369 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50634.log 2024-04-25T08:53:52.369 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33417.log.gz 2024-04-25T08:53:52.369 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.2.34553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.34553.log.gz 2024-04-25T08:53:52.370 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47903.log 2024-04-25T08:53:52.370 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38530.log 2024-04-25T08:53:52.370 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50634.log.gz 2024-04-25T08:53:52.371 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35601.log 2024-04-25T08:53:52.371 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47903.log.gz 2024-04-25T08:53:52.372 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48942.log 2024-04-25T08:53:52.372 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38530.log.gz 2024-04-25T08:53:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44303.log 2024-04-25T08:53:52.374 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35601.log.gz 2024-04-25T08:53:52.374 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46925.log 2024-04-25T08:53:52.374 INFO:teuthology.orchestra.run.smithi046.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.48942.log.gz 2024-04-25T08:53:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32579.log 2024-04-25T08:53:52.375 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44303.log.gz 2024-04-25T08:53:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50742.log 2024-04-25T08:53:52.376 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46925.log.gz 2024-04-25T08:53:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-04-25T08:53:52.377 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.32579.log.gz 2024-04-25T08:53:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35015.log 2024-04-25T08:53:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43972.log: gzip -5 --verbose 91.0% -- /var/log/ceph/ceph-client.admin.35921.log 2024-04-25T08:53:52.378 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50742.log.gz 2024-04-25T08:53:52.378 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-04-25T08:53:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2024-04-25T08:53:52.379 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35015.log.gz 2024-04-25T08:53:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41909.log 2024-04-25T08:53:52.380 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35921.log.gz 2024-04-25T08:53:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32139.log 2024-04-25T08:53:52.382 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2024-04-25T08:53:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48311.log 2024-04-25T08:53:52.382 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41909.log.gz 2024-04-25T08:53:52.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47113.log 2024-04-25T08:53:52.383 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32139.log.gz 2024-04-25T08:53:52.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42943.log 2024-04-25T08:53:52.384 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33246.log 2024-04-25T08:53:52.385 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48311.log.gz 2024-04-25T08:53:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51475.log 2024-04-25T08:53:52.385 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47113.log.gz 2024-04-25T08:53:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.42943.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.0.42943.log.gz --verbose 2024-04-25T08:53:52.385 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.42263.log 2024-04-25T08:53:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52239.log 2024-04-25T08:53:52.387 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51475.log.gz 2024-04-25T08:53:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42263.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T08:53:52.387 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42263.log.gz 2024-04-25T08:53:52.388 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27720.log 2024-04-25T08:53:52.388 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.3.33246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27792.log 2024-04-25T08:53:52.388 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33246.log.gz 2024-04-25T08:53:52.389 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.27720.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T08:53:52.389 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27720.log.gz 2024-04-25T08:53:52.389 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.27792.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T08:53:52.389 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27792.log.gz 2024-04-25T08:53:52.389 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27696.log 2024-04-25T08:53:52.389 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33251.log 2024-04-25T08:53:52.390 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.27696.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T08:53:52.390 INFO:teuthology.orchestra.run.smithi082.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27696.log.gz 2024-04-25T08:53:52.392 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37687.log 2024-04-25T08:53:52.392 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52239.log.gz 2024-04-25T08:53:52.392 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45928.log 2024-04-25T08:53:52.393 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40208.log 2024-04-25T08:53:52.393 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37687.log.gz 2024-04-25T08:53:52.393 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47760.log 2024-04-25T08:53:52.393 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45928.log.gz 2024-04-25T08:53:52.393 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41344.log 2024-04-25T08:53:52.393 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40208.log.gz 2024-04-25T08:53:52.393 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46221.log 2024-04-25T08:53:52.393 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47760.log.gz 2024-04-25T08:53:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.3.33251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33251.log.gz 2024-04-25T08:53:52.397 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27768.log 2024-04-25T08:53:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33152.log 2024-04-25T08:53:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.27768.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T08:53:52.398 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27768.log.gz 2024-04-25T08:53:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.1.33152.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T08:53:52.398 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33152.log.gz 2024-04-25T08:53:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27840.log 2024-04-25T08:53:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T08:53:52.398 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.27840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27744.log 2024-04-25T08:53:52.398 INFO:teuthology.orchestra.run.smithi082.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.27840.log.gz 2024-04-25T08:53:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43661.log 2024-04-25T08:53:52.400 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41344.log.gz 2024-04-25T08:53:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53197.log 2024-04-25T08:53:52.400 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46221.log.gz 2024-04-25T08:53:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.43661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43661.log.gz 2024-04-25T08:53:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36287.log 2024-04-25T08:53:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51568.log 2024-04-25T08:53:52.400 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53197.log.gz 2024-04-25T08:53:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36025.log 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi046.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36287.log.gz 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34446.log 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi046.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51568.log.gz 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43069.log 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36025.log.gz 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.34446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34446.log.gz 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48580.log 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50838.log 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.27744.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27744.log.gz 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T08:53:52.401 INFO:teuthology.orchestra.run.smithi082.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T08:53:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T08:53:52.403 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T08:53:52.404 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43069.log.gz 2024-04-25T08:53:52.404 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27667.log 2024-04-25T08:53:52.404 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48580.log.gz 2024-04-25T08:53:52.404 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.50838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43508.log 2024-04-25T08:53:52.404 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50838.log.gz 2024-04-25T08:53:52.404 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.27667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45046.log 2024-04-25T08:53:52.405 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27667.log.gz 2024-04-25T08:53:52.405 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36697.log 2024-04-25T08:53:52.405 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43508.log.gz 2024-04-25T08:53:52.405 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.45046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37817.log 2024-04-25T08:53:52.405 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45046.log.gz 2024-04-25T08:53:52.407 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36723.log 2024-04-25T08:53:52.407 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36697.log.gz 2024-04-25T08:53:52.407 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32788.log 2024-04-25T08:53:52.407 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37817.log.gz 2024-04-25T08:53:52.407 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45978.log 2024-04-25T08:53:52.407 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36723.log.gz 2024-04-25T08:53:52.407 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37583.log 2024-04-25T08:53:52.407 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32788.log.gz 2024-04-25T08:53:52.408 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T08:53:52.408 INFO:teuthology.orchestra.run.smithi082.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T08:53:52.408 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27816.log 2024-04-25T08:53:52.408 INFO:teuthology.orchestra.run.smithi082.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T08:53:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46738.log 2024-04-25T08:53:52.409 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45978.log.gz 2024-04-25T08:53:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33198.log 2024-04-25T08:53:52.409 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37583.log.gz 2024-04-25T08:53:52.409 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45723.log 2024-04-25T08:53:52.409 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46738.log.gz 2024-04-25T08:53:52.411 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27864.log/var/log/ceph/ceph-mon.b.log: 2024-04-25T08:53:52.411 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.27816.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27816.log.gz 2024-04-25T08:53:52.411 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.27864.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27864.log.gz 2024-04-25T08:53:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44331.log 2024-04-25T08:53:52.412 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33198.log.gz 2024-04-25T08:53:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50475.log 2024-04-25T08:53:52.412 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45723.log.gz 2024-04-25T08:53:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49443.log 2024-04-25T08:53:52.412 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44331.log.gz 2024-04-25T08:53:52.412 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50866.log 2024-04-25T08:53:52.414 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T08:53:52.414 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33170.log 2024-04-25T08:53:52.414 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49443.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.50475.log.gz 2024-04-25T08:53:52.415 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35999.log 2024-04-25T08:53:52.415 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49443.log.gz 2024-04-25T08:53:52.415 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52911.log 2024-04-25T08:53:52.415 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50866.log.gz 2024-04-25T08:53:52.415 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33053.log 2024-04-25T08:53:52.415 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35999.log.gz 2024-04-25T08:53:52.416 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.1.33170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33170.log.gz 2024-04-25T08:53:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47035.log 2024-04-25T08:53:52.417 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52911.log.gz 2024-04-25T08:53:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33053.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T08:53:52.417 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33053.log.gz 2024-04-25T08:53:52.417 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44024.log 2024-04-25T08:53:52.417 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47035.log.gz 2024-04-25T08:53:52.418 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46764.log 2024-04-25T08:53:52.418 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47367.log 2024-04-25T08:53:52.419 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44024.log.gz 2024-04-25T08:53:52.419 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47931.log 2024-04-25T08:53:52.420 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0%/var/log/ceph/ceph-client.admin.47367.log: -- replaced with /var/log/ceph/ceph-client.admin.46764.log.gz 2024-04-25T08:53:52.420 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45236.log 2024-04-25T08:53:52.420 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47367.log.gz 2024-04-25T08:53:52.421 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31774.log 2024-04-25T08:53:52.421 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47931.log.gz 2024-04-25T08:53:52.422 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47788.log 2024-04-25T08:53:52.422 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45236.log.gz 2024-04-25T08:53:52.422 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33859.log 2024-04-25T08:53:52.423 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31774.log.gz 2024-04-25T08:53:52.423 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38019.log 2024-04-25T08:53:52.424 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47788.log.gz 2024-04-25T08:53:52.424 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42465.log 2024-04-25T08:53:52.425 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33859.log.gz 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33599.log 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.42465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42465.log.gz 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46348.log 2024-04-25T08:53:52.427 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33204.log 2024-04-25T08:53:52.427 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33599.log.gz 2024-04-25T08:53:52.428 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37382.log 2024-04-25T08:53:52.428 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46348.log.gz 2024-04-25T08:53:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41701.log 2024-04-25T08:53:52.429 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33204.log.gz 2024-04-25T08:53:52.429 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35198.log 2024-04-25T08:53:52.430 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37382.log.gz 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37050.log 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41701.log.gz 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37609.log 2024-04-25T08:53:52.432 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35198.log.gz 2024-04-25T08:53:52.432 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34885.log 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37050.log.gz 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37888.log 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37609.log.gz 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48394.log 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34885.log.gz 2024-04-25T08:53:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44867.log 2024-04-25T08:53:52.436 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37888.log.gz 2024-04-25T08:53:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45335.log 2024-04-25T08:53:52.437 INFO:teuthology.orchestra.run.smithi046.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.48394.log.gz 2024-04-25T08:53:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34859.log 2024-04-25T08:53:52.437 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44867.log.gz 2024-04-25T08:53:52.437 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39205.log 2024-04-25T08:53:52.438 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45335.log.gz 2024-04-25T08:53:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37661.log 2024-04-25T08:53:52.439 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34859.log.gz 2024-04-25T08:53:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42470.log 2024-04-25T08:53:52.439 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39205.log.gz 2024-04-25T08:53:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2024-04-25T08:53:52.441 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37661.log.gz 2024-04-25T08:53:52.441 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.42470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41500.log 0.0% 2024-04-25T08:53:52.441 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.0.42470.log.gz 2024-04-25T08:53:52.442 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45673.log 2024-04-25T08:53:52.443 INFO:teuthology.orchestra.run.smithi046.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2024-04-25T08:53:52.443 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34911.log 2024-04-25T08:53:52.443 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41500.log.gz 2024-04-25T08:53:52.444 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36183.log 2024-04-25T08:53:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34911.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45673.log.gz 2024-04-25T08:53:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46318.log 2024-04-25T08:53:52.445 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34911.log.gz 2024-04-25T08:53:52.445 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2024-04-25T08:53:52.446 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36183.log.gz 2024-04-25T08:53:52.446 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34625.log 2024-04-25T08:53:52.447 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39283.log 2024-04-25T08:53:52.447 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2024-04-25T08:53:52.448 INFO:teuthology.orchestra.run.smithi046.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.46318.log.gz 2024-04-25T08:53:52.448 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41448.log 2024-04-25T08:53:52.448 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34625.log.gz 2024-04-25T08:53:52.449 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49703.log 2024-04-25T08:53:52.449 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39283.log.gz 2024-04-25T08:53:52.450 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32349.log 2024-04-25T08:53:52.450 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41448.log.gz 2024-04-25T08:53:52.451 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32167.log 2024-04-25T08:53:52.451 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49703.log.gz 2024-04-25T08:53:52.452 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36894.log 2024-04-25T08:53:52.453 INFO:teuthology.orchestra.run.smithi046.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.32349.log.gz 2024-04-25T08:53:52.453 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35575.log 2024-04-25T08:53:52.453 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32167.log.gz 2024-04-25T08:53:52.454 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46082.log 2024-04-25T08:53:52.454 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36894.log.gz 2024-04-25T08:53:52.454 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2024-04-25T08:53:52.454 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35575.log.gz 2024-04-25T08:53:52.455 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-04-25T08:53:52.455 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46082.log.gz 2024-04-25T08:53:52.456 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40519.log 2024-04-25T08:53:52.457 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2024-04-25T08:53:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35523.log 2024-04-25T08:53:52.457 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-04-25T08:53:52.458 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.40519.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.40519.log.gz 2024-04-25T08:53:52.458 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33989.log 2024-04-25T08:53:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35895.log 2024-04-25T08:53:52.459 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35523.log.gz 2024-04-25T08:53:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47849.log 2024-04-25T08:53:52.460 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33989.log.gz 2024-04-25T08:53:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53093.log 2024-04-25T08:53:52.461 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35895.log.gz 2024-04-25T08:53:52.461 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48087.log 2024-04-25T08:53:52.462 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47849.log.gz 2024-04-25T08:53:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35627.log 2024-04-25T08:53:52.463 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53093.log.gz 2024-04-25T08:53:52.463 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51911.log 2024-04-25T08:53:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35627.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.48087.log.gz 2024-04-25T08:53:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53145.log 2024-04-25T08:53:52.464 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35627.log.gz 2024-04-25T08:53:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52213.log 2024-04-25T08:53:52.465 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51911.log.gz 2024-04-25T08:53:52.466 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42991.log 2024-04-25T08:53:52.466 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53145.log.gz 2024-04-25T08:53:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33611.log 2024-04-25T08:53:52.467 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52213.log.gz 2024-04-25T08:53:52.467 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49755.log 2024-04-25T08:53:52.468 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42991.log.gz 2024-04-25T08:53:52.468 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49390.log 2024-04-25T08:53:52.469 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33611.log.gz 2024-04-25T08:53:52.469 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48671.log 2024-04-25T08:53:52.470 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49755.log.gz 2024-04-25T08:53:52.470 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52318.log 2024-04-25T08:53:52.470 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49390.log.gz 2024-04-25T08:53:52.471 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47471.log 2024-04-25T08:53:52.472 INFO:teuthology.orchestra.run.smithi046.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48671.log.gz 2024-04-25T08:53:52.472 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43328.log 2024-04-25T08:53:52.473 INFO:teuthology.orchestra.run.smithi082.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T08:53:52.473 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52318.log.gz 2024-04-25T08:53:52.474 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52623.log 2024-04-25T08:53:52.474 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47471.log.gz 2024-04-25T08:53:52.474 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50345.log 2024-04-25T08:53:52.474 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43328.log.gz 2024-04-25T08:53:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53275.log 2024-04-25T08:53:52.475 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52623.log.gz 2024-04-25T08:53:52.475 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51256.log 2024-04-25T08:53:52.475 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50345.log.gz 2024-04-25T08:53:52.476 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47875.log 2024-04-25T08:53:52.477 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53275.log.gz 2024-04-25T08:53:52.477 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48009.log 2024-04-25T08:53:52.477 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51256.log.gz 2024-04-25T08:53:52.478 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50178.log 2024-04-25T08:53:52.478 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47875.log.gz 2024-04-25T08:53:52.479 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34171.log 2024-04-25T08:53:52.479 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48009.log.gz 2024-04-25T08:53:52.479 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39591.log 2024-04-25T08:53:52.480 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50178.log.gz 2024-04-25T08:53:52.480 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42070.log 2024-04-25T08:53:52.480 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34171.log.gz 2024-04-25T08:53:52.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52265.log 2024-04-25T08:53:52.481 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39591.log.gz 2024-04-25T08:53:52.482 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35973.log 2024-04-25T08:53:52.482 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42070.log.gz 2024-04-25T08:53:52.483 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49599.log 2024-04-25T08:53:52.483 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52265.log.gz 2024-04-25T08:53:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37739.log 2024-04-25T08:53:52.484 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35973.log.gz 2024-04-25T08:53:52.484 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37862.log 2024-04-25T08:53:52.485 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49599.log.gz 2024-04-25T08:53:52.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46563.log 2024-04-25T08:53:52.486 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37739.log.gz 2024-04-25T08:53:52.486 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37024.log 2024-04-25T08:53:52.486 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37862.log.gz 2024-04-25T08:53:52.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.46563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34225.log 2024-04-25T08:53:52.487 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46563.log.gz 2024-04-25T08:53:52.488 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40664.log 2024-04-25T08:53:52.488 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37024.log.gz 2024-04-25T08:53:52.489 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40842.log 2024-04-25T08:53:52.489 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34225.log.gz 2024-04-25T08:53:52.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-04-25T08:53:52.490 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40842.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.40664.log.gz 2024-04-25T08:53:52.491 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38400.log 2024-04-25T08:53:52.491 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40842.log.gz 2024-04-25T08:53:52.491 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46467.log 2024-04-25T08:53:52.491 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-04-25T08:53:52.492 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35172.log 2024-04-25T08:53:52.493 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38400.log.gz 2024-04-25T08:53:52.493 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52500.log 2024-04-25T08:53:52.494 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35172.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46467.log.gz 2024-04-25T08:53:52.494 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47445.log 2024-04-25T08:53:52.494 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35172.log.gz 2024-04-25T08:53:52.495 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42419.log 2024-04-25T08:53:52.495 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52500.log.gz 2024-04-25T08:53:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32605.log 2024-04-25T08:53:52.496 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47445.log.gz 2024-04-25T08:53:52.496 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42419.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T08:53:52.497 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42419.log.gz 2024-04-25T08:53:52.497 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51419.log 2024-04-25T08:53:52.497 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32605.log.gz 2024-04-25T08:53:52.498 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33027.log 2024-04-25T08:53:52.499 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52781.log 2024-04-25T08:53:52.499 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51419.log.gz 2024-04-25T08:53:52.500 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42317.log 2024-04-25T08:53:52.500 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33027.log.gz 2024-04-25T08:53:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44709.log 2024-04-25T08:53:52.501 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52781.log.gz 2024-04-25T08:53:52.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38478.log 2024-04-25T08:53:52.502 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42317.log.gz 2024-04-25T08:53:52.502 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44709.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.35843.log 2024-04-25T08:53:52.503 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44709.log.gz 2024-04-25T08:53:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43202.log 2024-04-25T08:53:52.503 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38478.log.gz 2024-04-25T08:53:52.504 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48502.log 2024-04-25T08:53:52.504 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35843.log.gz 2024-04-25T08:53:52.505 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44995.log 2024-04-25T08:53:52.506 INFO:teuthology.orchestra.run.smithi046.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.43202.log.gz 2024-04-25T08:53:52.506 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44735.log 2024-04-25T08:53:52.506 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48502.log.gz 2024-04-25T08:53:52.507 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37455.log 2024-04-25T08:53:52.507 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44995.log.gz 2024-04-25T08:53:52.508 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52937.log 2024-04-25T08:53:52.508 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44735.log.gz/var/log/ceph/ceph-client.admin.37455.log: 2024-04-25T08:53:52.509 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37455.log.gz 2024-04-25T08:53:52.509 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50892.log 2024-04-25T08:53:52.510 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43656.log 2024-04-25T08:53:52.510 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52937.log.gz 2024-04-25T08:53:52.511 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35786.log 2024-04-25T08:53:52.511 INFO:teuthology.orchestra.run.smithi046.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50892.log.gz 2024-04-25T08:53:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.43656.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43656.log.gz 2024-04-25T08:53:52.512 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52729.log 2024-04-25T08:53:52.513 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41727.log 2024-04-25T08:53:52.513 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35786.log.gz 2024-04-25T08:53:52.514 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36593.log 2024-04-25T08:53:52.514 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52729.log.gz 2024-04-25T08:53:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53015.log 2024-04-25T08:53:52.515 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41727.log.gz 2024-04-25T08:53:52.515 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52571.log 2024-04-25T08:53:52.516 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36593.log.gz 2024-04-25T08:53:52.517 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32522.log 2024-04-25T08:53:52.517 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53015.log.gz 2024-04-25T08:53:52.517 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35388.log/var/log/ceph/ceph-client.admin.52571.log: 2024-04-25T08:53:52.518 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52571.log.gz 2024-04-25T08:53:52.518 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32029.log 2024-04-25T08:53:52.519 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37321.log 2024-04-25T08:53:52.519 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35388.log.gz 2024-04-25T08:53:52.520 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32029.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.32522.log.gz 2024-04-25T08:53:52.520 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46108.log 2024-04-25T08:53:52.520 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32029.log.gz 2024-04-25T08:53:52.521 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45206.log 2024-04-25T08:53:52.521 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37321.log.gz 2024-04-25T08:53:52.522 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33637.log 2024-04-25T08:53:52.522 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46108.log.gz 2024-04-25T08:53:52.523 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45206.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T08:53:52.523 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45206.log.gz 2024-04-25T08:53:52.524 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39787.log 2024-04-25T08:53:52.524 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33637.log.gz 2024-04-25T08:53:52.524 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34428.log 2024-04-25T08:53:52.525 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32736.log 2024-04-25T08:53:52.526 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39787.log.gz 2024-04-25T08:53:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.34428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.34428.log.gz 2024-04-25T08:53:52.526 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33403.log 2024-04-25T08:53:52.527 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35869.log 2024-04-25T08:53:52.527 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32736.log.gz 2024-04-25T08:53:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33403.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33403.log.gz 2024-04-25T08:53:52.528 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38337.log 2024-04-25T08:53:52.529 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39439.log 2024-04-25T08:53:52.529 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35869.log.gz 2024-04-25T08:53:52.530 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38337.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.38757.log 2024-04-25T08:53:52.530 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38337.log.gz 2024-04-25T08:53:52.531 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39361.log 2024-04-25T08:53:52.531 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39439.log.gz 2024-04-25T08:53:52.532 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39309.log 2024-04-25T08:53:52.532 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38757.log.gz 2024-04-25T08:53:52.533 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33407.log 2024-04-25T08:53:52.533 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39361.log.gz 2024-04-25T08:53:52.534 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36619.log 2024-04-25T08:53:52.534 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39309.log.gz 2024-04-25T08:53:52.535 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31872.log 2024-04-25T08:53:52.535 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33407.log.gz 2024-04-25T08:53:52.536 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53171.log 2024-04-25T08:53:52.536 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36619.log.gz 2024-04-25T08:53:52.537 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41649.log 2024-04-25T08:53:52.537 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31872.log.gz 2024-04-25T08:53:52.537 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35760.log 2024-04-25T08:53:52.538 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53171.log.gz 2024-04-25T08:53:52.538 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44763.log 2024-04-25T08:53:52.538 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41649.log.gz 2024-04-25T08:53:52.539 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44390.log 2024-04-25T08:53:52.539 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35760.log.gz 2024-04-25T08:53:52.540 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50525.log 2024-04-25T08:53:52.540 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44763.log.gz 2024-04-25T08:53:52.541 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42185.log 2024-04-25T08:53:52.542 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44390.log.gz 2024-04-25T08:53:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46899.log 2024-04-25T08:53:52.543 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50525.log.gz 2024-04-25T08:53:52.543 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40130.log 2024-04-25T08:53:52.543 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42185.log.gz 2024-04-25T08:53:52.544 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31807.log 2024-04-25T08:53:52.544 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46899.log.gz 2024-04-25T08:53:52.546 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53225.log 2024-04-25T08:53:52.546 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40130.log.gz 2024-04-25T08:53:52.546 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42494.log 2024-04-25T08:53:52.546 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.31807.log.gz 2024-04-25T08:53:52.547 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47706.log 2024-04-25T08:53:52.547 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53225.log.gz 2024-04-25T08:53:52.548 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33196.log 2024-04-25T08:53:52.550 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42494.log.gz 2024-04-25T08:53:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36671.log 2024-04-25T08:53:52.550 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47706.log.gz 2024-04-25T08:53:52.550 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33105.log 2024-04-25T08:53:52.550 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33196.log.gz 2024-04-25T08:53:52.551 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32866.log 2024-04-25T08:53:52.551 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36671.log.gz 2024-04-25T08:53:52.552 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33105.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33105.log.gz 2024-04-25T08:53:52.552 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45176.log 2024-04-25T08:53:52.552 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51339.log 2024-04-25T08:53:52.553 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32866.log.gz 2024-04-25T08:53:52.554 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48035.log 2024-04-25T08:53:52.554 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45176.log.gz 2024-04-25T08:53:52.554 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35414.log 2024-04-25T08:53:52.555 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51339.log.gz 2024-04-25T08:53:52.555 INFO:teuthology.orchestra.run.smithi082.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T08:53:52.556 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44422.log 2024-04-25T08:53:52.556 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48035.log.gz 2024-04-25T08:53:52.557 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39413.log 2024-04-25T08:53:52.557 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35414.log.gz 2024-04-25T08:53:52.558 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34703.log 2024-04-25T08:53:52.558 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44422.log.gz 2024-04-25T08:53:52.558 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49335.log 2024-04-25T08:53:52.559 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39413.log.gz 2024-04-25T08:53:52.559 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44538.log 2024-04-25T08:53:52.560 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34703.log.gz 2024-04-25T08:53:52.560 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51987.log 2024-04-25T08:53:52.561 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49335.log.gz 2024-04-25T08:53:52.561 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49833.log 2024-04-25T08:53:52.561 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44538.log.gz 2024-04-25T08:53:52.562 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46291.log 2024-04-25T08:53:52.562 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51987.log.gz 2024-04-25T08:53:52.563 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35120.log 2024-04-25T08:53:52.563 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49833.log.gz 2024-04-25T08:53:52.564 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37269.log 2024-04-25T08:53:52.565 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35120.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35120.log.gz 2024-04-25T08:53:52.565 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-04-25T08:53:52.565 INFO:teuthology.orchestra.run.smithi046.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.46291.log.gz 2024-04-25T08:53:52.566 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48554.log 2024-04-25T08:53:52.566 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37269.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37269.log.gz 2024-04-25T08:53:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51819.log 2024-04-25T08:53:52.567 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-04-25T08:53:52.568 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43894.log 2024-04-25T08:53:52.568 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48554.log.gz 2024-04-25T08:53:52.569 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51180.log 2024-04-25T08:53:52.569 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51819.log.gz 2024-04-25T08:53:52.570 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38959.log 2024-04-25T08:53:52.570 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43894.log.gz 2024-04-25T08:53:52.571 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2024-04-25T08:53:52.571 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51180.log.gz 2024-04-25T08:53:52.572 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47732.log 2024-04-25T08:53:52.572 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44050.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38959.log.gz 2024-04-25T08:53:52.573 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2024-04-25T08:53:52.573 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46582.log 2024-04-25T08:53:52.574 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50712.log 2024-04-25T08:53:52.574 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47732.log.gz 2024-04-25T08:53:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37217.log 2024-04-25T08:53:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50712.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46582.log.gz 2024-04-25T08:53:52.576 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42938.log 2024-04-25T08:53:52.576 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50712.log.gz 2024-04-25T08:53:52.577 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33419.log 2024-04-25T08:53:52.577 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37217.log.gz 2024-04-25T08:53:52.577 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.42938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42938.log.gz 2024-04-25T08:53:52.578 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49781.log 2024-04-25T08:53:52.579 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43252.log 2024-04-25T08:53:52.579 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33419.log.gz 2024-04-25T08:53:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34119.log 2024-04-25T08:53:52.580 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49781.log.gz 2024-04-25T08:53:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34015.log 2024-04-25T08:53:52.581 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz 2024-04-25T08:53:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49625.log 2024-04-25T08:53:52.582 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34119.log.gz 2024-04-25T08:53:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48285.log 2024-04-25T08:53:52.583 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34015.log.gz 2024-04-25T08:53:52.583 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36567.log 2024-04-25T08:53:52.584 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49625.log.gz 2024-04-25T08:53:52.585 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48233.log 2024-04-25T08:53:52.585 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48285.log.gz 2024-04-25T08:53:52.585 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34041.log 2024-04-25T08:53:52.586 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36567.log.gz 2024-04-25T08:53:52.586 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33807.log 2024-04-25T08:53:52.586 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48233.log.gz 2024-04-25T08:53:52.587 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40918.log 2024-04-25T08:53:52.587 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34041.log.gz 2024-04-25T08:53:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38784.log 2024-04-25T08:53:52.589 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33807.log.gz 2024-04-25T08:53:52.589 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52448.log 2024-04-25T08:53:52.590 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40918.log.gz 2024-04-25T08:53:52.590 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49416.log 2024-04-25T08:53:52.590 INFO:teuthology.orchestra.run.smithi046.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.38784.log.gz 2024-04-25T08:53:52.591 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34599.log 2024-04-25T08:53:52.592 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52448.log.gz 2024-04-25T08:53:52.592 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38045.log 2024-04-25T08:53:52.593 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49416.log.gz 2024-04-25T08:53:52.593 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34599.log: gzip 88.9% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.34599.log.gz -- 2024-04-25T08:53:52.593 INFO:teuthology.orchestra.run.smithi046.stderr: /var/log/ceph/ceph-client.admin.34989.log 2024-04-25T08:53:52.594 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39491.log 2024-04-25T08:53:52.595 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38045.log.gz 2024-04-25T08:53:52.595 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38110.log 2024-04-25T08:53:52.595 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34989.log.gz 2024-04-25T08:53:52.596 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48259.log 2024-04-25T08:53:52.596 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39491.log.gz 2024-04-25T08:53:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49573.log 2024-04-25T08:53:52.597 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38110.log.gz 2024-04-25T08:53:52.598 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36461.log 2024-04-25T08:53:52.598 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48259.log.gz 2024-04-25T08:53:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32762.log 2024-04-25T08:53:52.599 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49573.log.gz 2024-04-25T08:53:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40766.log 2024-04-25T08:53:52.600 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36461.log.gz 2024-04-25T08:53:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33157.log 2024-04-25T08:53:52.601 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32762.log.gz 2024-04-25T08:53:52.602 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45749.log 2024-04-25T08:53:52.602 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40766.log.gz 2024-04-25T08:53:52.603 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39387.log 2024-04-25T08:53:52.603 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33157.log.gz 2024-04-25T08:53:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39036.log 2024-04-25T08:53:52.604 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39387.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.45749.log.gz 2024-04-25T08:53:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46568.log 2024-04-25T08:53:52.605 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39387.log.gz 2024-04-25T08:53:52.605 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40286.log 2024-04-25T08:53:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39036.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.39036.log.gz 2024-04-25T08:53:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.46568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46568.log.gz 2024-04-25T08:53:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45041.log 2024-04-25T08:53:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42105.log 2024-04-25T08:53:52.608 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40286.log.gz 2024-04-25T08:53:52.608 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.45041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50018.log 2024-04-25T08:53:52.608 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45041.log.gz 2024-04-25T08:53:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27561.log 2024-04-25T08:53:52.610 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42105.log.gz 2024-04-25T08:53:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49887.log 2024-04-25T08:53:52.610 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50018.log.gz 2024-04-25T08:53:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.27561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50576.log 2024-04-25T08:53:52.611 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27561.log.gz 2024-04-25T08:53:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41208.log 2024-04-25T08:53:52.612 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49887.log.gz 2024-04-25T08:53:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.50576.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50576.log.gz 2024-04-25T08:53:52.613 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39755.log 2024-04-25T08:53:52.614 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47654.log 2024-04-25T08:53:52.614 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41208.log.gz 2024-04-25T08:53:52.615 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39755.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39755.log.gz 2024-04-25T08:53:52.615 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50122.log 2024-04-25T08:53:52.616 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50792.log 2024-04-25T08:53:52.616 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47654.log.gz 2024-04-25T08:53:52.617 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49031.log 2024-04-25T08:53:52.617 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50122.log.gz 2024-04-25T08:53:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42788.log 2024-04-25T08:53:52.618 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50792.log.gz 2024-04-25T08:53:52.619 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52396.log 2024-04-25T08:53:52.619 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49031.log.gz 2024-04-25T08:53:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32631.log 2024-04-25T08:53:52.620 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42788.log.gz 2024-04-25T08:53:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34807.log 2024-04-25T08:53:52.621 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52396.log.gz 2024-04-25T08:53:52.622 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45124.log 2024-04-25T08:53:52.622 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32631.log.gz 2024-04-25T08:53:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47497.log 2024-04-25T08:53:52.623 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34807.log.gz 2024-04-25T08:53:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50608.log 2024-04-25T08:53:52.624 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45124.log.gz 2024-04-25T08:53:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45259.log 2024-04-25T08:53:52.625 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47497.log.gz 2024-04-25T08:53:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50397.log 2024-04-25T08:53:52.626 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50608.log.gz 2024-04-25T08:53:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40156.log 2024-04-25T08:53:52.627 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45259.log.gz 2024-04-25T08:53:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40050.log 2024-04-25T08:53:52.628 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50397.log.gz 2024-04-25T08:53:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38732.log 2024-04-25T08:53:52.629 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40156.log.gz 2024-04-25T08:53:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40050.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T08:53:52.630 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40050.log.gz 2024-04-25T08:53:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.38732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38732.log.gz 2024-04-25T08:53:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41021.log 2024-04-25T08:53:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38556.log 2024-04-25T08:53:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40392.log 2024-04-25T08:53:52.633 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41021.log.gz 2024-04-25T08:53:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36079.log 2024-04-25T08:53:52.634 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38556.log.gz 2024-04-25T08:53:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37499.log 2024-04-25T08:53:52.636 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40392.log.gz 2024-04-25T08:53:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34937.log 2024-04-25T08:53:52.636 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36079.log.gz 2024-04-25T08:53:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41779.log 2024-04-25T08:53:52.637 INFO:teuthology.orchestra.run.smithi046.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.37499.log.gz 2024-04-25T08:53:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33613.log 2024-04-25T08:53:52.639 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34937.log.gz 2024-04-25T08:53:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47957.log 2024-04-25T08:53:52.639 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41779.log.gz 2024-04-25T08:53:52.640 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36313.log 2024-04-25T08:53:52.641 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33613.log.gz 2024-04-25T08:53:52.641 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41526.log 2024-04-25T08:53:52.642 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47957.log.gz 2024-04-25T08:53:52.643 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44841.log 2024-04-25T08:53:52.643 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36313.log.gz 2024-04-25T08:53:52.644 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43534.log 2024-04-25T08:53:52.644 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41526.log.gz 2024-04-25T08:53:52.644 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46632.log 2024-04-25T08:53:52.645 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44841.log.gz 2024-04-25T08:53:52.646 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32219.log 2024-04-25T08:53:52.646 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43534.log.gz 2024-04-25T08:53:52.647 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50918.log 2024-04-25T08:53:52.648 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32219.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46632.log.gz 2024-04-25T08:53:52.648 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43733.log 2024-04-25T08:53:52.648 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32219.log.gz 2024-04-25T08:53:52.649 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52181.log 2024-04-25T08:53:52.649 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50918.log.gz 2024-04-25T08:53:52.650 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41935.log 2024-04-25T08:53:52.650 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43733.log.gz 2024-04-25T08:53:52.651 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41097.log 2024-04-25T08:53:52.652 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52181.log.gz 2024-04-25T08:53:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34729.log 2024-04-25T08:53:52.652 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41935.log.gz 2024-04-25T08:53:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52755.log 2024-04-25T08:53:52.654 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41097.log.gz 2024-04-25T08:53:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42892.log 2024-04-25T08:53:52.655 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34729.log.gz 2024-04-25T08:53:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48528.log 2024-04-25T08:53:52.656 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52755.log.gz 2024-04-25T08:53:52.656 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51502.log 2024-04-25T08:53:52.657 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42892.log.gz 2024-04-25T08:53:52.657 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40015.log 2024-04-25T08:53:52.658 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48528.log.gz 2024-04-25T08:53:52.659 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44225.log 2024-04-25T08:53:52.659 INFO:teuthology.orchestra.run.smithi046.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.51502.log.gz 2024-04-25T08:53:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51004.log 2024-04-25T08:53:52.660 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40015.log.gz 2024-04-25T08:53:52.660 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35146.log 2024-04-25T08:53:52.661 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44225.log.gz 2024-04-25T08:53:52.662 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51717.log 2024-04-25T08:53:52.662 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51004.log.gz 2024-04-25T08:53:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44277.log 2024-04-25T08:53:52.663 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35146.log.gz 2024-04-25T08:53:52.664 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41988.log 2024-04-25T08:53:52.665 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33203.log 2024-04-25T08:53:52.665 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44277.log.gz 2024-04-25T08:53:52.665 INFO:teuthology.orchestra.run.smithi046.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.51717.log.gz 2024-04-25T08:53:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50044.log 2024-04-25T08:53:52.666 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41988.log.gz 2024-04-25T08:53:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.33203.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 2024-04-25T08:53:52.668 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33203.log.gz 2024-04-25T08:53:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38807.log 2024-04-25T08:53:52.668 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50044.log.gz 2024-04-25T08:53:52.669 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48183.log 2024-04-25T08:53:52.669 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2024-04-25T08:53:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36339.log 2024-04-25T08:53:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38807.log: /var/log/ceph/ceph-client.0.48183.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.38807.log.gz 2024-04-25T08:53:52.671 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38686.log 2024-04-25T08:53:52.672 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48183.log.gz 2024-04-25T08:53:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36397.log 2024-04-25T08:53:52.673 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36339.log.gz 2024-04-25T08:53:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.38686.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41474.log 2024-04-25T08:53:52.674 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38686.log.gz 2024-04-25T08:53:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50239.log 2024-04-25T08:53:52.675 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36397.log.gz 2024-04-25T08:53:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48419.log 2024-04-25T08:53:52.676 INFO:teuthology.orchestra.run.smithi046.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41474.log.gz 2024-04-25T08:53:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34677.log 2024-04-25T08:53:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50239.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50239.log.gz 2024-04-25T08:53:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45150.log 2024-04-25T08:53:52.678 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48419.log.gz 2024-04-25T08:53:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44512.log 2024-04-25T08:53:52.679 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34677.log.gz 2024-04-25T08:53:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47562.log 2024-04-25T08:53:52.680 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45150.log.gz 2024-04-25T08:53:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38426.log 2024-04-25T08:53:52.681 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44512.log.gz 2024-04-25T08:53:52.682 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43404.log 2024-04-25T08:53:52.682 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47562.log.gz 2024-04-25T08:53:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40514.log 2024-04-25T08:53:52.683 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38426.log.gz 2024-04-25T08:53:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37243.log 2024-04-25T08:53:52.685 INFO:teuthology.orchestra.run.smithi046.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.43404.log.gz 2024-04-25T08:53:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.40514.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T08:53:52.685 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40514.log.gz 2024-04-25T08:53:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35276.log 2024-04-25T08:53:52.686 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37243.log.gz 2024-04-25T08:53:52.687 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52545.log 2024-04-25T08:53:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33396.log 2024-04-25T08:53:52.689 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35276.log.gz 2024-04-25T08:53:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52545.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52545.log.gz 2024-04-25T08:53:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50213.log 2024-04-25T08:53:52.690 INFO:teuthology.orchestra.run.smithi082.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T08:53:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33396.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33396.log.gz 2024-04-25T08:53:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36365.log 2024-04-25T08:53:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39906.log 2024-04-25T08:53:52.693 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50213.log.gz 2024-04-25T08:53:52.693 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49952.log 2024-04-25T08:53:52.694 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36365.log.gz 2024-04-25T08:53:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40234.log 2024-04-25T08:53:52.695 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39906.log.gz 2024-04-25T08:53:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49952.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49952.log.gz 2024-04-25T08:53:52.696 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35817.log 2024-04-25T08:53:52.697 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40234.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40234.log.gz 2024-04-25T08:53:52.698 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33210.log 2024-04-25T08:53:52.699 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40468.log 2024-04-25T08:53:52.699 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35817.log.gz 2024-04-25T08:53:52.700 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41292.log 2024-04-25T08:53:52.700 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33210.log.gz 2024-04-25T08:53:52.701 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40468.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40468.log.gz 2024-04-25T08:53:52.702 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36261.log 2024-04-25T08:53:52.702 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41292.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41292.log.gz 2024-04-25T08:53:52.703 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52370.log 2024-04-25T08:53:52.704 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45570.log 2024-04-25T08:53:52.704 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36261.log.gz 2024-04-25T08:53:52.705 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49859.log 2024-04-25T08:53:52.705 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52370.log.gz 2024-04-25T08:53:52.707 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45570.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45570.log.gz 2024-04-25T08:53:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49859.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49859.log.gz 2024-04-25T08:53:52.708 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35068.log 2024-04-25T08:53:52.709 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33079.log 2024-04-25T08:53:52.711 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52038.log 2024-04-25T08:53:52.711 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35068.log.gz 2024-04-25T08:53:52.712 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33079.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33079.log.gz 2024-04-25T08:53:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.52038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52038.log.gz 2024-04-25T08:53:52.713 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34251.log 2024-04-25T08:53:52.714 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35302.log 2024-04-25T08:53:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47393.log 2024-04-25T08:53:52.717 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35302.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34251.log.gz 2024-04-25T08:53:52.717 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35302.log.gz 2024-04-25T08:53:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47393.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47393.log.gz 2024-04-25T08:53:52.718 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45427.log 2024-04-25T08:53:52.719 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35734.log 2024-04-25T08:53:52.721 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36840.log 2024-04-25T08:53:52.721 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45427.log.gz 2024-04-25T08:53:52.722 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35734.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35734.log.gz 2024-04-25T08:53:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36840.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36840.log.gz 2024-04-25T08:53:52.723 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43458.log 2024-04-25T08:53:52.724 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48870.log 2024-04-25T08:53:52.727 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39173.log 2024-04-25T08:53:52.727 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43458.log.gz/var/log/ceph/ceph-client.admin.48870.log: 2024-04-25T08:53:52.727 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50970.log 2024-04-25T08:53:52.727 INFO:teuthology.orchestra.run.smithi046.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.48870.log.gz 2024-04-25T08:53:52.728 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43707.log 2024-04-25T08:53:52.729 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39173.log.gz 2024-04-25T08:53:52.730 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40566.log 2024-04-25T08:53:52.730 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50970.log.gz 2024-04-25T08:53:52.731 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32683.log 2024-04-25T08:53:52.731 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43707.log.gz 2024-04-25T08:53:52.732 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40566.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.40566.log.gz 2024-04-25T08:53:52.732 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36053.log 2024-04-25T08:53:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32683.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32683.log.gz 2024-04-25T08:53:52.733 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39722.log 2024-04-25T08:53:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48341.log 2024-04-25T08:53:52.735 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36053.log.gz 2024-04-25T08:53:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39722.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39722.log.gz 2024-04-25T08:53:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44592.log 2024-04-25T08:53:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48341.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48341.log.gz 2024-04-25T08:53:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43354.log 2024-04-25T08:53:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49521.log 2024-04-25T08:53:52.739 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44592.log.gz 2024-04-25T08:53:52.739 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37352.log 2024-04-25T08:53:52.740 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49521.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43354.log.gz 2024-04-25T08:53:52.741 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49521.log.gz 2024-04-25T08:53:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34067.log 2024-04-25T08:53:52.742 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37352.log.gz 2024-04-25T08:53:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42520.log 2024-04-25T08:53:52.744 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48207.log 2024-04-25T08:53:52.745 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34067.log.gz 2024-04-25T08:53:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42520.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42520.log.gz 2024-04-25T08:53:52.746 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48207.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48207.log.gz 2024-04-25T08:53:52.747 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39959.log 2024-04-25T08:53:52.748 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39565.log 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37433.log 2024-04-25T08:53:52.750 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39959.log.gz 2024-04-25T08:53:52.750 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39565.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39565.log.gz 2024-04-25T08:53:52.751 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37433.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37433.log.gz 2024-04-25T08:53:52.752 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42842.log 2024-04-25T08:53:52.753 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33885.log 2024-04-25T08:53:52.755 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53119.log 2024-04-25T08:53:52.755 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33885.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42842.log.gz 2024-04-25T08:53:52.755 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33885.log.gz 2024-04-25T08:53:52.756 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53119.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53119.log.gz 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34755.log 2024-04-25T08:53:52.758 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42237.log 2024-04-25T08:53:52.760 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38309.log 2024-04-25T08:53:52.760 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34755.log.gz 2024-04-25T08:53:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39933.log 2024-04-25T08:53:52.761 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42237.log.gz 2024-04-25T08:53:52.761 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38309.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38309.log.gz 2024-04-25T08:53:52.762 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51126.log 2024-04-25T08:53:52.762 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39933.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39933.log.gz 2024-04-25T08:53:52.763 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32087.log 2024-04-25T08:53:52.764 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34466.log 2024-04-25T08:53:52.765 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51126.log.gz 2024-04-25T08:53:52.765 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32087.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32087.log.gz 2024-04-25T08:53:52.766 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39257.log 2024-04-25T08:53:52.767 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46004.log 2024-04-25T08:53:52.767 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34466.log.gz 2024-04-25T08:53:52.768 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2024-04-25T08:53:52.768 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39257.log.gz 2024-04-25T08:53:52.770 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46004.log: 90.9%/var/log/ceph/ceph-client.admin.44128.log: -- replaced with /var/log/ceph/ceph-client.admin.46004.log.gz 2024-04-25T08:53:52.770 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41370.log 2024-04-25T08:53:52.771 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2024-04-25T08:53:52.772 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37295.log 2024-04-25T08:53:52.773 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51643.log 2024-04-25T08:53:52.773 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41370.log.gz 2024-04-25T08:53:52.774 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37295.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37295.log.gz 2024-04-25T08:53:52.775 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38737.log 2024-04-25T08:53:52.776 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51643.log.gz 2024-04-25T08:53:52.776 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34963.log 2024-04-25T08:53:52.778 INFO:teuthology.orchestra.run.smithi082.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T08:53:52.778 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.38737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52155.log 2024-04-25T08:53:52.778 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38737.log.gz 2024-04-25T08:53:52.779 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40868.log 2024-04-25T08:53:52.779 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34963.log.gz 2024-04-25T08:53:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52155.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52155.log.gz 2024-04-25T08:53:52.780 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49262.log 2024-04-25T08:53:52.782 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32399.log 2024-04-25T08:53:52.782 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40868.log.gz 2024-04-25T08:53:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32814.log 2024-04-25T08:53:52.786 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1%/var/log/ceph/ceph-client.admin.32399.log: -- replaced with /var/log/ceph/ceph-client.admin.49262.log.gz 2024-04-25T08:53:52.786 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32399.log.gz 2024-04-25T08:53:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45098.log 2024-04-25T08:53:52.786 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32814.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32814.log.gz 2024-04-25T08:53:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52422.log 2024-04-25T08:53:52.787 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51307.log 2024-04-25T08:53:52.787 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45098.log.gz 2024-04-25T08:53:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52422.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52422.log.gz 2024-04-25T08:53:52.788 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42658.log 2024-04-25T08:53:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.51307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51307.log.gz 2024-04-25T08:53:52.789 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50371.log 2024-04-25T08:53:52.791 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42658.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T08:53:52.791 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42658.log.gz 2024-04-25T08:53:52.792 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50371.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50371.log.gz 2024-04-25T08:53:52.795 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48847.log 2024-04-25T08:53:52.796 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T08:53:52.796 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44683.log 2024-04-25T08:53:52.797 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45875.log 2024-04-25T08:53:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48847.log: /var/log/ceph/ceph-client.admin.44683.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48847.log.gz 2024-04-25T08:53:52.799 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49729.log 2024-04-25T08:53:52.800 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44683.log.gz 2024-04-25T08:53:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45875.log: /var/log/ceph/ceph-client.admin.49729.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.45875.log.gz 2024-04-25T08:53:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41396.log 2024-04-25T08:53:52.801 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49729.log.gz 2024-04-25T08:53:52.802 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-04-25T08:53:52.804 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2024-04-25T08:53:52.804 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41396.log.gz 2024-04-25T08:53:52.805 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53249.log 2024-04-25T08:53:52.805 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-04-25T08:53:52.806 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47419.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz 2024-04-25T08:53:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53249.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53249.log.gz 2024-04-25T08:53:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38452.log 2024-04-25T08:53:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48470.log 2024-04-25T08:53:52.810 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41422.log 2024-04-25T08:53:52.810 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38452.log.gz 2024-04-25T08:53:52.811 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.48470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48470.log.gz 2024-04-25T08:53:52.811 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52833.log 2024-04-25T08:53:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41422.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41422.log.gz 2024-04-25T08:53:52.813 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49085.log 2024-04-25T08:53:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52833.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52833.log.gz 2024-04-25T08:53:52.814 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46951.log 2024-04-25T08:53:52.816 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44173.log 2024-04-25T08:53:52.816 INFO:teuthology.orchestra.run.smithi046.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49085.log.gz 2024-04-25T08:53:52.817 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46951.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46951.log.gz 2024-04-25T08:53:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44173.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44173.log.gz 2024-04-25T08:53:52.818 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31977.log 2024-04-25T08:53:52.820 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51391.log 2024-04-25T08:53:52.821 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35682.log 2024-04-25T08:53:52.821 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31977.log.gz 2024-04-25T08:53:52.822 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51391.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51391.log.gz 2024-04-25T08:53:52.823 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38835.log 2024-04-25T08:53:52.824 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35682.log.gz 2024-04-25T08:53:52.824 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47061.log 2024-04-25T08:53:52.826 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43785.log 2024-04-25T08:53:52.827 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38835.log.gz 2024-04-25T08:53:52.827 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46983.log 2024-04-25T08:53:52.827 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47061.log.gz 2024-04-25T08:53:52.828 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43785.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43785.log.gz 2024-04-25T08:53:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46517.log 2024-04-25T08:53:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46983.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46983.log.gz 2024-04-25T08:53:52.830 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33609.log 2024-04-25T08:53:52.831 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42131.log 2024-04-25T08:53:52.831 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46517.log.gz 2024-04-25T08:53:52.832 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33609.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33609.log.gz 2024-04-25T08:53:52.833 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42131.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42131.log.gz 2024-04-25T08:53:52.833 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46688.log 2024-04-25T08:53:52.834 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36487.log 2024-04-25T08:53:52.837 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38281.log 2024-04-25T08:53:52.837 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36487.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46688.log.gz 2024-04-25T08:53:52.837 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35042.log 2024-04-25T08:53:52.837 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36487.log.gz 2024-04-25T08:53:52.839 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38281.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38281.log.gz 2024-04-25T08:53:52.839 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41156.log 2024-04-25T08:53:52.840 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35042.log.gz 2024-04-25T08:53:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44620.log 2024-04-25T08:53:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47314.log 2024-04-25T08:53:52.842 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41156.log.gz 2024-04-25T08:53:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42546.log 2024-04-25T08:53:52.843 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44620.log.gz 2024-04-25T08:53:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47314.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47314.log.gz 2024-04-25T08:53:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43674.log 2024-04-25T08:53:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52963.log 2024-04-25T08:53:52.846 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42546.log.gz 2024-04-25T08:53:52.847 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32271.log 2024-04-25T08:53:52.847 INFO:teuthology.orchestra.run.smithi046.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.43674.log.gz 2024-04-25T08:53:52.848 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52963.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52963.log.gz 2024-04-25T08:53:52.848 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49059.log 2024-04-25T08:53:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32271.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32271.log.gz 2024-04-25T08:53:52.850 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43868.log 2024-04-25T08:53:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51744.log 2024-04-25T08:53:52.851 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49059.log.gz 2024-04-25T08:53:52.852 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-04-25T08:53:52.852 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43868.log.gz 2024-04-25T08:53:52.855 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51744.log: /var/log/ceph/ceph-client.admin.44945.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.51744.log.gz 2024-04-25T08:53:52.855 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-04-25T08:53:52.855 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49920.log 2024-04-25T08:53:52.856 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36235.log 2024-04-25T08:53:52.858 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34145.log 2024-04-25T08:53:52.858 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49920.log.gz 2024-04-25T08:53:52.858 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36235.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36235.log.gz 2024-04-25T08:53:52.859 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52989.log 2024-04-25T08:53:52.860 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46134.log 2024-04-25T08:53:52.861 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34145.log.gz 2024-04-25T08:53:52.862 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49651.log 2024-04-25T08:53:52.862 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52989.log.gz 2024-04-25T08:53:52.863 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42369.log 2024-04-25T08:53:52.863 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46134.log.gz 2024-04-25T08:53:52.864 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49651.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49651.log.gz 2024-04-25T08:53:52.865 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.34548.log 2024-04-25T08:53:52.866 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33833.log 2024-04-25T08:53:52.866 INFO:teuthology.orchestra.run.smithi046.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.42369.log.gz 2024-04-25T08:53:52.867 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.2.34548.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.2.34548.log.gz 2024-04-25T08:53:52.867 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33418.log 2024-04-25T08:53:52.868 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50096.log 2024-04-25T08:53:52.869 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33833.log.gz 2024-04-25T08:53:52.870 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45072.log 2024-04-25T08:53:52.870 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33418.log.gz 2024-04-25T08:53:52.871 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50096.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50096.log.gz 2024-04-25T08:53:52.871 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46873.log 2024-04-25T08:53:52.873 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45072.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45072.log.gz 2024-04-25T08:53:52.874 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5/var/log/ceph/ceph-client.admin.46873.log: --verbose -- /var/log/ceph/ceph-client.admin.45399.log 2024-04-25T08:53:52.874 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46873.log.gz 2024-04-25T08:53:52.875 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34317.log 2024-04-25T08:53:52.876 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34199.log 2024-04-25T08:53:52.876 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45399.log.gz 2024-04-25T08:53:52.877 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34317.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34317.log.gz 2024-04-25T08:53:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34199.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34199.log.gz 2024-04-25T08:53:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37914.log 2024-04-25T08:53:52.879 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36920.log 2024-04-25T08:53:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43946.log 2024-04-25T08:53:52.882 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37914.log.gz 2024-04-25T08:53:52.882 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36920.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36920.log.gz 2024-04-25T08:53:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43946.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43946.log.gz 2024-04-25T08:53:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51883.log 2024-04-25T08:53:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33623.log 2024-04-25T08:53:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42628.log 2024-04-25T08:53:52.886 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51883.log.gz 2024-04-25T08:53:52.887 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2024-04-25T08:53:52.887 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33623.log.gz 2024-04-25T08:53:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42628.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42628.log.gz 2024-04-25T08:53:52.889 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46386.log 2024-04-25T08:53:52.890 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47217.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gz 2024-04-25T08:53:52.890 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34573.log 2024-04-25T08:53:52.891 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41266.log 2024-04-25T08:53:52.891 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46386.log.gz 2024-04-25T08:53:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34573.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34573.log.gz 2024-04-25T08:53:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41266.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41266.log.gz 2024-04-25T08:53:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32003.log 2024-04-25T08:53:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39987.log 2024-04-25T08:53:52.896 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32949.log 2024-04-25T08:53:52.897 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32003.log: 89.3%/var/log/ceph/ceph-client.admin.39987.log: -- replaced with /var/log/ceph/ceph-client.admin.32003.log.gz 2024-04-25T08:53:52.897 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39987.log.gz 2024-04-25T08:53:52.898 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37993.log 2024-04-25T08:53:52.899 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32949.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32949.log.gz 2024-04-25T08:53:52.899 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42211.log 2024-04-25T08:53:52.901 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43150.log 2024-04-25T08:53:52.901 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37993.log.gz 2024-04-25T08:53:52.902 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42211.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42211.log.gz 2024-04-25T08:53:52.903 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43150.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.43150.log.gz 2024-04-25T08:53:52.904 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33615.log 2024-04-25T08:53:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42816.log 2024-04-25T08:53:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49547.log 2024-04-25T08:53:52.907 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33615.log.gz 2024-04-25T08:53:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42816.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42816.log.gz 2024-04-25T08:53:52.908 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49547.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49547.log.gz 2024-04-25T08:53:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37940.log 2024-04-25T08:53:52.910 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41318.log 2024-04-25T08:53:52.911 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43100.log 2024-04-25T08:53:52.912 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37940.log.gz 2024-04-25T08:53:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41318.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41318.log.gz 2024-04-25T08:53:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43100.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43100.log.gz 2024-04-25T08:53:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38078.log 2024-04-25T08:53:52.915 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43584.log 2024-04-25T08:53:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37791.log 2024-04-25T08:53:52.917 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38078.log.gz 2024-04-25T08:53:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43584.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43584.log.gz 2024-04-25T08:53:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37791.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37791.log.gz 2024-04-25T08:53:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36751.log 2024-04-25T08:53:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37765.log 2024-04-25T08:53:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35094.log 2024-04-25T08:53:52.922 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36751.log.gz 2024-04-25T08:53:52.923 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44251.log 2024-04-25T08:53:52.923 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37765.log.gz 2024-04-25T08:53:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35094.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35094.log.gz 2024-04-25T08:53:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32923.log 2024-04-25T08:53:52.925 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44251.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44251.log.gz 2024-04-25T08:53:52.926 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33911.log 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48636.log 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32923.log.gz 2024-04-25T08:53:52.928 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33911.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33911.log.gz 2024-04-25T08:53:52.929 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48636.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48636.log.gz 2024-04-25T08:53:52.930 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2024-04-25T08:53:52.931 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37531.log 2024-04-25T08:53:52.932 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52597.log 2024-04-25T08:53:52.932 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2024-04-25T08:53:52.933 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37531.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37531.log.gz 2024-04-25T08:53:52.934 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40971.log 2024-04-25T08:53:52.935 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52597.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52597.log.gz 2024-04-25T08:53:52.935 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39231.log 2024-04-25T08:53:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48465.log 2024-04-25T08:53:52.937 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39231.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40971.log.gz 2024-04-25T08:53:52.937 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39231.log.gz 2024-04-25T08:53:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.48465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48465.log.gz 2024-04-25T08:53:52.938 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36972.log 2024-04-25T08:53:52.939 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51857.log 2024-04-25T08:53:52.941 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45620.log 2024-04-25T08:53:52.941 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36972.log.gz 2024-04-25T08:53:52.942 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51857.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51857.log.gz 2024-04-25T08:53:52.943 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41234.log 2024-04-25T08:53:52.944 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45620.log.gz 2024-04-25T08:53:52.945 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49005.log 2024-04-25T08:53:52.946 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36779.log 2024-04-25T08:53:52.946 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41234.log.gz 2024-04-25T08:53:52.947 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36998.log 2024-04-25T08:53:52.947 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49005.log.gz 2024-04-25T08:53:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36779.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36779.log.gz 2024-04-25T08:53:52.948 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50571.log 2024-04-25T08:53:52.950 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35708.log 2024-04-25T08:53:52.950 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36998.log.gz 2024-04-25T08:53:52.951 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.50571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50571.log.gz 2024-04-25T08:53:52.951 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32323.log 2024-04-25T08:53:52.952 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35708.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35708.log.gz 2024-04-25T08:53:52.953 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47009.log 2024-04-25T08:53:52.953 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32323.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32323.log.gz 2024-04-25T08:53:52.954 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37713.log 2024-04-25T08:53:52.955 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52703.log 2024-04-25T08:53:52.955 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47009.log.gz 2024-04-25T08:53:52.956 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37713.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37713.log.gz 2024-04-25T08:53:52.957 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52703.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52703.log.gz 2024-04-25T08:53:52.958 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36131.log 2024-04-25T08:53:52.959 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47087.log 2024-04-25T08:53:52.960 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.36131.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41597.log 2024-04-25T08:53:52.961 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36131.log.gz 2024-04-25T08:53:52.961 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47087.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47087.log.gz 2024-04-25T08:53:52.962 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41597.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41597.log.gz 2024-04-25T08:53:52.963 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37076.log 2024-04-25T08:53:52.964 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39617.log 2024-04-25T08:53:52.966 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35250.log 2024-04-25T08:53:52.966 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39617.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.37076.log.gz 2024-04-25T08:53:52.967 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39617.log.gz 2024-04-25T08:53:52.968 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35250.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35250.log.gz 2024-04-25T08:53:52.968 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32975.log 2024-04-25T08:53:52.969 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38374.log 2024-04-25T08:53:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42343.log 2024-04-25T08:53:52.971 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32975.log.gz 2024-04-25T08:53:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52103.log 2024-04-25T08:53:52.972 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38374.log.gz 2024-04-25T08:53:52.973 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42343.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42343.log.gz 2024-04-25T08:53:52.973 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38255.log 2024-04-25T08:53:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52103.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52103.log.gz 2024-04-25T08:53:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44789.log 2024-04-25T08:53:52.976 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38255.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T08:53:52.976 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38255.log.gz 2024-04-25T08:53:52.976 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44789.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44789.log.gz 2024-04-25T08:53:52.977 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T08:53:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34093.log 2024-04-25T08:53:52.981 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.34093.log: 89.4% 88.3% -- replaced with /var/log/ceph/ceph.audit.log.gz -- replaced with /var/log/ceph/ceph-client.admin.34093.log.gz 2024-04-25T08:53:52.981 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-25T08:53:52.981 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51365.log 2024-04-25T08:53:52.982 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53067.log 2024-04-25T08:53:52.984 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51365.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51365.log.gz 2024-04-25T08:53:52.985 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53067.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53067.log.gz 2024-04-25T08:53:52.985 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50843.log 2024-04-25T08:53:52.987 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41130.log 2024-04-25T08:53:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.50843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50843.log.gz 2024-04-25T08:53:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41130.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41130.log.gz 2024-04-25T08:53:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51771.log 2024-04-25T08:53:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T08:53:52.994 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51771.log: /var/log/ceph/ceph-mds.a.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.51771.log.gz 2024-04-25T08:53:52.994 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50660.log 2024-04-25T08:53:52.996 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51088.log 2024-04-25T08:53:52.997 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50660.log.gz 2024-04-25T08:53:52.999 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50070.log 2024-04-25T08:53:53.003 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50070.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50070.log.gz 2024-04-25T08:53:53.003 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51088.log.gz 2024-04-25T08:53:53.004 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43278.log 2024-04-25T08:53:53.006 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37477.log 2024-04-25T08:53:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37477.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.37477.log.gz 2024-04-25T08:53:53.011 INFO:teuthology.orchestra.run.smithi046.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43278.log.gz 2024-04-25T08:53:53.011 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44566.log 2024-04-25T08:53:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44566.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44566.log.gz 2024-04-25T08:53:53.014 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46412.log 2024-04-25T08:53:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37967.log 2024-04-25T08:53:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46412.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46412.log.gz 2024-04-25T08:53:53.018 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27448.log 2024-04-25T08:53:53.018 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37967.log.gz 2024-04-25T08:53:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.27448.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27448.log.gz 2024-04-25T08:53:53.021 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37172.log 2024-04-25T08:53:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.37172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37172.log.gz 2024-04-25T08:53:53.024 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33131.log 2024-04-25T08:53:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33131.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33131.log.gz 2024-04-25T08:53:53.026 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52292.log 2024-04-25T08:53:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40792.log 2024-04-25T08:53:53.030 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52292.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52292.log.gz 2024-04-25T08:53:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40792.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40792.log.gz 2024-04-25T08:53:53.033 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49138.log 2024-04-25T08:53:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49138.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49138.log.gz 2024-04-25T08:53:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47139.log 2024-04-25T08:53:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47139.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47139.log.gz 2024-04-25T08:53:53.039 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44815.log 2024-04-25T08:53:53.043 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49209.log 2024-04-25T08:53:53.043 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44815.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44815.log.gz 2024-04-25T08:53:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-04-25T08:53:53.044 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49209.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49209.log.gz 2024-04-25T08:53:53.047 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43811.log 2024-04-25T08:53:53.048 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-04-25T08:53:53.049 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49677.log 2024-04-25T08:53:53.051 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43811.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43811.log.gz 2024-04-25T08:53:53.052 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38504.log 2024-04-25T08:53:53.055 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49677.log: /var/log/ceph/ceph-client.admin.38504.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38504.log.gz 2024-04-25T08:53:53.056 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32897.log 2024-04-25T08:53:53.057 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50944.log 2024-04-25T08:53:53.058 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32897.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32897.log.gz 2024-04-25T08:53:53.059 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49677.log.gz 2024-04-25T08:53:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35362.log 2024-04-25T08:53:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50944.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50944.log.gz 2024-04-25T08:53:53.065 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35362.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35362.log.gz 2024-04-25T08:53:53.066 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46191.log 2024-04-25T08:53:53.068 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46191.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46191.log.gz 2024-04-25T08:53:53.069 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39696.log 2024-04-25T08:53:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44919.log 2024-04-25T08:53:53.072 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39696.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39696.log.gz 2024-04-25T08:53:53.074 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44919.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44919.log.gz 2024-04-25T08:53:53.074 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52677.log 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52677.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52677.log.gz 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39536.log 2024-04-25T08:53:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39536.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39536.log.gz 2024-04-25T08:53:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40076.log 2024-04-25T08:53:53.083 INFO:teuthology.orchestra.run.smithi046.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T08:53:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51546.log 2024-04-25T08:53:53.084 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40076.log.gz 2024-04-25T08:53:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50265.log 2024-04-25T08:53:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51546.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51546.log.gz 2024-04-25T08:53:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2024-04-25T08:53:53.087 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50265.log.gz 2024-04-25T08:53:53.088 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44199.log 2024-04-25T08:53:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47243.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz 2024-04-25T08:53:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35497.log 2024-04-25T08:53:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44199.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44199.log.gz 2024-04-25T08:53:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35497.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35497.log.gz 2024-04-25T08:53:53.093 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32709.log 2024-04-25T08:53:53.094 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39854.log 2024-04-25T08:53:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32709.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32709.log.gz 2024-04-25T08:53:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39854.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39854.log.gz 2024-04-25T08:53:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32057.log 2024-04-25T08:53:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40614.log 2024-04-25T08:53:53.100 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32057.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32057.log.gz 2024-04-25T08:53:53.101 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40614.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40614.log.gz 2024-04-25T08:53:53.101 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37177.log 2024-04-25T08:53:53.103 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42014.log 2024-04-25T08:53:53.104 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.37177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37177.log.gz 2024-04-25T08:53:53.106 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42014.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42014.log.gz 2024-04-25T08:53:53.106 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48606.log 2024-04-25T08:53:53.107 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47269.log 2024-04-25T08:53:53.108 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48606.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48606.log.gz 2024-04-25T08:53:53.109 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47269.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47269.log.gz 2024-04-25T08:53:53.111 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35224.log 2024-04-25T08:53:53.111 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47680.log 2024-04-25T08:53:53.113 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35224.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35224.log.gz 2024-04-25T08:53:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47680.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47680.log.gz 2024-04-25T08:53:53.114 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47191.log 2024-04-25T08:53:53.116 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51593.log 2024-04-25T08:53:53.117 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47191.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47191.log.gz 2024-04-25T08:53:53.118 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51593.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.51593.log.gz 2024-04-25T08:53:53.119 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51152.log 2024-04-25T08:53:53.120 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52344.log 2024-04-25T08:53:53.122 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51152.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51152.log.gz 2024-04-25T08:53:53.123 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52344.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52344.log.gz 2024-04-25T08:53:53.123 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46847.log 2024-04-25T08:53:53.125 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33404.log 2024-04-25T08:53:53.126 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46847.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46847.log.gz 2024-04-25T08:53:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33404.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33404.log.gz 2024-04-25T08:53:53.127 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40102.log 2024-04-25T08:53:53.129 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40366.log 2024-04-25T08:53:53.130 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40102.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40102.log.gz 2024-04-25T08:53:53.131 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40366.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40366.log.gz 2024-04-25T08:53:53.131 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41552.log 2024-04-25T08:53:53.133 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47628.log 2024-04-25T08:53:53.134 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41552.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41552.log.gz 2024-04-25T08:53:53.135 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47628.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47628.log.gz 2024-04-25T08:53:53.136 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38229.log 2024-04-25T08:53:53.137 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47165.log 2024-04-25T08:53:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38229.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38229.log.gz 2024-04-25T08:53:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47165.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47165.log.gz 2024-04-25T08:53:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53041.log 2024-04-25T08:53:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46256.log 2024-04-25T08:53:53.143 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.53041.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53041.log.gz 2024-04-25T08:53:53.145 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46256.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.46256.log.gz 2024-04-25T08:53:53.145 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48797.log 2024-04-25T08:53:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36946.log 2024-04-25T08:53:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48797.log: /var/log/ceph/ceph-client.admin.36946.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.48797.log.gz 2024-04-25T08:53:53.149 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36946.log.gz 2024-04-25T08:53:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51666.log 2024-04-25T08:53:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52807.log 2024-04-25T08:53:53.154 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51666.log: 91.6%/var/log/ceph/ceph-client.admin.52807.log: -- replaced with /var/log/ceph/ceph-client.admin.51666.log.gz 2024-04-25T08:53:53.154 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39335.log 2024-04-25T08:53:53.154 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52807.log.gz 2024-04-25T08:53:53.154 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42291.log 2024-04-25T08:53:53.156 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39335.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39335.log.gz 2024-04-25T08:53:53.157 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42291.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42291.log.gz 2024-04-25T08:53:53.157 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41844.log 2024-04-25T08:53:53.159 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48371.log 2024-04-25T08:53:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41844.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41844.log.gz 2024-04-25T08:53:53.161 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48371.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48371.log.gz 2024-04-25T08:53:53.162 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32113.log 2024-04-25T08:53:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39465.log 2024-04-25T08:53:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32113.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32113.log.gz 2024-04-25T08:53:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39465.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39465.log.gz 2024-04-25T08:53:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41812.log 2024-04-25T08:53:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41675.log 2024-04-25T08:53:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41812.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41812.log.gz 2024-04-25T08:53:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41675.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41675.log.gz 2024-04-25T08:53:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36814.log 2024-04-25T08:53:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35947.log 2024-04-25T08:53:53.173 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36814.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36814.log.gz 2024-04-25T08:53:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35947.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35947.log.gz 2024-04-25T08:53:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33400.log 2024-04-25T08:53:53.176 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40589.log 2024-04-25T08:53:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33400.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33400.log.gz 2024-04-25T08:53:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40589.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.40589.log.gz 2024-04-25T08:53:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31925.log 2024-04-25T08:53:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42598.log 2024-04-25T08:53:53.182 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31925.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31925.log.gz 2024-04-25T08:53:53.182 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42598.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42598.log.gz 2024-04-25T08:53:53.183 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36209.log 2024-04-25T08:53:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39069.log 2024-04-25T08:53:53.186 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36209.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36209.log.gz 2024-04-25T08:53:53.187 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39069.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39069.log.gz 2024-04-25T08:53:53.187 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37635.log 2024-04-25T08:53:53.190 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37635.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37635.log.gz 2024-04-25T08:53:53.190 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32297.log 2024-04-25T08:53:53.192 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33937.log 2024-04-25T08:53:53.193 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32297.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32297.log.gz 2024-04-25T08:53:53.194 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33937.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.33937.log.gz 2024-04-25T08:53:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48061.log 2024-04-25T08:53:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45373.log 2024-04-25T08:53:53.197 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48061.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48061.log.gz 2024-04-25T08:53:53.198 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45373.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45373.log.gz 2024-04-25T08:53:53.199 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2024-04-25T08:53:53.200 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31718.log 2024-04-25T08:53:53.202 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42965.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2024-04-25T08:53:53.202 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31718.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31718.log.gz 2024-04-25T08:53:53.203 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31951.log 2024-04-25T08:53:53.204 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38203.log 2024-04-25T08:53:53.206 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31951.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31951.log.gz 2024-04-25T08:53:53.207 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38203.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38203.log.gz 2024-04-25T08:53:53.207 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38177.log 2024-04-25T08:53:53.209 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2024-04-25T08:53:53.211 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38177.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38177.log.gz 2024-04-25T08:53:53.212 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46056.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2024-04-25T08:53:53.212 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50686.log 2024-04-25T08:53:53.213 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38906.log 2024-04-25T08:53:53.215 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50686.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50686.log.gz 2024-04-25T08:53:53.216 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38906.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.38906.log.gz 2024-04-25T08:53:53.216 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42724.log 2024-04-25T08:53:53.217 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36866.log 2024-04-25T08:53:53.219 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42724.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42724.log.gz 2024-04-25T08:53:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36866.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36866.log.gz 2024-04-25T08:53:53.220 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51302.log 2024-04-25T08:53:53.221 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47340.log 2024-04-25T08:53:53.223 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.51302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51302.log.gz 2024-04-25T08:53:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47340.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47340.log.gz 2024-04-25T08:53:53.224 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41623.log 2024-04-25T08:53:53.226 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42762.log 2024-04-25T08:53:53.227 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41623.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41623.log.gz 2024-04-25T08:53:53.227 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46160.log 2024-04-25T08:53:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42762.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42762.log.gz 2024-04-25T08:53:53.229 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44648.log 2024-04-25T08:53:53.230 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46160.log.gz 2024-04-25T08:53:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44648.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44648.log.gz 2024-04-25T08:53:53.232 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41047.log 2024-04-25T08:53:53.234 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32193.log 2024-04-25T08:53:53.236 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41047.log: /var/log/ceph/ceph-client.admin.32193.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41047.log.gz 2024-04-25T08:53:53.236 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32193.log.gz 2024-04-25T08:53:53.236 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45299.log 2024-04-25T08:53:53.238 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45549.log 2024-04-25T08:53:53.239 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43998.log 2024-04-25T08:53:53.240 INFO:teuthology.orchestra.run.smithi046.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.45299.log.gz 2024-04-25T08:53:53.241 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.45549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45549.log.gz 2024-04-25T08:53:53.241 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33211.log 2024-04-25T08:53:53.243 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43998.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43998.log.gz 2024-04-25T08:53:53.244 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33211.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33211.log.gz 2024-04-25T08:53:53.244 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48188.log 2024-04-25T08:53:53.245 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48915.log 2024-04-25T08:53:53.247 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.48188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48188.log.gz 2024-04-25T08:53:53.248 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48915.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.48915.log.gz 2024-04-25T08:53:53.249 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33194.log 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36541.log 2024-04-25T08:53:53.252 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33194.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33194.log.gz 2024-04-25T08:53:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36541.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36541.log.gz 2024-04-25T08:53:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44893.log 2024-04-25T08:53:53.254 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34833.log 2024-04-25T08:53:53.256 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44893.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44893.log.gz 2024-04-25T08:53:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34833.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34833.log.gz 2024-04-25T08:53:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22639.log 2024-04-25T08:53:53.259 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50423.log 2024-04-25T08:53:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.tmp-client.admin.22639.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22639.log.gz 2024-04-25T08:53:53.261 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50423.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50423.log.gz 2024-04-25T08:53:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43017.log 2024-04-25T08:53:53.263 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49807.log 2024-04-25T08:53:53.265 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43017.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43017.log.gz 2024-04-25T08:53:53.265 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49807.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49807.log.gz 2024-04-25T08:53:53.266 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42157.log 2024-04-25T08:53:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34781.log 2024-04-25T08:53:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42157.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42157.log.gz 2024-04-25T08:53:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34781.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34781.log.gz 2024-04-25T08:53:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45453.log 2024-04-25T08:53:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40716.log 2024-04-25T08:53:53.275 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45453.log: /var/log/ceph/ceph-client.admin.40716.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45453.log.gz 2024-04-25T08:53:53.275 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38608.log 2024-04-25T08:53:53.275 INFO:teuthology.orchestra.run.smithi046.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40716.log.gz 2024-04-25T08:53:53.275 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39121.log 2024-04-25T08:53:53.277 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38608.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38608.log.gz 2024-04-25T08:53:53.278 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39121.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39121.log.gz 2024-04-25T08:53:53.278 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50449.log 2024-04-25T08:53:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33207.log 2024-04-25T08:53:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50449.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50449.log.gz 2024-04-25T08:53:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33207.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33207.log.gz 2024-04-25T08:53:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47823.log 2024-04-25T08:53:53.284 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49137.log 2024-04-25T08:53:53.285 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47823.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47823.log.gz 2024-04-25T08:53:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49137.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49137.log.gz 2024-04-25T08:53:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37126.log 2024-04-25T08:53:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36157.log 2024-04-25T08:53:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37126.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37126.log.gz 2024-04-25T08:53:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36157.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36157.log.gz 2024-04-25T08:53:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42042.log 2024-04-25T08:53:53.292 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41182.log 2024-04-25T08:53:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42042.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42042.log.gz 2024-04-25T08:53:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.41182.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41182.log.gz 2024-04-25T08:53:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35440.log 2024-04-25T08:53:53.296 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39880.log 2024-04-25T08:53:53.298 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35440.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35440.log.gz 2024-04-25T08:53:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39880.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39880.log.gz 2024-04-25T08:53:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45554.log 2024-04-25T08:53:53.301 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40312.log 2024-04-25T08:53:53.302 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.45554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45554.log.gz 2024-04-25T08:53:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40312.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40312.log.gz 2024-04-25T08:53:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36645.log 2024-04-25T08:53:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47530.log 2024-04-25T08:53:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36645.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36645.log.gz 2024-04-25T08:53:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52859.log 2024-04-25T08:53:53.308 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47530.log.gz 2024-04-25T08:53:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35336.log 2024-04-25T08:53:53.310 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52859.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52859.log.gz 2024-04-25T08:53:53.311 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35336.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35336.log.gz 2024-04-25T08:53:53.311 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43759.log 2024-04-25T08:53:53.313 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40340.log 2024-04-25T08:53:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43759.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43759.log.gz 2024-04-25T08:53:53.315 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40340.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40340.log.gz 2024-04-25T08:53:53.316 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51937.log 2024-04-25T08:53:53.317 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43842.log 2024-04-25T08:53:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51937.log: /var/log/ceph/ceph-client.admin.43842.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51937.log.gz 2024-04-25T08:53:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48743.log 2024-04-25T08:53:53.320 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43842.log.gz 2024-04-25T08:53:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41961.log 2024-04-25T08:53:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48743.log: /var/log/ceph/ceph-client.admin.41961.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48743.log.gz 2024-04-25T08:53:53.323 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41961.log.gz 2024-04-25T08:53:53.324 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51524.log 2024-04-25T08:53:53.325 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40539.log 2024-04-25T08:53:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51524.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51524.log.gz 2024-04-25T08:53:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40539.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40539.log.gz 2024-04-25T08:53:53.328 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32245.log 2024-04-25T08:53:53.329 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33001.log 2024-04-25T08:53:53.330 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.32245.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32245.log.gz 2024-04-25T08:53:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33001.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33001.log.gz 2024-04-25T08:53:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34651.log 2024-04-25T08:53:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49469.log 2024-04-25T08:53:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.34651.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34651.log.gz 2024-04-25T08:53:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49469.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49469.log.gz 2024-04-25T08:53:53.336 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38636.log 2024-04-25T08:53:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46441.log 2024-04-25T08:53:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38636.log: /var/log/ceph/ceph-client.admin.46441.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38636.log.gz 2024-04-25T08:53:53.340 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39643.log 2024-04-25T08:53:53.340 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46441.log.gz 2024-04-25T08:53:53.341 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44486.log 2024-04-25T08:53:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39643.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39643.log.gz 2024-04-25T08:53:53.343 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38834.log 2024-04-25T08:53:53.344 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44486.log.gz 2024-04-25T08:53:53.345 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32657.log 2024-04-25T08:53:53.347 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38834.log: /var/log/ceph/ceph-client.admin.32657.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.38834.log.gz 2024-04-25T08:53:53.347 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42681.log 2024-04-25T08:53:53.347 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.32657.log.gz 2024-04-25T08:53:53.348 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38582.log 2024-04-25T08:53:53.350 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42681.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42681.log.gz 2024-04-25T08:53:53.351 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38582.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38582.log.gz 2024-04-25T08:53:53.351 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44357.log 2024-04-25T08:53:53.352 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43043.log 2024-04-25T08:53:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44357.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44357.log.gz 2024-04-25T08:53:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43043.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43043.log.gz 2024-04-25T08:53:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35656.log 2024-04-25T08:53:53.357 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52033.log 2024-04-25T08:53:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35656.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35656.log.gz 2024-04-25T08:53:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.52033.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52033.log.gz 2024-04-25T08:53:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50150.log 2024-04-25T08:53:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46814.log 2024-04-25T08:53:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.50150.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50150.log.gz 2024-04-25T08:53:53.363 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46814.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46814.log.gz 2024-04-25T08:53:53.363 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43610.log 2024-04-25T08:53:53.365 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31898.log 2024-04-25T08:53:53.366 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43610.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43610.log.gz 2024-04-25T08:53:53.367 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31898.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31898.log.gz 2024-04-25T08:53:53.367 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35549.log 2024-04-25T08:53:53.369 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39147.log 2024-04-25T08:53:53.370 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.35549.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35549.log.gz 2024-04-25T08:53:53.372 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48137.log 2024-04-25T08:53:53.372 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39147.log.gz 2024-04-25T08:53:53.373 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33963.log 2024-04-25T08:53:53.375 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48137.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48137.log.gz 2024-04-25T08:53:53.376 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33963.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33963.log.gz 2024-04-25T08:53:53.376 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33622.log 2024-04-25T08:53:53.377 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52129.log 2024-04-25T08:53:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.33622.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33622.log.gz 2024-04-25T08:53:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52129.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52129.log.gz 2024-04-25T08:53:53.380 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40260.log 2024-04-25T08:53:53.381 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51063.log 2024-04-25T08:53:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40260.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40260.log.gz 2024-04-25T08:53:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.51063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39095.log 2024-04-25T08:53:53.384 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51063.log.gz 2024-04-25T08:53:53.385 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49312.log 2024-04-25T08:53:53.386 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.39095.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39095.log.gz 2024-04-25T08:53:53.387 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.49312.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49312.log.gz 2024-04-25T08:53:53.387 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52077.log 2024-04-25T08:53:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37411.log 2024-04-25T08:53:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52077.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52077.log.gz 2024-04-25T08:53:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37411.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.37411.log.gz 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48672.log 2024-04-25T08:53:53.393 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43920.log 2024-04-25T08:53:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.48672.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48672.log.gz 2024-04-25T08:53:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43920.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43920.log.gz 2024-04-25T08:53:53.401 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40418.log 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52885.log 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.40418.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40418.log.gz 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.52885.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52885.log.gz 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37557.log 2024-04-25T08:53:53.403 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42572.log 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.37557.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37557.log.gz 2024-04-25T08:53:53.405 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42572.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T08:53:53.405 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42572.log.gz 2024-04-25T08:53:53.405 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36105.log 2024-04-25T08:53:53.406 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36513.log 2024-04-25T08:53:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36105.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36105.log.gz 2024-04-25T08:53:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.36513.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36513.log.gz 2024-04-25T08:53:53.421 INFO:teuthology.orchestra.run.smithi046.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T08:53:53.644 INFO:teuthology.orchestra.run.smithi082.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T08:53:53.713 INFO:teuthology.orchestra.run.smithi082.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T08:53:53.810 INFO:teuthology.orchestra.run.smithi082.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T08:53:53.822 INFO:teuthology.orchestra.run.smithi082.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T08:53:53.825 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T08:53:53.825 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m1.473s 2024-04-25T08:53:53.825 INFO:teuthology.orchestra.run.smithi082.stderr:user 0m5.839s 2024-04-25T08:53:53.825 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m0.498s 2024-04-25T08:53:54.050 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T08:53:54.175 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T08:53:54.198 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T08:53:54.242 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T08:53:54.243 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-04-25T08:53:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m1.910s 2024-04-25T08:53:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m8.205s 2024-04-25T08:53:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m1.688s 2024-04-25T08:53:54.245 INFO:tasks.ceph:Archiving logs... 2024-04-25T08:53:54.245 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/remote/smithi046/log 2024-04-25T08:53:54.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T08:53:56.043 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/remote/smithi082/log 2024-04-25T08:53:56.044 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T08:53:56.777 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T08:53:56.787 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T08:53:56.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T08:53:56.791 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T08:53:56.878 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T08:53:56.891 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T08:53:56.891 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:53:56.895 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:53:56.908 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T08:53:56.908 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-04-25T08:53:56.908 INFO:teuthology.orchestra.run.smithi046.stdout:+hv01.front.sepi 44.4.53.2 2 u 13 64 377 0.081 -2.748 0.250 2024-04-25T08:53:56.908 INFO:teuthology.orchestra.run.smithi046.stdout:+hv02.front.sepi 63.231.80.2 3 u 3 64 377 0.074 -0.493 0.294 2024-04-25T08:53:56.908 INFO:teuthology.orchestra.run.smithi046.stdout:*hv03.front.sepi 74.6.168.72 3 u 2 64 377 0.076 -1.878 0.285 2024-04-25T08:53:56.908 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-25T08:53:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T08:53:56.910 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================== 2024-04-25T08:53:56.910 INFO:teuthology.orchestra.run.smithi082.stdout:+hv01.front.sepi 44.4.53.2 2 u 9 64 377 0.109 -2.853 2.331 2024-04-25T08:53:56.910 INFO:teuthology.orchestra.run.smithi082.stdout:+hv02.front.sepi 63.231.80.2 3 u 5 64 377 0.107 -0.607 2.253 2024-04-25T08:53:56.910 INFO:teuthology.orchestra.run.smithi082.stdout:*hv03.front.sepi 74.6.168.72 3 u 8 64 377 0.090 -1.355 2.755 2024-04-25T08:53:56.910 INFO:teuthology.orchestra.run.smithi082.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-25T08:53:56.911 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T08:53:56.939 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T08:53:56.939 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T08:53:56.969 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T08:53:57.001 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T08:53:57.035 INFO:teuthology.task.internal:Duration was 1397.948967 seconds 2024-04-25T08:53:57.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T08:53:57.069 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T08:53:57.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T08:53:57.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T08:53:57.174 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T08:53:57.174 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-04-25T08:53:57.174 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T08:53:57.195 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T08:53:57.195 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T08:53:57.215 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T08:53:57.216 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T08:53:57.242 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T08:53:57.300 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T08:53:57.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T08:53:57.330 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T08:53:57.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T08:53:57.428 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T08:53:57.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T08:53:57.448 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T08:53:57.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T08:53:57.528 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:53:57.531 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:53:57.546 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-04-25T08:53:57.548 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-04-25T08:53:57.566 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:53:57.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:53:57.618 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:53:57.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:53:57.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T08:53:57.637 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T08:53:57.638 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/remote/smithi046 2024-04-25T08:53:57.638 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T08:53:57.691 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672993/remote/smithi082 2024-04-25T08:53:57.692 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T08:53:57.718 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T08:53:57.718 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T08:53:57.734 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T08:53:57.762 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T08:53:57.774 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T08:53:57.774 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T08:53:57.802 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T08:53:57.803 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T08:53:57.806 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T08:53:57.811 INFO:teuthology.orchestra.run.smithi046.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 08:53 /home/ubuntu/cephtest 2024-04-25T08:53:57.814 INFO:teuthology.orchestra.run.smithi082.stdout: 658122 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 08:53 /home/ubuntu/cephtest 2024-04-25T08:53:57.816 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T08:53:57.836 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T08:53:57.945 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1397.9489674568176 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T08:53:57.945 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:53:58.018 INFO:teuthology.run:pass